Tuesday, 2019-04-16

*** markvoelker has quit IRC00:00
*** lbragstad has joined #openstack-meeting00:07
*** yamamoto has joined #openstack-meeting00:13
*** bobh has quit IRC00:14
*** ijw has joined #openstack-meeting00:16
*** tetsuro has joined #openstack-meeting00:17
*** yamamoto has quit IRC00:22
*** bobh has joined #openstack-meeting00:23
*** cheng1 has quit IRC00:24
*** bobh has quit IRC00:24
*** bobh has joined #openstack-meeting00:25
*** cheng1 has joined #openstack-meeting00:26
*** erlon has joined #openstack-meeting00:28
*** bobh_ has joined #openstack-meeting00:28
*** bobh has quit IRC00:33
*** mattw4 has quit IRC00:37
*** lseki has quit IRC00:37
*** Liang__ has joined #openstack-meeting00:48
*** yamamoto has joined #openstack-meeting00:49
*** bobh_ has quit IRC00:54
*** bobh has joined #openstack-meeting00:58
*** bobh has quit IRC00:59
*** bobh has joined #openstack-meeting01:11
*** eharney has quit IRC01:14
*** baojg has joined #openstack-meeting01:35
*** jamesmcarthur has joined #openstack-meeting01:37
*** zhubx has quit IRC01:49
*** zhubx has joined #openstack-meeting01:49
*** whoami-rajat has joined #openstack-meeting01:53
*** erlon has quit IRC01:58
*** apetrich has quit IRC01:59
*** bobh has quit IRC02:01
*** ijw has quit IRC02:04
*** jamesmcarthur has quit IRC02:08
*** hongbin has joined #openstack-meeting02:12
*** iyamahat has quit IRC02:21
*** lbragstad has quit IRC02:21
*** yamahata has quit IRC02:21
*** bobh has joined #openstack-meeting02:32
*** bobh has quit IRC02:37
*** jamesmcarthur has joined #openstack-meeting02:39
*** ricolin has joined #openstack-meeting02:45
*** jamesmcarthur has quit IRC02:46
*** zhubx has quit IRC02:49
*** boxiang has joined #openstack-meeting02:49
*** baojg has quit IRC02:53
*** lbragstad has joined #openstack-meeting03:00
*** jamesmcarthur has joined #openstack-meeting03:02
*** enriquetaso has quit IRC03:24
*** jamesmcarthur has quit IRC03:39
*** jamesmcarthur has joined #openstack-meeting03:40
*** diablo_rojo has quit IRC03:41
*** jamesmcarthur has quit IRC03:45
*** shilpasd has joined #openstack-meeting03:56
*** tashiromt__ has joined #openstack-meeting04:01
*** samP___ has joined #openstack-meeting04:02
samP___#startmeeting masakari04:02
openstackMeeting started Tue Apr 16 04:02:58 2019 UTC and is due to finish in 60 minutes.  The chair is samP___. Information about MeetBot at http://wiki.debian.org/MeetBot.04:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:02
*** openstack changes topic to " (Meeting topic: masakari)"04:03
openstackThe meeting name has been set to 'masakari'04:03
samP___Hi all for Masakari04:03
shilpasdHi samP04:03
*** imsurit_ofc has joined #openstack-meeting04:03
samP___shilpasd: Hi04:03
samP___#topic critical bugs and patches04:04
*** openstack changes topic to "critical bugs and patches (Meeting topic: masakari)"04:04
samP___Any critical patches or bug need to discuss?04:05
*** _erlon_ has quit IRC04:05
samP___if any please bring up anytime in the meeting04:06
samP___#topic AOB04:06
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:06
samP___We planned to discuss Train work items this week IRC. However, I couldn't finish listing them. I would like to postpone it to next week04:07
samP___If you have any work items for Train, please add them to following etherpad04:07
samP___#link https://etherpad.openstack.org/p/masakari-train-workitems04:07
tashiromt__Thanks for sharing04:07
shilpasdOkay, thank you04:08
samP___No more topics from my side to discuss.04:08
samP___if no other topics, then we can finish the meeting today.04:08
tashiromt__I do not have any topics today04:10
*** jamesmcarthur has joined #openstack-meeting04:11
samP___Seems link no other topics to discuss. Let's finish today's meeting. Please use #openstack-masakari IRC @freenode or openstack-discuss@lists.openstack.org ML for further discussion.04:11
samP___Thank you all for join!04:11
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:12
openstackMeeting ended Tue Apr 16 04:11:59 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:12
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-04-16-04.02.html04:12
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-04-16-04.02.txt04:12
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-04-16-04.02.log.html04:12
*** shilpasd has quit IRC04:15
*** jamesmcarthur has quit IRC04:18
*** hongbin has quit IRC04:40
*** imsurit_ofc has quit IRC04:44
*** bobh has joined #openstack-meeting04:47
*** bobh has quit IRC04:51
*** imsurit_ofc has joined #openstack-meeting04:52
*** e0ne has joined #openstack-meeting04:53
*** e0ne has quit IRC04:56
*** gyee has quit IRC05:08
*** samP___ has quit IRC05:08
*** e0ne has joined #openstack-meeting05:21
*** jbadiapa has quit IRC05:25
*** e0ne has quit IRC05:25
*** dkushwaha has joined #openstack-meeting05:34
*** baojg has joined #openstack-meeting05:37
*** Luzi has joined #openstack-meeting05:37
*** lbragstad has quit IRC05:55
*** sridharg has joined #openstack-meeting05:58
*** lpetrut has joined #openstack-meeting06:02
*** kopecmartin|off is now known as kopecmartin06:07
*** lpetrut has quit IRC06:16
*** lpetrut has joined #openstack-meeting06:16
*** tashiromt__ has quit IRC06:20
*** pcaruana has joined #openstack-meeting06:26
*** masayukig has quit IRC06:27
*** kmalloc has quit IRC06:28
*** kmalloc has joined #openstack-meeting06:30
*** hogepodge has quit IRC06:32
*** johnsom has quit IRC06:32
*** kmalloc has quit IRC06:40
*** ralonsoh has joined #openstack-meeting06:43
*** markvoelker has joined #openstack-meeting06:43
*** xyang has quit IRC06:48
*** slaweq has joined #openstack-meeting06:49
*** xyang has joined #openstack-meeting06:52
*** kmalloc has joined #openstack-meeting06:53
*** johnsom has joined #openstack-meeting06:55
*** apetrich has joined #openstack-meeting06:55
*** masayukig has joined #openstack-meeting06:55
*** hogepodge has joined #openstack-meeting06:56
*** bobh has joined #openstack-meeting07:04
*** e0ne has joined #openstack-meeting07:06
*** bobh has quit IRC07:09
*** e0ne has quit IRC07:09
*** e0ne has joined #openstack-meeting07:16
*** rcernin has quit IRC07:20
*** dkushwaha has quit IRC07:28
*** tssurya has joined #openstack-meeting07:29
*** jbadiapa has joined #openstack-meeting07:31
*** bobh has joined #openstack-meeting07:37
*** bobh has quit IRC07:41
*** yamamoto has quit IRC07:41
*** yamamoto has joined #openstack-meeting07:48
*** e0ne has quit IRC07:52
*** dkehn has quit IRC07:55
*** e0ne has joined #openstack-meeting07:58
*** iyamahat has joined #openstack-meeting07:59
*** e0ne has quit IRC07:59
*** dkushwaha_ has joined #openstack-meeting08:02
*** xyang has quit IRC08:04
*** johnsom has quit IRC08:04
*** iyamahat has quit IRC08:04
dkushwaha_#startmeeting tacker08:05
openstackMeeting started Tue Apr 16 08:05:01 2019 UTC and is due to finish in 60 minutes.  The chair is dkushwaha_. Information about MeetBot at http://wiki.debian.org/MeetBot.08:05
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:05
*** openstack changes topic to " (Meeting topic: tacker)"08:05
openstackThe meeting name has been set to 'tacker'08:05
*** johnsom has joined #openstack-meeting08:05
dkushwaha_#topic Roll Call08:05
*** xyang has joined #openstack-meeting08:05
*** openstack changes topic to "Roll Call (Meeting topic: tacker)"08:05
*** masayukig_ has joined #openstack-meeting08:05
*** masayukig has quit IRC08:05
*** masayukig_ is now known as masayukig08:05
dkushwaha_who is here for Tacker weekly meeting?08:05
*** hjwon has joined #openstack-meeting08:08
dkushwaha_hello hyunsikyang JangwonLee08:08
*** priteau has joined #openstack-meeting08:09
*** e0ne has joined #openstack-meeting08:10
dkushwaha_seems no more members today08:10
dkushwaha_hyunsikyang: JangwonLee do you have something to discuss, otherwise we can skip it for today?08:11
hyunsikyangnot quite much, yet. We will update blueprint soon.08:12
hyunsikyangBecause we are considering the parameter for VNFD. So, you just check it later.08:13
hyunsikyangAnd I wanna discuss about presentation material.08:13
dkushwaha_ok, so lets close this meeting, and will be available on Tacker channel for further discussions08:14
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:14
openstackMeeting ended Tue Apr 16 08:14:37 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:14
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-04-16-08.05.html08:14
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-04-16-08.05.txt08:14
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-04-16-08.05.log.html08:14
*** ttsiouts has joined #openstack-meeting08:21
*** hokeeeeun has joined #openstack-meeting08:26
*** Lucas_Gray has joined #openstack-meeting08:26
*** janki has joined #openstack-meeting08:28
*** hokeeeeun has quit IRC08:28
*** dkushwaha_ has quit IRC08:30
*** ygbo has joined #openstack-meeting08:58
*** bhagyashris has joined #openstack-meeting09:11
*** bhagyashris has quit IRC09:34
*** yamamoto has quit IRC09:45
*** yamamoto has joined #openstack-meeting09:46
*** yamamoto has quit IRC09:46
*** yamamoto has joined #openstack-meeting09:47
*** yamamoto has quit IRC09:52
*** rcernin has joined #openstack-meeting10:02
*** Liang__ has quit IRC10:05
*** ttsiouts has quit IRC10:23
*** ttsiouts has joined #openstack-meeting10:24
*** ttsiouts has quit IRC10:28
*** yamamoto has joined #openstack-meeting10:30
*** yamamoto has quit IRC10:39
*** yamamoto has joined #openstack-meeting10:40
*** raildo has joined #openstack-meeting10:41
*** hjwon has quit IRC10:42
*** yamamoto has quit IRC10:47
*** baojg has quit IRC10:56
*** baojg has joined #openstack-meeting10:57
*** yamamoto has joined #openstack-meeting10:58
*** baojg has quit IRC11:01
*** ttsiouts has joined #openstack-meeting11:02
*** Lucas_Gray has quit IRC11:09
*** XSM_HUST has joined #openstack-meeting11:11
*** XSM_HUST has quit IRC11:12
*** Lucas_Gray has joined #openstack-meeting11:12
*** pcaruana has quit IRC11:17
*** baojg has joined #openstack-meeting11:23
*** bbowen has joined #openstack-meeting11:36
*** XSM_HUST has joined #openstack-meeting11:37
*** tetsuro has quit IRC11:37
*** XSM_HUST has joined #openstack-meeting11:43
*** imsurit_ofc has quit IRC11:43
*** XSM_HUST has quit IRC11:43
*** nguyenhai has joined #openstack-meeting11:47
*** bobh has joined #openstack-meeting11:48
*** XSM_HUST has joined #openstack-meeting11:49
*** XSM_HUST has quit IRC11:50
*** Lucas_Gray has quit IRC11:52
*** lpetrut has quit IRC11:52
*** nguyenhai has quit IRC11:52
*** bobh has quit IRC11:53
*** erlon has joined #openstack-meeting11:59
*** lpetrut has joined #openstack-meeting12:00
*** pcaruana has joined #openstack-meeting12:04
*** baojg has quit IRC12:05
*** bobh has joined #openstack-meeting12:10
*** priteau has quit IRC12:17
*** bobh has quit IRC12:18
*** priteau has joined #openstack-meeting12:19
*** gaoyan has joined #openstack-meeting12:21
*** jamesmcarthur has joined #openstack-meeting12:21
*** Luzi has quit IRC12:24
*** diga has joined #openstack-meeting12:28
*** jamesmcarthur has quit IRC12:30
*** jbadiapa has quit IRC12:32
*** Liang__ has joined #openstack-meeting12:45
*** Luzi has joined #openstack-meeting12:46
*** jamesmcarthur has joined #openstack-meeting12:48
*** enriquetaso has joined #openstack-meeting12:49
*** liuyulong has joined #openstack-meeting12:50
*** mjturek has joined #openstack-meeting12:58
*** jamesmcarthur has quit IRC12:58
*** yamamoto has quit IRC13:07
*** bobh has joined #openstack-meeting13:07
*** pcaruana has quit IRC13:07
*** lbragstad has joined #openstack-meeting13:10
*** mriedem has joined #openstack-meeting13:21
*** eharney has joined #openstack-meeting13:22
*** yaawang has quit IRC13:22
*** yaawang has joined #openstack-meeting13:23
*** yamamoto has joined #openstack-meeting13:24
*** yamamoto has quit IRC13:24
*** yamamoto has joined #openstack-meeting13:24
*** lseki has joined #openstack-meeting13:25
*** yamamoto has quit IRC13:25
*** yamamoto has joined #openstack-meeting13:25
*** mjturek has quit IRC13:27
*** efried_pto is now known as efried13:31
*** boxiang has quit IRC13:32
*** lennyb has quit IRC13:32
*** pcaruana has joined #openstack-meeting13:33
*** boxiang has joined #openstack-meeting13:33
*** lennyb has joined #openstack-meeting13:45
*** bobh has quit IRC13:50
*** yamamoto has quit IRC13:57
*** yamamoto has joined #openstack-meeting13:58
*** yamamoto has quit IRC13:58
*** awaugama has joined #openstack-meeting14:00
*** kopecmartin is now known as kopecmartin|off14:02
*** jrbalderrama has joined #openstack-meeting14:02
*** shintaro has joined #openstack-meeting14:03
*** markvoelker has quit IRC14:07
*** Luzi has quit IRC14:09
*** Lucas_Gray has joined #openstack-meeting14:13
*** shintaro has quit IRC14:13
*** jbadiapa has joined #openstack-meeting14:15
*** Liang__ has quit IRC14:17
*** priteau has quit IRC14:19
*** gaoyan has quit IRC14:20
*** ttsiouts has quit IRC14:23
*** ttsiouts has joined #openstack-meeting14:24
*** rcernin has quit IRC14:27
*** janki has quit IRC14:27
*** ttsiouts has quit IRC14:27
*** ttsiouts has joined #openstack-meeting14:28
*** munimeha1 has joined #openstack-meeting14:32
*** gaoyan has joined #openstack-meeting14:35
*** baojg has joined #openstack-meeting14:35
*** lpetrut has quit IRC14:36
*** priteau has joined #openstack-meeting14:37
*** bobh has joined #openstack-meeting14:39
*** mjturek has joined #openstack-meeting14:41
*** bobh has quit IRC14:41
*** diga has quit IRC14:44
*** dklyle has quit IRC14:49
*** dklyle has joined #openstack-meeting14:50
*** jamesmcarthur has joined #openstack-meeting14:56
*** Liang__ has joined #openstack-meeting15:06
*** Liang__ is now known as LiangFang15:08
*** enriquetaso has quit IRC15:09
*** gaoyan has quit IRC15:23
*** pcaruana has quit IRC15:27
*** yamahata has joined #openstack-meeting15:30
*** pcaruana has joined #openstack-meeting15:30
*** e0ne has quit IRC15:31
*** vishalmanchanda has joined #openstack-meeting15:33
*** pcaruana has quit IRC15:40
*** iyamahat has joined #openstack-meeting15:40
*** pcaruana has joined #openstack-meeting15:46
*** gyee has joined #openstack-meeting15:50
*** ayoung has joined #openstack-meeting15:50
*** ygbo has quit IRC15:52
*** pcaruana has quit IRC15:55
*** mlavalle has joined #openstack-meeting15:58
*** ttsiouts has quit IRC15:59
*** ttsiouts has joined #openstack-meeting15:59
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Apr 16 16:00:05 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
slaweqfirst of all16:00
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:00
slaweqplease open it now16:00
slaweq#topic Actions from previous meetings16:01
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:01
slaweqfirst one: mlavalle to debug reasons of neutron-tempest-plugin-dvr-multinode-scenario failures16:01
*** njohnsto_ has joined #openstack-meeting16:01
mlavalleslaweq: I spent not as much time as I wanted. so no conclusions here yet16:01
*** pcaruana has joined #openstack-meeting16:02
slaweqsure, lets assign it to You for next week as reminder, ok?16:02
slaweq#action mlavalle to debug reasons of neutron-tempest-plugin-dvr-multinode-scenario failures16:03
slaweqso lets go the next one16:03
slaweqralonsoh to report and triage new fullstack test_min_bw_qos_policy_rule_lifecycle failure16:03
ralonsohslaweq, I think that was solved16:03
ralonsohdo you have the launchpad id?16:03
slaweqno, I don't have it now16:04
*** ttsiouts has quit IRC16:04
*** jamesmcarthur has quit IRC16:04
*** njohnsto_ has quit IRC16:04
ralonsohslaweq, ok, let me check it, I'll report it later16:04
slaweqralonsoh: was it this one https://bugs.launchpad.net/neutron/+bug/1824138 ?16:05
openstackLaunchpad bug 1824138 in neutron "Fullstack QoS tests should not handle other tests port events" [High,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)16:05
slaweqok, so it's not fixed yet16:05
ralonsohand the patch: https://review.openstack.org/#/c/651523/16:05
slaweqplease review this patch if You will have some time mlavalle :)16:06
slaweqthx ralonsoh for fixing this bug :)16:06
mlavalleslaweq: ack16:06
ralonsohwe need to back port it to stein16:06
slaweqI saw You added tag for backport also, will You take care of it once it will be merged?16:07
slaweqthx a lot16:07
slaweqok, next one then16:07
slaweqslaweq to report and debug yet another db migration functional tests issue16:07
slaweqI reopened old bug https://bugs.launchpad.net/neutron/+bug/1687027 for it16:07
openstackLaunchpad bug 1687027 in neutron "test_walk_versions tests fail with "IndexError: tuple index out of range" after timeout" [High,Fix committed] - Assigned to Slawek Kaplonski (slaweq)16:07
slaweqAnd sent a patch https://review.openstack.org/65137116:07
slaweqit's merged now and I didn't saw same issues again in last couple of days16:07
slaweqso hopefully it will be better now16:08
slaweqbut if You would see something like that, please ping me :)16:08
*** pcaruana has quit IRC16:08
slaweqok, next one16:08
slaweq* mlavalle will send DNM patch which will add tcpdump in routers' namespaces to debug ssh issue16:09
bcafarelnice so these failing-again tests were "just" because of different raised exception? good to have fixed16:09
slaweqbcafarel: yes, it seems so :)16:10
slaweqmlavalle: thx nice, clean patch :)16:10
slaweqI hope we will spot this issue and we will be able to debug something from it16:10
slaweqnext one was:16:11
slaweqslaweq will send DNM patch to tempest to dump router's namespace state when ssh will fail16:11
slaweqand I didn't have time to do that :/ sorry16:11
slaweqI will add it for myself for this week16:11
slaweq#action slaweq will send DNM patch to tempest to dump router's namespace state when ssh will fail16:11
mlavalleslaweq: add an action to me to recheck the tcpdump patch and analyze output16:12
slaweq#action mlavalle to recheck tcpdump patch and analyze output from ci jobs16:12
slaweqhere You go :)16:12
slaweqand the last one (but not least)16:13
slaweqnjohnston move wsgi jobs to check queue nonvoting16:13
*** jamesmcarthur has joined #openstack-meeting16:13
slaweqI don't think he did it this week16:14
slaweqI will assign it to him for next time to not forget16:14
slaweq#action njohnston move wsgi jobs to check queue nonvoting16:14
slaweqok, that's all actions from last week16:14
slaweqanything You want to add/ask?16:15
slaweqok, so lets move on16:16
slaweq#topic Python 316:16
*** openstack changes topic to "Python 3 (Meeting topic: neutron_ci)"16:16
slaweqStadium projects etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status16:16
slaweqI don't think there was any progress on it recently16:16
slaweqI'm looking into this etherpad now and I see that most of those py27 jobs are still defined as legacy jobs, so we should also try to move them to zuulv316:17
slaweqI will probably have some time to work on those slowly after PTG16:18
slaweqbut I also think that for projects like midonet or odl we will need some help from teams which are responsible for it16:19
slaweqas e.g. I have no any knowledge about those projects, configs and so on16:19
slaweqand probably none of us have such knowledge16:19
slaweqmlavalle: do You think it is reasonable to add such (short) topic to PTG etherpad?16:20
mlavallewe can ask yamamoto and manjeets can help16:20
mlavalleand yes, let's discuss it in Denver16:20
*** Lucas_Gray has quit IRC16:20
slaweqok, I will add it to etherpad then16:20
slaweqanything You want to add regarding python3?16:21
mlavallenot from me16:21
slaweqok, lets move on then16:21
slaweqnext topic16:21
slaweq#topic tempest-plugins migration16:21
*** openstack changes topic to "tempest-plugins migration (Meeting topic: neutron_ci)"16:21
slaweqEtherpad: https://etherpad.openstack.org/p/neutron_stadium_move_to_tempest_plugin_repo16:21
bcafareljust checking the agenda can probably have a "stadium in general" section (even if it will probably be short)16:22
slaweqbcafarel: good point, I can change it :)16:22
slaweqso regarding to tempest plugins, I just send today first PS for bgpvpn project https://review.openstack.org/65299116:22
slaweqlets see how much I will be missing there16:23
*** samP has joined #openstack-meeting16:23
slaweqI also saw that bcafarel send new patch set for sfc today, right?16:23
bcafareland I finally sent first WIP or SFC https://review.openstack.org/#/c/65301216:23
mlavalleI updated my patch16:23
slaweqone question which I want to ask here16:24
bcafarelI kept the same layout as njohnston suggested in fwaas neutron_tempest_plugin/fwaas/{api|scenario|...}16:24
slaweqe.g. in bcafarel's patch I see that there is neutron_tempest_plugin/sfc/services/flowclassifier_client.py file16:24
slaweqI moved bgpvpn_client.py file to be in neutron_tempest_plugin/services/network/json/ directory16:25
slaweqdo You think that we should do it in some consistent way for all patches?16:25
mlavalleI originally did what slaweq did16:25
mlavallebut then I saw what njohnston and bcafarel did and I think it is better16:26
mlavalleThis is my latest revision https://review.openstack.org/#/c/649373/16:26
mlavalleI suggest we all stick to the same approach16:26
slaweqok, so I will move it back in my patch :)16:27
slaweqand we will be consistent with it16:27
mlavalleI also have a question....16:27
bcafarel"everything related to a stadium project tests in its own subfolder" right?16:27
mlavalleyeap bcafarel16:27
mlavallegoing back to my question16:27
mlavalleas you can see, Tempest is now attempting to run the vpnaas test: http://logs.openstack.org/73/649373/5/check/neutron-tempest-plugin-vpnaas/2019e3f/testr_results.html.gz16:28
mlavallebut they are skipped, becuase vpnaas is not confirgured in the tempest config16:28
mlavallewhre is this file?16:28
mlavalleit's been driving me crazy the past 30 minutes16:29
slaweqare You asking about http://logs.openstack.org/73/649373/5/check/neutron-tempest-plugin-vpnaas/2019e3f/controller/logs/tempest_conf.txt.gz ?16:29
*** LiangFang has quit IRC16:30
*** iyamahat has quit IRC16:31
mlavalleI think I have to add vpnaas as a service to the extensions, right?16:31
mlavalleto pass this check https://review.openstack.org/#/c/649373/5/neutron_tempest_plugin/vpnaas/api/test_vpnaas.py@4416:31
mlavallewhere do I add it?16:32
slaweqyes, to neutron extensions list I think16:32
mlavalleyeah, but where in the tempest plugin repo is that file?16:32
slaweqit is defined in .zuul.yaml file16:33
*** yamahata has quit IRC16:33
mlavalleok, thanks16:33
mlavalledriving me crazy the past 30 minutes16:34
slaweqYou should do it in similar way like in https://github.com/openstack/neutron-tempest-plugin/blob/master/.zuul.yaml#L30416:34
slaweqso use "standard" ones and add "vpnaas" to it16:34
* bcafarel takes notes for next sfc patchset too16:34
slaweqor set only vpnaas will be enough, I'm not sure16:34
mlavalleok, thanks!16:34
slaweqok, can we move on to the next topic then?16:35
bcafarelgood for me16:35
slaweq#topic Grafana16:36
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:36
mlavalleyes please16:36
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:36
*** pcaruana has joined #openstack-meeting16:36
slaweqI don't see anything what would be going bad recently16:37
slaweqin fact I think that it's quite good comparing to what was couple of weeks ago16:38
*** enriquetaso has joined #openstack-meeting16:38
mlavalleyes, I agree16:38
slaweqI was even going today through some results of recent jobs to find some examples of failures and I only found some issues with SSH (which is known)16:39
slaweqsome failures not related to neutron directly16:39
*** mattw4 has joined #openstack-meeting16:39
slaweqand some failures e.g. in fullstack or functional jobs related to patch on which it was running16:39
slaweqdo You see anything what You want to discuss now?16:40
mlavalleno, it looks good overall to me16:40
mlavallewhat do you think bcafarel ?16:40
bcafarelloosk good to me too :)16:41
bcafarelI will send patch soon to update http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 (and "older stable" too) to point to stein and rocky now that stein is out16:41
slaweqthx bcafarel :)16:41
slaweqso, lets talk a bit about tempest job now16:41
bcafarelnp, it should not be too complicated :)16:41
slaweq#topic Tempest/Scenario16:42
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"16:42
slaweqI saw one strange failure in http://logs.openstack.org/27/642527/4/check/neutron-tempest-plugin-api/9aeecb7/testr_results.html.gz16:42
slaweqit is strange for me because it looks like it failed because GET returned no body16:42
slaweqBut from logs it looks that all was fine: http://logs.openstack.org/27/642527/4/check/neutron-tempest-plugin-api/9aeecb7/controller/logs/screen-q-svc.txt.gz#_Apr_15_12_46_02_99504316:43
slaweqand what is the most strange for me is that it failed in line https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/admin/test_network_segment_range.py#L20916:43
slaweqand it somehow passed assertions in above lines16:43
slaweqbut I have no idea how it is possible16:43
slaweqmaybe someone wants to take a look at it and will find something what I missed there :)16:44
ralonsohI'll try to debug this tomorrow16:44
slaweqralonsoh: thx a lot16:44
ralonsohI'll ping you if I find something16:44
slaweqI will add it as an action for You to not forget about it, ok?16:45
slaweq#action ralonsoh to debug issue with neutron_tempest_plugin.api.admin.test_network_segment_range test16:45
slaweqthx a lot :)16:45
*** iyamahat has joined #openstack-meeting16:45
slaweqand that was the only new case which I found recently and wanted to raise here16:45
slaweqanything else You want to talk about?16:46
mlavallenot from me16:46
bcafarelnothing new either16:46
slaweqok, so last topic for today16:46
slaweq#topic Periodic16:46
*** openstack changes topic to "Periodic (Meeting topic: neutron_ci)"16:46
slaweqI have only one question regarding to periodic jobs16:47
slaweqwe are running those jobs with python 35 still16:47
slaweqshould we move them to py36?16:47
mlavallei think so16:47
slaweqthere was email from gmann about that recently16:47
bcafarelprobably yes, as they are getting dropped in "normal" jobs16:47
slaweqok, so I will propose patch for that this week16:47
*** tssurya has quit IRC16:47
slaweq#action slaweq to switch periodic jobs from py35 to py3616:48
gmannyeah, we need to move periodic job to py36.  thanks slaweq16:48
*** elox has joined #openstack-meeting16:48
slaweqgmann: thx for confirmation :)16:48
*** armax has quit IRC16:48
*** jamesmcarthur_ has joined #openstack-meeting16:48
slaweqok, thats all from my side for today16:48
slaweqif You don't have anything else I will give You 10 minutes back today16:49
mlavalleI take those 10 minute16:49
mlavallehow about you ralonsoh and bcafarel16:49
bcafarel10 minutes back sounds great indeed :)16:49
ralonsohnothing from me16:49
slaweqHappy Easter and see You next week then :)16:49
slaweqthx for attending16:49
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:49
openstackMeeting ended Tue Apr 16 16:49:57 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:49
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-04-16-16.00.html16:50
mlavallesame to you all16:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-04-16-16.00.txt16:50
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-04-16-16.00.log.html16:50
*** jamesmcarthur has quit IRC16:52
*** markvoelker has joined #openstack-meeting16:52
*** markvoelker has quit IRC16:56
*** ijw has joined #openstack-meeting17:00
*** yamahata has joined #openstack-meeting17:05
*** priteau has quit IRC17:11
*** diablo_rojo has joined #openstack-meeting17:21
*** baojg has quit IRC17:24
*** ricolin has quit IRC17:25
*** e0ne has joined #openstack-meeting17:32
*** ralonsoh has quit IRC17:34
*** armax has joined #openstack-meeting17:34
*** jrbalderrama has quit IRC17:39
*** ijw has quit IRC17:40
*** ijw has joined #openstack-meeting17:42
*** dklyle has quit IRC17:45
*** e0ne has quit IRC17:49
*** priteau has joined #openstack-meeting17:49
*** igordc has joined #openstack-meeting17:50
*** erlon has quit IRC17:54
*** erlon has joined #openstack-meeting17:56
*** mattw4 has quit IRC18:00
*** erlon has quit IRC18:07
*** electrofelix has quit IRC18:13
*** ijw has quit IRC18:15
*** ijw has joined #openstack-meeting18:16
*** absubram has joined #openstack-meeting18:20
*** ijw has quit IRC18:22
*** jamesmcarthur_ has quit IRC18:25
*** jamesmcarthur has joined #openstack-meeting18:26
*** vishalmanchanda has quit IRC18:33
*** priteau has quit IRC18:33
*** ijw has joined #openstack-meeting18:35
*** jamesmcarthur has quit IRC18:35
*** irclogbot_1 has quit IRC18:39
*** jamesmcarthur has joined #openstack-meeting18:40
*** irclogbot_2 has joined #openstack-meeting18:41
*** pabelanger has joined #openstack-meeting18:52
clarkbanyone else here for the infra meeting?18:59
clarkbwe will get started shortly18:59
clarkb#startmeeting infra19:01
openstackMeeting started Tue Apr 16 19:01:10 2019 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-April/006311.html19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
*** Shrews has joined #openstack-meeting19:01
clarkbWeek after next is the summit and ptg. I expect we'll be skipping the meeting on April 30 as a result19:02
clarkbIf those not attending want to have the meeting I have no objection. I just won't be able to put an agenda together or run it that day19:02
clarkb#topic Actions from last meeting19:02
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:02
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-04-09-19.01.txt minutes from last meeting19:03
clarkbThere were no recorded actions that meeting and I believe all the actiosn from the prior had been done19:03
clarkb#topic Specs approval19:03
*** openstack changes topic to "Specs approval (Meeting topic: infra)"19:03
clarkbI think we've all largely been focused on implementing the opendev git hosting spec, config mgmt update spaces and the LE implementation so nothing to add under this topic19:03
clarkb#topic Priority Efforts19:04
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:04
clarkbLets dive right into the fun stuff19:04
clarkb#topic OpenDev19:04
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:04
clarkbI wrote down some specific questions about our planned migration on friday in the agenda19:04
clarkbFirstup is we should probably set some specific times19:04
clarkbI think my preference would be to get started as early in the day as people find reasonable so that we have the whole day to clean up after ourselves :)19:05
corvusfirst thing pacific time?  what's that -- 1500?19:05
clarkbya 1500UTC is 8am pacific19:05
fungias noted earlier, it's my 5th wedding anniversary so i'll have all my stuff done and up for review ahead of time but will only be around off and on for friday so will chip in when i can19:06
fungithough i plan to be around most of the weekend to help unbreak whatever we missed19:06
corvusmordred: 1500 work for you?19:07
clarkbmaybe we say something like starting at 1500UTC the infra/opendev team will begin work to perform the repo hosting migration. Expect outages off and on with gerrit and git hosting services through the day, particularly earlier in the day.19:07
corvusclarkb: sounds good19:08
clarkbok I'll work to send some followup emails with those details19:08
clarkbNext on my list is "are we ready? what do we need to get done before friday?"19:08
fungii'm about to push up the script which *generates* the mapping of renames (i know that wasn't a task in the story, but it became necessary to correlate the dozen or so data sources used to make that decision) and will then send a link to a breakdown of that mapping to the relevant mailing lists19:09
mordredcorvus: ++19:09
fungiand i'm about halfway through composing the script to do the repository edits themselves19:09
clarkbok so waiting on the changes that gerneate those lists. I think we should also freeze new project creation at this point19:09
fungiwhich i'm planning to also benchmark with a full set of data from review.o.o19:09
mordredclarkb: ++19:09
corvusfungi: do you need anyone to jump in and pick up tasks, or are you okay?19:09
clarkbit will just simplify fungi's list generation to not have to accomodate new projects19:09
fungiwell, the list generation is dynamic19:10
clarkbfungi: I know but people will want to review them19:10
fungibut it relies on some data sources we may want to urge projects to quiesce, like openstack's projects.yaml in governance19:10
fungii'm fine with freezing project creation for sure19:10
fungijust thinking that, for example, since folks create new projects before they get added to governance there's a bit of a race where we may rename something we shouldn't19:11
fungiso maybe if jroll could encourage the tc to quickly flush any pending changes of that nature19:11
clarkbAnother thing that I think would be helpful is if we write down our plan in an etherpad like we've done in the past19:12
clarkbthere are fair number of moving parts here like dns updates and renaming things in gerrit and in gitea etc that having that written down will be good19:12
jrollguh, sorry, missed this meeting (and have another meeting shortly)19:12
jrollfungi: will do19:12
*** enriquetaso has quit IRC19:13
clarkbmy head is currently unhappy so I'm not sure I'd get to writing down that plan today, but I can probably start on that tomorrow19:13
clarkbif anyone else is interested in writing the steps down too feel free to start :)19:13
fungithe mappnig generator is just down to me working out how to consume an ethercalc export on the fly now19:13
clarkbWe have a repo rename request on the meeting agenda wiki page as well. Is that somethign we want to incorporate into this transition or do in a followup?19:14
fungiit can be incorporated (i guess by adding to the spreadsheet would be one way) or we can save it to exercise the rename tooling after the mass migration19:15
corvuseither works for me; i'll yield to fungi's opinion on that19:16
clarkbyielding to fungi wfm.19:16
fungithe way i'm incorporating the renames ethercalc, it's as a final override, so adding it there would take care of incorporating it19:17
*** eharney has quit IRC19:17
fungihrm, actually looking back at my script, no it's not unless it's not already part of another project19:18
clarkbOther things to consider: airship is apparently trying to do a release on monday. But they never brought this up in the emails we sent or in these meetings so I'm not seeing that as a blocker19:18
fungiso maybe we hold it for the week after?19:18
clarkbfungi: in that case asving it for later seems fine19:18
fungiyeah, i'm leaning toward that so i don't revisit my current logic19:18
clarkbAnything else we should be thinking about while we've got everyone here?19:19
* mordred feels like he shoul dhave more thoughts of things to think about, but cant' think of any such thoughts19:20
corvusare we set on tls?19:20
clarkbcorvus: for gerrit I guess? fungi did you get a cert for gerrit when you got one for gitea?19:21
fungiyep, it's already in the certs dir19:21
clarkber gitea was already done so the recent one must've been for gerrit. cool19:21
fungipulled that one just in case we wanted it19:21
corvusianw: there was talk of using LE -- is that ready, or should we stick with manual certs now?19:22
fungididn't want this to become a letsencrypt scramble on top of opendev migration19:22
ianwcorvus: yeah, best to leave that, it can be done asynchronously under less pressure.  it's the puppet-ish deployment bits that will need thinking on19:22
corvushttps://review.openstack.org/653108 needs approval :)19:23
clarkband we'll probably need a change to our apache config on review.openstack.org to redirect review.openstack to review.opendev ?19:24
corvusand i guess we should prepare a change to do redirects from review.openstack to review.opendev19:24
corvusclarkb: jinx19:24
clarkboh and update the canonical name of the server19:24
corvusthat's all i can think of19:25
*** baojg has joined #openstack-meeting19:25
*** bobh has joined #openstack-meeting19:25
clarkbseems like a plan is coming together. This is exciting19:25
clarkb#topic Update Config Management19:26
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:26
clarkbContinuing to make good progress on the puppet-4 transition. Ran into issues where there are no puppet-4 arm64 packages so we'll be excluding the arm servers from puppet-419:27
cmurphyi think we're almost done19:27
cmurphynodepool servers are next up19:27
clarkbianw has started work for removing support for puppet3 from stuff as well19:27
clarkbthis allows us to update puppet moduels to versions that require at least puppet 419:27
ianwyeah, if we could look at the grafana review, i can babysit that one19:28
*** bobh has quit IRC19:28
ianw#link https://review.openstack.org/#/c/652443/19:28
ianwthat's ontop of the skip puppet3 fix too19:29
clarkbon the docker side of things we've run into a few issues. Docker is still ignoring my PR for the ipv6 stuff (and the bug I filed). Skopeo has the same issue ( I think because it uses docker's parsing rules for docker:// urls). Also our insecure-ci-registry filled up with images so we'll have to set up garbage collection or different disk backing to address that19:29
cmurphyi think dropping puppet 3 will be easier to do in one big sweep after everything is migrated19:29
corvusclarkb: i think skopeo may literally use the docker code for that19:30
clarkbcorvus: yup I think so. I have a PR up to fix that code so hopefully eventually that gets addressed19:30
*** baojg has quit IRC19:30
corvusi'm concerned the docker project may be effectively dead; every problem we've encountered has been described by a 2-year old open issue19:30
corvuser, every *other* problem :)19:31
clarkbyes I've been wondering about that too especially given the lack of resposne on my PR and bug19:31
clarkbso we may want to investigate alternative tools for building, pushing, and hosting images19:31
corvusyeah; though if skopeo has the same problem.. yeesh19:32
corvusperhaps we should open a skopeo issue?19:32
mordredcorvus: that said - we know people on the skopeo side and they're actiev and responsive19:32
mordredlike - it seems like a place where we should be able to get the issue resolved19:33
corvusat least if they are faced with "docker is sitting on this bug" they may have more options19:33
corvusclarkb: would you mind opening that issue, then poke me so i can poke mordred to poke other people? :)19:33
clarkbsure. I can point them at my docker side issue too19:33
clarkbanything else around config management updates?19:34
*** awaugama has quit IRC19:34
clarkb#topic Storyboard19:36
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:36
clarkbfungi: diablo_rojo: I've been so heads down on other things recently. Anything new here19:36
fungioh, getting close with the telemetry migration at least19:36
*** sridharg has quit IRC19:36
fungiperformed a test import of all their deliverables from lp onto storyboard-dev last week19:37
fungithough right now the ptl lacks control of the lp projects to be able to close down bug reporting there, so is trying to coordinate access with former project members (the groups are not owned by openstack administrators)19:37
*** bobh has joined #openstack-meeting19:37
mordredof course they're not19:38
clarkbalright lets move on we have 20 minutes left for the remainder of our agenda19:39
clarkb#topic General Topics19:39
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:39
clarkbianw: did you want to give us a quick update on the letsencrypt progress? graphite01.opendev.org is letsencrypted now. What are the next steps etc19:40
ianwrestarting services when certificates change is probably the last bit of global infrastructure19:41
ianwi have a proposal up using handlers, and clarkb has a different approach too19:41
ianw#link https://review.openstack.org/65280119:41
clarkbI'm not tied to my approach and handlers are probably more ansibley19:42
ianw#link https://review.openstack.org/65038419:42
ianwi think we can sort that out in review19:42
ianwafter that, it's really just service by service, deciding on how to deploy the keys into production19:42
clarkbgreat progress. thank you for taking that on19:42
clarkbI'm actually really happy with the setup we ended up with. I think it gives us good flexibility and isn't terribly complicated19:43
clarkbnext up is the trusty upgrade backlog19:43
clarkb#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup19:43
clarkblists.openstack.org was upgraded last friday19:43
clarkbThe only minor hiccup was a stale lock file for the openstack vhost19:44
clarkbonce we cleared that out everything was happy19:44
clarkbNeed volunteers for static, ask, refstack, and wiki19:44
clarkber just static, ask and refstack now I think19:44
clarkbthank you mordred and fungi for volunteering to do status and wiki19:44
clarkbbut also I expect we'll be idle on this task this week as the opendev transition is the big focus19:44
cmurphyso that can be puppet-4'd now too right?19:45
fungicould use a second review on the next bit for the wiki19:45
clarkbcmurphy: yes19:45
fungi#link https://review.openstack.org/651352 Replace transitional package names for Xenial19:45
mordredclarkb: also - do we really need status as a separate host from static at this point?19:45
clarkbmordred: that I don't know19:45
mordredI'll loop back around on that with folks in channel19:45
fungimordred: status runs some services, at least some of elasticrecheck i think?19:45
mordredoh yeah. good point19:46
clarkbThe last item on the agenda comes from zbr19:46
mordredelasticrecheck and health are the only two things on status that continue to be useful things - the rest of the things are vestigal19:46
clarkbRemoval of ugo+rw chmod from default job definitions19:46
clarkbzbr: ^ are you here to talk about this?19:46
clarkb"Mainly this insanely unsafe chmod does prevent ansible from loading ansible.cfg files from our repos. That is happening from "Change zuul-cloner permissions" task which is run even for basic jobs like openstack-tox-py* ones." is the quote from the agenda19:47
corvusi'm all for it; is there a proposed solution?19:48
clarkbiirc pabelanger and mordred intended to address this by not runnign zuul-cloner compat by default19:48
clarkbso jobs would have to opt into the chmod sillyness19:48
ianwthe role says "Make repositories writable so that people can hardlink"19:48
mordredI think that distills down to "remove zuul-cloner from base job" yeah?19:48
clarkbianw: ya d-g in particular will hardlink those iirc19:48
clarkbmordred: ya19:48
fungiat least if it's on the same filesystem19:49
corvusright, so the action needed is for someone to figure out which jobs still need that, make sure they have it, then remove from base19:49
fungiis it still in base, or only in legacy-base?19:49
fungifor some reason i thought the latter19:49
clarkbI believe it is still in base19:49
clarkbbecause we realized we'd break a bunch of things a year or so ago when we first looked at moving it19:50
corvusalternatively, just make it be that ^ and let chips fall where they may19:50
mordredyeah - I'm kind of inclined to just do the swap to make sure it's in legacy-base19:51
mordredif a job breaks from missing it - it's an easy fix19:51
clarkbmaybe a post summit/ptg activity considering all the other ways we are going to break the jobs on friday :P19:51
clarkbbut ya that seems like a reasonable path forward19:52
clarkbzbr: ^ hopefully that clears things up for you19:52
mordredclarkb: what is the role in question here?19:52
mordredfetch-zuul-cloner does not show up in base - but does show up in legacy base19:52
clarkboh maybe we have moved it then?19:53
clarkbmordred: zbr claims openstack-tox-py* is affected by this19:53
clarkbare those legacy based jobs?19:53
pabelangerno, it is still in base. I seen it the zuul release job a few hours ago19:54
mordredpabelanger: really? it's not showing up in git-grep for me - so maybe I'm just bad at computers19:54
mordredah. I didn't grep opendev/base-jobs. whoops19:55
clarkbok we are almost out of time. I think we have a general plan forward on this19:57
clarkb#topic Open Discussion19:57
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:57
clarkbanything else before our hour is up?19:57
dtroyerI am looking to set up some github integration with Zuul for starlingx, eyeing the kata setup, a) is that the right model, b) is that best left for after Friday?19:57
clarkbdtroyer: we can discuss it after the meeting in the infra channel likely19:58
clarkbin general though I think we've found that we can't reasonably support github hosted projects19:58
*** dklyle has joined #openstack-meeting19:58
clarkb(we learned a lot with kata)19:58
dtroyerah, ok19:58
fungithere was an official statement we sent to the infra ml a few months ago?20:00
clarkband we are at time. Thanks everyone!20:00
*** jamesmcarthur has quit IRC20:00
clarkbfungi: yup  Isent a note to clarify our position on it to the infra list20:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"20:00
openstackMeeting ended Tue Apr 16 20:00:22 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-04-16-19.01.html20:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-04-16-19.01.txt20:00
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-04-16-19.01.log.html20:00
*** bbowen has quit IRC20:03
*** igordc has quit IRC20:05
*** eharney has joined #openstack-meeting20:14
*** pcaruana has quit IRC20:19
*** Shrews has left #openstack-meeting20:22
*** priteau has joined #openstack-meeting20:23
*** trandles has joined #openstack-meeting20:24
*** priteau has quit IRC20:27
*** oneswig has joined #openstack-meeting20:34
*** ijw has quit IRC20:42
*** ttsiouts has joined #openstack-meeting20:44
*** ijw has joined #openstack-meeting20:47
*** ijw has quit IRC20:54
*** bobh has quit IRC20:56
*** rbudden has joined #openstack-meeting20:56
*** wwriverrat has joined #openstack-meeting20:56
oneswig#startmeeting scientific-sig21:00
openstackMeeting started Tue Apr 16 21:00:17 2019 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: scientific-sig)"21:00
openstackThe meeting name has been set to 'scientific_sig'21:00
oneswigGreetings SIG21:00
oneswig#link Agenda for today https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_16th_201921:00
oneswigHey rbudden, hody21:00
oneswig(I mean, howdy)21:00
*** TheJulia has joined #openstack-meeting21:01
oneswigMartial sends apologies as he has family commitments this week21:01
eloxHello from Sweden, Erik Lönroth http://eriklonroth.com here.21:01
rbuddenunfortunately i have to bounce early to pickup my son, but i’ll catch up on the logs afterwards21:01
*** jamesmcarthur has joined #openstack-meeting21:01
oneswigelox: Hi, welcome :-)21:01
oneswigThanks for coming along21:02
*** martial has joined #openstack-meeting21:02
eloxHope to learn and contribute.21:02
oneswigSeems like it might be a short meeting this week though - with the run-up to Easter a few folks are out21:03
oneswigOK, let's get going...21:03
oneswig#topic Ironic and external DCIM21:03
*** openstack changes topic to "Ironic and external DCIM (Meeting topic: scientific-sig)"21:03
oneswigThe context here is there was some discussion last week with the CERN team (Arne in particular)21:04
martial(On phone as discussed)21:04
oneswigThey've got a big server enrollment imminent and are looking again at how they might streamline that21:04
oneswighey martial, you made it :-)21:04
oneswig#chair martial21:05
openstackCurrent chairs: martial oneswig21:05
oneswig#link previous week's discussion http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-04-10-10.59.log.html#l-1721:05
*** bdx has joined #openstack-meeting21:05
oneswigIf I can summarise correctly, the conclusion was that they'd like a means to store inspection data in an external infrastructure management DB (eg, Netbox, etc)21:06
oneswigAnd to update periodically if possible to keep the information current21:07
eloxIs there anyone from their team here?21:07
*** b1airo has joined #openstack-meeting21:07
oneswigArne's looking at candidate technologies and wants to consider things that people are actually using21:08
oneswigelox: Not as far as I know...21:08
oneswigHi b1airo :-)21:08
oneswig#chair b1airo21:08
openstackCurrent chairs: b1airo martial oneswig21:08
TheJuliaSo following up dtantsur's comment, we've also discussed downstream building a reporting agent to provide a greater detail, but it would only really be viable in self hosted environments and there are items that would need to be worked out to make something viable like that.21:08
oneswigHi TheJulia, thanks for joining21:09
oneswigWhat do you mean by self-hosted in this context?  Sorry.21:09
TheJuliaoperator that is willing to trust and deploy an agent with-in their workload21:10
oneswigAh, thanks.21:10
TheJuliaThe end idea, possibly being keeping ironic's inspection data fairly up to date.21:11
oneswigThat's not ideal, in many cases people wouldn't want that.21:11
TheJuliaOut of band... while seems perfect, we're also talking about embedded systems that are easy to put in weird states that cause them to lock up for a little while which is also not ideal21:12
oneswigTheJulia: I think there was some discussion around the cleaning phase being an option for periodic re-inspection?21:12
*** raildo has quit IRC21:12
TheJuliaoneswig: possibly, the case we have is we want to somehow capture a drive swap21:12
TheJuliaor the addition of a disk21:12
*** wwriverrat has quit IRC21:13
oneswigI see.  Hence wanting to track while an instance is live?21:13
TheJuliaFor which the usefulness is kind pointless. That being said during cleaning to have a check-in with inspector just seems like some minor-ish logic in inspector21:13
TheJuliaoneswig: exactly.21:13
oneswigThe sense I got was that within Ironic many of the pieces for a solution (that met many needs) were already within reach.21:14
oneswigAlthough it sounds like your problem case is still beyond that.21:15
oneswigArne was looking for a shortlist of systems people use for DCIM.21:16
oneswigSo far I recall the list included Xcat, Netbox, Infoblox, possibly OpenDCIM21:16
oneswigrbudden: do you know what you're using at GSFC?21:16
rbuddenwe use xcat21:17
oneswigArne's list was actually longer... i-doit, CMDBuild, SnipeIT, SysAid, Spiceworks, GLPI, NetBox, xcat21:17
oneswigthanks rbudden, I'll make sure that's fed back.  How about you elox?21:18
trandlesHi folks. Sorry, was logged in and STILL late to the meeting.21:18
oneswigHi trandles!21:18
oneswigtrandles: Just talking solutions for DCIM21:19
rbuddenironically at GSFC xcat is used for baremetal provisioning and some lightweight VMs that aren’t openstack controlled, but there is interest in entertaining Ironic for future HPC clusters21:19
trandlesDCIM is a hot topic21:20
eloxWe are using MaaS21:20
oneswigtrandles: the circles in which you move... :-)21:20
oneswigthanks elox, good to know21:20
trandlesI'd like to hear a lot about other's experiences. We don't have a comprehensive DCIM right now but we might be pushed in that direction. Would be beneficial to have a head start when it does come to the fore.21:21
*** jmlowe has joined #openstack-meeting21:21
oneswigtrandles: from the data gathered so far, it's a pretty diverse set of tools in use.21:21
eloxtrandles: We are really keen on sharing our experiences.21:22
jmloweHey, sorry I'm late21:22
oneswigHi jmlowe, no problem :-)21:22
oneswigCurrently surveying DCIMs in use - care to contribute a data point for IU?21:22
jmloweWe have one, I ignore it21:23
TheJuliaI'm kind of curious if there is a reason why21:23
oneswigfair enough :-) probably not a good recommendation21:23
TheJuliaWhich sounds awful, but it gives everyone a data point in terms of is there a process or use for DCIM that might not be ideal21:24
jmlowelooks like opendcim21:24
trandlesOur current issues with DCIM probably aren't unique. We have several siloed systems that came from vendors and only really work well with their equipment, where "equipment" is largely power and cooling. They call them "DCIM" but I don't think they know what that means in a larger sense.21:24
oneswigjmlowe: Is it bad, or just not relevant to your workflows?21:24
jmlowenot relevant, key info in when systems come and go, then ignore21:25
*** ijw has joined #openstack-meeting21:25
TheJuliaSo its use is more as asset tracking21:25
rbuddenjmlowe: that was largely how things were at PSC as well21:25
jmlowetypical lifespan of systems is 5-7 years21:26
rbuddenTheJulia: exactly21:26
oneswigI've seen Netbox play a useful role inbetween those events21:26
*** baojg has joined #openstack-meeting21:26
*** enriquetaso has joined #openstack-meeting21:26
rbuddenI know the problem with lifecycle at PSC was it was originally populated when new systems came in, then fell out of updates for one reason or another (I forget what DCIM was being used)21:27
oneswigTheJulia: do you know what the current state of play is with anomaly detection on inspection data?  I've used cardiff from python-hw for this before, but it was hard to setup21:27
rbuddenso then it wasn’t reliably useful for anything :P21:27
TheJuliaoneswig: like failed hardware detection?21:28
oneswigrbudden: jmlowe: if the DCIM was updated with Ironic inspection data, perhaps it would have more utility21:28
oneswigTheJulia: exactly - missing DIMMs, disks, etc.21:28
TheJuliaoneswig: for inside introspection data, likely the same as long as long as "extra-hardware" is set as a collector for introspection.21:28
oneswigThanks TheJulia, good to know.21:29
trandlesoneswig: we use NHC ( https://github.com/mej/nhc ) for that bit it doesn't integrate with a larger DCIM. We have custom splunk dashboards that communicate that level of brokenness.21:29
trandlesie. missing DIMMs, interconnect links at the wrong speed, CPUs stuck in a bad cstate, etc.21:30
oneswigbrokenness as a metric - how cool is that21:30
oneswiggood link trandles, thanks for that21:30
*** baojg has quit IRC21:30
rbuddensorry, have to run, this is all very interesting though, i’ll ask around about anything else in use at GSFC21:31
oneswigThanks rbudden - see you21:31
rbuddenbye all21:31
oneswigelox: What are your experiences with MaaS?21:31
trandlesit will detect things like wrong firmware levels on mobos, nics, hcas, hdds21:31
eloxoneswig: So far, it has provided us with a means to work with our hardware as if it was a cloud and adding in the Juju framework (https://jujucharms.com/) lets us provision a diversity of modern software previously out of our reach.21:33
oneswigelox: you're provisioning bare metal software deployments with it?  Neat.21:34
elox... We will likely deploy an openStack on top of MaaS which is a legit use-case for us. We already leverage AWS, vsphere and plan to use GCE, Azure and perhaps even Oracle.21:34
b1airook, other meeting done!21:35
b1airobusy morning here o_021:35
oneswigelox: A really diverse set of IaaS there.  I think you've got pretty much the full set? :-)21:35
oneswighey b1airo, good to have you21:36
eloxoneswig: We deploy the full software stack on top of MaaS: SLURM, OFED, application, middleware etc. We add in things as we need to meet a diverse set of use-cases. AI/BigData etc.21:36
oneswigSounds good.21:37
oneswigOK - we should move on with the agenda I guess.  Any more to add before we do?21:38
*** stevebaker has left #openstack-meeting21:38
eloxoneswig: Yes, its diverse. We need to meet different requirements from projects limited in time. For example, month #1, we might need the HPC cluster(s) to be tailored towards DeepLearning. Month #2 Data Analytics or a mix. We then rebuild the cluster explicitly for those workloads via Juju.21:38
b1airocatching up on DCIM discussion... Google Spreadsheet21:38
* b1airo ducks21:38
oneswigb1airo: Very true, certainly we've deployed from them21:39
trandlesI would be remiss if I didn't link this here:  https://github.com/hpc/kraken/21:40
trandlesit's a LANL thing21:40
oneswigDon't see Juju very often but I've heard when it works it's pretty smooth.  Conversely when it doesn't work...21:40
trandlesnot yet in production, very much in development, but looking for more interested use cases21:41
oneswigtrandles: sounds like a beast alright.  Keep us updated :-)21:42
eloxoneswig: I think we are pretty much first out here but its living up to our expectations and more so.21:42
b1airojust got to the "or will when fully implemented" parentheses :-)21:42
eloxoneswig: We have developed a SLURM bundle that deploys SLURM to "any" cloud and is in progress of tailor it to be as close to a reference implementation we can get for HPC with SLURM.21:43
trandlesb1airo: haha, indeed...the local dev has taken some heat for that21:44
oneswigelox: good to hear it.  Is that open source?21:44
eloxAll open source.21:44
eloxoneswig: https://jujucharms.com/slurm-core/bundle/021:45
oneswigThanks elox - share and enjoy :-)21:45
*** mlavalle has left #openstack-meeting21:45
oneswigOK, let's move on...21:45
oneswig#topic Denver - SIG "help most wanted" forum21:45
*** openstack changes topic to "Denver - SIG "help most wanted" forum (Meeting topic: scientific-sig)"21:45
eloxoneswig: I was hoping to. It feels a bit risky being out in this end so its really in our self interest to both get eyes on what we are doing and helping others that might want to try give it a spin.21:46
oneswigI'm aware of one or two others in the Juju ecosystem but given it's a small world you probably know them already too21:47
oneswigLast airing for this one, Rico's looking for input on cross-SIG needs to advocate21:48
oneswig#link Scientific SIG wish-list taking shape here https://etherpad.openstack.org/p/scientific-sig-denver2019-gaps21:48
oneswigPlease contribute if you have pain points21:48
oneswigThere is a forum session in Denver where the hope is the SIGs will gather and find common needs21:49
oneswig#link Forum session in Denver https://www.openstack.org/summit/denver-2019/summit-schedule/events/23612/help-most-needed-for-sigs-and-wgs21:50
oneswigOK, that's enough on that I think21:50
oneswig#topic Denver SIG meeting and BoF21:50
*** openstack changes topic to "Denver SIG meeting and BoF (Meeting topic: scientific-sig)"21:50
oneswigI noticed the room we are in (507) is not looking particularly big on the map...21:51
oneswig#link map of the convention center https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c087f27a43eea/www-assets-prod/summits/26/locations/428/floors/OSDEN-Map-FINAL-sm2.png21:52
oneswigI have a photo from Berlin which I might share with the organisers to see if we can upgrade.21:52
oneswig#link lightning talks wanted - sign up here https://etherpad.openstack.org/p/scientific-sig-denver19-bof21:53
oneswigI'm hoping Arkady will offer a prize as usual!21:53
oneswigFinally - evening beer social21:54
oneswigLast summit (Berlin) we co-incided with the Ironic gang, which was great21:54
oneswigI saw this go by on the Ops thread and wondered if we should tag along with them on the Monday night - http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005096.html21:55
b1airogood idea re room upgrade oneswig21:55
*** ijw has quit IRC21:55
*** ijw has joined #openstack-meeting21:55
trandlescould be a repeat of Barca when we jammed into that tiny room21:56
b1airoalso a reminder we're still looking for lightning talks: https://etherpad.openstack.org/p/scientific-sig-denver19-bof21:57
oneswigThanks b1airo21:58
oneswigOK we are nearly at time...21:58
oneswigAOB items?21:58
b1airooneswig: i'm onboard for the operators drinks21:59
oneswigSounds good to me too (jetlag permitting) - you'll be raring of course... :-)21:59
trandlesI'll be arriving Sunday around noon if anyone wants to get an early start ;)22:00
b1airoi arrive (after ~24 hours in-transit) about 11pm on Sunday, so i should be sparkling...22:00
oneswigI'm due in Sunday arvo some time.  Also fried.22:00
b1airobut i'll be around until following Sunday before heading on to Montreal for CUG22:00
oneswigOK we are at the hour22:00
jmloweI'm in Sunday as well22:00
eloxthanx for putting the meeting together.22:00
jmloweand like to drink22:00
oneswigOn that happy note!22:01
trandlesb1airo: if you're interested some of us are going to a baseball game Friday and Saturday22:01
oneswigThanks all, until next time22:01
trandleso/ see you all in Denver22:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:01
openstackMeeting ended Tue Apr 16 22:01:23 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-04-16-21.00.html22:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-04-16-21.00.txt22:01
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-04-16-21.00.log.html22:01
martialWent well ;) (Sorry now at swim lesson with kids)22:10
*** ttsiouts has quit IRC22:10
*** iyamahat has quit IRC22:11
*** yamahata has quit IRC22:11
*** ttsiouts has joined #openstack-meeting22:11
*** munimeha1 has quit IRC22:12
*** ttsiouts has quit IRC22:12
*** trandles has quit IRC22:13
*** oneswig has quit IRC22:27
*** rcernin has joined #openstack-meeting22:38
*** whoami-rajat has quit IRC23:02
*** enriquetaso has quit IRC23:05
*** jamesmcarthur has quit IRC23:06
*** jamesmcarthur has joined #openstack-meeting23:07
*** jamesmcarthur has quit IRC23:11
*** bbowen has joined #openstack-meeting23:26
*** baojg has joined #openstack-meeting23:27
*** baojg has quit IRC23:31
*** Liang__ has joined #openstack-meeting23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!