Tuesday, 2018-10-16

*** hongbin has quit IRC00:16
*** ykatabam has joined #openstack-meeting00:27
*** annabelleB has quit IRC00:29
*** macza has joined #openstack-meeting00:37
*** jamesmcarthur has joined #openstack-meeting00:39
*** longkb has joined #openstack-meeting00:40
*** macza has quit IRC00:42
*** tetsuro has joined #openstack-meeting00:43
*** cloudrancher has quit IRC00:47
*** jamesmcarthur has quit IRC00:47
*** ssbarnea has joined #openstack-meeting00:48
*** ssbarnea|bkp2 has quit IRC00:48
*** cloudrancher has joined #openstack-meeting00:49
*** geguileo_PTO has quit IRC00:49
*** jamesmcarthur has joined #openstack-meeting00:51
*** lhinds has quit IRC00:51
*** jamesmcarthur has quit IRC00:52
*** jamesmcarthur has joined #openstack-meeting00:52
*** lhinds has joined #openstack-meeting00:55
*** slaweq has joined #openstack-meeting01:11
*** slaweq has quit IRC01:16
*** cloudrancher has quit IRC01:19
*** cloudrancher has joined #openstack-meeting01:20
*** yamahata has quit IRC01:24
*** yamamoto has quit IRC01:41
*** yamamoto has joined #openstack-meeting01:41
*** mhen has quit IRC01:52
*** armax has quit IRC02:03
*** hongbin has joined #openstack-meeting02:09
*** diablo_rojo has quit IRC02:17
*** jamesmcarthur has quit IRC02:18
*** YanXing_an has joined #openstack-meeting02:26
*** jamesmcarthur has joined #openstack-meeting02:27
*** cao_bin has joined #openstack-meeting02:41
*** cao_bin has quit IRC02:42
*** felipemonteiro has joined #openstack-meeting02:42
*** Liu has joined #openstack-meeting02:43
*** cloudrancher has quit IRC02:44
*** Liu has left #openstack-meeting02:45
*** cloudrancher has joined #openstack-meeting02:45
*** Liu has joined #openstack-meeting02:45
*** Liu has left #openstack-meeting02:48
*** cao_bin has joined #openstack-meeting02:50
*** LiuYuanyuan has joined #openstack-meeting02:53
*** LiuYuanyuan has left #openstack-meeting02:55
*** macza has joined #openstack-meeting02:55
*** cao_bin has quit IRC02:56
*** macza has quit IRC02:59
*** jamesmcarthur has quit IRC03:02
*** cloudrancher has quit IRC03:08
*** jamesmcarthur has joined #openstack-meeting03:08
*** cloudrancher has joined #openstack-meeting03:09
*** slaweq has joined #openstack-meeting03:11
*** bobh has joined #openstack-meeting03:14
*** slaweq has quit IRC03:16
*** jamesmcarthur has quit IRC03:19
*** bobh has quit IRC03:31
*** jamesmcarthur has joined #openstack-meeting03:45
*** YanXing_an has quit IRC03:46
*** hongbin has quit IRC03:48
*** Bhujay has joined #openstack-meeting03:51
*** bswartz has quit IRC03:51
*** Bhujay has quit IRC03:52
*** lpetrut has joined #openstack-meeting03:52
*** Bhujay has joined #openstack-meeting03:52
*** sagara has joined #openstack-meeting03:59
samP#startmeeting masakari04:00
openstackMeeting started Tue Oct 16 04:00:09 2018 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:00
*** openstack changes topic to " (Meeting topic: masakari)"04:00
openstackThe meeting name has been set to 'masakari'04:00
samPHi all for masakari04:00
samPsagara: Hi04:00
samP#topic bugs/patches04:01
*** openstack changes topic to "bugs/patches (Meeting topic: masakari)"04:01
samPI have merged some pathes for master, stable/queens, and stable/rocky04:01
samPpushed patch to release a new version for masakari-monitors for stable/queens04:02
sagarasamP: OK, thanks. Is their any high priority bug?04:03
*** felipemonteiro has quit IRC04:04
samPsagara: need to fix rootwrap bug in masakari-monitors04:04
*** tpatil has joined #openstack-meeting04:05
tpatilSorry to join late04:05
samPtpatil: hi04:05
tpatilsamP: Hi04:06
samPDiscussing about rootwrap bug in masakari-monitors04:06
tpatilsamP: I have noticed you have merged 3-4 back ported patches. Thank you04:06
*** Bhujay has quit IRC04:06
samPtpatil: np, working for release new versions for them04:07
tpatilsamP: Ok04:07
sagarasamP: About fix rootwrap bug in masakari-monitors, please let me know the link.04:07
samPsagara: Im trying to find the link for that bug report04:09
samPwith no luck04:09
samPI will share more details in mail later04:10
samPAny bugs or pathes need to discuss?04:10
*** slaweq has joined #openstack-meeting04:11
sagaraI also don't have any topic04:11
samPtpatil: sagara: thanks04:11
samP#topic Stein work items04:12
*** openstack changes topic to "Stein work items (Meeting topic: masakari)"04:12
samPAny update on stein work items04:12
*** macza has joined #openstack-meeting04:12
tpatilNotification specs implementation is in progress04:13
samPtpatil: thanks04:13
tpatilWorking on drafting event notification specs04:13
samPtpatil: thanks04:14
samPAbout the functional test,04:15
tpatilWe will push the functional tests written in openstacksdk for failover_segments and hosts in this week. For that, we will also add a CI job which will run masakari related functional tests.04:15
samPtpatil: thanks04:16
*** slaweq has quit IRC04:16
samPAs a extention of the discussion we had last week, I woul prefer to put functional test in masakari repo04:17
samPAlso, I think we need some of those tests in sdk siede too.04:17
tpatilin openstacksdk, we will add few CRUD functional tests for segments and hosts04:19
samPtpatil: great, otherwise, there is no way to know whether the new changes gonna break the masakari command04:19
tpatilin masakari, we can add all sorts of functional tests including notifications for process/hosts/instances04:20
samPtpatil: great04:20
samPtpatil: thanks04:20
samPAnyother topics to discuss?04:21
tpatilNot from my end04:21
samPOK then, no other topics from my side04:23
samPLet's finish the meeting.04:24
samPThank you for your time04:24
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:24
openstackMeeting ended Tue Oct 16 04:24:22 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:24
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-16-04.00.html04:24
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-16-04.00.txt04:24
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-16-04.00.log.html04:24
*** samP has left #openstack-meeting04:24
*** sagara has quit IRC04:24
*** lpetrut has quit IRC04:34
*** jamesmcarthur has quit IRC04:44
*** tpatil has quit IRC05:01
*** tetsuro has quit IRC05:04
*** macza_ has joined #openstack-meeting05:08
*** Douhet has quit IRC05:09
*** slaweq has joined #openstack-meeting05:11
*** macza has quit IRC05:12
*** macza_ has quit IRC05:12
*** macza has joined #openstack-meeting05:13
*** slaweq has quit IRC05:15
*** macza has quit IRC05:20
*** macza has joined #openstack-meeting05:20
*** macza has quit IRC05:24
*** apetrich has quit IRC05:40
*** Douhet has joined #openstack-meeting05:44
*** janki has joined #openstack-meeting05:47
*** apetrich has joined #openstack-meeting06:00
*** Luzi has joined #openstack-meeting06:01
*** cao_bin has joined #openstack-meeting06:01
*** slaweq has joined #openstack-meeting06:11
*** slaweq has quit IRC06:16
*** apetrich has quit IRC06:23
*** ykatabam has quit IRC06:35
*** apetrich has joined #openstack-meeting06:44
*** e0ne has joined #openstack-meeting06:48
*** tommylikehu has joined #openstack-meeting06:54
*** slaweq has joined #openstack-meeting06:55
*** aojea has joined #openstack-meeting06:57
*** rcernin has quit IRC07:01
*** lpetrut has joined #openstack-meeting07:20
*** cao_bin has left #openstack-meeting07:23
*** lpetrut has quit IRC07:25
*** markvoelker has quit IRC07:29
*** markvoelker has joined #openstack-meeting07:29
*** ralonsoh has joined #openstack-meeting07:29
*** ttsiouts has joined #openstack-meeting07:30
*** dkushwaha has joined #openstack-meeting07:31
*** markvoelker has quit IRC07:34
*** yamahata has joined #openstack-meeting07:42
*** aojea has quit IRC07:42
*** cao_bin has joined #openstack-meeting07:49
*** joxyuki has joined #openstack-meeting07:56
*** YanXing_an has joined #openstack-meeting07:58
dkushwaha#startmeeting tacker08:02
openstackMeeting started Tue Oct 16 08:02:58 2018 UTC and is due to finish in 60 minutes.  The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot.08:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:03
*** openstack changes topic to " (Meeting topic: tacker)"08:03
openstackThe meeting name has been set to 'tacker'08:03
dkushwaha#topic Roll Call08:03
*** openstack changes topic to "Roll Call (Meeting topic: tacker)"08:03
dkushwahawho is here for Tacker weekly meeting ?08:03
dkushwahajoxyuki, hi08:06
*** hadai has joined #openstack-meeting08:06
dkushwahacao_bin, hi08:07
dkushwahaSeems no more friends available today :)08:09
dkushwahaok lets start a brief meeting..08:12
dkushwaha#topic Announcements08:12
*** openstack changes topic to "Announcements (Meeting topic: tacker)"08:12
dkushwahaThis time we have only Project update and project onboarding session in berlin summit08:13
dkushwaha#link https://www.openstack.org/summit/berlin-2018/summit-schedule/global-search?t=tacker+-+08:13
dkushwahaStein1 milestone is in next week.08:14
dkushwaha#link https://releases.openstack.org/stein/schedule.html08:14
dkushwahaWe needs to focus on specs to finish them asap08:15
dkushwahaYanXing_an, hi08:15
dkushwaha#topic bp/mistral-monitor-policy08:16
*** openstack changes topic to "bp/mistral-monitor-policy (Meeting topic: tacker)"08:16
dkushwaha#link https://review.openstack.org/#/c/486924/08:17
dkushwahaI resumed my work in this, but facing some design related issues in conductor08:17
dkushwahaI will update patch with raw code today08:18
dkushwahaYanXing_an, and all needs your help to review the patch08:18
joxyukidkushwaha, what is the problem?08:20
YanXing_andkushwaha, both li_jia_le and i are busy on projects these weeks08:20
*** tetsuro has joined #openstack-meeting08:21
dkushwahajoxyuki, conductor is running as service and in action part, conductor not able to fatch mistralclient with error ¨service catalog is empty¨08:21
*** LiuYuanyuan has joined #openstack-meeting08:22
*** yikun has quit IRC08:22
dkushwahaas of now i tried hardcoded workaround, but will change them further08:22
dkushwahaYanXing_an, ok08:22
*** dmacpher_ has joined #openstack-meeting08:23
dkushwahajoxyuki, i will update the patch today with current code08:23
joxyukiI will have a look it.08:23
dkushwaha#topic bp/force-delete-resources08:23
*** openstack changes topic to "bp/force-delete-resources (Meeting topic: tacker)"08:23
dkushwaha#link https://review.openstack.org/#/c/602528/08:24
YanXing_andkushwaha, you mean conductor can not invoke mistral's api using client?08:24
dkushwahano update on this week.08:24
dkushwahaYanXing_an, right08:24
dkushwahaYanXing_an, and i am not thinking to the same approach as we used for shared vim08:25
dkushwahaneeds suggestions on that from you folks08:26
dkushwaha#topic bp/vdu-auto-healing08:26
*** openstack changes topic to "bp/vdu-auto-healing (Meeting topic: tacker)"08:26
YanXing_anis it due to the "service" context can not access the keystone to get the endpoint?08:26
dkushwahajoxyuki, i have just commented on patch08:27
dkushwaha#link https://review.openstack.org/#/c/603558/08:27
*** dmacpher has quit IRC08:27
dkushwahaYanXing_an, yes, and at the same time project_name is empty while creating new tocken08:28
joxyukidkushwaha, I think existing `respawn` action are useful.08:29
joxyukiSo I want to remain it as is.08:29
*** markvoelker has joined #openstack-meeting08:30
joxyuki`respawn` action can be a last resort of trouble-shooting.08:30
dkushwahajoxyuki, do you have any usecase/plan for that?08:30
*** priteau has joined #openstack-meeting08:31
joxyukiumm,,, no idea for now...08:32
YanXing_anwhy we need conductor invoke mistral? I think tacker-server will create a workflow to mistrial to monitor vnf, and conductor only do action, such as "respawn"08:32
dkushwahajoxyuki, ok lets keep it for now.08:33
dkushwahaYanXing_an, when we call ¨call_vnf_policy_action¨, it runs under conducter context  and call execute_action08:36
*** e0ne has quit IRC08:37
dkushwahaYanXing_an, lets wait to update patch today08:37
dkushwahaIt will be clear to discuss on that08:37
YanXing_andkushwaha, ok, i need some time to understand that usecase :)08:38
dkushwahaYanXing_an, yup sure08:39
dkushwaha#topic OpenDiscussion08:40
*** openstack changes topic to "OpenDiscussion (Meeting topic: tacker)"08:40
dkushwahacao_bin, hi08:40
dkushwahacould you please introduce yourself08:41
cao_binhi, i am from chinamobile, YanXing_an's colleague08:43
dkushwahacao_bin, great :)08:44
YanXing_ancao_bin, expecting on your first patch08:44
joxyukihi cao_bin :-)08:44
*** e0ne has joined #openstack-meeting08:44
cao_binhi, joxyuki08:45
*** hadai has quit IRC08:45
dkushwahawe are looking volunteer to help help test cases addition in tacker08:46
dkushwahaIt will be great help if somebody can work on that08:47
dkushwahathis is complete from me for this week.08:48
cao_bini'm glad to do it08:48
dkushwahaYanXing_an, joxyuki do you have some other topics to discuss?08:48
*** a-pugachev has joined #openstack-meeting08:49
dkushwahacao_bin, Thanks for raising your hand. It will be really great help08:49
cao_binIt's my pleasure08:50
dkushwahaOk, Thanks Folks..08:52
dkushwahaclosing this meeting now08:52
dkushwaha#endmeeting tacker08:52
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:52
openstackMeeting ended Tue Oct 16 08:52:56 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:52
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-16-08.02.html08:52
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-16-08.02.txt08:52
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-16-08.02.log.html08:53
*** joxyuki has left #openstack-meeting08:53
*** cao_bin has left #openstack-meeting08:54
*** LiuYuanyuan has left #openstack-meeting08:54
*** YanXing_an has quit IRC08:58
*** markvoelker has quit IRC09:03
*** yamahata has quit IRC09:06
*** persia has quit IRC09:27
*** persia has joined #openstack-meeting09:28
*** tommylikehu has quit IRC09:33
*** a-pugachev has quit IRC09:53
*** imacdonn has quit IRC09:54
*** imacdonn has joined #openstack-meeting09:55
*** a-pugachev has joined #openstack-meeting09:57
*** longkb has quit IRC09:59
*** markvoelker has joined #openstack-meeting10:00
*** tetsuro has quit IRC10:03
*** erlon has joined #openstack-meeting10:07
*** ttsiouts has quit IRC10:08
*** tetsuro has joined #openstack-meeting10:14
*** tetsuro has quit IRC10:16
*** nguyenhai has joined #openstack-meeting10:26
*** felipemonteiro has joined #openstack-meeting10:27
*** ttsiouts has joined #openstack-meeting10:29
*** nguyenhai has quit IRC10:30
*** nguyenhai has joined #openstack-meeting10:30
*** markvoelker has quit IRC10:34
*** macza has joined #openstack-meeting10:43
*** macza has quit IRC10:47
*** ykatabam has joined #openstack-meeting10:51
*** jamesmcarthur has joined #openstack-meeting10:56
*** ttsiouts has quit IRC11:02
*** ttsiouts has joined #openstack-meeting11:03
*** electrofelix has joined #openstack-meeting11:04
*** macza has joined #openstack-meeting11:04
*** ttsiouts has quit IRC11:07
*** macza has quit IRC11:09
*** dkushwaha has quit IRC11:16
*** ttsiouts has joined #openstack-meeting11:17
*** e0ne has quit IRC11:18
*** felipemonteiro has quit IRC11:20
*** ykatabam has quit IRC11:24
*** markvoelker has joined #openstack-meeting11:31
*** ttsiouts has quit IRC11:31
*** ttsiouts has joined #openstack-meeting11:32
*** e0ne has joined #openstack-meeting11:32
*** ttsiouts has quit IRC11:34
*** ttsiouts has joined #openstack-meeting11:35
*** ttsiouts has quit IRC11:37
*** ttsiouts has joined #openstack-meeting11:38
*** cloudrancher has quit IRC11:39
*** ttsiouts has quit IRC11:46
*** ttsiouts has joined #openstack-meeting11:47
*** ttsiouts has quit IRC11:47
*** ttsiouts has joined #openstack-meeting11:47
*** ttsiouts has quit IRC11:50
*** ttsiouts has joined #openstack-meeting11:51
*** ttsiouts has quit IRC11:55
*** ttsiouts has joined #openstack-meeting12:00
*** markvoelker has quit IRC12:04
*** yamamoto has quit IRC12:04
*** yamamoto has joined #openstack-meeting12:05
*** yamamoto has quit IRC12:05
*** yamamoto has joined #openstack-meeting12:05
*** dtrainor has joined #openstack-meeting12:14
*** electrofelix has quit IRC12:18
*** e0ne has quit IRC12:24
*** electrofelix has joined #openstack-meeting12:24
*** e0ne has joined #openstack-meeting12:27
*** felipemonteiro has joined #openstack-meeting12:32
*** ttsiouts has quit IRC12:32
*** raildo has joined #openstack-meeting12:35
*** cloudrancher has joined #openstack-meeting12:38
*** priteau has quit IRC12:39
*** priteau has joined #openstack-meeting12:40
*** dmacpher__ has joined #openstack-meeting12:46
*** dmacpher_ has quit IRC12:49
*** macza has joined #openstack-meeting12:53
*** macza has quit IRC12:58
*** bobh has joined #openstack-meeting13:04
*** efried has joined #openstack-meeting13:10
*** yamamoto has quit IRC13:10
*** yamamoto has joined #openstack-meeting13:11
*** e0ne has quit IRC13:12
*** yamamoto has quit IRC13:15
*** e0ne has joined #openstack-meeting13:16
*** mriedem has joined #openstack-meeting13:23
*** cloudrancher has quit IRC13:27
*** felipemonteiro has quit IRC13:27
*** awaugama has joined #openstack-meeting13:37
*** ociuhandu has joined #openstack-meeting13:37
*** ociuhandu has quit IRC13:38
*** mjturek has joined #openstack-meeting13:42
*** eharney has joined #openstack-meeting13:45
*** e0ne has quit IRC13:53
*** janki has quit IRC13:58
*** yamamoto has joined #openstack-meeting14:00
*** Luzi has quit IRC14:00
*** e0ne has joined #openstack-meeting14:01
*** yamahata has joined #openstack-meeting14:01
*** annabelleB has joined #openstack-meeting14:03
*** hongbin has joined #openstack-meeting14:05
*** munimeha1 has joined #openstack-meeting14:18
*** eharney has quit IRC14:33
*** armax has joined #openstack-meeting14:43
*** annabelleB has quit IRC14:45
*** annabelleB has joined #openstack-meeting14:45
*** eharney has joined #openstack-meeting14:48
*** felipemonteiro has joined #openstack-meeting14:55
*** manjeets_ has joined #openstack-meeting15:19
*** macza has joined #openstack-meeting15:39
*** macza has quit IRC15:44
*** felipemonteiro has quit IRC15:49
*** e0ne has quit IRC15:50
*** macza has joined #openstack-meeting15:57
*** mlavalle has joined #openstack-meeting15:58
*** macza has quit IRC15:58
*** macza has joined #openstack-meeting16:00
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Oct 16 16:00:08 2018 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
slaweqnjohnston: haleyb: are You around for CI meeting?16:01
haleybhi, missed reminder16:02
slaweqok, lets start then16:02
slaweq#topic Actions from previous meetings16:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:02
slaweqnjohnston look into fullstack compilation of ovs still needed on bionic16:02
slaweqI think njohnston is not around now so lets move to the next one16:03
slaweqslaweq will create scenario/tempest/fullstack/functional jobs running on Bionic in experimental queue16:03
slaweqI today pushed a patch for that https://review.openstack.org/61099716:03
slaweqit's in experimental queue now and I will check how it will go16:04
slaweqI will update status of it next week, ok?16:04
slaweq#action slaweq to continue checking how jobs will run on Bionic nodes16:05
slaweqok, next one:16:05
slaweqmlavalle will check failing trunk scenario test16:05
mlavalleI've been working on this bug all this morning16:05
mlavallewe have 33 hits over the past 7 days16:06
mlavalleMost of those (~27) are in neutron-tempest-plugin-dvr-multinode-scenario16:06
slaweqat least this job is non voting :)16:06
mlavalleand the failure occurs after creating the servers when trying to check connectivity over the fip16:07
mlavalleto one of the servers16:07
mlavallethe test hasn't tried to do anything with the trunk yet16:07
mlavalleother than creating the trunk16:07
mlavalleso at this point I suspect the fip wiring in dvr16:08
slaweqdid You check how long vm was booting? Maybe it's some timeout again?16:08
mlavalleno, the vms finish booting correctly16:08
mlavalleboth of them16:08
mlavalleand then we go and try to ssh to one of them using the fip16:09
mlavallechecking connectivity before starting to play with the trunk16:09
slaweqI think it is quite common cause of failures in scenario tests - FIP is not reachable16:09
mlavalleat this point it seems to me to be a generic failure of the fip16:10
slaweqnow the question is: how we can try to debug this issue?16:10
mlavallebut I am taking it as a good opporutnity to debug the neutron-tempest-plugin-dvr-multinode-scenario job16:10
haleybmlavalle: so you can't ping the FIP?16:11
mlavallehaleyb: we can't ssh to it16:11
*** cloudrancher has joined #openstack-meeting16:11
mlavalleat this point I am chacking to see if this has to do with in what node the instance / port lands16:12
mlavallecontroller vs compute16:12
haleybit sounds like the problem we were seeing before the revert... i guess we can start tracing the packets w/tcpdump, etc16:12
mlavallehaleyb: yesh, I suspect that16:12
mlavalleI need to work this a little bit longer, though16:13
*** macza has quit IRC16:13
slaweqmlavalle: yes, You can ask infra-root to set such job on hold and then if job will fail You will have ssh access to nodes16:13
*** macza has joined #openstack-meeting16:13
mlavalleslaweq: yeap, I might try that16:13
slaweqor do something "ugly" which I was doing when debugging dvr-multinode grenade issue16:13
mlavalleat this point I am gathering more evidence16:13
slaweqI send DNM patch where I was adding my ssh key to node16:14
slaweqand then You can e.g. add some sleep() in test code and login to nodes and debug16:14
slaweqwithout asking infra-root for holding job for You :)16:14
* mordred also has timeout issues with floating ip functional tests for openstacksdk - but I haven't ruled out just needing to rework those tests16:15
slaweqI wonder if this will still be not reachable after login to host or will all works fine then :)16:15
mlavallewill keep you posted mordred16:15
mordredmlavalle: thanks! also let me know if I can be of any help16:16
mlavallethat's as far as I've gotten with this. I'll continue working it16:16
slaweqthx mlavalle for working on this issue :)16:16
slaweq#action mlavalle to continue debugging issue with not reachable FIP in scenario jobs16:17
slaweqok, next one then16:18
slaweqslaweq will try to reproduce and triage https://bugs.launchpad.net/neutron/+bug/168702716:18
openstackLaunchpad bug 1687027 in neutron "test_walk_versions tests fail with "IndexError: tuple index out of range" after timeout" [High,In progress] - Assigned to Slawek Kaplonski (slaweq)16:18
mlavallewe got a fix, didn't we?16:18
slaweqPatch is in the gate already https://review.openstack.org/61000316:18
slaweqmlavalle: yes, it's kind of fix :)16:18
*** apetrich has quit IRC16:18
slaweqbut we should be good with it I hope16:18
slaweqand the last one from last week was:16:19
slaweqmlavalle to send an email about moving tempest plugins from stadium to separate repo16:19
mlavalledang, I forgot that one16:19
slaweqno problem :)16:19
mlavalleI'll do it at the end of this meeting16:19
slaweq#action mlavalle to send an email about moving tempest plugins from stadium to separate repo16:20
slaweqok, that's all from last week on my list16:20
slaweq#topic Grafana16:20
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:20
*** manjeets_ has quit IRC16:21
*** d0ugal has quit IRC16:21
slaweqfirst of all - we are much better with grenade-multinode-dvr job now :)16:21
slaweqfrom other things I see that functional tests were on quite high failure level recently16:22
slaweqbut it's related to this issue with db migrations which should be fine with my patch16:22
mlavalleoverall, the picture looks good to me16:23
*** d0ugal has joined #openstack-meeting16:23
slaweqyes, most of jobs are below 20% of failures16:23
slaweqso lets talk about few specific issues which I found last week and want to raise here16:24
*** yamahata has quit IRC16:24
slaweq#topic fullstack/functional16:24
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"16:24
slaweqabout fullstack tests, I found few times issue with cleanup processes16:24
slaweqso various tests but similar failure16:25
slaweqhere it looks that some process wasn't exited properly: http://logs.openstack.org/68/564668/7/check/neutron-fullstack-python36/c4223c2/logs/dsvm-fullstack-logs/TestOvsConnectivitySameNetwork.test_connectivity_GRE-l2pop-arp_responder,openflow-native_.txt.gz#_2018-10-16_02_43_49_75516:26
slaweqat least for me16:26
slaweqand in this example it looks that it is openvswitch-agent: http://logs.openstack.org/68/564668/7/check/neutron-fullstack-python36/c4223c2/logs/dsvm-fullstack-logs/TestOvsConnectivitySameNetwork.test_connectivity_GRE-l2pop-arp_responder,openflow-native_/neutron-openvswitch-agent--2018-10-16--02-42-43-987526.txt.gz16:27
slaweqI don't know if that is an issue with tests or with ovs-agent process really16:28
slaweqLooking at logs of this ovs agent it looks that there is no log like "Agent caught SIGTERM, quitting daemon loop." at the end16:30
slaweqit is in other agents' logs but not in this one16:30
mlavalleso, are these cleanup issues causing tests to fail?16:30
slaweqmlavalle: yes, it looks that test if waiting for process to be exited and then timeout exception is raised16:30
*** apetrich has joined #openstack-meeting16:30
mlavallethat makes sense16:30
*** tssurya has joined #openstack-meeting16:31
slaweqI was thinking that I will open a bug for that to not forget about this and maybe there will be some volunteer to work on it :)16:32
slaweqif I will have some cycles, I will try to debug it more16:32
mlavallethat's a good idea16:32
mlavalleopen the bug and if nobody volunteers, please remind us in the next meeting16:33
slaweq#action slaweq to report a bug about issue with process ending in fullstack tests16:33
mlavalleif I'm done with the trunk one16:33
*** diablo_rojo has joined #openstack-meeting16:33
slaweqok, thx mlavalle16:33
mlavalleand there is nothing more urgent, I'll take it16:33
slaweqI will remind it for sure if will be necessary16:34
slaweqok, lets move on to scenario jobs now16:34
slaweq#topic Tempest/Scenario16:35
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"16:35
slaweqfirst of all we still have this bug: https://bugs.launchpad.net/neutron/+bug/178943416:35
openstackLaunchpad bug 1789434 in neutron "neutron_tempest_plugin.scenario.test_migration.NetworkMigrationFromHA failing 100% times" [High,Confirmed] - Assigned to Manjeet Singh Bhatia (manjeet-s-bhatia)16:35
slaweqand I know that manjeets is working on it16:35
slaweqany updates manjeets?16:35
slaweqok, I guess no16:37
slaweqso let's move on16:37
slaweqin neutron-tempest-plugin-scenario-linuxbridge I spotted at least two times issue with failed to transition FIP to down16:37
slaweqfor example:16:38
slaweqhaleyb: I think You were debugging similar issue some time ago, right?16:39
haleybslaweq: yes, and i thought the change merged16:39
slaweqand You found that this BUILD state is coming from nova16:39
*** a-pugachev has quit IRC16:39
slaweqI think Your patch was merged, right16:39
slaweqbut it looks that this problem still happens16:39
haleybit's actually the state on the associated compute port16:39
haleybi think it did merge as that looks like the new code16:40
mlavalleit merged16:40
mlavalleI remember we got stuck a little with naming a function16:41
*** zaneb has quit IRC16:41
slaweqand what is in logs, is that this FIP was set to DOWN here: http://logs.openstack.org/97/602497/5/gate/neutron-tempest-plugin-scenario-linuxbridge/52c2890/controller/logs/screen-q-svc.txt.gz#_Oct_16_01_28_04_49827916:41
slaweqand test failed around 2018-10-16 01:27:59,89316:41
slaweqmaybe node was overloaded and something took longer time simply?16:42
haleybthe test still checks the port the FIP is attached to, not the FIP itself, so maybe it was not the best fix?16:42
haleybwe wait 120 seconds16:42
slaweqhaleyb: can I assign it to You as an action for next week?16:43
slaweqwill You be able to take a look on it?16:43
haleybslaweq: sure16:43
slaweqthx a lot16:43
slaweq#action haleyb to check issue with failing FIP transition to down state16:43
slaweqbasically I didn't found other failures related to neutron in jobs which I was checking16:44
slaweqthere are some issues with volumes and some issues with host mapping to cells in multinode jobs but that's not neutron issues16:45
slaweqI have one more short thing to mention about scenario jobs16:45
slaweqI have patch to make such jobs run faster: https://review.openstack.org/#/c/609762/16:45
slaweqbasically I proposed to remove config option "is_image_advanced" and use "advanced_image_ref" instead16:46
slaweqand use this advanced image only in tests which really needs it16:46
slaweqwe have 3 such tests currently16:46
slaweqall other tests will use Cirros image16:46
slaweqcomparing jobs' running time:16:47
slaweqneutron-tempest-plugin-dvr-multinode-scenario - 2h 11m 27s without my patch and 1h 22m 34s on my patch16:47
*** a-pugachev has joined #openstack-meeting16:47
slaweqneutron-tempest-plugin-scenario-linuxbridge - 1h 39m 48s without this patch and 1h 09m 23s with patch16:48
slaweqso difference is quite big IMO16:48
slaweqplease review this patch if You will have some time :)16:48
*** a-pugachev has quit IRC16:48
mlavallewill do16:48
*** cloudrancher has quit IRC16:48
haleybme too16:48
slaweqok, that's all from my side for today16:49
slaweq#topic Open discussion16:49
*** openstack changes topic to "Open discussion (Meeting topic: neutron_ci)"16:49
slaweqanyone wants to talk about something?16:49
clarkbaccording to e-r, http://status.openstack.org/elastic-recheck/data/integrated_gate.html, there are some neutron fullstack failures too (the functional job at the top of the list should have a fix out there now)16:49
*** annabelleB has quit IRC16:50
manjeetsslaweq sorry I was on other internal meeting as i discussed with I need to get into l3 and l2 agents code to point the notification issue16:50
clarkbmore of an fyi than anything else since you were talking about neutron fails above16:51
slaweqclarkb: we already talked about issue like is e.g. in http://logs.openstack.org/04/599604/10/gate/neutron-fullstack/3c364de/logs/testr_results.html.gz16:51
*** annabelleB has joined #openstack-meeting16:51
slaweqand it looks that it's the reason of 3 of 4 issues pointed there16:51
slaweqand those 2: http://logs.openstack.org/03/610003/2/gate/neutron-fullstack-python36/b7f131a/logs/testr_results.html.gz and http://logs.openstack.org/61/608361/2/gate/neutron-fullstack-python36/39f72ae/logs/testr_results.html.gz16:52
slaweqlooks at first glance that it's again the same culprit16:52
slaweqtest fails as it tried to kill agent and that was not finished with success16:53
*** yamahata has joined #openstack-meeting16:53
slaweqso as I said, I will open a bug report for that for now, and will try to debug it if I will have time for it16:53
clarkbok, you might also want to add e-r queries for known issues if they will persist for longer than a few days16:54
clarkbhelps others identify problems that have direct effect on our ability to merge code and hopefully motivates people to fix them too :)16:54
slaweqclarkb: ok, I will add it, thx16:54
mlavalleclarkb: thanks for bringing this up16:54
mlavallethis is the perfect forum for CI issues16:54
mlavalleand we meet every week at this time16:55
mlavalleyou are always welcome16:55
slaweq#action slaweq to add e-r query for know fullstack issue (when bug will be reported)16:55
*** chkumar|off has joined #openstack-meeting16:56
slaweqanything else or can we finish few minutes before time? :)16:56
mlavallenot from me16:56
*** chkumar|off is now known as chandankumar16:57
slaweqthx for attending16:57
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:57
openstackMeeting ended Tue Oct 16 16:57:39 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:57
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-16-16.00.html16:57
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-16-16.00.txt16:57
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-16-16.00.log.html16:57
*** jamesmcarthur has quit IRC17:00
*** jamesmcarthur has joined #openstack-meeting17:00
*** jesusaur has quit IRC17:02
*** kopecmartin|ruck is now known as kopecmartin|off17:05
*** annabelleB has quit IRC17:07
*** e0ne has joined #openstack-meeting17:08
*** mlavalle has left #openstack-meeting17:08
*** munimeha1 has quit IRC17:10
*** whoami-rajat has quit IRC17:13
*** ralonsoh has quit IRC17:28
*** zaneb has joined #openstack-meeting17:33
*** electrofelix has quit IRC17:41
*** e0ne has quit IRC17:59
*** annabelleB has joined #openstack-meeting18:01
*** diablo_rojo has quit IRC18:03
*** diablo_rojo has joined #openstack-meeting18:10
*** e0ne has joined #openstack-meeting18:15
*** gyee has joined #openstack-meeting18:19
*** e0ne has quit IRC18:22
*** annabelleB has quit IRC18:29
*** annabelleB has joined #openstack-meeting18:37
*** jamesmcarthur has quit IRC18:38
*** jamesmcarthur has joined #openstack-meeting18:52
*** tssurya has quit IRC18:52
*** Shrews has joined #openstack-meeting18:54
*** bobh has quit IRC18:54
*** bobh has joined #openstack-meeting18:56
*** jamesmcarthur has quit IRC18:56
*** ssbarnea_ has joined #openstack-meeting18:57
*** gagehugo has joined #openstack-meeting18:58
clarkbanyone else here for the infra meeting?18:59
clarkbI'll give it a few minutes for people to trickle in18:59
gary_perkinsHi o/18:59
corvusi am!  and have eaten my sandwich *before* the meeting today18:59
fungihere taking a break from housework19:00
*** annabelleB has joined #openstack-meeting19:00
*** bobh has quit IRC19:01
clarkb#startmeeting infra19:01
openstackMeeting started Tue Oct 16 19:01:07 2018 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting19:01
clarkbI cleaned up the meeting agenda based on what was discussed during last meeting and what we have going on this week.19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
clarkbI'm likely to be afk tomorrow enjoying some of the last "good" weather before winter invades. My brothers demand I go fishing with them19:02
clarkbfungi is also afk dealing with things like sea grass aiui19:02
corvusfungi: can you see a lot of sea grass from the sea shore?19:02
fungimost of the sea grass is back outside again where it belongs19:02
clarkb#link https://www.openstack.org/summit/berlin-2018/summit-schedule/#track=262 Berlin Forum schedule up for feedback19:02
clarkbAs we get closer to the summit and forum the schedule is up and probably worth looking over for any hard to manage conflicts19:03
*** prometheanfire has joined #openstack-meeting19:03
ssbarnea_i am gere19:03
clarkb#topic Actions from last meeting19:04
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:04
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-09-19.00.txt minutes from last meeting19:04
clarkbwe don't appear to have recorded any formal actions, but ssbarnea_ had some items to talk about. I wanted to follow up and ask if we need to talk about any more of those today?19:05
clarkbssbarnea_: ^ or is there any other input you need to make some of those actionable?19:05
fungiworth discussing the suggestion of git-review prereleases if it's not been added to the agenda yet19:05
ssbarnea_clarkb: nope, not for now.19:05
fungior can bring it up during open discussion at the end19:06
clarkbfungi: and adding new cores. We should have time to go through that during the end of the meeting19:06
clarkb#topic Specs approval19:06
*** openstack changes topic to "Specs approval (Meeting topic: infra)"19:06
clarkbI approved the third party ci direction spec last week and it merged19:06
clarkbwe can now point to that when interested parties want to get involved running third party ci setups19:06
clarkbthe other spec people should be aware of (but is still WIP) is the storyboard attachments spec19:07
clarkb#link https://review.openstack.org/#/c/607377/ Storyboard Attachments Spec. Would be good toget input on that from an operational perspective. Concerns like overal disk usage and not wanting to host spam etc19:07
clarkb#topic Priority Efforts19:08
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:08
clarkb#topic Update Config Management19:09
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:09
clarkb#info topic:puppet-4 and topic:update-cfg-mgmt to find changes to review19:09
clarkbI've got all but one of the outstanding futureparser changes merged at this point. Thanks to cmurphy for continuing to push on that19:09
clarkbThere are other outstanding puppet-4 related changes to make it possible to futureparser more things19:10
clarkbmordred: has started looking at deploying gerrit with containers and how a gerrit 2.14/2.15 might fit into that19:10
clarkbianw: is looking at graphite deployment from containers too aiui19:10
clarkbFor the continuous deployment work I think we are still waiting on https://review.openstack.org/#/c/604925/719:11
clarkbcmurphy: mordred ianw corvus any other related work you'd like to mention?19:11
corvusno, it's back-burner for me right now, but soon i'd like to catch up with ianw on container stuff19:13
mordredoh look. it's the infra meeting19:13
* mordred sucks at clocks19:13
corvussince i've spent a week dockering docker dockers i may be able to docker docker more docker than i could docker before dockering19:13
clarkbthat is a lot of docker19:14
cmurphyclarkb: i had set that aside for a bit so now that more future parser changes are in i'll try to finish the list19:14
corvuson second thought, i may not be any help19:14
mordredhttps://review.openstack.org/#/c/610395/ <-- there is first stab at building a docker container of gerrit 2.1519:14
clarkb#link https://review.openstack.org/#/c/610395/ Docker containers for gerrit 2.1519:14
cmurphyclarkb: i updated https://review.openstack.org/#/c/576262/ but fungi had already +2'd the version before19:14
ianwi was looking at reusing mostly an upstream thing, it will be good to have it being looked at from a few different angles19:15
clarkbcmurphy: thanks I think between fungi ianw and myself we should be able to get htat one in shortly19:15
clarkbfungi: ianw can you help review https://review.openstack.org/#/c/576262/ since you've reviewed it before?19:15
fungiyou bet19:15
cmurphyalso wondering about the state of ask.o.o because last i looked at it ask-staging was turned off so we didn't want to experiment on ask.o.o19:15
clarkbcmurphy: ya I think ianw had some stuff in motion around that, we should resync on that and make sure we are making progress (or at least not holding up other efforts)19:16
clarkbthat might be good for after the meeting though?19:16
mordredit's worth noting that gerrit 2.14 does not build for  me locally - so am looking in to the idea of using a pure upstream built war for the 2.13->2.14 - and then _immediately_ upgrade to 2.1519:16
ianwyeah, let's sync.  i had a memory of turning ask-staging into a xenial host19:16
mordredrather than spend time fighting the outdated 2.14 build to make an artifact we don't really care about long term19:17
corvusclarkb: what happened to the idea of getting the authorized keys from a url?  (maybe with a new ansible module?)19:17
cmurphyand one other thing is that i think the refstack module is broken, i haven't looked into it too deeply but i think someone needs to help get https://review.openstack.org/#/c/557539/ into shape so that we can have tests for it19:18
clarkbcorvus: its done https://review.openstack.org/#/c/604932/ is child change to the other change because I figured we'd want more review on this one than the simpler first change19:18
corvusclarkb: thx19:18
clarkb#link https://review.openstack.org/#/c/557539/ Needs to be brought into shape so that we can test refstack and get it future parsered19:19
clarkbTo summarize there are a number of efforts in progress largely blocked on reviews. If you can find time to review those two topics that were #info'd above that would be much appreciated19:20
clarkbAnything else before we jump to the next item?19:21
clarkb#topic Storyboard19:21
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:21
clarkbAs mentioned the WIP spec for story attachments could use our input.19:22
clarkbThis is a feature often requested by users since paste.o.o has a paste size limit and logs.o.o rotate out logs relatively quickly19:22
clarkbfungi: diablo_rojo anything else in storyboard land we should be aware of?19:23
diablo_rojoclarkb, nothing right now19:23
diablo_rojothe WIP spec was all I had19:23
diablo_rojoand you already mentioned that :)19:23
fungii'm out of touch wrt sb this week19:23
*** weshay has joined #openstack-meeting19:24
fungiwe just had another request (this time from prometheanfire) for constraining the project and project group story views to base theri activity decision only on tasks for the selected project(s)19:24
fungithat's probably a low-hanging-fruit thing for sb though i'm not sure whether anyone's opened a story for it yet19:24
diablo_rojofungi, not that I know of19:25
clarkbprometheanfire: ^ you might consider opening a story for this item19:25
*** jamesmcarthur has joined #openstack-meeting19:25
clarkb#topic General Topics19:26
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:26
clarkb#topic OpenDev19:26
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:26
clarkbI managed to find a time for us to meet with the foundation leadership/marketing staff and that time is 1800UTC Monday October 2219:26
clarkbI know this is likely not ideal for everyone but when trying to schedule around foundation travel it was a reasonable option that popped up in the near future19:27
clarkbif you'd like to talk opendev messaging please join us in #openstack-infra at that time19:27
clarkbcorvus: I was also going to ask about the dns bootstrapping. There are changes that need review for that?19:27
corvusclarkb: my most recent re-attempt failed and i haven't looked at why yet: https://review.openstack.org/60509219:28
clarkb#link https://review.openstack.org/#/c/605092/ bootstrap opendev dns servers19:28
corvusthat is closer to the top of my list, but not quite there yet; so if someone has a minute to find/fix that problem, is okay with me19:28
clarkbwith the new etherpad servers I had considered calling them "etherpad-dev01.opendev.org" and similar but realized we need to update the system config site.pp to support that19:29
clarkbany thoughts on maybe updating the regexes in there across the board to make it easy going forward?19:29
clarkbwe could do something like foo\.(opendev|openstack).org19:30
mordredclarkb: foo.open.*.org :)19:30
clarkbmordred: ya that might be simpler :)19:30
corvusclarkb: do you think we will run into $fqdn problems?19:30
clarkbcorvus: I don't think so because most (maybe all at this point?) of our services require us to set the server name in apache due to the 01/02/03 etc hosts names19:31
corvusk, then i think that should be fine19:31
clarkbso we can call the server foo.opendev.org in nova then point foo.openstack.org dns at the service with apache config set to foo.openstack.org19:31
*** bobh has joined #openstack-meeting19:32
mordredclarkb: do we want to make apache redirect .openstack.org to .opendev.org once we have both dns entries? or just let the vhost serve as either name?19:32
clarkbmordred: I think we'll end up tackling that on a case per case basis depending on the service? for etherpad I expect we'll redirect openstack.org to opendev.org19:33
clarkbgit.openstack.org the opposite as that provides an identity to oepnstack and others19:34
mordredclarkb: nod. good point19:34
clarkband we'll have to work through that over time. I do think as much as possible the hostnames should be opendev.org though then we can do the majority of dns management through the code reviewed dns system19:34
mordredclarkb: ++19:34
*** jamesmcarthur has quit IRC19:35
corvusi think we'll need to talk more about git.*19:35
corvusif we're going to focus on opendev as a brand/service/hosting platform, it may be counterproductive19:35
mordredagree (that we need to talk more about it - I don't have a strong opinion one way of the other)19:36
clarkbya the details there are definitely worth working out and probably a good topic for the messaging meeting as the dns records take a part in the messaging19:36
corvusi'm comping from the pov that anything which confuses the 'canonical' repo name is bad -- this makes things confusing for zuul and other systems/folks that use golang format repo names19:36
*** bobh has quit IRC19:37
clarkbcorvus: we may need to consider the things that k8s does too in that case (I don't think they are the only one that alias github)19:37
mordredcorvus: yah. otoh - our fine friends at k8s store their source code in a neutral branded location (github) and use weird aliasing things to make their golang paths be based on k8s.io ...19:38
mordredclarkb: jinx19:38
corvusmaybe we need to make zuul smarter about supporting per-project (rather than per-connection) canonical names19:38
mordredwhich is to say - there are a pile of use cases / requirements here that we should think carefully about19:38
clarkbalright anyhting else on opendev before we go to the next item?19:39
corvuson the face of it, if we go forward with the status quo of git.opendev.org and git.openstack.org both existing without any other changes, i think we'll be in a bad spot.  so we should do something.  :)19:39
clarkbcorvus: might also help to write down the concerns and constraints? maybe as a followup to my schedulign email for the messaging meeting? (not sure thats the best venue but it is time set aside to talk about related stuff)19:40
fungii just assumed we're going to need to do a lot of project renames into another namespace to "move" repos from git.openstack.org to git.opendev.org19:40
fungiperhaps map namespaces to domains 1:119:41
fungibut this discussion can come later19:41
clarkb#topic Upgrade sprint19:41
*** openstack changes topic to "Upgrade sprint (Meeting topic: infra)"19:41
clarkbLets keep the meeting moving. About a month ago I scribbled this week in as an upgrade sprint week. I've been focused on getting etherpad servers updated. The dev server is now running on xenial and I expect to get the prod server to xenial this afternoon19:42
clarkbthis week worked out as it is between conferences and not in a bad spot for the release cycle19:42
clarkbI expect I'll look at logstash.o.o on thursday19:43
clarkbIf others can grab a service/server or two and work on figuring out what platform upgrades look like that would be helpful to19:43
clarkb#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup server upgrade todo list19:43
mordredyou know - I'll take storyboard - that one seems potentially fun19:44
clarkbmordred: thanks.19:44
clarkband do ping me if you need help with reviews or puppet or ansible. I actually started the week fixing ansible so that puppet would run :)19:45
* mordred warns diablo_rojo that he's likely about to start screwing stuff up for everybody19:45
clarkb#topic Zuul Queue Backlogs19:45
*** openstack changes topic to "Zuul Queue Backlogs (Meeting topic: infra)"19:45
mordredclarkb: oh - I have a (potentially) related question that's also potentially related to the opendev topic ...19:45
mordred(sorry, on the previous topic)19:46
openstackRemoving item from minutes: #topic Zuul Queue Backlogs19:46
clarkbmordred: go for it19:46
*** raildo has quit IRC19:46
mordredclarkb: a few weeks ago I grabbed opendevorg dockerhub namespace so we could publish images we build for our services19:46
mordredI put the creds into the password file - but haven't added a secret for it yet ...19:46
mordredare we good with that as a location?19:46
mordred(opendev is taken)19:46
clarkbworks for me19:47
mordredkk. cool. just realized we hadn't actually talked about it :)19:47
corvusmordred: is opendev used?19:47
corvusor just squatted?19:47
mordredcorvus: just squatted19:47
prometheanfirehttps://opendev.com is taken19:47
corvusmordred: maybe we can also ask dockerhub about a transfer?  probably best after we've actually pushed an image up :)19:48
corvusthen we can demonstrate an actual intent to use it19:48
clarkbcorvus: ya we don't want to be squatters asking for other squatter namespaces :)19:48
mordredcorvus: totes - it's def worth asking19:48
clarkb#topic Zuul Queue Backlogs19:48
*** openstack changes topic to "Zuul Queue Backlogs (Meeting topic: infra)"19:48
clarkbreally quickly. Wanted to point out that we continue to be aware of this and are working on various related things to try and make it better19:49
clarkbianw has been looking at the port leaks in ovh with slaweg and dpawlik19:49
clarkbovh is aware of the problems there and is apparntly working to correct it19:49
ianwok, i've had to disable gra1 because it doesn't seem to boot servers19:50
corvusclarkb: the node-accounting-in-logs change has merged; needs a scheduler restart.  that would let us more accurately get a gauge for what projects/jobs are using the system the most.19:50
clarkbwe merged a zuul gate window floor value change to 10 to see if that helps, initial data is that it doesn't but I figured it was worht a shot19:50
clarkbI continue to push projects to fix the bugs in their tests and update e-r as necessary to better track longer term issues19:50
clarkbOverall I think we are headed in the right direction it is just a slow turning ship19:51
clarkbcorvus: thanks, I'll keep that in mind (restart in particular)19:51
clarkbat this point a lot of the flakyness is outside of our control I think. Looking at e-r data jobs are failing for non infra valid reasons19:51
clarkbso other than adding capacity to reduce the impact of ^ we are largely counting on projects to identify and fix their test bugs19:52
clarkb#topic git-review19:52
*** openstack changes topic to "git-review (Meeting topic: infra)"19:52
corvusclarkb: the gate floor reduction not helping the check backlog seems to suggest supply/demand is important19:53
clarkbcorvus: yup19:53
clarkbfungi: ssbarnea_ quickly before we run out of time. There are two git-review items. First is psuhing prerelease git review tags so people can opt in to testing changes to git-review before the rest of the world sees them19:53
fungithis seems like a no-brainer19:53
fungii can push one today and will happily push another any time someone asks19:54
clarkbthe other is adding core reviewers. stephenfin and ssbarnea_ have offered to be new core reviewers on git-review. I'd like to add them in with an agreement that "stabilizing" git review is the goal. Update tests, fix known bugs, etc19:54
fungimight make it slightly easier for people who aren't comfortable cloning the repo and doing a pip install .19:54
*** jamesmcarthur has joined #openstack-meeting19:54
clarkbthen if we end up in a place where git-review is stable wtih reliable testing reconsider adding features19:54
clarkbis there any objection to adding them if we agree to that? (I think that should be global git-review dev agrement not just for the new core reviewrs)19:55
ssbarnea_fungi: yeah, installing with "pip install --pre git-review" is more friendly IMHO19:55
clarkband ya I htink pushing pre release git reviews is a great idea19:55
fungiadding a couple more git-review-core reviewers sounds like a fine plan to me, curious what others think19:55
*** jamesmcarthur has quit IRC19:56
*** jamesmcarthur has joined #openstack-meeting19:56
ianw++ to new reviewers19:57
clarkbwe are almost out of time. But please let me knwo if you have concerns about adding new core reviewrs to git review in this way19:57
clarkb#topic Open Discussion19:57
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:57
clarkband now ~2 minutes for anything else19:57
corvusi'm happy with new reviewers if we're all on board with it being a really stable tool that doesn't change much and not a new feature magnet19:58
corvusand to be really clear about that, i think any change which requires us to alter our contributor docs should have a nearly impossible hill to climb for acceptance.19:59
fungiunless the alteration is "remove some unnecessary things we've automated" maybe19:59
corvusbut new features like supporting the wip plugin, etc, would be great20:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"20:00
openstackMeeting ended Tue Oct 16 20:00:30 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-16-19.01.html20:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-16-19.01.txt20:00
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-16-19.01.log.html20:00
*** prometheanfire has left #openstack-meeting20:00
clarkbour hour is up. Find is in #openstack-infra for more discussion or on the openstack-infra@lists.openstack.org mailing list20:00
clarkbthank you everyone20:00
fungithanks clarkb!!!20:00
gary_perkinscheers clarkb :)20:01
ssbarnea_thanks! time to close the lid for me ;)20:01
clarkbgary_perkins: hopefully ianw's response to the email about credentials gives you what you were looking for?20:01
clarkbgary_perkins: feel free to followup as needed if not20:01
gary_perkinsclarkb: yep, it's fine. thnaks :)20:02
gary_perkinsthanks even!20:02
*** b1air has joined #openstack-meeting20:02
*** mjturek has quit IRC20:04
ianwgary_perkins: cool, i'll have a look at that now20:04
*** bobh has joined #openstack-meeting20:06
*** Shrews has left #openstack-meeting20:07
*** bobh has quit IRC20:12
*** jamesmcarthur has quit IRC20:12
*** annabelleB has quit IRC20:16
*** annabelleB has joined #openstack-meeting20:18
*** jamesmcarthur has joined #openstack-meeting20:19
*** macza has quit IRC20:25
*** macza_ has joined #openstack-meeting20:25
*** macza_ has quit IRC20:28
*** macza has joined #openstack-meeting20:29
*** macza has quit IRC20:31
*** macza has joined #openstack-meeting20:32
*** slaweq has quit IRC20:33
*** bobh has joined #openstack-meeting20:38
*** ssbarnea has quit IRC20:42
*** ssbarne__ has joined #openstack-meeting20:42
*** ssbarnea has joined #openstack-meeting20:42
*** ssbarnea_ has quit IRC20:44
*** erlon has quit IRC20:48
*** jesusaur has joined #openstack-meeting20:48
*** awaugama has quit IRC20:51
*** bobh has quit IRC20:52
*** oneswig has joined #openstack-meeting20:55
*** martial__ has joined #openstack-meeting20:55
*** priteau has quit IRC20:57
*** ssbarne__ has quit IRC20:57
martial__Hello team20:59
oneswigG'day martial__21:00
oneswigWould you like to drive today? :-)21:00
martial__#startmeeting Scientific-SIG21:00
openstackMeeting started Tue Oct 16 21:00:32 2018 UTC and is due to finish in 60 minutes.  The chair is martial__. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: Scientific-SIG)"21:00
openstackThe meeting name has been set to 'scientific_sig'21:00
*** priteau has joined #openstack-meeting21:00
oneswig#link agenda for today https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_October_16th_201821:01
oneswigIt's somewhat short today21:01
*** mriedem is now known as mriedem_away21:02
martial__(shoot too late was looking for that :) )21:02
oneswigmartial__: are you travelling today?21:02
*** janders has joined #openstack-meeting21:02
*** trandles has joined #openstack-meeting21:02
jandersg'day - sorry for being late21:02
martial__#topic Topics for the agenda for Berlin21:03
*** openstack changes topic to "Topics for the agenda for Berlin (Meeting topic: Scientific-SIG)"21:03
trandlesI'm here, sorta21:03
*** eharney has quit IRC21:03
oneswighey b1air janders trandles21:03
b1airI was an hour early :-(21:03
martial__stig: nope am talking to a colleague at NIST who is looking at deploying an OpenStack for Public Safety type work21:03
martial__am trying to get him to meet people and come to Berlin :)21:03
oneswigb1air: always the keener21:04
martial__Will introduce him after, maybe he can setup a couple meetings with people to see how to best do his work :)21:04
b1airNot sure which timezone I'm in, but it's not the same as my calendars21:04
oneswigmartial__: be very happy to meet them21:04
oneswigb1air: you're in Sydney?21:04
*** annabelleB has quit IRC21:05
oneswigSo - we should get on track I guess.21:05
jandersyeah daylight savings made the evening session tricker, but the morning session is so much easier now :)21:05
martial__(see guest speaker and all that :) )21:05
*** munimeha1 has joined #openstack-meeting21:05
oneswigWe have SIG sessions in the schedule now21:06
martial__So team, the dates have been set21:06
martial__Stig, you go :)21:06
oneswigThey are not consecutive, but at least on the same day (and in the right order)21:06
oneswigVery happy to see it, in the back of my mind I had a nagging feeling I needed to jump through one more hoop to get the paperwork in order :-)21:07
martial__have you thought of your co-hosts? :)21:08
oneswigmartial__: fancy a video link? :-)21:08
oneswigI hadn't thought of co-hosts tbh21:08
martial__if it is not at the same time as the SC18 panels/BoF or similar, I am happy to participate if I can21:09
oneswigDallas - I think it may be 5am...21:09
martial__For you, sure21:09
martial__Which reminds me, the dates in questions are21:09
martial__SIG Meeting Wed 14, 11:00am - 11:40am21:10
martial__SIG BoF Wed 14, 1:40pm - 2:20pm21:10
martial__SIG Social Wednesday evening, TBD21:10
martial__(that was interesting)21:10
trandlesthose times are all CET?21:10
oneswigAs in, Central European Time, I am not sure what they do for DST in Germany21:11
jandersI just looked it up, in November it should be CET21:12
janderssummer would be CEST21:12
trandlesUS switches to standard time night of November 3-421:12
oneswigOur summer is not so long and neither is our summer time :-)21:12
trandlesso Berlin should be 7 hours ahead of Dallas21:12
trandlesI guess we'll see just how dedicated martial__ is ;)21:13
oneswighmmm. I know martial__ is a night owl but that's maybe too far!21:13
martial__5am ... no prob-zzzzzzzz21:14
jandersBerlin (Germany - Berlin) Wednesday, 14 November 2018 at 11:00:00 am CET UTC+1 hour   Dallas (USA - Texas)      Wednesday, 14 November 2018 at 4:00:00 am  CST UTC-6 hours  Corresponding UTC (GMT)   Wednesday, 14 November 2018 at 10:00:0021:14
oneswigb1air: martial__: I shall expect a video message as minimum (recorded in some Dallas bar, perhaps)21:14
b1airJust need to find a bar that stays open 24 hours ;-)21:14
martial__I can set up the GoToMeeting link even ;)21:14
martial__actually why not ...21:14
jandersBerlin (Germany - Berlin) Wednesday, 14 November 2018 at 2:00:00 pm CET UTC+1 hour   Dallas (USA - Texas)      Wednesday, 14 November 2018 at 7:00:00 am CST UTC-6 hours  Corresponding UTC (GMT)   Wednesday, 14 November 2018 at 13:00:0021:14
jandersat least BoF falls within more civilised hours21:15
martial__Sitg, let's talk after the end of the meeting21:15
*** a-pugachev has joined #openstack-meeting21:15
martial__Btw, Jim said he can speak about his cloud approach to Public Safety, so we have a guest speaker :)21:15
oneswigsounds good.  The final matter on the SIG events is that Nick from our team has offered/been conscripted to get us a beer social on Wednesday evening.21:16
oneswigIt is Germany, after all :-)21:16
oneswigmartial__: That Jim Golden?21:16
martial__but you got there in "November"21:16
martial__another Jim21:16
martial__the first Jim will be in Germany btw21:17
oneswigGood to have him, all the same21:17
jandersoneswig: subtle hint: how about a venue with craft beer + open fire roasted meat? :)21:17
jandersalways a highlight at ISCs21:17
jandersunfortunately I don't know Berlin at all21:17
oneswigjanders: seems like you've done this before :-)21:17
martial__surprised Jim (Lowe) did not jump on this one ;)21:18
jandersyeah but only in Frankfurt and Leipzig21:18
trandlesMike Lowe?21:18
trandlessounds like janders is volunteering to organize the outing, good man21:18
martial__yes Mike (is it already that late? :) )21:18
martial__yeah janders!!!!21:19
janderstrandles: I don't know Berlin well enough (or at all) so just hinting the idea :)21:19
janderscan ask around though21:19
oneswigBerlin attendees, I'll be receptive to any google-based suggestions of local places on the theme of das-beer-and-roasted-meat-haus21:19
oneswig(although at pains to point out that other dietary requirements must also be met)21:20
*** jamesmcarthur has quit IRC21:20
trandlesjanders: I'll be in Dallas unfortunately :(  Definitely be at the Denver Summit in the spring though...21:20
martial__asked my German friend Lukas for recommendations :)21:20
jandersmartial_: good man! :)21:20
oneswigmartial__: couldn't ask for more than that.  Provided Lukas is not a monk.21:21
oneswigBTW how are your panel questions coming on for SC?21:21
martial__I worked on the "panel" questions, not the "BoF" ones yet21:21
martial__need to get this started21:22
martial__will start a Google Doc or Etherpad and share it here21:22
*** jamesmcarthur has joined #openstack-meeting21:22
oneswigThis came up among our team this week and may be interesting for your crowd: https://kubernetes.io/blog/2017/08/kubernetes-meets-high-performance/21:22
martial__Lukas via chat recommend looking at https://awesomeberlin.net/attractions/biergarten-berlin-best-beer-garden21:23
martial__Jim is next to me btw, so when we get to AOB, I will give him my keyboard21:24
oneswigI get the impression Docker's runtime is getting squeezed out of the Kubernetes ecosystem - lots of energy going into projects like podman and cri-o21:24
jandersoneswig: VERY interesting link21:25
jandersthanks for sharing21:25
oneswigMight be a good opener for Christian :-)21:25
janderson the SIG sessions front - I did have a quick chat to Bright last week21:25
jandersas it stands, they don't seem to be targetting running on top of a bare-metal OpenStack, they (still) insist on running their own OpenStack (and I don't think they are far with baremetal on that)21:26
jandersI wonder if you guys see value in trying to get some HPC cluster management solution vendors to join us, so that they get a better understanding of what we're doing?21:26
jandersI can ping Bright again asking if they have anyone at the Summit - we could also maybe try with others?21:27
oneswigjanders: that is always good.21:27
janderswe have (or have access to) four systems - two are bright, one is SGI stack and one is a Cray21:28
*** bobh has joined #openstack-meeting21:28
jandersso Cray would be my second thought21:28
oneswigThe Cray's probably half Bright anyway?21:28
jandersit's hard to keep up with them21:28
janderstwo years ago they were going all-OpenStack, now they backed out21:28
oneswigMuch of the management infra is deployed with that.21:28
trandlesAFAIK is Cray even doing any OpenStack any more?21:29
jandersthey are making crazy changes to their interconnects (details - NDA but you likely heard)21:29
janderstrandles: last time I heard they admitted failure and dropped the ball21:29
trandlesjanders: +121:29
jandersI hope maybe if they listen to us talking they'll realise what they're missing out on21:30
oneswigtrandles: no21:30
jandersand we'll be fine without them, but it would be good to have more options for running HPC on OpenStack21:30
jandersok.. I will ping Bright21:31
jandersdo you guys know anyone at Cray that could be interested?21:31
jandersI only talk to salesy people on their end unfortunately21:31
bolligI know a couple guys across the river that used to work on OS at Cray. I’ll ask21:31
oneswigjanders: I know some people in Europe but I don't think they will be in Berlin - I can ask21:31
*** gagehugo has left #openstack-meeting21:32
*** bobh has quit IRC21:33
jandersI dropped Martijn a quick note, let21:34
janders's see if anyone from Bright is heading to Berlin21:34
jandershow about SchedMD? Do you know if they'll have any presence in Berlin?21:34
trandlesjanders: highly doubtful since SC is the same week21:35
jandersvery fair point21:35
oneswigI've pinged my contacts in Cray but suspect it'll be a similar story21:35
trandlesdoes IBM have any openstack things going on?  You might use that as a way to get in touch with LSF folks.21:36
trandlesalthough since IBM != x86 any more, I doubt it21:36
jandersall I know is that their GPFS integration is struggling to keep up with the new OpenStack releases21:36
janderswhich isn't a good sign21:36
trandlesthat basically leaves Adaptive (torque/moab) and Altair (PBS)21:37
trandlesI wouldn't be surprised if Altair doesn't have something going on.21:37
jandersRHATs are doing some interesting work with Power and Ironic here in Canberra, but that's completely non-HPC21:37
martial__seems we are slowing down on this topic21:40
martial__should I get Jim Horan to introduce himself?21:40
oneswiggo for it21:40
*** lbragstad is now known as lbragstad-50321:42
martial__Hi my name is Jim Horan. I am working at NIST in Gaithersburg, MD in the area of public safety analytics. We are currently working with NIST Public Safety Communications Research Lab in our Bolder facility to further the use of analytics by public safety stakeholders and first responders. One of the areas we have been investigating is having HPC capability at the edge for use by public safety operators. We are21:44
martial__also looking at cloud-based analytics and hybrid cloud capabilities to optimize the distribution of high and low computational intensity analytics.21:44
*** munimeha1 has quit IRC21:45
martial__We are funding several research grants to develop analytics as well as internal projects to support reference frameworks to test and evaluate analytic performance.21:45
martial__We are currently developing a stack to support this reference capability and Openstack has been an area of interest to support our research.21:46
oneswigSounds good.  There was a keynote by AT&T (IIRC) on a similar theme at the last summit.  Did you see that?21:47
martial__No, but if I can track it down, I'll check it out.. Thanks!21:47
oneswigIt was a movie clip, started with a car going off a bridge21:48
oneswigIt will be very interesting to hear about the use case - edge computing work is a hot subject these days!21:50
martial__Ah.. I think somebody did tell me about that movie clip.  Our PS folks have been working to refine requirements of use cases (such as car or person off a bridge) and put them against state of the art development of analytics to see what can be done in the near term.. and what capabilities are required to support them.21:50
jandersfew years back, we built a POC where we had a small HPC OpenStack in Singapore which would also manage resources around the world21:51
martial__we have been keeping a close eye on the progress in edge computing.. since with the onset of 5G the edge can go from being a group or cluster of functionality to a fabric21:51
janderscontrollers + few computes in Singapore (along with central storage ) and then more computes in Australia, Europe, US21:52
jandersworked reasonably well, was quite interesting. not sure if this is directly relevant to your use case but thought I will mention in case it is21:53
*** slaweq has joined #openstack-meeting21:53
martial__how did the POC hold up in terms of robustness with all the variables of comms/network etc?21:53
*** trandles has quit IRC21:53
oneswigJim would you be able to talk on this some more with the SIG in Berlin?21:53
martial__Martial has been trying to get me to talk about it. I am checking to see if I can make it out.. but I would be glad to talk more about it.21:54
jandersit was quite solid. wan setup was different for different sites and those without QoSed circuits would sometimes cause trouble. Those with solid QoS were pretty good.21:54
oneswigjanders: sounds like an idea ahead of its time21:55
*** mriedem_away has quit IRC21:55
martial__did you do any network active adaptation based on the QoS changing? I think this will become super important as the edge clusters start to load balance etc21:55
*** smcginnis is now known as smcginnis_vaca21:55
jandersmartial_: nope21:56
janderswe would have different host aggregates mapping to AZs and depending on the requirements the user could spin up resources within a specific site, or scattered around the World21:56
jandersbut there wasn't any other form of load balancing or placement optimisation21:57
martial__sometimes not the luxury of having spare capacity/circuits..   that sounds like an interesting system21:57
martial__(and we are reaching the end time wise :) )21:59
martial__(taking back my keyboard)21:59
martial__so I am trying to get Jim to come to the Berlin Summit :)21:59
janders+1! :)21:59
oneswigmartial__: be good to see him!22:00
martial__will try to get him to come for sure :)22:01
oneswigOK we should wrap up for today22:01
martial__and on that, thanks all for attending our Scientific SIG meeting22:01
jandersthanks all! have a good day / good night :)22:02
*** a-pugachev has quit IRC22:02
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:02
openstackMeeting ended Tue Oct 16 22:02:16 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:02
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-16-21.00.html22:02
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-16-21.00.txt22:02
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-16-21.00.log.html22:02
*** oneswig has quit IRC22:03
*** priteau has quit IRC22:03
*** slaweq has quit IRC22:09
*** slaweq has joined #openstack-meeting22:11
*** janders has quit IRC22:24
*** jamesmcarthur has quit IRC22:26
*** jamesmcarthur has joined #openstack-meeting22:38
*** jamesmcarthur has quit IRC22:43
*** slaweq has quit IRC22:44
*** rcernin has joined #openstack-meeting22:49
*** ykatabam has joined #openstack-meeting22:51
*** jamesmcarthur has joined #openstack-meeting22:54
*** macza has quit IRC23:01
*** hongbin has quit IRC23:07
*** jamesmcarthur has quit IRC23:07
*** jamesmcarthur has joined #openstack-meeting23:07
*** slaweq has joined #openstack-meeting23:11
*** jamesmcarthur has quit IRC23:18
*** jamesmcarthur has joined #openstack-meeting23:21
*** annabelleB has joined #openstack-meeting23:27
*** jamesmcarthur has quit IRC23:32
*** annabelleB has quit IRC23:39
*** slaweq has quit IRC23:44
*** jamesmcarthur has joined #openstack-meeting23:44
*** ssbarnea has quit IRC23:46
*** ssbarnea|bkp2 has joined #openstack-meeting23:46
*** erlon has joined #openstack-meeting23:55
*** cloudrancher has joined #openstack-meeting23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!