Tuesday, 2018-10-23

*** ykatabam has quit IRC00:04
*** rcernin has joined #openstack-meeting00:11
*** jamesmcarthur has joined #openstack-meeting00:11
*** rcernin_ has quit IRC00:12
*** rcernin has quit IRC00:13
*** rcernin has joined #openstack-meeting00:14
*** jamesmcarthur has quit IRC00:16
*** mjturek has quit IRC00:22
*** ssbarnea has joined #openstack-meeting00:22
*** ykatabam has joined #openstack-meeting00:23
*** jamesmcarthur has joined #openstack-meeting00:30
*** longkb has joined #openstack-meeting00:37
*** ykatabam has quit IRC00:39
*** ykatabam has joined #openstack-meeting00:40
*** tetsuro has joined #openstack-meeting00:45
*** gouthamr has joined #openstack-meeting00:47
*** dmellado has joined #openstack-meeting00:51
*** tetsuro has quit IRC01:06
*** tetsuro_ has joined #openstack-meeting01:06
*** stevebaker has joined #openstack-meeting01:06
*** imacdonn has quit IRC01:23
*** imacdonn has joined #openstack-meeting01:23
*** markvoelker has joined #openstack-meeting01:25
*** hongbin has joined #openstack-meeting01:27
*** cloudrancher has quit IRC01:33
*** cloudrancher has joined #openstack-meeting01:34
*** hongbin has quit IRC01:36
*** hongbin has joined #openstack-meeting01:37
*** yamahata has quit IRC01:40
*** jamesmcarthur has quit IRC01:41
*** hongbin_ has joined #openstack-meeting01:41
*** cloudrancher has quit IRC01:41
*** cloudrancher has joined #openstack-meeting01:42
*** hongbin has quit IRC01:43
*** mhen has quit IRC01:44
*** mhen has joined #openstack-meeting01:47
*** jamesmcarthur has joined #openstack-meeting02:00
*** hongbin has joined #openstack-meeting02:05
*** jamesmcarthur has quit IRC02:05
*** hongbin_ has quit IRC02:07
*** jamesmcarthur has joined #openstack-meeting02:18
*** jamesmcarthur has quit IRC02:34
*** jamesmcarthur has joined #openstack-meeting02:39
*** luobin-smile has joined #openstack-meeting02:52
*** psachin has joined #openstack-meeting02:57
*** jamesmcarthur has quit IRC02:59
*** munimeha1 has quit IRC03:01
*** shilpasd_ has joined #openstack-meeting03:02
*** shilpasd_ has quit IRC03:08
*** yamahata has joined #openstack-meeting03:49
*** tpatil has joined #openstack-meeting03:50
*** hongbin has quit IRC03:52
*** sagara has joined #openstack-meeting04:01
*** samP has joined #openstack-meeting04:01
sagarasamP: Hi04:02
samP#startmeeting masakari04:02
openstackMeeting started Tue Oct 23 04:02:34 2018 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:02
*** openstack changes topic to " (Meeting topic: masakari)"04:02
openstackThe meeting name has been set to 'masakari'04:02
samPHi all04:02
samPsorry for the delay04:02
samPLet's start04:02
samP#topic Critical Bugs/Patches04:03
*** openstack changes topic to "Critical Bugs/Patches (Meeting topic: masakari)"04:03
*** ShilpaSD has joined #openstack-meeting04:03
samPAny critical bugs or patches need to discuss?04:04
samP#link https://review.openstack.org/#/c/606652/104:04
samPPlease review this ^^, LGTM04:04
openstackLaunchpad bug 1798496 in masakari "masakari-dasboard : DateTime parameters are displayed in UTC format" [Undecided,New]04:05
tpatilThis bug is not critical04:05
tpatilIn masakari dashboard and elsewhere in horizon, date is displayed in UTC format04:05
*** mrhillsman is now known as openlab04:05
tpatilAs an user, I would like to see the date time parameters in local timezone04:06
*** openlab is now known as mrhillsman04:06
samPtpatil: Can it change from horizon side?04:07
tpatilLet me check04:07
tpatilIt's there in settings04:08
tpatilWill mark this bug as invalid04:08
samPtpatil: Thanks.04:09
tpatilThere are some bugs whose status is not marked correctly even after the proposed patches are merged04:10
tpatile.g. https://bugs.launchpad.net/masakari/+bug/177975204:10
openstackLaunchpad bug 1779752 in masakari-monitors "masakari segment-list returns error (openstack queens)" [Critical,Confirmed] - Assigned to Shilpa Devharakar (shilpasd)04:10
samPtpatil: yes, specially those queens issues.04:10
samPWe need to check them and re-set the status04:11
tpatilok, Can I mark it's status to fix committed?04:13
samPtpatil: sure, thanks04:14
samPI will check for python-masakariclient and masakari-monitors04:14
tpatilOk, will do04:15
samPtpatil: thanks04:16
*** yamamoto has quit IRC04:17
*** yamamoto has joined #openstack-meeting04:17
samPI am looking for the rootwrap issue in masakari monitors, with no luck04:17
*** jamesmcarthur has joined #openstack-meeting04:18
tpatilit's not reproducible04:18
*** yamahata has quit IRC04:19
samPtpatil: OK, let's discuss that if it surface again.04:19
*** yamahata has joined #openstack-meeting04:19
samP#topic Stein Work Items04:20
*** openstack changes topic to "Stein Work Items (Meeting topic: masakari)"04:20
tpatilFunctional test in openstacksdk04:20
samPPlease share if you have any updates from last week04:20
tpatilcouldn't push patches in the last week. Will target in this week04:20
samPtpatil: thanks04:21
tpatilin openstacksdk, we will add minimal functional tests04:21
*** tashiromt_ has joined #openstack-meeting04:21
samPtpatil: sure04:21
tpatilNotification workflow events, In the last meeting, it was decided to write a specs for this04:22
*** jamesmcarthur has quit IRC04:22
samPneed to update spec repo04:22
tpatilI checked the task flow library and it has support to persist task flow details in backend04:22
tpatilI think  we should leverage that functionality instead of adding our own04:23
tpatilchecking how to pass masakari backend to task flow so that the required db tables can be added in masakari db04:23
samPtpatil: great. No need to add same functionality to masakari, if we can use one in faskflow lib04:24
tpatilsamP: Presently, understanding what changes we would need to do to pass useful information in each task04:25
tpatilonce it's clear, I will write a specs for this04:26
samPBTW, Greg Hill started the discussion about moving taskflow out of openstack/oslo to github.04:26
tpatilsamP: Yes , I was that mailing thread04:26
samPDo we have to worry about this?04:26
tpatilI don' think so04:27
tpatilAdd event notification feature to masakari: https://review.openstack.org/#/c/47305704:27
tpatilInternal review is in progress. Will push patches soon04:28
samPtpatil: sure. Thanks04:28
*** rfolco|rover has quit IRC04:30
samPAny other updates?04:31
sagarasamP: for my side, nothing04:32
samPNo updates from my side.04:33
samP#topic AOB04:33
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:33
samPAny other issues to discuss?04:33
samPOK then, if any please use openstack-dev ML with [masakari] or #openstack-masakari @freenode04:34
samPLet's finish today's meeting04:34
samPThank you for your time04:35
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:35
openstackMeeting ended Tue Oct 23 04:35:12 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:35
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-23-04.02.html04:35
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-23-04.02.txt04:35
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-23-04.02.log.html04:35
*** sagara has quit IRC04:37
*** mahatic has joined #openstack-meeting04:41
*** tpatil has left #openstack-meeting04:44
*** janki has joined #openstack-meeting04:59
*** jiaopengju has quit IRC05:20
*** jiaopengju has joined #openstack-meeting05:23
*** ralonsoh has joined #openstack-meeting05:40
*** ttsiouts has quit IRC05:42
*** ttsiouts has joined #openstack-meeting05:43
*** Luzi has joined #openstack-meeting05:44
*** ttsiouts has quit IRC05:47
*** tetsuro_ has quit IRC05:50
*** tashiromt_ has quit IRC06:35
*** slaweq has joined #openstack-meeting06:43
*** njohnston has quit IRC06:47
*** pcaruana has joined #openstack-meeting06:56
*** ykatabam has quit IRC07:03
*** felipemonteiro has joined #openstack-meeting07:04
*** armax has quit IRC07:04
*** rcernin has quit IRC07:06
*** felipemonteiro has quit IRC07:22
*** jamesmcarthur has joined #openstack-meeting07:34
*** aojea has joined #openstack-meeting07:37
*** jamesmcarthur has quit IRC07:38
*** tpatil has joined #openstack-meeting07:49
*** bhagyashris has joined #openstack-meeting07:53
*** phuoch has joined #openstack-meeting08:04
*** phuoch has quit IRC08:09
*** ttsiouts has joined #openstack-meeting08:17
*** e0ne has joined #openstack-meeting08:23
*** a-pugachev has joined #openstack-meeting08:25
*** electrofelix has joined #openstack-meeting08:29
*** gouthamr has quit IRC08:32
*** dmellado has quit IRC08:34
*** stevebaker has quit IRC08:35
*** priteau has joined #openstack-meeting08:45
*** yinweiishere has joined #openstack-meeting08:58
*** dtrainor has quit IRC08:59
jiaopengju#startmeeting karbor09:01
openstackMeeting started Tue Oct 23 09:01:01 2018 UTC and is due to finish in 60 minutes.  The chair is jiaopengju. Information about MeetBot at http://wiki.debian.org/MeetBot.09:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.09:01
*** openstack changes topic to " (Meeting topic: karbor)"09:01
openstackThe meeting name has been set to 'karbor'09:01
jiaopengjuhi guys09:01
jiaopengjuhi yinweiishere, luobin-smile09:01
yinweiisherehi pengju09:01
yinweiisherehave we got any schedule for S version new features?09:02
jiaopengjuI saw the meeging agenda has been updated here, https://wiki.openstack.org/wiki/Meetings/Karbor09:02
yinweiisherewe wanna propose add snapshot feature there09:03
yinweiishereand just now luobin and I checked current karbor code09:03
jiaopengjuyinweiishere, I have do some plan for the S version, mainly focus on optimization09:03
yinweiisherewe found some problems for snapshot there09:03
yinweiishereany wiki page for S version plan?09:03
jiaopengjuI will send an etherpad link after meeting09:04
yinweiishereOK, good09:05
jiaopengjuWe can talk about the feature in the agenda first09:05
jiaopengju#topic add snapshot feature to Karbor to support crash consistency and app consistency further09:05
*** openstack changes topic to "add snapshot feature to Karbor to support crash consistency and app consistency further (Meeting topic: karbor)"09:05
yinweiishereI'd also like to hear your ideas for optimization09:05
yinweiisheresince as I know Yuval has done a lot opt there :)09:06
yinweiishereso for snapshot09:06
jiaopengjunow you can describe the snapshot feature09:06
yinweiishereas we know, snapshot has two usages09:06
yinweiishereone to restore another instance from snapshot09:07
yinweiisherethe other is to rollback current instance to the snapshot09:07
yinweiisherehere, the instance I mean server or volume or volume groups09:07
jiaopengjuget it09:07
yinweiisherewe checked current implementation in karbor09:07
yinweiisherewe only provide restore API for snapshot, where we can't rollback current instance09:08
yinweiishereif you check actual nova libvirt snapshot or ceph rbd snapshot, the backends all provide rbd snapshot rollback09:09
yinweiishereso, for the API aspect, the snapshot feature is missing there09:10
jiaopengjuThis means we should add rollback operation in the protection plugins?09:10
jiaopengjujust like verify, copy and so on09:10
yinweiisherefirst add rollback API in restore09:10
*** gouthamr has joined #openstack-meeting09:11
yinweiisherethen, for snapshot protection plugin, support it09:11
yinweiishereso this is the API part09:11
yinweiisheresecond, snapshot has a consistency semantics09:11
yinweiisherenow Karbor's checkpoint doesn't support any level's consistency09:12
yinweiisheredo you agree?09:12
*** geguileo has joined #openstack-meeting09:12
jiaopengjuyes, agree09:13
*** tetsuro has joined #openstack-meeting09:14
yinweiisherewhen we initially started this project, the founder agreed to postpone the consistency issue. But without consistency, neither the snapshot or the checkpoint (what ever you call) it doesn't satisfy 'protect' semantic in fact.09:14
yinweiishereactually, there're two levels for consistensy09:15
yinweiisherethe initial level: crash consistency09:15
yinweiisherethe higher level: app consistency09:15
yinweiisherealthough Karbor has analyzed dependencies among resources, it still fails to support even crash consistency09:16
jiaopengjudo you have specs for these two level?09:17
jiaopengjuor some docmentation link09:17
yinweiisherelook at how we protect server and its volumes/volume groups, we didn't maintain the consistency there09:18
*** dmellado has joined #openstack-meeting09:18
yinweiisherewe propose to support snapshot with two levels consistency step by step09:18
yinweiisherefirst crash consistency and then app level consistency09:18
yinweiishereyes, we do have some ideas there09:19
yinweiisherewant to achieve consensus first before writing it to spec09:19
*** bzhao__ has quit IRC09:20
yinweiisherepengju, are you there?09:20
jiaopengjuyes, can you give more messages about crash consistency?09:21
yinweiishereactually, I'm thinking that we need a pair of APIs for snapshot:take_snapshot(consistency_level) and rollback_snapshot(checkpoint_id)09:22
yinweiishereto differenciate existed protect/restore API, which support loose restriction on consistency09:23
jiaopengjudo you mean that, if we take a snapshot in karbor, then the checkpoints info that matched to this snapshot will be saved?09:23
yinweiishereyes, the snapshot will be the backend resource of the checkpoint09:24
yinweiisheresimilar to the backup id09:24
jiaopengjuok, I understand, that sounds useful09:24
yinweiishereor we can merge it with protect API with consistency level params, none means current way09:25
yinweiisherethose are details, we can discuss it in spec09:25
yinweiishereand for crash consistency09:25
*** tpatil has quit IRC09:26
yinweiisherewe need pause the server, flush the memory to disk, and then call volume/volume group's backend snapshot methods09:26
yinweiishereon contrast, currently we never pause server, but only backup each volumes attached to the server if it booted from a volume. If it booted from an image, we never backup the changes in system volume.09:28
yinweiisherethat's the crash consistency09:28
yinweiisherewhere the system could really boot from the snapshot, and apps09:28
yinweiishere won't crash09:28
jiaopengjuI have talked the vms that booted from an image with yuval and chenying before09:29
jiaopengjuBut if we pause server in karbor, this will a few intrusive to the end user09:31
yinweiishereour current way may backup at a wrong moment,that the system and the apps haven't saved the necessary status to disk and they may fail to boot without those status.09:31
yinweiisherethat's the protection plugin09:31
jiaopengjupart of karbor09:32
yinweiisherethat's dependent on the consistency param09:32
yinweiishereif user asks for crash consistency, it means he/she understands what does it mean09:33
yinweiisherefor crash consistency itself, that's the way it should be09:33
jiaopengjuok, understand, this is the detail info, you can write it in the spec09:34
jiaopengjuapp consistency?09:35
*** stevebaker has joined #openstack-meeting09:36
jiaopengjuyinweiishere are you there?09:37
yinweiishereyes, I'm here09:37
yinweiishereapp consistency is a bit complicated09:37
yinweiishereLuobin, could you pls. elaborate here?09:38
yinweiishereit's about global consistent snapshot09:39
yinweiisherewhich requires some background knowledge09:39
jiaopengjuapp , the resource that matched to openstack is a couple of resources, or a group of resources?09:39
*** bhagyashris has quit IRC09:39
yinweiisherehave you heard of chandy-lamport algorithm for distributed snapshot?09:39
yinweiisherethis algorithm is enhanced as ABS algorithm and applied in stream engine in flink09:40
yinweiishereour idea is to enhance ABS and apply it in karbor09:41
yinweiisherewe can put it as a more long term target09:41
yinweiisherethe paper of ABS is here09:42
yinweiisherefor your reference09:42
*** jamesmcarthur has joined #openstack-meeting09:43
jiaopengjuactually I think we should define which resource that mapped to openstack we protected in app consistency scene09:43
jiaopengjuand then how09:43
yinweiishereresources are the same09:44
yinweiisherethe problem is how to make the app aware of the snapshot09:45
yinweiishereand, as there are many apps in one plan's servers09:45
yinweiisherehow to make sure all apps are consistent09:45
yinweiisherethe distributed snapshot issue is to make sure the causal ordering is correct09:46
yinweiishereas, APP1 is the input of APP209:46
yinweiisherewe take snapshot on the whole system09:47
*** jamesmcarthur has quit IRC09:47
yinweiisherebut the snapshot command may get triggered at different timming on server1 and server209:47
jiaopengjuok, a little understand, I will see the reference you send after meeting09:47
yinweiisheresince there're events on the fly09:47
yinweiisheresimilar to crash consistency09:48
yinweiisherewe need maintain the dependencies among APPs09:48
yinweiisherethat's all for my part09:48
jiaopengjuthanks for giving this useful idea09:49
yinweiishereyou can send out the schedule link09:50
jiaopengjuwe can add this to stein plan09:50
yinweiishereand we can put the effort there to see if more people will get interested there09:50
yinweiishereyes, we can support it step by step09:50
jiaopengjuI can easily describe the plan in stein cycle here: optimization of multiple nodes of operation engine (yuval provide the first version)09:52
jiaopengju2. cross-site backup and restore (cross keystone)09:52
jiaopengju3. documentation09:52
yinweiisherecross site is what I proposed long long ago09:53
jiaopengjuyes, some people has asked questions about it09:53
yinweiishereit's really useful, right?09:53
jiaopengjucross keystone seems not09:53
yinweiishereI think cross region/AZ is necessary09:54
yinweiisherecross keystone is a bit difficult09:54
jiaopengjuat the same time, we do not have documentation enough about it09:54
yinweiishereagain, step by step is more feasible09:55
jiaopengjuyes, agree09:55
yinweiishereok, I think the time is run out09:55
jiaopengjuyeah, so I will end the meeting soon and we can talk about it in karbor channel09:55
yinweiishereif you have made the schedule link, pls. let us know in karbor channel09:56
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"09:56
openstackMeeting ended Tue Oct 23 09:56:33 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)09:56
openstackMinutes:        http://eavesdrop.openstack.org/meetings/karbor/2018/karbor.2018-10-23-09.01.html09:56
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/karbor/2018/karbor.2018-10-23-09.01.txt09:56
openstackLog:            http://eavesdrop.openstack.org/meetings/karbor/2018/karbor.2018-10-23-09.01.log.html09:56
*** rossella_s has quit IRC09:57
*** a-pugachev has quit IRC10:03
*** a-pugachev has joined #openstack-meeting10:04
*** rfolco has joined #openstack-meeting10:12
*** ssbarnea_ has joined #openstack-meeting10:16
*** ianychoi has quit IRC10:22
*** ianychoi has joined #openstack-meeting10:25
*** apetrich has quit IRC10:38
*** dtrainor has joined #openstack-meeting10:48
*** apetrich has joined #openstack-meeting10:54
*** erlon has joined #openstack-meeting10:58
*** yamamoto has quit IRC11:03
*** yamamoto has joined #openstack-meeting11:04
*** ttsiouts has quit IRC11:04
*** yamamoto has quit IRC11:08
*** yikun has quit IRC11:09
*** tetsuro has quit IRC11:12
*** longkb has quit IRC11:19
*** yamamoto has joined #openstack-meeting11:21
*** rfolco is now known as rfolco|rucker11:23
*** janki has quit IRC11:41
*** ttsiouts has joined #openstack-meeting11:43
*** eharney has joined #openstack-meeting11:45
*** markvoelker has quit IRC11:45
*** ttsiouts has quit IRC11:48
*** mmethot has joined #openstack-meeting11:54
*** ttsiouts has joined #openstack-meeting12:02
*** a-pugachev has quit IRC12:09
*** janki has joined #openstack-meeting12:14
*** cloudrancher has quit IRC12:19
*** cloudrancher has joined #openstack-meeting12:19
*** janki has quit IRC12:25
*** janki has joined #openstack-meeting12:25
*** janki has quit IRC12:27
*** tobberydberg has quit IRC12:30
*** janki has joined #openstack-meeting12:34
*** rubasov has joined #openstack-meeting12:36
*** markvoelker has joined #openstack-meeting12:36
*** markvoelker has quit IRC12:37
*** jchhatbar has joined #openstack-meeting12:38
*** janki has quit IRC12:40
*** jchhatbar is now known as janki12:50
*** dustins has joined #openstack-meeting12:51
*** dustins is now known as dschoenb|worksho12:52
*** dschoenb|worksho is now known as dustins12:52
*** yamamoto has quit IRC13:02
*** bnemec has joined #openstack-meeting13:08
*** psachin has quit IRC13:10
*** a-pugachev has joined #openstack-meeting13:13
*** e0ne has quit IRC13:19
*** longkb has joined #openstack-meeting13:28
*** kailun has joined #openstack-meeting13:29
*** bobh has joined #openstack-meeting13:31
*** mriedem has joined #openstack-meeting13:34
*** yamamoto has joined #openstack-meeting13:35
*** awaugama has joined #openstack-meeting13:42
*** cloudrancher has quit IRC13:44
*** cloudrancher has joined #openstack-meeting13:45
*** mjturek has joined #openstack-meeting13:47
*** liuyulong has joined #openstack-meeting13:48
*** kiennt26 has joined #openstack-meeting13:50
*** mlavalle has joined #openstack-meeting13:53
*** kiennt26 has quit IRC13:56
*** a-pugachev has quit IRC13:58
*** Leo_m has joined #openstack-meeting13:58
*** edmondsw has joined #openstack-meeting14:00
*** hongbin has joined #openstack-meeting14:00
mlavalle#startmeeting networking14:01
openstackMeeting started Tue Oct 23 14:01:29 2018 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.14:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:01
*** openstack changes topic to " (Meeting topic: networking)"14:01
openstackThe meeting name has been set to 'networking'14:01
longkbhi :)14:01
mlavalle#topic Announcements14:02
*** a-pugachev has joined #openstack-meeting14:02
*** openstack changes topic to "Announcements (Meeting topic: networking)"14:02
mlavalleThis week we reached the Stein-1 milestone14:03
*** annp_ has joined #openstack-meeting14:03
mlavalleAnd we are a little more than two weeks away from the Berlin Summit14:04
mlavalleAny other announcements to share with the team?14:04
*** njohnston has joined #openstack-meeting14:04
*** lajoskatona_ has joined #openstack-meeting14:04
mlavalleok, let's move on14:05
mlavalle#topic Blueprints14:05
*** openstack changes topic to "Blueprints (Meeting topic: networking)"14:05
mlavalleThese are the blueprints we are targeting: https://launchpad.net/neutron/+milestone/stein-114:05
mlavalleDoes anybode have an update or request?14:06
mlavalleregarding blueprints14:06
*** Luzi has quit IRC14:06
mlavalleamotoki: all good to cut the milestone?14:07
amotokimlavalle: yeah. I will prepare a release patch around Thu14:07
mlavalleamotoki: are we also releasing the neutron stable branches?14:07
amotokimlavalle: we can, but it depends on pending patches.14:08
amotokii haven't checked the status. I need to sync with you.14:08
mlavalleamotoki: ok14:08
mlavalleLast week I was approached by someone asking a Queens release14:09
mlavalleso let's do it14:09
mlavalleok, moving on14:09
mlavalle#topic Community goals14:10
*** openstack changes topic to "Community goals (Meeting topic: networking)"14:10
mlavalleamotoki: any updates with policy in code?14:10
amotokiregarding policy-in-code, I just pushed a new patch set. sorry for late, but I believe neutron side will be in good shape.14:10
mlavalledo you have the pointer handy?14:11
mlavalleThanks amotoki :-)14:11
amotokiI think I am in limited bandwidth, so once I prepared neutron-fwaas patch, I would like to gather volunteers for other stadium projects.14:11
amotoki* to make the progress fast14:12
amotokineutron-fwaas patch would be a good example for other projects.14:12
mlavalledo we need a volunteer for each Stadium project?14:13
amotokiI don't think we need each per project14:13
mlavalleok, let us know when the fwaas patch is ready and we can use it as a model14:14
mlavalleand to request volunteers14:14
mlavallethanks for the update14:14
mlavallenjohnston: anything on py3?14:14
njohnstonNothing at present; there has been a thread on openstack-dev about how to handle py35 vs py36 vs py37, and so I have been waiting to make sure the aproach is not going to change.  I think a relative consensus has been reached.14:15
njohnstonSo I am revisiting some of those changes this morning.14:16
mlavalleThanks for the update, njohnston14:16
amotokithis thread is related to python3 future http://lists.openstack.org/pipermail/openstack-dev/2018-October/135892.html14:17
amotokithis is just FYI14:17
njohnstonThanks amotoki, I was just looking for the link14:17
*** boden has joined #openstack-meeting14:17
mlavalleslaweq: anything worth mentioning on Support Pre Upgrade Checks?14:17
slaweqonly that patch https://review.openstack.org/#/c/608444/ with base cli tool is ready for review14:18
slaweqnext step will be for me to add support for checks to be loaded using stevedore that e.g. stadium projects or some 3rd party plugins will be able to add own checks there14:18
mlavalleok, I will take a look at this patch today. Thanks14:20
slaweqthx mlavalle14:20
mlavalle#topic Bugs14:20
*** openstack changes topic to "Bugs (Meeting topic: networking)"14:20
mlavallehaleyb was our bugs deputy last week14:21
mlavalleHe sent this update to the ML: http://lists.openstack.org/pipermail/openstack-dev/2018-October/135969.html14:21
*** janki has quit IRC14:21
mlavalleI thought we had gotten rid of this one a couple of weeks ago: https://bugs.launchpad.net/neutron/+bug/179868814:23
openstackLaunchpad bug 1798688 in neutron "iptables_hybrid tests tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_shelve_shelved_server failed " [Undecided,Incomplete]14:23
mlavalleI will take a look at it14:24
slaweqmlavalle: few weeks ago it was visible mostly in dvr tests14:24
slaweqand then You couldn't reproduce it14:25
slaweqbut it looks that it still happen from time to time14:25
mlavalleyeah, I will look again14:26
mlavalleany other comments on the bugs report?14:27
mlavalleok, this week, our deputy is armax, so we are covered14:27
mlavalleand next week the deputy is Zhao Bo. I will send him an email to make sure he is ready14:28
mlavallemoving on14:28
mlavalle#topic os-ken14:28
*** openstack changes topic to "os-ken (Meeting topic: networking)"14:28
*** e0ne has joined #openstack-meeting14:29
mlavallehongbin: any updates on this?14:29
hongbinso last week, there are a couple of progress on os-ken14:29
hongbin1. the pep8 job is setup: https://review.openstack.org/#/c/609121/14:29
hongbin2. the unit test is setup: https://review.openstack.org/#/c/609148/14:30
hongbin3. the lower constratnt is setup: https://review.openstack.org/#/c/609100/14:30
hongbin4. the doc building job is under review14:30
hongbin#link https://review.openstack.org/#/q/project:openstack/os-ken14:30
hongbinfor the requirement side, we have a patch up for review14:31
amotokihongbin: I added a minor question on doc fix patch https://review.openstack.org/#/c/610706/4/doc/source/snort_integrate.rst@9014:31
hongbin#link https://review.openstack.org/#/c/609018/14:31
hongbinamotoki: thanks, i will get back to it14:31
amotokii am okay to merge this as is, but I would like you to check it before it :)14:32
hongbini am fine with it14:32
hongbinso, right now, we are basically waiting for the requirement patch14:33
mlavalleshould we ping Mathew?14:33
hongbinsure, we can ping him to look at it again14:34
mlavalleanything else?14:34
hongbinnothing else from my side14:34
mlavalle#topic CLI / SDK14:35
*** openstack changes topic to "CLI / SDK (Meeting topic: networking)"14:35
mlavalleamotoki: any updates on this topic?14:35
*** aojeagarcia has joined #openstack-meeting14:35
amotokinothing from me this week.14:35
mlavalleok, moving on14:35
*** aojea has quit IRC14:35
mlavalle#topic neutron-lib14:35
*** openstack changes topic to "neutron-lib (Meeting topic: networking)"14:35
mlavalleboden: you are up14:36
bodenmuch of the same as last week; working on the db consumption patches along with the switch to payload style args for callbacks14:36
bodena few patches I would like to ask for reviews on that would help keep things moving14:36
bodenfor the db decouple: https://review.openstack.org/#/c/612527/  should be ready14:37
bodenand for payloads: https://review.openstack.org/#/c/597561/  and https://review.openstack.org/#/c/596959/14:37
bodenif folks get time, it would help to keep these moving :)14:37
bodenthat's all the exciting news I have for neutron-lib unless others have something14:37
mlavalleadded all of them to my pile14:38
slaweqI wanted to ask about one thing14:38
mlavalleI think we need to ask amotoki to take a look at https://review.openstack.org/#/c/596959/14:38
*** felipemonteiro has joined #openstack-meeting14:38
mlavalledon't we?14:38
slaweqdo we backport patches to stable branches in neutron-lib? or it don't work like that there?14:39
amotokimlavalle: boden: sorry for that... we discussed it with boden and we agree that we can go as-is. will check it.14:39
mlavalleamotoki: thanks ;-)14:39
bodenslaweq is there some bug we need fixed?14:40
slaweqboden: I was thinking about backport of https://review.openstack.org/#/c/609633/14:40
slaweqbut I don't know if that makes sense so I'm asking here :)14:40
bodenslaweq honestly I'm not sure unless we were to re-release an old version of neutron-lib since everything is pulled from pypi??14:41
bodenmay need ot ask the release team if we do need that backported... not sure we've done this to date that I'm aware of14:42
mlavalleI don't think we have done it14:42
slaweqthat's why I'm asking, I saw stable/XXX branches in neutron-lib repo but I'm not sure if we are releasing such stable releases with bugfixes only14:42
amotokiin theory, if we have bug fixes worth backported in neutorn-lib stable branches, we can backport and release it.14:43
amotokiwe didn't have such so far, but we can if needed.14:43
bodenI think we can, but woudl be a minor release (of whatever version we released for rocky say) right???14:43
amotokiit would be a patch release14:44
amotokiit means only Z can be incremented in X.Y.Z14:44
slaweqok, so that would be more question to stable core team then, if this patch is worth to be backported :)14:44
bodenyes, that's what I Was trying to say14:44
bodenslaweq do you need me to follow-up with them, or you will?14:45
slaweqmlavalle: can You take a look at this patch and check if that is worth to be backported? :)14:45
mlavalleslaweq: sure14:45
mlavalleanything else on neutron-lib14:46
bodennot from me14:46
mlavalleok, let's move on14:46
mlavalle#topic On demand agenda14:46
*** openstack changes topic to "On demand agenda (Meeting topic: networking)"14:46
mlavalleanything else we should bring up to the team today?14:46
mlavalleok, thanks for attending14:47
mlavalleHave a great week!14:47
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:47
openstackMeeting ended Tue Oct 23 14:47:41 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:47
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2018/networking.2018-10-23-14.01.html14:47
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2018/networking.2018-10-23-14.01.txt14:47
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2018/networking.2018-10-23-14.01.log.html14:47
*** boden has left #openstack-meeting14:47
bcafarelo/ (yep joining after the meeting end)14:52
mlavallebcafarel: LOL14:59
slaweq#startmeeting neutron_qos15:00
openstackMeeting started Tue Oct 23 15:00:34 2018 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: neutron_qos)"15:00
openstackThe meeting name has been set to 'neutron_qos'15:00
* mlavalle is not feeling very well today. stomach flue15:01
slaweqmlavalle: :(15:02
slaweqok, let's start15:02
slaweq#topic RFEs15:02
*** openstack changes topic to "RFEs (Meeting topic: neutron_qos)"15:02
slaweqfirst one on the list is:15:02
*** Leo_m_ has joined #openstack-meeting15:02
slaweq#link https://bugs.launchpad.net/neutron/+bug/156096315:02
openstackLaunchpad bug 1560963 in neutron "[RFE] Minimum bandwidth support (egress)" [Wishlist,In progress]15:02
slaweqand there is still no any progress on this one15:02
slaweqso I think we can quickly go to the next one, which is:15:03
slaweq#link https://bugs.launchpad.net/neutron/+bug/172757815:03
openstackLaunchpad bug 1727578 in neutron "[RFE]Support apply qos policy in VPN service" [Wishlist,Triaged]15:03
mlavalleZhao Bo will be working on this one this cycle15:04
slaweqI think there wasn't any update on this one since last meeting: patches by topic: https://review.openstack.org/#/q/topic:qos-vpn+(status:open+OR+status:merged)15:04
mlavalleyes, he'll get to it soon15:04
slaweqgreat, thx mlavalle for info15:04
*** Leo_m has quit IRC15:04
*** vishalmanchanda has quit IRC15:04
slaweqnext is:15:05
slaweq#link https://bugs.launchpad.net/neutron/+bug/175704415:05
openstackLaunchpad bug 1757044 in neutron "[RFE] neutron L3 router gateway IP QoS" [Wishlist,In progress] - Assigned to LIU Yulong (dragon889)15:05
slaweqthere are patches for that one:15:05
liuyulongI will rebase the patch set since there is a conflicts patch is getting merged.15:05
slaweq#link https://review.openstack.org/#/c/424468/ - server side patch,15:05
slaweq    #link https://review.openstack.org/#/c/568526/ - agent side patch,15:05
slaweqin server side patch I posted some comments recently15:05
liuyulongYes, waiting for a rebase.15:05
slaweqI'm now not sure if it is necessary to add additional binding between qos and router's gateway15:06
slaweqas router's gateway has got port in neutron so why we can't just use qos policy associated to this port?15:06
liuyulongBinding to the gateway port is already done.15:07
liuyulongI've explained this before in the patch set. We are try to do QoS stuff for L3 IPs.15:08
liuyulong`Binding qos to the gateway port` is something in L2 view.15:08
slaweqbut can gateway IP have more than one IP address?15:09
*** Leo_m_ has quit IRC15:09
liuyulongAnd that will influence all the traffic through the router gateway port.15:09
*** Leo_m has joined #openstack-meeting15:09
liuyulongYep, the patch now handle all the gateway IPv4 set.15:10
slaweqok, I will have to take a look at it once again then15:11
*** longkb has quit IRC15:11
*** cloudrancher has quit IRC15:12
slaweqlets move on then15:12
slaweqnext one: #link https://bugs.launchpad.net/neutron/+bug/177762715:12
openstackLaunchpad bug 1777627 in neutron "RFE: QoS – rule's management (set,delete, show…) requires <qos-policy> parameter in addition to <rule-id>" [Wishlist,In progress] - Assigned to Miguel Lavalle (minsel)15:12
slaweqWe have merged neutron-lib's api definition for that: https://review.openstack.org/#/q/topic:bug/1777627+(status:open+OR+status:merged)15:12
slaweqthx mlavalle :)15:12
slaweqNow we need implementation on neutron's side also, right?15:12
mlavalleI have patch in my local system15:12
*** jamesmcarthur has joined #openstack-meeting15:12
mlavalleclose to b pushed15:12
slaweqgreat :)15:12
mlavalleplan was to push last night, but then the stomach flu stroke15:13
*** cloudrancher has joined #openstack-meeting15:13
mlavallehoping to push it later today15:13
slaweqno worries, pleaese push it when You will feel better :)15:13
slaweqand the last one on my list is:15:14
slaweq#link https://bugs.launchpad.net/neutron/+bug/157898915:14
openstackLaunchpad bug 1578989 in neutron "[RFE] Strict minimum bandwidth support (egress)" [Wishlist,In progress] - Assigned to Bence Romsics (bence-romsics)15:14
slaweqrubasov: lajoskatona_: any updates?15:14
rubasovlajoskatona_: do you want to give an update this week?15:14
lajoskatona_rubasov: sure15:14
lajoskatona_slaweq: I think we ar eprogresing slowly, as I checked we have some merged patches in neutron side, thatnks for the review.15:15
lajoskatona_slaweq: otherwise we started to concentrate more on the Berlin demo, rubasov and gibi has some slides, and I try to have some working lab environment, hope that will be ok15:16
slaweqsure, lets focus on that for now :)15:16
slaweqwe can continue work on patches after summit15:16
lajoskatona_slaweq: by progressing slowly, I meant I feel it quite good, but we need some "mini retro" to see if we will be ready as we planned :-)15:16
lajoskatona_slaweq: thanks15:17
slaweqok, thx for update lajoskatona_15:19
slaweqthat's all about RFEs which I have for today15:19
slaweqanyone wants to add something?15:19
* mlavalle has https://review.openstack.org/#/c/581364/ next in his pile15:19
rubasovmlavalle: cool, thanks15:20
slaweqok, lets move on to the next topic then15:22
slaweq#topic Bugs15:22
*** openstack changes topic to "Bugs (Meeting topic: neutron_qos)"15:22
slaweqthere isn't anything new related to qos here15:22
slaweqso let's only talk about few existing bugs15:22
slaweq#link https://bugs.launchpad.net/neutron/+bug/178400615:22
openstackLaunchpad bug 1784006 in neutron "Instances miss neutron QoS on their ports after unrescue and soft reboot" [Medium,Confirmed] - Assigned to Miguel Lavalle (minsel)15:22
slaweqmlavalle: any progress on this one?15:22
mlavalleno, I haven't made progress on this one15:23
slaweqnext one is:15:24
slaweq#link https://bugs.launchpad.net/neutron/+bug/177616015:25
openstackLaunchpad bug 1776160 in neutron "'burst' does not take effect for neutron egress qos bindlimit by ovs" [Undecided,New]15:25
slaweqand this is still not confirmed as I didn't have any time for that15:25
slaweqif there is anyone who wants to verfify (and then fix) it, that would be great :)15:25
*** armax has joined #openstack-meeting15:25
ralonsohslaweq: maybe at the end of this week15:26
slaweqralonsoh: sure, thx a lot15:26
slaweqit's already waiting many weeks so that should be not a problem at all ;)15:26
slaweqok, and the last one on the list is:15:27
slaweq#link https://bugs.launchpad.net/neutron/+bug/178518915:27
openstackLaunchpad bug 1785189 in neutron "Floatingip and router bandwidth speed limit failure" [Undecided,In progress] - Assigned to Brian Haley (brian-haley)15:27
slaweqpatch for it is proposed: https://review.openstack.org/#/c/596637/15:27
slaweqbut it's waiting for reply for liuyulong's questions still15:28
slaweqand I'm not sure if author of this patch is really interested in contining work on it15:29
slaweq*continue work on it15:29
liuyulongWe set 64kb for a really long time since then.15:29
slaweqliuyulong: IIRC packets bigger than this MTU value will be simply dropped by tc from qdisc15:30
liuyulongslaweq, yes, it should be.15:32
slaweqso why You are asking about "what happened to this 68130 length packet,"?15:32
*** kopecmartin is now known as kopecmartin|off15:33
slaweqliuyulong: so why You think that changing this value is not good idea?15:35
slaweqI didn't test that but from what I can see in https://launchpadlibrarian.net/383646973/result.pdf it looks that this solves some problem in fact, no?15:35
liuyulongTC does not drop it since the MTU is now 70kb, but what happened on the physical host NIC or the physical switch?15:35
*** ianychoi_ has joined #openstack-meeting15:36
liuyulongBecause I can still see some RETY according to the author's test result.15:36
slaweqok, lets wait for reply from author of the patch maybe15:37
slaweqthat's all from what I have for today15:38
slaweqanyone wants to talk about something else?15:38
slaweq#topic Open Discussion15:38
*** openstack changes topic to "Open Discussion (Meeting topic: neutron_qos)"15:38
slaweqif no, I think we can finish a bit earlier today15:38
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:39
openstackMeeting ended Tue Oct 23 15:39:21 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:39
slaweqthx for attending :)15:39
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_qos/2018/neutron_qos.2018-10-23-15.00.html15:39
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_qos/2018/neutron_qos.2018-10-23-15.00.txt15:39
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_qos/2018/neutron_qos.2018-10-23-15.00.log.html15:39
*** e0ne has quit IRC15:39
*** lajoskatona_ has quit IRC15:39
*** rubasov has left #openstack-meeting15:40
*** ianychoi has quit IRC15:40
liuyulongbye team, and good night~15:40
*** liuyulong is now known as liuyulong|away15:41
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Oct 23 16:00:36 2018 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
*** ShilpaSD has left #openstack-meeting16:00
*** rfolco|rucker is now known as rfolco|brb16:01
slaweq#topic Actions from previous meetings16:01
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:01
slaweqlets start16:01
slaweqslaweq to continue checking how jobs will run on Bionic nodes16:01
slaweqI didn't have time to work on this during last week16:02
slaweqsorry for that16:02
slaweqI will assign it to my for next week also16:02
slaweq#action slaweq to continue checking how jobs will run on Bionic nodes16:02
slaweqnext one:16:02
slaweqmlavalle to continue debugging issue with not reachable FIP in scenario jobs16:02
mlavalleI went over a coouple more failures16:03
mlavalleand left notes in the bug16:03
mlavalleI created an enviroment to test it loclly16:03
mlavalleso I am running it16:03
mlavallein my local environment16:03
slaweqand was You able to reproduce it locally?16:04
mlavallenot yet16:04
mlavalleI'll try today and tomorrow16:04
mlavalleif I don't succeed then I will try to debug in zuul16:04
mlavallefollowing slaweq's recipe16:04
slaweqok, thx for update and for working on this mlavalle16:05
slaweq#action mlavalle to continue debugging issue with not reachable FIP in scenario jobs16:05
slaweqok, next one is16:05
slaweq* mlavalle to send an email about moving tempest plugins from stadium to separate repo16:05
mlavalleI did earlier today16:05
slaweqthx mlavalle16:06
slaweqlets wait for response from stadium projects now16:07
slaweqnext one was:16:07
slaweq* slaweq to report a bug about issue with process ending in fullstack tests16:07
slaweqDone: https://bugs.launchpad.net/neutron/+bug/179847216:07
openstackLaunchpad bug 1798472 in neutron "Fullstack tests fails because process is not killed properly" [High,Confirmed]16:07
slaweqI only reported it but I didn't have possibility to work on this16:07
slaweqso it's free to take if someone wants :)16:08
slaweqit's maybe not "low-hanging-fruit" but may be interesting to debug ;)16:08
mlavalleI might take it after I finsih debugging the previous to do that I have16:08
slaweqthx mlavalle16:08
slaweqok, next one:16:09
slaweq* haleyb to check issue with failing FIP transition to down state16:09
slaweqI think haleyb is not around today16:09
haleybi just got here16:09
*** felipemonteiro has quit IRC16:09
slaweqohh, hi haleyb :)16:09
haleybi looked into it, test seems good still, don't know why associated port gets into BUILD state, still looking16:09
slaweqso it's port to which FIP is associated is in build state?16:10
haleybright, so fip stays in ACTIVE (from my memory)16:11
haleybso i'll continue the fight16:12
slaweqcan it be maybe related to https://review.openstack.org/#/c/606827/ somehow?16:13
slaweqthere is info that port may stuck in DOWN state but I don't know, maybe it's something similar?16:14
haleybi'll look, but in the original issue there is no live migration, it just removes the device from the VM16:15
slaweqahh, ok16:15
slaweqso maybe that other issue then16:15
slaweqfrom what I know port is in BUILD if L2 agent will not send info that port is ACTIVE (or DOWN)16:15
slaweqso You should check in L2 agent's logs if port was wired properly16:16
slaweqand dig from there IMHO16:16
*** ttsiouts has quit IRC16:16
haleybsure, will look there16:16
*** annp_ has quit IRC16:16
*** gyee has joined #openstack-meeting16:16
slaweqok, move on then16:17
slaweqnext was:16:17
*** ttsiouts has joined #openstack-meeting16:17
slaweqslaweq to add e-r query for know fullstack issue (when bug will be reported)16:17
slaweqDone: https://review.openstack.org/#/c/611529/16:17
slaweqpatch is merged already16:17
slaweqand that's all for actions from previous week16:18
slaweqanyone wants to add anything?16:18
mlavallenot me16:18
slaweqok, lets move on then16:18
slaweq#topic Grafana16:18
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:18
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:19
slaweqthere was some spike during the weekend on some jobs, but as it was similar on man different jobs I suspect it's related to some infra issue maybe16:20
slaweqI don't know about anything on our side which could cause such issue16:21
*** ttsiouts has quit IRC16:21
slaweqanything specific You want to discuss now?16:21
mlavalleno, I'm good16:22
slaweqor can we go to talk about different jobs as usually?16:22
mlavalleyes, let's do that16:22
slaweq#topic fullstack/functional16:22
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"16:22
slaweqspeaing about fullstack, I found 2 issues recently16:23
slaweqone was mentioned before: * https://bugs.launchpad.net/neutron/+bug/179847216:23
openstackLaunchpad bug 1798472 in neutron "Fullstack tests fails because process is not killed properly" [High,Confirmed]16:23
slaweqand second is * https://bugs.launchpad.net/neutron/+bug/179847516:23
openstackLaunchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,Confirmed]16:23
slaweqwhere looks that restart of L3 agents with ha routers again cause some packet loss sometimes16:24
mlavalleThey are in haleyb's bug report16:24
slaweqmlavalle: yes, both are there16:24
slaweqcurrently it's not hitting as very often but if this second issue will be very annoying we can mark this test as unstable16:26
slaweqbut I would like first to try to fix it maybe :)16:26
slaweqand speaking about functional tests, we also have still some issues there16:26
slaweqI still saw sometimes that db migration tests are still failing, even with timeout set to 300 seconds16:27
slaweqI checked in logs that those tests can take even up to 400 seconds (I found such numbers at least)16:28
slaweqso I proposed another patch: https://review.openstack.org/#/c/612505/16:28
slaweqI know that it's not the best "solution" but I don't think we can do anything else with it :/16:28
slaweqso please check that patch16:28
mlavalleI just +2ed it16:29
slaweqthx mlavalle16:29
mlavalleand I will push it when zuul returns green16:29
slaweqand second issue which I found from time to time is failing test: neutron.tests.functional.agent.l3.test_ha_router.LinuxBridgeL3HATestCase.test_ha_router_namespace_has_ipv6_forwarding_disabled16:29
slaweqlike e.g. in http://logs.openstack.org/37/610737/2/gate/neutron-functional/286402b/logs/testr_results.html.gz16:29
slaweqI think that it may be related to failing fullstack test: https://bugs.launchpad.net/neutron/+bug/1798475 but that should be checked in logs first16:30
openstackLaunchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,Confirmed]16:30
slaweqand I will check that this week16:30
slaweq#action slaweq to check if failing test_ha_router_namespace_has_ipv6_forwarding_disabled is related to bug https://bugs.launchpad.net/neutron/+bug/179847516:31
slaweqand that's all about fullstack/functional for today on my side16:31
slaweqdo You want to add something?16:31
slaweqok, so next topic16:32
slaweq#topic Tempest/Scenario16:32
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"16:32
slaweqI was looking at tempest jobs this week16:32
slaweqand I found out that we have some timeouts reached for neutron-tempest-xxx jobs from time to time16:33
slaweqit hits jobs like:16:33
*** iyamahat has joined #openstack-meeting16:33
slaweqall of them has got timeout configured for 7800 seconds16:34
slaweqmaybe we should increase it?16:34
mlavallewe can give it a try16:34
slaweqI can imagine that sometimes job can go to cloud node which is under heavy load and tests runs slower there16:35
slaweqso I think it's better to wait e.g. 3h for results then recheck16:35
slaweqand from what I was checking neutron related tests aren't in top of longest ones in those jobs :)16:35
slaweqso mlavalle, would it be good to add 1h to those timeouts?16:36
slaweqok, I will do that16:36
*** yamahata has quit IRC16:36
slaweq#action slaweq to increase neutron-tempest jobs timeouts16:37
slaweqI hope it will make jobs at least a bit more stable16:37
slaweqthat's all from me about scenario/tempest jobs for this week16:37
slaweqdo You have anything to add?16:37
slaweqso last topic for today is16:38
slaweq#topic Open discussion16:38
*** openstack changes topic to "Open discussion (Meeting topic: neutron_ci)"16:38
slaweqand I wanted to ask here about few things16:38
slaweqfirst of all: we have currently 2 quite big topics related to CI jobs:16:38
slaweqrunning jobs with python 316:38
slaweqand second16:38
slawequse of Bionic instead of Xenial16:39
slaweqwhat do You think to add to agenda of this meeting topics about those two things?16:39
slaweqto track progress on both of them weekly?16:39
njohnstonsure, I think that is a good idea16:39
mlavallethat's a good proposal16:39
slaweqok, so I will include it in next meeting then, thx :)16:40
slaweqand also I have one more question16:40
slaweqrelated to patch https://review.openstack.org/#/c/573933/16:40
slaweqit's waiting for review since long time16:40
*** iyamahat has quit IRC16:41
slaweqbut I'm not sure if we really want to maintain something like that in our repo16:41
slaweqso I wanted to ask You about opinions about it16:41
slaweqit You didn't saw it yet, please add it to Your review list and check it this week maybe16:41
mlavalleI'll look at it slowly and leave my comments16:41
slaweqthx mlavalle16:42
njohnstonSince it's meant for developers to run by hand, it seems to me it's more of a dev tool than a CI tool at least at this point16:42
*** a-pugachev has quit IRC16:42
njohnstonI'll have to run it and see what he's trying to do16:42
slaweqnjohnston: yes, for me it's some kind of UT framework for bash16:42
slaweqand I'm not sure it is something what should land in our repo but I would like to know others' opinion about it :)16:44
slaweqok, and that's all from my side for today16:44
slaweqanything else You want to discuss?16:44
mlavallenot from me16:44
* mlavalle is going to rest for a while16:44
ralonsohget well soon16:44
slaweqok, thx for attending guys16:45
slaweqand feel better soon mlavalle :)16:45
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:45
openstackMeeting ended Tue Oct 23 16:45:33 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:45
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-23-16.00.html16:45
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-23-16.00.txt16:45
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-23-16.00.log.html16:45
*** armstrong has joined #openstack-meeting16:46
*** jamesmcarthur has quit IRC16:46
*** a-pugachev has joined #openstack-meeting16:47
*** a-pugachev has quit IRC16:50
*** vishalmanchanda has joined #openstack-meeting16:52
*** raildo has joined #openstack-meeting16:55
*** dtruong has quit IRC16:56
*** iyamahat has joined #openstack-meeting17:05
*** iyamahat has quit IRC17:05
*** iyamahat has joined #openstack-meeting17:06
*** bobh has quit IRC17:08
*** e0ne has joined #openstack-meeting17:08
*** bobh has joined #openstack-meeting17:09
*** bobh has quit IRC17:14
*** iyamahat_ has joined #openstack-meeting17:15
*** rfolco|brb is now known as rfolco|rucker17:16
*** lbragstad is now known as lbragstad_f00d17:17
*** aojeagarcia has quit IRC17:18
*** iyamahat has quit IRC17:18
*** yamahata has joined #openstack-meeting17:24
*** tpsilva has joined #openstack-meeting17:40
*** e0ne has quit IRC17:45
*** lbragstad_f00d is now known as lbragstad17:45
*** felipemonteiro has joined #openstack-meeting17:45
*** ralonsoh has quit IRC17:57
*** gary_perkins has quit IRC18:00
*** bobh has joined #openstack-meeting18:07
*** jamesmcarthur has joined #openstack-meeting18:07
*** jamesmcarthur has quit IRC18:08
*** jamesmcarthur has joined #openstack-meeting18:09
*** cloudrancher has quit IRC18:10
*** cloudrancher has joined #openstack-meeting18:10
*** bobh has quit IRC18:12
*** apetrich has quit IRC18:16
*** apetrich has joined #openstack-meeting18:17
*** bobh has joined #openstack-meeting18:28
*** e0ne has joined #openstack-meeting18:31
*** e0ne has quit IRC18:31
*** felipemonteiro has quit IRC18:34
*** irclogbot_3 has joined #openstack-meeting18:35
*** chandankumar is now known as chkumar|off18:37
*** gary_perkins has joined #openstack-meeting18:41
*** Shrews has joined #openstack-meeting18:51
*** bobh has quit IRC18:57
*** dmsimard has joined #openstack-meeting18:59
clarkbHello infra, anyone here for the meeting?19:00
gary_perkinso/ Hi!19:00
Shrewsi'm here for the cookies19:00
clarkboh no I forgot the cookies19:01
clarkb#startmeeting infra19:01
openstackMeeting started Tue Oct 23 19:01:10 2018 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
clarkbJust another friendly reminder that the summit and forum fast appraoch and you may wish to double check the schedule19:01
clarkb#link https://www.openstack.org/summit/berlin-2018/summit-schedule/#track=262 Berlin Forum schedule up for feedback19:01
fungiso fast19:02
clarkbI think it is fairly solid at this point so mostly just a reminder to go look at the schedule if you haven't arleady. Unsure if changes can be made easily now19:02
clarkbfungi: There is rumor that food awaits me after the meeting. Good motivation :)19:02
fungioh, i was simply commenting on the fast approach of the summit and forum19:03
clarkboh that :)19:03
fungibut yes, fast food too19:03
clarkbindeed it is like 3 weeks away19:03
clarkbI'll be visiting the dentist tomorrow midday, but don't expect any drilling or drugs to happen so should be around. Otherwise not seen any announcements19:03
clarkb#topic Actions from last meeting19:04
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:04
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-16-19.01.txt minutes from last meeting19:04
clarkbNo explicit actions called out but we did some work on things we talked about (which we'll get to later in the agenda)19:04
*** weshay has joined #openstack-meeting19:05
clarkb#topic Specs approval19:05
*** openstack changes topic to "Specs approval (Meeting topic: infra)"19:05
clarkbAny specs we should be aware of? I think we are mostly head down on implementing work that is already captured by specs or general maintenance19:05
clarkbI guess the storyboard spec is still outstanding. And I keep meaning to read it19:06
fungiyou and a bunch of us19:06
fungiwell, at least me anyway19:06
clarkb#link https://review.openstack.org/#/c/607377/ Storyboard attachments spec. Not ready, but input would be helpful19:06
clarkbfrickler has review comments. diablo_rojo not sure if you've seen those19:07
clarkb#topic Priority Efforts19:08
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:08
clarkb#topic Storyboard19:08
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:08
clarkbThat makes a good transition to storyboard topic19:08
diablo_rojoclarkb, I saw them, was waiting for some more before I did an update19:08
diablo_rojojust poked SotK for comments19:09
clarkbdiablo_rojo: ok, I'll try to take a look at it today after the nodepool builder work19:09
diablo_rojoclarkb, that would be suuuper helpful19:09
diablo_rojofungi, too if you have some free time in between the house stuff.19:09
clarkbmordred: not sure if you are here, but I think you started looking at using pbrx with storyboard to work on server upgrades?19:09
* clarkb will try to be a stand in for mordred19:10
clarkbmordred volunteered to work on the storyboard trusty server upgrades and seems to have taken the appraoch of using storaybord(-dev) as an early container deployment system19:10
diablo_rojoSeems like a good approach to me19:10
clarkbThe upside to this is as a python + js application it is actually fairly similar to zuul which means pbrx should work for it19:11
fungiit's a fairly self-contained service so might be a good test case19:11
*** vishalmanchanda has quit IRC19:11
*** caboucha has joined #openstack-meeting19:11
clarkbI want to say the early work has been in udpating bindep.txt type files to have pbrx produce working container images for storyboard19:11
clarkbif you are interested in this aspect of the config mgmt updates this is probably a good place to follow along and/or participate19:11
fungithe api server is straightforward and the webclient is just a wad of files that need to be splatted somewhere a webserver can find19:11
clarkbianw: ^ you may have thoughts in particular since you've looked at similar with graphite19:12
fungithough there's also the need to have rabbitmq running, and also the worker process19:12
fungiso i guess not completely straightforward19:12
clarkbfungi: it should be a good real world application though as it isn't trivial19:13
clarkbwhile still tying into the toolin we've already built19:13
ianwclarkb: yeah, i need to get back to the actual "get docker on control plane server bit", some prelim reviews out there that didn't quite work19:13
clarkbdiablo_rojo: fungi any other storyboard topics to bring up before we move on to the config mgmt udpates?19:14
diablo_rojoI don't think so.19:14
diablo_rojoThank you :)19:14
clarkb#topic Config Mgmt Update19:14
*** openstack changes topic to "Config Mgmt Update (Meeting topic: infra)"19:14
clarkbAs mentioned in the context of storyboard the work of updating our config management processes continues to happen19:15
*** e0ne has joined #openstack-meeting19:15
fungii guess there are a couple base classes of work going on there? 1. contanierizing things, and 2. automatic deployment/replacement of containers?19:16
clarkb#link https://review.openstack.org/#/c/604925/ add zuul user to bridge.o.o for CD activities could use a second review19:16
corvusalso https://review.openstack.org/60955619:16
clarkbfungi: ya part of the spec is the idea that we'll build images regularly to pick up updates and to avoid being stuck on insecure software unexpectedly19:16
corvusi guess i should have used the topic on that19:16
clarkbfungi: to make that useful we also need to update the running deployments with the newi mages19:16
clarkb#link https://review.openstack.org/#/c/609556/ Install ansible 2.7.0 release on bridge.openstack.org19:17
clarkbmostly I think the work here needs a few reviews. The topic:puppet-4 stuff is largely blocked on reviews as well19:18
clarkbif we can find time to take a look at topic:puppet-4 and topic:update-cfg-mgmt we should be able to make some pretty big progress over the short term19:18
dmsimardFWIW I sent patches to see what it would look like to enable ara on bridge.o.o, would love feedback https://review.openstack.org/#/q/topic:ara-on-bridge19:18
clarkb#link https://review.openstack.org/#/q/topic:ara-on-bridge changes to run ara on the bridge server to visualize ansible runs there19:19
corvusshould we change the topic to update-cfg-mgmt?19:19
corvusi think ara is in scope for that19:19
clarkbcorvus: dmsimard ++ I think this is an aid for that spec and could use the topic19:19
clarkbOther than "please review as you can" any other items for this topic?19:21
corvusre ara --19:22
corvusdo we need a mysql db for that or are we going to stick with sqlite for now?19:22
dmsimardsqlite works well in the context of executors, each job is running on it's own database19:22
dmsimardthe problem with sqlite is write concurrency19:22
fungisounded like there was a concurrent write issue with sqlite?19:22
fungiyeah, that19:22
*** jamesmcarthur has quit IRC19:22
corvusso we should have an infra-root set up a mysql db before we land those ara changes?19:23
* mordred waves19:23
*** pcaruana has quit IRC19:23
fungido we want trove or just orchestrate a mysql service onto the bridge server?19:23
clarkbor add that to the changes and run one locally?19:23
*** jamesmcarthur has joined #openstack-meeting19:23
dmsimardcorvus: the legwork is done in hostvars on bridge.o.o, pabelanger helped19:23
clarkbdmsimard: meaning there is already a mysql db running somewhere?19:24
dmsimardyes, a trove instance19:24
corvusdmsimard: did he provide the connection info so we can update that change to use it?19:25
dmsimardWe can definitely start with sqlite, I'm just not sure how things might behave with concurrency19:25
corvuslet's not start with sqlite, let's use the trove which apparently exists :)19:25
clarkband would be helpful if we remember to #status log changes like that that happen outside of config mgmt19:26
dmsimardclarkb: my default19:26
clarkbcorvus: ++19:26
dmsimardcorvus: the connection information is secret though, so it needs to live on bridge.o.o ?19:26
dmsimardI guess we could add a node to the job, install mysql on it and use that for the testinfra tests19:27
corvusdmsimard: only the password i think; paul should have added that to private hiera and given you the key he put it under so the change can reference thta key19:27
corvusi think sqlite is ok for testinfra19:27
*** jamesmcarthur has quit IRC19:27
*** david-lyle has joined #openstack-meeting19:27
fungiif it's a trove instance, we usually keep the hostname secret too19:27
corvusok 2 things then :)19:28
fungisince there is no connection security there19:28
*** dklyle has quit IRC19:28
dmsimardcorvus: I'm the one who committed the changes, they're in hostvars for bridge.o.o as well as ara.o.o19:28
*** jamesmcarthur has joined #openstack-meeting19:28
ianwis there some potential for keys/passwords to make it into this?  do we care?19:28
fungiany machine for any tenant in that same region can open sockets to the trove mysql service for it, after all19:28
dmsimardpabelanger helped me through the process of doing that, he didn't do it for me, sorry for the confusion19:28
clarkbianw: that was going to be my next question. Do we need to use no_log type attributes on tasks to avoid data exposure?19:28
corvusdmsimard: oh i thought you said paul did it.  ok then :)19:29
clarkbmaybe to start we should run it locally only and we have to ssh port forward into bridge to see the web server?19:29
clarkbto get a feel for what data is exposed19:29
corvusclarkb: wfm19:29
dmsimardclarkb: yes19:29
dmsimardianw: and yes, data can be exposed19:29
fungisounds fine19:30
fungi(not publishing the ara reports i mean)19:30
clarkbok lets start with localhost only webserver to see what we are exposing, remediate it and learn from there then maybe make it public19:30
*** jamesmcarthur has quit IRC19:30
corvusand go ahead and have bridge report to mysql19:30
clarkb(goal should be making it open in some capacity though)19:30
clarkbcorvus: yup19:30
ianwoh, i wasn't really thinking publishing the website, i was more back at "data at rest not on bridge.o.o" tbh19:31
fungiyeah, eventually it would be nice to get back what we lost with puppetboard being abandoned19:31
dmsimardclarkb: the original spec was to replace puppetboard19:31
corvusdmsimard: thanks for this, it's going to help a lot :)19:31
clarkbianw: ah the actual db contents themselves19:32
fungithat's also a great point19:32
*** jamesmcarthur has joined #openstack-meeting19:32
dmsimardare the trove instances on the public internet ?19:32
clarkbdmsimard: no19:32
fungipossible we're compromising the security of bridge.o.o by putting the database somewhere not locally on that server19:32
clarkbthey are on the rax private network19:32
*** bobh has joined #openstack-meeting19:32
fungidmsimard: they might as well be though19:32
clarkbwhich is "private"19:33
fungithey're on the shared "private" network which all rackspace tenants can access19:33
dmsimardif this is a concern, putting mysql on bridge.o.o is another option which would also happen to reduce latency and improve performance (i.e, no roundtrip to put stuff in a remote database far away)19:33
fungiso you need access to a rackspace account/vm and to know (or guess) the database address and credentials19:33
fungior a pre-authentication zero-day vulnerability in mysql19:34
dmsimardbut then we might as well set up the web application there as well (instead of a dedicated ara.o.o server)19:34
ianwyeah, also possibly the communications aren't encrypted?19:34
corvusbridge is not sized correctly to even run ansible; there's no way we can add mysql there without a rebuild.19:34
fungiianw: no possible about it. the database connections aren't encrypted, so someone with the ability to subvert that network could also sniff them maybe? i can't remember whether they're sent in the clear or as a challenge/response19:35
clarkbanother option would be to run a mysql off host ourselves19:35
clarkbthen add encryption and such19:35
fungibut also possible to inject into an established socket with some effort i guess19:35
clarkbthen we don't need to rebuild bridge.o.o for this19:36
dmsimardNeed to drop momentarily to pick up kids19:36
corvusmeh, we need to rebuild bridge anyway, so if we want to stick with on-host, i wouldn't count that as a strike against it19:36
clarkbalso possible trove has grown the feature to allow us to use tls/ssl for connections19:36
clarkbcorvus: fair neough19:36
fungimaybe we use trove initially with the understanding that when (not if) we resize bridge (because it's already not appropriately-sized for ansible anyway) we add enough capacity to also put mysql on it?19:36
fungior also what corvus said19:36
clarkbwhy don't we pick this back up in #openstack-infra when dmsimard can rejoin? and we can continue with the meeting agenda?19:37
ianwit may also be that we consider it low risk enough that we are leaking in the first place, it doesn't matter19:37
ianwor what fungi said :) ++19:37
clarkbI do think being careful around this is worthwhile while we work to understand what we are exposing19:37
corvusyeah, we're not *supposed* to be putting private stuff in this db, the question is what happens if we accidentally do19:37
fungiwe're all just repeating one another at this point anyway, so yeah we can pick it back up in #-infra later ;)19:37
corvusmaybe we should pick it back up in #-infra later19:38
clarkb#topic General topics19:38
*** openstack changes topic to "General topics (Meeting topic: infra)"19:38
*** bobh has quit IRC19:38
clarkbI'll take that as a cue19:38
clarkbOpenDev is a thing and we now have some concrete early task items planned out. Step 0 is getting dns servers running for opendev.org (thank your corvus for getting this going)19:39
clarkbbefore we can start using opendev.org everywhere we need to communicate what we mean by opendev and what the goals we have are19:39
clarkb#link https://etherpad.openstack.org/p/infra-to-opendev-messaging is the document we started working with yesterday between the infra team and the foundation staff to try and capture some of that19:40
clarkbI will be working with the foundation staff to write up the short message with a Q&A and use that to send a community wide email type thing. There is also plan to talk about this at the summit at least at the board meeting19:40
clarkbMy hunch is once we get through the summit we'll be able to start concretely using opendev and maybe etherpad becomes etherpad.opendev.org the week after summit19:41
clarkbStill a lot of details to sort out, I expect we'll be reviewing documents in the near future to make sure they make sense from our perspective19:41
clarkband for anyone wondering the current working one liner is: "Community hosted tools and infrastructure for developing and maintaining open source software."19:41
clarkbquestions, concerns about ^?19:42
fungii like the specificity of "free/libre open source software" but as a first go it's not bad19:43
fungi"community hosted and operated" might also be a good expansion19:44
clarkbFeel free to reach out to me directly as well if this venue doesn't work for you19:44
clarkbI'm reachable via email and irc19:44
fungisomething to make it more obvious we're running these things as a community, not just hosting them for communities19:44
clarkbfungi: you should add those thoughts to the etherpad :)19:44
clarkbNext item is the trusty upgrade sprint. Last week ended up being a wierd week for this as various individuals had less availabiltiy then initially anticipated19:45
*** armstrong has quit IRC19:46
clarkbThat said I did manage to upgrade logstash.o.o, etherpad-dev.o.o, and etherpad.o.o and am working with shrews this week to delete nodepool.o.o and use our newer zookeeper cluster instead19:46
clarkbThank you to all of you that helped me with random reviews to make that possible19:46
*** mjturek has quit IRC19:46
clarkbI'd like to keep pushing on this as a persistent thing because trusty EOL is about 6 months away19:46
clarkbif we are able to do a handful of servers a week we should be done relatively quickly19:46
clarkb#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup list of trusty servers that need to be upgraded. Please put your name next to those you can help with19:47
clarkbThat brings up the last thing I wanted to talk about which is the zookeeper cluster move19:48
clarkbI got the zookeeper cluster up and running as a cluster yesterday. Shrews and I will be moving the nodepool builders to the cluster after this meeting19:48
clarkbThose will then run for a day and a half or so to populate images on the new db19:48
clarkbThen on thursday I'd like us to update the zookeeper config on the nodepool launchers and zuul scheduler to use the new cluster19:49
clarkbWe will likely implement this as a full cluster shutdown (may as well get everything running latest code, and maybe zuul can make a release based on that)19:49
clarkbhrm apparently thursday is the stein-1 milestone day though :/19:50
fungisounds great, thanks for working on that19:50
clarkbwe might have to wait for the milestone stuff to happen first?19:50
clarkbI'll talk to release team after htis meeting19:50
fungimilestone 1 is usually a non-event, especially now that openstack is no longer tagging milestones19:50
*** lbragstad has quit IRC19:50
clarkboh in that case ya should be fine. But I will double check19:51
fungithough they may force releases for cycle-with-intermediary projects which haven't released prior to the milestone week19:51
clarkbOnce all that is done we'll want to go back through and cleanup any images and nodepool nodes that are not automatically cleaned up19:51
clarkbI'm volunteering myself to drive and do a lot of that because I do think this is an important switch and early cycle is the time to do it19:52
clarkbyou are now all warned and if you want to follow along or help let me know :)19:52
clarkb#topic Open Discussion19:52
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:52
clarkbAnd now ~7 minutes for anything else19:52
Shrewswe need to get them cleaned up since the image numbers will reset (and thus avoid name collisions), but we should have plenty of leeway on doing that19:52
*** lbragstad has joined #openstack-meeting19:53
*** david-lyle is now known as dklyle19:53
clarkbthats a good point, but ya ~1 year of nodepool with zk should give us a big runway for that19:53
Shrewssmallest sequence number i saw was 300 some, but still plenty19:54
corvuswe will definitely.  almost certainly.  probably.  get it all cleaned up in a year.19:54
fungii like your optimism!19:54
clarkbif anyone is wondering zk has weird way of determining which ip address to listen to19:56
clarkbyou configure a list of all the cluster members and for the entry that belongs to the current host it binds to the address in that entry19:57
clarkbso if you use hostnames that resolve to localhost because /etc/hosts then you only listen to localhost and can't talk to your cluster19:57
*** jamesmcarthur has quit IRC19:58
clarkblooks/sounds like we are basically done here. Thank you everyone. Find us on the openstack-infra mailing list or in #openstack-infra for further discussion19:58
clarkbI'll go ahead and end the meeting now19:58
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"19:58
openstackMeeting ended Tue Oct 23 19:58:25 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:58
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-23-19.01.html19:58
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-23-19.01.txt19:58
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-23-19.01.log.html19:58
fungithanks clarkb!19:58
*** Shrews has left #openstack-meeting19:58
*** jamesmcarthur has joined #openstack-meeting20:02
*** luobin-smile has quit IRC20:03
*** bobh has joined #openstack-meeting20:05
*** jamesmcarthur has quit IRC20:06
*** jamesmcarthur has joined #openstack-meeting20:09
*** bobh has quit IRC20:09
*** mjturek has joined #openstack-meeting20:14
*** cloudrancher has quit IRC20:21
*** cloudrancher has joined #openstack-meeting20:22
*** SWDevAngel has joined #openstack-meeting20:28
*** ssbarnea_ has quit IRC20:36
*** mjturek has quit IRC20:39
*** mjturek has joined #openstack-meeting20:40
*** mjturek has quit IRC20:41
*** mjturek has joined #openstack-meeting20:44
*** ttsiouts has joined #openstack-meeting20:56
*** bobh has joined #openstack-meeting20:56
*** bobh has quit IRC20:59
*** bobh_ has joined #openstack-meeting20:59
*** bobh_ has quit IRC21:00
*** bobh has joined #openstack-meeting21:01
*** erlon has quit IRC21:03
*** mlavalle has left #openstack-meeting21:04
*** e0ne has quit IRC21:04
*** bobh has quit IRC21:06
*** priteau has quit IRC21:09
*** felipemonteiro has joined #openstack-meeting21:13
*** dustins has quit IRC21:19
*** dklyle has quit IRC21:19
*** dklyle has joined #openstack-meeting21:20
*** raildo has quit IRC21:23
*** jamesmcarthur has quit IRC21:29
*** cloudrancher has quit IRC21:35
*** cloudrancher has joined #openstack-meeting21:36
*** slaweq has quit IRC21:39
*** awaugama has quit IRC21:39
*** rtjure has quit IRC21:39
*** cloudrancher has quit IRC21:40
*** cloudrancher has joined #openstack-meeting21:40
*** dustins has joined #openstack-meeting21:41
*** munimeha1 has joined #openstack-meeting21:44
*** felipemonteiro has quit IRC21:52
*** felipemonteiro has joined #openstack-meeting21:54
*** slaweq has joined #openstack-meeting22:04
*** ttsiouts has quit IRC22:06
*** ttsiouts has joined #openstack-meeting22:06
*** mjturek has quit IRC22:08
*** munimeha1 has quit IRC22:10
*** ttsiouts has quit IRC22:11
*** felipemonteiro has quit IRC22:12
*** eharney has quit IRC22:13
*** dustins has quit IRC22:15
*** ykatabam has joined #openstack-meeting22:23
*** rcernin has joined #openstack-meeting22:24
*** bnemec has quit IRC22:25
*** mriedem has quit IRC22:26
*** slaweq has quit IRC22:38
*** caboucha has quit IRC22:44
*** diablo_rojo has quit IRC22:57
*** diablo_rojo has joined #openstack-meeting23:08
*** hongbin has quit IRC23:10
*** slaweq has joined #openstack-meeting23:11
*** tpsilva has quit IRC23:11
*** ykatabam has quit IRC23:21
*** ykatabam has joined #openstack-meeting23:37
*** slaweq has quit IRC23:45
*** gyee has quit IRC23:46
*** mjturek has joined #openstack-meeting23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!