Tuesday, 2017-02-07

*** tonytan4ever has joined #openstack-meeting00:00
*** spzala has joined #openstack-meeting00:01
*** armstrong has joined #openstack-meeting00:02
*** ykatabam1 has joined #openstack-meeting00:03
*** ykatabam has quit IRC00:03
*** sacharya has joined #openstack-meeting00:03
*** salv-orl_ has quit IRC00:05
*** spzala has quit IRC00:06
*** bobh has joined #openstack-meeting00:07
*** sacharya has quit IRC00:07
*** jamesdenton has quit IRC00:08
*** aeng has quit IRC00:09
*** yangyapeng has quit IRC00:09
*** tonytan_brb has joined #openstack-meeting00:09
*** lamt has quit IRC00:10
*** tonytan4ever has quit IRC00:12
*** iyamahat has quit IRC00:16
*** iyamahat has joined #openstack-meeting00:18
*** thorst_ has joined #openstack-meeting00:19
*** spzala has joined #openstack-meeting00:21
*** aeng has joined #openstack-meeting00:22
*** bobh has quit IRC00:24
*** bobh has joined #openstack-meeting00:27
*** adisky_ has quit IRC00:29
*** hongbin_ has quit IRC00:30
*** iceyao has joined #openstack-meeting00:31
*** sdake has joined #openstack-meeting00:32
*** ansmith has quit IRC00:33
*** jamesdenton has joined #openstack-meeting00:35
*** zz_dimtruck is now known as dimtruck00:40
*** bobh has quit IRC00:43
*** zhurong has joined #openstack-meeting00:45
*** electrichead is now known as rerobot00:45
*** sdake has quit IRC00:45
*** VW has joined #openstack-meeting00:47
*** pvaneck has quit IRC00:50
*** jamesdenton has quit IRC00:51
*** chris_morrell has quit IRC00:51
*** VW has quit IRC00:52
*** adiantum has quit IRC00:54
*** adiantum has joined #openstack-meeting00:54
*** donghao has joined #openstack-meeting00:55
*** spzala has quit IRC00:57
*** iceyao_ has joined #openstack-meeting00:57
*** xingchao has joined #openstack-meeting00:59
*** Julien-zte has joined #openstack-meeting00:59
*** aeng has quit IRC01:00
*** iceyao has quit IRC01:01
*** donghao has quit IRC01:02
*** pleia2_ is now known as pleia201:04
*** ykatabam1 has quit IRC01:05
*** kylek3h_away has joined #openstack-meeting01:06
*** david-lyle has quit IRC01:06
*** spzala has joined #openstack-meeting01:09
*** thorst_ has quit IRC01:09
*** gyee has quit IRC01:10
*** aeng has joined #openstack-meeting01:14
*** yangyapeng has joined #openstack-meeting01:15
*** kevinz has joined #openstack-meeting01:16
*** alexpilo_ has quit IRC01:16
*** yangyapeng has quit IRC01:17
*** yangyapeng has joined #openstack-meeting01:18
*** sacharya has joined #openstack-meeting01:19
*** Julien-zte has quit IRC01:20
*** Julien-zte has joined #openstack-meeting01:20
*** Julien-zte has quit IRC01:21
*** Julien-z_ has joined #openstack-meeting01:21
*** sacharya has quit IRC01:24
*** xgerman has joined #openstack-meeting01:24
*** aweeks has joined #openstack-meeting01:24
*** jkilpatr has quit IRC01:25
*** aeng has quit IRC01:27
*** alexpilotti has joined #openstack-meeting01:27
*** Apoorva has quit IRC01:29
*** ykatabam has joined #openstack-meeting01:31
*** baoli has joined #openstack-meeting01:31
*** takahara has joined #openstack-meeting01:31
*** alexpilotti has quit IRC01:31
*** alexpilotti has joined #openstack-meeting01:34
*** alexpilotti has quit IRC01:38
*** takahara has left #openstack-meeting01:39
*** spzala has quit IRC01:41
*** spzala has joined #openstack-meeting01:42
*** aeng has joined #openstack-meeting01:42
*** mmalik4 has quit IRC01:44
*** guoshan has joined #openstack-meeting01:44
*** xingchao has quit IRC01:44
*** spzala has quit IRC01:46
*** spzala has joined #openstack-meeting01:53
*** huanxuan has joined #openstack-meeting01:56
*** sdague_ has quit IRC01:57
*** sacharya has joined #openstack-meeting01:59
*** xingchao has joined #openstack-meeting02:00
*** dmorita has quit IRC02:01
*** dmorita has joined #openstack-meeting02:03
*** jamesdenton has joined #openstack-meeting02:04
*** julim has joined #openstack-meeting02:07
*** gcb has joined #openstack-meeting02:12
*** armstrong has quit IRC02:15
*** unicell1 has quit IRC02:19
*** jkilpatr has joined #openstack-meeting02:20
*** japestinho has joined #openstack-meeting02:24
*** japestinho has quit IRC02:26
*** japestinho has joined #openstack-meeting02:26
*** gongysh has joined #openstack-meeting02:31
*** alexpilotti has joined #openstack-meeting02:32
*** lhx__ has joined #openstack-meeting02:33
*** jamesdenton has quit IRC02:36
*** alexpilotti has quit IRC02:37
*** VW has joined #openstack-meeting02:38
*** galstrom_zzz is now known as galstrom02:39
*** spzala has quit IRC02:41
*** VW has quit IRC02:43
*** nijaba has quit IRC02:44
*** nijaba has joined #openstack-meeting02:45
*** nijaba has quit IRC02:45
*** nijaba has joined #openstack-meeting02:45
*** s3wong has quit IRC02:46
*** thorst_ has joined #openstack-meeting02:48
*** thorst_ has quit IRC02:48
*** jkilpatr has quit IRC02:48
*** ricolin has joined #openstack-meeting02:49
*** Wenzhi has joined #openstack-meeting02:54
*** hongbin has joined #openstack-meeting02:55
*** sdague has joined #openstack-meeting02:55
*** sudipto_ has joined #openstack-meeting02:55
*** sudipto has joined #openstack-meeting02:55
*** spzala has joined #openstack-meeting02:55
*** ianychoi has quit IRC02:59
*** lakerzhou has joined #openstack-meeting02:59
*** ianychoi has joined #openstack-meeting02:59
hongbin#startmeeting zun03:00
openstackMeeting started Tue Feb  7 03:00:07 2017 UTC and is due to finish in 60 minutes.  The chair is hongbin. Information about MeetBot at http://wiki.debian.org/MeetBot.03:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.03:00
*** spzala has quit IRC03:00
*** openstack changes topic to " (Meeting topic: zun)"03:00
hongbin#link https://wiki.openstack.org/wiki/Zun#Agenda_for_2017-02-07_0300_UTC Today's agenda03:00
*** Namrata has joined #openstack-meeting03:00
openstackThe meeting name has been set to 'zun'03:00
hongbin#topic Roll Call03:00
*** lhx__ has quit IRC03:00
*** openstack changes topic to "Roll Call (Meeting topic: zun)"03:00
mkraiMadhuri Kumari03:00
hongbinthanks for joining the meeting Namrata sudipto lakerzhou kevinz mkrai Wenzhi03:01
*** lhx__ has joined #openstack-meeting03:01
hongbinlet's get started03:01
hongbin#topic Announcements03:01
*** openstack changes topic to "Announcements (Meeting topic: zun)"03:01
hongbin#topic Review Action Items03:01
*** openstack changes topic to "Review Action Items (Meeting topic: zun)"03:01
hongbin1. hongbin create a spec for host capability (DONE)03:01
hongbin#link https://blueprints.launchpad.net/zun/+spec/expose-host-capabilities03:01
hongbinany comment on this bp?03:01
mkraiIs there any spec posted for it?03:02
hongbinmkrai: sudipto has drafted a spec about cpu policy03:02
lakerzhouI haven't got chance to read it, but will do it tomorrow.03:02
mkraiI read the bp and the idea is clear03:02
hongbinhey eliqiao_03:03
eliqiao_I am late ,sorry, in a meeting.03:03
hongbinlakerzhou: thanks for your comment on the spec, it is helpful03:03
hongbinok, moving on03:03
hongbin#topic Cinder integration (diga)03:03
*** openstack changes topic to "Cinder integration (diga) (Meeting topic: zun)"03:03
hongbin#link https://blueprints.launchpad.net/zun/+spec/cinder-zun-integration The BP03:03
hongbin#link https://review.openstack.org/#/c/417747/ The design spec03:03
hongbinit looks diga is not here03:04
hongbini saw he submitted a patch for that03:04
hongbin#link https://review.openstack.org/#/c/429943/03:04
*** thorst_ has joined #openstack-meeting03:04
hongbinfeel free to comment on the patch if you like03:04
*** thorst_ has quit IRC03:04
hongbinany comment on this bp?03:04
*** mickeys has quit IRC03:04
*** vishnoianil has quit IRC03:05
hongbinok, next topic03:05
hongbin#topic Kuryr integration (hongbin)03:05
*** openstack changes topic to "Kuryr integration (hongbin) (Meeting topic: zun)"03:05
hongbin#link https://blueprints.launchpad.net/zun/+spec/kuryr-integration The BP03:05
hongbini am working on this one, there is a spec that seems to be closed03:05
*** jkilpatr has joined #openstack-meeting03:06
hongbin#link https://review.openstack.org/#/c/425883/03:06
hongbinfeel free to comment on it if you interest03:06
hongbinany comment?03:06
hongbinok, next one03:07
hongbin#topic Support interactive mode (kevinz)03:07
*** openstack changes topic to "Support interactive mode (kevinz) (Meeting topic: zun)"03:07
*** janonymous has joined #openstack-meeting03:07
hongbin#link https://blueprints.launchpad.net/zun/+spec/support-interactive-mode The BP03:07
hongbin#link https://review.openstack.org/#/c/396841/ The design spec03:07
hongbinkevinz: ^^03:07
kevinzHi hongbin, I've submitted a new patch set after address you and Pradeep's comments03:08
*** epico has joined #openstack-meeting03:08
kevinzIf the API and code is Ok, I will add more test cases for this03:08
hongbincool, will find some time tomorrow to go through it again03:09
kevinzOK Thanks :D03:09
hongbinkevinz: a question for you, which version of docker-py you required for this feature?03:09
*** mickeys has joined #openstack-meeting03:10
kevinzhongbin: Now it's fine with currently version.03:10
hongbinkevinz: ack03:10
hongbinany other comment on this feature?03:11
sudipto_I will go through the code today.03:11
hongbinthanks kevinz sudipto_03:11
kevinzsudipto_: Cool, thanks03:12
hongbinok, next one03:12
hongbin#topic Introduce host capabilities and cpusets03:12
*** openstack changes topic to "Introduce host capabilities and cpusets (Meeting topic: zun)"03:12
hongbin#link https://review.openstack.org/#/c/427007/ The spec03:12
hongbinsudipto_: you want to drive this one?03:12
sudiptoOk, so this is the work for exclusive cpusets that drove us to think about a light weight scheduler to begin with.03:12
*** david-lyle has joined #openstack-meeting03:13
*** sdague has quit IRC03:13
sudiptolakerzhou, recommended that we do this work keeping in mind the nova's new placement API.03:13
lakerzhou"ZUN won't support a compute host to have both dedicated and a shared policy at the same time applicable to it. " sounds to me, this assumption is too strong03:13
mkraiIs there any link for nova's new placement API? What is it?03:13
hongbin#link http://docs.openstack.org/developer/nova/placement.html03:14
sudiptolakerzhou, the reason why this is dicey is because, for a shared case - you wouldn't know which cpusets are being shared - and then if the user requests dedicated - then you may take out the already in use cpusets03:14
sudiptothe same behaviour is in Nova as well today.03:14
lakerzhoucan you please explain a little bit why to make such assumption? we can talk offline if you prefer03:14
sudiptoNova's new placement APIs are still getting ironed out, so i have made a few 'ease of use' assumptions for us.03:15
lakerzhouThe assumption is different than existing nova's behavior.03:16
*** VW has joined #openstack-meeting03:16
lakerzhoudo you think it is final design, or temporary solution?03:17
sudiptolakerzhou, http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html#proposed-change - In the proposed change section - the second last paragraph03:17
*** diga has joined #openstack-meeting03:18
lakerzhouok, I will check again. let 's take it offline. It sounds a big change to me.03:18
hongbindiga: hey, thanks for joining03:18
sudiptolakerzhou, this behaviour can be changed, given we can isolate NUMA nodes within a given host that would be used for pinning. However, if you have a solution w.r.t how this mixed case can be achieved, then why not.03:18
digasorry got late03:18
sudiptolakerzhou, sounds good. Your reviews have been helpful.03:18
sudiptoRequest everyone to take a look and comment as much as possible. Esp on the implementation details.03:19
sudiptohongbin, i am done :-)03:19
digasudipto: I hv gone through your spec, we are only considering NUMA nodes for this BP ?03:19
hongbinsudipto: thx03:20
sudiptodiga, you could do a normal pinning as well - on a system that does not come with the NUMA capabilities (older generation) - but i thought supporting hyper threading for such systems is a must.03:20
digasudipto: okay03:20
sudiptodiga, however, these things can be worked out once we have the first draft of the code out.03:21
sudiptoI was just taking the most important/simplistic case to begin with.03:21
digasudipto: let me know if you need any help, I am working with nova-placement & have good exp in placement scenarios03:21
*** iyamahat has quit IRC03:22
*** yamahata has quit IRC03:22
sudiptodiga, great, first of all, i would need your help in reviewing the spec as brutally as possible :-)03:22
digasudipto: sure, will go through it today03:22
sudiptohongbin, we are done i think.03:23
sudiptoas in on this topic03:23
*** cloud_player has joined #openstack-meeting03:23
hongbinany further comment on this topic?03:23
hongbinseems not03:23
hongbindiga: you want to give a brief update about the cinder volume work?03:23
digahongbin: yes03:23
hongbindiga: go ahead03:24
digahongbin: cinder  & docker related work in remaining, API/DB level work is done03:24
*** julim has quit IRC03:25
digahongbin: I am targeting cinder & Docker to complete in 2 days03:25
hongbindiga: cool03:25
digahongbin: simultaneosly working on zunclient for volume create/show/list/delete03:25
digahongbin: I think i will try to complete everything by next week core code, test case i will try to complete but not sure03:26
hongbindiga: sound great03:26
*** takahara has joined #openstack-meeting03:26
*** gouthamr has quit IRC03:26
hongbindiga: thanks for your hard work on this one03:27
sudiptodiga, thank you!03:27
sudiptoI would ben more than happy to review the code.03:27
digasudipto: hongbin: welcome!03:27
digasudipto: :) sure03:27
hongbinok, then let's enter open discussion03:27
hongbin#topic Open Discussion03:27
*** openstack changes topic to "Open Discussion (Meeting topic: zun)"03:27
hongbinNamrata: you want to give a brief update about your Heat resource work?03:28
Namratayeah sure.03:28
digahongbin: I think we should plan mid-cycle meeting or remote PTG in this month (one or two days), to plan Pike cycle features03:28
*** julim has joined #openstack-meeting03:29
NamrataI have added zun resourceshttps://review.openstack.org/#/c/426210/03:29
hongbindiga: one second03:29
Namratahttps://review.openstack.org/#/c/429259/ zun client plugin too03:29
hongbin#link https://review.openstack.org/#/c/426210/03:29
hongbin#link https://review.openstack.org/#/c/429259/03:30
Namratai will join the heat meeting tommorow and ask for reviewing the patches03:30
hongbinNamrata: thanks03:30
hongbinNamrata: the patches look good to me so far03:30
Namratafurther i will be working on container resource03:30
Namratathanks hongbin03:31
hongbinyes, it will be a very useful feature03:31
hongbini knew solum team is looking for that03:31
digaNamrata: hongbin : this work should go to heat correct ? zun driver in Heat ?03:31
mkraidiga: Yes03:32
*** alexpilotti has joined #openstack-meeting03:32
hongbinok, does anyone else want to bring a topic for team discussion?03:33
eliqiao_yes hongbin03:33
hongbineliqiao_: go ahead03:33
eliqiao_would like to talk the disadvantage of some detail03:34
hongbindisadvantage of ?03:34
eliqiao_I tried zun recently, and found that if I create a sandbox with novadocker driver03:34
eliqiao_we will ensure sandbox is ready (a poll with timeout)03:34
eliqiao_but if I delete the container before the sandbox is ready, the poll is still going.03:35
hongbini see03:35
eliqiao_the correct logic is need to notice it and stop polling...03:35
hongbineliqiao_: i think you filed a bug for that already?03:36
eliqiao_seems no good solution to resolve this :(03:36
digabetter we should fix timeout issue, for every driver we use, this is typical scenarios we should handle03:36
eliqiao_not yet03:36
*** spzala has joined #openstack-meeting03:36
hongbineliqiao_: mind filing one?03:36
eliqiao_will do it later.03:36
hongbin#action eliqiao_ create a bug for polling issue03:37
*** alexpilotti has quit IRC03:37
eliqiao_besides, another issue is that python-zunclent doesn't give detail error message03:37
eliqiao_but lower prority03:37
eliqiao_it's about uer experence, will file bug later..03:38
hongbineliqiao_: great, thanks for that03:38
*** ihrachys has joined #openstack-meeting03:38
eliqiao_I am done, will try zun more later and hope can give more feedback03:39
hongbineliqiao_: thx03:39
eliqiao_thanks for listening :)03:39
*** ihrachys has quit IRC03:39
hongbinany other topic ?03:39
hongbinok, all. thanks for joining the meeting03:40
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"03:40
openstackMeeting ended Tue Feb  7 03:40:06 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)03:40
openstackMinutes:        http://eavesdrop.openstack.org/meetings/zun/2017/zun.2017-02-07-03.00.html03:40
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/zun/2017/zun.2017-02-07-03.00.txt03:40
openstackLog:            http://eavesdrop.openstack.org/meetings/zun/2017/zun.2017-02-07-03.00.log.html03:40
*** jkilpatr has quit IRC03:40
*** iyamahat has joined #openstack-meeting03:41
*** iyamahat has quit IRC03:41
*** links has joined #openstack-meeting03:41
*** iyamahat has joined #openstack-meeting03:41
*** spzala has quit IRC03:41
*** yamahata has joined #openstack-meeting03:42
*** galstrom is now known as galstrom_zzz03:47
*** sagara has joined #openstack-meeting03:52
*** jungleboyj has joined #openstack-meeting03:53
*** spzala has joined #openstack-meeting03:53
*** lakerzhou has quit IRC03:53
*** sdague has joined #openstack-meeting03:54
*** zhurong has quit IRC03:57
*** spzala has quit IRC03:58
*** galstrom_zzz is now known as galstrom03:58
*** ijw has joined #openstack-meeting03:58
*** thorst_ has joined #openstack-meeting03:59
*** tpatil has joined #openstack-meeting03:59
*** samP has joined #openstack-meeting03:59
*** takashi has joined #openstack-meeting03:59
*** rkmrHonjo has joined #openstack-meeting04:00
*** thorst_ has quit IRC04:00
*** donghao has joined #openstack-meeting04:01
*** abhishekk has joined #openstack-meeting04:01
samPhi all..04:01
samPlest start04:01
*** Dinesh_Bhor has joined #openstack-meeting04:01
samP#startmeeting masakari04:01
openstackMeeting started Tue Feb  7 04:01:22 2017 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:01
*** openstack changes topic to " (Meeting topic: masakari)"04:01
openstackThe meeting name has been set to 'masakari'04:01
*** diablo_rojo has quit IRC04:01
samP#topic Bugs04:01
*** openstack changes topic to "Bugs (Meeting topic: masakari)"04:01
samPAny Bug to discuss?04:01
samPOK, rkmrHonjo pls go first04:02
*** guoshan has quit IRC04:02
rkmrHonjoReturn user-friendly error message when monkey_patch enabled https://review.openstack.org/#/c/426648/04:02
rkmrHonjo , Switch to oslo_log https://review.openstack.org/#/c/421172/04:02
*** ijw has quit IRC04:03
rkmrHonjoI and Takahara commented to above patches, but there were no replies.04:03
rkmrHonjoIf author or co-developper of these patch are joining this meeting, please check  and reply our comments.04:03
*** VW has quit IRC04:03
*** galstrom is now known as galstrom_zzz04:03
abhishekkrkmrHonjo: we are working on adding hacking checks, will push a patch soon04:04
*** VW has joined #openstack-meeting04:04
rkmrHonjoahhishekk: OK, thanks.04:04
samPabhishekk: thanks..04:04
*** dmorita has quit IRC04:04
abhishekkno issues04:04
abhishekkjenkins is failing on masakai, please refer http://logs.openstack.org/38/407538/5/check/gate-masakari-python27-ubuntu-xenial/f4435c2/console.html04:05
abhishekkthis is because maskari is not synced with upper-constraints of openstack/requirements04:05
*** ykatabam has quit IRC04:06
*** fnaval_ has joined #openstack-meeting04:06
abhishekktest is failing as invalid version of jsonchema library (2.6.0) is getting installed04:06
takashiabhishekk: you mean we need something like https://github.com/openstack/nova/blob/master/tox.ini#L13, right?04:06
takashito set upper constraints04:06
*** donghao has quit IRC04:07
samPthat would be nice04:07
abhishekksamP, takashi, I will create a patch, test this and then push it for review ASAP04:08
takashiabhishekk: sure. I don't like to keep this blocker for long time... :-(04:09
abhishekktakashi san: yes04:09
*** Namrata has quit IRC04:09
takashiwill keep my eyes on gerrit :-)04:09
samPI found another issue with oslo.log, in current req is oslo.config>=3.10.0 for masakari. minimum version should be oslo.config==3.12.004:09
samPsorry oslo.config04:10
samPabhishekk: thanks..04:10
samPThe reason is URIOpt is introduced in oslo.config==3.12.004:11
abhishekktakashi san, sure04:11
takashisamP: As far as I seen in global requreiemts, it should be 3.11.0 https://github.com/openstack/requirements/blob/master/global-requirements.txt#L12604:11
*** bkopilov_ has joined #openstack-meeting04:12
*** bkopilov has joined #openstack-meeting04:12
rkmrHonjosamP: oslo.config is the target of your opinion, don't you? It should be >=3.14.0, !=3.18.0 https://github.com/openstack/requirements/blob/master/global-requirements.txt#L12104:12
*** fnaval has joined #openstack-meeting04:13
tpatilsamP: we should enable OpenStack proposal bot to sync it with global requirements.txt04:13
*** baoli has quit IRC04:13
samPtakashi: thanks, I think masakari need oslo.config>=3.12.0, but global req is 3.11.004:13
takashisamP: ok04:13
takashitpatil: +104:13
tpatilto resolve this issue permanently. for the time being, let's sync it manually as per global requirements04:14
samPtakashi: +1 and thanks04:14
takashitpatil: +1, we need to sync global requirements before we release our Ocata version. IMO it would be great if we can set up proposal bot in the beggining of next cycle.04:15
*** fnaval_ has quit IRC04:15
samPOk, then, lets move to next bug, abhishekk ?04:15
abhishekksamP: Ihave talked about jenkins failure just above04:16
samPtakashi: agree.04:16
abhishekks/Ihave/I have/04:16
samPabhishekk: great..ok then04:16
abhishekksamP: jsut reported it, https://bugs.launchpad.net/masakari/+bug/166240804:16
openstackLaunchpad bug 1662408 in masakari "Masakari is not synced with upper-contraints of global requirements" [Undecided,New]04:16
*** rajinir has quit IRC04:16
samPabhishekk: thanks04:17
takashisamP: Can I ask you to confirm the bug?04:17
samPAny other bugs?04:17
takashiI mean, change its status to 'confirmed'04:18
*** Nakato_ is now known as Nakato04:18
samPtakashi: I think I just did04:18
takashisamP: thanks!04:18
takashinothing else from my side04:19
samPtakashi: np04:19
*** tonytan_brb has quit IRC04:19
samPOK then, lest move to Discussion Points04:19
*** thorst_ has joined #openstack-meeting04:19
*** thorst_ has quit IRC04:19
*** ricolin has quit IRC04:19
samP#topic Discussion04:19
*** openstack changes topic to "Discussion (Meeting topic: masakari)"04:19
samPI raised one issue about, deprecating bash scripts of host and process monitors04:20
samP#link https://review.openstack.org/#/c/427584/04:20
*** armax has quit IRC04:21
samPsince we have new python monitors, we will not need these bash script monitors,04:21
samPHowever, some ppl are still using these for testing and evaluate masakari,04:22
samPTherefore, my proposal is lets leave them as it is for Ocata and deprecate/remove them in Pike04:23
*** dmorita has joined #openstack-meeting04:23
rkmrHonjosamP: I don't have any objections.04:23
samPwe can add this message in each bash script, so users will know this.04:24
rkmrHonjosamP: Agree. I'll implement it.04:25
*** hongbin has quit IRC04:26
samPif no objections, lets add message to each bash script and leave them as it is for Ocata release as deprecate candidates.04:26
*** ykatabam has joined #openstack-meeting04:26
samPrkmrHonjo: OK then04:26
samP#action rkmrHonjo  add message to each bash monitor scripts and leave them as it is for Ocata release as deprecate candidates.04:26
samP#topic AOB04:27
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:27
samPany other topics to discuss?04:27
*** dmorita has quit IRC04:27
tpatilImplement reserved_host recovery action04:27
tpatil#link : https://review.openstack.org/#/c/423072/04:28
tpatilIn the specs, Abhishek has proposed to use tooz library for acquiring locks04:28
tpatiltooz library is meant to use if you want to access locks across distributed services04:29
tpatilbut in our case, masakari-engine will run on a single host04:29
tpatilso it's better to add tooz support when we decide to run masakari-engine on multiple hosts04:30
tpatilwhat you guys suggest?04:30
takashilet me talk a little about its background04:30
takashito implement reserved host recovery method, we need to implement lock mechanism over reserved host04:31
takashiso that we don't use same host for multiple failure notifications04:31
takashito implement lock mechanism there are two ways04:31
takashi1. Implement very simple lock mechanism using lock file(?). This is very simple, but can not manage lock among multiple nodes04:32
takashiso we need to act/stb setup for masakari engine04:32
takashi2. Implement lock mechanis using tooz, which is used in cinder-volume act/act04:32
takashiso that we realize act/act setup about masakari-engine04:32
tpatiltakashi: you are correct04:33
Dinesh_Bhorreference: tooz use in openstack: http://codesearch.openstack.org/?q=tooz%3E%3D&i=nope&files=&repos=04:33
samPtakashi: make sense04:33
tpatiltooz does supports both04:33
takashiI had some discussions with tpatil in this morning, and IMO 1 looks better, because 2 takes some more time to complete, and we can do that migration to tooz later04:34
*** alexpilotti has joined #openstack-meeting04:35
samPMake masakari Act/Act would be nice, but we have consider other issues such as recovery flow management and state management.04:37
*** unicell has joined #openstack-meeting04:38
samPI agree with takashi, we can implement lock with 1 and move to tooz later04:38
samPas a part of make masakari act/act04:38
*** Daisy has joined #openstack-meeting04:39
*** takahara_ has joined #openstack-meeting04:40
takashisamP: +104:40
takashidoes it make sense to other guys?04:40
*** alexpilotti has quit IRC04:40
tpatilsamP: +104:40
rkmrHonjosamP: +1.04:41
sagarasamP: +104:41
*** takahara has quit IRC04:42
samPOK, any objections? if so, raise now04:42
samPseems not, you may raise them later. for now lets make it confirm04:42
samPabhishekk: could you please add above discussion to spec?04:43
abhishekksamP: yes04:43
samPabhishekk: great thanks04:43
tpatilsamP: when are you planning to cut stable/ocata branch?04:44
samP#action abhishekk update the Implement reserved_host recovery action with locking method details04:44
samPtpatil: 2/15 end of the date04:45
tpatilsamP: OK, thanks04:45
samPso we can make the announce on 2/1604:46
tpatilsounds good04:46
samPtpatil: we have 2-3 days for just in case04:46
takashiwould be great if we can check merge status in next irc meeting04:47
samPtakashi: that would be great..04:48
samPdo we need a place to write them down before next irc?04:49
*** jrobinson is now known as jrobinson-afk04:50
tpatilsamP: I will create an etherpad with Ocata priorities and add it to the agenda04:50
samPtpatil: great thanks04:50
*** sudipto_ has quit IRC04:50
*** sudipto has quit IRC04:50
*** sudipto_ has joined #openstack-meeting04:51
*** sudipto has joined #openstack-meeting04:51
*** sudipto has quit IRC04:51
samP#action tpatil Create an etherpad with Ocata priorities and add it to the next irc agenda04:51
*** sudipto_ has quit IRC04:51
*** sdake has joined #openstack-meeting04:52
samPtakashi: in previous meeting you mentioned about #openstack-masakari channel04:52
*** alexpilotti has joined #openstack-meeting04:53
takashisamP: I remember that I was just concerned about meeting time and proposed you to move to our local channel (because this channel is shared04:55
takashisamP: do you mean that?04:55
samPtakashi: oh...ok then. I thought you was proposing to close that channel.04:56
takashisamP: no. I don't know whether we should merge our channel to openstack-ha, but I don't feel big requirement for that04:56
takashibecause now it is working04:57
takashifor us04:57
*** alexpilotti has quit IRC04:58
samPtakashi: yep.. IMO, lets leave it for a while, because some ppl come seek for help there04:58
takashisamP: yes04:59
samPOK then, is almost time.. any other things to discuss?04:59
*** ayogi has joined #openstack-meeting04:59
tpatilnothing from my end05:00
*** csomerville has quit IRC05:00
samPLets end this meeting. Thank you all...05:00
*** csomerville has joined #openstack-meeting05:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"05:00
openstackMeeting ended Tue Feb  7 05:00:54 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)05:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2017/masakari.2017-02-07-04.01.html05:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2017/masakari.2017-02-07-04.01.txt05:00
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2017/masakari.2017-02-07-04.01.log.html05:00
*** sankarshan has joined #openstack-meeting05:01
samPthank you all....05:01
*** sankarshan has quit IRC05:01
*** sankarshan has joined #openstack-meeting05:01
*** VW has quit IRC05:01
*** rkmrHonjo has left #openstack-meeting05:02
*** ykatabam has quit IRC05:02
*** sagara has left #openstack-meeting05:03
*** rkmrHonjo has joined #openstack-meeting05:03
*** rkmrHonjo has quit IRC05:03
*** tpatil has quit IRC05:04
*** bswartz has quit IRC05:08
*** Daisy has quit IRC05:08
*** trozet has quit IRC05:08
*** Daisy has joined #openstack-meeting05:08
*** sankarshan has left #openstack-meeting05:10
*** Daviey has quit IRC05:11
*** sudipto has joined #openstack-meeting05:12
*** sudipto_ has joined #openstack-meeting05:12
*** Daisy has quit IRC05:13
*** anilvenkata has joined #openstack-meeting05:13
*** spzala has joined #openstack-meeting05:15
*** ykatabam has joined #openstack-meeting05:17
*** nadya has joined #openstack-meeting05:19
*** nadya has quit IRC05:19
*** spzala has quit IRC05:20
*** trozet has joined #openstack-meeting05:22
*** trinaths has joined #openstack-meeting05:22
*** mickeys has quit IRC05:23
*** markvoelker_ has quit IRC05:28
*** markvoelker has joined #openstack-meeting05:28
*** takahara has joined #openstack-meeting05:29
*** maishsk has quit IRC05:31
*** jrobinson-afk is now known as jrobinson05:32
*** markvoelker has quit IRC05:33
*** takahara_ has quit IRC05:33
*** sdake_ has joined #openstack-meeting05:34
*** Daisy has joined #openstack-meeting05:36
*** dmorita has joined #openstack-meeting05:36
*** sdake has quit IRC05:36
*** fnaval has quit IRC05:38
*** fnaval has joined #openstack-meeting05:39
*** aeng has quit IRC05:39
*** aeng has joined #openstack-meeting05:39
*** salv-orlando has joined #openstack-meeting05:39
*** david-lyle has quit IRC05:40
*** Daisy has quit IRC05:41
*** trozet has quit IRC05:42
*** fnaval has quit IRC05:44
*** salv-orlando has quit IRC05:44
*** dmorita has quit IRC05:47
*** sacharya has quit IRC05:48
*** trozet has joined #openstack-meeting05:49
*** takashi has quit IRC05:49
*** donghao has joined #openstack-meeting05:51
*** links has quit IRC05:51
*** Daisy has joined #openstack-meeting05:52
*** lpetrut has joined #openstack-meeting05:54
*** donghao has quit IRC05:55
*** liuyulong has quit IRC05:59
*** r-mibu has quit IRC05:59
*** rcernin has joined #openstack-meeting06:01
*** dmorita has joined #openstack-meeting06:01
*** rcernin has quit IRC06:03
*** rcernin has joined #openstack-meeting06:04
*** r-mibu has joined #openstack-meeting06:05
*** mickeys has joined #openstack-meeting06:05
*** dmorita has quit IRC06:07
*** links has joined #openstack-meeting06:07
*** benj_ has quit IRC06:13
*** benj_ has joined #openstack-meeting06:14
*** lhx__ has quit IRC06:17
*** thorst_ has joined #openstack-meeting06:20
*** links has quit IRC06:22
*** sdake_ has quit IRC06:22
*** nkrinner_afk is now known as nkrinner06:22
*** zhurong has joined #openstack-meeting06:23
*** thorst_ has quit IRC06:25
*** Dinesh_Bhor has left #openstack-meeting06:27
*** ekuris__ has joined #openstack-meeting06:28
*** markvoelker has joined #openstack-meeting06:28
*** ricolin has joined #openstack-meeting06:30
*** markvoelker has quit IRC06:33
*** Hosam has joined #openstack-meeting06:35
*** vishnoianil has joined #openstack-meeting06:35
*** ktychkova has quit IRC06:36
*** Daisy has quit IRC06:38
*** links has joined #openstack-meeting06:38
*** Daisy has joined #openstack-meeting06:39
*** salv-orlando has joined #openstack-meeting06:40
*** Hosam_ has joined #openstack-meeting06:41
*** lhx__ has joined #openstack-meeting06:42
*** jrobinson has quit IRC06:42
*** Hosam has quit IRC06:43
*** salv-orlando has quit IRC06:44
*** Daisy_ has joined #openstack-meeting06:45
*** lpetrut has quit IRC06:46
*** Daisy has quit IRC06:48
*** alexpilotti has joined #openstack-meeting06:55
*** alexpilotti has quit IRC06:59
*** ykatabam has quit IRC07:02
*** takahara has quit IRC07:04
*** dimtruck is now known as zz_dimtruck07:06
*** mickeys has quit IRC07:06
*** links has quit IRC07:07
*** nkrinner is now known as nkrinner_afk07:08
*** lhx__ has quit IRC07:09
*** lhx__ has joined #openstack-meeting07:10
*** unicell1 has joined #openstack-meeting07:10
*** unicell has quit IRC07:10
*** salv-orlando has joined #openstack-meeting07:11
*** abhishekk has left #openstack-meeting07:13
*** jungleboyj has quit IRC07:16
*** makowals has joined #openstack-meeting07:18
*** links has joined #openstack-meeting07:19
*** rasca has joined #openstack-meeting07:20
*** makowals_ has quit IRC07:21
*** sridharg has joined #openstack-meeting07:21
*** ricolin has quit IRC07:24
*** kaisers has joined #openstack-meeting07:27
*** markvoelker has joined #openstack-meeting07:29
*** xingchao_ has joined #openstack-meeting07:29
*** trinaths has quit IRC07:30
*** nkrinner_afk is now known as nkrinner07:30
*** links has quit IRC07:30
*** zhurong has quit IRC07:31
*** tesseract has joined #openstack-meeting07:31
*** xingchao has quit IRC07:33
*** abhishekk has joined #openstack-meeting07:34
*** abhishekk has left #openstack-meeting07:34
*** makowals has quit IRC07:34
*** markvoelker has quit IRC07:35
*** zhurong has joined #openstack-meeting07:35
*** guoshan has joined #openstack-meeting07:37
*** Hosam_ has quit IRC07:38
*** yamamoto has quit IRC07:38
*** Hosam has joined #openstack-meeting07:38
*** Hosam has quit IRC07:40
*** Hosam has joined #openstack-meeting07:40
*** andreas_s has joined #openstack-meeting07:41
*** ricolin has joined #openstack-meeting07:41
*** links has joined #openstack-meeting07:43
*** makowals has joined #openstack-meeting07:44
*** pcaruana has joined #openstack-meeting07:46
*** pcaruana has quit IRC07:48
*** salv-orl_ has joined #openstack-meeting07:49
*** makowals has quit IRC07:51
*** salv-orlando has quit IRC07:52
*** cody-somerville has quit IRC07:52
*** trinaths has joined #openstack-meeting07:53
*** matrohon has joined #openstack-meeting07:53
*** pcaruana has joined #openstack-meeting07:55
*** eranrom has left #openstack-meeting07:58
*** eranrom has joined #openstack-meeting07:58
*** fzdarsky has joined #openstack-meeting07:59
*** akihito has joined #openstack-meeting07:59
*** takashi has joined #openstack-meeting07:59
eranrom#startmeeting storlets08:00
openstackMeeting started Tue Feb  7 08:00:13 2017 UTC and is due to finish in 60 minutes.  The chair is eranrom. Information about MeetBot at http://wiki.debian.org/MeetBot.08:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:00
*** openstack changes topic to " (Meeting topic: storlets)"08:00
openstackThe meeting name has been set to 'storlets'08:00
eranromI have updated the agenda: https://wiki.openstack.org/wiki/Meetings/Storlets08:01
eranromplease ignore #108:01
eranromIt is a leftover from last week08:02
eranromPlease let me know if you have anything else.08:02
takashieranrom: thanks08:02
akihitoThank you . I'm looking.08:02
eranrom#topic Ocata release08:02
*** openstack changes topic to "Ocata release (Meeting topic: storlets)"08:03
*** makowals has joined #openstack-meeting08:03
eranromSwift will have a branch during the week of the 14th. Once a branch is created for Swift I will create one for Storlets08:04
*** nijaba has quit IRC08:04
eranromso that the Swift dependency would be on that branch08:04
*** alexpilotti has joined #openstack-meeting08:04
takashieranrom: makes sense08:05
eranromSo we still have some time to land the top priority patches. I guess we mainly nee kota_ here...08:05
takashieranrom: I think so08:05
eranromAnything else on Ocata?08:06
takashieranrom: Can I ask one thing related to your container id fix?08:07
eranromtakashi: please do08:07
*** mickeys has joined #openstack-meeting08:07
*** dmorita has joined #openstack-meeting08:07
takashieranrom: I previously tried to pass whole tenant id to scope as gateway, https://review.openstack.org/42580708:08
takashieranrom: As I think it is more safe because we don't need to consider possible collisions between tenant id when we only use first some characters08:09
*** rossella_s has joined #openstack-meeting08:09
*** alexpilotti has quit IRC08:09
takashido you have any ideas why it fails...? I guess there are some limitations that we need to use part of tenant id08:09
eranromtakashi: right. Originally, we indeed used the whole hash. As far as I recall this caused a problem with path strigns getting too long. although I am not sure.08:11
eranromI can try your patch and find out.08:11
eranromI agree it is better to use the whole hash08:11
*** mickeys has quit IRC08:11
eranromLet me try your patch08:11
*** mickeys has joined #openstack-meeting08:11
takashieranrom: thx08:12
eranromtakashi: sure08:12
*** dmorita has quit IRC08:12
takashiI think that we need to fix before Ocata release, so let me know if you find any difficulties08:12
*** nijaba has joined #openstack-meeting08:13
*** nijaba has quit IRC08:13
*** nijaba has joined #openstack-meeting08:13
takashieranrom: I'll try it by my side too, but honestly speaking I'm not sure that I can take some time for that...08:13
eranromtakashi: sure, will post my findings thru Gerrit. If this works well, I will abondon my patch and place yours in the priority reviews08:13
*** nadya has joined #openstack-meeting08:13
*** makowals has quit IRC08:13
eranromAnything else on Ocata?08:14
takashinothing from my side08:14
*** makowals has joined #openstack-meeting08:15
takashieranrom: I saw your great work about host-only installation yesterday08:15
takashijust confirmation. Do you plan to land that 'AFTER' Ocata release, right?08:15
*** salv-orlando has joined #openstack-meeting08:15
*** yuval has joined #openstack-meeting08:15
eranromtakashi: right. I believe none of you will have time to review before :-)08:15
*** sagara has joined #openstack-meeting08:16
takashieranrom: I love to review it, but yeah I need some time... :-(08:16
eranromsagara: Hi08:16
eranromtakashi: right.08:17
sagaraeranrom: Hi08:17
eranrommoving on.08:17
eranrom#topic PTG08:17
*** openstack changes topic to "PTG (Meeting topic: storlets)"08:17
eranromI do not have anything special to say.08:17
eranromJust to remind the etherpad :-)08:17
eranromplease add topics as you see fit.08:17
takashiI have one news from my side08:17
*** hashar has joined #openstack-meeting08:18
takashibecause of the project assignment in my team, I can not join to PTG this time, sorry.08:18
takashito tell you the truth, we have another urgent project, and I'm assignet to it recently.08:19
*** rossella_s has quit IRC08:19
eranromtakashi: Thanks for the update. I understand08:19
takashieranrom: sagara and akihito will join (I know kota_ also will join), so I think you can discuss with them08:19
eranromtakashi: sure. Thanks.08:20
sagaraeranrom: you know akihito, he writes funtional tests for storlets08:20
eranromtakashi: absolutely.08:20
eranromsagara: Absolutely.08:21
*** salv-orl_ has quit IRC08:21
*** thorst_ has joined #openstack-meeting08:21
takashieranrom kota_ know both of two persons, and I think you met sagara at Barcelona.08:21
eranromSure, we have met in Barcelona08:21
takashilooking forward to hear about great discussion among you later :-)08:22
*** rossella_s has joined #openstack-meeting08:22
eranromtakashi: Sure. Will you still be around or are you going to be totally busy with the new project?08:23
takashieranrom: honestly I'm not sure now. I'd like to keep in touch with this project, storlets, personally08:23
eranromtakashi: sure, fair enough. I would love you to keep in touch.08:24
takashieranrom: thx :-)08:24
*** thorst_ has quit IRC08:26
eranromtakashi: So I am not saying goodbye or anything now :-) Hoping that you will still be able to keep in touch.08:26
takashieranrom: thanks again. :-)08:27
eranromtakashi: :-)08:27
eranromI have no further topics for today08:28
*** Wenzhi has quit IRC08:28
takashithat is all about my update related to ptg. fyi, kota_ already knows my situation08:28
takashiand noting elae from my side08:28
*** Wenzhi has joined #openstack-meeting08:28
eranromtakashi: ok. Thanks.08:28
takashimany typos... :-P08:29
eranromakihito: sagara: Anything else before we end the meeting?08:29
akihitoyeh. nothing..08:30
eranromok, so thanks very much for joining.08:30
eranromtalk to you later.08:30
takashihave a nice day, guys!08:30
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"08:30
openstackMeeting ended Tue Feb  7 08:30:47 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:30
openstackMinutes:        http://eavesdrop.openstack.org/meetings/storlets/2017/storlets.2017-02-07-08.00.html08:30
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/storlets/2017/storlets.2017-02-07-08.00.txt08:30
openstackLog:            http://eavesdrop.openstack.org/meetings/storlets/2017/storlets.2017-02-07-08.00.log.html08:30
eranromtakashi: thanks08:31
sagaraeranrom: thanks08:31
akihitoThank you.  If I notice topic to discuss on PTG, I add it to etherpad..08:31
eranromakihito: great. thanks!08:31
*** markvoelker has joined #openstack-meeting08:32
*** yamamoto has joined #openstack-meeting08:32
*** lpetrut has joined #openstack-meeting08:32
*** Wenzhi has quit IRC08:32
*** alexpilotti has joined #openstack-meeting08:34
*** markvoelker has quit IRC08:38
*** alexpilotti has quit IRC08:38
*** ykatabam has joined #openstack-meeting08:38
*** makowals has quit IRC08:42
*** priteau has joined #openstack-meeting08:45
*** sagara has quit IRC08:49
*** akihito has quit IRC08:50
*** toscalix has joined #openstack-meeting08:54
*** dbecker has joined #openstack-meeting08:54
*** makowals has joined #openstack-meeting08:55
*** makowals has quit IRC08:56
*** chenying has joined #openstack-meeting09:01
*** shintaro has joined #openstack-meeting09:02
*** edisonxiang has joined #openstack-meeting09:03
*** adisky_ has joined #openstack-meeting09:03
*** salv-orlando has quit IRC09:04
*** zhangshuai has joined #openstack-meeting09:05
*** phil_ has joined #openstack-meeting09:06
*** salv-orlando has joined #openstack-meeting09:07
*** makowals has joined #openstack-meeting09:12
*** kota_ has joined #openstack-meeting09:13
*** yangyapeng has quit IRC09:13
*** kota_ has left #openstack-meeting09:13
*** yangyapeng has joined #openstack-meeting09:13
*** yangyape_ has joined #openstack-meeting09:15
*** yangyapeng has quit IRC09:15
*** makowals has quit IRC09:15
*** yamahata has quit IRC09:17
*** makowals has joined #openstack-meeting09:18
*** iyamahat has quit IRC09:18
*** Hosam has quit IRC09:19
*** aloga_ has joined #openstack-meeting09:19
*** huanxuan has quit IRC09:20
*** huanxuan has joined #openstack-meeting09:20
*** wujiajun has joined #openstack-meeting09:21
*** Daviey has joined #openstack-meeting09:21
*** wujiajun has left #openstack-meeting09:21
*** megm has quit IRC09:22
*** wujiajun has joined #openstack-meeting09:22
wujiajunmeeting now?09:23
*** megm has joined #openstack-meeting09:23
*** Alex_Staf has joined #openstack-meeting09:24
*** adiantum has quit IRC09:25
*** donghao has joined #openstack-meeting09:26
*** Daisy_ has quit IRC09:27
*** adiantum has joined #openstack-meeting09:27
*** gongysh has quit IRC09:29
*** strigazi1AFK is now known as strigazi09:29
*** gongysh has joined #openstack-meeting09:29
*** zhangshuai has left #openstack-meeting09:30
*** gongysh has quit IRC09:30
*** ricolin has quit IRC09:30
*** yamamoto has quit IRC09:30
*** wujiajun has left #openstack-meeting09:30
*** liuyulong has joined #openstack-meeting09:31
*** donghao has quit IRC09:31
*** adiantum has quit IRC09:32
*** yamamoto has joined #openstack-meeting09:33
*** markvoelker has joined #openstack-meeting09:34
*** yamamoto has quit IRC09:39
*** markvoelker has quit IRC09:40
*** zhurong has quit IRC09:44
*** Julien-z_ has quit IRC09:45
*** ricolin has joined #openstack-meeting09:47
*** mickeys has quit IRC09:53
*** jschwarz has quit IRC10:01
*** jschwarz has joined #openstack-meeting10:01
*** yangyape_ has quit IRC10:03
*** yangyapeng has joined #openstack-meeting10:04
*** iceyao_ has quit IRC10:04
*** yangyapeng has quit IRC10:08
*** kevinz has quit IRC10:17
*** salv-orl_ has joined #openstack-meeting10:19
*** salv-orlando has quit IRC10:21
*** thorst_ has joined #openstack-meeting10:22
*** shintaro has quit IRC10:25
*** Daisy has joined #openstack-meeting10:26
*** thorst_ has quit IRC10:26
*** donghao has joined #openstack-meeting10:29
*** Daisy has quit IRC10:30
*** adiantum has joined #openstack-meeting10:30
*** guoshan has quit IRC10:34
*** donghao has quit IRC10:34
*** ricolin has quit IRC10:35
*** markvoelker has joined #openstack-meeting10:37
*** markvoelker has quit IRC10:43
*** sambetts_ is now known as sambetts10:43
*** huanxuan has quit IRC10:50
*** jkraj has joined #openstack-meeting10:51
*** mickeys has joined #openstack-meeting10:54
*** adiantum has quit IRC10:55
*** adiantum has joined #openstack-meeting10:58
*** mickeys has quit IRC10:58
*** llu has quit IRC11:01
*** sudipto has quit IRC11:05
*** sudipto_ has quit IRC11:05
*** rfolco has joined #openstack-meeting11:06
*** rossella_s has quit IRC11:09
*** Hosam has joined #openstack-meeting11:15
*** fzdarsky is now known as fzdarsky|lunch11:19
*** makowals_ has joined #openstack-meeting11:24
*** yangyapeng has joined #openstack-meeting11:25
*** makowals has quit IRC11:25
*** liusheng has quit IRC11:26
*** epico has quit IRC11:26
*** Hosam_ has joined #openstack-meeting11:26
*** liusheng has joined #openstack-meeting11:26
*** diga has quit IRC11:27
*** Hosam has quit IRC11:28
*** ntpttr has quit IRC11:30
*** ntpttr has joined #openstack-meeting11:31
*** Wenzhi has joined #openstack-meeting11:34
*** markvoelker has joined #openstack-meeting11:39
*** markvoelker has quit IRC11:46
*** salv-orl_ has quit IRC11:47
*** salv-orlando has joined #openstack-meeting11:48
*** Wenzhi has quit IRC11:49
*** xingchao_ has quit IRC11:52
*** xingchao has joined #openstack-meeting11:52
*** ansmith has joined #openstack-meeting11:52
*** xingchao has quit IRC11:54
*** xingchao has joined #openstack-meeting11:54
*** mickeys has joined #openstack-meeting11:54
*** xingchao has quit IRC11:55
*** takashi has quit IRC11:56
*** xingchao has joined #openstack-meeting11:57
*** mickeys has quit IRC11:59
*** jschwarz has quit IRC12:00
*** jkilpatr has joined #openstack-meeting12:00
*** xingchao has quit IRC12:04
*** xingchao has joined #openstack-meeting12:04
*** brault is now known as brault|away12:06
*** ansmith has quit IRC12:08
*** xingchao has quit IRC12:10
*** astarove has quit IRC12:10
*** xingchao has joined #openstack-meeting12:11
*** xingchao has quit IRC12:12
*** nadya has quit IRC12:12
*** Cibo_ has quit IRC12:22
*** Cibo_ has joined #openstack-meeting12:22
*** bkopilov_ has quit IRC12:28
*** acoles_ is now known as acoles12:28
*** bkopilov has quit IRC12:28
*** sdague_ has joined #openstack-meeting12:32
*** gcb has quit IRC12:36
*** sdake has joined #openstack-meeting12:37
*** gcb has joined #openstack-meeting12:37
*** xingchao has joined #openstack-meeting12:37
*** kevinz has joined #openstack-meeting12:39
*** thorst_ has joined #openstack-meeting12:39
*** markvoelker has joined #openstack-meeting12:41
*** sdake_ has joined #openstack-meeting12:42
*** sdake has quit IRC12:42
*** sandanar has joined #openstack-meeting12:43
*** nadya has joined #openstack-meeting12:43
*** gcb has quit IRC12:45
*** trinaths has quit IRC12:45
*** kevinz has quit IRC12:46
*** kevinz has joined #openstack-meeting12:47
*** markvoelker has quit IRC12:48
*** links has quit IRC12:48
*** rossella_s has joined #openstack-meeting12:49
*** aloga_ has quit IRC12:49
*** Alex_Staf has quit IRC12:50
*** lhx__ has quit IRC12:51
*** sdague has quit IRC12:52
*** gcb has joined #openstack-meeting12:53
*** yanyanhu has joined #openstack-meeting12:55
*** mickeys has joined #openstack-meeting12:55
*** kevinz has quit IRC12:56
*** kevinz has joined #openstack-meeting12:57
*** swapnil-nilangek has joined #openstack-meeting12:58
*** elynn has joined #openstack-meeting12:59
yanyanhu#startmeeting senlin13:00
openstackMeeting started Tue Feb  7 13:00:11 2017 UTC and is due to finish in 60 minutes.  The chair is yanyanhu. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
*** openstack changes topic to " (Meeting topic: senlin)"13:00
openstackThe meeting name has been set to 'senlin'13:00
yanyanhuhi, Qiming13:00
yanyanhuI guess some guys are still in vacation :)13:01
*** mickeys has quit IRC13:01
yanyanhuhi, elynn13:01
*** bobh has joined #openstack-meeting13:02
*** bobh has quit IRC13:02
elynnSaw xinhui's email, she might be on vacation.13:02
yanyanhulets wait for a while for other attenders13:02
*** bobh has joined #openstack-meeting13:02
yanyanhuelynn, yep13:02
yanyanhuok, lets get started13:03
yanyanhuhere is the agenda, please feel free to add items13:04
yanyanhujust a short one :)13:04
*** jmckind has joined #openstack-meeting13:04
yanyanhu#topic Ocata final RC13:04
*** openstack changes topic to "Ocata final RC (Meeting topic: senlin)"13:04
yanyanhuOcata final RC will be in two weeks13:04
yanyanhuand we have cut the rc1 release and the stable/ocata branch was built on it13:05
yanyanhuif there is any critical fix coming in next two weeks, we can consider to port them back13:05
yanyanhuso pike cycle will come soon :)13:05
QimingI've already merged two backports13:05
yanyanhuQiming, cool :)13:06
yanyanhuso the back port will be done automatically?13:06
yanyanhuat the end of O cycle13:06
Qiming... no13:06
yanyanhuso it needs release team's help?13:06
Qimingcherry pick, propose, review, gate, approve, merge13:07
*** VW has joined #openstack-meeting13:07
yanyanhuI see13:07
yanyanhuand I noticed that doug send a mail said they will wait for a while for several projects who haven't had ocata release candidiate yet13:08
yanyanhuso after that, the cycle will be switched to pike?13:08
*** gcb has quit IRC13:09
*** salv-orlando has quit IRC13:09
Qimingafter official release13:09
yanyanhuI see13:09
yanyanhuok, lets move to next topic13:10
yanyanhu#topic Pike workitems13:10
*** openstack changes topic to "Pike workitems (Meeting topic: senlin)"13:10
yanyanhuI just initialized an etherpad for senlin pike workitem13:10
yanyanhuand moved those backlogs from ocata workitems list13:11
Qimingem ... we are not in pike cycle yet13:11
Qimingbut it is fine if we just keep a single etherpad for progress tracking13:12
yanyanhuQiming, yes, so before pike cycle starts, lets keep using ocata page :)13:12
*** sudipto has joined #openstack-meeting13:12
*** sudipto_ has joined #openstack-meeting13:12
yanyanhuthey are same now13:13
yanyanhuso I will revise the pike list after ocata cycle is done13:13
*** XueFeng has joined #openstack-meeting13:13
yanyanhulets quick go through the list?13:13
yanyanhuhi, XueFeng13:13
yanyanhu"Feature Rich" Nova Server13:14
XueFengSorry for late13:14
yanyanhuXueFeng, no problem13:14
*** rbowen has joined #openstack-meeting13:14
yanyanhuelynn is working on it13:14
yanyanhuand one of our summit proposal will be based on it13:14
QimingI tried to review it ... but it is difficult13:14
elynnbasic support for rich network properties is done13:14
Qimingit is not easy to tell which are newly added code13:15
elynnQiming, use diff and compare it with original server profile...13:15
elynnOr should I just change the original server profile?13:15
Qimingthat means the comparison cannot be done using gerrit directly?13:16
yanyanhuhmm, it is difficult if it is totally a new file13:16
Qiminganother workaround is to propose a copy of server profile13:16
yanyanhuor we can copy the original nova server profile to contribute dir in the first patch13:16
Qimingthen rebase your changes onto it13:16
yanyanhuthen merge it13:16
*** fzdarsky|lunch is now known as fzdarsky13:17
elynna copy of server profile?13:17
yanyanhuthen propose the current change based on it?13:17
elynnhow to do it with git?13:17
yanyanhuQiming, :)13:17
yanyanhuexact what I mean13:17
elynnokay, I see.13:17
elynnThat would be a workaround.13:17
yanyanhuthen we can easily tell the difference13:18
elynncould do that.13:18
*** nilanges has joined #openstack-meeting13:18
elynnbtw, I'm working on the handle_update part for networks.13:18
elynnand bran from vmware is testing it in mitaka version of openstack.13:19
*** nilanges_ has joined #openstack-meeting13:19
yanyanhuit is for updating network resource property?13:19
*** nilanges_ has left #openstack-meeting13:19
elynnyes, yanyanhu13:19
elynnupdate is not yet done.13:19
*** nilanges has quit IRC13:19
yanyanhuI see13:19
elynnand volume is not added yet.13:19
*** lhx__ has joined #openstack-meeting13:20
*** nilanges has joined #openstack-meeting13:20
*** nilanges_ has joined #openstack-meeting13:20
*** askb has quit IRC13:20
elynnI'm not sure if volume is necessary.13:20
*** VW has quit IRC13:20
yanyanhumay need to talk with xinhui about it13:20
*** nilanges has quit IRC13:20
yanyanhuto see what is the requirement from nfv app13:20
elynnyes, after her vacation :)13:21
yanyanhuyep :)13:21
yanyanhuok, if there is no more question about this work, lets move on13:21
Qiminghave any of you encountered this ? https://bugs.launchpad.net/senlin/+bug/166205313:21
openstackLaunchpad bug 1662053 in senlin "node-show -D image error" [Undecided,New] - Assigned to Yuanbin.Chen (yuanbin.chen)13:21
Qimingit is also related to nova server profile13:22
*** nilanges_ has quit IRC13:22
*** bobh has quit IRC13:22
yanyanhuwe haven't used block_device_mapping_v2 before13:22
Qimingthe complaint is that if a server has a volume attached13:22
*** nilanges has joined #openstack-meeting13:22
Qimingthe returned server data JSON will look different13:22
yanyanhuyes, seems so13:23
Qimingthe image key doesn't have an id entry any more13:23
*** nilanges_ has joined #openstack-meeting13:23
*** ralonsoh has joined #openstack-meeting13:23
yanyanhuelynn, have you met this issue before?13:23
elynnnever used this property13:23
yanyanhuwhen vm node has extra volume attached13:23
*** nilanges_ has left #openstack-meeting13:23
*** nilanges_ has quit IRC13:23
*** dprince has joined #openstack-meeting13:24
Qimingneed to check nova api docs there13:24
*** nilanges is now known as swaapnil-nilange13:24
yanyanhuso next work item13:25
yanyanhuImprove tempest API test13:25
yanyanhuruijie has been working on it13:25
yanyanhuhe is not here now, so will talk with him to see the progress13:25
yanyanhusome works have been done before spring festival13:26
yanyanhuEngine Improvement13:26
yanyanhuautomatic action purge13:26
yanyanhuhaven't picked up the patch yet13:26
yanyanhuwill revisit it soon13:27
yanyanhuand hongbao has started working on db purge13:27
Qimingthat patch is still very dirty13:27
yanyanhuthis one13:27
yanyanhuyes, still some questions there13:28
QimingI don't get the requirement about so many parameters13:28
Qimingpurge events based on object status?13:28
yanyanhuactually me neither, so maybe we can consider to start from a simple one13:28
Qimingbased on object names?13:28
yanyanhuand add more params support if there is requirement13:29
Qimingin the latest patch, the age is still a 'nargs='?'' parameter13:29
Qimingthat means I can specify --age 3 --age 413:29
yanyanhuthat is not beautiful13:30
*** lhx__ has quit IRC13:30
Qimingit won't work actually13:30
Qimingit has to be a single value13:30
*** japestinho has quit IRC13:30
yanyanhuok, lets talk with hongbao to see how to improve this patch13:31
yanyanhuok next one13:32
yanyanhucluster check13:32
*** cody-somerville has joined #openstack-meeting13:32
yanyanhuhi, XueFeng13:32
yanyanhuthis patch https://review.openstack.org/#/c/421615/13:32
*** pradk has joined #openstack-meeting13:33
XueFengWill modify again13:33
yanyanhuit is for decreasing the action records13:33
*** swapnil-nilangek has quit IRC13:33
yanyanhuXueFeng, thanks, I guess it is close to be done13:34
yanyanhuand also this one :)13:34
*** swaapnil-nilange has quit IRC13:35
*** jungleboyj has joined #openstack-meeting13:35
yanyanhuok, next13:35
yanyanhuHealth Management13:35
yanyanhuwill sync with xinhui after she comes back13:36
*** hoangcx has joined #openstack-meeting13:36
XueFengAbout health management sitll has some problem13:36
*** sdake_ is now known as sdkae13:37
XueFengThe new field in health_registry 'enabled' is confused13:37
Qimingit is an internal field for use by the health manager only13:37
Qimingthe health registry table is designed to be used by health manager only13:38
Qimingthe long term goal is to separate health registry into a separate service13:38
Qimingwe are trying to make it as independent as possible13:39
XueFengNow we only set this field =113:39
yanyanhuQiming, you mean splitting it from engine?13:39
XueFengThat mean we do prepare for later?13:39
Qiminghealth manager, when doing polling, may introduce a lot of CPU overhead13:40
Qimingit will hurt the senlin-engine performance13:40
yanyanhuit is reasonable13:40
yanyanhumaybe we can start a bp for it in pike cycle13:41
Qimingyep, if there are hands working on it13:41
XueFengI mentioned this before13:41
Qiminghealth management and container support, these are the top two priorities for Pike in my minde13:41
yanyanhuyes, ha definitely should be13:42
XueFengAlso, another problem.In current health_registry pick desinge13:42
*** ansmith has joined #openstack-meeting13:42
XueFengIt can't support concurrent mechanism13:43
*** sdkae is now known as sdake13:43
*** cdelatte has joined #openstack-meeting13:43
*** lhx_ has joined #openstack-meeting13:43
Qimingyou mean ? two health manager pick the same cluster for monitoring?13:44
*** markvoelker has joined #openstack-meeting13:44
XueFengwe can't loadbalance health mangager to many engine13:44
*** amotoki has joined #openstack-meeting13:44
*** Hosam_ has quit IRC13:44
*** egarbade- has quit IRC13:44
*** trown|brb has joined #openstack-meeting13:45
yanyanhuyou mean?13:45
*** trown|brb is now known as trown|mtg13:45
*** trown|mtg is now known as trown13:45
*** rbowen has quit IRC13:46
yanyanhuok, lets talk more about it offline13:46
Qimingthe notify() method in the health-manager module has a engine_id which can be set to None13:46
XueFengwill discuss more later13:46
Qimingif it is set to None, any health manager can pick that cluster for monitoring13:47
*** VW has joined #openstack-meeting13:47
yanyanhuengine_id=none means broadcasting?13:47
Qimingit is a trivial issue13:47
*** mtanino has quit IRC13:47
*** zz_dimtruck is now known as dimtruck13:47
*** egarbade- has joined #openstack-meeting13:47
*** baoli has joined #openstack-meeting13:48
*** Julien-zte has joined #openstack-meeting13:48
yanyanhuok, maybe we can have further disucssion later. we have only 10 minutes left13:48
*** VW has quit IRC13:48
yanyanhudocument for HA policy13:49
yanyanhuhas been done13:49
*** VW has joined #openstack-meeting13:49
*** jmckind_ has joined #openstack-meeting13:49
yanyanhuok next one13:49
yanyanhufunctional test for client13:49
yanyanhuhasn't been started yet13:50
*** markvoelker has quit IRC13:50
yanyanhumaybe we can postpone it to next cycle13:50
yanyanhuno maybe, the final release of senlinclient has been done actually :)13:50
yanyanhuok, next one13:50
yanyanhucontainer profile13:50
*** salv-orlando has joined #openstack-meeting13:50
yanyanhuwill be another important work item in pike cycle13:51
*** jmckind has quit IRC13:51
yanyanhuwill talk with haiwe about it later13:51
*** bswartz has joined #openstack-meeting13:51
yanyanhuok, that's all items in the list13:51
yanyanhuand please pay attention on those new items added to TODO.rst13:51
*** bkopilov_ has joined #openstack-meeting13:51
yanyanhuand pick up them freely if you are interested on them13:52
*** bkopilov has joined #openstack-meeting13:52
yanyanhuthanks :)13:52
yanyanhu#topic PTL election result13:52
*** openstack changes topic to "PTL election result (Meeting topic: senlin)"13:52
yanyanhupike ptl election has been done13:52
yanyanhuQiming will play the role in next cycle :)13:52
elynncongratulates ;)13:53
yanyanhuthanks god, u're back, haha13:53
QimingI figured ... that no one was stepping up13:53
elynnThe king is back.13:53
Qimingand the deadline has passed, ...13:53
yanyanhuit was in spring vacation13:54
Qimingseems no one enjoys this servant role, :)13:54
elynnI figured ... that some one will step up ;)13:54
yanyanhuanyone, welcome back, to lead us :)13:55
Qimingsomeone has to jump into this13:55
Qiminglet it be me >:13:55
*** armstrong has joined #openstack-meeting13:55
elynnQiming! Qiming! Qiming!13:55
elynnSee, you are the best.13:56
Qimingit is always about a team work13:56
Qimingbe prepared, next one could be you13:57
elynnWe will be with you :)13:57
*** mickeys has joined #openstack-meeting13:57
yanyanhuok, those are all topics in the agenda13:57
yanyanhuopen discussion now13:57
yanyanhu#open discussion13:57
Qimingnext week is final rc13:58
Qimingand the week after that would be final release13:58
*** yuval has quit IRC13:58
yanyanhu#topic open discussion13:58
*** openstack changes topic to "open discussion (Meeting topic: senlin)"13:58
QimingI'll try get listener to heat events implemented13:58
XueFengAbout rdo if we need to do?13:58
Qimingso the loop is closed13:58
yanyanhuthe first two patches have landed13:58
*** baoli has quit IRC13:58
yanyanhuyes, looks some guys need it13:59
QimingXueFeng, we need volunteers to work on that13:59
QimingRDO needs a maintainer for a project13:59
*** baoli has joined #openstack-meeting13:59
yanyanhuno have to be redhat guy?13:59
*** rbowen has joined #openstack-meeting13:59
Qimingtime's up, maybe can continue on #senlin13:59
Qimingdoesn't have to be RH employee13:59
yanyanhuthanks you guys for joining, lets move back to senlin channel13:59
yanyanhuI see13:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"14:00
openstackMeeting ended Tue Feb  7 14:00:08 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/senlin/2017/senlin.2017-02-07-13.00.html14:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/senlin/2017/senlin.2017-02-07-13.00.txt14:00
openstackLog:            http://eavesdrop.openstack.org/meetings/senlin/2017/senlin.2017-02-07-13.00.log.html14:00
*** XueFeng has quit IRC14:02
*** mickeys has quit IRC14:02
*** lamt has joined #openstack-meeting14:09
*** claudiub has joined #openstack-meeting14:10
*** chenhuayi has joined #openstack-meeting14:10
*** yanyanhu has quit IRC14:10
*** eglynn has joined #openstack-meeting14:10
*** ihrachys has joined #openstack-meeting14:11
*** eric_young has joined #openstack-meeting14:18
*** lamt has quit IRC14:23
*** jamesdenton has joined #openstack-meeting14:26
*** mjturek has joined #openstack-meeting14:27
*** mriedem has joined #openstack-meeting14:27
*** Daisy has joined #openstack-meeting14:28
*** xingchao has quit IRC14:30
*** jamesden_ has joined #openstack-meeting14:32
*** Daisy has quit IRC14:32
*** jamesdenton has quit IRC14:33
*** zhurong has joined #openstack-meeting14:34
*** aloga_ has joined #openstack-meeting14:35
*** kevinz has quit IRC14:36
*** lamt has joined #openstack-meeting14:40
*** cdelatte has quit IRC14:43
*** donghao has joined #openstack-meeting14:46
*** markvoelker has joined #openstack-meeting14:46
*** gouthamr has joined #openstack-meeting14:47
*** sudipto_ has quit IRC14:48
*** sudipto has quit IRC14:48
*** dimtruck is now known as zz_dimtruck14:51
*** markvoelker has quit IRC14:52
*** mtanino has joined #openstack-meeting14:54
*** jamesden_ has quit IRC14:55
*** fnaval has joined #openstack-meeting14:57
*** rbak has joined #openstack-meeting14:57
*** jamesdenton has joined #openstack-meeting14:57
*** elynn has quit IRC14:58
*** mickeys has joined #openstack-meeting14:58
*** ramishra has joined #openstack-meeting15:00
*** shintaro has joined #openstack-meeting15:01
*** cody-somerville has quit IRC15:01
*** eharney has joined #openstack-meeting15:01
*** mickeys has quit IRC15:03
*** ramishra has left #openstack-meeting15:03
*** yuval has joined #openstack-meeting15:04
*** mcunietti has joined #openstack-meeting15:04
yuval#startmeeting Karbor15:04
openstackMeeting started Tue Feb  7 15:04:22 2017 UTC and is due to finish in 60 minutes.  The chair is yuval. Information about MeetBot at http://wiki.debian.org/MeetBot.15:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:04
*** openstack changes topic to " (Meeting topic: Karbor)"15:04
openstackThe meeting name has been set to 'karbor'15:04
yuvalHello everybody15:04
yuvalhey :)15:04
*** dmacpher has joined #openstack-meeting15:04
yuvalchenying: anyone else joining us from over there?15:05
chenyingyuval I don't know. I am at home.15:05
yuvalhey chenhuayi15:05
chenyinghi chenhuayi15:05
chenhuayihey yuval15:06
yuvalright, let's begin15:06
yuval#topic Ocata RC15:06
*** openstack changes topic to "Ocata RC (Meeting topic: Karbor)"15:06
yuvalWe are almost in the Ocata release15:07
yuvalmost projects have branched out ocata15:07
yuvalthe only reason we haven't done so in karbor repo is because of the gate failure which we solved today15:07
yuvalkarbor-dashboard and python-karborclient were branched out and released15:07
chenyingSo we can branch out ocata taday or tomorrow.15:08
*** zhangshuai_ has joined #openstack-meeting15:08
yuvalafter we merge the remaining patches today and tomorrow, I'll release and branch15:08
yuvalchenying: yes :)15:08
*** david-lyle has joined #openstack-meeting15:08
yuvaland a patch to release repo is required15:08
*** dmorita has joined #openstack-meeting15:08
*** egallen has joined #openstack-meeting15:09
yuvalany patches which you think are a must for ocata (besides the nova protection plugin)15:09
chenyingchenhuayi The network plugins also need be merged to ocata branch :)15:09
yuvalchenying: ?15:10
yuvalchenhuayi: ?15:10
*** david-lyle has quit IRC15:10
*** david-lyle has joined #openstack-meeting15:11
*** tidwellr has joined #openstack-meeting15:11
chenyingIf the patch that we think it is important for ocata, we could merged to this branch later.15:11
chenhuayithe network plugins protectable has been summit.15:11
yuvalchenhuayi: I see only protectable, not protection plugin15:12
chenyingchenhuayi After the protection plugins is ready, they also can be submitted to ocata branch.15:12
*** dmorita has quit IRC15:13
chenyinghi zhangshuai15:13
yuvalchenhuayi: you can submit half-baked patch for reviews and mark it WIP15:13
yuvalchenhuayi: usually I wouldn't say that, but we are almost in release15:14
chenyingDo you have any patch that you think it is a must for ocata?15:14
chenhuayibut the protection still working.15:14
chenhuayiyeah, protection plugin still working on the tests.15:14
chenhuayichenying, got it.15:14
yuvalright, next15:15
yuval#topic Mascot15:15
*** openstack changes topic to "Mascot (Meeting topic: Karbor)"15:15
chenhuayiyuval, i will submit in 1 or 2 days.15:15
yuvalAs you might have seen in the mailing list, we received two options for our beaver mascot15:15
*** david-lyle has quit IRC15:16
chenyingYes I have seen.15:16
yuvalsee http://lists.openstack.org/pipermail/openstack-dev/2017-February/111666.html15:16
yuvalOption 1: http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/c9c312a4/attachment.png15:16
yuvalOption 2: http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/c9c312a4/attachment-0001.png15:16
yuvalLet's have a quick vote, please write your favorite alternative: 1 or 215:17
*** sdague_ is now known as sdague15:17
*** aloga_ has quit IRC15:18
yuvalchenying: chenhuayi: zhangshuai_: ?15:18
chenhuayivote 215:18
chenyingDo we need gather other guy's choice?15:19
*** sacharya has joined #openstack-meeting15:19
chenyingvote 115:19
*** bauzas has quit IRC15:19
yuvalchenying: whose choice?15:19
*** sshnaidm has quit IRC15:19
*** zz_dimtruck is now known as dimtruck15:19
chenyingMaybe we can gather the vote about the Mascot in the IRC meeting?15:20
yuvalchenying: we are in the IRC meeting15:20
*** sshnaidm has joined #openstack-meeting15:20
chenyingsorry in thr karbor irc channel :(15:21
*** trown is now known as trown|brb15:21
*** hongbin has joined #openstack-meeting15:21
*** gouthamr has quit IRC15:22
yuvalalright, let's gather the votes from this meeting with the votes cast in the Karbor IRC channel tomorrow between 08:00-09:00 UTC15:22
*** bauzas has joined #openstack-meeting15:22
yuvalwhich is 16:00-17:00 in China15:22
yuvalat 09:00 UTC I'll collect the results and announce the mascot15:22
*** trown|brb is now known as trown15:23
yuval#topic PTG15:23
*** openstack changes topic to "PTG (Meeting topic: Karbor)"15:23
yuvalremineder that the PTG is coming. Myself and saggi will be there15:23
chenyingyuval Ok I will call yinwei to join the channel.15:23
yuvalif you have a topic you want us to discuss, inside Karbor team or with other team (such as Tiramisu), please add to the etherpad15:24
chenyingI have seen the topic about karbor PTG15:24
*** nadya has quit IRC15:24
*** hoangcx has quit IRC15:24
*** chenhuayi_ has joined #openstack-meeting15:25
chenyingI have added two topics to the etherpad.15:25
yuvalchenying: great :)15:25
yuval#topic Open Discussion15:26
*** openstack changes topic to "Open Discussion (Meeting topic: Karbor)"15:26
yuvalanything else you'd like to discuss?15:26
*** chenhuayi has quit IRC15:27
chenyingChina Moblie has some interests about karbor, they have a requirement about protecting file in the vm.15:27
yuvalchenying: files in general?15:29
*** gouthamr has joined #openstack-meeting15:29
chenyingSo we could discuss the solution about file protection in PTG.15:29
*** gouthamr has quit IRC15:30
chenyingyuval Yes  I will discuss with them about the detail of file protection.15:30
yuvalchenying: can you add that to the etherpad?15:30
yuvalchenying: will they attend the PTG?15:30
*** anilvenkata has quit IRC15:30
chenyingNO I don't think so.15:30
chenyingI will add a topic about file protection. It seem that freezer can protect the general files in vm.15:31
yuvalchenying: ok, but if they want to push something like this, they should attend IRC meetings at least15:32
chenyingOK next IRC meeting They may attend.15:33
*** mcunietti has quit IRC15:33
yuvalchenying: great15:33
yuvalanything else?15:34
*** armax has joined #openstack-meeting15:34
chenyingThis guy is from China Moblie. https://review.openstack.org/41822515:34
*** armax has quit IRC15:34
yuvalthank you for attending, albeit the late hour in China15:34
*** armax has joined #openstack-meeting15:35
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"15:35
openstackMeeting ended Tue Feb  7 15:35:45 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:35
openstackMinutes:        http://eavesdrop.openstack.org/meetings/karbor/2017/karbor.2017-02-07-15.04.html15:35
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/karbor/2017/karbor.2017-02-07-15.04.txt15:35
openstackLog:            http://eavesdrop.openstack.org/meetings/karbor/2017/karbor.2017-02-07-15.04.log.html15:35
*** yuval has quit IRC15:36
*** chenhuayi_ has quit IRC15:36
*** egallen has quit IRC15:38
*** cdelatte has joined #openstack-meeting15:40
*** Cibo_ has quit IRC15:40
*** tovin07 has joined #openstack-meeting15:42
*** jaugustine has joined #openstack-meeting15:43
*** yamahata has joined #openstack-meeting15:43
*** shintaro has quit IRC15:44
*** egarbade- has quit IRC15:44
*** Cibo_ has joined #openstack-meeting15:45
*** larainema has quit IRC15:46
*** salv-orlando has quit IRC15:46
*** egarbade- has joined #openstack-meeting15:47
*** markvoelker has joined #openstack-meeting15:49
*** jaugustine has quit IRC15:49
*** Cibo_ has quit IRC15:50
*** Cibo_ has joined #openstack-meeting15:51
*** david-lyle has joined #openstack-meeting15:53
*** esberglu has joined #openstack-meeting15:53
*** Cibo_ has quit IRC15:55
*** markvoelker has quit IRC15:55
*** jdurgin1 has joined #openstack-meeting15:55
*** sandanar has quit IRC15:56
*** lpetrut has quit IRC15:56
*** hieulq_ has joined #openstack-meeting15:58
*** mickeys has joined #openstack-meeting15:59
*** zhangshuai_ has quit IRC16:00
*** adiantum has quit IRC16:00
ihrachys#startmeeting neutron_ci16:00
openstackMeeting started Tue Feb  7 16:00:54 2017 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
ihrachyshi everyone16:01
* ihrachys gives a minute for everyone to gather around a fire in a circle16:01
* haleyb gets some marshmallows for the fire16:02
*** donghao has quit IRC16:02
*** iyamahat has joined #openstack-meeting16:03
ihrachysok let's get it started. hopefully people will get on board. :)16:03
*** donghao has joined #openstack-meeting16:03
*** mickeys has quit IRC16:03
ihrachys#link https://wiki.openstack.org/wiki/Meetings/NeutronCI Agenda16:03
*** VW has quit IRC16:03
ihrachysI guess we can start with looking at action items from the previous meeting16:04
*** VW has joined #openstack-meeting16:04
ihrachys#topic Action items from previous meeting16:04
*** openstack changes topic to "Action items from previous meeting (Meeting topic: neutron_ci)"16:04
*** apetrich has quit IRC16:04
*** jmckind_ has quit IRC16:04
*** salv-orlando has joined #openstack-meeting16:04
ihrachys"armax to make sure periodic functional test job shows up in grafana"16:04
*** apetrich has joined #openstack-meeting16:04
*** electrocucaracha has joined #openstack-meeting16:04
ihrachysarmax: I still don't see the job in periodic dashboard in grafana. any news on that one?16:05
*** ricolin has joined #openstack-meeting16:05
*** phil_ has quit IRC16:05
ihrachysok, I guess armax is not up that early. I will follow up with him offline.16:06
ihrachys#action ihrachys to follow up with armax on periodic functional job not showing up in grafana16:06
ihrachysnext is: "ihrachys to follow up on elastic-recheck with e-r cores"16:06
ihrachysso I sent this email to get some answers from e-r folks: http://lists.openstack.org/pipermail/openstack-dev/2017-January/111288.html16:07
*** AlanClark has joined #openstack-meeting16:07
*** donghao has quit IRC16:08
ihrachysmtreinish gave some answers; basically, 1) check queue is as eligible for e-r queries as gate one so functional job can be captured with it; 2) adding cores is a matter of doing some reviews before getting a hammer; 3) there is a recheck bot that we can enable for neutron channel if we feel a need for that.16:08
ihrachysso from our perspective, we should be fine pushing more queries into the repo, and we should probably try to get some review weight in the repo16:09
ihrachys#action ihrachys to look at e-r bot for openstack-neutron channel16:10
manjeetsihrachys, question : do we have a dashboard for ci specific patches ?16:10
*** diablo_rojo has joined #openstack-meeting16:10
ihrachysmanjeets: we don't, though we have LP that captures bugs with gate-failure and similar tags16:10
mtreinishihrachys: I said check queue jobs are covered by e-r queries. But we normally only add queries for failures that occur in gate jobs. If it's just the check queue filtering noise from patches makes it tricky16:11
armaxihrachys: I am here16:11
armaxbut in another meeting16:11
*** nadya has joined #openstack-meeting16:11
ihrachysmanjeets: we probably can look at writing something to generate such a dashboard16:11
*** tovin07 has left #openstack-meeting16:11
*** nadya has quit IRC16:11
mtreinishihrachys: I can give you a hand updating the e-r bot config, it should be a simple yaml change16:11
electrocucarachaihrachys: do we have a criteria for adding a new query to e-r, like more than x number of hits?16:11
ihrachysmtreinish: that's a price of having functional neutron job in check only. we may want to revisit that.16:11
manjeetsihrachys, yes thanks16:12
ihrachyselectrocucaracha: I don't think we have it, but the guidelines would be - it's some high profile gate bug, and we can't quickly come up with a fix.16:12
*** galstrom_zzz is now known as galstrom16:12
ihrachysmanjeets: do you want to look at creating such a dashboard?16:12
manjeetsihrachys, sure i'll take a look16:12
*** cdelatte has quit IRC16:12
ihrachysmanjeets: rossella_s wrote one for patches targeted for next release, you probably could reuse her work16:13
*** markvoelker has joined #openstack-meeting16:13
mtreinishelectrocucaracha: if you want to get your feet wet everything here http://status.openstack.org/elastic-recheck/data/integrated_gate.html needs categorization16:13
electrocucarachaihrachys: regarding the point 3, it seems like only adding a new entry in the yams file https://github.com/openstack-infra/project-config/blob/master/gerritbot/channels.yaml#L83116:13
manjeetscool thanks for example16:13
ihrachysmanjeets: see Gerrit Dashboard Links at the top at http://status.openstack.org/reviews/16:13
electrocucarachathanks mtreinish16:13
rossella_sihrachys, manjeets I can help with that16:14
mtreinishif you find a fingerprint for any of those failures thats something we'd definitely accept an e-r query for16:14
*** acoles is now known as acoles_16:14
*** markvoelker has quit IRC16:14
*** ekuris__ has quit IRC16:14
ihrachys#action manjeets to produce a gerrit dashboard for gate and functional failures16:14
*** markvoelker has joined #openstack-meeting16:14
manjeetsthanks rossella_s  i'll go through and will ping you if any help needed16:14
mtreinishelectrocucaracha: that's actually not the correct irc bot. The elastic recheck cofnig lives in the puppet-elastic_recheck repo16:15
mtreinishI linked to it on my ML post16:15
ihrachysarmax: np, I will ping you later to see what we can do with the dashboard16:15
ihrachysoverall, some grafana dashboards are in bad shape, we may need to have a broader look at them16:15
*** ricolin has quit IRC16:16
*** rcernin has quit IRC16:16
*** ricolin has joined #openstack-meeting16:16
ihrachysok next action was: "ihrachys to follow up with infra on forbidding bare gerrit rechecks"16:16
ihrachysI haven't actually followed up with infra, though I checked project-config code16:17
*** Cibo_ has joined #openstack-meeting16:17
*** pcaruana has quit IRC16:17
mtreinishihrachys: if you have ideas on how to make: http://status.openstack.org/openstack-health/#/g/project/openstack~2Fneutron more useful that's somethign we should work on too16:17
ihrachysbasically, the gerrit comment recheck filter is per queue, not per project16:17
*** zhurong has quit IRC16:17
ihrachyshere: https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L2016:18
ihrachysso I believe it would require some more work on infra side to make it per project. but I will still check with infra to make sure.16:18
ihrachysmtreinish: what do you mean? is it somehow project specific? or do you mean just general improvements that may help everyone?16:19
mtreinishyou were talking about dashboards, and I would like to make sure that your needs are being met with openstack-health. Whatever improvements neutron needed likely would benefit everyone16:19
mtreinishso instead of doing it in a corner, I just wanted to see if there was space to work on that in o-h16:20
ihrachysyou see -health as a replacement for grafana?16:20
mtreinishI see it as something that can consume grafana as necessary. But I want to unify all the test results dashboards to a single place16:20
mtreinishinstead of jumping around between a dozen web pages16:21
ihrachysmakes sense. it's just we were so far set on grafana, probably it's time to consider -health first for any new ideas.16:22
ihrachysok, next action was "ihrachys check if we can enable dstat logs for functional job"16:23
ihrachysthat's to track system load during functional test runs that sometimes produce timeouts in ovsdb native16:23
ihrachysI posted this https://review.openstack.org/427358, please have a look16:23
*** artom_ has joined #openstack-meeting16:24
ihrachyson related note, I also posted https://review.openstack.org/427362 to properly index per-testcase messages in logstash, that should also help us with elastic-recheck queries, and overall with understanding impact of some failures16:25
manjeetsihrachys, where it will the dump the logs a separate screen window ?16:25
manjeetsi mean separate file ?16:25
ihrachyssorry, not project-config patch, I wanted to post https://review.openstack.org/430316 instead16:25
*** xinli has joined #openstack-meeting16:25
ihrachysmanjeets: yes, it should go in screen-dstat as in devstack runs16:25
ihrachysfinally, there is some peakmem-tracker service in devstack that I try to enable here: https://review.openstack.org/430289 (not sure if it will even work, I haven't found other repos that use the service)16:26
ihrachysfinally, the last action item is "jlibosva to explore what broke scenario job"16:26
ihrachyssadly I don't see Jakb16:27
*** artom has quit IRC16:27
ihrachysbut afaik the failures were related to bad ubuntu image contents16:27
ihrachysso we pinned the image with https://review.openstack.org/#/c/425165/16:27
*** matrohon has quit IRC16:28
ihrachysand Jakub also has a patch to enable console logging for connectivity failures in scenario jobs: https://review.openstack.org/#/c/427312/, that one needs second +2, please review16:28
ihrachyssadly grafana shows that scenario jobs are still at 80% to 100% failure rate, something that did not happen even a month ago16:29
ihrachysso there is still something to follow up on16:29
ihrachys#action jlibosva to follow up on scenario failures16:29
ihrachysoverall, those jobs will need to go voting, or it will be another fullstack job broken once in a while :)16:30
*** ayogi has quit IRC16:30
*** egarbade- has quit IRC16:32
*** baoli has quit IRC16:32
ihrachysok let's have a look at bugs now16:32
ihrachys#topic Known gate failures16:32
*** openstack changes topic to "Known gate failures (Meeting topic: neutron_ci)"16:32
ihrachys#link https://goo.gl/IYhs7k Confirmed/In progress bugs16:32
ihrachysok, so first is ovsdb native timeout16:33
ihrachysotherwiseguy was kind to produce some patch that hopefully mitigates the issue: https://review.openstack.org/#/c/429095/16:33
ihrachysand it already has +W, nice16:33
*** nkrinner is now known as nkrinner_afk16:33
ihrachysthere is a backport for the patch for Ocata found in: https://review.openstack.org/#/q/I26c7731f5dbd3bd2955dbfa18a7c41517da63e6e,n,z16:34
ihrachysso far rechecks in gerrit show some good results16:34
ihrachyswe will monitor the failure rate after it lands16:34
*** lpetrut has joined #openstack-meeting16:35
ihrachysanother bug that lingers our gates is bug 164391116:36
openstackbug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] https://launchpad.net/bugs/164391116:36
*** cody-somerville has joined #openstack-meeting16:36
*** cody-somerville has quit IRC16:36
*** cody-somerville has joined #openstack-meeting16:36
ihrachysthe last time I checked, armax suspected it to be the same as the oom-killer spree bug in gate, something discussed extensively in http://lists.openstack.org/pipermail/openstack-dev/2017-February/111413.html16:37
ihrachysarmax made several attempts to lower memory footprint for neutron, like the one merged https://review.openstack.org/42906916:38
*** aloga_ has joined #openstack-meeting16:38
ihrachysit's not a complete solution, but hopefully buys us some time16:38
*** csomerville has quit IRC16:38
*** diablo_rojo has quit IRC16:38
*** artom_ is now known as artom16:38
ihrachysthere is an action item to actually run a memory profiler against neutron services and see what takes the most16:38
ihrachysafaiu armax won't have time in next days for that, so, anyone willing to try it out?16:39
ihrachysI may give some guidance if you are hesitant about tools to try :)16:41
ihrachysanyway, reach out if you have cycles for this high profile assignment :16:41
manjeetswouldn't enabling dtsat will help ?16:41
ihrachysit will give us info about how system behaved while tests were running, but it won't give us info on which data structures use the memory16:42
manjeetsohk gotcha16:43
armaxihrachys: the devstsack changes landed at last16:44
armaxpreliminary logstash results seem promising16:44
ihrachysarmax: do we see rate going down?16:44
armaxbut that only bought us a few more days16:44
ihrachysmmm, good16:44
ihrachysarmax: few more days? you are optimistic about what the community can achieve in such a short time :P16:44
armaxihrachys: I haven’t looked in great detail, but the last failure was yesterday lunchtime PST16:45
ihrachyswhat was it? like 300 mb freed?16:45
armaxihrachys: between 350 and 400 MB of RSS memory, yes16:45
armaxsome might be shared, but it should be enough to push the ceiling a bit further up and help aboid oom-kills and libvirt barfing all over the place16:45
mtreinisharmax: fwiw, harlowja and I have been playing tracemalloc to try and profile the memory usage16:46
*** rossella_s has quit IRC16:46
armaxmtreinish: nice16:46
armaxmtreinish: for libvirt?16:46
mtreinishwell I started with turning the profiling on for the neutron api server16:47
*** jprovazn_bbl has joined #openstack-meeting16:47
ihrachysmtreinish: any results to consume so far? I see all red in neutron patch.16:48
*** tmorin has joined #openstack-meeting16:48
mtreinishthe neutron patch won't work, it's just a dnm to setup stuff for testing. Look at the oslo.service patch's tempest job16:48
mtreinishthere are memory snapshots there16:48
mtreinishif you want your browser to hate you, this kinda thing is the end goal: http://blog.kortar.org/wp-content/uploads/2017/02/flame.svg16:48
*** cdelatte has joined #openstack-meeting16:49
* ihrachys puts a life vest on and clicks16:49
mtreinishwe're still debugging the memory snapshot collection, because what we're collecting doesn't match what ps says the process is consuming16:49
*** jdurgin1 has quit IRC16:49
mtreinishihrachys: heh, it's 26MB IIRC16:49
ihrachyscool, let me capture the link in the notes16:50
ihrachys#link https://review.openstack.org/#/q/status:open+topic:tracemalloc Attempt to trace memory usage for Neutron16:50
ihrachys#link http://blog.kortar.org/wp-content/uploads/2017/02/flame.svg Neutron API server memory trace attempt16:51
armaxmtreinish: would apply the same patches to nova help correlate/spot a pattern?16:51
ihrachysarmax: and maybe also a service that does not seem to be as obese16:51
mtreinisharmax: that's the theory16:51
mtreinishonce we can figure out a successful pattern for collecting and visualizing where things are eating memory we can apply it to all the things16:52
armaxmtreinish: understood16:52
armaxmtreinish: thanks for looking into this16:52
ihrachysis the tracer usable in gate? does it slow down/destabilize jobs?16:52
armaxI have been struggling to find time to go deeper into this headache16:52
* mtreinish prepares his little vm for the traffic flood16:53
mtreinishihrachys: there didn't seem to be too much of an overhead, but I wasn't watching it closely16:53
mtreinishit's still very early in all of this (I just started playing with it yesterday afternoon :) )16:53
*** lhx_ has quit IRC16:54
ihrachysgotcha, thanks a lot for taking it upon yourself16:54
ihrachyson related note, I am not sure we got to the root of why we don't use all the swap16:55
*** hieulq_ has quit IRC16:55
mtreinishihrachys, armax: oh if you want to generate that flame graph locally: http://paste.openstack.org/show/597889/16:55
ihrachyswe played with swappiness knob of no affect I believe: https://review.openstack.org/#/c/425961/16:55
mtreinishjust take the snapshots from the oslo.service log dir16:55
*** lpetrut has quit IRC16:56
ihrachys#action ihrachys to read about how swappiness is supposed to work, and why it doesn't in gate16:58
ihrachys#topic Open discussion16:58
*** openstack changes topic to "Open discussion (Meeting topic: neutron_ci)"16:58
ihrachyswe are almost at the top of the hour. anything worth mentioning before we abrupt?16:58
*** andreas_s has quit IRC16:59
*** mickeys has joined #openstack-meeting16:59
ihrachysok thanks everyone for joining, and working on making the gate great again17:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"17:00
openstackMeeting ended Tue Feb  7 17:00:09 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-02-07-16.00.html17:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-02-07-16.00.txt17:00
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-02-07-16.00.log.html17:00
*** electrocucaracha has left #openstack-meeting17:00
*** eharney_ has joined #openstack-meeting17:01
*** eharney has quit IRC17:03
*** eharney_ is now known as eharney17:03
*** mickeys has quit IRC17:04
*** sridharg has quit IRC17:04
*** nadya has joined #openstack-meeting17:06
*** unicell has joined #openstack-meeting17:08
*** unicell1 has quit IRC17:08
*** lpetrut has joined #openstack-meeting17:08
*** trown is now known as trown|lunch17:09
*** Cibo_ has quit IRC17:09
*** priteau has quit IRC17:10
*** Cibo_ has joined #openstack-meeting17:10
*** matrohon has joined #openstack-meeting17:11
*** iyamahat has quit IRC17:15
*** Apoorva has joined #openstack-meeting17:15
*** yamahata has quit IRC17:16
*** jkilpatr_ has joined #openstack-meeting17:17
JayF/win 3417:17
JayFwhoops, sorry17:17
*** jkilpatr has quit IRC17:19
*** Cibo_ has quit IRC17:19
*** tmorin has quit IRC17:20
*** Cibo_ has joined #openstack-meeting17:20
*** jgrassler has quit IRC17:22
*** jgr has joined #openstack-meeting17:23
*** pradk has quit IRC17:23
*** VW has quit IRC17:25
*** unicell has quit IRC17:26
*** Cibo_ has quit IRC17:27
*** efried has quit IRC17:28
*** VW has joined #openstack-meeting17:33
*** kaisers has quit IRC17:33
*** dprince has quit IRC17:34
*** dprince has joined #openstack-meeting17:35
*** Apoorva has quit IRC17:35
*** Apoorva has joined #openstack-meeting17:36
*** Swami has joined #openstack-meeting17:36
*** kaisers has joined #openstack-meeting17:37
*** Swami has quit IRC17:37
*** rfolco has quit IRC17:38
*** esberglu has quit IRC17:38
*** esheffield has joined #openstack-meeting17:38
*** nikhil|afk has joined #openstack-meeting17:38
*** Swami has joined #openstack-meeting17:38
*** venkatesh_ has joined #openstack-meeting17:38
*** iyamahat has joined #openstack-meeting17:38
*** toscalix has quit IRC17:39
*** efried has joined #openstack-meeting17:41
*** dmorita has joined #openstack-meeting17:41
*** Julien-zte has quit IRC17:44
*** egarbade- has joined #openstack-meeting17:47
*** Julien-zte has joined #openstack-meeting17:47
*** Julien-zte has quit IRC17:48
*** baoli has joined #openstack-meeting17:49
*** efried has quit IRC17:50
*** esberglu has joined #openstack-meeting17:52
*** efried has joined #openstack-meeting17:54
*** nadya has quit IRC17:55
*** spilla has joined #openstack-meeting17:55
*** sudipto_ has joined #openstack-meeting17:55
*** sudipto has joined #openstack-meeting17:55
*** antwash has joined #openstack-meeting17:56
*** browne has joined #openstack-meeting17:57
*** jaugustine has joined #openstack-meeting17:58
stevemarping agrebennikov, amakarov, annakoppad, antwash, ayoung, bknudson, breton, browne, chrisplo, crinkle, davechen, dolphm, dstanek, edmondsw, edtubill, gagehugo, henrynash, hrybacki, jamielennox, jaugustine, jgrassler, knikolla, lamt, lbragstad, kbaikov, ktychkova, morgan, nishaYadav, nkinder, notmorgan, raildo, ravelar, rderose, rodrigods, roxanaghe, samueldmq, shaleh, spilla, srwilkers, StefanPaetowJisc, steve17:59
stevemarmar, topol, portdirect, SamYaple17:59
*** gagehugo has joined #openstack-meeting17:59
stevemar#startmeeting keystone17:59
openstackMeeting started Tue Feb  7 17:59:50 2017 UTC and is due to finish in 60 minutes.  The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot.17:59
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:59
*** openstack changes topic to " (Meeting topic: keystone)"17:59
openstackThe meeting name has been set to 'keystone'17:59
*** hashar has quit IRC18:00
*** ralonsoh has quit IRC18:00
*** rfolco has joined #openstack-meeting18:00
stevemaragenda is at https://etherpad.openstack.org/p/keystone-weekly-meeting18:00
stevemar#link https://etherpad.openstack.org/p/keystone-weekly-meeting18:00
* morgan goes back to sleep. :P18:00
*** mickeys has joined #openstack-meeting18:00
* stevemar gives morgan a pillow18:00
*** dmorita has quit IRC18:01
stevemarmay as well get comfy18:01
stevemarnot much was added to the agenda !18:01
*** tesseract has quit IRC18:01
stevemarhopefully we can make this short :)18:01
*** gouthamr has joined #openstack-meeting18:02
*** alexpilotti has joined #openstack-meeting18:02
*** abalutoiu__ is now known as abalutoiu18:02
*** dmorita has joined #openstack-meeting18:02
stevemarokay, lets get going18:02
stevemar#topic announcements18:02
*** openstack changes topic to "announcements (Meeting topic: keystone)"18:02
*** yamahata has joined #openstack-meeting18:02
stevemarwe're in RC week18:02
*** aloga_ has quit IRC18:03
stevemarnext week is the final RC candidates and the week after we're out the door!18:03
*** krtaylor has quit IRC18:03
stevemarhope we don't get any upgrade bugs :(18:03
stevemaralso, PTG in 2 weeks, yay!18:04
stevemarin case anyone missed it, stable/ocata branch was created, we can work on pike goals now18:04
stevemarand lastly, you'll all have a new PTL in ~6 hours18:05
*** mickeys has quit IRC18:05
stevemar#topic functional/integration tests prior merging features18:05
*** openstack changes topic to "functional/integration tests prior merging features (Meeting topic: keystone)"18:05
stevemarrodrigods: ^18:05
stevemaryou're up boss18:05
samueldmqstevemar: lbragstad \o/18:06
rodrigodsthis is a request and hopefully you will all agree :)18:06
*** pradk has joined #openstack-meeting18:06
rodrigodshas been a while that i'm implementing integration tests for keystone and it is not uncommon to find bugs in features that are there for some time18:06
rodrigodsyou can see examples in the agenda18:07
ayoungStill need a mechanism for LDAP and Federation.18:07
rodrigodsin Barcelona, me and stevemar discussed about requiring a functional/integration test submitted prior approving new features18:07
morganI disagree, mostly to be contrary... and only in the meeting...and only when I am pre-coffee. and only when I am not being serious. :P18:07
rodrigodsayoung, right... federation we already have18:07
ayoungAh, good...SAML via Shib?18:08
rodrigodsayoung, yes18:08
morganrodrigods: a lot of cases, functional was not possible before. FTR. this is not unreasonable now.18:08
rodrigodsmorgan, it is totally doable as of today18:08
stevemarwe made functional tests required in OSC18:09
rodrigodswhat the team think about this?18:09
rodrigodsi mean... this is not very hard18:09
morganeh. depends on the feature18:09
rodrigodsand can help *a lot* in the quality of stuff we delivery18:09
ayoungSo...say the RBAC thing...something that really needs a remote service to test...are we going to do an echo server thing for functional tests, to do a complete round trip with middelware too?18:10
morganbut honestly, most of our recent "features" aren't features like federation/ldap18:10
rodrigodsmorgan, ++18:10
morganso, I am hesitant to make it blanket. PCI DSS functional tests are really already covered... but not in the integration manner you are speaking, nor would they be18:11
morganfor example.18:11
rodrigodsi can always help as well18:11
rodrigodsmorgan, it is, for some settings18:11
rodrigodsi've implemented myself18:11
*** jmckind has joined #openstack-meeting18:11
rodrigodsmorgan, https://review.openstack.org/#/c/378624/18:12
morgannot in the way you're describing. most of the code is Keystone specific and unit test covered18:12
morganso, let's step back18:12
rodrigodsfunctional/integration tests are not supposed to be exhaustive18:12
rodrigodsexhaustion is done in unit testing18:13
*** jprovazn_bbl has quit IRC18:13
morganlet's do a clear delineation of functional vs unit and outline further where we are breaking our tests up before we require a new type of test for features18:13
ayoungrodrigods, exhaustion is done trying to get a patch through review18:13
rodrigodsayoung, not always, but usually18:13
morganright. now we still have a total mishmash and mismatched and impossible to follow test suite that is not clearly defined what the test types should be18:14
dstaneki'm trying to pay attention here as much as i can (in another meeting), but if we require them what is the criteria to say they are done18:14
dstanekif they are not exhaustive...is it that at least one is included?18:14
rodrigodsmorgan, functional is tempest-like, unit is everything else that lands inside our /tests folder18:14
rodrigodsdstanek, something that shows in a high level the feature18:14
rodrigodsan API call18:14
morganright. explain to me how to pick what test to Implement where.18:14
morganwe have tests in unit that should be functional18:14
*** trown|lunch is now known as trown18:15
rodrigodsunit is everything, all edge cases (including negative tests)18:15
dstanekrodrigods: so only happy path is required? and for every API call?18:15
morganwe have never cleaned up our tests.18:15
rodrigodsdstanek, basically18:15
*** diablo_rojo has joined #openstack-meeting18:15
dstanekif we do mandate this we have to have clear guidelines...otherwise reviewers won't know what to do or won't be consistent18:15
morganso.. I am -1 on this. we havent done basic work to fix our current tests to.conform or identify what should be moved over/addressed.18:15
rodrigodsthe guidelines are basic what tempest requires18:15
morganwe are so inconsistent we can't mandate a. we test type right now18:16
stevemaron the topic of "fixing our current tests"18:16
stevemarthis was proposed to the TC as a goal: https://review.openstack.org/#/c/369749/18:16
morganreviews will be inconsistent.18:16
dstanekrodrigods: by guidelines i don't mean how to write tests, but something that tells us what tests to write18:16
stevemarwe would need to create a keystone-tempest repo18:16
rodrigodsexample of tempest-like tests: https://review.openstack.org/#/c/425927/18:16
rodrigodsdstanek, this is what i meant... tempest doesn't accept negative tests anymore, for example18:17
rodrigodsso a happy API path should be enough for most features18:17
morganI'm still -1 on this for the above stated reasons.18:17
rodrigodsmorgan, it is not necessary to move over old tests18:17
rodrigodsjust to require a simple functional tests for new stuff18:17
dstaneki don't think we should enforce this yet. we should encourage it and see how that goes18:17
rodrigodsthis has lots of consequences18:18
stevemardstanek: that might be the right call18:18
rodrigodswe frequently break APIs18:18
rodrigodsas an example18:18
morganmandating new test types when we don't even know what tests we have should be moved over, and there is no clear way to say move these tests (identify the ones, don't do it, backlog it) AND not having g a clearly written case, is not going f to help18:18
rodrigodsand the fact that find bugs is not uncommon, is a red flag for me18:18
dstanekmaybe we can push back a little on reviews and see if we can get them to include the tests18:19
morgan"like this test *point*", -218:19
rodrigodsmorgan, i guess i can write a spec?18:19
stevemarrodrigods: bugs are always going to happen18:19
stevemarrodrigods: no need for a spec IMO18:19
morganclear descriptor, identity tests that are incorrect in our suit, specification, and documentation on the right way, +1, and more.convo on making it mandatory18:20
stevemarrodrigods: i think the issue people have is that they don't want to slow down an already slow process18:20
rodrigodsstevemar, although bugs are always going to happen, they can get fix prior the feature is merged18:20
rodrigodsinstead of finding issues later, and having to backport stuff18:20
stevemari think we should encourage new APIs to include functional tests18:20
stevemarnot make it required tho18:21
rderosestevemar: ++18:21
stevemari don't know how to add new tests18:21
stevemari assume i'm not alone?18:21
rderoseyou're not18:21
stevemarhaven't looked into it much at all18:21
*** dprince has quit IRC18:21
morganso, what I am asking for is a clear description on what is wanted, documentation on what we are expecting (encouraged), identifications of tests to move over (backlog work)18:21
rodrigodsso we just found a problem that can be solved :)18:21
morganthen we can discuss making it mandatory.18:21
rodrigodslbragstad already told me about our functional tests18:21
rodrigodsthere is no doc around them18:22
samueldmqmorgan: ++18:22
morganbut for now, like I said -1 or -2 based on this convo.18:22
*** donghao has joined #openstack-meeting18:22
*** dprince has joined #openstack-meeting18:22
stevemarstart off with a doc that says how to add a test?18:22
morganfilling in the requests I just made before we make it mandatory +218:22
*** sudipto_ has quit IRC18:23
*** sudipto has quit IRC18:23
rodrigodsstevemar, cool18:23
rodrigodswill add a section about our tempest plugin18:23
rodrigodsdevstack plugin18:23
samueldmqlooks like everybody agrees it's good to have those tests, it's just about the way we get to/enforce that18:23
stevemarsamueldmq: yep18:23
morganI would expect the mandatory part to be for the Q cycle, but highly encouraged, documentation, and identified tests to convert for pike18:23
morgandoc first.18:23
rodrigodsand how to add new tests to tempest18:24
rodrigodstempest repo ^18:24
dstaneksamueldmq: ++18:24
rodrigodsok, so i think i'm done here18:24
rodrigodswith the topic, i mean :)18:25
* morgan is also done and wants coffee. :p18:25
stevemarrodrigods: i look forward to adding new tests myself in P :)18:25
* topol wondering if morgan is using his french press18:25
stevemar#topic open discussion18:25
*** openstack changes topic to "open discussion (Meeting topic: keystone)"18:25
rodrigodsstevemar, ++18:26
morgantopol: not yet, soon.18:26
stevemaranyone have anything they want to talk about?18:26
rodrigodswho submitted talks to the next summit?18:26
rodrigodssubmitted one about federation, again18:26
stevemarapparently the deadline was extended :D18:26
morganSomeone was doing pci-dss. make sure to talk with rderose .18:26
* lbragstad sets https://etherpad.openstack.org/p/keystone-pike-ptg on the desk18:27
topolrodrigods I have a talk and a panel submitted on Interop and a talk on OpenStack Kube integration18:27
morganif not collaborations already.18:27
*** donghao has quit IRC18:27
rodrigodstopol, cool! mine is a panel too (not a talk)18:27
stevemari haven't submitted anything yet, i should probably do something...18:27
rderosemorgan: ++18:27
rodrigodsstevemar, you are usually invited :)18:27
topolstevemar I think you missed the deadline18:27
lbragstadoh! how'd https://etherpad.openstack.org/p/keystone-pike-ptg get here?! just a reminder to make sure folks fill it out if they have feedback ;)18:27
dstanekwhat's the new deadline?18:27
morgantopol: doesn't like me, he never wants to collaborate on a talk with me :P (I would prob say no.. but...I should be asked) :P (j/k) ^_^18:28
rodrigodslol lbragstad18:28
morganthe 8th?18:28
stevemartopol: they extended it18:28
samueldmqmorgan: it's myself and rderose doing pci-dss18:28
topolstevemar til when???18:28
morganyep, the 8th18:28
dstanekneat, maybe i'll come up with something then18:28
stevemartopol: https://twitter.com/OpenStack/status/82901898300935782418:28
* topol groan more internal abstracts to review #wantmylifeback :-018:28
stevemarFeb 8, 11:59pm PT18:28
gagehugoI'll have to see the pci-dss talk then if I go18:29
stevemartopol: can i skip the internal review?18:29
*** s3wong has joined #openstack-meeting18:29
topoljust submit something please18:29
rodrigodslet's submit a talk: "requiring integration tests for new features" :P18:29
stevemarjust trying to save you from doing work18:29
lbragstadnow that we've been performance testing data - we should do a performance talk18:29
lbragstadI have the data and would be willing to tag team it with someone18:30
dstaneklbragstad: to report on how we've been doing?18:30
* topol always fun doing a talk with Morgan... Sad our paths have diverged to diff topics of late18:31
topoldrinking with him in a nice consolation prize. hope that happens18:31
lbragstaddstanek well - we have a bunch of data18:31
lbragstadcould just start asking questions like "what did it help us accomplish?" "was it effective?" "has our performance improved or degraded?" "how can we expand on what we've built?" etc...18:33
dstaneklbragstad: i may be interested...let's talk offline?18:33
lbragstaddstanek ++18:33
*** aloga_ has joined #openstack-meeting18:33
*** unicell has joined #openstack-meeting18:33
lbragstadi would assume we'd be able to come up with enough things to make an interesting talk18:33
rodrigodsdstanek, lbragstad me too18:35
stevemarwell if no one else has anything to discuss we can finish this off early18:35
rodrigodsnot to be a presenter, but to help with it in general18:35
lbragstadrodrigods ++18:35
*** rasca has quit IRC18:35
* stevemar will celebrate chairing his last meeting as ptl with a big cup of coffee18:35
dstanekrodrigods: ++18:35
*** gouthamr has quit IRC18:35
rodrigodsstevemar, s/coffee/beer18:36
lbragstadstevemar you're ending your last meeting as PTL early?!18:36
dstanekstevemar: seems like we can call it a day18:36
topolstevemar CONGRATS. You earned that coffee!!18:36
*** mickeys has joined #openstack-meeting18:36
stevemarlbragstad: i don't see a better reason to end it early :D18:36
lbragstadtopol ++18:36
rodrigodsthanks for everything stevemar18:36
lbragstadstevemar thanks for all the hard work, sir18:36
stevemarlooking forward to a change of pace :)18:36
*** AJaeger has joined #openstack-meeting18:36
* topol I'll save the I remember when Steve first started on OpenStack stories for the PTG18:37
lbragstadstevemar are you excited to write code again?18:37
lbragstadtopol ++18:37
lbragstadtopol I remember the first call I had with you two on OpenStack stuff ;)18:37
stevemarlbragstad: haha, that's right, you taught us!18:37
topollbragstad, good times18:37
stevemarlbragstad: my code will be terrible, it'll receive many -1's18:38
lbragstadstevemar topol ++18:38
*** sambetts is now known as sambetts|afk18:38
stevemaralright, thanks folks!18:38
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"18:38
openstackMeeting ended Tue Feb  7 18:38:47 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:38
openstackMinutes:        http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.2017-02-07-17.59.html18:38
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.2017-02-07-17.59.txt18:38
* topol no one wants to ask an idiotic question on IRC and look like the moron??? Okay then. Hold my beer...18:38
*** spilla has left #openstack-meeting18:38
openstackLog:            http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.2017-02-07-17.59.log.html18:38
*** krtaylor has joined #openstack-meeting18:39
*** antwash has left #openstack-meeting18:40
*** jdurgin has quit IRC18:41
*** jkraj has quit IRC18:41
*** rbak has quit IRC18:41
*** hrybacki is now known as hrybacki____18:42
*** aloga_ has quit IRC18:44
*** Shrews has joined #openstack-meeting18:45
*** rbak has joined #openstack-meeting18:49
*** vishnoianil has quit IRC18:51
*** alexpilo_ has joined #openstack-meeting18:55
*** VW has quit IRC18:55
*** alexpilotti has quit IRC18:55
*** jdurgin has joined #openstack-meeting18:57
*** tidwellr has quit IRC18:57
*** bobh has joined #openstack-meeting18:57
*** adiantum has joined #openstack-meeting18:58
fungiinfra team, aggregate!19:00
*** gagehugo has left #openstack-meeting19:00
* jeblair settles down19:00
* jeblair watches closely to see if fungi got it19:00
* fungi grins19:00
* jeblair is relieved19:01
pabelangerit is time19:01
fungiat least i didn't say "precipitate"19:01
fungimaybe next week19:01
jeblairthat'll give me time to work on it19:01
*** bobh has quit IRC19:01
* fungi considers making some joke about solutions, and then loses interest19:01
*** VW has joined #openstack-meeting19:02
* mordred waves19:02
*** ricolin has quit IRC19:02
*** ociuhandu has joined #openstack-meeting19:03
fungithis week we have topics proposed by jeblair and fungi (so far anyway)19:04
fungi#startmeeting infra19:04
openstackMeeting started Tue Feb  7 19:04:38 2017 UTC and is due to finish in 60 minutes.  The chair is fungi. Information about MeetBot at http://wiki.debian.org/MeetBot.19:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:04
*** openstack changes topic to " (Meeting topic: infra)"19:04
*** nadya has joined #openstack-meeting19:04
openstackThe meeting name has been set to 'infra'19:04
fungi#link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting19:04
fungi#topic Announcements19:04
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:04
*** nadya has quit IRC19:04
fungi#info Big thanks to ianw for chairing last week while I was otherwise occupied19:05
fungii read through the logs later in the week and looks like it was a productive mee19:05
fungi#info Please muster a hearty welcome to David Shrewsbury as our newest Infra root sysadmin and returning core reviewer19:05
* mordred hands Shrews a somewhat unused pie19:05
fungihe was actually in infra-core before we renamed ourselves "infra" (and before i joined the project for that matter)19:05
pabelangerShrews: congrats19:06
jeblairShrews: yay!19:06
*** pvaneck has joined #openstack-meeting19:06
jeblairreboot all the things19:06
pabelangerold is new again19:06
fungias i understand it, he's planning to focus mostly on nodepool for now, but has been instrumental so far in shade and the latest improvements on the road to zuul v3 support in nodepool (among many other things)19:06
clarkbdarn I was hoping he would deal with Gerrit again19:07
fungialso he loves writing java ;)19:07
* jeblair takes cover19:07
fungi(for those who missed our early history, he was the author of our original "work in progress" implementation for gerrit)19:08
* Shrews glares menacingly19:08
mordredthat work can be directly blamed for Shrews going and doing non-infra things for quite some time :)19:08
* fungi wonders what ever happened to libra19:08
fungi#action Shrews propose a change adding yourself to modules/openstack_project/manifests/users*.pp in system-config19:09
fungii suspect we're going to chew up a lot of time on ptg discussion today, so i'll move on with the meeting19:10
fungias always, feel free to hit me up with announcements you want included in future meetings19:10
fungi#topic Actions from last meeting19:10
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:10
fungi#link http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-01-31-19.01.html19:10
fungiianw ensure gerritbot upgrade is deployed for new release (0.3.0)19:11
fungifungi@review:~$ pip list|grep bot19:11
fungigerritbot (0.3.0)19:11
ianwyep, that's all working19:11
fungilgtm, thanks ianw!19:11
ianwgerritbot messages now have the branch in them19:11
fungii noticed, very nice19:11
AJaegerI wonder whether we should remove the "master" - otherwise it's nice.19:11
fungithat was your handiwork, right jlvillal?19:11
fungipabelanger update on zuul-launcher to openstack-ci move and zuul-dev for gerrit upgrade19:12
fungiwe can probably talk about this in just a sec during the priority efforts part of the agenda instead19:12
jlvillalThat was my first patch to not display master.19:12
jlvillalBut was suggested to always display branch19:12
pabelangerfungi: I am working on that now, for our PTG zuul effort, same code will apply for zaro19:12
AJaegerjlvillal: ok19:12
fungijlvillal: yeah, i expect it's a matter of personal taste. i could go either way19:13
jlvillalAJaeger, I'm happy to change if consensus19:13
fungiexplicitness there doesn't bother me, and the implementation is simpler for it19:13
mordredI like it19:13
jlvillalYes implementation is simpler19:13
fungipabelanger: oh, thanks! we can skip adding it to the priority efforts discussion today then19:13
ianw(it was my comment, and i figured it's better to be consistent with the message, especially if people might have a regex etc)19:13
fungi#topic Specs approval19:14
*** openstack changes topic to "Specs approval (Meeting topic: infra)"19:14
fungiwe don't seem to have anything new up this week, which is good since the ptg is bearing down on us19:14
fungi#topic Priority Efforts19:15
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:15
funginothing on the agenda for these today, though we'll talk about zuul v3 stuff for the ptg next anyway19:15
fungi#topic Zuul related PTG prep (jeblair)19:15
*** openstack changes topic to "Zuul related PTG prep (jeblair) (Meeting topic: infra)"19:15
fungi#link https://etherpad.openstack.org/p/pike-ptg-zuul19:15
jeblairoh whew i was just looking up that link19:16
fungithis is a continuation/rehoming of the discussion from the zuul meeting yesterday19:16
jeblairyeah, we talked a bit out what we need to do to prepare for the ptg, and hopefully meet our goal of actually running some sort of zuulv3 job19:16
jeblairthat etherpad has a list of things to get done before we arrive19:16
fungithat's an attractive approach in my opinion. i like the idea that we have a solid goal for something we can show coming out of our two days19:17
jeblairhopefully it's fairly self-explanatory; one thing on there i have signed up for is to write some notes on what we'll have in place, what still needs to be done, and what building a job at the ptg might look like.19:17
*** Rockyg has joined #openstack-meeting19:18
*** jmckind has quit IRC19:18
jeblairso we should all be on the same page, or maybe the same chapter, when we get there19:18
jeblairi'll try to do that this week19:18
fungia few of us assigned prep tasks from that pad to ourselves19:19
*** adiantum1 has joined #openstack-meeting19:19
pabelangerI have some patches up already for nodepool-launcher (nl01.o.o) that could use some reviews19:19
fungiif there's anything anyone else thinks about that is probably a prerequisite for making it happen (even if you won't be at the ptg yourself), add it to the prep list and feel free to solve/research/whatever to help us for when we get there19:20
*** adiantum has quit IRC19:20
*** adiantum1 is now known as adiantum19:20
fungiit's definitely a way to be involved even if you won't be present19:20
fungianything else we want to cover about zuul prep for the ptg before we move on to more general last-minute ptg planning?19:22
fungior are people still digesting that etherpad?19:22
* mordred is digesting a bowl of taco meat19:23
*** jmckind has joined #openstack-meeting19:23
* fungi pictures mordred face-down in a bowl of taco meat19:23
* jeblair wonders whether the taco was raised humanely19:23
* mordred wonders if fungi has a camera aimed at him19:23
* fungi distracts everyone from his surveillance activities by spontaneously changing the topic of conversation19:24
fungi#topic General PTG planning (fungi)19:24
*** openstack changes topic to "General PTG planning (fungi) (Meeting topic: infra)"19:24
fungi#link https://etherpad.openstack.org/p/infra-ptg-pike19:24
*** adiantum has quit IRC19:24
fungii like that more of the community has started throwing things at this19:25
fungiildikov, sdague and mriedem want to collaborate on localrc to local.conf conversion in devstack-gate, for example19:25
*** adiantum has joined #openstack-meeting19:25
fungimy plan so far is to chunk monday and tuesday up into a total of four large blocks (before/after lunch each day)19:27
*** dprince has quit IRC19:27
fungiand do brief unconference pitching at the start, with wrap-up regrouping discussions either at the end of each or at the beginning of the next block19:27
*** noslzzp has joined #openstack-meeting19:28
mordredfungi: the localrc/local.conf discussion might also be a good time to get those folks visibility on to v3 job structure and stuff19:28
fungioh, excellent point19:29
jeblairyeah, the discussion this morning has given us more data to help shape what a good v3 config for devstack would look like19:29
fungithis is mostly uncharted territory... i'm hoping it works out more or less like the unstructured time we've had at design summits in the past, but since it's longer we cat get better group arcs over the duration more like some of our focused mid-cycle events have been19:29
mordredif we come out of the week with a running v3 job and sdague understanding how v3 relates to devstack-gate I'll be over the moon19:30
fungiit's a little hard to know what the dynamic will be in the room and how we'll be able to break out within the space and/or taking over various corners of lounge areas or reserving short blocks in some of the cross-project spaces19:31
jeblairfungi: right, and i think that we should try to focus on things where we can make a big difference having a bunch of people in a room together19:31
*** trandles has joined #openstack-meeting19:31
*** vishnoianil has joined #openstack-meeting19:32
fungii'd like to accommodate items that only a handful of people are interested in collaborating on, but odds are they'll need to do it in other spaces to keep the main room for large efforts19:32
jeblairsounds reasonable19:32
jheskethI'm wondering how we can best be helpful to other projects for the rest of the week. With no schedules it'll be difficult to know which rooms/discussions to attend19:32
jheskethShould we be trying to schedule 'infra time' with groups?19:33
clarkbjhesketh: I think the plan/goal/hope is to use the new ethercalc server to advertise when large chunks of things are hjappening and where19:33
fungijhesketh: yep, i've wondered that myself. one method which will _probably_ work is if people in infra have a general affinity for a particular vertical then hang out with them and alert others if our help is needed19:33
jeblairpossibly shouting on irc could be a thing19:34
clarkbWe should have backups working today and I was planning to send mail to list tomorrow about it once I confirmed backups happened19:34
fungialso if a lot of us are hanging out in #openstack-ptg then we can be raised easily19:34
fungier, what jeblair said ;)19:34
jeblairthat even lets us hang out in a bar on thursday and show up when needed :)19:34
* fungi bets there is a hotel bar19:35
jheskethYep, I'm guess it's going to be a learning experience and something we iterate on but that seems like a fine place to start :-)19:35
fungiright, i'm setting my expectations to "reasonable" for this first ptg, since we'll be feeling a lot of it out for the first time (all of our community, not just infra)19:35
mordredjeblair: can we hang out in a bar monday-wednesday too?19:35
* fungi wonders if we could talk erin into just making the lounge into a bar ;)19:36
* jhesketh nods19:36
*** gordc has joined #openstack-meeting19:37
fungiokay, so anyway, throw anything you want to collaborate on into the etherpad19:39
*** Daisy has joined #openstack-meeting19:39
fungiit might be software development, or planning type things that benefit from having a lot of faces in one room19:39
fungia little variety will also probably help us avoid falling asleep as much19:40
* fungi thinks some of us may be falling asleep now19:41
*** gouthamr has joined #openstack-meeting19:41
mordredwait - we're allowed to sleep at these things?19:41
fungiif you don't mind waking up with things written on your face in permanent marker, sure ;)19:41
*** gouthamr has quit IRC19:41
* fungi notes to pack permanent markers19:42
clarkbI am distracted by all the colors on the etherpad19:42
*** Daisy has quit IRC19:43
ShrewsI'm glad I'm only staying for the first half of PTG. I hear Atlanta really lets you down for any second half things.19:43
fungithe owl19:44
*** xinli has quit IRC19:44
pabelangerI get that reference19:44
fungiShrews: that's why we're headed to boston for the summit, of course19:44
Shrewsyay sportsball19:44
clarkbfungi: double wow19:44
jeblaireven i get that since i read about it in my newspaperotron19:45
fungicertainly a strange coincidence that we booked our events into the two towns who sent their gladiators to the big event19:45
fungiwell, anyway, does anyone have any specific items on the planning etherpad they want to discuss during the last 15 minutes of our meeting today?19:46
fungiotherwise we can go to open discussion (since we mostly seem to have done at this point)19:47
clarkbwe really need to light a fire under the "stop using precise" task19:47
clarkbsince precise has like 2 more months of life19:47
fungiright, i wondered if some people wanted to maybe work together on moving mailman19:47
pabelangerya, sign me up for that to help19:48
fungipuppetboard is the only other one, right? and it's already broken anyway19:48
pabelangerya, not sure the status of that to be honest19:48
clarkbwe can ask ansible for the listing19:48
clarkbI think once I can get zanata upgrades behind me I will try to focus on ^19:48
jeblairi'd like to do this one as a virtual sprint19:49
fungii could get behind that option as well19:49
jeblairi would volunteer for that19:49
fungiit may not really benefit from ptg-ness anyway19:49
pabelangerit worked out well last time19:49
*** salv-orl_ has joined #openstack-meeting19:50
fungii was also recently taking a fresh look at mm3 but seems like the packaging story there is still a little meh19:50
jeblairfungi: mm3 is still the future of mailman?19:51
fungigreat question19:51
mordredMailman 3.0 was released on April 28, 2015.19:51
mordredhttp://docs.mailman3.org/en/latest/ fwiw19:52
fungii think the point is it's been called into question whether mm3 is too dissimilar and people will stick woth a continuation of more traditional 2.x19:52
*** salv-orlando has quit IRC19:52
jeblairi don't really know what the word on the street is.19:53
* jeblair assumes mailman is whatever apt-get installs19:53
clarkbfungi: its not different like google groups is different is it?19:53
clarkbI can still send email and get email?19:53
mordredclarkb: maybe?19:54
fungii get the impression it's a heavy rewrite, but since it's not really packaged by any distros (there is ongoing drama trying to get the dependencies into debian for example) i have no experience using any lists running on it19:54
mordredhere's a fun bug: https://bugs.launchpad.net/mailman/+bug/96553219:54
openstackLaunchpad bug 965532 in GNU Mailman "Need a script to upgrade from MM2 to MM3" [Critical,Confirmed] - Assigned to Richard Wackerbarth (wacky)19:54
mordredreading the description of hte thoughts, it seems like mm3 is just a new piece of software19:55
* persia has heard good things about ponymail and frustrsting stories about mailmanv3, but has deployed neither.19:55
fungi#link https://bugs.debian.org/79928119:56
openstackDebian bug 799281 in wnpp "ITP: mailman3-core -- Mailing list management system" [Wishlist,Open]19:56
fungi(for the record)19:56
fungisomething called "hyperkitty"19:56
ttxmordred: yes, it's different enough19:56
fungiin positive news, it's still python-based19:57
fungipy3k-only in fact19:57
ttxand still written by Barry last I checked19:57
jeblairpersia: ponymail requires a mailing list system, such as mailman319:58
fungi#topic Open discussion19:58
*** openstack changes topic to "Open discussion (Meeting topic: infra)"19:58
fungi(with one minute left!)19:59
persiaApologies for the noise then.19:59
AJaegerfungi, what needs to be done to redirect http to https on docs.o.o and developer.o.o?19:59
fungiAJaeger: a change updating the vhost configs for them19:59
fungiAJaeger: assuming everyone's okay with serving only https versions of those sites going forward19:59
fungii didn't see much else out of the docs team on that item19:59
fungiand we're out of time20:00
* AJaeger will ask again20:00
fungithanks everyone!20:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"20:00
openstackMeeting ended Tue Feb  7 20:00:07 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-02-07-19.04.html20:00
*** cdent has joined #openstack-meeting20:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-02-07-19.04.txt20:00
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-02-07-19.04.log.html20:00
* jroll lurks20:00
* fungi refrains from heading to the bar, sticks around for one more hour20:00
* Rockyg wanders to the beer tap and pours one20:00
* edleafe waits in line behind Rockyg 20:01
* mordred o/20:01
*** ihrachys_ has joined #openstack-meeting20:01
ttxdims, johnthetubaguy, mtreinish: around ?20:01
ttx#startmeeting tc20:01
openstackMeeting started Tue Feb  7 20:01:20 2017 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.20:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.20:01
*** openstack changes topic to " (Meeting topic: tc)"20:01
openstackThe meeting name has been set to 'tc'20:01
ttxHi everyone!20:01
ttxOur agenda for today is at:20:01
ttx#link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee20:01
ttxremember to use #info #idea and #link liberally to make for a more readable summary20:01
*** xinli has joined #openstack-meeting20:01
ttx#topic Deprecate postgresql in OpenStack20:02
*** openstack changes topic to "Deprecate postgresql in OpenStack (Meeting topic: tc)"20:02
ttx#link https://review.openstack.org/42788020:02
ttxsdague: o/20:02
ttxquick intro ?20:02
sdagueright, this came up on the mailing list again recently20:02
sdaguethe basic issue is that pg is really very unevenly supported in our community20:02
sdaguewe dropped the bulk of the gate testing 5 months ago20:02
fungiand almost entirely untested20:03
fungiright, that20:03
sdaguethe number of actual users is small (per survey results)20:03
ttxfrom the comments on the thread and review, it appears some people still use it though20:03
AJaegerSUSE's OpenStack product uses postgres and tests if heavily20:03
johnthetubaguysdague: +1 making this explicit for users, it keeps surpising people20:03
*** ansmith has quit IRC20:03
sdagueand with only so many resources to go around, I think we should say once and for all that this isn't an upstream supported thing20:03
*** ihrachys has quit IRC20:03
sdaguettx: some people still use a number of things20:04
*** salv-orl_ has quit IRC20:04
sdagueI'm sure more use nova-net than postgresql20:04
Rockygquick point of info...Huawei uses a DB based on Postgres and that mean DT, China Mobile, etc.20:04
RockygSo, so big users, or soon to be big.20:04
dhellmannthis seems like one of those things like stable branches and requirements, where folks want to use it but don't contribute to it20:04
flaper87We've done this with other tools technologies in the community20:04
fungithe trick will be coming up with a soft enough way to say we're not actively testing it and won't know what we're breaking without coming out and implying that it just won't work20:04
mordredwhat dhellmann said20:04
dhellmannRockyg : what is "a database based on postgres"?20:05
mordredthe problem isn't anything specifically against postgres - it's that it breaks and nobody cares20:05
flaper87we have removed support for qpidd, for example20:05
fungidhellmann: sounds like a postgres fork20:05
RockygIwait a sec...20:05
mordredpostgres is obviously a fine database platform20:05
flaper87so, nothing against postgres but unless people raise their hands and help maintaining it, I think I'm fine with removing support for it20:05
johnthetubaguymordred: +1 this is more our inability to support both20:05
mordredbut if people think it's important from an _upstream_ perspective, people need to pony up to support it20:06
dhellmannI still prefer postgres over mysql because I don't get angry every time I have to figure out how to set up a user20:06
EmilienMflaper87: +120:06
gordcmordred: i'm not sure that's accurate. i think it just takes longer to raise since it doesn't get surfaced as early20:06
edleafeSo is there an option for one of these users to step up and offer to fully support it?20:06
jrollto be fair, if someone wanted to come upstream and "contribute to making postgres work", it's super unclear what that person should do, other than waiting for it to break and jumping on it20:06
mordreddhellmann: funny, that's one of my big beefs when I try to use postgres :)20:06
dhellmannmordred : heh20:06
ttxI wish we'd use an abstraction layer that lets us ignore it20:06
mtreinishdhellmann: heh, I have the opposite problem. I can never remember how to add postgres users :)20:06
dhellmannttx: ideally that's what sqlalchemy would be20:06
mordredttx: well - so this is my other opinino, where I become more opinionated20:06
RockygSo, this just got raised with Huawei by Gordon.  I'll need to spin them up for support in community20:06
jrolle.g. should that person go contribute patches to optimize better for postgres, if that's what is chosen?20:06
ttxit feels like it leaks implementation details up though20:07
fungii think we also need to keep in mind that the term "support" in our (upstream) case just means we do or don't actively try to avoid breaking something, distinct from distributors "supporting" some combined solution they're actively testing and taking contracts from customers over20:07
flaper87ttx: should be oslo.db and/or sqlalchemy for that matter20:07
sdaguettx: abstraction layers are fine when you don't need optimizations or are using basic common parts20:07
mordredlarge scale things on rdbms' should be able to use advanced features of their rdbms20:07
sdaguebut at some point you have to decide you are either using your technology stack effectively20:07
flaper87but apparently it's not abstract enough because there are specific optimizations20:07
ttxsdague: the question then becomes, how much do we rely on those advanced features20:07
mordredunfortunately, that is not the place where databases are similar enough to abstract20:07
sdagueor just abstracting for abstracting sakes20:07
dimscontribs/patches welcome, 3rd party ci welcome, no CI jobs in community?20:08
jrollmordred: sdague: +1, I think that is the real reason to do this, not that people aren't willing to work on it20:08
dimsbut does that let us optimize for mysql?20:08
ttxI mean, currently we are (about to) saying that projects can assume an oslo.db database is present in an openstack installation20:08
fungigiven that there are distributors preferring postgresql over mysql for openstack, i wonder whether us officially stating that we don't support it will cause them to rethink that or will just resort in more downstream patching/forks so they don't have to switch to mysql20:08
mordreddims: I would like to open the doors for our team to optimize for _a_ database20:08
ttxShoud we say "MySQL" instead ?20:08
sdaguemordred: ++20:09
dimsmordred : agree20:09
mordredI believe the inertia is there for "A database" to be mysql20:09
sdagueespecially because that also opens up optimizing the HA scenarios into the environment20:09
mordredbut I would be just as happy if a database was postgres20:09
mordredsdague: exactly20:09
ttxAnd if we say that we write code for MySQL only, then should ditching PG support (in projects that support it) follow deprecation policies ?20:09
edleafeIf we have to specify one, can it at least be an Open database? Like MariaDB?20:09
johnthetubaguyso I had a question, currently its MySQL "like" as in does it include MySQL+galera, MariaDB, MariaDB+galera, etc20:09
sdaguejohnthetubaguy: I think mysql-like is the right answer20:10
mordrededleafe: the differences between maria and mysql are miniscule20:10
*** annegentle has joined #openstack-meeting20:10
sdaguethe optimizations across those sets would be pretty close20:10
mordrededleafe: if we get to the point where they matter, let's deal with it then20:10
dtroyermordred:  how long will that be the case?20:10
ttxsdague: doesn't it expose some implementation details that would require extra testing though ?20:10
johnthetubaguysdague: yeah, I wasn't sure how to expess that I guess20:10
edleafemordred: in production, sure. I meant the licensing20:10
fungiover time i have a feeling mysql, mariadb and the other mysql forks will diverge to the point where we would still need to "pick one"20:10
sdaguettx: possibly, but we have to start somewhere20:10
ttxfungi: ++20:10
dimsso 2 more releases with no postgres in our CI, then we optimize for mysql-like databases?20:10
dtroyerwe'll have the same problem again just withing a smaller family20:11
mordrededleafe: has mysql stopped being gpl and I didn't noticed?20:11
dtroyerfungi: ++20:11
gordcis this 'optimising' still all hypothetical?20:11
ttxSo we could do a two-stage thing. Say now that in the future only MySQL will be supported. Then in two releases ditch all PG support20:11
*** Shrews has left #openstack-meeting20:11
cdentthe concern with future divergence supports my assertion that the resolution should state "yes mysql" not "no postgresql"20:11
dhellmannif we say "mysql-like" does that include the clustering thing the folks from oracle are interested in us supporting?20:11
ttxcdent: yes20:11
mordredcdent: ++20:11
dhellmannbecause that seems to have some schema constraints20:12
EmilienMcdent: yes, I thought the same thing20:12
sdagueso, I'm fine adjusting the language as long as we are getting to the same place before we all rage quit20:12
cdent(not that I really want to vote for mysql if the reason for it is because we want "optimizations" for our little databases ;) )20:12
mordreddhellmann: it's a great question. I would hope so - but I think we need to think about it20:12
ttxcdent: I would likely retrofit it in the "base services" stuff, so it would be "yes MySQL"20:12
persiagordc: I have read reports of some tools not working with pg.  Nothing very widely deployed.20:12
edleafemordred: IANAL, but stuff like this makes me concerned: https://www.mysql.com/about/legal/licensing/foss-exception/20:12
mordredcdent: when I say "optimizations" I do not mean like speed optimizations20:12
mordrededleafe: that's been there literally for forever20:12
johnthetubaguydhellmann: I would rather we picked a clustering one because thats more restrictive, like MariaDB + galera, but yeah, there are differences20:13
EmilienMsome operators won't want to remove pg from their cloud, they probably have customers running clouds in production, they'll have to maintain the deployments. Not sure we've thought about it until now20:13
edleafemordred: And I've not liked it for literally forever20:13
mordrededleafe: it allows foss projects that link against libmysqlclient to not need to gpl their application20:13
gordcpersia: it is being used so i imagine it's just the lag between product team noticing and patching it upstream20:13
ttxsdague: does the two-stage thing I posted above make sense ?20:13
sdagueEmilienM: but I think they did that based on an unrealistic expectation of what is happening in upstream20:13
dhellmannjohnthetubaguy : right, my point is if we're worried about managing support for 1 database in order to optimize for it, having 2 clustering solutions with different requirements seems like it's going to be a hard thing to sell20:13
*** ansmith has joined #openstack-meeting20:13
dtroyerdhellmann: that feels like the same problem in a slightly different flavor20:14
mordredcdent: when I say optimizations- I mean what sdague mentioned - being able to bake in more assumptions about how HA and friends work20:14
dhellmanndtroyer : yep20:14
EmilienMAJaeger: what is your longterm roadmap at Suse, to keep pg? Have you already prepare a migration etc?20:14
ttxsdague: i.e. make a statement now, then start actively ditching things starting in two releases20:14
mordredbecause HA stories and whatnot are where the differences between things like mysql and postgres diverge MASSIVELY20:14
johnthetubaguydhellmann: in my head I was supporting the top 4 most deployed databases, as best we can, I know that means we will have to do things differently to whats happening today20:14
dhellmannmordred : it's also a matter of training folks in the community to recognize the sorts of things that can/must be done for optimization20:14
sdaguettx: I think that the dev tooling should dump things now, we stopped testing with it in Newton20:14
EmilienM(it would be great to also hear from operators)20:14
mordreddhellmann: yup20:14
AJaegerEmilienM: no migration planned yet.20:15
gordcEmilienM: we have no plans to migrate from postgres either20:15
sdaguein pike we should remove it from devstack20:15
ttxsdague: apparently doesn't prevent people from using it. They are surely at risk, but not a reason to remove features with no warning20:15
EmilienMwhat gordc and AJaeger said is a concern we can't ignore it20:15
AJaegerEmilienM: but this discussion here started already discussion20:15
dhellmannjohnthetubaguy : I guess we need to be clear about what "support" means there, because I think 4 may be too big of a number20:15
jrollsdague: "we stopped testing with it in Newton" <- which we is this? nova? openstack?20:15
ttxsdague: not sure what you mean by dev tooling, might be saying the same thing I say20:15
sdaguejroll: openstack20:15
mordredjroll: openstack20:16
mtreinishjroll: we removed the dsvm postgres jobs20:16
johnthetubaguydhellmann: +1 I meant to ask that question, I am not quite sure what that means yet either20:16
sdaguethe main base layer jobs were deleted20:16
jrollall of them?20:16
jrollhuh, TIL20:16
sdaguepretty much20:16
jrollironic tests it, apparently20:16
dhellmannexcept apparently for ceilometer, since that's how the issue came up20:16
gordcjroll: we had it in ceilometer which is where we always find the postgres issues20:16
ihrachys_neutron has a periodic job for psql20:16
AJaegerand ceilometer has a job up to remove it as well20:16
mtreinishjroll: we didn't delete *postgres* just the job that ran everywhere20:16
AJaegerhas a change I mean20:16
fungijroll: this came up because ceilometer still tries to do some tests with postgres and there were recent changes which broke it because we don't generally integration test with that any longer20:16
mtreinishso projects that had one offs kept them20:17
jrollmtreinish: so "we" is the integrated gate. not openstack.20:17
fungier, what gordc said20:17
gordcfungi: :)20:17
jrollfungi: yep, I've been reading the thread20:17
dhellmannISTM that there's a lot more going on with postgresql in the community, dev and deployer, than we originally thought20:17
thingeereference discussion of dropping gate job http://lists.openstack.org/pipermail/openstack-dev/2016-August/thread.html#10189220:17
sdaguedhellmann: I actually don't think that's true20:17
jrollthingee: thanks for that20:17
ttxsdague: basically I agree that we should align documentation with reality (now). I'd like to give a transition period before we actively break anything for people for which that may actually be working fine20:17
dhellmannsdague : I hear you saying we're not testing it, and I hear 3 projects saying they are20:18
ttxof which there seems to be a lot20:18
sdaguettx: ok, but define actively break20:18
fungittx: we seem to be breaking things anyway20:18
dimsdhellmann : the main thing i am worrying about is moving all the existing user deployments..20:18
mordredthe lack of postgres representation in the user survey has been very consistent20:18
fungi(because of not testing it)20:18
jrollto be clear, I'm not advocating for or against dropping postgres here, I just want to be clear that we're talking about the integrated gate, not openstack as a whole20:18
sdagueright, we took away all that testing20:18
sdaguemordred: ++20:18
ttxsdague: like remove code that works around pgsql glitches20:18
thingeemordred: ++20:18
EmilienMdims: what about the ones who don't want to leave pg?20:18
ttx(not in tests, in service code)20:18
dhellmannjroll : my impression is that we're talking about encouraging all projects to drop psql support20:18
dhellmannthat's what the end of the resolution says20:19
dimsEmilienM : right20:19
flaper87I think, like with everything else, we should provide a transition plan out of psql20:19
ttxdhellmann: I think the resolution encourages all projects to ignore it20:19
fungiit would be interesting to get more information on wy some operators are deploying with postgresql, and even moreso why some distributions chose to standardize their openstack designs on it20:19
ttxflaper87: my thoughts too20:19
* flaper87 doesn't like to assume it'll all be good20:19
jrolldhellmann: sorry, I mean when we talk here about how we've stopped testing it20:19
thingeedhellmann: yeah I prefer to think the way cdent put it. yes mysql.20:19
sdagueso, I don't think there will be a transition plan20:19
dhellmannttx: "And that the recommendation is they remove it from their code, documentation, and testing at the earliest convenience."20:19
EmilienMI understand the "why" of the proposal. I'm just very worried about our community who use pg, like Suse or Huawei and probably much more20:19
pabelangersdague: ++20:20
sdaguethe fact is, it's level of community support is very low20:20
johnthetubaguywhen we say drop support, do we mean actively break it, or make it low priority and let fixes in still, or deprecate then break it?20:20
mtreinishEmilienM: that's why we have a deprecation procedure20:20
EmilienMflaper87: yes that but until it's planned, we shouldn't remove support or jobs like we're doing20:20
ttxdhellmann: right, that's the part I disagree with.20:20
gordcjohnthetubaguy: i think the current state is the 2nd20:20
sdaguesaying "we have to have a transition plan" means we actually need a ramp up on it, to have any idea that it works, plus transition tooling, plus... docs?20:20
flaper87sdague: transition plan, 1 cycle notice.. tomatos, tomatos20:20
stevemarjohnthetubaguy: " make it low priority and let fixes in " seems the least harmful20:20
flaper87EmilienM: yup20:20
mtreinishin reality it's a small percentage of users, and we can drop support for it gracefully20:20
bswartzmanila gate tests do ensure postgres compatibility, but if the rest of the projects start allowing postgres bugs to go unfixed we'll eventually have to drop support too because we won't be able to test20:20
ttxdhellmann: I disagree with actively breaking it without a deprecation period20:20
dimssdague : so another way is to get folks to show up to help? (dunno how, but...)20:20
sdaguedims: but we don't want other folks to show up20:21
dhellmannsdague : it is not clear that everyone feels that way. I'm unconvinced.20:21
* jroll points out that switching database engines is WAY harder to do than a normal feature or config deprecation, which is what our deprecation process is designed for20:21
thingeeI'm also not advocating for keeping pgsql, but this decision will further prevent the user survey changing or for it to become the defacto20:21
johnthetubaguyttx: I think I am with you, I would also say "formal deprecation" period20:21
mordredright. we _want_ to be able to do better things with HA and similar stories so that we can provide a better product for our operators20:21
stevemarjroll: ++20:21
dimsjroll : true20:21
sdaguemordred: ++20:21
AJaegerI suggest to have at least "low prio and get bugs in " for the transition period.20:21
dimsfolks probably have more things built on top of openstack20:21
EmilienMbecause of the fact we did support pgsql (probably without many testing), we can't just throw it away like this20:22
sdagueand better compatibility on the API20:22
johnthetubaguyjroll: +120:22
ttxmordred: I agree with the end goal. I disagree with actively breaking people for which that might accidentally work20:22
* AJaeger likes to be able to upstream patches20:22
mordredsdague: yes compat on the API20:22
sdagueEmilienM: you assume that we did support pg officially20:22
ttxat least not without a deprecation notice/period20:22
dimsAJaeger ++20:22
mordredttx: sure. I do not have any problem with deprecation periods20:22
sdagueThat's the crux of my issue with the requirement of making this super graceful to get rid of it20:22
EmilienMsdague: we did or we didn't. The reality is some Clouds probably in production are using it and we can't ignore it now20:22
flaper87EmilienM: ++20:23
sdaguethe reason pg was in the gate, was because I spent 2 months making that happen in 2013 over some unrelated licensing concerns20:23
johnthetubaguythe way I see this, the deprecation period here would help concerned folks get together I find the best way forward, the really important job we need to do here is wave the big red flag that something needs doing20:23
dhellmannjohnthetubaguy : ++20:23
mordredjohnthetubaguy: ++20:23
fungiwe collectively allowed people who figured out how to get openstack running with postgres to add sections to official deployment documentation saying how to do it, for example20:23
dimsagree johnthetubaguy20:23
ttxwe could also say "unless someone shows up to actually test it, we'll remove support for it"20:23
dhellmannfungi : ++20:23
sdaguefungi: we did, without any resolution20:23
ttx"because in OpenStack, what's not tested is broken"20:24
gordcttx: what does 'test it' mean?20:24
sdagueand this is about setting direction that we want out of that game20:24
cdentis it allowed to directly go ask people who we know are running it to pony up resource to support it? the passive option of waiting doesn't seem great20:24
ttxgordc: maintain functioning gate ?20:24
mordredttx: I'd rather we didn't say that, to be quite honest, because there is still a ceiling on how good we can make openstack when we're trying to engineer around 2 different wildly-divergent database backends20:24
sdaguecdent: I honestly don't want to go down that road20:24
sdaguemordred: ++20:24
EmilienMttx: "test it" and also "proactively maintain it"20:24
ttxmordred: that is a good point. Like I said, I agree with the end goal20:24
gordcttx: hardware resources? or development? iirc, the original reason for removing was mainly due to hardware constraints20:24
mordrednobody in the world who is running LARGE systems does it on "either mysql or postgres, kinda whichever"20:24
*** mickeys has quit IRC20:24
dhellmannI'd be OK with the idea of saying we don't want maintenance if we have an extended deprecation period20:25
sdaguethat's the big point, by making this an abstraction layer instead of a foundation, we fundamentally limit openstack progress20:25
fungiworse, when trying to engineer around nuances of johnthetubaguy's four different database backends ;)20:25
mordredsdague: ++20:25
johnthetubaguyfungi: better than five though right ;)20:25
ttxsdague: so we should not say we remove pgsql because it's broken, but because it prevents us from relying on advanced DB features20:25
dhellmannfungi : right, I think we're going to end up having to pick a clustering solution, too20:25
sdagueand, I hear, people get frustrated and rant about pace of openstack progress some times20:25
EmilienMgordc: I think the biggest challenge is to maintain CI jobs running well with pg (and deal with failures when they happen, like we currently do with mysql)20:25
flaper87so, I think there's agreement on the goal but not entirely on how we get there20:25
mordreddhellmann: yes. being able to pick one of those would be amazing and allow us to move the needle forward20:26
flaper87should we proceed with voting either here or on the review?20:26
ttxsdague: are they saying we are too fast or too slow ? Because I hear both20:26
*** rfolco has quit IRC20:26
mordredttx: I hear both at the same time many times :)20:26
gordcEmilienM: periodic gate like neutron has now? i know the bugs we find at product end have been pushed back into upstream20:26
dtroyerBeing more opinionated about things like this (messaging, etc too) will let us speed up somewhat20:26
gordcbut there's a noticeable lag between upstream patch to product team20:26
johnthetubaguysdague: +1 we will have to be more opinionated to go faster, and keep going20:26
flaper87I feel we're starting to say the same things so I'd rather move onto the next phase of the discussion20:27
mordredflaper87: ++20:27
ttxflaper87: yes please20:27
johnthetubaguyso what do we write on the big red flag?20:27
mordredjohnthetubaguy: a sonnet?20:27
ttxSounds like there is some consensus on the end goal, but diagreement on the way to state it and how fast to proceed20:27
flaper87so, should we vote on whether we should have a deprecation period here or on the review ?20:27
johnthetubaguymordred: if that makes people read it, it could totally work :)20:27
ttxflaper87: I propose we continue on the review20:27
jrollI think a good first step would be make sure install guide says mysql20:27
mordredjohnthetubaguy: :)20:27
mordredttx: ++20:27
flaper87ttx: I like that20:27
jrollI don't think any of us would disagree with that20:27
ttxit will take some time to hammer out20:27
jrollthe changing existing pg deployments is the hard part20:28
AJaegerjroll: It does already20:28
mordredttx: maybe we should also tee up a session at the Forum to discuss this20:28
dhellmannmordred : ++20:28
AJaegerjroll: and only MySQL/MariaDB20:28
fungialso as has been pointed out, this will serve as prior art for the message bus discussions still to come20:28
Rockygmordred, ++20:28
jrollAJaeger: it doesn't mention postgres at all?20:28
jrolloh nice20:28
mordredfungi: ++20:28
Rockygalso at ops midcycle20:28
AJaegerjroll: Correct -t he Install Guide shows basically one path - and opinoined way to install20:28
gordcjroll: it says postgres is also supported in install guide20:28
ttx#action mordred to propose a pgsql discussion at PTG (and probably another one at Forum) when time comes20:28
sdaguejroll: again, what gets exposed in the community around pg is really inconsistent20:29
flaper87mordred: thanks20:29
* jroll reads conflicting information20:29
gordcbut all the steps are for mysql20:29
mordredttx: wow. I shoudl have seen that coming20:29
ttxmordred: I'm fast20:29
*** ansmith has quit IRC20:29
ttxOK, I propose we move on20:29
jrollsdague: yeah so we should clean up stuff that new deployments would use to just mysql20:29
johnthetubaguyI think we need something before then, though, but we can talk on the review about that20:29
jrolland go from there with existing stuff20:29
sdaguejroll: yep20:29
fungii still mostly want to know _why_ people are chosing the less popular option here, so that we have better data on whether it's just an arbitrary choice or there are definite behavior/performance needs not being met by the default option20:29
ttxlet's iterate on review, and discuss next topic20:29
ttxdolphm: around ?20:29
fungisounds good20:29
johnthetubaguyfungi: +1 me too20:30
dolphmttx: o/20:30
ttx#topic Disallow downtime of controlled resources during upgrades20:30
*** openstack changes topic to "Disallow downtime of controlled resources during upgrades (Meeting topic: tc)"20:30
ttx#link https://review.openstack.org/40436120:30
ttxThe language in the latest draft looks good to me20:30
ttxI guess the main unknown is how exactly you would validate "accessibility of a controlled resource" throughout the upgrade process20:30
ttxSome grenade code ?20:31
*** cody-somerville has quit IRC20:31
*** AJaeger has left #openstack-meeting20:31
stevemarttx: some tests, yes, could be grenade20:31
*** cody-somerville has joined #openstack-meeting20:31
*** cody-somerville has quit IRC20:31
*** cody-somerville has joined #openstack-meeting20:31
johnthetubaguyan ssh session being open, kept coming up, just saying that for reference really20:31
mtreinishttx: grenade does poll things while the services are shutdown20:31
dolphmi'm hoping to answer that at the PTG, but yes... my thought is to extend/adapt the resources that grenade creates beforre an upgrade, to also be exercised during the upgrade20:31
dolphmmtreinish: so, if that becomes an asynchronous an ongoing validation process, i think ti satisfies the tag for the projects it covers20:32
dhellmannthat seems like a reasonable approach, and I think we can add details to the tag with some examples later when we have some20:32
ttxmtreinish: ok, so enough to assess "accessibility of a controlled resource"20:32
sdagueI think that's going to be the basic challenge, everything we do in testing today is sequential20:32
sdaguebecause it's not just the validation, it's how the actual upgrades are done as well20:33
johnthetubaguysdague: +1 was just tying something like that20:33
*** dprince has joined #openstack-meeting20:33
mtreinishttx: well it's not a continous check more after thigns are shutdown can I still ping server x20:34
ttxmtreinish: I think that the new wording is good with that20:34
dolphmttx: i disagree, the keyword being "throughout" the upgrade process20:35
* ttx looks up latest patchset20:35
dolphmtoday, the accessibility of controlled resources is tested basically once during the upgrade process20:35
sdaguedolphm: well, 3 times20:35
dolphmttx: L59 https://review.openstack.org/#/c/404361/8/reference/tags/assert_supports-accessible-upgrade.rst20:35
sdaguebefore services drop, when they are down, after they are upgraded20:36
dolphmsdague: once before the upgrade begins, once during the upgrade process, and once after the upgrade?20:36
ttxShould we add some precision so that testing 3 times is enough ?20:36
ttxI just want to remove the uncertainty there20:36
mordredone could imagine perhaps opening an ssh connection to a vm in a thread/subprocess, then doing the upgrade, then making sure the ssh session doesn't drop20:36
fungion the ssh session idea... ssh can resume seamlessly after rather lengthy periods of total packet loss20:36
ttxmordred: or that20:36
dhellmannI don't think we need to solve the implementation ideas in this meeting, do we?20:36
mordrednot evne a little bit :)20:36
sdaguewell, I think we do want to have an implementation before the tag though20:37
ttxok, so we can judge that when the tag is proposed20:37
ttxto a specific project20:37
dolphmthat's true20:37
mordredI'm more musing on that by way of exploring what sorts of things this could be wanting to require20:37
sdaguebecause if we think it's testable, we should work that out first20:37
dhellmannI'm not sure we do. We've clearly agreed on "throughout". Different projects may have different ways of testing that.20:37
fungiyou can ssh to an instance, suspend the instance for an hour or more (in the right network environments at least), resume the session and then go back to using that same socket without it ever disconnecting20:37
johnthetubaguywe get to revise the tag until someone gets the tag?20:37
ttxI suspect we'll refine the tag definition when a project actually asks for it :)20:37
flaper87dhellmann: ++20:37
sdagueplus, we all need to realize that testing is a model20:37
johnthetubaguysdague: I still wonder if it has value in saying "hey we don't test this thing yet"20:37
dhellmannjohnthetubaguy : or after, if it's a backwards-compatible change20:37
sdaguethe point isn't to catch everything20:37
* ttx is good with that. Baby steps20:38
mordredsdague: ++20:38
sdaguethe point is to catch representative errors that people missed in code review20:38
ttxIf you feel good, pile up +1s and I'll approve now20:38
johnthetubaguysdague: +120:38
fungiright, which is why i think it's sufficient to just test sample (once, twice, whatever) during the process and assume things weren't offline in between and miraculously came back just in time for the next poll20:38
fungithough that's also a good way to sneak nondeterminsitic failures in20:39
ttxOne more +1 to approval20:39
ttxAny objection ?20:39
johnthetubaguyI feel we should merge what we have, and adjust it if we need to before we approve the first project to get the tag20:39
ttxjohnthetubaguy: yes20:39
sdaguejohnthetubaguy: that seems fine20:39
dhellmannfungi : ++ to using reasoning to apply good, minimal testing practices20:39
EmilienMjohnthetubaguy: +1, baby steps20:39
ttxAlright, approved20:39
ttxThanks dolphm !20:39
fungidhellmann: and keeping an eye out for races. polling-related test like that are a great surface area to grow race conditions on20:39
stevemar++ thanks dolphm20:40
ttx#topic Progress on stalled reviews20:40
*** openstack changes topic to "Progress on stalled reviews (Meeting topic: tc)"20:40
dhellmannfungi : true20:40
ttx* Status update on Golang CTI20:40
ttx#link https://review.openstack.org/41035520:40
ttxdtroyer: how is that going ?20:40
dolphmcool, landing these tags has certainly driven a lot of conversations in the right direction (reasonable testing practices, as someone just mentioned)20:40
dolphmso thank you all!20:40
Rockygfungi, ++20:40
dtroyerI need to get back to it, runup to release has consumed me, but I think it's pretty close20:40
dhellmanndolphm : thanks for working on this20:40
* dims needs to review that golang_cti in depth20:41
ttxdtroyer: ok, so WIP but you'll pick it up soon20:41
dtroyerI kept flaper87's doc in mind and IIRc there may only be minor changes to fit that20:41
ttxOn that same topic...20:41
ttxflaper87: do we want to grandfather in that step 1 for golang addition (the use case analysis) based on the Swift case ?20:41
ttxDid you find any volunteer to file that ?20:41
flaper87I sent the email out and reached out but no one signed up for it AFAICT20:42
flaper87unless I missed something entirely obvious20:42
*** mickeys has joined #openstack-meeting20:42
flaper87I would love to do a first runtest of this process with Swift20:42
dtroyerI've been operating under the assumption that we can refer to that as necessary20:42
ttxok, maybe reach out to the ones that should be interested see if they still care20:42
flaper87ttx: erm, done that already20:42
* flaper87 shrugs20:42
ttxok. Maybe we could tie loose ends at PTG20:42
ttx* Status update on Driver discoverability / need for driver team recognition20:43
ttxA couple weeks ago we tabled that discussion while waiting for (some) progress on driver discoverability20:43
flaper87yes, I'm hoping for that and feel free to action me on it20:43
ttxI was wondering when we would be comfortable to reconsider those various options for driver team recognition20:43
ttx#action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis)20:43
ttxSo when is that ? post-PTG ? Later ?20:43
ttxthingee: I suspect there won't be much visible progress on discoverability before a while ?20:44
flaper87+!1 for post PTG to take the drivers discussion again20:44
ttxAlso a bit of PTG beers might convince the ones or the others20:44
stevemaryeah, post-PTG20:44
*** ansmith has joined #openstack-meeting20:44
stevemarbeers works too20:44
fungiit sounded like there was a fair amount of interest from the www site admins on revamping/improving the driver marketplace and solving the need to automatically aggregate driver details from structured data under the control of individual teams20:45
thingeettx: not yet20:45
ttxfungi: yes, but will likely take some time to implement20:45
fungiit seems we have a path forward on that front at least, if not a timeline (yet anyway)20:46
ttxok, let's revive it post-PTG and use that week to informally discuss it20:46
thingeettx: current noticable progress is neutron moving towards the same common format that nova started for listing drivers. I would like to start communication with other projects to follow20:46
ttxthingee: sounds like a good step forward!20:46
thingeecinder as I believed from various people in the team back in barcelona are also going to be moving towards this same common format20:46
ttx* Progress on Introduce assert:supports-api-compatibility20:46
dhellmannthingee : is neutron listing drivers that are outside of its repos?20:46
*** dimtruck is now known as zz_dimtruck20:47
openstackRemoving item from minutes: #action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis)20:47
thingeeonce that's in place the foundation will work with the community on displaying things in the market place with the common format.20:47
ttx#action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis)20:47
mtreinishttx: hah :)20:47
flaper87ttx: stop taking work away from me20:47
flaper87no wait, do it! take work away from me20:47
ttxI need #subtopic20:47
thingeedhellmann: yes20:47
mtreinishthat meetingbot always surprises20:47
dhellmannthingee : ok, good20:47
mtreinisheverytime I undo it's never what I want either20:47
ttx#link https://review.openstack.org/41801020:47
mordredttx: patches accepted :)20:47
ttxThis one was blocked while the API-WG refreshes their "API changes guidelines" that this proposal references20:48
cdentyeah, still working on that20:48
mordredit's a fun mailing list thread20:48
cdentin the rush of release, has been slow generating the necessary input to get a full picture of the context20:48
ttxmordred: already need to add Slack-like 'notify me when I'm mentioned in another channel the bot lurks in" feature20:48
thingeedhellmann: https://review.openstack.org/#/c/318192/20:48
cdentbut it's getting there20:48
mordredthingee: EDONOTWANT20:48
mordredttx: EDONOTWANT20:48
thingeemordred: ?20:49
ttxmordred: would let us get rid of common meeting channels though :)20:49
dhellmannthingee : maybe we can move some of that css and templating into oslosphinx when we have 2 that are similar enough20:49
ttxcdent: ok cool, the other review is stalled while we wait for that step to complete20:49
*** zioproto has joined #openstack-meeting20:49
thingeedhellmann: yes there is already duplication happening with the script that generates it.20:50
cdentttx: yeah, that's the somewhat regrettable but desired outcome20:50
mtreinishttx: to be fair a lot of projects have been living under the current guideline for a while20:50
*** goldenfri has joined #openstack-meeting20:50
mtreinishit was a tc approved thing back in the day20:50
smcginnisCinder patch for driver info: https://review.openstack.org/#/c/371169/20:50
ttxmtreinish: so technically we could pursue both in parallel ?20:50
thingeesmcginnis: excellent!20:50
mtreinisherr or ppb: https://wiki.openstack.org/wiki/Governance/Approved/APIStability20:50
cdentttx, mtreinish I don't think that a good idea20:50
dhellmannthingee : either oslosphinx or a new project, whatever's easier20:50
cdentthe existing guidelines are flat out wrong in many ways20:51
smcginnisI'd also like to point out we have driver information automatically pulled out of the source and published here: http://docs.openstack.org/developer/cinder/drivers.html20:51
ttxmtreinish: archeowiking is not that great20:51
cdentand i think the modern reality of the big tent and the concept of tags changes the concepts of stability quite a bit20:51
* ttx keeps on adding ETOOOLD notices to pages that some people exhume20:51
*** ansmith has quit IRC20:51
mtreinishttx: sure, I'm just saying this tag was more my attempt to modernize the rules projects have been using20:51
cdentat least the discussion thus far certainly suggests that to be the case20:51
johnthetubaguysmcginnis: very nice20:51
*** amotoki has quit IRC20:51
*** priteau has joined #openstack-meeting20:52
ttxok, so let's see where cdent's modernization effort takes us first20:52
cdentshouldn't be too much longer20:52
ttx#topic Open discussion20:52
dimsgo cdent!20:52
*** openstack changes topic to "Open discussion (Meeting topic: tc)"20:52
ttx* Board+TC workshop on "OpenStack Futures"20:52
ttxSo finally it looks like this will happen March 8-9 in Boston20:52
thingeesmcginnis: although hmm, I was kind of hoping we'd have one way to generate these. This seems different from neutron did with the .ini file20:52
sdaguecdent: to be fair, I think that 90% of the guideline is pretty much going to be the same20:52
ttxa.k.a. "week of March 6 in Boston" in the poll20:52
dimsnice thanks ttx20:52
flaper87mmh, does dates are horrible for me :(20:52
sdaguecdent: only 10% really gets tweaked20:52
ttxThere would be a 1-day workshop with the board on March 8, followed by some common evening activity (which should include food)20:52
flaper87I won't be able to attend20:52
thingeesmcginnis: how is nova generating theirs?20:52
cdentsdague: that's not the impression I've had from discussion, actually20:53
dhellmann#info all milestone-based projects have tagged their RC1 and branched stable/ocata20:53
* flaper87 thought we were voting for March 6th20:53
*** rtheis has joined #openstack-meeting20:53
cdentwhich is why I've been stalling following through on rubber stamping it20:53
ttxOn March 9 the Board will hold a traditional board meeting, while interested TC members would work on the "TC vision", maybe with a facilitator from ZingTrain if gothicmindfood can arrange that20:53
smcginnisthingee: I'll have to see what they did. I'd rather not maintain an ini file, but if that makes everyone consistent, then all for it.20:53
cdentbecause there's lots of input that contradicts the cw20:53
ttxflaper87: frankly, no date was perfect20:53
EmilienMflaper87: :(20:53
*** rtheis has quit IRC20:53
ttxbut yes, we assumed it was ok with you based on that vote20:53
ttxThose TC members not interested in working on the TC vision nor in the board meeting could go home early20:53
ttxI'm confirming all the details and will send something out20:54
ttxas we need to RSVP20:54
thingeesmcginnis: I'll take a look at nova and bring it up in the thread.20:54
smcginnisthingee: Cool!20:54
flaper87bah :(20:54
ttxWill very likely happen in DLA Piper offices, which means we have to RSVP20:54
sdaguethingee: nova's is generated with a custom sphinx extension, johnthetubaguy wrote it20:54
sdagueoh, sorry, danpb originally wrote it20:54
ttxThe address is 33 Arch Street20:54
thingeesdague: https://review.openstack.org/#/c/371169/14/doc/ext/driver_support_matrix.py ?20:54
johnthetubaguyyeah, its evolved from dan's20:54
sdaguethingee: yeh20:54
mtreinishttx: ok cool20:55
sdagueini file was used because parser was already in python20:55
sdagueand, the alternatives were actually pretty much just a gross20:55
johnthetubaguythingee: the plan was to move that into some oslo lib, not quite done that bit yet20:55
ttxlet me know if you have any questions. We coupled both things to make the travel more worthwhile and avoid yet another trip for TC visioning20:55
dhellmannthingee: I haven't looked at the rendering code for the driver stuff, but it sounds like the sort of thing sphinxcontrib-datatemplates would be good for20:56
*** oneswig has joined #openstack-meeting20:56
dhellmannttx: how firm are the dates?20:56
stevemarttx: will a formal invite (or something else) be sent out ?20:56
fungiyeah, it's right by boston common. at least there are lots of places to eat ;)20:56
*** ijw has joined #openstack-meeting20:57
ttxdhellmann: my understanding is that they are final. I'd like to doublecheck that though, since it's VERY fresh20:57
ttxlike 5min ago20:57
dhellmannttx; ok20:57
EmilienMttx: can you send details asap so we can book flights, etc?20:57
dhellmannI'll wait to book a flight :-)20:57
ttxThat is my plan. After all I'm the one coming from the furthest away20:57
johnthetubaguydhellmann: thingee: there are some nova OSIC folks looking at tidying ours up again next cycle, let me know if there is a better direction to consider20:57
* stevemar still needs to be book his ptg flight20:57
* flaper87 is literally going on PTO on the 8th, something that doesn't happen often20:57
thingeesmcginnis: http://git.openstack.org/cgit/openstack/nova/tree/doc/source/support-matrix.ini20:57
fungittx: well, some of the board members are probably attending from asia ;)20:57
dhellmannflaper87 : enjoy the time off!20:58
ttxfungi: I won't send them any confirmation though20:58
EmilienMflaper87: well deserved20:58
fungittx: fair enough20:58
smcginnisthingee: Looks like I could probably adapt our automated way to generate most of that ini.20:58
ttxflaper87: I don't expect anything final to come out of those two days. Will be missing you though20:58
ttxAnything else, anyone ?20:58
*** rbudden has joined #openstack-meeting20:59
ttxOh, re: PTG we have an ethercalc instance now, so I'll be setting up booking forms for the discussion room and other shared resources20:59
ttxnot forms, more like tables20:59
ttxso we'll be able to start booking extra stuff21:00
mordredhttps://ethercalc.openstack.org/ <-- yay21:00
*** rfolco has joined #openstack-meeting21:00
stevemarttx: the info will all be here right? https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting21:00
ttxand we are out of time21:00
ttxstevemar: mayyyybe ? I'll send something to the TC list21:00
stevemarttx: thanks21:00
ttxmordred: yes, please test and crash it so that we know if we can rely on it or not :)21:00
*** lizhong has joined #openstack-meeting21:00
ttxand we are out of time21:01
ttxThanks everyone21:01
EmilienMthanks ttx for chairing21:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"21:01
openstackMeeting ended Tue Feb  7 21:01:08 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tc/2017/tc.2017-02-07-20.01.html21:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tc/2017/tc.2017-02-07-20.01.txt21:01
openstackLog:            http://eavesdrop.openstack.org/meetings/tc/2017/tc.2017-02-07-20.01.log.html21:01
johnthetubaguysmcginnis: I was planning to try and generate it from the code, I should take a peak at what you put together21:01
*** martial has joined #openstack-meeting21:01
smcginnisjohnthetubaguy: Sure. You might hate it. Or you might love it. Worth looking. ;)21:01
smcginnisjohnthetubaguy: I'll find my patches for that to make it a little easier.21:01
johnthetubaguysmcginnis: but its largely not just implementing a method call, its a bit nasty to tell if something is supported right now21:02
johnthetubaguysmcginnis: that would be cool, thanks21:02
*** jonmills_nasa has joined #openstack-meeting21:02
*** gordc has left #openstack-meeting21:02
oneswigLets get this show on the road?21:02
oneswig#startmeeting scientific-wg21:02
openstackMeeting started Tue Feb  7 21:02:22 2017 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.21:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:02
*** openstack changes topic to " (Meeting topic: scientific-wg)"21:02
openstackThe meeting name has been set to 'scientific_wg'21:02
trandleshi everyone21:02
*** ykatabam has quit IRC21:02
oneswigHi zioproto - up late!21:02
* ildikov is lurking :)21:02
martialHi Stig21:02
oneswigHi all21:02
oneswig#chair martial21:02
openstackCurrent chairs: martial oneswig21:02
martialHi all21:02
* cdent waves 21:02
rbuddenhello everyone21:02
*** cdent has left #openstack-meeting21:02
martial(sorry dual meeting right now -- in person + IRC)21:03
*** raildo has quit IRC21:03
oneswigwe'll try not to cause you any sudden outbursts in your real meeting Martial :-)21:03
oneswig#link Agenda for today https://wiki.openstack.org/wiki/Scientific_working_group#IRC_Meeting_February_7th_201721:03
oneswigfreshly minted21:03
*** ansmith has joined #openstack-meeting21:03
martialoneswig: thanks will be listening in both :) [working my multitasking skills :) ]21:03
oneswigPerfect.  Anyone seen Blair yet?21:04
*** b1airo has joined #openstack-meeting21:04
oneswig#chair b1airo21:04
openstackCurrent chairs: b1airo martial oneswig21:05
oneswigI rubbed the lamp - that's the truth21:05
b1airomorning, bit tardy sorry21:05
b1airowhat have i just walked in on?21:05
oneswigHi b1airo21:05
oneswigonly the minutes can answer that!21:05
oneswigReady to talk HPL and hypervisors?21:06
b1airorubbed the lamp... did you win something, or literally find a lamp?21:06
oneswig... aladdin21:06
oneswig#topic HPL and hypervisors21:06
*** openstack changes topic to "HPL and hypervisors (Meeting topic: scientific-wg)"21:06
*** ianw has quit IRC21:07
b1airook now that i say yes i need to paste some graphs21:07
*** ianw has joined #openstack-meeting21:08
b1airothis one is SMP linpack:21:08
b1airo#link http://pasteboard.co/vBwRsur2Q.png21:08
oneswigTaking a while to reach me here...21:09
zioprotois slow21:09
zioprotook I have loaded it21:09
oneswigaha, got it21:09
*** harlowja has quit IRC21:09
b1airothis one is HPL with OpenMPI (still single host though): http://pasteboard.co/vByv44w1d.png21:09
b1airo#link http://pasteboard.co/vByv44w1d.png21:09
*** rfolco has quit IRC21:10
b1airoacks to my colleague Phil Chan for generating these21:10
oneswigThese are hostnames along the x-axis?21:10
*** sdake has quit IRC21:11
b1airothe shortname yeah - they are not publicly accessible though so not too worried about security if that was your thinking?21:12
oneswigJust wondering what they were - hostnames, parameter variations, ?21:12
b1airoso you have guest name and hypervisor name along the x21:12
*** powerd has joined #openstack-meeting21:13
b1airoso basically the story goes like this...21:13
oneswigWhat are the five 120k outliers in the middle of the HPL chart due to?21:13
b1airoindeed, what?21:13
oneswigI came here tonight for answers!21:13
*** armstrong has quit IRC21:14
*** eglynn has quit IRC21:14
*** zz_dimtruck is now known as dimtruck21:14
b1airowe only put these nodes into production semi-recently as mellanox had been kind of using them as a RoCE MPI testbed for a while (there's a bunch of fixes and improvements in MOFED 4.0 thanks to this)21:14
b1airoanyway, some of our users complained of jobs running much slower on these nodes than on the other 4 year old cluster, so we started digging21:15
*** eglynn has joined #openstack-meeting21:16
*** ijw has quit IRC21:16
b1airojust thought i'd share what we have so far as it is a nice sample showing how tight BM-guest can be21:17
oneswigVMs appear to have a performance edge for hpl - any theories on why?21:18
b1airothe next thing is to figure out what is going on with those outliers - presumably something on the hypervisor is getting in the way at the highest memory utilisation levels, but not sure what yet as they are all the same21:18
b1airooneswig, at this stage i'm guessing it is just the difference in task scheduling between kernel versions21:18
b1airohost is Ubuntu with 4.4, guest is CentOS7 with 3.10(?)21:19
oneswigIs the NIC involved at all here, or is it memory-copy between VMs?21:19
b1airothe guest is pinned etc so host task scheduling is somewhat taken out of the picture there21:19
oneswigOr is it just one VM21:20
b1airoyeah just single host runs, no IPC21:20
*** ijw has joined #openstack-meeting21:20
*** gouthamr has joined #openstack-meeting21:21
b1airoi started looking at whether the guest memory was pinned like i thought it was, but seems the idea i had that pci passthrough forces all guest memory to be pinned was wrong21:21
oneswigperf in the hypervisor any help?21:21
b1airoat least the only place i've found to look for that is Mlocked memory in /proc/meminfo21:22
oneswigYou can apparently get perf traces of stacks from both hypervisor and guest, which is pretty cool (but I've never done it)21:22
b1airooneswig, yes that's one of the next things to look at now that we have a known set of misbehaving machines21:23
b1airoi have never used it yet either, just started reading21:23
*** galstrom is now known as galstrom_zzz21:23
oneswigHave you compared bios settings with racadm?  Might not explain why only the large-block cases are problematic21:24
b1airothink i heard about an ansible role or two that might be useful for that ;-)21:24
oneswigFunny you should say that!21:24
b1airohave not compared them with a fine-tooth comb just yet, but the big "performance" items have been checked21:25
*** delattec has joined #openstack-meeting21:25
oneswigYou'll report back if you figure it out I hope!21:25
b1airoin particular, all these hosts now have highest pstate forced via kernel command line21:26
*** cdelatte has quit IRC21:26
*** jkilpatr_ has quit IRC21:26
b1airoso they idle at a higher frequency than when they are actually working (AVX is a bit disconcerting like that)21:26
*** jrobinson has joined #openstack-meeting21:27
oneswigBecause AVX heats the chip up?21:27
b1airoyes, hopefully that'll be next week! i feel like an amateur detective at the moment, i'm just bouncing from one performance issue to another21:27
oneswigTime for the lowdown on ECMP?21:27
b1airooneswig, yes all the modern Intel CPUs have lower base frequencies for AVX instructions (you won't find that anywhere prominent on the product pages though!)21:28
b1airoyes ECMP21:28
oneswig#topic RoCE over layer-3 ECMP fabrics21:28
*** openstack changes topic to "RoCE over layer-3 ECMP fabrics (Meeting topic: scientific-wg)"21:28
zioprotowhat RoCE stands for ?21:29
oneswigSo this is layer-2 RoCE over VXLAN over ECMP, right?21:29
zioprotomemory over ethernet ?21:29
oneswigalso known as Infinband-on-ethernet21:29
oneswigIt's low latency networking21:29
b1airoa couple of weeks ago we started our big datacentre migration, in the process we replaced most of the switching in our research compute infrastructure21:29
*** jrobinson has quit IRC21:30
*** matrohon_ has joined #openstack-meeting21:30
*** nicolasbock has joined #openstack-meeting21:30
b1airowe were previously mostly using Mellanox SX1036 40/56GbE in a layer-2 configuration21:30
*** jrobinson has joined #openstack-meeting21:31
oneswigb1airo: what caused you to change?21:32
*** matrohon has quit IRC21:32
*** sdake has joined #openstack-meeting21:32
b1airosorry burnt toast emergency21:33
b1airoback now21:33
b1airothe size of the fabric we could build was limited to 2x spine switches21:33
oneswigDue to MLAG?21:33
b1airousing either MLAG or a multi-STP config21:33
b1airoand the MLAG had a habit of being a bit fragile21:34
zioprotoso now you have a L3 leaf/spine architecture ?21:34
*** ansmith has quit IRC21:34
b1airoso all our new gear is Mellanox Spectrum (100/50/25GbE)21:34
zioprotobut with more than 2 spines ?21:34
b1airowith switches all running Cumulus21:34
b1airozioproto, actually not yet - but yes we could have more spines or insert another aggregation layer21:35
*** harlowja has joined #openstack-meeting21:35
jonmills_nasaWe presently running with MLAG, all Cumulus, 2 spines and 10 leaf switches.  So far so good.21:36
b1airothe switches speak BGP to each other and VXLAN encap/decap traffic to/from the edge ports21:36
b1airo(in hardware)21:36
oneswigWe are old school - 2 core, 9 edge, MLNX-OS + NEO21:36
jonmills_nasaIs this Cumulus 3.2.0?21:36
zioprotob1airo, are you using puppet or ansible to configure the cumulus switches ?21:37
b1airojonmills_nasa, yep21:37
b1airozioproto, ansible21:37
b1airohonestly i don't think puppet really belongs with switching21:37
*** salv-orlando has joined #openstack-meeting21:37
jonmills_nasaI sorta thought that hardware accelerated VTEP wasn't supported in Cumulus until 3.3 release or even 3.4....21:37
oneswigb1airo: how are the VXLAN VNIs configured?  Is there a physical-aware ML2 driver behind that?21:38
b1airoanyway, it was a huge effort to get it all built and configured - we actually started moving hosts before the network configs were completed21:38
*** askb has joined #openstack-meeting21:39
*** mgoddard has joined #openstack-meeting21:39
b1airooneswig, no at this stage we haven't added any features just done the migration - so the automation has port profiles setup which configure the VNIs appropriately for the required host-facing VLANs21:39
oneswigAh OK, so the host's vlans are mapped to ranges of vnis in the vtep?21:40
b1airojonmills_nasa, this is using Cumulus LNV to coordinate the VNIs across switches, switching definitely happening in hardware21:40
b1airooneswig, yep21:40
*** sdake_ has joined #openstack-meeting21:41
b1airoonce we had all the gear migrated to the new DC we started properly testing the new network21:41
jonmills_nasab1airo, this is great stuff, this is my long-term goal -- just didn't think I could get there on first pass, so we started with all L221:41
*** dbecker has quit IRC21:41
b1airojonmills_nasa, cool - glad to have some company! Cumulus is very nice to work with I have to say21:41
jonmills_nasaI am loving it21:41
b1airowe used Lustre LNET self test21:42
jonmills_nasaI generated 100% of my L2 edge port config using a very small shell script with NCLU commands21:42
*** salv-orlando has quit IRC21:42
b1airowe run Lustre over the o2ib driver as usual but on RoCEv1 instead of IB21:42
martial(so far very cool)21:42
zioprotob1airo, we also have Cumulus at SWITCH21:42
zioprotob1airo, but we are using puppet and OSPF unnumbered for the fabric21:43
*** yamamoto has joined #openstack-meeting21:43
*** jkilpatr has joined #openstack-meeting21:43
b1airowe firstly had to ensure the network could handle RoCEv1 at all, which means needing flow-control everywhere (RoCEv1 does not like lossy networks)21:43
*** sdake has quit IRC21:44
b1airothe first hurdle there was that we couldn't just turn on global pause like we had previously on the MLNX-OS L2 fabric21:44
oneswigI hadn't realised that SR-IOV can be lossy - flow control ends at the phy on Mellanox NICs21:44
b1airooneswig, actually we should go into that more in a moment as they haven't told us anything particular about that (but of course I am interested to hear it!)21:45
*** gouthamr has quit IRC21:45
b1airothe Cumulus folks were leery about enabling pause on the inter-switch links because they were worried BGP peer exchange messages would might get held up long enough that it would then impact the fabric topology21:46
b1airoso we had to have priority flow-control configured inter-switch with switch-switch traffic at the highest CoS21:47
b1airothen we start pushing large streams around and immediately noticed only a single link utilised up and down when crossing spines21:48
b1airoturns out that the VXLAN encap happens before the packet is routed to the next hop, so the ECMP hash is done on the VXLAN'd packet, and for RoCEv1 at least that 5-tuple looks exactly the same for every packet on the same VNI21:49
zioprotoblair, that is a typical hashing problem21:50
zioprotomost hashing stuff expect UDP or TCP21:50
*** salv-orlando has joined #openstack-meeting21:50
zioprotowhen you has other protocols hashing algorithm is dumb and sends all to the same link21:51
b1airohence, one out of four 100G paths used and crappy performance - we actually had a simple cluster test we used to simulate what some user workloads to, we call it "crazy cat lady" - it basically just cats a bunch of large files into /dev/null in a parallel job21:51
priteaulove that name!21:51
oneswigwhy is the destination IP of the VXLAN frame not different for flows to different machines?21:51
b1airozioproto, it is particularly bad with flow-control, as the link gets highly congested with pause frames as well21:52
b1airooneswig, this is for flows from any host in rack A to any host in rack B on the storage VLAN - that's a single VNI for each ToR21:53
oneswigahhh, got it, thanks21:53
oneswigwhat's the fix?21:53
*** krtaylor has quit IRC21:54
b1airoanyway, the Cumulus support guys spent a couple of days in video conference with us working through testing and workaround21:54
oneswigCan you hash on the decapsulated packet?21:54
martial(how are we doing on time versus agenda?)21:54
oneswigmartial: not very well...21:55
b1airothe fix was to run a little python script that they hacked up which altered some base port settings in the ASIC and introduced some randomness to the hash21:55
oneswigb1airo: is this going to be your gift to Cumulus Linux?21:56
b1airotheir engineering wrote it, but i'll claim i provided the inspiration21:56
*** fguillot has quit IRC21:56
b1airoi believe this is going into 3.2.1 permanently21:56
oneswigIs there a long-term fix in the works, I assume?21:56
oneswigWe should cover some other matters before the clock strikes...21:57
b1airolooks like i monopolised somewhat today (sorry!)21:57
trandlesblame the burnt toast ;)21:57
oneswig#topic Boston Cloud Declaration21:57
*** openstack changes topic to "Boston Cloud Declaration (Meeting topic: scientific-wg)"21:57
martialMay 11 & 1221:57
oneswigHooray - the date is moved to the same week as the summit :-)21:58
oneswig#topic Repo for WG docs21:58
*** openstack changes topic to "Repo for WG docs (Meeting topic: scientific-wg)"21:58
oneswig#link http://git.openstack.org/cgit/openstack/scientific-wg/21:58
zioprotoguys I have to go21:58
priteauoneswig: that's good news. I assume the wiki page will be updated? https://wiki.openstack.org/wiki/Boston-Cloud-Congress21:58
oneswigwe've got somewhere where we can write things now21:58
zioprotobye :) sorry for leaving earlier21:58
oneswigpriteau: I think so - should have been already21:58
oneswigthanks zioproto21:58
*** zioproto has quit IRC21:58
oneswig#topic SC2017 workshops21:59
*** openstack changes topic to "SC2017 workshops (Meeting topic: scientific-wg)"21:59
oneswigI filed for two workshops at SC201721:59
oneswigone on infrastructure and one on platforms/apps21:59
oneswig#link as per discussion at https://etherpad.openstack.org/p/SC17WorkshopWorksheet21:59
b1airooneswig, nice one thanks for getting that done. i saw the notification email come by21:59
oneswigThere's still a week (apparently) before the submissions ship, any more volunteers please?21:59
*** tesseract has joined #openstack-meeting21:59
*** tesseract- has joined #openstack-meeting22:00
oneswigThanks b1airo, no problem22:00
oneswigSome improvement on the wording would help!22:00
b1airooneswig, we haven't pinged the MLs about that have we?22:00
oneswigIf anyone else will be at SC and can contribute, please put your names in the etherpad22:00
rbuddenoneswig: i’ll read it over and can likely volunteer to help22:01
oneswig#action oneswig to put the workshops onto the ML22:01
rbuddeni’m always at SC22:01
oneswigthanks rbudden22:01
b1airorbudden, that'd be great!22:01
oneswigwe are out of time, alas22:01
jonmills_nasaI'm honestly hoping to avoid SC17.  I'll be at Boston Summit tho22:01
rbuddenand as always we have the PSC booth for other SC content22:01
oneswigThanks all and thanks b1airo, I like my toast to be high roast22:01
*** thorst_ has quit IRC22:01
b1airoyou should all prioritise the Sydney Summit!22:01
rbuddenb1airo: i’m trying to pull strings ;)22:02
oneswigThat's my plan b1airo!22:02
trandlesSydney's a go for me22:02
oneswigtime to close, alas22:02
jonmills_nasaGov would never pay for me to go there22:02
rbuddenthe wife has always wanted to visit as well!22:02
b1airoawesome, i should start looking for venues!22:02
oneswigjonmills_nasa: not since they've fingered you as @RogueNASA?22:02
oneswigreally gotta close now :-)22:02
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"22:02
openstackMeeting ended Tue Feb  7 22:02:55 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:02
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_wg/2017/scientific_wg.2017-02-07-21.02.html22:02
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_wg/2017/scientific_wg.2017-02-07-21.02.txt22:02
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_wg/2017/scientific_wg.2017-02-07-21.02.log.html22:02
rbuddensee ya’ll later!22:03
*** jonmills_nasa has quit IRC22:03
*** thorst_ has joined #openstack-meeting22:03
*** b1airo has quit IRC22:03
*** powerd has quit IRC22:05
*** jmckind has quit IRC22:07
*** oneswig has quit IRC22:07
*** yamamoto has quit IRC22:07
*** ykatabam has joined #openstack-meeting22:07
*** tesseract has quit IRC22:07
*** tesseract- has quit IRC22:07
*** thorst_ has quit IRC22:08
*** tesseract has joined #openstack-meeting22:08
*** lizhong has quit IRC22:09
*** ihrachys_ has quit IRC22:09
*** trandles has quit IRC22:09
*** martial has quit IRC22:09
*** yamamoto has joined #openstack-meeting22:12
*** priteau has quit IRC22:12
*** mmalik4 has joined #openstack-meeting22:16
*** thorst_ has joined #openstack-meeting22:20
*** sdake_ is now known as sdake22:23
*** baoli has quit IRC22:24
*** trown is now known as trown|outtypewww22:24
*** baoli has joined #openstack-meeting22:24
*** thorst_ has quit IRC22:25
*** tesseract is now known as tesseract-RH22:26
*** mgoddard has quit IRC22:28
*** baoli has quit IRC22:29
*** ykatabam has quit IRC22:30
*** tesseract-RH is now known as tesseract22:31
*** ijw has quit IRC22:32
*** matrohon_ has quit IRC22:34
*** ykatabam has joined #openstack-meeting22:34
*** alexpilotti has joined #openstack-meeting22:38
*** alexpilo_ has quit IRC22:38
*** powerd has joined #openstack-meeting22:39
*** powerd has quit IRC22:40
*** powerd has joined #openstack-meeting22:41
*** powerd has quit IRC22:42
*** dprince has quit IRC22:43
*** gouthamr has joined #openstack-meeting22:45
*** mriedem has quit IRC22:51
*** eharney has quit IRC22:51
*** Swami has quit IRC22:57
*** gyee has joined #openstack-meeting22:57
*** donghao has joined #openstack-meeting22:59
*** esberglu has quit IRC23:00
*** esberglu has joined #openstack-meeting23:00
*** tesseract has quit IRC23:00
*** tesseract has joined #openstack-meeting23:00
*** tesseract-RH has joined #openstack-meeting23:01
*** esberglu has quit IRC23:01
*** tesseract has quit IRC23:01
*** tesseract-RH has quit IRC23:01
*** goldenfri has quit IRC23:03
*** donghao has quit IRC23:05
*** jamesdenton has quit IRC23:06
*** lpetrut has quit IRC23:07
*** gyee has quit IRC23:11
*** sdague has quit IRC23:12
*** ijw has joined #openstack-meeting23:14
*** ijw has quit IRC23:16
*** esberglu has joined #openstack-meeting23:16
*** esberglu has quit IRC23:18
*** sdake_ has joined #openstack-meeting23:25
*** lamt has quit IRC23:25
*** zhurong has joined #openstack-meeting23:26
*** sdake has quit IRC23:26
*** sdake_ has quit IRC23:27
*** sdake has joined #openstack-meeting23:28
*** zhurong has quit IRC23:28
*** xinli has quit IRC23:28
*** sdake_ has joined #openstack-meeting23:31
*** sdake has quit IRC23:32
*** dimtruck is now known as zz_dimtruck23:33
*** Swami has joined #openstack-meeting23:44
*** Swami has quit IRC23:45
*** baoli has joined #openstack-meeting23:45
*** Julien-zte has joined #openstack-meeting23:45
*** jaugustine has quit IRC23:48
*** unicell has quit IRC23:52
*** Julien-zte has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!