Tuesday, 2017-12-12

*** polvi has joined #openstack-meeting-500:00
*** tdasilva has joined #openstack-meeting-500:05
*** spzala has joined #openstack-meeting-500:10
*** spzala has quit IRC00:14
*** spzala has joined #openstack-meeting-500:23
*** iyamahat has quit IRC00:23
*** mjturek has quit IRC00:26
*** spzala has quit IRC00:28
*** portdirect has joined #openstack-meeting-500:29
*** sshank has quit IRC00:33
*** spzala has joined #openstack-meeting-500:34
*** spzala has quit IRC00:38
*** spzala has joined #openstack-meeting-500:42
*** spzala has quit IRC00:46
*** spzala has joined #openstack-meeting-500:57
*** spzala has quit IRC01:02
*** spzala has joined #openstack-meeting-501:05
*** spzala has quit IRC01:09
*** yamahata has joined #openstack-meeting-501:10
*** spzala has joined #openstack-meeting-501:10
*** spzala has quit IRC01:20
*** yamahata has quit IRC01:21
*** spzala has joined #openstack-meeting-501:23
*** spzala has quit IRC01:28
*** spzala has joined #openstack-meeting-501:30
*** sshank has joined #openstack-meeting-501:30
*** spzala has quit IRC01:35
*** spzala has joined #openstack-meeting-501:38
*** yamamoto has joined #openstack-meeting-501:41
*** spzala has quit IRC01:42
*** spzala has joined #openstack-meeting-501:45
*** spzala has quit IRC01:50
*** spzala has joined #openstack-meeting-501:52
*** sshank has quit IRC01:56
*** spzala has quit IRC01:56
*** spzala has joined #openstack-meeting-502:04
*** spzala has quit IRC02:09
*** spzala has joined #openstack-meeting-502:18
*** spzala has quit IRC02:22
*** spzala has joined #openstack-meeting-502:31
*** kei-ichi has quit IRC02:31
*** lukepatrick has joined #openstack-meeting-502:32
*** lukepatrick has quit IRC02:32
*** lukepatrick has joined #openstack-meeting-502:33
*** kei-ichi has joined #openstack-meeting-502:33
*** lukepatrick has quit IRC02:34
*** kei-ichi has quit IRC03:05
*** kei-ichi has joined #openstack-meeting-503:07
*** ricolin_ has joined #openstack-meeting-503:43
*** v1k0d3n has quit IRC03:56
*** v1k0d3n has joined #openstack-meeting-503:57
*** markvoelker_ has joined #openstack-meeting-504:12
*** markvoelker has quit IRC04:12
*** spzala has quit IRC05:09
*** spzala has joined #openstack-meeting-505:11
*** spzala has quit IRC05:16
*** spzala has joined #openstack-meeting-505:24
*** iyamahat has joined #openstack-meeting-505:25
*** iyamahat has quit IRC05:26
*** spzala has quit IRC05:29
*** spzala has joined #openstack-meeting-505:30
*** spzala has quit IRC05:34
*** fungi has quit IRC05:45
*** fungi has joined #openstack-meeting-505:47
*** yamahata has joined #openstack-meeting-505:57
*** spzala has joined #openstack-meeting-506:10
*** spzala has quit IRC06:14
*** dkehn has quit IRC06:54
*** dkehn has joined #openstack-meeting-506:54
*** MarkBaker_ has joined #openstack-meeting-507:29
*** MarkBaker has quit IRC07:34
*** MarkBaker_ is now known as MarkBaker07:34
*** MarkBaker has quit IRC08:12
*** MarkBaker has joined #openstack-meeting-508:51
*** MarkBaker has quit IRC08:57
*** MarkBaker has joined #openstack-meeting-509:27
*** MarkBaker_ has joined #openstack-meeting-509:41
*** MarkBaker has quit IRC09:44
*** derekh has joined #openstack-meeting-509:47
*** MarkBaker_ has quit IRC10:09
*** spzala has joined #openstack-meeting-510:11
*** spzala has quit IRC10:15
*** yamamoto has quit IRC10:35
*** yamamoto has joined #openstack-meeting-510:48
*** ricolin_ has quit IRC11:31
*** yamamoto has quit IRC11:54
*** yamamoto has joined #openstack-meeting-511:58
*** yamamoto has quit IRC12:03
*** spzala has joined #openstack-meeting-512:19
*** spzala has quit IRC12:25
*** yamamoto has joined #openstack-meeting-512:35
*** ricolin_ has joined #openstack-meeting-513:17
*** MarkBaker_ has joined #openstack-meeting-513:34
*** MarkBaker_ has quit IRC13:46
*** kiennt26 has joined #openstack-meeting-513:48
*** MarkBaker_ has joined #openstack-meeting-513:50
*** spzala has joined #openstack-meeting-513:57
*** spzala has quit IRC14:01
*** spzala has joined #openstack-meeting-514:02
*** spzala has quit IRC14:06
*** mjturek has joined #openstack-meeting-514:09
*** mab__ has joined #openstack-meeting-514:20
*** yamamoto has quit IRC14:24
*** MarkBaker_ has quit IRC14:24
*** felipemonteiro_ has joined #openstack-meeting-514:24
*** yamamoto has joined #openstack-meeting-514:24
*** felipemonteiro__ has joined #openstack-meeting-514:26
*** felipemonteiro_ has quit IRC14:30
*** mab__ has quit IRC14:43
*** spzala has joined #openstack-meeting-514:47
*** marst has joined #openstack-meeting-514:50
mattmceuen#startmeeting openstack-helm15:00
openstackMeeting started Tue Dec 12 15:00:30 2017 UTC and is due to finish in 60 minutes.  The chair is mattmceuen. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: openstack-helm)"15:00
openstackThe meeting name has been set to 'openstack_helm'15:00
mattmceuen#topic Rollcall15:00
*** raymaika has joined #openstack-meeting-515:00
*** openstack changes topic to "Rollcall (Meeting topic: openstack-helm)"15:00
mattmceuenGM everyone!15:00
raymaikao/15:00
srwilkerso/15:00
srwilkersits morning15:00
srwilkerswont say if its good yet or not15:01
mattmceuenHere's the agenda - I'll give folks a couple mins to fill it out https://etherpad.openstack.org/p/openstack-helm-meeting-2017-12-1215:01
mattmceuensrwilkers:  it's gonna be a great day man15:02
portdirecto/15:02
srwilkersour optimistic captain at the helm would never steer us wrong15:02
mattmceuen#titanic15:02
portdirecttoo soon mattmceuen15:02
portdirecttoo soon15:02
srwilkersoof15:03
srwilkersim more of a hunt for red october kinda guy15:03
mattmceuenalrighty let's get this show on the road15:03
mattmceuen#topic Dependencies on non-default images15:03
*** openstack changes topic to "Dependencies on non-default images (Meeting topic: openstack-helm)"15:03
mattmceuenLet me lay this out for y'all.  I want to accomplish two things here:15:04
mattmceuen1) Come up with a general principle for us OSH engineers to apply15:04
mattmceuen2) Come up with a tactical plan for Hyunsun's PS15:04
mattmceuenFirst the problem statement:  Hyunsun has a PS that has a feature (lbaas plugin), which is turned off by default15:05
mattmceuenThe feature doesn't work with the default neutron kolla 3.2.0 image that we have configured15:05
mattmceuenIt doesn't cause any issues unless you turn the feature on, but if you turn it on, you also have to switch out the image to support the feature15:06
mattmceuenThat's something that either needs to be documented very well (with a reference to an image you can use which supports the feature), or, we should apply a "feature must wait till the default images support it" rule15:06
mattmceuenSo that's the RFC for you all.  I've heard both opinions.15:07
*** cleong has joined #openstack-meeting-515:07
mattmceuenI am personally leaning toward "don't merge until the default image supports"15:07
mattmceuenPerhaps leaving the door open for "... unless there is a really special circumstance we haven't thought of yet"15:08
portdirectAgreed, though we should confirm if the 4.0.0 image works ok with newton15:08
mattmceuenOtherwise we could end up with spaghetti dependencies15:08
mattmceuenyou're skipping ahead portdirect!15:08
portdirectlol - I'll be quite ;)15:09
mattmceuenAny dissenting or reaffirming opinions?15:09
srwilkersim skeptical of having a default image that doesnt work.  as it shouldn't be considered the default at that point15:09
mattmceuenEveryone on xmas vacation already? :-D15:09
srwilkersits nothing but a placeholder then15:10
portdirect++15:10
mattmceuenAlrighty:15:10
mattmceuen#agreed features should not be merged until they are supported by the default images, even if they're turned off by default15:11
mattmceuenNext:  let's get tactical15:11
mattmceuenlbaas is supposed to be supported since kilo, and Hyunsun has or will file a bug with kolla for not supporting15:11
portdirectI suspect that will not yeild joy as newton is eol.15:12
mattmceuenWe could potentially swap in the kolla 4.0.0 image just for the needful image, or swap in a loki image if it supports it out of box15:12
srwilkersin the past, we've had issues getting kolla to provide fixes to images that don't work with the charts we're building15:12
srwilkersplus what portdirect said15:12
portdirectI think the first step would be to see if a 4.0.0 image works with 3.0.3 - if it does great15:12
srwilkersportdirect: ++15:13
mattmceuenAgree - I will pas that on to Hyunsun.  Thanks guys.15:13
portdirecton that ps as well while we are here15:13
*** tlam__ has joined #openstack-meeting-515:13
srwilkerstlam__: late to the party!15:13
portdirectwe will need to test it a bit - as lbaas agent with haproxy used to be prone to leaving zombie processes about15:13
tlam__o/ sorry was running late15:13
portdirectso we may need an init system in that pod to reap them...15:14
mattmceuen(the PS by the way is https://review.openstack.org/#/c/522162/)15:14
* portdirect finished rant15:14
* srwilkers thinks the patchset needs more cowbell15:14
mattmceuenshortest rant I've heard out of you yet portdirect, you've been refining your style15:14
mattmceuenYup Hyunsun affirmed the zombie apocalype this morning15:15
portdirectI just want LBaaS in :D its a great thing for us to use with magnum :)15:15
mattmceuenamen15:15
portdirectKubernetes on Openstack on Kubernetes awaits :D15:15
* srwilkers groans15:15
mattmceuenturtles all the way down15:15
* portdirect dares not abbreviate that.15:15
mattmceuenNext:15:16
mattmceuen#topic Fluentd Chart15:16
*** openstack changes topic to "Fluentd Chart (Meeting topic: openstack-helm)"15:16
mattmceuenTake it away swilkers!15:16
srwilkersthe patchset in question is: https://review.openstack.org/#/c/519548/15:16
srwilkerssungil and jayahn have done a great job at getting this work started, and i feel bad that it's moved destinations twice as we've worked to get osh-infra sorted15:17
srwilkersi think the works almost there, but it might need some tweaking to really shine.  i think the charts need to be separated to appropriately handle rbac for both services without getting too confusing15:18
mattmceuendo we have jayahn?15:18
srwilkersi also think the configuration files need to be defined in values.yaml to allow for customization of the filters and matches for complex use cases15:18
srwilkersi dont think we do :(15:18
portdirecthow come split for rbac?15:19
portdirectwould it not just be a rbac-fluent.yaml, and rbac-fluentbit.yaml ?15:19
srwilkersthe helm-toolkit function names the entrypoints by release15:19
portdirecttotally agree on moving configs to values.15:19
srwilkersso splitting them out in the way you mentioned results in duplicate names15:20
portdirectbut the entrypoint service account would be the same for both15:20
srwilkersokay, thats a misunderstanding on my part then15:21
portdirectthough it does touch on tins rbac work - and how much simpler that will make things - can we add that to parking lot15:21
mattmceuenyup15:22
mattmceuenCan the values file configurability be done in a follow-on PS?15:22
srwilkersit could be.  the prime value add there in my mind is that we could then configure fluentd to capture the logs running in the osh-infra gates15:23
*** kiennt26 has quit IRC15:23
mattmceuenCool - I'm  looking forward to getting the great work to date merged if possible15:24
mattmceuenSo where did you land srwilkers - do you think we need to split the fluentd chart after all?15:24
srwilkersits my opinion that it'd make things cleaner and i dont think the collector and aggregator need to be coupled in the same chart, but thats just my opinion15:26
srwilkersim not entirely stuck on it15:26
*** MarkBaker has joined #openstack-meeting-515:27
mattmceuenWould that be overly difficult to change later if we went down the single-chart path today?15:27
srwilkersnah15:27
MarkBakero/15:28
srwilkershey MarkBaker15:28
mattmceuenawesome - I am in git'er merged mode as the holidays approach :-D15:28
mattmceuenGM MarkBaker!15:28
srwilkerslet me make sure nothing else needs to be cleaned up in that patchset then should be good to go15:28
alanmeadowsEgg Nog in one hand, +2 mouse in the other -- sounds dangerous.15:28
mattmceuenalanmeadows same hand15:28
alanmeadowsnice15:29
portdirectwhy does `merica not understand the benefits of mulled wine?15:29
mattmceuensounds like a cultural learning opportunity15:29
mattmceuensrwilkers you keep the talking stick15:29
MarkBakeralanmeadows, drinks egg nogs that require 2 hands?15:29
mattmceuen#topic Prometheus 2.015:29
*** openstack changes topic to "Prometheus 2.0 (Meeting topic: openstack-helm)"15:29
mattmceuenalanmeadows is a legit pro at egg nog15:29
alanmeadowsIt comes in a stein15:29
srwilkersso prometheus 2.0 was released a bit ago.  it brought some benefits im happy to see15:30
srwilkersthe storage layer was drastically reworked to improve performance and reduce resource consumption15:30
srwilkersit also changed the rules format from gotpl to yaml, which makes me especially happy15:31
* portdirect does happy dance15:31
srwilkersive got a patchset to change the prometheus chart in osh-infra to use prometheus 2.0 by default15:31
*** mjturek1 has joined #openstack-meeting-515:31
srwilkersthere are a few other items i want to get merged first before looking to merge it, but it works currently15:32
alanmeadowsThat would be fantastic, one primary concern surrounding prometheus up until this point was its resource consumption15:32
srwilkersone of the new storage features added was the ability to snapshot the time series database15:32
srwilkersalanmeadows: yeah, i've had a few instances running at home and it wasnt uncommon for prometheus to fall over after chewing through resources15:32
srwilkersi was curious if there was appetite for including a cron job in the prometheus chart for snapshotting the database at configured intervals15:33
*** mjturek has quit IRC15:33
portdirectsrwilkers: what would the objective of the cron job be? backup?15:35
alanmeadowsBeyond that, we should think about how we might trigger that action as well, and how we might apply the same approach to things like mariadb - preupgrade actions across all of these data warehouses15:35
srwilkersportdirect: yep15:35
mattmceuenso prometheus can have multiple servers replicating the same data in case one goes down.  Would we be using it that way?15:35
srwilkersalanmeadows: also agree15:35
portdirectwe could really do with a `helm fire-hook foo`15:35
portdirectthat operates the same way test does15:36
alanmeadowsyes15:36
alanmeadowsreally the ask would just be make 'test' arbitrary15:36
portdirectshould we look into the feasibility of making a ps for that?15:37
portdirectohh github - s/ps/pr15:37
mattmceuenI like that idea15:37
srwilkersi think that'd make sense15:37
alanmeadowsIt satisfies two outstanding asks, being able to break tests apart into impacting vs non-impacting15:37
alanmeadowsand arbitrary actions like backups/snapshots/reversions/... ?15:37
portdirecti think so15:38
portdirectgive us a new hammer, and we'll find nails...15:38
srwilkers:)15:38
alanmeadowsor just hit things15:38
srwilkersthat too15:38
mattmceuenAny other prom bits you want to cover now srwilkers?  I'm looking fw to 2.015:39
srwilkersbut that concludes my points there15:39
srwilkersnope, thats it for me15:39
mattmceuencool.  portdirect get ready15:39
mattmceuen#topic The Future of Ceph!15:39
*** openstack changes topic to "The Future of Ceph! (Meeting topic: openstack-helm)"15:39
alanmeadowsIs this, the Ceph of Tomorrow, Today?15:39
mattmceuengive us a glimpse of this amazing future, technologist15:40
portdirectits the ceph of the futrure, tomorrow.15:40
* alanmeadows sips some nog.15:40
portdirectat kubecon i had some good chats with the ceph peeps re the various ceph-chart efforts15:40
portdirectand I think (ok hope) that we all have the same desire for there to be one well maintained chart that deploys ceph15:41
portdirectrather than the 3 or so versions I know of today.15:41
mattmceuenvariety is the spice of maintenance15:41
portdirectthe chart used by ceph/ceph-helm is actually a fork of ours, which in turn is a chartified version of seb-hans work15:42
portdirectI put a summary of the steps that we hashed out to get to a single chart in the etherpad15:42
portdirectfor the sake of meeting logging I'll paste them here15:42
portdirectAs ceph goes much further than just OpenStack, it makes sense for this to be hosted either by Ceph, or in K8s/charts15:44
portdirectceph/ceph-helm is based on the osh ceph chart from approx 3 months ago15:44
portdirectWe met with the ceph maintainers (core team) at kubecon and discussed their desires/issues with both of our charts and come up with the following proposals:15:44
portdirect1) Split Keystone endpoint creation out of the ceph chart and into its own thing (that would live in OSH)15:44
portdirect2) Merge the healthchecks from OSH into Ceph-Helm15:44
portdirect3) Merge the luminous support from Ceph-Helm into OSH15:44
portdirect4) Update the loopback device creation scripts from bash to ansible15:44
portdirect5) Combine the disc targetting efforts from both OSH and Ceph-Helm into a single effort that brings the reliability of RH's approach with the OSD by bus-id from OSH15:44
portdirect6) The Ceph-Helm chart will then be moved/mirrored to k8s/charts15:44
portdirect7) At this point, add an OSH gates to experimentally use the Ceph-Helm chart15:44
portdirect8) Once stabilised and we have confidence, depreciate the OSH ceph chart15:44
portdirectthe order is obviously somewhat flexible - but as a general outline how does this seem?15:44
mattmceuendigesting...15:47
alanmeadowsWhat is the destination, for example in #2 -- ceph/ceph-helm or K8s/charts?15:47
portdirectceph/ceph-helm15:47
alanmeadowsis this mismash of combination in various targets before aligning on one target because this spans a large period of time?15:48
portdirectand then once the majority of big changes are done we move to k8s/charts15:48
portdirecti would like us at 7 by eoy15:48
alanmeadowsi.e. #2 does work in ceph-helm, #3 in osh15:48
portdirectand 8 in the first two weeks of next15:48
portdirectyup - I have merge rights in ceph/ceph-helm to faciliate this moving faster15:49
jayahnHi late15:49
mattmceuenhey jayahn!15:49
mattmceuenportdirect:  s/disc/disk/ and I then I like the plan15:50
srwilkershey jayahn15:50
jayahnJust fell a sleep. :)15:50
jayahnWhile waiting for the meetinf15:51
srwilkersjust curious portdirect, as i havent paid much attention to the ceph work.  does the luminous support include enabling the built-in prometheus metrics exporter via ceph-mgr?15:51
srwilkersas that makes the ceph-exporter work something we can drop once that's accomplished i think15:52
portdirectsrwilkers: it does :D15:53
srwilkersnice :)15:53
mattmceuenI think your plan is the plan portdirect, unless there are any other thoughts15:54
alanmeadowsIt gets us to a unified chart the community owns, I'm all good15:54
mattmceuent minus 5 mins15:54
mattmceuenand still agenda items - may have to punt till next week.  alanmeadows, will yours fit in 5?15:55
jayahnunified ceph chart. sounds really good to me15:55
portdirectif we can fit in alanmeadows's topic that would be great15:56
mattmceuen#topic Holistic etcd approach15:56
*** openstack changes topic to "Holistic etcd approach (Meeting topic: openstack-helm)"15:56
mattmceuento quote alanmeadows:     Holistic etcd approach15:57
mattmceuen    Various charts trying to use etcd, can we (and should we) unify an approach, or let etcds sprinkle the cloud?15:57
mattmceuen    e.g. https://review.openstack.org/#/c/525752/15:57
mattmceuen    Rabbit would likely follow in approach at some point15:57
mattmceuen    Calico ....15:57
alanmeadowsI see a few different etcds popping up15:57
alanmeadowsThis seems like we need to tackle this one if nothing else to be cognizant of what we're doing15:58
*** yamahata has quit IRC15:58
portdirectagreed - I'd like us to get a solid etcd chart that we can use15:59
mattmceuenStart with a spec of one etcd chart to rule them all?15:59
alanmeadowsI think so, with a few harder needs in mind15:59
alanmeadowsnot just resiliency but backups, disaster recovery, and so on15:59
mattmceuenLet's let this marinate and continue to discuss next time - we're out of time friends15:59
mattmceuenthanks everyone15:59
mattmceuensee y'all in the #openstack-helm !16:00
mattmceuen#endmeeting16:00
*** openstack changes topic to "OVO patches (Meeting topic: neutron_upgrades)"16:00
openstackMeeting ended Tue Dec 12 16:00:10 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_helm/2017/openstack_helm.2017-12-12-15.00.html16:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_helm/2017/openstack_helm.2017-12-12-15.00.txt16:00
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_helm/2017/openstack_helm.2017-12-12-15.00.log.html16:00
*** raymaika has quit IRC16:00
*** mjturek has joined #openstack-meeting-516:27
*** mjturek1 has quit IRC16:30
*** felipemonteiro__ has quit IRC16:32
*** MarkBaker has quit IRC16:36
*** tlam__ has quit IRC17:08
*** tlam_ has joined #openstack-meeting-517:08
*** MarkBaker has joined #openstack-meeting-517:17
*** yamahata has joined #openstack-meeting-518:18
*** ricolin_ has quit IRC18:21
*** yamahata has quit IRC18:43
*** yamahata has joined #openstack-meeting-518:45
*** yamahata has quit IRC18:51
*** mjturek has quit IRC18:58
*** mjturek1 has joined #openstack-meeting-518:58
*** mjturek1 has quit IRC20:29
*** mjturek has joined #openstack-meeting-520:29
*** tlam__ has joined #openstack-meeting-520:33
*** notmyname has quit IRC20:34
*** tlam_ has quit IRC20:35
*** benj_ has quit IRC20:35
*** amrith has quit IRC20:35
*** vkmc has quit IRC20:36
*** notmyname has joined #openstack-meeting-520:41
*** benj_ has joined #openstack-meeting-520:41
*** amrith has joined #openstack-meeting-520:43
*** vkmc has joined #openstack-meeting-520:45
*** vkmc has quit IRC20:46
*** vkmc has joined #openstack-meeting-520:46
*** mjturek has quit IRC21:35
*** felipemonteiro_ has joined #openstack-meeting-521:46
*** cleong has quit IRC21:59
*** spzala has quit IRC22:48
*** spzala has joined #openstack-meeting-522:48
*** spzala has quit IRC22:53
*** marst has quit IRC23:08
*** tlam__ has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!