Monday, 2017-10-23

« Sunday, 2017-10-22 Index Tuesday, 2017-10-24 »
*** lukepatrick has joined #openstack-helm00:18
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407700:23
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407700:40
*** lukepatrick has quit IRC01:06
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Add Elasticsearch and Kibana  https://review.openstack.org/51288101:08
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407701:11
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407701:30
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407701:35
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407701:45
SamYaplewin 101:46
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407701:52
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407702:16
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407702:19
SamYaplework work work portdirect. need that OSH loci periodic job!02:38
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407703:06
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407703:34
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407703:55
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407703:56
*** lukepatrick has joined #openstack-helm03:56
*** yamamoto has joined #openstack-helm04:16
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407704:28
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407704:33
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407704:56
*** lukepatrick has quit IRC05:03
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407705:32
*** yamamoto has quit IRC06:33
*** yamamoto has joined #openstack-helm06:52
*** yamamoto_ has joined #openstack-helm07:11
*** yamamoto has quit IRC07:13
*** jistr is now known as jistr|mtgs07:26
*** yamamoto_ has quit IRC07:29
*** yamamoto has joined #openstack-helm07:29
*** yamamoto has quit IRC07:43
*** yamamoto has joined #openstack-helm07:57
*** yamamoto has quit IRC08:03
*** MarkBaker has joined #openstack-helm08:14
*** yamamoto has joined #openstack-helm08:36
*** krypto has joined #openstack-helm08:38
*** krypto has joined #openstack-helm08:38
*** yamamoto has quit IRC08:38
*** yamamoto has joined #openstack-helm08:50
*** yamamoto has quit IRC09:03
*** yamamoto has joined #openstack-helm09:13
*** yamamoto has quit IRC09:58
*** yamamoto has joined #openstack-helm09:59
*** yamamoto has quit IRC10:06
*** yamamoto has joined #openstack-helm10:06
osh-chatbot1<mateuszb> @jayahn That's a great summary. I agree with the approaches but I'd avoid developing a new plugin (that would require additional work to be done in fluentbit). Instead, I rather lean towards using the existing software.10:33
osh-chatbot1<mateuszb> @srwilkers Thanks for pointing out, I didn't take into consideration we can do it the same way cadvisor (or node-exporter) is done. That would resolve the issue I described earlier. And we would indeed need multiline support to avoid double-alarms which may happen if the same multiline log is propagated into different fluentd instances. However, I'd be really interested in your feedback on what @jay-ahn said about generating an10:33
osh-chatbot1alert either at the collecting edge or at log store :slightly_smiling_face:10:33
osh-chatbot1<mateuszb> Additional thing that comes to my mind is the integration capabilities with Nagios. I can see that Nagios is already able to query Prometheus for some basic stuff. Would it be difficult to add new 'checks', for example to see if the number of specified errors in logs increased? I'm adding @alanmeadows to the discussion (Alan was interested in integration with Nagios).10:34
*** yamamoto_ has joined #openstack-helm10:37
*** yamamoto has quit IRC10:40
*** yamamoto_ has quit IRC10:48
*** yamamoto has joined #openstack-helm10:50
*** yamamoto has quit IRC10:55
*** yamamoto has joined #openstack-helm10:55
*** krypto has quit IRC11:03
*** yamamoto_ has joined #openstack-helm11:30
*** yamamoto_ has quit IRC11:31
*** yamamoto has quit IRC11:32
*** yamamoto has joined #openstack-helm11:33
*** yamamoto has quit IRC11:52
*** yamamoto has joined #openstack-helm11:53
*** julim has quit IRC11:56
*** yamamoto has quit IRC11:58
*** felipemonteiro has joined #openstack-helm12:16
*** felipemonteiro_ has quit IRC12:17
*** yamamoto has joined #openstack-helm12:36
*** yamamoto has quit IRC12:46
*** yamamoto has joined #openstack-helm13:02
*** julim has joined #openstack-helm13:14
*** alraddarla has joined #openstack-helm13:32
*** alraddarla has left #openstack-helm13:33
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update values to allow simple parse of images being used  https://review.openstack.org/51432514:02
osh-chatbot1<srwilkers> @mateuszb wrt Nagios, I've got a patchset up for Nagios that includes the Prometheus plugin.  It can trigger alarms based off whatever scalar and vector metrics it can gather from Prometheus.  as long as the log errors exist in prometheus, nagios should be able to gather them.  The service checks are currently defined in values, so can be overriden for organizational needs14:05
osh-chatbot1<srwilkers> it needs some additional work to make it better in my opinion, but it's the best approach I could come up with for now14:06
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433114:11
*** felipemonteiro_ has joined #openstack-helm14:18
*** hachi_ has joined #openstack-helm14:18
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433114:19
*** felipemonteiro__ has joined #openstack-helm14:19
*** gouthamr has joined #openstack-helm14:20
*** felipemonteiro_ has quit IRC14:23
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433114:30
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433114:39
*** hachi_ has quit IRC14:58
*** hachi_ has joined #openstack-helm14:58
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433115:01
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update values to allow simple parse of images being used  https://review.openstack.org/51432515:05
*** lukepatrick has joined #openstack-helm15:10
*** hachi_ has quit IRC15:19
*** alraddarla has joined #openstack-helm15:23
*** alraddarla has left #openstack-helm15:26
*** RuiChen has quit IRC16:05
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: add pull all images to makefile  https://review.openstack.org/51439717:27
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236517:36
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236517:43
openstackgerritMerged openstack/openstack-helm master: Images: Update values to allow simple parse of images being used  https://review.openstack.org/51432517:49
*** mickyt has joined #openstack-helm17:52
osh-chatbot1<luke_> nova doing ok?  ```$ helm install --name=nova ./nova --namespace=openstack \ >   --values=./tools/overrides/mvp/nova.yaml \ >   --set conf.nova.libvirt.nova.conf.virt_type=qemu Error: render error in "nova/templates/deployment-scheduler.yaml": template: nova/templates/deployment-scheduler.yaml:36:62: executing "nova/templates/deployment-scheduler.yaml" at <include "helm-toolki...>: error calling include: template:17:54
osh-chatbot1nova/charts/helm-toolkit/templates/utils/_hash.tpl:22:4: executing "helm-toolkit.utils.hash" at <include $wtf $contex...>: error calling include: template: nova/templates/configmap-etc.yaml:143:3: executing "nova/templates/configmap-etc.yaml" at <include "helm-toolki...>: error calling include: template: nova/charts/helm-toolkit/templates/utils/_to_oslo_conf.tpl:25:7: executing "helm-toolkit.utils.to_oslo_conf" at <eq $value.type "mult...>:17:54
osh-chatbot1error calling eq: invalid type for comparison ```17:54
anticwluke_: this is master?17:57
osh-chatbot1<luke_> yes17:57
osh-chatbot1<v1k0d3n> @luke_ is this with Helm v2.6.2 as well, or v2.6.1? (just checking to be sure).18:02
osh-chatbot1<luke_> 2.6.218:03
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236518:07
openstackgerritPete Birley proposed openstack/openstack-helm master: Images: Update names to reflect service specific functions  https://review.openstack.org/51433118:07
anticwluke_: previous versions work?18:10
osh-chatbot1<luke_> I would have to go back in time, I think I had been using commits from early september. Trying to get back to the latest on master18:12
anticwfair enough, whilst not using gate directly i have verified nova works as of last week18:13
portdirectwhat does your '/tools/overrides/mvp/nova.yaml' look like?18:15
portdirect@luke_ ^^18:15
osh-chatbot1<luke_> no local changes, same as on master18:15
portdirectluke - just confirmed18:18
portdirect2 sec18:18
portdirectwhere are you getting `--set conf.nova.libvirt.nova.conf.virt_type=qemu` from?18:19
osh-chatbot1<luke_> kube-AIO docs18:19
portdirectshould be `--set conf.nova.libvirt.virt_type=qemu`18:19
portdirectcan you update?18:20
osh-chatbot1<luke_> yes18:20
portdirectcheers dude18:20
portdirectwe missed it in this :( https://review.openstack.org/#/c/505386/18:21
osh-chatbot1<luke_> thanks. confirmed nova ok now on my side.18:21
osh-chatbot1<luke_> man.. who +2'd that? ;)18:22
*** pyvpx_ has joined #openstack-helm18:27
*** pyvpx has quit IRC18:29
*** sghosh__ has joined #openstack-helm18:30
*** sghosh_ has quit IRC18:33
*** lukepatrick has quit IRC18:33
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236518:36
*** lukepatrick has joined #openstack-helm18:48
openstackgerritLuke Philips proposed openstack/openstack-helm master: Update AIO docs for nova examples and Docker image  https://review.openstack.org/51443218:52
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236519:09
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236519:12
*** lukepatrick has quit IRC19:28
*** hachi_ has joined #openstack-helm19:33
*** MarkBaker has quit IRC19:42
*** MarkBaker has joined #openstack-helm19:44
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: WIP: Zuul V3 gate  https://review.openstack.org/51407720:01
*** hachi_ has quit IRC20:22
osh-chatbot1<akorzeni> hi all, I'm trying to use ceph as pvc backend but maria_db is not starting due to  PersistentVolumeClaim is not bound: "mysql-data-mariadb-0"20:34
*** lukepatrick has joined #openstack-helm20:34
osh-chatbot1<akorzeni> any ideas where to look? the ceph -s is returning health-ok20:35
osh-chatbot1<akorzeni> ubuntu@node1:~/helm/openstack-helm$ kubectl exec -n ceph ${MON_POD} -- ceph -s     cluster 0f903bb3-0a9d-4ba4-b7d5-ff6cea0609e9      health HEALTH_OK      monmap e1: 1 mons at {node1=172.16.8.5:6789/0}             election epoch 3, quorum 0 node1       fsmap e5: 1/1/1 up {0=mds-ceph-mds-1628000934-z2sch=up:active}      osdmap e15: 1 osds: 1 up, 1 in             flags sortbitwise,require_jewel_osds       pgmap v252: 184 pgs, 1020:35
osh-chatbot1pools, 3656 bytes data, 191 objects             12174 MB used, 67159 MB / 79349 MB avail                  184 active+clean20:35
osh-chatbot1<raymaika> @akorzeni are your hosts correctly resolving `ceph-mon.ceph`?20:43
osh-chatbot1<akorzeni> ubuntu@node1:~/helm$ kubectl describe storageclass/general Name:           general IsDefaultClass: No Annotations:    <none> Provisioner:    ceph.com/rbd Parameters:     adminId=admin,adminSecretName=pvc-ceph-conf-combined-storageclass,adminSecretNamespace=ceph,monitors=ceph-mon.ceph.svc.cluster.local:6789,pool=rbd,userId=admin,userSecretName=pvc-ceph-client-key Events:         <none>20:46
osh-chatbot1<akorzeni> ceph-mon.ceph not, ceph-mon.ceph.svc.cluster.local yes20:47
osh-chatbot1<akorzeni> @raymaika ^20:47
osh-chatbot1<raymaika> I think some requests are sent to just ceph-mon or ceph-mon.ceph. Try adding some search params to your /etc/resolv.conf eg: `search openstack.svc.cluster.local svc.cluster.local cluster.local ceph.svc.cluster.local`20:48
osh-chatbot1<akorzeni> ok, and which pod from ceph should I look for logs?20:53
osh-chatbot1<raymaika> might be able to catch a request in the ceph-mon logs, other than that just have to check status of the pvc in the mariadb pod description. typically when ceph volumes are not provisioned but ceph -s show healthy, it's related to DNS20:56
osh-chatbot1<akorzeni> ok, thanks, will test the possible configs20:57
osh-chatbot1<akorzeni> ubuntu@node1:~/helm$ kubectl describe pvc ceph-test -n ceph Name:           ceph-test Namespace:      ceph StorageClass:   general Status:         Pending Volume: Labels:         <none> Annotations:    volume.beta.kubernetes.io/storage-provisioner=ceph.com/rbd Capacity: Access Modes: Events:   FirstSeen     LastSeen        Count   From                            SubObjectPath   Type            Reason       Message   ---------20:57
osh-chatbot1--------        -----   ----                            -------------   --------        ------       -------   4m            11s             19      persistentvolume-controller                     Normal          ExternalProvisioning  waiting for a volume to be created, either by external provisioner "ceph.com/rbd" or manually created by system administrator20:57
osh-chatbot1<akorzeni> some test ceph-test pvc20:58
openstackgerritDarla Ahlert proposed openstack/openstack-helm master: [WIP] Add Tempest Chart  https://review.openstack.org/49892921:08
*** julim has quit IRC21:09
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: Zuul V3 gate  https://review.openstack.org/51407721:16
*** lukepatrick has quit IRC21:17
*** lukepatrick has joined #openstack-helm21:19
*** lukepatrick has quit IRC21:20
*** lukepatrick has joined #openstack-helm21:22
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: Zuul V3 gate  https://review.openstack.org/51407721:35
osh-chatbot1<luke_> eh, still having a hard time getting nova up ``` $ kubectl logs po/nova-bootstrap-9vt54 -n openstack init  Entrypoint WARNING: 2017/10/23 22:04:42 entrypoint.go:70: Resolving dependency Service nova-api in namespace openstack failed: Service nova-api has no endpoints . ```22:07
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236522:34
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: WIP - Grafana Chart  https://review.openstack.org/51396422:36
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Add Nagios Chart for OpenStack-Helm  https://review.openstack.org/51380622:36
*** julim has joined #openstack-helm22:59
*** felipemonteiro__ has quit IRC23:13
openstackgerritPete Birley proposed openstack/openstack-helm-infra master: Zuul V3 gate  https://review.openstack.org/51407723:14
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Add Elasticsearch and Kibana  https://review.openstack.org/51288123:43
*** lukepatrick has quit IRC23:44
openstackgerritSteve Wilkerson proposed openstack/openstack-helm master: Prometheus Chart  https://review.openstack.org/51236523:45
« Sunday, 2017-10-22 Index Tuesday, 2017-10-24 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!