| *** iceyao has joined #openstack-helm | 00:51 | |
| *** iceyao_ has joined #openstack-helm | 00:53 | |
| *** iceyao has quit IRC | 00:55 | |
| *** iceyao_ has quit IRC | 00:57 | |
| *** iceyao has joined #openstack-helm | 01:10 | |
| *** julim has quit IRC | 01:36 | |
| *** irenab has quit IRC | 01:56 | |
| *** irenab has joined #openstack-helm | 01:57 | |
| *** gouthamr has quit IRC | 02:07 | |
| *** deep-book-gk has joined #openstack-helm | 02:20 | |
| *** deep-book-gk has left #openstack-helm | 02:21 | |
| openstackgerrit | Renis Makadia proposed openstack/openstack-helm master: WIP: Add Helm Chart for Designate https://review.openstack.org/469981 | 03:15 |
|---|---|---|
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: WIP: Loopback device support https://review.openstack.org/484982 | 03:24 |
| *** renmak__ has quit IRC | 03:48 | |
| *** renmak_ has quit IRC | 03:48 | |
| *** iceyao has quit IRC | 03:55 | |
| *** iceyao has joined #openstack-helm | 04:22 | |
| *** iceyao has quit IRC | 04:27 | |
| *** iceyao has joined #openstack-helm | 04:28 | |
| *** nix-wei has quit IRC | 05:25 | |
| *** renmak_ has joined #openstack-helm | 05:43 | |
| *** renmak__ has joined #openstack-helm | 05:43 | |
| *** korzen has joined #openstack-helm | 05:52 | |
| *** renmak__ has quit IRC | 06:40 | |
| *** renmak_ has quit IRC | 06:40 | |
| *** renmak__ has joined #openstack-helm | 06:42 | |
| *** renmak_ has joined #openstack-helm | 06:42 | |
| *** renmak_ has quit IRC | 07:27 | |
| *** renmak_ has joined #openstack-helm | 07:27 | |
| *** renmak__ has quit IRC | 07:28 | |
| *** renmak__ has joined #openstack-helm | 07:29 | |
| openstackgerrit | Renis Makadia proposed openstack/openstack-helm master: WIP: Add Helm Chart for Designate https://review.openstack.org/469981 | 07:30 |
| openstackgerrit | Renis Makadia proposed openstack/openstack-helm master: WIP: Add Helm Chart for Designate https://review.openstack.org/469981 | 08:08 |
| openstackgerrit | Renis Makadia proposed openstack/openstack-helm master: WIP: Add Helm Chart for Designate https://review.openstack.org/469981 | 08:09 |
| openstackgerrit | Michal Dulko proposed openstack/openstack-helm-addons master: Update ElasticSearch and Kibana to 5.4.2 https://review.openstack.org/487051 | 08:27 |
| *** renmak_ has quit IRC | 09:11 | |
| *** renmak__ has quit IRC | 09:11 | |
| *** jistr is now known as jistr|mtg | 11:12 | |
| *** jistr|mtg is now known as jistr | 11:39 | |
| *** julim has joined #openstack-helm | 11:40 | |
| *** iceyao has quit IRC | 11:55 | |
| *** nix-wei has joined #openstack-helm | 12:39 | |
| *** aric49 has joined #openstack-helm | 12:57 | |
| *** iceyao has joined #openstack-helm | 13:08 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Ingress: add default backend https://review.openstack.org/487544 | 13:09 |
| *** randomhack has joined #openstack-helm | 13:10 | |
| *** julim has quit IRC | 13:16 | |
| *** gouthamr has joined #openstack-helm | 13:31 | |
| *** korzen has quit IRC | 13:34 | |
| *** spsurya has quit IRC | 13:43 | |
| *** julim has joined #openstack-helm | 13:45 | |
| v1k0d3n | hey @srwilkers, is OSH going to have an icon before the PTG (an Albatross was discussed at one point)? | 13:45 |
| osh-chatbot3 | <leeuwenrjj> Hello, 2 newbie question: 1) How do we make sure writes are only going to one of the 3 mariadb nodes? (because of select for update issues with writing to multiple nodes) | 13:46 |
| osh-chatbot3 | <leeuwenrjj> 2) How do you make the ingress rules respond on the fqdn instead of the shortname e.g. ingress responds to keystone or keystone.openstack but no other hostnames (or how are people making it work with shortnames only) | 13:46 |
| *** julim_ has joined #openstack-helm | 13:49 | |
| *** marst_ has quit IRC | 13:49 | |
| *** julim has quit IRC | 13:50 | |
| osh-chatbot3 | <leeuwenrjj> e.g. it looks like it needs some modifications to the ingress template to support that but I could miss something obvious | 13:50 |
| *** marst has joined #openstack-helm | 14:03 | |
| *** jaypipes has joined #openstack-helm | 14:03 | |
| *** openstackgerrit has quit IRC | 14:33 | |
| osh-chatbot3 | <portdirect> @leeuwenrjj curently to support fqdn resolution of names via the ingress controllerwould require some modification | 14:39 |
| *** spsurya has joined #openstack-helm | 14:39 | |
| osh-chatbot3 | <portdirect> I have a ps locally that I'm hoping to get in today that does that - along with optionally creating dummy interfaces and ips for services. | 14:40 |
| osh-chatbot3 | <portdirect> re the mariadb - issue - this is being worked on by @srwilkers atm - the current implementation is far from ideal | 14:41 |
| osh-chatbot3 | <portdirect> we should be using prozysql to improve this - though if you are running high volumes of queries today I'd recommend modifying the service to attach to a single pod in the galera cluster as a short term fix | 14:42 |
| *** lrensing has joined #openstack-helm | 14:59 | |
| *** alraddarla has joined #openstack-helm | 15:02 | |
| *** openstackgerrit has joined #openstack-helm | 15:03 | |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 15:03 |
| *** randomhack has quit IRC | 15:05 | |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 15:08 |
| *** randomhack has joined #openstack-helm | 15:09 | |
| *** iceyao has quit IRC | 15:13 | |
| *** cloudnull has quit IRC | 15:14 | |
| *** cloudnull has joined #openstack-helm | 15:15 | |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm master: WIP -- Add proxysql to mariadb https://review.openstack.org/487588 | 15:22 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Gate: Loopback device support https://review.openstack.org/484982 | 15:24 |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Gate: Loopback device support https://review.openstack.org/484982 | 15:25 |
| *** renmak_ has joined #openstack-helm | 15:34 | |
| *** renmak__ has joined #openstack-helm | 15:34 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Gate: Loopback device support https://review.openstack.org/484982 | 15:34 |
| *** iceyao has joined #openstack-helm | 15:39 | |
| jayahn | hi | 15:40 |
| lrensing | o/ | 15:40 |
| portdirect | sup jayahn | 15:40 |
| jayahn | does anyone know galera issue? if all the nodes in the clustr is gone, then you cannot automatically restore the cluster. | 15:41 |
| portdirect | yes :( | 15:41 |
| jayahn | it is not container-specific one. we had the same problem before | 15:41 |
| jayahn | any plan? do we need to write a galera-operator? :) | 15:42 |
| portdirect | getting galera up after a total outage can be fun (meaning no fun at all) | 15:42 |
| jayahn | i know. | 15:42 |
| portdirect | its possible - though I think we should be able to distribute much of the logic out to the pods themselves | 15:43 |
| jayahn | probably, long term, we should develop a kubernetes operator for this. | 15:43 |
| *** iceyao has quit IRC | 15:43 | |
| portdirect | SamYaple you have any input here? (I'm not opposed to an operator - though would want to keep it as 'light' as possible) | 15:44 |
| jayahn | probably a extra discussion on PTG. :) | 15:45 |
| SamYaple | reading scrollback | 15:45 |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 15:46 |
| jayahn | anyway, do you guys have a hands-on workshop proposal for the summit this time as well? | 15:46 |
| SamYaple | jayahn: so that is actually not true. there are 7 different failure conditions for a galera cluster, and all can be automatically recovered from *if* there is no dataloss | 15:47 |
| jayahn | SamYaple: hi :) | 15:47 |
| SamYaple | unfortunately, we would need an operator of some sort to make that work | 15:47 |
| SamYaple | jayahn: hi | 15:47 |
| SamYaple | it doesnt have to be an operator.... but it would need to be extranal api or similiar | 15:47 |
| jayahn | yeap. some external one | 15:47 |
| SamYaple | basically the node coming up needs a way to question the other nodes, whether they are up or not | 15:48 |
| SamYaple | that said, we can do some single node trickery to recover from like 80% of outages | 15:48 |
| SamYaple | basically if gvwstate.dat file exists, start up the container and everything iwll be fine | 15:48 |
| jayahn | hm.. okay. | 15:49 |
| SamYaple | i have a bunch of stuff for this, but didnt have time to keep up with kubernetes | 15:49 |
| jayahn | is there any good reference documents on "there are 7 different failure conditions for a galera cluster, and all can be automatically recovered"? | 15:49 |
| portdirect | SamYaple: a copy of your working notes would be greatly appreated | 15:49 |
| SamYaple | i can detail the steps needed to recover for discussion at the PTG | 15:49 |
| jayahn | so that we can put our own effort on this for a bit. | 15:49 |
| jayahn | okay, that would be great. | 15:50 |
| SamYaple | jayahn: probably not. to figure out how glaera worked i had to swim in the source code | 15:50 |
| SamYaple | documentation is poor and bad | 15:50 |
| jayahn | just one thing, I need to attend that PTG. ;) | 15:50 |
| SamYaple | haha | 15:50 |
| openstackgerrit | Huamin Chen proposed openstack/openstack-helm master: fix BUG 1707670: randomize cleaner job https://review.openstack.org/489261 | 15:50 |
| openstack | bug 1707670 in openstack-helm "failed to clean ceph package: Error: jobs.batch "ceph-namespace-client-key-cleaner" already exists" [Undecided,New] https://launchpad.net/bugs/1707670 | 15:50 |
| jayahn | yeap.. got it. I like swimming. | 15:50 |
| jayahn | at least, one of our devs will be there to talk with you. :) | 15:51 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm master: WIP -- Add proxysql to mariadb https://review.openstack.org/487588 | 15:52 |
| SamYaple | jayahn: it should intrest you to note that better automatic recovery is on the galera roadmap and is actively being worked on | 15:53 |
| jayahn | SamYaple: I like "actively" part | 15:54 |
| jayahn | It is one of our ops main requests; if you say your way is so improved one, pls improve galera auto recovery once for all. :) | 15:55 |
| SamYaple | jayahn: i refuse! | 15:55 |
| SamYaple | for a very good reason though | 15:56 |
| SamYaple | this isn't possible without outside orchestration | 15:56 |
| SamYaple | so either we bake that into galera by adding in etcd or something, or galera doesnt have all teh auto-recovery stuff | 15:56 |
| SamYaple | so right now, if your cluster suffers from a 100% full power outage, you can just `service mysql start` all the nodes and the cluster will reform. magic! | 15:57 |
| *** schwicht has joined #openstack-helm | 15:57 | |
| SamYaple | but if even one of those nodes that was in the previously existing cluster stays down, the cluster wont reform | 15:57 |
| SamYaple | this is all outside info that would be very hard to let non-orchestration tools know about in my opinion | 15:57 |
| jayahn | yeap, ture. so we know this is not a problem we can solve with openstack-helm. :) but, our homework to come up with better way working with ops. | 15:59 |
| SamYaple | jayahn: not true! we can solve it with helm via k8s | 16:00 |
| SamYaple | at least mostly | 16:00 |
| SamYaple | if you have dataloss, then no we cant solve it | 16:00 |
| SamYaple | i need a mechanism for one node to query the other nodes in the cluster. i need to know if grastate.dat exists (and if it does, what is the content), if gvwstate.dat exists, if mysql is currently running, if /var/lib/mysql/ has any data in it, i need this per node. I need to know that on each startup per node | 16:02 |
| SamYaple | so i dont know how to coordinate fetching that info | 16:02 |
| openstackgerrit | Steve Wilkerson proposed openstack/openstack-helm master: WIP -- Add proxysql to mariadb https://review.openstack.org/487588 | 16:03 |
| jayahn | SamYaple: thanks for the comments; we now have a better starting point to think about this issue. | 16:05 |
| jayahn | we... (i mean my team) | 16:06 |
| jayahn | pordirect: SamYaple: thanks for the chat. it is time for me to sleep here in Korea. see you at weekly irc meeting. | 16:12 |
| *** randomhack has quit IRC | 16:19 | |
| *** schwicht has quit IRC | 16:25 | |
| *** aric49 has quit IRC | 16:32 | |
| osh-chatbot3 | <leeuwenrjj> @portdirect thx for the responses on the fqdn / galera questions. I will eagerly await the changes for the fqdn support :slightly_smiling_face: | 17:01 |
| *** randomhack has joined #openstack-helm | 17:14 | |
| *** randomhack has quit IRC | 17:15 | |
| *** randomhack has joined #openstack-helm | 17:19 | |
| *** aric49 has joined #openstack-helm | 17:30 | |
| openstackgerrit | Huamin Chen proposed openstack/openstack-helm master: Closes-Bug: 1707670: randomize cleaner job https://review.openstack.org/489261 | 17:44 |
| openstack | bug 1707670 in openstack-helm "failed to clean ceph package: Error: jobs.batch "ceph-namespace-client-key-cleaner" already exists" [Undecided,Confirmed] https://launchpad.net/bugs/1707670 | 17:44 |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 18:03 |
| *** aric49 has quit IRC | 18:30 | |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 19:02 |
| openstackgerrit | Larry Rensing proposed openstack/openstack-helm master: WIP - Panko chart https://review.openstack.org/469180 | 19:15 |
| openstackgerrit | Larry Rensing proposed openstack/openstack-helm master: WIP - Panko chart https://review.openstack.org/469180 | 19:15 |
| openstackgerrit | Tim Heyer proposed openstack/openstack-helm master: Armada OpenStack deployment yaml https://review.openstack.org/481234 | 19:24 |
| *** aric49 has joined #openstack-helm | 19:25 | |
| openstackgerrit | Darla Ahlert proposed openstack/openstack-helm-addons master: [WIP] Add influxDB Chart https://review.openstack.org/481703 | 19:26 |
| *** julim has joined #openstack-helm | 19:49 | |
| *** julim_ has quit IRC | 19:51 | |
| *** renmak__ has quit IRC | 20:00 | |
| *** renmak_ has quit IRC | 20:00 | |
| openstackgerrit | Larry Rensing proposed openstack/openstack-helm-addons master: Gnocchi chart https://review.openstack.org/472348 | 20:10 |
| openstackgerrit | Larry Rensing proposed openstack/openstack-helm master: WIP - Panko chart https://review.openstack.org/469180 | 20:11 |
| openstackgerrit | Darla Ahlert proposed openstack/openstack-helm-addons master: Add influxDB Chart https://review.openstack.org/481703 | 20:13 |
| *** randomhack has quit IRC | 20:19 | |
| *** renmak__ has joined #openstack-helm | 20:24 | |
| *** renmak_ has joined #openstack-helm | 20:24 | |
| *** spsurya has quit IRC | 20:26 | |
| *** Stacey has joined #openstack-helm | 20:28 | |
| *** julim has quit IRC | 20:45 | |
| *** felipemonteiro has joined #openstack-helm | 21:02 | |
| *** felipemonteiro_ has joined #openstack-helm | 21:09 | |
| *** alraddarla has quit IRC | 21:18 | |
| *** lrensing has quit IRC | 21:23 | |
| *** renmak__ has quit IRC | 21:31 | |
| *** renmak_ has quit IRC | 21:31 | |
| *** felipemonteiro_ has quit IRC | 21:38 | |
| *** renmak_ has joined #openstack-helm | 21:39 | |
| *** renmak__ has joined #openstack-helm | 21:39 | |
| *** aric49 has quit IRC | 21:40 | |
| *** renmak_ has quit IRC | 21:40 | |
| *** renmak_ has joined #openstack-helm | 21:40 | |
| *** renmak__ has joined #openstack-helm | 21:40 | |
| *** renmak_ has quit IRC | 21:42 | |
| *** renmak__ has quit IRC | 21:44 | |
| *** renmak_ has joined #openstack-helm | 21:46 | |
| *** 7GHABNWIQ has joined #openstack-helm | 21:46 | |
| *** schwicht has joined #openstack-helm | 22:04 | |
| openstackgerrit | Pete Birley proposed openstack/openstack-helm master: Gate: Loopback device support https://review.openstack.org/484982 | 22:12 |
| *** schwicht has quit IRC | 22:24 | |
| *** gouthamr has quit IRC | 22:46 | |
| *** gouthamr has joined #openstack-helm | 23:15 | |
| *** randomhack has joined #openstack-helm | 23:16 | |
| *** randomhack has quit IRC | 23:20 | |
| *** marst has quit IRC | 23:26 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!