*** zhiwei has joined #openstack-kolla | 00:04 | |
kfox1111 | wirehead_: those few patches you pointed me at got horizon to a point of being accessable via a browser. :) | 00:05 |
---|---|---|
wirehead_ | awesome | 00:06 |
kfox1111 | Thanks. :) | 00:06 |
wirehead_ | Not a prob. Glad to see that I’m not making something work that nobody wants. :) | 00:06 |
kfox1111 | heh. far from it. :) | 00:07 |
kfox1111 | I think this may go a long way to making openstack much more tractable. | 00:07 |
kfox1111 | I've managed many openstack clouds over the years, and have dozens of upgrades under my belt, and its always been painful. :/ | 00:08 |
kfox1111 | This is looking much much less so. :) | 00:08 |
kfox1111 | thanks for working on it. :) | 00:08 |
*** zhiwei has quit IRC | 00:09 | |
kfox1111 | wirehead_: one other thing I was thinking about... for rolling upgrades across major horizon version, since it dynamically loads content, | 00:13 |
kfox1111 | we may need a way to segment old from new horizons. | 00:14 |
wirehead_ | Hm. | 00:14 |
kfox1111 | maybe a haproxy cookie or something, and maybe seperate token databases for version N and Version n+1. | 00:14 |
kfox1111 | with the seperate token db, they would get a logout when they finally switch. | 00:15 |
kfox1111 | you could maybe use haproxy to tag new connections with a cookie, and then force them to the horizon pool of that version, | 00:16 |
kfox1111 | update it to point to the new one for 24 hours or something, until everyone's off the old pool, | 00:16 |
kfox1111 | then just remove the old rc/deployment. | 00:17 |
wirehead_ | So, you just want to keep the old Horizon around, not whole parallel stacks, right? | 00:17 |
kfox1111 | right. | 00:17 |
kfox1111 | since the horizon kind of sticks around in part in the users browser. | 00:17 |
wirehead_ | Yeah. | 00:17 |
kfox1111 | if you did a rolling upgrade from n -> n+1, users might get multiple versions loaded into the browser at the same time, breaking things. | 00:18 |
kfox1111 | so have a way to when going to n+1, keep n around long enough for logins to expire so they can be drained out. | 00:18 |
kfox1111 | kind of like what we were talking about with the api services, but I think only horizon would have the issue. | 00:19 |
wirehead_ | I think you’d have to do some hacking to create a second service and a proxy. | 00:19 |
wirehead_ | You can create the service with session affinity off of the client’s IP, but that doesn’t really keep track of that part. | 00:19 |
wirehead_ | Oh, and trivia point. If you want to create a Kube service that maps to a non-kube set of servers, here’s how: http://kubernetes.io/docs/user-guide/services/#services-without-selectors | 00:20 |
kfox1111 | I think something like this would work: https://docs.oracle.com/cd/E37670_01/E41138/html/section_jyh_zhz_4r.html | 00:20 |
kfox1111 | but use the new and old service ip's as the backends? | 00:20 |
wirehead_ | Yeah. | 00:20 |
kfox1111 | interesting. | 00:21 |
wirehead_ | So, the Kube people are working on adding session persistance, probably mirroring what haproxy can do because one of the options for Ingress is using haproxy, in 1.3 and beyond. | 00:23 |
kfox1111 | yeah. but from the look of it, its a ways out yet. | 00:23 |
wirehead_ | yeah | 00:23 |
wirehead_ | Then again 1.3 is almost out. | 00:23 |
kfox1111 | plus on the roadmap for it, I didn't see this sort of persistance, I think. | 00:23 |
kfox1111 | yeah. I'm running something very close to it now. :) | 00:23 |
kfox1111 | has some awesome stuff in it. :) | 00:24 |
kfox1111 | subpath mounts have been invaluable. :) | 00:24 |
*** salv-orlando has joined #openstack-kolla | 00:25 | |
kfox1111 | I'm still trying to figure out exactly how scaling is going to work without some kind of external load balancer. | 00:27 |
wirehead_ | Well, I’m behind on my blueprint there. | 00:28 |
kfox1111 | for now, I'm just using a single instance of a haproxy labeled to a particular node, and port.hostPort something specific. | 00:28 |
kfox1111 | it then exposes it on the right ip/port. not fully ha, but works ok for now. | 00:28 |
wirehead_ | So, we’ve got ingress resources. | 00:29 |
kfox1111 | yeah, but it seems to fire up an haproxy on one of the nodes at random as far as I can tell. | 00:29 |
kfox1111 | so you gota go update dns if it moves. | 00:29 |
kfox1111 | and for us, that's a painful experience. :/ | 00:29 |
kfox1111 | different teams.... one of which has IT in its name. | 00:30 |
wirehead_ | Ew. | 00:30 |
kfox1111 | so being able to stick with an ip as long as possible has huge benifits. :/ | 00:30 |
wirehead_ | Yeah. | 00:30 |
kfox1111 | shortish term, I'm thinking, keepalived to have the ip on one of a pair of nodes for ha, and then have the keepalive script relable the nodes with a haproxy lablel, and delete the existing one, or something like that. | 00:31 |
kfox1111 | a little sketchy, but I think it would work. | 00:31 |
*** salv-orlando has quit IRC | 00:32 | |
kfox1111 | if they get requiredDuringSchedulingRequiredDuringExecution implemented, the last step wouldn't be needed I think. | 00:32 |
wirehead_ | Or just set up a haproxy with a known config on a set of servers running kube-proxy. | 00:33 |
kfox1111 | yeah. that would work too. | 00:33 |
kfox1111 | just trying to containerize all I can at this point. :) | 00:33 |
wirehead_ | Yeah. | 00:33 |
kfox1111 | the other option might be pacemaker. they have a docker run plugin now. works pretty well. so you can have the ip and a haproxy container move together. | 00:34 |
kfox1111 | but I'd prefer to use k8s entirely if I can. multiple systems is a pain too. :/ | 00:34 |
wirehead_ | Yeah, or one of the other foofy SDN products for K8s. | 00:34 |
kfox1111 | yeah. I'd like to use kuryr too, but its k8s support is nonexistant right now. | 00:35 |
wirehead_ | Yeah. Hm. So that is a good thing to keep in mind - whatever we do about external LB’s, we need to be careful about IT-operated DNS. :) | 00:36 |
kfox1111 | well, gota head out. thanks for the help, and the chat. :) | 00:36 |
sdake_ | hey kfox1111 | 00:36 |
kfox1111 | yeah. for sure. :/ | 00:36 |
kfox1111 | sdake_: hey. :) | 00:37 |
wirehead_ | Not a problem. Thanks for some victories to point to. :) | 00:37 |
*** daneyon has joined #openstack-kolla | 00:38 | |
sdake_ | 118F today | 00:41 |
sdake_ | 119F yesterday | 00:41 |
sdake_ | my ac sensor says 114f | 00:42 |
*** sdake_ has quit IRC | 00:44 | |
*** dwalsh has joined #openstack-kolla | 00:46 | |
*** huikang has joined #openstack-kolla | 00:52 | |
*** huikang has quit IRC | 00:52 | |
Mech422 | wirehead_: I think 'be careful about IT-operated' basically sums up Kolla/K8S | 00:56 |
*** daneyon_ has joined #openstack-kolla | 00:56 | |
Mech422 | wirehead_: we can't really assume we 'own' anything..dhcp, vlans, bridges, disks, whateer | 00:56 |
Mech422 | err..whatever | 00:57 |
*** daneyon has quit IRC | 00:57 | |
Mech422 | wirehead_: as existing infrastructures may have dibs on it | 00:57 |
*** huikang has joined #openstack-kolla | 00:59 | |
*** daneyon has joined #openstack-kolla | 01:00 | |
*** phuongnh has joined #openstack-kolla | 01:00 | |
*** daneyon__ has joined #openstack-kolla | 01:01 | |
*** daneyon_ has quit IRC | 01:01 | |
*** daneyon has quit IRC | 01:05 | |
*** jeh has quit IRC | 01:06 | |
*** jeh has joined #openstack-kolla | 01:09 | |
*** jeh has joined #openstack-kolla | 01:09 | |
*** jtriley has quit IRC | 01:10 | |
*** jtriley has joined #openstack-kolla | 01:14 | |
*** zhiwei has joined #openstack-kolla | 01:25 | |
*** daneyon has joined #openstack-kolla | 01:25 | |
*** dwalsh has quit IRC | 01:26 | |
*** daneyon_ has joined #openstack-kolla | 01:26 | |
*** daneyon__ has quit IRC | 01:27 | |
*** jtriley has quit IRC | 01:29 | |
*** daneyon has quit IRC | 01:30 | |
*** jtriley has joined #openstack-kolla | 01:31 | |
*** asalkeld has joined #openstack-kolla | 01:34 | |
*** salv-orlando has joined #openstack-kolla | 01:37 | |
*** dwalsh has joined #openstack-kolla | 01:41 | |
*** salv-orlando has quit IRC | 01:44 | |
*** jtriley has quit IRC | 01:48 | |
*** dwalsh has quit IRC | 01:49 | |
*** daneyon_ has quit IRC | 01:53 | |
*** asalkeld has quit IRC | 01:58 | |
*** sacharya has joined #openstack-kolla | 01:59 | |
*** sacharya has quit IRC | 01:59 | |
*** sacharya has joined #openstack-kolla | 02:00 | |
*** daneyon has joined #openstack-kolla | 02:06 | |
*** fragatina has quit IRC | 02:14 | |
*** daneyon_ has joined #openstack-kolla | 02:15 | |
*** fragatina has joined #openstack-kolla | 02:15 | |
*** daneyon has quit IRC | 02:18 | |
*** jianqing has joined #openstack-kolla | 02:19 | |
*** daneyon_ has quit IRC | 02:23 | |
*** daneyon has joined #openstack-kolla | 02:23 | |
*** daneyon_ has joined #openstack-kolla | 02:24 | |
*** rhallisey has quit IRC | 02:25 | |
*** daneyon has quit IRC | 02:28 | |
*** caowei has joined #openstack-kolla | 02:29 | |
*** daneyon_ has quit IRC | 02:33 | |
*** jtriley has joined #openstack-kolla | 02:46 | |
*** klint has joined #openstack-kolla | 02:47 | |
*** yuanying has quit IRC | 02:51 | |
*** yuanying has joined #openstack-kolla | 02:54 | |
*** caowei has quit IRC | 03:04 | |
*** caowei has joined #openstack-kolla | 03:04 | |
*** ssurana has quit IRC | 03:05 | |
*** jianqing has quit IRC | 03:07 | |
*** jtriley has quit IRC | 03:08 | |
*** salv-orlando has joined #openstack-kolla | 03:08 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add customizations for apt keys and packages https://review.openstack.org/326688 | 03:14 |
*** salv-orlando has quit IRC | 03:14 | |
*** dave-mccowan has quit IRC | 03:32 | |
*** dave-mccowan has joined #openstack-kolla | 03:32 | |
*** dave-mccowan has quit IRC | 03:36 | |
*** daneyon has joined #openstack-kolla | 03:38 | |
*** tfukushima has joined #openstack-kolla | 03:39 | |
*** daneyon has quit IRC | 03:43 | |
*** huikang has quit IRC | 03:44 | |
*** caowei has quit IRC | 03:54 | |
*** yuanying has quit IRC | 04:02 | |
*** huikang has joined #openstack-kolla | 04:02 | |
*** yuanying has joined #openstack-kolla | 04:03 | |
*** salv-orlando has joined #openstack-kolla | 04:22 | |
openstackgerrit | Merged openstack/kolla: Add customizations for apt keys and packages https://review.openstack.org/326688 | 04:22 |
*** salv-orlando has quit IRC | 04:29 | |
*** coolsvap has joined #openstack-kolla | 04:32 | |
*** Jeffrey4l_ has quit IRC | 04:33 | |
*** fragatina has quit IRC | 04:47 | |
*** fragatina has joined #openstack-kolla | 04:48 | |
*** fragatina has quit IRC | 04:50 | |
*** fragatina has joined #openstack-kolla | 04:51 | |
*** Jeffrey4l_ has joined #openstack-kolla | 04:55 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Use MAC address for looking for DHCP leases for Vagrant https://review.openstack.org/306284 | 04:58 |
*** daneyon has joined #openstack-kolla | 05:01 | |
*** daneyon_ has joined #openstack-kolla | 05:02 | |
*** jtriley has joined #openstack-kolla | 05:05 | |
*** daneyon has quit IRC | 05:06 | |
*** JianqingJiang has joined #openstack-kolla | 05:11 | |
*** jtriley has quit IRC | 05:11 | |
*** tfukushima has quit IRC | 05:14 | |
*** salv-orlando has joined #openstack-kolla | 05:15 | |
*** daneyon has joined #openstack-kolla | 05:27 | |
*** daneyon has quit IRC | 05:31 | |
*** caowei has joined #openstack-kolla | 05:32 | |
coolsvap | nihilifer, around? | 05:32 |
nihilifer | coolsvap: yes | 05:32 |
*** huikang has quit IRC | 05:33 | |
coolsvap | nihilifer, got a min just wanted to understand some confusion related to comment in a patch 321802 | 05:33 |
patchbot | coolsvap: https://review.openstack.org/#/c/321802/ - kolla - Add a cloud-init RHEL(ish) bootstrap + run kolla s... | 05:33 |
coolsvap | https://review.openstack.org/#/c/321802/4/tools/bootstrap_kolla.sh | 05:33 |
patchbot | coolsvap: patch 321802 - kolla - Add a cloud-init RHEL(ish) bootstrap + run kolla s... | 05:33 |
coolsvap | why not curl for installing docker-engine | 05:34 |
nihilifer | really? doing "curl something | bash" on production? | 05:34 |
nihilifer | you really want me to explain such an obvious think? | 05:35 |
nihilifer | thing* | 05:35 |
nihilifer | imagine that server become unavailable | 05:35 |
nihilifer | and you get some 404 in pipe for bash | 05:35 |
nihilifer | if the concrete step like adding repo or installing package breaks in your script, then it's much easier to debug | 05:36 |
nihilifer | and unavailability of some server is much lesser pain | 05:36 |
coolsvap | nihilifer, got you but i dont wanted to remove the hard-coded addition of repo file | 05:37 |
coolsvap | as well | 05:38 |
coolsvap | i think i typed it wrong | 05:38 |
coolsvap | i want to remove the hard-coded addition of repo file as well | 05:38 |
*** salv-orl_ has joined #openstack-kolla | 05:39 | |
nihilifer | well, if we can fetch this repo file from somewhere, then i'll be fine with removing that | 05:39 |
coolsvap | alright let me check that | 05:39 |
nihilifer | but if this file isn't available for download anywhere (if i recall correctly, it isn't), then hmm... doing that in script is a lesser evil imo | 05:40 |
nihilifer | of course we can just create this repo file | 05:40 |
nihilifer | and copy it in script | 05:40 |
nihilifer | to get rid of "cat [...] EOF" | 05:40 |
*** salv-orlando has quit IRC | 05:41 | |
nihilifer | brb, will review this patch again today | 05:42 |
coolsvap | nihilifer, sure thanks | 05:42 |
*** huikang has joined #openstack-kolla | 05:46 | |
openstackgerrit | Merged openstack/kolla: Add named volume for nova-libvirt https://review.openstack.org/305471 | 05:50 |
*** huikang has quit IRC | 05:50 | |
*** tfukushima has joined #openstack-kolla | 05:51 | |
*** huikang has joined #openstack-kolla | 05:56 | |
openstackgerrit | Merged openstack/kolla: Makes ml2_conf.ini configuration consistent https://review.openstack.org/329760 | 05:58 |
openstackgerrit | Merged openstack/kolla: Use MAC address for looking for DHCP leases for Vagrant https://review.openstack.org/306284 | 05:58 |
*** ssurana has joined #openstack-kolla | 06:06 | |
*** huikang has quit IRC | 06:11 | |
*** ssurana has quit IRC | 06:14 | |
*** ssurana has joined #openstack-kolla | 06:16 | |
*** Jianqing_ has joined #openstack-kolla | 06:29 | |
*** JianqingJiang has quit IRC | 06:29 | |
*** JianqingJiang has joined #openstack-kolla | 06:31 | |
*** Jianqing_ has quit IRC | 06:31 | |
*** Jianqing_ has joined #openstack-kolla | 06:35 | |
*** JianqingJiang has quit IRC | 06:35 | |
*** JianqingJiang has joined #openstack-kolla | 06:41 | |
*** Jianqing_ has quit IRC | 06:41 | |
*** jmccarthy has joined #openstack-kolla | 06:42 | |
*** Jianqing_ has joined #openstack-kolla | 06:52 | |
*** JianqingJiang has quit IRC | 06:52 | |
*** ssurana has quit IRC | 06:54 | |
*** JianqingJiang has joined #openstack-kolla | 06:55 | |
*** Jianqing_ has quit IRC | 06:55 | |
*** mewald has joined #openstack-kolla | 06:56 | |
*** Jianqing_ has joined #openstack-kolla | 07:00 | |
*** JianqingJiang has quit IRC | 07:00 | |
*** belmoreira has joined #openstack-kolla | 07:04 | |
*** Jianqing_ has quit IRC | 07:05 | |
*** JianqingJiang has joined #openstack-kolla | 07:06 | |
*** jtriley has joined #openstack-kolla | 07:07 | |
*** matrohon has joined #openstack-kolla | 07:09 | |
*** Serlex has joined #openstack-kolla | 07:10 | |
*** daneyon_ has quit IRC | 07:10 | |
*** jtriley has quit IRC | 07:12 | |
*** sacharya has quit IRC | 07:13 | |
*** gfidente|afk has quit IRC | 07:15 | |
*** daneyon has joined #openstack-kolla | 07:15 | |
*** daneyon has quit IRC | 07:20 | |
*** Jianqing_ has joined #openstack-kolla | 07:25 | |
*** JianqingJiang has quit IRC | 07:25 | |
*** athomas has joined #openstack-kolla | 07:35 | |
*** JianqingJiang has joined #openstack-kolla | 07:37 | |
*** Jianqing_ has quit IRC | 07:37 | |
*** Jianqing_ has joined #openstack-kolla | 07:41 | |
*** JianqingJiang has quit IRC | 07:41 | |
*** JianqingJiang has joined #openstack-kolla | 07:41 | |
*** Jianqing_ has quit IRC | 07:41 | |
*** Jianqing_ has joined #openstack-kolla | 07:47 | |
*** JianqingJiang has quit IRC | 07:47 | |
*** JianqingJiang has joined #openstack-kolla | 07:50 | |
*** Jianqing_ has quit IRC | 07:51 | |
*** mewald has quit IRC | 07:51 | |
apuimedo | kfox1111: we have a prototype | 07:52 |
apuimedo | https://github.com/midonet/kuryr/tree/k8s | 07:52 |
*** salv-orl_ has quit IRC | 07:53 | |
*** mewald has joined #openstack-kolla | 07:57 | |
*** shardy has joined #openstack-kolla | 08:05 | |
*** matrohon has quit IRC | 08:12 | |
*** Jianqing_ has joined #openstack-kolla | 08:13 | |
*** JianqingJiang has quit IRC | 08:13 | |
*** sacharya has joined #openstack-kolla | 08:14 | |
*** dmk0202 has joined #openstack-kolla | 08:18 | |
*** mgoddard has joined #openstack-kolla | 08:19 | |
*** sacharya has quit IRC | 08:19 | |
*** salv-orlando has joined #openstack-kolla | 08:19 | |
*** shardy has quit IRC | 08:32 | |
*** JianqingJiang has joined #openstack-kolla | 08:35 | |
*** Jianqing_ has quit IRC | 08:35 | |
*** dmk0202 has quit IRC | 08:42 | |
*** williamcaban has joined #openstack-kolla | 08:49 | |
*** daneyon has joined #openstack-kolla | 09:03 | |
*** jtriley has joined #openstack-kolla | 09:08 | |
*** daneyon has quit IRC | 09:08 | |
*** tfukushima has quit IRC | 09:08 | |
*** jtriley has quit IRC | 09:12 | |
*** matrohon has joined #openstack-kolla | 09:15 | |
*** coolsvap has quit IRC | 09:15 | |
*** coolsvap has joined #openstack-kolla | 09:17 | |
*** openstackgerrit has quit IRC | 09:18 | |
*** openstackgerrit has joined #openstack-kolla | 09:18 | |
*** dwalsh has joined #openstack-kolla | 09:21 | |
*** dwalsh has quit IRC | 09:30 | |
*** caowei has quit IRC | 09:33 | |
openstackgerrit | Merged openstack/kolla-kubernetes: Fix Docker version in quickstart https://review.openstack.org/331337 | 09:35 |
*** sacharya has joined #openstack-kolla | 09:37 | |
*** tfukushima has joined #openstack-kolla | 09:40 | |
*** pbourke_ has quit IRC | 09:41 | |
*** sacharya has quit IRC | 09:41 | |
*** pbourke_ has joined #openstack-kolla | 09:42 | |
*** gfidente has joined #openstack-kolla | 09:51 | |
*** gfidente has joined #openstack-kolla | 09:51 | |
*** JianqingJiang has quit IRC | 10:03 | |
*** JeroenBo has joined #openstack-kolla | 10:10 | |
JeroenBo | Goodday, anyone available to discuss some multinode glance challenges ? | 10:19 |
*** jmccarthy has left #openstack-kolla | 10:19 | |
*** athomas has quit IRC | 10:20 | |
*** athomas has joined #openstack-kolla | 10:21 | |
*** zhiwei has quit IRC | 10:25 | |
mewald | JeroenBo: I am quite new to Kolla but would be interested to hear what it's about :D I will wait here for the pros to appear ;) | 10:28 |
*** phuongnh has quit IRC | 10:30 | |
JeroenBo | When you deploy glance on more than one container (with local storage), they share the same database.. But the image is only stored on one of the api hosts and the metadata is available on all glance api's | 10:33 |
JeroenBo | if the request is made to the api where only the metadata is available, it will (as expected) result in an error | 10:34 |
*** daneyon has joined #openstack-kolla | 10:52 | |
*** daneyon has quit IRC | 10:56 | |
mewald | JeroenBo: But that's the case with and without Kolla. Local storage is meant for single instance deployments only. For multiple instances you will always need shared storage, for example NFS oder a Ceph cluster | 11:01 |
JeroenBo | mewald: that’s why I call it a challenge ;-) We don’t want to use ceph anymore due to major outages we experienced in the past. | 11:06 |
*** jtriley has joined #openstack-kolla | 11:08 | |
*** jtriley has quit IRC | 11:14 | |
*** tfukushima has quit IRC | 11:14 | |
*** ppowell has joined #openstack-kolla | 11:16 | |
*** dwalsh has joined #openstack-kolla | 11:20 | |
*** Mr_Broken has joined #openstack-kolla | 11:28 | |
*** mliima has joined #openstack-kolla | 11:30 | |
*** salv-orl_ has joined #openstack-kolla | 11:39 | |
*** salv-orlando has quit IRC | 11:41 | |
*** dwalsh has quit IRC | 11:44 | |
*** caowei has joined #openstack-kolla | 11:46 | |
*** salv-orl_ has quit IRC | 11:54 | |
*** shardy has joined #openstack-kolla | 12:04 | |
*** dwalsh has joined #openstack-kolla | 12:07 | |
*** dave-mccowan has joined #openstack-kolla | 12:11 | |
*** jtriley has joined #openstack-kolla | 12:14 | |
*** jtriley has quit IRC | 12:19 | |
*** salv-orlando has joined #openstack-kolla | 12:20 | |
Mr_Broken | Hey Guys | 12:21 |
Mr_Broken | can anyone explain how should Glance work with multinode setup and haproxy enabled ? | 12:21 |
Mr_Broken | i mean, if you have set files storage | 12:22 |
Mr_Broken | and try to image-create, it will RANDOMLY pick one of 3 glances and put it on /var/lib/glance/images (default) | 12:22 |
Mr_Broken | then when you try to deploy - 2/3 deployments fail since they look for image on different glance hosts | 12:23 |
Mr_Broken | is that intentional ? | 12:23 |
*** b_bezak has joined #openstack-kolla | 12:23 | |
*** dwalsh has quit IRC | 12:24 | |
*** caowei has quit IRC | 12:30 | |
JeroenBo | Mr_Broken: I have the same issue at the moment | 12:38 |
Mr_Broken | @JeroenBo: I've managed to workaround that a bit, by configuring swift backend for Glance (external) | 12:40 |
*** daneyon has joined #openstack-kolla | 12:40 | |
Mr_Broken | but then I need glance-cache configured and working ... and the same problem ;] | 12:40 |
JeroenBo | i haven’t got a swift deployment | 12:40 |
JeroenBo | but it’s by design i think | 12:40 |
Mr_Broken | well me neither on the local openstack | 12:41 |
*** sbezverk has quit IRC | 12:41 | |
Mr_Broken | but i just connected to some swift backend in the internet | 12:41 |
Mr_Broken | thats why the need for glance-cache .;] | 12:41 |
Mr_Broken | and circle closes | 12:41 |
JeroenBo | maybe it’s possible to rsync /var/lib/docker/volumes/glance/_data/images => other docker host (just for testing purposes) | 12:42 |
Mr_Broken | its stored inside glance_api docker :/ | 12:43 |
JeroenBo | i know, but thats the path from the docker host | 12:44 |
JeroenBo | (do not do this in production ;-) ) | 12:44 |
Mr_Broken | true | 12:44 |
Mr_Broken | yeah ;] | 12:44 |
*** daneyon has quit IRC | 12:45 | |
mewald | JeroenBo: I still don't see how this is a Kolla issue. If you need disk to disk replication, I suggest you take a look at DRBD | 12:47 |
JeroenBo | it’s not a kolla issue, but it’s possible that you run into this issue when configuring glance on multiple nodes with local storage | 12:49 |
Mr_Broken | well, what kolla could do is change HAproxy configs, to make glance active/passive (like mariadb) | 12:52 |
Mr_Broken | because glance in active/active on 3 nodes - dont work as intented | 12:52 |
Mr_Broken | or I dont know how to make it to work consistently :/ | 12:52 |
*** rhallisey has joined #openstack-kolla | 12:52 | |
JeroenBo | Mr_Broken: it think the best option, is to give every glance_api host it’s own database (Nova will poll every glance_api for its images) | 12:53 |
JeroenBo | Downside .. it’s not HA | 12:54 |
rhallisey | with Kubernetes it would be | 12:57 |
*** klint has quit IRC | 12:59 | |
Mr_Broken | Is Kolla/Kubernetes ready to deploy ? :P | 12:59 |
rhallisey | Mr_Broken, nope | 13:00 |
rhallisey | :) | 13:00 |
Mr_Broken | :> | 13:00 |
Mr_Broken | yeah... cant wait for that one to get my hands dirty . | 13:00 |
rhallisey | there also isn't a db per service yet | 13:00 |
rhallisey | but it would be possible | 13:01 |
Mr_Broken | yeah, so how do we glance in multinode and haproxy till those days ? :] | 13:01 |
Mr_Broken | or even if we decide to move to outside glance - how do we manage cache then ? ;] | 13:03 |
JeroenBo | Mr_Broken: Rsync doesn’t work ;-) | 13:03 |
*** caowei has joined #openstack-kolla | 13:03 | |
Mr_Broken | JeroenBo: I imagine ImageID with a pointer to glance server, is stored in DB ? | 13:04 |
Mr_Broken | (not sure here) | 13:04 |
rhallisey | Mr_Broken, good questions | 13:04 |
JeroenBo | Everything is stored in the database | 13:04 |
Mr_Broken | image itself isnt right ? ;] | 13:05 |
JeroenBo | The problem is, the glance api has the metadata but not the data in /var/lib/glance | 13:05 |
*** salv-orlando has quit IRC | 13:07 | |
*** zhiwei has joined #openstack-kolla | 13:10 | |
*** dwalsh has joined #openstack-kolla | 13:21 | |
*** Mr_Broken has quit IRC | 13:23 | |
*** sbezverk has joined #openstack-kolla | 13:23 | |
*** Mr_Broken has joined #openstack-kolla | 13:24 | |
*** inc0 has joined #openstack-kolla | 13:24 | |
*** Mr_Broken has quit IRC | 13:28 | |
*** Mr_Broken has joined #openstack-kolla | 13:31 | |
*** jtriley has joined #openstack-kolla | 13:31 | |
*** Mr_Broken has quit IRC | 13:33 | |
*** Mr_Broken has joined #openstack-kolla | 13:34 | |
*** sdake has joined #openstack-kolla | 13:36 | |
*** Mr_Broken has quit IRC | 13:38 | |
sdake | morning folks | 13:39 |
sdake | just a heads up I was out of the office yesterday for a semi-emergency | 13:39 |
sdake | so if people were trying to find me or need something, please let me know | 13:39 |
*** huikang has joined #openstack-kolla | 13:40 | |
coolsvap | sdake, hi good morning | 13:41 |
sdake | hey coolsvap | 13:42 |
coolsvap | sdake, we need to push the oraclelinux images for mitaka to dockerhub the gates are failing due to their unavailability | 13:42 |
coolsvap | mitaka oraclelinux gates | 13:42 |
sdake | coolsvap the gate should not be using the dockerhub | 13:42 |
sdake | but that said, i agree we should push images for all distros | 13:43 |
coolsvap | yeah but its giving the same error | 13:43 |
sdake | we now have access to "kolla" namespace | 13:43 |
sdake | so I'd like to switch us over to using that | 13:43 |
coolsvap | sdake, yeah i saw the mail | 13:43 |
sdake | nobody asked to be added | 13:43 |
coolsvap | i asked | 13:44 |
sdake | unless I somehow mised the emaill | 13:44 |
coolsvap | :) | 13:44 |
sdake | one of my filters is busted and some of my mail gos to who knows where | 13:44 |
sdake | ok i'll add you moment | 13:44 |
coolsvap | i actually push images to my own repo very frequently | 13:44 |
coolsvap | i can push to kolla as well | 13:44 |
sdake | coolsvap pbourke_ inc0 rhallisey please provide email so I can add yo uto docker hub registry for "kolla" namespace | 13:45 |
sdake | and any other cores that happen to be awake that I missed | 13:45 |
sdake | coolsvap beffore pushing the images, can you sort out why the gate thinks it needs the registry | 13:46 |
sdake | it should be pulling from the docker cache | 13:46 |
*** huikang has quit IRC | 13:46 | |
coolsvap | sdake, yes i will do | 13:46 |
sdake | email is so I dont hae to go hunt for it coolsvap? :) | 13:46 |
sdake | mandre you may be awake too | 13:47 |
sdake | if i didn't list a core reviewer above, its because I think you are aslep - if you see this record pelase ping me and i'll be happy to addd you | 13:47 |
*** openstackgerrit has quit IRC | 13:48 | |
*** openstackgerrit has joined #openstack-kolla | 13:48 | |
coolsvap | sdake, didnt get the above question | 13:48 |
sdake | coolsvap give me your email :) | 13:49 |
*** wmiller has quit IRC | 13:49 | |
coolsvap | oh sorry | 13:49 |
sdake | thanks pbourke_ coolsvap | 13:50 |
sdake | if you dont want to write in channel private message will do | 13:50 |
sdake | remember, new namespace is "kolla" | 13:50 |
sdake | so we need to transition the code to use that instead of kollaglue | 13:50 |
sdake | I can do that, or someone else can | 13:50 |
sdake | any takers | 13:50 |
coolsvap | i have a patch up for review | 13:50 |
sdake | i've got a pretty full plate atm :( | 13:50 |
coolsvap | its marked in progress | 13:51 |
sdake | thanks rhallisey | 13:51 |
coolsvap | https://review.openstack.org/#/c/329289/ | 13:51 |
patchbot | coolsvap: patch 329289 - kolla - Update kollaglue repo on docker to kolla | 13:51 |
sdake | coolsvap cool eviewing moment | 13:51 |
sdake | thanks for the bug id :) | 13:51 |
*** wmiller has joined #openstack-kolla | 14:02 | |
sdake | coolsvap patch looks good not sure why its in wip | 14:02 |
sdake | you want ot test I assume? | 14:02 |
sdake | ot/to | 14:03 |
sdake | coolsvap any results on investigation o using the docker build service | 14:03 |
coolsvap | sdake, in a meeting will check right after it | 14:03 |
sdake | roger | 14:03 |
coolsvap | i removed workflow | 14:03 |
coolsvap | sdake, its working we can see it from the gate logs | 14:04 |
coolsvap | its using kolla instead of kollaglue | 14:05 |
sdake | deploy isn't working | 14:05 |
coolsvap | gate failure is not related to the fix | 14:06 |
coolsvap | http://logs.openstack.org/89/329289/1/check/gate-kolla-dsvm-deploy-centos-binary/d316c87/console.html.gz#_2016-06-14_07_32_08_325 | 14:06 |
*** salv-orlando has joined #openstack-kolla | 14:08 | |
coolsvap | this is a recurring issue that we are getting in gates recently in around 50% of gate runs | 14:08 |
coolsvap | sdake, ^ | 14:08 |
sdake | annoying | 14:08 |
* sdake beats the gate | 14:08 | |
*** thumpba has joined #openstack-kolla | 14:09 | |
sdake | coolsvap needle meet haystack | 14:11 |
*** salv-orlando has quit IRC | 14:14 | |
*** williamcaban has quit IRC | 14:19 | |
sdake | rhallisey did you fix wirehead_ s last comment on the k8s spec | 14:20 |
rhallisey | ya | 14:20 |
sdake | cool so ready for merge? | 14:20 |
rhallisey | ya | 14:20 |
sdake | cool i'll ask the core review team to review it and ack it then | 14:20 |
sdake | thanks! | 14:20 |
rhallisey | it's a solid outline of the plan | 14:20 |
sdake | link? | 14:21 |
rhallisey | will likely change, but it describes the problem well | 14:21 |
rhallisey | https://review.openstack.org/#/c/304182/ | 14:21 |
patchbot | rhallisey: patch 304182 - kolla - Spec: Deploy Kolla images using Kubernetes | 14:21 |
sdake | specs are a point in time plan for us | 14:21 |
sdake | thanks rhallisey | 14:21 |
rhallisey | no problem | 14:21 |
mewald | Is there any way to deploy per domain backends with kolla? | 14:25 |
sdake | pbourke_ mandre inc0 coolsvap please have a look at rhallisey and +2 if there are no glaring errors | 14:26 |
sdake | if there are typos or pronouns or stuff like that, recommend just ignoring | 14:26 |
*** JeroenBo has left #openstack-kolla | 14:28 | |
sdake | domain backend? | 14:28 |
sdake | would you mind expanding mewald | 14:28 |
sdake | my inital answer would be no, becaue I'm not sure what your tlaking about | 14:29 |
sdake | if your talking about multi-domain (as in multi-keystone) no that isnt implemented, but is planned | 14:29 |
sdake | the idea there is we will have one master keystone but permit multiple regions to be edeployed | 14:29 |
mewald | sdake: what you are describing is not "multi-domain". I want to have multiple domains in keystone and each with a different identity backend configuration. This way I can connect to different LDAPs for each keystone domain. | 14:33 |
mewald | It would require to copy several files to /etc/keystone/domains so it's pretty simple to implement | 14:33 |
sdake | mewald right - ya we don't have that atm | 14:34 |
sdake | mewald but we are also working on that | 14:34 |
sdake | via customizations | 14:34 |
sdake | mewald some work has hit the rpeo related to customizations already but we have 130 containers so its going to take awhile | 14:34 |
sdake | 4-6 weeks | 14:34 |
mewald | sdake: who is working on that? I would like to team up with him/her and help | 14:34 |
sdake | mewald i plan to involve the whole community | 14:34 |
sdake | mewald but for the moment its mostly been inc0 prototying | 14:34 |
mewald | ok | 14:35 |
sdake | mewald that said, if you could describe exactly how to implement it, it might help guidecustommiations | 14:36 |
mewald | Actually, all I would need would be in addition to the ini merge function to copy say the contents of /etc/kolla/keystone/domains/* into the container. Thats all. I don't think we need kolla to actually create the domains via API. What's a bit uncomfortable: Whenever a new domains is created, new images would have to be deployed. Dunno if there's a way around that | 14:39 |
sdake | moment processing | 14:39 |
sdake | 5 mins to think about that | 14:40 |
*** huikang has joined #openstack-kolla | 14:42 | |
mewald | sdake: In case your to familiar with this part of keystone: http://docs.openstack.org/developer/keystone/configuration.html Take a look at the "Domain-specific Drivers" section | 14:42 |
*** Mr_Broken has joined #openstack-kolla | 14:42 | |
mewald | *not | 14:42 |
sdake | i am not | 14:42 |
sdake | reading | 14:42 |
sdake | so my initial reactoin to redeploy on a config change is "ugh" | 14:43 |
mewald | exactly :D | 14:43 |
sdake | it almost seems lilke the keystone case requires ssome special magic | 14:43 |
sdake | inc0 thoughts on above^^ | 14:43 |
sdake | mewald the problem with getting merge inis to do the copying is ansible specifies which files to copy | 14:44 |
*** williamcaban has joined #openstack-kolla | 14:44 | |
* inc0 reading log | 14:44 | |
sdake | we could easily just tell it to copy the doains directory | 14:44 |
*** zhiwei has quit IRC | 14:44 | |
sdake | if that is all that is needed are cnofig files | 14:44 |
inc0 | so multi domain' | 14:45 |
inc0 | mewald, we have it | 14:45 |
sdake | mewald see, shows what I know :) | 14:45 |
inc0 | you can specify per-node config | 14:45 |
inc0 | using merge config and config overrides | 14:45 |
sdake | inc0 mewald needs to copy the contents of a subdirectory though | 14:46 |
sdake | inc0 the merge code doesn't do that at present | 14:46 |
mewald | inc0: from what I understand that only works as long as everything does into the keystone.conf | 14:46 |
sdake | alll filenames are hardcoded in ansible | 14:46 |
inc0 | https://github.com/openstack/kolla/blob/master/doc/advanced-configuration.rst#openstack-service-configuration-in-kolla | 14:46 |
*** huikang has quit IRC | 14:46 | |
*** daneyon has joined #openstack-kolla | 14:49 | |
Mr_Broken | inc0: yeah thats a fine functionality, but how do you comment things out from existing configs ? or remove a line ? :] | 14:49 |
inc0 | mewald, yeah we dont add files | 14:49 |
mewald | inc0: so you're saying I could create a file "/etc/kolla/config/keystone/domains/keystone.mydomain.conf" and it would copy that to "/etc/keystone/domains/keystone.mydomain.conf" on every keystone node? | 14:50 |
inc0 | Mr_Broken, you can always override it to default from code | 14:50 |
inc0 | mewald, no, you need sth like that? if so, you need to mess with ansible | 14:50 |
*** daneyon has quit IRC | 14:50 | |
inc0 | not a big mess, but mess nonetheless | 14:50 |
mewald | it's not like I need it, but Keystone needs it :D | 14:50 |
inc0 | I didn't know of this use case | 14:50 |
sdake | inc0 I was thinking since this is a standard keystone thing, we should implement a copy for it | 14:51 |
inc0 | I don't blame you;) | 14:51 |
inc0 | yeah | 14:51 |
inc0 | I agree | 14:51 |
mewald | inc0: take a look at the link I posted for sdake about keystone. They describe how it works :) | 14:51 |
*** jeh has quit IRC | 14:52 | |
inc0 | kk will take a look | 14:52 |
inc0 | so sdake was right after all;) we don't have it | 14:52 |
inc0 | we could use conf.d'ish mechanism for other stuff too | 14:53 |
sdake | mewald does anything else need to go in the container filesystem besides config files? | 14:53 |
sdake | mewald note i have no had a chance ot read yoru link yet - have 6 conversations going on at once :( | 14:53 |
mewald | sdake: no thats it, only the "domains" directory in /etc/keystone and the files that go there | 14:54 |
*** mliima has quit IRC | 14:57 | |
*** Jeffrey4l_ has quit IRC | 15:00 | |
*** athomas has quit IRC | 15:01 | |
Mr_Broken | inc0: the problem is , default value is bad for my deployment, need to remove the line totaly, and add other backend ( cinder and glance configs ) - so a way to remove or comment a line would be a great feature ! ;] | 15:01 |
Mr_Broken | cant show you example now, since someone broke network on the way to my openstack deployment ;] | 15:02 |
inc0 | Mr_Broken, if you remove it, it will be default in code | 15:02 |
mewald | Mr_Broken: inc0 is right, check the configuration reference for what the default value is and set that explicitly | 15:02 |
Mr_Broken | I'll need to demonstrate on example | 15:03 |
Mr_Broken | but in glance config, when you want to change from files to swift backend | 15:04 |
Mr_Broken | and then disable the files completely | 15:04 |
Mr_Broken | you actually need to remove 1 line from original default Kolla created configs | 15:04 |
sdake | Mr_Broken do you have exmple configs | 15:04 |
Mr_Broken | yeah in an hour | 15:04 |
sdake | or link the config in question | 15:04 |
sdake | you said its a default kolla config | 15:04 |
sdake | which file? | 15:05 |
*** b_bezak has quit IRC | 15:05 | |
Mr_Broken | glance-api.conf if i remember correctly | 15:05 |
Mr_Broken | another thing is - it would be super usefull, to have a config file added during reconiguration | 15:05 |
*** b_bezak has joined #openstack-kolla | 15:05 | |
Mr_Broken | for example , i would like to create glance-swift.conf on all 3 controllers | 15:05 |
inc0 | Mr_Broken, so I was thinking of this case | 15:06 |
sdake | Mr_Broken https://github.com/openstack/kolla/blob/master/ansible/roles/glance/templates/glance-api.conf.j2 | 15:06 |
inc0 | to decouple config generation from actual deployment | 15:06 |
sdake | Mr_Broken can you tell me which line # is the problem | 15:06 |
inc0 | so you can generate configs and then edit them | 15:06 |
Mr_Broken | default_store = file | 15:06 |
Mr_Broken | filesystem_store_datadir = /var/lib/glance/images/ | 15:06 |
sdake | inc0 i think we hae neough stuff to deal with atm | 15:07 |
sdake | inc0 can we table that for next cycle :) | 15:07 |
Mr_Broken | @sdake: i want to change default_store (and i can) and i want to remove completely or comment out filesystem_store_datadir | 15:07 |
inc0 | well, this is important | 15:07 |
Mr_Broken | inc0: that would be awsome ! :] | 15:07 |
sdake | Mr_Broken filesystem_store_datadir has some default value in openstack | 15:07 |
inc0 | so if somebody would like to take on it, I'd love to provide some thoughts I already have on that matter | 15:07 |
inc0 | I'm not sure if I'll have bandwidth to pursue it | 15:08 |
sdake | Mr_Broken if you find it and set it as mewald suggested | 15:08 |
sdake | that should get you going | 15:08 |
Mr_Broken | sdake: i like my config files tidy, and i want to handover to other teams in future | 15:08 |
inc0 | so...that's a call for help guys, who wants to be a contributor? | 15:08 |
Mr_Broken | having meaningless lines creates confusion | 15:08 |
sdake | Mr_Broken i hear what your saying | 15:08 |
Mr_Broken | but thats a secondary feature , can deal with that | 15:08 |
sdake | we keep the files super minimal on purpose | 15:08 |
Mr_Broken | my biggest problem right now is : | 15:08 |
sdake | i know its annoying there is no key/value deletion feataure | 15:09 |
Mr_Broken | glance and glance cache in multinode topology behind haproxy ! | 15:09 |
sdake | Mr_Broken that isnt working for you? | 15:09 |
Mr_Broken | its RANDOMLY picking up a controller to store images on | 15:09 |
sdake | Mr_Broken it should round robin | 15:09 |
sdake | and then when it reads glance imges it should roundrobin again until it finds the correct image | 15:09 |
Mr_Broken | well, 1st image-create - goes on 1 controller | 15:09 |
sdake | i know, not ideal | 15:09 |
*** b_bezak has quit IRC | 15:09 | |
Mr_Broken | then when you deploy from image - it connects to different one | 15:09 |
Mr_Broken | ans Image Not Found ... | 15:10 |
sdake | are you mising cinder in with this? | 15:10 |
Mr_Broken | after 2 retries - it founds image and deploys | 15:10 |
sdake | or straight nova+glance | 15:10 |
Mr_Broken | yes there is cinder in the mix | 15:10 |
*** mliima has joined #openstack-kolla | 15:10 | |
sdake | Mr_Broken yup t hat is how it is designed although cinder is either broken tm or our understanding of it is broken | 15:10 |
Mr_Broken | LVM ISCSI + XIV Driver for Spectrum Accelerate (also ISCSI) | 15:10 |
inc0 | ceph is really nice project | 15:11 |
inc0 | it really is | 15:11 |
inc0 | and solves so many issues;) | 15:11 |
Mr_Broken | yeah, I am deploying on a with ceph as well | 15:11 |
inc0 | Mr_Broken, then you should use ceph backend for glance | 15:11 |
Mr_Broken | so I managed to workaround the issue with glance itself (by using external swift, as backend for glance) | 15:11 |
Mr_Broken | but then glance-cache has the same issues :/ | 15:11 |
sdake | Mr_Broken note https://review.openstack.org/#/c/306662/ | 15:12 |
patchbot | sdake: patch 306662 - kolla - Make Cinder access glance in round robin fashion | 15:12 |
inc0 | Mr_Broken, if you deploy ceph it should use ceph backend...and really you dont use glance cache | 15:12 |
Mr_Broken | and list-queued round-robins to the glance controllers | 15:12 |
inc0 | as image is CoW snaphot in ceph | 15:12 |
Mr_Broken | well the glance-cache was a neccesity , since this swift backend for glance is far away ;] | 15:12 |
Mr_Broken | over internet | 15:12 |
Mr_Broken | and my XIV Cinder driver , timeouts from time to time , when trying to fetch image :/ | 15:12 |
*** ppowell has quit IRC | 15:13 | |
Mr_Broken | its madness :D | 15:13 |
*** vhosakot has joined #openstack-kolla | 15:13 | |
inc0 | yeah, now I get why you picked this irc handle | 15:13 |
sdake | inc0 pbourke_ looks like this review has lost some attention, would you midn looking again: https://review.openstack.org/#/c/306662/ | 15:13 |
patchbot | sdake: patch 306662 - kolla - Make Cinder access glance in round robin fashion | 15:13 |
Mr_Broken | @sdake: so you guys are aware, and there was some work on that already - great news! | 15:14 |
sdake | Mr_Broken yes although the cinder team may remove the cinder-api_servers key entirely | 15:14 |
sdake | Mr_Broken because they dont like how we hae used it specifically | 15:14 |
sdake | Mr_Broken we have fixed this already for nova | 15:15 |
sdake | nova has the api servers thing | 15:15 |
sdake | the root of the problem is glance doesn't replicate, and it shoudl do so | 15:16 |
Mr_Broken | that would solve a lot | 15:16 |
Mr_Broken | there was someone on the channel earlier, that tried to rsync those files between docker volumes on the host | 15:16 |
Mr_Broken | but said it didnt work for him ;] | 15:16 |
sdake | sbezverk you up | 15:17 |
inc0 | changed my -1 to +@ | 15:17 |
inc0 | +@ | 15:17 |
inc0 | +2 ffs | 15:17 |
sdake | inc0 tia :) | 15:17 |
sdake | Mr_Broken thats a shame, I think that would be a viable workaround | 15:18 |
sdake | then we could get rid of these api server usages in our codebase | 15:18 |
*** salv-orlando has joined #openstack-kolla | 15:18 | |
sdake | by making a synchronization container | 15:18 |
Mr_Broken | inc0: hehe, yeah everything i touch in our project gets broken ! | 15:18 |
sdake | fuel does synchronization of glance | 15:19 |
sdake | everyone else uses the api server thing | 15:19 |
Mr_Broken | sdake: i might try to develop some "hack" around it during the next days , but looking forward for some extra attention on https://review.openstack.org/#/c/306662/ :D | 15:19 |
patchbot | Mr_Broken: patch 306662 - kolla - Make Cinder access glance in round robin fashion | 15:19 |
sdake | Mr_Broken note a cinder patch is required too | 15:19 |
Mr_Broken | yeah, cinder needs some love in Mitaka | 15:20 |
Mr_Broken | how are your kubernetes thingie going ? | 15:20 |
sdake | https://review.openstack.org/#/c/306756/ | 15:20 |
patchbot | sdake: patch 306756 - cinder - Retry when glance returns None on fetch | 15:20 |
inc0 | Mr_Broken, in mitaka we assumed ceph pretty much | 15:20 |
inc0 | as we didn't really have iscsi support | 15:21 |
Mr_Broken | fun fact - my architects were saying hard no to ceph for months - > went to 1 customer meeting -> and now I am to broke mitaka + ceph next week ;] | 15:22 |
Mr_Broken | yep, didnt make iscsi work on contenerized cinder | 15:23 |
sdake | Mr_Broken note cinder cats are not going to fix the bug | 15:23 |
Mr_Broken | so i just installed it on separate "storage" node | 15:23 |
inc0 | Mr_Broken, it works in Newton | 15:23 |
inc0 | but only in Newton | 15:23 |
Mr_Broken | and no backports?! | 15:23 |
inc0 | no... | 15:23 |
Mr_Broken | fml | 15:23 |
inc0 | it's big feature | 15:23 |
inc0 | well you can do manual backport of this stuff | 15:24 |
inc0 | some people did | 15:24 |
inc0 | and it works | 15:24 |
inc0 | we'll even point you the way | 15:24 |
inc0 | however stable is stable... | 15:24 |
Mr_Broken | thing is this PoC is scheduled for production few months ahead | 15:24 |
inc0 | cinder iscsi is not nearly tested enough to backport it to stable branch | 15:24 |
Mr_Broken | cant deliver a service based on workarounds :D | 15:24 |
*** shardy has joined #openstack-kolla | 15:25 | |
inc0 | well, I strongly advice going with ceph for Mitaka | 15:25 |
Mr_Broken | yep | 15:25 |
inc0 | if you have hard requirement on iscsi, that's one of ways | 15:25 |
Mr_Broken | thats scheduled for next week ;] | 15:25 |
Mr_Broken | will keep you posted | 15:25 |
Mr_Broken | yeah, hardware cost are approved , some devices already bought | 15:25 |
Mr_Broken | need ISCSI support :/ | 15:25 |
*** The_Ball has quit IRC | 15:26 | |
inc0 | well these workaround won't be that horrible | 15:26 |
inc0 | it's just few patches to be cherry picked | 15:26 |
Mr_Broken | sure for a while, but when i leave project, and handover things | 15:26 |
Mr_Broken | how will they "update" in future | 15:26 |
inc0 | Mr_Broken, if they upgrade kolla | 15:27 |
Mr_Broken | ehh dont want to leave such a mess behind :/ | 15:27 |
inc0 | they will have iscsi in main tree | 15:27 |
Mr_Broken | true, newton incoming to stable soon ? | 15:27 |
inc0 | this is relevant only for transition between mitaka->newton | 15:27 |
inc0 | well, when newton releases, which is ~September if my memory serves | 15:27 |
Mr_Broken | cool | 15:27 |
inc0 | Oct 3-7 | 15:28 |
inc0 | that's full newton release | 15:28 |
inc0 | and all will be supported in tree by then | 15:28 |
inc0 | I mean it is now, but we do dev now so master is not super stable | 15:28 |
*** b_bezak has joined #openstack-kolla | 15:29 | |
*** belmoreira has quit IRC | 15:29 | |
sdake | inc0 that date is wrong | 15:30 |
sdake | we have a 2 week lag from the ret o the openstack schedule | 15:30 |
inc0 | ok, forgot about that | 15:30 |
sdake | it is rigth in the schedule :) | 15:31 |
*** caowei has quit IRC | 15:31 | |
*** caowei has joined #openstack-kolla | 15:31 | |
*** ppowell has joined #openstack-kolla | 15:32 | |
*** salv-orlando has quit IRC | 15:33 | |
*** Serlex has quit IRC | 15:36 | |
*** Lyncos has joined #openstack-kolla | 15:36 | |
Lyncos | Ping inc0 | 15:36 |
sdake | Mr_Broken I'm really not sure how to help you - the cinder community doesnt want to cooperate with your use case | 15:37 |
sdake | Mr_Broken the glance community doesn't think its a problem | 15:37 |
inc0 | Lyncos, wassup? | 15:37 |
sdake | Mr_Broken everyoen thinks some type of single storage solution should be used (such as ceph or swift) | 15:37 |
kfox1111 | morning | 15:38 |
sdake | yo kfox1111 | 15:38 |
inc0 | yo kfox1111 long time no see;) | 15:38 |
kfox1111 | apuimedo: nice. thanks. :) | 15:39 |
*** vdo_ is now known as vdo | 15:39 | |
*** vdo has joined #openstack-kolla | 15:39 | |
kfox1111 | inc0: Yeah. :/ | 15:39 |
kfox1111 | been way too busy of late. finally getting some time freed back up. | 15:39 |
kfox1111 | been playing a lot with k8s recently. | 15:40 |
kfox1111 | really good stuff. :) | 15:40 |
sdake | kfox1111 has alot of potential - has a long way to go ;) | 15:40 |
Mr_Broken | sdake: well I should be able to use it however i want, wasnt modularity one of the main ideas of openstack ? :] | 15:41 |
Mr_Broken | also having more than 1 solution is supported by documentation | 15:41 |
sdake | Mr_Broken feel free to chime in on the launchpad tracker where the cinder bug was closed wontfix | 15:41 |
sdake | Mr_Broken it couldn't hurt | 15:41 |
Mr_Broken | will do! | 15:41 |
sdake | also sking on #openstack-glance | 15:41 |
sdake | asking | 15:41 |
Mr_Broken | will put some project-managers on this task as well ;] | 15:41 |
sdake | Mr_Broken also sync up with sbezverk - he did the negotations with the cinder people | 15:42 |
sdake | Mr_Broken note its the second link that needs attention | 15:42 |
kfox1111 | sdake: yeah. but beats the hell out of doing it manually with docker-compose/heat. :) | 15:42 |
sdake | I am happy eough to merge it as is | 15:42 |
*** daneyon has joined #openstack-kolla | 15:42 | |
sdake | Mr_Broken https://review.openstack.org/#/c/306756/7 | 15:43 |
patchbot | sdake: patch 306756 - cinder - Retry when glance returns None on fetch | 15:43 |
sdake | https://launchpad.net/bugs/1571211 | 15:43 |
openstack | Launchpad bug 1571211 in Cinder "In multinode setup VM fails to launch due to cinder not checking with all glance api servers" [Medium,Won't fix] - Assigned to John Griffith (john-griffith) | 15:43 |
Mr_Broken | thanks | 15:43 |
openstackgerrit | Dave McCowan proposed openstack/kolla: Fernet Key Implementation [WIP] https://review.openstack.org/291539 | 15:44 |
sdake | Mr_Broken explain your use case in the launchpad bug above | 15:48 |
sdake | Mr_Broken if you have evidence it is supported by documentation that helps :) | 15:49 |
*** caowei has quit IRC | 15:50 | |
*** salv-orlando has joined #openstack-kolla | 15:51 | |
sdake | mewald hey apologies for losing track of our conversation | 15:52 |
sdake | mewald i'd suggest filing a blueprint for multidomain if one doesn't exist | 15:52 |
*** v1k0d3n has joined #openstack-kolla | 15:52 | |
sdake | link it here | 15:52 |
sdake | and i'll place it in the corrct state | 15:52 |
*** sacharya has joined #openstack-kolla | 15:53 | |
*** dmk0202 has joined #openstack-kolla | 15:54 | |
*** v1k0d3n has quit IRC | 16:00 | |
openstackgerrit | Steven Dake proposed openstack/kolla: Notify operators that cinder + LVM doesn't work https://review.openstack.org/332228 | 16:02 |
sdake | Mr_Broken just for you ^^ | 16:02 |
*** v1k0d3n has joined #openstack-kolla | 16:03 | |
*** v1k0d3n has quit IRC | 16:04 | |
-openstackstatus- NOTICE: Gerrit is being restarted now to apply an emergency security-related configuration change | 16:05 | |
wirehead_ | It takes magical skills to understand where the boundry between “configurable enough to fit any possible need” and the buffet at a Las Vegas casino lies. | 16:06 |
Mr_Broken | @sdake: thanks a lot | 16:10 |
Mr_Broken | thing is the issue is not only with glance and file backend | 16:10 |
Mr_Broken | there is also a problem with glance-cache having same issues when trying to queue-cached etc | 16:11 |
Mr_Broken | even if the backend for glance is on external swift | 16:12 |
Mr_Broken | not sure if its glance or cinder problem ;] | 16:12 |
Mr_Broken | seems both should cooperate ... | 16:13 |
wirehead_ | hey, inc0… About https://review.openstack.org/#/c/327925/ — Is there any reason why it’s in tasks/config.yml instead of groups/all.yml ? | 16:13 |
patchbot | wirehead_: patch 327925 - kolla - Enable kolla k8s to override bind api bind address... | 16:13 |
*** pbourke_ is now known as pbourke | 16:14 | |
inc0 | wirehead_, so in all.yml we normally don't put any logic | 16:15 |
inc0 | and this is logic | 16:15 |
Lyncos | Hi Inc0 sorry was at lunch | 16:15 |
*** daneyon_ has joined #openstack-kolla | 16:17 | |
jgriffith | sdake: oh... good reminder :) | 16:20 |
sdake | jgriffith ya it has fallen of my radar too | 16:21 |
sdake | but now more and more peole ask about ti in channel | 16:21 |
jgriffith | sdake: I don't recall... does that current version work? | 16:21 |
sdake | it works with that small patch i wrote | 16:21 |
*** daneyon_ has quit IRC | 16:21 | |
sdake | that yu reworked - not sure ifyour current patch works | 16:21 |
sdake | but the bug has been closed as wontfix | 16:21 |
jgriffith | sdake: That's what i mean, let's make sure the path in the review works and I"ll work on trying to get it merged | 16:22 |
jgriffith | sdake: so the bottom line is that none of us are happy that Glance has this flaw, but I'd rather see things *work* than try and push the rope | 16:22 |
*** dmk0202 has quit IRC | 16:22 | |
sdake | jgriffith i dont like it either fwiw | 16:23 |
jgriffith | sdake: yeah | 16:24 |
sdake | jgriffith I dont know anyone in glance circles, but i'm sure they know this is a problem and has been for years | 16:24 |
sdake | and if it hasn't been fixed yet, there is probbly a reason | 16:24 |
jgriffith | sdake: ok, based on sergui's testing sounds like it needs to be put back to patch 5 | 16:24 |
patchbot | jgriffith: https://review.openstack.org/#/c/5/ - openstack-infra/system-config - Fix problem with jenkins known_hosts url. (MERGED) | 16:24 |
jgriffith | hmmm well nice to meet you patchbot | 16:24 |
sdake | jgriffith i'll ask sbezverk to dobule check which patch works correctly | 16:25 |
sdake | he seems to be afk atm | 16:25 |
jgriffith | sdake: ok... I put the data method back in https://review.openstack.org/#/c/306756/ | 16:27 |
patchbot | jgriffith: patch 306756 - cinder - Retry when glance returns None on fetch | 16:27 |
sdake | cool i'll ping sbezverk to test it | 16:28 |
sbezverk | sdake: jgriffith: as far as I know glance refused to support this model. they even went on records with this.. | 16:36 |
sdake | not asing for support | 16:37 |
sdake | asking for corrct implementation | 16:37 |
sdake | implementation and support are two orthogonal things | 16:37 |
*** shardy has quit IRC | 16:37 | |
sdake | if cinder wants to deprecate those keys, and follow the normal deprecation cycle w e will cope with that | 16:37 |
*** david-lyle has quit IRC | 16:38 | |
sdake | its not permitted to deprecate stuff that already exists in a prevoius cycle in that previous cycle :) | 16:38 |
*** david-lyle has joined #openstack-kolla | 16:38 | |
*** huikang has joined #openstack-kolla | 16:43 | |
openstackgerrit | Ken Wronkiewicz proposed openstack/kolla-kubernetes: Adding debugging documentation for Memcached https://review.openstack.org/330766 | 16:46 |
*** huikang has quit IRC | 16:48 | |
*** fragatina has quit IRC | 16:48 | |
*** sacharya has quit IRC | 16:51 | |
*** huikang has joined #openstack-kolla | 16:54 | |
*** ssurana has joined #openstack-kolla | 16:56 | |
*** dmsimard is now known as dmsimard|afk | 16:59 | |
*** inc0 has quit IRC | 17:01 | |
*** inc0 has joined #openstack-kolla | 17:02 | |
*** gfidente has quit IRC | 17:09 | |
*** Serlex has joined #openstack-kolla | 17:12 | |
*** Serlex has quit IRC | 17:17 | |
*** sdake_ has joined #openstack-kolla | 17:24 | |
*** Mech422 has quit IRC | 17:26 | |
*** sdake has quit IRC | 17:27 | |
*** sacharya has joined #openstack-kolla | 17:29 | |
*** ayoung has quit IRC | 17:33 | |
*** ppowell has quit IRC | 17:33 | |
*** salv-orl_ has joined #openstack-kolla | 17:38 | |
*** sacharya has quit IRC | 17:41 | |
*** salv-orlando has quit IRC | 17:41 | |
*** zhiwei has joined #openstack-kolla | 17:42 | |
*** zhiwei has quit IRC | 17:46 | |
*** ppowell has joined #openstack-kolla | 17:48 | |
*** coolsvap has quit IRC | 17:51 | |
*** daneyon has quit IRC | 17:54 | |
*** daneyon has joined #openstack-kolla | 17:54 | |
*** Mr_Broken has quit IRC | 17:59 | |
*** Mr_Broken has joined #openstack-kolla | 17:59 | |
*** Mech422 has joined #openstack-kolla | 18:00 | |
*** Mr_Broken has quit IRC | 18:03 | |
*** Mech422__ has joined #openstack-kolla | 18:04 | |
*** Mech422 has quit IRC | 18:05 | |
*** daneyon_ has joined #openstack-kolla | 18:05 | |
*** ayoung has joined #openstack-kolla | 18:05 | |
*** daneyon_ has quit IRC | 18:10 | |
*** thumpba has quit IRC | 18:25 | |
*** thumpba has joined #openstack-kolla | 18:27 | |
*** huikang has quit IRC | 18:33 | |
Mech422__ | inc0: for multiple domains in keystone - you might need multiple drivers? Like we use local and LDAP ? | 18:38 |
*** sacharya has joined #openstack-kolla | 18:41 | |
*** sean-k-mooney has joined #openstack-kolla | 18:43 | |
*** sacharya has quit IRC | 18:47 | |
Mech422__ | inc0: just as a datapoint - my 3 ceph node glance/cinder seem to work fine | 18:47 |
inc0 | 10 node ceph works too;) | 18:48 |
*** ayoung has quit IRC | 18:54 | |
Mech422__ | inc0: cool - the multiple domain stuff for keystone is interesting... we currently use a native/mysql backend for admin and LDAP for users | 18:54 |
*** stvnoyes has quit IRC | 18:55 | |
*** stvnoyes has joined #openstack-kolla | 18:55 | |
rhallisey | inc0, ok so I have ansible deploying mariadb | 18:56 |
*** sdake has joined #openstack-kolla | 18:56 | |
rhallisey | this has been a little frustrating :) | 18:56 |
inc0 | how so? | 18:56 |
rhallisey | the kube ansible module doesn't deploy jobs. The yaml error checking is non existent | 18:57 |
rhallisey | creating configmaps doesn't like multiple files has input | 18:58 |
rhallisey | don't know how to get any status from the native kube module even if I could create a job :/ | 18:58 |
rhallisey | so I have my own getting pod status | 18:58 |
rhallisey | but it works | 18:59 |
*** ayoung has joined #openstack-kolla | 18:59 | |
*** sdake_ has quit IRC | 18:59 | |
rhallisey | I don't think the kube module was meant for heavy application/cluster wide orchestraion | 19:00 |
sdake | rhallisey they want it to be | 19:00 |
*** ayoung has quit IRC | 19:00 | |
sdake | rhallisey they are kicking that into overdrive | 19:00 |
rhallisey | the ansible module? | 19:00 |
sdake | yup | 19:00 |
rhallisey | I'll have to make a bunch of PRs then | 19:01 |
rhallisey | we need a bunch of stuff from it | 19:01 |
sdake | whatever works | 19:01 |
*** erhudy has joined #openstack-kolla | 19:03 | |
*** daneyon has quit IRC | 19:04 | |
*** daneyon has joined #openstack-kolla | 19:04 | |
*** JoseMello has joined #openstack-kolla | 19:12 | |
sean-k-mooney | ever get the feeling that your over thinking things. im about to try to deploy fuel in a vm ,with vagrent, in a vm, hosted on openstack, provisioned by kolla... | 19:12 |
*** dmsimard|afk is now known as dmsimard | 19:15 | |
wirehead_ | Maybe you could add Hercules and S/390 Linux, for style points, sean-k-mooney. | 19:16 |
*** fragatina has joined #openstack-kolla | 19:16 | |
sean-k-mooney | haha well at least i have nested virtualisation turned on so hopefully it wont just die when i booth the vagrent vms | 19:18 |
*** b_bezak has quit IRC | 19:19 | |
*** b_bezak has joined #openstack-kolla | 19:20 | |
*** b_bezak has quit IRC | 19:25 | |
*** cu5 has quit IRC | 19:29 | |
*** inc0_ has joined #openstack-kolla | 19:29 | |
*** david-lyle has quit IRC | 19:30 | |
*** wmiller has quit IRC | 19:30 | |
*** dwalsh has quit IRC | 19:30 | |
*** mandre has quit IRC | 19:30 | |
*** cinerama has quit IRC | 19:30 | |
*** inc0 has quit IRC | 19:31 | |
*** wmiller has joined #openstack-kolla | 19:31 | |
*** huikang has joined #openstack-kolla | 19:34 | |
*** cinerama has joined #openstack-kolla | 19:34 | |
*** cu5 has joined #openstack-kolla | 19:35 | |
*** dave-mccowan has quit IRC | 19:36 | |
*** mandre has joined #openstack-kolla | 19:36 | |
*** mliima has quit IRC | 19:36 | |
*** huikang has quit IRC | 19:38 | |
*** dave-mccowan has joined #openstack-kolla | 19:40 | |
*** ayoung has joined #openstack-kolla | 19:40 | |
*** david-lyle has joined #openstack-kolla | 19:41 | |
*** zhiwei has joined #openstack-kolla | 19:43 | |
*** dwalsh has joined #openstack-kolla | 19:43 | |
*** dave-mccowan has quit IRC | 19:45 | |
*** dave-mcc_ has joined #openstack-kolla | 19:45 | |
*** zhiwei has quit IRC | 19:47 | |
*** daneyon_ has joined #openstack-kolla | 19:53 | |
*** haplo37_ has joined #openstack-kolla | 19:58 | |
*** daneyon_ has quit IRC | 19:58 | |
*** huikang has joined #openstack-kolla | 20:11 | |
*** ppowell has quit IRC | 20:31 | |
*** jtriley has quit IRC | 20:38 | |
*** ayoung has quit IRC | 20:38 | |
sean-k-mooney | well that didnt work. looks like the vagrent file for fule is broke. the vm worked well though. | 20:44 |
openstackgerrit | David Wang proposed openstack/kolla-kubernetes: WIP: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 20:44 |
*** huikang has quit IRC | 20:58 | |
*** thumpba has quit IRC | 20:59 | |
*** mark-casey has joined #openstack-kolla | 21:04 | |
*** inc0_ has quit IRC | 21:10 | |
openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 21:12 |
*** rhallisey has quit IRC | 21:14 | |
openstackgerrit | Ken Wronkiewicz proposed openstack/kolla: WIP: Glance interface address, memcached override, and glance override. https://review.openstack.org/332481 | 21:21 |
*** sean-k-mooney has quit IRC | 21:33 | |
*** fragatina has quit IRC | 21:33 | |
*** jpeeler has quit IRC | 21:36 | |
*** haplo37_ has quit IRC | 21:37 | |
*** JoseMello has quit IRC | 21:39 | |
openstackgerrit | Joshua Harlow proposed openstack/kolla: Add a cloud-init RHEL(ish) bootstrap + run kolla script https://review.openstack.org/321802 | 21:40 |
*** ayoung has joined #openstack-kolla | 21:41 | |
*** sacharya has joined #openstack-kolla | 21:44 | |
openstackgerrit | Joshua Harlow proposed openstack/kolla: Add a cloud-init RHEL(ish) bootstrap + run kolla script https://review.openstack.org/321802 | 21:44 |
*** clayton has quit IRC | 21:46 | |
*** clayton has joined #openstack-kolla | 21:47 | |
*** sacharya has quit IRC | 21:49 | |
*** zhiwei has joined #openstack-kolla | 21:50 | |
*** zhiwei has quit IRC | 21:55 | |
*** mbound has joined #openstack-kolla | 21:55 | |
*** dwalsh has quit IRC | 21:59 | |
*** huikang has joined #openstack-kolla | 21:59 | |
*** huikang has quit IRC | 22:03 | |
*** jrist has quit IRC | 22:06 | |
*** v1k0d3n has joined #openstack-kolla | 22:13 | |
*** severion has joined #openstack-kolla | 22:14 | |
*** v1k0d3n has quit IRC | 22:18 | |
*** vhosakot has quit IRC | 22:19 | |
*** huikang has joined #openstack-kolla | 22:22 | |
openstackgerrit | Joshua Harlow proposed openstack/kolla: Stop using a global logger for all the things https://review.openstack.org/321884 | 22:25 |
*** huikang has quit IRC | 22:30 | |
*** williamcaban has quit IRC | 22:31 | |
*** daneyon_ has joined #openstack-kolla | 22:36 | |
*** rcarrillocruz has quit IRC | 22:39 | |
*** rcarrillocruz has joined #openstack-kolla | 22:40 | |
*** daneyon_ has quit IRC | 22:41 | |
*** severion has quit IRC | 22:45 | |
harlowja | man, the way to tell kolla to build off a custom git location is really weird to figure out | 23:07 |
harlowja | the usage of oslo.config doesn't really seem to be helping .... | 23:07 |
harlowja | in that there is some yaml files, there is some cli options, there is some conf files, like madness anyone? | 23:10 |
harlowja | i'd prefer like 1 damn thing, lol | 23:10 |
*** erhudy has quit IRC | 23:11 | |
harlowja | its not annoying to others that there is like multiple ways to configure stuff? | 23:12 |
*** jrist has joined #openstack-kolla | 23:12 | |
*** daneyon has quit IRC | 23:14 | |
*** daneyon has joined #openstack-kolla | 23:15 | |
harlowja | for example, just trying to get the following to run | 23:15 |
harlowja | # kolla-build glance --type source --config-file kolla-glance.tmp | 23:15 |
harlowja | with file | 23:15 |
harlowja | [glance-base] | 23:15 |
harlowja | type=git | 23:15 |
harlowja | source=git@github.com:openstack/glance.git | 23:15 |
harlowja | reference=stable/mitaka | 23:15 |
harlowja | and it appears to run all the things, now am i just not passing in things right, is there some other thing happening, is this supposed to be via a CLI option, who knows | 23:15 |
*** mark-casey has quit IRC | 23:30 | |
*** Jeffrey4l_ has joined #openstack-kolla | 23:32 | |
harlowja | so ya, i'm a little confused as what the actual params/config/cli/or who knows what to get it do actually just build glance, lol | 23:33 |
harlowja | http://paste.openstack.org/show/521011/ is where i got | 23:33 |
harlowja | why its trying to clone a tarball, idk | 23:33 |
harlowja | also whats up with the following automatically getting pulled in | 23:35 |
harlowja | RUN curl http://trunk.rdoproject.org/centos7/current-passed-ci/delorean.repo /etc/yum.repos.d/kibana.yum.repo /etc/yum.repos.d/elasticsearch.repo delorean-deps.repo even though i'm doing a source install | 23:35 |
harlowja | (of just glance) | 23:35 |
*** salv-orlando has joined #openstack-kolla | 23:39 | |
*** salv-orl_ has quit IRC | 23:42 | |
*** sacharya has joined #openstack-kolla | 23:45 | |
*** salv-orlando has quit IRC | 23:46 | |
*** sacharya has quit IRC | 23:49 | |
*** ssurana has quit IRC | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!