| *** gyee has quit IRC | 00:08 | |
| *** rebase has quit IRC | 00:13 | |
| *** karthikpr has joined #openstack-operators | 00:17 | |
| *** mriedem has quit IRC | 00:38 | |
| klindgren | Stupid question - but does anyoine know if on openstack rpc services (thinking neutron-ovs/linuxbridge, nova-compute), you can spinup another instances on the same box | 00:47 |
|---|---|---|
| klindgren | for a short (less than a minute) | 00:47 |
| klindgren | without it blowing stuff up? | 00:47 |
| *** Apoorva has quit IRC | 00:48 | |
| klindgren | with containerized control plane trying to do a blue/green deployment. But unlike API services - you can't do anything against the RPC servers to see if they actually work or not. | 00:48 |
| klindgren | IE you can't hit an API and do some tests before you place that API in to active rotation | 00:49 |
| mnaser | rpc services would probably be problematic klindgren | 00:52 |
| mnaser | i can imagine a few reasons such as multiple instances reporting state at the same time | 00:52 |
| klindgren | right now I have it stopping the original rpc container, starting the new one, doing as much checking that I can that it started, | 00:53 |
| mnaser | klindgren id poll for openstack nova servie list | 00:53 |
| mnaser | *service | 00:53 |
| klindgren | if any of the tests fail, stop the new rpc container, and start the old one | 00:53 |
| mnaser | that's probably one of the more accurate ways to know if it went up | 00:53 |
| klindgren | eh - thats a non-start in our env | 00:53 |
| mnaser | what do you mean by non-start? | 00:53 |
| klindgren | we have thousands of compute nodes, so polling that nova service list per compute node to ensure it started is bascially a non-starter | 00:54 |
| mnaser | klindgren i believe you're looking to get into the kolla land? :) | 00:54 |
| klindgren | we are using kolla containers | 00:54 |
| klindgren | but are passing on koll-ansible | 00:54 |
| klindgren | kolla-ansible* | 00:54 |
| mnaser | we're doing the same, i'd schedule this in groups, ex: do the process on 10 nodes, wait for all 10 to get up, and keep rolling out | 00:55 |
| mnaser | (we're using kolla-ansible) | 00:55 |
| mnaser | but if you're using ansible you should be able to say, do updates in groups of 50 or 100. | 00:55 |
| mnaser | you also dont want to reboot *, your control plane will not have a great time | 00:56 |
| klindgren | kolla-ansible is not specific enough for us. IE trying to deploy a glance service to 2 nodes, and it logs into a bunch of nodes to get information | 00:56 |
| mnaser | klindgren i do agree it has its shortcomings but we pretty much chose to try and support it and add improvements | 00:57 |
| mnaser | i do agree with you its not great at component deployment (we're doing the same roll out) | 00:57 |
| klindgren | The lack of any sort of checking/tests/extensions is also a non-starter | 00:58 |
| mnaser | klindgren how so? we've found it to be pretty extendible to do all sorts of things we want | 00:58 |
| klindgren | it doesn't manage haproxy, so if I want to start a new API container. I want the new api container to be started. Tests ran against it. When its ready I want it to be added to rotation in haproxy | 01:00 |
| klindgren | but first I want to drain off the connections to the old container | 01:00 |
| klindgren | IE allow glance image downloads/uploads to finish | 01:00 |
| klindgren | by manage haproxy I mean maintenace the active container, allow connections to drop to 0, then when its not recieving traffic, swap in the new container | 01:01 |
| klindgren | and un maintenace it | 01:01 |
| klindgren | So I extended the haproxy ansible module to do draining support. So it will wait/poll upto some timeout value for connections to drop to 0 and when they do, it will tell ansilbe that its at the desired state. IE out of rotation with no active connections, or its been x minutes and we are moving on. | 01:03 |
| klindgren | https://github.com/ansible/ansible/pull/21420 - need to fix some minor nits - but its there. | 01:04 |
| *** manheim has joined #openstack-operators | 01:06 | |
| *** saneax is now known as saneax-_-|AFK | 01:12 | |
| mnaser | klindgren sorry, i was a bit afk, didnt see this, but obviously those are shortcomings but i mean, all contributions are weelcome but i understand that's not always easy | 01:15 |
| klindgren | we honestly weighed out the doing in kolla, vs's do it our selves. Because I dont like re-investing the wheel | 01:23 |
| klindgren | but at the end of the day re-inventing the wheel was actually much easier. | 01:23 |
| klindgren | with the rest of the tooling that we already have in place. | 01:23 |
| *** manheim has quit IRC | 01:24 | |
| mnaser | klindgren bummer, would have liked another operator bring in work like this | 01:24 |
| mnaser | hopefully our work will eventually cross paths and we can get you to make the jump :) | 01:24 |
| *** markvoelker has joined #openstack-operators | 01:26 | |
| klindgren | IE I needed something did a rockstar job of deploying a container to a host. I dont need config management, I dont need something to figure out what services should be running where, I dont need something to attempt to do database setup and the like. | 01:26 |
| klindgren | I have things that already do that stuff for me | 01:26 |
| klindgren | We were planning on sharing the deployment stuff directly with the kolla people. So if they wanted to make use of the concepts and or code that works. | 01:29 |
| klindgren | but want to actually run it in production first | 01:29 |
| *** markvoelker has quit IRC | 01:31 | |
| klindgren | Some reading: https://openstack.nimeyo.com/104734/re-openstack-dev-kolla-ansible-kolla-am-i-doing-this-wrong | 01:32 |
| klindgren | specifically about what we noticed and some other things when trying it out. | 01:33 |
| *** jamesden_ has quit IRC | 01:37 | |
| *** fragatin_ has joined #openstack-operators | 01:42 | |
| *** fragati__ has joined #openstack-operators | 01:44 | |
| *** fragatina has quit IRC | 01:44 | |
| *** mriedem has joined #openstack-operators | 01:46 | |
| *** fragatin_ has quit IRC | 01:46 | |
| *** fragati__ has quit IRC | 01:49 | |
| *** markvoelker has joined #openstack-operators | 02:03 | |
| *** arif-ali has quit IRC | 02:54 | |
| *** arif-ali has joined #openstack-operators | 02:55 | |
| *** armax has quit IRC | 02:59 | |
| *** cemason1 has joined #openstack-operators | 03:02 | |
| *** cemason has quit IRC | 03:02 | |
| *** furlongm has quit IRC | 03:47 | |
| *** mriedem has quit IRC | 03:55 | |
| *** fragatina has joined #openstack-operators | 04:42 | |
| *** fragatina has quit IRC | 04:46 | |
| *** karthikpr has quit IRC | 06:26 | |
| *** rcernin has joined #openstack-operators | 06:48 | |
| *** rcernin has quit IRC | 06:48 | |
| *** rcernin has joined #openstack-operators | 07:17 | |
| *** arnewiebalck__ has joined #openstack-operators | 07:41 | |
| *** rcernin has quit IRC | 07:42 | |
| *** fragatina has joined #openstack-operators | 07:42 | |
| *** fragatina has quit IRC | 07:42 | |
| *** fragatina has joined #openstack-operators | 07:43 | |
| *** manheim has joined #openstack-operators | 08:01 | |
| *** manheim has quit IRC | 08:06 | |
| *** arnewiebalck__ has quit IRC | 08:10 | |
| *** aojea has joined #openstack-operators | 08:12 | |
| *** markvoelker has quit IRC | 08:56 | |
| *** arnewiebalck__ has joined #openstack-operators | 09:18 | |
| *** manheim has joined #openstack-operators | 09:23 | |
| *** arnewiebalck__ has quit IRC | 09:55 | |
| *** markvoelker has joined #openstack-operators | 09:56 | |
| *** markvoelker has quit IRC | 10:02 | |
| *** arnewiebalck__ has joined #openstack-operators | 10:40 | |
| *** saneax-_-|AFK is now known as saneax | 11:06 | |
| *** manheim has quit IRC | 11:07 | |
| *** manheim has joined #openstack-operators | 11:07 | |
| *** fragatina has quit IRC | 11:09 | |
| *** alexpilotti has quit IRC | 11:38 | |
| *** alexpilotti has joined #openstack-operators | 11:39 | |
| *** manheim has quit IRC | 11:49 | |
| *** arnewiebalck__ has quit IRC | 11:50 | |
| *** markvoelker has joined #openstack-operators | 11:57 | |
| *** haplo37 has quit IRC | 12:00 | |
| *** markvoelker has quit IRC | 12:02 | |
| *** haplo37 has joined #openstack-operators | 12:09 | |
| *** retr0h has quit IRC | 12:25 | |
| *** trw[m] has quit IRC | 12:25 | |
| *** trw[m] has joined #openstack-operators | 12:34 | |
| *** tanee is now known as tanee_away | 12:40 | |
| *** racedo has quit IRC | 13:24 | |
| *** racedo has joined #openstack-operators | 13:26 | |
| *** mriedem has joined #openstack-operators | 13:27 | |
| *** racedo has quit IRC | 13:29 | |
| *** racedo has joined #openstack-operators | 13:30 | |
| *** retr0h has joined #openstack-operators | 13:52 | |
| *** saneax is now known as saneax-_-|AFK | 14:00 | |
| *** fragatina has joined #openstack-operators | 14:11 | |
| *** racedo has quit IRC | 14:13 | |
| *** saneax-_-|AFK is now known as saneax | 14:28 | |
| *** fragatina has quit IRC | 14:34 | |
| *** rarcea has joined #openstack-operators | 14:39 | |
| *** arnewiebalck__ has joined #openstack-operators | 14:41 | |
| *** rarcea has quit IRC | 14:52 | |
| *** aojea has quit IRC | 14:54 | |
| *** jamesdenton has joined #openstack-operators | 14:54 | |
| *** aojea has joined #openstack-operators | 14:55 | |
| *** saneax is now known as saneax-_-|AFK | 14:56 | |
| *** armax has joined #openstack-operators | 14:58 | |
| *** aojea has quit IRC | 14:59 | |
| *** cemason1 has quit IRC | 15:00 | |
| *** cemason has joined #openstack-operators | 15:02 | |
| *** armax has quit IRC | 15:15 | |
| *** aojea has joined #openstack-operators | 15:53 | |
| *** armax has joined #openstack-operators | 16:15 | |
| *** cemason has quit IRC | 16:17 | |
| *** rcernin has joined #openstack-operators | 16:17 | |
| *** cemason has joined #openstack-operators | 16:18 | |
| *** karthikpr has joined #openstack-operators | 16:30 | |
| *** rcernin has quit IRC | 16:45 | |
| *** Apoorva has joined #openstack-operators | 16:45 | |
| *** rcernin has joined #openstack-operators | 16:45 | |
| *** Apoorva has quit IRC | 16:50 | |
| *** armax has quit IRC | 16:51 | |
| *** tanee_away is now known as tanee | 17:00 | |
| *** tanee is now known as tanee_away | 17:00 | |
| *** armax has joined #openstack-operators | 17:00 | |
| *** pwnall1337 is now known as zz_pwnall1337 | 17:21 | |
| *** mriedem1 has joined #openstack-operators | 17:35 | |
| *** armax has quit IRC | 17:36 | |
| *** mriedem has quit IRC | 17:37 | |
| *** cemason has quit IRC | 17:37 | |
| *** racedo has joined #openstack-operators | 17:42 | |
| *** arnewiebalck__ has quit IRC | 17:49 | |
| *** rcernin has quit IRC | 17:53 | |
| *** jamesdenton has quit IRC | 17:58 | |
| *** rcernin has joined #openstack-operators | 18:09 | |
| *** rcernin has quit IRC | 18:23 | |
| *** manheim has joined #openstack-operators | 18:26 | |
| *** manheim has quit IRC | 18:31 | |
| *** tanee_away is now known as tanee | 18:42 | |
| *** tanee is now known as tanee_away | 18:42 | |
| *** racedo has quit IRC | 19:06 | |
| *** racedo has joined #openstack-operators | 19:44 | |
| *** racedo has quit IRC | 19:58 | |
| *** racedo has joined #openstack-operators | 19:59 | |
| *** slaweq has joined #openstack-operators | 20:02 | |
| *** cnf has quit IRC | 20:13 | |
| *** markvoelker has joined #openstack-operators | 20:14 | |
| *** manheim has joined #openstack-operators | 20:15 | |
| *** fragatina has joined #openstack-operators | 20:19 | |
| *** manheim has quit IRC | 20:19 | |
| *** ircuser-1 has quit IRC | 20:26 | |
| *** cnf has joined #openstack-operators | 20:31 | |
| *** racedo has quit IRC | 20:33 | |
| *** racedo has joined #openstack-operators | 20:36 | |
| *** racedo has quit IRC | 20:42 | |
| *** racedo has joined #openstack-operators | 20:46 | |
| *** racedo has quit IRC | 20:48 | |
| *** fragatina has quit IRC | 21:25 | |
| *** mriedem1 is now known as mriedem | 21:39 | |
| *** simon-AS559 has joined #openstack-operators | 22:01 | |
| *** racedo has joined #openstack-operators | 22:43 | |
| *** aojea has quit IRC | 22:54 | |
| *** saneax-_-|AFK is now known as saneax | 23:15 | |
| *** markvoelker has quit IRC | 23:17 | |
| *** IRCFrEAK has joined #openstack-operators | 23:18 | |
| *** IRCFrEAK has quit IRC | 23:24 | |
| *** racedo has quit IRC | 23:43 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!