*** ricolin has joined #openstack-containers | 00:59 | |
*** hongbin has joined #openstack-containers | 02:02 | |
*** ykarel has joined #openstack-containers | 03:22 | |
*** hongbin has quit IRC | 03:58 | |
*** udesale has joined #openstack-containers | 04:03 | |
*** ramishra has joined #openstack-containers | 04:45 | |
*** janki has joined #openstack-containers | 04:46 | |
openstackgerrit | Ghanshyam Mann proposed openstack/magnum master: DNM: Testing all legacy jobs on bionic https://review.openstack.org/642339 | 05:15 |
---|---|---|
*** jaewook_oh has joined #openstack-containers | 05:15 | |
*** jaewook_oh has quit IRC | 05:22 | |
*** sapd1 has joined #openstack-containers | 06:08 | |
*** ivve has joined #openstack-containers | 06:28 | |
*** ianychoi has quit IRC | 06:32 | |
*** ianychoi has joined #openstack-containers | 06:32 | |
*** ianychoi has quit IRC | 06:35 | |
*** ianychoi has joined #openstack-containers | 06:36 | |
*** pcaruana has joined #openstack-containers | 07:00 | |
*** rcernin has quit IRC | 07:03 | |
*** belmoreira has joined #openstack-containers | 07:41 | |
*** belmoreira has quit IRC | 07:42 | |
*** belmoreira has joined #openstack-containers | 07:42 | |
*** flwang1 has joined #openstack-containers | 07:55 | |
flwang1 | strigazi: around for a catch up? | 07:56 |
openstackgerrit | Feilong Wang proposed openstack/magnum master: (WIP) Add cluster upgrade to the API https://review.openstack.org/514959 | 07:56 |
*** gsimondon has joined #openstack-containers | 08:10 | |
*** sapd1 has quit IRC | 08:15 | |
*** gsimondo1 has joined #openstack-containers | 08:29 | |
*** gsimondon has quit IRC | 08:31 | |
openstackgerrit | Merged openstack/python-magnumclient master: Add hidden property to cluster template https://review.openstack.org/634955 | 08:47 |
*** adrianreza has quit IRC | 09:00 | |
*** adrianreza has joined #openstack-containers | 09:00 | |
*** ykarel has quit IRC | 09:10 | |
*** ykarel has joined #openstack-containers | 09:11 | |
*** ttsiouts has joined #openstack-containers | 09:26 | |
*** janki has quit IRC | 09:30 | |
*** jchhatbar has joined #openstack-containers | 09:30 | |
*** ykarel is now known as ykarel|lunch | 09:37 | |
*** alisanhaji has joined #openstack-containers | 09:55 | |
openstackgerrit | Lingxian Kong proposed openstack/magnum master: Improve floating_ip_enabled https://review.openstack.org/641547 | 09:58 |
*** ykarel|lunch is now known as ykarel | 10:07 | |
*** udesale has quit IRC | 10:34 | |
strigazi | flwang1: hello | 10:49 |
strigazi | flwang1: are you here? | 10:49 |
flwang1 | yes | 10:49 |
strigazi | tell me | 10:49 |
flwang1 | i think lingxian has discussed with you about the floating ip design, are you happy with that? | 10:50 |
strigazi | look ok | 10:50 |
flwang1 | nice | 10:51 |
flwang1 | strigazi: and here is the NPD patch https://review.openstack.org/641902 | 10:51 |
strigazi | i have resize and NPD for today | 10:51 |
flwang1 | nice | 10:52 |
strigazi | FYI I'll push a path to disable the CI | 10:52 |
strigazi | we should not burn resources | 10:52 |
flwang1 | oh, sure, we have to disable it for now | 10:52 |
flwang1 | we just need to be more careful | 10:52 |
openstackgerrit | Feilong Wang proposed openstack/magnum master: (WIP) Add cluster upgrade to the API https://review.openstack.org/514959 | 10:53 |
strigazi | flwang1: https://ci.centos.org/artifacts/fedora-coreos/prod/builds/latest/ | 10:54 |
strigazi | https://ci.centos.org/artifacts/fedora-coreos/prod/builds/latest/fedora-coreos-29.753-openstack.qcow2.gz | 10:54 |
flwang1 | wow | 10:54 |
flwang1 | do you have plan to move? | 10:55 |
flwang1 | https://review.openstack.org/#/c/514959/10/magnum/drivers/k8s_fedora_atomic_v1/driver.py@58 here is my design about the upgrade api | 10:55 |
strigazi | flwang1: do you mind if we set a time for tmr morning you, (evening today for me) to discuss? | 10:55 |
flwang1 | strigazi: sure | 10:55 |
flwang1 | strigazi: it would be nice if you can take a look then we can discuss | 10:56 |
flwang1 | 9 utc? | 10:56 |
strigazi | I will | 10:56 |
flwang1 | 21 i mean | 10:56 |
strigazi | this https://www.timeanddate.com/worldclock/fixedtime.html?msg=chat&iso=20190311T2100 ? | 10:57 |
flwang1 | yep, is that ok for you? | 10:57 |
strigazi | yeap | 10:57 |
flwang1 | cool | 10:57 |
strigazi | see you then | 10:57 |
flwang1 | cool | 10:57 |
strigazi | I have to go, I'll take a look in the patches | 10:58 |
flwang1 | thanks | 10:58 |
*** ricolin has quit IRC | 10:59 | |
*** dave-mccowan has joined #openstack-containers | 11:10 | |
*** ttsiouts has quit IRC | 11:12 | |
*** dave-mccowan has quit IRC | 11:14 | |
*** dave-mccowan has joined #openstack-containers | 11:20 | |
*** ykarel_ has joined #openstack-containers | 11:22 | |
*** ykarel has quit IRC | 11:24 | |
*** ivve has quit IRC | 11:31 | |
*** dave-mccowan has quit IRC | 11:42 | |
*** ykarel_ is now known as ykarel | 11:49 | |
*** dave-mccowan has joined #openstack-containers | 11:56 | |
*** jchhatbar has quit IRC | 11:58 | |
*** ttsiouts has joined #openstack-containers | 12:02 | |
*** ivve has joined #openstack-containers | 12:14 | |
*** janki has joined #openstack-containers | 12:35 | |
*** ianychoi has quit IRC | 13:28 | |
*** ianychoi has joined #openstack-containers | 13:29 | |
*** sapd1 has joined #openstack-containers | 13:31 | |
*** dave-mccowan has quit IRC | 13:33 | |
*** henriqueof has quit IRC | 13:41 | |
*** lxkong has quit IRC | 13:42 | |
*** henriqueof has joined #openstack-containers | 14:05 | |
*** dave-mccowan has joined #openstack-containers | 14:20 | |
*** alisanhaji has quit IRC | 14:30 | |
*** mkuf has quit IRC | 14:31 | |
*** FlorianFa has joined #openstack-containers | 14:50 | |
*** FlorianFa has quit IRC | 14:52 | |
*** FlorianFa has joined #openstack-containers | 14:52 | |
*** udesale has joined #openstack-containers | 14:56 | |
*** sapd1 has quit IRC | 15:05 | |
*** alisanhaji has joined #openstack-containers | 15:22 | |
*** ykarel is now known as ykarel|afk | 15:28 | |
*** janki has quit IRC | 15:33 | |
*** flwang1 has quit IRC | 15:42 | |
*** ramishra has quit IRC | 16:02 | |
*** gsimondo1 has quit IRC | 16:06 | |
*** ykarel_ has joined #openstack-containers | 16:07 | |
*** ykarel|afk has quit IRC | 16:09 | |
*** pcaruana has quit IRC | 16:23 | |
*** pcaruana has joined #openstack-containers | 16:23 | |
*** ykarel_ is now known as ykarel | 16:23 | |
*** ykarel is now known as ykarel|away | 16:29 | |
*** ivve has quit IRC | 16:32 | |
*** udesale has quit IRC | 16:52 | |
*** ttsiouts has quit IRC | 17:24 | |
*** ykarel|away has quit IRC | 17:29 | |
*** ivve has joined #openstack-containers | 17:35 | |
*** hongbin has joined #openstack-containers | 17:40 | |
*** flwang has joined #openstack-containers | 19:44 | |
openstackgerrit | Lingxian Kong proposed openstack/magnum master: Improve floating_ip_enabled https://review.openstack.org/641547 | 19:47 |
flwang | NobodyCam: i don't think you can access the atomic container, why do you want to do that? | 19:54 |
*** lxkong has joined #openstack-containers | 20:00 | |
*** itlinux has joined #openstack-containers | 20:30 | |
*** itlinux has quit IRC | 20:34 | |
*** itlinux has joined #openstack-containers | 20:56 | |
strigazi | NobodyCam runc exec -t kube-apiserver bash | 21:02 |
strigazi | flwang: hey, I'm testing the patches | 21:02 |
strigazi | flwang: Will get back to you in a bit | 21:02 |
flwang | strigazi: sure | 21:04 |
flwang | i'm around, just let me know when you're ready to catch up | 21:04 |
*** henriqueof has quit IRC | 21:08 | |
NobodyCam | strigazi: Awesome thank you :) | 21:13 |
NobodyCam | can some help me set a trusted root ca for Kube-controler-manager? | 21:14 |
NobodyCam | I'm getting : `controllermanager.go:169] error building controller context: cloud provider could not be initialized: could not init cloud provider "openstack": Post https://10.21.100.198:5000/v3/auth/tokens: x509: certificate signed by unknown authority` | 21:14 |
NobodyCam | How ever I not sure what Need to be set to trust my cert | 21:15 |
strigazi | https://docs.openstack.org/magnum/latest/configuration/sample-config.html | 21:16 |
strigazi | # Path to the OpenStack CA-bundle file to pass and install in all cluster | 21:16 |
strigazi | # nodes. (string value) | 21:16 |
strigazi | #openstack_ca_file = | 21:16 |
eandersson | strigazi, flwang lets get this merged https://review.openstack.org/#/c/619148/ | 21:19 |
eandersson | clean up, but a good one imo | 21:19 |
strigazi | +2 | 21:21 |
strigazi | flwang: we missed that https://github.com/helm/charts/tree/master/stable/node-problem-detector | 21:21 |
flwang | strigazi: you want to install with helm? | 21:22 |
NobodyCam | strigazi: Again Thank you :) testing that now | 21:22 |
flwang | NobodyCam: better setup a devstack env and then you can compare the config | 21:23 |
strigazi | now it is done, we could have less code, mainly helm helps on that | 21:24 |
*** alisanhaji has quit IRC | 21:24 | |
strigazi | NPD patch looks ok, as you want | 21:25 |
flwang | strigazi: we should have a policy to encourage using helm if we think that's the direction | 21:27 |
flwang | strigazi: i mean some documents for developer about this addons implementation | 21:28 |
strigazi | What do you think? | 21:30 |
strigazi | Docs are helpful, and we should tell developers how to add addons. | 21:32 |
flwang | strigazi: you mean do the NPD with helm now? | 21:32 |
eandersson | flwang, strigazi lets get this one in as well https://review.openstack.org/#/c/637266/ | 21:34 |
eandersson | py3.7 test job | 21:34 |
eandersson | https://review.openstack.org/#/c/637267/ | 21:35 |
*** dave-mccowan has quit IRC | 21:36 | |
flwang | eandersson: thank you for all the good work | 21:36 |
eandersson | Anytime flwang | 21:36 |
eandersson | It's good to keep the review backlog clean so we can focus on the important stuff. | 21:36 |
flwang | eandersson: true | 21:37 |
strigazi | I would prefer it this way. If you are up for it, you can change it. I don't want to tell you to do it again :) | 21:38 |
flwang | strigazi: thanks, let's get it in this time. and we will pay more attention next time about how to choose the right way for new addons, agree? | 21:41 |
openstackgerrit | Spyros Trigazis proposed openstack/magnum master: Support <ClusterID>/actions/resize API https://review.openstack.org/638572 | 21:43 |
strigazi | I rebased to test resize we vm uuid | 21:44 |
strigazi | ok | 21:44 |
*** pcaruana has quit IRC | 21:45 | |
*** ivve has quit IRC | 21:54 | |
*** itlinux has quit IRC | 21:55 | |
flwang | strigazi: did you get a chance to review the upgrade api patch? | 21:56 |
flwang | strigazi: still around? | 22:01 |
strigazi | flwang: I'm testing resize now. I see you remove the helper script for testing upgrade. You don't test? | 22:17 |
openstackgerrit | Merged openstack/magnum master: Add missing ws separator between words https://review.openstack.org/619148 | 22:17 |
strigazi | flwang: oh, you put it in an rst | 22:18 |
strigazi | flwang: resize gives me 405 | 22:18 |
strigazi | flwang: 202 finally, but I passed as nodegroup: "error" and it worked | 22:21 |
flwang | strigazi: since i didn't use the nodegroup parameter now since we don't support it yet | 22:22 |
strigazi | flwang: resize for scale up, is it working? | 22:23 |
strigazi | flwang: works, but I need to pass empty list | 22:24 |
flwang | strigazi: yep, scale up works as well. we could set a default value for nodes_to_remove | 22:25 |
strigazi | scale up while UPDATE_IN_PROGRESS returnes 202 but has no effect | 22:26 |
flwang | so you think 204 could be better? | 22:27 |
strigazi | if it has no effect, shouldn't return 4xx ? | 22:27 |
strigazi | or we can make to have an effect | 22:27 |
flwang | what do you mean 'no effect' then? | 22:27 |
strigazi | resize from 2 to 3 -> 202 and heat receives the request | 22:28 |
flwang | since the stack/cluster is "UPDATE_IN_PROGRESS", so i think it's effecting, isn't it? | 22:28 |
strigazi | right after resize from 3 to 4 -> 202 but heat doesn't receive the request | 22:28 |
flwang | ah, i think it's because we can't update a cluster which is in 'update_in_progress' | 22:29 |
flwang | you should get a warning https://review.openstack.org/#/c/638572/16/magnum/conductor/handlers/cluster_conductor.py@196 | 22:30 |
flwang | strigazi: i will dig, this could be a bug | 22:30 |
flwang | normally, if the cluster is being updated, then user can't issue another update | 22:30 |
strigazi | we could make it work, heat accepts it | 22:31 |
flwang | ok | 22:31 |
flwang | but i don't think that's good user case anway | 22:31 |
strigazi | it is what the autoscaler wants though | 22:32 |
flwang | good point, i can see the idea now | 22:33 |
flwang | yep, the workload could increase very fast, and CA may send request very often | 22:34 |
flwang | i will do more test about that case | 22:35 |
flwang | and you're welcome to submit patch set if you can sort it faster | 22:35 |
strigazi | have you seen the nodegroup patches? | 22:36 |
flwang | yep, i saw that | 22:36 |
flwang | it's on my list, i will start to test it this week | 22:37 |
strigazi | resize is close, needs a bit more testing | 22:38 |
flwang | is the NPD patch OK for you? | 22:38 |
strigazi | yes, one nit, I left a comment | 22:39 |
openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora-atomic-k8s] Adding Node Problem Detector https://review.openstack.org/641902 | 22:40 |
flwang | cool, i just fixed it | 22:40 |
flwang | strigazi: let's discuss the rolling upgrade? | 22:40 |
flwang | one problem when i testing the upgrade api, i found i can't get those default versions/tags if user didn't specified as labels | 22:42 |
flwang | that's OK, the only cons is we can't check the downgrade to raise 400 | 22:42 |
strigazi | shall we make all the container tags another field? | 22:44 |
strigazi | or leave as a 2nd step | 22:45 |
strigazi | resize seems to work | 22:45 |
strigazi | let's check resize when already in progress and we are good | 22:45 |
flwang | you mean take the npd_tag off from current patch? | 22:46 |
flwang | I will test the resize api again for the update-in-progress case | 22:47 |
flwang | strigazi: i can't see much benefit to put all tags into the 2nd patch | 22:49 |
flwang | given we will merge them all finally in this release | 22:50 |
*** threestrands has joined #openstack-containers | 22:50 | |
strigazi | question about fips | 22:51 |
strigazi | you want to remove fips for all workers? | 22:51 |
strigazi | I didn't realize that | 22:51 |
strigazi | in our cloud, it doesn't matter | 22:51 |
strigazi | but for other sites it seem extremly radical | 22:51 |
flwang | strigazi: we'd like to encourage customer to expose services via lb | 22:52 |
flwang | and user can bind FIP for worker manually | 22:52 |
strigazi | wasn't it an option to have fips master_only, not fips, and all? | 22:52 |
flwang | strigazi: that's the thing i discussed with lingxian, and we | 22:52 |
flwang | like to get comments from the community | 22:52 |
strigazi | it is best practice | 22:53 |
lxkong | hi strigazi | 22:53 |
strigazi | but when people upgrade it is very unexpected | 22:53 |
flwang | strigazi: yep, i know it's a bit aggressive | 22:53 |
strigazi | again, for our cloud we don't care | 22:53 |
lxkong | yeah, the patch removed the fip for workers. But users can still allocate fip for workers manually | 22:53 |
strigazi | but for other might be really bad | 22:54 |
flwang | eandersson: jakeyip: how do you guys think? | 22:54 |
lxkong | strigazi: that's why i sent an email to mailing list to gather feedback | 22:54 |
lxkong | but for now, no one replied | 22:54 |
eandersson | We don't use this feature at the moment. | 22:54 |
lxkong | i see Jim added himself to the reviewers, let's see how he will say | 22:54 |
strigazi | lxkong: flwang you really think it is a good move? | 22:55 |
eandersson | I added Jim | 22:55 |
lxkong | strigazi: i think so. We don't want to expose the workers to the public internet by default | 22:55 |
strigazi | this is for your cloud ^^ | 22:55 |
lxkong | yeah | 22:55 |
lxkong | so it's important to know if it affects otheres | 22:56 |
strigazi | and an email for a couple of days will solve this? | 22:56 |
lxkong | strigazi: your suggestion? | 22:56 |
strigazi | why not make the three options as we discussed in the past? | 22:56 |
eandersson | My only concern is that while no one might use the "current' option today | 22:56 |
eandersson | but what if someone installs Queens? | 22:56 |
eandersson | Could we back-port this behavior if we made the change? | 22:57 |
lxkong | eandersson: in Queens, workers still have fip if floating_ip_enabled | 22:57 |
eandersson | Sure - but what about when they upgrade. | 22:57 |
flwang | strigazi: what's the problem when you deal with this issue? i mean any blocker for your old patch to change the fip enabled from bool to enum? | 22:57 |
lxkong | eandersson: what's your concern for upgrade? | 22:58 |
strigazi | there is not blocker, I didn't have time and it was low priority | 22:58 |
flwang | strigazi: do you think a new label 'master_only' or something could be a balanced option? | 22:59 |
strigazi | could be | 23:00 |
strigazi | more friendly | 23:00 |
strigazi | for catalyst, isn't even more economical not fips at all and only in the lbaas for the api? | 23:01 |
strigazi | for catalyst, isn't even more economical no fips at all and only in the lbaas for the api? | 23:01 |
strigazi | like this, it one fip per cluster | 23:02 |
flwang | strigazi: that's related to security | 23:02 |
strigazi | how? | 23:02 |
lxkong | strigazi: the attach surface | 23:02 |
flwang | reduced more exposure | 23:02 |
lxkong | attack | 23:02 |
strigazi | I don't think you understoop what I said. I was comparing | 23:03 |
strigazi | having fips in the master(s) nodes to having only a fip for the API LB | 23:03 |
strigazi | in a cluster with three masters, you would have only the LB exposed, how is that less secure? | 23:04 |
strigazi | and the most secure is zero fips | 23:04 |
lxkong | the lb only allows traffic to 6443 port | 23:04 |
strigazi | which is bad? | 23:05 |
flwang | strigazi: user can also have 0 FIP | 23:05 |
lxkong | strigazi: but we need to allow the user to access the cluster from internet | 23:05 |
lxkong | the fip needs to be included in the api cerficiate | 23:05 |
strigazi | it is | 23:05 |
lxkong | if floating_ip_enabled is false, then the cluster is only for internal usage | 23:06 |
lxkong | users can never access the cluster even allocate fip to the vip later | 23:06 |
strigazi | hmm, the FIP of the LB is not | 23:06 |
strigazi | so, if we add the LB FIP to the api cert we are good, no? | 23:07 |
lxkong | we are good | 23:07 |
strigazi | isn't that even easier? | 23:07 |
lxkong | any api could do this? or manually? | 23:07 |
strigazi | I mean , when users asks for cluster with LB, the LB will have a fip | 23:08 |
lxkong | no matter if 'floating_ip_enabled' is set or not? | 23:08 |
flwang | i think that's not a bad idea | 23:09 |
strigazi | good point, maybe add a label for that? so floating_ip_enabled controls only the fips on the vms | 23:09 |
lxkong | my proposal is, if floating_ip_enabled is set, only allocate fip to the lb vip, otherwise, no fip at all | 23:10 |
strigazi | and for LB fip have a label? | 23:10 |
flwang | strigazi: i kind of like the ideea | 23:10 |
lxkong | strigazi: no a bad idea | 23:10 |
flwang | strigazi: lxkong: are we all good for that proposal? | 23:11 |
lxkong | strigazi: but if we should allocate fip for the master in the cluster only has one master? | 23:11 |
strigazi | the proposal is good in theory, but from experience all openstack clouds are weird :) | 23:11 |
flwang | strigazi: hah, i won't argue that | 23:11 |
lxkong | lb_fip label is true and floating_ip_enabled is false, how we deal with the cluster with only one master? | 23:12 |
strigazi | you can have single master cluster with LB? not much value on that but it works like this atm | 23:12 |
flwang | lxkong: depends on if master lb is enabled | 23:12 |
lxkong | ah, yes | 23:12 |
flwang | lxkong: if you have only one master, but you do want to enable lb, your call | 23:12 |
flwang | everything works as the same | 23:12 |
lxkong | one master cluster can not have lb, right? | 23:13 |
strigazi | at the moment it can | 23:13 |
flwang | lxkong: it can if you enable the master_lb_enabled | 23:13 |
strigazi | (it was like this because we were planning to scale up the masters) | 23:13 |
lxkong | ok, then all good | 23:13 |
lxkong | we add a label to control the lb vip | 23:14 |
lxkong | leave floating_ip_enabled to control the vms | 23:14 |
strigazi | +1 | 23:14 |
lxkong | flwang: agree? | 23:14 |
flwang | +1 | 23:14 |
lxkong | cool | 23:15 |
strigazi | IMO, we should tell user to not expose the k8s api to the internet | 23:15 |
flwang | strigazi: still have time to discuss rolling upgrade? or you have to goto bed | 23:15 |
strigazi | that is just suggestion, I agree with what we just said | 23:16 |
flwang | lxkong: can you add a small document about that ^ ? | 23:16 |
lxkong | i will | 23:16 |
flwang | lxkong: thank you | 23:16 |
flwang | strigazi: i'm ok to let you go now ;) but you have to promise you will leave comments on https://review.openstack.org/#/c/514959/ | 23:17 |
strigazi | flwang: for upgrade, I think it is good to upgrade the addons/k8s-resources separately. We can achieve this with CTs that have only changes for tags | 23:18 |
strigazi | I'll leave comments | 23:19 |
strigazi | Before going to bed, just smth to think about | 23:19 |
flwang | strigazi: cool. thank you very much. and I really appreciate for your time to stay late with us | 23:20 |
flwang | have a good night, my friend | 23:20 |
strigazi | At CERN, we have some custom plugings (four for storage and two for special site certs and kerberos keytabs) | 23:20 |
flwang | so i think you are setting the tags explicitly? | 23:21 |
strigazi | we plan to write a helm chart for each one of those and then a meta-chart that will control all of them | 23:21 |
flwang | that would be cool | 23:21 |
strigazi | this way we only release one chart that rules them all | 23:22 |
flwang | we could do that for magnum as well | 23:23 |
flwang | in the future | 23:23 |
strigazi | we can do this upstream too, control all plugins with many charts and one meta chart which has only dependencies | 23:23 |
strigazi | exactly | 23:23 |
flwang | i'm happy to review it | 23:23 |
strigazi | cool | 23:23 |
flwang | cool, i think we're all good for today? | 23:24 |
strigazi | see you in the meeting tmr :) have a nice day lxkong flwang and evening eandersson | 23:24 |
strigazi | flwang: yeap | 23:24 |
lxkong | have a good night | 23:25 |
strigazi | cheers | 23:25 |
eandersson | Sorry had to step away for a bit | 23:31 |
eandersson | strigazi, have a good one! | 23:31 |
*** threestrands_ has joined #openstack-containers | 23:42 | |
*** threestrands has quit IRC | 23:45 | |
*** hongbin has quit IRC | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!