| *** goldyfruit___ has joined #openstack-containers | 00:11 | |
| *** iokiwi has joined #openstack-containers | 00:58 | |
| iokiwi | Hey all, wondering if anyone can guide me on getting a devstack with magnum setup? I'm following the quick start here https://docs.openstack.org/magnum/latest/contributor/quickstart.html#exercising-the-services-using-devstack but during the setup neutron modifies the network interfaces and I lose access to my host | 01:01 |
|---|---|---|
| iokiwi | http://paste.openstack.org/show/775013/ | 01:01 |
| iokiwi | The result I am trying to achieve is to stand up a devstack with a public ip address which a remote team can build magnum-ui against | 01:02 |
| *** flwang has quit IRC | 01:44 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: Test k8s CI https://review.opendev.org/677581 | 02:04 |
| *** FlorianFa has quit IRC | 02:15 | |
| *** flwang has joined #openstack-containers | 02:22 | |
| *** FlorianFa has joined #openstack-containers | 02:35 | |
| *** ykarel|away has joined #openstack-containers | 03:08 | |
| *** PrinzElvis has quit IRC | 03:39 | |
| *** PrinzElvis has joined #openstack-containers | 03:45 | |
| *** udesale has joined #openstack-containers | 04:00 | |
| *** ykarel|away has quit IRC | 04:08 | |
| *** ianychoi_ has joined #openstack-containers | 04:24 | |
| *** ykarel|away has joined #openstack-containers | 04:25 | |
| *** ykarel|away is now known as ykarel | 04:25 | |
| *** ianychoi has quit IRC | 04:27 | |
| *** dave-mccowan has quit IRC | 04:29 | |
| *** pcaruana has joined #openstack-containers | 04:42 | |
| *** pcaruana has quit IRC | 05:12 | |
| *** rcernin has quit IRC | 05:22 | |
| openstackgerrit | Jake Yip proposed openstack/magnum master: Return default quota from API https://review.opendev.org/673782 | 05:37 |
| *** rcernin has joined #openstack-containers | 05:39 | |
| *** rcernin has quit IRC | 05:51 | |
| *** rcernin has joined #openstack-containers | 06:09 | |
| *** pcaruana has joined #openstack-containers | 06:21 | |
| *** ricolin has joined #openstack-containers | 06:38 | |
| *** ricolin has quit IRC | 06:39 | |
| *** lpetrut has joined #openstack-containers | 06:52 | |
| *** ykarel is now known as ykarel|lunch | 06:57 | |
| *** trident has quit IRC | 07:08 | |
| *** ivve has joined #openstack-containers | 07:09 | |
| *** trident has joined #openstack-containers | 07:17 | |
| *** trident has quit IRC | 07:22 | |
| brtknr | squarebracket: i believe those are passed to qemu by nova? I dont have a good understanding of that layer | 07:26 |
| *** trident has joined #openstack-containers | 07:31 | |
| *** rcernin has quit IRC | 07:38 | |
| *** ykarel|lunch is now known as ykarel | 07:45 | |
| *** ykarel is now known as ykarel|meeting | 08:12 | |
| *** ykarel_ has joined #openstack-containers | 08:14 | |
| *** ykarel|meeting has quit IRC | 08:16 | |
| *** ykarel_ is now known as ykarel|meeting | 08:18 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora-atomic][k8s] Support operating system upgrade https://review.opendev.org/669593 | 08:30 |
| openstackgerrit | Spyros Trigazis proposed openstack/magnum master: k8s_fedora_atomic: Add PodSecurityPolicy https://review.opendev.org/681013 | 08:38 |
| openstackgerrit | Spyros Trigazis proposed openstack/magnum master: k8s_fedora: Set rp_filter=1 for calico https://review.opendev.org/681244 | 08:38 |
| *** flwang1 has joined #openstack-containers | 08:50 | |
| flwang1 | strigazi: let's have meeting in 10 mins? | 08:50 |
| flwang1 | brtknr: ? | 08:50 |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/magnum master: [WIP] ng-6: Add new fields to nodegroup objects https://review.opendev.org/667088 | 09:00 |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/magnum master: [WIP] ng-7: Adapt parameter and output mappings https://review.opendev.org/667089 | 09:00 |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/magnum master: [WIP] ng-8: APIs for nodegroup CRUD operations https://review.opendev.org/647792 | 09:00 |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/magnum master: [WIP] ng-9: Driver for nodegroup operations https://review.opendev.org/667090 | 09:00 |
| *** ttsiouts has joined #openstack-containers | 09:00 | |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/python-magnumclient master: Add nodegroup CRUD commands https://review.opendev.org/647793 | 09:00 |
| *** lpetrut has quit IRC | 09:04 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora-atomic][k8s] Support operating system upgrade https://review.opendev.org/669593 | 09:06 |
| *** ianychoi_ has quit IRC | 09:09 | |
| *** rcernin has joined #openstack-containers | 09:11 | |
| strigazi | hello | 09:12 |
| flwang1 | hi | 09:13 |
| strigazi | brtknr: ping | 09:13 |
| ttsiouts | hi! | 09:14 |
| strigazi | flwang1: psp and calico fix, looks good to you? | 09:15 |
| strigazi | flwang1: conformance passes. | 09:15 |
| flwang1 | calico fix looks good for me | 09:19 |
| flwang1 | psp looks good as well, but i'd like a manual test for psp | 09:19 |
| strigazi | nothing chaned for calico. just a RoleBinding os that privilaged: true works. | 09:20 |
| strigazi | nothing chaned for calico. just a RoleBinding so that privilaged: true works. | 09:21 |
| flwang1 | strigazi: ok | 09:24 |
| flwang1 | how do you think the fedora coreos work? | 09:24 |
| strigazi | i don't think the software_config user data will work if we don't patch heat. | 09:25 |
| strigazi | If I was an ignition developer I wouldn't add multipart mime support | 09:25 |
| strigazi | we could patch heat so that it appends the creds to ignition | 09:26 |
| flwang1 | strigazi: hmm... patch heat to do special changes only for ignition? | 09:28 |
| flwang1 | i'm not sure if heat folks buy in that | 09:28 |
| *** ykarel_ has joined #openstack-containers | 09:32 | |
| brtknr | hello | 09:32 |
| strigazi | flwang1: why not? coreos is popular. rhel will support it | 09:32 |
| brtknr | sorry i didnt realise we confirmed 10am on wednesday | 09:32 |
| *** ykarel|meeting has quit IRC | 09:34 | |
| strigazi | flwang1: otherwise we need to do something else without heat SD | 09:34 |
| flwang1 | strigazi: i see | 09:35 |
| flwang1 | strigazi: we can try to propose change in heat to support ignition | 09:35 |
| strigazi | flwang1: I think the change is relatively small and it will be an opt-in option | 09:36 |
| flwang1 | unless we get a fully stop sign from the heat team, i think it's worthy to try comparing the effort to rework all the stuff | 09:36 |
| flwang1 | strigazi: let's do that then | 09:36 |
| flwang1 | strigazi: do you have any idea where is the code we should start? | 09:38 |
| strigazi | https://github.com/openstack/heat/blob/master/heat/engine/clients/os/nova.py#L327 | 09:39 |
| *** ykarel_ is now known as ykarel | 09:39 | |
| strigazi | if user_data_format == 'IGN3_SOFTWARE_CONFIG': | 09:40 |
| strigazi | IGN3 = ignition verison 3.0.0. | 09:40 |
| *** rcernin has quit IRC | 09:41 | |
| flwang1 | oh, that's simpler that i thought | 09:41 |
| flwang1 | it's doable i think | 09:41 |
| strigazi | it just needs to put the file with the creds in the correcy place in the ignition json | 09:42 |
| flwang1 | strigazi: yep, i understand now | 09:44 |
| flwang1 | i will try to catch up with ricolin to get comments from him first, how do you think? | 09:45 |
| strigazi | sounds good | 09:46 |
| flwang1 | it's just another opt-in option, i can't see why heat team reject it | 09:47 |
| flwang1 | especially given that ignition will be another boostrap way like cloud-init | 09:47 |
| flwang1 | it even could be a benefit for heat | 09:48 |
| strigazi | I hope so | 09:51 |
| flwang1 | strigazi: i still have a question before testing | 09:51 |
| flwang1 | for example, i have the ignition file and assuming heat should be able to support injecting the credentials into the igntion file | 09:52 |
| flwang1 | then how about the other software_deployment scripts? | 09:52 |
| flwang1 | will they need to be injected into the ignition json file as well? | 09:53 |
| flwang1 | otherwise, i think ignition can't read the format correctly without the multi part support | 09:53 |
| strigazi | the scripts for SD are like before | 09:53 |
| strigazi | nothing to do with ignition | 09:54 |
| strigazi | they will be executed in a container | 09:54 |
| strigazi | the heat agent container | 09:54 |
| strigazi | flwang1: makes sense? | 09:54 |
| flwang1 | so they won't be shipped in user-data when booting, but poll by the heat container agent later, right? | 09:55 |
| strigazi | ye | 09:55 |
| flwang1 | same as current fedora atomic way? | 09:55 |
| strigazi | s | 09:55 |
| strigazi | yes | 09:55 |
| flwang1 | ok, then good | 09:55 |
| strigazi | brtknr: flwang1 about NGs. Can you test the latest patchset? | 09:56 |
| strigazi | everything is there, client too | 09:56 |
| flwang1 | strigazi: sure | 09:56 |
| strigazi | brtknr: flwang1 do you have any questions about it? | 09:57 |
| flwang1 | btw, i had a quick glance about current patches, nothing big design change since those initial patches | 09:57 |
| strigazi | ttsiouts: ^^ | 09:57 |
| flwang1 | the only question is, how will it work with the resize | 09:57 |
| flwang1 | user can only resize a ng? | 09:57 |
| brtknr | strigazi: yes i can do, I've been battling with my devstack deployment yesterday, so hence the lack of feedback | 09:57 |
| flwang1 | not overally worker nodes, is it? | 09:58 |
| strigazi | flwang1: yes, user can resize an ng | 09:58 |
| ttsiouts | flwang1: yes only one NG at a time | 09:59 |
| flwang1 | and all master nodes will be in one ng? | 09:59 |
| flwang1 | strigazi: do you have a patch for the master resize? | 09:59 |
| brtknr | apologies again, im in the middle of our weekly standup | 09:59 |
| strigazi | flwang1: I was strongly against working in master resize now. | 09:59 |
| flwang1 | interesting, why? | 10:00 |
| strigazi | it will tank developement again and we won't do anything | 10:00 |
| strigazi | one step at a time | 10:00 |
| flwang1 | so just resource issue | 10:00 |
| flwang1 | not design issue, right? | 10:00 |
| strigazi | the current direction is a step to the correcy direction | 10:00 |
| strigazi | design is done with maste resize in mind | 10:01 |
| strigazi | makes sense? | 10:01 |
| flwang1 | ok, cool | 10:01 |
| strigazi | it is no difficult but I think we need to take all the great work done so far merged | 10:01 |
| flwang1 | agree | 10:02 |
| brtknr | i thought we were having a conversation about out-of-tree driver at one point? | 10:02 |
| strigazi | brtknr: no resources I would say | 10:03 |
| flwang1 | brtknr: i'm not a big fan of that, but i can see its value | 10:03 |
| strigazi | but it can be done | 10:03 |
| strigazi | I am a fan, but someone needs to do it :) | 10:03 |
| flwang1 | strigazi: btw, as for the coreos driver, i'd like to support boot from value from day 1, how do you think? | 10:03 |
| strigazi | sure | 10:04 |
| strigazi | I'll be back in a bit. Are you guys staying? | 10:04 |
| flwang1 | strigazi: and as for this patch https://review.opendev.org/#/c/621734/ did you see the scenario with image based? | 10:04 |
| flwang1 | i will be off in next 10 mins | 10:04 |
| flwang1 | it's 22:05 now | 10:05 |
| flwang1 | strigazi: and as for this patch https://review.opendev.org/#/c/621734/ did you test the scenario with image based? | 10:05 |
| flwang1 | when i say image based, it means with label boot_volume_size=0 | 10:05 |
| flwang1 | it didn't work for me, i tested several times | 10:06 |
| flwang1 | brtknr: it would be nice if you can help test this https://review.opendev.org/#/c/621734/ | 10:06 |
| brtknr | flwang1: i remember testing that patch, but i can see that I didnt leave a comment, will retest it | 10:06 |
| flwang1 | i think it can improve the cluster create performance with boot from volume | 10:07 |
| flwang1 | benefit from ceph's CoW | 10:07 |
| brtknr | hmm enabling boot_from_volume but setting boot_volume_size=0 seems like a nasty trick | 10:08 |
| brtknr | we ought to be able to identify this in the code | 10:08 |
| flwang1 | brtknr: take a look the patch and feel free post your comments there ;) | 10:09 |
| flwang1 | i have to go | 10:09 |
| flwang1 | ttyl, folks | 10:09 |
| brtknr | take care flwang1 | 10:09 |
| *** ttsiouts has quit IRC | 10:23 | |
| *** ttsiouts has joined #openstack-containers | 10:23 | |
| *** ttsiouts has quit IRC | 10:28 | |
| *** ianychoi has joined #openstack-containers | 10:30 | |
| openstackgerrit | Spyros Trigazis proposed openstack/magnum master: k8s_fedora: Set rp_filter=1 for calico https://review.opendev.org/681244 | 10:36 |
| brtknr | strigazi: I am only working for another 1hr today. I will do most of the testing tomorrow | 10:44 |
| brtknr | sadly the cluster is still coming as unhealthy | 10:44 |
| brtknr | trying to work out why | 10:44 |
| *** ianychoi has quit IRC | 10:45 | |
| brtknr | strigazi: any idea why the train-dev heat container agent is not pushed yet? | 10:45 |
| *** ianychoi has joined #openstack-containers | 10:45 | |
| brtknr | the change got merged some time ago | 10:47 |
| *** ttsiouts has joined #openstack-containers | 11:03 | |
| *** ttsiouts has quit IRC | 11:07 | |
| *** ttsiouts has joined #openstack-containers | 11:12 | |
| *** ykarel is now known as ykarel|afk | 11:20 | |
| *** udesale has quit IRC | 11:31 | |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/magnum master: [WIP] ng-9: Driver for nodegroup operations https://review.opendev.org/667090 | 11:38 |
| *** spsurya has joined #openstack-containers | 11:58 | |
| *** dave-mccowan has joined #openstack-containers | 12:05 | |
| *** lpetrut has joined #openstack-containers | 12:13 | |
| *** goldyfruit___ has quit IRC | 12:15 | |
| *** ykarel|afk is now known as ykarel | 12:38 | |
| *** jmlowe has quit IRC | 12:42 | |
| *** jmlowe has joined #openstack-containers | 12:59 | |
| *** jmlowe has joined #openstack-containers | 13:00 | |
| *** goldyfruit___ has joined #openstack-containers | 13:29 | |
| *** spiette has quit IRC | 13:36 | |
| *** spiette has joined #openstack-containers | 13:41 | |
| *** ricolin has joined #openstack-containers | 14:18 | |
| *** ttsiouts has quit IRC | 14:37 | |
| *** ykarel is now known as ykarel|away | 14:37 | |
| *** ttsiouts has joined #openstack-containers | 14:37 | |
| openstackgerrit | Merged openstack/magnum master: k8s_fedora: Set rp_filter=1 for calico https://review.opendev.org/681244 | 14:39 |
| *** ttsiouts has quit IRC | 14:42 | |
| *** ykarel|away has quit IRC | 14:46 | |
| *** ykarel has joined #openstack-containers | 15:03 | |
| *** jmlowe has quit IRC | 15:04 | |
| *** ArchiFleKs has joined #openstack-containers | 15:06 | |
| *** jmlowe has joined #openstack-containers | 15:09 | |
| *** ivve has quit IRC | 15:29 | |
| *** ykarel is now known as ykarel|away | 15:51 | |
| *** jmlowe has quit IRC | 16:13 | |
| *** ricolin has quit IRC | 16:22 | |
| *** lpetrut has quit IRC | 16:40 | |
| *** ivve has joined #openstack-containers | 16:42 | |
| *** ykarel|away has quit IRC | 16:52 | |
| *** ykarel|away has joined #openstack-containers | 17:08 | |
| *** spsurya has quit IRC | 17:32 | |
| *** jmlowe has joined #openstack-containers | 17:36 | |
| *** ramishra has quit IRC | 18:00 | |
| *** ykarel|away has quit IRC | 18:00 | |
| *** ykarel|away has joined #openstack-containers | 18:00 | |
| *** ykarel|away has quit IRC | 18:28 | |
| *** hogepodge has left #openstack-containers | 18:50 | |
| colby_ | Hey Everyone. Is there a guide on how to upgrade kubernetes versions on running clusters. I see in Stein there is an upgrade procedure but we are currently on rocky. It would be nice to be able to easily update kube versions to address security issues. | 19:49 |
| *** henriqueof has joined #openstack-containers | 19:51 | |
| *** flwang1 has quit IRC | 20:30 | |
| andrein | hello everyone, I think my magnum deployed kubernetes cluster is failing because the following script isn't running. https://opendev.org/openstack/magnum/src/branch/stable/stein/magnum/drivers/common/templates/kubernetes/fragments/write-kube-os-config.sh#L12 how can I debug this? | 20:36 |
| *** ivve has quit IRC | 21:05 | |
| *** ivve has joined #openstack-containers | 21:05 | |
| *** rcernin has joined #openstack-containers | 21:15 | |
| goldyfruit___ | andrein, https://opendev.org/openstack/magnum/src/branch/master/magnum/drivers/common/templates/kubernetes/fragments/write-kube-os-config.sh | 21:27 |
| goldyfruit___ | Seems to be fixed in master | 21:27 |
| andrein | goldyfruit___: thanks for looking into this, but I think the issue is that TRUST_ID is empty | 21:28 |
| andrein | just added `cluster_user_trust=True` as described in https://ask.openstack.org/en/question/114339/magnum-enable-cloud-controller-manager/ | 21:30 |
| andrein | and it looks like I have running pods now, as opposed to pending | 21:30 |
| *** goldyfruit_ has joined #openstack-containers | 21:32 | |
| *** goldyfruit___ has quit IRC | 21:34 | |
| goldyfruit_ | andrein, cool | 21:37 |
| andrein | I understand why the setting is off by default, but is it possible to run a kubernetes cluster at all with the default setting? | 21:40 |
| flwang | colby_: it's a new feature only supported in stein | 21:41 |
| flwang | colby_: sorry, train | 21:41 |
| andrein | ok, let me rephrase that: is it possible to run a kubernetes cluster at all with the default setting on stein? | 21:44 |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora-atomic][k8s] Support operating system upgrade https://review.opendev.org/669593 | 21:45 |
| flwang | andrein: why not? | 21:45 |
| flwang | andrein: we're using stable/stein on our prod | 21:46 |
| andrein | flwang: are you using the default cluster_user_trust=False in magnum.conf? | 21:47 |
| colby_ | flwang: So to upgrade current clusters I have to SSH into them and use atomic pull and atomic container update to update kubernetes? | 21:47 |
| flwang | it should be True | 21:47 |
| flwang | colby_: you can refer this https://github.com/openstack/magnum/blob/master/magnum/drivers/common/templates/kubernetes/fragments/upgrade-kubernetes.sh | 21:48 |
| goldyfruit_ | andrein, we are running stein in prod too | 21:48 |
| goldyfruit_ | andrein, https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/magnum/templates/magnum.conf.j2 | 21:48 |
| goldyfruit_ | This is the configuration we are using (deployed/configured by Kolla) | 21:49 |
| goldyfruit_ | Then we are enabling enable_cluster_user_trust | 21:50 |
| flwang | andrein: unless you don't need the cinder as volume driver or the swift registry support | 21:50 |
| andrein | Ah, I see | 21:51 |
| *** henriqueof1 has joined #openstack-containers | 21:51 | |
| *** henriqueof has quit IRC | 21:52 | |
| colby_ | flwang: Thanks! So this wont be more automated until Train then? | 21:56 |
| flwang | unless you want cherry pick | 21:57 |
| flwang | as we did :) | 21:57 |
| colby_ | ha yea...that gets problematic with updates :) | 21:58 |
| andrein | flwang this should be better documented either in the kolla or in the magnum docs. In the magnum docs the only reference to it is in the sample user config. Me and our colleagues have been chasing our tails for days until someone pointed this out to me. | 21:59 |
| flwang | andrein: the good reference is the doc from devstack I would say | 21:59 |
| colby_ | adrein: I had the same problem. Took me a while to get that set to True and get things working | 22:00 |
| flwang | but yes, we should document it well | 22:00 |
| flwang | we can change the default value to True | 22:00 |
| andrein | what about the security aspects? | 22:00 |
| andrein | as I said before, I understand why the default is false, but I would expect to be able to deploy a cluster, but not be able to use PVs backed by cinder or swift registry. instead, my cluster was effectively broken with all nodes tainted because the cloud-controller couldn't launch and untaint them (I assume) | 22:05 |
| flwang | andrein: could you please file a story on the story board so that we can track the issue deeper? | 22:13 |
| andrein | I will do that first thing tomorrow morning. | 22:14 |
| flwang | andrein: thank you, i appreciate that | 22:17 |
| flwang | andrein: pls feel free post your question here | 22:18 |
| andrein | https://bugs.launchpad.net/kolla-ansible/+bug/1842449 there's this issue in the kolla-ansible launchpad | 22:18 |
| openstack | Launchpad bug 1842449 in kolla-ansible "Magnum "enable_cluster_user_trust" documentation" [Undecided,New] | 22:18 |
| flwang | cool, thanks | 22:28 |
| *** threestrands has joined #openstack-containers | 22:37 | |
| colby_ | whats the highest kubernetes version that works with rocky? | 22:37 |
| andrein | flwang: https://storyboard.openstack.org/#!/story/2006531 | 22:45 |
| flwang | colby_: i think v1.13.x should work with rocky | 22:46 |
| flwang | better give it a try | 22:46 |
| flwang | andrein: thank you very much | 22:46 |
| andrein | now I'm really going to sleep. If I can be of any assistance, please let me know. | 22:46 |
| flwang | andrein: cheers | 22:47 |
| *** dtruong has quit IRC | 22:55 | |
| *** dtruong has joined #openstack-containers | 22:55 | |
| *** goldyfruit_ has quit IRC | 22:57 | |
| openstackgerrit | Merged openstack/magnum master: k8s_fedora_atomic: Add PodSecurityPolicy https://review.opendev.org/681013 | 23:12 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!