| *** ttsiouts has joined #openstack-containers | 00:52 | |
| *** ttsiouts has quit IRC | 01:25 | |
| *** ricolin has joined #openstack-containers | 01:58 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: [k8s][fcos] Fix docker storage https://review.opendev.org/718296 | 02:29 |
|---|---|---|
| *** k_mouza has joined #openstack-containers | 02:31 | |
| *** k_mouza has quit IRC | 02:35 | |
| *** ttsiouts has joined #openstack-containers | 03:22 | |
| *** vishalmanchanda has joined #openstack-containers | 03:53 | |
| *** ttsiouts has quit IRC | 03:56 | |
| *** ykarel|away is now known as ykarel | 04:24 | |
| *** udesale has joined #openstack-containers | 05:22 | |
| *** udesale has quit IRC | 05:23 | |
| *** udesale has joined #openstack-containers | 05:23 | |
| *** ttsiouts has joined #openstack-containers | 05:53 | |
| *** ttsiouts has quit IRC | 06:27 | |
| cosmicsound | good day | 06:40 |
| flwang1 | cosmicsound: 18:40 here ;) | 06:40 |
| cosmicsound | it seems there is a issue with addig prometheus on v1.18.0 coreos | 06:40 |
| cosmicsound | good evening flwang1 | 06:40 |
| cosmicsound | got these labels https://mdb.uhlhost.net/uploads/961096b6a643538c/image.png | 06:41 |
| flwang1 | cosmicsound: are you using montoring_enabled? or prometheus_montoring? | 06:41 |
| cosmicsound | if i add prometheus_monitoring=true will fail | 06:41 |
| cosmicsound | yes it is enabled | 06:41 |
| flwang1 | no | 06:41 |
| flwang1 | can you see the tiller job? | 06:42 |
| *** ttsiouts has joined #openstack-containers | 06:45 | |
| brtknr | flwang1: hii | 06:47 |
| flwang1 | brtknr: hello | 06:47 |
| flwang1 | brtknr: do you still have concern with https://review.opendev.org/710384 ? | 06:48 |
| brtknr | No I thought I +2 it | 06:49 |
| cosmicsound | flwang1 , the tiller job on k8s? | 06:50 |
| brtknr | flwang1: looks like I didn’t, dorryt | 06:50 |
| brtknr | sorry | 06:50 |
| flwang1 | cosmicsound: yes | 06:50 |
| cosmicsound | let me check | 06:51 |
| flwang1 | brtknr: and could you please review https://review.opendev.org/#/c/718296/ ? | 06:51 |
| flwang1 | brtknr: would you like to join today's meeting? | 06:52 |
| cosmicsound | NAMESPACE NAME COMPLETIONS DURATION AGE | 06:55 |
| cosmicsound | magnum-tiller job.batch/install-metrics-server-job 0/1 9h 9h | 06:55 |
| cosmicsound | magnum-tiller job.batch/install-prometheus-adapter-job 0/1 9h 9h | 06:55 |
| cosmicsound | magnum-tiller job.batch/install-prometheus-operator-job 0/1 9h 9h | 06:55 |
| cosmicsound | 8:55:26 [cosmic:~/Devstack/zun] % | 06:55 |
| cosmicsound | magnum-tiller replicaset.apps/tiller-deploy-cdf847b8c 1 1 1 9h | 06:55 |
| cosmicsound | magnum-tiller pod/install-metrics-server-job-wkg5j 1/1 Running 0 9h | 06:56 |
| cosmicsound | magnum-tiller pod/install-prometheus-adapter-job-ccpwr 1/1 Running 0 9h | 06:56 |
| cosmicsound | magnum-tiller pod/install-prometheus-operator-job-l8rqc 1/1 Running 0 9h | 06:56 |
| cosmicsound | magnum-tiller pod/tiller-deploy-cdf847b8c-7lvlx 1/1 Running 0 9h | 06:56 |
| cosmicsound | sry for the long paste | 06:56 |
| *** xinliang has joined #openstack-containers | 06:59 | |
| cosmicsound | flwang1 , should it be that it is either montoring_enabled or prometheus_montoring? | 07:00 |
| cosmicsound | not both together? | 07:00 |
| flwang1 | monitoring_enabled will need tiller to install the prometheus by helm | 07:01 |
| cosmicsound | right | 07:02 |
| cosmicsound | so in this case i am good | 07:02 |
| cosmicsound | since. the prometheur operator and adapter are there | 07:03 |
| brtknr | flwang1: yes I’ll be there | 07:14 |
| *** born2bake has joined #openstack-containers | 07:37 | |
| cosmicsound | flwang1 , seems tiller did not installed on coreos prometheus | 07:54 |
| cosmicsound | just made some pods to install it | 07:55 |
| cosmicsound | or i cant seem to find the ip of prometheus among services | 07:55 |
| brtknr | flwang1: anything to add to the agenda | 07:56 |
| cosmicsound | checked https://docs.openstack.org/magnum/latest/user/ kube_dashboard_version is not documented any idea why? | 08:03 |
| brtknr | cosmicsound: you're right, flwang1 you should document it in your patch: https://review.opendev.org/#/c/714021/2/magnum/drivers/k8s_fedora_coreos_v1/templates/kubecluster.yaml | 08:06 |
| *** dioguerra has joined #openstack-containers | 08:10 | |
| cosmicsound | kube_dashboard_version=v1.8.3 its only working seems i canot overwrite it with latest | 08:13 |
| cosmicsound | tried v2.0.0-rc7 | 08:14 |
| cosmicsound | still sees old one | 08:14 |
| brtknr | cosmicsound: 2 is not supported yet, the patch hasnt merged yet | 08:14 |
| brtknr | cosmicsound: btw how did you get magnum working in the end? | 08:15 |
| brtknr | are you using master branch? | 08:15 |
| cosmicsound | seems magnum -> magnum-base-source/magnum-9.1.0.dev212 did the job | 08:16 |
| cosmicsound | and also had to change from scsi in virtio | 08:16 |
| cosmicsound | since disks were not read properly | 08:16 |
| brtknr | what is magnum-base-source/magnum-9.1.0.dev212? | 08:17 |
| brtknr | kolla image? | 08:17 |
| cosmicsound | thats the deployment via master in kolla | 08:17 |
| cosmicsound | when i used train tag i had 9.2.0 | 08:17 |
| cosmicsound | with master i get this | 08:17 |
| brtknr | cosmicsound: hmm interesting | 08:19 |
| *** rcernin has quit IRC | 08:27 | |
| *** xinliang has quit IRC | 08:32 | |
| cosmicsound | brtknr , if i updated the docker of magnum with the patched version for latest kubernetes dashboard | 08:44 |
| cosmicsound | should i restart the container to take effect or? | 08:45 |
| cosmicsound | brtknr , does magnum needs a separate cert manager like https://cert-manager.io/docs/ or it handles it all in barbican? | 08:48 |
| cosmicsound | i see a cert_manager_api yet not sure on it | 08:49 |
| brtknr | cosmicsound: yes restart should do it | 08:52 |
| *** ykarel is now known as ykarel|lunch | 08:53 | |
| cosmicsound | I will document now all labels that work | 08:55 |
| cosmicsound | and will share them maybe we can update a better compatibility matrix | 08:55 |
| cosmicsound | with main most used labels out there and their counterpart services | 08:55 |
| cosmicsound | this i found big mess in past to match right versions maybe for me was also the way i used to have the scsi issues yet anyhow it could be improved | 08:56 |
| flwang1 | strigazi: are you around? | 08:59 |
| flwang1 | brtknr: seems we don't have strigazi today | 09:00 |
| flwang1 | brtknr: are you around? | 09:00 |
| brtknr | flwang1: yes im here | 09:01 |
| flwang1 | let's cancel the meeting and we can have a casual discussion | 09:01 |
| brtknr | no problem | 09:02 |
| flwang1 | brtknr: anything you want to discuss? | 09:03 |
| brtknr | flwang1: yesterday we were discussing the specs: http://eavesdrop.openstack.org/irclogs/%23openstack-containers/%23openstack-containers.2020-04-07.log.html#t2020-04-07T12:07:50 | 09:04 |
| brtknr | the label one | 09:04 |
| flwang1 | ok, what's your idea? | 09:05 |
| flwang1 | did i miss a comment from your side on that spec? | 09:05 |
| brtknr | flwang1: just wondering if you are okay with a flag? | 09:05 |
| flwang1 | what's the flag? | 09:06 |
| brtknr | i will pass on your thoughts to ttsiouts when he comes back later | 09:06 |
| brtknr | instead of having a secondary labels field | 09:07 |
| brtknr | e.g. --override-labels flag if you want to merge labels provided on cluster scope with labels provided in cluster template scope | 09:08 |
| brtknr | it would default to false to emulate the current behaviour | 09:08 |
| brtknr | but if --override-labels flag is provided, the server updates the cluster template labels with cluster labels and cluster labels with nodegroup labels | 09:09 |
| brtknr | flwang1: make sense? | 09:09 |
| flwang1 | did you see my comments in https://review.opendev.org/#/c/621611/ ? | 09:10 |
| brtknr | which comments? there are many :) | 09:11 |
| flwang1 | see my first one | 09:11 |
| cosmicsound | what tags do i need to update cluster nodes? | 09:11 |
| *** k_mouza has joined #openstack-containers | 09:11 | |
| cosmicsound | when i try to update from 1 nnode to 3 it says it cannot via horizo | 09:11 |
| cosmicsound | should i use cli? | 09:11 |
| cosmicsound | or i need some labels for this to work | 09:11 |
| flwang1 | cosmicsound: you can use resize command | 09:12 |
| flwang1 | or the latest magnum ui can do that as well | 09:12 |
| brtknr | flwang1: with the label_merging_policy? | 09:12 |
| flwang1 | brtknr: yep, it's basically samething as the --override-labels you mentioned above | 09:12 |
| brtknr | yes, it is a similar concept except its a first class parameter rather than a label and handled on the server side | 09:13 |
| flwang1 | but the policy can be vary to support different scenarios | 09:13 |
| flwang1 | no it's a label | 09:13 |
| flwang1 | but that's not the key point | 09:13 |
| flwang1 | i mean I'm ok with that direction | 09:13 |
| brtknr | ok cool | 09:13 |
| flwang1 | brtknr: pls review https://review.opendev.org/#/c/718296/ when you have time, thanks | 09:14 |
| flwang1 | anything else? | 09:15 |
| brtknr | flwang1: are you okay with an opt-in --override-labels to support the new behaviour | 09:15 |
| flwang1 | i will have to offline earlier today cause we just done a big production release today | 09:15 |
| brtknr | e.g. providing --override-labels without supplying --labels at the same time would throw an API error | 09:16 |
| brtknr | flwang1: i will review https://review.opendev.org/#/c/718296/ this friday, I have to prepare for a presention tomorrw | 09:16 |
| brtknr | flwang1: whats the production release? | 09:17 |
| brtknr | i tried to create an account on catalyst cloud but didnt get accepted hehe | 09:17 |
| flwang1 | brtknr: we're upgrading heat to latest stable version and a horzion release | 09:17 |
| flwang1 | brtknr: heh, really? just try again ;) | 09:17 |
| flwang1 | why we need an 400 when both labels provided? | 09:18 |
| brtknr | providing --override-labels when there is nothing to override with... thats why | 09:19 |
| brtknr | if --override-labels is not provided, it evaluates to false in DB | 09:19 |
| brtknr | the migration also defaults this field to false in the DB | 09:20 |
| flwang1 | hold on | 09:20 |
| brtknr | and the field is only evaluated when --override-labels is provided | 09:21 |
| flwang1 | i'm a bit confused | 09:21 |
| brtknr | the difference with idea in the spec is that --override-labels is a boolean rather than a dict | 09:21 |
| flwang1 | what's the flag you're talking about | 09:21 |
| flwang1 | i know the override_labels are the new filed we need to introduce | 09:21 |
| flwang1 | what's the flag then? | 09:21 |
| brtknr | in the CLI, this would be --override-labels | 09:22 |
| dioguerra | the default behaviour should occur when --override-labels is True then. This label name seems that its enabling what the current --labels do | 09:22 |
| brtknr | dioguerra: override==merge==combine==update | 09:23 |
| brtknr | so default behaviour is override=False | 09:23 |
| brtknr | so default/current behaviour is override=False | 09:24 |
| flwang1 | and the new --over-labels is a label or just a client side flag? | 09:24 |
| brtknr | its a client side flag which tells the server what to do with the provided labels | 09:25 |
| brtknr | the labels do not get combined on the client side, | 09:25 |
| brtknr | this is different to the spec ttsiouts has currently up but he is planning to update it | 09:26 |
| flwang1 | then, when it's a merging, how can we know what's the lables user passed or we don't care? | 09:26 |
| brtknr | but not if you disagree with the idea :) | 09:26 |
| brtknr | the user passed labels are stored under the current labels field | 09:27 |
| flwang1 | i would like to see it's proposed as the alternative solution in the spec | 09:27 |
| brtknr | but the override_labels boolean tells the server what to do with the labels | 09:27 |
| openstackgerrit | Tobias Urdin proposed openstack/python-magnumclient master: Check response type in _extract_error_json https://review.opendev.org/718356 | 09:27 |
| flwang1 | no, i mean the original labels user passed in | 09:27 |
| brtknr | whether to merge it or to relace it | 09:27 |
| flwang1 | i understand this idea now | 09:28 |
| brtknr | :) | 09:28 |
| brtknr | great! | 09:28 |
| flwang1 | i'd like to see it's proposed in the spec and we can discuss it from there | 09:28 |
| brtknr | flwang1: sweet! | 09:28 |
| flwang1 | i know this one is simpler ;) | 09:28 |
| brtknr | but are you okay with this direction compared to adding a separate dict field? | 09:28 |
| flwang1 | i can't make the decision now. because i'd like to take this opportunity to get this issue done beautifully | 09:29 |
| flwang1 | i know we need to get this one done quickly for several reasons | 09:29 |
| *** ricolin_ has quit IRC | 09:29 | |
| brtknr | ok | 09:30 |
| flwang1 | i think we can make the decision by this week or early next week | 09:30 |
| *** ricolin_ has joined #openstack-containers | 09:30 | |
| flwang1 | sorry i can't give you the clear answer now | 09:30 |
| *** ricolin has quit IRC | 09:30 | |
| brtknr | ok thats fine | 09:30 |
| flwang1 | cheers, i have to go | 09:32 |
| brtknr | flwang1: | 09:32 |
| brtknr | 1 last thing | 09:32 |
| brtknr | can we cut another release with the zincati patch? | 09:32 |
| *** k_mouza has quit IRC | 09:32 | |
| brtknr | 9.3.0 is basically unusable without it | 09:32 |
| brtknr | for fedora coreos | 09:33 |
| flwang1 | brtknr: we have to | 09:33 |
| brtknr | 9.3.1 | 09:33 |
| flwang1 | i'm waiting for the gate fixed | 09:33 |
| brtknr | ? | 09:33 |
| brtknr | or 9.4.0? | 09:33 |
| brtknr | what gate fix? | 09:33 |
| flwang1 | at least 9.3.1 | 09:33 |
| flwang1 | py2 | 09:33 |
| flwang1 | https://review.opendev.org/#/c/717428/ | 09:34 |
| flwang1 | hopefully it can be fixed by https://review.opendev.org/#/c/718130/ | 09:34 |
| flwang1 | will see | 09:34 |
| *** k_mouza has joined #openstack-containers | 09:35 | |
| brtknr | flwang1: okay | 09:35 |
| brtknr | cool thanks for the updat | 09:35 |
| brtknr | speak soon! | 09:36 |
| flwang1 | brtknr: :) cheers, take care my friend | 09:36 |
| brtknr | flwang1: you too :) goodnight | 09:36 |
| *** flwang1 has quit IRC | 09:41 | |
| openstackgerrit | Tobias Urdin proposed openstack/python-magnumclient master: Check response type in _extract_error_json https://review.opendev.org/718356 | 09:47 |
| *** ricolin_ has quit IRC | 09:57 | |
| *** ricolin_ has joined #openstack-containers | 09:57 | |
| *** ricolin has joined #openstack-containers | 10:00 | |
| *** ricolin_ has quit IRC | 10:02 | |
| *** ykarel|lunch is now known as ykarel | 10:12 | |
| *** dioguerra has quit IRC | 10:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/magnum-ui master: Imported Translations from Zanata https://review.opendev.org/718380 | 10:30 |
| *** ricolin has quit IRC | 10:39 | |
| *** k_mouza has quit IRC | 10:47 | |
| *** k_mouza has joined #openstack-containers | 10:55 | |
| born2bake | cosmicsound did you deploy your cluster with lb? | 11:15 |
| cosmicsound | born2bake , not yet | 11:16 |
| cosmicsound | now working on building lb | 11:16 |
| cosmicsound | do you have some hints/good guide for this | 11:16 |
| born2bake | not really... I need to figure out if octavia is deployed correctly first | 11:16 |
| cosmicsound | ok so you did not made it yet working in k8s | 11:21 |
| *** ttsiouts has quit IRC | 12:53 | |
| *** ykarel is now known as ykarel|afk | 12:56 | |
| *** ttsiouts has joined #openstack-containers | 12:57 | |
| *** ricolin has joined #openstack-containers | 12:57 | |
| cosmicsound | anyone managed login with kubelet config? Not enough data to create auth info structure. is what i get | 13:17 |
| *** udesale_ has joined #openstack-containers | 13:18 | |
| *** udesale has quit IRC | 13:20 | |
| *** ykarel|afk is now known as ykarel | 13:37 | |
| *** ttsiouts has quit IRC | 14:06 | |
| *** ttsiouts has joined #openstack-containers | 14:06 | |
| *** udesale_ has quit IRC | 14:46 | |
| *** ykarel is now known as ykarel|away | 15:04 | |
| tobias-urdin | anybody else seen a issue where /etc/os-collect-config.conf gets | 15:13 |
| tobias-urdin | [heat] | 15:13 |
| tobias-urdin | auth_url = http://public:5000/v3/v3 | 15:13 |
| tobias-urdin | fault endpoint, something is appending my http://public:5000/v3 endpoint | 15:13 |
| born2bake | brtknr hello, I ve tried calico version without lb and just 1 master-1node, its still failing :( https://seashells.io/v/27wRXmgg master https://seashells.io/v/qczaxkKa worker | 15:16 |
| tobias-urdin | hm heat stack says correct, auth_url: http://public:5000/v3 | 15:17 |
| *** Pimpek has joined #openstack-containers | 15:35 | |
| born2bake | brtknr thats weird that magnum sends failed message :/ and I just login into master... k8s-calico-coreos-7fdoejglxqvo-master-0 Ready master 36m v1.17.4 k8s-calico-coreos-7fdoejglxqvo-node-0 Ready <none> 30m v1.17.4 | 15:46 |
| brtknr | born2bake: sorry I cannot help today, got too much to do, will be back on Friday | 15:47 |
| born2bake | ok no worries :P | 15:47 |
| brtknr | you can add a list of issues to the etherpad | 15:47 |
| brtknr | but /var/log/heat-config is your best friend | 15:48 |
| brtknr | you will see the reason in there | 15:48 |
| brtknr | if the cluster is deployed, it is probably a configuration issue | 15:48 |
| brtknr | e.g. unsupported labe | 15:48 |
| brtknr | e.g. unsupported label | 15:48 |
| tobias-urdin | brtknr: do you know where os-collect-config gets content in /var/lib/os-collect-config/heat_local.json from? | 15:59 |
| *** ttsiouts has quit IRC | 16:00 | |
| cosmicsound | brtknr , about helm and tiller, is tiller running helm inside itself or using helm from the vm where hoster. in this case helm is not shipped in coreos | 16:22 |
| cosmicsound | magnum-tiller job.batch/install-metrics-server-job 0/1 164m 164m | 16:22 |
| cosmicsound | magnum-tiller job.batch/install-prometheus-adapter-job 0/1 164m 164m | 16:22 |
| cosmicsound | magnum-tiller job.batch/install-prometheus-operator-job 0/1 164m 164m | 16:22 |
| cosmicsound | i am not an expert just would say these jobs never completed | 16:23 |
| *** k_mouza has quit IRC | 18:46 | |
| *** k_mouza has joined #openstack-containers | 18:47 | |
| *** k_mouza has quit IRC | 18:59 | |
| *** k_mouza has joined #openstack-containers | 19:27 | |
| *** vishalmanchanda has quit IRC | 19:32 | |
| *** k_mouza has quit IRC | 19:36 | |
| *** k_mouza has joined #openstack-containers | 19:37 | |
| *** k_mouza has quit IRC | 19:41 | |
| Pimpek | hi everyone. I'm stuck creating a new kubernetes cluster | 20:01 |
| Pimpek | long story short, I have one controller and one compute node | 20:01 |
| Pimpek | log checks show that the reason for cluster creation failure is that neutron can't create the required port | 20:02 |
| Pimpek | but it's creating the kube-master on the controller node instead of the compute node | 20:03 |
| Pimpek | where the networking is different | 20:03 |
| Pimpek | spinning up regular kvm VMs works fine | 20:03 |
| Pimpek | am I missing something? how do I get it to create the kube-master on the compute node instead of the master? | 20:04 |
| *** k_mouza has joined #openstack-containers | 21:05 | |
| *** k_mouza has quit IRC | 21:06 | |
| *** k_mouza has joined #openstack-containers | 21:06 | |
| *** k_mouza has quit IRC | 21:19 | |
| *** k_mouza has joined #openstack-containers | 22:30 | |
| *** k_mouza has quit IRC | 22:31 | |
| *** rcernin has joined #openstack-containers | 22:32 | |
| *** born2bake has quit IRC | 22:51 | |
| *** threestrands has joined #openstack-containers | 23:10 | |
| openstackgerrit | Merged openstack/magnum-ui master: Imported Translations from Zanata https://review.opendev.org/718380 | 23:52 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
