Monday, 2020-04-20

*** threestrands has joined #openstack-kolla00:35
*** k_mouza has joined #openstack-kolla02:17
*** xinliang has joined #openstack-kolla02:18
*** k_mouza has quit IRC02:22
*** k_mouza has joined #openstack-kolla02:26
*** k_mouza has quit IRC02:33
*** k_mouza has joined #openstack-kolla02:38
openstackgerritya.wang proposed openstack/kolla-ansible master: Fix that cyborg agent failed to start privesp daemon.  https://review.opendev.org/72113902:38
*** k_mouza has quit IRC02:46
*** k_mouza has joined #openstack-kolla02:52
*** k_mouza has quit IRC02:57
*** k_mouza has joined #openstack-kolla03:01
*** kevinz has joined #openstack-kolla03:06
*** k_mouza has quit IRC03:08
*** k_mouza has joined #openstack-kolla03:15
openstackgerritya.wang proposed openstack/kolla-ansible master: Fix taht cyborg conductor failed to communicate with placement  https://review.opendev.org/72115803:15
*** k_mouza has quit IRC03:20
*** k_mouza has joined #openstack-kolla03:22
*** k_mouza has quit IRC03:28
*** k_mouza has joined #openstack-kolla03:31
*** k_mouza has quit IRC03:37
*** k_mouza has joined #openstack-kolla03:40
*** k_mouza has quit IRC03:45
*** ricolin has joined #openstack-kolla03:48
*** k_mouza has joined #openstack-kolla04:02
*** ykarel|away is now known as ykarel04:09
*** k_mouza has quit IRC04:10
*** xinliang has quit IRC04:11
*** k_mouza has joined #openstack-kolla04:16
*** k_mouza has quit IRC04:20
*** k_mouza has joined #openstack-kolla04:22
*** evrardjp has quit IRC04:23
*** k_mouza has quit IRC04:27
*** skramaja has joined #openstack-kolla04:29
*** k_mouza has joined #openstack-kolla04:38
*** k_mouza has quit IRC04:43
*** k_mouza has joined #openstack-kolla04:46
*** k_mouza has quit IRC04:50
*** k_mouza has joined #openstack-kolla04:56
*** cah_link has joined #openstack-kolla05:00
*** k_mouza has quit IRC05:02
*** k_mouza has joined #openstack-kolla05:13
*** k_mouza has quit IRC05:17
*** k_mouza has joined #openstack-kolla05:23
*** k_mouza has quit IRC05:28
*** k_mouza has joined #openstack-kolla05:33
*** k_mouza has quit IRC05:38
*** prernadembla05 has joined #openstack-kolla05:39
*** k_mouza has joined #openstack-kolla05:41
*** evrardjp has joined #openstack-kolla05:41
*** k_mouza has quit IRC05:45
*** vishalmanchanda has joined #openstack-kolla05:46
*** shyamb has joined #openstack-kolla05:48
osmanlicilegimorning05:49
*** k_mouza has joined #openstack-kolla05:51
*** k_mouza has quit IRC05:55
*** shyamb has quit IRC06:04
*** seco has joined #openstack-kolla06:04
*** shyamb has joined #openstack-kolla06:05
*** k_mouza has joined #openstack-kolla06:11
*** k_mouza has quit IRC06:16
*** numans has joined #openstack-kolla06:29
*** jbadiapa has joined #openstack-kolla06:49
yoctozeptomorning06:51
*** shyamb has quit IRC06:53
*** shyamb has joined #openstack-kolla06:53
*** k_mouza has joined #openstack-kolla06:56
*** k_mouza has quit IRC07:01
*** shyamb has quit IRC07:03
*** dougsz has joined #openstack-kolla07:05
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible master: Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821307:10
*** bengates has joined #openstack-kolla07:11
*** rpittau|afk is now known as rpittau07:12
*** hjensas has quit IRC07:13
*** hjensas has joined #openstack-kolla07:16
*** threestrands_ has joined #openstack-kolla07:18
*** threestrands_ has quit IRC07:18
*** bengates has quit IRC07:21
*** threestrands has quit IRC07:21
*** bengates has joined #openstack-kolla07:21
sri_morning07:21
*** k_mouza has joined #openstack-kolla07:31
*** spsurya_ has joined #openstack-kolla07:35
*** k_mouza has quit IRC07:36
mnasiadkamorning07:46
*** negronjl has quit IRC07:58
*** xinliang has joined #openstack-kolla08:00
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ansible lint related fixes  https://review.opendev.org/72119108:01
*** negronjl has joined #openstack-kolla08:01
*** xinliang has quit IRC08:12
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Manage nova scheduler workers count  https://review.opendev.org/72119808:13
*** ykarel is now known as ykarel|lunch08:17
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Add ansible-lint to tox  https://review.opendev.org/69477908:17
*** shyamb has joined #openstack-kolla08:23
mgoddardmorning08:29
sri_mnasiadka: hi , quick question,  Is ceph-ansible supported in ussuri release ?08:42
sri_mnasiadka: from kolla-ansible08:42
mnasiadkasri_: external ceph is supported in Ussuri release (no more kolla ceph deployment), you can use any deployment tool - we use ceph-ansible in the CI for testing08:43
*** shyam89 has joined #openstack-kolla08:45
sri_mnasiadka:  ack,  is there any way to tell  ceph-ansible to skip the docker install ? I've tried to " --skip-tags docker"  but it's not working,08:47
mnasiadkasri_: --skip-tags=with_pkg08:47
*** shyamb has quit IRC08:48
sri_mnasiadka: Thanks :)08:50
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Manage nova scheduler workers count  https://review.opendev.org/72119808:51
*** vishalmanchanda has quit IRC08:56
*** ykarel|lunch is now known as ykarel09:00
*** k_mouza has joined #openstack-kolla09:00
mgoddardthanks for the ML post yoctozepto. I'm going to let it sink for a while before replying09:03
hrwelo09:05
oklhostHi @all. Anybody facing issues with latest Google Chrome and "Launch Instance" button in horizon? The modal won't open...09:06
hrwoklhost: #openstack-horizon?09:08
oklhostthx09:08
*** shyam89 has quit IRC09:20
*** shyam89 has joined #openstack-kolla09:20
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible master: Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821309:25
yoctozeptomgoddard: thanks, I had your poc starred already :-)09:27
*** ricolin has quit IRC09:28
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: CI: add Debian source upgrade job on AArch64  https://review.opendev.org/71385409:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: CI: add Debian source upgrade job on AArch64  https://review.opendev.org/71385409:30
hrwrebased and dropped depends-on09:30
*** shyam89 has quit IRC09:32
*** shyamb has joined #openstack-kolla09:35
hrwyoctozepto: +2 on both python27 cleanups09:37
*** e0ne has joined #openstack-kolla09:42
*** shyam89 has joined #openstack-kolla09:45
*** shyamb has quit IRC09:48
*** ricolin has joined #openstack-kolla09:48
*** ricolin_ has joined #openstack-kolla09:48
yoctozeptohrw: thanks, k-a has to wait for fixing (and need to check why infra c8 images are far from vanilla - I love how introducing bugs reveals other bugs too)09:53
*** vishalmanchanda has joined #openstack-kolla09:53
*** gfidente has joined #openstack-kolla09:55
stingrayzahey all09:57
stingrayzamy k-a isn't picking up my nova.conf override in /etc/kolla/config/nova/nova.conf09:58
stingrayzaI'm doing a "kolla-ansible reconfigure --tags nova" - that should pick up the file and run with it, right?09:59
yoctozeptonova.conf or nova/nova-api.conf etc.10:00
yoctozeptomany folks seem to be mistaken, maybe we should just pick up either...10:01
*** ricolin_ has quit IRC10:02
stingrayzahttps://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration.html#openstack-service-configuration-in-kolla might need an update - that's perhaps the source of the confusion...?10:02
* stingrayza tries again with the file one level higher...10:04
*** e0ne has quit IRC10:05
*** e0ne has joined #openstack-kolla10:05
openstackgerritMark Goddard proposed openstack/kayobe master: Cleanup py27 support  https://review.opendev.org/72076410:09
*** e0ne has quit IRC10:12
*** e0ne has joined #openstack-kolla10:13
*** rpittau is now known as rpittau|bbl10:15
openstackgerritya.wang proposed openstack/kolla-ansible master: Fix that cyborg conductor failed to communicate with placement  https://review.opendev.org/72115810:25
*** shyam89 has quit IRC10:33
openstackgerritya.wang proposed openstack/kolla-ansible master: Fix that cyborg agent failed to start privsep daemon.  https://review.opendev.org/72113910:48
stingrayzathanks yoctozepto - moving it 'up' one dir worked. I'm gonna log a bug for a fix (and take it)10:50
stingrayzalogged (https://bugs.launchpad.net/kolla-ansible/+bug/1873782)10:51
openstackLaunchpad bug 1873782 in kolla-ansible "nova.conf override not used from documented location" [Undecided,New] - Assigned to Raimund Hook (stingrayza)10:51
*** e0ne has quit IRC10:51
*** e0ne has joined #openstack-kolla10:52
stingrayzahappy to fix this, - the question is do I just do a quick doc clean-up, or do we fix the conf pick up to additionally pick up "{{ node_custom_config }}/nova/nova.conf" ? (and if the latter, then do I do that for _all_ services?)10:54
*** dougsz has quit IRC11:02
mgoddardstingrayza: I think typically we don't support that path, although there may be exceptions. Worth checking a few to be sure, but I'd update docs11:08
stingrayzamgoddard: cool. doc update inbound :)11:10
*** dougsz has joined #openstack-kolla11:17
hrwmeh.11:18
hrwI look at kolla-toolbox and while it can be moved to use binary packages for most of stuff the problem stays due to ubuntu shipping ancient ansible ;(11:19
hrwdebian:buster has 2.9.6 in backports, centos has 2.9.7 in some repos11:20
hrwand it would also allow to drop compiler from it11:21
hrwor we add https://launchpad.net/~ansible/+archive/ubuntu/ansible and use binary package 2.9.x11:23
openstackgerritRaimund Hook (stingrayza) proposed openstack/kolla-ansible master: Update Advanced Config guide to clarify paths  https://review.opendev.org/72124111:25
mgoddardhrw: true, but ansible doesn't require upper constraints11:27
*** hjensas has quit IRC11:28
*** sean-k-mooney has joined #openstack-kolla11:29
hrwmgoddard: and we want 2.8+ while bionic has 2.511:31
mgoddardhrw: yeah, I'm saying we could just pip install ansible11:31
mgoddardhrw: will still be an improvement over today11:32
hrwmgoddard: and then we have to deal with virtualenv paths so have to install all python stuff in venv. which means building them. installing compilers, headers...11:32
mnasiadkahrw: well, using the ppa is not bad I guess, especially that it seems Red Hat is maintaining it?11:32
mgoddardhrw: we wouldn't use a virtualenv for binary, install ansible to system11:32
hrwmgoddard: kolla-toolbox is infra image.11:33
mgoddardhrw: not yet :)11:33
hrwit is. just not called that11:33
mgoddardbut I see your point11:33
hrwnormal/debian-binary-kolla-toolbox                  9.1.0               0d7a331a881b        29 minutes ago      979MB11:37
hrwbinaryonly/debian-binary-kolla-toolbox              9.1.0               637522ed9bf9        5 seconds ago       619MB11:37
hrw360MB11:37
*** e0ne has quit IRC11:43
*** e0ne has joined #openstack-kolla11:43
hrwINFO:kolla.common.utils.kolla-toolbox:Setting up ansible (2.9.7-1ppa~bionic) ...11:43
hrwbinaryonly/ubuntu-binary-kolla-toolbox              9.1.0               d73aed9b0b32        29 seconds ago       648MB11:44
*** dking has joined #openstack-kolla11:48
*** Fl1nt has joined #openstack-kolla11:50
Fl1ntHi everyone!11:51
Fl1ntHow are you?11:51
hrwmgoddard: hm. what for we needed python-mysqlclient in k-a?11:52
dkingGood morning/evening, Fl1nt. I'm just getting started for the day early.11:52
hrwmgoddard: it is the only thing missing in centos ;d11:52
mgoddardhrw: pass11:52
mgoddardhrw: https://docs.ansible.com/ansible/latest/modules/mysql_db_module.html11:53
mgoddarduses PyMySQL11:53
Fl1nt@dking, it's the afternoon in here ^^11:53
hrwgracias11:53
dkingFl1nt: Then, that might make it easier for in this channel. I find that folks are usually going away about the time that I get started for the day.11:55
hrw~curse rpm distros for PacKAgeSwIThuPPerCAseLETTersInNamES11:56
hrwFl1nt, dking: google for UGT.11:57
hrwUniversal Greeting Time11:57
dkingKayobe question: So, I'm a bit more familiar with the configuration now. However, when I mentioned before that I was wanting multiple isolated overclouds, I may have forgotten to mention that I was wanting those overclouds to be able to be unique. It looks like the configuration here implies that they are all the same kolla-ansible setup? Would that mean that I would have to deploy them as baremetal servers instead?11:57
Fl1nt@dking, that's for sure as many of the active members are actually located un europe.11:58
hrwbinaryonly/centos-binary-kolla-toolbox              9.1.0               321b14f10ad7        48 seconds ago      769MB11:58
*** e0ne has quit IRC11:58
dkinghrw: That's pretty useful. I suppose that it's a different world these days. I think I'll adopt that.11:58
Fl1nt@dking, you need to setup one repository per overcloud that you want to deploy11:59
hrwdking: it simplifies greetings a lot11:59
mgoddarddking: Fl1nt is right - use a separate kayobe-config repo/branch per environment12:00
dkingIs that one kayobe-config directory per overcloud?12:00
mgoddarddking: yes12:00
dkingOkay. That might work, then.12:00
mgoddarddking: we have a feature proposed to support multiple envs in one, but haven't got there yet12:00
Fl1ntthat's simplier, cleaner and except if you're Google's sized, you shouldn't have hundreds of overclouds ^^12:00
hrwmgoddard: now kolla-toolbox has few layers less too due to all RUN calls done at once12:01
dkingAnother question, then. I see that it mentions how to setup a bond, and then how to setup VLAN. Each of my servers have two NICs that need to be bonded, and then will use different VLANs on that bond. Would I setup a dummy interface to make the bond, and then set the VLANs for the actual *_interface?12:02
mgoddardhrw: nice12:02
dkingFl1nt: Actually, even though we're no where near Google sized, we do plan to have hundreds of overclouds. We're looking at doing many smaller isolated overclouds.12:03
*** rpittau|bbl is now known as rpittau12:03
mgoddarddking: it depends if you want an IP on the native VLAN. If so, that network will define the bond. Otherwise you'll need a dummy to represent the bond12:04
hrwmgoddard:  5 files changed, 35 insertions(+), 64 deletions(-)12:04
mgoddarddking: another thing to consider - for OVS we currently require a bridge, and kayobe will plumb a veth pair from that bridge to the OVS bridge12:04
dkingmgoddard: Do you have some info on that project? I won't need it starting out, but if we do end up going with Kayobe, that's something we would likely be interested in.12:05
mgoddarddking: this patch allows you to plug an interface straight into OVS: https://review.opendev.org/70503712:05
patchbotpatch 705037 - kayobe - Allow OVS bridges to connect directly to interface - 6 patch sets12:05
Fl1nt@dking, ok depending on you constraints regarding isolation/security I do understand that plan, however, I can advise you from experience, that using that much spread isn't going to be easy to manage for operators, you would likely rather have a giant overcloud with segmentation12:05
Fl1ntusing12:05
Fl1ntcell/zone/region/aggregates12:05
mgoddarddking: here's the story for multiple envs: https://storyboard.openstack.org/#!/story/200200912:06
dkingThat's great! I'm sure that we'll be looking at that a little more soon.12:10
dkingFl1nt: Yes, that's true. But we're actually wanting to sell overclouds. And yes, that is going to be a lot to manage, but that's part of the new direction.12:12
Fl1nt@dking, oh ok, I do understand your business case, make sense and that's actually easier to manager with one/x overcloud splitted per customer as you can use your repository project/subproject tree to cleanly store them.12:14
dkingmgoddard: I'm having a bit of trouble understanding the part about OVS. Once we get the overclouds up, we won't want them using the Ansible or seed hosts for the networking, so would that still apply? Of course, the overcloud controllers will eventually be setting up their own OVS for Neutron.12:14
mgoddarddking: I'm referring to the OVS networking within a cloud. No seed involved. It's just about how the network interfaces need to be wired up12:16
dkingFl1nt: Yes, that's one way. We've actually been tossing around ideas about different deploy methods, and there's even some consideration given to using VMs for deploy. There will be an entire separate system for managing the details, regardless how we implement the deploy.12:16
mgoddarddking: with a bond, you'll want physical -> bond -> bridge12:16
mgoddardthen kayobe will plumb a veth from that bridge to OVS12:16
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: use Ansible 2.9+ binary packages  https://review.opendev.org/72125112:17
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: kolla-toolbox: use only binary packages  https://review.opendev.org/72125212:17
hrwlet CI test was I right ;d12:17
Fl1nt@dking, interesting ^^ let us know about how things evolves ^^12:18
dkingmgoddard: At the moment, the physical networks are connected at the switch using port groups.12:18
mgoddarddking: when I say bridge, I mean a linux bridge not hardware12:19
dkingUp to this point, I don't think that I've been using a bridge, other than the bridges that Neutron sets up. This is a way that Kayobe handles networking while deploying, and this patch allows us to bypass that because we don't need an IP on the native VLAN?12:27
Fl1nt@dking, even a bond don't need an IP, only your vnic need one.12:30
dkingI'm going to be needing IPs on the various VLAN interfaces on the bond, corresponding to the various networks. I just won't need an IP on the native VLAN.12:32
yoctozeptomgoddard, hrw: I see kolla-toolbox relying on ansible as a kind of accidental complexity; I mean, what kolla-toolbox does is perfectly doable without the weaponry of ansible12:32
Fl1ntyeah that's ok, just set up your vlan interfaces.12:32
hrwyoctozepto: ansible/roles/common/handlers/main.yml:  command: docker exec -t "{{ common_services['kolla-toolbox']['container_name'] }}" ansible --version12:35
yoctozeptostingrayza: for the love of kolla, I did not notice there was this outrageous mistake in the docs, yikes12:35
hrwyoctozepto: so it is used :D12:35
yoctozeptohrw: yeah, it is used12:35
yoctozeptohrw: I meant to say it does not have to be by design12:35
hrwyoctozepto: you know that I try to stay away from k-a as much as possible ;D12:35
yoctozeptohrw: don't drop ansible right away :D12:35
yoctozeptohrw: sure, I know12:36
hrwyoctozepto: I do not plan to drop it D:12:36
dkingOkay. Another question. I've got the seed deployed and running. I'm going to be trying to set up a single control node in the overcloud to test things out. Because we're still using the switch for some other things at the moment, I'm not going to be using discover, but manually adding the server with kolla_bifrost_servers. Would I still need to make entries for the physical network? At the moment, my next step is real12:37
dkingsee the node available in bifrost.12:37
dking(or technically, even manageable, for the moment)12:38
*** Luzi has joined #openstack-kolla12:42
*** priteau has joined #openstack-kolla12:50
stingrayzayoctozepto: hopefully that will fix the 'many folks seem to be mistaken' :)12:55
*** ykarel is now known as ykarel|afk12:56
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: Improve CI ansible rules to pass ansible-lint  https://review.opendev.org/72066413:02
hrwimproved commit message a bit13:02
*** irclogbot_0 has quit IRC13:06
*** irclogbot_1 has joined #openstack-kolla13:07
yoctozeptostingrayza: hahaha, yeah13:08
*** kevinz has quit IRC13:09
*** kevinz has joined #openstack-kolla13:14
hrwSwitched to a new branch 'wip/train-use-c-r-o-t'13:16
hrwnow just have to wait for c-r-o-t package to appear ;d13:16
*** skramaja has quit IRC13:17
*** dswebb has joined #openstack-kolla13:22
dswebbHi all, quick one.  just saw that OVN support has been merged into master, is there any plan to back port this to train or will it be a ussari only feature?13:23
kplantusually features aren't backported, only fixes; i'll let someone else confirm13:24
yoctozeptodswebb: ussuri only13:28
yoctozepto(and forward of course)13:28
dswebbperfect, cheers13:42
*** TrevorV has joined #openstack-kolla13:50
*** prernadembla05 has quit IRC13:52
mgoddarddking: to be honest I haven't used kolla_bifrost_servers in a long time. You could give it a try, or if you are familiar with ironic just enroll the node manually13:52
mgoddarddking: what do you mean by 'make entries for the phyiscal network'? As in switches?13:53
-openstackstatus- NOTICE: Zuul is temporarily offline; service should be restored in about 15 minutes.13:59
*** hongbin has joined #openstack-kolla14:21
*** ykarel|afk is now known as ykarel14:22
*** cah_link has quit IRC14:27
*** cdearborn has joined #openstack-kolla14:34
dmsimardyoctozepto: btw, reversed the truncate for improved relevantness https://api.trunk.demo.recordsansible.org/14:42
openstackgerritMark Goddard proposed openstack/kayobe master: Fix passwords.yml generation with vault encryption on Python 3  https://review.opendev.org/72128114:43
yoctozepto dmsimard: yay, looks way better, thanks14:44
openstackgerritMark Goddard proposed openstack/kayobe master: Cleanup py27 support  https://review.opendev.org/72076414:47
*** jimcrowleyibm has joined #openstack-kolla14:49
Fl1nt@mgoddard, so, I've just stumble upon a funny side effect of k-a that plan to not deploy/manage ceph anymore in futur release.14:50
Fl1ntas our infrastructure is "hyperconverged" we've all our openstack host to install ceph using ceph-ansible with a containerized deployment.14:51
Fl1ntC-A use python-docker-py package from YUM (installed system wide) and K-A use docker pip package.14:52
Fl1ntboth packages conflict14:52
Fl1ntas python-docker-py take precedence over docker pip package14:53
openstackgerritMark Goddard proposed openstack/kayobe master: Revove support for CentOS 7 and Python 2  https://review.opendev.org/72081814:53
Fl1ntwhen performing a precheck step your deployment fail as docker.__version__ is 1.10.6 instead of > 2.x14:53
*** hjensas has joined #openstack-kolla14:54
openstackgerritMark Goddard proposed openstack/kayobe master: Remove support for CentOS 7 and Python 2  https://review.opendev.org/72081814:54
mgoddardFl1nt: :( haven't come across that one before14:55
Fl1ntsooo, all in all, I'll fill an issue with C-A as they don't use venv and just rely on YUM by assuming they're the only citizen in town, which isn't really gentle at all ^^14:55
mgoddardpython-docker-py is pretty old, sure there is no way to configure it?14:55
Fl1ntunfortunately not, it's hardcoded within the prerequisite role14:55
yoctozeptoFl1nt: k-a can use virtualenv14:56
mgoddardFl1nt: you can use a virtualenv for k-a14:56
Fl1ntit need a patch from C-A tbh14:56
Fl1ntI do14:56
yoctozeptomgoddard: agreed then14:56
mgoddardFl1nt: for remote hosts?14:56
yoctozeptoFl1nt: on the target hosts14:56
yoctozeptopff14:56
yoctozepto:D14:56
mgoddardyoctozepto: too slow14:56
Fl1nthehehe let me check.14:56
mgoddardI assume yoctozepto is an echo bot that just rephrases what I say14:56
yoctozeptomgoddard: nah, I was second faster on the first statement14:57
mgoddardhmm, right14:57
mgoddardtime travelling echo bot?14:57
yoctozeptocount me in!14:57
Fl1nthum... ok, seems we don't use venv, which vars define that ?14:58
Fl1ntthat could do the trick14:58
Fl1ntbut yet14:58
Fl1ntC-A shouldn't assume to be the only one to live in host especially when dealing with containerized installation.14:58
yoctozeptoFl1nt: it shouldn't14:58
yoctozeptoFl1nt: but it's not our baby :-)14:59
Fl1ntyep14:59
sri_Fl1nt: I've faced that issue when i am deploying c-a and k-a in the same nodes14:59
yoctozeptoyou can go there and tell folks kolla boys are doing it right ;D14:59
Fl1ntok, I'll be back in few hours, happy to propose something regarding ceph as containers images are pretty solid.14:59
Fl1nt@sri_, yep, that's the scenario.15:00
sri_the quick fix found is pip uninstall all docker packages, exp: docker, docker-py, docker-*, then run the pip install docker15:01
mgoddardFl1nt: https://docs.openstack.org/kolla-ansible/latest/user/virtual-environments.html15:01
mgoddardFl1nt: kayobe does this by default, which might explain why I haven't seen it15:01
kplant^ that's how i run my clouds15:04
kplantmuch safer imo15:04
openstackgerritMark Goddard proposed openstack/kayobe master: Remove nameservers with any IP in overcloud resolv.conf workaround  https://review.opendev.org/72128515:06
sri_mgoddard: this looks more safer and cleaner, thanks for link :)15:07
dmsimardmgoddard: btw during the weekend I sent https://review.opendev.org/#/c/720902/ to test the newest release of ara, this is the data we've got across all jobs: https://api.trunk.demo.recordsansible.org/?name=&path=kolla-ansible&label=15:07
patchbotpatch 720902 - kolla-ansible - DNM: Test ara 1.4 with kolla-ansible - 1 patch set15:07
dmsimardthe equivalent in API form: https://api.trunk.demo.recordsansible.org/api/v1/playbooks?path=kolla-ansible15:07
*** born2bake has joined #openstack-kolla15:07
*** dougsz has quit IRC15:09
mgoddarddmsimard: neat, looks nice!15:09
dkingmgoddard: Sorry, I was away. Yes, I meant switches.15:18
*** ykarel is now known as ykarel|away15:18
mgoddarddking: ok, that's all optional - kayobe doesn't assume it manages them. You can still run discovery15:18
mgoddarddking: the requirements are the same as for provisioning - PXE boot15:19
mgoddarddking: assume your bonds have some fallback config for PXE?15:19
dkingNo, PXE only understands unbonded, untagged. I was assuming that Kayobe would be creating the bonds/VLANs after booting from PXE.15:20
*** well100 has quit IRC15:21
mgoddarddking: I mean on the switch side15:22
mgoddardhrw, yoctozepto, mnasiadka, osmanlicilegi: let's land this thing: https://review.opendev.org/#/c/713283/ https://review.opendev.org/#/c/713282/15:26
patchbotpatch 713283 - kolla-ansible - Remove support for Python 2 on the host - 5 patch sets15:26
patchbotpatch 713282 - kolla-ansible - Remove support for CentOS 7 - 2 patch sets15:26
dkingYes, they should be able to handle the packets without a bond for PXE. We have a separate switch for the BMC network and ports. The servers can also receive untagged packets to a single interface.15:26
*** vishalmanchanda has quit IRC15:26
mgoddarddking: ok, should be fine15:26
*** hongbin has quit IRC15:28
hrwmgoddard: next step: remove easy_install15:31
mgoddardhrw: go for it :)15:31
hrw+2 on both15:32
cosmicsounddmsimard , will ara be included in kolla-ansible?15:37
mgoddardhrw: thanks15:37
dmsimardcosmicsound: define included ? it's enabled in CI but other than that I am not aware15:38
dmsimardlike there could be a kolla container with the ara API server in it maybe ?15:38
cosmicsoundyep, thats what i meant15:38
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: Drop handling of 'easy_install'  https://review.opendev.org/72129015:40
dmsimardcosmicsound: it's not on my personal to-do list at this time but if anyone wants to do it I can point them in the right direction15:40
dmsimardfwiw I am testing container image releases, this is the first one: https://hub.docker.com/r/recordsansible/ara-api15:40
dmsimardbuilt from https://review.opendev.org/#/c/687905/5/contrib/containers/ara-api-container-source.sh15:41
patchbotpatch 687905 - recordsansible/ara - Add scripts to build and run ARA API containers - 5 patch sets15:41
dkingGreat. I'm giving it a try. I'm receiving an error for the overcloud node deploy because kolla_external_vip_interface is not set. kolla_external_vip_interface is only set when {{ public_net_name }} is in network_interfaces. network_interfaces seems like it should include public_net_name. I'm trying to track things back, but if anybody has a hint at what I might have left out, I would appreciate it.15:42
*** happyhemant has joined #openstack-kolla15:45
-openstackstatus- NOTICE: Gerrit will be restarted to correct a misconfiguration which caused some git mirrors to have outdated references.15:47
mgoddarddking: public_net_name should be in network_interfaces if the host is in the 'network' group15:48
dkingYes, I'm seeing that. I'm trying to dump variables. I see that groups['network'] contains the host.15:53
dkingOdd. controller_network_host_network_interfaces is empty.15:53
*** hjensas has quit IRC15:54
openstackgerritMerged openstack/kolla master: Cleanup py27 support  https://review.opendev.org/72075115:55
dkingSo, for some reason, this: "{{ ([public_net_name, tunnel_net_name] + external_net_names) | reject('none') | unique | list }}" is empty, even though public_net_name is "external". I'm dumping that just a few lines after I print out public_net_name.15:56
*** cah_link has joined #openstack-kolla15:56
*** KeithMnemonic has joined #openstack-kolla16:00
openstackgerritMark Goddard proposed openstack/kayobe master: CentOS 8: separate kolla build tag from deploy tag  https://review.opendev.org/71213416:00
*** bengates has quit IRC16:02
*** bengates has joined #openstack-kolla16:02
mgoddarddking: btw, I don't read all IRC logs, so if you don't mention me by name I might miss your message16:03
*** hongbin has joined #openstack-kolla16:05
mgoddarddking: that is quite confusing16:05
mgoddarddking: have you discovered kayobe configuration dump?16:05
*** bengates has quit IRC16:07
*** rpittau is now known as rpittau|afk16:07
dkingThanks for the tip. I don't know that I've used that yet. I've just been dumping stuff from Ansible.16:08
dkingI think it has something to do with those filters, but I don't understand why yet.16:08
*** seco has quit IRC16:09
openstackgerritMerged openstack/kolla master: Zun: add CNI loopback binary to the image  https://review.opendev.org/72104416:10
*** Luzi has quit IRC16:15
dkingOh, it was my fault! I must have reverted things somewhere. I ended up setting external_net_name as a list rather than actually setting external_net_names.16:18
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Remove support for Python 2 on the host  https://review.opendev.org/71328316:18
openstackgerritMerged openstack/kolla-ansible master: Fix kibana deployment with openstack_cacert unset  https://review.opendev.org/70909116:19
openstackgerritMerged openstack/kayobe stable/train: Use upper constraints when installing Tenks  https://review.opendev.org/72108216:20
dkingNow, I'm receiving an error about kolla_internal_vip_address being invalid for localhost. It seems that internal_net_name is empty (""), even though networks.yml has: public_net_name: external16:29
dking...I meant, I also have: internal_net_name: cloud16:30
dkingmgoddard: Does networks.yml get picked up for localhost also?16:34
*** evrardjp has quit IRC16:35
*** evrardjp has joined #openstack-kolla16:35
*** jbadiapa has quit IRC16:37
openstackgerritJames Kirsch proposed openstack/kolla master: Add stunnel container for Glance service  https://review.opendev.org/72085016:41
*** cah_link has quit IRC16:45
*** k_mouza has quit IRC16:46
*** gfidente is now known as gfidente|afk16:53
*** kklimonda has quit IRC16:54
*** kklimonda has joined #openstack-kolla16:55
openstackgerritMerged openstack/kolla-ansible master: Remove support for CentOS 7  https://review.opendev.org/71328216:57
sean-k-mooneymgoddard: quick question. i have just deployed an all in one deployment with train-centos8 using master kolla-ansible and it worked fine. but i was wondering if i want to enable ironic can i add it to the same node and have both nova/libvirt and ironic deployed on the same host17:00
sean-k-mooneyim not sure if i want/need ironic in my home deployment but it would be nice to be able to proviosu my other 4 servers too even though i will have those off 99% of the time17:02
sean-k-mooneyso i was just wondering if kolla support colocating nova-ironic and nova-libvirt on the same system17:04
sean-k-mooneyif not ill just create a vm and install ironic in that if i need it in the future17:05
*** seco has joined #openstack-kolla17:06
*** e0ne has joined #openstack-kolla17:07
*** seco has quit IRC17:12
*** e0ne has quit IRC17:16
*** priteau has quit IRC17:19
*** ricolin has quit IRC17:29
*** portdirect has quit IRC17:30
*** portdirect has joined #openstack-kolla17:30
yoctozeptosean-k-mooney: yes, nova-compute-ironic will be deployed on controllers by default17:35
sean-k-mooneyyoctozepto: right but in a deployment with only 1 host can you deploy two nova compte conatienr on the same host with different config17:37
sean-k-mooneyone useing ironic virt driver an the other using libvirt17:37
sean-k-mooneywhen i tried to do this 2-3 release ago you could not17:37
sean-k-mooneye.g. you need a minium of 2 hosts if you wanted 2 virt dirvers17:37
sean-k-mooneydeploying the ironic serivce will work but i dont think kolla-ansible has support for deploying both compute agents on the same host17:38
yoctozeptosean-k-mooney: our CI does it so it should work17:38
yoctozeptoit runs both libvirt and ironic on same nodes17:38
sean-k-mooneyso a singel vm with both nova libvirt and nova ironic deployed17:39
sean-k-mooneyok i might try that17:39
yoctozeptoironic has different name17:39
yoctozeptohost = primary for libvirt17:40
yoctozeptohost = primary-ironic  for ironic17:40
sean-k-mooneyso i need to map both hostnames to the same host17:40
yoctozeptoand they are discovered on deploy undert those different names17:40
yoctozeptono, no need to17:40
sean-k-mooneycan you point me to the job defintion so i can see what is added to the global.yaml and inventory17:41
sean-k-mooneyi could not find any docs for how to do this17:41
yoctozeptosean-k-mooney: docs should be enough but here you go: https://opendev.org/openstack/kolla-ansible/src/commit/9f54c6fd1b9b25e052c48a0dec90d38ff29cbcae/tests/templates/globals-default.j2#L81-L8417:43
sean-k-mooney yoctozepto the ironci docs dont cover colocation https://docs.openstack.org/kolla-ansible/latest/reference/bare-metal/ironic-guide.html17:44
*** evrardjp has quit IRC17:44
sean-k-mooneywhen you set "enable_ironic: yes" in the past it prevented you form deploying libvirt17:44
yoctozeptosean-k-mooney: because it's by default17:44
yoctozeptothen it has changed in the meantime17:44
sean-k-mooneyyep as i said its been about 3 releases since i tried17:45
yoctozeptocould use doc update most likely17:45
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for encrypting glance/heat api  https://review.opendev.org/70713117:45
*** seco has joined #openstack-kolla17:45
*** evrardjp has joined #openstack-kolla17:49
*** seco has quit IRC17:50
mgoddardhi sean-k-mooney, as yoctozepto says it should work. The reason we don't document it is that it only really applies to the all-in-one case17:52
sean-k-mooneyyep i have one server that is a tower server that is pretty power effeicnt and quite so i leave that running17:53
sean-k-mooneybut i have 4 other servers taht are all in a single 2u rack mount chassis that i use for sriov work17:54
sean-k-mooneyso im trying to confiur it so my tower server which will run my dev vms can optionally use ironic to provision my other servers17:54
mgoddardthat should work just fine17:55
sean-k-mooneyit looks like i just missed this being added17:55
*** happyhemant has quit IRC17:55
sean-k-mooneyit predates teh refactor to launching containers via the role handles17:55
sean-k-mooneylast time i tried was Newton i belive and it had issue back then.17:55
sean-k-mooneyanyway thanks :)17:56
sean-k-mooneyunfortuetly i guess kolla ansible droped the requiremetns for docs for all new features/roles at some point as it seam like it can do much more then is documented17:57
sean-k-mooneyby the way that is not intended as a dig, i just noticed that as i try to enable service like octavia there are not docs for it and the same is true of many of the services. some just work but often you need to read the ansible playbooks to figure out how it works.18:02
openstackgerritMark Goddard proposed openstack/kayobe master: Cleanup py27 support  https://review.opendev.org/72076418:02
openstackgerritMark Goddard proposed openstack/kayobe master: Cleanup py27 support  https://review.opendev.org/72076418:04
openstackgerritMark Goddard proposed openstack/kayobe master: Remove support for CentOS 7 and Python 2  https://review.opendev.org/72081818:09
*** hongbin has quit IRC18:21
*** seco has joined #openstack-kolla18:22
*** hongbin has joined #openstack-kolla18:26
*** hongbin has quit IRC18:27
*** seco has quit IRC18:27
*** cah_link has joined #openstack-kolla18:42
*** k_mouza has joined #openstack-kolla18:48
hrwstable/train with official centos mirrors build goes quite nice18:50
*** k_mouza has quit IRC18:52
*** seco has joined #openstack-kolla18:54
Fl1nt@mgoddard, oh neat! thanks a lot, I'll definitely switch to venv.18:58
*** Fl1nt has quit IRC18:58
sean-k-mooneyyoctozepto: mgoddard as an fyi it looks like there is some issue with ironic and iptables on centos 8 http://paste.openstack.org/show/792432/18:59
sean-k-mooneyif i run the command myself i get the same output19:00
sean-k-mooneysudo iptables --flush ironic-inspector19:00
sean-k-mooneyiptables: Operation not supported.19:00
sean-k-mooneywhats a little bit strange is --flush is supported but i do know that iptables/nftables implemenation changed between centos 7 a and 8 so maybe there is a step i missed that is required to make that work19:02
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/train: [WIP] switch to official CentOS 8 repositories  https://review.opendev.org/72132919:02
hrwwe still lack centos-release-* packages to enable those repos but we no longer use delorean19:03
* hrw off19:04
sean-k-mooneyimguessing centos 8 is default to  iptables-nft but we need iptables-legacy19:05
sean-k-mooneyya so on the host centos 8 is using the nf tables version iptables v1.8.2 (nf_tables)19:10
sean-k-mooneyit looks like the issue is related to https://opendev.org/openstack/kolla-ansible/commit/86e83faeb1fd088d44c5108a5ec835eba6316b2d19:20
*** hjensas has joined #openstack-kolla19:23
*** e0ne has joined #openstack-kolla19:35
*** dciabrin has quit IRC19:37
*** dciabrin has joined #openstack-kolla19:37
sean-k-mooneymgoddard: ok so if i remove the deploy.yml change introduced in ^  i think i can deploy ironic properly on centos-8 its passed where its was failing but is still running19:38
sean-k-mooneygiven that was added for upgrading betwen using iptables to dnsmasq can it now be removed?19:38
*** e0ne has quit IRC19:40
*** e0ne has joined #openstack-kolla19:41
*** e0ne has quit IRC19:46
sean-k-mooneyreported as a bug here https://bugs.launchpad.net/kolla-ansible/+bug/187395119:49
openstackLaunchpad bug 1873951 in kolla-ansible "ironinc inspector deploy fails due to iptables --flush" [Undecided,New]19:49
r3ap3rsean-k-mooney: do you happen to be using CentOS 8 Stream or CentOS 8.1?19:50
sean-k-mooneycentos 8 stream19:50
sean-k-mooneysince that should most closly track the next version fo rhel19:50
sean-k-mooneyill actully update the bug with that just incase there is a difference19:53
r3ap3rOk, thanks. Just trying to get an idea of what people are using. From my understanding, there will be two different versions of RHEL and CentOS from here on out, Stream and non-Stream. Stream is meant to be semi-rolling release model the more closely aligns with Fedora while the "non-Stream" will stick with Major and Minor version releases. I may be mistaken but that is my current understanding.19:53
sean-k-mooneyr3ap3r: well i normaly use ubunutu but since i work for redhat im at least trying to use centos19:56
r3ap3rsean-k-mooney: haha, understood.19:56
*** seco has quit IRC20:00
r3ap3rsean-k-mooney: for what it is worth, I have a C6145 that I am installing CentOS 8 on and have to use the drivers for the LSI 2008 from the dud elrepo to get them to work. If I am trying to install CentOS 8 Stream, it doesn't pick up the dud driver from the other USB drive I have plugged in but after flashing CentOS 8.1 it does pick up the dud drivers. Definitely a difference between the two. ;-)20:11
*** k_mouza has joined #openstack-kolla20:15
r3ap3rmgoddard: yoctozepto: do you happen to be running CentOS 8 Stream or CentOS 8.1 for testing your CI jobs if you don't mind me asking?20:16
yoctozeptolast time I checked - stream20:18
* yoctozepto goes to bed20:18
r3ap3rCool, thanks.20:19
*** k_mouza has quit IRC20:20
sean-k-mooneyr3ap3r: oh i am not using the centos kernel i imendetly replaced it with the kernel-ml package from elrepo20:20
sean-k-mooneyas i said i normally use ubuntu and in used to using ukuu to run the latest upstream longterm support kernel or upstream stable kernel20:21
sean-k-mooneyi do not like distro patched kernels in general. i do use them but i have been bitten by disto kernels braking nested virt and dpdk in the past so i prefer to use a vanilla kernel where i can20:22
r3ap3rsean-k-mooney: Hmm, never had that problem with CentOS or Arch Linux but hey, there is a first time for everything. Thanks for the info, I always love learning how other people do things so I can better my processes.20:25
sean-k-mooneyi have had mixed success with centos and nested virt20:25
sean-k-mooneyi dislike that centos 8 was based on 4.18 when nested virt was enable upstream by defaul in linux in 4.1920:26
sean-k-mooneymany of the nested virt change have been backported but iw ould have prefered it to be based on 4.19 or 5.020:27
r3ap3rI have been using oVirt for the past year but decided to try my hand at OpenStack which led me here. I like Openstack better than oVirt for building training environments. Ah, I see what you are saying. That does make sense, missed it by one minor version update.20:28
sean-k-mooneyyep so when rhel 8.0 lanuched or not everything had been backported. in 8.1 its more or less fine bar some edgecase that will be adress in 8.2 related to secure boot20:29
sean-k-mooneybut as a result i like to track the latest sable upstream kernel to get those fixes as soon as they are merged and released20:30
r3ap3rI've always been curious about utilizing the latest stable upstream kernel on CentOS but a little hesitant due to "unexpected" results from using the "non-standard" kernel but I've been thinking about giving it a go due to moving to OpenStack.20:32
sean-k-mooneyya i should point out this is for my dev servers20:34
sean-k-mooneyi work on nova mainly and do most of my dev on a openstack deployed with kolla20:34
sean-k-mooneyso while i want it to be stable for my own work i am the only user of that install20:35
sean-k-mooneythat said i have never really had issue on ubuntu with ukuu20:35
r3ap3rThat is cool. I like to live a little dangerous so I may roll my production that way once I get my deployment process down.20:36
sean-k-mooneyhehe well as long as you are prepared to fix it if it explodes. one of the benifts of ukuu that elrepo does not provide is all the upstream kerenls. e.g. you can pick to saty on jsut eh 5.4.x kernel with elrepo20:37
sean-k-mooneyukuu if you are not familar with it is the ubuntu kernel update utility and automate installing kerenls form the ubuntu-mainline ppa which has ever point release of the kernels avaiable20:38
r3ap3rMy main distro everywhere but my servers is Arch Linux so I am well prepared for explosions, breakage and everything in between. ;-D20:39
r3ap3rI even thought about rolling OpenStack on Arch but I have to learn OpenStack better before I try that. lol20:39
sean-k-mooneyi tried manjaro for a while and ya the same tought crossed my mind. i was suprissed with how outdate the pacakges were for arch20:40
sean-k-mooneyhavent tried vanilla arch but i might at somepoint20:41
r3ap3rYeah, Manjaro is supposed to be "user friendly" Arch but I don't care for it. I have had Manjaro break on me 10 times more than Vanilla Arch.20:43
r3ap3rStarted on Antergos before they retired the project and then went the Arch Way and haven't looked back.20:44
r3ap3rArch Linux + i3 + Vim = productivity for me.20:44
sean-k-mooney:) unfortunetly devstack and kolla dont support arch so i wont be using it anytime soon form my day to day dev. since my laptop has an nvida gpu i just stick with popos on that front too since it more or less just works with nvida20:46
*** cah_link has quit IRC20:47
r3ap3rNot intending to start a "flame war" but I'm an AMD guy myself and ever since the opensourced pretty much every aspect of their drivers for their GPU's, it has made the AMD GPU support in Linux worlds better in my opinion.20:48
sean-k-mooneyoh ya if you are on lunux you should use amd but we had a limited choice of laptops and this was the only option with a gpu20:50
sean-k-mooneywell dedicated gpu20:50
r3ap3rMy mindset is if it works, it works, use whatever you need to get the job done. At the end of the day, everything is a tool for us to use to accomplish our goal/task so use what works for YOU.20:51
sean-k-mooneyyep20:51
r3ap3rVery true, Intel/Nvidia do tend to jump into every retail as quickly as possible.20:52
openstackgerritJames Kirsch proposed openstack/kolla master: Add stunnel container for Glance service  https://review.opendev.org/72085020:52
*** e0ne has joined #openstack-kolla20:54
*** jimcrowleyibm has quit IRC20:59
*** jimcrowleyibm has joined #openstack-kolla20:59
*** jimcrowleyibm has quit IRC21:04
*** mcnugit has joined #openstack-kolla21:16
*** e0ne has quit IRC21:42
*** dosaboy has quit IRC21:46
*** dosaboy has joined #openstack-kolla22:02
*** e0ne has joined #openstack-kolla22:06
*** mcnugit has quit IRC22:12
*** born2bake has quit IRC22:33
*** TrevorV has quit IRC22:42
*** threestrands has joined #openstack-kolla23:18
*** hjensas has quit IRC23:49
*** hjensas has joined #openstack-kolla23:51
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for encrypting glance/heat api  https://review.opendev.org/70713123:53
*** e0ne has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!