| *** goldyfruit has quit IRC | 00:42 | |
| *** goldyfruit has joined #openstack-kolla | 00:44 | |
| *** zhanglong has joined #openstack-kolla | 00:45 | |
| *** xinliang has joined #openstack-kolla | 00:54 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla-ansible master: Adapt to Octavia Certificate Configuration Guide. https://review.opendev.org/706203 | 01:02 |
|---|---|---|
| *** cloudnull has quit IRC | 01:04 | |
| *** cloudnull has joined #openstack-kolla | 01:05 | |
| *** openstackstatus has joined #openstack-kolla | 01:16 | |
| *** ChanServ sets mode: +v openstackstatus | 01:16 | |
| *** dking_desktop has quit IRC | 01:35 | |
| *** zhanglong has quit IRC | 01:42 | |
| *** zhanglong has joined #openstack-kolla | 01:45 | |
| *** zhanglong has quit IRC | 02:22 | |
| *** pbing19 has joined #openstack-kolla | 04:18 | |
| *** ricolin has joined #openstack-kolla | 04:34 | |
| *** Tony31 has joined #openstack-kolla | 04:36 | |
| *** skramaja has joined #openstack-kolla | 04:46 | |
| *** pbing19 has quit IRC | 04:57 | |
| osmanlicilegi | morning | 05:29 |
| Tony31 | good morning | 05:32 |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #openstack-kolla | 05:34 | |
| Tony31 | how could I debug the nova_compute container keep restarting? This was during deployment and deployment failed due to waiting for the compute service to register | 05:40 |
| Tony31 | 2nd problem which I am facing is the compute node is not getting the storage network provisioned onto it | 05:40 |
| Tony31 | `FAILED - RETRYING: Waiting for nova-compute services to register themselves (1 retries left). | 05:41 |
| Tony31 | "--os-project-domain-name", "default", "--os-tenant-name", "admin", "--os-username", "admin", "--os-password", "removed", "--os-user-domain-name", "default", "--os-region-name", "RegionOne", "compute", "service", "list", "--format", "json", "--column", "Host", "--service", "nova-compute"], "delta": "0:00:03.594549", "end": "2020-02-10 | 05:41 |
| Tony31 | 12:47:13.118327", "rc": 0, "start": "2020-02-10 12:47:09.523778", "stderr": "", "stderr_lines": [], "stdout": "[]", "stdout_lines": ["[]"]}` | 05:41 |
| osmanlicilegi | Tony31: is it train? | 05:58 |
| osmanlicilegi | Tony31: are you using k-a provided ceph or something else? | 06:00 |
| Tony31 | hi, it's Train and is kayobe | 06:02 |
| Tony31 | p.s. - could be user error on my part - this is my first time going through the deployment | 06:02 |
| Tony31 | I wonder if the compute node is trying to contact the DNS `fqdn:35357` | 06:05 |
| Tony31 | dns was not resolving | 06:08 |
| *** shyamb has joined #openstack-kolla | 06:16 | |
| *** pbing19 has joined #openstack-kolla | 06:17 | |
| osmanlicilegi | Tony31: ah kayobe... unfortunately I'm not a kayobe guy yet :| | 06:31 |
| Tony31 | :) | 06:32 |
| Tony31 | no worries - thanks for your help. I think adding the /etc/hosts has fixed that problem, the deploy is still running now | 06:32 |
| Tony31 | I'm really excited over this kayobe/openstack deployment. I hope to contribute as much as I can. | 06:34 |
| osmanlicilegi | Tony31: glad to hear that :] | 06:35 |
| Tony31 | deployment completed - adminrc.sh files are created now... i'll continue on to post deployment | 06:35 |
| Tony31 | :) | 06:35 |
| osmanlicilegi | kayobe will rule my deployments too at future | 06:36 |
| openstackgerrit | Merged openstack/kolla master: aarch64: enable influxdb and telegraf for Debian/Ubuntu https://review.opendev.org/704970 | 06:37 |
| Tony31 | While going through this, it's so much easier than the tripleo. I had a thought - I think it would be easy to create a web configuration front-end. Like some web page that you can use, to fill out the boxes or drag/drop config to create the openstack design | 06:37 |
| Tony31 | Hooray I now have a horizon login page :) | 06:38 |
| osmanlicilegi | great! now it's time to enjoy your cloud :] | 06:40 |
| Tony31 | almost - I need to figure out: 1) why the controller does not have a storage network added (I understand it's required for building instances) 2) why the compute node doesnt have a storage network (required for cinder isci for instances) 3) how to configure cinder 4) how to configure LDAP/Active Directory | 06:46 |
| Tony31 | Then I can look at minor things :) | 06:46 |
| *** shyamb has quit IRC | 06:56 | |
| xinliang | Hello all, anyone knows is it possible to upgrade kolla oenstack from rocky directly to train? Or the right upgrade way is rocky->stein->train ? | 06:58 |
| Tony31 | Hi Xinliang, I believe that all openstack need to upgrade to each release. So you need Stein > Train. You cannot miss any release. The only time you can miss a release, is if there are specific instruction for how to achieve it. | 07:02 |
| *** numans_ has joined #openstack-kolla | 07:02 | |
| Tony31 | https://docs.openstack.org/kayobe/train/upgrading.html#upgrading-the-overcloud | 07:02 |
| xinliang | Tony31: thanks, I do tried upgrade from rocky directly to train. The upgrade process repoorts no error, but some service, such as cinder, horizon does not work properly. | 07:12 |
| xinliang | If there is this limiation, really hope we can document it here:https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html#limitations-and-recommendations | 07:13 |
| Tony31 | Are you able to wait for the experts to come online, so that you can speak with them? | 07:14 |
| Tony31 | I am sorry that I am not an expert with this. | 07:14 |
| *** shyamb has joined #openstack-kolla | 07:14 | |
| xinliang | Sure, thanks Tony31 | 07:14 |
| *** sri_ has joined #openstack-kolla | 07:26 | |
| sri_ | Hi Tony31, quick question kayobe work with vmbc ? | 07:29 |
| Tony31 | Hi sri_ - virtual bmc? You mean like for pxe boot VMs ? | 07:30 |
| sri_ | Tony31, Yes | 07:30 |
| sri_ | tto test baremetal provisioning | 07:31 |
| Tony31 | Actually, I had this same chat with the devs here last week - They said to me that there's no point pretending that VMs are real bare metal. And they told me to just install the VM myself manually, set up the SSH access key and then add the VM(s) to the inventory. So I done just that. And it saved me the trouble of messing around with any vbmc | 07:31 |
| *** cah_link has joined #openstack-kolla | 07:31 | |
| sri_ | hmm, ok | 07:32 |
| Tony31 | If you just want to test real bare metal with vbmc then I dont know... but I guess you would need to fake the ipmi (or other interface). I was leaning to go down this path but it's not required for a deployment onto VMs. | 07:32 |
| Tony31 | vbmc should work - all you would be doing is faking the ipmi or other management interface. | 07:33 |
| sri_ | I was thinking to use tripleo for baremetal provisioning and just kolla-ansible | 07:33 |
| sri_ | to deploy openstack | 07:33 |
| sri_ | I've not give try with kayobe yet | 07:34 |
| Tony31 | I have tripleO already deployed, I have the controller as a VM. At that time (3 years ago) I used the fake driver for ipmi and then powered on the VM manually during the deployment. Was a pain. :) | 07:34 |
| Tony31 | With kayobe - I did not need any of that pain. | 07:34 |
| sri_ | ack | 07:35 |
| Tony31 | when you say bare metal provisioning, are you talking about deploying openstack onto bare metal? Or are you talking about using openstack to deploy instances onto entire bare metal ? | 07:35 |
| sri_ | deploying openstack to baremetal | 07:36 |
| Tony31 | ok | 07:37 |
| *** dciabrin has joined #openstack-kolla | 07:37 | |
| sri_ | Tony31, Thanks for the info, Last week I saw you mentioned about the DR with ovirt and openstack, that's really interesting | 07:43 |
| *** Tengu has quit IRC | 07:44 | |
| Tony31 | Yes - so basically where I am at the moment is trying to create an openstack that will allow me to implement a DR plan for my org. with it. The tripleO deployment which I have has problems, like 1) can't upgrade it 2) running Pike version 3) can't access the tools I need for DR 4) can't maintain it 5) lack of documentation or rather, lots of | 07:45 |
| Tony31 | documentation errors or confusing documentations | 07:45 |
| Tony31 | so I've gone back to the beginning and working on re-deploying and migratiom | 07:45 |
| *** Tengu has joined #openstack-kolla | 07:46 | |
| Tony31 | oVirt has a DR plan which I have set up and tested as working. I havent decided how I'll achieve it with Openstack as yet. But I have 2 x baremetal compute in the DR site only. I'd prefer to keep 2 x compute tied in with oVirt so that oVirt has resilience. And then run a virtual compute node on top of that, which would be used in DR scenario only. | 07:47 |
| Tony31 | But I have yet to dive into it, or plan / test. | 07:47 |
| Tony31 | ...virtual compute node for openstack, I mean | 07:48 |
| sri_ | I am also working on same thing DR for openstack , we were using Tripleo as you told it's very hard to customize | 07:48 |
| Tony31 | I would accept lower performance in the DR situation. Although, the DR site is new and the CPUs and memory are also new and much faster than my production site. So I am expecting not much difference in performance. | 07:48 |
| Tony31 | The way I set up tripleO was to deploy it as best I could. Then do manual post-deployment of cinder, horizon, ldap and add projects, routers and networks. I was unable to achieve that during the deployment for various reasons. | 07:50 |
| Tony31 | I would have tried to stick with tripleo if the documentation supported me. But I'm on Pike and the documentation which is there doesnt align. | 07:51 |
| sri_ | true, i think it's apply for all big opensource source projects | 07:53 |
| sri_ | it's really interesting approach the way you are planning to DR. I will also try to do some POC | 07:55 |
| Tony31 | yes - test, test and test again :) | 07:55 |
| sri_ | Tony31, i really appreciate your input :) | 07:56 |
| Tony31 | my idea is to layer the design. With oVirt managing the lower bare-metal and the failover of the VMs. | 07:57 |
| Tony31 | Actually, it's Red Hat Virtualisation, where I have support. It's just red hat branded, supported oVirt. | 07:57 |
| sri_ | Is it same as Red Hat Virtualization ? | 07:58 |
| Tony31 | yes, redhat = support | 07:59 |
| Tony31 | and different logo ;) | 07:59 |
| sri_ | Got it :) | 07:59 |
| yoctozepto | xinliang: openstack requires upgrade from release to release, skipping releases is not supported | 08:00 |
| Tony31 | experts incoming ^ :) | 08:00 |
| *** shyamb has quit IRC | 08:01 | |
| yoctozepto | Tony31: thanks | 08:01 |
| yoctozepto | 06:40:21 <Tony31> how could I debug the nova_compute container keep restarting? This was during deployment and deployment failed due to waiting for the compute service to register | 08:01 |
| yoctozepto | 06:41:19 <Tony31> `FAILED - RETRYING: Waiting for nova-compute services to register themselves (1 retries left). | 08:01 |
| yoctozepto | these are the very same problem | 08:01 |
| yoctozepto | look into /var/log/kolla/nova/nova-compute.log | 08:01 |
| Tony31 | thanks - that issue was due to dns. the compute node could not contact the controller due to missing dns resolution. Fixed with /etc/hosts entry | 08:02 |
| *** cah_link has quit IRC | 08:02 | |
| Tony31 | i successfully deployed | 08:02 |
| yoctozepto | (I'm writing these paths from the top of my head) | 08:02 |
| yoctozepto | ah, understood | 08:02 |
| Tony31 | :) | 08:02 |
| yoctozepto | well, now you also know how to debug | 08:02 |
| Tony31 | thanks :) | 08:02 |
| yoctozepto | 07:46:17 <Tony31> almost - I need to figure out: 1) why the controller does not have a storage network added (I understand it's required for building instances) 2) why the compute node doesnt have a storage network (required for cinder isci for instances) 3) how to configure cinder 4) how to configure LDAP/Active Directory | 08:02 |
| Tony31 | my controller and compute node does not have storage network - do I need to enable cinder and iscsi in kolla.yml ? | 08:02 |
| *** mleone87 has joined #openstack-kolla | 08:02 | |
| yoctozepto | ad 1) no idea, depends on storage configuration - it might not be necessary there | 08:02 |
| yoctozepto | ah, yeah, storage is for cinder | 08:03 |
| Tony31 | ok - no problem. This is what I like. Deploy - see a problem - fix problem, move on to the next. Always learning | 08:03 |
| yoctozepto | but I don't know kayobe flow so not sure if it should be there already or not | 08:03 |
| *** bengates has joined #openstack-kolla | 08:04 | |
| Tony31 | is it right that kayobe is just layered over the top of kolla? | 08:04 |
| yoctozepto | ad 3) enable_cinder is a good start, then decide on the backend | 08:04 |
| yoctozepto | Tony31: yeah, kayobe (provision of baremetal with bifrost) -> kolla-ansible (deployment of openstack) -> kolla (container images) | 08:04 |
| Tony31 | thank you | 08:05 |
| yoctozepto | np | 08:05 |
| Tony31 | for LDAP with keystone, to enable the `domain` - along with the modification of the horizon to enable the `domain` field on the login page - is this a manual task with kolla-ansible? I mean, do I need to log in to the containers and make the config change then restart the container? | 08:06 |
| xinliang | yoctozepto: I see, thanks. | 08:07 |
| *** cah_link has joined #openstack-kolla | 08:09 | |
| sri_ | Tony31, I don't think you don't have to login to the containers, All the configuration is out of container, exp: /etc/kolla/keystone/ you make changes to config and restart the container | 08:09 |
| sri_ | yoctozepto, correct me if i am wrong^ | 08:10 |
| Tony31 | wow thanks sri_ - I only checked `ls -l /etc/key` and it wasnt there. So I then logged into the container | 08:10 |
| yoctozepto | Tony31: sri_ is right, then you just need to reconfigure | 08:10 |
| yoctozepto | or whatever kayobe calls that :-) | 08:11 |
| Tony31 | but indeed, on the controller, the config file is there `/etc/kolla/keystone` | 08:11 |
| yoctozepto | on the deployment host you add such overrides | 08:11 |
| mnasiadka | morning | 08:11 |
| yoctozepto | and kolla-ansible merges them | 08:11 |
| Tony31 | what I am worried about is making the config change on the controller directly, then running the `kayobe overcloud service reconfigure` will wipe out the config | 08:11 |
| yoctozepto | mnasiadka welcome, our timezone-friendly local kayobe expert | 08:11 |
| Tony31 | morning | 08:11 |
| yoctozepto | Tony31: yeah, you should not do that | 08:12 |
| yoctozepto | the overrides are on the deployment host | 08:12 |
| *** sorin-mihai has quit IRC | 08:12 | |
| Tony31 | my deployment host is the ansible control host | 08:12 |
| mnasiadka | yoctozepto: I wouldn't put the kayobe expert close to my name, I just use it :) | 08:12 |
| yoctozepto | mnasiadka: shh | 08:12 |
| yoctozepto | ;D | 08:12 |
| Tony31 | but I cant see where to add such config - do I need to make up ansible files for it? | 08:12 |
| mnasiadka | it's hard to read from history what are you trying to achieve - can you write it once again? | 08:15 |
| *** tonythomas has joined #openstack-kolla | 08:16 | |
| *** gfidente has joined #openstack-kolla | 08:20 | |
| kevinz | hrw: ^ | 08:27 |
| yoctozepto | mnasiadka: override keystone config for kayobe | 08:29 |
| kevinz | hrw: there? xinliang met a problem about upgrade from Rocky-> Train. It looks that we can't do skip version upgrade from R->T, just can do R->S->T | 08:29 |
| *** carli has joined #openstack-kolla | 08:31 | |
| mnasiadka | yoctozepto, Tony31: I think this describes how to override globals and per service config in kayobe - https://docs.openstack.org/kayobe/latest/configuration/kolla-ansible.html | 08:31 |
| mnasiadka | especially this: https://docs.openstack.org/kayobe/latest/configuration/kolla-ansible.html#service-configuration | 08:32 |
| yoctozepto | kevinz: yeah, no matter the deployment method | 08:32 |
| Tony31 | thank you - reading | 08:33 |
| yoctozepto | kevinz: it's also necessary to adjust any config as mentioned in the upgrade notes | 08:33 |
| kevinz | hi yoctozepto, sure thanks, upgrading action is dangerous | 08:34 |
| mnasiadka | kevinz: we do not support fast-forward upgrades, also known as skip-level upgrades | 08:34 |
| mnasiadka | there was some work around it - but nobody picked it up | 08:34 |
| *** ivve has joined #openstack-kolla | 08:36 | |
| kevinz | <mnasiadka> : yes I see. If we must to do so, I guess much configurations need to be sort out | 08:37 |
| mnasiadka | kevinz: nobody said OpenStack is easy :) | 08:38 |
| kevinz | mnasiadka: absolutely :D | 08:38 |
| yoctozepto | mnasiadka: does any deployment project support it anyway? | 08:39 |
| mnasiadka | yoctozepto: tripleo - they have to | 08:40 |
| yoctozepto | mnasiadka: mhm, interesting | 08:40 |
| mnasiadka | yoctozepto: since RHOS supports LTS releases - and it's every third release I think | 08:40 |
| *** shyamb has joined #openstack-kolla | 08:40 | |
| Tony31 | so I do not have storage network configured on compute or controller. I have just enabled Cinder service in `kolla.yml` and I can see the cinder containers now - but still no storage network. I think I need to run `kayobe overcloud host configure` -> can anyone confirm? | 08:40 |
| yoctozepto | ah, because they ensure specific paths | 08:40 |
| mnasiadka | so in most customer envs - every upgrade is a FFU... | 08:40 |
| yoctozepto | so it's not like they allow just any update to happen anyway | 08:40 |
| *** rpittau|afk is now known as rpittau | 08:41 | |
| yoctozepto | or, well, "allow" is not the best | 08:41 |
| mnasiadka | yoctozepto: I survived doing N->Q in TripleO for my previous employer | 08:41 |
| yoctozepto | they don't offer "unb0rkability promise" | 08:42 |
| Tony31 | tripleo is similar (if not same) to RHOS. I used RHOS docs to fill in the tripleO documentation gaps. | 08:42 |
| yoctozepto | Tony31: RHOS is commercial offering of tripleo exactly | 08:42 |
| Tony31 | got it, thanks | 08:42 |
| mnasiadka | yoctozepto: need your review on https://review.opendev.org/#/c/704309 :-) | 08:42 |
| patchbot | patch 704309 - kolla-ansible - Remove kolla-ceph - 9 patch sets | 08:42 |
| yoctozepto | "mnasiadka: I need to learn to apologise for the apologising :-)" | 08:43 |
| yoctozepto | did you? | 08:43 |
| mnasiadka | not yet | 08:43 |
| yoctozepto | then no +2 | 08:43 |
| mnasiadka | only working with the Brits since August 2019 :D | 08:43 |
| yoctozepto | ;D | 08:44 |
| yoctozepto | oh my, the change is brutal | 08:44 |
| yoctozepto | +90, -3155 | 08:44 |
| *** Luzi has joined #openstack-kolla | 08:45 | |
| *** pbing19 has quit IRC | 08:49 | |
| *** pbing19 has joined #openstack-kolla | 08:49 | |
| *** mleone87 has quit IRC | 08:52 | |
| Tony31 | ""mnasiadka: I need to learn to apologise for the apologising :-)" 🤣🤣 you know us well :) | 08:53 |
| mnasiadka | yoctozepto: I guess it's not easy to rip apart years of code :) | 08:53 |
| Tony31 | I ran again `kayobe overcloud host configure` and i am still not getting storage network on compute or controller | 08:56 |
| Tony31 | confused about it | 08:56 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 08:56 |
| mnasiadka | Tony31: https://docs.openstack.org/kayobe/latest/configuration/network.html#network-to-host-mapping | 08:57 |
| Tony31 | in `networks.yml` i have, all of these assigned to the storage network: `storage_net_name:` / `ceph_storage_net_name` / `swift_storage_net_name:` / `swift_storage_replication_net_name` | 08:58 |
| Tony31 | mnasiadka - I was expecting the storage aspect to be covered in the networks.yml - is it not the case? | 08:59 |
| *** dougsz has joined #openstack-kolla | 08:59 | |
| Tony31 | have to rush off - i'll come back in a few hours. If you reply to my Q I'll check the logged messages :) Thanks all - tachau | 09:01 |
| *** Tony31 has quit IRC | 09:06 | |
| yoctozepto | mnasiadka: reviewed, uff | 09:07 |
| mnasiadka | thanks :) | 09:08 |
| *** sri_ has quit IRC | 09:17 | |
| *** k_mouza has joined #openstack-kolla | 09:17 | |
| *** bjolo has joined #openstack-kolla | 09:19 | |
| *** k_mouza has quit IRC | 09:22 | |
| *** rgogunskiy has joined #openstack-kolla | 09:43 | |
| bjolo | morning | 09:45 |
| mgoddard | morning | 09:47 |
| bjolo | hi marc, how are you>? | 09:47 |
| *** k_mouza has joined #openstack-kolla | 09:50 | |
| hrw | morning | 09:53 |
| mgoddard | hi bjolo. Long time no see. I'm good thanks. And you? | 09:55 |
| *** dking_desktop has joined #openstack-kolla | 09:57 | |
| bjolo | good good | 09:58 |
| bjolo | yah i know. been a while since i was around :) | 09:58 |
| bjolo | been busy with other stuff than OS | 10:00 |
| bjolo | mostly non tech | 10:03 |
| bjolo | but now im back to handson and pending upgrades in the near future | 10:03 |
| bjolo | kolla still seems alive and thriving :) | 10:04 |
| *** shyamb has quit IRC | 10:04 | |
| yoctozepto | bjolo: yeah, it is | 10:04 |
| yoctozepto | ever-blooming | 10:04 |
| bjolo | nice :) | 10:04 |
| bjolo | we run 2 clouds on kolla rocky, going to stein in a 1-2 months | 10:05 |
| bjolo | i have a silly question... perhaps google fu is bad this monday morning but.. where do i report a bug on openstackclient? | 10:06 |
| bjolo | and also for kolla. is that still launchpad? | 10:07 |
| hrw | kolla yes | 10:07 |
| mnasiadka | kolla uses launchpad | 10:07 |
| yoctozepto | mgoddard, mnasiadka: as far as upgrades are concerned, I upgraded production from stein to train; obviously dropped horizon vncproxy sessions due to haproxy restart but otherwise no user impact, woop woop | 10:07 |
| mnasiadka | openstackclient probably uses storyboard | 10:07 |
| mnasiadka | yoctozepto: probably you were just lucky, it can't be so straightforward :) | 10:08 |
| yoctozepto | mnasiadka: thought so, my bad! | 10:08 |
| *** dking_desktop has quit IRC | 10:12 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 10:16 |
| *** Tengu has quit IRC | 10:17 | |
| *** Tengu has joined #openstack-kolla | 10:18 | |
| *** cah_link has quit IRC | 10:18 | |
| *** shyamb has joined #openstack-kolla | 10:19 | |
| bjolo | gee storyboard is slow | 10:21 |
| yoctozepto | bjolo: complaints go to #storyboard | 10:22 |
| *** cah_link has joined #openstack-kolla | 10:23 | |
| *** Tengu has quit IRC | 10:31 | |
| hrw | storage-sig managed to rebuild / crosstag all ceph nautilus deps into | 10:33 |
| hrw | storage8-ceph-nautilus-candidate [1] target thanks to the new el8 buildroot | 10:33 |
| hrw | centos-release-ceph-nautilus will arrive soon | 10:33 |
| *** Tengu has joined #openstack-kolla | 10:40 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 10:42 |
| mnasiadka | hrw: that's interesting, given the previous statement by Sage that Nautilus will not be supported on centos8 :) | 10:44 |
| mnasiadka | and these are only deps, still I can't find any ceph package built on el8 using cbs | 10:46 |
| hrw | mnasiadka: ach, right. need more coffe | 10:49 |
| hrw | -ENOCOFFEE | 10:49 |
| *** shyamb has quit IRC | 10:58 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Remove kolla-ceph https://review.opendev.org/704309 | 11:10 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [ussuri][goal] Drop python 2.7 support and testing https://review.opendev.org/695617 | 11:17 |
| *** Tony31 has joined #openstack-kolla | 11:18 | |
| Tony31 | Hi guys | 11:19 |
| Tony31 | :) | 11:19 |
| mgoddard | hi Tony31 | 11:20 |
| Tony31 | maybe I can word it this way - what are the storage networks for, in networks.yml ? Why are they there and why do we have to configure them a part of the first steps? | 11:20 |
| Tony31 | Are any of those used for cinder / iscsi? | 11:20 |
| Tony31 | Are any of them used for glance image download during instance build process ? | 11:20 |
| Tony31 | Hi Mark :) | 11:21 |
| mgoddard | First, I saw you were asking about SSH users. Try the *_bootstrap_user variables (e.g. controller_bootstrap_user). These are used when first accessing the hosts to setup the stack user that is used from that point onwards | 11:24 |
| mgoddard | storage networks are used by ceph and swift | 11:24 |
| Tony31 | Sorry, i should probably provide an update :) | 11:25 |
| Tony31 | I managed to successfully deploy today :) | 11:25 |
| mgoddard | oh, great work! | 11:25 |
| Tony31 | the issue which I am facing now at this point (which means I am misunderstanding / not done something, but i dont know what, yet) is that the controller and compute does not have the storage subnet configured onto them | 11:25 |
| Tony31 | i am planning to use cinder/iscsi. no storage nodes. I have external storage that I would like to use | 11:26 |
| Tony31 | the storage network via compute will be used to mount the instance volumes | 11:26 |
| Tony31 | and from past experience, the storage subnet is required on the controller node, because during the instance build process, the controller mounts the storage, downloads the image from glance and then moves the storage over to a compute node to run the instance | 11:27 |
| *** jopdorp has joined #openstack-kolla | 11:27 | |
| Tony31 | i've enabled cinder in kolla.yml and re-deployed using the kayobe configure | 11:27 |
| mgoddard | Tony31: ok. compute nodes should be on the storage network by default | 11:27 |
| *** shyamb has joined #openstack-kolla | 11:28 | |
| mgoddard | have you set storage_net_name and defined a subnet, vlan etc? | 11:28 |
| Tony31 | ok, I thought this... Also I had a inventory issue which meant the compute node was not built at first. Once I fixed the inventory issue I went back and re-ran kayobe overcloud host configure and provision | 11:29 |
| *** tony3170 has joined #openstack-kolla | 11:32 | |
| tony3170 | sorry my vpn kicked me off | 11:32 |
| tony3170 | yes I have that | 11:32 |
| *** Tony31 has quit IRC | 11:33 | |
| *** sri_ has joined #openstack-kolla | 11:34 | |
| tony3170 | I also have `compute_network_interfaces: - inside - osiapi - osipip - ossana` | 11:34 |
| tony3170 | last one is OS-SAN-A `ossana` | 11:34 |
| mgoddard | tony3170: you could try looking at `kayobe configuration dump`. You can run it for one host, and even dump a single variable. The network_interfaces variable is the list of neteworks the host is mapped to | 11:35 |
| tony3170 | and in hos_vars/group_vars I have which network name corresponds to which ethX interfacw | 11:35 |
| tony3170 | ok great I'll try that :) thanks | 11:35 |
| tony3170 | when I make host changes, do I need to run kayobe host configure | 11:36 |
| hrw | TASK [common : Pulling common images] ******************************************************************************************************************** | 11:40 |
| hrw | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named ssl_match_hostname | 11:40 |
| hrw | meh... | 11:40 |
| tony3170 | i dont see the config there in the config dump | 11:43 |
| tony3170 | it shows the allocation pool but not a host IP for any node | 11:43 |
| *** zhanglong has joined #openstack-kolla | 11:44 | |
| tony3170 | wait - let me check these variables have the correct spelling | 11:44 |
| tony3170 | i think it would have worked if i didnt have a typo in the host config 🙈 | 11:46 |
| tony3170 | re-running conigure now | 11:46 |
| tony3170 | thanks Mark | 11:46 |
| tony3170 | looking good so far - storage node has a storage IP | 11:48 |
| *** rgogunskiy has quit IRC | 11:48 | |
| tony3170 | fixed - thank you | 11:53 |
| hrw | why ansible produces so unreadable error messages? | 11:57 |
| hrw | https://pastebin.com/HW8F2Kcc | 11:57 |
| mnasiadka | that's jinja | 11:59 |
| mnasiadka | template error while templating string: no filter named 'kolla_address' | 11:59 |
| mnasiadka | pip install kolla_ansible :) | 11:59 |
| hrw | mnasiadka: 'pip install .' was done.. | 11:59 |
| hrw | HEAD of master | 12:00 |
| mnasiadka | yeah well, so then ansible can't find it | 12:00 |
| hrw | (venv3) 13:01 (1s) hrw@puchatek:kolla-ansible$ grep -r "def kolla_address" ../venv3/ | 12:01 |
| hrw | ../venv3/lib/python3.7/site-packages/kolla_ansible/kolla_address.py:def kolla_address(context, network_name, hostname=None): | 12:01 |
| mnasiadka | are you sure ansible uses python3, not python2? :) | 12:02 |
| hrw | mnasiadka: first checkd for py3 then py2 so should py3 | 12:03 |
| hrw | checked. py3 | 12:04 |
| mnasiadka | so that's purely interesting | 12:05 |
| *** rgogunskiy has joined #openstack-kolla | 12:05 | |
| hrw | meh. will then move to target machine and do all-in-one on localhost | 12:06 |
| *** rpittau is now known as rpittau|bbl | 12:08 | |
| hrw | so fresh os, fresh venv | 12:11 |
| *** kplant has joined #openstack-kolla | 12:13 | |
| mgoddard | tony3170: great! | 12:13 |
| hrw | meh. another set of stein fixes needed ;( | 12:13 |
| tony3170 | now to figure out applying .conf config. I have some things to read through tomorrow | 12:14 |
| admin0 | "Kayobe is a subproject of Kolla that uses Kolla Ansible and Bifrost to deploy an OpenStack control plane to bare metal." - this means no docker containers ? | 12:21 |
| *** shyamb has quit IRC | 12:21 | |
| kplant | no it still uses docker | 12:21 |
| *** tony3170 has quit IRC | 12:22 | |
| admin0 | i am trying for days to get a all-in-one up .. failing .. who can help ? https://asciinema.org/a/ig2c0AOq8xenGdOgzZxf99wNM | 12:22 |
| hrw | admin0: kayobe uses kolla-ansible to deploy containers built with kolla. | 12:22 |
| kplant | kolla-ansible takes systems that have already been provisioned with OS and networking and goes from there | 12:22 |
| kplant | kayobe starts from almost nil and loads the OS... then kicks off k-a | 12:22 |
| admin0 | so kayobe will build the docker images locally and then deploy .. while kolla-ansible will use pre-built images ? | 12:22 |
| kplant | no, that's not the difference | 12:23 |
| hrw | admin0: kayobe does not call kolla to build containers. | 12:23 |
| kplant | kayobe will use bifrost to deploy the OS on the nodes in your inventory | 12:23 |
| admin0 | ok .. as an operator, who just wants openstack installed, how do I decide which route i need to take ? | 12:24 |
| kplant | do you want to install the operating system yourself? | 12:24 |
| kplant | or do you want something else to do it for you | 12:24 |
| admin0 | oh oh .. now i get it | 12:24 |
| admin0 | overcloud / undercloud type | 12:24 |
| kplant | you could say it's similar to tripleo - yes | 12:24 |
| admin0 | thanks for the clarification | 12:25 |
| admin0 | please review my terminal recording .. i am not sure why i am getting issues .. i *think* I followed the documentation as mentioned | 12:26 |
| mgoddard | admin0: where did you get the all-in-one inentory from? | 12:30 |
| admin0 | i copied it | 12:30 |
| admin0 | its in the instructions | 12:30 |
| mgoddard | from where? | 12:30 |
| mgoddard | ok | 12:30 |
| admin0 | our kolla readme | 12:31 |
| admin0 | let me | 12:31 |
| mgoddard | I see | 12:31 |
| admin0 | https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html Copy all-in-one and multinode inventory files to the current directory. .. | 12:31 |
| kplant | also, what version of ansible are you running? | 12:31 |
| mgoddard | it says that localhost has not hostvars | 12:31 |
| mgoddard | what version of ansible are you using? | 12:32 |
| mgoddard | :) | 12:32 |
| admin0 | i am supposed to follow the docs and get it working :) | 12:32 |
| admin0 | let me check the versions | 12:32 |
| kplant | i've copied and pasted the quickstart guide several times with success | 12:32 |
| admin0 | which is why I recorded it this time to present here | 12:33 |
| admin0 | all steps done in a brand new 18.04 install | 12:33 |
| mgoddard | "Currently, Kolla Ansible requires Ansible 2.6+." | 12:33 |
| mgoddard | in the quickstart :) | 12:33 |
| mgoddard | often the version provided by the distro is old. You might be better using pip | 12:33 |
| hrw | and 18.04 has 2.5.broken | 12:34 |
| admin0 | ansible 2.5.1 | 12:34 |
| mgoddard | very old | 12:34 |
| admin0 | so where in our docs give the steps to upgrade this ? | 12:34 |
| mgoddard | pip install 'ansible>2.6' | 12:34 |
| kplant | also: 'Install Ansible. Kolla Ansible requires at least Ansible 2.8 and supports up to 2.9.' | 12:34 |
| admin0 | its now downloading 2.9.4 | 12:35 |
| kplant | https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html | 12:35 |
| hrw | fsck. stein and it's 2.7 python... | 12:35 |
| kplant | fsck failed: please repair manually and reboot | 12:36 |
| admin0 | this step is not in "Install dependencies not using virtual environment" where the docs say apt-get install ansible instead of this | 12:36 |
| kplant | `Install Ansible. Kolla Ansible requires at least Ansible 2.8 and supports up to 2.9.` | 12:36 |
| hrw | admin0: send patch to the docs? | 12:36 |
| kplant | that's right in "Install dependencies not using a virtual environment¶" | 12:36 |
| admin0 | yeah .. planning to do that | 12:36 |
| admin0 | yes .. 3. Install Ansible. Kolla Ansible requires at least Ansible 2.8 and supports up to 2.9. and then in the steps to copy paste for ubuntu run: apt-get install ansible | 12:37 |
| admin0 | i will try to send a patch note and put more comments to the docs | 12:37 |
| * hrw votes for marking ubuntu as not supported distribution | 12:37 | |
| kplant | ^ | 12:37 |
| kplant | canonical is the absolute devil | 12:37 |
| kplant | can i say that in here without getting shot? | 12:38 |
| hrw | kplant: I do not care ;D | 12:38 |
| admin0 | failed at the exact same error | 12:38 |
| admin0 | The conditional check 'inventory_hostname not in groups['bifrost'] or 'ansible_' + hostvars[inventory_hostname]['api_interface'] | 12:38 |
| hrw | mgoddard: why we did not had debian-source CI in stein? :( | 12:39 |
| *** zhanglong has quit IRC | 12:40 | |
| admin0 | its a single system .. i ran all-in-one .. why its complaining about bifrost | 12:41 |
| kplant | that's probably the first group, alphabetically | 12:42 |
| kplant | it has nothing to do with bifrost, just the first group that got hit and failed | 12:42 |
| *** zhanglong has joined #openstack-kolla | 12:42 | |
| hrw | 2020-02-10 13:43:28.676 6 ERROR nova.compute.manager [instance: 7d62960f-ecc4-458a-a6f5-a418dee52ec2] libvirt.libvirtError: invalid argument: could not find capabilities for domaintype=kvm | 12:44 |
| hrw | nasty... | 12:44 |
| admin0 | i am going to remove the apt installed ansible, do a apt clean, remove all dependencies and retry again | 12:45 |
| admin0 | it moved forward \o/ | 12:48 |
| admin0 | does kolla also manage the external_vip address and add it automatically ? | 12:52 |
| admin0 | or is the external address supposed to be NAT to the internal vip address | 12:52 |
| * admin0 starts to read global file :) | 12:54 | |
| hrw | ~curse py2 | 12:54 |
| mgoddard | admin0: kolla manages the VIP | 12:55 |
| admin0 | yeah.. i see it now in the configs | 12:56 |
| admin0 | kolla-ansible -i ./all-in-one deploy -- this finished .. but i do not see any service docker containers .. i only see rabbit, memcached, db, keepalived, haproxy, chrony, cron, kolla_toolbox and fluentd | 12:57 |
| admin0 | shouldn't i be seeing neutron, glance etc also ? | 12:57 |
| mnasiadka | yoctozepto: have any useful conclusions with keepalived? | 12:58 |
| hrw | folks: can rocky debian/source be migrated to stein debian/binary to train debian/source? | 12:59 |
| *** shyamb has joined #openstack-kolla | 12:59 | |
| admin0 | rabbit failed .. redoing | 13:00 |
| admin0 | RUNNING HANDLER [rabbitmq : Waiting for rabbitmq to start on first node] -- always fails .. known issue or a workaround here to proceed ? | 13:02 |
| mnasiadka | hrw: as long as you don't use services that rely on unbuildable images - I guess you should be fine | 13:02 |
| mgoddard | hrw: need to set nova_compute_virt_type: qemu? | 13:02 |
| hrw | mgoddard: probably. | 13:02 |
| hrw | mnasiadka: yep. | 13:02 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ironic: fix documentation https://review.opendev.org/706822 | 13:05 |
| mgoddard | admin0: probably need to remove an entry from /etc/hosts. Although I thought we had a fix for this | 13:06 |
| mgoddard | admin0: see https://bugs.launchpad.net/kolla-ansible/+bug/1855935 | 13:07 |
| openstack | Launchpad bug 1855935 in kolla-ansible "Kolla-ansible deploy fails at rabbitmq" [Undecided,Incomplete] | 13:07 |
| devfaz | mgoddard: I also run into this issue. I fixed it in our branch but not jet upstream! | 13:07 |
| *** shyamb has quit IRC | 13:07 | |
| devfaz | admin0: fyi | 13:07 |
| devfaz | admin0: https://github.com/noris-network/kolla-ansible/commit/3058f601c9b593b7455cc5a383c86270afe2cd26 | 13:08 |
| admin0 | devfaz, thanks .. so if hostname points to the api interface IP, that is good to go ? | 13:09 |
| devfaz | admin0: correct | 13:09 |
| admin0 | done ..rerunning | 13:10 |
| admin0 | assuming this is up, what is the process to add more stuff ? just edit in globals file and rerun the bootstrap,precheck and deploy ? | 13:11 |
| *** sorin-mihai has joined #openstack-kolla | 13:11 | |
| admin0 | \o/ now i see keystone being configured .. | 13:11 |
| kplant | stuff: more nodes, yes bootstrap them | 13:11 |
| kplant | stuff: mode services, no bootstrap | 13:11 |
| kplant | more* | 13:11 |
| admin0 | but if no new nodes, the same ones, still bootstrap is required ? | 13:11 |
| kplant | no | 13:11 |
| admin0 | ok | 13:12 |
| admin0 | got it | 13:12 |
| *** rpittau|bbl is now known as rpittau | 13:12 | |
| kplant | make config changes, deploy | 13:12 |
| kplant | that should be it | 13:12 |
| admin0 | my notes: 1. - follow the docs, but not apt-get install ansible. Instead pip install 'ansible>2.6' 2. - make sure hostname does not point to loopback, but point to internal ip. | 13:14 |
| mgoddard | devfaz: would appreciate that commit upstream! | 13:14 |
| mgoddard | and a bug report if possibel | 13:14 |
| *** vesper11 has quit IRC | 13:15 | |
| *** vesper has joined #openstack-kolla | 13:15 | |
| devfaz | mogddard: your wish is my command :) | 13:16 |
| r3ap3r | admin0: that is why the docs reccomend doing the the install in a Python Virtual Environment. That way you can "isolate" your Python requirements for Kolla from the requirements of the Host system. ;-) | 13:18 |
| admin0 | r3ap3r, with that, i struggled for a whole day . before selecting this non virtualhost one | 13:19 |
| mgoddard | thanks devfaz :) | 13:19 |
| admin0 | non virtualenv* | 13:19 |
| sorin-mihai | that wasn't really clear for me, venv vs system packages, but now i get it. though, are there things that i should install in the host(s) before trying to use kolla-ansible? if yes, where can i see the whole list? when i tried the non-venv method it worked, the venv method had some hick-ups | 13:20 |
| admin0 | r3ap3r, https://gist.github.com/a1git/9fc701e4d2aaec41f46499cefe2ede78 --- documentation related to the virtualenv one | 13:20 |
| *** skramaja has quit IRC | 13:22 | |
| sorin-mihai | admin0, i assume you're using ubuntu. i'm using centos here, but had same problem. ended up installing docker-ce manually, and then had some other (minor) problems, sort of sorted them afterwards | 13:24 |
| *** gary_perkins has joined #openstack-kolla | 13:26 | |
| admin0 | sorin-mihai, i tend to stick to the docs and when the docs not work, complain .. because we know and we can do stuff manually .. but for a first time operator who heard kolla is awesome and wants to us it, he may not be able to get even the basic setup up which is a big putoff | 13:26 |
| admin0 | sorin-mihai, did you submitted the fixes ? | 13:26 |
| sorin-mihai | yeah, same here. doing a clean install now, trying to get everything sorted with venv, as normal user with sudo, will post the findings once done | 13:28 |
| admin0 | the deploy failed in nova-cell .. i will run deploy again .. is there a way to tell the script not to re-run keystone etc but start at nova ? | 13:30 |
| admin0 | fails on: TASK [service-rabbitmq : nova-cell | Ensure RabbitMQ users exist] -- * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server | 13:38 |
| *** dougsz has quit IRC | 13:38 | |
| r3ap3r | admin0: I use CentOS 7 for my deployment, haven't tried deploying on Ubuntu yet but plan to try giving it a go now that I have the process down for my CentOS environment. IDK when I will have the time though? | 13:39 |
| admin0 | fails on: TASK [service-rabbitmq : nova-cell | Ensure RabbitMQ users exist] -- * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server | 13:40 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [ussuri][goal] Drop python 2.7 support and testing https://review.opendev.org/695617 | 13:42 |
| r3ap3r | admin0: you may also want to try "pip install -U kolla-ansible" after doing a normal install. In my experience the normal install only grabs the base Kolla-Ansible version and not any minor version updates. Mayber the Kolla team pushed out a fix for your issues and you just haven't grabbed them yet? | 13:43 |
| r3ap3r | admin0: I mean running that directly after "pip install kolla-ansible". Not after the deployment. ;-) | 13:44 |
| admin0 | i added to the notes | 13:45 |
| admin0 | right now , trying to redo the deploy so that it might fix rabbit and allow me to proceed with the installation from where its stuck .. nova-cell | 13:45 |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Allow Kayobe to wipe virtualized disks https://review.opendev.org/704613 | 13:48 |
| *** tony31 has joined #openstack-kolla | 13:49 | |
| *** kplant has quit IRC | 13:49 | |
| *** priteau has joined #openstack-kolla | 13:50 | |
| hrw | mnasiadka: if someone has <ussuri with kolla-ceph and does upgrade to ussuri then how to use ceph? the existing one will work as external or does it need to be migrated to some external ceph? | 13:50 |
| *** kplant has joined #openstack-kolla | 13:52 | |
| *** tony31 has quit IRC | 13:53 | |
| admin0 | i think its official .. kolla aio train is broken :D | 13:54 |
| admin0 | on ubuntu | 13:54 |
| hrw | if you say so... | 13:54 |
| r3ap3r | admin0: why don't you give CentOS a go? After all, Linux is nothing more than a tool, the end goal is to teach in your case. ;-) | 13:56 |
| admin0 | https://gist.github.com/a1git/29ad3a1919fbf7b2dae1310ca61eb708 - stuck here again and again | 13:56 |
| admin0 | r3ap3r, i have had my fair share of rpm cyclic dependency hell :) | 13:56 |
| r3ap3r | lol | 13:58 |
| hrw | I did aio on debian 10 but with master | 13:58 |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Stop running kolla-target-venv in seed hypervisor host upgrade https://review.opendev.org/704615 | 13:59 |
| admin0 | for me to give it a try in office and have one kolla-ansible on top of every other openstack-ansible, i have to document it working on a latest release ( train) and then develop some sort of roadmap to test upgrades, updates etc | 14:00 |
| admin0 | i can start with rocky/stein if they work out of the box | 14:00 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Fix deploy guide jobs https://review.opendev.org/706834 | 14:01 |
| *** dougsz has joined #openstack-kolla | 14:02 | |
| *** rgogunskiy has quit IRC | 14:06 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Ignore zuul.d https://review.opendev.org/706836 | 14:07 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Fix deploy guide jobs https://review.opendev.org/706834 | 14:07 |
| hrw | admin0: so, you got fresh 18.04 system, cloned kolla-ansible stable/train and then deployed all-in-one and it did not worked for you? | 14:08 |
| hrw | admin0: which distro/install_type images? | 14:08 |
| admin0 | distro is ubuntu 18.04 | 14:09 |
| admin0 | source based installed | 14:09 |
| hrw | ok | 14:10 |
| admin0 | hrw, my server prep is recorded here: https://asciinema.org/a/ig2c0AOq8xenGdOgzZxf99wNM | 14:10 |
| yoctozepto | admin0: looks like rmq is really down or under different address | 14:10 |
| yoctozepto | admin0: did you get past that "localhost" being undefined? | 14:11 |
| yoctozepto | it is really weird | 14:11 |
| hrw | ffs... py2/py3 is mess | 14:12 |
| admin0 | yes .. hostname should not be mapped to loopback, and i had to remove apt ansible but do pip install ansible>2.6 instead to get past that | 14:12 |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Fix use of IPA images in Bifrost without a checksum https://review.opendev.org/704619 | 14:13 |
| hrw | I am waiting for moment when we get c8 working and drop c7 | 14:13 |
| hrw | then I plan to hunt for every py2 use and kill it | 14:13 |
| yoctozepto | admin0: ah, ansible from apt will not work with venv | 14:14 |
| admin0 | its not inside venv | 14:14 |
| admin0 | venv gave this issue: https://gist.github.com/a1git/29ad3a1919fbf7b2dae1310ca61eb708 which could not solve | 14:15 |
| admin0 | wrong one .. this one : https://gist.github.com/a1git/9fc701e4d2aaec41f46499cefe2ede78 | 14:16 |
| yoctozepto | admin0: ah, it's a very old ansible :D 2.5 | 14:16 |
| yoctozepto | yeah, ansible from apt is just too old | 14:16 |
| yoctozepto | and the venv problem I could not reproduce | 14:16 |
| yoctozepto | whatever I did it did the right thing, very weird | 14:16 |
| yoctozepto | obviously I am doing that one thing differently | 14:17 |
| yoctozepto | just no idea which | 14:17 |
| r3ap3r | hrw: kill with extreme prejudice right? lol | 14:18 |
| yoctozepto | r3ap3r: berserk mode 8-) | 14:18 |
| r3ap3r | Oh yeah!! | 14:18 |
| hrw | r3ap3r: I have to deal with rocky/stain/train/ussuri - each has own 'py2 or py3' issues. | 14:19 |
| hrw | r3ap3r: once ussuri will be c7 free there will be no place for py2 in it. | 14:19 |
| r3ap3r | Heat seekers and all! Grab my BFG from Doom and we should be good! ;-D | 14:19 |
| hrw | r3ap3r: IDKFA/IDDQD and we can work on py2 removal | 14:19 |
| yoctozepto | you, you... cheater! | 14:20 |
| yoctozepto | ;D | 14:20 |
| mnasiadka | hrw: so, oficially you need to convert to external ceph - which will be a nice challenge | 14:20 |
| hrw | yoctozepto: better cheat was to edit save file to have 65535/65535 ammo so idkfa was more useful | 14:21 |
| mnasiadka | Unofficially - you can extract keys from kolla-ceph and use them in external ceph mode ;) | 14:21 |
| r3ap3r | hrw: lol | 14:22 |
| hrw | mnasiadka: I will point kevinz to you for docs then ;d | 14:22 |
| *** vmixor has joined #openstack-kolla | 14:22 | |
| admin0 | https://asciinema.org/a/bRJ3SMAoqgEMimqF5WA90SmOs prechecks 2:25 deploy: 3:20 | 14:24 |
| mnasiadka | hrw: he'll be delighted - you have a production kolla-ceph deployment? | 14:27 |
| hrw | mnasiadka: more than one | 14:28 |
| hrw | mnasiadka: openstack infra uses two of them | 14:29 |
| mnasiadka | good one | 14:29 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 14:31 |
| hrw | admin0: out of luck I am then... | 14:35 |
| *** sri_ has quit IRC | 14:35 | |
| hrw | admin0: fresh 18.04 VM, kolla-ansible from git stable/train, ansible from pip, ubuntu-source:train images and deployment finished. now initrunonce and ready for vm | 14:35 |
| openstackgerrit | Isaac Prior proposed openstack/kayobe master: WIP: Apply sane defaults to Monasca configuration https://review.opendev.org/706853 | 14:36 |
| admin0 | if anyone can make me a setup.sh file that i can just run and get it up, that would be great | 14:36 |
| *** zhanglong has quit IRC | 14:38 | |
| hrw | admin0: https://pastebin.com/Z7Qrm8fB can be? | 14:38 |
| admin0 | works for me | 14:38 |
| hrw | (vevn3) hrw@ubuntu:~/kolla-ansible$ openstack server list | 14:39 |
| hrw | +--------------------------------------+-------+--------+--------------------+--------+---------+ | 14:39 |
| hrw | | ID | Name | Status | Networks | Image | Flavor | | 14:39 |
| hrw | +--------------------------------------+-------+--------+--------------------+--------+---------+ | 14:39 |
| hrw | | e4bed5c1-e1b1-4627-b4d9-d1974a0bdf89 | demo1 | ACTIVE | demo-net=10.0.0.60 | cirros | m1.tiny | | 14:39 |
| hrw | +--------------------------------------+-------+--------+--------------------+--------+---------+ | 14:39 |
| hrw | https://pastebin.com/cGnVXsLg shows running containers | 14:41 |
| hrw | http://paste.ubuntu.com/p/XcfjXgWQHd/ is globals.yml I used (set ubuntu/source/train, defined proper names instead of eth0/eth1, disabled haproxy) | 14:42 |
| admin0 | --configdir $PWD/hrw -- hrw is globals ? | 14:42 |
| *** peeejayz has joined #openstack-kolla | 14:42 | |
| hrw | admin0: (vevn3) hrw@ubuntu:~/kolla-ansible$ ls hrw | 14:43 |
| hrw | admin-openrc.sh all-in-one globals.yml passwords.yml | 14:43 |
| admin0 | ok | 14:43 |
| hrw | admin0: system is VM with two network cards | 14:44 |
| admin0 | same as mine | 14:44 |
| hrw | admin0: I prefer to use directory for all thsoe files instead of keeping them in /etc/kolla/ | 14:45 |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Fix kayobe overcloud provision without seed in limit https://review.opendev.org/704617 | 14:46 |
| peeejayz | Anyone running Kolla-Ansible in production here? | 14:46 |
| hrw | peeejayz: who would be so insane??? production can only be run with devstack! | 14:47 |
| kplant | devstack?? you cowboy | 14:47 |
| kplant | packstack only | 14:47 |
| peeejayz | oh dear... | 14:47 |
| kplant | peeejayz: yes, what's up | 14:47 |
| hrw | kplant: venv tarballs! | 14:48 |
| hrw | packed into rpms! | 14:48 |
| kplant | inside apks so you can run them on android | 14:48 |
| peeejayz | I'm just evaluating it and was wondering how many are running it and how it is after day 0 | 14:48 |
| kplant | peeejayz: pretty great | 14:48 |
| kplant | probably the most flexible thing i've found | 14:48 |
| peeejayz | Any major pitfalls that you'd highlight to stay clear of? | 14:48 |
| kplant | python sucks | 14:49 |
| kplant | so try to run everything in venvs :-) | 14:49 |
| kplant | that includes both the ansible host and target machines | 14:49 |
| peeejayz | I've deployed a few clusters, but we are about to go down a new cluster and I'm evaluating what is the current best deployment method | 14:49 |
| peeejayz | oh thats very interesting about the venvs | 14:49 |
| kplant | i also advise grabbing kolla & kolla-ansible from git | 14:50 |
| kplant | not from pypi | 14:50 |
| kplant | i currently run stable/stein and stable/train in my environment | 14:50 |
| hrw | +1 for 'from git' | 14:50 |
| kplant | just need to schedule an upgrade of the last site | 14:50 |
| peeejayz | how large are the clusters your running in it? | 14:50 |
| hrw | we do releases for pypi but git nearly always has some fixes which are not yet on pypi | 14:51 |
| kplant | pretty mediocre: 1 build, 3 ctl, 2 net, 3 monitoring, N compute | 14:51 |
| kplant | i only need 1 nova-cell (for now) per site | 14:51 |
| kplant | and then i have separate ceph clusters | 14:51 |
| *** ivve has quit IRC | 14:51 | |
| hrw | peeejayz: no idea, I am just using deployed cloud. And build images to get them deployed. | 14:51 |
| *** dking_desktop has joined #openstack-kolla | 14:56 | |
| admin0 | i removed localhost, put it my hostname, mapped it to an interface ip and now redploying | 15:00 |
| admin0 | i see how this will never be an issue in multinode setups . | 15:01 |
| *** sorin-mihai has quit IRC | 15:02 | |
| *** Luzi has quit IRC | 15:03 | |
| *** dougsz has quit IRC | 15:04 | |
| *** TrevorV has joined #openstack-kolla | 15:04 | |
| *** KeithMnemonic has joined #openstack-kolla | 15:06 | |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Use internal API endpoints in overcloud API interaction https://review.opendev.org/704621 | 15:12 |
| mnasiadka | yoctozepto: https://review.opendev.org/#/c/704309/ - happier now? | 15:16 |
| patchbot | patch 704309 - kolla-ansible - Remove kolla-ceph - 10 patch sets | 15:16 |
| *** primeministerp has joined #openstack-kolla | 15:17 | |
| vmixor | Hi, I have a question about variable naming convention in Kolla-Ansible | 15:17 |
| vmixor | should I use: audit_notifications_topic_cinder or cinder_audit_notifications_topic | 15:18 |
| vmixor | and | 15:18 |
| vmixor | enable_audit_notifications_cinder or enable_cinder_audit_notifications or cinder_enable_audit_notifications | 15:19 |
| admin0 | it has not stuck now .. still running.. | 15:19 |
| mnasiadka | vmixor: cinder_audit_notifications_topic and enable_cinder_audit_notifications gets +1 from me, mgoddard yoctozepto hrw osmanlicilegi? | 15:20 |
| hrw | vmixor: COMPONENT_something and enable_COMPONENT_something | 15:24 |
| osmanlicilegi | +1 for mnasiadka | 15:25 |
| vmixor | hrw, mnasiadka, osmanlicilegi: thanks | 15:25 |
| *** numans_ has quit IRC | 15:28 | |
| admin0 | still running .. looks like its finally through | 15:36 |
| admin0 | so question that went unanswered .. in case deploy fails for any reason, is there a way to pick it up from the role it failed | 15:38 |
| admin0 | like --start-at-role --start-at-step etc | 15:38 |
| admin0 | say keystone, glance, nova, neutron were OK and it failed at cinder .. how to tell it to restart at cinder and below | 15:38 |
| admin0 | so that time is not wasted redoing keystone .. neutron | 15:39 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 15:41 |
| *** sorin-mihai has joined #openstack-kolla | 15:41 | |
| hrw | admin0: as you may seen in my script I just restart everything. but that makes sense with all-in-one setups only iirc | 15:43 |
| admin0 | so this feature is not there yet ? | 15:44 |
| admin0 | is there a way to make a feature requst ? is it via git issues ? | 15:44 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 15:44 |
| mnasiadka | admin0: launchpad.net/kolla-ansible | 15:44 |
| *** dougsz has joined #openstack-kolla | 15:45 | |
| admin0 | https://bugs.launchpad.net/kolla-ansible/+bug/1862649 -- hopefully the details is enough | 15:51 |
| openstack | Launchpad bug 1862649 in kolla-ansible "kolla-ansible start at a particular task " [Undecided,New] | 15:51 |
| *** carli has quit IRC | 15:56 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [ussuri][goal] Drop python 2.7 support and testing https://review.opendev.org/695617 | 15:57 |
| *** Trevor_V has joined #openstack-kolla | 16:02 | |
| *** TrevorV has quit IRC | 16:06 | |
| admin0 | i thought it was working .. but during openvswitch , the network went down ( i was doing it via ssh) .. so have to login to the system using console and check again :D | 16:06 |
| admin0 | i forgot to tmux this time | 16:06 |
| admin0 | but i think the setup/packages issues are gone | 16:06 |
| *** pbing19 has quit IRC | 16:09 | |
| *** bengates has quit IRC | 16:29 | |
| *** bengates has joined #openstack-kolla | 16:30 | |
| *** bengates has quit IRC | 16:35 | |
| mgoddard | admin0: sounds like openvswitch swallowed your IP | 16:49 |
| mgoddard | admin0: neutron_external_interface can't have an IP | 16:49 |
| mgoddard | because it gets plugged into an OVS bridge | 16:49 |
| *** priteau has quit IRC | 16:56 | |
| *** rpittau is now known as rpittau|afk | 17:07 | |
| *** peeejayz has quit IRC | 17:17 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Docs: fix cells upgrade guide https://review.opendev.org/706669 | 17:25 |
| *** pbing19 has joined #openstack-kolla | 17:25 | |
| jopdorp | hi! | 17:26 |
| jopdorp | anyone know about ceilometer deployment failures related to gnocchiclient.exceptions.NotFound | 17:26 |
| jopdorp | ? | 17:26 |
| admin0 | ;) | 17:32 |
| admin0 | i have to redo-restart | 17:32 |
| admin0 | which is OK | 17:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Docs: fix cells upgrade guide https://review.opendev.org/706917 | 17:32 |
| admin0 | so quick question .. if everything kolla can setup is enabled , in an aio, how big of a resource would it need ? | 17:33 |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #openstack-kolla | 17:34 | |
| jopdorp | https://pastebin.com/f5zVjAY0 | 17:38 |
| jopdorp | that is my /var/log/kolla/ceilometer/ceilometer-upgrade.log | 17:38 |
| *** k_mouza has quit IRC | 17:39 | |
| jopdorp | any idea about that deployment failure? | 17:40 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: Allow OVS bridges to connect directly to interface https://review.opendev.org/705037 | 17:41 |
| jopdorp | it fails at step "ceilometer : Running Ceilometer bootstrap container" | 17:43 |
| *** jopdorp has quit IRC | 17:53 | |
| *** gfidente is now known as gfidente|afk | 17:55 | |
| *** openstackstatus has quit IRC | 17:57 | |
| *** openstack has joined #openstack-kolla | 18:00 | |
| *** ChanServ sets mode: +o openstack | 18:00 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Construct service configuration urls using kolla_internal_fqdn https://review.opendev.org/706627 | 18:24 |
| openstackgerrit | Merged openstack/kayobe stable/rocky: Don't list tasks for config dump https://review.opendev.org/704611 | 18:28 |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone. https://review.opendev.org/674579 | 18:36 |
| *** dcapone2004 has joined #openstack-kolla | 18:40 | |
| dking_desktop | Looking at ironic-dnsmasq.conf.j2, I noticed that the tftp options use api_interface_address. Is that generally useful? Isn't that typically going to be on a different network/VLAN than the baremetal servers are accessing during boot? | 18:44 |
| *** vedup19 has joined #openstack-kolla | 18:46 | |
| *** pbing19 has quit IRC | 18:48 | |
| *** jopdorp has joined #openstack-kolla | 18:49 | |
| yoctozepto | dking_desktop: I had similar suspicions when doing ipv6, but I think there is one more place this would need to change - the tftp server itself | 18:49 |
| dking_desktop | The tftp server seems to listen on 0.0.0.0, so if it had the right IP to reach it, then it seems to me that it should be fine. | 18:50 |
| yoctozepto | dking_desktop: and it would have to be routable anyway | 18:51 |
| dking_desktop | Yes, it would. ironic-pxe.json.j2 has "/usr/sbin/in.tftpd --verbose --foreground --user root --address 0.0.0.0:69 ...". I was able to reach it by putting in the IP of my controller. So, I can reach it that way, but just not over the separate internal api vlan. | 18:54 |
| dking_desktop | I was wondering what is commonly done, and if it might be better to allow that IP to be set separately. | 18:55 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add --clean to kolla-mergepwd https://review.opendev.org/706668 | 19:00 |
| yoctozepto | mgoddard could help ^ | 19:03 |
| *** vedup19 has quit IRC | 19:12 | |
| *** reph has joined #openstack-kolla | 19:21 | |
| dking_desktop | I've heard that most people in here are in Europe, and it's late there. | 19:34 |
| *** igordc has joined #openstack-kolla | 19:35 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 19:37 |
| openstackgerrit | Dincer Celik proposed openstack/kolla stable/rocky: Fixes fluentd for rocky https://review.opendev.org/706713 | 19:42 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 19:44 |
| openstackgerrit | Dincer Celik proposed openstack/kolla stable/rocky: Fixes fluentd for rocky https://review.opendev.org/706713 | 19:45 |
| *** numans has joined #openstack-kolla | 19:48 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Remove kolla-ceph https://review.opendev.org/704309 | 19:48 |
| *** igordc has quit IRC | 19:53 | |
| *** reph has quit IRC | 19:54 | |
| *** umbSublime has joined #openstack-kolla | 19:55 | |
| *** jopdorp has quit IRC | 19:55 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Docs: fix cells upgrade guide https://review.opendev.org/706917 | 20:01 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 20:04 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Tidy up tools/ subdirectory https://review.opendev.org/706496 | 20:10 |
| yoctozepto | dking_desktop: yeah, I am one of those "people in Europe" ;-) | 20:14 |
| kplant | people live there? | 20:15 |
| kplant | gross | 20:15 |
| *** cah_link has quit IRC | 20:24 | |
| *** cah_link has joined #openstack-kolla | 20:24 | |
| yoctozepto | kplant: yeah, happens | 20:30 |
| kplant | how do you function without an orange emperor? | 20:34 |
| *** kplant has quit IRC | 20:45 | |
| admin0 | here in NL we have the church of flying spaghetti monster to guide us | 21:19 |
| hrw | looks like first all-in-one using infra images will be deployed ;D | 21:26 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: WIP: Create 'infra' type of images https://review.opendev.org/698753 | 21:46 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Use 'infra' type of images https://review.opendev.org/706982 | 21:46 |
| hrw | maybe their dependencies should be done in reverse | 21:46 |
| hrw | or k-a part gets build of images enabled | 21:46 |
| hrw | https://pastebin.com/CM76qD7N - created VM in infra type deployment | 21:47 |
| * hrw off | 21:52 | |
| dking_desktop | yoctozepto, mgoddard: There's also 'ironic_inspector_internal_fqdn: "{{ kolla_internal_fqdn }}"' in group_vars/all.yml | 22:07 |
| *** vmixor has quit IRC | 22:09 | |
| *** k_mouza has joined #openstack-kolla | 22:16 | |
| *** k_mouza has quit IRC | 22:20 | |
| *** cah_link has quit IRC | 22:33 | |
| *** openstackgerrit has quit IRC | 22:46 | |
| *** Trevor_V has quit IRC | 22:56 | |
| *** jaicaa has quit IRC | 23:47 | |
| *** jaicaa has joined #openstack-kolla | 23:48 | |
| *** iniazi_ has joined #openstack-kolla | 23:53 | |
| *** iniazi has quit IRC | 23:55 | |
| *** openstackgerrit has joined #openstack-kolla | 23:59 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!