| *** masber has joined #openstack-ansible | 00:03 | |
| *** masber has quit IRC | 00:03 | |
| *** masber has joined #openstack-ansible | 00:03 | |
| *** masber has quit IRC | 00:04 | |
| *** masber has joined #openstack-ansible | 00:04 | |
| *** masber has quit IRC | 00:04 | |
| *** masber has joined #openstack-ansible | 00:06 | |
| *** masber has quit IRC | 00:06 | |
| *** masber has joined #openstack-ansible | 00:08 | |
| *** vnogin has quit IRC | 00:18 | |
| *** DanyC has joined #openstack-ansible | 02:01 | |
| *** DanyC has quit IRC | 02:05 | |
| openstackgerrit | Chason Chan proposed openstack/openstack-ansible-os_nova master: Remove useless code-block tag https://review.openstack.org/568170 | 02:06 |
|---|---|---|
| openstackgerrit | lei zhang proposed openstack/openstack-ansible-os_neutron master: Doc improvements https://review.openstack.org/568171 | 02:17 |
| *** vnogin has joined #openstack-ansible | 02:42 | |
| *** DanyC has joined #openstack-ansible | 03:01 | |
| *** DanyC has quit IRC | 03:06 | |
| *** vnogin has quit IRC | 03:15 | |
| *** vnogin_ has joined #openstack-ansible | 03:15 | |
| *** vnogin_ has quit IRC | 03:20 | |
| *** udesale has joined #openstack-ansible | 03:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 03:48 |
| *** radeks_ has joined #openstack-ansible | 03:58 | |
| *** threestrands has joined #openstack-ansible | 03:59 | |
| *** threestrands has quit IRC | 03:59 | |
| *** threestrands has joined #openstack-ansible | 03:59 | |
| *** threestrands_ has joined #openstack-ansible | 04:00 | |
| *** threestrands_ has quit IRC | 04:01 | |
| *** threestrands_ has joined #openstack-ansible | 04:02 | |
| *** DanyC has joined #openstack-ansible | 04:02 | |
| *** threestrands has quit IRC | 04:04 | |
| *** kevinbenton has joined #openstack-ansible | 04:05 | |
| *** DanyC has quit IRC | 04:07 | |
| *** radeks_ has quit IRC | 04:08 | |
| *** ethfci has joined #openstack-ansible | 04:21 | |
| *** dxiri has joined #openstack-ansible | 04:35 | |
| *** bhujay has joined #openstack-ansible | 04:37 | |
| olivierb- | Good morning. Could someone point me to some guidelines on how to setup partitions on multi-node OSA hosts please. The bootstrap-aio playbook does a lot of things for single node and I did not find (quick look) doc sections for this (as comparted to networking setup which can be easily found). Thanks | 04:37 |
| olivierb- | I guess that this also depends on the node role (controller, compute, storage) but was wonderning if there was some references setup written somewhere | 04:38 |
| *** threestrands_ has quit IRC | 04:39 | |
| *** armaan has joined #openstack-ansible | 04:50 | |
| *** mma has quit IRC | 04:51 | |
| *** udesale has quit IRC | 04:51 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Add service params to keep systems in sync https://review.openstack.org/568188 | 04:52 |
| *** armaan has quit IRC | 04:52 | |
| *** armaan has joined #openstack-ansible | 04:52 | |
| cloudnull | olivierb- idk of any specific partitioning guide | 04:53 |
| cloudnull | we have https://docs.openstack.org/project-deploy-guide/openstack-ansible/ocata/overview-requirements.html#disk-recommendations | 04:54 |
| cloudnull | we also have these preseeds https://github.com/openstack/openstack-ansible-ops/blob/master/multi-node-aio/playbooks/pxe/configs/debian/compute.config.j2#L109-L154 | 04:56 |
| cloudnull | which are used with test instances however may be useful when creating the partition layout | 04:56 |
| *** udesale has joined #openstack-ansible | 04:56 | |
| olivierb- | many thanks cloudnull | 04:57 |
| cloudnull | hope it helps | 04:59 |
| *** DanyC has joined #openstack-ansible | 05:02 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 05:03 |
| *** threestrands has joined #openstack-ansible | 05:04 | |
| *** threestrands has quit IRC | 05:07 | |
| *** DanyC has quit IRC | 05:19 | |
| *** udesale_ has joined #openstack-ansible | 05:23 | |
| *** udesale has quit IRC | 05:25 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 05:28 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Add service params to keep systems in sync https://review.openstack.org/568188 | 05:31 |
| *** dxiri has quit IRC | 05:33 | |
| *** radeks_ has joined #openstack-ansible | 05:34 | |
| *** radek__ has joined #openstack-ansible | 05:35 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Add Bionic testing https://review.openstack.org/567042 | 05:37 |
| *** evin has quit IRC | 05:42 | |
| *** vnogin has joined #openstack-ansible | 05:49 | |
| *** radeks_ has quit IRC | 05:50 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 06:00 |
| *** vnogin has quit IRC | 06:07 | |
| *** vnogin has joined #openstack-ansible | 06:10 | |
| *** evin has joined #openstack-ansible | 06:11 | |
| *** adrianreza has quit IRC | 06:12 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 06:13 |
| *** DanyC has joined #openstack-ansible | 06:14 | |
| *** vnogin has quit IRC | 06:15 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Implement clouds.yaml on test hosts https://review.openstack.org/568139 | 06:16 |
| openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_cloudkitty master: Configure cloudkitty-api to use uwsgi https://review.openstack.org/567168 | 06:16 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Begin testing opensuse and centos with nspawn https://review.openstack.org/568197 | 06:18 |
| *** DanyC has quit IRC | 06:30 | |
| *** pcaruana has joined #openstack-ansible | 06:31 | |
| evrardjp | olivierb-: indeed the project deployment guide is the documentation on how to deploy openstack ansible for production | 06:32 |
| evrardjp | oh great the patch to unblock master is blocked by package conflicts in epel | 06:37 |
| evrardjp | with* | 06:37 |
| *** aojea has joined #openstack-ansible | 06:38 | |
| *** mma has joined #openstack-ansible | 06:40 | |
| *** jbadiapa has quit IRC | 06:49 | |
| evrardjp | what the actual F*** ? | 06:49 |
| *** aojea has quit IRC | 06:51 | |
| evrardjp | woops sorry | 06:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-plugins stable/ocata: Remove distro package indexing for Ansible 2.0+ https://review.openstack.org/568208 | 06:57 |
| olivierb- | evrardjp thx, I have read that more carefully now but it seems that it talks more about disk requirements rather than partitioning guidelines and FS type choices (XFS vs EXTx vs ... ). For instance, I have seen in the AIO bootstrap playbook changes for defaults fs types and usage of GPT vs MSDOS disk labels. | 06:58 |
| olivierb- | is this documented/commented elsewhere ? | 06:58 |
| evrardjp | you do whatever you want | 07:00 |
| evrardjp | we do not recommend anything particularily | 07:00 |
| evrardjp | follow what upstream recommends | 07:00 |
| olivierb- | ok thx | 07:01 |
| evrardjp | cloudnull: wow thanks on the update of the lxc_hosts patch | 07:02 |
| *** epalper has joined #openstack-ansible | 07:04 | |
| evrardjp | you dealt with apparmor changes... I wasreluctant to do what you did, but at the end it works, which is the most important for now | 07:04 |
| evrardjp | so ty | 07:04 |
| *** armaan_ has joined #openstack-ansible | 07:05 | |
| *** armaan has quit IRC | 07:09 | |
| *** jbadiapa has joined #openstack-ansible | 07:14 | |
| *** jbadiapa has quit IRC | 07:27 | |
| *** armaan has joined #openstack-ansible | 07:29 | |
| *** armaan_ has quit IRC | 07:32 | |
| *** mbuil has joined #openstack-ansible | 07:35 | |
| *** sawblade6 has quit IRC | 07:37 | |
| *** tosky has joined #openstack-ansible | 07:37 | |
| *** sawblade6 has joined #openstack-ansible | 07:38 | |
| *** DanyC has joined #openstack-ansible | 07:39 | |
| *** rpittau has joined #openstack-ansible | 07:53 | |
| *** kevinbenton has quit IRC | 07:56 | |
| *** jbadiapa has joined #openstack-ansible | 08:05 | |
| *** ivve has quit IRC | 08:11 | |
| *** erolg has joined #openstack-ansible | 08:12 | |
| *** udesale_ has quit IRC | 08:41 | |
| *** udesale has joined #openstack-ansible | 08:43 | |
| odyssey4me | olivierb- if you can work out something you'd like to see in the docs, it'd be great if you could push a patch up | 08:51 |
| odyssey4me | evrardjp and yeah, it's a bit annoying that we have that centos package issue - that's why https://review.openstack.org/567904 was pushed up, but that did the wrong thing, so we did https://review.openstack.org/568135 | 08:52 |
| olivierb- | odyssey4me yep will do eventually, no pb, once I figure out everything/most what's related to this topic | 08:52 |
| odyssey4me | now we have to find an appropriate solution | 08:52 |
| odyssey4me | olivierb- I guess the most important thing to know from an OSA standpoint is that everything on the hosts goes into /openstack | 08:52 |
| odyssey4me | anything else is general openstack stuff, not OSA specific | 08:53 |
| odyssey4me | some things for OSA depend on what backing store you're using for the containers - btrfs, directory, lvm, etc | 08:53 |
| olivierb- | true, thx for the notice | 08:53 |
| odyssey4me | then others depend on what backing store you're using for glance/images and nova/ephemeral | 08:53 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567796 | 08:55 |
| odyssey4me | evrardjp are you going to try and figure it out, or shall I fire up a host to try? not sure if hwoarang is around yet | 08:56 |
| *** harryhuang_ has joined #openstack-ansible | 08:56 | |
| odyssey4me | another option, of course, is to mark those tests non-voting for now | 08:56 |
| hwoarang | what happened? | 08:56 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 08:57 |
| odyssey4me | hwoarang well, the cinder role is broken - and https://review.openstack.org/#/c/568134/ fixes it - but the issue which got you submitting https://review.openstack.org/567904 didn't fix that problem, so it's still there - and we had to do https://review.openstack.org/568135 because it got worse | 08:57 |
| hwoarang | aha | 08:59 |
| hwoarang | broken because of the distro install on centos | 08:59 |
| odyssey4me | yeah, and I think that's due to packages already installed | 08:59 |
| hwoarang | it seems an epel package requires something that doesn't exist on epel | 08:59 |
| hwoarang | i have no submitted https://review.openstack.org/567870 which makes centos -nv | 08:59 |
| odyssey4me | do we still have a task to uninstall known problem packages - I think we do in the neutron role, but not cinder | 08:59 |
| hwoarang | *now | 08:59 |
| hwoarang | yeah i have a vm now running to investigate :( | 09:00 |
| hwoarang | but this needs to be done on glance and any other role which installs packages | 09:00 |
| evrardjp | we could probably do it in openstack_hosts | 09:00 |
| evrardjp | to do it accross the board | 09:00 |
| hwoarang | yeah i was waiting for mnaser to wait up to ask him about that | 09:01 |
| hwoarang | but i will spend some time on this this morning | 09:01 |
| odyssey4me | I guess it depends on the packages - some of the known problem packages will be general, but some specific to the role. | 09:01 |
| odyssey4me | hwoarang commented on another fix which is needed for cinder to make it work right | 09:01 |
| odyssey4me | in https://review.openstack.org/#/c/567870/3 | 09:01 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 09:02 |
| evrardjp | I think we should have it nv for a while due to 7.5 release | 09:02 |
| evrardjp | until someone picks that up | 09:03 |
| evrardjp | I liked the idea of using cloud prio | 09:03 |
| hwoarang | yeah this is definitely new because that test was passing in the initial distro install commit | 09:03 |
| evrardjp | it's sad we had to rollback | 09:03 |
| odyssey4me | cloud priorities is already in place | 09:03 |
| hwoarang | odyssey4me: evrardjp regarding the rollback , now that everything is on prio50 cloud is still not the prefered repo right? | 09:03 |
| odyssey4me | read the revert commit message more carefully | 09:03 |
| hwoarang | we should prob make it 40 or something | 09:03 |
| hwoarang | because it's 50 similar to every other repo. so it doesn't make it high prio | 09:04 |
| odyssey4me | could try that - you'd have to trawl through mhayden's commits which worked on all this stuff to figure out whether that was already tried | 09:05 |
| odyssey4me | I remember there were many, many attempts to get it where it is now | 09:05 |
| odyssey4me | The current conflict is with an epel package though, not one from another one of those repositories... | 09:06 |
| odyssey4me | I don't know if the priorities tell yum to uninstall and replace? | 09:06 |
| *** masber has quit IRC | 09:06 | |
| hwoarang | nooooo clue... | 09:06 |
| evrardjp | not a clue either | 09:06 |
| ArchiFleKs | Hi, is anyone using octavia in master, it was working a couple month ago, and now I get SSL error (http://paste.openstack.org/show/720889/), I noticed there were changes in the role and self generating cert, I destroyed the old amphorae afterward, I'm using the self generated cert. xgerman_ | 09:08 |
| evrardjp | 3s | 09:10 |
| evrardjp | woops | 09:10 |
| odyssey4me | evrardjp see my notes in https://review.openstack.org/#/c/568146 - specifically about that play placement... I was wondering where that's best suited myself... any thoughts for alternatives? maybe the utility container playbook? | 09:12 |
| evrardjp | I think the note is the most important. | 09:13 |
| evrardjp | utility is weird. I thought of using the openstack_hosts | 09:14 |
| odyssey4me | ok, so leave it there and just add a note to explain what it is | 09:14 |
| evrardjp | yes I think that's what gives you my positive vote | 09:14 |
| odyssey4me | I added it to keystone, because it's not useful until keystone is there | 09:14 |
| evrardjp | else it's easy to loose | 09:14 |
| evrardjp | yup that was what I said in my comment | 09:14 |
| evrardjp | I think it solely depends on what we do there | 09:14 |
| evrardjp | I guess cinder standalone is not a reality right now | 09:15 |
| evrardjp | so there is no point of taking this into consideration until later | 09:15 |
| evrardjp | and at that moment we can think of moving this | 09:15 |
| odyssey4me | well, clouds.yaml or openrc are not useful without keystone | 09:15 |
| evrardjp | yeah | 09:15 |
| evrardjp | I guess | 09:15 |
| evrardjp | thanks for the comments I understand the depends-on path better. We did very bad to have both :) | 09:16 |
| evrardjp | I mean both meta and running the role in plays | 09:16 |
| odyssey4me | yeah, not sure how we missed that | 09:17 |
| odyssey4me | also the playbooks had them running after the role, which is funny because the role needed the openrc file to be able to do the service setup | 09:17 |
| odyssey4me | so it was a waste anyway | 09:17 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Only implement openrc/clouds.yaml on a designated host https://review.openstack.org/568146 | 09:20 |
| odyssey4me | hopefully that's better :) | 09:21 |
| evrardjp | yeah, I wonder how much time we'll shave with that, probably a minute or so | 09:24 |
| evrardjp | per aio | 09:24 |
| evrardjp | hwoarang: if you could quickly vote on https://review.openstack.org/#/c/568208/ that would unblock O | 09:36 |
| evrardjp | see also: https://review.openstack.org/#/c/568155/ | 09:36 |
| evrardjp | oh I remember why we had that openrc everywhere in the past, it's because we were using a lot of commands in the roles themselves, like cinder blablabla... | 09:38 |
| evrardjp | odyssey4me: ^ | 09:38 |
| evrardjp | there is a risk when it's gonna be removed :) | 09:38 |
| evrardjp | from meta I mean | 09:38 |
| *** armaan has quit IRC | 09:43 | |
| *** armaan has joined #openstack-ansible | 09:44 | |
| odyssey4me | evrardjp yes, and as I work through every role I'll sort those out | 09:49 |
| odyssey4me | the plan is to have all of those run from a designated host | 09:49 |
| odyssey4me | where possible | 09:49 |
| evrardjp | move to use shade if need be | 09:50 |
| evrardjp | :) | 09:50 |
| odyssey4me | for some that may not be possible, in which case we need to implement the clouds.yaml on that node | 09:50 |
| evrardjp | yea | 09:50 |
| evrardjp | but I am pretty sure many things can now be moved to use openstack CLI | 09:50 |
| evrardjp | from any node | 09:50 |
| odyssey4me | anything shade-based should be able to use the designated node... and yeah, perhaps we could work on patches to shade/ansible to extend its functionality to include more things so that we can reduce the host install footprint | 09:51 |
| evrardjp | the only few remaining ones probably don't need keystone things | 09:51 |
| evrardjp | yes | 09:51 |
| openstackgerrit | Merged openstack/openstack-ansible-plugins stable/ocata: Remove distro package indexing for Ansible 2.0+ https://review.openstack.org/568208 | 09:51 |
| odyssey4me | hwoarang hmm, looks like https://review.openstack.org/567870 is failing all distro builds | 09:53 |
| odyssey4me | http://logs.openstack.org/70/567870/4/check/openstack-ansible-functional-distro_install-ubuntu-xenial/defd619/logs/ara-report/result/eb20df83-2105-40a2-ab8d-b10ce4aff508/ | 09:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.22 https://review.openstack.org/568155 | 09:54 |
| odyssey4me | hwoarang oh man, I'm confused about cinder_wsgi_bin, _cinder_uwsgi_bin and _cinder_wsgi_bin now | 09:58 |
| hwoarang | i understand... | 10:00 |
| odyssey4me | ah, the glance one is more consistent - and makes a little more sense | 10:01 |
| odyssey4me | so I'm guessing you'll fix the wsgi_bin vars to be uwsgi_bin vars | 10:01 |
| hwoarang | yep | 10:01 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 10:01 |
| odyssey4me | hwoarang I think one more edit is needed: https://review.openstack.org/#/c/567870/5/vars/distro_install.yml | 10:05 |
| hwoarang | yep | 10:06 |
| evrardjp | I was reviewing | 10:06 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 10:06 |
| evrardjp | this is hard to read :) | 10:06 |
| evrardjp | hwoarang: question for you: do we have vars that have different values if they are distro or source installed AND that are different based on the OS? | 10:08 |
| evrardjp | so a matrix of source/distro - os version | 10:09 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for using distribution packages for OpenStack services https://review.openstack.org/567808 | 10:10 |
| hwoarang | evrardjp: probably not | 10:11 |
| hwoarang | well | 10:11 |
| hwoarang | different vars are managed in /vars/$os files | 10:12 |
| hwoarang | those in defaults/* are the same | 10:12 |
| hwoarang | so the deployer only needs to choose distro or source install and the rest is magic | 10:13 |
| *** tomcsi has quit IRC | 10:15 | |
| *** tomcsi has joined #openstack-ansible | 10:16 | |
| *** tnogisto has joined #openstack-ansible | 10:21 | |
| evrardjp | hwoarang: ok | 10:24 |
| evrardjp | so why not simplify those vars then? | 10:24 |
| evrardjp | in other words, don't have the uwsgi var in vars/distro_install.yml, but keep it in vars/source_install.yml to override the distro version | 10:25 |
| evrardjp | that seems so convoluted :) | 10:26 |
| hwoarang | i dont know what to tell you | 10:29 |
| hwoarang | in theory the distro_install.yml could be merged into var/$os and then let source_install change these variables | 10:29 |
| *** nicolasbock has joined #openstack-ansible | 10:29 | |
| hwoarang | but it was done like that for keystone so I am just keeping the same pattern | 10:29 |
| hwoarang | i personally like the idea of keeping the distro_install and source_install files | 10:30 |
| hwoarang | be more explicit | 10:30 |
| hwoarang | instead of dumping everythin in vars/$os and override them somewhere else. it prob makes it somewhat harder to follow | 10:30 |
| evrardjp | I agree with you | 10:30 |
| evrardjp | that's not what I meant | 10:31 |
| evrardjp | maybe it is | 10:31 |
| evrardjp | please check my review comment | 10:31 |
| hwoarang | you also commented on the patchset #4 maybe #6 is better? | 10:31 |
| evrardjp | and let's discuss this | 10:31 |
| evrardjp | yeah | 10:31 |
| evrardjp | it's probably out of date now | 10:31 |
| Tahvok | Hey everyone! Can anyone tell me how their rabbitmq is holding up? We have a cluster with 2 infra machines and ~100 compute nodes. Currently, apart from running instances, nothing else is going on in this cluster, and rabbitmq is constantly taking 80-90% of a cpu core | 10:31 |
| odyssey4me | I suppose if the distro vars are imported first, then the install type vars, then the second can override the first | 10:31 |
| Tahvok | Is it normal behaviour? Or should I optimize rabbitmq somehow? | 10:32 |
| hwoarang | odyssey4me: yes but sometimes you need to do more stuff in vars/distro_install so if you drop that file you would have to copy that on every $os file | 10:32 |
| evrardjp | odyssey4me: that's what I mean: source install is basically not respecting the distro vars. We are, ourselves, overriding the default distro locations | 10:32 |
| evrardjp | hwoarang: we do not mean dropping the file | 10:33 |
| evrardjp | we mean if something, is overriding the default <os><version> file based on some scenario, it should be in its scenario file | 10:34 |
| evrardjp | it makes sense for me to say "by default for the OS, the uwsgi location is there" | 10:34 |
| evrardjp | and | 10:34 |
| evrardjp | "oh, but you're using source installs? Then the location is the location of your source, in /openstack/ ... " | 10:34 |
| hwoarang | well that could be extended to everything in distro_install | 10:35 |
| hwoarang | ie the default package list is this, oh but this is source install, so the default package list is that | 10:35 |
| evrardjp | Tahvok: cloudnull could help you there. | 10:35 |
| evrardjp | he wasn't using telemetry though | 10:35 |
| evrardjp | hwoarang: that sounds valid reasoning. | 10:36 |
| Tahvok | cloudnull: please look at my question above | 10:36 |
| odyssey4me | evrardjp I think that cloudnull is using telemetry on his own personal cluster | 10:36 |
| evrardjp | odyssey4me: I was refering to OSIC where he got the chance to see larger scale clouds :) | 10:37 |
| Tahvok | I don't use telemetry, I'm speaking of just looking at 'top' | 10:37 |
| evrardjp | ok | 10:37 |
| evrardjp | well | 10:37 |
| evrardjp | I haven't got an env of 100 nodes, so I can't say | 10:38 |
| Tahvok | does cloudnull has one? | 10:38 |
| odyssey4me | no, cloudnull's personal one is much, much smaller | 10:38 |
| odyssey4me | Tahvok not sure, but this may help? https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/master/defaults/main.yml#L106-L129 | 10:39 |
| evrardjp | Tahvok: cloudnull had one :) | 10:39 |
| odyssey4me | those could be adapted too - if you're not using notifications then perhaps set an expiry on them too | 10:40 |
| *** vnogin has joined #openstack-ansible | 10:41 | |
| evrardjp | hwoarang: anyway that's details. | 10:41 |
| evrardjp | TIL find: module doesn't fail if path doesn't exist. | 10:48 |
| *** armaan has quit IRC | 10:49 | |
| *** armaan has joined #openstack-ansible | 10:49 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client master: [WIP] Add Bionic testing https://review.openstack.org/567029 | 10:56 |
| *** armaan has quit IRC | 10:59 | |
| *** armaan has joined #openstack-ansible | 10:59 | |
| *** sar_ has joined #openstack-ansible | 11:01 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 11:01 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567796 | 11:02 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for using distribution packages for OpenStack services https://review.openstack.org/567808 | 11:02 |
| *** bhujay has quit IRC | 11:02 | |
| *** bhujay has joined #openstack-ansible | 11:02 | |
| *** evin has quit IRC | 11:03 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: Add support for using distribution packages for OpenStack services https://review.openstack.org/560308 | 11:04 |
| *** bhujay has quit IRC | 11:04 | |
| *** bhujay has joined #openstack-ansible | 11:05 | |
| *** bhujay has quit IRC | 11:12 | |
| *** vnogin has quit IRC | 11:13 | |
| *** vnogin has joined #openstack-ansible | 11:26 | |
| *** vnogin has quit IRC | 11:26 | |
| *** tomtom001 has quit IRC | 11:28 | |
| *** tomtom001 has joined #openstack-ansible | 11:28 | |
| *** vnogin has joined #openstack-ansible | 11:30 | |
| *** kaiokmo has joined #openstack-ansible | 11:36 | |
| *** vnogin has quit IRC | 11:36 | |
| *** tnogisto has quit IRC | 11:42 | |
| *** nicolasbock has quit IRC | 11:49 | |
| *** sar_ has quit IRC | 11:49 | |
| *** ansmith has quit IRC | 11:51 | |
| Miouge | hwoarang: OSA with openSUSE has the same features as with Ubuntu? | 11:51 |
| openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_cloudkitty master: Configure cloudkitty-api to use uwsgi https://review.openstack.org/567168 | 11:51 |
| *** sar_ has joined #openstack-ansible | 11:52 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 11:59 |
| mgariepy | morning everyone | 12:09 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Remove unused radosgw_ssl variable https://review.openstack.org/565693 | 12:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Move radosgw keystone config tasks to their own playbook https://review.openstack.org/565701 | 12:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 12:10 |
| Tahvok | Is there some place in osa to add my ssh key to connect to hosts? | 12:11 |
| *** nicolasbock has joined #openstack-ansible | 12:12 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: Add support for using distribution packages for OpenStack services https://review.openstack.org/560308 | 12:13 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 12:14 |
| odyssey4me | Tahvok by default it grabs the pub key from your deploy host, but you can also set lxc_container_ssh_key with your pubkey instead. https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/tasks/main.yml#L16-L32 | 12:14 |
| Tahvok | odyssey4me: I don't want to replace the key, I want to add additional key | 12:15 |
| odyssey4me | Tahvok ah, then just use an ansible ad-hoc command to copy the key everywhere | 12:15 |
| *** vnogin has joined #openstack-ansible | 12:17 | |
| Tahvok | odyssey4me: well, that's what I would do anyway | 12:19 |
| Tahvok | I was just hoping there is some hidden key management implemented somewhere | 12:19 |
| Tahvok | Thanks anyway! | 12:19 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 12:21 |
| *** harryhuang_ has quit IRC | 12:25 | |
| *** radek__ has quit IRC | 12:29 | |
| *** radeks has joined #openstack-ansible | 12:30 | |
| *** guilhermesp has joined #openstack-ansible | 12:37 | |
| *** tnogisto has joined #openstack-ansible | 12:42 | |
| mnaser | hey hwoarang | 12:45 |
| mnaser | can i help, i noticed you highlighted me | 12:45 |
| hwoarang | hello mnaser | 12:45 |
| hwoarang | yep we are seeing centos failures like this | 12:45 |
| hwoarang | http://logs.openstack.org/96/567796/5/check/openstack-ansible-functional-distro_install-centos-7-nv/8f95c4e/job-output.txt.gz#_2018-05-14_11_49_00_081839 | 12:46 |
| hwoarang | any clue? to me it seems like something in the epel repo is broken... | 12:46 |
| * mnaser looks | 12:47 | |
| mnaser | hwoarang: are you adding delorean-deps repo when installing from packages on centos? | 12:47 |
| hwoarang | no | 12:48 |
| mnaser | so question is are you adding the repos manually or are you installing the centos-release-openstack-queens package | 12:48 |
| mnaser | if you add the packages that should come fine, if you are adding the repo using ansible, you will need the extra repo which contains dependencies | 12:49 |
| hwoarang | mnaser: i am not adding any repos | 12:49 |
| hwoarang | any *new* repos | 12:49 |
| hwoarang | i am using whatever centos repos are being prepared in the openstack_hosts role | 12:49 |
| hwoarang | but why do you think it's a queens repo problem? the log seems to suggest that the package is coming from epel | 12:50 |
| hwoarang | Error: Package: python2-qpid-proton-0.21.0-3.el7.x86_64 (epel) | 12:50 |
| mnaser | hwoarang: because the deps repo might have a newer python2-qpid which works with that set of packages | 12:51 |
| mnaser | i think i see whats happening | 12:51 |
| mnaser | epel probably updated python2-qpid | 12:52 |
| mnaser | queens has python-qpid-proton-0.17.0-4.el7.x86_64.rpm | 12:52 |
| hwoarang | ok... but i have now clue how to resolve that on centos and i normally break stuff when i touch it so if you can submit something that fixes it that would be nice :) | 12:52 |
| mnaser | hwoarang: yes i'm thinking about this :) | 12:52 |
| hwoarang | great :) | 12:53 |
| *** dmsimard is now known as dmsimard|off | 12:58 | |
| mnaser | hwoarang: right, so, epel + rdo = not recommended and supported as per rdo team | 12:59 |
| hwoarang | :( | 13:00 |
| mnaser | i will submit a patch to clean this up | 13:00 |
| mnaser | especially that we're using containers | 13:00 |
| mnaser | i don't think we have a reason for epel | 13:00 |
| mnaser | we can only install it when we need it? | 13:00 |
| hwoarang | i was about to ask that | 13:00 |
| hwoarang | what does epel offer to us? | 13:00 |
| cloudnull | mornings | 13:00 |
| hwoarang | hello cloudnull | 13:00 |
| cloudnull | o/ | 13:00 |
| odyssey4me | as far as I remember, we need epel for nginx/uwsgi - but that's a vague memory | 13:01 |
| hwoarang | :/ | 13:01 |
| mnaser | yes im pretty sure you need epel for uwsgi | 13:01 |
| odyssey4me | if we can ditch epel, great - that's one less thing changing upstream that we need to care about | 13:01 |
| hwoarang | mnaser: but why would mixing the repos cause troubles if rdo has higher priority? | 13:01 |
| mnaser | but does rdo actually have higher priority? | 13:02 |
| odyssey4me | otherwise, we'll have to figure out how to do package filtering or how to properly prefer one repo over another, regardless of version | 13:02 |
| hwoarang | it's prio is 50 epel is at 99 | 13:02 |
| odyssey4me | yes it does | 13:02 |
| hwoarang | i guss that package 'foo' can exist in both repos but pulling 'A' from rdo and 'B' from epel and A and B do not like each other | 13:03 |
| hwoarang | just a theory... | 13:03 |
| cloudnull | Tahvok: rabbit can run faily hot but 90% on 100 nodes sounds like its doing too much or something is generally off. I'd look into checking out why. likely something has gone crazy in HA replication, there are too many queues without consumers, some client is hammering the cluster, etc. | 13:03 |
| cloudnull | I used cluster_metrics https://github.com/openstack/openstack-ansible-ops/tree/master/cluster_metrics/ when I was operating the OSIC, we collected a lot of data on Rabbit to help with troubleshooting. | 13:04 |
| *** radek__ has joined #openstack-ansible | 13:04 | |
| mnaser | hwoarang: i think thats whats happening as well | 13:05 |
| mnaser | is bigger priority = higher? | 13:06 |
| *** ansmith has joined #openstack-ansible | 13:06 | |
| mnaser | "The repositories with the lowest numerical priority number have the highest priority." | 13:06 |
| cloudnull | Tahvok I've got the elk_metrics_6x https://github.com/openstack/openstack-ansible-ops/tree/master/elk_metrics_6x deployed in a few environments now and the insight we've been able to gain into the environments has been great. | 13:07 |
| cloudnull | either solution is great in terms of observability | 13:07 |
| odyssey4me | evrardjp happy with https://review.openstack.org/#/c/567870/8 now? | 13:09 |
| Tahvok | cloudnull: thanks a lot! Will take a look at all this | 13:10 |
| cloudnull | none of those tools solves the immediate problem, but may help diagnose issue in the future. | 13:12 |
| cloudnull | rabbitmq has a nice blog full of useful information - http://www.rabbitmq.com/blog/2011/10/27/performance-of-queues-when-less-is-more - you might want to have a read there too. | 13:13 |
| *** geb has quit IRC | 13:17 | |
| *** admin0 has joined #openstack-ansible | 13:17 | |
| *** qwebirc31273 has joined #openstack-ansible | 13:17 | |
| qwebirc31273 | hi guys! i've deployed osa queens . now i'm trying to deploy auto scaling as a test on my environment but after set everything up.it gives an issue!! I dont know what to do !! could anyone help me?http://paste.openstack.org/show/720912/ | 13:19 |
| *** rmart04 has joined #openstack-ansible | 13:20 | |
| *** geb has joined #openstack-ansible | 13:21 | |
| *** vnogin has quit IRC | 13:23 | |
| odyssey4me | hwoarang evrardjp no use rechecking the integrated build for master right now - it is blocked by the cinder fact setting bug, which is fixed in https://review.openstack.org/567870 | 13:23 |
| evrardjp | yes I have realized later | 13:25 |
| evrardjp | I didn't really think there, my bad | 13:25 |
| evrardjp | oh its nv now | 13:25 |
| odyssey4me | yes, centos is nv for now seeing as it's blocking and broken | 13:26 |
| cloudnull | qwebirc31273 is ceilometer deployed in the environment? | 13:29 |
| *** cmart has joined #openstack-ansible | 13:30 | |
| cloudnull | I believe all of the auto scaling components are tied to ceilometer still. | 13:30 |
| * cloudnull could be totally wrong though, I don't use ceilometer or heat all that often | 13:30 | |
| odyssey4me | I believe it still needs aodh and ceilometer, yes | 13:30 |
| qwebirc31273 | cloudnull yes ceilometer aodh and genuchi deployed in my environnemt | 13:31 |
| odyssey4me | never done it personally, but saw some docs this w/end about it | 13:31 |
| *** vnogin has joined #openstack-ansible | 13:34 | |
| *** esberglu has joined #openstack-ansible | 13:35 | |
| cloudnull | qwebirc31273 I guess the next thing would be check that those services are running | 13:36 |
| *** jwitko_ has joined #openstack-ansible | 13:36 | |
| *** jwitko_ has quit IRC | 13:40 | |
| *** jwitko_ has joined #openstack-ansible | 13:40 | |
| *** armaan has quit IRC | 13:42 | |
| *** armaan has joined #openstack-ansible | 13:43 | |
| *** armaan has quit IRC | 13:47 | |
| *** armaan has joined #openstack-ansible | 13:47 | |
| *** armaan has quit IRC | 13:48 | |
| *** armaan has joined #openstack-ansible | 13:49 | |
| qwebirc31273 | cloudnull should i go inside every container and check the services ? | 13:52 |
| cloudnull | qwebirc31273 you could. if you have the logging node setup you can check the logs from there. | 13:53 |
| cloudnull | also check the lb make sure the backends are up | 13:53 |
| cloudnull | use the various openstack clients to run some basic commands against the cluster, etc. | 13:53 |
| *** sar_ has quit IRC | 13:54 | |
| qwebirc31273 | cloudnull thanks | 13:54 |
| *** devx has joined #openstack-ansible | 13:55 | |
| *** devx has quit IRC | 13:56 | |
| *** devx has joined #openstack-ansible | 13:59 | |
| qwebirc31273 | cloudnull i don't know what are the service names when i connect to the ceilometer container to check wheter it is up or not . is there any documentation? | 13:59 |
| *** dxiri has joined #openstack-ansible | 13:59 | |
| *** vnogin has quit IRC | 14:00 | |
| cloudnull | the service names will start with ceilometer, aodh, etc. docs for each role can be found here https://docs.openstack.org/openstack-ansible-os_ceilometer | 14:03 |
| cloudnull | just change the url to match the role name | 14:04 |
| *** mma has quit IRC | 14:14 | |
| *** udesale has quit IRC | 14:14 | |
| *** mma has joined #openstack-ansible | 14:15 | |
| *** throwsb1 has joined #openstack-ansible | 14:16 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: vars: Do not install build dependencies for distro installs https://review.openstack.org/567870 | 14:18 |
| *** mma has quit IRC | 14:19 | |
| *** cjloader has joined #openstack-ansible | 14:20 | |
| *** cjloader has quit IRC | 14:20 | |
| *** cjloader has joined #openstack-ansible | 14:21 | |
| cloudnull | odyssey4me looks like https://review.openstack.org/#/c/568134/ is in merge conflict. mind rebasing it ? | 14:28 |
| cloudnull | that's still blocking the gate, right? | 14:29 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Add mount options for better machinectl performance https://review.openstack.org/568303 | 14:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Doc improvements https://review.openstack.org/568171 | 14:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca stable/pike: Non-ascii characters in files https://review.openstack.org/567607 | 14:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca stable/queens: Non-ascii characters in files https://review.openstack.org/567609 | 14:36 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts stable/queens: Setup /dev/random and /dev/urandom device in cache prep (ubuntu/suse) https://review.openstack.org/568136 | 14:37 |
| *** cmart has quit IRC | 14:38 | |
| *** vnogin has joined #openstack-ansible | 14:42 | |
| *** dcdamien has quit IRC | 14:43 | |
| qwebirc31273 | cloudnull i've checked all services are up and working :((( | 14:47 |
| *** weezS has joined #openstack-ansible | 14:47 | |
| *** dcdamien has joined #openstack-ansible | 14:47 | |
| *** vnogin has quit IRC | 14:53 | |
| *** vnogin has joined #openstack-ansible | 14:53 | |
| *** armaan has quit IRC | 14:53 | |
| *** armaan has joined #openstack-ansible | 14:53 | |
| *** kstev has joined #openstack-ansible | 14:57 | |
| *** cmart has joined #openstack-ansible | 15:01 | |
| admin0 | if i need to add just magnum service on a running osa .. how do i run it using tags ? | 15:01 |
| *** pcaruana has quit IRC | 15:02 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_aodh master: Add support for using distribution packages for OpenStack services https://review.openstack.org/568311 | 15:04 |
| admin0 | hmm.. how do I add a new role .. it does not create the containers :D | 15:04 |
| *** DanyC has quit IRC | 15:13 | |
| *** tomtom001 has quit IRC | 15:14 | |
| *** tomtom001 has joined #openstack-ansible | 15:14 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/pike: Add a rule to prevent passwords to be logged https://review.openstack.org/561882 | 15:16 |
| *** mma has joined #openstack-ansible | 15:16 | |
| hwoarang | evrardjp: odyssey4me could you give your blessing to https://review.openstack.org/#/c/560308/ one more time? | 15:20 |
| *** mma has quit IRC | 15:20 | |
| evrardjp | ofc | 15:25 |
| hwoarang | merci beaucoup | 15:26 |
| evrardjp | hwoarang: commented there just real quick | 15:27 |
| evrardjp | I don't see a problem for inclusion | 15:28 |
| evrardjp | great jon | 15:28 |
| evrardjp | job | 15:28 |
| hwoarang | evrardjp: what kind of simplification are you talking about? | 15:30 |
| hwoarang | this code already exists on keystone | 15:30 |
| hwoarang | look at the diff https://review.openstack.org/#/c/560308/24/tasks/keystone_install.yml | 15:31 |
| *** vnogin has quit IRC | 15:36 | |
| evrardjp | I think I am confused | 15:38 |
| evrardjp | https://review.openstack.org/#/c/568134/2/tasks/cinder_install_source.yml | 15:38 |
| evrardjp | odyssey4me: did you see that patch got a merge conflict now? | 15:39 |
| *** vnogin has joined #openstack-ansible | 15:40 | |
| *** vnogin has quit IRC | 15:40 | |
| *** erolg has quit IRC | 15:40 | |
| hwoarang | evrardjp: the code in cinder is new. the code in keystone was there already | 15:41 |
| hwoarang | maybe that's why it's confusing | 15:42 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_cinder master: Add new voting jobs to gates https://review.openstack.org/568320 | 15:42 |
| evrardjp | sarcasm? :) | 15:42 |
| evrardjp | I thought this code was necessary :) | 15:42 |
| hwoarang | not sure | 15:42 |
| evrardjp | because that was the issue in the integrated | 15:43 |
| *** e-vad has joined #openstack-ansible | 15:43 | |
| evrardjp | so if that's causing integrated issues, I'd rather make sure we don't duplicate the error first. | 15:43 |
| evrardjp | reproduce* | 15:43 |
| hwoarang | i think you are confused | 15:43 |
| *** e-vad has left #openstack-ansible | 15:43 | |
| hwoarang | the new code in cinder was not very good and it broke stuff | 15:43 |
| hwoarang | which i have fixed ever since | 15:43 |
| evrardjp | omg and my tiny brain forgot that I already did a check experimental | 15:43 |
| odyssey4me | evrardjp I think you're mixing up my patch and hwoarang's | 15:44 |
| odyssey4me | hwoarang's fixed the issue for the cinder role, so I need to revise my patch to make it easier to read | 15:44 |
| evrardjp | odyssey4me: ok | 15:44 |
| evrardjp | it was just readability | 15:44 |
| evrardjp | ok one can go without the other then | 15:45 |
| evrardjp | great | 15:45 |
| evrardjp | hwoarang: if you change the gate jobs (see my last comment, I am good then) | 15:45 |
| hwoarang | evrardjp: none of the other roles have these jobs in gate pipeline :( | 15:45 |
| hwoarang | i wanted to do that when all the roles are fixed to use distro packages because i consider this as a half-baked feature right now | 15:46 |
| evrardjp | half baked but voting? | 15:46 |
| evrardjp | ok | 15:47 |
| evrardjp | that's weird | 15:47 |
| evrardjp | but ok | 15:47 |
| hwoarang | it's like the neutron voting jobs | 15:47 |
| hwoarang | that only exist in the check pipeline | 15:47 |
| evrardjp | that's something we should fix :D | 15:47 |
| hwoarang | ok i will do another round of review for all the roles | 15:47 |
| evrardjp | I can do it | 15:47 |
| hwoarang | to add them to both pipelines | 15:48 |
| evrardjp | I have to do it anyway | 15:48 |
| evrardjp | I was planning to add bionic, so it can be done at the same time. | 15:48 |
| hwoarang | ok | 15:48 |
| *** nicolasbock has quit IRC | 15:50 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Ensure interface ordering is consistent https://review.openstack.org/568321 | 15:50 |
| cloudnull | admin0 did you get it going? | 15:50 |
| admin0 | yep :) | 15:50 |
| *** vnogin has joined #openstack-ansible | 15:50 | |
| admin0 | only fedoratomic-25 works though .. 26/27/28 just fails | 15:51 |
| *** MikeW has joined #openstack-ansible | 15:51 | |
| cloudnull | interesting. DimGR might have some thoughts on that? | 15:52 |
| *** nicolasbock has joined #openstack-ansible | 15:53 | |
| *** vnogin has quit IRC | 15:56 | |
| *** vnogin has joined #openstack-ansible | 15:56 | |
| * admin0 is gone .. will be back later | 15:57 | |
| *** admin0 has quit IRC | 15:57 | |
| *** radek__ has quit IRC | 15:58 | |
| DimGR | cloudnull ping | 16:01 |
| DimGR | ah admin0 left | 16:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts stable/pike: Setup /dev/random and /dev/urandom device in cache prep (ubuntu/suse) https://review.openstack.org/568323 | 16:07 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Update all SHAs for 17.0.5 https://review.openstack.org/568149 | 16:08 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts stable/ocata: Setup /dev/random and /dev/urandom device in cache prep (ubuntu/suse) https://review.openstack.org/568325 | 16:09 |
| *** vnogin has quit IRC | 16:10 | |
| *** gyee has joined #openstack-ansible | 16:13 | |
| *** vnogin has joined #openstack-ansible | 16:18 | |
| *** vnogin has quit IRC | 16:18 | |
| *** pcaruana has joined #openstack-ansible | 16:20 | |
| *** dave-mccowan has joined #openstack-ansible | 16:23 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.14 https://review.openstack.org/568153 | 16:25 |
| *** armaan has quit IRC | 16:27 | |
| *** armaan has joined #openstack-ansible | 16:28 | |
| *** dave-mccowan has quit IRC | 16:28 | |
| *** dave-mccowan has joined #openstack-ansible | 16:31 | |
| *** MikeW has quit IRC | 16:32 | |
| *** rmart04 has quit IRC | 16:33 | |
| *** sar_ has joined #openstack-ansible | 16:39 | |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Add mount options for better machinectl performance https://review.openstack.org/568303 | 16:43 |
| *** CobHeadI is now known as CobHead | 16:43 | |
| *** CobHead has joined #openstack-ansible | 16:43 | |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Ensure interface ordering is consistent https://review.openstack.org/568321 | 16:46 |
| *** cmart has quit IRC | 17:00 | |
| *** epalper has quit IRC | 17:01 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Use fan out when deploying with nspawn https://review.openstack.org/568331 | 17:02 |
| *** DanyC has joined #openstack-ansible | 17:04 | |
| *** mugsie has quit IRC | 17:06 | |
| *** mugsie has joined #openstack-ansible | 17:06 | |
| *** mugsie has quit IRC | 17:06 | |
| *** mugsie has joined #openstack-ansible | 17:06 | |
| *** DanyC has quit IRC | 17:08 | |
| *** mugsie has quit IRC | 17:08 | |
| *** radeks has quit IRC | 17:09 | |
| *** mugsie has joined #openstack-ansible | 17:12 | |
| *** mugsie has quit IRC | 17:12 | |
| *** mugsie has joined #openstack-ansible | 17:12 | |
| *** mbuil has quit IRC | 17:17 | |
| cloudnull | ^ All of the containers online using ~900MB - http://paste.openstack.org/show/720940/ | 17:18 |
| evrardjp | that's nice :) | 17:24 |
| evrardjp | we need to solve the issue with disk space though, as I see that it's using btrfs | 17:24 |
| cloudnull | yup. | 17:25 |
| cloudnull | nspawn uses btrfs | 17:25 |
| cloudnull | I've never been able to replicate the disk space issue. I really have no idea what caused that. | 17:26 |
| cloudnull | it doesnt seem to be a big issue though and im hoping that the recent changes in mount options will solve it. | 17:27 |
| cloudnull | evrardjp is this still blockng us https://review.openstack.org/#/c/568134/ ? | 17:28 |
| cloudnull | in the integrated gate that is ? | 17:29 |
| evrardjp | I wasn't really fully aware of the issue. I think hwoarang has now fixed the integrated gates, and now this is bonus. | 17:30 |
| *** tosky has quit IRC | 17:30 | |
| evrardjp | I've checked experimental on your patch | 17:31 |
| evrardjp | so we'll see | 17:31 |
| jrosser | i was seeing cinder failures across * earlier, is that fixed? | 17:31 |
| cloudnull | evrardjp also - https://review.openstack.org/#/q/owner:jean-philippe%2540evrard.me+status:open - odyssey4me had a question on making bionic voting are you going to change the prs or can we get them moving forward as is ? | 17:31 |
| cloudnull | evrardjp " checked experimental " ? | 17:32 |
| *** radeks has joined #openstack-ansible | 17:33 | |
| evrardjp | cloudnull: I am talking about https://review.openstack.org/#/c/466179/29 | 17:33 |
| cloudnull | ah nice! | 17:34 |
| evrardjp | for bionic, I haven't got the chance to update them with that. I was busy with the galera_client at some point in the day, but paperwork kept me from finishing all those | 17:34 |
| cloudnull | it'd be good to get that knocked out. then i can do the rest. | 17:34 |
| *** tosky has joined #openstack-ansible | 17:34 | |
| cloudnull | I still have https://gist.github.com/cloudnull/401efe182310884505f7b26ad8e44282 | 17:34 |
| evrardjp | let me write that in an etherpad | 17:35 |
| cloudnull | ok. | 17:35 |
| evrardjp | wait | 17:35 |
| evrardjp | not sure we are talking about the same thing | 17:35 |
| evrardjp | I didn't push anything for the os_* yet, they are all in my environment if I am not wrong | 17:35 |
| cloudnull | ? | 17:36 |
| evrardjp | so I guess your link is about using systemd role? | 17:36 |
| cloudnull | yes | 17:36 |
| evrardjp | not about bionic support | 17:36 |
| evrardjp | ok :) | 17:36 |
| evrardjp | got it :) | 17:36 |
| cloudnull | I have bionic deployed within my hardware lab | 17:36 |
| cloudnull | most of the things are working with the patches you've pushed. | 17:37 |
| odyssey4me | I expect that it's just a matter of getting the deps right to make the patches pass. | 17:38 |
| odyssey4me | Especially the low level roles, those just just go voting straight up. | 17:38 |
| cloudnull | https://twitter.com/cloudnull/status/995058259462680577 - tested with the pxelinux deployment things we have in the ops repo and cherry-picking most of your PRs I have a functional host. | 17:39 |
| cloudnull | a lot of them just work (-tm) with the var file in place. | 17:40 |
| cloudnull | i think it makes sense to get rid of the "-nv" in the job name, but otherwise those PRs all lgtm :) | 17:40 |
| osnaya | quit | 17:58 |
| *** osnaya has quit IRC | 17:58 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add cleanup process when using embedded ansible https://review.openstack.org/568339 | 18:03 |
| *** lvdombrkr has joined #openstack-ansible | 18:09 | |
| *** aludwar1 has joined #openstack-ansible | 18:21 | |
| *** aludwar1 has quit IRC | 18:21 | |
| *** Miouge has quit IRC | 18:22 | |
| *** idlemind has joined #openstack-ansible | 18:27 | |
| *** hamzy has quit IRC | 18:30 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_aodh master: Add support for using distribution packages for OpenStack services https://review.openstack.org/568311 | 18:31 |
| *** hamzy has joined #openstack-ansible | 18:33 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-config_template master: Add Bionic testing https://review.openstack.org/567014 | 18:40 |
| *** Guest16323 is now known as mgagne | 18:40 | |
| *** mgagne has joined #openstack-ansible | 18:41 | |
| *** markvoelker_ has joined #openstack-ansible | 18:42 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_mount master: Add Bionic testing https://review.openstack.org/567015 | 18:42 |
| *** markvoelker has quit IRC | 18:43 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_networkd master: Add Bionic testing https://review.openstack.org/567016 | 18:44 |
| *** DanyC has joined #openstack-ansible | 18:46 | |
| *** DanyC_ has joined #openstack-ansible | 18:46 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_service master: Add Bionic testing https://review.openstack.org/567017 | 18:46 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-apt_package_pinning master: Add Bionic testing https://review.openstack.org/567018 | 18:48 |
| *** radeks has quit IRC | 18:48 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 18:48 |
| *** markvoelker has joined #openstack-ansible | 18:49 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client master: Add Bionic testing https://review.openstack.org/567024 | 18:49 |
| *** DanyC has quit IRC | 18:50 | |
| *** markvoelker_ has quit IRC | 18:50 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client master: Add Bionic testing https://review.openstack.org/567029 | 18:52 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-haproxy_server master: Add Bionic testing https://review.openstack.org/567031 | 18:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-memcached_server master: Add Bionic testing https://review.openstack.org/567035 | 18:55 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Add Bionic testing https://review.openstack.org/567038 | 18:56 |
| *** vnovakov has joined #openstack-ansible | 18:57 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_openrc master: Add Bionic testing https://review.openstack.org/567041 | 18:57 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-pip_install master: Add Bionic testing https://review.openstack.org/567045 | 18:58 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rsyslog_client master: Add Bionic testing https://review.openstack.org/567047 | 18:59 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rsyslog_server master: Add Bionic testing https://review.openstack.org/567048 | 19:00 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server master: Add Bionic testing https://review.openstack.org/567030 | 19:02 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: Add Bionic testing https://review.openstack.org/567034 | 19:04 |
| *** DanyC_ has quit IRC | 19:06 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing https://review.openstack.org/567032 | 19:06 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-nspawn_hosts master: Add Bionic testing https://review.openstack.org/567042 | 19:10 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-plugins master: Add Bionic testing https://review.openstack.org/567044 | 19:13 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible-ops master: Ansible inventory and keepalive support https://review.openstack.org/568354 | 19:19 |
| openstackgerrit | Merged openstack/openstack-ansible-apt_package_pinning master: Add Bionic testing https://review.openstack.org/567018 | 19:21 |
| *** lvdombrkr89 has joined #openstack-ansible | 19:21 | |
| *** lvdombrkr has quit IRC | 19:24 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Add task to regather facts once the role completes https://review.openstack.org/568355 | 19:25 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible-ops master: Ansible inventory and keepalive support https://review.openstack.org/568354 | 19:26 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Move radosgw keystone config tasks to their own playbook https://review.openstack.org/565701 | 19:26 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 19:26 |
| openstackgerrit | Merged openstack/ansible-config_template master: Add Bionic testing https://review.openstack.org/567014 | 19:29 |
| openstackgerrit | Merged openstack/ansible-role-systemd_mount master: Add Bionic testing https://review.openstack.org/567015 | 19:29 |
| openstackgerrit | Merged openstack/ansible-role-systemd_service master: Add Bionic testing https://review.openstack.org/567017 | 19:29 |
| openstackgerrit | Merged openstack/ansible-role-systemd_networkd master: Add Bionic testing https://review.openstack.org/567016 | 19:29 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Add option to run a full config update if required https://review.openstack.org/568356 | 19:30 |
| *** radeks has joined #openstack-ansible | 19:32 | |
| *** kysse has quit IRC | 19:33 | |
| *** radeks_ has joined #openstack-ansible | 19:34 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Add Bionic testing https://review.openstack.org/567042 | 19:34 |
| *** dave-mccowan has quit IRC | 19:36 | |
| *** radeks has quit IRC | 19:37 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 19:39 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 19:39 |
| jrosser | win 23 | 19:40 |
| *** kysse has joined #openstack-ansible | 19:41 | |
| cloudnull | jrosser missing a couple bits on your win32 environment? | 19:42 |
| cloudnull | :D | 19:42 |
| cloudnull | o/ kysse | 19:42 |
| *** devx has quit IRC | 19:43 | |
| jrosser | cloudnull: a version of windows from teh future surely? | 19:44 |
| cloudnull | haha | 19:44 |
| *** devx has joined #openstack-ansible | 19:44 | |
| cloudnull | 64bit is SO last year! | 19:44 |
| evrardjp | that's a terrible password | 19:45 |
| evrardjp | jrosser: please change your password now :p | 19:45 |
| evrardjp | hahah | 19:45 |
| jrosser | you lot and your new fangled irc clients | 19:46 |
| evrardjp | haha | 19:47 |
| evrardjp | I am using a good old ncurses client | 19:47 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_server master: Add Bionic testing https://review.openstack.org/567048 | 19:48 |
| evrardjp | wow things are merging | 19:49 |
| jrosser | evrardjp: speaking of which is the cinder fix merged yet? | 19:49 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing https://review.openstack.org/567032 | 19:49 |
| evrardjp | jrosser: to be honest I am little lost in the changes, but I think it's fine now | 19:50 |
| *** radeks__ has joined #openstack-ansible | 19:51 | |
| DimGR | HI | 19:51 |
| cloudnull | o/ DimGR | 19:52 |
| DimGR | i missed it when you pinged me | 19:52 |
| jrosser | evrardjp: ok, i'll recheck some stuff then.... | 19:52 |
| cloudnull | no worries. | 19:52 |
| cloudnull | i think admin0 figured it out. | 19:52 |
| *** radeks_ has quit IRC | 19:53 | |
| *** dave-mccowan has joined #openstack-ansible | 19:53 | |
| DimGR | atomics are very locked down | 19:54 |
| odyssey4me | jrosser yeah, the fix for the general cinder failure is in | 19:55 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Add Bionic testing https://review.openstack.org/567029 | 19:55 |
| openstackgerrit | Merged openstack/openstack-ansible-memcached_server master: Add Bionic testing https://review.openstack.org/567035 | 19:55 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_openrc master: Add Bionic testing https://review.openstack.org/567041 | 19:55 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client master: Add Bionic testing https://review.openstack.org/567047 | 19:55 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Add Bionic testing https://review.openstack.org/567024 | 19:56 |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server master: Add Bionic testing https://review.openstack.org/567031 | 19:56 |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install master: Add Bionic testing https://review.openstack.org/567045 | 19:56 |
| odyssey4me | so the integrated build on master should work fine again - barring the sometimes tempest failures | 19:56 |
| jrosser | ah excellent | 19:56 |
| jrosser | i had a *massive* hack at the limited connectivity docs too if someone wants a look https://review.openstack.org/#/c/568023/ | 19:56 |
| *** fghaas has joined #openstack-ansible | 19:59 | |
| openstackgerrit | Florian Haas proposed openstack/openstack-ansible-os_neutron stable/pike: Add default for neutron_ovs_service_name https://review.openstack.org/568360 | 20:01 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Add Bionic testing https://review.openstack.org/567038 | 20:03 |
| evrardjp | I am done for the day | 20:04 |
| evrardjp | tty tomorrow | 20:04 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_aodh master: Add support for using distribution packages for OpenStack services https://review.openstack.org/568311 | 20:04 |
| *** ansmith has quit IRC | 20:04 | |
| *** radeks_ has joined #openstack-ansible | 20:06 | |
| *** aludwar1 has joined #openstack-ansible | 20:07 | |
| *** aludwar1 has quit IRC | 20:07 | |
| *** radeks__ has quit IRC | 20:08 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Correct galera cluster maintainance example wording https://review.openstack.org/568362 | 20:10 |
| *** vnovakov has quit IRC | 20:20 | |
| *** klamath has quit IRC | 20:20 | |
| *** vnovakov has joined #openstack-ansible | 20:20 | |
| *** qwebirc31273 has quit IRC | 20:23 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/ansible-role-systemd_networkd master: Make the networkd role distro agnostic https://review.openstack.org/568367 | 20:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Add new voting jobs to gates https://review.openstack.org/568320 | 20:28 |
| *** vnovakov has quit IRC | 20:29 | |
| *** devx has quit IRC | 20:30 | |
| *** devx has joined #openstack-ansible | 20:30 | |
| *** vnovakov has joined #openstack-ansible | 20:30 | |
| *** vnovakov has quit IRC | 20:30 | |
| *** devx has quit IRC | 20:31 | |
| *** devx has joined #openstack-ansible | 20:31 | |
| *** devx has quit IRC | 20:34 | |
| *** devx has joined #openstack-ansible | 20:34 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Allow integration of externally deployed RadosGW https://review.openstack.org/568276 | 20:36 |
| *** devx has quit IRC | 20:38 | |
| *** devx has joined #openstack-ansible | 20:38 | |
| *** devx has quit IRC | 20:38 | |
| *** klamath has joined #openstack-ansible | 20:39 | |
| *** devx has joined #openstack-ansible | 20:39 | |
| *** pcaruana has quit IRC | 20:43 | |
| *** lvdombrkr89 has quit IRC | 20:45 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Add cleanup process when using embedded ansible https://review.openstack.org/568339 | 20:51 |
| *** devx has quit IRC | 20:53 | |
| *** devx has joined #openstack-ansible | 20:53 | |
| idlemind | anyone else seeing venv's failing to build for the repo_build in osa/pike (latest - as of a few hours ago) | 20:54 |
| idlemind | so far argparse and cryptography failing to be found | 20:54 |
| idlemind | on centos | 20:54 |
| cloudnull | nope. | 20:55 |
| guilhermesp | idlemind: I'm facing a problem with repo_build today | 20:55 |
| guilhermesp | let me paste some of the outputs using -vvvvv | 20:55 |
| idlemind | cloudnull thx guilhermesp | 20:55 |
| idlemind | thx too | 20:55 |
| guilhermesp | http://paste.openstack.org/show/720955/ | 20:56 |
| guilhermesp | here | 20:56 |
| guilhermesp | so, the task uses all the retries for all services (heat, magnum, glance, etc | 20:57 |
| idlemind | interesting, mine outright fail on missing dependencies | 20:57 |
| idlemind | i might just start w/cloudnull's favorite and drop the repo container before i go too deep | 20:58 |
| cloudnull | ++ | 20:58 |
| idlemind | assuming this run through fails as well | 20:58 |
| cloudnull | nuke it ! | 20:58 |
| cloudnull | we're both of these failures new deployments ? | 20:58 |
| idlemind | but i never know with you clever folks; always changin' knobs :) so gotta check first ... mine was a minor minor version bump that's having issues | 20:58 |
| idlemind | within osa/pike | 20:59 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Add Bionic testing https://review.openstack.org/567034 | 20:59 |
| cloudnull | idlemind was that a minor bump to the latest stable tag? | 20:59 |
| idlemind | ya | 20:59 |
| cloudnull | or some sha within the branch ? | 20:59 |
| cloudnull | ok. | 20:59 |
| * cloudnull goes to try things | 21:00 | |
| idlemind | lol thx | 21:00 |
| guilhermesp | I can't go deep right now about my issue, but I've seeing some topics about it in eavesdrop. So, gonna try a few things tomorrow and anything I hope you guys can help me. | 21:04 |
| guilhermesp | here's an example: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2017-06-27.log.html | 21:06 |
| cloudnull | guilhermesp are you on the latest stable/pike tag too ? | 21:07 |
| *** ansmith has joined #openstack-ansible | 21:08 | |
| *** armaan has quit IRC | 21:08 | |
| *** armaan has joined #openstack-ansible | 21:08 | |
| guilhermesp | cloudnull: no, I am using 16.0.0 | 21:10 |
| cloudnull | guilhermesp is this a new deployment ? | 21:10 |
| cloudnull | or an update to an existing one? | 21:10 |
| *** armaan has quit IRC | 21:11 | |
| guilhermesp | cloudnull: nops, I deployed a few months ago. I'm trying to add monasca manually using os_monasca role, so I droped all extra contents in the right places, but I don't think this is a problem of the monasca | 21:11 |
| *** armaan has joined #openstack-ansible | 21:11 | |
| guilhermesp | following this doc https://docs.openstack.org/openstack-ansible-os_monasca/pike/configure-monasca.html | 21:12 |
| cloudnull | guilhermesp ok. | 21:12 |
| *** guilhermesp has quit IRC | 21:13 | |
| *** guilhermesp has joined #openstack-ansible | 21:14 | |
| guilhermesp | I'm gonna continue trying, maybe later, any news or workaround I tell here. | 21:16 |
| openstackgerrit | Merged openstack/ansible-role-systemd_networkd master: Make the networkd role distro agnostic https://review.openstack.org/568367 | 21:19 |
| cloudnull | guilhermesp we'll be around. let us know how it goes. | 21:21 |
| cloudnull | idlemind i have a greenfield deployment running i'll know more about the latest tag in a few | 21:22 |
| *** mmercer has joined #openstack-ansible | 21:23 | |
| mmercer | lo folks | 21:23 |
| mmercer | running on a *clean* ubuntu 16.04 install, attempting to use the aio installer -- TASK [lxc_hosts : Ensure that the LXC cache has been prepared] ***************** Friday 11 May 2018 18:49:01 -0700 (0:00:00.205) 0:05:17.613 ************ fatal: [aio1]: FAILED! => {"ansible_job_id": "284330344628.17991", "attempts": 1, "changed": true, "cmd": "chroot /var/lib/machines/ubuntu-xenial-amd64 /usr/local/bin/cache-prep-comm | 21:24 |
| *** throwsb1 has quit IRC | 21:25 | |
| mmercer | oh.... input trimming... not used to that, heh | 21:25 |
| mmercer | https://paste.fedoraproject.org/paste/RBTUCjP0saUPR0UTdUT1bw | 21:27 |
| cloudnull | mmercer mind having a look at /var/log/lxc-cache-prep-commands.log | 21:30 |
| mmercer | that seems rather impressive | 21:32 |
| mmercer | oops.. | 21:32 |
| mmercer | - /etc/resolv.conf: no such file or directory | 21:32 |
| mmercer | ( will be nice to be back on weechat in the near future ) | 21:32 |
| cloudnull | there was an upstream image change we had to adjust that | 21:32 |
| cloudnull | https://review.openstack.org/#/c/565933/ | 21:32 |
| cloudnull | mmercer if you pull in that patch it should fix the issue for you | 21:33 |
| mmercer | and then rerun the whole stack or how do you resume 'in the middle' so to speak ? | 21:34 |
| cloudnull | you can also workaround it like so https://review.openstack.org/#/c/566304/1/releasenotes/notes/lxc-cache-prep-fail-resolveconf-bbfe1705259a461d.yaml | 21:34 |
| cloudnull | mmercer is this a new deployment ? | 21:34 |
| cloudnull | what release? | 21:34 |
| mmercer | yes -- queens | 21:34 |
| mmercer | and had thought i had done the user_Variables | 21:35 |
| mmercer | user_variables | 21:35 |
| cloudnull | I'd personally pull in the head of stable/queens and rerun. | 21:35 |
| cloudnull | if you kill that task you can then run `openstack-ansible containers-deploy.yml setup-infrastructure.yml setup-openstack.yml` | 21:36 |
| cloudnull | that will pick up where the failure left off. | 21:36 |
| cloudnull | you could run `setup-everything.yml` that failure is pretty early in the build. | 21:36 |
| cloudnull | but you don't have to | 21:36 |
| *** jwitko__ has joined #openstack-ansible | 21:37 | |
| cloudnull | idlemind pike: moment of truth :) | 21:37 |
| mmercer | pulled stable/queens, starting with containers-deploy now | 21:38 |
| mmercer | hmmm, peculiar -- same error, even with running from origin/stable/queens | 21:39 |
| *** jwitko_ has quit IRC | 21:40 | |
| *** jwitko__ has quit IRC | 21:42 | |
| *** cmart has joined #openstack-ansible | 21:45 | |
| *** cmart has quit IRC | 21:47 | |
| *** cjloader has quit IRC | 21:50 | |
| *** cjloader_ has joined #openstack-ansible | 21:50 | |
| mmercer | cloudnull: https://paste.fedoraproject.org/paste/Tonr13EdQ9X89h48nWMNRA | 21:51 |
| *** cjloader has joined #openstack-ansible | 21:55 | |
| *** cjloader_ has quit IRC | 21:55 | |
| *** vnogin has joined #openstack-ansible | 21:58 | |
| *** cjloader has quit IRC | 21:59 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 22:04 |
| cloudnull | mmercer, hum. | 22:05 |
| cloudnull | "http://us.archive.ubuntu.com/ubuntu xenial/main amd64 xz-utils amd64 5.1.1alpha+20120614-2ubuntu2", " Temporary failure resolving 'us.archive.ubuntu.com'"" that's odd | 22:05 |
| cloudnull | mind running `lxc-veth-check` ? | 22:05 |
| cloudnull | if that command returns anything indicating there were broken veth's that's the issue. | 22:06 |
| cloudnull | otherwies something else is going on in terms of not being able to talk to "us.archive.ubuntu.com" | 22:06 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible-ops master: Ansible inventory and keepalive support https://review.openstack.org/568354 | 22:08 |
| mmercer | cloudnull: 2.0.8 | 22:10 |
| mmercer | nothing else | 22:10 |
| *** cmart has joined #openstack-ansible | 22:12 | |
| *** esberglu has quit IRC | 22:12 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Add support for using distribution packages for OpenStack services https://review.openstack.org/560308 | 22:12 |
| *** esberglu has joined #openstack-ansible | 22:13 | |
| cloudnull | mmercer well that's good. | 22:13 |
| cloudnull | means nothing is broken in the network path from the host to the container | 22:14 |
| cloudnull | mmercer from within the repo container can you ping that domain ? | 22:14 |
| mmercer | nope. unable to resolve, but i can ping 8.8.8.8 | 22:16 |
| mmercer | no /etc/resolv.conf (and from the sounds of it, thats normal ) | 22:16 |
| *** threestrands has joined #openstack-ansible | 22:16 | |
| *** threestrands has quit IRC | 22:16 | |
| *** threestrands has joined #openstack-ansible | 22:16 | |
| *** radeks_ has quit IRC | 22:17 | |
| *** esberglu has quit IRC | 22:17 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 22:18 |
| mmercer | cloudnull: so there is a resolv.conf under /var/run/resolveconf/resolv.conf but its not @/etc/resolv.conf -- so right now, its not resolving anything since there is no /etc/resolv.conf in general | 22:19 |
| cloudnull | you should have /etc/resolv.conf and by default it should be pointed at "nameserver 10.0.3.1" | 22:19 |
| cloudnull | which is the lxcbr0 interface / address | 22:20 |
| mmercer | hrum | 22:20 |
| cloudnull | the resolver bug was that the base image didn't have it and we were expecting it to be there. | 22:20 |
| *** cmart has quit IRC | 22:20 | |
| mmercer | nope, nothing at /etc/resolv.conf at all, and the one under /var/run is pointed at 10.255.255.1 | 22:20 |
| cloudnull | what does your host have ? | 22:21 |
| cloudnull | I assume that one is all good/working ? | 22:21 |
| mmercer | yep | 22:21 |
| mmercer | its just repo that does not have a resolv.conf, it has the resolveconf/ dir at /etc/ but no resolv.conf file | 22:22 |
| cloudnull | to fix it thoughout the containers you could run something like `ansible -m copy 'src=/etc/resolv.conf dest=/etc/resolv.conf ' all_containers` | 22:22 |
| cloudnull | which would ship your host resolver into the containers. | 22:22 |
| cloudnull | or you could just write out a new one. | 22:22 |
| mmercer | ill write out a new one on repo right quick | 22:22 |
| cloudnull | ++ | 22:22 |
| cloudnull | do all of your other containers have it ? | 22:23 |
| cloudnull | `ansible -m shell -a 'ls -al /etc/resolv.conf ' all_containers` | 22:23 |
| cloudnull | if not, you may need to write out a general file and ship it out everywhere. | 22:24 |
| mmercer | looks like none of them have it, actually | 22:24 |
| cloudnull | I think the image prep failed in a big way. you might need to recreate that? | 22:25 |
| mmercer | possible, i dont recall seeing any prior issues | 22:25 |
| cloudnull | ok. | 22:25 |
| mmercer | whats the starting point for the image ? | 22:26 |
| cloudnull | containers-deploy.yml will cover it . | 22:26 |
| cloudnull | though you might need to expunge the cache | 22:26 |
| mmercer | is there a playbook for cache expunge as well, or is that manual ? | 22:27 |
| cloudnull | https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/defaults/main.yml#L189-L190 | 22:27 |
| cloudnull | just a var | 22:27 |
| mmercer | ( i have a TON to learn to be able to start contributing, but i do plan on lurking, learning, and wanting to contribute, since I am a constant ansible user ) | 22:27 |
| cloudnull | `openstack-ansible containers-deploy.yml setup-infrastructure.yml setup-openstack.yml -e lxc_image_cache_refresh=yes` should do it | 22:28 |
| cloudnull | if you checked out the latest stable queens, make sure you first rebootstrap the environment else your roles wont be up-to-date | 22:28 |
| cloudnull | ./scripts/bootstrap-ansible.sh | 22:29 |
| mmercer | ahhh | 22:29 |
| cloudnull | then rerun the playbooks. | 22:29 |
| mmercer | i had wondered about that =D | 22:29 |
| cloudnull | yea that one gets a lot of folks. sorry about that. | 22:30 |
| cloudnull | i should've mentioned that before. | 22:30 |
| cloudnull | idlemind so fresh install of pike, using the head of the stable/pike branch, is all go so far. | 22:30 |
| cloudnull | My MNAIO is running the nova playbooks ATM . | 22:31 |
| mmercer | looks like at this point, the bootstrap-ansible is picking up the modified pip.conf index address, as its looking for 172.29.236.100:8181 (and failing) | 22:31 |
| cloudnull | idlemind im on dd2d5f5e4f1ffc81dab654c6b390cb32056825fe | 22:31 |
| cloudnull | yup, that's a thing. `rm ~/.pip/pip.conf` | 22:32 |
| cloudnull | it'll get recreated | 22:32 |
| cloudnull | you normally dont have to do that. | 22:32 |
| cloudnull | but if it failed in an odd place something things are not all totally wired up. | 22:32 |
| mmercer | * nods * | 22:33 |
| mmercer | i dont recall having seen any other mentions of fails prior to the one i joined with | 22:33 |
| mmercer | originally i was trying to get it installing under centos, but since it sounds like its better supported on ubuntu, i said screw it, the host platform doesnt matter to me, ultimately | 22:33 |
| *** dxiri_ has joined #openstack-ansible | 22:34 | |
| cloudnull | yea, centos is getting better but ubuntu / suse are the two most used platforms for us at this time. | 22:37 |
| *** dxiri has quit IRC | 22:37 | |
| * cloudnull imho | 22:37 | |
| *** jwitko_ has joined #openstack-ansible | 22:39 | |
| SamYaple | that suprises me about suse, but is really cool | 22:42 |
| mmercer | hrum..... i reran bootstrap-ansible, ran the deploy-containers with the reset cache.... and it failed with the exact same us.archive.ubuntu issue | 22:47 |
| mmercer | with *no* prior errors | 22:47 |
| mmercer | that really doesnt make sense to me | 22:47 |
| mmercer | if i refresh the image cache, then it should in theory 'fix' the issue... right ? | 22:48 |
| mmercer | there is some greater power that seems to not want me playing with openstack, lol | 22:49 |
| cloudnull | haha | 22:49 |
| cloudnull | SamYaple yup the suse folks (hwoarang) made it all happen | 22:50 |
| cloudnull | mmercer maybe the best course of action will be to ship out a resolve.conf file | 22:51 |
| cloudnull | that cache reset flag should have purged it all though | 22:51 |
| * cloudnull might spin up a queens lab to try it out just in case. | 22:51 | |
| mmercer | no resolve.conf on any of them | 22:52 |
| mmercer | where is resolv.conf copied at ? | 22:52 |
| mmercer | or is it built into the base image and then the base image is what the lxc containers are built from ? | 22:53 |
| mmercer | if i look at the /etc/resolv.conf under /var/lib/machines/ubuntu-xenial-amd64, it has the nameserver 10.255.255.1 that i expected id see everywhere else | 22:54 |
| cloudnull | yes. that's right. | 22:54 |
| cloudnull | you have this pr in the lxc_hosts role https://review.openstack.org/#/c/565933/ ?V | 22:54 |
| cloudnull | you could manually purge the cache and rerun the playbooks or change the lxc backend store to machinectl which would produce COW clones of that base image. | 22:56 |
| cloudnull | oh... | 22:56 |
| cloudnull | after the rebuild did you by change destory the containers ? | 22:56 |
| mmercer | cloudnull: yes, that pr is present ( just checked ) | 22:56 |
| *** jwitko_ has quit IRC | 22:56 | |
| mmercer | cloudnull: nope, i wasnt sure if i needed to | 22:56 |
| mmercer | i had thought about it | 22:56 |
| cloudnull | I think im batting 1000 in telling you what not to do | 22:56 |
| cloudnull | :) | 22:56 |
| cloudnull | you could try it all with one container just to make sure it's fixed | 22:57 |
| mmercer | meh, whats the fun in that =D go big or go home! lol | 22:57 |
| mmercer | this is my 'workstation server' at the office anyway | 22:57 |
| cloudnull | openstack-ansible containers-lxc-destroy.yml containers-deploy.yml --limit "hosts:repo_all[0]" | 22:57 |
| mmercer | and once i have it running, i dont plan on doing anything... destructive (keyword == plan) | 22:57 |
| cloudnull | ah, then run that without the limit | 22:58 |
| cloudnull | it'll destroy / build all the things | 22:58 |
| mmercer | that was the playbook i couldnt remember... of course, one of these days ill go through and read through everything in detail | 22:58 |
| cloudnull | if after the rebuild it still has no resolvers. then I likely broken something along the way. | 22:59 |
| cloudnull | but if your seeing it in the machines cache it should all be good to go | 22:59 |
| mmercer | * nods * | 23:00 |
| mmercer | boom! bwahahaha | 23:00 |
| mmercer | should enable cowsay on that task and have it be an ascii bomb for just that playset, lol | 23:00 |
| mmercer | would be rather amusing | 23:00 |
| cloudnull | hahah | 23:02 |
| *** dave-mccowan has quit IRC | 23:02 | |
| mmercer | woohooo, blacklisted package == working | 23:04 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Add task to regather facts once the role completes https://review.openstack.org/568355 | 23:06 |
| cloudnull | mmercer great success so far ? | 23:09 |
| cloudnull | is this an all in one? | 23:09 |
| cloudnull | if so, you should see all of the container images within `machinectl list-images` | 23:10 |
| cloudnull | you can tinker with those containers directly though that interface, back them up, clone, etc. | 23:10 |
| * cloudnull assuming it's an AIO w/ the machinectl backend in service | 23:11 | |
| mmercer | cloudnull: yep, its at 'install distro packages' currently | 23:12 |
| mmercer | meh, i can look at the images just by cd'ing to /var/lib/machines | 23:12 |
| mmercer | lol | 23:12 |
| mmercer | they are essentially 'raw' images | 23:12 |
| cloudnull | if there are multiple images there, then its using the machinectl backend. | 23:12 |
| mmercer | there are, each one has its own raw directory | 23:13 |
| cloudnull | ++ they should all be btrfs subvolumes . | 23:13 |
| cloudnull | `btrfs sub list /var/lib/machines` | 23:13 |
| mmercer | oooh, interesting! | 23:14 |
| cloudnull | you can do all kinds of fun things :) | 23:14 |
| mmercer | jeeze, why does anyone use the puppet / rdo installer.... ( not that i can't appreciate rdo/packstack/etc ).... but.... this one just seems much better thought out | 23:14 |
| cloudnull | Thanks! They're all good in they're own right. Everything has it's pros and cons. | 23:16 |
| mmercer | i suppose that one is good for someone who wants to practice and work on puppet in general | 23:16 |
| mmercer | * nods * | 23:16 |
| cloudnull | I will say that we have a pretty great community of deployers/operators. :D | 23:17 |
| cloudnull | anyway, I gotta run. time to go take the dog for a walk. | 23:17 |
| cloudnull | have a good one. | 23:17 |
| mmercer | you too | 23:17 |
| mmercer | ill see where i end up when this is finished =D probably resume tomorrow depending on where it lands | 23:17 |
| cloudnull | let us know how things go and if you have questions. cheers | 23:17 |
| cloudnull | ++ | 23:17 |
| mmercer | * nods * I'll put up some blog thoughts on it shortly as well | 23:18 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Add Bionic testing https://review.openstack.org/567042 | 23:20 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_container_create master: Update role to gate using all supported distros https://review.openstack.org/547158 | 23:24 |
| cloudnull | mmercer i look forward to reading it :) | 23:26 |
| cloudnull | cheers ! | 23:26 |
| *** lihi has quit IRC | 23:37 | |
| *** oanson has quit IRC | 23:37 | |
| *** lihi has joined #openstack-ansible | 23:45 | |
| *** oanson has joined #openstack-ansible | 23:48 | |
| *** tosky has quit IRC | 23:52 | |
| *** kstev has quit IRC | 23:57 | |
| *** weezS has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!