| *** Kennan_Vacation is now known as Kennan | 00:03 | |
| *** dims has quit IRC | 00:04 | |
| *** cloudnau_ has joined #kolla | 00:05 | |
| sdake | not sure why fastestmirror was added back into the distro | 00:07 |
|---|---|---|
| sdake | no bueno | 00:07 |
| *** cloudnautique has quit IRC | 00:08 | |
| *** cloudnau_ has quit IRC | 00:10 | |
| *** dims has joined #kolla | 00:11 | |
| SamYaple | sdake: something probably pulls it in as a dep | 00:12 |
| sdake | rebuilding | 00:12 |
| sdake | may have been that pip imploded during build | 00:12 |
| SamYaple | instead of cleaning every time i think centos should just not cache the packages sdake | 00:12 |
| SamYaple | in ubuntu we dont 'apt-get update' everytime | 00:12 |
| SamYaple | with centos if packages change mid-build your build will be mixed packages | 00:13 |
| SamYaple | not the case with ubuntu, i made sure of it | 00:13 |
| sdake | i'm not sure how to do that with yum | 00:13 |
| sdake | apt is a better tool then yum | 00:13 |
| sdake | that is why there is this new fangled fedora thing for package managemenet | 00:14 |
| SamYaple | you can do it with yum | 00:14 |
| sdake | you mean not do a cache clean each time? | 00:14 |
| SamYaple | http://linux.die.net/man/5/yum.conf | 00:14 |
| SamYaple | keepcache 0 | 00:14 |
| SamYaple | you shouldnt need to clean after that | 00:15 |
| SamYaple | i was thinking of doing that for apt as well | 00:15 |
| sdake | yup that would be a big improvement | 00:15 |
| SamYaple | instead of needing apt-clean | 00:15 |
| SamYaple | though yum clean is way more invloved than apt-clean | 00:15 |
| dave-mccowan | sdake INFO:__main__:keystone: yum-plugin-fastestmirror is needed by (installed) yum-3.4.3-132.el7.centos.0.1.noarch | 00:16 |
| sdake | rpm -e --nodeps | 00:16 |
| dave-mccowan | sdake same error on httpd | 00:30 |
| sdake | dave-mccowan not sure what is going on with the centos mirros | 00:30 |
| sdake | but i am reubilding atm | 00:30 |
| sdake | and everything is bust | 00:31 |
| sdake | probably related to us buidling ;) | 00:31 |
| sdake | kolla puts alot of network pressure on the centos mirrors | 00:31 |
| sdake | dave-mccowan d_code this is for example why we can't make the gate voting until we have our own mirrors in openstack infra | 00:32 |
| dave-mccowan | sdake no worries. i'll try again in few hours. | 00:32 |
| *** jtriley has quit IRC | 00:32 | |
| sdake | dave-mccowan i find it odd that it is jus tthat one package | 00:33 |
| sdake | must have recently updated | 00:33 |
| sdake | minus the temporary impairment, kolla is looking pretty good | 00:34 |
| *** salv-orl_ has quit IRC | 00:38 | |
| *** iceyao has joined #kolla | 00:41 | |
| *** iceyao has quit IRC | 00:41 | |
| *** Allen_Gao has joined #kolla | 00:45 | |
| SamYaple | we are close to making the ubuntu gate voting | 00:46 |
| sdake | 1 > 0 | 00:47 |
| *** aginwala has quit IRC | 00:51 | |
| *** iceyao has joined #kolla | 01:00 | |
| *** dims has quit IRC | 01:04 | |
| *** dims has joined #kolla | 01:08 | |
| *** iceyao_ has joined #kolla | 01:10 | |
| *** iceyao has quit IRC | 01:13 | |
| *** achanda has quit IRC | 01:13 | |
| sdake | samyaple would you mind rebasing https://review.openstack.org/#/c/276328/ so we can see if it goes through the gate properly once we get all this mess sorted out with the dev bindmounts? | 01:14 |
| sdake | oh nm thats stable/liberty | 01:15 |
| sdake | SamYaple the stable/liberty gate is currently passing all tests, can you rebase that patch now so we can see if your change passes the gate | 01:16 |
| *** aginwala has joined #kolla | 01:17 | |
| *** achanda has joined #kolla | 01:17 | |
| *** achanda has quit IRC | 01:20 | |
| *** tzn has joined #kolla | 01:26 | |
| *** kejlly_ has quit IRC | 01:26 | |
| sdake | boy we got a whole lot of conflicting changes and a slew of review backlog | 01:39 |
| sdake | i'm off to dinner, enough fun for one day | 01:39 |
| sdake | bbl possibly | 01:39 |
| *** Allen_Gao has quit IRC | 01:41 | |
| openstackgerrit | Merged openstack/kolla: Ensure that /var/lib/zookeeper is owned by user https://review.openstack.org/280445 | 01:41 |
| *** Allen_Gao has joined #kolla | 01:41 | |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix nova tasks https://review.openstack.org/280464 | 01:42 |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix nova group name in nova.conf.j2 https://review.openstack.org/280465 | 01:42 |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix glance-registry enable typo https://review.openstack.org/280466 | 01:42 |
| *** tfukushima has joined #kolla | 01:50 | |
| *** Allen_Gao has quit IRC | 01:53 | |
| *** jasonsb has quit IRC | 01:56 | |
| *** jasonsb has joined #kolla | 01:57 | |
| *** tzn has quit IRC | 02:00 | |
| *** Allen_Gao has joined #kolla | 02:09 | |
| *** daneyon_ has quit IRC | 02:09 | |
| *** iceyao has joined #kolla | 02:15 | |
| *** iceyao_ has quit IRC | 02:17 | |
| *** rhallisey has quit IRC | 02:22 | |
| mandre | dave-mccowan, what's your docker storage backend? the issue you have is typically caused by an old aufs | 02:24 |
| mandre | https://github.com/docker/docker/issues/6980 | 02:24 |
| d_code | sdake: SamYaple: keepcache=0 doesn’t affect pulling latest package data | 02:25 |
| d_code | SamYaple: yum actually enforces consistency…it’s transactional in that you can revert changes that occur with it | 02:25 |
| d_code | dnf is really the next evolution of yum, and is more of a restructure, but the transactional nature of it stays | 02:26 |
| d_code | if you want to install packages solely from whatever is in the local repo cache metadata, you use `yum --cacheonly install foo` | 02:27 |
| d_code | prior to that you can do `yum makecache fast` and it will pull down all the repodata…I do this with Chef because chef will pull down all the metadata everytime, which takes forever | 02:27 |
| dave-mccowan | mandre aufs. yes, that looks on point. | 02:28 |
| d_code | SamYaple: the keepcache=0 effectively creates a local yum repo for all the packages you’ve downloaded. By default they are removed once the package is installed successfully (full transaction is complete) to save disk space | 02:28 |
| dave-mccowan | mandre solution is to use centos instead of ubuntu? | 02:29 |
| mandre | dave-mccowan, no need to go that far, simply use a different storage backend | 02:30 |
| mandre | http://docs.openstack.org/developer/kolla/image-building.html#known-issues | 02:30 |
| d_code | you can also specify “metadata_expire” in yum.conf. It defaults to 90 minutes, in which case it shouldn’t download new package metadata | 02:30 |
| mandre | dave-mccowan, I'm using devicemapper myself from a centos box provisioned with vagrant | 02:31 |
| d_code | sorry…yum is one of the reasons I left apt distributions… as a developer, packager, and automated deployer…I find yum much more predictable … could just be familiarity bias though…except the transactions…those have saved my life | 02:31 |
| d_code | ite…headed to bed… | 02:31 |
| mandre | d_code, interesting we can potentially speed up build time by keeping yum metadata on the base image | 02:36 |
| mandre | d_code, is there a way to tell yum not to keep downloaded packages around? | 02:37 |
| dave-mccowan | mandre thanks! that did it. DOCKER_OPTS="-s devicemapper" | 02:43 |
| *** iceyao_ has joined #kolla | 02:46 | |
| *** iceyao has quit IRC | 02:50 | |
| mandre | dave-mccowan: np, glad you fixed it | 02:53 |
| mandre | what version of ubuntu are you using? was aufs the default backend? | 02:53 |
| *** tzn has joined #kolla | 02:57 | |
| dave-mccowan | mandre 14.04 with 3.16 kernel. aufs was the default. | 02:58 |
| *** salv-orlando has joined #kolla | 03:16 | |
| *** neophy has joined #kolla | 03:17 | |
| *** kexiaodong has joined #kolla | 03:18 | |
| *** aginwala has quit IRC | 03:19 | |
| *** Marga__ has joined #kolla | 03:20 | |
| *** Marga_ has quit IRC | 03:23 | |
| *** Marga__ has quit IRC | 03:25 | |
| *** salv-orlando has quit IRC | 03:28 | |
| *** tzn has quit IRC | 03:30 | |
| *** tfukushima has quit IRC | 03:33 | |
| *** dims has quit IRC | 03:37 | |
| *** aginwala has joined #kolla | 03:38 | |
| sdake | evening folks | 03:40 |
| *** klint has joined #kolla | 03:41 | |
| *** chandankumar has joined #kolla | 03:41 | |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix glance-registry enable typo https://review.openstack.org/280466 | 03:41 |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix nova group name in nova.conf.j2 https://review.openstack.org/280465 | 03:41 |
| *** archjun has joined #kolla | 04:00 | |
| mdnadeem | sdake, good eve | 04:04 |
| mdnadeem | sdake, kolla deploy fail on TASK: [mariadb | Waiting for MariaDB service to be ready] | 04:05 |
| mdnadeem | with error : Container mariadb is restarting, wait until the container is running | 04:05 |
| mdnadeem | docker logs mariadb shows "/usr/bin/mysqld_safe: line 139: /var/lib/mysql//kolla.err: Permission denied" | 04:09 |
| *** tzn has joined #kolla | 04:27 | |
| *** iceyao has joined #kolla | 04:44 | |
| *** dave-mccowan has quit IRC | 04:47 | |
| *** iceyao_ has quit IRC | 04:47 | |
| *** achanda has joined #kolla | 04:48 | |
| *** tzn has quit IRC | 05:00 | |
| *** subscope has joined #kolla | 05:09 | |
| *** jasonsb has quit IRC | 05:09 | |
| *** jasonsb_ has joined #kolla | 05:10 | |
| *** iceyao_ has joined #kolla | 05:12 | |
| *** iceyao has quit IRC | 05:16 | |
| *** iceyao has joined #kolla | 05:18 | |
| *** iceyao_ has quit IRC | 05:21 | |
| *** iceyao has quit IRC | 05:33 | |
| *** alisonh has quit IRC | 05:38 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 05:43 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 05:43 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Remove bootstrap playbook for ZooKeeper https://review.openstack.org/280502 | 05:43 |
| *** kjelly has joined #kolla | 05:45 | |
| *** coolsvap|away is now known as coolsvap | 05:56 | |
| *** iceyao has joined #kolla | 05:56 | |
| *** iceyao_ has joined #kolla | 05:57 | |
| *** tzn has joined #kolla | 05:57 | |
| *** iceyao_ has quit IRC | 05:58 | |
| *** iceyao__ has joined #kolla | 05:59 | |
| *** tfukushima has joined #kolla | 05:59 | |
| *** iceyao has quit IRC | 06:01 | |
| *** iceyao__ has quit IRC | 06:02 | |
| *** iceyao has joined #kolla | 06:02 | |
| *** alisonh has joined #kolla | 06:15 | |
| *** neophy has quit IRC | 06:15 | |
| *** tzn has quit IRC | 06:30 | |
| openstackgerrit | MD NADEEM proposed openstack/kolla: Aodh ubuntu binary container https://review.openstack.org/280515 | 06:32 |
| *** cemason has joined #kolla | 06:32 | |
| *** subscope has quit IRC | 06:34 | |
| *** aginwala has quit IRC | 06:38 | |
| *** achanda has quit IRC | 06:39 | |
| *** salv-orlando has joined #kolla | 06:43 | |
| *** tzn has joined #kolla | 06:45 | |
| *** tzn has quit IRC | 06:45 | |
| *** alisonh has quit IRC | 06:45 | |
| *** tzn has joined #kolla | 06:46 | |
| *** salv-orlando has quit IRC | 06:46 | |
| *** alisonh has joined #kolla | 06:49 | |
| *** subscope has joined #kolla | 06:52 | |
| *** tzn has quit IRC | 06:55 | |
| *** subscope has quit IRC | 06:55 | |
| *** coolsvap is now known as coolsvap|away | 06:58 | |
| *** achanda has joined #kolla | 06:59 | |
| *** unicell has joined #kolla | 07:00 | |
| *** Marga_ has joined #kolla | 07:00 | |
| *** Marga__ has joined #kolla | 07:02 | |
| *** Marga_ has quit IRC | 07:02 | |
| *** Serlex has joined #kolla | 07:04 | |
| *** Liuqing has joined #kolla | 07:05 | |
| *** Marga__ has quit IRC | 07:06 | |
| *** Marga_ has joined #kolla | 07:06 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 07:11 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 07:11 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Look for private interface lease https://review.openstack.org/280524 | 07:11 |
| nihilifer | asalkeld: if you are available, could you please look at these 2 commits? https://review.openstack.org/#/c/280502/ https://review.openstack.org/#/c/280524/ | 07:13 |
| nihilifer | the first one fixed mesos deployment at all (for docker 1.10) | 07:13 |
| nihilifer | the second one fixes vagrant under ubuntu | 07:13 |
| *** iceyao has quit IRC | 07:19 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 07:20 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 07:20 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Look for private interface lease https://review.openstack.org/280524 | 07:20 |
| *** iceyao has joined #kolla | 07:20 | |
| asalkeld | done nihilifer | 07:27 |
| nihilifer | thx | 07:27 |
| *** guillaume_ has quit IRC | 07:41 | |
| *** ravla has quit IRC | 07:45 | |
| *** ravla has joined #kolla | 07:46 | |
| *** tzn has joined #kolla | 07:50 | |
| *** fgimenez has joined #kolla | 07:59 | |
| *** chandankumar has quit IRC | 08:00 | |
| *** tfukushima has quit IRC | 08:03 | |
| *** coolsvap|away is now known as coolsvap | 08:09 | |
| *** unicell has quit IRC | 08:11 | |
| *** unicell has joined #kolla | 08:11 | |
| *** chandankumar has joined #kolla | 08:15 | |
| *** shardy has joined #kolla | 08:16 | |
| *** archjun has quit IRC | 08:17 | |
| *** tfukushima has joined #kolla | 08:17 | |
| *** mikelk has joined #kolla | 08:18 | |
| *** tzn has quit IRC | 08:19 | |
| *** tzn has joined #kolla | 08:20 | |
| *** subscope has joined #kolla | 08:21 | |
| *** chandankumar has quit IRC | 08:21 | |
| *** chandankumar has joined #kolla | 08:25 | |
| *** coolsvap is now known as coolsvap|away | 08:26 | |
| *** mikelk has quit IRC | 08:28 | |
| *** salv-orlando has joined #kolla | 08:30 | |
| *** salv-orlando has quit IRC | 08:32 | |
| *** tzn has quit IRC | 08:34 | |
| *** tzn has joined #kolla | 08:34 | |
| *** tzn has quit IRC | 08:34 | |
| *** archjun has joined #kolla | 08:39 | |
| *** mikelk has joined #kolla | 08:39 | |
| sdake | mdnadeem thanks for the debug on that | 08:41 |
| sdake | mdnadeem can you please file a bug | 08:41 |
| *** opennode has joined #kolla | 08:42 | |
| ajafo | o/ | 08:43 |
| mdnadeem | sdake, I am trying to do it freshly by deleting all container and its volumes, if it again produce then i will lodge a bug | 08:44 |
| *** kproskurin has joined #kolla | 08:45 | |
| *** opennode has quit IRC | 08:45 | |
| *** athomas has joined #kolla | 08:45 | |
| *** subscope has quit IRC | 08:50 | |
| *** iceyao_ has joined #kolla | 08:51 | |
| mdnadeem | ajafo, hi | 08:51 |
| mdnadeem | ajafo, Your yesterday issue get resolved? | 08:52 |
| *** iceyao has quit IRC | 08:52 | |
| ajafo | mdnadeem: hi, yes | 08:53 |
| mdnadeem | ajafo, cool, have you ever encounter this kind of issue : http://pastebin.com/cY6fVjrQ | 08:54 |
| *** mbound has joined #kolla | 09:02 | |
| *** achanda has quit IRC | 09:03 | |
| *** subscope has joined #kolla | 09:04 | |
| openstackgerrit | Allen Gao proposed openstack/kolla: Keep consistent with others for ironic containers name https://review.openstack.org/280557 | 09:04 |
| *** achanda has joined #kolla | 09:06 | |
| mdnadeem | ajafo, sdake , hi the above issue resolve , i just try it freshly by removing containers and their volumes :) | 09:08 |
| *** salv-orlando has joined #kolla | 09:08 | |
| *** pbourke has joined #kolla | 09:13 | |
| openstackgerrit | Merged openstack/kolla: A few silly grammar fixes. https://review.openstack.org/280363 | 09:15 |
| *** pbourke has quit IRC | 09:18 | |
| *** macsz has joined #kolla | 09:19 | |
| *** achanda has quit IRC | 09:28 | |
| *** akwasnie has joined #kolla | 09:29 | |
| *** archjun has quit IRC | 09:29 | |
| ajafo | mdnadeem: for me it looks like you've two problems mysql can't access pid file and and can't get port | 09:30 |
| ajafo | my some old container is working or sthg like this? | 09:31 |
| *** akscram has quit IRC | 09:33 | |
| mdnadeem | ajafo, Now the problem get resolved by applying work aorund >> removed mariadb container and its volume and again run sudo kolla-ansible deploy | 09:33 |
| *** akscram has joined #kolla | 09:33 | |
| *** subscope has quit IRC | 09:40 | |
| *** Liuqing has quit IRC | 09:41 | |
| *** pbourke has joined #kolla | 09:47 | |
| *** subscope has joined #kolla | 09:47 | |
| *** tzn has joined #kolla | 09:49 | |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Add priviliged flag to nova-api https://review.openstack.org/280576 | 09:50 |
| *** tzn has quit IRC | 09:51 | |
| *** tzn has joined #kolla | 09:51 | |
| *** pbourke has quit IRC | 09:54 | |
| *** pbourke has joined #kolla | 09:54 | |
| *** tzn has quit IRC | 09:59 | |
| *** tzn has joined #kolla | 10:01 | |
| *** mbound has quit IRC | 10:01 | |
| *** salv-orl_ has joined #kolla | 10:02 | |
| openstackgerrit | Merged openstack/kolla-mesos: Remove bootstrap playbook for ZooKeeper https://review.openstack.org/280502 | 10:03 |
| openstackgerrit | Merged openstack/kolla-mesos: Look for private interface lease https://review.openstack.org/280524 | 10:03 |
| *** salv-orlando has quit IRC | 10:05 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 10:05 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 10:05 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Fix Chronos tasks cleanup https://review.openstack.org/280585 | 10:05 |
| *** mbound has joined #kolla | 10:06 | |
| *** Liuqing has joined #kolla | 10:06 | |
| *** tfukushima has quit IRC | 10:11 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 10:26 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Fix Chronos tasks cleanup https://review.openstack.org/280585 | 10:26 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 10:26 |
| *** alyson_ has joined #kolla | 10:27 | |
| openstackgerrit | Marek Zawadzki proposed openstack/kolla: Added note about root permissions, removed sudos. https://review.openstack.org/278532 | 10:36 |
| *** subscope has quit IRC | 10:47 | |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Fix glance registry conf path https://review.openstack.org/280600 | 10:49 |
| *** dims has joined #kolla | 10:51 | |
| kproskurin | Guys, neutron has “shared” volumes in agents: https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/tasks/start.yml#L184 | 10:53 |
| kproskurin | I cant find docker dockumentation about this “:shared” option - https://docs.docker.com/engine/userguide/containers/dockervolumes/ | 10:53 |
| kproskurin | From there does it came from? | 10:54 |
| nihilifer | from 1.10 probably | 11:00 |
| kproskurin | Well, it doesnt work for me in 1.10 ;-\ and I cant find any docs about it. | 11:01 |
| nihilifer | how it doesn't work? | 11:02 |
| nihilifer | or another question - you're doing this in mesos? | 11:02 |
| kproskurin | Yes in mesos, it doesnt start containers with this :shared thing with “error 125”. I cant event find anything about this shared thingy here: https://github.com/docker/docker/releases/tag/v1.10.0 | 11:03 |
| kproskurin | Ubuntu btw, this could be the case | 11:03 |
| nihilifer | kproskurin: does mesos-slave container have docker cli in 1.10 version? | 11:03 |
| kproskurin | yes | 11:03 |
| nihilifer | you sure? when it was built? | 11:03 |
| *** dave-mccowan has joined #kolla | 11:04 | |
| kproskurin | http://fpaste.org/323411/45562071/ | 11:05 |
| nihilifer | in /usr/lib/systemd/system/docker.service | 11:06 |
| nihilifer | you shoud have | 11:06 |
| nihilifer | MountFlags=shared | 11:06 |
| nihilifer | you have it? | 11:06 |
| kproskurin | ubuntu 14.04 dosnt use systemd yet… iirc | 11:06 |
| asalkeld | also make sure the docker in the mesos-slave is 1.10 too | 11:06 |
| kproskurin | oh wait. on mesos slave right? | 11:07 |
| nihilifer | yep | 11:07 |
| nihilifer | umm... nope :P | 11:07 |
| nihilifer | sorry | 11:07 |
| kproskurin | %-) | 11:07 |
| nihilifer | i mean, shared in settings should be only on host | 11:07 |
| nihilifer | so, i don't know how to set the MountFlags in upstart | 11:07 |
| kproskurin | Looks like this is the case, ill dig how to set it up in ubuntu | 11:08 |
| nihilifer | but that seems to be thing which should be done | 11:08 |
| asalkeld | kproskurin:https://github.com/openstack/kolla/blob/master/dev/vagrant/ubuntu-bootstrap.sh#L25-L35 | 11:12 |
| *** dims has quit IRC | 11:12 | |
| asalkeld | that's the ubuntu setup | 11:12 |
| kproskurin | it’s a bit strange one. Ubuntu 14.04 by default doesnt use systemd it even dosnt have systemd cli installed | 11:13 |
| kproskurin | bash: systemctl: command not found | 11:14 |
| kproskurin | Imo it’s a bad copy paste :-) | 11:15 |
| kproskurin | from centos | 11:15 |
| *** mbound has quit IRC | 11:15 | |
| *** tzn has quit IRC | 11:16 | |
| openstackgerrit | Allen Gao proposed openstack/kolla: Install python-dogpile-cache for python-ironicclient https://review.openstack.org/280615 | 11:18 |
| *** subscope has joined #kolla | 11:18 | |
| openstackgerrit | Merged openstack/kolla-mesos: Fix Chronos tasks cleanup https://review.openstack.org/280585 | 11:19 |
| openstackgerrit | Angus Salkeld proposed openstack/kolla-mesos: Fix nova tasks https://review.openstack.org/280464 | 11:19 |
| *** mbound has joined #kolla | 11:20 | |
| *** dims has joined #kolla | 11:24 | |
| *** achanda has joined #kolla | 11:28 | |
| *** tzn has joined #kolla | 11:30 | |
| *** achanda has quit IRC | 11:34 | |
| ajafo | kproskurin: systemd is from 15.04 in ubuntu | 11:38 |
| *** dims has quit IRC | 11:38 | |
| kproskurin | I know | 11:38 |
| *** akwasnie has quit IRC | 11:39 | |
| *** ccesario has quit IRC | 11:44 | |
| *** fgimenez has quit IRC | 11:48 | |
| *** fgimenez has joined #kolla | 11:50 | |
| *** akwasnie has joined #kolla | 12:01 | |
| *** sdake has quit IRC | 12:06 | |
| *** sdake has joined #kolla | 12:06 | |
| kproskurin | Gound the solution, not sure if it’s ok: mount --make-shared /run | 12:09 |
| *** dwalsh has joined #kolla | 12:09 | |
| kproskurin | potentialy security risks, imo | 12:09 |
| sdake | hey folks | 12:10 |
| sdake | anyone know how horizon produces the console in openstack | 12:10 |
| sdake | is it via vnc? | 12:10 |
| ajafo | hi sdake | 12:11 |
| sdake | hey ajao | 12:12 |
| ajafo | I suppose vnc or spice but I'm not sure | 12:12 |
| *** dwalsh has quit IRC | 12:14 | |
| *** dwalsh has joined #kolla | 12:14 | |
| *** Jeffrey4l has quit IRC | 12:18 | |
| *** dwalsh has quit IRC | 12:19 | |
| Serlex | "To provide a remote console or remote desktop access to guest virtual machines, use VNC or SPICE HTML5..." | 12:28 |
| Serlex | thats from the docs sdake | 12:28 |
| sdake | thanks serlax | 12:29 |
| Serlex | Actually that was misleading | 12:30 |
| Serlex | "OpenStack Compute supports VNC consoles to guests. The VNC protocol is fairly limited" - " SPICE is a new protocol that aims to address the limitations in VNC and provide good remote desktop support" | 12:30 |
| *** Jeffrey4l has joined #kolla | 12:31 | |
| *** ccesario has joined #kolla | 12:31 | |
| *** achanda has joined #kolla | 12:31 | |
| sdake | yup i know what spice is in detail | 12:33 |
| sdake | worked with it in the past | 12:33 |
| sdake | and vnc as well | 12:33 |
| sdake | i just wasn't sure if horizon used it for the console | 12:33 |
| sdake | i guess it must ;) | 12:33 |
| *** ccesario has quit IRC | 12:36 | |
| *** achanda has quit IRC | 12:37 | |
| openstackgerrit | Allen Gao proposed openstack/kolla: WIP: Move ironic-discoverd to ironic-inspector https://review.openstack.org/280655 | 12:39 |
| *** dwalsh has joined #kolla | 12:43 | |
| *** rhallisey has joined #kolla | 12:43 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: install the openstack-designate-api in the base stage https://review.openstack.org/280658 | 12:44 |
| openstackgerrit | Allen Gao proposed openstack/kolla: Keep consistent with others for ironic containers name https://review.openstack.org/280557 | 12:55 |
| *** chandankumar has quit IRC | 12:58 | |
| *** iceyao_ has quit IRC | 13:04 | |
| *** chandankumar has joined #kolla | 13:09 | |
| sdake | Jeffrey4l quick question | 13:12 |
| Jeffrey4l | sdake, pong | 13:13 |
| Jeffrey4l | sup | 13:13 |
| sdake | Jeffrey4l we a a bunch of at risk items in mitaka-3 | 13:13 |
| sdake | things that need to be i n the release that may need R&D attention | 13:13 |
| limamauricio | sdake, init-runonce is updated? | 13:14 |
| sdake | limamauricio no, but we have a solution for the problem of the /dev bindmount ordering problem | 13:14 |
| sdake | limamauricio if you care, its not a kolla problem, its a docker-py problem (docker-py doesn't maintain ordering) | 13:14 |
| Jeffrey4l | yea. I know the /dev bindmount issue. | 13:14 |
| sdake | if you don't care, I can point you at the WIP patch | 13:14 |
| *** chandankumar has quit IRC | 13:15 | |
| Jeffrey4l | But i have no idea how to fix this. | 13:15 |
| sdake | Jeffrey4l its fixed | 13:15 |
| Jeffrey4l | docker can not support this case. | 13:15 |
| sdake | sam and i spent all day debugging it | 13:15 |
| sdake | it can indeed support i t | 13:15 |
| sdake | docker can do it on command line | 13:15 |
| Jeffrey4l | cool | 13:15 |
| Jeffrey4l | what's it? | 13:15 |
| sdake | the problem is docker-py reorders | 13:15 |
| sdake | it converts the volume list about 10 different times to lists and dicts etc | 13:16 |
| sdake | all that conversion changes ordering | 13:16 |
| limamauricio | tools/init-runonce: line 49: neutron: command not found | 13:16 |
| limamauricio | :/ | 13:16 |
| sdake | limamauricio cool you got through a deploy | 13:16 |
| sdake | sudo pip install python-neutronclient | 13:16 |
| Jeffrey4l | limamauricio, you need install python-neutronclient | 13:16 |
| limamauricio | :) | 13:17 |
| sdake | Jeffrey4l can you work on the at risk items? | 13:17 |
| limamauricio | ok | 13:17 |
| Jeffrey4l | sdake, fix the docker-py? push a PS? | 13:17 |
| Jeffrey4l | Or? | 13:17 |
| sdake | Jeffrey4l hang tight | 13:17 |
| sdake | https://etherpad.openstack.org/p/kolla-mitaka-midcycle-roadmap | 13:18 |
| sdake | Jeffrey4l if you want to work on something big impact, reconfigure playbooks would rock | 13:18 |
| sdake | right now we have actions for deploy | 13:18 |
| sdake | we need one for reconfigure | 13:18 |
| sdake | nobody is working on thi problem atm | 13:19 |
| sdake | we have 3 weeks to sort this out | 13:19 |
| limamauricio | sdake, can init-runonce install neutronclient? | 13:19 |
| Jeffrey4l | (saw the midcycle discuss whole afternoon) | 13:19 |
| sdake | limamauricio I prefer nt | 13:19 |
| limamauricio | hmm | 13:19 |
| limamauricio | ok | 13:19 |
| sdake | limamauricio we are working on a tool for centos atleast that configures a host for the first time | 13:19 |
| sdake | d_code has a centos script | 13:20 |
| sdake | maybe it can be put in there, not sure | 13:20 |
| sdake | that scriptis meant for the deploy targets, not the client nodes | 13:20 |
| sdake | how I use my openstack environment is I install my clients on my macintosh laptop | 13:20 |
| Jeffrey4l | sdake, what's mean for "reconfigure services on a redeploy" any details? | 13:20 |
| sdake | and keep a copy of init-runonce on my mac laptop | 13:20 |
| sdake | and deploy to baremetal | 13:20 |
| limamauricio | ok | 13:20 |
| sdake | Jeffrey4l you know how we have the deploy action ? | 13:21 |
| Jeffrey4l | yes | 13:21 |
| sdake | we want a reconfigure action | 13:21 |
| sdake | and we want kolla-ansible to be able to trigger it | 13:21 |
| sdake | kolla-ansible reconfigure | 13:21 |
| Jeffrey4l | change the configure, then restart the service? | 13:21 |
| Jeffrey4l | Got it. | 13:21 |
| sdake | roger | 13:21 |
| sdake | not update the containers | 13:21 |
| sdake | not pull the containers | 13:22 |
| sdake | i'd go with reconfig personally | 13:22 |
| Jeffrey4l | np. I can handle this. | 13:22 |
| sdake | eta? :) | 13:22 |
| sdake | i think we need a blueprint, let me check on that | 13:22 |
| Jeffrey4l | sdake, one week is enough? I think I can finish it befor the end of this week. | 13:23 |
| sdake | sounds fantastic | 13:23 |
| Jeffrey4l | sdake, let us back to the /dev bind mount issue? | 13:23 |
| sdake | gotta work correclty with copy once and copy always | 13:23 |
| sdake | Jeffrey4l that is solved | 13:23 |
| Jeffrey4l | sdake, could u explain more for me? what's the docker cli parameter can solve this/ | 13:24 |
| sdake | Jeffrey4l if you want to help out there, test SamYaple's patch in the review queue | 13:24 |
| * Jeffrey4l is finding | 13:24 | |
| sdake | working on finding the reconfig blueprint if ther eis one | 13:25 |
| sdake | moment | 13:25 |
| Jeffrey4l | sdake, this one? https://review.openstack.org/#/c/280429/ | 13:28 |
| sdake | Jeffrey4l yup thats it | 13:28 |
| rhallisey | sdake, does this mean copy_once functionality is being left as is? | 13:28 |
| limamauricio | sdake, my deploy kolla + manila + ceph ran perfectly | 13:28 |
| limamauricio | :) | 13:28 |
| limamauricio | AIO :) | 13:28 |
| sdake | rhallisey I assume Jeffrey4l is going to fix that ;) | 13:28 |
| rhallisey | and the reconfigure action would replace what we talked would be confi_once | 13:28 |
| sdake | rhallisey these are actions in the playbooks | 13:29 |
| sdake | the strategies don't change | 13:29 |
| sdake | we still have copy_once and copy_always | 13:29 |
| sdake | the problem with copy_once is as implemented, it doesn't match the spec, or atleast what I had in mind ;) | 13:29 |
| rhallisey | so I suppose this goes hand in hand with copy_once then | 13:29 |
| sdake | yes | 13:29 |
| Jeffrey4l | copy_once should work as: do not copy if the the configuration file don't change, right? | 13:29 |
| sdake | rhallisey yes | 13:30 |
| sdake | Jeffrey4l let me think for a moment | 13:30 |
| rhallisey | Jeffrey4l, I don't think ansible can detect that | 13:30 |
| rhallisey | oh wait | 13:30 |
| sdake | rhallisey it could | 13:30 |
| rhallisey | we came up with a way | 13:30 |
| Jeffrey4l | it could | 13:30 |
| sdake | Jeffrey4l I think that would work, but at the moment the containers have a broad problem | 13:31 |
| Jeffrey4l | but if so, the result of copy_once and copy_always will be the same? | 13:31 |
| sdake | the result of copy_once and copy_always from the deployment host is supposed to be the same! | 13:31 |
| Jeffrey4l | Oh sorry, i am wrong. | 13:31 |
| sdake | the result of copy_once and copy_always from a deployment target is supposed to be different | 13:31 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 13:31 |
| *** salv-orl_ has quit IRC | 13:32 | |
| Jeffrey4l | sdake, yea, u are right. | 13:32 |
| sdake | well I filed a bug on copy_once | 13:33 |
| sdake | now cana't find it | 13:33 |
| Jeffrey4l | at the deployment stage, the copy_once and copy_always works the same like now. at the reconfiguration stage, the copy_once works like copy_always. | 13:33 |
| sdake | checking email moment | 13:33 |
| Jeffrey4l | rhallisey, you asked a good question. thanks. :D | 13:35 |
| rhallisey | copy_once = pick up config if there's a change copy_always = pick up config no matter what | 13:35 |
| rhallisey | Jeffrey4l, sure no problem. We talked a bit about this at the midcycle so I figured I would bring up the same points | 13:36 |
| *** salv-orlando has joined #kolla | 13:36 | |
| Jeffrey4l | rhallisey, I do not think so. copy_once should mean pick up the only once. | 13:36 |
| rhallisey | Jeffrey4l, yes that's the way I see it too | 13:36 |
| Jeffrey4l | so which one is expected? | 13:37 |
| rhallisey | ^^^ that is the way it's intended | 13:37 |
| rhallisey | sdake, can you clarify | 13:37 |
| rhallisey | right now, copy_once literally is copy only one time | 13:37 |
| SamYaple | yo | 13:38 |
| Jeffrey4l | morning SamYaple | 13:38 |
| SamYaple | morning Jeffrey4l | 13:38 |
| sdake | Jeffrey4l copy_once is a misnomer | 13:38 |
| rhallisey | but pretty sure when it came up first it was intended to be copy when config chages | 13:38 |
| rhallisey | SamYaple, hey | 13:38 |
| sdake | it should probably b e called copy_immutable | 13:38 |
| SamYaple | rhallisey: no it was intended to be immutability | 13:38 |
| SamYaple | but yea | 13:38 |
| sdake | rhallisey is correct | 13:38 |
| SamYaple | i dont think so | 13:39 |
| SamYaple | look at the ansible spec | 13:39 |
| sdake | did someone closee the bug i filed on this point? | 13:39 |
| sdake | I can't find it | 13:39 |
| rhallisey | where's the original text! XD | 13:39 |
| Jeffrey4l | if rhallisey is right, the result of the copy_once and copy_always will be the same. | 13:39 |
| rhallisey | Jeffrey4l, yup | 13:39 |
| sdake | i put the original text in there, and basically said it was not implemented according to spec ;) | 13:39 |
| SamYaple | sdake: well youre wrong then. spec says it should work like it does | 13:40 |
| Jeffrey4l | and meaningless to have two configure options. | 13:40 |
| sdake | well I filed bug to clear it up | 13:40 |
| SamYaple | https://launchpad.net/bugs/1544287 | 13:40 |
| openstack | Launchpad bug 1544287 in kolla-mesos "unable to launch containers, Unexpected Env format for 'ContainerConfig.Env'" [Critical,Confirmed] | 13:40 |
| sdake | I guess I file another | 13:40 |
| SamYaple | ignore that bug | 13:40 |
| SamYaple | rhallisey: https://github.com/openstack/kolla/blob/master/specs/ansible-multi.rst | 13:40 |
| sdake | but i think someone closed it and i can't find it | 13:40 |
| SamYaple | "The major difference between these two models is that one offers immutability and single source of truth (CONFIG_OUTSIDE_COPY_ONCE)" | 13:40 |
| sdake | right | 13:40 |
| SamYaple | COPY_ONCE has always acted like the spec meant it to | 13:40 |
| sdake | you are leaving out the part where it says "reconfigurable" | 13:41 |
| rhallisey | so I'm not arguing one way or another here just trying to understand the sides | 13:41 |
| SamYaple | sdake: thats because it doesnt say that | 13:41 |
| rhallisey | the issue with copy_once is if you input the wrong config and your container restarts. Now you can't get another conifg to replace it XD | 13:41 |
| SamYaple | rhallisey: yea you have to destroy container | 13:42 |
| SamYaple | this is how ive always expected it to work based on that spec | 13:42 |
| SamYaple | its why i use COPY_ALWAYAS | 13:42 |
| rhallisey | which I'm fine with | 13:42 |
| SamYaple | COPY_ONCE is awful | 13:42 |
| sdake | The CONFIG_OUTSIDE_COPY_ONCE model of configuration maintains the immutable and declarative nature of the Kolla containers, as defined by our current Kolla best practices while introducing completely customizable configuration. | 13:42 |
| Jeffrey4l | does reconfigure action could save u rhallisey | 13:42 |
| sdake | yes clearly there was a misunderstanding in how copy_once was implemented | 13:42 |
| sbezverk | Hello, are you aware of a traceback generated during kolla-build, http://paste.openstack.org/show/487106 | 13:42 |
| rhallisey | Jeffrey4l, oh I use copy_always | 13:42 |
| rhallisey | Jeffrey4l, :) | 13:42 |
| SamYaple | sdake: i think your memory is being selective again. i can assure you this is what you wanted (i have logs) | 13:43 |
| Jeffrey4l | I will COPY_ALWAYS too. ;D | 13:43 |
| rhallisey | Jeffrey4l, reconfig would work, but I think it would break the intention of copy_once | 13:43 |
| SamYaple | rhallisey: no because it would recreate teh container | 13:43 |
| SamYaple | so it would be a new container | 13:43 |
| sdake | ok guys lets focus on the future here | 13:43 |
| SamYaple | so immutability works | 13:43 |
| sdake | who gives a shit what happened i n the past | 13:43 |
| SamYaple | im pointing out this is how everyone expected it to work | 13:44 |
| rhallisey | see I was under the impression it would reload configs and restart | 13:44 |
| SamYaple | i said it was a bad idea but meh | 13:44 |
| rhallisey | but true, it could replace | 13:44 |
| SamYaple | reload on restart would be COPY_ALWAYS though, no? | 13:44 |
| sdake | what *I* want with copy_once is for on a config change on the master node, the configs to be reloaded | 13:44 |
| sdake | but if a config change on the non-master nodes happen, I dont want a configs tobe reloaded | 13:45 |
| SamYaple | you mean the containers recreated? | 13:45 |
| sdake | right recreated | 13:45 |
| sdake | call that what you want | 13:45 |
| sdake | copy_ocne | 13:45 |
| sdake | copy_immutable seems better | 13:45 |
| *** fgimenez has quit IRC | 13:45 | |
| SamYaple | well thats fine, we just need playbooks for that | 13:45 |
| SamYaple | it was a point at the kolla metting | 13:45 |
| *** dims has joined #kolla | 13:45 | |
| sdake | that is single source of truth with immutability | 13:46 |
| SamYaple | so at this point in time, COPY_ALWAYS would need a restart/nohup, COPY_ONCE would require a recreate | 13:46 |
| SamYaple | playbooks can handle that | 13:46 |
| sdake | that seems good | 13:46 |
| rhallisey | perfect | 13:46 |
| sdake | I did file a bug here defining the intentions of copy_once once I found out its a pile of shit as is :) | 13:46 |
| sdake | but it has disappared | 13:46 |
| SamYaple | yup this is what I thought we agreed to at the midcycle (and original ansible spec), sorry if it wasny made clear | 13:46 |
| sdake | oh | 13:47 |
| sdake | bingo | 13:47 |
| sdake | playbooks don't recreate atm | 13:47 |
| *** fgimenez has joined #kolla | 13:47 | |
| SamYaple | thats teh one | 13:48 |
| SamYaple | they only recreate if image/container meta changes | 13:48 |
| SamYaple | not openstack configs | 13:48 |
| SamYaple | to reload configs without COPY_ALWAYS requires nuking container and running playbooks again as it always has | 13:48 |
| sdake | so this can all be rolled into the reconfig action | 13:48 |
| SamYaple | yup | 13:48 |
| sdake | cool | 13:49 |
| Jeffrey4l | clearly. | 13:49 |
| rhallisey | :D | 13:49 |
| sdake | Jeffrey4l did you catch all that? | 13:50 |
| sdake | Jeffrey4l filing blueprint now, so you have something to work with | 13:51 |
| *** dims has quit IRC | 13:51 | |
| d_code | sdake: my script installs neutron client | 13:51 |
| *** subscope has quit IRC | 13:53 | |
| Jeffrey4l | sdake, yea. | 13:54 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make a final check to validate mysql works https://review.openstack.org/276851 | 13:54 |
| openstackgerrit | Merged openstack/kolla: Install python-dogpile-cache for python-ironicclient https://review.openstack.org/280615 | 13:57 |
| Jeffrey4l | SamYaple, sdake it the /dev bindmount ordering problem is this https://bugs.launchpad.net/kolla/+bug/1544156 ? | 13:57 |
| openstack | Launchpad bug 1544156 in kolla "Kolla does not survive reboots" [Undecided,New] | 13:57 |
| sdake | unforutnately nobody has filed a bug yet | 13:58 |
| sdake | but the patch appears to work for several people | 13:58 |
| sdake | limamauricio in particular says it works | 13:58 |
| sdake | i am stuck on NAT + virtualbox not giving me a free VIp to work with | 13:59 |
| SamYaple | Jeffrey4l: no _however_ my bind mount fixes that :D | 13:59 |
| sdake | but will get to that this morning after the school rush finishes | 13:59 |
| SamYaple | or at least greatly reduces that problem | 13:59 |
| SamYaple | Jeffrey4l: https://review.openstack.org/280615 | 13:59 |
| limamauricio | sdake: i'm testing my deploy | 13:59 |
| openstackgerrit | Merged openstack/kolla: Added note about root permissions, removed sudos. https://review.openstack.org/278532 | 13:59 |
| Jeffrey4l | sdake, try use virtio-net adapter | 14:00 |
| openstackgerrit | Allen Gao proposed openstack/kolla: WIP: Move ironic-discoverd to ironic-inspector https://review.openstack.org/280655 | 14:01 |
| sdake | Jeffrey4l https://blueprints.launchpad.net/kolla/+spec/kolla-reconfig | 14:01 |
| Jeffrey4l | sdake, roger | 14:01 |
| sdake | Jeffrey4l I am on a mac | 14:01 |
| sdake | all my linux gear is in the garage atm | 14:02 |
| *** klint has quit IRC | 14:02 | |
| sdake | and my remodel is stalled on upstream ;) | 14:02 |
| sdake | circular dependency there | 14:02 |
| SamYaple | Jeffrey4l: lets get an initial patch worked out for something like keystone for the reconfigure | 14:02 |
| SamYaple | make sure everyone agrees on implementation | 14:02 |
| SamYaple | once base is done it will be simple to throw up the rest | 14:03 |
| sdake | Jeffrey4l ya start with one | 14:03 |
| Jeffrey4l | SamYaple, np | 14:03 |
| Jeffrey4l | SamYaple, really do not think You PS can reduce this. Anyway, i will make some test later. | 14:04 |
| sdake | ok folks I gotta jet for an hour to take my kids to school | 14:06 |
| sdake | posibly 1.5 hours | 14:06 |
| sdake | school madness ftw! | 14:06 |
| *** salv-orlando has quit IRC | 14:08 | |
| *** ccesario has joined #kolla | 14:08 | |
| SamYaple | for all of those with the kolla_toolbox issue, please try this patch and comment on it. 09 | 14:13 |
| SamYaple | https://review.openstack.org/#/c/280429/ | 14:13 |
| SamYaple | its marked WIP because I can't validate it fixes the issue since i dont have that issue, but it works on ubuntu source | 14:14 |
| SamYaple | its a much cleaner implem,entation too | 14:14 |
| SamYaple | rhallisey: Jeffrey4l mandre pbourke ^^ | 14:14 |
| SamYaple | pbourke: a reboot did _not_ fix this toolbox issue for most people like it did for me that night | 14:14 |
| Jeffrey4l | roger | 14:16 |
| sbezverk | SamYaple, after your diff kolla_toolbox needs to be rebuilt? | 14:18 |
| SamYaple | sbezverk: correct | 14:19 |
| SamYaple | techinically the playbooks run without a rebuild, but you wont have logging | 14:19 |
| SamYaple | openstack will work though | 14:19 |
| *** subscope has joined #kolla | 14:22 | |
| sbezverk | SamYaple I also see a traceback almost everytime when run kolla-build, are you aware of this issue? | 14:23 |
| SamYaple | what traceback? | 14:23 |
| SamYaple | (pastebin please) | 14:24 |
| sbezverk | SamYaple this one: http://paste.openstack.org/show/487106 | 14:24 |
| SamYaple | sbezverk: i dont think anyone has seen that before | 14:24 |
| SamYaple | what version of docker and docker-yp | 14:24 |
| SamYaple | ? | 14:24 |
| sbezverk | SamYaple Docker 1.10.1 | 14:25 |
| sbezverk | Version: 1.7.0 | 14:25 |
| sbezverk | for Docker-py | 14:25 |
| SamYaple | hmmm | 14:26 |
| SamYaple | host os? | 14:26 |
| sbezverk | centos 7.2 | 14:26 |
| sdake | sbezverk are you pushing to a registry? | 14:26 |
| sbezverk | sdake yes | 14:26 |
| sdake | do you ahve the insecure registry flag set? | 14:26 |
| sdake | ps -ef | grep docker | 14:27 |
| SamYaple | sdake: that flag isnt used for registry v2 | 14:27 |
| sdake | oh really? | 14:27 |
| Serlex | This is strange. I had neutron-dhcp-agent failure, I ran it again and now it works | 14:27 |
| SamYaple | and 1.10 doesnt work with v1 | 14:27 |
| sdake | yes I know that | 14:27 |
| sdake | but didn't know insecure registry was a 1.0 only flag | 14:27 |
| sbezverk | sdake http://paste.fedoraproject.org/323493/32846145 | 14:27 |
| SamYaple | sdake: hmm ill double check that | 14:27 |
| sdake | sbezverk ok the problem is your using registry 1.0 | 14:28 |
| SamYaple | they deprecated the docker-py insecure flag | 14:28 |
| sdake | sbezverk registry 1.0 is not compatible with docker 1.10 | 14:28 |
| sdake | you need registry 2.3 | 14:28 |
| sbezverk | sdake what are my options then? new master required 1.10.1 based on yesterday's discussions.. | 14:28 |
| sdake | sbezverk uninstall docker registry rpm | 14:29 |
| sdake | then run | 14:29 |
| sdake | docker run -d -p 4000:5000 --restart=always --name registry registry:2:3 | 14:29 |
| sdake | then proceed as normal | 14:29 |
| sdake | doc fix incoming today for that | 14:29 |
| sbezverk | got it thank you | 14:29 |
| *** athomas has quit IRC | 14:29 | |
| SamYaple | you know it wasn't blocked by the docker team | 14:29 |
| SamYaple | so v1 registry not working would be a bug for you guys | 14:29 |
| sdake | well registery 2.3 is far superiorr in terms of perf why not just use that? | 14:30 |
| sdake | registry 1 is unmaintained | 14:30 |
| SamYaple | its not faster than v1 by much | 14:30 |
| SamYaple | but im not advocating for it | 14:30 |
| SamYaple | just saying its not a hard switch | 14:31 |
| sdake | i see | 14:31 |
| sdake | i thought it was cmopletely disabled because i t didn't work for me with master ;-) | 14:31 |
| sdake | so that sounds like a docker bug | 14:31 |
| sdake | anyway have to jet for a bit | 14:31 |
| sdake | to drive kiddo to school | 14:31 |
| SamYaple | yea pushing to v1 registry works for me | 14:33 |
| SamYaple | might be busted in docker-py.... but thats still a bug | 14:33 |
| SamYaple | docker daemon still allows push | 14:33 |
| *** achanda has joined #kolla | 14:35 | |
| Serlex | sdake, any ideas how kolla deploy worked in second run after failing on neutron-dhcp-agent failure | 14:35 |
| Serlex | very strange | 14:35 |
| Serlex | Horizon doesn't seem to respond though | 14:36 |
| SamYaple | Serlex: what was the time between run? | 14:36 |
| Serlex | 3 days | 14:36 |
| SamYaple | dhcp failures are associated with not having the MountFlags=shared with the docker daemon | 14:36 |
| Serlex | I gave up about 3 days ago, I thought I do some kolla tinkering today, and it worked -_- | 14:37 |
| *** jtriley has joined #kolla | 14:37 | |
| Serlex | oh! that says it then. I just changed that after reading the conversation on here | 14:37 |
| Serlex | thanks SamYaple | 14:38 |
| Serlex | I made the change blindly lol | 14:38 |
| SamYaple | Serlex: always good to trust the random internet people | 14:39 |
| *** achanda has quit IRC | 14:40 | |
| *** athomas has joined #kolla | 14:41 | |
| sdake | Serlex was afk ad going afk for about an hour - looks like your sorted | 14:42 |
| limamauricio | 2016-02-15 18:30:09.932 22 INFO cinder.volume.manager [req-ae123e1a-2b66-4f2a-9257-83d1a65ed216 - - - - -] Starting volume driver RBDDriver (1.2.0) 2016-02-15 18:30:09.933 22 ERROR cinder.volume.manager [req-ae123e1a-2b66-4f2a-9257-83d1a65ed216 - - - - -] Failed to initialize driver. | 14:42 |
| limamauricio | enable_ceph: "yes" | 14:43 |
| limamauricio | VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: rados and rbd python libraries not found | 14:44 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Move socket binding to named_volume https://review.openstack.org/280429 | 14:44 |
| SamYaple | limamauricio: is this on master? | 14:44 |
| limamauricio | yes | 14:44 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 14:45 |
| SamYaple | limamauricio: ubuntu/centos? | 14:45 |
| limamauricio | centos | 14:45 |
| limamauricio | source | 14:45 |
| limamauricio | AIO | 14:45 |
| SamYaple | limamauricio: can fixing it now | 14:45 |
| SamYaple | ok* | 14:45 |
| limamauricio | this is a known bug? | 14:46 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Fix missing ceph-common package in cinder https://review.openstack.org/280738 | 14:47 |
| SamYaple | limamauricio: it is now. thats teh fix | 14:47 |
| SamYaple | i dont test centos so i could not have caught that before limamauricio | 14:47 |
| SamYaple | i guess no one has done centos + ceph + source | 14:47 |
| limamauricio | then I'm the guy ;P | 14:48 |
| SamYaple | iwannabetheguy | 14:49 |
| limamauricio | ok, you can be the guy | 14:50 |
| SamYaple | lol its a game | 14:50 |
| SamYaple | a very hard game | 14:50 |
| *** jtriley has quit IRC | 14:52 | |
| SamYaple | Jeffrey4l: could you help me with a review here? I need this to finish out the "functional" gate work https://review.openstack.org/280738 | 14:56 |
| Jeffrey4l | SamYaple, np | 14:56 |
| * Jeffrey4l is checking | 14:56 | |
| SamYaple | if we are ok with this approach I can move on | 14:57 |
| *** coolsvap|away has quit IRC | 14:57 | |
| Jeffrey4l | SamYaple, reviewed +2 | 14:59 |
| *** subscope has quit IRC | 14:59 | |
| mkoderer | limamauricio: pls let me know if you need help with testing https://review.openstack.org/#/c/269688/ | 14:59 |
| *** chandankumar has joined #kolla | 14:59 | |
| *** jtriley has joined #kolla | 15:01 | |
| *** coolsvap|away has joined #kolla | 15:01 | |
| SamYaple | Jeffrey4l: did you submit the review? or are you just letting me know? | 15:01 |
| Jeffrey4l | SamYaple, just let you know :D | 15:02 |
| SamYaple | cool | 15:03 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Reduce gate resource usage https://review.openstack.org/280029 | 15:05 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 15:05 |
| Jeffrey4l | SamYaple, I tested serveral times for your rsyslog named volume socket PS. it seem that the issue is really fixed (or at least reduced). But I still have no idea why? could you show the magic? | 15:05 |
| SamYaple | Jeffrey4l: i moved the socket into a named_volume | 15:05 |
| SamYaple | the issue before (I think) is a mount ordering issue from docker-py to docker | 15:06 |
| *** subscope has joined #kolla | 15:06 | |
| SamYaple | so _sometimes_ /dev/log would try to bind in before /dev | 15:06 |
| Jeffrey4l | I saw that. But I think use a named_volume and mount a host folder should be the same. | 15:06 |
| SamYaple | no because look at base/start.sh | 15:06 |
| SamYaple | we dont bind to /dev/log | 15:07 |
| SamYaple | we symlink after the contaienr starts | 15:07 |
| Jeffrey4l | OK. got it. | 15:07 |
| Serlex | someone help me get horizon working so I can call this successful kolla AIO deployment :D | 15:08 |
| Jeffrey4l | Serlex, error message? | 15:08 |
| SamYaple | Serlex: sure an particular issue? | 15:09 |
| SamYaple | not deploying? | 15:09 |
| SamYaple | not building? | 15:09 |
| Serlex | "The connection was reset" via the browser. Horizon container /var/log/http/error_log | 15:09 |
| Serlex | [Tue Feb 16 15:00:53.535568 2016] [auth_digest:notice] [pid 1] AH01757: generating secret for digest authentication ... | 15:09 |
| Serlex | [Tue Feb 16 15:00:53.536990 2016] [lbmethod_heartbeat:notice] [pid 1] AH02282: No slotmem from mod_heartmonitor | 15:09 |
| Serlex | [Tue Feb 16 15:00:53.551720 2016] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.6 () mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations | 15:09 |
| Serlex | [Tue Feb 16 15:00:53.551766 2016] [core:notice] [pid 1] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' | 15:09 |
| Serlex | Built, deployed | 15:09 |
| Serlex | more openstack problem than kolla i guess | 15:10 |
| Serlex | or misconfigured networking | 15:10 |
| Jeffrey4l | (use a paste service pls, paste.openstack.org) | 15:10 |
| Serlex | sure | 15:10 |
| Serlex | http://pastebin.com/34uBffNQ | 15:11 |
| Serlex | I ran a tcpdump and I can see the HTTP requests coming in | 15:11 |
| SamYaple | Serlex: i do not see this issue on ubuntu | 15:11 |
| SamYaple | ill let Jeffrey4l help you out | 15:11 |
| Serlex | browser request times out after couple minutes | 15:11 |
| Serlex | I probably did something wrong | 15:11 |
| SamYaple | its hard to mess up this stage of kolla | 15:11 |
| SamYaple | if the other stuff works, its probably not you | 15:12 |
| *** fgimenez has quit IRC | 15:12 | |
| Jeffrey4l | Serlex, try use `nova list` on the host | 15:12 |
| Jeffrey4l | test other service is ok. | 15:12 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 15:12 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Reduce gate resource usage https://review.openstack.org/280029 | 15:12 |
| sdake | I was thinking - we should limit enabled by default services to basically compute kit | 15:13 |
| sdake | which heat is not in | 15:13 |
| sdake | any thoughts? | 15:13 |
| SamYaple | i agree with no heat, i think horizon is needed | 15:13 |
| sdake | gotta run my kids to school | 15:13 |
| SamYaple | heat doesnt fully work | 15:13 |
| sdake | which part is broken? | 15:14 |
| SamYaple | when it does it needs ceilometer | 15:14 |
| sdake | trusts still busted? | 15:14 |
| SamYaple | you never merged it... | 15:14 |
| sdake | I never wrote it, nihilifer did | 15:14 |
| sdake | i am pretty sure nihilifer fixed that problem | 15:14 |
| SamYaple | ok its got enough cores ill merge | 15:14 |
| *** fgimenez has joined #kolla | 15:14 | |
| *** fgimenez has quit IRC | 15:14 | |
| *** fgimenez has joined #kolla | 15:14 | |
| sdake | ya we shouldn't be single threading on anyone in the community especially me :) | 15:15 |
| SamYaple | you specifically asked for this | 15:15 |
| SamYaple | I rebased this for you sdake http://pastebin.com/34uBffNQ | 15:15 |
| SamYaple | oops | 15:15 |
| SamYaple | https://review.openstack.org/#/c/276851/ | 15:15 |
| SamYaple | so please remove -1 when you get a chance | 15:16 |
| sdake | done and w+1 | 15:16 |
| sdake | thanks again | 15:16 |
| SamYaple | anytime | 15:16 |
| openstackgerrit | Merged openstack/kolla: Use trusts in heat.conf https://review.openstack.org/236198 | 15:16 |
| sdake | folks, if you ahve patches awaiting review and they have gate failures, please rebase them on master so they will recheck | 15:17 |
| sdake | it takes a long ime to leave a comment in every patch just for that | 15:17 |
| SamYaple | of note- centos binary will still fail on designate | 15:18 |
| SamYaple | jeffrey has a fix for that | 15:18 |
| *** blahRus has joined #kolla | 15:18 | |
| sdake | the only gate that should be failing is build-centos-binary | 15:18 |
| sdake | SamYaple have pointer to review? | 15:18 |
| SamYaple | 280658 | 15:18 |
| openstackgerrit | Merged openstack/kolla: Make a final check to validate mysql works https://review.openstack.org/276851 | 15:18 |
| SamYaple | https://review.openstack.org/#/c/280658/ | 15:18 |
| Serlex | running nova-list | 15:18 |
| sdake | thanks SamYaple thanks Jeffrey4l | 15:19 |
| sdake | ok folks wait for that patch to merge | 15:19 |
| sdake | then please rebase | 15:19 |
| sdake | i'll send a note to the list | 15:19 |
| sdake | many notes coming to the list today ;) | 15:19 |
| sdake | after I get done dealing with school | 15:19 |
| sdake | bbiaf | 15:19 |
| sdake | SamYaple odd how the original code installed designate-api in the base image, and someone otok it out, with that big comment above | 15:20 |
| sdake | bbiaf | 15:20 |
| openstackgerrit | Merged openstack/kolla: install the openstack-designate-api in the base stage https://review.openstack.org/280658 | 15:20 |
| *** Liuqing has quit IRC | 15:21 | |
| *** Liuqing has joined #kolla | 15:22 | |
| *** Liuqing has quit IRC | 15:22 | |
| Serlex | nova list on a AIO should be fairly quick right? | 15:23 |
| SamYaple | Serlex: it should be quick everywhere | 15:24 |
| SamYaple | but if its hanging there could be any number of reasons | 15:24 |
| SamYaple | its probably an issue with kolla, but it could be swpaping or a slow system | 15:24 |
| sdake | Serlex make sure your VIP is an unused IP address on your network | 15:25 |
| sdake | Serlex if not, kolla will behave exactly as you describe - nova-list being pokey | 15:25 |
| sdake | or not working | 15:25 |
| Jeffrey4l | Serlex, try `keystone --debug user-list` first, and check why it is blocked. | 15:26 |
| Jeffrey4l | s/why/where/ | 15:26 |
| sdake | serlex docker ps | fpaste | 15:26 |
| Serlex | ok one sec | 15:26 |
| sdake | serlex lets see if all yourcontainers are running | 15:26 |
| sdake | ok really gotta jet | 15:26 |
| Serlex | all but toolbox | 15:26 |
| sdake | toolbox not running? | 15:27 |
| sdake | docker logs the toolbox id | 15:27 |
| Serlex | looks like exited | 15:27 |
| sdake | it should not exit | 15:27 |
| Serlex | guessing thats bit crucial | 15:27 |
| Jeffrey4l | toolbox is not big deal. Shoul exit due to the `/dev/log` issue. | 15:27 |
| sdake | gotta go | 15:27 |
| Serlex | ok cya | 15:28 |
| Serlex | http://pastebin.com/48ffjwkG | 15:28 |
| Jeffrey4l | neutron-openvswitch-agent is restarting. | 15:28 |
| Serlex | "keystone --debug user-list" is showing its trying to connect to proxy im behind off | 15:28 |
| Jeffrey4l | neutron-openvswitch-agent should not be the current issue, too. | 15:29 |
| Jeffrey4l | try to fix the proxy issue first. | 15:29 |
| Serlex | sure | 15:29 |
| Jeffrey4l | should remove the proxy? | 15:29 |
| Serlex | I will try without | 15:29 |
| Serlex | DEBUG:keystoneclient.session:Request returned failure status: 404 | 15:36 |
| Jeffrey4l | Serlex, more info? | 15:37 |
| Serlex | hmm I don't think my openrc is misconfigured | 15:37 |
| Serlex | sorry that was the output from keystone user-list - http://pastebin.com/AfBTVpii | 15:38 |
| limamauricio | mkoderer: I'll update the patch | 15:38 |
| limamauricio | and you can try to deploy it and create a share :) | 15:38 |
| Serlex | I can see via netsat 10.0.3.17:35357 is listening | 15:39 |
| *** salv-orl_ has joined #kolla | 15:40 | |
| *** salv-orl_ has quit IRC | 15:40 | |
| *** salv-orlando has joined #kolla | 15:41 | |
| Jeffrey4l | Serlex, it should be right. the keystoneclient (i think) should not handle the new type endpoint ( without version ) properly. | 15:41 |
| Jeffrey4l | wait for a moment. Or just try `openstack --debug user list` | 15:42 |
| Serlex | I got a success with openstack --debug user list | 15:43 |
| Jeffrey4l | Serlex, great. Try `openstack --debug server list` | 15:44 |
| Jeffrey4l | I think your root cause is the proxy. After removed that, you can try the dashboard again now. | 15:45 |
| Serlex | "IndexError: list index out of range" for server list | 15:45 |
| Serlex | I can see neutron openswiitch-agent container is in restarting state again | 15:46 |
| Serlex | logs show | 15:46 |
| Serlex | "2016-02-16 15:43:30.915 1 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-7b980d98-1b95-43b8-b624-ac7f8438cc02 - - - - -] Bridge br-ex for physical network physnet1 does not exist. Agent terminated!" | 15:46 |
| Serlex | I will try dashboard again | 15:46 |
| Jeffrey4l | are you using the latest code? this br-ex issue should be fixed now. Serlex | 15:47 |
| Serlex | oh - no I'm not | 15:47 |
| Serlex | woops | 15:47 |
| Serlex | But would that stop dashboard from appearing? | 15:48 |
| Jeffrey4l | Serlex, no. | 15:48 |
| Jeffrey4l | It just break the boot vms. | 15:48 |
| Serlex | AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain. Set the 'ServerName' directive globally to suppress this message | 15:48 |
| *** guillaume_ has joined #kolla | 15:49 | |
| Jeffrey4l | this is not a issue. | 15:49 |
| *** dave-mccowan has quit IRC | 15:53 | |
| SamYaple | Jeffrey4l: with the logging patch, im not sure what you are saying | 15:54 |
| SamYaple | its probably still waiting on the socket to appear | 15:55 |
| Jeffrey4l | A simple test, run `logger a` in kolla_toolbox container. this message will be received by the host rsyslog rather than the rsyslog container. | 15:56 |
| Jeffrey4l | SamYaple, ^ | 15:56 |
| SamYaple | i think you miss my point | 15:57 |
| SamYaple | check whats running in keystone container | 15:57 |
| SamYaple | its probably looping waiting for socket to appear, no | 15:57 |
| SamYaple | unless you didnt do a full rebuild of your containers | 15:57 |
| SamYaple | and then logging just wont work | 15:57 |
| sbezverk | SamYaple your fix seems helping with kolla-toolbox, at least it was created and started properly | 15:57 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add ansible support for Manila https://review.openstack.org/269688 | 15:58 |
| sbezverk | but I hit an issue with neutron-dhcp-agent, the container was created but by some reason it has not started.. | 15:58 |
| Jeffrey4l | SamYaple, keystone container works. Because it doesn't mount the host `/dev` folder. | 15:58 |
| Jeffrey4l | kolla_toolbox mount the `/dev/` folder and the `/dev/log` file is the hosts. | 15:59 |
| sbezverk | SamYaple when I try to start it manually I get this error: Error response from daemon: Cannot start container 7d273c63087cad3be5d51f023c568072f12e55967ead661e804b2a6354bd7358: Path /run/netns is mounted on /run but it is not a shared mount. | 15:59 |
| Serlex | sbezvek I think that was solved with shared MountFlags in docker.service | 16:00 |
| Serlex | MountFlags=shared | 16:00 |
| Serlex | Jeffrey4l | 16:00 |
| *** akwasnie has quit IRC | 16:00 | |
| Serlex | I'm not sure what is stopping horizon, something isn't right, probably network issue | 16:00 |
| Jeffrey4l | Serlex, what's the error now? | 16:01 |
| sbezverk | Serlex Thank you, so I change it and restart docker, right? | 16:01 |
| Serlex | horizon times out in browser still | 16:01 |
| Jeffrey4l | try to use a curl command to test ` curl <ip>:80 -vvvv` | 16:02 |
| Jeffrey4l | still the proxy issue? | 16:02 |
| Serlex | not proxy | 16:02 |
| *** salv-orl_ has joined #kolla | 16:02 | |
| Serlex | keystone --debug user-list is given me 404 | 16:02 |
| Jeffrey4l | are u sure the proxy is removed totally? include the environments in the containers. | 16:03 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add ansible support for Manila https://review.openstack.org/269688 | 16:03 |
| Jeffrey4l | Serlex, does `nova list` works? | 16:03 |
| Serlex | I've checked various places, I've only added it to .bash_profile today to have a look at kolla at work | 16:04 |
| Serlex | ermm | 16:04 |
| Serlex | it does now | 16:04 |
| Serlex | but its empty | 16:04 |
| Serlex | :-/ | 16:05 |
| *** salv-orlando has quit IRC | 16:05 | |
| Jeffrey4l | Serlex, try `curl <ip>:80 -vvv` | 16:05 |
| *** jmccarthy has quit IRC | 16:05 | |
| *** haplo37 has joined #kolla | 16:06 | |
| Serlex | * Connected to 10.0.3.17 (10.0.3.17) port 80 (#0) | 16:06 |
| Serlex | hmm | 16:06 |
| *** jmccarthy has joined #kolla | 16:06 | |
| Serlex | Thanks for the help Jeffrey4l, I got to go | 16:07 |
| Serlex | I will give it a go again later | 16:07 |
| SamYaple | Jeffrey4l: Oh so you are saying the other containers work, just not the ones mounting /dev | 16:07 |
| Jeffrey4l | Serlex, root cause is? | 16:07 |
| SamYaple | ? | 16:07 |
| Jeffrey4l | SamYaple, yup | 16:07 |
| *** inc0 has joined #kolla | 16:08 | |
| SamYaple | we may have to live with that until heka | 16:08 |
| Jeffrey4l | SamYaple, wait a minute. I think it should, but not tested. | 16:08 |
| SamYaple | i made a note in there | 16:08 |
| inc0 | hey, good morning | 16:08 |
| SamYaple | Jeffrey4l: with heka we don't need ot bind over /dev/log | 16:08 |
| SamYaple | morning inc0 | 16:08 |
| Jeffrey4l | SamYaple, yea. | 16:08 |
| Jeffrey4l | inc0, morning | 16:08 |
| inc0 | SamYaple, yesterdays weather made us spend night in Dallas | 16:09 |
| SamYaple | inc0: ouch | 16:09 |
| *** stvnoyes has quit IRC | 16:09 | |
| SamYaple | back home now? | 16:09 |
| *** Serlex has quit IRC | 16:09 | |
| inc0 | yeah, we arrived like an hour ago | 16:09 |
| *** stvnoyes has joined #kolla | 16:10 | |
| SamYaple | man that explains why you didnt respond to my messages | 16:10 |
| inc0 | what messages? | 16:10 |
| inc0 | I didn't get any | 16:11 |
| SamYaple | i asked if you arrived | 16:11 |
| SamYaple | on signal | 16:11 |
| inc0 | did not recieve. | 16:11 |
| SamYaple | hmm | 16:11 |
| SamYaple | wierd | 16:11 |
| SamYaple | signal has never been flaky for us before... | 16:12 |
| inc0 | anyway, you have your answer now;) | 16:12 |
| inc0 | good thing tho that american airlines gave us hotel | 16:12 |
| inc0 | so we didn't have to sleep on airport | 16:12 |
| *** dave-mccowan has joined #kolla | 16:12 | |
| inc0 | sdake, around? | 16:13 |
| *** Marga_ has quit IRC | 16:13 | |
| SamYaple | yea | 16:13 |
| *** JoseMello has joined #kolla | 16:13 | |
| SamYaple | Jeffrey4l: looking at the Heka stuff... that should work | 16:14 |
| SamYaple | anything requiring /dev/log (haproxy/mariadb) can specifcy an alternate log location | 16:15 |
| SamYaple | and the openstack services write to files | 16:15 |
| Jeffrey4l | ok | 16:16 |
| inc0 | SamYaple, did you figute out what was wrong with docker 1.10? | 16:16 |
| SamYaple | inc0: nothing with docker 1.10 | 16:16 |
| SamYaple | it *appears* that somewhere in the kolla_docker->docker-py->docker stack the mounts are getting reordered | 16:16 |
| SamYaple | so /dev/log tries to mount before /dev | 16:17 |
| inc0 | meh, thats strange | 16:17 |
| inc0 | let me see one thing | 16:17 |
| SamYaple | since the process for the binds is a conersation from string>list>dict>string>list>dict>rest-api it doesnt suprise me | 16:17 |
| sbezverk | Do not beleive my eyes, deployment has been completed sucessfully :-) | 16:17 |
| SamYaple | thats the path it takes from the yaml to the docker-egine inc0 | 16:18 |
| SamYaple | sbezverk: horizon and booting vm? | 16:18 |
| inc0 | and dict in between messes up ordering I'd bet | 16:18 |
| SamYaple | inc0: we already attempted an ordereddict fix | 16:18 |
| sbezverk | SamYaple Horizon is up | 16:18 |
| SamYaple | in both docker-py and kolla_docker | 16:18 |
| SamYaple | i confirmed it still doesnt break for me with a non-ordereddict | 16:18 |
| sbezverk | networking config does not look right | 16:18 |
| SamYaple | sbezverk: well youre getting there | 16:19 |
| sbezverk | but it is minor | 16:19 |
| *** cloudnautique has joined #kolla | 16:19 | |
| *** cloudnautique has quit IRC | 16:19 | |
| *** cloudnautique has joined #kolla | 16:20 | |
| sbezverk | SamYaple, normally for AIO you do not create br-ex and you plug "external" interface directly to br-int | 16:20 |
| SamYaple | sbezverk: thats not normal, no | 16:21 |
| SamYaple | you _can_ do that I suppose... | 16:21 |
| sbezverk | SamYaple :-) believe me it is, I have done about 100 all in one, and it is the only way to get connectivity | 16:21 |
| SamYaple | that doesnt make it normal | 16:22 |
| SamYaple | I have done 100's as well and never done that | 16:22 |
| SamYaple | doing it that way prevents scaling to multinode, prevents testing vlan | 16:22 |
| SamYaple | prevents alot actually | 16:23 |
| d_code | sdake: so…still have plenty of things to work on…but I’ve got networking up, provisioned a couple of boxes…. no avc denials at all on CentOS | 16:23 |
| d_code | I wonder if the new docker selinux package addressed many of the things | 16:23 |
| *** fgimenez has quit IRC | 16:24 | |
| sbezverk | SamYaple I will try to get connectivity with current setup built by kolla-deploy if it does not work I will ping you for suggestions | 16:24 |
| SamYaple | sbezverk: for a single interface deploy it can get complicated | 16:24 |
| SamYaple | is that what you are doign? | 16:24 |
| sbezverk | SamYaple, nope I have 1 interface "external" mapped to routable vlan, 2nd interface is 8021q trunk used for tennats and 3rd interface is management interface | 16:26 |
| *** fgimenez has joined #kolla | 16:26 | |
| *** fgimenez has quit IRC | 16:26 | |
| *** fgimenez has joined #kolla | 16:26 | |
| *** daneyon has joined #kolla | 16:27 | |
| SamYaple | sbezverk: then im confused as to the issue | 16:30 |
| SamYaple | do you have an ip that you use on your "external" interface? | 16:30 |
| SamYaple | if its just L2 then you can create a br-ex without issue | 16:30 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 16:33 |
| sbezverk | SamYaple If you check Red Hat packstack for opensatck installation, when AIO is selected br-ex is not getting created, so I do not know whether it is "normal" or not but it is how we were advised by RH since havana.. | 16:34 |
| SamYaple | sbezverk: but see thats different than what you said | 16:36 |
| limamauricio | Sam, ERROR cinder.backup.manager [req-dcc698ac-678e-40da-9b01-1f44799a0f9f - - - - -] Error encountered during initialization of driver: RBDDriver | 16:36 |
| SamYaple | but i am not talking about those dev tools | 16:36 |
| limamauricio | cinder_backup logs | 16:36 |
| SamYaple | this is meant for production grade stuff and AIO is the same as multinode | 16:36 |
| SamYaple | the br-ex stuff is required for neutron-l3-agent only | 16:37 |
| limamauricio | I did your patch, and cinder_volume is ok | 16:37 |
| limamauricio | :) | 16:37 |
| SamYaple | limamauricio: so cinder-backup has issue though... | 16:37 |
| SamYaple | whats the full past for that limamauricio ? | 16:38 |
| *** achanda has joined #kolla | 16:38 | |
| SamYaple | sbezverk: what is it you are looking to get out of Kolla in this case? as it stands we need br-ex for external communication for vms, even on multinode. that likely wont change | 16:39 |
| limamauricio | SamYaple -> http://pastebin.com/mk4LbVji | 16:39 |
| SamYaple | we can scale from AIO to multinode to production multinode and back down to AIO with the system we have, and that means br-ex has to be there even in AIO | 16:39 |
| *** dwalsh has quit IRC | 16:39 | |
| SamYaple | limamauricio: I found it | 16:40 |
| SamYaple | ill fix it, moment | 16:40 |
| sbezverk | SamYaple it is ok, if it works the way kolla deploy builds is it is perfectly fine.., but I wan to change some neutron parameters , to activate them I just restart neutron server container? I selected COPY_ALWAYS in globals.yml | 16:40 |
| limamauricio | Ok | 16:40 |
| SamYaple | sbezverk: yes | 16:42 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Use "optional" for ceph confs https://review.openstack.org/280808 | 16:42 |
| SamYaple | limamauricio: test that out | 16:42 |
| SamYaple | cinder-backup was broken for all it would appear :) | 16:42 |
| sbezverk | SamYaple if I need to add a custom neutron plugin, example sriov, any guidance how to accomplish it? | 16:42 |
| SamYaple | we do not have a plugin architecture thats fully flushed out yet sbezverk | 16:43 |
| SamYaple | right now they would have to be build into the containers | 16:43 |
| sbezverk | SamYaple :-( | 16:43 |
| SamYaple | built* | 16:43 |
| SamYaple | so we can do plugins, but it requires rebuilds | 16:43 |
| limamauricio | Thank Sam | 16:43 |
| *** achanda has quit IRC | 16:44 | |
| *** dwalsh has joined #kolla | 16:45 | |
| SamYaple | sbezverk: im afraid you may have taken that the wrong way, we _do_ have plugins, but you have to rebuild the container with a different conf rather than just start a new container | 16:46 |
| SamYaple | i use sr-iov myself | 16:46 |
| sdake | back in | 16:49 |
| sbezverk | SamYaple, I see great, would you have a wiki or some steps tp build custom container? By some reason I could not find where ovs maps physical server interfaces to ovs bridges.. normally it would be sitting openvswitch-neutron-agent.ini | 16:49 |
| *** dwalsh has quit IRC | 16:50 | |
| *** tzn has quit IRC | 16:50 | |
| inc0 | sdake, just came back from SC, did you ask people about backporting thin containers? | 16:51 |
| *** ssurana has joined #kolla | 16:53 | |
| *** mikelk has quit IRC | 16:53 | |
| *** subscope has quit IRC | 16:56 | |
| SamYaple | sbezverk: again, those files are just random files beople use. openvswitch-neutron-agent.ini though hasn't been used since ml2 came around | 16:59 |
| sdake | inc0 going out today with thatemail | 16:59 |
| SamYaple | youll find that info in ml2_conf.ini | 16:59 |
| sdake | i was waiting until tuesday so folks would actually read the email :) | 16:59 |
| inc0 | kk thanks | 16:59 |
| SamYaple | sdake: that logging patch | 17:00 |
| SamYaple | socket* | 17:00 |
| sdake | the one you wrote? | 17:00 |
| sdake | right | 17:00 |
| SamYaple | yea | 17:00 |
| SamYaple | so Jeffrey4l pointed out somethign great | 17:00 |
| SamYaple | when binding in host /dev you cant symlink like the patch does | 17:00 |
| sdake | i guess i should read scrollback ;) | 17:00 |
| SamYaple | basically what this means is we lose logging for cinder-volume | 17:00 |
| SamYaple | _until_ Heka | 17:00 |
| SamYaple | with heka (as I made a note of) we shouldnt need that symlink | 17:01 |
| sdake | got it | 17:01 |
| SamYaple | also logging for kolla_toolbox, but I can fix that | 17:01 |
| SamYaple | i think | 17:01 |
| SamYaple | either way after heka both logging issues are solved | 17:01 |
| SamYaple | i think its reasonable to merge now and then focus on heka | 17:01 |
| sdake | merge your patch? | 17:02 |
| sdake | I thought I +2'ed it already | 17:02 |
| sdake | just got back in office haven't read email | 17:02 |
| SamYaple | sdake: no you have a +2 jeffreys comment was after your +2 | 17:03 |
| sdake | got it so it needs another review on it | 17:03 |
| SamYaple | well it needs everyone to understand why it broke | 17:04 |
| sdake | reading review and processing omemnt | 17:04 |
| *** Marga_ has joined #kolla | 17:05 | |
| SamYaple | sdake: i just added comment | 17:05 |
| SamYaple | make sure to refresh before secodn review | 17:05 |
| *** Marga_ has quit IRC | 17:05 | |
| *** Marga_ has joined #kolla | 17:05 | |
| sdake | Jeffrey4l can you ahve a look at the review here; https://review.openstack.org/#/c/280429/2 | 17:06 |
| sdake | so do you need to remove the symlink entirely? | 17:06 |
| SamYaple | well it wont be needed after heka | 17:07 |
| SamYaple | i dont have to do anything to the patch as-is | 17:07 |
| SamYaple | with the outcome I say | 17:07 |
| SamYaple | but after heka no need to use /dev/log | 17:07 |
| *** blahRus1 has joined #kolla | 17:07 | |
| SamYaple | we can point haproxy and keepalived to a socket anywhere, not just /dev/log | 17:07 |
| *** blahRus has quit IRC | 17:08 | |
| *** aginwala has joined #kolla | 17:09 | |
| SamYaple | fyi sdake im removing lots fo the /run mounts | 17:12 |
| SamYaple | named_volumes for all! | 17:12 |
| sdake | wfm | 17:13 |
| sdake | i hated those run mounts | 17:13 |
| sdake | be careful with libvirt and nova however | 17:13 |
| sdake | it needs to communicate with systemd on teh host | 17:13 |
| SamYaple | libvirt and nova are tcp based | 17:13 |
| sdake | oh right that got changed | 17:13 |
| SamYaple | il probably change it back after the socket thing | 17:13 |
| SamYaple | maybe. if i can address nihilifer concerns | 17:13 |
| sdake | but libvirt still may use systemd's dbus docket | 17:13 |
| *** chandankumar has quit IRC | 17:13 | |
| sdake | I would suggest double checking on a system with systemd your patch for that if you intend to named volumes the run for libvirt and friends | 17:14 |
| sdake | specifically an upgrade of the container | 17:14 |
| *** fthiagogv_ has quit IRC | 17:14 | |
| sdake | or separate out the nova patch from the others that way one goes in fast and the other can block for testing | 17:14 |
| SamYaple | ill start simple with openvswitch and haproxy | 17:15 |
| SamYaple | you can review the nova patch | 17:15 |
| sdake | if you test with systemd that is good enough for me | 17:16 |
| sdake | if you want me to test it i can do so | 17:16 |
| *** macsz has quit IRC | 17:16 | |
| sdake | if I can get my env going ;-) | 17:16 |
| SamYaple | im running 15.10 | 17:16 |
| SamYaple | so its all systemd | 17:16 |
| sdake | nice | 17:16 |
| sdake | right | 17:16 |
| *** mbound has quit IRC | 17:19 | |
| kproskurin | and what about poor ubuntu 14.04 users? :-\ | 17:22 |
| SamYaple | kproskurin: no issue! | 17:23 |
| kproskurin | Btw, right now ubuntu 14.04 is busted, ‘cos it doesnt have systemd so there is no MountFlags=shared fix | 17:23 |
| SamYaple | works perfect | 17:23 |
| Jeffrey4l | SamYaple, i have removed my -1 | 17:23 |
| SamYaple | kproskurin: no its all busted sometimes | 17:23 |
| SamYaple | thats waht my patch is for | 17:23 |
| kproskurin | I did mount --make-shared /run to fix that, but it’s need to be a better way, like upstart job or something | 17:23 |
| SamYaple | oh that | 17:24 |
| SamYaple | kproskurin: d you think we should document modifying the upstrart job to do that/ | 17:28 |
| sbezverk | SamYaple After server reboot, kolla-toolbox fails to start even with the patch | 17:28 |
| SamYaple | I understand and agree thats a bit annoying | 17:28 |
| kproskurin | SamYaple: well if we want to support ubuntu 14.04 - yeah we should :-) | 17:28 |
| SamYaple | thats fair | 17:28 |
| kproskurin | ‘cos it not working right now | 17:28 |
| SamYaple | and we absolutely do | 17:28 |
| SamYaple | i just added a second upstart job to change it | 17:29 |
| SamYaple | but maybe modifying the docker one is better? | 17:29 |
| kproskurin | could you send a link to review? | 17:29 |
| *** fgimenez has quit IRC | 17:29 | |
| SamYaple | i dont have one | 17:29 |
| SamYaple | i did it locally | 17:29 |
| sdake | sbezverk can you run docker logs on the kolla-toolbox container id plz | 17:30 |
| kproskurin | I have no idea how to fix it “right”, ‘cos I not fammiliar with upstart. I tried to find something like MountFlags in upstart and looks like there is none | 17:30 |
| sdake | anad pipe into fpaste | 17:30 |
| SamYaple | kproskurin: we could require a second mount on /run/netns and have only that one be shared | 17:31 |
| SamYaple | systemd takes care of making mounts shared i think | 17:31 |
| kproskurin | yep, mount -bind or something | 17:32 |
| SamYaple | well run is a tmpfs anyway | 17:32 |
| SamYaple | we could just do a second tmpfs | 17:32 |
| sbezverk | sdake here you go: http://paste.fedoraproject.org/323634/14556439 | 17:33 |
| limamauricio | SamYaple, i think the "true" value must be enclosed in quotes https://review.openstack.org/#/c/280808/ | 17:33 |
| kproskurin | SamYaple: just to save your time: findmnt -o TARGET,PROPAGATION in ubuntu 14.04 doesnt know about PROPAGATION flag, so if you want to check is mount is shared you have to look at /proc/self/mountinfo | 17:34 |
| *** aginwala has quit IRC | 17:34 | |
| sdake | sbezverk that is with dockage images of the docker-toolbox id? | 17:34 |
| SamYaple | kproskurin: shoot lol tahts where i was heading | 17:34 |
| SamYaple | ok so direct to proc | 17:34 |
| *** dims has joined #kolla | 17:34 | |
| kproskurin | have to go, bay all! | 17:35 |
| sbezverk | sdake yes | 17:35 |
| sdake | sbezverk journalctl -u docker | fpaste | 17:35 |
| *** vhosakot has joined #kolla | 17:35 | |
| sdake | by kproskurin | 17:35 |
| SamYaple | thanks kproskurin | 17:35 |
| sdake | 89 people in hannel | 17:35 |
| sdake | winning | 17:35 |
| sbezverk | http://paste.fedoraproject.org/323635/64414714 | 17:36 |
| vhosakot | hi all, still working on getting my dev env going.... I filed https://bugs.launchpad.net/bugs/1545926 to update the quickstart guide with missing commands | 17:36 |
| openstack | Launchpad bug 1545926 in kolla "Update doc/quickstart.rst" [Undecided,In progress] - Assigned to Vikram Hosakote (vhosakot) | 17:36 |
| SamYaple | limamauricio: can you test that real quick? youre probably right | 17:36 |
| *** aginwala has joined #kolla | 17:36 | |
| SamYaple | vhosakot: fyi, no need for a bug to fix docs | 17:36 |
| SamYaple | vhosakot: but no harm doing so | 17:36 |
| limamauricio | I tested, and i have a invalid json file | 17:36 |
| *** jtriley has quit IRC | 17:36 | |
| SamYaple | yea thats probably right | 17:36 |
| vhosakot | SamYaple: ok, didn't know... so the patch set to update doc need not have a track bug in Launchpad ? | 17:37 |
| SamYaple | let me look at code | 17:37 |
| SamYaple | vhosakot: correct | 17:37 |
| vhosakot | tracking* | 17:37 |
| vhosakot | cool | 17:37 |
| sdake | sandbox issue SamYaple after a reboot it looks like with your patch | 17:37 |
| SamYaple | vhosakot: for docs-only patches | 17:37 |
| vhosakot | cool | 17:37 |
| SamYaple | sdake: ? | 17:37 |
| sdake | sbezverk can you do git shortlog -5 | fpaste | 17:37 |
| sdake | SamYaple see sbezverk last paste | 17:37 |
| sdake | he rebooted | 17:37 |
| sdake | and kolla toolbox container fails to start | 17:37 |
| sdake | with a sandbox error | 17:38 |
| sdake | he previously had a deployment rolling | 17:38 |
| sdake | sbezverk arey ou running the named mount patch in the patch queue (that is what he git shortlog is meant to determine) | 17:39 |
| *** kproskurin has quit IRC | 17:39 | |
| sbezverk | sdake git shortlog does not produce anything, for the patch I dowloaded diff and manually applied with patch command | 17:39 |
| *** Jeffrey4l has quit IRC | 17:40 | |
| sdake | sbezverk there is a far superior way to do it, use cherrypick option in the review screen - copy and paste the operation | 17:40 |
| sdake | it is possible your patch didn't get applied correctly | 17:41 |
| *** jtriley has joined #kolla | 17:41 | |
| vhosakot | In the quickstart guide, the part about making Docker use an insecure registry (other_args), the commands are for CentOS.... for Ubuntu, to do this (make docker use insecure registry), I need to have "DOCKER_OPTS="-H fd:// $other_args" in the file /etc/init.d/docker, right ? After adding this, I need to retstart the docker service (sudo service docker restart).. Is this right ? | 17:41 |
| sdake | or its possible reboot is busted | 17:41 |
| SamYaple | vhosakot: thats correct for 14.04 ubuntu, not past taht | 17:41 |
| SamYaple | in 15.10 you have to edit systemd file | 17:42 |
| sdake | master docs in registry are wrong | 17:42 |
| vhosakot | ok... yes.. I'm on 14.04... thanks! | 17:42 |
| sdake | on registry | 17:42 |
| sdake | someone is working on a patch atm | 17:42 |
| sbezverk | openvswitch does not read or use new config. I changed /etc/kolla/neutron-server/ml2_conf.ini to add mapping for tenants interface bridge_mappings = physnet1:br-ex,physnet2:br-tenants_b0 I do not see openvswitch agent plugging br-tenants_b0 into br-int bridge? I rebooted a server but no help. ANy ideas what I am doing wrong? | 17:43 |
| vhosakot | to see how many containers the "kolla-build" command built, I can check it in "docker images" right ? | 17:44 |
| sdake | sbezverk well, if your using copy-once config strategy which is teh default, it unfortunately wont read the config until we get done with the reconfig blueprint | 17:44 |
| sdake | sbezverk another short-term option is to use config-strategy config_always | 17:44 |
| sdake | or COPY_ALWAYS that is | 17:44 |
| sbezverk | sdake I used COPY_ALWAYS | 17:45 |
| sdake | did you redeploy or just reboot? | 17:47 |
| SamYaple | sbezverk: you have to create br-tenants_b0 | 17:47 |
| sdake | you ahve to redeploy | 17:47 |
| sdake | it will upgrade and reconfig | 17:47 |
| SamYaple | did you do that? | 17:47 |
| sbezverk | sdake just rebooted | 17:47 |
| sdake | just a reboot wont do it | 17:47 |
| sdake | you dont want to reboot your cloud nodes anyway :) | 17:47 |
| SamYaple | limamauricio: no i meant can you test your suggested change | 17:47 |
| sdake | sbezverk try a kolla-ansible deploy again | 17:48 |
| sdake | your existing cloud will not be disrupted | 17:48 |
| sbezverk | sdake well, I have a problem then as re-deploy will not work due to a screw up with kolla-toolbox | 17:48 |
| sdake | but new config options will be put in place | 17:48 |
| vhosakot | sorry for duplicate message.... to see how many containers the "kolla-build" command built, I can check it in "docker images" right ? | 17:48 |
| sdake | i hear you with the kolla-toolbox problem, I susicion is the patch was not applied correctly | 17:48 |
| *** shardy has quit IRC | 17:48 | |
| sdake | wipe out all existing containers | 17:49 |
| sdake | (the ones running) | 17:49 |
| sdake | cleanup-containers | 17:49 |
| sdake | in the tools directory | 17:49 |
| sdake | docker rm -f kolla-toolbox | 17:49 |
| sdake | in your repo git reset --hard | 17:49 |
| sdake | and run the cherrypick command from gerrit | 17:49 |
| *** chandankumar has joined #kolla | 17:49 | |
| sdake | or wait for the patch to pull into master | 17:49 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Use "optional" for ceph confs https://review.openstack.org/280808 | 17:50 |
| sbezverk | sdake ETA for this patch in the master? | 17:50 |
| SamYaple | sbezverk: soon as another core approves | 17:50 |
| SamYaple | inc0: rhallisey ^^ http://paste.fedoraproject.org/323634/14556439 | 17:51 |
| sbezverk | sdake I will wait then.. thank you | 17:51 |
| sdake | https://review.openstack.org/#/c/280429/ needs another core review to review | 17:51 |
| vhosakot | how can I check the list of containers that kolla-build built ? is "docker images" the right command ? | 17:51 |
| SamYaple | vhosakot: yes | 17:52 |
| vhosakot | cool | 17:52 |
| SamYaple | also the output from the kolla-build command | 17:52 |
| SamYaple | it tells you what it built | 17:52 |
| vhosakot | yeah... at the end of the output, right ? | 17:52 |
| vhosakot | I see it | 17:52 |
| SamYaple | yup | 17:52 |
| sdake | SamYaple what is with this gate failure | 17:53 |
| sdake | http://logs.openstack.org/29/280429/2/check/gate-kolla-dsvm-deploy-centos-binary/621a359/console.html#_2016-02-16_15_59_21_169 | 17:53 |
| rhallisey | SamYaple, this is the named_volume patch? | 17:53 |
| sdake | yes but it is failing gate, don't +w just yet | 17:53 |
| rhallisey | I wanted to test it | 17:53 |
| rhallisey | still having issue with my setup tho | 17:53 |
| sdake | cool please do | 17:53 |
| vhosakot | I see there is no "kolla-build status" command | 17:53 |
| rhallisey | getting closer | 17:53 |
| rhallisey | I think someone had this issue earlier..: msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/neutron_dhcp_agent/start',),) | 17:54 |
| SamYaple | sdake: i prefer to say the gate is failing my patch | 17:54 |
| SamYaple | rhallisey: MountFlags=shared in your docker.service file | 17:54 |
| SamYaple | the restart docker | 17:55 |
| SamYaple | its unrelated. its for the thin contaienrs patch | 17:55 |
| rhallisey | SamYaple, did you put that in docs somewhere? /me looks through the q | 17:55 |
| sdake | have 1 hour meeting, bbiaf | 17:56 |
| SamYaple | rhallisey: sorry man. i have teh patch on my desktop at home | 17:56 |
| SamYaple | its not pushed | 17:56 |
| rhallisey | hehe | 17:56 |
| rhallisey | it's ok | 17:56 |
| SamYaple | i wrote it last night | 17:56 |
| SamYaple | i cant remote access my machine yet | 17:56 |
| SamYaple | i thought about it this morning :/ i promised it yesterday | 17:56 |
| sdake | rhallisey do this: | 17:56 |
| sdake | ur1.ca/oj0ii | 17:57 |
| sdake | should replace /usr/lib/systemd/system/docker.service | 17:58 |
| *** achanda has joined #kolla | 17:59 | |
| rhallisey | cool thx | 17:59 |
| sdake | oh meeting ins't until noon | 17:59 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 17:59 |
| *** cemason has quit IRC | 18:00 | |
| limamauricio | SamYaple: same error :/ | 18:00 |
| limamauricio | Error encountered during initialization of driver: RBDDriver. | 18:00 |
| *** cemason has joined #kolla | 18:01 | |
| limamauricio | SamYaple -> http://pastebin.com/tq7eJ6Vp | 18:01 |
| *** jtriley has quit IRC | 18:01 | |
| *** Marga__ has joined #kolla | 18:01 | |
| SamYaple | limamauricio: can you confirm the ceph files transfered in appropriatly? | 18:05 |
| limamauricio | yes, i can | 18:05 |
| *** ssurana1 has joined #kolla | 18:05 | |
| *** Marga_ has quit IRC | 18:05 | |
| *** achanda has quit IRC | 18:06 | |
| SamYaple | so they are in the container? | 18:06 |
| *** ssurana has quit IRC | 18:06 | |
| SamYaple | oh nvm limamauricio yea so its in there | 18:06 |
| SamYaple | but i bet its not the right file | 18:07 |
| SamYaple | let me check | 18:07 |
| *** subscope has joined #kolla | 18:07 | |
| limamauricio | Sam, the files are there | 18:07 |
| SamYaple | limamauricio: but cinder-backup keyring isnt... right? | 18:08 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Use "optional" for ceph confs https://review.openstack.org/280808 | 18:08 |
| SamYaple | check this patch out | 18:08 |
| limamauricio | ok | 18:08 |
| SamYaple | cinder and cinder-backup have different keyrings | 18:08 |
| SamYaple | that was only copying in cinder keyring, it needs cinder-abckup | 18:08 |
| limamauricio | oh | 18:09 |
| limamauricio | :) | 18:09 |
| limamauricio | I'll test now | 18:09 |
| *** Serlex has joined #kolla | 18:22 | |
| *** achanda has joined #kolla | 18:23 | |
| *** jtriley has joined #kolla | 18:25 | |
| *** jasonsb_ has quit IRC | 18:28 | |
| *** gfidente has quit IRC | 18:29 | |
| sdake | inc0 email away | 18:34 |
| inc0 | sdake, thanks | 18:35 |
| sdake | feel free to correct mistatements | 18:35 |
| openstackgerrit | Merged openstack/kolla-mesos: Add priviliged flag to nova-api https://review.openstack.org/280576 | 18:35 |
| sdake | that is how i interpreted thigns :) | 18:35 |
| openstackgerrit | Merged openstack/kolla-mesos: Fix glance registry conf path https://review.openstack.org/280600 | 18:35 |
| openstackgerrit | Merged openstack/kolla-mesos: Fix nova tasks https://review.openstack.org/280464 | 18:35 |
| *** jtriley has quit IRC | 18:35 | |
| *** jtriley has joined #kolla | 18:39 | |
| inc0 | sdake, you didnt call for vote for thin neutron containers:P | 18:39 |
| sdake | inc0 writing that email - emails take a long time to write ;) | 18:39 |
| limamauricio | SamYaple, same error on cinder_backup, but i created a volume successfully http://pastebin.com/YKmDU4fD | 18:39 |
| sdake | i wanted to keep the threads separate if that makes sense to you | 18:40 |
| sdake | There was not a majority approval for another proposal to backport thin containers for neutron which I will handle in a separate email. | 18:40 |
| inc0 | sure, no problem | 18:40 |
| limamauricio | maybe im forgetting something | 18:41 |
| sdake | i say right there its a seprate email ;) | 18:41 |
| *** gfidente has joined #kolla | 18:41 | |
| vhosakot | does anyone here successfully use Vagrant for day-to-day kolla dev work... just wanted to check if going the Vagrant route is good.... I;m seeing some issues realted to SSH key mis-match everyt ime I do "vagrant ssh" | 18:42 |
| sdake | vhosakot pbourke and mandre use it | 18:42 |
| SamYaple | limamauricio: ill have t otest now, nothing else major sticks out | 18:43 |
| SamYaple | limamauricio: you may try a new deploy | 18:43 |
| vhosakot | pbourke, mandre do you giys use the Vagrantfile at https://github.com/openstack/kolla/blob/master/dev/vagrant/Vagrantfile | 18:43 |
| vhosakot | guys* | 18:43 |
| vhosakot | I have used Vagrant before... but the SSH key mis-match issue I'm seeing is after I installed kolla, ran kolla-build, and logged out and tried to SSH in back | 18:44 |
| vhosakot | I dont want to debug non-kolla realted Vagrant issues :) | 18:44 |
| vhosakot | related* | 18:44 |
| SamYaple | sdake: successful vm launch in gate | 18:46 |
| SamYaple | after | 18:46 |
| SamYaple | Sam Yaple | 18:46 |
| SamYaple | after * https://review.openstack.org/#/c/280029/ | 18:46 |
| sdake | vhosakot mandre and pbourke are likely asleep or on a plane atm | 18:48 |
| inc0 | SamYaple, looks cool | 18:48 |
| vhosakot | oh ok.. cool.. np.. | 18:48 |
| vhosakot | will keep the team posted the moment I'm up with kolla | 18:49 |
| *** chandankumar has quit IRC | 18:49 | |
| vhosakot | sdake: do you think a "kolla-build status" is needed ? or, this is covered by "docker status | 18:49 |
| vhosakot | forgot to add ? at the end | 18:49 |
| sdake | SamYaple looking sweet:) | 18:50 |
| sdake | vhosakot please expand on what kolla-build status would do | 18:50 |
| openstackgerrit | Merged openstack/kolla: Reduce gate resource usage https://review.openstack.org/280029 | 18:51 |
| SamYaple | can someone give me access to a box with the toolbox sandbox issue? | 18:51 |
| *** ssurana1 has quit IRC | 18:52 | |
| *** athomas has quit IRC | 18:52 | |
| sdake | you mean sans your patch? | 18:53 |
| vhosakot | sdake: ok, "kolla-build" builds all the containers and fails few times for bad/flakey mirrors... I am thinking if there should be a kolla command that shows all the docker containers built... Since this info is in "docker images", a command like "kolla-build status" may not be needed... | 18:53 |
| SamYaple | sdake: to test a few things about the underlying issue | 18:53 |
| SamYaple | i j ust need a sysmtem that breaks | 18:53 |
| vhosakot | as a new user, I did not know how to find what kolla-build did | 18:54 |
| vhosakot | until I found "docker images" | 18:54 |
| vhosakot | found == used | 18:54 |
| sdake | SamYaple yup i can try that this afternoon - but be aware, its on my laptop :) | 18:54 |
| SamYaple | sdake: even better | 18:54 |
| sdake | i have meeting in 6 minutes | 18:54 |
| sdake | and meeting at 2pm | 18:54 |
| sdake | and kids at 3pm | 18:54 |
| SamYaple | vhosakot: so the builds are a one-off operation | 18:54 |
| sdake | i'll try to get it setup as soon as possible | 18:55 |
| vhosakot | sdake: ah... got it... not every day | 18:55 |
| SamYaple | kolla-build status wouldn't work because the images on your system dont neccesarily match the dockerfiles you want to build | 18:55 |
| vhosakot | SamYaple: because, the images depend on what registry the user is uwing ? | 18:55 |
| vhosakot | using* | 18:55 |
| SamYaple | no they depend on the Dockerfile content | 18:56 |
| sdake | SamYaple so basically i have between 1 and 2 to get you setup | 18:56 |
| sdake | or 2-:245 | 18:56 |
| SamYaple | i just need someone with a failing system | 18:56 |
| vhosakot | got it.. cool | 18:56 |
| SamYaple | i cant reproduce | 18:56 |
| sdake | it may take a couple hours to sort how to get you an ip into this vm | 18:56 |
| sdake | atm the vm is in nat mode rather then getting an ip from my network | 18:56 |
| sdake | i'll sort it out, but need couple hours because of meetings etc | 18:57 |
| sdake | i can definately give my laptop a static ip | 18:57 |
| *** athomas has joined #kolla | 18:58 | |
| *** gfidente has quit IRC | 19:00 | |
| *** haplo37 has quit IRC | 19:03 | |
| SamYaple | can anyone with the kolla_toolbox issue give me the output of `docker inspect kolla_toolbox` | 19:05 |
| Serlex | Hi SamYaple, is that where toolbox container exits? | 19:07 |
| SamYaple | Serlex: if you are asking if toolbox conatiner == kolla_toolbox, the answer is yes | 19:08 |
| Serlex | ok running now | 19:08 |
| *** dwalsh has joined #kolla | 19:08 | |
| SamYaple | Serlex: its important that your deploy has broken and the kolla_toolbox container is in the "Created" state | 19:09 |
| Serlex | ahh ok, not me then | 19:09 |
| *** jtriley has quit IRC | 19:11 | |
| *** dwalsh_ has joined #kolla | 19:15 | |
| *** dwalsh has quit IRC | 19:16 | |
| *** ssurana has joined #kolla | 19:18 | |
| *** dwalsh_ has quit IRC | 19:19 | |
| *** jtriley has joined #kolla | 19:19 | |
| *** mbound has joined #kolla | 19:23 | |
| sbezverk | SamYaple Do you still need output you requested earlier? | 19:24 |
| *** unicell has joined #kolla | 19:26 | |
| SamYaple | sbezverk: from a kolla_toolbox container in the Created state, yes | 19:27 |
| sbezverk | SamYaple, two minutes then I will bring up a server with broken container | 19:27 |
| SamYaple | w00t | 19:28 |
| SamYaple | mind running a few tests for me too? | 19:28 |
| sbezverk | hmmm, it is in exited state, would it still be ok? | 19:29 |
| SamYaple | probably not, but why is it in an exited state.... | 19:30 |
| SamYaple | it should never be there | 19:30 |
| Serlex | devaju | 19:30 |
| Serlex | dejavu | 19:30 |
| sbezverk | SamYaple check this http://paste.fedoraproject.org/323714/51020145 | 19:30 |
| sbezverk | 4f9cfa12ddf2 192.168.80.240:4000/kollaglue/centos-source-kolla-toolbox:2.0.0 "/bin/sleep infinity" 3 hours ago Exited (-1) 2 hours ago kolla_toolbox | 19:31 |
| *** limamauricio has quit IRC | 19:31 | |
| SamYaple | sbezverk: looks like you rebooted or something | 19:31 |
| SamYaple | whats `docker logs kolla_toolbox` say? | 19:31 |
| *** dwalsh_ has joined #kolla | 19:32 | |
| Serlex | its the same state that I have | 19:32 |
| SamYaple | surely thats after a reboot Serlex | 19:32 |
| SamYaple | it _cant_ deploy in that state | 19:32 |
| SamYaple | all it does is run /bin/sleep | 19:32 |
| sbezverk | SamYaple, that is correct, the server was rebooted | 19:33 |
| SamYaple | sbezverk: if you say this server gets the toolbox issue, would you be opposed to wiping kolla from it and rekicking kolla? | 19:34 |
| SamYaple | i know you just got it setup so if you want to play with it i understand | 19:34 |
| SamYaple | i just cant replicate this issue anywhere | 19:34 |
| *** jtriley has quit IRC | 19:35 | |
| sbezverk | SamYaple, anything for you to get this issue resolved.. I am opened for a debugging session where I can share access to my server via web conferencing. | 19:35 |
| vhosakot | "curl -sSL https://get.docker.io | bash" in the quickstart guide installs docker version 1.10.1.. Is docker 1.10 ok ? | 19:35 |
| sbezverk | vhosakot yes | 19:36 |
| vhosakot | cool | 19:36 |
| SamYaple | vhosakot: 1.10+ | 19:37 |
| SamYaple | or rather >=1.10 | 19:37 |
| vhosakot | so, I don't have to run "sudo apt-get install docker-engine=1.8.2-0~trusty" to downgrade docker as docker 1.10 is good... am I right ? | 19:37 |
| SamYaple | vhosakot: correct | 19:38 |
| SamYaple | 1.8.2 is for liberty | 19:38 |
| vhosakot | according to the quickstart guide, running "sudo apt-get install docker-engine=1.8.2-0~trusty" after "curl -sSL https://get.docker.io | bash" downgrades docker from 1.10 to 1.8.2...this is needed for Liberty... | 19:38 |
| SamYaple | vhosakot: are you running liberty? | 19:38 |
| SamYaple | master==mitaka | 19:38 |
| vhosakot | no, following quickstart... | 19:39 |
| vhosakot | I want latest | 19:39 |
| vhosakot | MItaka | 19:39 |
| vhosakot | I will not run "sudo apt-get install docker-engine=1.8.2-0~trusty" and use docker 1.10 on Mitaka... | 19:39 |
| vhosakot | thanks! :) | 19:39 |
| SamYaple | thats it! | 19:40 |
| vhosakot | :) | 19:40 |
| sbezverk | b54f552c9123 192.168.80.240:4000/kollaglue/centos-source-kolla-toolbox:2.0.0 "/bin/sleep infinity" 13 seconds ago Created kolla_toolbox | 19:41 |
| SamYaple | sbezverk: that looks like the breakage i need to test! | 19:42 |
| sbezverk | http://paste.fedoraproject.org/323725/45565172 | 19:42 |
| sbezverk | SamYaple anything else tp collect? | 19:42 |
| SamYaple | sbezverk: sure enough! | 19:42 |
| SamYaple | theres the issue | 19:43 |
| Serlex | ? | 19:43 |
| SamYaple | Serlex: i see the issue i predicted in the output sbezverk provided | 19:44 |
| SamYaple | look at mounts between his and mine http://paste.openstack.org/show/487163/ | 19:45 |
| SamYaple | err wrong post | 19:45 |
| *** nihilifer_mobile has joined #kolla | 19:45 | |
| SamYaple | but the Mounts are different | 19:45 |
| Serlex | awesome SamYaple, you the man | 19:45 |
| SamYaple | nah it doesnt help to predict a problem i cant fi | 19:46 |
| SamYaple | x* | 19:46 |
| Serlex | I should really read what the toolbox container is but the fact my container has exited is a concern/ | 19:46 |
| Serlex | or not? | 19:46 |
| SamYaple | Serlex: you reboot though right? | 19:46 |
| Serlex | yes | 19:46 |
| SamYaple | yea thats why | 19:47 |
| SamYaple | rebooting the containers don't start back up in the correct order | 19:47 |
| Serlex | that means nothing to me, all my other containers are running after reboot | 19:47 |
| SamYaple | we talked about that as an action point at the midcycle | 19:47 |
| Serlex | oh ok | 19:47 |
| SamYaple | you should be able to start the kolla_toolbox container | 19:47 |
| SamYaple | `docker start kolla_toolbox` | 19:47 |
| SamYaple | worst case you can remove and recreate | 19:47 |
| SamYaple | `docker rm -f kolla_toolbox` then just rerun playbooks | 19:47 |
| sbezverk | SamYaple, do you mean right now or after it is fixed? | 19:48 |
| SamYaple | sbezverk: no not for you that was for Serlex | 19:48 |
| SamYaple | i have a patch in the queue to work around your issue | 19:48 |
| sbezverk | ok | 19:48 |
| SamYaple | but i still need to _fix_ it, but i need access to a box to do so | 19:48 |
| SamYaple | at least to investigate further | 19:48 |
| sbezverk | SamYaple, and Webex to me server does not work for you? | 19:49 |
| *** haplo37 has joined #kolla | 19:49 | |
| Serlex | woop I have a horizon | 19:50 |
| Serlex | thank god after hours of dicking around | 19:50 |
| SamYaple | sbezverk: hmm webex work on linux? | 19:51 |
| SamYaple | its been a while since ive used it | 19:52 |
| *** nihilifer_mobile has quit IRC | 19:52 | |
| SamYaple | sbezverk: that might work | 19:52 |
| vhosakot | which docker registry version should I use ? v1 or v2 ? Should I run "docker run -d -p 4000:5000 --restart=always --name registry registry:2" per the quickstart | 19:53 |
| Serlex | 2.3 | 19:53 |
| sbezverk | SamYaple I know it works on Mac, never tried to join from linux ws though | 19:53 |
| vhosakot | so, I need to "docker run -d -p 4000:5000 --restart=always --name registry registry:2" right ? | 19:54 |
| Serlex | add .3 | 19:55 |
| SamYaple | vhosakot: that would start a v2 registry listening on port 4000 yes | 19:55 |
| vhosakot | ok.. | 19:55 |
| Serlex | or whatever SamYaple says | 19:55 |
| Serlex | I ran 2.3 | 19:55 |
| vhosakot | I will run "docker run -d -p 4000:5000 --restart=always --name registry registry:2.3" then... thanks! | 19:55 |
| SamYaple | Serlex: vhosakot you can do registry:2.3 if you want registry:2 is the same thing | 19:55 |
| vhosakot | cool! | 19:55 |
| SamYaple | see the current tags at https://hub.docker.com/_/registry/ | 19:55 |
| vhosakot | yep | 19:56 |
| vhosakot | thanks! | 19:56 |
| SamYaple | with registry:2 when, say 2.4.1 comes out it will retag as :2 | 19:56 |
| *** Marga__ has quit IRC | 19:58 | |
| *** Marga_ has joined #kolla | 19:58 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 19:59 |
| sdake | SamYaple webex will not work on linux | 19:59 |
| SamYaple | sdake: webex says it will | 19:59 |
| sdake | serlex nice! :) | 19:59 |
| sbezverk | SamYaple, so do you want to give it a try with webex? | 20:01 |
| SamYaple | sbezverk: yea lets give it a try | 20:02 |
| sdake | SamYaple re sbezverk wrong bind order, you think that is caused by the reboot? | 20:02 |
| SamYaple | sdake: no this is back to the sandbox issue | 20:03 |
| SamYaple | still trying to troubleshoot it | 20:03 |
| sdake | yes i thought I said that hours ago :) | 20:03 |
| sdake | maybe its a different sandbox issue | 20:03 |
| SamYaple | no he is helping me trouble shoot it | 20:03 |
| sdake | keep in mind sbezverk hand applied the upstream patch | 20:03 |
| sdake | rather then git cherrypicking it | 20:03 |
| SamYaple | again. this is not whats going on | 20:03 |
| sdake | ok | 20:04 |
| SamYaple | he is helping me debug not his issue but _THE_ issue | 20:04 |
| sdake | well I dont have a problem after you patch | 20:04 |
| SamYaple | since i cant reproduce | 20:04 |
| SamYaple | without the patch | 20:04 |
| SamYaple | its fine sdake | 20:04 |
| sdake | i see, you want to get to root cause because you think its still there | 20:05 |
| *** aginwala has quit IRC | 20:05 | |
| SamYaple | well the root cause is still there, we are just mostly working around it | 20:05 |
| sdake | what I dont get is why sbezverk has two /dev mounts in his host config, since your patch only has one | 20:05 |
| sdake | i think in _HIS_ case, he is not running the patch ;-) | 20:06 |
| sbezverk | https://cisco.webex.com/join/sbezverk | 20:06 |
| *** aginwala has joined #kolla | 20:08 | |
| SamYaple | sbezverk: its crashing for me :/ | 20:09 |
| SamYaple | sdake: i know he isnt, i didnt want him to. i asked him not too | 20:09 |
| SamYaple | sdake: its covered | 20:09 |
| sdake | sounds good | 20:09 |
| sdake | ya webex is bust on linux | 20:10 |
| SamYaple | ive done it once or twice | 20:10 |
| SamYaple | but java always breaks | 20:10 |
| sdake | you can use a different webex tool | 20:10 |
| sdake | i forget what its called | 20:10 |
| sdake | why dont you try google hangouts | 20:10 |
| sdake | webex is dependent on a componet of java that open source jdk has deprecated | 20:10 |
| sdake | something related to gcc | 20:10 |
| sdake | you can theoretically get it to work | 20:10 |
| SamYaple | im running oracle java | 20:11 |
| sdake | but its a real pain in the ass | 20:11 |
| sdake | i was never actually able to get jwebex to work on linux | 20:11 |
| sdake | and I tried and tried | 20:11 |
| sdake | SamYaple one thing you cand o is install a 32 bit vm | 20:11 |
| sdake | and run webex in that | 20:11 |
| SamYaple | i did at rax, but its PITA | 20:11 |
| sdake | that will work from my experience | 20:11 |
| *** rajathagasthya has joined #kolla | 20:12 | |
| SamYaple | sorry sbezverk i think ill have to pass on the webex for now | 20:12 |
| SamYaple | before the beginning of next week ill have my private openstack cluster running so i should be able to reproduce this in a vm there | 20:13 |
| sbezverk | SamYaple sure, I understand, just was trying to be helpful | 20:13 |
| *** cemason has quit IRC | 20:13 | |
| SamYaple | sbezverk: you were very much so. thanks! | 20:13 |
| SamYaple | sdake: looging patch turned green | 20:14 |
| SamYaple | rhallisey: did you get it working? | 20:14 |
| *** cemason has joined #kolla | 20:15 | |
| *** sdake_ has joined #kolla | 20:15 | |
| rhallisey | SamYaple, not yet. kolla-toolbox error. APIError(HTTPError(u'500 Server Error: Internal Server Error | 20:16 |
| *** dwalsh_ has quit IRC | 20:16 | |
| rhallisey | I had to work on something else so I haven't touched it since I got that | 20:16 |
| rhallisey | I think someone has had that issue before | 20:17 |
| *** dwalsh has joined #kolla | 20:17 | |
| *** sdake has quit IRC | 20:17 | |
| SamYaple | rhallisey: thats the issue the patch is solving, no? | 20:17 |
| SamYaple | what state is the container in? | 20:17 |
| SamYaple | kolla_toolbox that is | 20:17 |
| rhallisey | created status | 20:18 |
| rhallisey | stuck there | 20:18 |
| SamYaple | yea you either didnt clean the environemnt or you didnt rebuild your containers with the patch | 20:18 |
| SamYaple | i would think maybe you didnt clean the failed kolla_ansible container | 20:19 |
| vhosakot | is there a recommended value for the argument --retries for kolla-build.... or, the default number of retries (3) is used... how many retries does everyone do ? | 20:20 |
| rhallisey | ya I'll get back to it later | 20:20 |
| rhallisey | SamYaple, thanks | 20:20 |
| SamYaple | rhallisey: yup thanks for testing | 20:21 |
| SamYaple | vhosakot: for the build, I actually do 0 | 20:22 |
| SamYaple | i dont want it to retry if there is an issue | 20:22 |
| SamYaple | in the gate however we are more lienient | 20:22 |
| SamYaple | we want it to retry | 20:22 |
| SamYaple | thats why the default is 3 | 20:22 |
| vhosakot | cool | 20:22 |
| SamYaple | it really _shouldnt_ be failing, that indicates a problem | 20:22 |
| vhosakot | right | 20:22 |
| vhosakot | I will do with 0 then | 20:22 |
| vhosakot | "kolla-build --retries 0 " is this the right way ? | 20:23 |
| vhosakot | ? | 20:25 |
| SamYaple | vhosakot: yes | 20:25 |
| vhosakot | cool! | 20:25 |
| SamYaple | or you can set it in /etc/kolla/kolla-build.conf to make it more permanant | 20:25 |
| sdake_ | SamYaple ssh samyaple@broked.selfip.net | 20:25 |
| guillaume_ | hello guys, is the kolla-toolbox bug fixed? | 20:25 |
| sdake_ | lets see if that par tworks | 20:25 |
| sdake_ | guillaume_ yes fixed not merged | 20:25 |
| SamYaple | sdake_: still have my pubkey? | 20:25 |
| sdake_ | SamYaple yes | 20:26 |
| guillaume_ | when it will be merged? | 20:26 |
| sdake_ | its in my gmail ;) | 20:26 |
| sdake_ | I have it forever | 20:26 |
| sdake_ | if you sen dto gmail, i can find it | 20:26 |
| sdake_ | if you send to cisco email, well, outlook blows ;) | 20:26 |
| sdake_ | see ify ou can login | 20:26 |
| SamYaple | sdake_: refused | 20:26 |
| SamYaple | guillaume_: it will be merged when another core approves it | 20:27 |
| sdake_ | samyaple ocnnection refused or login refused? | 20:27 |
| SamYaple | guillaume_: https://review.openstack.org/#/c/280429/ | 20:27 |
| SamYaple | sdake_: connection | 20:27 |
| SamYaple | port num? | 20:27 |
| guillaume_ | SamYaple and sdake: thanks for you info. I will modified it by hand for today ? | 20:27 |
| *** dwalsh has quit IRC | 20:27 | |
| SamYaple | guillaume_: sounds good. that patch should be merged tomorrow | 20:28 |
| *** sdake_ has quit IRC | 20:32 | |
| sbezverk | Anybody managed to get cinder volume container running? | 20:32 |
| *** sdake has joined #kolla | 20:32 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Move haproxy socket to volume https://review.openstack.org/280918 | 20:33 |
| SamYaple | sbezverk: with ceph, yes | 20:33 |
| SamYaple | sbezverk: cinder+nfs patch is in the queue | 20:33 |
| SamYaple | cinder+lvm is not actively being wokred on | 20:34 |
| sbezverk | SamYaple how about LVM? | 20:34 |
| SamYaple | i had a PoC many moons ago, its technically possible but no patches are up for it | 20:34 |
| sbezverk | SamYaple I see.. anything I could do to make it happen? | 20:35 |
| sbezverk | will it make to official Mitaka? | 20:35 |
| sdake | SamYaple try again plz | 20:35 |
| sbezverk | after all LVM is still a reference confo | 20:35 |
| sbezverk | configuration.. | 20:35 |
| sdake | to ssh | 20:36 |
| SamYaple | sbezverk: doubtful, unless someone picks it up | 20:36 |
| SamYaple | it requires us ot containerize iscsi and no one has done that yet | 20:36 |
| SamYaple | i am too overlaoded to start that again | 20:36 |
| SamYaple | sdake: im in. dont know how to elevate | 20:38 |
| sdake | your on a mac | 20:38 |
| sdake | you need to ssh into my vm | 20:38 |
| sdake | i am working on that next | 20:38 |
| SamYaple | oh this is mac ok | 20:38 |
| sbezverk | SamYaple that puts a big red stop on my activities, all our test beds are using AIO and LVM as a storage.. | 20:38 |
| SamYaple | sbezverk: thats disappointing. I wish I could give you better news. I would say its about a week of my time dedicated to get it setup | 20:39 |
| SamYaple | i wish I could give you better news, but I dont have a week to give before Mar 3rd | 20:39 |
| SamYaple | and Mar3rd is mitaka-3 tag | 20:39 |
| *** cemason has quit IRC | 20:40 | |
| sbezverk | SamYaple I understand | 20:41 |
| *** cemason has joined #kolla | 20:41 | |
| *** dwalsh has joined #kolla | 20:41 | |
| *** subscope has quit IRC | 20:42 | |
| *** aginwala has quit IRC | 20:46 | |
| SamYaple | ill tell you what sbezverk give me until tomorrow and ill see if I can get the work started for others to finish | 20:46 |
| *** aginwala has joined #kolla | 20:49 | |
| vhosakot | I see 112 Dockerfiles in kolla/docker.... does this mean kolla-build builds all of these 112 containers ? | 20:53 |
| *** dwalsh has quit IRC | 20:54 | |
| SamYaple | vhosakot: it would build 112 images in total if run in full, it only produces ~60 or so services and of those kolla-ansible only deploys ~40 | 20:55 |
| vhosakot | "if run in full" is this an option to kolla-build to build all the 112 containers ? or "if run in full" means kolla-build + kolla-ansible | 20:58 |
| SamYaple | kolla-build with no paramaters builds 112 images | 20:59 |
| vhosakot | ok... that is what I'm doing per the quickstart... kolla-build | 21:00 |
| SamYaple | yup | 21:00 |
| vhosakot | cool | 21:02 |
| daneyon | SamYaple do you have a moment for me to bounce a question off you? | 21:07 |
| SamYaple | daneyon: for you, always | 21:07 |
| daneyon | thx | 21:07 |
| d_code | has anyone done PCI passthrough with Kolla? | 21:07 |
| SamYaple | d_code: me! but it requires kolla codebase hacking | 21:08 |
| SamYaple | d_code: cant do it with native kolla last i checked | 21:08 |
| daneyon | SamYaple I am trying to get Ansible to work with CoreOS | 21:08 |
| d_code | I can hackz | 21:08 |
| daneyon | SamYaple the 1st issue is that coreos does not include a python interpreter | 21:08 |
| d_code | the pci passthrough wiki page on openstack.org seems like it’s old…but it suggests that it’s built into nova now | 21:08 |
| d_code | is the hack to enable the API endpoints? | 21:08 |
| d_code | I saw the github repo for that, last commit was 3 years ago | 21:09 |
| daneyon | SamYaple i have worked around it in my lab by installing a script on coreos that pulls the python src and does some magic voodoo | 21:09 |
| daneyon | SamYaple I want to get away from that and run python in a container | 21:09 |
| d_code | SamYaple: just nobody wants to merge it in, or what? | 21:09 |
| SamYaple | daneyon: so you want to run python in a container on core os? | 21:09 |
| daneyon | so everytime i issue python this happens: docker run --interactive --rm --net=none python:2.7-slim python | 21:09 |
| daneyon | SamYaple yes | 21:10 |
| SamYaple | and have ansible commands run _in_ that container? | 21:10 |
| daneyon | SamYaple the above works just fine when I create an alias to the docker run command for python | 21:10 |
| daneyon | SamYaple but when I run Ansible it runs the shell in non-interactive mode and does not respect the alias that I put in /etc/environment | 21:11 |
| SamYaple | yea its not going to. lots of stuff in ansible is hardcoded as /usr/bin/python | 21:12 |
| SamYaple | im not sure there is a way to do what you want to be honest | 21:12 |
| daneyon | SamYaple with the alias set in /etc/environment I can type python --version and docker runs the container and passes the --version command and runs it | 21:12 |
| SamYaple | i did alot of this with the kolla-ansible/kolla-toolbox container | 21:12 |
| *** jtriley has joined #kolla | 21:13 | |
| SamYaple | yea but ansible is going to talk directly to the interpreter | 21:13 |
| daneyon | SamYaple even when I changes the ansible_python_interpreter: "python" it does not work. From my understanding it's b/c ansible uses /bin/sh which does not support aliases | 21:14 |
| SamYaple | im not sure thats the correct reason daneyon | 21:14 |
| SamYaple | what yo uneed to test out is if you can have a test.py file that will run python code in a container | 21:15 |
| SamYaple | i couldnt figure out a way | 21:15 |
| daneyon | SamYaple: http://unix.stackexchange.com/questions/52471/how-to-add-system-alias | 21:15 |
| vhosakot | SamYaple: kolla-build built 87 images and 25 images failed to build. Here is the output http://paste.openstack.org/show/487173/. Should I manually rebuild each of the failed 25 images ? | 21:16 |
| daneyon | SamYaple check out the answer with the green checkmark | 21:16 |
| SamYaple | daneyon: but this isn't about aliases | 21:17 |
| SamYaple | ansible actually spits out a file with the shebang line that says #!/usr/bin/python followed by python code | 21:17 |
| *** cemason has quit IRC | 21:17 | |
| SamYaple | you need to get a file like that to run in a docker container without hcanging the file, and I couldnt figure out how to do that | 21:17 |
| SamYaple | the only other answer was to hack on ansible | 21:17 |
| SamYaple | you can absolute hack on ansible to get that done to, but its a much bigger commitment | 21:18 |
| daneyon | SamYaple when I change ansible_python_interpreter: "python" ansible uses python instead of /usr/bin/python. I alias alias python="docker run --interactive --rm --net=none python:2.7-slim python"' | 21:18 |
| daneyon | so everytime i python --version, the docker run command is executed | 21:19 |
| SamYaple | ill put together a pastebin daneyon so we are on the same page | 21:19 |
| daneyon | SamYaple if you have time to jump on a webex I could share my screen so you see what i'm up to | 21:20 |
| *** cemason has joined #kolla | 21:20 | |
| *** Marga_ has quit IRC | 21:21 | |
| SamYaple | daneyon: i cant webex from linux :/ its borked | 21:21 |
| SamYaple | daneyon: http://paste.openstack.org/show/487175/ | 21:21 |
| *** Marga_ has joined #kolla | 21:21 | |
| SamYaple | without modifying test.py, you need to make that run inside a container, how do you do it? | 21:22 |
| *** rajathagasthya has quit IRC | 21:22 | |
| larsks | SamYaple: daneyon : https://tmate.io/ is nice for sharing terminal sessions. It's just tmux w/ remote support. | 21:22 |
| SamYaple | i like tmate | 21:22 |
| SamYaple | larsks: this thought problem may also interest you | 21:23 |
| SamYaple | see the paste above? and the following question | 21:23 |
| *** sdake_ has joined #kolla | 21:24 | |
| SamYaple | larsks: daneyon in this problem, you can modify /usr/bin/python, but not test.py itself | 21:24 |
| larsks | SamYaple: the goal is run with a different python? | 21:25 |
| *** Marga_ has quit IRC | 21:25 | |
| *** sdake has quit IRC | 21:25 | |
| SamYaple | larsks: the goal is to run with the python interpreter inside a docker container | 21:25 |
| daneyon | SamYaple larsks check this out: https://gist.github.com/danehans/1597a3743cd4848e57cd | 21:25 |
| larsks | I wonder if you can just bind-mount a script that would Do the Right Thing: mount -o bind /path/to/wrapper /usr/bin/python | 21:25 |
| SamYaple | daneyon: but thats an alias, its not the same thing. | 21:26 |
| vhosakot | SamYaple: so, should I rebuild the 25 failed images ? | 21:26 |
| SamYaple | you aren't running a script | 21:26 |
| SamYaple | vhosakot: im not sure why they failed, but you can try | 21:26 |
| larsks | daneyon: aliases are really only good for interactive shells, right? Because anything that calls "exec" doesn't care about your alias. | 21:26 |
| daneyon | ^ works as expected when I ssh to my coreos box, but not with ansible, even after changing ansible_python_interpreter: "python" | 21:26 |
| SamYaple | ok but even if it does work, its not going to execute the python instructions in the file | 21:27 |
| vhosakot | SamYaple: where can I find the logs why those 25 containers failed ? /var/log/upstart/docker.log has just "ERRO[3732] Handler for GET /v1.22/containers/nova-compute/json returned error: No such container: nova-compute" | 21:27 |
| SamYaple | it will execute everything on the command line | 21:27 |
| SamYaple | vhosakot: when you built the containers, it spits out the failure message | 21:27 |
| vhosakot | SamYaple: no logs are written by kolla-build ? | 21:27 |
| daneyon | According to http://unix.stackexchange.com/questions/52471/how-to-add-system-alias it's b/c the alias is not used when the shell is in non interactive mode | 21:27 |
| vhosakot | right.. I saw that | 21:27 |
| vhosakot | ok | 21:27 |
| d_code | docker run --interactive --rm --net=none python:2.7-slim python -c "$(cat test.py)" | 21:28 |
| d_code | is that cheating? | 21:28 |
| *** Marga_ has joined #kolla | 21:28 | |
| SamYaple | daneyon: i understand your point, what im saying is if aliases worked like you are asking them to it _still_ wouldnt work like you want because ansible doesnt stream files like that | 21:28 |
| SamYaple | d_code: actually no | 21:28 |
| d_code | :) | 21:28 |
| SamYaple | but getting that to work with ansible is the issue | 21:28 |
| larsks | SamYaple: In this situation, does a /usr/bin/python already exist and the goal is get *everything* to run inside a container? Or just specific things, like ansible modules? | 21:28 |
| SamYaple | larsks: /usr/bin/python does not exist | 21:29 |
| larsks | SamYaple: Because daneyon is correct that you can tell ansible which python interpreter to use and it will use that instead of /usr/bin/python. | 21:29 |
| daneyon | larsks that's correct... the alias works great from an interactive shell, but not from Ansible | 21:29 |
| SamYaple | but will that launch the code from the test.py file? | 21:29 |
| vhosakot | SamYaple: I scrolled up and see this... ERROR:kolla.cmd.build:nova-compute:Parent image error'd with message "error" | 21:29 |
| SamYaple | mind you, without modifying the test.py file larsks it needs to send it to an interpreter in the docker container | 21:29 |
| larsks | SamYaple: I don't know, it's not clear to me if test.py is something hypothetical, or an ansible module, or... | 21:30 |
| SamYaple | its what the ansible module would look like | 21:30 |
| daneyon | SamYaple larsks I am basicly spawning the python container everytime I issue a python command from the shell | 21:30 |
| SamYaple | python from the shell is different than python from a file | 21:30 |
| SamYaple | ansible streams it to /usr/bin/python (by default) you cant stream into a docker container like that | 21:30 |
| larsks | SamYaple: So in that case, setting ansible_python_interpreter should work, right? That could be a path to a wrapper script that runs python in a container... | 21:31 |
| SamYaple | it would not, because ansible streams into interpretur | 21:31 |
| SamYaple | like `cat test.py | /usr/bin/python` | 21:31 |
| daneyon | SamYaple larsks maybe another option would be to set an env that Ansible uses for all the playbook runs. Something like export PYTHON="docker run --interactive --rm --net=none python:2.7-slim python" | 21:32 |
| larsks | Can you not stream stdin to a docker container? | 21:32 |
| larsks | I thought that worked just fine. | 21:32 |
| SamYaple | not that I am aware of | 21:32 |
| SamYaple | oh well wait | 21:32 |
| daneyon | SamYaple larsks then set the ansible_python_interpreter to $PYTHON | 21:32 |
| SamYaple | with interactive yea | 21:32 |
| SamYaple | i guess you can larsks | 21:32 |
| larsks | SamYaple: e.g.: echo hello | docker run -i fedora sed s/hello/goodbye/ | 21:32 |
| larsks | Yeah. | 21:32 |
| SamYaple | yea i just tested too | 21:32 |
| SamYaple | you cant without -i | 21:32 |
| SamYaple | but that makes sense | 21:33 |
| larsks | So that seems like you've got everything you need. | 21:33 |
| SamYaple | ok daneyon give me a moment to test this | 21:33 |
| *** Marga_ has quit IRC | 21:33 | |
| vhosakot | SamYaple: I scrolled up and see this... ERROR:kolla.cmd.build:nova-compute:Parent image error'd with message "error" .... Is this because "FROM {{ namespace }}/{{ image_prefix }}nova-base:{{ tag }}" in nova-compute's Dockerfile is incorrect | 21:35 |
| *** dims has quit IRC | 21:37 | |
| *** JoseMello has quit IRC | 21:38 | |
| d_code | SamYaple: so…do you have any pointers on PCI Passthrough? is it not built into Nova as noted here: https://wiki.openstack.org/wiki/Pci_passthrough | 21:40 |
| d_code | there’s nothing on that page that says how old it is | 21:41 |
| vhosakot | I'm building nova-compute individually | 21:42 |
| SamYaple | daneyon: progress, working through some bugs | 21:42 |
| daneyon | SamYaple :-) | 21:42 |
| *** macsz has joined #kolla | 21:43 | |
| SamYaple | daneyon: wow. | 21:43 |
| SamYaple | i marked this path out a long time ago due to us not using interactive | 21:44 |
| SamYaple | alright so its not pretty | 21:44 |
| SamYaple | you have to bind in ~/.ansible/ into your container | 21:44 |
| SamYaple | so if you are root that means | 21:44 |
| SamYaple | -v /root/.ansible/:/root/.ansible/ | 21:44 |
| daneyon | SamYaple let me try | 21:45 |
| SamYaple | hold on ive got to post you more things | 21:46 |
| *** jtriley has quit IRC | 21:46 | |
| SamYaple | root@y-work-laptop01:~# cat inv | 21:46 |
| SamYaple | localhost ansible_python_interpreter="docker run -u root -v /root/.ansible/tmp/:/root/.ansible/tmp/ --interactive --net=none registry:8182/kollaglue/ubuntu-source-nova-api:2.0.0 python" | 21:46 |
| SamYaple | you need that in your inventory file | 21:46 |
| SamYaple | now the fun part is you cant use a pure python container | 21:46 |
| SamYaple | you have to install python in, say, and ubuntu container | 21:47 |
| SamYaple | a pure python container is missing things that youll need | 21:47 |
| *** dwalsh has joined #kolla | 21:47 | |
| SamYaple | hence why im running with a kolla image up there | 21:47 |
| vhosakot | SamYaple: is running kolla-build multiple times a good idea to rebuild failed images ? | 21:47 |
| SamYaple | vhosakot: it depends on the failure, it wont hurt | 21:47 |
| vhosakot | ok | 21:48 |
| vhosakot | SamYaple: Any idea why nova-compute failed with the "Parent image error'd with message "error"" error... I dont see any other error anywhere | 21:48 |
| larsks | SamYaple: daneyon : this seems to work for me: https://gist.github.com/larsks/c1eefeee8f69e77c8849 | 21:48 |
| daneyon | SamYaple and larsks let me try both solutions and I'll let you know how it goes. | 21:49 |
| larsks | daneyon: looks like that is basically the same thing. SamYaple is just including the docker invocation directly. | 21:50 |
| larsks | I missed SamYaple 's message up there... | 21:50 |
| *** dims has joined #kolla | 21:50 | |
| SamYaple | larsks: youd actually want to do `cat /proc/$$/fd/0 | docker run --interactive --net=none python:2.7-slim python` | 21:51 |
| *** banix has joined #kolla | 21:51 | |
| SamYaple | otherwise blobs will get parsed by bash i think | 21:51 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/280943 | 21:51 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-mesos: Updated from global requirements https://review.openstack.org/280944 | 21:51 |
| *** agasthya has joined #kolla | 21:51 | |
| larsks | SamYaple: I don't think bash will do anything horrific with stdin? I could be wrong. | 21:51 |
| larsks | I only tested it with '-m ping'... | 21:51 |
| SamYaple | it might if you do $@ | 21:51 |
| SamYaple | thats bash processing it | 21:52 |
| SamYaple | maybe idk | 21:52 |
| *** agasthya has quit IRC | 21:52 | |
| larsks | But that's command line arguments, not stdin... | 21:52 |
| SamYaple | oh yea | 21:52 |
| SamYaple | nvm | 21:52 |
| SamYaple | long day | 21:52 |
| SamYaple | im actually piping stdin for the script to python directly | 21:53 |
| SamYaple | so for you python wrapper you could do `echo something.py | python-wrapper` and it would not work, yes? | 21:53 |
| *** jtriley has joined #kolla | 21:53 | |
| *** rhallisey has quit IRC | 21:55 | |
| openstackgerrit | Merged openstack/kolla: Add manila perms to /var/cache/manila https://review.openstack.org/280306 | 21:58 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/280943 | 21:59 |
| vhosakot | SamYaple: build nova-compute alone worked | 21:59 |
| vhosakot | building* | 21:59 |
| SamYaple | vhosakot: you can try reducing the number of build threads | 21:59 |
| SamYaple | but without logs of why it failed i cant help | 22:00 |
| vhosakot | good point... may be too many threads... will tune it | 22:00 |
| SamYaple | larsks: daneyon if you set your ansible.cnf to have "pipelining=true" you dont need to do that bind mount! | 22:01 |
| vhosakot | when nova-compute failed to build, I scrolled up, and didn't see any logs other than "Parent image error'd with message "error"" | 22:01 |
| larsks | SamYaple: I didn't need a bind mount anyway...because I was explicitly setting ansible_interpreter. | 22:01 |
| SamYaple | larsks: you did | 22:01 |
| SamYaple | because by default it doesn't pipeline | 22:01 |
| SamYaple | it sends file to ~/.ansible/tmp | 22:02 |
| larsks | Actually, no, I didn't! | 22:02 |
| larsks | That worked without any bind mount magic. | 22:02 |
| SamYaple | you must be using pipelining then | 22:02 |
| SamYaple | otehr wise it literally ships a file over for your interpretrt to run | 22:02 |
| SamYaple | you should check your results | 22:02 |
| larsks | NNNNnnnnoooo...it just ran "sh -c /usr/local/bin/python-wrapper /home/centos/.ansible/tmp/...." | 22:03 |
| larsks | Yeah, exactly. | 22:03 |
| larsks | It runs that file using the value of ansible_python_interpreter. | 22:03 |
| *** salv-orlando has joined #kolla | 22:03 | |
| SamYaple | hmmm | 22:03 |
| larsks | Oh wait, I think we are talking about different bind mounts. | 22:04 |
| larsks | I thought you were talking about a bind mount on /usr/bin/python? But if you mean the one $HOME, then yeah, you're write, I did need trhat. | 22:04 |
| larsks | And I see how pipelining would avoid that. | 22:04 |
| SamYaple | yea im talking about home | 22:04 |
| SamYaple | good | 22:04 |
| SamYaple | im not crazy | 22:04 |
| larsks | You are also right. In addition to being write. | 22:04 |
| SamYaple | lol | 22:04 |
| *** cemason has quit IRC | 22:04 | |
| SamYaple | but with pipelining.... man this could do some things | 22:04 |
| *** salv-orl_ has quit IRC | 22:05 | |
| SamYaple | pipeliniging would work with your script too larsks if you use `cat /proc/$$/fd/0 | docker....` | 22:06 |
| SamYaple | otherwise you would have to use the interpreter straight | 22:06 |
| *** cemason has joined #kolla | 22:06 | |
| larsks | Yah. | 22:07 |
| *** aginwala has quit IRC | 22:08 | |
| *** mbound has quit IRC | 22:09 | |
| *** Serlex has quit IRC | 22:10 | |
| SamYaple | ok daneyon i think ive fleshed out all the angles here, even with pipelining you would still need to bind ~/.ansible/:/root/ansible/ into the container | 22:11 |
| SamYaple | but this is entirely doable | 22:11 |
| daneyon | SamYaple Sweet. I'm still playing around with the options | 22:12 |
| SamYaple | thanks larsks for correnting me on the stdin stuff | 22:12 |
| SamYaple | i always forget -i opens stdin | 22:12 |
| SamYaple | daneyon: so you think this will work for you? | 22:12 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 22:15 |
| *** Marga_ has joined #kolla | 22:16 | |
| *** Marga_ has quit IRC | 22:17 | |
| *** cemason has quit IRC | 22:17 | |
| *** Marga_ has joined #kolla | 22:17 | |
| *** vhosakot has quit IRC | 22:17 | |
| *** Marga_ has quit IRC | 22:18 | |
| *** Marga_ has joined #kolla | 22:18 | |
| *** vhosakot has joined #kolla | 22:19 | |
| daneyon | SamYaple I think this will work. Big thanks to you and larsks | 22:19 |
| SamYaple | nah just larsks I would have told you no :) | 22:20 |
| *** cemason has joined #kolla | 22:20 | |
| SamYaple | let me know how that works for oyu long term, im curious | 22:20 |
| larsks | \o/ | 22:21 |
| *** bmace has quit IRC | 22:23 | |
| *** bmace has joined #kolla | 22:24 | |
| *** tzn has joined #kolla | 22:25 | |
| *** dave-mccowan has quit IRC | 22:26 | |
| *** banix has quit IRC | 22:26 | |
| *** banix has joined #kolla | 22:27 | |
| *** jtriley has quit IRC | 22:28 | |
| *** banix has quit IRC | 22:29 | |
| *** tzn has quit IRC | 22:30 | |
| *** vhosakot has quit IRC | 22:30 | |
| *** vhosakot has joined #kolla | 22:31 | |
| *** macsz has quit IRC | 22:32 | |
| *** vhosakot has quit IRC | 22:34 | |
| *** vhosakot has joined #kolla | 22:34 | |
| *** bmace has quit IRC | 22:35 | |
| *** bmace has joined #kolla | 22:35 | |
| openstackgerrit | Merged openstack/kolla: Use variables to specify http or https when constructing URLs https://review.openstack.org/279707 | 22:37 |
| *** rhallisey has joined #kolla | 22:38 | |
| *** cemason has quit IRC | 22:41 | |
| *** cemason has joined #kolla | 22:41 | |
| *** athomas has quit IRC | 22:46 | |
| *** dwalsh has quit IRC | 22:47 | |
| *** dims has quit IRC | 22:47 | |
| *** bmace has quit IRC | 22:49 | |
| *** haplo37 has quit IRC | 22:52 | |
| *** stvnoyes has quit IRC | 22:53 | |
| d_code | SamYaple: regarding the PCI passthrough, what needs to be hacked? is it just including the server API? | 22:54 |
| d_code | https://wiki.openstack.org/wiki/Pci_passthrough would suggest it’s already available in Nova | 22:55 |
| d_code | and that https://github.com/yjiang5/pci_api just needs to be patched into the nova server API | 22:55 |
| SamYaple | d_code: thats correct | 22:56 |
| d_code | k…so need to modify the ansible / dockerfile for the nova-server container? | 22:56 |
| SamYaple | that and a bit of hacking on the pilicy file and you should be good | 22:56 |
| SamYaple | i dont think you do | 22:56 |
| d_code | which policy? | 22:56 |
| SamYaple | nova policy.json | 22:56 |
| d_code | so where does the server side of the API go if not in the container? | 22:57 |
| SamYaple | im confused | 22:57 |
| SamYaple | what do you mean? | 22:57 |
| d_code | :) | 22:57 |
| *** unicell has quit IRC | 22:57 | |
| d_code | yjiang5/pci_api has 2 dirs… client and server | 22:57 |
| *** unicell has joined #kolla | 22:58 | |
| SamYaple | oh for the cli stuff | 22:58 |
| SamYaple | i never used that | 22:58 |
| d_code | client obviously just goes against wherever you’re accessing the api | 22:58 |
| SamYaple | but the client stuff would go on the host | 22:58 |
| SamYaple | the server stuff needs to patch in the container | 22:58 |
| SamYaple | this is what i meant by hacking | 22:58 |
| d_code | k…that’s what I was thinking | 22:58 |
| d_code | but…idealy…could modify the build script for the container to happen at build time | 22:59 |
| d_code | rather than manually doing it each time | 22:59 |
| SamYaple | yea put it in the Dockerfile.j2 | 22:59 |
| d_code | and submitting a patch to Kolla? | 22:59 |
| SamYaple | no | 22:59 |
| d_code | is this something that could be merged in? | 22:59 |
| SamYaple | nope | 22:59 |
| d_code | :-( | 22:59 |
| SamYaple | unfortunately not | 22:59 |
| d_code | sad panda | 22:59 |
| SamYaple | not until nova accepts it | 22:59 |
| SamYaple | afaik they havent | 22:59 |
| d_code | yeah…the last commit on that repo was 3 years ago | 22:59 |
| d_code | /sigh | 22:59 |
| SamYaple | hoenstly that patch won't change much | 23:00 |
| SamYaple | there is probably a current review out there someone is rebasing | 23:00 |
| *** aginwala has joined #kolla | 23:00 | |
| d_code | interesting…yeah..the patch is literally just providing the api…it’s hooking existing nova.objects.pci_device | 23:01 |
| vhosakot | SamYaple: should I have all 112 containers built successfully in order to run "kolla-ansible deploy" ? | 23:01 |
| d_code | k…well…I’ll play with this and see if I can get it to work | 23:01 |
| SamYaple | d_code: you know its been a while since ive gone down this path, nova may have merged it back before even 2015 | 23:01 |
| vhosakot | Or, is there a minimum set of containers needed to be built before running "kolla-ansible deploy".... if so, I will make sure that this minimum set of cotnaienrs are built | 23:02 |
| d_code | k…it isn’t in my nova client…so I’ll try to patch that first and see if it blows up :) | 23:02 |
| SamYaple | d_code: yea see pci_passthrough_whitelist | 23:03 |
| SamYaple | i think the only thing not merged is the api | 23:03 |
| SamYaple | the rest can be configured through nova.conf | 23:03 |
| d_code | k | 23:03 |
| SamYaple | i still hack in the api so i can use that on the fly | 23:04 |
| d_code | be back tomorrow…. dinner time! | 23:04 |
| *** cemason has quit IRC | 23:05 | |
| *** cemason has joined #kolla | 23:07 | |
| *** guillaume_ has quit IRC | 23:07 | |
| *** sdake_ has quit IRC | 23:09 | |
| *** daneyon has quit IRC | 23:10 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 23:13 |
| vhosakot | SamYaple: keystone failed with error "ERROR:kolla.cmd.build:keystone:The command '/bin/sh -c yum -y install openstack-keystone python-keystoneclient httpd mod_wsgi && yum clean all' returned a non-zero code: 1 " | 23:19 |
| vhosakot | SamYaple: all the failed containers failed in the RUN command that installs the service . is this an issue ? | 23:20 |
| *** cemason has quit IRC | 23:23 | |
| *** cemason has joined #kolla | 23:25 | |
| *** inc0_ has joined #kolla | 23:27 | |
| *** jtriley has joined #kolla | 23:29 | |
| *** dave-mccowan has joined #kolla | 23:31 | |
| *** tpeoples_ has joined #kolla | 23:31 | |
| *** stvnoyes has joined #kolla | 23:33 | |
| *** jtriley has quit IRC | 23:34 | |
| *** mandre_ has joined #kolla | 23:35 | |
| *** ByPasS_ has joined #kolla | 23:35 | |
| *** another_larsks has joined #kolla | 23:36 | |
| *** clayton_ has joined #kolla | 23:36 | |
| *** inc0 has quit IRC | 23:37 | |
| *** ccesario has quit IRC | 23:37 | |
| *** ByPasS has quit IRC | 23:37 | |
| *** mandre has quit IRC | 23:37 | |
| *** clayton has quit IRC | 23:37 | |
| *** tpeoples has quit IRC | 23:37 | |
| *** larsks has quit IRC | 23:37 | |
| *** bmace has joined #kolla | 23:37 | |
| *** mandre_ is now known as mandre | 23:37 | |
| *** clayton_ is now known as clayton | 23:37 | |
| *** vhosakot has quit IRC | 23:37 | |
| *** ByPasS_ is now known as ByPasS | 23:37 | |
| *** tpeoples_ is now known as tpeoples | 23:40 | |
| *** ccesario has joined #kolla | 23:43 | |
| *** dims_ has joined #kolla | 23:44 | |
| ayoung | SamYaple, the patch you need has been queueiong since 1:30 this afternoon: https://review.openstack.org/#/c/280329/ | 23:44 |
| *** banix has joined #kolla | 23:50 | |
| *** sdake has joined #kolla | 23:53 | |
| *** yuanying has quit IRC | 23:56 | |
| *** yuanying has joined #kolla | 23:58 | |
| sdake | d_code re above, we don't have a way in kolla to cusotmize policy files atm | 23:59 |
| SamYaple | ayoung: what patch? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!