| *** akwasnie has quit IRC | 00:01 | |
| *** achanda has joined #kolla | 00:05 | |
| *** weiyu_ has quit IRC | 00:12 | |
| *** shadower has quit IRC | 00:23 | |
| *** shadower has joined #kolla | 00:23 | |
| *** bmace has quit IRC | 00:33 | |
| *** jruano has quit IRC | 00:36 | |
| *** bmace has joined #kolla | 00:45 | |
| sdake | https://bugs.launchpad.net/kolla/+bug/1477351 | 00:53 |
|---|---|---|
| openstack | Launchpad bug 1477351 in kolla "openvswitch-agent must have tty to run sudo" [Critical,Triaged] | 00:53 |
| *** weiyu_ has joined #kolla | 01:01 | |
| *** erkules has joined #kolla | 01:10 | |
| *** erkules_ has quit IRC | 01:12 | |
| *** weiyu_ has quit IRC | 01:19 | |
| sdake | samyaple when you arise ping me | 01:31 |
| sdake | samyaple neutron is busted - I think one problem besides we are not starting containers with tty=true, is that br-ex is not being created | 01:31 |
| sdake | in yaodu you create it, in kolla it is uncreated and isn't in the bridge list in ovs | 01:32 |
| *** jruano has joined #kolla | 01:32 | |
| sdake | you also create a port, when I try to do this manually I get an error | 01:32 |
| sdake | the l3 agent needs the br-ex, but its in the namespace of the openvswitch container | 01:33 |
| sdake | also getting error about ovs-vsctl not found in the neutron-agent container | 01:35 |
| sdake | I think we probably need to merge the ovs agent into the neutron agents container | 01:35 |
| sdake | so they run in the same namespace | 01:35 |
| *** stvnoyes has left #kolla | 01:45 | |
| *** Haomeng|2 has quit IRC | 01:46 | |
| *** dolpher has quit IRC | 01:52 | |
| *** weiyu_ has joined #kolla | 01:56 | |
| *** blahRus has quit IRC | 02:07 | |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Enable TTY mode for neutron agents for sudo use https://review.openstack.org/204809 | 02:11 |
| *** unicell has quit IRC | 02:29 | |
| *** dolpher has joined #kolla | 02:54 | |
| *** bradjones has quit IRC | 03:03 | |
| *** bradjones has joined #kolla | 03:04 | |
| *** bradjones has quit IRC | 03:04 | |
| *** bradjones has joined #kolla | 03:04 | |
| sdake | https://bugs.launchpad.net/kolla/+bug/1477375 | 03:14 |
| openstack | Launchpad bug 1477375 in kolla "ERROR The external network bridge 'br-ex' does not exist" [Critical,Confirmed] | 03:14 |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Allows neutron-agents l3 agent to communicate with OVS https://review.openstack.org/204832 | 03:19 |
| *** coolsvap|away is now known as coolsvap | 03:34 | |
| *** jasonsb has joined #kolla | 03:35 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Add Gnocchi service to kolla https://review.openstack.org/180790 | 03:43 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Removed unused hautoproxy code https://review.openstack.org/204567 | 03:51 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Glance installation from source https://review.openstack.org/201665 | 03:59 |
| *** mordred has quit IRC | 04:11 | |
| *** weiyu_ has quit IRC | 04:13 | |
| *** unicell has joined #kolla | 04:14 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Removes Host Cleanup from Container Cleanup Script https://review.openstack.org/182493 | 04:18 |
| *** unicell has quit IRC | 04:18 | |
| *** unicell has joined #kolla | 04:19 | |
| *** daneyon_ has joined #kolla | 04:22 | |
| sdake | samyaple awake yet? | 04:23 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Removes Host Cleanup from Container Cleanup Script https://review.openstack.org/182493 | 04:24 |
| *** mordred has joined #kolla | 04:24 | |
| *** daneyon has quit IRC | 04:24 | |
| *** diga has quit IRC | 04:26 | |
| coolsvap | sdake, around? | 04:28 |
| sdake | shoot | 04:28 |
| coolsvap | sdake, regarding the horizon dependency, mod-wsgi, i am bit confused | 04:29 |
| sdake | in which way | 04:29 |
| coolsvap | i added a change set to update it from http-mod-wsgi -> mod-wsgi in binary | 04:29 |
| coolsvap | it was -1 | 04:29 |
| coolsvap | now you are asking to change it in source | 04:29 |
| sdake | who -1'ed it? | 04:30 |
| coolsvap | you :) https://review.openstack.org/#/c/201497/ | 04:30 |
| sdake | i am not perfect | 04:30 |
| sdake | I wasn't sure so today I ran yum install mod-wsgi | 04:30 |
| sdake | and it installs | 04:30 |
| coolsvap | :( :( | 04:30 |
| sdake | yum install httpd-mod-wsgi does not | 04:30 |
| coolsvap | alright updating source and binary both | 04:31 |
| sdake | maybe centos is different then fedora 21 | 04:31 |
| sdake | could you try building the container | 04:31 |
| sdake | and see what it says | 04:31 |
| sdake | it will tell you if it can't find the deepndency | 04:31 |
| sdake | but it will *NOT* fail to build the container | 04:32 |
| sdake | which sucks | 04:32 |
| sdake | whatever is correct is in keystone | 04:32 |
| sdake | I know because I fixed keystone and got it working personally | 04:32 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update horizon dockerfine for dependencies https://review.openstack.org/201497 | 04:33 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Add build from source for Horizon https://review.openstack.org/202446 | 04:35 |
| coolsvap | yes building it, it was working fine when i proposed the change, checkign it again | 04:35 |
| *** diga has joined #kolla | 04:37 | |
| coolsvap | yes does not fail while building the container | 04:44 |
| sdake | https://ask.openstack.org/en/question/30333/error-unexpected-vif_typebinding_failed/ | 04:50 |
| sdake | coolsvap what do yo uthink that sed command does? | 04:50 |
| *** dolpher has quit IRC | 04:50 | |
| *** dolpher has joined #kolla | 04:52 | |
| coolsvap | i think it changes the plugins/openvswitch/ovs_neutron_plugin.ini to plugin.ini, need to double check though | 04:54 |
| *** dolpher has quit IRC | 04:57 | |
| *** gfidente has joined #kolla | 04:57 | |
| *** dolpher has joined #kolla | 04:58 | |
| *** diga has quit IRC | 04:59 | |
| * sdake wishes someone would get neutron working | 05:00 | |
| * sdake groans | 05:00 | |
| *** weiyu_ has joined #kolla | 05:11 | |
| * coolsvap having a look at it | 05:11 | |
| sdake | i'm pretty sure the mechanism driver isn't being set properly | 05:11 |
| *** diga has joined #kolla | 05:11 | |
| coolsvap | sdake, maybe a pretty dumb question, but how can i keep different versions of source and binary images in the same host? | 05:12 |
| sdake | what do you mean different versions | 05:12 |
| sdake | you can use the tag feature | 05:12 |
| *** weiyu_ has quit IRC | 05:14 | |
| coolsvap | i was thinking of adding a script which builds images only for a single service | 05:20 |
| coolsvap | rather than building all images or building each image individually in the service | 05:20 |
| *** weiyu_ has joined #kolla | 05:27 | |
| sdake | expand build.py | 05:27 |
| coolsvap | hmm | 05:32 |
| sdake | we dont need more tools we need the tools weh ave to be better | 05:37 |
| sdake | the only reason we went with a python builder is because the shell script was too complicated to comprehend ;) | 05:37 |
| coolsvap | yes, i see the TODOs in the build.py, will ask SamYaple if there's anything he's currently not working on | 05:38 |
| coolsvap | also need to start with ansible | 05:39 |
| sdake | yes we need ansible finished asap | 05:39 |
| sdake | the main services are all more or less done although neutron is busted | 05:39 |
| sdake | making progress - got the ml2 config file loading now ;-) | 05:49 |
| coolsvap | i need to get my first ansible patch working, then i can build up on that | 05:51 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1477399 | 05:53 |
| openstack | Launchpad bug 1477399 in kolla "ml2_conf.ini was not being loaded into image" [Critical,Confirmed] - Assigned to Steven Dake (sdake) | 05:53 |
| sdake | probably the place to start is to deploy aio with ansible | 05:53 |
| *** inc0 has joined #kolla | 05:53 | |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Copy ml2_conf.ini to correct location https://review.openstack.org/204891 | 05:55 |
| *** jruano has quit IRC | 06:06 | |
| harmw | lol was that realy it sdake ? | 06:46 |
| sdake | harmw its still busted | 06:47 |
| harmw | oh, your working on ansible | 06:47 |
| sdake | but i'e fixed 3 problems so far | 06:47 |
| sdake | i keep hoping samyaple will show up to rescue me :) | 06:47 |
| harmw | i had that vif_bind issue last week | 06:48 |
| sdake | you fixed it? | 06:48 |
| harmw | couldn't realy find where it was busted though | 06:48 |
| harmw | no | 06:48 |
| harmw | sad story | 06:48 |
| sdake | it is broken in atlaest 3 places | 06:48 |
| sdake | there are patches up for the 3 I know about | 06:48 |
| harmw | ok | 06:48 |
| harmw | well it made me move to checking out ansible instead :p | 06:49 |
| harmw | which brings completely different bugs on the table | 06:49 |
| sdake | ya that is what i'm using, ansible deosn't nova boot | 06:49 |
| sdake | vif bind problem | 06:49 |
| harmw | mechanism drivers were complaining in neutron logs iirc | 06:49 |
| harmw | btw, barbican is failing to build because of some pip dependency issue | 06:50 |
| harmw | swift fails because it can't find files or whatever | 06:50 |
| harmw | all the others should be pushed | 06:50 |
| harmw | *centos-rdo-binary | 06:50 |
| sdake | nice | 06:50 |
| sdake | would have been nice to just use linuxbridge since that worked ;( | 06:51 |
| harmw | nah, OVS should just work | 06:51 |
| sdake | it does not even after the implementation bugs are fixed | 06:51 |
| harmw | I had it working in some old deployment like months ago :p | 06:51 |
| sdake | must be a configuration prolem | 06:51 |
| sdake | yes I mean we don't know how to configure ovs | 06:51 |
| sdake | we have linuxbridge down | 06:51 |
| *** bmace has quit IRC | 06:53 | |
| sdake | prior to my patches the openvswitch code wasn't even loading in the nasible implementation of neutron | 06:53 |
| harmw | hehe I can see that | 06:53 |
| harmw | nice | 06:53 |
| harmw | it realy is nice to see people start playing with the ansible code now | 06:54 |
| harmw | and uncovering bugs as they go | 06:54 |
| sdake | id' prefer it just work | 06:55 |
| sdake | I've ben throug this rpocess bout 3 or 4 times with various implementations | 06:55 |
| * sdake tired of debugging neutron + nova integration | 06:55 | |
| harmw | hehe | 06:55 |
| harmw | oh yes | 06:55 |
| harmw | arent' we all | 06:55 |
| sdake | the worst part is i'm stuck, I don't know how to proceed | 06:56 |
| sdake | the logs have nothing useful | 06:56 |
| harmw | that was my issue aswell | 06:56 |
| harmw | hurray for neutron and nova | 06:57 |
| sdake | see daneyon figured all this shit outwith linuxbridge ;) | 06:57 |
| harmw | linuxbridge didn't have much shit, that also helps :p | 06:57 |
| sdake | precisely | 06:57 |
| sdake | http://openvswitch.org/pipermail/discuss/2010-November/004516.html | 07:01 |
| sdake | this is where I'm at | 07:01 |
| harmw | are the right OVS sockets shared between containers? | 07:02 |
| inc0 | good morning | 07:03 |
| harmw | hi inc0 | 07:04 |
| harmw | god I hate being behind a proxy | 07:04 |
| inc0 | I know the pain | 07:04 |
| *** bmace has joined #kolla | 07:05 | |
| inc0 | let me read through your discussion because it sounds interesting;) | 07:05 |
| harmw | can't be | 07:05 |
| harmw | it involved sdake | 07:05 |
| sdake | the annoying part is ovs-vsctl executed via docker exec works fantastically | 07:05 |
| inc0 | how about having ovs-db outside container? | 07:06 |
| inc0 | let me check how ovs is configured currently | 07:07 |
| sdake | oh snap i figured something out | 07:07 |
| sdake | running ovs-vsctol works | 07:07 |
| sdake | running within neutron-rootwrap does not! | 07:07 |
| sdake | this is the problem | 07:07 |
| inc0 | does neutron-rootweap even have ovs-vsctl? | 07:07 |
| harmw | uhm, why isn't neutron complaining about that | 07:07 |
| sdake | there apparently must be some filter files | 07:12 |
| sdake | probably not installed | 07:12 |
| sdake | definately not installed | 07:17 |
| inc0 | so, installation of vsctl might help | 07:19 |
| inc0 | I didn't know neutron rootwrap even uses it | 07:19 |
| sdake | i already installed openvswitch which includes ovsctl dude | 07:20 |
| sdake | the filters come from the neutron-server package | 07:20 |
| sdake | i am just going to copy them in | 07:20 |
| *** daneyon_ has quit IRC | 07:20 | |
| *** daneyon has joined #kolla | 07:21 | |
| sdake | spending 4 minutes running ansible has the same feeling as building images........................ | 07:30 |
| sdake | harmw can you check if from source installs filters? | 07:32 |
| sdake | /etc/neutron/rootwrap.d | 07:33 |
| *** jmccarthy has quit IRC | 07:33 | |
| *** jmccarthy has joined #kolla | 07:33 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Make variables in OVS container configurable https://review.openstack.org/204939 | 07:55 |
| *** achanda has quit IRC | 08:13 | |
| *** achanda has joined #kolla | 08:16 | |
| *** zhiwei has joined #kolla | 08:18 | |
| *** achanda has quit IRC | 08:21 | |
| harmw | hm, some docs on the whole build-from-source would be nice | 08:26 |
| harmw | god I should just reset my LP password and start filing bugs this instant, instead of tonight | 08:26 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Removed unused hautoproxy code https://review.openstack.org/204567 | 08:28 |
| SamYaple | sdake: ping | 08:29 |
| sdake | hey sam | 08:29 |
| sdake | i fixed 3 bugs with neutron | 08:29 |
| SamYaple | whats up | 08:30 |
| SamYaple | oh yea | 08:30 |
| sdake | and I have another one i'm working on | 08:30 |
| sdake | ya so it should actually work now vs not as it didn't before ;-) | 08:30 |
| SamYaple | thats probably right. i only tested the agents started | 08:30 |
| sdake | ya gotta tet nova boots | 08:30 |
| SamYaple | gotta get more timei n the day! | 08:31 |
| SamYaple | its my weekend now | 08:31 |
| *** shardy_ has joined #kolla | 08:31 | |
| sdake | running ansible over and over is like building images | 08:32 |
| SamYaple | for fresh reruns, yea | 08:32 |
| *** shardy has quit IRC | 08:32 | |
| sdake | 4 minutes 9 seconds | 08:33 |
| sdake | wait around | 08:33 |
| sdake | samyaple is shade supposed tobe installed on every node inteh cluster? | 08:35 |
| harmw | yep | 08:35 |
| SamYaple | yea | 08:35 |
| SamYaple | but i dont like that, i want a container that handles this | 08:36 |
| sdake | we need to put it in a container in some way then, it has a million dependencies | 08:36 |
| SamYaple | yup | 08:36 |
| *** shardy_ has quit IRC | 08:36 | |
| SamYaple | issue is ansible modules wont work like that | 08:36 |
| SamYaple | we would have to run an ssh server inside that container | 08:37 |
| SamYaple | im not sure how i want to solve it, but i really dont like that idea | 08:37 |
| sdake | I think that is ok tradeoff comapredt o installing a ton of deps jus tfor 1 library | 08:37 |
| *** shardy has joined #kolla | 08:37 | |
| SamYaple | you dont understand the implications | 08:37 |
| inc0 | SamYaple, how about ssh to host and docker exec stuff? | 08:40 |
| inc0 | instead of ssh to container? | 08:40 |
| SamYaple | that breaks ansible modules inc0 | 08:41 |
| sdake | well we dont want to install 100's of packages just to deploy openstack | 08:41 |
| inc0 | why? I meant that ansible can do that | 08:41 |
| sdake | that nullifies the entire point of containers | 08:42 |
| SamYaple | inc0: that breaks ansible modules. they cannot do that. | 08:42 |
| inc0 | ansible modules can do docker exec... | 08:42 |
| sdake | samyaple these commands need to be done | 08:42 |
| sdake | # ovs-vsctl add-br br-eth0 | 08:42 |
| sdake | # ovs-vsctl add-port br-eth0 eth0 | 08:42 |
| SamYaple | sdake: yea i just remembered about those | 08:42 |
| SamYaple | inc0: no we are speaking differently here. ansible modulescannot work like that | 08:43 |
| sdake | also the interface for ml2_conf.ini is neutron_interface, but it needs to be a bridge | 08:43 |
| sdake | I wasn't quite sure how to fix it | 08:43 |
| SamYaple | sdake: no it doesnt | 08:43 |
| sdake | it does need to be a bridge I can assure you | 08:43 |
| sdake | if it is not, the agent blows up | 08:44 |
| SamYaple | you are wrong or we are talkign about different things | 08:44 |
| *** achanda has joined #kolla | 08:44 | |
| sdake | why dont you pull my patch stream and try a nova boot | 08:44 |
| sdake | the docs say [ovs] bridge_interfaces has to be a bridge, not a regualr nic | 08:45 |
| sdake | the code bombs out if its not a bridge | 08:45 |
| SamYaple | oh that. yea but neutron_interface is supposed to be a bridge | 08:45 |
| SamYaple | how else would you be able to name it? | 08:46 |
| SamYaple | fyi guys i just woke up | 08:46 |
| inc0 | sdake, which line of code checks that? Could you please point me to it? | 08:46 |
| SamYaple | https://github.com/stackforge/kolla/blob/master/ansible/roles/neutron/templates/ml2_conf.ini.j2#L35 | 08:47 |
| inc0 | because as I understand, it doesn't show as bridge in neutron container | 08:47 |
| inc0 | even if it might be one due to net-host and ovs being configured in other container? | 08:47 |
| SamYaple | inc0: no that line needs to be a bridge | 08:47 |
| SamYaple | its the ovs bridge you create | 08:47 |
| SamYaple | neutron does not create that ovs bridge | 08:47 |
| inc0 | yes, but in which container you create bridge? | 08:48 |
| inc0 | manually? | 08:48 |
| SamYaple | that is the way it goes, yes | 08:48 |
| inc0 | let me answer that, ovs container | 08:48 |
| SamYaple | and not in the container | 08:48 |
| inc0 | outside container? | 08:48 |
| SamYaple | i mean yea but it doesnt stay there | 08:48 |
| *** mickt has joined #kolla | 08:48 | |
| inc0 | I know, net-ost | 08:48 |
| SamYaple | oh jeez ill be back in 10 | 08:48 |
| inc0 | and I'll run your code sdake - I want to see traceback;) | 08:49 |
| mickt | Qusetion regarding build versus build --release; What is the difference, i.e. what is "release" supposed to have above other? | 08:49 |
| SamYaple | fuck | 08:49 |
| harmw | mickt: it tags it as latest | 08:50 |
| inc0 | mickt, it also marks your new image as latest | 08:50 |
| sdake | mickt --release tags with "latest" | 08:50 |
| SamYaple | sdake: i didnt have my latest neutron shit uploaded beforei t merged | 08:50 |
| sdake | without release, it tags with whatever the hash is | 08:50 |
| inc0 | its equivalent to docker tag your-img-id latest | 08:50 |
| SamYaple | sdake: https://review.openstack.org/#/c/204891/ | 08:50 |
| sdake | well i've spent lsat 8 hours fixing bugs | 08:50 |
| SamYaple | can i take over that patch? i have the code for it | 08:50 |
| pbourke | i think what mickt is asking is why not just tag everything latest | 08:50 |
| SamYaple | sdake: it needs 4 more ections | 08:50 |
| mickt | so it just identifies it as latest, doesn't mean it's fully tested or better etc. | 08:50 |
| sdake | knock yurself out | 08:51 |
| sdake | samyaple there are other patches inthte stream | 08:51 |
| inc0 | mickt, it just add tag, nothing more | 08:51 |
| SamYaple | sdake: yea ive reviewed them | 08:51 |
| sdake | there is also one more patch I have | 08:51 |
| SamYaple | already reviews | 08:51 |
| sdake | letm e add it on | 08:51 |
| sdake | so you can do what you like with it, i'm going to bed shortly | 08:51 |
| harmw | TASK: [nova | Cleaning up boostrap container] ********************************* | 08:51 |
| harmw | <support01> REMOTE_MODULE docker state=absent name=bootstrap_nova image=kollaglue/centos-rdo-nova-api:latest | 08:51 |
| harmw | wonder why that takes forever | 08:52 |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Add filters so neutron rootwrap doesn't timeout https://review.openstack.org/204955 | 08:53 |
| sdake | samyaple if you ilke you can take over that patch stream and get nova into booting state - i'd love you long time for it :) | 08:54 |
| SamYaple | yea this is all my screwup | 08:54 |
| sdake | so I can do heat support | 08:54 |
| SamYaple | theres a bunch of code here i didnt git-review | 08:54 |
| sdake | its all good I learned alot about how all this stuff works | 08:55 |
| SamYaple | i feel like a tool | 08:55 |
| SamYaple | all that wasted time | 08:55 |
| mickt | to rephrase, why bother, why just not use latest as default | 08:55 |
| mickt | don't see much value | 08:55 |
| SamYaple | mickt: the new build script does use latest as the default | 08:55 |
| harmw | mickt: we can also tag our releases :) | 08:55 |
| harmw | kilo, juno, etc | 08:55 |
| harmw | that way you can pull containers from a certain release | 08:55 |
| sdake | mickt I agree it would maek more sense to tag latest as default | 08:56 |
| harmw | instead of the latest and gratest, which has a higer probability of failing | 08:56 |
| inc0 | sdake, you use ansible all-in-one in your tests right? | 08:56 |
| sdake | mickt and offer the other type of tagging as non-default | 08:56 |
| sdake | however, that isn't how the code was written ;-) | 08:56 |
| SamYaple | sdake: the filters don't need to be hardcoded like that. those files exist in the container and can be copied | 08:56 |
| SamYaple | right? | 08:56 |
| sdake | inc0 ya aio | 08:56 |
| sdake | samyaple I couldn't find them in the container | 08:56 |
| sdake | samyaple they come from the neutron-server package only | 08:57 |
| sdake | not the agents package... | 08:57 |
| sdake | not sure about from source, but from binary they have to be hard coded or neutron-server needs to be installed in all containers | 08:57 |
| SamYaple | sdake: /usr/share/neutron/rootwrap/ | 08:57 |
| sdake | that may be in neutron-server | 08:58 |
| SamYaple | nope | 08:58 |
| sdake | but i didn't see it in the agent | 08:58 |
| SamYaple | its in the agents | 08:58 |
| SamYaple | filters_path=/etc/neutron/rootwrap.d,/usr/share/neutron/rootwrap | 08:58 |
| sdake | pretty sure it isn't because once I put it in, i stopped having error 14 timeouts | 08:58 |
| SamYaple | and the filter paths already check that | 08:58 |
| sdake | yes but there are no filter files in that directory | 08:58 |
| SamYaple | sdake: im looking at it | 08:58 |
| sdake | in which agent container? | 08:59 |
| SamYaple | root@server02:~/kolla# docker run --rm -it kollaglue/centos-binary-neutron-agents bash | 08:59 |
| SamYaple | [root@6c3e3df11f18 /]# ls /usr/share/neutron/rootwrap/ | 08:59 |
| SamYaple | debug.filters dhcp.filters ebtables.filters ipset-firewall.filters iptables-firewall.filters l3.filters | 08:59 |
| sdake | wrong agent container | 08:59 |
| sdake | try ovs agent container | 08:59 |
| SamYaple | ah ok ill check | 08:59 |
| SamYaple | we need thin containers | 08:59 |
| SamYaple | root@server02:~/kolla# docker run --rm -it kollaglue/centos-binary-neutron-openvswitch-agent bash | 09:00 |
| SamYaple | [root@d6bffe6c8bb1 /]# ls /usr/share/neutron/rootwrap | 09:00 |
| SamYaple | debug.filters ebtables.filters iptables-firewall.filters openvswitch-plugin.filters | 09:00 |
| SamYaple | dhcp.filters ipset-firewall.filters l3.filters | 09:00 |
| sdake | interesting | 09:00 |
| sdake | well i'm off to bed - 2am here :) | 09:00 |
| sdake | feel free to rebase and drop patches/abandon patches as necessary | 09:00 |
| sdake | if you get nova booting I'll love you long time :) | 09:00 |
| SamYaple | will do | 09:01 |
| SamYaple | thanks for all the debug | 09:01 |
| SamYaple | sorry for missing code | 09:01 |
| SamYaple | nova should be up by morning | 09:01 |
| SamYaple | inc0: re: ansible modules ssh | 09:01 |
| SamYaple | ansible modules run on the host they ssh to. you cannot docker exec and run the ansible module in that docker container | 09:02 |
| SamYaple | you have to ssh into that docker container which we cannot do for a whole slew of reasons | 09:02 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Copy ml2_conf.ini to correct location https://review.openstack.org/204891 | 09:14 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Removed unused hautoproxy code https://review.openstack.org/204567 | 09:19 |
| inc0 | SamYaple, but you can ssh to host and execute commands in docker using docker exec | 09:27 |
| SamYaple | inc0: that beaks modules | 09:27 |
| SamYaple | where is the disconnect here | 09:27 |
| inc0 | thats how I hot reload configs in haproxy | 09:27 |
| SamYaple | yea, but thats not an ansible module you are using | 09:28 |
| inc0 | I run ansible's command module | 09:28 |
| SamYaple | great. now how do you do this without the command module? | 09:28 |
| SamYaple | say, for the modules that need to use the shade library? | 09:28 |
| inc0 | why modules *need* to use shade? | 09:30 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Glance installation from source https://review.openstack.org/201665 | 09:30 |
| SamYaple | im not prepared to have this discussion again. the mailing list should hold all the answers for you | 09:31 |
| inc0 | ok...I'll dig through this and we'll talk later | 09:31 |
| inc0 | also I'll play around this neutron bug | 09:31 |
| inc0 | afk for now | 09:31 |
| SamYaple | which one | 09:31 |
| inc0 | lack of ovs bridge | 09:31 |
| SamYaple | yea i have that working. i just have to create teh bridge like i do in yaodu | 09:32 |
| inc0 | cool, I'll take a look;) | 09:32 |
| SamYaple | always good to have more eyes | 09:32 |
| inc0 | brb for now | 09:32 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update horizon dockerfile for dependencies https://review.openstack.org/201497 | 09:36 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update service helper scripts for variable changes https://review.openstack.org/203935 | 09:38 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update service helper scripts for variable changes https://review.openstack.org/203935 | 09:38 |
| openstackgerrit | Merged stackforge/kolla: Add memcached image (required by Swift) https://review.openstack.org/202493 | 09:38 |
| *** diga has quit IRC | 09:50 | |
| *** erkules has quit IRC | 09:53 | |
| *** erkules has joined #kolla | 09:53 | |
| *** weiyu_ has quit IRC | 09:53 | |
| *** weiyu_ has joined #kolla | 09:54 | |
| dolpher | Anyone can take a look at this simple changes? it needs another +2 review. https://review.openstack.org/#/c/203939/ | 09:55 |
| SamYaple | dolpher: done | 09:56 |
| dolpher | Thanks SamYaple | 09:57 |
| openstackgerrit | Merged stackforge/kolla: Add get_or_create_domain for designate sink https://review.openstack.org/203939 | 09:58 |
| *** gfidente has quit IRC | 09:59 | |
| *** dolpher has quit IRC | 10:00 | |
| *** gfidente has joined #kolla | 10:01 | |
| *** athomas has quit IRC | 10:02 | |
| inc0 | SamYaple, merge_configs is not a legal parameter in an Ansible task or handler - how do you add library dir to ansibles PATH? | 10:06 |
| openstackgerrit | Paul Bourke proposed stackforge/kolla: Install correct crux in centos-source-base https://review.openstack.org/204977 | 10:06 |
| SamYaple | inc0: its a relative path (though i think there is a global) | 10:07 |
| harmw | inc0: ~/.ansible.cfg, or make sure the library folder lives in your . | 10:07 |
| inc0 | hmm...it should | 10:08 |
| harmw | http://docs.ansible.com/ansible/intro_configuration.html#library | 10:08 |
| inc0 | thanks | 10:08 |
| harmw | he SamYaple, you've tested the multinode setup? | 10:08 |
| harmw | (ansible) | 10:08 |
| inc0 | however, it is in ./library so that should work too | 10:09 |
| inc0 | debug time! | 10:09 |
| harmw | yep inc0 :) | 10:09 |
| *** athomas has joined #kolla | 10:10 | |
| SamYaple | harmw: yea. except for neutron. working on that now | 10:10 |
| harmw | any thoughts on when it'll be done? | 10:14 |
| SamYaple | youll be able to boot an instance multinode by L2 | 10:14 |
| *** zhiwei has quit IRC | 10:15 | |
| harmw | ah, thats probably not today :p | 10:15 |
| SamYaple | i plan on having it working today, but i wont promise it until L2 | 10:16 |
| harmw | I'm just eager to try it out | 10:16 |
| SamYaple | this is the first chance ive had to work on it uninteruppted in almost 2 weeks | 10:16 |
| vbel | Good morning/afternoon/evening folks. I am going to fix naming error. Are we to use CONFIG_INTERNAL or CONFIG_INSIDE? | 10:18 |
| SamYaple | vbel: internal and external at the names | 10:19 |
| vbel | I mean: | 10:19 |
| vbel | > grep -r CONFIG_INSIDE . | 10:19 |
| vbel | ./specs/ansible-multi.rst:CONFIG_INSIDE. | 10:19 |
| vbel | ./specs/ansible-multi.rst:CONFIG_INSIDE model, the community will repair it. | 10:19 |
| vbel | ./specs/ansible-multi.rst:The CONFIG_INSIDE model of configuration maintains the immutable, | 10:19 |
| vbel | ./specs/ansible-multi.rst: CONFIG_STRATEGY="CONFIG_INSIDE" | 10:19 |
| vbel | ./specs/ansible-multi.rst:CONFIG_INSIDE will match the current crudini.sh implementation. | 10:19 |
| vbel | ./specs/ansible-multi.rst: CONFIG_INSIDE) | 10:19 |
| vbel | ./specs/ansible-multi.rst: CONFIG_INSIDE - no bind mount | 10:19 |
| vbel | > grep -r CONFIG_INTERNAL . | 10:19 |
| vbel | ./compose/openstack.env:KOLLA_CONFIG_STRATEGY=CONFIG_INTERNAL | 10:19 |
| vbel | ./docker/common/base/kolla-common.sh: CONFIG_INTERNAL) | 10:19 |
| vbel | ./docker/common/neutron/neutron-dhcp-agent/start.sh: CONFIG_INTERNAL) | 10:19 |
| vbel | ./docker/common/neutron/neutron-l3-agent/start.sh: CONFIG_INTERNAL) | 10:19 |
| vbel | ./docker/common/neutron/neutron-metadata-agent/start.sh: CONFIG_INTERNAL) | 10:19 |
| vbel | ./tools/genenv:KOLLA_CONFIG_STRATEGY=CONFIG_INTERNAL | 10:19 |
| vbel | SamYaple: ok! | 10:19 |
| SamYaple | vbel: yea the spec is outdated. done worry about fixing that | 10:20 |
| SamYaple | the code is correct | 10:20 |
| vbel | ok, great :) | 10:20 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: sudoers script was not properly running https://review.openstack.org/204994 | 10:21 |
| openstackgerrit | Paul Bourke proposed stackforge/kolla: Install correct crux in centos-source-base https://review.openstack.org/204977 | 10:24 |
| *** coolsvap is now known as coolsvap|away | 10:27 | |
| *** akscram has quit IRC | 10:33 | |
| *** zhiwei has joined #kolla | 10:34 | |
| *** akscram has joined #kolla | 10:35 | |
| inc0 | which version of ansible are you running? | 10:36 |
| SamYaple | 1.9.2 | 10:40 |
| SamYaple | 2.x is broked | 10:40 |
| *** dims_ has joined #kolla | 10:52 | |
| *** zhiwei has quit IRC | 10:52 | |
| *** rhallisey has joined #kolla | 11:16 | |
| *** weiyu_ has quit IRC | 11:36 | |
| *** inc0_ has joined #kolla | 11:44 | |
| *** inc0 has quit IRC | 11:44 | |
| *** vbel has quit IRC | 11:50 | |
| *** vbel has joined #kolla | 11:50 | |
| *** dims_ has quit IRC | 12:02 | |
| *** dwalsh has joined #kolla | 12:04 | |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Adds the appropriate tools to addin the ovs bridge https://review.openstack.org/205029 | 12:12 |
| harmw | SamYaple: why don't you just use $? on L7 of ovs_ensure_bla? | 12:15 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Adds the appropriate tools to addin the ovs bridge https://review.openstack.org/205029 | 12:15 |
| SamYaple | harmw: thats really not best practice | 12:16 |
| SamYaple | if you do that and you refactor you can screw up the code its checking without knowing it | 12:16 |
| harmw | if thats the case one should be a bit more careful in their actions when refactoring | 12:18 |
| harmw | but fair enough | 12:18 |
| SamYaple | or you can write better code like this | 12:18 |
| SamYaple | preferably both happens | 12:18 |
| harmw | with this neutron just works? | 12:19 |
| SamYaple | this and 3-4 other patches | 12:19 |
| SamYaple | they are sitting in review | 12:19 |
| harmw | cool | 12:19 |
| harmw | I didn't look into any of them yet though | 12:19 |
| harmw | does this involve multinode or is that a completely different set? | 12:20 |
| SamYaple | i have no clue. havent tested a clean system yet | 12:20 |
| SamYaple | this should work with multinode | 12:20 |
| SamYaple | there is no mutlinode specific code if thats what youre asking | 12:21 |
| harmw | ill probably spin something up tonight | 12:21 |
| harmw | see what happens | 12:21 |
| SamYaple | yea just document what breaks | 12:21 |
| harmw | ofcourse | 12:21 |
| harmw | LP is my friend :) | 12:22 |
| SamYaple | also youll probably want to wait for the patches to merge | 12:22 |
| harmw | oh I'll get them in first | 12:22 |
| harmw | not gonna wait for a proper merge, because of $time | 12:22 |
| SamYaple | i feel bad for sdake | 12:23 |
| harmw | hehe | 12:24 |
| SamYaple | i made him test neutron when half the patchset was sitting in my local branch | 12:24 |
| SamYaple | :/ | 12:24 |
| SamYaple | i never git reviewed | 12:24 |
| *** weiyu has joined #kolla | 12:27 | |
| *** prad has joined #kolla | 12:41 | |
| *** diogogmt has quit IRC | 12:44 | |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Add filters so neutron rootwrap doesn't timeout https://review.openstack.org/204955 | 12:54 |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Allows neutron-agents l3 agent to communicate with OVS https://review.openstack.org/204832 | 12:54 |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Enable TTY mode for neutron agents for sudo use https://review.openstack.org/204809 | 12:54 |
| openstackgerrit | Steven Dake proposed stackforge/kolla: Copy ml2_conf.ini to correct location https://review.openstack.org/204891 | 12:54 |
| SamYaple | sdake: why are you breaking my patches? | 12:54 |
| sdake | morning | 12:56 |
| sdake | 4 hrs of ssleep = groan | 12:56 |
| sdake | you should feel bad for me samyaple | 12:57 |
| SamYaple | i only kinda do | 12:57 |
| sdake | samyaple I was just putting https://review.openstack.org/#/c/204832/ at the front | 12:58 |
| sdake | and then was going to abandon the others | 12:58 |
| sdake | would you prefer some other course of action | 12:58 |
| sdake | harmw if youw ant to ack https://review.openstack.org/#/c/204832/ that would work - or rhallisey if your around | 12:58 |
| SamYaple | or pbourke | 12:58 |
| SamYaple | i got your back pbourke | 12:58 |
| SamYaple | i wont forget about you | 12:58 |
| sdake | it woudl make the most sense if you edited my change ids with your solutions to hte problems | 12:58 |
| sdake | like the removing tty support requirement for sudo, I have no idea how to do that | 12:58 |
| SamYaple | sdake: i meant to, i accidentally made a new patchset ;) | 12:59 |
| sdake | samyaple are you working on a rebase of that patch stream or your own patch stream | 12:59 |
| pbourke | will ahve a look | 12:59 |
| pbourke | *have | 12:59 |
| SamYaple | sdake: as you can see, you overrode my patch here https://review.openstack.org/#/c/204891/ | 12:59 |
| sdake | oh my bad | 13:00 |
| sdake | can you resubmit with that one patch reordered | 13:00 |
| SamYaple | i can indeed | 13:00 |
| SamYaple | everything is functional, testing booting now btw | 13:00 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Copy ml2_conf.ini to correct location https://review.openstack.org/204891 | 13:01 |
| harmw | sdake: I will ack stuff once I get home | 13:01 |
| harmw | some hours from now | 13:02 |
| SamYaple | sdake: the tty thing we fixed in config-internal a long time ago | 13:02 |
| SamYaple | sdake: i just wasnt sourcing it properly. we share that code now | 13:02 |
| sdake | ok | 13:02 |
| sdake | are all your patches in the review queue? | 13:02 |
| SamYaple | yup | 13:02 |
| sdake | cool i'll hit it | 13:02 |
| sdake | and nova boots now? | 13:03 |
| sdake | or still no boot | 13:03 |
| SamYaple | im checking it now | 13:03 |
| SamYaple | setting up images and what not | 13:03 |
| SamYaple | ports work and communicate properly | 13:03 |
| sdake | pbourke mind hitting the review queue with me so we can get this stuff merged | 13:04 |
| pbourke | sdake: sure | 13:04 |
| pbourke | can I get a one liner of context? was scanning logs from last night but only picked up something something neutron | 13:06 |
| pbourke | just beating it into shape? | 13:06 |
| SamYaple | damn vif_binding failed. lemme figure out why | 13:06 |
| SamYaple | i hate this error | 13:07 |
| pbourke | sdake: ^ | 13:07 |
| sdake | pbourke related to neutron not working with nova booting an instance | 13:07 |
| sdake | I spent about 8 hours debugging various problems with it | 13:07 |
| sdake | and fixed about 3 or 4 things, apparently incorrectly :) | 13:08 |
| SamYaple | that i had sitting in my local branch :( | 13:08 |
| SamYaple | i really feel bad about that one | 13:08 |
| sdake | well if you fix the current vif binding failed, then no hard feelings :) | 13:08 |
| sdake | the last place I got was that the agents kept restarting | 13:09 |
| SamYaple | nah thats all good now | 13:09 |
| SamYaple | no filters needed | 13:09 |
| SamYaple | well filters needed, but they are in teh right place | 13:09 |
| sdake | pbourke https://review.openstack.org/#/c/204994/ | 13:09 |
| pbourke | ok sounds good | 13:09 |
| sdake | pbourke you can do search field "owner:sam@yaple.net" | 13:09 |
| SamYaple | https://www.youtube.com/watch?v=Gtffv9bpB-U | 13:10 |
| sdake | samyaple one of those containers I had to add openvswitch to to get ovs-vsctl | 13:11 |
| sdake | did you do that? | 13:11 |
| *** jruano has joined #kolla | 13:11 | |
| sdake | the openvswitch-agent container | 13:11 |
| SamYaple | sdake: yea its in one of those patches | 13:12 |
| sdake | I see you fixed bridge_mappings :) | 13:13 |
| SamYaple | yea i realized you were right and that would get confusing | 13:13 |
| sdake | samyaple that introduction of openvswitch.rpm is not present in any of your patches to the openvswitch_agent | 13:15 |
| openstackgerrit | Merged stackforge/kolla: sudoers script was not properly running https://review.openstack.org/204994 | 13:15 |
| SamYaple | in the neutron-base | 13:16 |
| sdake | you only had patches outstanding that I saw | 13:17 |
| sdake | and neither of them modified neutron_base | 13:17 |
| SamYaple | lies at least the sudoers did. im looking for it | 13:17 |
| sdake | sudoers script just merged | 13:18 |
| sdake | pbourke https://review.openstack.org/#/c/204891/ | 13:18 |
| sdake | there is no change to any dockerfiles in 204994 | 13:18 |
| SamYaple | its there sdake | 13:20 |
| SamYaple | its just in your patchset ;) | 13:20 |
| SamYaple | https://review.openstack.org/#/c/204832/ | 13:20 |
| sdake | pbourke openvswitch is needed in that review because it brings in ovs-vsctl | 13:21 |
| sdake | ovs-vsctl is run via sudo rootwrap by the agent python code | 13:22 |
| SamYaple | sdake: i think he is saying we can remove it other places | 13:22 |
| SamYaple | but we dont need to remove them other places | 13:22 |
| SamYaple | i understand its some dup right now, but it will be needed when we go to thin containers again | 13:22 |
| sdake | i think we dont want ipc we want rpc | 13:22 |
| sdake | but anyway discussion for later | 13:22 |
| SamYaple | thats not the discussion | 13:23 |
| sdake | I dont see how that agent is supposed to work on a different node | 13:23 |
| SamYaple | its not | 13:23 |
| SamYaple | thats not what were talking about | 13:23 |
| sdake | i know, I hanged subjects | 13:23 |
| pbourke | SamYaple: if its in base it will work for thin | 13:23 |
| SamYaple | pbourke: but i dont want it in all containers | 13:23 |
| pbourke | ok | 13:24 |
| SamYaple | what you arre saying is not wrong pbourke, i just dontw want to remove it other places at this time | 13:24 |
| pbourke | that's fine | 13:24 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Make variables in OVS container configurable https://review.openstack.org/204939 | 13:24 |
| sdake | pbourke can you chane your vote then on https://review.openstack.org/#/c/204832/2 | 13:25 |
| pbourke | done | 13:27 |
| openstackgerrit | Merged stackforge/kolla: Copy ml2_conf.ini to correct location https://review.openstack.org/204891 | 13:27 |
| pbourke | openvswitch_vswitchd | 13:27 |
| pbourke | SamYaple: I'm trying to figure out where this container comes from? | 13:27 |
| pbourke | https://review.openstack.org/#/c/205029/2/ansible/roles/neutron/tasks/start.yml | 13:28 |
| openstackgerrit | Merged stackforge/kolla: Allows neutron-agents l3 agent to communicate with OVS https://review.openstack.org/204832 | 13:28 |
| sdake | its buit in the ovs directory | 13:28 |
| SamYaple | pbourke: thats the name of the container. the image is ovs-vswitchd | 13:29 |
| pbourke | where is it started | 13:30 |
| SamYaple | ansible starts it | 13:30 |
| pbourke | kolla_openvswitch_vswitchd_container_name: "ovs-vswitchd" | 13:31 |
| pbourke | name is wrong? | 13:31 |
| openstackgerrit | James McCarthy proposed stackforge/kolla: Swift Containers https://review.openstack.org/202171 | 13:31 |
| SamYaple | pbourke: no | 13:31 |
| *** inc0_ has quit IRC | 13:33 | |
| pbourke | ok I can take your word on it | 13:33 |
| pbourke | final question - should you be using the variable for this command | 13:33 |
| *** jtriley has joined #kolla | 13:34 | |
| SamYaple | ? which command | 13:36 |
| pbourke | the docker exec in https://review.openstack.org/#/c/205029/2/ansible/roles/neutron/tasks/start.yml | 13:36 |
| SamYaple | no the container names are not configurable | 13:36 |
| SamYaple | that variable should be "kolla_openvswitch_vswitchd_image_name" | 13:37 |
| SamYaple | i noticed that a while back but ill clean it up all at once | 13:37 |
| sdake | all /usr/share and /opt needs to go in /var/lib while your about it ;) | 13:38 |
| SamYaple | sdake: what? | 13:38 |
| SamYaple | /opt/kolla is staying /opt/kolla or goign to /etc/kolla | 13:38 |
| SamYaple | those are configs | 13:38 |
| sdake | /var/lib is where persistent data is stored | 13:39 |
| sdake | /usr/share is where shared data that doesn't change is stored | 13:39 |
| SamYaple | /usr/share is wrong | 13:39 |
| SamYaple | there is a todo about that already | 13:39 |
| sdake | /var/lib would be better then /opt | 13:39 |
| SamYaple | its for configs that the enduser can modify | 13:39 |
| SamYaple | /etc | 13:39 |
| sdake | yes I suppose although they are loaded on the remote system so they aren't exactly /etc files | 13:40 |
| sdake | but anyway /opt is definately wrong ;) | 13:40 |
| SamYaple | they are becaue you can modify them on the remote system | 13:40 |
| SamYaple | eh i dont disagree | 13:40 |
| openstackgerrit | Merged stackforge/kolla: Adds the appropriate tools to addin the ovs bridge https://review.openstack.org/205029 | 13:40 |
| SamYaple | both those changes are two lines in an ansible config | 13:41 |
| SamYaple | and probably a few hours of fighting | 13:41 |
| *** shardy_ has joined #kolla | 13:42 | |
| *** pbourke has quit IRC | 13:43 | |
| *** pbourke has joined #kolla | 13:44 | |
| *** dims_ has joined #kolla | 13:44 | |
| *** shardy has quit IRC | 13:44 | |
| pbourke | sdake: I think I've reviewed everything except the rootwrap filters which I will need to do some reading on to understand. ping me any I've missed | 13:44 |
| sdake | sam claims we dont need the rootwrap filters | 13:44 |
| sdake | so i'lll test master now | 13:44 |
| sdake | and likely find i still can't boot nova | 13:45 |
| SamYaple | sdake: i dont claim that! | 13:45 |
| *** dims_ has quit IRC | 13:45 | |
| SamYaple | i say they are already there | 13:45 |
| SamYaple | and they are | 13:45 |
| sdake | thats what i mean | 13:45 |
| *** dims_ has joined #kolla | 13:45 | |
| SamYaple | you should check | 13:45 |
| sdake | where are they preciesely? | 13:45 |
| SamYaple | /etc/neutron/rootwrap.conf | 13:45 |
| sdake | they need to be in /etc/neutron/rootwrap.d | 13:45 |
| SamYaple | that will tell you where it looks | 13:45 |
| SamYaple | no they dont | 13:45 |
| SamYaple | they _can_ be there | 13:46 |
| sdake | pretty sure that is what rootwrap.conf says | 13:46 |
| sdake | or alternatelively /usr/share | 13:46 |
| sdake | as I recall they were in neither place | 13:46 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update scipts for variable changes https://review.openstack.org/203935 | 13:46 |
| SamYaple | there are in /usr/share | 13:46 |
| SamYaple | you shoudl ook | 13:46 |
| sdake | will do | 13:46 |
| SamYaple | i showed you last night | 13:46 |
| sdake | when at 2am? | 13:46 |
| SamYaple | indeed | 13:46 |
| SamYaple | Stderr: ip6tables-restore v1.4.21: ip6tables-restore: unable to initialize table 'raw' | 13:47 |
| SamYaple | tis why vif_failed | 13:47 |
| *** shardy_ has quit IRC | 13:48 | |
| *** shardy has joined #kolla | 13:48 | |
| sdake | nice | 13:49 |
| sdake | which container was that in | 13:50 |
| SamYaple | neutron-openvswitch-agent | 13:50 |
| sdake | i am rebuilding atm | 13:51 |
| sdake | you need /lib/modules i suspect | 13:52 |
| sdake | samyaple ^^ | 13:52 |
| SamYaple | yea already added | 13:52 |
| SamYaple | (i cheated, i checked yaodu) | 13:52 |
| sdake | lol | 13:52 |
| sdake | is openvswitch gent run on every compute node? | 13:53 |
| SamYaple | yes | 13:53 |
| SamYaple | well unless they want linuxbridge of course | 13:53 |
| *** prad has left #kolla | 13:53 | |
| sdake | how does it communicate with the db - ovs-vsctl expects to use unix sockets for comms | 13:54 |
| SamYaple | openvswitch runs on every compute node | 13:54 |
| sdake | the ovs db does as well? | 13:54 |
| SamYaple | /run is bound around | 13:54 |
| SamYaple | yes | 13:54 |
| SamYaple | they are a pair | 13:54 |
| sdake | got it thanks | 13:54 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Update service helper scripts for variable changes https://review.openstack.org/203935 | 13:55 |
| sdake | kolla requires setting 5 config values | 13:59 |
| sdake | we ought to make password.yml autogenerate securely at oem point | 13:59 |
| sdake | if someone is looking for work to do ;) | 13:59 |
| openstackgerrit | James McCarthy proposed stackforge/kolla: Swift Containers https://review.openstack.org/202171 | 14:00 |
| SamYaple | sdake: yea we really should | 14:03 |
| SamYaple | Stderr: execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:135 | 14:03 |
| SamYaple | 2015-07-23 14:01:51.880 1 DEBUG neutron.agent.linux.utils [req-dac20874-aa98-455c-9dc5-9ccbec26b805 - - - - -] Running command: ['ps', '--ppid', '178', '-o', 'pid='] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85 | 14:03 |
| SamYaple | 2015-07-23 14:01:51.911 1 DEBUG neutron.agent.linux.utils [req-dac20874-aa98-455c-9dc5-9ccbec26b805 - - - - -] | 14:03 |
| SamYaple | Command: ['ps', '--ppid', '178', '-o', 'pid='] | 14:03 |
| SamYaple | thats breaking neutron openvsiwthc agent | 14:03 |
| SamYaple | Exit code: 1 | 14:03 |
| sdake | which ps --ppid 178 -o pid=? | 14:03 |
| *** inc0 has joined #kolla | 14:04 | |
| sdake | hey inc0 | 14:04 |
| SamYaple | root 178 0.0 0.0 45544 4416 ? S 13:56 0:00 /bin/ovsdb-client monitor Interface name,ofport,external_ids --format=json | 14:04 |
| inc0 | hello | 14:04 |
| sdake | why do you think its breaking the agent it didn't return an error | 14:04 |
| SamYaple | 14:03 < SamYaple> Exit code: 1 | 14:05 |
| SamYaple | are you saying that is a good error code? | 14:05 |
| * SamYaple doesnt know | 14:05 | |
| SamYaple | its normally bad | 14:05 |
| sdake | i am not sure | 14:06 |
| sdake | it should say "ERRROR" in big caps if there is a problem | 14:06 |
| SamYaple | lol | 14:07 |
| SamYaple | not neutron | 14:07 |
| sdake | what happens when you run ps --ppid 178 -o pid= | 14:07 |
| sdake | and type echo $? | 14:07 |
| *** unicell1 has joined #kolla | 14:16 | |
| *** unicell has quit IRC | 14:17 | |
| *** stvnoyes has joined #kolla | 14:17 | |
| sdake | running ainsible now samyaple | 14:18 |
| SamYaple | ok | 14:18 |
| SamYaple | ovs is being wierd | 14:18 |
| SamYaple | not sure if its ovs thats the issue though | 14:18 |
| sdake | man canonical never giong to give up on juju | 14:19 |
| sdake | got to hand it to them, they got thier ass handed to them by heat and docker and still keep trying ;-) | 14:19 |
| SamYaple | they really like juju | 14:20 |
| sdake | juju had the same vision docker has executed | 14:20 |
| vincent_vdk | anyone who has actually used it? | 14:20 |
| sdake | like 2 years before docker existed | 14:20 |
| sdake | juju is highly unreliale | 14:20 |
| SamYaple | i tried it | 14:20 |
| SamYaple | it is bad | 14:20 |
| vincent_vdk | :) | 14:20 |
| sdake | its a packaging format | 14:21 |
| vincent_vdk | had to deal with Ubuntu today | 14:21 |
| sdake | think what docker is | 14:21 |
| vincent_vdk | it;s a bit the same i guess | 14:21 |
| sdake | but the thing is, it doesn't have a docker ecosystem | 14:21 |
| *** diogogmt has joined #kolla | 14:22 | |
| openstackgerrit | James McCarthy proposed stackforge/kolla: Swift Containers https://review.openstack.org/202171 | 14:23 |
| sdake | master no bueno | 14:25 |
| SamYaple | ? | 14:25 |
| sdake | still get binding_failed | 14:27 |
| sdake | after adding /lib/modules to hte bindmounts | 14:27 |
| sdake | no errors in the ovs agent | 14:27 |
| SamYaple | yup | 14:27 |
| SamYaple | i think i have a thread on the issue | 14:28 |
| * SamYaple checked all the leftover uncommited code | 14:28 | |
| *** diogogmt has quit IRC | 14:29 | |
| sdake | paste diff? | 14:29 |
| SamYaple | im not copying any of the l3_agent.ini dhcp_agent_ini metadata_agent.ini files ;) | 14:29 |
| SamYaple | i mean i am in the uncommited branch... but not on master xD | 14:29 |
| sdake | can you git diff | paste? | 14:30 |
| sdake | can't you I mean | 14:30 |
| SamYaple | why am i git diffiing? | 14:30 |
| sdake | you aid you had it fixed in your branch? | 14:31 |
| SamYaple | in the original neutron branch | 14:31 |
| SamYaple | i never git-reviewed the final commit i did | 14:31 |
| sdake | my question is do you have a solution to nova booting or are we back in debug mode | 14:32 |
| *** weiyu has quit IRC | 14:32 | |
| SamYaple | i dont know im testing | 14:32 |
| SamYaple | do you see a clear problem? | 14:32 |
| SamYaple | because i dont | 14:32 |
| sdake | just the vif thing | 14:33 |
| sdake | other then that, I dont see anything helpful | 14:33 |
| SamYaple | thats not a clear problem :P | 14:33 |
| SamYaple | vif just means go check neutron | 14:33 |
| sdake | ml2.managers failed to bind port | 14:34 |
| sdake | the "physical network" is "None" in the debug logs | 14:35 |
| sdake | that could possibly be a problem | 14:35 |
| SamYaple | are you doing a flat network? | 14:36 |
| *** weiyu has joined #kolla | 14:37 | |
| sdake | i am diong whatever the default is for init-runonce | 14:38 |
| SamYaple | i dont know what that is so find out if you are doign a flat | 14:38 |
| SamYaple | flat is not configured | 14:38 |
| sdake | tools/init-runonce | 14:38 |
| sdake | its the thing that sets up the network | 14:38 |
| SamYaple | ok. as a flat or vlan? | 14:38 |
| sdake | says flat | 14:39 |
| SamYaple | yea thats not configured | 14:39 |
| SamYaple | you cant use the same interface for flat and vlan networks | 14:39 |
| SamYaple | vlan is whats configured, thats half the issue | 14:39 |
| SamYaple | for oyu at least | 14:39 |
| *** mickt has quit IRC | 14:39 | |
| sdake | so should i search and replace flat with vlan? | 14:40 |
| SamYaple | nah ill switch the default to flat ot match existing | 14:40 |
| SamYaple | ive got a patch incoming | 14:40 |
| sdake | ok | 14:40 |
| openstackgerrit | Mark Duggan proposed stackforge/kolla: Adding cinder source image https://review.openstack.org/205086 | 14:41 |
| SamYaple | ugh this config stuff is horrible | 14:41 |
| sdake | i noticed ovs-vswitch has a bunch of not found issues | 14:41 |
| SamYaple | i really need to clean it up | 14:41 |
| sdake | like its not finding taps and stuff | 14:41 |
| SamYaple | when you purge your environment, do you `rmmod openvswitch`? | 14:42 |
| sdake | no | 14:42 |
| SamYaple | oyu should | 14:42 |
| sdake | ok will do | 14:42 |
| SamYaple | itll cleanup all thos interfaces | 14:42 |
| *** weiyu has quit IRC | 14:48 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed stackforge/kolla: Make variables in OVS container configurable https://review.openstack.org/204939 | 14:49 |
| sdake | samyapel the init-runonce script creates a flat lan and a vxlan | 14:52 |
| SamYaple | yep thats normal | 14:52 |
| sdake | flat for public | 14:52 |
| sdake | and vxlan for private | 14:52 |
| SamYaple | were close. this new patch will fix more config issues | 14:55 |
| SamYaple | still ahvent booted | 14:55 |
| sdake | cool well get it up and I'll pull it down and help debug | 14:56 |
| *** jasonsb has quit IRC | 15:00 | |
| *** jasonsb has joined #kolla | 15:00 | |
| *** jasonsb has quit IRC | 15:05 | |
| SamYaple | hey sdake did you see ML thread about neutron and the docker plugins? | 15:08 |
| sdake | yes i did | 15:08 |
| SamYaple | i need more info | 15:08 |
| sdake | i think they want to do what we are doing with linuxbridge and ovs | 15:08 |
| SamYaple | lol | 15:08 |
| sdake | except for all the agents | 15:08 |
| SamYaple | done | 15:08 |
| sdake | there are a bunch of proprietary agents as well | 15:09 |
| SamYaple | I woudl expect more response to people who already do it... but no one has replied :( | 15:09 |
| sdake | I was waiting for openstack namesapce to get into a battle over it | 15:10 |
| sdake | if they want to duplicate the work wfm ;) | 15:10 |
| sdake | I really dont care - waste of their time but its their time to allocate | 15:10 |
| openstackgerrit | Paul Bourke proposed stackforge/kolla: Adding cinder source image https://review.openstack.org/205086 | 15:11 |
| SamYaple | they even said they dont want to reinvent the wheel | 15:11 |
| *** diogogmt has joined #kolla | 15:11 | |
| SamYaple | and yet... | 15:11 |
| sdake | openstack is a busy space not everyone knows what everyone else is working on | 15:11 |
| SamYaple | but i said we did it like 8 months ago, and no one responded :( | 15:12 |
| sdake | they are going to do what they are going to do | 15:15 |
| sdake | saying its already done probably isn't going to stop them from reinventing the wheel ;) | 15:15 |
| sdake | and here its pure wheel reinvention - ther eisn't much wiggle room on containers | 15:16 |
| pbourke | I would try and let them know more clearly | 15:16 |
| pbourke | they probably have no idea, given a lot of the work is still in progress | 15:17 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Properly use all configs with neutron https://review.openstack.org/205112 | 15:22 |
| SamYaple | sdake: that doesnt solve the issue. i know i know whtat this is... | 15:22 |
| SamYaple | anyway configs still need configing | 15:22 |
| sdake | hey guys | 15:23 |
| sdake | kolla has 1000 commits ! | 15:23 |
| bmace | happy 1000 commits day kolla! you need a cake at the midcycle for that :) | 15:23 |
| sdake | yes i'm informing now pbourke | 15:23 |
| SamYaple | I GOT the 1000th! | 15:23 |
| *** pradk has joined #kolla | 15:23 | |
| rhallisey | nice! | 15:24 |
| sdake | no jenkins got the 1000th ;) | 15:24 |
| SamYaple | alright thats it for me | 15:24 |
| SamYaple | time to step down | 15:24 |
| sdake | sam | 15:24 |
| bmace | nite sam | 15:24 |
| sdake | real quick | 15:24 |
| sdake | any debug tip on neutron | 15:24 |
| sdake | you said you may hae an idea waht the problem is ? | 15:24 |
| SamYaple | oh im not leaving | 15:24 |
| SamYaple | i was making a joke at daneyon expense | 15:24 |
| sdake | oh | 15:25 |
| SamYaple | ive still got more debug left in me | 15:25 |
| daneyon | congrats!!! | 15:26 |
| SamYaple | we have the power to summon him at will people! | 15:26 |
| SamYaple | lets not abuse it | 15:26 |
| rhallisey | lol | 15:26 |
| bmace | just say daneyon 3 times? :) | 15:27 |
| SamYaple | once seems to do it | 15:27 |
| daneyon | https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMzMg8zI8cYCFUoviAodagcCtw&url=http%3A%2F%2Fwww.scriptmag.com%2Ffeatures%2Fspecs-city-trickster-archetype-beetlejuice&ei=EwixVYz3M8reoATqjoi4Cw&bvm=bv.98476267,d.cGU&psig=AFQjCNHr9iF6KpdIil-yrCpn46lO1SAujg&ust=1437751689260058 | 15:28 |
| pbourke | can somebody else +2 https://review.openstack.org/#/c/201665/ please | 15:28 |
| SamYaple | daneyon: come fix neutron for us. vif_binding issues | 15:32 |
| openstackgerrit | Merged stackforge/kolla: Glance installation from source https://review.openstack.org/201665 | 15:34 |
| daneyon | SamYaple when I ran into that before, neutron-api was not actually reading the conf file. Have you enabled debug logging on neutron-api and actually observe the config file being loaded and read? I fixed the issue my specifying the conf file in the exec. Like here: https://github.com/stackforge/kolla/blob/stable/juno/docker/neutron/neutron-server/start.sh#L98 | 15:38 |
| SamYaple | pretty srue we are doign that, ill check | 15:39 |
| SamYaple | yup we specify it on the command line | 15:40 |
| SamYaple | all the agents work ust fine | 15:40 |
| SamYaple | nova just vif_bind fials. no neutron bits have errors | 15:40 |
| SamYaple | ugh wait. dont libvirt and nova_compute need to share volumes? | 15:42 |
| SamYaple | they do | 15:42 |
| *** bmace has quit IRC | 15:46 | |
| sdake | hrm | 15:49 |
| sdake | they need to share an /etc directory! | 15:49 |
| sdake | and its networking related | 15:49 |
| SamYaple | no they dont | 15:49 |
| SamYaple | nova-net only | 15:49 |
| sdake | oh | 15:50 |
| sdake | they need to share /var/lib/nova, /var/lib/libvirt, and possibly some /etc/ directory for nova | 15:50 |
| SamYaple | sdake: they don't need to share libvirt | 15:51 |
| SamYaple | /var/lib/nova/instances only | 15:51 |
| SamYaple | /etc stuffs for nova-net | 15:52 |
| SamYaple | but thats not whats causing the vif issues anyway | 15:52 |
| sdake | so is it accurate that youdu works and can boot a nova vm? | 15:52 |
| SamYaple | on the 0.1 tag, yea | 15:52 |
| openstackgerrit | Merged stackforge/kolla: Removes Host Cleanup from Container Cleanup Script https://review.openstack.org/182493 | 15:52 |
| SamYaple | but that hanst been done in months | 15:53 |
| SamYaple | docker probably broke it by now | 15:53 |
| sdake | one thing we can try is implement linuxbridge and use that ;-) | 15:54 |
| SamYaple | i have no experince with it in containers :( i dont know what would be needed from config-external to make it work | 15:54 |
| sdake | i was getting vif binding errors without the /etc dir shared in the past | 15:54 |
| sdake | but not sure if I tried with nova net or neutron | 15:55 |
| pbourke | how are Kuryr in the big tent already? | 15:56 |
| sdake | i am pretty sure you need dnsmasq.conf to operate | 15:56 |
| SamYaple | pbourke: i think enutron said yea we want that | 15:56 |
| pbourke | have they been around for a while and keeping quiet or do I have no idea how big tent works | 15:56 |
| sdake | i think i got vif bidning errors there too | 15:56 |
| SamYaple | they said they are neutron big-tent | 15:56 |
| SamYaple | sdake: you dont | 15:57 |
| pbourke | so there are different tents | 15:57 |
| SamYaple | that has nothing to do with vif-binding | 15:57 |
| SamYaple | look at the port sdake | 15:57 |
| SamYaple | neutron port-list | 15:57 |
| sdake | neutron has its own tent structure | 15:57 |
| SamYaple | neutron port-show <portid> | 15:57 |
| SamYaple | when the port gets created it bad | 15:57 |
| SamYaple | all neutron stuffs | 15:57 |
| *** bmace has joined #kolla | 15:57 | |
| sdake | let me try taht with your patch | 15:58 |
| sdake | 10 mins | 15:58 |
| sdake | why is vxlan in 239 address space? | 15:59 |
| SamYaple | thats multicast | 16:00 |
| sdake | yes i know | 16:02 |
| sdake | init-runonce creates a vxlan interface, but its not in the 239 address space | 16:03 |
| sdake | rather a vxlan network | 16:03 |
| * sdake fucking hates networking | 16:03 | |
| SamYaple | thats not what that is | 16:03 |
| SamYaple | ignore it | 16:03 |
| sdake | as my wife says "Cisco didn't build its empire because networking is easy." | 16:03 |
| sdake | samyaple can you ahe a look at init-runonce | 16:04 |
| sdake | and make sure it makes sense for ovs | 16:04 |
| *** inc0 has quit IRC | 16:04 | |
| *** mstachow has quit IRC | 16:07 | |
| SamYaple | 2015-07-23 16:06:32.666 28 DEBUG neutron.plugins.ml2.db [req-ca48af2f-c916-41b5-9085-5c0b5b4840e4 562681f2e69a4d039290610d20b229da 9753a054c6cd4c089973224d00d248f1 - - -] Attempted to set empty binding levels set_binding_levels /usr/lib/ | 16:07 |
| SamYaple | python2.7/site-packages/neutron/plugins/ml2/db.py:164 | 16:07 |
| sdake | that looks problamtic | 16:09 |
| sdake | becaue if levels is defined something is added to sessions.* | 16:10 |
| sdake | whtaever that is | 16:10 |
| sdake | ok guys, debugging neutron and getting nova booting in ansible is job #1 for the next couple of days | 16:14 |
| sdake | if folks could focus on that, I'd appreciate it (until its done :) | 16:14 |
| SamYaple | pff thats not optimistic | 16:14 |
| SamYaple | day | 16:14 |
| SamYaple | few hours at most | 16:14 |
| SamYaple | argggg i think i found it | 16:17 |
| SamYaple | im just a dummy | 16:17 |
| SamYaple | neutron-server isnt reading ml2_conf | 16:17 |
| *** Slower has quit IRC | 16:18 | |
| SamYaple | rebuilding and checking now | 16:18 |
| sdake | i think daneyon suggested that ;-) | 16:18 |
| SamYaple | yea he said neutron.conf | 16:18 |
| sdake | oh | 16:18 |
| SamYaple | actually he didnt say | 16:18 |
| SamYaple | but yea we were reading neutron.conf just fine | 16:18 |
| SamYaple | not ml2_confi.ini | 16:18 |
| SamYaple | i think i remember thinking about this and saying "why owuld neutron_server need this" | 16:19 |
| * SamYaple sighs | 16:19 | |
| sdake | ya doesit need it? | 16:19 |
| SamYaple | apparently | 16:19 |
| SamYaple | the server logs were complaining about ml2 things | 16:19 |
| SamYaple | and thats why the binging was bad | 16:19 |
| SamYaple | binding* | 16:20 |
| SamYaple | 2015-07-23 16:15:02.243 32 DEBUG neutron.plugins.ml2.managers [req-a2563b0a-a643-4b1b-87b3-f7aad444548e 6e77bffc7d1b4f2b9e04273abf8e1c66 a7a80483976a48f49d553a003bfbf0f5 - - -] Attempting to bind port 495c824c-db4e-41ca-980c-6e52c47efc3c | 16:20 |
| SamYaple | on host server02 at level 0 using segmens [{'segmentation_id': None, 'physical_network': None, 'id': u'fc210d08-246c-4f44-ade6-7e39b3f8df1c', 'network_type': u'llocal'}] _bind_port_level /usr/lib/python2.7/site-packages/neutron/plugins/ | 16:20 |
| SamYaple | ml2/managers.py:651 | 16:20 |
| SamYaple | network_type would be ovs if it read ml2_conf.ini | 16:20 |
| SamYaple | but hey this diversion made me see a few little bugs that would be hard to find | 16:21 |
| SamYaple | so not all lost | 16:21 |
| sdake | why are there empty ini files in /etc? | 16:21 |
| sdake | can't we put those in /usr/share? | 16:21 |
| sdake | or is the idea someone shoudl put something in them? | 16:22 |
| sdake | samyaple can you paste a git diff so I can see how you fixed the ml2 config file read | 16:23 |
| *** achanda_ has joined #kolla | 16:26 | |
| *** achanda has quit IRC | 16:26 | |
| *** absubram has joined #kolla | 16:27 | |
| *** unicell1 has quit IRC | 16:28 | |
| SamYaple | sdake i have answered those questions for you many times before | 16:30 |
| SamYaple | lets not worry about them right now | 16:30 |
| SamYaple | they are literally a single ansible variable to change | 16:30 |
| SamYaple | they can be changed later | 16:30 |
| sdake | i guess I dont understand why there are empty files | 16:31 |
| sdake | and you haven't explained that one ;) | 16:31 |
| SamYaple | i know and ive explained it before | 16:31 |
| SamYaple | i have | 16:31 |
| SamYaple | several times | 16:31 |
| sdake | would you mind explaining one more - i'll pay attention this time | 16:31 |
| sdake | like dhcp_blah.ini | 16:31 |
| SamYaple | they are teh augment files. ansible requires the files to exist to use them for templating. i need to file them with comments and examples for how to use so they arent just blank | 16:32 |
| SamYaple | for now its just blank | 16:32 |
| sdake | I get the empty augementation files - those are meant to be changed | 16:32 |
| SamYaple | dhcp_agent.ini is an autment file as well | 16:32 |
| sdake | i see | 16:32 |
| sdake | ok that wasn't clear | 16:32 |
| SamYaple | all those files are | 16:32 |
| SamYaple | they need comments | 16:32 |
| sdake | ok etc makes sense then | 16:32 |
| sdake | i thought they were needed for implementation purpose | 16:33 |
| sdake | as in, need a empty file to copy from | 16:33 |
| SamYaple | the /usr/share stuff is | 16:33 |
| sdake | in which case they should go in /usr/share | 16:33 |
| SamYaple | but that can move to /var/lib | 16:33 |
| sdake | the current /usr/share shold def go to /var/lib | 16:33 |
| SamYaple | agreed | 16:33 |
| sdake | my only concern is how does someone upgrade the /etc path | 16:33 |
| sdake | when new files are added | 16:33 |
| SamYaple | those files are modified by ansible only | 16:33 |
| SamYaple | the /usr/share ones i mena | 16:34 |
| sdake | yup | 16:34 |
| sdake | i understand how /usr/share and /opt work | 16:34 |
| sdake | 10 hours of debugging neutron, remember ;) | 16:34 |
| sdake | I have good indepth understanding of how ansible works in our implementation now | 16:34 |
| sdake | all the cores should take on a ansible project to understand how the stuff works | 16:35 |
| SamYaple | yay | 16:35 |
| *** Slower has joined #kolla | 16:36 | |
| sdake | i think what would make sense is if a config file isnt' found, our merger treats it as an empty file | 16:38 |
| sdake | since they are all empty anyway | 16:38 |
| sdake | that way ansible doesn't bomb out on kolla upgrades | 16:38 |
| SamYaple | that wont work | 16:39 |
| SamYaple | ansible is the merger | 16:39 |
| SamYaple | it bombs | 16:39 |
| sdake | doesn't it have python code that does the merging? | 16:39 |
| sdake | if file not found - file = '' :) | 16:39 |
| SamYaple | its not like that | 16:39 |
| sdake | well i can tlel upgrades of kolla will be painful | 16:40 |
| SamYaple | this is the best way forlots of reasons. those files need comments | 16:40 |
| SamYaple | no | 16:40 |
| SamYaple | script will update the yamls | 16:40 |
| sdake | you mentioned that prior | 16:40 |
| sdake | i guess that would work but seems fragile | 16:40 |
| SamYaple | its really not | 16:41 |
| SamYaple | python yaml parser | 16:41 |
| sdake | someone very likely not to actuallly run the upgrade script and then bork their install | 16:41 |
| SamYaple | well it wont break their data | 16:41 |
| SamYaple | it doesnt matter anyway. thats they way it is | 16:41 |
| SamYaple | sdake, you know this will all be dont with kolla start, right? | 16:44 |
| SamYaple | a check at the beginning for version/sanity? | 16:44 |
| sdake | that is a workable approach | 16:45 |
| sdake | I hadn't thought of that though, no | 16:45 |
| SamYaple | oh then that explains the disconnect | 16:45 |
| SamYaple | that is why im not worried | 16:45 |
| sdake | ok that wfm | 16:45 |
| SamYaple | i mean yea if they run the ansible stuff dircetly they might break things, but thats thier own fault is my point | 16:46 |
| SamYaple | SUCCESS | 16:47 |
| SamYaple | kinda | 16:47 |
| SamYaple | all ports are creating, but nova_compute needs ovs utils | 16:47 |
| SamYaple | with the next 10 minutes i bet i have a successful boot | 16:49 |
| *** unicell has joined #kolla | 16:55 | |
| sdake | samyaple I used your latest patch + made neutron-server load ml2_conf, port still says unbound | 16:57 |
| sdake | with port-show | 16:57 |
| sdake | rather binding-failed | 16:58 |
| SamYaple | libvirtError: internal error: process exited while connecting to monitor: Could not access KVM kernel module: Permission denied | 17:01 |
| SamYaple | weve solved this already | 17:01 |
| SamYaple | that was root:kvm perm issues right | 17:01 |
| sdake | wow your gtting far | 17:02 |
| sdake | there is a chmod in /dev/kvm in config-internal | 17:03 |
| sdake | but probably not config-external | 17:03 |
| SamYaple | i see it | 17:03 |
| SamYaple | im also not binding /dev into the container | 17:03 |
| sdake | the dev bind doesn't matter | 17:03 |
| sdake | we use it for cinder | 17:03 |
| SamYaple | but i need /dev/kvm | 17:04 |
| sdake | /dev/kvm is already in the container, it just has the wrong permissions | 17:05 |
| SamYaple | hmm | 17:05 |
| SamYaple | thats from devtmpfs | 17:05 |
| sdake | yup | 17:05 |
| sdake | we will want to bindmount /dev - so might as well do that anyway -for cinder | 17:05 |
| *** absubram has quit IRC | 17:06 | |
| SamYaple | are you sure we have to? | 17:06 |
| sdake | not certain, rhallisey claims it is necessary | 17:06 |
| SamYaple | rhallisey: what was the verdict of that? do we have to bind host | 17:06 |
| rhallisey | yes we have to | 17:06 |
| SamYaple | i refuse! | 17:07 |
| rhallisey | without it I was having issues | 17:07 |
| SamYaple | hmm ok so bind /dev from host | 17:09 |
| SamYaple | so the issue wit hthat is will the group 'kvm' in the container match the guid on the outside of the container? | 17:10 |
| SamYaple | that seems dangerous | 17:10 |
| *** dims_ has quit IRC | 17:12 | |
| SamYaple | what was the reasoning for not tweaking the libvirt conf to run as root in the contaienr? | 17:12 |
| sdake | no reasoning | 17:14 |
| sdake | but ya the gid needs to match - agree its subotpimal as is | 17:15 |
| sdake | although libvirt really shouldn't run as root either | 17:15 |
| SamYaple | when we get around to dropping CAP itll be fine | 17:15 |
| SamYaple | i mean the whole contain is privileged anyway | 17:15 |
| SamYaple | conatiner* | 17:15 |
| sdake | right | 17:15 |
| SamYaple | guid and uid mapping is coming for dokcer | 17:16 |
| sdake | can you paste an updated patch | 17:17 |
| sdake | or do a new git erview | 17:17 |
| SamYaple | im working on it | 17:17 |
| sdake | i want to see if there is something wrong with my environment that wont create ports | 17:17 |
| sdake | since i think I've done what you did | 17:17 |
| SamYaple | yay its working | 17:18 |
| sdake | booted a vm? | 17:18 |
| SamYaple | yep | 17:18 |
| sdake | vm have network connectivity? | 17:18 |
| SamYaple | no idea | 17:18 |
| SamYaple | im going to up the patch | 17:19 |
| sdake | probably not without the dnsmasq setup | 17:19 |
| sdake | cool i'll test it here | 17:19 |
| SamYaple | dnsmasq works fine dammit | 17:19 |
| SamYaple | that config file is just extra | 17:19 |
| SamYaple | we need to add it but it wont break things | 17:19 |
| sdake | in linuxbridge I had to set dnsmasq to 1450 to get outside vm connectivity to work | 17:21 |
| sdake | becauseof the vxlan overhead | 17:21 |
| SamYaple | you dont have to | 17:21 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Fix compute permissions and start params https://review.openstack.org/205183 | 17:21 |
| sdake | i definately had to do so | 17:21 |
| SamYaple | you do not | 17:21 |
| SamYaple | im telling you | 17:21 |
| SamYaple | it just fragments | 17:21 |
| SamYaple | i know the issues and you can work around it | 17:22 |
| SamYaple | without dnsmasq | 17:22 |
| *** coolsvap|away is now known as coolsvap | 17:22 | |
| sdake | is that the ony patch we should apply samyaple | 17:23 |
| SamYaple | sdake youll need https://review.openstack.org/#/c/205112/ and https://review.openstack.org/#/c/205183/ | 17:23 |
| sdake | or also apply your previous one? | 17:23 |
| sdake | ok | 17:23 |
| SamYaple | yea i forgot to depend them | 17:23 |
| SamYaple | on a clean enironment it gest stuck on BUILDING | 17:28 |
| SamYaple | probably some external configuration i forgot | 17:28 |
| *** absubram has joined #kolla | 17:31 | |
| openstackgerrit | James McCarthy proposed stackforge/kolla: Swift Containers https://review.openstack.org/202171 | 17:33 |
| *** Slower has quit IRC | 17:34 | |
| SamYaple | have you tried it yet sdake? | 17:37 |
| sdake | buidling | 17:38 |
| sdake | slow network | 17:38 |
| sdake | in a vm | 17:38 |
| SamYaple | awesome | 17:38 |
| sdake | over wireless | 17:38 |
| sdake | just finished cmopute now for libvirt ;-) | 17:41 |
| jmccarthy | baseline swift looks to be close (it was working!), seems like some new config dependencies got in for proxy server somehow, hopefully it'll be sorted tomorrow (and some other stuff can be bugged up from the todo) | 17:41 |
| sdake | jmccarthy if your in for an adventure, you could add ansible support for swift ;-) | 17:42 |
| openstackgerrit | Harm Weites proposed stackforge/kolla: Don't try to chown root:root /tmp/kolla_mariadb_cluster https://review.openstack.org/205195 | 17:43 |
| jmccarthy | Hehe :) I would give it a whirl but some of the supporting services might need sorting out first ? Depends I guess - I'll be coordinating with Paul tomorrow as I'm away on holidays next week | 17:44 |
| sdake | nice enjoy the holiday | 17:44 |
| sdake | im getting ready to go hiking myself here shortly | 17:44 |
| jmccarthy | Don't bring any devices with you ;) | 17:45 |
| sdake | i'll take my phone in case i get lost | 17:45 |
| harmw | guys, please look at that review up above :) nothing big, but still | 17:45 |
| *** dims has joined #kolla | 17:45 | |
| SamYaple | i reviewed it harmw | 17:45 |
| SamYaple | that was for security reasons | 17:45 |
| jmccarthy | Hehe ok phone is allowed | 17:45 |
| SamYaple | and running as root is required for all kinds of kolla things | 17:45 |
| harmw | ansible is doing it's tricks as root, ofcourse | 17:46 |
| jmccarthy | laters o/ | 17:46 |
| harmw | but I am running ansible, as me | 17:46 |
| SamYaple | thats not how that works harmw | 17:46 |
| sdake | at minimum yo uwould hae ot run ansible as the kolla user | 17:46 |
| sdake | so it could write to /var/lib and /usr/share and whatnot | 17:47 |
| SamYaple | there is no kolla user | 17:47 |
| sdake | i recognize it | 17:47 |
| sdake | what i'm saying is it could be done without root | 17:47 |
| harmw | I'm just running as me sdake, which works great | 17:47 |
| harmw | ansible_ssh_user=root | 17:47 |
| harmw | thats all | 17:47 |
| SamYaple | harmw: for now, but it wont when everything is in place | 17:47 |
| harmw | but $USER is just me | 17:47 |
| SamYaple | we chown lots of things | 17:47 |
| harmw | uhm yes, we do, but on the support and cluster nodes | 17:48 |
| harmw | right? | 17:48 |
| SamYaple | yea but this is a local_action | 17:48 |
| SamYaple | look im ok with the change but sdake threw a security fit over it | 17:48 |
| harmw | indeed, which takes place on the operator node | 17:48 |
| sdake | i only care if the file is written as root :) | 17:49 |
| harmw | in that case it should be chmodded to 600 aswell | 17:49 |
| SamYaple | sdake: it never was | 17:49 |
| SamYaple | i explained that | 17:49 |
| harmw | it's written by $USER | 17:49 |
| harmw | so not root, atleast not here :p | 17:50 |
| SamYaple | harmw: the file is set as 0600 | 17:50 |
| harmw | oh ok, excuse that | 17:50 |
| harmw | I thought I saw a 660 there | 17:50 |
| SamYaple | honestly i never saw this a a big security thing. the only issue was i used shell original which is not best practice anyway | 17:51 |
| harmw | I'm not seeing it either, all I know is I need my patch in order to have ansible deploy the database | 17:52 |
| sdake | what is written by user? | 17:53 |
| harmw | that file, to /tmp | 17:53 |
| sdake | I thought it was written by ansible as root | 17:53 |
| harmw | /tmp/kolla_mariadb_cluster | 17:53 |
| SamYaple | sdake: only if you run ansible as root | 17:53 |
| harmw | it's writing the file and doing a chown to root | 17:53 |
| SamYaple | so for your AIO stuff you are running ansible as root | 17:54 |
| sdake | dont we ned to run ansible as root? | 17:54 |
| SamYaple | only for AIO | 17:54 |
| harmw | SamYaple: AHA | 17:54 |
| harmw | now that makes sense | 17:54 |
| harmw | and I, ofcourse, am playing with multihost :) | 17:54 |
| SamYaple | AIO doesnt connect over ssh | 17:54 |
| SamYaple | yes | 17:54 |
| sdake | can we make aio connect over ssh? | 17:54 |
| sdake | then it wouldn't have to run as root? | 17:54 |
| SamYaple | that would require additional setup | 17:55 |
| sdake | running ansible atm | 17:55 |
| sdake | t-3 minutes | 17:55 |
| SamYaple | you would have to make a passwordless ssh key for the current user to ssh to root on the local host | 17:55 |
| sdake | oh i did that already - i thought that was mandatory ;-) | 17:55 |
| harmw | so in it's current form you run as root and deploy using root, creating a file /tmp and chown to root (which is redundant) | 17:56 |
| SamYaple | harmw: dont complain to me. i had to fight to fix what i thought wasnt a big deal | 17:56 |
| harmw | all because AiO works on localhost, without ssh/sudo whatever | 17:56 |
| harmw | SamYaple: I never complain | 17:56 |
| harmw | only to sdake :) | 17:56 |
| harmw | I didn't know about the consideration regarding AiO :) | 17:56 |
| harmw | I'm just seeing an issue when trying to play with multinode | 17:57 |
| sdake | dont blame me blame the hackers | 17:57 |
| sdake | and the NSA | 17:57 |
| SamYaple | but theres nothing to hack is my point | 17:57 |
| SamYaple | at the VERY WORST you could potentially start the database on the wrong host | 17:57 |
| harmw | plus, since AiO runs as root already (albeight just for now) I see no value in that additional chown to root (which ultimatly screws my multinode testing) | 17:57 |
| SamYaple | thats the worst | 17:58 |
| harmw | chmod 600 is already in place, as it should | 17:58 |
| harmw | I see no harm in accepting that patch :) | 17:58 |
| sdake | listen here is the deal - I dont care but as soon as people start distributing our software they will audit it for security red flags | 17:59 |
| sdake | so we will have to respond to those as they come in | 17:59 |
| sdake | i'd prefer not to have any audits come back negatively as it affects the project negatively | 17:59 |
| SamYaple | sdake: as i tried to explain before. there is no security risk here | 18:00 |
| SamYaple | i would argue this makes it less secure | 18:00 |
| harmw | I'm with SamYaple on this one :) | 18:01 |
| sdake | i haven't looked at the patch -busy atm | 18:01 |
| sdake | hang tight | 18:01 |
| SamYaple | sdake: i see the issue that i missed | 18:03 |
| SamYaple | dont approve that patch just yet | 18:03 |
| SamYaple | why its stuck in building i mean | 18:04 |
| sdake | say when we do upgrade, arew e going to make a site-upgrade.yml file? | 18:05 |
| sdake | and have it basically do the same thing except force bootstrapping? | 18:06 |
| SamYaple | probably, but not quite used like that | 18:06 |
| SamYaple | `kolla upgrade` | 18:06 |
| sdake | right | 18:06 |
| SamYaple | it will have logics in it specific to that version of the upgrade | 18:06 |
| sdake | but that will run a site-upgrade.yml playbook? | 18:06 |
| SamYaple | maybe | 18:06 |
| SamYaple | i dont know. it will probably differ with each openstack upgrade too | 18:06 |
| sdake | any time there is a new container we need to rerun the db migration | 18:07 |
| SamYaple | the upgrade process changes | 18:07 |
| SamYaple | no | 18:07 |
| SamYaple | the dont do db changes between versions | 18:07 |
| SamYaple | they* | 18:07 |
| sdake | on master ? sure do | 18:07 |
| SamYaple | on master yea | 18:07 |
| SamYaple | thats not stable | 18:07 |
| sdake | the 2015.1.1 thing is going byebye | 18:08 |
| sdake | people are going to maintain a stable branch and backport to it | 18:08 |
| SamYaple | yea but tags are sticking around | 18:08 |
| sdake | no more tagging | 18:08 |
| SamYaple | so db upgraede new tag, yes not new container | 18:08 |
| SamYaple | lol yea theres still tagging sdake | 18:08 |
| SamYaple | its just whatever tagging scheme the project wants | 18:09 |
| SamYaple | unstead of openstack-wide | 18:09 |
| sdake | ya tagging only at db schema changes | 18:09 |
| sdake | we are having a design sesision on upgrade I guess we can tackle it then | 18:09 |
| SamYaple | thats probably best | 18:09 |
| SamYaple | i dont think anyone has thought it all out yet | 18:10 |
| SamYaple | i know i havent | 18:10 |
| sdake | error state | 18:11 |
| sdake | port status = active tho! | 18:12 |
| SamYaple | w00t | 18:13 |
| SamYaple | got ACTIVE again | 18:13 |
| SamYaple | going to test ping | 18:13 |
| SamYaple | no response :( | 18:14 |
| SamYaple | going to check the console | 18:14 |
| SamYaple | oh wait | 18:15 |
| SamYaple | response! | 18:15 |
| SamYaple | secgroup rules | 18:15 |
| SamYaple | yay | 18:15 |
| sdake | i get "libvirtError: internal error: no supported architecture for os type 'hvm' | 18:15 |
| SamYaple | youre running in a vm | 18:15 |
| sdake | i set qemu | 18:15 |
| SamYaple | idk what else youd have to set | 18:15 |
| sdake | wierd my libvirt settings went byebye | 18:16 |
| sdake | i must have erased it accidentally | 18:16 |
| SamYaple | probably | 18:16 |
| sdake | how do I set that again? | 18:16 |
| SamYaple | going to submit patchset t2o | 18:16 |
| SamYaple | [libvirt] | 18:17 |
| SamYaple | virt_type=qemu | 18:17 |
| SamYaple | i think | 18:17 |
| SamYaple | in /etc/kolla/config/nova/nova-compute.conf | 18:17 |
| sdake | lets see if rerunning ansible will configure the new value | 18:18 |
| SamYaple | it wont | 18:18 |
| SamYaple | i dont have the 'if changed config restart container' logic in yet | 18:19 |
| SamYaple | plus the config method would prevent that anyway | 18:19 |
| SamYaple | but you can rm the container | 18:19 |
| SamYaple | thgat woudl work | 18:19 |
| sdake | ya it doesn't | 18:20 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Fix compute permissions and start params https://review.openstack.org/205183 | 18:20 |
| SamYaple | das what got me bootin | 18:20 |
| sdake | is that a new patch or a rebase of an existing patch | 18:21 |
| SamYaple | rebase | 18:21 |
| SamYaple | er update | 18:21 |
| SamYaple | can someone ack this https://review.openstack.org/#/c/204567/ | 18:22 |
| *** Slower has joined #kolla | 18:22 | |
| *** Slower has quit IRC | 18:22 | |
| *** Slower has joined #kolla | 18:22 | |
| harmw | done | 18:23 |
| SamYaple | danke | 18:23 |
| harmw | sdake: can you ack those other neutron bits? | 18:23 |
| harmw | and what to do with my ansible patch :p | 18:24 |
| *** jtriley has quit IRC | 18:24 | |
| harmw | SamYaple: you mind me changing your ansible_ssh_user=sam line to root? | 18:24 |
| sdake | i am going to test the neutron bits | 18:24 |
| harmw | one less file that needs tinkering with in demos | 18:24 |
| sdake | then ack them | 18:24 |
| SamYaple | sdake: that would be recommended | 18:25 |
| *** stvnoyes has quit IRC | 18:28 | |
| sdake | running ansible twice breaks things | 18:29 |
| SamYaple | lies | 18:30 |
| harmw | lol no sdake ofcourse not | 18:30 |
| SamYaple | i consistently run it twice in a row | 18:30 |
| SamYaple | its habit now | 18:30 |
| harmw | thats the beauty | 18:30 |
| harmw | https://www.youtube.com/watch?v=7AlEvy0fJto woohoo! | 18:30 |
| *** jtriley has joined #kolla | 18:30 | |
| SamYaple | sdake: good respons from the ML | 18:30 |
| SamYaple | my name holds no wieght | 18:31 |
| *** stvnoyes has joined #kolla | 18:35 | |
| openstackgerrit | Merged stackforge/kolla: Removed unused hautoproxy code https://review.openstack.org/204567 | 18:36 |
| sdake | well I'd probably win a popularity contest | 18:37 |
| sdake | but so what :) | 18:37 |
| SamYaple | depends on whos there | 18:37 |
| harmw | who's winning? | 18:38 |
| harmw | sdake: done with neutron? | 18:38 |
| sdake | nearly | 18:38 |
| sdake | almost ready to boot | 18:38 |
| harmw | 20:24:40 < harmw> SamYaple: you mind me changing your ansible_ssh_user=sam line to root? | 18:38 |
| sdake | so my public network is not being created samyaple | 18:39 |
| sdake | only the demo network | 18:39 |
| sdake | from init-runonce | 18:39 |
| *** absubram has quit IRC | 18:39 | |
| harmw | hm, is there no matching physnet sdake ? | 18:39 |
| SamYaple | harmw: changing it to what? | 18:39 |
| harmw | to root | 18:39 |
| harmw | (or completely removing it) | 18:40 |
| SamYaple | yea, cause this is all meant to be sudo'd and not run directly as root | 18:40 |
| SamYaple | were just not there yet | 18:40 |
| sdake | vm went active yay ;-) | 18:40 |
| SamYaple | :) | 18:40 |
| harmw | nice sdake | 18:41 |
| SamYaple | i told you it would be done today | 18:41 |
| harmw | and SamYaple, also | 18:41 |
| harmw | now | 18:41 |
| harmw | ack! | 18:41 |
| harmw | and I'll start pushing again :p | 18:41 |
| harmw | and run some multinode tests | 18:41 |
| SamYaple | harmw: you are pushing from the L1 tagged branch right? | 18:41 |
| sdake | console-log gets stuck at sending discover... | 18:41 |
| harmw | no SamYaple latest | 18:41 |
| harmw | sdake: thats not stuck, its dhcp that fails | 18:42 |
| SamYaple | uhh thats not what were supposed to be doing | 18:42 |
| SamYaple | we only push 3 times a cycle | 18:42 |
| harmw | we do? | 18:42 |
| SamYaple | thats why we had the stablility push for L1 | 18:42 |
| SamYaple | yea | 18:42 |
| harmw | thats not what sdake told me... | 18:42 |
| harmw | so sry, I guess | 18:42 |
| SamYaple | oh then im on a differnet page | 18:42 |
| SamYaple | when i was pushing it was supposed to be from L1 | 18:43 |
| SamYaple | like i said half this stuff is broken | 18:43 |
| sdake | ya we have l1 l2 l3 | 18:43 |
| harmw | I thought we were just pushing master to latest every now and then | 18:43 |
| sdake | ya i guess we should hae pushed l1 | 18:43 |
| sdake | oh well | 18:43 |
| sdake | l1 was never pushed iirc | 18:43 |
| SamYaple | correct | 18:43 |
| harmw | hm, so I did bad | 18:43 |
| SamYaple | we can just hold off and push L2 | 18:44 |
| harmw | indeed | 18:44 |
| SamYaple | i mean honestly | 18:44 |
| harmw | close anyway | 18:44 |
| SamYaple | L2 should have a working ansible deploy too | 18:44 |
| sdake | samyaple dhcp no worky | 18:44 |
| SamYaple | sdake: ill look at init-runonce | 18:44 |
| sdake | but looking much better so i'll approve the two changes | 18:44 |
| sdake | yes please do | 18:44 |
| sdake | this stuff is all black magic to me | 18:44 |
| SamYaple | dhcp works for me ;) | 18:44 |
| sdake | daneyon wrote that script | 18:44 |
| SamYaple | im going to hammer on multinode now | 18:45 |
| SamYaple | hopeflly no changes | 18:45 |
| sdake | init-runonce also doesn't create the public network | 18:45 |
| sdake | our public network is called public, but the compose has physnet1 iirc | 18:45 |
| harmw | yep | 18:45 |
| sdake | so we shoudl probably change that | 18:45 |
| SamYaple | sdake: no | 18:45 |
| SamYaple | thats not what that is | 18:45 |
| SamYaple | i wish i could transfer my network knowlege into your brain | 18:46 |
| sdake | me too | 18:46 |
| SamYaple | yea that neutron line wont work | 18:46 |
| sdake | samyaple i'm going off what harmw said | 18:46 |
| sdake | it works on compose ;-) | 18:47 |
| harmw | wait wut | 18:47 |
| harmw | what'd I say? | 18:47 |
| SamYaple | i mean it _can_ work but i have to change the defaults | 18:47 |
| SamYaple | but we can just drop that param | 18:47 |
| SamYaple | its not required | 18:47 |
| SamYaple | then it woudl work for both i suppose | 18:47 |
| sdake | i dont know which your tlaking about | 18:47 |
| SamYaple | --provider:physical_network physnet1 | 18:48 |
| SamYaple | thats not required | 18:48 |
| harmw | it isn't? | 18:48 |
| SamYaple | well not required when there is only one mapping | 18:48 |
| harmw | uhm yes | 18:48 |
| SamYaple | um no | 18:48 |
| harmw | was about to say that :) | 18:48 |
| SamYaple | its not required for two either | 18:48 |
| SamYaple | you just cant select the one you want ;) | 18:48 |
| SamYaple | --provider:network_type vxlan --provider:segmentation_id 10 | 18:49 |
| SamYaple | also not required | 18:49 |
| harmw | but if we leave it out, then how it works? | 18:49 |
| SamYaple | magic! | 18:49 |
| harmw | it won't select vxlan by itself? | 18:49 |
| harmw | ah | 18:49 |
| SamYaple | it will | 18:49 |
| harmw | we have defaults set? | 18:49 |
| SamYaple | no it selects tenant network types when not explicitly set | 18:50 |
| SamYaple | vxlan is the only one configured | 18:50 |
| harmw | yep | 18:50 |
| SamYaple | cause gre be bad | 18:50 |
| harmw | thats default enough for me | 18:50 |
| harmw | :p | 18:50 |
| SamYaple | but again, if would select one for you if you dont specify | 18:50 |
| harmw | got it | 18:51 |
| harmw | did today's fixes merge yet sdake? :p | 18:51 |
| SamYaple | just rebuilt all containers on fresh host | 18:51 |
| SamYaple | going to reattmept | 18:51 |
| harmw | I'll be doing that as well, later tonight | 18:51 |
| harmw | once all is merged :P | 18:51 |
| SamYaple | i need to test multinode | 18:52 |
| sdake | they hae been approved but haven't gone through zuul yet | 18:52 |
| harmw | SamYaple: you could you Vagrant for that | 18:52 |
| SamYaple | ugh no | 18:53 |
| harmw | why not | 18:53 |
| SamYaple | i have real hardware for multihost :P | 18:53 |
| SamYaple | plus i do most of my testing on Openstack | 18:53 |
| harmw | ah | 18:53 |
| SamYaple | openstack on openstack on openstack | 18:53 |
| harmw | I don't have that luxury :p | 18:53 |
| SamYaple | ive done it, dont laugh | 18:53 |
| SamYaple | yea well i paid out for this luxury | 18:54 |
| SamYaple | it didnt come cheap | 18:54 |
| harmw | ehe | 18:54 |
| *** bitblt has joined #kolla | 18:55 | |
| SamYaple | oh jeez | 18:56 |
| SamYaple | lol sdake dhcp is busted on a clean host | 18:56 |
| SamYaple | let me figure out why | 18:56 |
| sdake | fix init-runonce while yoru at it iplz | 18:57 |
| SamYaple | i dont wanna | 18:57 |
| SamYaple | it aint broke | 18:57 |
| openstackgerrit | Merged stackforge/kolla: Fix compute permissions and start params https://review.openstack.org/205183 | 18:57 |
| openstackgerrit | Merged stackforge/kolla: Properly use all configs with neutron https://review.openstack.org/205112 | 18:57 |
| openstackgerrit | Harm Weites proposed stackforge/kolla: [WIP] Vagrantfile and docs https://review.openstack.org/204238 | 18:58 |
| *** bitblt has quit IRC | 18:58 | |
| SamYaple | sdake: nvm dhcp works fine. i setup the network with --disable-dhcp | 19:00 |
| sdake | try with init-runonce | 19:01 |
| SamYaple | no thats busted | 19:01 |
| sdake | I would really like one network setup script | 19:01 |
| sdake | so that eveyrone is doing the same thing re neutron | 19:01 |
| sdake | you said above its nto busted | 19:02 |
| SamYaple | but i am not going to use that... | 19:02 |
| sdake | you dont have to use it | 19:02 |
| SamYaple | i said dhcp is not busted | 19:02 |
| sdake | but i use it all the time | 19:02 |
| sdake | and most other devs do as well | 19:02 |
| SamYaple | yea but that doesnt even use the proper client | 19:02 |
| sdake | [11:57:09] <sdake>fix init-runonce while yoru at it iplz | 19:02 |
| sdake | [11:57:20] <SamYaple>i dont wanna | 19:02 |
| sdake | [11:57:27] <SamYaple>it aint broke | 19:02 |
| sdake | you said right there it aint broke | 19:03 |
| SamYaple | for compose ;) | 19:03 |
| SamYaple | and you can change the neutron config to make it wrok.... | 19:03 |
| sdake | i would like one script for both models if that is at all possible | 19:03 |
| SamYaple | it is | 19:03 |
| SamYaple | im already working on it | 19:03 |
| SamYaple | BUT | 19:03 |
| sdake | I dont know *how* to do it | 19:03 |
| sdake | thanks :) | 19:03 |
| SamYaple | if you want it done you file a bug | 19:03 |
| sdake | will file | 19:04 |
| SamYaple | good. i hate that lol | 19:04 |
| harmw | so the init-runonce differs between ansible and compose? | 19:04 |
| SamYaple | nah just the bridge_mapping name | 19:04 |
| SamYaple | sdake could just update his config but eh | 19:05 |
| harmw | then the fix is to use the same name in both setups.. right? | 19:05 |
| SamYaple | no | 19:05 |
| SamYaple | its not not call the name out explicitly | 19:05 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1477725 | 19:05 |
| openstack | Launchpad bug 1477725 in kolla "init-runonce doesn't work with ansible neutron" [High,Confirmed] - Assigned to Sam Yaple (s8m) | 19:05 |
| SamYaple | that init-runonce is a basics barebones script not meant to be run in a production deployment | 19:05 |
| sdake | its absolutely a development only tool | 19:06 |
| sdake | an argument could be made we need something to get people rolling that is production worthy | 19:06 |
| sdake | but that isn't a priority atm ;-) | 19:06 |
| sdake | rhallisey how is tripleo integration coming along | 19:08 |
| rhallisey | sdake, it going | 19:08 |
| rhallisey | it's going | 19:08 |
| sdake | using config external or config internal? | 19:08 |
| harmw | sdake: which containers need rebuilding after those merges? | 19:08 |
| sdake | harmw neutron* nova* | 19:09 |
| harmw | ok | 19:09 |
| sdake | and the ovs* I thiink ;-) | 19:09 |
| harmw | ah | 19:09 |
| harmw | nova-compute and neutron* | 19:09 |
| sdake | and nova-libvirt | 19:09 |
| harmw | so nova* | 19:10 |
| harmw | fair enough :p | 19:10 |
| sdake | theere may be a base file edit in there, i dont recall | 19:10 |
| SamYaple | i need to change the keystone client to openstack client in init-runonce | 19:10 |
| sdake | i just rebuilt em all | 19:10 |
| SamYaple | keystone is deprecated and unusable | 19:10 |
| sdake | samyaple sounds good | 19:10 |
| sdake | the main reason i need init-runonce is because i am going to implement heat in ansible | 19:11 |
| *** achanda_ has quit IRC | 19:14 | |
| rhallisey | sdake, external | 19:17 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Update init-runonce https://review.openstack.org/205247 | 19:19 |
| SamYaple | sdake: that should fix it. youll need to rekick since i updated the br-ex to match compose for consistency | 19:19 |
| SamYaple | physnet1 | 19:20 |
| *** apuimedo has joined #kolla | 19:27 | |
| apuimedo | SamYaple: sdake: Hi! | 19:28 |
| apuimedo | Thanks for reaching out! | 19:28 |
| *** gfidente has quit IRC | 19:29 | |
| SamYaple | hi! antoni? | 19:29 |
| apuimedo | Yup, that's me | 19:30 |
| SamYaple | welcome! | 19:30 |
| harmw | something networking, our way has come | 19:30 |
| apuimedo | may the source be with us | 19:31 |
| harmw | peering established | 19:31 |
| harmw | nice :p | 19:31 |
| SamYaple | thats definetly better than 'may the fork be with us' | 19:31 |
| harmw | I always like spoons better | 19:31 |
| apuimedo | I prefer to chew at the food like an animal :-) | 19:32 |
| SamYaple | well apuimedo you have caught us at a good time. the ansible bits have just landed in place so a complete and working openstack environment comes up using Ansible now | 19:32 |
| apuimedo | specially when the pig leg is starting to be quite finished | 19:32 |
| apuimedo | s/pig/pork/ | 19:33 |
| bmace | i like that one apuimedo. i usually go with just "use the source" | 19:33 |
| apuimedo | SamYaple: that's great to hear | 19:33 |
| apuimedo | can you point me to it | 19:33 |
| apuimedo | we use ansible quite a bit | 19:33 |
| SamYaple | https://github.com/stackforge/kolla/tree/master/ansible | 19:34 |
| apuimedo | and we are very much looking forward to have a kolla based deployment | 19:34 |
| SamYaple | https://github.com/stackforge/kolla/blob/master/docs/ansible-deployment.md | 19:34 |
| apuimedo | oh, so it's in master | 19:34 |
| apuimedo | I thought it would be a feature branch :P | 19:34 |
| harmw | aren't we all in master | 19:34 |
| SamYaple | hah as if! | 19:34 |
| SamYaple | the docs will get you started, i hope you dont mind bleeding edge ;) | 19:35 |
| apuimedo | there is only one thing when I tried kolla that left me puzzled | 19:35 |
| apuimedo | a month ago | 19:35 |
| SamYaple | sure | 19:35 |
| apuimedo | that it automatically set up the compose for restart | 19:35 |
| apuimedo | so every reboot my laptop would start all of kolla :P | 19:35 |
| SamYaple | youll have to talk to sdake about that. i also dont like that ;) | 19:35 |
| harmw | systemctl disable docker :p | 19:36 |
| SamYaple | its configurable for ansible | 19:36 |
| apuimedo | I hope that by deleting all the yml of compose I finally got rid of it | 19:36 |
| apuimedo | harmw: +1 for using systemd | 19:36 |
| harmw | hah | 19:36 |
| SamYaple | as if we have a choice these days | 19:36 |
| harmw | he SamYaple how exactly do you use a custom docker registry | 19:36 |
| apuimedo | there is only one init and poettering is its prophet | 19:37 |
| harmw | talking usecase here | 19:37 |
| harmw | yep apuimedo | 19:37 |
| SamYaple | apuimedo: i actually do like systemd, I just wish it was more modular | 19:37 |
| apuimedo | harmw: what do you mean custom docker registry? | 19:37 |
| harmw | running my own registry | 19:37 |
| SamYaple | we support private registries | 19:37 |
| harmw | to push the new images to, so they can easily be fetched from the other cluster nodes | 19:37 |
| apuimedo | I come from Red Hat, I was spreading the word to my older sys admins :P | 19:37 |
| SamYaple | harmw: https://github.com/stackforge/kolla/blob/master/etc/kolla/globals.yml#L43-49 | 19:38 |
| harmw | but thats for loading SamYaple , right? | 19:38 |
| harmw | pulling | 19:38 |
| apuimedo | SamYaple: kolla is all rdo based for now, right? | 19:38 |
| SamYaple | you mean for pushing? | 19:38 |
| SamYaple | apuimedo: acutally we have many distros now | 19:38 |
| SamYaple | i beleive ubuntu is almost done | 19:38 |
| SamYaple | we currently only gate rdo | 19:39 |
| apuimedo | oh, it seems I missed quite a bit in the last month | 19:39 |
| SamYaple | we have come a long way yes | 19:39 |
| apuimedo | too busy setting up Kuryr and working on Juju | 19:39 |
| harmw | SamYaple: the workflow is probably building and pulling from dockerhub, then push to on premise registry | 19:39 |
| harmw | right? | 19:39 |
| harmw | and have other nodes pull from internal registry | 19:39 |
| SamYaple | I actually expect most people to build thier own images harmw | 19:39 |
| apuimedo | SamYaple: for networking plugins, what is your idea, to make separate image names | 19:39 |
| apuimedo | or just tag differentiation? | 19:40 |
| harmw | I do, but never pushed to internal registry | 19:40 |
| SamYaple | apuimedo: i think we can get by with simply installing the plugin in the existing container. Then configuration would be handled by ansible after the build | 19:40 |
| apuimedo | modify the container after the build? | 19:41 |
| SamYaple | apuimedo: oh im sorry, i meant for nova container. the nova bits can be dropped in the existing nova container | 19:41 |
| SamYaple | neutron will get its own neutron-midonet-agent container | 19:41 |
| apuimedo | you mean the nova-compute container? | 19:42 |
| SamYaple | yes | 19:42 |
| *** athomas has quit IRC | 19:43 | |
| apuimedo | SamYaple: how I had thought about it was that neutron-server would have different builds for the different plugins | 19:43 |
| apuimedo | then | 19:43 |
| apuimedo | make a nova-compute-vendorname for each of the vendors | 19:44 |
| SamYaple | we can certainly go that path if needed. we do break up the neutron plugin agents so far we havent had a need to split of nova-compute | 19:45 |
| SamYaple | https://github.com/stackforge/kolla/tree/master/docker/centos/binary/neutron | 19:45 |
| apuimedo | SamYaple: it's very good that you split the neutron agents | 19:45 |
| apuimedo | since some vendors do not use some of them ;-) | 19:46 |
| SamYaple | yes so we could easily add neutron-midonet-agent | 19:46 |
| SamYaple | i am worried a bit about spliting compute so as not to violate DRY | 19:46 |
| SamYaple | but if we do it as a seperate layer, that may work well | 19:46 |
| apuimedo | the midonet agent has to run on the nova-compute | 19:46 |
| apuimedo | think of it as the same as ovs | 19:47 |
| apuimedo | nova-compute, when talking to libvirt to start a VM | 19:47 |
| apuimedo | tells libvirt to start the container with a specific virtual port uuid | 19:47 |
| apuimedo | libvirt then calls ovs-vsctl | 19:47 |
| apuimedo | for midonet it's the same, but libvirt calls mm-ctl | 19:48 |
| apuimedo | SamYaple: does nova-compute run on the --net=host? | 19:48 |
| SamYaple | apuimedo: all containers do | 19:48 |
| SamYaple | for performance reasons | 19:48 |
| apuimedo | oh, that's great | 19:48 |
| apuimedo | well, in that case | 19:48 |
| apuimedo | midonet-host-agent can run completely standalone | 19:48 |
| *** stvnoyes has quit IRC | 19:49 | |
| SamYaple | currently we install openvswitch innside nova-compute container itself, i can see the benefit to spliting that off | 19:49 |
| SamYaple | oh ok then thats great too | 19:49 |
| apuimedo | we could just have nova-compute have the mm-ctl binary and that's it | 19:49 |
| SamYaple | thats perfect | 19:49 |
| SamYaple | its exactly what we do with openvswitch | 19:49 |
| SamYaple | opevswitch is a seperate container | 19:49 |
| apuimedo | midonet-host-agent would receive all the data it needs from zookeeper | 19:49 |
| apuimedo | :-) | 19:49 |
| SamYaple | i love it when a plan comes together | 19:49 |
| apuimedo | :-) | 19:50 |
| SamYaple | we can likely implement this in record time to | 19:50 |
| apuimedo | I think so | 19:50 |
| apuimedo | as soon as I can spare myself some time or of some of my team | 19:50 |
| SamYaple | i can wrok with you on the ansible bits. those are still in flux | 19:50 |
| apuimedo | :-) | 19:50 |
| apuimedo | great | 19:50 |
| SamYaple | would you care to start a blueprint for adding midonet? | 19:51 |
| SamYaple | perhaps some kolla folks can assist as well | 19:51 |
| harmw | hm? | 19:51 |
| apuimedo | I'll have to change our binary binding packaging | 19:51 |
| apuimedo | sure thing | 19:51 |
| apuimedo | I'll do that! | 19:51 |
| SamYaple | have we achieved synergy? | 19:51 |
| SamYaple | is this what that feels like? | 19:51 |
| apuimedo | SamYaple: should I create it in launchpad blueprints or send the spec to gerrit or both? | 19:51 |
| SamYaple | apuimedo: a lunchpad blueprint should be fine | 19:52 |
| SamYaple | thats all we have done for most containers | 19:52 |
| apuimedo | good | 19:52 |
| SamYaple | honestly midonet doesnt sound much different from OVS from an implementation standpoint | 19:52 |
| SamYaple | so this should be easy-peasy | 19:53 |
| apuimedo | SamYaple: https://blueprints.launchpad.net/kolla | 19:53 |
| apuimedo | SamYaple: it's simpler, since it doesn't need the dhcp agent | 19:53 |
| SamYaple | tahts the one | 19:53 |
| apuimedo | and soon it won't need the metadata agent | 19:53 |
| SamYaple | you had my curiosity, but now you have my attention | 19:53 |
| apuimedo | we only will need to have a custom neutron-api that selects the right plugin, of course | 19:53 |
| apuimedo | SamYaple: well, I'm pushing for reducing moving parts | 19:54 |
| apuimedo | and having them closer to the VMs | 19:54 |
| SamYaple | so neutron-server-midonet? | 19:54 |
| apuimedo | I guess so | 19:54 |
| SamYaple | is this not a selectable plugin with neutron-server? | 19:54 |
| apuimedo | neutron.conf has to be different | 19:54 |
| SamYaple | that can be handled with ansible | 19:55 |
| SamYaple | if its just the conf the container doesnt need to change | 19:55 |
| apuimedo | and have a midonet pypi package installed | 19:55 |
| SamYaple | thats fine | 19:55 |
| apuimedo | networking-midonet | 19:55 |
| apuimedo | alright then | 19:55 |
| apuimedo | that makes me happy | 19:55 |
| apuimedo | I don't want separate containers | 19:55 |
| apuimedo | if possible | 19:55 |
| SamYaple | as long as the container can work normally with the package installed thats great | 19:55 |
| SamYaple | as in if we install midonet, we can still use OVS thats perfectly fine | 19:56 |
| apuimedo | sure, of course it can ;-) | 19:56 |
| SamYaple | then perfect we are good to go on our end | 19:56 |
| apuimedo | soon we'll even have an ML2 driver | 19:56 |
| apuimedo | to make it even easier | 19:56 |
| SamYaple | even better | 19:56 |
| apuimedo | how does the ansible part work, for configuration? | 19:58 |
| SamYaple | id rather not say... :) | 19:58 |
| apuimedo | it puts ansible agents on the containers? | 19:58 |
| SamYaple | its still being flushed out | 19:58 |
| SamYaple | no | 19:58 |
| *** dims has quit IRC | 19:58 | |
| apuimedo | glad to read that :P | 19:58 |
| SamYaple | we drop the configs in a location on the host | 19:58 |
| apuimedo | of course | 19:59 |
| apuimedo | volumes | 19:59 |
| SamYaple | we bind mount them in the container and it copies them in to fix the permissions | 19:59 |
| apuimedo | I'm too hungry | 19:59 |
| apuimedo | and my brain lacks fodder | 19:59 |
| apuimedo | I know that what I'm about to say is a bit crazy | 19:59 |
| apuimedo | but, have we considered having nova-docker as well? | 19:59 |
| SamYaple | lol i like you apuimedo | 20:00 |
| SamYaple | i have indeed | 20:00 |
| SamYaple | i even have some code to support it | 20:00 |
| SamYaple | but there is a bit of an issue with the nova-docker driver being... bad | 20:00 |
| apuimedo | good | 20:00 |
| apuimedo | SamYaple: heh, I have some experience with it | 20:00 |
| apuimedo | I was debugging it a bit for Juno for some customer | 20:00 |
| apuimedo | :P | 20:00 |
| SamYaple | having the driver running in the container and managing that same docker host is troublesome | 20:00 |
| apuimedo | had to backport a couple of things | 20:00 |
| apuimedo | it could | 20:01 |
| apuimedo | there's always the docker on docker :P | 20:01 |
| SamYaple | thats what im saying | 20:01 |
| SamYaple | we share the docker socket into the container | 20:01 |
| SamYaple | then bam | 20:01 |
| apuimedo | no, I meant to use another docker inside | 20:02 |
| SamYaple | oh | 20:02 |
| SamYaple | yea... no | 20:02 |
| apuimedo | I've not tried it | 20:02 |
| SamYaple | lol | 20:02 |
| SamYaple | that would have lots of namespace issues | 20:02 |
| apuimedo | but it sounds to me like it could be hacked out | 20:02 |
| *** vinkman has joined #kolla | 20:02 | |
| apuimedo | I've heard of people using docker on lxc | 20:02 |
| *** stvnoyes has joined #kolla | 20:02 | |
| SamYaple | plus then the data would be very not persistent, which might be ok i guess | 20:02 |
| apuimedo | (not seen it, so my belief on it is limited) | 20:03 |
| SamYaple | let me put it this way, i am open to all working implementations | 20:03 |
| SamYaple | whatever those may be | 20:03 |
| apuimedo | SamYaple: I did worn it would sound crazy | 20:03 |
| apuimedo | s/worn/warn/ | 20:03 |
| SamYaple | haha yea i was talking about it a few days back in the channel | 20:03 |
| SamYaple | it is going to happen i promise you | 20:03 |
| SamYaple | just maybe later rather than sooner | 20:04 |
| apuimedo | I have the suspicion that nova-docker (if it had more devs) | 20:04 |
| SamYaple | wait for things to stablize | 20:04 |
| apuimedo | could be hacked enough to share the docker server | 20:04 |
| SamYaple | thats the key, it would have to | 20:04 |
| SamYaple | if it could then Kolla wouldnt hurt the instances | 20:05 |
| SamYaple | kolla can coexist with teh containers | 20:05 |
| apuimedo | ;-) | 20:06 |
| apuimedo | gotta go for a while, talk to you later or tomorrow ;-) | 20:06 |
| SamYaple | looking forward to it | 20:06 |
| *** apuimedo is now known as apuimedo|away | 20:06 | |
| *** apuimedo|away is now known as apuimedo|dinner | 20:07 | |
| *** vinkman has left #kolla | 20:07 | |
| *** rhallisey has quit IRC | 20:28 | |
| *** stvnoyes has quit IRC | 20:34 | |
| *** bmace has quit IRC | 20:38 | |
| *** pradk has quit IRC | 20:41 | |
| *** bmace has joined #kolla | 20:50 | |
| *** shardy has quit IRC | 20:54 | |
| sdake | yo | 20:55 |
| sdake | well that was a good power nap | 21:00 |
| *** blahRus has joined #kolla | 21:01 | |
| *** jtriley has quit IRC | 21:02 | |
| openstackgerrit | Harm Weites proposed stackforge/kolla: [WIP] Vagrantfile and docs https://review.openstack.org/204238 | 21:03 |
| openstackgerrit | Sam Yaple proposed stackforge/kolla: Update init-runonce https://review.openstack.org/205247 | 21:05 |
| SamYaple | good news sdake | 21:05 |
| SamYaple | blank brand new host with that patch works | 21:05 |
| SamYaple | you are welcome | 21:05 |
| sdake | it didn't work previously? | 21:05 |
| SamYaple | the init patch i mean | 21:06 |
| SamYaple | i tested it for you | 21:06 |
| SamYaple | init-runonce | 21:06 |
| SamYaple | anyway its late im going to bed | 21:06 |
| SamYaple | tanksfor all the help | 21:06 |
| SamYaple | we did accomplish alot today | 21:06 |
| sdake | we solved a binary equation | 21:07 |
| sdake | not working -> working | 21:07 |
| *** jtriley_ has joined #kolla | 21:15 | |
| *** absubram has joined #kolla | 21:15 | |
| *** jtriley__ has joined #kolla | 21:45 | |
| *** jtriley_ has quit IRC | 21:49 | |
| *** achanda has joined #kolla | 21:49 | |
| sdake | samyaple dhcp no worky with master | 21:53 |
| *** jtriley__ has quit IRC | 21:54 | |
| *** achanda has quit IRC | 22:23 | |
| *** absubram has quit IRC | 22:26 | |
| *** dims has joined #kolla | 22:39 | |
| *** dwalsh has quit IRC | 22:48 | |
| *** coolsvap is now known as coolsvap|away | 23:00 | |
| *** diogogmt has quit IRC | 23:04 | |
| *** weiyu has joined #kolla | 23:06 | |
| *** ashuk has joined #kolla | 23:10 | |
| *** weiyu has quit IRC | 23:23 | |
| *** dims_ has joined #kolla | 23:24 | |
| *** dims has quit IRC | 23:27 | |
| *** jruano has quit IRC | 23:34 | |
| *** blahRus has quit IRC | 23:45 | |
| *** dims_ has quit IRC | 23:46 | |
| *** weiyu has joined #kolla | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!