| *** marmat has joined #kolla | 00:00 | |
| *** marmat has quit IRC | 00:01 | |
| *** salv-orl_ has joined #kolla | 00:02 | |
| *** dims_ has joined #kolla | 00:03 | |
| *** dims has quit IRC | 00:04 | |
| *** salv-orlando has quit IRC | 00:05 | |
| *** salv-orl_ has quit IRC | 00:07 | |
| *** phuongnh has quit IRC | 00:09 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add PBR generation of version information https://review.openstack.org/296124 | 00:12 |
|---|---|---|
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 00:12 |
| sdake | reno is pretty sweet | 00:13 |
| *** Marga_ has quit IRC | 00:14 | |
| *** Marga_ has joined #kolla | 00:15 | |
| mlima | sdake where can i find some doc about kolla achitecture, deployment strategy ? | 00:16 |
| sdake | mlima actually taht is next up on my plate after i finish with heat doa bug | 00:16 |
| sdake | mainly for the threat nalysis for we can obtaint he VMT tag | 00:16 |
| sdake | but as of now there is no such document | 00:17 |
| mlima | ok | 00:17 |
| mlima | no problem | 00:17 |
| mlima | i have an idea | 00:17 |
| mlima | "add support for many network interfaces" | 00:18 |
| *** achanda has quit IRC | 00:18 | |
| mlima | but I'm still thinking about the best way to do this | 00:19 |
| *** achanda has joined #kolla | 00:21 | |
| mlima | not even know if this idea makes sense for you haha | 00:21 |
| *** weiyu_ has joined #kolla | 00:22 | |
| *** aginwala has quit IRC | 00:28 | |
| *** unicell has joined #kolla | 00:31 | |
| *** aginwala has joined #kolla | 00:32 | |
| *** achanda has quit IRC | 00:40 | |
| *** alisonh has quit IRC | 00:43 | |
| *** sdake has quit IRC | 00:50 | |
| *** sdake has joined #kolla | 00:51 | |
| sdake | dave-mccowan https://bugs.launchpad.net/kolla/+bug/1560744 | 00:56 |
| openstack | Launchpad bug 1560744 in kolla mitaka "drop root for crontab" [Critical,Confirmed] - Assigned to Steven Dake (sdake) | 00:56 |
| *** daneyon has quit IRC | 01:00 | |
| *** unicell has quit IRC | 01:00 | |
| *** alisonh has joined #kolla | 01:01 | |
| *** alisonh has quit IRC | 01:09 | |
| *** achanda has joined #kolla | 01:11 | |
| *** alisonh has joined #kolla | 01:13 | |
| *** phuongnh has joined #kolla | 01:14 | |
| *** aginwala has quit IRC | 01:16 | |
| *** aginwala has joined #kolla | 01:19 | |
| *** mlima has quit IRC | 01:20 | |
| *** daneyon has joined #kolla | 01:29 | |
| sdake | this needs fixing https://bugs.launchpad.net/kolla/+bug/1560751 | 01:30 |
| openstack | Launchpad bug 1560751 in kolla mitaka "logs are not all owned by the heka user" [High,Confirmed] | 01:30 |
| *** Marga_ has quit IRC | 01:30 | |
| *** ayoung has joined #kolla | 01:32 | |
| SamYaple | sdake: gotta disagree with your take on that bug | 01:33 |
| *** salv-orlando has joined #kolla | 01:35 | |
| *** daneyon has quit IRC | 01:35 | |
| *** daneyon has joined #kolla | 01:36 | |
| sdake | the problem is the logs are all 660 | 01:36 |
| sdake | they should be 600 and owned by heka only | 01:36 |
| sdake | the only reason they are readale is the permissions heka applies to the logs is difficient | 01:37 |
| SamYaple | heka doesnt own them | 01:37 |
| SamYaple | the only thing that is supose to match here is the group | 01:37 |
| SamYaple | and it does | 01:37 |
| SamYaple | thats the design | 01:37 |
| sdake | i see | 01:37 |
| sdake | well the logs should definately be 640 not 660 | 01:37 |
| SamYaple | and they should defintely group writes | 01:37 |
| SamYaple | nope | 01:37 |
| SamYaple | group writes are needed | 01:37 |
| SamYaple | group perms are how we address all of these | 01:38 |
| sdake | isn't it heka that writes teh log files out? | 01:38 |
| SamYaple | no | 01:38 |
| sdake | we dont want one container overwritng the logs of anotehr container in a seucrity situation | 01:38 |
| SamYaple | heka only writes out the logs for things it has a socket for (haproxy/keepalived) | 01:39 |
| SamYaple | but the group writes are needed for the cron container | 01:39 |
| SamYaple | tis the design we have | 01:39 |
| sdake | the cron container has to go | 01:39 |
| sdake | cron cannot be run as nonroot | 01:39 |
| SamYaple | cron is not going. it is also needed | 01:39 |
| SamYaple | otherwise logs grow and grow and grow.... | 01:40 |
| sdake | it needs a replacement that can run as non root | 01:40 |
| sdake | i understand the need for logrotate | 01:40 |
| *** dims_ has quit IRC | 01:40 | |
| sdake | crond will only run as root | 01:41 |
| sdake | on startup it attempts seteuid (0) | 01:41 |
| sdake | i gues if the logs need to be 660 thats fine | 01:42 |
| sdake | if heka isnt writign the logs itself | 01:42 |
| SamYaple | not to worried. crond has no external interface | 01:42 |
| *** salv-orlando has quit IRC | 01:42 | |
| SamYaple | no entry point to copromise | 01:42 |
| sdake | crontab -e ?? | 01:42 |
| sdake | files on the filesystem? | 01:42 |
| sdake | it hs plenty of external entrypoints | 01:42 |
| SamYaple | like what? | 01:43 |
| sdake | one attack vector of crontab is put whatevery u ike in the cro ndirectories | 01:43 |
| SamYaple | how? | 01:43 |
| SamYaple | that could be said for anything | 01:44 |
| SamYaple | or you could replace /run_command | 01:44 |
| sdake | that file is owned by root | 01:44 |
| SamYaple | so is the cron directories | 01:44 |
| sdake | you must surely admit crontab running as nonroot would be preferable to crontab running as root | 01:46 |
| *** jasonsb has joined #kolla | 01:46 | |
| SamYaple | anything as non-root is preferable with no privs is preferable | 01:47 |
| SamYaple | i just think we have better things to focus on than non-security risks | 01:47 |
| SamYaple | like haproxy drop root | 01:47 |
| SamYaple | would love to see that done right | 01:47 |
| sdake | i guess we will find out with our threat analysis | 01:47 |
| *** jasonsb has quit IRC | 01:54 | |
| *** haplo37 has joined #kolla | 02:03 | |
| *** vhosakot has joined #kolla | 02:08 | |
| *** haplo37 has quit IRC | 02:13 | |
| *** iceyao has joined #kolla | 02:24 | |
| *** iceyao has quit IRC | 02:25 | |
| *** iceyao has joined #kolla | 02:31 | |
| sdake | asalkeld you around | 02:33 |
| asalkeld | Hi sdake | 02:33 |
| sdake | hey quick python question | 02:33 |
| sdake | i've got kolla package which has releasenotes/source/conf.py | 02:34 |
| sdake | i am getting error no module kolla.versions found | 02:34 |
| sdake | ifi instll kolla it of course works | 02:34 |
| sdake | but it can't seem to find it in the python conf.py file | 02:34 |
| sdake | is there some trick to make it find it? | 02:34 |
| sdake | nova seems to suffer from the same problem | 02:35 |
| asalkeld | Add __init__.py ? | 02:35 |
| sdake | let me try that | 02:35 |
| sdake | same deal | 02:36 |
| sdake | heat also suffers from this problem | 02:37 |
| asalkeld | It's from doc/source right? | 02:37 |
| sdake | its in releaenotes/* | 02:37 |
| sdake | releaesnotes/source | 02:37 |
| asalkeld | I see, not sure ... | 02:37 |
| asalkeld_ | sdake: has kolla been installed at this point? | 02:39 |
| sdake | it is not installed | 02:39 |
| sdake | if I install it, it works | 02:39 |
| *** haplo37 has joined #kolla | 02:39 | |
| asalkeld_ | so if you need kolla.versions, you need kolla in the python path somewhere | 02:40 |
| asalkeld_ | how is this run? | 02:40 |
| sdake | sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html | 02:40 |
| sdake | but i want to run it from tox | 02:40 |
| asalkeld_ | PYTHON_PATH=... | 02:40 |
| sdake | when i run from tox it works.. | 02:40 |
| sdake | when it runs from tox in the gate, it does not work.. | 02:41 |
| *** salv-orlando has joined #kolla | 02:41 | |
| asalkeld_ | well tox installs the kolla package right? | 02:41 |
| asalkeld_ | well that's odd | 02:42 |
| sdake | agree it is odd | 02:42 |
| sdake | i just added rm to the whitelist going to try thato ut | 02:42 |
| sbezverk | sdake I tested iscsi patch on brand new master and it works. I do see some issues with swift/heka but I do not think it is related.. | 02:42 |
| sdake | without those /udev bindmounts? | 02:43 |
| sbezverk | I must have missed your comment about udev | 02:44 |
| sbezverk | trying to find it to see what you meant.. | 02:46 |
| openstackgerrit | Merged openstack/kolla: Check relevant logging mechanism in the gate https://review.openstack.org/295909 | 02:47 |
| *** tfukushima has joined #kolla | 02:47 | |
| sbezverk | sdake got it.. /lib/udev is required | 02:48 |
| sbezverk | especially for nova | 02:48 |
| * asalkeld_ bbl - going to gym | 02:48 | |
| sbezverk | nova when it starts an instance uses scsi_id binary | 02:48 |
| sbezverk | which is located there | 02:48 |
| sdake | why not use the scsi_id in the container? | 02:49 |
| *** salv-orlando has quit IRC | 02:51 | |
| sdake | groan that is part of the systemd package | 02:51 |
| sdake | why on earth would they put that dependnecy in such a bloated package | 02:51 |
| *** aginwala has quit IRC | 02:51 | |
| sdake | sbezverk try installing systemd in the container and then running without the udev bindmount | 02:52 |
| sdake | bloated packaging in containers is better then bindmounts ;) | 02:52 |
| sbezverk | sdake seriously?!?!? I would think having systemd is much worse than just use ro bindmount.. | 02:53 |
| sdake | bindmounts are a security problem - the less we have the better | 02:53 |
| sbezverk | :-( | 02:53 |
| sdake | and in the case of running binarie they dont deliver a immutable experience | 02:54 |
| sdake | security is pretty much the only thing I dont negotiate on in patches :) | 02:54 |
| sbezverk | sdake I do not know what impact will be on everything from installing systemd | 02:55 |
| sdake | try it | 02:55 |
| *** iceyao has quit IRC | 02:55 | |
| sbezverk | also where do you want me to add it? base or nova base? | 02:55 |
| sdake | just install to ge tth ebinary | 02:55 |
| sdake | in the containers it is needed in only | 02:56 |
| sdake | to pull in the iscsi_id binary | 02:56 |
| sbezverk | ok then just for nova compute | 02:56 |
| sbezverk | ok I will take care of it tomorrow | 02:57 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 03:04 |
| *** iceyao has joined #kolla | 03:07 | |
| sdake | https://bugs.launchpad.net/kolla/+bug/1560779 | 03:12 |
| openstack | Launchpad bug 1560779 in kolla "heka elasticsearch is busted" [High,Confirmed] | 03:12 |
| vhosakot | wow, I need working ES+heka to write new lua parser for libvirt.. is this on Ubuntu or centos | 03:18 |
| *** yuanying has quit IRC | 03:21 | |
| *** yuanying_ has joined #kolla | 03:21 | |
| *** yuanying_ has quit IRC | 03:24 | |
| *** yuanying has joined #kolla | 03:24 | |
| *** Marga_ has joined #kolla | 03:26 | |
| *** yuanying has quit IRC | 03:29 | |
| *** aginwala has joined #kolla | 03:29 | |
| *** Marga_ has quit IRC | 03:30 | |
| *** alisonh has left #kolla | 03:31 | |
| *** aginwala has quit IRC | 03:33 | |
| sdake | sbezverk https://bugs.launchpad.net/kolla/+bug/1560788 | 03:37 |
| openstack | Launchpad bug 1560788 in kolla mitaka "heka and swift are busted together" [Critical,Confirmed] | 03:37 |
| sdake | this is master t-24 hours ago | 03:37 |
| *** haplo37 has quit IRC | 03:39 | |
| *** Marga_ has joined #kolla | 03:39 | |
| *** Marga_ has quit IRC | 03:40 | |
| *** Marga_ has joined #kolla | 03:40 | |
| *** iceyao_ has joined #kolla | 03:40 | |
| sbezverk | sdake so heka got screwed up for last 24 hours? | 03:41 |
| sbezverk | I meant during last 24 hours | 03:41 |
| sdake | no it was probably broken prio to that | 03:41 |
| sbezverk | sdake it complains about syntax error, but since I am not strong in this syntax I cannot spot what exactly wrong | 03:43 |
| sbezverk | syntax error: last token: { pos: 27 | 03:43 |
| *** iceyao has quit IRC | 03:43 | |
| sbezverk | could it be that it requires double {{}} ?? | 03:44 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add PBR generation of version information https://review.openstack.org/296124 | 03:45 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 03:45 |
| sdake | let me go take a bio and have a smoke | 03:46 |
| sdake | and i'll look at it | 03:46 |
| sdake | when was the last time it was working for you - prior to your rebase? | 03:48 |
| sdake | sbezverk ^^ | 03:50 |
| sbezverk | sdake just before rebase swift was working perfectly fine | 03:52 |
| sdake | gold up to 1249 | 03:52 |
| sdake | I knew buying 100 ozes when it was at 1000 was a good idea ;-) | 03:52 |
| sdake | what file conflicted in the rebase | 03:52 |
| sbezverk | man you are lucky, after renovation you still have something keft to buy gold?!? | 03:52 |
| sdake | my ennovation was only 16k | 03:53 |
| sbezverk | sdake ansible/roles/common/tasks/config.yml | 03:53 |
| *** coolsvap has joined #kolla | 03:53 | |
| sdake | 0a1486a2c73275009f7158c911e7bfd683e40a58 | 03:54 |
| sdake | let me try reverting that change | 03:54 |
| sbezverk | yep, that was the one.. I notced it added these swift related statements | 03:56 |
| *** tfukushima has quit IRC | 03:57 | |
| *** jtriley has quit IRC | 03:57 | |
| sdake | we need a storage gate | 03:58 |
| sbezverk | sdake who hosts servers acting as gates? | 03:59 |
| sdake | cloud copanies | 03:59 |
| sbezverk | sdake I have 5108 blade server chassis with bunch of server, sitting in my lab idle ;-) | 04:01 |
| *** yuanying has joined #kolla | 04:01 | |
| sdake | hey yuanying how goes it bro | 04:01 |
| sbezverk | but because it so damn noisy and eats lots of electricity I do not use it much | 04:02 |
| yuanying | hi sdake what's up? | 04:03 |
| sdake | release time | 04:03 |
| sdake | busy busy | 04:03 |
| yuanying | haha | 04:04 |
| sdake | you working on magnum these days? | 04:04 |
| yuanying | yes, but furthermore my boss orderd me another works | 04:04 |
| sdake | why | 04:04 |
| sdake | barbican dependency cluster foxtrot ?: ) | 04:05 |
| yuanying | foxtrot? | 04:05 |
| *** achanda has quit IRC | 04:06 | |
| sdake | sbezverk please read https://bugs.launchpad.net/kolla/+bug/1560788 | 04:06 |
| openstack | Launchpad bug 1560788 in kolla mitaka "heka and swift are busted together" [Critical,Confirmed] | 04:06 |
| sdake | foxtrot is a polite way of swearing | 04:06 |
| yuanying | difficult to understand for me | 04:09 |
| *** salv-orlando has joined #kolla | 04:10 | |
| *** salv-orlando has quit IRC | 04:14 | |
| vhosakot | I need ES+heka working for the new lua parser I'm writing for libvirt.log.. sdake: can I submit the patch wihtout testing (since ES+heka is broken) in lastest Newton | 04:20 |
| sdake | heka broken all over the place | 04:20 |
| sdake | can you file a bug i one isn't alrady filed | 04:20 |
| vhosakot | sdake: https://review.openstack.org/#/c/296012/... there is already a bug https://bugs.launchpad.net/kolla/+bug/1560298 | 04:22 |
| openstack | Launchpad bug 1560298 in kolla mitaka "Make heka collect libvirtd.log in nova_libvirt container" [Critical,New] - Assigned to Vikram Hosakote (vhosakot) | 04:22 |
| *** jasonsb has joined #kolla | 04:22 | |
| vhosakot | wow, this lua syntax is different! | 04:23 |
| sdake | already a bug for es+heka being bust? | 04:23 |
| sdake | i filed one an hour or so ago | 04:23 |
| sdake | we need to fix the bugs before we introduce new implementations | 04:24 |
| sdake | so we can test without compounding problems | 04:24 |
| sdake | if you want feedback on patch set workflow to -1 | 04:24 |
| vhosakot | sdake: heka, es, kibana... all worked 100% 4 days ago | 04:24 |
| vhosakot | sdake: good point.. I will first -1 myself | 04:24 |
| *** dave-mccowan has quit IRC | 04:24 | |
| sdake | ya try reverting 0a1486a2c73275009f7158c911e7bfd683e40a58 | 04:24 |
| vhosakot | sdake: ok, let me do it | 04:25 |
| *** weiyu_ has quit IRC | 04:26 | |
| sdake | commit f4175f04 may be promatic too | 04:30 |
| vhosakot | sdake: was the es container up when you saw this bug https://bugs.launchpad.net/kolla/+bug/1560779 | 04:31 |
| openstack | Launchpad bug 1560779 in kolla "heka elasticsearch is busted" [High,Confirmed] | 04:31 |
| sdake | no idea i just ran docker logs heka on it | 04:31 |
| *** coolsvap has quit IRC | 04:31 | |
| *** coolsvap has joined #kolla | 04:32 | |
| vhosakot | ok, the others logs look good.. only es logs failed to get indexed/decoded.. 2016/03/21 20:34:06 Plugin 'elasticsearch_output' error: can't index: HTTP request failed: Post http://192.168.1.148:9200/_bulk: dial tcp 192.168.1.148:9200: no route to host.. | 04:33 |
| vhosakot | will look into this bug... I need this working to implement lua parser for libvirt.log | 04:33 |
| sdake | i am trying a ever t- will let you know in 10 mins | 04:37 |
| vhosakot | ok | 04:41 |
| *** aginwala has joined #kolla | 04:41 | |
| *** aginwala has quit IRC | 04:43 | |
| openstackgerrit | Merged openstack/kolla: Enable horizon domain logins https://review.openstack.org/296052 | 04:44 |
| sdake | vhosakot how late are you styaing up tonight | 04:48 |
| vhosakot | another 30 mins may be.... I am trying to reach a stage where I can start writing the lua parser | 04:49 |
| vhosakot | why u ask ? | 04:49 |
| *** aginwala has joined #kolla | 04:49 | |
| *** jtriley has joined #kolla | 04:52 | |
| *** achanda has joined #kolla | 04:53 | |
| *** jtriley has quit IRC | 04:58 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 05:00 |
| sdake | vhosakot that revert i tried didn't work on fixing elasticsearch | 05:04 |
| vhosakot | sdake: 0a1486a2c73275009f7158c911e7bfd683e40a58 ? | 05:04 |
| sdake | one of two things happened | 05:04 |
| sdake | elasticserch changed their code | 05:04 |
| vhosakot | the es package you mean ? | 05:04 |
| sdake | yes | 05:05 |
| sdake | or there is a regressoin i nthe code base | 05:05 |
| sdake | reverting that patch fixes swift, so that patch is definately fucked up | 05:05 |
| *** weiyu_ has joined #kolla | 05:08 | |
| vhosakot | but, 0a1486a2c73275009f7158c911e7bfd683e40a58 passed gate https://review.openstack.org/#/c/294206/ | 05:09 |
| vhosakot | sdake: ^^ | 05:09 |
| sdake | our gate doesn't test swift | 05:09 |
| sdake | our gate doesn't test much of anything tbh | 05:09 |
| vhosakot | right... also gate does do "docker logs heka" | 05:10 |
| sdake | it tests deploy and tests a vm boots | 05:10 |
| vhosakot | not* | 05:10 |
| sdake | it doesn't even deploy swift atm | 05:10 |
| sdake | or cinder | 05:10 |
| sdake | or heat | 05:10 |
| sdake | or manila | 05:10 |
| sdake | or anything but compute kit | 05:10 |
| vhosakot | yep, just init_runonce | 05:10 |
| sdake | we need seprate gates for swift etc | 05:11 |
| sdake | or we need multiple nodes and deploy storage on one node and compute on that node and controller on another | 05:11 |
| sdake | there is a 8x2 gate hardware we can use | 05:11 |
| vhosakot | sdake: all the patch sets that touch heka in the past couple of days | 05:12 |
| vhosakot | https://review.openstack.org/#/q/project:openstack/kolla+message:heka | 05:12 |
| sdake | this is the part that looks prematic | 05:14 |
| sdake | 2016/03/23 05:12:36 Pre-loading: [rabbitmq_logstreamer_input] | 05:14 |
| sdake | 2016/03/23 05:12:36 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-auditor'}_file_output] | 05:14 |
| sdake | 2016/03/23 05:12:36 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-reaper'}_file_output] | 05:14 |
| sdake | 2016/03/23 05:12:36 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-replicator'}_file_output] | 05:14 |
| sdake | 2016/03/23 05:12:36 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-server'}_file_output] | 05:14 |
| sdake | src: swift enabled: yes name: whatever is not being processes in teh pre-loading | 05:14 |
| vhosakot | sdake: you talking about https://review.openstack.org/#/c/294206/3/ansible/roles/common/tasks/config.yml | 05:16 |
| vhosakot | ? | 05:16 |
| sdake | yup | 05:16 |
| sdake | is this correct syntax | 05:16 |
| sdake | src: "heka-{{ item.src|default(item.name) }}.toml.j2" | 05:17 |
| coolsvap | there is one more review https://review.openstack.org/#/c/283118 | 05:17 |
| coolsvap | related to swift logging to heka | 05:17 |
| sdake | that ws merged in february | 05:17 |
| vhosakot | I see it | 05:17 |
| sdake | it is definatel ythis patch | 05:17 |
| sdake | i reverted it, and swift works after the revert | 05:17 |
| sdake | here is what this line is suppose dto do | 05:19 |
| sdake | src: "heka-{{ item.src|default(item.name) }}.toml.j2" | 05:19 |
| sdake | its supposed to tak eheka and append item.src to it if its defined, if not then it supposed to append default.name | 05:19 |
| sdake | rather item.name | 05:20 |
| sdake | vhosakot do you think that line does that? | 05:20 |
| vhosakot | sdake: wait, to check if the syntax works, check dest folder on host for *.toml files in /etc/kolla/heka | 05:20 |
| sdake | i can tell the syntax is wrong vhosakot | 05:21 |
| sdake | what i want to know is what is the correct syntax | 05:21 |
| sdake | here is how I know the syntax is wrong | 05:21 |
| sdake | <sdake>2016/03/23 05:12:36 Pre-loading: [rabbitmq_logstreamer_input] | 05:21 |
| sdake | [22:14:06] <sdake>2016/03/23 05:12:36 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-auditor'}_file_output] | 05:21 |
| *** weiyu_ has quit IRC | 05:21 | |
| vhosakot | the or | | 05:21 |
| sdake | you see rabbitmq_logstreamer_input is pulled in correctly | 05:22 |
| vhosakot | ah | 05:22 |
| sdake | pre-loading is pulling in src: swift enabled: yes name: swit-account-auditor_file_output | 05:22 |
| vhosakot | so, src: "heka-{{ item.src|default(item.name) }}.toml.j2" is failing failing to get src: for swift | 05:24 |
| vhosakot | https://github.com/openstack/kolla/blob/master/ansible/roles/common/templates/heka-swift.toml.j2 should be copied when swift is enabled.. if it is not happening.. it is bug | 05:25 |
| elemoine_ | vhosakot sdake, I can't help right now but I'll be in the office in about one hour | 05:26 |
| sdake | soundsgood elemoine_ | 05:26 |
| vhosakot | elemoine_: cool :) | 05:26 |
| sdake | vhosakot bug in what | 05:26 |
| sdake | bug in ansible? | 05:27 |
| sdake | or bug in the implementation | 05:27 |
| vhosakot | src: "heka-{{ item.src|default(item.name) }}.toml.j2" not copying https://github.com/openstack/kolla/blob/master/ansible/roles/common/templates/heka-swift.toml.j2 when swift is enabled | 05:27 |
| vhosakot | sdake: if you can pastebin the output of the task "Copying over heka config files" when swift is enabled, I can check | 05:28 |
| sdake | moment | 05:29 |
| openstackgerrit | Allen Gao proposed openstack/kolla-mesos: Fix the deprecated security_group_api and network_api_class https://review.openstack.org/290454 | 05:31 |
| *** phuongnh has quit IRC | 05:34 | |
| sdake | vhosakot http://paste.fedoraproject.org/344024/58711354/ -> http://paste.fedoraproject.org/344024/58711354 | 05:36 |
| sdake | looks busted to me | 05:36 |
| vhosakot | it skipped everything | 05:36 |
| coolsvap | i am also getting similar issue | 05:36 |
| vhosakot | hmm | 05:37 |
| sdake | its supposed to skip that is the point of sam's change | 05:39 |
| vhosakot | the condition "when: item.enabled | bool" is true for each item.. why was each item skipped | 05:39 |
| sdake | hmm good point | 05:40 |
| vhosakot | coolsvap: do you have swift and rabbit enabled ? | 05:40 |
| vhosakot | I'd add a debug and print item.enabled... | 05:40 |
| coolsvap | i have swift enabled i wanted to test the drop root for swift | 05:40 |
| coolsvap | let me stop deploy and start again | 05:41 |
| vhosakot | should it be enabled: {{ enable_swift }} without quotes instead of enabled: "{{ enable_swift }}" | 05:41 |
| vhosakot | let me check other yml files | 05:41 |
| *** salv-orlando has joined #kolla | 05:42 | |
| vhosakot | coolsvap: this affects copying only heka files... I think the rest of swift should be ok | 05:42 |
| * coolsvap checkint | 05:43 | |
| coolsvap | checking* | 05:43 |
| vhosakot | sdake: the quotes are correct... for some reason, when: item.enabled | bool is false | 05:44 |
| sdake | vhosakot let me try | 05:44 |
| sdake | moment | 05:44 |
| elemoine_ | vhosakot sdake probably unrelated but the mariadb line has enable_rabbitmq. Sounds like a bug. | 05:45 |
| sdake | elemoine_ good catch | 05:46 |
| vhosakot | sdake: you could to - debug: msg="value of when is {{item.enabled }}" | 05:46 |
| vhosakot | elemoine_: yes, good catch...unrelated bug | 05:47 |
| sdake | vhosakot so your suggetion is to remove the quotes | 05:47 |
| vhosakot | sdake: no, the quotes are fine | 05:47 |
| vhosakot | sdake: do "kolla-ansible deploy -t swift -vvvv" for extra verbose output | 05:48 |
| *** salv-orlando has quit IRC | 05:49 | |
| sdake | http://paste.fedoraproject.org/344025/58712173/ -> http://paste.fedoraproject.org/344025/58712173 | 05:49 |
| vhosakot | sdake: let me check... do you see heka-swift.toml in /etc/kolla/heka on your host ? | 05:50 |
| sdake | 2016/03/23 05:44:34 Pre-loading: [rabbitmq_log_decoder] | 05:50 |
| sdake | 2016/03/23 05:44:34 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-auditor'}_file_output] | 05:50 |
| sdake | here does the "Pre-loading" data come from | 05:51 |
| sdake | that is what is busted | 05:51 |
| sdake | http://paste.fedoraproject.org/344026/87123021/ -> http://paste.fedoraproject.org/344026/87123021 | 05:51 |
| sdake | answer: no | 05:51 |
| sdake | contents of one of the files: http://paste.fedoraproject.org/344027/87123481/ -> http://paste.fedoraproject.org/344027/87123481 | 05:52 |
| vhosakot | I see | 05:52 |
| sdake | the contents of the file re where the problem is | 05:53 |
| vhosakot | yes, [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-auditor'}_file_output] does not look correct... | 05:53 |
| sdake | definately not correct | 05:53 |
| sdake | that is what is cuasing the problem | 05:53 |
| vhosakot | https://review.openstack.org/#/c/294206/3/ansible/roles/common/tasks/config.yml is causing that | 05:54 |
| *** weiyu has joined #kolla | 05:55 | |
| sdake | i found problem | 05:55 |
| sdake | [{{ item }}_file_output] | 05:55 |
| sdake | it should be item.something | 05:55 |
| sdake | rather then just item | 05:55 |
| vhosakot | https://github.com/openstack/kolla/blob/master/ansible/roles/common/templates/heka-swift.toml.j2#L4 | 05:55 |
| vhosakot | huh | 05:55 |
| sdake | in the template file that is copied | 05:55 |
| sdake | heka-swift.toml.j2 | 05:55 |
| vhosakot | yeah here | 05:55 |
| vhosakot | https://github.com/openstack/kolla/blob/master/ansible/roles/common/templates/heka-swift.toml.j2#L4 | 05:55 |
| vhosakot | great catch | 05:56 |
| sdake | yup | 05:56 |
| sdake | i'm not sure what it shoudl be though | 05:56 |
| sdake | item.name? | 05:56 |
| vhosakot | let me check... the variable {{ item }} does not even exist in that file | 05:57 |
| sdake | it does exist in th tfile | 05:58 |
| sdake | template imports it | 05:58 |
| sdake | ansible magic ftw | 05:58 |
| sdake | i am trying item.name | 05:58 |
| sdake | will report after deploy | 05:58 |
| openstackgerrit | Kuo-tung Kao proposed openstack/kolla: add unit tests for build script https://review.openstack.org/290423 | 06:01 |
| sdake | i wish i had hair like jim kirk | 06:01 |
| sdake | his hair rocks | 06:01 |
| vhosakot | haha | 06:01 |
| sdake | ok well that bug is fixed | 06:01 |
| vhosakot | sdake: look here.... https://github.com/openstack/kolla/blob/master/ansible/roles/common/templates/heka-haproxy.toml.j2#L11 just try [swift__file_output] | 06:02 |
| sdake | nah i got it | 06:02 |
| sdake | moment posting patch | 06:02 |
| vhosakot | item.name ? | 06:02 |
| *** jasonsb has quit IRC | 06:06 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Swift busted with Heka https://review.openstack.org/296190 | 06:09 |
| sdake | elastic search still broken | 06:12 |
| *** achanda_ has joined #kolla | 06:13 | |
| vhosakot | nice fix!... I will pull latest code look into ES tomorrow... | 06:13 |
| *** achanda has quit IRC | 06:14 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Mariad is nto copied if enable_rabitmq is disaled https://review.openstack.org/296194 | 06:16 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Mariad is not copied if enable_rabitmq is disaled https://review.openstack.org/296194 | 06:17 |
| sdake | ok everyone working on swift, your now unblocked ;-) | 06:17 |
| sdake | enjoy | 06:17 |
| coolsvap | thx sdake | 06:18 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Mariadb is not copied if enable_rabitmq is disabled https://review.openstack.org/296194 | 06:18 |
| sdake | i already submitted that coolsvap | 06:18 |
| coolsvap | minor nit in commit | 06:18 |
| coolsvap | message | 06:19 |
| *** vhosakot has quit IRC | 06:25 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 06:25 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 06:30 |
| sdake | ok off to bed | 06:30 |
| sdake | enjoy | 06:30 |
| sdake | elemoine if you want to look into the eleastic search heka problem that would be fantastic :) | 06:30 |
| sdake | i looked and dont see the same item problem | 06:31 |
| *** sdake_ has joined #kolla | 06:35 | |
| *** sdake has quit IRC | 06:36 | |
| *** ssurana has quit IRC | 06:40 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: MariaDB is not copied if enable_rabitmq is disaled https://review.openstack.org/296194 | 06:42 |
| coolsvap | sdake_: s/disaled/disabled :) | 06:42 |
| *** vhosakot has joined #kolla | 06:42 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: MariaDB is not copied if enable_rabitmq is disaled https://review.openstack.org/296194 | 06:43 |
| *** sdake_ has quit IRC | 06:44 | |
| *** sdake has joined #kolla | 06:44 | |
| *** vhosakot has quit IRC | 06:48 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: MariaDB is not copied if enable_rabitmq is disabled https://review.openstack.org/296194 | 06:48 |
| *** jtriley has joined #kolla | 06:53 | |
| *** salv-orlando has joined #kolla | 06:58 | |
| *** jtriley has quit IRC | 06:59 | |
| *** mdnadeem_ has joined #kolla | 07:07 | |
| openstackgerrit | Merged openstack/kolla: Changes horizon log location to kolla_logs volume https://review.openstack.org/295568 | 07:09 |
| *** gfidente has joined #kolla | 07:10 | |
| *** vishwanathj is now known as vishwanathj_away | 07:12 | |
| openstackgerrit | Merged openstack/kolla: Added note about nested environment https://review.openstack.org/295389 | 07:13 |
| openstackgerrit | Merged openstack/kolla: Reconfigure for haproxy https://review.openstack.org/288248 | 07:14 |
| *** salv-orlando has quit IRC | 07:15 | |
| *** weiyu has quit IRC | 07:15 | |
| *** cristicalin has joined #kolla | 07:17 | |
| *** weiyu_ has joined #kolla | 07:19 | |
| *** sdake has quit IRC | 07:23 | |
| *** achanda_ has quit IRC | 07:26 | |
| *** aginwala has quit IRC | 07:29 | |
| openstackgerrit | MD NADEEM proposed openstack/kolla: Reconfigure for haproxy https://review.openstack.org/296229 | 07:30 |
| *** mikelk has joined #kolla | 07:33 | |
| *** achanda has joined #kolla | 07:35 | |
| *** Serlex has joined #kolla | 07:49 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Use debian repos for debian base docker image https://review.openstack.org/288936 | 07:55 |
| *** coolsvap has quit IRC | 07:56 | |
| *** achanda has quit IRC | 08:06 | |
| *** stvnoyes has quit IRC | 08:06 | |
| *** stvnoyes has joined #kolla | 08:07 | |
| *** achanda has joined #kolla | 08:08 | |
| *** achanda has quit IRC | 08:11 | |
| *** achanda has joined #kolla | 08:12 | |
| *** achanda has quit IRC | 08:12 | |
| *** achanda has joined #kolla | 08:13 | |
| *** achanda has quit IRC | 08:13 | |
| *** athomas has joined #kolla | 08:20 | |
| *** mbound has joined #kolla | 08:34 | |
| *** shardy has joined #kolla | 08:37 | |
| *** weiyu_ has quit IRC | 08:46 | |
| *** weiyu_ has joined #kolla | 08:46 | |
| *** jmccarthy has quit IRC | 08:49 | |
| *** jmccarthy has joined #kolla | 08:50 | |
| *** jtriley has joined #kolla | 08:55 | |
| *** jtriley has quit IRC | 09:00 | |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** openstackgerrit has joined #kolla | 09:04 | |
| *** salv-orlando has joined #kolla | 09:06 | |
| *** achanda has joined #kolla | 09:14 | |
| *** achanda has quit IRC | 09:20 | |
| *** cristicalin has quit IRC | 09:33 | |
| *** dims has joined #kolla | 10:00 | |
| *** mikelk has quit IRC | 10:05 | |
| *** mikelk has joined #kolla | 10:05 | |
| *** pbourke has quit IRC | 10:14 | |
| *** pbourke has joined #kolla | 10:15 | |
| *** iceyao_ has quit IRC | 10:16 | |
| *** Jeffrey4l has joined #kolla | 10:49 | |
| *** weiyu_ has quit IRC | 10:53 | |
| *** jtriley has joined #kolla | 10:56 | |
| *** jtriley has quit IRC | 11:02 | |
| *** dims_ has joined #kolla | 11:05 | |
| *** dims has quit IRC | 11:06 | |
| elemoine_ | sdake what is the elasticsearch heka problem you mentioned? | 11:08 |
| *** Jeffrey4l has quit IRC | 11:10 | |
| *** rhallisey has quit IRC | 11:17 | |
| *** Jeffrey4l has joined #kolla | 11:17 | |
| *** rhallisey has joined #kolla | 11:17 | |
| *** achanda has joined #kolla | 11:19 | |
| *** mlima has joined #kolla | 11:21 | |
| *** mlima_ has joined #kolla | 11:23 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: iscsi container with lvm2 support https://review.openstack.org/291285 | 11:23 |
| *** achanda has quit IRC | 11:24 | |
| *** mlima has quit IRC | 11:26 | |
| *** Jeffrey4l has quit IRC | 11:28 | |
| *** kproskurin has joined #kolla | 11:29 | |
| *** Jeffrey4l has joined #kolla | 11:47 | |
| *** haplo37 has joined #kolla | 11:50 | |
| openstackgerrit | Merged openstack/kolla: Remove ironic images from excluded list https://review.openstack.org/296033 | 11:54 |
| *** Jeffrey4l has quit IRC | 11:58 | |
| *** salv-orl_ has joined #kolla | 12:02 | |
| *** haplo37 has quit IRC | 12:04 | |
| *** salv-orlando has quit IRC | 12:04 | |
| ccesario | morning! | 12:10 |
| *** dwalsh has joined #kolla | 12:20 | |
| *** Jeffrey4l has joined #kolla | 12:25 | |
| openstackgerrit | Merged openstack/kolla: Add PBR generation of version information https://review.openstack.org/296124 | 12:33 |
| openstackgerrit | Merged openstack/kolla: Swift busted with Heka https://review.openstack.org/296190 | 12:36 |
| *** Jeffrey4l has quit IRC | 12:38 | |
| *** Jeffrey4l has joined #kolla | 12:40 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Bump ansible container deploy version https://review.openstack.org/296412 | 12:42 |
| *** mathlin has joined #kolla | 12:44 | |
| *** Jeffrey4l has quit IRC | 12:47 | |
| *** mbound has quit IRC | 12:49 | |
| *** dave-mccowan has joined #kolla | 12:52 | |
| *** athomas has quit IRC | 13:20 | |
| *** mbound has joined #kolla | 13:20 | |
| *** achanda has joined #kolla | 13:22 | |
| *** athomas has joined #kolla | 13:24 | |
| sbezverk | Good morning, I still see issue with heka and swift even with the latest merged fix. http://paste.openstack.org/show/491577/ | 13:24 |
| elemoine_ | sbezverk: has the patch being merged? | 13:25 |
| sbezverk | elemoine_ I think so | 13:25 |
| elemoine_ | yes, it was | 13:26 |
| elemoine_ | strange | 13:26 |
| sbezverk | yep, just checked the source I have | 13:26 |
| sbezverk | and it has fixed code | 13:26 |
| *** achanda has quit IRC | 13:27 | |
| elemoine_ | sbezverk: try to docker rm -f heka | 13:28 |
| elemoine_ | and then re-deploy | 13:28 |
| sbezverk | I will, but what is strange if you check the log I pasted | 13:29 |
| sbezverk | toml files are correctly generated, the issue happens when heka tries to load them.. | 13:30 |
| elemoine_ | this can happen if the Heka is restarted with the same (bad) config files inside the container | 13:30 |
| *** absubram has joined #kolla | 13:32 | |
| *** absubram_ has joined #kolla | 13:33 | |
| sbezverk | elemoine_ anyway to enable additional debug in heka to nail which one is causing it? | 13:35 |
| elemoine_ | not sure right now | 13:35 |
| elemoine_ | I can try to reproduce the problem | 13:36 |
| sbezverk | elemoine_ sdake managed to repro exactly the same issue as I am seeing now.. | 13:37 |
| *** absubram has quit IRC | 13:37 | |
| *** absubram_ is now known as absubram | 13:37 | |
| *** jtriley has joined #kolla | 13:38 | |
| *** Serlex has quit IRC | 13:39 | |
| elemoine_ | sbezverk: I am new reproducing with sdake's patch | 13:40 |
| elemoine_ | s/new/not | 13:41 |
| *** iceyao has joined #kolla | 13:41 | |
| sbezverk | elemoine_ hmm, I did what you suggested and still the same for me.. | 13:46 |
| elemoine_ | are you sure you have the latest code? | 13:47 |
| elemoine_ | it is strange | 13:47 |
| sbezverk | elemoine_ sure 100% after I saw merge for that fix I clone master | 13:47 |
| elemoine_ | but it looks like th heka container is started with bad config files | 13:48 |
| sbezverk | elemoine_ what is even mode strange I have swift disabled, but still it gets installed.. | 13:49 |
| elemoine_ | what!? | 13:49 |
| elemoine_ | is the heka container running? or did it exit because it couldn't start properly? | 13:50 |
| elemoine_ | if it runs you could run docker exec to check the config within the container | 13:51 |
| SamYaple | morning | 13:51 |
| sbezverk | Good morning SamYaple | 13:52 |
| sbezverk | elemoine_ heka container keeps restarting | 13:52 |
| elemoine_ | I see | 13:52 |
| elemoine_ | did you try to remove everything using ./tools/cleanup-containers? | 13:53 |
| sbezverk | elemoine_ clean-containers, clean-host, clean-images, reboot 2 times | 13:54 |
| sbezverk | elemoine_ I checked the config file and I do not see anywhere enable swift | 13:55 |
| elemoine_ | do you have swift files in /etc/kolla/heka? | 13:56 |
| elemoine_ | you probably have incorrect files in there | 13:56 |
| elemoine_ | from your previous deploy with swift_enabled | 13:56 |
| elemoine_ | and now you're not generating new files because you have swift disabled | 13:57 |
| sbezverk | elemoine_ that is a possibility, but if it is the case it sounds like a bug to me | 13:57 |
| *** haplo37 has joined #kolla | 13:57 | |
| SamYaple | sbezverk: which container is breaking? | 13:57 |
| sbezverk | if swift is disabled, everything related to it must be just ignored. | 13:57 |
| sbezverk | SamYaple heka | 13:58 |
| elemoine_ | we know that we have bugs when changing the settings (the enable_ flags) | 13:58 |
| SamYaple | i think i fixed that, its all optional now | 13:58 |
| elemoine_ | SamYaple: different issue here | 13:58 |
| SamYaple | oh | 13:58 |
| sbezverk | elemoine_ I am moving all swift related from /etc/kolla/config on me deploying machine and will re-try.. | 14:01 |
| *** absubram has quit IRC | 14:02 | |
| *** kproskurin has quit IRC | 14:02 | |
| *** dims has joined #kolla | 14:04 | |
| *** dims_ has quit IRC | 14:04 | |
| *** achanda has joined #kolla | 14:05 | |
| *** achanda has quit IRC | 14:07 | |
| *** salv-orl_ has quit IRC | 14:07 | |
| openstackgerrit | Merged openstack/kolla: Reconfigure for haproxy https://review.openstack.org/296229 | 14:08 |
| *** openstack has joined #kolla | 14:22 | |
| sbezverk | SamYaple deployment is done on my server in less than 5 minutes :-) and the only rpm with conntrac I see is this one: libnetfilter_conntrack-1.0.4-2.el7.x86_64 | 14:24 |
| SamYaple | ok thanks | 14:26 |
| rhallisey | SamYaple, multinode is going well so far :) | 14:26 |
| *** sdake has joined #kolla | 14:27 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add conntrack-tools https://review.openstack.org/296481 | 14:27 |
| SamYaple | rhallisey: awesome! | 14:27 |
| SamYaple | glad to hear it | 14:27 |
| sdake | morning | 14:27 |
| sbezverk | elemoine_ I am really puzzeled. I cleaned up everything swift related I do se during deployment it is skipped, but heka still flapping.. | 14:27 |
| SamYaple | morning sdake please see contrack patch above, need it to land in mitaka doo | 14:28 |
| SamYaple | too* | 14:28 |
| sdake | we got a whole bunch of patches neeed to land | 14:28 |
| SamYaple | yea but this is fairly high | 14:28 |
| SamYaple | its not breaking, but it will cause problems for longer rnning things | 14:29 |
| *** mgoddard has quit IRC | 14:30 | |
| *** mgoddard has joined #kolla | 14:32 | |
| rhallisey | my host name was longer than 64 chars XD | 14:35 |
| rhallisey | blows up rabbit | 14:35 |
| SamYaple | rhallisey: actually individual parts of the hostname can only be 63 chars, with the entire FQDN bing 255 | 14:38 |
| SamYaple | erlang is just enforcing the RFS | 14:38 |
| SamYaple | RFC* | 14:38 |
| SamYaple | rhallisey: https://tools.ietf.org/html/rfc1123 | 14:39 |
| SamYaple | "string of labels each containing up to 63 8-bit octets" | 14:39 |
| SamYaple | thats the section you want to read | 14:39 |
| *** Serlex has joined #kolla | 14:39 | |
| rhallisey | neat | 14:42 |
| sdake | folksgive me 15 mins to wake up | 14:43 |
| sdake | caffeine free diet means i'm not zero to hero in 3 minutes of slamming a pepsi | 14:44 |
| *** sdake_ has joined #kolla | 14:48 | |
| *** inc0 has quit IRC | 14:48 | |
| *** sdake has quit IRC | 14:49 | |
| *** athomas has quit IRC | 14:49 | |
| *** inc0 has joined #kolla | 14:49 | |
| *** Serlex has quit IRC | 14:50 | |
| elemoine_ | sbezverk: what do you have in /etc/kolla/heka? | 14:51 |
| *** iceyao has quit IRC | 14:52 | |
| sbezverk | elemoine_ on the deployment host or on the target host? | 14:52 |
| *** Jeffrey4l has joined #kolla | 14:52 | |
| *** sdake has joined #kolla | 14:52 | |
| elemoine_ | on the deployment host? | 14:53 |
| *** sdake_ has quit IRC | 14:53 | |
| *** athomas has joined #kolla | 14:53 | |
| sdake | note its an even week this week, our meeting is at 16:30 UTC | 14:53 |
| sbezverk | elemoine_ on the deployment host /etc/kolla/heka does not exist | 14:54 |
| elemoine_ | that's a suprise to me | 14:54 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Fix OpenStack services registration https://review.openstack.org/291251 | 14:54 |
| sdake | sbezverk your definately running withou the patch | 14:56 |
| sdake | but let me double check its actually working on my end with a fresh setup | 14:56 |
| sbezverk | elemoine_ it seems cleaning tools do not clean /etc/kolla directory it has all sorts of old junk | 14:56 |
| sdake | sbezverk the way i can tell is 2016/03/23 14:14:18 Pre-loading: [{'src': 'swift', 'enabled': u'yes', 'name': 'swift-account-auditor'}_file_output] | 14:56 |
| sbezverk | I will remove /etc/kolla from the target and re-deploy | 14:56 |
| sdake | from docker logs | 14:56 |
| sdake | keep a copy of globals.yml | 14:57 |
| sbezverk | sdake how is it possible if when I do git log I see your patch merged? | 14:58 |
| sbezverk | and the folder I use for dpeloyment is in sync with master | 14:58 |
| sdake | it is possible that your heka config files are not being copied over | 14:58 |
| sdake | to /etc/kolla/heka/* | 14:58 |
| sdake | i noticed this bheavior when debugging the patch lat night | 14:59 |
| sdake | that sometimes the files are not copied | 14:59 |
| sbezverk | sdake ok, I will remove all kolla traces on the target server and re-deloy | 14:59 |
| sdake | ya its pretty straightforwar to clean up | 14:59 |
| sdake | but i am dobule checking on my end to make sure it sitll works | 14:59 |
| sbezverk | sdake I would suggest to add a step in clean-host to kill /etc/kolla on the target host | 14:59 |
| SamYaple | who would be interested in a kolla_kill script that just thoughtlessly removes any files kolla related? | 15:00 |
| sdake | elemoine did you fix the elastic search | 15:00 |
| sdake | SamYaple +3 | 15:00 |
| sdake | or make that partof kolla-host | 15:01 |
| sbezverk | SamYaple that would be awsome, it will prevent problems like this one.. | 15:02 |
| sdake | sbezverk why is ringbuild not in the repoistory | 15:02 |
| sdake | rather kolla-clenaup-host | 15:02 |
| elemoine_ | sdake: what elasticsearch bug are your referring to? | 15:02 |
| sdake | i think our kolla-cleanup-host needs love, neesd to remove traces of ceph as well | 15:02 |
| sdake | elemoine_ heka and elasticsearch are broken together | 15:02 |
| sdake | its i the bug tracker, need a ink? | 15:02 |
| elemoine_ | yes, I do, because the tests I ran today did not reveal any problem | 15:03 |
| sdake | i didn't get to it last night | 15:03 |
| sdake | maybe broken is the wrong word | 15:03 |
| sbezverk | sdake not sure what you are refering too? | 15:03 |
| sdake | "returning errors' is th e right statement | 15:03 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1560779 | 15:04 |
| openstack | Launchpad bug 1560779 in kolla "heka elasticsearch is busted" [High,Confirmed] - Assigned to Vikram Hosakote (vhosakot) | 15:04 |
| sdake | elemoine_ 6^ | 15:04 |
| sdake | sbezverk here is my docker logs http://paste.fedoraproject.org/344196/45478145/ -> http://paste.fedoraproject.org/344196/45478145 | 15:04 |
| sdake | you see gone are the swift things | 15:04 |
| sdake | I just did a git pull --rebase | 15:04 |
| *** absubram has joined #kolla | 15:05 | |
| elemoine_ | sdake: the elasticsearch problem is related to Elasticsearch not being started | 15:05 |
| sdake | i backed up the two yml files in /etc/kolla and rm -rf /etc/kolla | 15:05 |
| elemoine_ | deploy elasticsearch and the error logs will go away | 15:05 |
| sdake | elemoine_ is that just a warning or a critical error? | 15:05 |
| sdake | so heka's files shoudlnt deploy if eleasticsearch is disabled then i suppose is the fix? | 15:06 |
| elemoine_ | it's a warning basiccally | 15:06 |
| sdake | ok - i dont like warnings but we can mke it medium priority | 15:06 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Fix OpenStack services registration https://review.openstack.org/291251 | 15:06 |
| elemoine_ | sdake: we already have the code to deal with that | 15:06 |
| elemoine_ | but if you do say : kolla-ansible -t haproxy deploy | 15:07 |
| elemoine_ | heka will start, while es won't | 15:07 |
| elemoine_ | and you'll have to do kolla-ansible -t elasticsearch deploy to start es | 15:07 |
| elemoine_ | then heka will manage to conect to es and the error will go away | 15:08 |
| elemoine_ | makes sense? | 15:08 |
| sdake | elemoine_ i am deploying master | 15:09 |
| sdake | and it hapepns | 15:09 |
| sdake | http://paste.fedoraproject.org/344199/74580314/ -> http://paste.fedoraproject.org/344199/74580314 | 15:10 |
| sdake | my globalsyml file^ | 15:10 |
| sdake | i am doing straight up sudo ./kolla-ansible deploy | 15:10 |
| sdake | elemoine_ there was some rework of heka recently | 15:10 |
| sdake | and i think it got busted | 15:10 |
| *** openstackstatus has joined #kolla | 15:11 | |
| *** ChanServ sets mode: +v openstackstatus | 15:11 | |
| sdake | SamYaple which bug needs attention now? | 15:11 |
| *** Serlex has joined #kolla | 15:11 | |
| elemoine_ | sdake: is elasticsearch started? | 15:11 |
| sdake | SamYaple i had a dc for some reason - flakey wireless | 15:11 |
| sdake | so not sure which one you said needs attention | 15:11 |
| SamYaple | https://review.openstack.org/#/c/296481/ | 15:11 |
| sdake | elemoine_ no its nto started | 15:11 |
| elemoine_ | sdake: if it is Heka should stop outputting these logs | 15:11 |
| sdake | elemoine_ my point is it shoudln't output these logs if eleasticsearch is disabled - but i'll fix it up | 15:12 |
| elemoine_ | sdake: probably because es is not started by default | 15:12 |
| elemoine_ | sdake: hmm, it should be | 15:12 |
| *** mgoddard has quit IRC | 15:14 | |
| *** mgoddard_ has joined #kolla | 15:14 | |
| sdake | elemoine_ i'll dig itno it | 15:14 |
| sdake | after i fix these god awful pep8 errors in autogenerated code | 15:14 |
| elemoine_ | sdake: I just ran a straight kolla-ansible deploy and elasticsearch did start | 15:14 |
| sdake | your right elasticserach is started in my environment | 15:15 |
| sdake | i dont know why i thought it wasn't | 15:15 |
| sdake | i think i looked for "elastic_search" | 15:15 |
| elemoine_ | so you should not see more than a few "Plugin 'elasticsearch_output' error" lines in the Heka logs | 15:16 |
| sdake | SamYaple nice catch | 15:16 |
| sdake | elemoine_ here is my heka log | 15:16 |
| sdake | http://paste.fedoraproject.org/344207/87462011/ -> http://paste.fedoraproject.org/344207/87462011 | 15:16 |
| *** mgoddard__ has joined #kolla | 15:17 | |
| sdake | so i think this is a startup error | 15:17 |
| elemoine_ | if you tail the heka logs do you see more errors showing up? | 15:17 |
| sdake | from what yu describe | 15:17 |
| sdake | no the errors stop once elasticsearch is fully up | 15:17 |
| sdake | can't we start es first? | 15:18 |
| sdake | or alternatively get heka not to compain until ater a bit | 15:18 |
| sdake | here is the dela about warnings | 15:18 |
| elemoine_ | sdake: I mentioned this problem in my ES/Heka integration commit message | 15:18 |
| sdake | when people see wanrings in log they say "hey look something broken must fix" | 15:18 |
| sdake | elemoine_ ok | 15:18 |
| elemoine_ | and I don't have a solution to the problem right now | 15:18 |
| sdake | elemoine_ can it be fixed | 15:19 |
| sdake | do you have any solution that would work? | 15:19 |
| elemoine_ | but I agree we should look at fixing this | 15:19 |
| elemoine_ | I don't know :) | 15:19 |
| sdake | the problem is again, operators see problem, must fix | 15:19 |
| sdake | since es is a dep of elastic search, es should start prior to heka | 15:20 |
| sdake | the problem is the ordering is wrong | 15:20 |
| elemoine_ | does kibana work you? I'd like to check that ESÂ does collect logs | 15:20 |
| *** dwalsh has joined #kolla | 15:20 | |
| sdake | what is the port again | 15:20 |
| sdake | 5601? | 15:20 |
| elemoine_ | sdake: I know, but Heka already belongs to the common playbook, which is executed first | 15:20 |
| elemoine_ | sdake: yes | 15:20 |
| *** mgoddard_ has quit IRC | 15:21 | |
| inc0 | sdake, can you please review patch for rabbitmq upgrade? | 15:21 |
| sbezverk | elemoine_ sdake the mistery is solved, by cleaning /etc/kolla on a target machine | 15:21 |
| inc0 | you -1 it but Sam commented | 15:21 |
| sdake | elemoine_ yest here are logs | 15:21 |
| sdake | inc0 link and yes | 15:21 |
| sdake | sbezverk ya thats a problem | 15:21 |
| inc0 | sdake, https://review.openstack.org/#/c/287969/ | 15:21 |
| sdake | sbezverk please filea bug | 15:22 |
| *** ssurana has joined #kolla | 15:22 | |
| sdake | a cleanup and deploy should result in a working deploy | 15:22 |
| *** vhosakot has joined #kolla | 15:22 | |
| sdake | and by cleanup, i dont rmean removing all files in e/tc/kolla | 15:22 |
| sdake | inc0 i already revieweed this patch days ago | 15:23 |
| sdake | but i forgot to hit the reply button | 15:23 |
| inc0 | lol | 15:23 |
| sdake | i thnk | 15:23 |
| sdake | my main compalint was shell-true | 15:23 |
| sbezverk | yes I removed completely /etc/kolla | 15:23 |
| inc0 | sdake, but that is from inside container to inside container | 15:24 |
| sdake | inc0 nevermind that isn't what happeened, i am sstill waking up ;) | 15:24 |
| inc0 | this shell never leaves container | 15:24 |
| sdake | inc0 let me read your response | 15:24 |
| inc0 | sure | 15:24 |
| sdake | inc0 ok so just ot clarify shell=true is mandatory? | 15:26 |
| sdake | raed my response real quick | 15:27 |
| sdake | i'm not arguing debates of whether its good idea or not or secure or not | 15:27 |
| sdake | i just need to know if its mandatory | 15:27 |
| sdake | because of bandit | 15:28 |
| inc0 | SamYaple, around? | 15:28 |
| inc0 | so I tried with shell false, can't remember why I didn't do shell false;) | 15:28 |
| inc0 | let me double check | 15:28 |
| sdake | if it doesn't work with shell=false then we can roll with that | 15:28 |
| sdake | i'd like confirmation though and something in the review tracker | 15:28 |
| sbezverk | sdake https://bugs.launchpad.net/kolla/+bug/1561055 | 15:29 |
| openstack | Launchpad bug 1561055 in kolla "enhancing target host clean script" [Undecided,New] | 15:29 |
| sbezverk | sdake can we go back for a minute to discussion about systemd | 15:29 |
| SamYaple | inc0: sdake yea shell false breaks it | 15:30 |
| *** dwalsh has quit IRC | 15:30 | |
| SamYaple | because rabbitmqctl is a wrapper of a wrapper or soemthing | 15:30 |
| SamYaple | in the end it calls erlang things i believe | 15:30 |
| SamYaple | specfically erl | 15:30 |
| SamYaple | shell=True is not dangeours here since none of the commands being executred are variable | 15:31 |
| vhosakot | elemoine: SamYaple: could you check my reply to the comments in https://review.openstack.org/#/c/296012/1 and let me know your thoughts | 15:32 |
| sdake | SamYaple not debating if its secure or not | 15:32 |
| sdake | i'll change my vote to +2 | 15:33 |
| *** mgoddard has joined #kolla | 15:33 | |
| *** mgoddard__ has quit IRC | 15:33 | |
| sdake | inc0 or SamYaple can one of you take responsibility for backporting the blueprints related to rabbitmq upgrade | 15:34 |
| sdake | SamYaple you touched on something above, - shell=true is only dangerous when commands are variable passsed to the operation? | 15:37 |
| sdake | we will eventually need comments in our code around these bandit complaint points where we #nosec them | 15:38 |
| elemoine_ | vhosakot: you're suggestion makes sense to me | 15:38 |
| vhosakot | elemoine: thanks for the feedback! | 15:39 |
| vhosakot | elemoine: so, I will add lua parser for libvirtd.log for both master and Mitaka as a separate effort and patch set | 15:40 |
| sdake | it should be a backport | 15:40 |
| vhosakot | sdake: yep, backport to mitaka.. right | 15:41 |
| sdake | cn you expalin why you would violate the master first backport policy | 15:41 |
| vhosakot | and implementtation for master | 15:41 |
| sdake | ok i misunderstood you | 15:41 |
| vhosakot | no, I will first implement and test in Newton master.. then, backport the parser to mitaka :) | 15:41 |
| vhosakot | I won't forwardport from Mitaka to Newton :) | 15:42 |
| *** akwasnie1 has joined #kolla | 15:42 | |
| sdake | vhosakot alot o peple operate that way - to their dismay it doesn't work at all | 15:42 |
| vhosakot | sdake: wow, I like new dev in master and backporting nice features/fixes to old release... so, the lua parser for libvirts.log will go in master first and then I'll backport it to mitaka | 15:43 |
| vhosakot | sdake: so, the ES bug is not high pri as the warninigs as expected when centrol_looging is disbaled ? | 15:44 |
| sdake | is common a magic role in ansible that always gets deployed first? | 15:44 |
| sdake | SamYaple i don't see it in the site.yml file | 15:44 |
| sdake | vhosakot i tihnk we need to fix it | 15:44 |
| vhosakot | sdake: conditional copy in config.yml ? | 15:45 |
| vhosakot | I'm heads down in new parser for libvirt | 15:45 |
| sdake | vhosakot there is an ordering problem | 15:45 |
| vhosakot | sdake: ah ok | 15:45 |
| sdake | vhosakot ya i'm taing this bug over | 15:45 |
| vhosakot | cool | 15:45 |
| sdake | vhosakot do yo uknow if common is some special magic role - i dont see it in site.yml | 15:46 |
| vhosakot | yes, I see common run when ES, heka, haproxy, anything is run | 15:47 |
| vhosakot | common containers have no tag | 15:47 |
| sdake | vhosakot yes but do you understand where common is programaticaly defined to run | 15:48 |
| sdake | i want to start es prior to common | 15:48 |
| sdake | or alteernaviely heka needs too be taken out of common | 15:48 |
| sdake | elemoine thoughts ^ | 15:49 |
| vhosakot | sdake: it is a dep in meta for ES | 15:51 |
| vhosakot | sdake: https://github.com/openstack/kolla/blob/master/ansible/roles/elasticsearch/meta/main.yml#L3 | 15:51 |
| openstackgerrit | Merged openstack/kolla: MariaDB is not copied if enable_rabitmq is disabled https://review.openstack.org/296194 | 15:51 |
| SamYaple | sdake: common is a meta role | 15:54 |
| sdake | vhosakot thanks | 15:54 |
| sdake | SamYaple thanks | 15:54 |
| sdake | i get it now | 15:54 |
| vhosakot | sdake: cool | 15:54 |
| vhosakot | ok, back to lua | 15:55 |
| * vhosakot is liking the emails about Heat vs Ansible :) | 15:56 | |
| sdake | heat solves a different problem then ansible | 15:56 |
| sdake | SamYaple looking for a recommendation here | 15:56 |
| sdake | SamYaple elasticsearch must start prior to heka | 15:56 |
| sdake | heka is in common | 15:57 |
| *** mgoddard_ has joined #kolla | 15:57 | |
| SamYaple | it depends on what is needed then | 15:58 |
| SamYaple | does it need the log directories already created? | 15:58 |
| SamYaple | you can start elasticsearc hwithout the common role dep | 15:58 |
| sdake | let me check o n the named volume access | 15:59 |
| sdake | it does nto depend on any named volumes but eleasticserach | 16:00 |
| sdake | and obviously the copying of the cifngs | 16:00 |
| *** daneyon has quit IRC | 16:00 | |
| *** mgoddard has quit IRC | 16:01 | |
| SamYaple | then move it to the first thing in site.yml and remove its meta entry | 16:01 |
| sdake | it may depend on database | 16:01 |
| sdake | i am not sure | 16:01 |
| sdake | i will need to look at the code | 16:01 |
| akwasnie1 | hi guys, I see there is a problem with elasticsearch/heka? | 16:02 |
| sdake | but that sounds good if i t has no other deps | 16:02 |
| SamYaple | why does elasticsaerch need to start first? | 16:02 |
| sdake | SamYaple heka ctries to connect to es and generatess a spew of 30 warnings | 16:02 |
| sdake | so es is a dep of heka | 16:02 |
| SamYaple | i mean you can make meta roles conditional | 16:03 |
| elemoine_ | heka can wait no problem, it's just "embarassing" to see these error messages in the heka logs | 16:03 |
| SamYaple | oh thats not a big issue then | 16:03 |
| sdake | ya this isnt' critical | 16:03 |
| SamYaple | ok cool | 16:03 |
| sdake | but i'd like it fixed for mitaka | 16:03 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: MariaDB is not copied if enable_rabitmq is disabled https://review.openstack.org/296565 | 16:03 |
| sdake | its medium prioritory in the tracker atm | 16:03 |
| sdake | trust me from years of support experience any log message needs to be explained ;) | 16:03 |
| *** mgoddard_ has quit IRC | 16:04 | |
| *** mgoddard has joined #kolla | 16:04 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Swift busted with Heka https://review.openstack.org/296566 | 16:04 |
| elemoine_ | sdake: I agree with you, I'll see if I can come up with a work-around, but I don't think we should require ES to be started before Heka | 16:04 |
| sdake | elemoine_ can you expand on that | 16:05 |
| gmmaha | hi, has anyone seen a scenario where the docker containers are stuck at restarting and they dont come up.. | 16:05 |
| sdake | gmmaha how old areyour containers | 16:05 |
| gmmaha | is there a way i can get to some sorta logs that will tell us whats going on | 16:05 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add creation of new log folder for Horizon https://review.openstack.org/296569 | 16:05 |
| vhosakot | gmmaha: which containers.. name please | 16:05 |
| gmmaha | sdake: probably 5 days max | 16:05 |
| sdake | gmmaha you can use docker logs | 16:05 |
| gmmaha | vhosakot: keystone and horizon | 16:05 |
| sdake | centos binary? | 16:05 |
| vhosakot | gmmaha: stale pid files.. there was a fix recetnly | 16:05 |
| gmmaha | sdake: ubuntu binaries | 16:06 |
| sdake | elemoine can you expand why reorodering isnt' viable? | 16:06 |
| gmmaha | vhosakot: aah | 16:06 |
| elemoine_ | sdake: enforcing containers start with Ansible is a bad idea in my opinion, think reboots for example | 16:06 |
| elemoine_ | I mean "enforcing containers start order" | 16:06 |
| sdake | i dont know how well kolla works with a reboot situation | 16:07 |
| sdake | last time i tested that it wasn't good | 16:07 |
| elemoine_ | sdake: I tested it as well, and it was not good | 16:07 |
| gmmaha | sdake: vhosakot: they are both stuck with these errors http://paste.openstack.org/show/491615/ | 16:07 |
| elemoine_ | it's a major issue IMO | 16:07 |
| sdake | the reason its not good is because of ordering | 16:07 |
| vhosakot | gmmaha: it is fixed in master now https://bugs.launchpad.net/kolla/+bug/1557036 | 16:07 |
| openstack | Launchpad bug 1557036 in kolla "httpd (pid 1) already running when starting keystone or horizon container" [Critical,Fix released] - Assigned to Sam Yaple (s8m) | 16:07 |
| sdake | elemoine_ agree | 16:07 |
| SamYaple | for the record, i run with restart policy == never for this reason guys | 16:08 |
| sdake | docker has no dependency managmeent built into its daemon | 16:08 |
| vhosakot | gmmaha: delete everything in /var/run/apache2/* in horizon and restart | 16:08 |
| gmmaha | vhosakot: aaah | 16:08 |
| gmmaha | cook | 16:08 |
| gmmaha | cool | 16:08 |
| gmmaha | let me try that | 16:08 |
| gmmaha | thanks | 16:08 |
| * gmmaha makes a note to update to the latest release with the fix | 16:08 | |
| gmmaha | thanks vhosakot sdake | 16:08 |
| vhosakot | gmmaha: cool, let me know if it works | 16:08 |
| sdake | gmmaha happy to help | 16:08 |
| sdake | restart=never doesn't fix the reboot problem | 16:09 |
| vhosakot | gmmaha: I update to master multiple times a day | 16:09 |
| vhosakot | gmmaha: don't lose out the fixes :) | 16:09 |
| gmmaha | vhosakot: :) | 16:09 |
| gmmaha | indeed.. | 16:09 |
| SamYaple | vhosakot: that fix should be in mitaka | 16:09 |
| SamYaple | it merged before branch | 16:09 |
| vhosakot | SamYaple: ah... | 16:10 |
| gmmaha | vhosakot: when you say delete everything in horizin under /var/run/apache2, might i ask how do i do that? The container doesnt respond to any bash/exec commands :( | 16:10 |
| gmmaha | sorry very novice with docker containers | 16:10 |
| vhosakot | SamYaple: so, should it be forwardported ? shouldn't the fix come to master as well ? | 16:10 |
| vhosakot | SamYaple: is forwardport a thing ? or the fix needs to be redone in master | 16:10 |
| SamYaple | vhosakot: its in master and mitaka | 16:12 |
| vhosakot | SamYaple: nice! | 16:12 |
| vhosakot | SamYaple: i just checked it.. cool | 16:12 |
| vhosakot | gmmaha: will send the commands .. 1 sec | 16:13 |
| gmmaha | vhosakot: thanks a ton! | 16:13 |
| inc0 | gmmaha, you shouldn't delete it in docker container | 16:13 |
| inc0 | you should remove docker container itself | 16:13 |
| inc0 | basically containers should be immutable, so you don't change anything inside | 16:13 |
| gmmaha | inc0: aah ok.. | 16:14 |
| inc0 | what's the issue you're having anyway? | 16:14 |
| gmmaha | inc0: the base machine restarted and now horizon and keystone containers are stuck inrestarting mode | 16:14 |
| gmmaha | inc0: vhosakot just ponited me to this bug. https://bugs.launchpad.net/kolla/+bug/1557036 | 16:14 |
| openstack | Launchpad bug 1557036 in kolla "httpd (pid 1) already running when starting keystone or horizon container" [Critical,Fix released] - Assigned to Sam Yaple (s8m) | 16:15 |
| sdake | that has been fixed in our code base about 5different times | 16:15 |
| sdake | i dont get why it keeps oppign up | 16:15 |
| SamYaple | sdake: not for ubuntu | 16:15 |
| inc0 | gmmaha, try upgrading to master | 16:15 |
| vhosakot | gmmaha: so, running "docker exec horizon em -rf /var/run/apache2/*" will delete stalepid files in horizon | 16:15 |
| sdake | oh for centos only then? | 16:15 |
| inc0 | this fix is out there | 16:15 |
| vhosakot | em==rm | 16:15 |
| SamYaple | vhosakot: not when the container is not running | 16:15 |
| vhosakot | SamYaple: right right.. stop it forst | 16:16 |
| vhosakot | first | 16:16 |
| SamYaple | vhosakot: no that doesnt work when the contaienr is stopped | 16:16 |
| inc0 | vhosakot, opposite, it won't work | 16:16 |
| SamYaple | exec requires container is running | 16:16 |
| inc0 | if container is in restarting loop, you should remove-recreate it | 16:16 |
| gmmaha | guess ihave to blow away the container | 16:16 |
| inc0 | yeah, do it | 16:16 |
| inc0 | but rebuild your contiainers to latest master | 16:17 |
| gmmaha | inc0: sdake: vhosakot: thanks. will destroy them and get them bck again | 16:17 |
| inc0 | so you'll do effective upgrade | 16:17 |
| vhosakot | I meant, exec to delete stale piu files, stop, restart... it worked for me the other day | 16:17 |
| inc0 | and this bug is already fixed | 16:17 |
| vhosakot | gmmaha: yep, delete and recreate. that is the best | 16:17 |
| gmmaha | vhosakot: thanks.. | 16:17 |
| inc0 | also you can try to run kolla-ansible upgrade :) | 16:17 |
| inc0 | instead of full deplo | 16:17 |
| gmmaha | inc0: sure | 16:17 |
| inc0 | just rebuild containers prior to that | 16:18 |
| vhosakot | inc0: to run "kolla-ansible upgrade", we need to build the latest images first ? is this right ? how do we upgrade ? | 16:18 |
| vhosakot | inc0: pull master, kolla-build, kolla-ansible upgrade ... is this workflow correct ? | 16:18 |
| inc0 | yes | 16:19 |
| inc0 | this will upgrade running infra to latest master | 16:19 |
| vhosakot | all containers ? | 16:19 |
| sdake | vhosakot nearly all containes * | 16:19 |
| vhosakot | or only containers that have their Dockerfile _changed_ in master | 16:19 |
| inc0 | vhosakot, really those which you want upgraded | 16:19 |
| vhosakot | inc0: ah ok, cool | 16:20 |
| inc0 | but I'd suggest all | 16:20 |
| vhosakot | inc0: cool | 16:20 |
| inc0 | since you don't want to have different base images | 16:20 |
| inc0 | (for example different version of apt-get installed packages in base) | 16:20 |
| openstackgerrit | Merged openstack/kolla: Add conntrack-tools https://review.openstack.org/296481 | 16:20 |
| vhosakot | yep, agrred... upgarde all the things | 16:20 |
| inc0 | that's the way it should be handled | 16:21 |
| vhosakot | cool.. thanks for the info | 16:21 |
| inc0 | but upgrade will only change if image differs from what is out there | 16:21 |
| inc0 | so if you specifically want to keep container x not-upgraded, just don't build new image | 16:21 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add conntrack-tools https://review.openstack.org/296587 | 16:21 |
| inc0 | or use kolla-ansible upgrade -t horizon to upgrade just horizon for example | 16:22 |
| inc0 | (which is impossible without containers \o/) | 16:22 |
| sdake | groan heka is dependent on haproxy as well | 16:24 |
| sdake | elemoine_ ^^ | 16:24 |
| sdake | i think heka needs to go out of the common meta | 16:25 |
| *** dwalsh has joined #kolla | 16:25 | |
| vhosakot | inc0: cool | 16:26 |
| SamYaple | sdake: no | 16:29 |
| SamYaple | heka is a requirement | 16:29 |
| SamYaple | it setups the logs directories and what not | 16:29 |
| vhosakot | meeting in 1 min ? | 16:29 |
| SamYaple | it _needs_ to be started | 16:29 |
| *** jasonsb has joined #kolla | 16:30 | |
| sdake | circular dependendcies ftl | 16:30 |
| sdake | ok folks meeting time | 16:31 |
| sdake | inc0 | 16:31 |
| elemoine_ | heka creates the syslog unix socket that other services/containers may use | 16:31 |
| sdake | rhallisey jpeeler elemoine_ | 16:31 |
| elemoine_ | so it needs to be started first | 16:31 |
| rhallisey | sdake, already said hi | 16:31 |
| *** salv-orlando has joined #kolla | 16:31 | |
| elemoine_ | if heka relies on resources created by other services then it should wait :) | 16:32 |
| SamYaple | im not worried about a few errors that pop up on start up right now | 16:32 |
| *** jasonsb has quit IRC | 16:38 | |
| *** salv-orlando has quit IRC | 16:38 | |
| *** jasonsb has joined #kolla | 16:39 | |
| *** akwasnie1 has quit IRC | 16:39 | |
| *** vishwanathj_away is now known as vishwanathj | 16:39 | |
| *** akwasnie1 has joined #kolla | 16:40 | |
| sbezverk | which one is meeting channel? | 16:40 |
| *** mikelk has quit IRC | 16:41 | |
| vhosakot | sbezverk: #openstack-meeting-4 | 16:42 |
| *** salv-orlando has joined #kolla | 16:42 | |
| vhosakot | sbezverk: /join #openstack-meeting-4 | 16:42 |
| *** aginwala has joined #kolla | 16:44 | |
| *** aginwala has quit IRC | 16:49 | |
| *** salv-orlando has quit IRC | 16:54 | |
| *** achanda has joined #kolla | 16:58 | |
| *** Pierre_Carette has joined #kolla | 16:58 | |
| mlima_ | hello guys | 17:02 |
| mlima_ | i have this error | 17:02 |
| mlima_ | TASK: [common | Starting heka bootstrap container | 17:02 |
| mlima_ | failed: [localhost] => {"changed": true, "failed": true} | 17:02 |
| mlima_ | msg: Container exited with non-zero return code | 17:02 |
| SamYaple | mlima_: docker logs? | 17:03 |
| mlima_ | ERROR:__main__:The source to copy does not exist: /var/lib/kolla/config_files/heka-haproxy.toml | 17:03 |
| mlima_ | i set enable_haproxy: "no" | 17:03 |
| *** achanda has quit IRC | 17:03 | |
| mlima_ | full log http://pastebin.com/Aaikig8b SamYaple | 17:04 |
| *** dims_ has quit IRC | 17:05 | |
| *** dims has joined #kolla | 17:06 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Enable horizon domain logins https://review.openstack.org/296619 | 17:09 |
| *** jasonsb has quit IRC | 17:09 | |
| SamYaple | mlima_: file a bug against master | 17:09 |
| mlima_ | ok | 17:10 |
| SamYaple | i know the issue. there is a partial fix here https://review.openstack.org/#/c/294206/ | 17:10 |
| SamYaple | ill get it fixed and into mitaka for you | 17:10 |
| mlima_ | ok | 17:10 |
| mlima_ | https://bugs.launchpad.net/kolla/+bug/1561111 SamYaple | 17:13 |
| openstack | Launchpad bug 1561111 in kolla " Starting heka bootstrap container failed in deployment" [Undecided,New] | 17:13 |
| *** achanda has joined #kolla | 17:14 | |
| *** mgoddard has quit IRC | 17:16 | |
| *** dwalsh has quit IRC | 17:16 | |
| *** mgoddard has joined #kolla | 17:16 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Cleanup volumes on multiple nodes https://review.openstack.org/291116 | 17:22 |
| *** dougs has joined #kolla | 17:23 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: iscsi container with lvm2 support https://review.openstack.org/291285 | 17:24 |
| *** gfidente has quit IRC | 17:24 | |
| SamYaple | mlima_: thanks will get it fixed up for you! | 17:25 |
| *** aginwala has joined #kolla | 17:26 | |
| openstackgerrit | Merged openstack/kolla: MariaDB is not copied if enable_rabitmq is disabled https://review.openstack.org/296565 | 17:27 |
| dougs | Does anyone use Kolla with Pacemaker? | 17:27 |
| dougs | (For HA) | 17:28 |
| SamYaple | dougs: we use haproxy and keepalived | 17:29 |
| SamYaple | then for rabbitmq we cluster that, mariadb + galera handeles the database | 17:29 |
| *** achanda has quit IRC | 17:29 | |
| dougs | SamYaple, how do you handle fencing? | 17:30 |
| sdake | dougs kolla does no isolation as part of its HA system | 17:31 |
| *** akwasnie1 has quit IRC | 17:31 | |
| dougs | sdake, thanks | 17:32 |
| sdake | dougs i've worked on ha my entiere life | 17:32 |
| sdake | i invented corosync for example | 17:32 |
| sdake | i can say that fencing isn't absolutely necessary | 17:32 |
| sdake | but in the case of kolla isolation woudl be helfpul | 17:32 |
| sdake | what isolation is really needed is network isolation though, not fencing | 17:33 |
| *** achanda has joined #kolla | 17:33 | |
| gmmaha | inc0: tried kolla-ansible upgrade and hitting this error.. http://paste.openstack.org/show/491619/ | 17:33 |
| sdake | gmmaha is that master? | 17:34 |
| gmmaha | sdake: yes sir | 17:34 |
| * sdake groans | 17:34 | |
| sdake | dare you familiar with launchpad enough to file a bug? | 17:34 |
| gmmaha | yes i am :) | 17:35 |
| * gmmaha goes to file a bug with some logs | 17:35 | |
| gmmaha | well, with the tiny logs | 17:35 |
| sdake | would you do so | 17:35 |
| sdake | and link when done | 17:35 |
| gmmaha | aye aye | 17:35 |
| sdake | inc0 may have some suggestions for information he woudl like in the log | 17:35 |
| gmmaha | inc0: anything else uou would like | 17:35 |
| sdake | i'd recommend running kolla-ansible upgrade -vvv though | 17:35 |
| gmmaha | i can run the same ansible with much more verbose logging | 17:35 |
| gmmaha | sdake: you are quick.. yes will do that | 17:36 |
| sdake | kolla-ansible takes --vvv flags | 17:36 |
| *** mbound has quit IRC | 17:36 | |
| sdake | i dont recall 1 dash or 2 ;) | 17:36 |
| gmmaha | sdake: one dash | 17:36 |
| gmmaha | and i have 4 vs.. not sure if that will help.. | 17:36 |
| * gmmaha isnt sure of the limit on the v's | 17:37 | |
| sdake | try to be dettailed | 17:37 |
| sdake | include your inventory file please | 17:37 |
| gmmaha | sure | 17:37 |
| sdake | let me try upgrade with master on my aio setup | 17:37 |
| gmmaha | sdake: sure.. thsi is multinode if that matters. | 17:37 |
| gmmaha | will put it all in the logs | 17:37 |
| *** unicell has joined #kolla | 17:38 | |
| *** mbound has joined #kolla | 17:38 | |
| *** akwasnie1 has joined #kolla | 17:39 | |
| vhosakot | elemoine_: SamYaple: sdake: I have replied to the comments and uploaded new patch set... would you reconsider voting again on the review :) https://review.openstack.org/#/c/296012/ | 17:40 |
| sdake | gmmaha fwiw master works with aio upgrade | 17:40 |
| gmmaha | sdake: ohhhh | 17:40 |
| gmmaha | getting that bug your way in the next few mins | 17:40 |
| sdake | gmmaha i am gong to setup my other 2 nodes today in my duaghthers room if I can find the right cabling | 17:40 |
| sdake | and will be able to test multinode | 17:41 |
| vhosakot | just wanted to make sure it (libvirtd.log going to /var/lok/kolla/libvirt) gets into mikata | 17:41 |
| sdake | not sure on inc0's abillity to test multinode atm | 17:41 |
| sdake | (my gear is down because of a remodel) | 17:41 |
| gmmaha | sdake: aah.. | 17:41 |
| *** achanda has quit IRC | 17:41 | |
| SamYaple | vhosakot: thats not how cherry-picks work | 17:42 |
| SamYaple | if you are changing the cherry-pick its not a cherry-pick | 17:42 |
| SamYaple | it needs to be fixed in master first | 17:42 |
| sdake | cherry-picks that resolve merge conflicts are acceptable | 17:43 |
| SamYaple | yes thats true | 17:43 |
| SamYaple | in this case thats not whats happening | 17:43 |
| vhosakot | sure, I will revert heka-openstack.toml.j2 for libvirt in master also then | 17:43 |
| sdake | SamYaple no idea what context is :) | 17:43 |
| sdake | trying to finish reno before people start adding features | 17:44 |
| sdake | so we can do proper tracking of features :) | 17:44 |
| gmmaha | sdake: https://bugs.launchpad.net/kolla/+bug/1561132 | 17:44 |
| openstack | Launchpad bug 1561132 in kolla "Kolla upgrade on multi-node with master fails" [Undecided,New] | 17:44 |
| gmmaha | sdake: inc0: attached the multinode inventory file along with it | 17:45 |
| sdake | gmmaha thanks | 17:46 |
| sdake | inc0 may be at lucnh atm | 17:46 |
| gmmaha | aah yes.. | 17:46 |
| gmmaha | will wait him to come back on that | 17:46 |
| *** salv-orlando has joined #kolla | 17:48 | |
| *** mbound has quit IRC | 17:51 | |
| *** achanda has joined #kolla | 17:51 | |
| *** daneyon has joined #kolla | 17:54 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: iscsi container with lvm2 support https://review.openstack.org/291285 | 17:55 |
| sdake | ok reno! | 17:58 |
| sdake | bbiaf | 17:58 |
| *** salv-orl_ has joined #kolla | 18:01 | |
| dougs | sdake: Do you think that using Pacemaker + a docker agent for health checking / cloning containers would be a reasonable way to get HA with Kolla including a mechanism for isolation? Or in practice is the lack of isolation not a significant problem, in which case keepalived should be enough? | 18:04 |
| *** dwalsh has joined #kolla | 18:04 | |
| sdake | dougs having worked on corosync and pacemaker for 9 years of my life, i can say with confidence they are total overkill for our needs | 18:04 |
| sdake | as in wrong tool for the job | 18:04 |
| *** salv-orlando has quit IRC | 18:04 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Revert libvirt change in heka-openstack.toml.j2 in master https://review.openstack.org/296649 | 18:05 |
| dougs | sdake: thanks a lot, much appreciated | 18:05 |
| dougs | so you've never experienced problems with split-brain due to lack of isolation? | 18:06 |
| openstackgerrit | Alicja Kwasniewska proposed openstack/kolla: Add kibana authentication using HAproxy https://review.openstack.org/296650 | 18:07 |
| *** yuanying_ has joined #kolla | 18:07 | |
| openstackgerrit | Alicja Kwasniewska proposed openstack/kolla: Add kibana authentication using HAproxy https://review.openstack.org/296650 | 18:08 |
| *** achanda has quit IRC | 18:09 | |
| *** yuanying has quit IRC | 18:09 | |
| vhosakot | SamYaple: elemoine_: I have reverted the libvirt change in ansible/roles/common/templates/heka-openstack.toml.j2 in master as well.... once https://review.openstack.org/#/c/296012/ and https://review.openstack.org/#/c/296649/ are merged, heka-openstack.toml.j2 will be consistent and same in both master and stable/mitaka. | 18:09 |
| inc0 | gmmaha, back | 18:12 |
| gmmaha | inc0: hey | 18:14 |
| inc0 | sooo | 18:14 |
| gmmaha | :) | 18:14 |
| inc0 | SamYaple, I might need your help | 18:14 |
| * gmmaha likes it when smeoen starts with sooo | 18:14 | |
| SamYaple | inc0: if this is the mariadb one i can handle it | 18:16 |
| inc0 | SamYaple, indeed mariadb | 18:17 |
| gmmaha | SamYaple: inc0: thanks in advance for the help | 18:17 |
| SamYaple | inc0: yea im like 99% sure i know the issue | 18:17 |
| SamYaple | im struglling with arp and neutron ATM | 18:17 |
| sdake | dougs it is possible in our architecture for keepalived to place a vip on two nodes in a split brain | 18:22 |
| sdake | dougs that is why we need *network* isolation not machine isolation | 18:26 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 18:26 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 18:27 |
| *** aginwala has quit IRC | 18:30 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add reno support https://review.openstack.org/296125 | 18:33 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add PBR generation of version information https://review.openstack.org/296662 | 18:34 |
| *** unicell has quit IRC | 18:34 | |
| *** unicell has joined #kolla | 18:34 | |
| *** aginwala has joined #kolla | 18:37 | |
| *** sdake_ has joined #kolla | 18:41 | |
| *** akwasnie1 has quit IRC | 18:43 | |
| *** sdake has quit IRC | 18:44 | |
| *** sdake has joined #kolla | 18:47 | |
| *** aginwala has quit IRC | 18:47 | |
| *** cemason has joined #kolla | 18:49 | |
| *** sdake_ has quit IRC | 18:49 | |
| *** openstack has joined #kolla | 19:08 | |
| notmyname | heh. just dropping in to give a patchbot overview ;-) | 19:08 |
| sdake | i tested it just last night | 19:08 |
| SamYaple | notmyname: :D | 19:08 |
| notmyname | cool | 19:08 |
| SamYaple | sdake: im just joking around | 19:08 |
| *** openstack has joined #kolla | 19:21 | |
| *** openstackstatus has joined #kolla | 19:22 | |
| *** ChanServ sets mode: +v openstackstatus | 19:22 | |
| sdake | SamYaple ar eyou running kolla with vlans? | 19:22 |
| ccesario | vhosakot, no no..... for while only experimental tests .... based docker-py documentation http://docker-py.readthedocs.org/en/stable/api/?highlight=hostname https://hub.docker.com/r/cloudaku/docker-py/ | 19:23 |
| SamYaple | sdake: of course | 19:23 |
| *** serverascode_ is now known as serverascode | 19:23 | |
| SamYaple | also jumbo frames | 19:23 |
| vhosakot | ccesario: got it thanks! | 19:23 |
| ccesario | vhosakot, remembering that I'm not python expert :P | 19:24 |
| *** DuncanT_ is now known as DuncanT | 19:24 | |
| *** harmw has quit IRC | 19:24 | |
| *** harmw has joined #kolla | 19:25 | |
| *** Guest69253 is now known as zeus` | 19:25 | |
| vhosakot | well well python is very big | 19:27 |
| *** aginwala has joined #kolla | 19:32 | |
| *** zeus` is now known as zeus | 19:33 | |
| *** zeus has quit IRC | 19:34 | |
| *** zeus has joined #kolla | 19:34 | |
| *** achanda has joined #kolla | 19:35 | |
| sdake | getting a 503 service unavailable with horizon on master with your network patch SamYaple | 19:36 |
| sdake | from a freshly cleaned and rebooted environment | 19:36 |
| SamYaple | sdake: surely you recognize that horizon is unrelated to that external network option | 19:37 |
| SamYaple | and 503 is haproxy not apache | 19:37 |
| sdake | yup so its likely master horizon is broken in general | 19:38 |
| SamYaple | sdake: it is not | 19:39 |
| SamYaple | works fine | 19:39 |
| SamYaple | well | 19:39 |
| SamYaple | im using ubuntu-source | 19:39 |
| SamYaple | so maybe centos | 19:39 |
| sdake | i am using centos source | 19:39 |
| SamYaple | idk | 19:39 |
| sdake | it was working a few days ago | 19:39 |
| sdake | i have not changed my image since | 19:39 |
| openstackgerrit | Merged openstack/kolla: Add creation of new log folder for Horizon https://review.openstack.org/296569 | 19:39 |
| *** jtriley has joined #kolla | 19:40 | |
| *** aginwala has quit IRC | 19:42 | |
| sdake | when connecting directly to the service, I receive This site can’t provide a secure connection | 19:42 |
| sdake | 192.168.1.148 sent an invalid response. | 19:42 |
| sdake | ERR_SSL_PROTOCOL_ERROR | 19:42 |
| mlima_ | sdake, kolla uses vlan or vxlan? | 19:43 |
| *** dims_ has joined #kolla | 19:43 | |
| mlima_ | SamYaple ^ | 19:43 |
| sdake | mlima_ it is configurable | 19:43 |
| mlima_ | but the default value is vlan? | 19:44 |
| *** daneyon has quit IRC | 19:44 | |
| *** aginwala has joined #kolla | 19:44 | |
| mlima_ | sdake ^ | 19:45 |
| *** daneyon has joined #kolla | 19:45 | |
| *** dims has quit IRC | 19:45 | |
| vhosakot | mlima_: https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/templates/ml2_conf.ini.j2#L9 | 19:45 |
| mlima_ | oh, thank you vhosakot | 19:45 |
| vhosakot | the default tenant network uses vxlan tagging | 19:46 |
| vhosakot | ml2 has vlan configs | 19:46 |
| mlima_ | ok | 19:46 |
| vhosakot | mlima_: https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/templates/ml2_conf.ini.j2#L18-L34 | 19:46 |
| SamYaple | mlima_: to do vlans correctly youll need to use augment files | 19:46 |
| vhosakot | vxlan is configured for tenant network | 19:47 |
| *** daneyon_ has joined #kolla | 19:47 | |
| mlima_ | ok guys | 19:47 |
| vhosakot | mlima_: change type_drivers=vlan in https://github.com/openstack/kolla/blob/a755aa88e4454b2b4540c7b1e48f3e6d4e963edb/ansible/roles/neutron/templates/ml2_conf.ini.j2#L8 and reconfig neutron if you want vlan | 19:47 |
| sdake | vhosakot nah just use an augment file | 19:48 |
| sdake | non eed to chane the codebase | 19:48 |
| vhosakot | yep, augment is best | 19:48 |
| vhosakot | in /etc/kolla/config | 19:48 |
| mlima_ | i don't want to use it now, it was just a doubt | 19:48 |
| mlima_ | :) | 19:48 |
| vhosakot | also, configure network_vlan_ranges in https://github.com/openstack/kolla/blob/a755aa88e4454b2b4540c7b1e48f3e6d4e963edb/ansible/roles/neutron/templates/ml2_conf.ini.j2#L22 | 19:49 |
| vhosakot | cool | 19:49 |
| *** daneyon has quit IRC | 19:50 | |
| mlima_ | sdake, SamYaple, vhosakot, it makes sense to you https://blueprints.launchpad.net/kolla/+spec/add-data-network-interface ? | 19:50 |
| sdake | bbiaf - meeting | 19:51 |
| sdake | mlima_ i'll look when i return | 19:51 |
| SamYaple | mlima_: we already have that | 19:51 |
| vhosakot | let me read it mlima_ | 19:51 |
| mlima_ | really SamYaple? | 19:52 |
| SamYaple | mlima_: yes | 19:53 |
| SamYaple | since liberty | 19:53 |
| ccesario | the storage_interface: "{{ network_interface }}" is not for this purpose ? | 19:53 |
| SamYaple | https://github.com/openstack/kolla/blob/stable/mitaka/ansible/roles/ceph/templates/ceph.conf.j2#L5 | 19:53 |
| SamYaple | ccesario: it iss | 19:53 |
| SamYaple | mlima_: ^ see link | 19:53 |
| *** dims_ has quit IRC | 19:53 | |
| ccesario | :) | 19:54 |
| mlima_ | hmm | 19:55 |
| *** dims has joined #kolla | 19:56 | |
| mlima_ | ok guys | 19:56 |
| *** jmccarthy1 has quit IRC | 19:58 | |
| mlima_ | SamYaple, is a new ovs bridge created for storage_interface ? | 20:06 |
| sdake | SamYaple ya horizon is doa with or without yoru patch so i guess it causes no harm | 20:07 |
| *** jtriley has quit IRC | 20:07 | |
| *** shardy_ has joined #kolla | 20:07 | |
| SamYaple | mlima_: no, storage is unrelated to neutron and ovs | 20:08 |
| *** shardy has quit IRC | 20:08 | |
| *** jtriley has joined #kolla | 20:09 | |
| sbezverk | sdake why horizon is doa? I saw it working fine or it is some specific config? | 20:11 |
| openstackgerrit | Merged openstack/kolla: Swift busted with Heka https://review.openstack.org/296566 | 20:14 |
| *** achanda has quit IRC | 20:18 | |
| SamYaple | sbezverk: works for me, ubuntu-source | 20:18 |
| *** openstack has joined #kolla | 20:31 | |
| *** jtriley_ has joined #kolla | 20:38 | |
| *** jtriley has quit IRC | 20:41 | |
| *** haplo37 has quit IRC | 20:41 | |
| *** mlima_ has quit IRC | 20:47 | |
| sbezverk | vhosakot did you get any doc for log parsers development? | 20:48 |
| vhosakot | no... just googling | 20:51 |
| *** jtriley_ has quit IRC | 20:51 | |
| *** jtriley has joined #kolla | 20:54 | |
| *** shardy_ has quit IRC | 20:55 | |
| *** aginwala has quit IRC | 20:56 | |
| *** vhosakot has quit IRC | 21:00 | |
| *** achanda has joined #kolla | 21:01 | |
| *** aginwala has joined #kolla | 21:01 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: iscsi container with lvm2 support https://review.openstack.org/291285 | 21:01 |
| *** aginwala has quit IRC | 21:01 | |
| *** aginwala has joined #kolla | 21:03 | |
| *** vhosakot has joined #kolla | 21:03 | |
| sbezverk | vhosakot I see, please let me know if you find something useful | 21:05 |
| vhosakot | sbezverk: sure.. libvirt.log is not oslo based... so, it needs new logic to parse and display in Kibana UI | 21:05 |
| vhosakot | sbezverk: what log are you trying to parse ? horizon ? | 21:05 |
| vhosakot | SamYaple: sdake: https://review.openstack.org/#/c/296012/ and https://review.openstack.org/#/c/296649/ both passed gate | 21:07 |
| patchbot | vhosakot: patch 296012 - kolla (stable/mitaka) - Make heka collect libvirtd.log in nova_libvirt con... | 21:07 |
| patchbot | vhosakot: patch 296649 - kolla - Revert libvirt change in heka-openstack.toml.j2 in... | 21:07 |
| vhosakot | ah, patchbot | 21:07 |
| vhosakot | sbezverk: what log are you trying to parse ? horizon ? | 21:08 |
| *** mlima has joined #kolla | 21:08 | |
| ccesario | end shift!!! .... guys... if you test the patch, please, leave me a message .... pls... | 21:08 |
| ccesario | SamYaple, vhosakot ^^ | 21:09 |
| vhosakot | ok ccesario, to test it, I need to pass the hostname flag when creating the container... | 21:09 |
| ccesario | vhosakot, yes... | 21:10 |
| vhosakot | cool | 21:10 |
| ccesario | add hostname param | 21:10 |
| sdake | hey cr's please review reno support here https://review.openstack.org/#/c/296125/ | 21:10 |
| patchbot | sdake: patch 296125 - kolla - Add reno support | 21:10 |
| ccesario | in the task file | 21:10 |
| ccesario | vhosakot, thank you (Y) | 21:11 |
| vhosakot | np | 21:11 |
| *** inc0 has quit IRC | 21:13 | |
| *** unicell has quit IRC | 21:15 | |
| *** unicell has joined #kolla | 21:15 | |
| sbezverk | vhosakot horizon | 21:17 |
| sbezverk | for now I took the parser used for keystone | 21:17 |
| *** ccesario_ has joined #kolla | 21:18 | |
| sbezverk | should be more or less the same | 21:18 |
| vhosakot | sbezverk: ah ok | 21:18 |
| *** sdake has quit IRC | 21:27 | |
| *** dwalsh has quit IRC | 21:28 | |
| *** rhallisey has quit IRC | 21:44 | |
| *** ccesario_ has quit IRC | 21:44 | |
| *** mlima has quit IRC | 21:44 | |
| *** achanda has quit IRC | 21:46 | |
| *** ccesario__ has joined #kolla | 21:47 | |
| *** ccesario__ has quit IRC | 21:53 | |
| *** ccesario__ has joined #kolla | 21:55 | |
| *** sdake has joined #kolla | 21:55 | |
| openstackgerrit | Merged openstack/kolla: Fix inconsistancy in globals.yml https://review.openstack.org/294392 | 22:08 |
| *** alyson_ has quit IRC | 22:12 | |
| *** Serlex has quit IRC | 22:13 | |
| *** achanda has joined #kolla | 22:16 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Reduce log errors from heka https://review.openstack.org/296818 | 22:17 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Reduce log errors from heka https://review.openstack.org/296818 | 22:19 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Reduce log errors from heka https://review.openstack.org/296818 | 22:20 |
| *** vhosakot has quit IRC | 22:21 | |
| *** vhosakot has joined #kolla | 22:21 | |
| *** sdake_ has joined #kolla | 22:29 | |
| *** sdake has quit IRC | 22:30 | |
| *** manjeets has joined #kolla | 22:38 | |
| manjeets | while doing /tools/build.py it failed to build few images | 22:40 |
| Pierre_Carette | I am using mikata/stable with -t source -b ubuntu and so far so good | 22:41 |
| manjeets | Pierre_Carette has mitaka stable released for kolla | 22:43 |
| manjeets | i guess I am using master | 22:43 |
| manjeets | I am getting this when i run tools/build.py | 22:44 |
| manjeets | http://paste.openstack.org/show/491650/ | 22:44 |
| sbezverk | manjeets these images are not availble if your source is binary | 22:44 |
| Pierre_Carette | I used docker-build instead | 22:45 |
| Pierre_Carette | sorry kolla-build | 22:45 |
| Pierre_Carette | kolla-build -t source -b ubuntu | 22:45 |
| manjeets | sbezverk how do i get these working | 22:47 |
| *** Jeffrey4l has quit IRC | 22:49 | |
| *** aginwala has quit IRC | 22:49 | |
| Pierre_Carette | manjeets: yes there is a stable/mitaka branch in kolla now | 22:49 |
| SamYaple | Pierre_Carette: there is a stable mitaka branch, but it is not official yet, just fyi | 22:50 |
| SamYaple | it wont be until mid-to-late april, after all other projects have released | 22:50 |
| sbezverk | manjeets if you switch to use "source" instead of binaries you should be able to build all images | 22:51 |
| Pierre_Carette | SamYaple: it looks like the stable/liberty is missing features and I guess stable/mitaka is better than the master branch | 22:51 |
| sdake_ | Pierre_Carette a branch is not a release artiifact - those are tags | 22:55 |
| sdake_ | Pierre_Carette we will tag kolla when its ready for release | 22:55 |
| Pierre_Carette | Thanks for letting me know | 22:56 |
| sdake_ | happy to helpt | 22:56 |
| *** aginwala has joined #kolla | 22:56 | |
| sdake_ | the mai np oint of that is that there are atleast 50 bugs out standing to hit the stable/mitaka branch before we tag | 22:56 |
| Pierre_Carette | anyway, I am testing stable/mitaka and I will give feedbacks as needed | 22:57 |
| sdake_ | nice ;) | 22:57 |
| sdake_ | we are backporting as often as we find bugs and solutions for them | 22:57 |
| *** aginwala has quit IRC | 22:58 | |
| sdake_ | SamYaple this needs an ack https://review.openstack.org/#/c/296125/ | 22:59 |
| patchbot | sdake_: patch 296125 - kolla - Add reno support | 22:59 |
| *** sdake_ is now known as sdake | 22:59 | |
| sdake | jpeeler ^^ | 22:59 |
| sdake | asalkeld might be round not too ^^ | 22:59 |
| sdake | not/now | 22:59 |
| *** aginwala_ has joined #kolla | 23:03 | |
| *** aginwala_ has quit IRC | 23:05 | |
| *** aginwala has joined #kolla | 23:06 | |
| *** jtriley has quit IRC | 23:11 | |
| *** jtriley has joined #kolla | 23:17 | |
| *** absubram has quit IRC | 23:21 | |
| *** openstack has joined #kolla | 23:23 | |
| *** Pierre_Carette has quit IRC | 23:28 | |
| *** vhosakot has quit IRC | 23:34 | |
| *** ccesario__ has quit IRC | 23:41 | |
| *** aginwala has quit IRC | 23:47 | |
| *** aginwala has joined #kolla | 23:49 | |
| *** aginwala has quit IRC | 23:50 | |
| manjeets | is there a way to force delete all the build images ? | 23:55 |
| manjeets | i tried /tools/cleanup-images which is unable to remove all | 23:56 |
| *** jtriley has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!