| *** cloudnautique has quit IRC | 00:06 | |
| *** diogogmt has quit IRC | 00:42 | |
| *** sdake has joined #kolla | 00:43 | |
| *** jasonsb has quit IRC | 00:48 | |
| sdake | SamYaple around? | 00:51 |
|---|---|---|
| *** sdake has quit IRC | 00:58 | |
| *** sdake has joined #kolla | 01:03 | |
| *** blahRus has quit IRC | 01:04 | |
| *** dims has quit IRC | 01:33 | |
| *** dims has joined #kolla | 01:34 | |
| openstackgerrit | Merged openstack/kolla: Bump ansible to 1.9.2 https://review.openstack.org/233983 | 01:41 |
| openstackgerrit | Merged openstack/kolla: Fix issue with su and bad kernels https://review.openstack.org/233984 | 01:42 |
| openstackgerrit | Merged openstack/kolla: Pin docker to 1.8.2 in gate https://review.openstack.org/234125 | 01:42 |
| *** rhallisey has quit IRC | 01:42 | |
| *** tpot has joined #kolla | 01:46 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Make Ceph work for RPM based distributions https://review.openstack.org/234020 | 01:49 |
| *** daneyon has joined #kolla | 01:53 | |
| *** daneyon_ has joined #kolla | 01:56 | |
| openstackgerrit | Merged openstack/kolla: Also copy over Swift builder files for rings https://review.openstack.org/234117 | 01:58 |
| openstackgerrit | Merged openstack/kolla: Update Vagrant docs https://review.openstack.org/233371 | 01:59 |
| *** daneyon has quit IRC | 01:59 | |
| *** shardy has quit IRC | 02:01 | |
| *** dolpher has joined #kolla | 02:01 | |
| *** vinkman has quit IRC | 02:05 | |
| *** sdake has quit IRC | 02:06 | |
| openstackgerrit | Kuo-tung Kao proposed openstack/kolla: update doc for root is needed when deploying https://review.openstack.org/232381 | 02:07 |
| *** sdake has joined #kolla | 02:08 | |
| *** alisonh has quit IRC | 02:09 | |
| *** alisonh has joined #kolla | 02:24 | |
| *** sdake_ has joined #kolla | 02:29 | |
| *** sdake has quit IRC | 02:30 | |
| *** alisonh has quit IRC | 02:33 | |
| *** alisonh has joined #kolla | 02:43 | |
| *** tfukushima has joined #kolla | 02:49 | |
| *** klint has joined #kolla | 02:51 | |
| *** unicell has quit IRC | 02:53 | |
| *** CBR09 has joined #kolla | 02:54 | |
| *** achanda has joined #kolla | 02:57 | |
| *** alisonh has quit IRC | 03:02 | |
| *** alisonh has joined #kolla | 03:09 | |
| *** sdake_ has quit IRC | 03:14 | |
| *** sdake has joined #kolla | 03:23 | |
| *** diga has joined #kolla | 03:25 | |
| sdake | evening | 03:26 |
| openstackgerrit | Martin André proposed openstack/kolla: Remove outdated workaround in documentation https://review.openstack.org/234539 | 04:08 |
| SamYaple | morning | 04:10 |
| SamYaple | sdake: im not sure im ok with the multiple gate thing | 04:10 |
| SamYaple | we have a build gate. then a build+deploy gate. then a build+deploy+openstack testing gate? | 04:10 |
| SamYaple | times the number of distros we test against? | 04:11 |
| SamYaple | that quickly gets out of control | 04:11 |
| *** vinkman has joined #kolla | 04:18 | |
| sdake | i was thinkign build + deploy + test gate | 04:19 |
| sdake | and build gate only | 04:19 |
| sdake | it does become alot of gates, but makes understanding failures faster | 04:20 |
| sdake | vs | 04:20 |
| sdake | wellt he gate is bustd, f it | 04:20 |
| sdake | also I suspect the deploy gate will be flakey for some time until it becomes stabilized | 04:21 |
| sdake | i dont want the build gate to look flakey as well | 04:21 |
| sdake | because then it becomes untrusted | 04:21 |
| sdake | if it really is overwhelmig in terms of too much info down the road I'd be open to merging it once the deploy gate was as stable as the build gate | 04:22 |
| sdake | i dont think that will be the case for several months | 04:22 |
| *** diga has quit IRC | 04:24 | |
| SamYaple | I understand your point. im a bit weary of doing it though. Perhaps a compromise? A build-all-images gate (what we currently have) and a "build only the images we are going to use and deploy ansible" gate | 04:26 |
| SamYaple | this would prevent the need for an "aux" gate and a "main" gate | 04:26 |
| SamYaple | sdake: ^ | 04:27 |
| sdake | wfm | 04:27 |
| sdake | i really llike that idea | 04:28 |
| sdake | got my 10gig switch today but they only shipped 1 of 4 network cards | 04:29 |
| sdake | so i can't test it out yet | 04:29 |
| *** diga has joined #kolla | 04:29 | |
| *** jasonsb_ has joined #kolla | 04:31 | |
| sdake | what brand of nic did ou get | 04:31 |
| SamYaple | intel 520-t2 | 04:31 |
| sdake | i think i got a 420 t2 but not sure | 04:31 |
| sdake | it was a dual port copper | 04:31 |
| sdake | 400 bucks a crack | 04:31 |
| SamYaple | thats what i got, x520-t2 dual port copper | 04:31 |
| SamYaple | that sounds about right | 04:32 |
| *** tfukushima has quit IRC | 04:35 | |
| sdake | SamYaple re ceph, why is it necessary to scale downt he # of whatevers when running in aio | 04:37 |
| sdake | vs just setting it to 1 and scling it up based upon the number of osds? | 04:37 |
| SamYaple | its either that or modify the crush map | 04:39 |
| SamYaple | crush map == swift hash ring more or less for what it accomplishes | 04:39 |
| SamYaple | basically the crush map says "I wont duplicate objects on the same host" | 04:39 |
| *** jasonsb_ has quit IRC | 04:40 | |
| SamYaple | you can change that behaviour but its a bit complicated | 04:40 |
| SamYaple | easiest cleanest way is ot just not duplicate objects | 04:40 |
| *** Kennan2 has quit IRC | 04:40 | |
| sdake | not sure if i am using correct terminology, but the pools defaults to 3? | 04:41 |
| *** Kennan has joined #kolla | 04:41 | |
| SamYaple | yea the default is three copies of the object/chunk/data on three seperate hosts | 04:42 |
| sdake | can't that be dhnamic based upon the number of hosts? | 04:44 |
| sdake | or is that a pita to implement | 04:44 |
| SamYaple | so this goes back to what i was saying before, i am -2 on building out the ceph implemntation like you are talking about | 04:45 |
| SamYaple | were not here to auomate ceph | 04:45 |
| sdake | i wasn't talking about doing that | 04:45 |
| sdake | i was just asking a quesiton :) | 04:45 |
| SamYaple | but you are | 04:45 |
| SamYaple | thats what yo uare asking | 04:45 |
| sdake | today i gave a demo and cratered it because i only had one storage node | 04:46 |
| sdake | and didn't reduce the number of copies | 04:46 |
| SamYaple | you should really test that stuff | 04:46 |
| SamYaple | and that info is in the docs | 04:46 |
| sdake | yes i know | 04:46 |
| sdake | i forgot to type the command | 04:46 |
| sdake | i even hvae a cript to do it | 04:47 |
| sdake | called mp :) | 04:47 |
| SamYaple | fyi sdake docker 1.8.3 _did_ change the api mid minor release without bumping the version | 04:50 |
| SamYaple | and that broke ansible module | 04:50 |
| sdake | fail | 04:50 |
| SamYaple | so 1.8.2 -> 1.8.3 had an api change | 04:50 |
| SamYaple | so our cap for docker is 1.8.2 | 04:51 |
| SamYaple | its a hard cap | 04:51 |
| SamYaple | probably for the life of ansible 1.x | 04:51 |
| SamYaple | might be able to get it backported but no garauntee | 04:51 |
| sdake | what about docker 1.9? | 04:52 |
| SamYaple | no its a hard cap | 04:53 |
| SamYaple | no new version of docker will fix this | 04:53 |
| SamYaple | unless of course i can get the ansible docker patch backported, but got to get it merged upstream first | 04:53 |
| SamYaple | https://github.com/ansible/ansible-modules-core/pull/2258 | 04:53 |
| *** diga has quit IRC | 04:54 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Bump ansible to 1.9.2 https://review.openstack.org/234556 | 04:58 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Fix issue with su and bad kernels https://review.openstack.org/234557 | 04:58 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Pin docker to 1.8.2 in gate https://review.openstack.org/234558 | 04:58 |
| *** vinkman has quit IRC | 05:01 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 05:05 |
| openstackgerrit | Martin André proposed openstack/kolla: Cleanup outdated documentation https://review.openstack.org/234539 | 05:10 |
| *** tfukushima has joined #kolla | 05:17 | |
| sdake | SamYaple you are #12 for commits during the liberty cycle | 05:47 |
| sdake | I was #14 ;=) | 05:47 |
| SamYaple | i saw that | 05:48 |
| SamYaple | im not sure when the mitaka stats start getting calulated sdake | 05:49 |
| SamYaple | i see some projects have them already | 05:49 |
| sdake | not sure | 05:49 |
| sdake | SamYaple can you get your backport ptches in the queue | 05:50 |
| SamYaple | pretty sure theyre already up | 05:50 |
| SamYaple | sdake: https://jenkins06.openstack.org/job/gate-kolla-build-images-ubuntu-source/138/consoleFull | 05:51 |
| SamYaple | thats why the bootstrap container is existing | 05:51 |
| SamYaple | weird :/ ive had a few successfuly deploy | 05:51 |
| SamYaple | but never on hp hardware | 05:51 |
| openstackgerrit | Merged openstack/kolla: Cleanup outdated documentation https://review.openstack.org/234539 | 05:52 |
| sdake | looks like the db looses info | 05:52 |
| SamYaple | no the apache process is crashing | 05:53 |
| sdake | oh i didn't see that let me check it out | 05:53 |
| SamYaple | HTTP 500 is from apache | 05:53 |
| SamYaple | or keystone trying to reach a dead apache should i say | 05:53 |
| SamYaple | i have to cat the logs now | 05:53 |
| SamYaple | we need to have a neutron/nova logging structure so we can view the services logs | 05:53 |
| SamYaple | http://logs.openstack.org/35/234435/2/check/gate-neutron-dsvm-api/8e10e0a/ | 05:54 |
| SamYaple | look there, lots of logs | 05:54 |
| sdake | it returns a 500 | 05:54 |
| sdake | i think apache is up and keystone is down | 05:54 |
| SamYaple | thats not how that works at all | 05:54 |
| SamYaple | apache is keystone | 05:54 |
| SamYaple | thats its only task | 05:55 |
| SamYaple | it cant be up without the other | 05:55 |
| sdake | ok | 05:55 |
| sdake | maybe oom or out of disk | 05:55 |
| SamYaple | maybe _but_ ive had successful deploys | 05:55 |
| SamYaple | and hp actually has 500gb disks where rax has 80 | 05:56 |
| sdake | but only on one hardware type? | 05:56 |
| SamYaple | yea on in rax environment | 05:56 |
| sdake | they are different images iirc | 05:56 |
| sdake | hp and rax hae completely differnet bsae images | 05:57 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 05:57 |
| SamYaple | yea xen vs kvm | 05:57 |
| sdake | hp is running xen? | 05:57 |
| SamYaple | you have that backwards | 05:58 |
| SamYaple | rax runs xen | 05:58 |
| SamYaple | (look at the device names /dev/xvde vs /dev/vda or /dev/sda) | 05:58 |
| sdake | that is why it was a quesstio nrather then a statement ;) | 05:58 |
| *** jmccarthy has joined #kolla | 05:59 | |
| sdake | did you upgrade the kernels on the box at all? | 05:59 |
| SamYaple | cant upgrade kernels in the gate | 05:59 |
| sdake | same docker storage driver type? | 06:00 |
| SamYaple | yea doubtful that its anything like that | 06:01 |
| sdake | i have seen wierd failures if there is not enough extra disk space beyond what is actually needed with lvm | 06:01 |
| SamYaple | see http://logs.openstack.org/81/231881/44/check/gate-kolla-build-images-ubuntu-source/444e2c6/console.html | 06:01 |
| SamYaple | thats a successful run | 06:01 |
| SamYaple | ah that one was hp | 06:01 |
| SamYaple | wierd | 06:01 |
| sdake | 2015-10-12 10:22:57.144 | + docker exec rabbitmq cat '/var/log/rabbitmq/*' | 06:02 |
| sdake | i have never seen docker exec with a wildcard work correclty | 06:02 |
| SamYaple | it does | 06:03 |
| SamYaple | but that was just debug info | 06:03 |
| SamYaple | oh you need to not be expanding * on your shell for it to work sdake | 06:03 |
| SamYaple | you probably expand it external to the container | 06:03 |
| sdake | is tht some type of environment variable setting? | 06:04 |
| SamYaple | do you use bash? | 06:04 |
| sdake | yup | 06:04 |
| SamYaple | eya you probably want to leave it alone | 06:04 |
| SamYaple | you could make it work but itll break other things | 06:04 |
| sdake | probalby need to run the gate again to get the hp failure | 06:06 |
| sdake | so it can be debugged | 06:06 |
| *** jmccarthy1 has joined #kolla | 06:10 | |
| *** jmccarthy has quit IRC | 06:10 | |
| *** diga has joined #kolla | 06:11 | |
| *** unicell has joined #kolla | 06:13 | |
| *** jmccarthy1 has quit IRC | 06:13 | |
| openstackgerrit | Merged openstack/kolla: Bump ansible to 1.9.2 https://review.openstack.org/234556 | 06:16 |
| openstackgerrit | Merged openstack/kolla: Fix issue with su and bad kernels https://review.openstack.org/234557 | 06:17 |
| SamYaple | yea the container is dying | 06:17 |
| openstackgerrit | Merged openstack/kolla: Pin docker to 1.8.2 in gate https://review.openstack.org/234558 | 06:20 |
| sdake | i need to get the het template updated | 06:25 |
| sdake | thi will allow scale testing of kolla | 06:25 |
| sdake | on my hardware alone i could test 30 nodes i think | 06:25 |
| SamYaple | my two nodes i can do ~100 instances before ram is an issue | 06:26 |
| SamYaple | the disk is only an issue if they are hammering it | 06:26 |
| sdake | no i mean 25+ compute nodes | 06:26 |
| SamYaple | you have 30 nodes at your house? | 06:27 |
| sdake | i could deploy a 3 node openstack | 06:27 |
| sdake | and then deploy 30 node openstack on top of that using the heat template | 06:27 |
| *** tpot has quit IRC | 06:27 | |
| sdake | or in rackspace cloud spend 30 bucks for a day of 100 nodes with 8gb ram each | 06:28 |
| sdake | i want a measurement of how long it takes to deploy 100 nodes | 06:30 |
| sdake | and whether kolla works at that scale | 06:31 |
| *** jmccarthy has joined #kolla | 06:32 | |
| sdake | hey jmccarthy | 06:32 |
| jmccarthy | o/ | 06:33 |
| SamYaple | i mean thats just not going ot be accurate sdake | 06:33 |
| jmccarthy | Hiyas ! | 06:34 |
| sdake | it will show worst case | 06:34 |
| SamYaple | it wont even show that | 06:34 |
| sdake | why not? | 06:34 |
| SamYaple | baremetal _could_ be slower and its dependant on so many other factors | 06:34 |
| SamYaple | bandwidth no withstanding | 06:34 |
| sdake | i dont see how baremetal coul d be slower | 06:35 |
| SamYaple | slower disks and/or resource contention | 06:35 |
| sdake | if it did work at 100 nodes, then we know for certain it would work on bare metal | 06:36 |
| sdake | (100 nodes virt) | 06:36 |
| SamYaple | define work | 06:36 |
| SamYaple | eh it doesnt matter | 06:36 |
| sdake | launch steak-rg with 100 vms | 06:36 |
| sdake | as in not fall over | 06:36 |
| SamYaple | its just not a valid test for real world is my point | 06:37 |
| SamYaple | no kvm (or tainted kvm) | 06:37 |
| SamYaple | no vlan | 06:37 |
| sdake | ya agree its not apple to apples | 06:37 |
| sdake | but its not apple to oranges either ;-) | 06:38 |
| SamYaple | apples to pears maybe? | 06:38 |
| sdake | i was just thinking that! | 06:38 |
| SamYaple | great minds think alike. and apparently ours do too | 06:38 |
| sdake | sever your connection as soon as possible samyaple !:) | 06:39 |
| sdake | i played with fio today | 06:40 |
| sdake | pretty cool tool | 06:40 |
| SamYaple | oh youve never used it? | 06:40 |
| sdake | nope | 06:40 |
| SamYaple | i think its the goto for most if not all disk benchmarks in linux | 06:40 |
| sdake | my 750 1.2tb does 100k iops | 06:40 |
| sdake | 100k read 25k writ eiirc | 06:41 |
| SamYaple | esh | 06:41 |
| SamYaple | i do far more writes | 06:41 |
| sdake | perhaps we hace different parameters | 06:41 |
| sdake | ./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75 | 06:42 |
| SamYaple | yea i do ~200k writes and 400k reads | 06:42 |
| *** sdake_ has joined #kolla | 06:43 | |
| sdake_ | with that command? | 06:43 |
| sdake_ | read : io=3071.7MB, bw=640739KB/s, iops=160184, runt= 4909msec | 06:43 |
| sdake_ | write: io=1024.4MB, bw=213672KB/s, iops=53418, runt= 4909msec | 06:43 |
| SamYaple | fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=5 | 06:43 |
| SamYaple | thats the one i do | 06:44 |
| SamYaple | well sorta | 06:44 |
| SamYaple | i jack up the size | 06:44 |
| sdake_ | read : io=210300KB, bw=41768KB/s, iops=10441, runt= 5035msec | 06:44 |
| sdake_ | write: io=3890.7MB, bw=791262KB/s, iops=197815, runt= 5035msec | 06:44 |
| SamYaple | oh yea what filesystem are you running? | 06:45 |
| sdake_ | ext4 | 06:45 |
| sdake_ | read ios slow onext4 it appears | 06:45 |
| sdake_ | iops | 06:45 |
| *** sdake has quit IRC | 06:46 | |
| SamYaple | yea that test is way to short for you | 06:46 |
| sdake_ | how should i change it | 06:47 |
| SamYaple | --size=40G or something | 06:47 |
| SamYaple | 5 seconds is not enough time to run | 06:47 |
| sdake_ | is that the rwmixread=5? | 06:47 |
| SamYaple | i typically do a 60 second cap if im comparing different hardware types | 06:47 |
| SamYaple | nah you can do 75 | 06:47 |
| sdake_ | read : io=3071.7MB, bw=616864KB/s, iops=154215, runt= 5099msec | 06:48 |
| sdake_ | write: io=1024.4MB, bw=205710KB/s, iops=51427, runt= 5099msec | 06:48 |
| sdake_ | how do you change the runtime, it still took like 5 seconds | 06:48 |
| SamYaple | you need to change the size | 06:48 |
| SamYaple | you didnt | 06:48 |
| sdake_ | i see tryign now | 06:48 |
| SamYaple | --rwmixread= thats your read/write profile | 06:49 |
| SamYaple | its a percentage | 06:49 |
| SamYaple | 75 means 75 percent read/25% write | 06:49 |
| sdake_ | read : io=30723MB, bw=627555KB/s, iops=156888, runt= 50132msec | 06:50 |
| sdake_ | write: io=10237MB, bw=209097KB/s, iops=52274, runt= 50132msec | 06:50 |
| sdake_ | with size=40gb | 06:50 |
| SamYaple | seems more reasonable | 06:50 |
| SamYaple | mind that you are mixing reads and writes and not getting a benchmark on either | 06:51 |
| sdake_ | my vm did like 1k iops.. lol | 06:51 |
| SamYaple | you need to into tuning | 06:52 |
| SamYaple | you are hitting the qemu virtio driver limit | 06:52 |
| SamYaple | you need to use the scsi driver for sure | 06:52 |
| sdake_ | well i did have a 1gb network | 06:52 |
| SamYaple | doesnt matter | 06:52 |
| SamYaple | you arent bandwidth bound on 4k writes | 06:52 |
| SamYaple | 4k * 1000 == 4MB | 06:52 |
| SamYaple | so you were using 32mbit of 1000mbit | 06:53 |
| sdake_ | virtio has built in limits? | 06:53 |
| SamYaple | no it has software limitations | 06:53 |
| SamYaple | you need to us ethe scsi driver | 06:53 |
| sdake_ | is that a config option in openstak? | 06:54 |
| SamYaple | glance image-update --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi <glance_image_id> | 06:54 |
| SamYaple | then boot a new instance on that image | 06:54 |
| SamYaple | you should see a rise to ~5k iops | 06:54 |
| sdake_ | cool well i'd do it now | 06:55 |
| sdake_ | but i've hit the wall | 06:55 |
| sdake_ | time fo rbed | 06:55 |
| sdake_ | night all | 06:55 |
| SamYaple | night | 06:55 |
| *** shardy has joined #kolla | 06:57 | |
| *** sdake_ has quit IRC | 06:57 | |
| *** openstackgerrit has quit IRC | 07:01 | |
| *** openstackgerrit has joined #kolla | 07:01 | |
| *** unicell1 has joined #kolla | 07:09 | |
| *** unicell has quit IRC | 07:10 | |
| SamYaple | ughghhhghghgh | 07:11 |
| SamYaple | keystone is worked mandre | 07:11 |
| SamYaple | 2015-10-14 07:10:25.933600 ContextualVersionConflict: (WebOb 1.5.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('WebOb<1.5.0,>=1.2.3'), set(['keystonemiddleware'])) | 07:11 |
| *** tfukushima has quit IRC | 07:14 | |
| *** tfukushima has joined #kolla | 07:14 | |
| *** jmccarthy has quit IRC | 07:21 | |
| *** achanda has quit IRC | 07:26 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 07:27 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Use openstack/requirements for repeatable builds https://review.openstack.org/233734 | 07:27 |
| *** achanda has joined #kolla | 07:27 | |
| *** achanda has quit IRC | 07:33 | |
| kjelly | Now when building centos-binary, it show "Cannot open: http://www.percona.com/redir/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm. Skipping.", Does anyone have the same problem ? | 07:33 |
| kjelly | No, maybe my docker problem. | 07:33 |
| SamYaple | kjelly: you can check the gate logs | 07:34 |
| *** tfukushima has quit IRC | 07:34 | |
| mandre | SamYaple, the deployment gate already proves to be super useful | 07:35 |
| kjelly | SamYaple: Ok, thanks. | 07:36 |
| mandre | i'm confused... it's going to pull oslo.log==1.11.0, isn't is going to break? | 07:37 |
| mandre | should we revert the revert? | 07:37 |
| SamYaple | mandre: yes. the correct action here is to go upstream to requirements and submit a patch | 07:37 |
| SamYaple | problably to block 1.11.0 | 07:37 |
| SamYaple | ill work on that now | 07:37 |
| *** gfidente has joined #kolla | 07:38 | |
| mandre | thx | 07:38 |
| *** akwasnie has joined #kolla | 07:41 | |
| *** inc0 has joined #kolla | 07:48 | |
| *** tfukushima has joined #kolla | 07:53 | |
| *** athomas has joined #kolla | 08:08 | |
| *** mbound has joined #kolla | 08:15 | |
| openstackgerrit | Kuo-tung Kao proposed openstack/kolla: add "registry" flag to "tools/build.py" https://review.openstack.org/234629 | 08:26 |
| SamYaple | mandre: can you confirm centos is working? | 08:26 |
| SamYaple | I think the sudo patch may have busted centos | 08:26 |
| SamYaple | since i cant test centos i have no way of knowing | 08:27 |
| *** pbourke has quit IRC | 08:57 | |
| *** pbourke has joined #kolla | 08:58 | |
| mandre | SamYaple, rebuilding now | 09:00 |
| mandre | i'm heading home, bbl | 09:00 |
| *** athomas has quit IRC | 09:05 | |
| SamYaple | mandre: cool cool | 09:06 |
| SamYaple | mandre: i have a fix if that is broken btw | 09:06 |
| *** athomas has joined #kolla | 09:14 | |
| *** dims_ has joined #kolla | 09:17 | |
| *** dims has quit IRC | 09:20 | |
| *** tpot has joined #kolla | 09:26 | |
| *** jmccarthy has joined #kolla | 09:27 | |
| *** Kennan has quit IRC | 09:47 | |
| *** Kennan has joined #kolla | 09:48 | |
| *** jmccarthy1 has joined #kolla | 09:49 | |
| *** jmccarthy1 has quit IRC | 09:50 | |
| *** jmccarthy1 has joined #kolla | 09:50 | |
| *** jmccarthy1 has quit IRC | 09:52 | |
| *** jmccarthy2 has joined #kolla | 09:52 | |
| *** jmccarthy has quit IRC | 09:52 | |
| *** inc0 has quit IRC | 09:56 | |
| *** inc0 has joined #kolla | 09:57 | |
| *** inc0 has quit IRC | 09:59 | |
| *** inc0_ has joined #kolla | 09:59 | |
| *** jmccarthy2 has quit IRC | 10:04 | |
| *** tfukushima has quit IRC | 10:04 | |
| *** tfukushima has joined #kolla | 10:04 | |
| *** athomas has quit IRC | 10:05 | |
| *** inc0_ has quit IRC | 10:08 | |
| *** dolpher has quit IRC | 10:09 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 10:12 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add tty to containers https://review.openstack.org/234703 | 10:12 |
| *** jmccarthy has joined #kolla | 10:17 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add tty to containers https://review.openstack.org/234703 | 10:21 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 10:21 |
| *** tpot has quit IRC | 10:23 | |
| SamYaple | mandre: yea that appears to be the case. centos is bust | 10:52 |
| SamYaple | pbourke: you around? | 10:52 |
| pbourke | SamYaple: hi | 10:52 |
| SamYaple | so the sudo patch from yesterday appears to have busted centos | 10:53 |
| pbourke | tty issue? | 10:53 |
| SamYaple | i have a patch that ive tested in the gate that makes it work | 10:53 |
| SamYaple | yea | 10:53 |
| pbourke | already +2'ed it | 10:53 |
| SamYaple | right but i have to rebase away from your global reqs patch | 10:53 |
| SamYaple | i was just testing the gate | 10:53 |
| SamYaple | good news is gate 100% works with that patch! | 10:54 |
| SamYaple | full deploy for ubuntu and centos | 10:54 |
| SamYaple | source and binary | 10:54 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add tty to containers https://review.openstack.org/234703 | 10:54 |
| SamYaple | oh cool it preserved your +2 | 10:54 |
| *** shardy is now known as shardy_afk | 10:55 | |
| SamYaple | pbourke: if you could review https://review.openstack.org/#/c/234020/ | 10:56 |
| SamYaple | we need that to merge upstream before we backport | 10:56 |
| SamYaple | and we tag tomorrow | 10:56 |
| *** diga has quit IRC | 10:57 | |
| openstackgerrit | Merged openstack/kolla: Make Ceph work for RPM based distributions https://review.openstack.org/234020 | 10:58 |
| SamYaple | thanks ill backport it now | 10:58 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Ceph work for RPM based distributions https://review.openstack.org/234717 | 10:59 |
| SamYaple | pbourke: https://review.openstack.org/#/c/233993/ | 11:07 |
| SamYaple | look at that! | 11:07 |
| SamYaple | full ansible deploy | 11:07 |
| SamYaple | all green | 11:07 |
| pbourke | awesome | 11:08 |
| pbourke | ubuntu a fair bit slower than centos? | 11:08 |
| *** tfukushima has quit IRC | 11:08 | |
| SamYaple | the package fetching fluctuates for both of them | 11:09 |
| SamYaple | sometimes ubuntus faster sometimes not | 11:09 |
| SamYaple | its always the package fetching | 11:09 |
| SamYaple | hp has really slow mirrors too | 11:09 |
| pbourke | yeah was thinking | 11:09 |
| SamYaple | if you watch which environment they get launched in you can predict the run time | 11:10 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Also copy over Swift builder files for rings https://review.openstack.org/234721 | 11:11 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add small reminder to use latest vagrant https://review.openstack.org/234722 | 11:11 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Neutron-server should be on control group https://review.openstack.org/234723 | 11:11 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Remove some outdated info from swift-guide https://review.openstack.org/234724 | 11:11 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Swift JSON files looking for rings in the wrong place https://review.openstack.org/234725 | 11:11 |
| SamYaple | so im coming close to a oneliner that will lookup all backports and purpose them to the stable branch | 11:12 |
| SamYaple | that was done with 2 commands and just a bit of manual ^ | 11:13 |
| openstackgerrit | Merged openstack/kolla: update doc for root is needed when deploying https://review.openstack.org/232381 | 11:21 |
| *** tfukushima has joined #kolla | 11:24 | |
| *** tfukushima has quit IRC | 11:24 | |
| *** dwalsh has joined #kolla | 11:27 | |
| *** bizarrochristy has joined #kolla | 12:03 | |
| *** cemason has quit IRC | 12:03 | |
| *** bizarrochristy has quit IRC | 12:12 | |
| *** achanda has joined #kolla | 12:16 | |
| *** achanda has quit IRC | 12:17 | |
| *** rhallisey has joined #kolla | 12:18 | |
| *** achanda has joined #kolla | 12:18 | |
| *** achanda has quit IRC | 12:22 | |
| *** cemason has joined #kolla | 12:25 | |
| *** inc0 has joined #kolla | 12:29 | |
| *** cemason has quit IRC | 12:30 | |
| *** cemason has joined #kolla | 12:33 | |
| *** athomas has joined #kolla | 12:34 | |
| *** cemason has quit IRC | 12:37 | |
| *** shardy_a1k has joined #kolla | 12:39 | |
| *** shardy_afk has quit IRC | 12:41 | |
| *** shardy_a1k has quit IRC | 12:45 | |
| *** shardy_afk has joined #kolla | 12:46 | |
| *** inc0 has quit IRC | 12:46 | |
| *** klint has quit IRC | 13:18 | |
| *** shardy_afk is now known as shardy | 13:47 | |
| *** Hui has joined #kolla | 13:50 | |
| Hui | Good morning | 13:50 |
| *** Hui has quit IRC | 13:54 | |
| *** dims_ has quit IRC | 14:01 | |
| *** dims has joined #kolla | 14:01 | |
| *** Hui has joined #kolla | 14:10 | |
| *** cemason has joined #kolla | 14:26 | |
| *** nihilifer has quit IRC | 14:32 | |
| *** jasonsb_ has joined #kolla | 14:32 | |
| *** nihilifer has joined #kolla | 14:33 | |
| *** diogogmt has joined #kolla | 14:39 | |
| *** CBR09 has quit IRC | 14:43 | |
| SamYaple | morning harmw | 14:43 |
| SamYaple | morning Hui | 14:44 |
| *** cemason has quit IRC | 14:45 | |
| Hui | Morning Sam | 14:47 |
| *** jruano has joined #kolla | 14:48 | |
| SamYaple | pbourke: re: memcache | 14:50 |
| *** cemason has joined #kolla | 14:50 | |
| SamYaple | it looks like we are going to have to use it for nova-consoleauth so im going to break it out into a role on its own and not a dependancy like it is | 14:50 |
| Hui | Hi, Sam, I met a problem. Could you help take a look at this https://bugs.launchpad.net/kolla/+bug/1506099 | 14:51 |
| openstack | Launchpad bug 1506099 in kolla "Ansible creating duplicate container causes error" [Undecided,New] | 14:51 |
| SamYaple | Hui: docker screwed up upstream | 14:52 |
| SamYaple | docker module with ansible wont work with docker 1.8.3 or above | 14:53 |
| SamYaple | use docker 1.8.2 | 14:53 |
| SamYaple | https://github.com/ansible/ansible-modules-core/pull/2258 | 14:53 |
| Hui | ok | 14:55 |
| Hui | let me try that | 14:55 |
| *** shardy_ has joined #kolla | 14:56 | |
| *** shardy has quit IRC | 14:57 | |
| *** shardy_ has quit IRC | 15:01 | |
| *** cemason has quit IRC | 15:01 | |
| *** shardy has joined #kolla | 15:02 | |
| *** cemason has joined #kolla | 15:03 | |
| mandre | SamYaple, I had a strange issue during ansible deploy, containers failing because they couldn't find config file | 15:04 |
| mandre | i've kicked a new build from a fresh environment but it's taking ages | 15:04 |
| SamYaple | mandre: see above | 15:04 |
| SamYaple | use docker 1.8.2 docker screwed up | 15:05 |
| mandre | ah, i haven't realized it was related | 15:05 |
| *** inc0 has joined #kolla | 15:06 | |
| SamYaple | mandre: feel free to put presure on the PR to fix the issue https://github.com/ansible/ansible-modules-core/pull/2258 | 15:07 |
| *** cemason has quit IRC | 15:07 | |
| inc0 | o/ | 15:07 |
| SamYaple | mandre: i also confirmed the centos issue and provided a fix here https://review.openstack.org/#/c/234703/ | 15:07 |
| SamYaple | morning inc0 | 15:07 |
| inc0 | afternoon Sam | 15:07 |
| Hui | Hi, SamYaple, docker 1.8.2 works | 15:07 |
| SamYaple | :) | 15:07 |
| nihilifer | akwasnie: inc0: will you be on PyCon PL? :) | 15:08 |
| mandre | i'll give it a try tomorrow, my build should be finished by then | 15:08 |
| inc0 | nihilifer, unfortunately no, too much happening right now | 15:08 |
| *** mbound has quit IRC | 15:27 | |
| SamYaple | hey everyone. we are tagging tomorrow so if everyone can spare some time on the queue and bugs today that would be fantastic | 15:28 |
| *** sdake has joined #kolla | 15:29 | |
| SamYaple | fyi docker 1.8.3 is broken with ansible and centos is broken in both liberty _and_ master without this patch https://review.openstack.org/#/c/234703/ | 15:29 |
| SamYaple | so i would say that patch is priorty | 15:29 |
| sdake | morning | 15:29 |
| SamYaple | morning sdake | 15:30 |
| SamYaple | did you see my above messages? | 15:30 |
| sdake | last one about a aptch | 15:30 |
| sdake | lookign now | 15:30 |
| *** jtriley has joined #kolla | 15:30 | |
| sdake | did you want me to test that or just approve it | 15:31 |
| *** cemason has joined #kolla | 15:31 | |
| SamYaple | i have tested it in a successful gate deploy | 15:32 |
| SamYaple | https://review.openstack.org/#/c/233993/ | 15:33 |
| SamYaple | feel free to review the logs and or test | 15:33 |
| SamYaple | the keystone bootstrap issue was upstream bug btw | 15:33 |
| *** bizarrochristy has joined #kolla | 15:36 | |
| *** sdake has quit IRC | 15:36 | |
| openstackgerrit | Merged openstack/kolla: Add tty to containers https://review.openstack.org/234703 | 15:37 |
| *** sdake has joined #kolla | 15:37 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add tty to containers https://review.openstack.org/234843 | 15:38 |
| *** cemason has quit IRC | 15:39 | |
| *** cemason_ has joined #kolla | 15:39 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: In gate functional testing https://review.openstack.org/233993 | 15:41 |
| SamYaple | sdake: nova just fixed upstream and it merged so the gate _should_ succeed now | 15:41 |
| *** bizarrochristy has quit IRC | 15:43 | |
| *** sdake_ has joined #kolla | 16:00 | |
| *** sdake has quit IRC | 16:01 | |
| *** jtriley has quit IRC | 16:03 | |
| *** inc0 has quit IRC | 16:06 | |
| *** inc0 has joined #kolla | 16:07 | |
| *** cemason_ has quit IRC | 16:10 | |
| *** cemason has joined #kolla | 16:12 | |
| SamYaple | the meeting is today right? | 16:17 |
| SamYaple | in like 10 minutes? | 16:17 |
| SamYaple | pbourke: sdake_ inc0 ? | 16:17 |
| pbourke | yes | 16:17 |
| SamYaple | ok cool | 16:17 |
| sdake_ | yup | 16:17 |
| SamYaple | thanks im really tired | 16:17 |
| *** sdake_ is now known as sdake | 16:18 | |
| inc0 | sdake, we have ops summit session about infra containers at the same time as kolla work session | 16:19 |
| *** akwasnie has quit IRC | 16:19 | |
| inc0 | anyway, damn, out of 3 ops summit session I want/need to be, all of them conflicts with kolla work sessions :/ | 16:20 |
| SamYaple | which ones | 16:21 |
| inc0 | infra containers, end-user containers and upgrades | 16:22 |
| *** jtriley has joined #kolla | 16:22 | |
| *** shardy has quit IRC | 16:25 | |
| *** dwalsh has quit IRC | 16:34 | |
| *** jasonsb_ has quit IRC | 16:37 | |
| *** cemason has quit IRC | 16:38 | |
| *** SamYaple_ has joined #kolla | 16:39 | |
| *** bigjools_ has joined #kolla | 16:40 | |
| *** jtriley has quit IRC | 16:40 | |
| *** sdake has quit IRC | 16:43 | |
| *** thomasem_ has joined #kolla | 16:44 | |
| *** another_larsks has joined #kolla | 16:44 | |
| *** pbourke-mobile has joined #kolla | 16:46 | |
| *** sdake has joined #kolla | 16:49 | |
| *** jmccarthy has quit IRC | 16:50 | |
| *** bmace__ has quit IRC | 16:50 | |
| *** SamYaple has quit IRC | 16:50 | |
| *** larsks has quit IRC | 16:50 | |
| *** vincent_vdk has quit IRC | 16:50 | |
| *** bigjools has quit IRC | 16:50 | |
| *** thomasem has quit IRC | 16:50 | |
| *** Hui has quit IRC | 16:50 | |
| *** unicell1 has quit IRC | 16:50 | |
| *** shardy has joined #kolla | 16:50 | |
| *** vinkman has joined #kolla | 16:51 | |
| *** SamYaple_ is now known as SamYaple | 16:53 | |
| *** another_larsks is now known as larsks | 16:53 | |
| *** cemason has joined #kolla | 16:56 | |
| *** jtriley has joined #kolla | 16:59 | |
| *** blahRus has joined #kolla | 16:59 | |
| *** Hui has joined #kolla | 16:59 | |
| *** pbourke_ has joined #kolla | 17:02 | |
| *** jmccarthy has joined #kolla | 17:02 | |
| *** jtriley has quit IRC | 17:03 | |
| *** pbourke-mobile has quit IRC | 17:04 | |
| *** pbourke has quit IRC | 17:06 | |
| *** blahRus has quit IRC | 17:07 | |
| *** Hui has quit IRC | 17:07 | |
| *** vincent_vdk has joined #kolla | 17:07 | |
| *** bmace__ has joined #kolla | 17:07 | |
| *** cemason has quit IRC | 17:08 | |
| *** blahRus has joined #kolla | 17:10 | |
| *** Hui has joined #kolla | 17:10 | |
| openstackgerrit | Merged openstack/kolla: Add tty to containers https://review.openstack.org/234843 | 17:10 |
| *** shardy has quit IRC | 17:14 | |
| *** athomas has quit IRC | 17:16 | |
| *** unicell has joined #kolla | 17:17 | |
| *** achanda has joined #kolla | 17:17 | |
| *** Hui has quit IRC | 17:18 | |
| *** cemason has joined #kolla | 17:23 | |
| inc0 | I'm going off guys, cya tomorrow | 17:24 |
| rhallisey | inc0, see ya | 17:24 |
| SamYaple | me too | 17:25 |
| *** pbourke_ has quit IRC | 17:25 | |
| SamYaple | bye everyone | 17:25 |
| rhallisey | bye | 17:25 |
| *** Hui has joined #kolla | 17:25 | |
| Hui | hi, INFO:__main__:The config strategy prevents copying new configs | 17:26 |
| Hui | Running command: 'sudo -H -u rabbitmq /usr/sbin/rabbitmq-server' | 17:26 |
| Hui | sudo: sorry, you must have a tty to run sudo | 17:26 |
| Hui | is there any change related to rabbitmq image? | 17:26 |
| jpeeler | that was just fixed i believe | 17:26 |
| *** jasonsb has joined #kolla | 17:26 | |
| SamYaple | Hui: that was all just fixed and merged upstream | 17:26 |
| SamYaple | Hui: no need to rebuild containers, just `git pull` and rerun asnbiel | 17:27 |
| Hui | hi, jpeeler, SamYaple, nice | 17:27 |
| jpeeler | specifically 17 minutes ago | 17:27 |
| Hui | let me try them | 17:27 |
| *** inc0 has quit IRC | 17:29 | |
| *** cemason has quit IRC | 17:32 | |
| Hui | mkdir: cannot create directory '/var/lib/rabbitmq/mnesia': Permission denied | 17:37 |
| Hui | Hi, SamYaple, still some error with the rabbitmq container | 17:37 |
| *** sdake has quit IRC | 17:39 | |
| *** dwalsh has joined #kolla | 17:49 | |
| Hui | the rabbitmq problem is solved after restarting the rabbitmq_data container | 17:50 |
| *** thumpba has joined #kolla | 17:51 | |
| *** jruano has quit IRC | 18:06 | |
| *** jruano has joined #kolla | 18:14 | |
| *** vinkman has quit IRC | 18:23 | |
| *** achanda has quit IRC | 18:25 | |
| *** thumpba has quit IRC | 18:29 | |
| *** sdake has joined #kolla | 18:38 | |
| *** sdake has quit IRC | 18:38 | |
| *** sdake has joined #kolla | 18:39 | |
| *** jruano has quit IRC | 18:42 | |
| *** sdake_ has joined #kolla | 18:43 | |
| *** sdake has quit IRC | 18:43 | |
| *** vinkman has joined #kolla | 18:45 | |
| *** sdake_ has quit IRC | 18:51 | |
| *** sdake has joined #kolla | 18:51 | |
| *** dwalsh has quit IRC | 18:57 | |
| *** thumpba has joined #kolla | 19:04 | |
| *** achanda has joined #kolla | 19:09 | |
| *** dwalsh has joined #kolla | 19:18 | |
| *** akwasnie has joined #kolla | 19:28 | |
| *** sdake has quit IRC | 19:35 | |
| *** sdake has joined #kolla | 19:36 | |
| *** thumpba has quit IRC | 19:55 | |
| *** achanda has quit IRC | 20:00 | |
| *** dwalsh has quit IRC | 20:01 | |
| Hui | Hi, neurtron fails; it shows n raise errorclass, errorvalue | 20:14 |
| Hui | 2015-10-14 20:11:41.780 1 ERROR neutron ProgrammingError: (ProgrammingError) (1146, "Table 'neutron.ml2_vlan_allocations' doesn't exist") 'SELECT ml2_vlan_allocations.physical_network AS ml2_vlan_allocations_physical_network, ml2_vlan_allocations.vlan_id AS ml2_vlan_allocations_vlan_id, ml2_vlan_allocations.allocated AS ml2_vlan_allocations_allocated \nFROM ml2_vlan_allocations FOR UPDATE' () | 20:14 |
| *** gfidente has quit IRC | 20:17 | |
| nihilifer | Hui: it means that migration didn't went well | 20:28 |
| nihilifer | didn't ansible deployment show any error? | 20:28 |
| *** akwasnie has quit IRC | 20:31 | |
| Hui | no | 20:33 |
| Hui | hi, nihilifer, ansible play finishes correctly | 20:34 |
| Hui | Shall I file a bug about this? | 20:34 |
| *** achanda has joined #kolla | 20:41 | |
| *** achanda has quit IRC | 20:41 | |
| *** akwasnie has joined #kolla | 20:42 | |
| *** achanda has joined #kolla | 20:44 | |
| *** akwasnie has quit IRC | 20:48 | |
| *** alisonh has quit IRC | 20:51 | |
| *** tpot has joined #kolla | 20:52 | |
| *** achanda has quit IRC | 20:56 | |
| *** sdake_ has joined #kolla | 20:57 | |
| *** sdake has quit IRC | 20:59 | |
| *** rhallisey has quit IRC | 20:59 | |
| *** dwalsh has joined #kolla | 20:59 | |
| *** sdake_ has quit IRC | 21:02 | |
| *** thumpba has joined #kolla | 21:05 | |
| *** alisonh has joined #kolla | 21:09 | |
| *** achanda has joined #kolla | 21:12 | |
| *** dwalsh has quit IRC | 21:33 | |
| *** achanda has quit IRC | 21:48 | |
| *** bigjools_ is now known as bigjools | 21:52 | |
| *** bigjools has quit IRC | 21:52 | |
| *** bigjools has joined #kolla | 21:52 | |
| *** bkosick has joined #kolla | 21:55 | |
| *** bkosick has quit IRC | 22:00 | |
| *** dtturner has joined #kolla | 22:01 | |
| *** achanda has joined #kolla | 22:15 | |
| *** alisonh has quit IRC | 22:22 | |
| *** alisonh has joined #kolla | 22:27 | |
| *** jtriley has joined #kolla | 22:45 | |
| *** achanda has quit IRC | 22:45 | |
| *** openstackgerrit has quit IRC | 22:46 | |
| *** openstackgerrit has joined #kolla | 22:46 | |
| *** achanda has joined #kolla | 22:47 | |
| *** achanda has quit IRC | 22:47 | |
| *** jtriley has quit IRC | 23:03 | |
| *** achanda has joined #kolla | 23:09 | |
| *** jtriley has joined #kolla | 23:14 | |
| *** blahRus has quit IRC | 23:18 | |
| *** Hui has quit IRC | 23:19 | |
| *** dims_ has joined #kolla | 23:19 | |
| *** dims has quit IRC | 23:22 | |
| *** jtriley has quit IRC | 23:36 | |
| *** jtriley has joined #kolla | 23:51 | |
| *** rhallisey has joined #kolla | 23:53 | |
| *** rhallisey has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!