*** jasonsb has quit IRC | 00:08 | |
*** achanda has joined #kolla | 00:11 | |
*** weiyu_ has joined #kolla | 00:15 | |
*** achanda has quit IRC | 00:19 | |
*** Haomeng has joined #kolla | 00:19 | |
*** rhallisey has joined #kolla | 00:25 | |
*** achanda has joined #kolla | 00:25 | |
*** alisonh has quit IRC | 00:30 | |
*** zhiwei has joined #kolla | 00:34 | |
*** sdake has quit IRC | 00:39 | |
*** alisonh has joined #kolla | 00:48 | |
*** Haomeng|2 has joined #kolla | 00:52 | |
*** Haomeng has quit IRC | 00:55 | |
*** SourabhP has quit IRC | 01:17 | |
*** unicell has quit IRC | 01:19 | |
*** jtriley has joined #kolla | 01:19 | |
*** jasonsb has joined #kolla | 01:27 | |
*** jasonsb has quit IRC | 01:28 | |
*** jasonsb has joined #kolla | 01:29 | |
*** erkules_ has joined #kolla | 01:30 | |
*** erkules has quit IRC | 01:32 | |
*** jasonsb has quit IRC | 01:32 | |
*** zhiwei has quit IRC | 01:33 | |
*** zhiwei has joined #kolla | 01:33 | |
*** rhallisey has quit IRC | 01:37 | |
*** dims has joined #kolla | 01:43 | |
*** amuller has joined #kolla | 02:18 | |
amuller | anyone alive? I have a few questions about the current state of things, specifically with respect to Neutron and building from source | 02:19 |
---|---|---|
*** amuller has left #kolla | 02:30 | |
*** amuller has quit IRC | 02:30 | |
*** jasonsb has joined #kolla | 02:35 | |
*** jtriley has quit IRC | 02:44 | |
*** loth has quit IRC | 02:51 | |
*** dims has quit IRC | 02:56 | |
*** alisonh has quit IRC | 03:21 | |
*** diga has joined #kolla | 03:23 | |
*** alisonh has joined #kolla | 03:27 | |
*** weiyu_ has quit IRC | 04:04 | |
*** achanda has quit IRC | 04:08 | |
*** achanda has joined #kolla | 04:44 | |
*** unicell has joined #kolla | 04:46 | |
*** unicell has quit IRC | 04:53 | |
*** unicell has joined #kolla | 04:53 | |
*** SourabhP has joined #kolla | 05:01 | |
*** SourabhP_ has joined #kolla | 05:03 | |
*** sdake has joined #kolla | 05:05 | |
*** SourabhP has quit IRC | 05:06 | |
*** SourabhP_ is now known as SourabhP | 05:06 | |
*** sdake_ has joined #kolla | 05:06 | |
*** blakec has joined #kolla | 05:07 | |
*** sdake has quit IRC | 05:10 | |
*** openstackgerrit has quit IRC | 05:16 | |
*** openstackgerrit has joined #kolla | 05:16 | |
sdake_ | evening | 05:18 |
*** sdake_ is now known as sdake | 05:18 | |
coolsvap | sdake, hi | 05:27 |
sdake | hey coolsvap | 05:28 |
coolsvap | sorry could not attend yesterday's meeting | 05:28 |
* coolsvap was busy with family on my bday | 05:28 | |
sdake | all good coolsvap | 05:31 |
mandre | hi | 05:31 |
sdake | family ist ;) | 05:31 |
mandre | happy bday coolsvap | 05:31 |
coolsvap | thanks mandre :) | 05:31 |
*** chandankumar has joined #kolla | 05:34 | |
*** weiyu has joined #kolla | 05:34 | |
*** dims has joined #kolla | 05:46 | |
sdake | im off to bed | 05:50 |
sdake | night falks | 05:50 |
SamYaple | night sdake | 05:50 |
*** dims has quit IRC | 05:52 | |
*** blakec has quit IRC | 05:56 | |
*** inc0 has joined #kolla | 05:58 | |
inc0 | good mornign | 05:59 |
SamYaple | morning inc0 | 05:59 |
SamYaple | talks are out | 05:59 |
SamYaple | only one Kolla :( | 06:00 |
inc0 | yeah, I saw | 06:00 |
inc0 | I hope this won't cause problems with me getting to Tokyo | 06:01 |
inc0 | but it shouldn't...I hope;) | 06:01 |
inc0 | very little technical-meaty sessions landed | 06:01 |
SamYaple | Hey would you mind review https://review.openstack.org/#/c/216691/ inc0 ? I believe you gave it the thumbs up in IRC, but can you give it a +1 so we have a record of agreement? | 06:01 |
inc0 | I feel summits are more and more marketing oriented | 06:01 |
SamYaple | they are | 06:01 |
SamYaple | but the designones have some nice stuff | 06:01 |
inc0 | it's on you guys to promote kolla then!:P | 06:02 |
SamYaple | yea i was really hoping for a lab to do just that | 06:03 |
inc0 | I was kinda hoping for "move to kolla howto" session | 06:03 |
SamYaple | yea that would have been awesome | 06:04 |
SamYaple | next summit for sure | 06:04 |
sdake | i am highlysurprised weddidt get a labb | 06:07 |
SamYaple | its cause they switched my tracks last second | 06:07 |
SamYaple | my lab was in containers and then got moved sdake | 06:07 |
SamYaple | or thats what im blaming! | 06:08 |
sdake | the whitepape s==from the fount is out imy witter steam | 06:11 |
SamYaple | yea id like to help with that | 06:11 |
*** inc0 has quit IRC | 06:36 | |
*** inc0 has joined #kolla | 07:05 | |
*** blakec has joined #kolla | 07:05 | |
*** achanda has quit IRC | 07:06 | |
*** blakec1 has joined #kolla | 07:10 | |
*** blakec has quit IRC | 07:13 | |
*** inc0 has quit IRC | 07:18 | |
*** inc0 has joined #kolla | 07:24 | |
inc0 | SamYaple, heat+ansible is working? (apart of broken heat) | 07:25 |
*** sdake has quit IRC | 07:29 | |
SamYaple | inc0: yea | 07:30 |
SamYaple | and with db upgrade not applitd, it just works | 07:30 |
*** shardy has joined #kolla | 07:31 | |
*** dims has joined #kolla | 07:35 | |
*** inc0 has quit IRC | 07:38 | |
*** inc0 has joined #kolla | 07:38 | |
*** jmccarthy has joined #kolla | 07:39 | |
*** dims has quit IRC | 07:40 | |
*** achanda has joined #kolla | 07:51 | |
*** SourabhP has quit IRC | 07:54 | |
*** blakec1 has quit IRC | 07:55 | |
inc0 | SamYaple, few comments, mostly post-development leftovers | 07:58 |
SamYaple | thanks inc0 | 08:03 |
SamYaple | coolsvap: https://review.openstack.org/#/c/213397/ | 08:04 |
SamYaple | would you mind reveiwing now that Kennan has commented? | 08:04 |
*** weiyu has quit IRC | 08:20 | |
*** weiyu has joined #kolla | 08:23 | |
*** weiyu has quit IRC | 08:26 | |
*** mandre has quit IRC | 08:34 | |
*** weiyu_ has joined #kolla | 08:37 | |
*** akwasnie has joined #kolla | 08:44 | |
*** weiyu_ has quit IRC | 08:47 | |
pbourke | sad face on summit talks | 08:49 |
pbourke | really can't believe only one got picked | 08:49 |
SamYaple | indeed | 08:49 |
SamYaple | there was some last minute track suffling that really hurt us i think | 08:50 |
*** achanda has quit IRC | 08:50 | |
pbourke | the containers on openstack topic seems to get more interest than openstack in containers | 08:51 |
*** shardy_ has joined #kolla | 08:51 | |
SamYaple | which makes sense when you think about it | 08:51 |
pbourke | are all ops that happy these days? :) | 08:51 |
SamYaple | most everyone already has a way they deploy | 08:51 |
SamYaple | no but change is long and hard, they want moer from openstack | 08:52 |
pbourke | I think I saw you saying once we have stable working images on dockerhub it will help a lot | 08:52 |
pbourke | which I agree with | 08:52 |
SamYaple | there are very few "private cloud" type business out there | 08:52 |
coolsvap | SamYaple, done, was away for lunch | 08:52 |
SamYaple | coolsvap: no worries! | 08:52 |
*** shardy has quit IRC | 08:52 | |
pbourke | the idea was that it should be easier than devstack, right now that's not the case I think | 08:52 |
openstackgerrit | Merged stackforge/kolla: Setup the gate for cross-distro support https://review.openstack.org/213397 | 08:53 |
SamYaple | well yea, but then you have to get the individual companies to change the way they deploy | 08:54 |
SamYaple | very difficult to do | 08:54 |
*** weiyu_ has joined #kolla | 08:56 | |
*** shardy_ has quit IRC | 08:56 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Remove tiered building in build.py https://review.openstack.org/216691 | 08:57 |
*** shardy has joined #kolla | 08:57 | |
*** weiyu_ has quit IRC | 08:59 | |
*** weiyu_ has joined #kolla | 09:03 | |
*** mfalatic has quit IRC | 09:12 | |
inc0 | especially that there are less and less ops on summits | 09:23 |
inc0 | which is sad | 09:23 |
inc0 | hopefully Austin will be more welcoming | 09:23 |
Kennan | inc0: where can find all summits selected this year? | 09:24 |
inc0 | Kennan, you mean talk schedule? | 09:24 |
inc0 | or summit locations? | 09:24 |
inc0 | https://www.openstack.org/summit/tokyo-2015/schedule/ <- schedule | 09:24 |
Kennan | talk schedule. seems selected talks is finalized | 09:24 |
inc0 | it is | 09:25 |
*** mandre has joined #kolla | 09:30 | |
*** diga has quit IRC | 09:33 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Keystone https://review.openstack.org/217601 | 09:35 |
SamYaple | please review the above change, I am about to make alot of those changes and dont want wasted effort | 09:36 |
inc0 | SamYaple, you've said yesterday that you have idea what was wrong with Stevens ansible - ansible not finding interfaces | 09:38 |
inc0 | care to share what's the idea? | 09:38 |
SamYaple | whats wrong is he was trying to lookup interfaces on hosts he wasn't addressing in that role | 09:39 |
SamYaple | a few ways to solve that issue | 09:39 |
SamYaple | fact caching is one, giving all host to the haproxy role is another | 09:39 |
SamYaple | ill submit a patch for it tonight | 09:39 |
SamYaple | its easily reproducable | 09:40 |
inc0 | yeah, we've just hit something similar | 09:40 |
inc0 | hence my question | 09:40 |
SamYaple | best this i can say for now is add all of your hosts to the network group | 09:40 |
SamYaple | thats the workaroudn that works with no code change | 09:40 |
SamYaple | otherwise ill have a patch up tonight | 09:40 |
SamYaple | (a few hours) | 09:40 |
inc0 | ahh, I see now...yeah that makes sense | 09:41 |
inc0 | but that being said, I don't think its case for us | 09:41 |
inc0 | hold on | 09:42 |
SamYaple | if you post your error ill look at it | 09:42 |
inc0 | nah, let me have some fun | 09:42 |
SamYaple | go for it! | 09:42 |
SamYaple | but if its an issue post it someone for others too | 09:42 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Glance https://review.openstack.org/217605 | 09:45 |
inc0 | ehh if I'd have +2 keystone would be merged already :( | 09:47 |
SamYaple | nah i gots to test them all. its fine | 09:50 |
*** dims has joined #kolla | 09:50 | |
coolsvap | inc0, we need to utilize/calibrate/test the gates we are setting up | 09:52 |
SamYaple | ^ | 09:52 |
openstackgerrit | Merged stackforge/kolla: Remove tiered building in build.py https://review.openstack.org/216691 | 09:52 |
inc0 | yeah, functional testing would be cool actually | 09:53 |
inc0 | I hope we'll get them early M | 09:53 |
SamYaple | inc0: im going to put them in rc1, sdake doesnt think i have time but i do | 09:54 |
inc0 | somehow I don't doubt that you will Sam | 09:55 |
inc0 | anyway, we all need to discuss how exactly we want them to look like | 09:56 |
inc0 | it's not trivial | 09:56 |
inc0 | and I for one dislike tempest, its faulty | 09:56 |
SamYaple | doubt me all you want, ive failed nothing! | 09:57 |
SamYaple | inc0: im talking about different functinoal testing | 09:57 |
SamYaple | sorry for the confusion | 09:57 |
inc0 | which one then? | 09:58 |
SamYaple | ansible deploy in the gate and boot instance | 09:58 |
inc0 | well, something as simpe as running rally stuff over deployed kolla might be good | 09:58 |
SamYaple | sure, but the fancy testing will probably be in M | 09:58 |
inc0 | will we still be bound by 40mins time limit tho? because well... | 09:58 |
SamYaple | barebones, does it deploy? does it boot? | 09:58 |
SamYaple | inc0: 60 mintutes are the gate | 09:59 |
SamYaple | we can get 90 if needed | 09:59 |
SamYaple | but we can do it in 60 | 09:59 |
SamYaple | build in ~25, deploy in ~5, test == ~30 minutes | 09:59 |
inc0 | more complex test would be to bood volume-backed instance with heat and add floating ip to it | 10:00 |
inc0 | we test pretty much all core projects with this one | 10:01 |
inc0 | and it's single heat stack deploy | 10:01 |
SamYaple | yea initial test will be cirros - does it boot? does it ping internet? can i ssh to it over floating ip? | 10:01 |
SamYaple | heat no worky | 10:01 |
SamYaple | but im sure sdake will want that :) | 10:01 |
inc0 | I'm kinda afraid we'll end up being CI for openstack | 10:07 |
inc0 | whole openstack | 10:07 |
SamYaple | afraid? | 10:07 |
inc0 | someone breaks their project, kolla gates are busted | 10:07 |
SamYaple | thats been my goal from the start | 10:07 |
SamYaple | not the way i have it in mind :) | 10:07 |
inc0 | what do you have in mind? | 10:08 |
SamYaple | well the kolla gate by its very nature can't be voting. its up to the cores to review the logs of the failed gate | 10:09 |
SamYaple | additionally we are going to hit issues with stable and oldstable branches | 10:09 |
SamYaple | with versions of things | 10:09 |
SamYaple | since we always pull the newest that _will_ break things eventually | 10:10 |
SamYaple | version capping will help, but stil itll happen | 10:10 |
SamYaple | now for the openstack as a whole ci stuff, each project wont have to build all of openstack, just thier project | 10:11 |
SamYaple | and there could be daily/hour cached and updated kolla images | 10:11 |
SamYaple | that would be far greater for functional testing than anything we currently have | 10:12 |
inc0 | well, if it's non voting | 10:12 |
SamYaple | it would have to be | 10:12 |
SamYaple | to many projects, too many changes | 10:13 |
SamYaple | neutron shouldnt be able to break glance devel | 10:13 |
SamYaple | but hey if it works 80% of the time, thats _way_ better than the nothing we have right now | 10:13 |
inc0 | that's essentially my point | 10:17 |
inc0 | we'll have failing gate even if kolla is ok | 10:17 |
inc0 | and it will be increasingly hard for cores to differ between incorrect kolla code and broken service | 10:17 |
inc0 | look at you with heat | 10:17 |
inc0 | I mean it could be doable, we could have repeated CI for kolla master and if something breaks there, that means it's broken in service | 10:18 |
SamYaple | wait wait | 10:19 |
SamYaple | you think kolla code is going to grow? | 10:19 |
inc0 | then all changes to existing containers could be compared to stable kolla errors | 10:19 |
inc0 | actually I do | 10:19 |
inc0 | because openstack changes | 10:19 |
SamYaple | well that right there is the problem | 10:19 |
pbourke | SamYaple: why do a lot of the services seem to have home directories now? | 10:19 |
pbourke | just wondering | 10:19 |
SamYaple | pbourke: all services always had home directories pbourke | 10:19 |
pbourke | ummmm | 10:19 |
pbourke | I dont think so :) | 10:19 |
SamYaple | useradd on centos creates them, but not on ubuntu | 10:19 |
pbourke | ah | 10:20 |
pbourke | maybe we should be doing --system | 10:20 |
SamYaple | no su for you then | 10:20 |
inc0 | SamYaple, software is finished with death of its last user | 10:21 |
inc0 | kolla will have to be maintained, expanded | 10:21 |
SamYaple | i disagree on expanded | 10:21 |
SamYaple | thats feature and scope creep | 10:21 |
inc0 | well right now we have very simplistic ansible | 10:21 |
inc0 | people will want more | 10:22 |
SamYaple | people can write more | 10:22 |
SamYaple | this was always a basic ansible template | 10:22 |
inc0 | people will want coreos compatibility | 10:22 |
*** weiyu_ has quit IRC | 10:22 | |
inc0 | yes, but if they write, why not put it upstream as long as it's correct? | 10:22 |
inc0 | and useful? | 10:22 |
inc0 | I would hate to create another "we don't want it here" project, like nova is right now | 10:23 |
inc0 | I'm not saying accept all, but keep an open mind | 10:23 |
pbourke | SamYaple: gotcha | 10:24 |
SamYaple | open mind, sure. but you are describing scope creep | 10:24 |
SamYaple | and feature creep | 10:24 |
SamYaple | you started talkign about distrubuted building the other day | 10:24 |
SamYaple | never gonna happen | 10:24 |
inc0 | there is a thin line and its up to cores to not cross it | 10:24 |
pbourke | its causing failures in OL, Im not sure about centos but id be surprised if not | 10:24 |
pbourke | as /home/blah already exists | 10:24 |
pbourke | when it tries to mkdir | 10:24 |
SamYaple | is that a _failure_? | 10:25 |
pbourke | tis | 10:25 |
SamYaple | it should just be a warning | 10:25 |
pbourke | let me check recent CI builds | 10:25 |
SamYaple | the home stuff has been in the code for quite a while | 10:25 |
SamYaple | (in kolla times) | 10:25 |
inc0 | ok, about distributed build, I throw 400 ideas a minute, 99.9% of them won't ever happen and I'm not saying it should | 10:25 |
SamYaple | fyi i stopped pushing patches while i test locally | 10:26 |
inc0 | but there is always something that actually should land | 10:26 |
SamYaple | i dont disagree, but it will slow down was my point | 10:26 |
inc0 | anyway, afk for 40mins, meeting \\o o// | 10:26 |
SamYaple | and it will | 10:26 |
*** Kennan has quit IRC | 10:38 | |
*** Haomeng|2 has quit IRC | 10:39 | |
*** zhiwei has quit IRC | 10:40 | |
*** zhiwei has joined #kolla | 10:40 | |
SamYaple | pbourke: https://review.openstack.org/#/c/217601/ | 10:43 |
SamYaple | it appears to be passing the gate just fine | 10:43 |
SamYaple | thats centos | 10:43 |
SamYaple | and it works fine on ubuntu | 10:43 |
pbourke | SamYaple: I didnt think it was that review? | 10:43 |
pbourke | SamYaple: anyway it might an oraclelinux only thing, im testing atm | 10:43 |
SamYaple | yea its in there | 10:44 |
SamYaple | but from like 2 weeks ago | 10:44 |
SamYaple | its been merged for a while | 10:44 |
pbourke | why are you -2'ing the above review though | 10:44 |
pbourke | its unrelated to the home dir | 10:44 |
SamYaple | oh right, was just trying to make sure a breaking oraclelinux change didnt slip through | 10:45 |
pbourke | the build.py output is bizarre | 10:45 |
pbourke | really must look at it | 10:46 |
SamYaple | its because the logging doesnt spit out each line but saves it all up until the end of the build | 10:46 |
SamYaple | i want to change that to print everything out as it gets it | 10:46 |
SamYaple | i can grep for the image name and see the logs | 10:46 |
pbourke | well theres that, but there's also -T 1 | 10:46 |
SamYaple | yea but thats sllooooo | 10:46 |
SamYaple | w | 10:46 |
pbourke | which tries to give real time logs, but seems to mix up the output | 10:46 |
pbourke | SamYaple: does it ever seem to just hang indefinitely on you | 10:52 |
SamYaple | pbourke: whats youre docker storage backed? | 10:52 |
SamYaple | only time it hangs like that is when docker is applying layer | 10:53 |
SamYaple | you should see that if you `ps aux | grep layer` | 10:53 |
pbourke | whichever comes default on centos - devicemapper? | 10:53 |
SamYaple | `docker info` | 10:53 |
SamYaple | it will tell you | 10:53 |
pbourke | yeah its devicemapper | 10:55 |
pbourke | nothing about layers in ps | 10:55 |
*** Kennan has joined #kolla | 10:55 | |
SamYaple | well thats all i got! | 10:55 |
pbourke | we've been experiencing a lot here since moving to build.py | 10:55 |
pbourke | maybe yesterdays patch will improve | 10:55 |
pbourke | it | 10:55 |
SamYaple | are you running with -T1 | 10:56 |
SamYaple | ? | 10:56 |
SamYaple | to me it sounds like your storage backend cant keep up | 10:56 |
SamYaple | whats your iowait times? | 10:56 |
vbel | good morning/evening everyone. I am missing generic 'data' container - is there such today? :) | 10:57 |
vbel | I mean the one with volumes for maria/rabbit/cinder | 10:57 |
SamYaple | vbel: we dont specify VOLUMES in the dockerfiles anymore | 10:58 |
SamYaple | we use ansible to launch a generic data container wit hte needed volumes | 10:58 |
jmccarthy | TASK: [mariadb | Starting MariaDB data container] ***************************** | 10:58 |
jmccarthy | failed: [localhost] => {"changes": ["{\"status\":\"Pulling repository docker.io/kollaglue/centos-binary-data\"}\r\n", "{\"errorDetail\":{\"message\":\"Error: image kollaglue/centos-binary-data:latest not found\"},\"error\":\"Error: image kollaglue/centos-binary-data:latest not found\"}\r\n"], "failed": true, "status": ""} | 10:58 |
vbel | ok, I don't see it is build | 10:58 |
jmccarthy | I suppose I missed one of the arguments to build.py | 10:59 |
pbourke | jmccarthy: build data | 10:59 |
jmccarthy | ok added data | 10:59 |
vbel | ok, I see, need to build.py 'data' | 11:00 |
vbel | thanks! | 11:01 |
jmccarthy | D'oh sorry for the noise | 11:01 |
vbel | if you don't do full build it can be confusing to find there is no data part ;) | 11:02 |
SamYaple | if you dont specify any options, it will build all containers | 11:02 |
SamYaple | including data | 11:02 |
*** zhiwei has quit IRC | 11:18 | |
*** rhallisey_ has joined #kolla | 11:18 | |
pbourke | SamYaple: what backend do you use? | 11:21 |
pbourke | SamYaple: also do you build on bare metal? | 11:21 |
SamYaple | i do build on baremetal | 11:21 |
SamYaple | i use btrfs or zfs | 11:21 |
SamYaple | btrfs is more stable and a bit faster | 11:22 |
SamYaple | than zfs* | 11:22 |
SamYaple | git branch | 11:22 |
SamYaple | git branch | 11:22 |
SamYaple | oops sorry | 11:22 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Fixes any() in an if statement build-py https://review.openstack.org/217646 | 11:27 |
SamYaple | guys ^^ there was a syntax issue in the tiering patch | 11:27 |
SamYaple | i guess i never tested locally replacing len with any pbourke | 11:27 |
pbourke | my bad also for not giving correct example | 11:27 |
SamYaple | its on me, i know how to use nay :( | 11:28 |
pbourke | arg | 11:28 |
pbourke | "blah returned a non zero error code" | 11:28 |
*** pbourke has quit IRC | 11:28 | |
*** pbourke has joined #kolla | 11:29 | |
SamYaple | awesome | 11:42 |
SamYaple | all of the incoming patches have been tested | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Ceilometer https://review.openstack.org/217655 | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Cinder https://review.openstack.org/217656 | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Designate https://review.openstack.org/217657 | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Glance https://review.openstack.org/217605 | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Gnocchi https://review.openstack.org/217658 | 11:43 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Heat https://review.openstack.org/217659 | 11:43 |
inc0 | Sam... | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Horizon https://review.openstack.org/217660 | 11:44 |
inc0 | really.. | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Ironic https://review.openstack.org/217661 | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Keystone https://review.openstack.org/217601 | 11:44 |
inc0 | stahp | 11:44 |
inc0 | Sam | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Magnum https://review.openstack.org/217662 | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Murano https://review.openstack.org/217663 | 11:44 |
SamYaple | never | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Nova https://review.openstack.org/217664 | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Swift https://review.openstack.org/217665 | 11:44 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Zaqar https://review.openstack.org/217666 | 11:44 |
SamYaple | is it over...? | 11:44 |
inc0 | here goes stackalitics review boost | 11:44 |
SamYaple | actually i hate this many patches | 11:45 |
SamYaple | but anytime it gets slighty large people complain that i should have broken it up | 11:45 |
inc0 | it's worse when they touch code that changes often | 11:45 |
SamYaple | build.py latest patch for example | 11:45 |
inc0 | trust me, I know | 11:45 |
inc0 | many patches are good, one big patch is bad | 11:45 |
SamYaple | i know, but then you end up with this thing that just happened | 11:46 |
inc0 | it's not bad, it's better than single | 11:46 |
SamYaple | agreed, but you understand | 11:46 |
inc0 | I do | 11:47 |
inc0 | I just noticed that there are keyboard shortcuts in gerrit | 11:48 |
SamYaple | ill never catch up with sdake on reviews :( | 11:48 |
inc0 | brave new world guys, brave new world | 11:48 |
SamYaple | what... | 11:48 |
SamYaple | what do they do | 11:48 |
inc0 | get to code review and press ? | 11:49 |
SamYaple | coolsvap mandre if one of you could ack this, it breaks build.py summary https://review.openstack.org/#/c/217646/ | 11:49 |
SamYaple | awesome! | 11:49 |
SamYaple | wowzers | 11:50 |
inc0 | all I need now is shortcut to +1 | 11:51 |
SamYaple | pbourke: have you tested horizon? | 11:51 |
inc0 | -1 I need to do manually, but thats only moment when I need to touch mouse | 11:51 |
SamYaple | pbourke: it looks like there might still be some icons missing. i thought it was just my browser but idk | 11:51 |
SamYaple | i would like to start building openvswitch from source | 11:53 |
SamYaple | what are your opinions on this? | 11:54 |
SamYaple | it wont be the module, that is still on the host system | 11:54 |
SamYaple | but the user-utils don't have to match the kernel version (they just should be newer) | 11:54 |
SamYaple | well gain benefits from the newer utils even on older kernels accordning to the openvswitch folks | 11:55 |
inc0 | SamYaple, https://review.openstack.org/#/c/217656/1 why do you unlink builds out there? | 11:57 |
SamYaple | oh thats leftovers from a bad docker -> docker_templates move | 11:57 |
SamYaple | those build scripts wont work with the template | 11:57 |
SamYaple | thats for bash | 11:57 |
inc0 | SamYaple, about ovs...is there concrete benefit of having that? | 11:58 |
SamYaple | http://openvswitch.org/releases/NEWS-2.4.0 | 11:58 |
inc0 | I know they won't, but we still have non-templates right? | 11:58 |
SamYaple | you tell me | 11:58 |
SamYaple | inc0: yea but thats only in the templates directory inc0 | 11:58 |
SamYaple | not the other directory | 11:58 |
inc0 | a yeah, makes sense | 11:59 |
inc0 | I missed that it's templates directory | 11:59 |
SamYaple | i just saw them as i was going and removed them | 11:59 |
SamYaple | horizon is being super slow in this container | 11:59 |
inc0 | I blame keystone | 12:00 |
SamYaple | its weird. we are actually getting to the 'i have to debug openstack proper' stage of this project | 12:00 |
inc0 | but that's like automatic reaction for me...somethings broken == neutron fault, somethings slow == keystone fault | 12:00 |
SamYaple | thats cool | 12:00 |
SamYaple | im going to do a WIP for openvswitch from source | 12:01 |
SamYaple | we already have the build tools installed | 12:01 |
SamYaple | oh and neutron is now using the ovs python bindings! | 12:01 |
SamYaple | they arent wrapping ovs-vsctl anymore! | 12:01 |
inc0 | then do we even need newer ovs? | 12:01 |
SamYaple | always | 12:02 |
SamYaple | you always need newer ovs | 12:02 |
inc0 | in container I mean | 12:02 |
SamYaple | yea? | 12:02 |
inc0 | ovs itself will be on host | 12:02 |
SamYaple | i mean you could do that, yes | 12:02 |
SamYaple | even now | 12:02 |
SamYaple | but boooo | 12:02 |
SamYaple | ovs works way better in a container | 12:02 |
inc0 | eee? | 12:03 |
SamYaple | seperating the db from the vswitchd is awesome | 12:03 |
inc0 | that's a new for me...why? | 12:03 |
inc0 | ah, yeah.. | 12:03 |
SamYaple | because ovs doesnt really do anything | 12:03 |
SamYaple | it just talks to the kernel module | 12:03 |
SamYaple | so we have far greater control over it in a container | 12:03 |
SamYaple | the vswitchd can fail, while neutron still updates the db | 12:03 |
SamYaple | when the vswitchd comes back, it will pull in those changes | 12:04 |
SamYaple | neutron never freaks out | 12:04 |
SamYaple | thanks rhallisey_ | 12:07 |
rhallisey_ | inc0, were using ovs in a container dewd | 12:07 |
*** rhallisey_ is now known as rhallisey | 12:07 | |
inc0 | I know, but I didn't know why its "better" | 12:07 |
SamYaple | inc0: ive found it to be more stable | 12:08 |
rhallisey | inc0, it actually it very tricky because of the networking stuff we're doing | 12:08 |
inc0 | yeah I know, I mean I know it works | 12:09 |
inc0 | just Sam said it's better and since it's kernel module really, I was wondering what's the difference between container and non-container | 12:10 |
inc0 | but separation of ovsdb makes sense, right | 12:10 |
rhallisey | don't know | 12:10 |
SamYaple | also that ovsdb is a sqlite database that never gets truncated | 12:10 |
SamYaple | a new container will truncate it | 12:10 |
SamYaple | woo | 12:10 |
SamYaple | old installs get wieghed down by that in a large environemtn | 12:11 |
SamYaple | (after years of lots of port changes) | 12:11 |
openstackgerrit | Vladislav Belogrudov proposed stackforge/kolla: Add placeholder to unset http proxies https://review.openstack.org/217678 | 12:11 |
*** jasonamyers has joined #kolla | 12:12 | |
*** dims has quit IRC | 12:14 | |
openstackgerrit | Vladislav Belogrudov proposed stackforge/kolla: Add placeholder to unset http proxies https://review.openstack.org/217678 | 12:16 |
SamYaple | akwasnie: have you started on https://bugs.launchpad.net/kolla/+bug/1485742 ? | 12:20 |
openstack | Launchpad bug 1485742 in kolla "global search and replace /opt/kolla with /var/lib/kolla" [Critical,Confirmed] - Assigned to Alicja Kwasniewska (alicja-kwasniewska) | 12:20 |
SamYaple | if not, I can take over | 12:20 |
SamYaple | its due pertty much now for L3 | 12:20 |
openstackgerrit | Merged stackforge/kolla: Fixes any() in an if statement build-py https://review.openstack.org/217646 | 12:25 |
rhallisey | SamYaple, how can I create a file on a host in ansible | 12:25 |
rhallisey | have trouble finding an example | 12:26 |
SamYaple | What content do you want? | 12:26 |
SamYaple | copy a file? | 12:26 |
SamYaple | just touch | 12:26 |
SamYaple | ? | 12:26 |
rhallisey | SamYaple, this would be to create the json file on a host | 12:26 |
SamYaple | where are you getting the inital content? | 12:26 |
SamYaple | a string? another file? | 12:26 |
rhallisey | I'm thinking a string | 12:27 |
openstackgerrit | Vladislav Belogrudov proposed stackforge/kolla: Add placeholder to unset http proxies https://review.openstack.org/217678 | 12:27 |
SamYaple | copy module | 12:27 |
SamYaple | copy content={{ string }} dest=/file/on/dest/host | 12:27 |
rhallisey | ok cool | 12:27 |
SamYaple | coolsvap: regarding https://bugs.launchpad.net/kolla/+bug/1480231 | 12:28 |
openstack | Launchpad bug 1480231 in kolla "change nova tar url to master" [High,Confirmed] - Assigned to Swapnil Kulkarni (coolsvap) | 12:28 |
SamYaple | can you update and/or close that bug? | 12:28 |
SamYaple | we do pull nova from master in build.ini | 12:28 |
*** dims has joined #kolla | 12:29 | |
*** openstackgerrit has quit IRC | 12:31 | |
*** openstackgerrit has joined #kolla | 12:32 | |
*** inc0 has quit IRC | 12:38 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Fixes matching issue build.py https://review.openstack.org/217689 | 12:38 |
pbourke | SamYaple: could you do me a favour? | 12:41 |
SamYaple | for you, anything! | 12:42 |
pbourke | :P | 12:42 |
pbourke | does this work for you | 12:42 |
pbourke | tools/build.py --base centos --type source --template nova-base -T 1 -d | 12:42 |
SamYaple | testing | 12:42 |
SamYaple | root@server02:~/kolla/tools# tools/build.py --base centos --type source --template nova-base -T 1 -d | 12:42 |
SamYaple | -bash: tools/build.py: No such file or directory | 12:42 |
SamYaple | im kidding ;) | 12:42 |
SamYaple | its building now pbourke | 12:43 |
pbourke | kk | 12:43 |
SamYaple | what am i looking for | 12:43 |
pbourke | the mkdir failing the build | 12:43 |
SamYaple | ok | 12:43 |
SamYaple | im building on master | 12:44 |
pbourke | yeah | 12:44 |
SamYaple | ok | 12:44 |
pbourke | i would just stick mkdir -p in and be done with it, but curious why its building in CI | 12:44 |
SamYaple | i updated all my patches with a -p | 12:44 |
SamYaple | btw | 12:44 |
pbourke | also keep an eye out for the logs being jumbled | 12:45 |
SamYaple | youll have to defin jumled a bit more | 12:45 |
pbourke | I think there's something wrong with the way we're stashing them into image['logs'] | 12:45 |
SamYaple | oh yea im remving that in the patchset incoming | 12:45 |
pbourke | it seems you get lots of duplicate output | 12:45 |
SamYaple | im udating loging | 12:45 |
SamYaple | be can discuss it there | 12:46 |
SamYaple | people* | 12:46 |
SamYaple | jeez why cant i type | 12:46 |
pbourke | cool | 12:46 |
SamYaple | vbel: fixed your namespace issue (i think) | 12:46 |
SamYaple | https://review.openstack.org/#/c/217689/ | 12:46 |
pbourke | for all these --no-cache-dir patches | 12:47 |
SamYaple | yes | 12:47 |
pbourke | are we sure all dependencies are installed in base | 12:47 |
pbourke | how can we be sure | 12:47 |
SamYaple | pip install installs the requirements file | 12:47 |
SamYaple | thats why currenlty you see a bajillsion "already installed" in the logs | 12:48 |
SamYaple | because we install the requirements, then install the folder (which installs the requirements) | 12:48 |
pbourke | so the install -r requirements step was redundant | 12:48 |
pbourke | ? | 12:48 |
SamYaple | yes | 12:48 |
pbourke | gotcha | 12:48 |
SamYaple | INFO:__main__:nova-base:mkdir: cannot create directory '/home/nova': File exists | 12:57 |
SamYaple | yes pbourke it failes | 12:57 |
SamYaple | it wont fail with -p though | 12:57 |
SamYaple | thats the point of that flag kinda | 12:57 |
SamYaple | -p, --parents | 12:58 |
SamYaple | no error if existing, make parent directories as needed | 12:58 |
pbourke | yeah I see the -p's added now, going through those patches | 12:59 |
pbourke | why is upstream building though | 12:59 |
SamYaple | ? | 12:59 |
SamYaple | oh yea | 12:59 |
pbourke | sorry, CI | 12:59 |
SamYaple | i know what youre saying | 12:59 |
pbourke | also horizon is busted | 13:00 |
SamYaple | yea the CI is wierd | 13:00 |
pbourke | but not in CI | 13:00 |
pbourke | :/ | 13:00 |
SamYaple | i dont know whats happening there | 13:00 |
SamYaple | i notcied it too | 13:00 |
SamYaple | like it using old version of kolla or something | 13:00 |
pbourke | mickt is fixing the horizon bug | 13:01 |
pbourke | patch incoming | 13:01 |
SamYaple | its working for ubuntu fyi | 13:01 |
pbourke | i dont see how | 13:01 |
pbourke | one second | 13:01 |
pbourke | https://github.com/stackforge/kolla/blob/master/docker_templates/horizon/Dockerfile.j2#L47 | 13:02 |
SamYaple | yup | 13:02 |
pbourke | /etc/openstack-dashboard/local_settings | 13:02 |
pbourke | doesn't exist | 13:02 |
SamYaple | of course it doesnt | 13:02 |
SamYaple | its copied in with config-external | 13:02 |
pbourke | but the build runs first | 13:03 |
pbourke | what am i missing here | 13:03 |
SamYaple | that doesnt matter | 13:03 |
SamYaple | you can create symlinks pointing to nonexistant things | 13:03 |
pbourke | ah ok | 13:03 |
*** mandre has quit IRC | 13:03 | |
pbourke | thanks | 13:04 |
*** vbel has quit IRC | 13:04 | |
pbourke | need to get all these -p patches in | 13:05 |
*** vbel has joined #kolla | 13:05 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Remove CONFIG_STRATEGY prefix https://review.openstack.org/217700 | 13:07 |
akwasnie | SamYaple, you asked if I had started on https://bugs.launchpad.net/kolla/+bug/1485742 ? | 13:09 |
openstack | Launchpad bug 1485742 in kolla "global search and replace /opt/kolla with /var/lib/kolla" [Critical,Confirmed] - Assigned to Alicja Kwasniewska (alicja-kwasniewska) | 13:09 |
SamYaple | yes | 13:09 |
akwasnie | I haven't | 13:09 |
SamYaple | Do you plan to have it done by L3? | 13:09 |
SamYaple | (2-3 days?) | 13:09 |
akwasnie | better take it over | 13:10 |
*** diogogmt has joined #kolla | 13:10 | |
SamYaple | will do | 13:10 |
akwasnie | good | 13:10 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Glance https://review.openstack.org/217605 | 13:20 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Improve cleanup script https://review.openstack.org/217706 | 13:25 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Remove CONFIG_STRATEGY prefix https://review.openstack.org/217700 | 13:26 |
SamYaple | btw did you guys notice our gate runs 20% faster since the build tiers were removed? | 13:27 |
SamYaple | that translates to 5 minutes! | 13:27 |
rhallisey | cool | 13:28 |
rhallisey | anyone else having issuing with haproxy/mariadb lately? | 13:29 |
rhallisey | haproxy sits on 3306 and mariadb never comes up | 13:29 |
*** chandankumar has quit IRC | 13:30 | |
vbel | haproxy should only listen vip | 13:30 |
vbel | and mariadb shoudl only listen host interface ip | 13:31 |
rhallisey | err ok so it's something in globals.yml | 13:31 |
vbel | there still is a problem with unsetting http proxies - kolla ansible container works for local commands but not for things like creation of endpoints | 13:32 |
vbel | because it does not read kolla-common.sh or anything else | 13:32 |
*** jasonamyers has quit IRC | 13:33 | |
openstackgerrit | Merged stackforge/kolla: Fixes matching issue build.py https://review.openstack.org/217689 | 13:35 |
*** jtriley has joined #kolla | 13:40 | |
*** diogogmt has quit IRC | 13:47 | |
rhallisey | vbel, ya I think both a looking at vip | 13:49 |
rhallisey | how can I fix this | 13:49 |
rhallisey | I've changed the IP a bunch no difference | 13:49 |
vbel | may be mariadb in /etc/my.cnf should only listen to ip, usually folks set bind = 0.0.0.0 or similar, need to check template | 13:50 |
SamYaple | vbel: yea ansible container wont work if you unset in kolla-common :( | 13:53 |
SamYaple | we really need a footer | 13:53 |
*** ftcpops has joined #kolla | 13:53 | |
vbel | ok, i have idea - just add entrypoint to kolla-ansible which sources kolla-common.sh and exec "$@" | 13:53 |
rhallisey | this use to work for me what happened that I have to set that manually? | 13:57 |
rhallisey | SamYaple, did something change in allinone that would cause haproxy and mariadb to both look at the same ip? | 13:59 |
SamYaple | vbel: i really dont like that idea, lets see if we can come up with another one and circle back to that one if we cant | 13:59 |
*** ccrouch has quit IRC | 13:59 | |
rhallisey | setting a vip not letting the two run | 13:59 |
SamYaple | rhallisey: what is your kolla_internal_address? | 13:59 |
rhallisey | default | 14:00 |
SamYaple | what is default? | 14:00 |
pbourke | id love to know why its faster | 14:00 |
SamYaple | i mean it has to be an ip | 14:00 |
SamYaple | pbourke: the building? | 14:00 |
pbourke | yeah | 14:00 |
rhallisey | 10.10.10.254 | 14:00 |
pbourke | just because its reading from a single queue rather than multiple ones | 14:00 |
SamYaple | because before at the end of the tier only one thread would be running | 14:00 |
SamYaple | then the next tier would go | 14:00 |
rhallisey | use to work on 192.168.1.148 | 14:00 |
SamYaple | rhallisey: whats the first ip address of the interface on your box? | 14:01 |
rhallisey | 10.18.57.20 | 14:01 |
SamYaple | im not sure then are you positive its trying to bind the same address? | 14:01 |
SamYaple | can you post the galera.cnf? | 14:01 |
SamYaple | pbourke: now all threads are always active until the last images, before it was bursty | 14:02 |
pbourke | ah ok | 14:02 |
*** Kennan has quit IRC | 14:02 | |
rhallisey | ya just a sec | 14:02 |
SamYaple | and globals.yml too i suppose | 14:02 |
*** Kennan has joined #kolla | 14:03 | |
rhallisey | globals http://paste.fedoraproject.org/260101/84228144 | 14:04 |
rhallisey | container logs -> http://fpaste.org/260104/40684294/ | 14:05 |
rhallisey | from mariadb | 14:05 |
SamYaple | what about the galera.cnf? | 14:05 |
SamYaple | /etc/kolla/galera/galera.cnf | 14:05 |
SamYaple | i think | 14:05 |
SamYaple | /etc/kolla/mariadb/galera.cnf | 14:06 |
SamYaple | that one | 14:06 |
rhallisey | need to stop haproxy just a second | 14:06 |
SamYaple | i actually have to go to dmv | 14:07 |
SamYaple | ill be back around just tag me on any pastes | 14:07 |
SamYaple | ill look at them | 14:07 |
rhallisey | SamYaple, kk | 14:07 |
rhallisey | I know what it is, but don't know how this happened | 14:08 |
rhallisey | wow this is funny.. | 14:09 |
SamYaple | the vip is the first ip? | 14:09 |
SamYaple | thats one ive seen | 14:09 |
rhallisey | no the file just say the name of the var I"m using for json | 14:10 |
rhallisey | I have no idea how | 14:11 |
SamYaple | lol | 14:11 |
rhallisey | that was a surprise lol.. | 14:11 |
vbel | SamYaple: entrypoint does not help because we exec into container directly | 14:11 |
SamYaple | vbel: i know, but the reason we call the binary directly is to parse the output | 14:11 |
rhallisey | thanks SamYaple I'm good now | 14:12 |
SamYaple | passing it through other things like a start.sh script gave differnet output | 14:12 |
SamYaple | id like to call ansible directly if possible | 14:12 |
SamYaple | gotta go | 14:13 |
rhallisey | SamYaple, good luck | 14:13 |
rhallisey | dmv sucks | 14:14 |
*** jasonamyers has joined #kolla | 14:14 | |
*** sdake has joined #kolla | 14:16 | |
*** diogogmt has joined #kolla | 14:21 | |
*** jtriley has quit IRC | 14:24 | |
*** diogogmt has quit IRC | 14:25 | |
*** achanda has joined #kolla | 14:31 | |
*** jtriley has joined #kolla | 14:32 | |
*** ccrouch has joined #kolla | 14:39 | |
*** achanda has quit IRC | 14:40 | |
*** dims has quit IRC | 14:51 | |
*** dims has joined #kolla | 14:51 | |
vbel | what if we just squash containers to remove whatever build env stuff is included in whatever layers? Pros - clean, 15+% smaller image, Cons - registry will not share common parts | 15:08 |
vbel | + security after squashing | 15:10 |
vbel | I mean it is not for build.py but a guide to clean up stuff from images | 15:10 |
*** mfalatic has joined #kolla | 15:11 | |
*** chandankumar has joined #kolla | 15:16 | |
pbourke | what do you mean by squashing? | 15:24 |
*** blakec has joined #kolla | 15:24 | |
pbourke | vbel: ^ | 15:25 |
vbel | we basically remove all layers and ENV, images become 1 layer (sometimes much smaller) | 15:26 |
pbourke | is it what's described here? http://jasonwilder.com/blog/2014/08/19/squashing-docker-images/ | 15:27 |
vbel | yes | 15:27 |
vbel | I do it in bash with couple of commands | 15:27 |
vbel | ID=$(docker run -d oracle/ol-openstack-keystone /bin/bash) | 15:28 |
vbel | docker export $ID | docker import - oracle/ol-openstack-keystone:latest | 15:28 |
vbel | and docker rm $ID | 15:28 |
vbel | that's it. NO PROXIES :))))) | 15:29 |
vbel | no pypi mirrors or magic build flags with internal ips | 15:29 |
*** mandre has joined #kolla | 15:30 | |
vbel | there is disadvantage that it takes more space in registry because sharing of layers is away. The same storage problem would happen with dockers saved as tarballs | 15:30 |
pbourke | it seems a bit more complex than just doing the unsets in kolla-common | 15:33 |
sdake | i'd rather not hack thei mages | 15:34 |
sdake | just an fyi ;) | 15:34 |
sdake | vbel ^^ | 15:34 |
vbel | we cannot workaround ansible containers - they don't read kolla-common.sh and we do exec things into them | 15:35 |
SamYaple | rhallisey: its worse! | 15:36 |
SamYaple | my social security card was stolen a while back | 15:36 |
rhallisey | holy | 15:36 |
SamYaple | so now im at teh social security office | 15:36 |
SamYaple | pbourke: i figured out a problem | 15:36 |
SamYaple | with the build script | 15:36 |
vbel | so I have found two ways - to run ansible inside thru a script that sources common.sh (SamYaple disagreed here) or stick ENV HTTP* "" at the end of kolla-ansible template | 15:36 |
SamYaple | your gonna laugh | 15:37 |
*** sdake_ has joined #kolla | 15:38 | |
SamYaple | oh never mind i already fixed that | 15:39 |
SamYaple | morning sdake | 15:39 |
*** SourabhP has joined #kolla | 15:40 | |
SamYaple | vbel: does your custom docker 1.8 that accepts buildtime env args have api bindings? | 15:40 |
*** sdake has quit IRC | 15:40 | |
SamYaple | if so i could extend docker-py for you and have that work | 15:40 |
SamYaple | that sounds like the only answer here to be honest | 15:41 |
vbel | SamYaple: we want abandon that patch :) Ansible container worked well for ENV HTTP* = "" | 15:41 |
vbel | so I wil check glance and keystone and if they can work with empty proxies I would prefer include footer | 15:42 |
*** jruano has joined #kolla | 15:43 | |
SamYaple | vbel: i like the idea of a footer | 15:43 |
SamYaple | super flexibility | 15:43 |
vbel | also it worked well with "command: docker exec -t kolla_ansible /entrypoint.sh /usr/bin/ansible localhost" . Yes, I like footer idea, will check now | 15:44 |
sdake_ | hey samyaple | 15:45 |
openstackgerrit | Merged stackforge/kolla: Remove CONFIG_STRATEGY prefix https://review.openstack.org/217700 | 15:46 |
SamYaple | sdake im goldplating what are you gonna do about it? | 15:47 |
SamYaple | :P | 15:47 |
SamYaple | no im looking at fixing logging | 15:47 |
sdake_ | goldplating is not an anti-pattern | 15:49 |
*** akwasnie has quit IRC | 15:49 | |
sdake_ | it is just not an optimal use of resources | 15:49 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Nova https://review.openstack.org/217664 | 15:49 |
sdake_ | anyway that config strategy patch looks like goodgold platingto me :) | 15:49 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Gnocchi https://review.openstack.org/217658 | 15:49 |
*** SourabhP has quit IRC | 15:50 | |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Designate https://review.openstack.org/217657 | 15:50 |
sdake_ | say samyaple | 15:51 |
sdake_ | can you copy the config option for neutron network | 15:51 |
sdake_ | neutron agent\ | 15:51 |
*** SourabhP has joined #kolla | 15:51 | |
*** SourabhP_ has joined #kolla | 15:52 | |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Cinder https://review.openstack.org/217656 | 15:53 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Heat https://review.openstack.org/217659 | 15:53 |
SamYaple | i dont follow sdake_ ? | 15:54 |
SamYaple | you mean openvswitch? | 15:54 |
sdake_ | yes | 15:54 |
SamYaple | sure thing | 15:54 |
SamYaple | moment | 15:54 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Make logging more better https://review.openstack.org/217774 | 15:54 |
SamYaple | everyone, try out that logging patch | 15:54 |
SamYaple | with grep and searching it should be less confusing | 15:55 |
SamYaple | but its the only way i can think of to do proper logging multithreaded | 15:55 |
SamYaple | its up to you to filter it yourself | 15:55 |
SamYaple | also check the gate output once that finishes running | 15:55 |
*** SourabhP has quit IRC | 15:56 | |
*** SourabhP_ is now known as SourabhP | 15:56 | |
sdake_ | samyaple the problem with the grep to filter logs is that isnt buidt into the gate | 15:58 |
sdake_ | the bug was about the gate logs not being helpful ;) | 15:58 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Display neutron plugin options in globals.yml https://review.openstack.org/217777 | 15:58 |
SamYaple | sdake_: but now you can actually use find in a browser | 15:59 |
SamYaple | which you could not before | 15:59 |
SamYaple | there is no way these _THREADED_ logs are going to be super readable sdake, filtering what we want is the only option | 15:59 |
SamYaple | now we can properly do that filtering | 15:59 |
sdake_ | can't you just filter for the user at the end | 16:00 |
*** jasonsb has quit IRC | 16:00 | |
*** jasonsb has joined #kolla | 16:01 | |
SamYaple | what do you mean? | 16:01 |
SamYaple | thats what everyone is complaining about | 16:01 |
sdake_ | i mean put each log message in a queue for that image | 16:01 |
SamYaple | thats what we do | 16:01 |
sdake_ | and print out the queues at the end | 16:01 |
SamYaple | we collect all logs, printing them all out at the end means no output at all for 10-15 minutes | 16:02 |
SamYaple | this is the biggest complaints we get | 16:02 |
sdake_ | i think the fact that th elogs are totally not hepful in a failure should be the biggest complaint ;) | 16:02 |
SamYaple | now they will be helpful | 16:03 |
SamYaple | oh look keystone failed | 16:03 |
SamYaple | ctrl + f keystone | 16:03 |
SamYaple | how thats why | 16:03 |
SamYaple | you cant do that now | 16:03 |
sdake_ | agree its an improvement if it works propery - hen't tested it | 16:04 |
sdake_ | samyaple i deplyed yesterday during m y demo and ovs locked up | 16:04 |
sdake_ | the part where it says preparing ovs blah blah | 16:05 |
sdake_ | any ideas on that one? | 16:05 |
*** jasonsb has quit IRC | 16:05 | |
SamYaple | would need proper logs for sure | 16:05 |
sdake_ | ok | 16:06 |
*** sdake_ is now known as sdake | 16:06 | |
sdake | let me get my day rolling and i'll see what i can produce | 16:06 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Murano https://review.openstack.org/217663 | 16:07 |
*** jpeeler has quit IRC | 16:08 | |
*** SourabhP has quit IRC | 16:09 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Make logging more better https://review.openstack.org/217774 | 16:10 |
pbourke | SamYaple: what was the problem? | 16:11 |
SamYaple | pbourke: nah i had already fixed that previously | 16:12 |
SamYaple | i thought the while loop in the thread was running while | 16:12 |
pbourke | I know there's been some disucssion around paths | 16:12 |
SamYaple | but it blocks on queue.get() | 16:12 |
pbourke | I propose we use /etc/kolla everywhere | 16:12 |
pbourke | locally, on remote, and in containers | 16:12 |
pbourke | I found it extremely confusing when implementing ansible swift, and when explaining to jmccarthy it was not any easier | 16:13 |
SamYaple | well pbourke the stuff in containers arent configs | 16:13 |
SamYaple | and that isn't LSB if we use /etc/kolla | 16:13 |
SamYaple | we can _mount_ intinto /etc/kolla for configs | 16:13 |
SamYaple | but the scripts have to be in /usr/share/kolla | 16:13 |
sdake | yes scripts need to be in /usr/share/kolla | 16:14 |
sdake | basically whatever is in /opt should be reworked | 16:14 |
SamYaple | sdake: ill be submitting that patch tonight | 16:14 |
SamYaple | itll merge before L3 | 16:14 |
sdake | i htink /usr/share/kolla/templates should be in /var/lib/kolla/templates | 16:14 |
SamYaple | how so? | 16:14 |
sdake | /usr/share is supposed to be read only | 16:14 |
SamYaple | curious, not fighting | 16:14 |
SamYaple | ok | 16:15 |
SamYaple | as log as it is out of the way, ill make that change as well! | 16:15 |
sdake | ya lets get all the path changes done in l3 | 16:15 |
*** jasonamyers has quit IRC | 16:16 | |
jmccarthy | Saaam ! Yea agree pbourke - the paths and piecing stuff together broke my tired brain | 16:18 |
sdake | well i'd rather stick to lsb standards here rather then jam stuff in oen path to make it easier no developers ;) | 16:19 |
sdake | no/on | 16:19 |
pbourke | well, also ops, I'm sure will want to understand this stuff | 16:19 |
pbourke | but im not suggesting violating LSB | 16:20 |
sdake | agree- should be documented | 16:20 |
jmccarthy | freudian typo ;P | 16:20 |
pbourke | hopefully the removal of /opt will simplfy things somewhat | 16:20 |
*** unicell has quit IRC | 16:20 | |
sdake | so just a heads up, i'll let the release go until the 3rd for blueprints for liberty | 16:24 |
sdake | for librty-3 | 16:24 |
sdake | but 3rd is cutoff | 16:24 |
sdake | so geter done befor then plz | 16:24 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Zaqar https://review.openstack.org/217666 | 16:29 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Magnum https://review.openstack.org/217662 | 16:30 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Swift https://review.openstack.org/217665 | 16:30 |
sdake | samyaple this is why the mysql contianer fails to work properly multinode - i'll provide full evidence via docker exec mariadb_data ls -l /var/lib/mysql and ls -ld /var/lib/mysql | 16:33 |
sdake | https://bugs.launchpad.net/kolla/+bug/1489537 | 16:33 |
openstack | Launchpad bug 1489537 in kolla "mariadb+galera doesn't work multinode" [Critical,Confirmed] - Assigned to Steven Dake (sdake) | 16:33 |
sdake | i've got a wip patch to sort it out | 16:34 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Horizon https://review.openstack.org/217660 | 16:34 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Ironic https://review.openstack.org/217661 | 16:36 |
sdake | samyaple before you begin rework on https://review.openstack.org/#/c/217706/ please read my comments thnaks :) | 16:37 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Keystone https://review.openstack.org/217601 | 16:39 |
jmccarthy | Hmm anyone see 'ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))' type errors when building containers ? | 16:44 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Ceilometer https://review.openstack.org/217655 | 16:45 |
*** jtriley has quit IRC | 16:45 | |
pbourke | the logging is miles better | 16:45 |
pbourke | SamYaple: ^ | 16:45 |
sdake | image build down to 22 minutes in the gate = nice work on the gate patch samyaple | 16:46 |
sdake | hopefully those lvm changes are reliable ;) | 16:46 |
pbourke | im a very happy camper with this logging patch | 16:47 |
*** jtriley has joined #kolla | 16:52 | |
sdake | samyaple re the network in all hosts required, you aid you had a fix for that? | 16:53 |
*** unicell has joined #kolla | 16:59 | |
*** sdake_ has joined #kolla | 16:59 | |
openstackgerrit | Merged stackforge/kolla: Display neutron plugin options in globals.yml https://review.openstack.org/217777 | 16:59 |
*** chandankumar has quit IRC | 17:00 | |
*** sdake has quit IRC | 17:02 | |
*** jpeeler has joined #kolla | 17:03 | |
*** jpeeler has quit IRC | 17:03 | |
*** jpeeler has joined #kolla | 17:03 | |
SamYaple | sorry, back | 17:06 |
SamYaple | social_security_card.get() | 17:06 |
SamYaple | sdake_: im almost positive the mysql issue youre seeing will go away when you dont install mysql in the base image | 17:07 |
SamYaple | which is probably why i dont have the issue in ubuntu at app | 17:07 |
SamYaple | all* | 17:07 |
SamYaple | pbourke: yea i thought that would be better :) | 17:07 |
SamYaple | sdake_: i said i know how to fix it, i will fix the networking things | 17:07 |
SamYaple | it will be in RC1 | 17:07 |
sdake_ | samyaple undersood thanks | 17:08 |
SamYaple | i have to think on the best way to do it (there are 3-4 options) | 17:08 |
sdake_ | for centos we install mariadb in the baes? | 17:08 |
SamYaple | indeed | 17:08 |
SamYaple | i opened a bug for it | 17:08 |
SamYaple | then submitted a patch | 17:08 |
*** jasonsb has joined #kolla | 17:08 | |
SamYaple | but i didnt test centos so i blocked it | 17:08 |
SamYaple | i probably wont get around to testing centos ever | 17:08 |
SamYaple | ever* being relative to this cycle | 17:09 |
sdake_ | i remember seeing your patch and thinking the reaon we did that was probbly for dependencies | 17:09 |
SamYaple | sdake_: https://bugs.launchpad.net/kolla/+bug/1486374 | 17:09 |
openstack | Launchpad bug 1486374 in kolla "Remove mysql-server from base" [Medium,Triaged] | 17:09 |
SamYaple | well you should not do that :P | 17:09 |
sdake_ | your proposing just mysql in the mariadb container? | 17:09 |
SamYaple | pull those dependaciesi n better | 17:09 |
SamYaple | also move the mariadb repo to the base so all libraries match throughout the environment | 17:10 |
SamYaple | i install python-mysqldb in my base | 17:10 |
SamYaple | i think thats all you need | 17:10 |
SamYaple | no idea what thats calling in centosland | 17:10 |
sdake_ | ok i'll hae a look at your patch and see if that fixes it | 17:10 |
SamYaple | its old and outdated | 17:10 |
SamYaple | i just abandonded this morning | 17:11 |
sdake_ | yes i understand | 17:11 |
sdake_ | mysql-server really shouldn't be in the bse in centos | 17:11 |
sdake_ | the fact that our dependencis dont' match may explain why my ssytem blows up on mariadb beyond just the permissions issue | 17:12 |
sdake_ | if you are installing mysql in your own image and not rm -rf /var/lib/mysql this permission problem wont happen | 17:12 |
sdake_ | its the rm -rf /var/lib/mysql that causes the problem because those efiles already hae correct permissions iiuc | 17:13 |
sdake_ | data container copies existing files to new data container | 17:13 |
SamYaple | yea thats what happened to me in yaodu | 17:13 |
SamYaple | anyway. I would think that would be highly related. good place to start anyway | 17:13 |
sdake_ | its just a different way to solve the same problem | 17:14 |
*** sdake_ is now known as sdake | 17:14 | |
SamYaple | but the way that doesnt affect ubuntu which works great | 17:14 |
sdake | i'll look into putting mysql into the proper container | 17:14 |
SamYaple | i went on a little bug squashing trip earlier sdake. hammered out a few | 17:19 |
SamYaple | looks like they have mostly merged now | 17:19 |
sdake | fwiw your the one that made me delete that directory in the first place ;-) | 17:19 |
SamYaple | sdake: 19 minute gate time for source! | 17:19 |
SamYaple | i said remove the contents, not delete | 17:20 |
sdake | how do you remove the contents without deleting? | 17:20 |
SamYaple | rm -rf /folder/* | 17:20 |
sdake | yes that is what is in master | 17:20 |
SamYaple | if you want to be fancy... | 17:20 |
SamYaple | ok thats not removeing the folder | 17:20 |
SamYaple | just the content | 17:21 |
SamYaple | well non-hidden content | 17:21 |
SamYaple | but mysql doesnt have any hidden content | 17:21 |
SamYaple | if it does we could fix that too | 17:21 |
sdake | ls -a didn't turn anything up | 17:21 |
SamYaple | yea i checked the first time i told you to use that | 17:21 |
SamYaple | anyway. i think the issue is related to the base first and foremost | 17:22 |
SamYaple | and while we will certainyl adjust teh scripts if needed, im hesitant because they work for ubuntu perfectly right now | 17:22 |
*** achanda has joined #kolla | 17:22 | |
sdake | well i'll try your approach first | 17:23 |
sdake | but we may need to modify the scripts | 17:23 |
SamYaple | i highly doubt we will ever _need_ to | 17:23 |
sdake | the chown i think needs to be outside the conditional | 17:23 |
sdake | time will tell | 17:23 |
*** sdake_ has joined #kolla | 17:26 | |
*** sdake has quit IRC | 17:29 | |
*** blakec has quit IRC | 17:30 | |
*** blakec has joined #kolla | 17:32 | |
*** blakec has left #kolla | 17:32 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Make logging more better https://review.openstack.org/217774 | 17:37 |
*** loth has joined #kolla | 17:37 | |
SamYaple | pbourke: ^ that now catches the remaing multi line outputs i believe | 17:38 |
SamYaple | all lines are prefixed now, even if the message has a \n char | 17:38 |
sdake_ | pbourke vbel did the pull from git work finish? | 17:38 |
SamYaple | yes sdake | 17:38 |
SamYaple | its merged | 17:38 |
sdake_ | docker dir is consistenty moving out of sync | 17:38 |
sdake_ | ok time to remove docker dir then ;-) | 17:39 |
sdake_ | what else are we missing | 17:39 |
SamYaple | agreed | 17:39 |
SamYaple | i think thats it | 17:39 |
SamYaple | lets give it a day to confirm though | 17:39 |
sdake_ | pbourke a can you ack that | 17:39 |
sdake_ | what is the percona client equivalent to mariadb package | 17:40 |
sdake_ | mariadb is the client not the server | 17:40 |
SamYaple | what do you mean? | 17:40 |
SamYaple | wrong... | 17:40 |
sdake_ | maridb-server is the server in centos | 17:40 |
SamYaple | mariadb is the server | 17:40 |
SamYaple | it has clients | 17:40 |
SamYaple | percona stuff is for galera | 17:40 |
SamYaple | and there is a repo for centos if you want to add it i believe | 17:41 |
sdake_ | http://ur1.ca/nkrbo -> http://paste.fedoraproject.org/260215/14406973 | 17:42 |
sdake_ | samyaple ^^ | 17:42 |
sdake_ | two yum commands in there | 17:42 |
sdake_ | one lists out mariadb contents one lists out mariadb-server contents | 17:42 |
SamYaple | on youre tlaking about names | 17:42 |
SamYaple | yea package names i dont know | 17:43 |
jpeeler | SamYaple: did people really not like having non-realtime that much to want jumbled logs? | 17:43 |
SamYaple | i was saying mariadb _is- a server | 17:43 |
SamYaple | jpeeler: yea | 17:43 |
SamYaple | i cant say i completely disagree | 17:43 |
sdake_ | i dont care about nonrealtime | 17:43 |
sdake_ | i care about the fact that the logs don't sync up with the errors coming out of docker! | 17:43 |
SamYaple | but that base image takes ~10 minutes to build on alot of systems | 17:43 |
SamYaple | thats 10minutes of no logs at all | 17:43 |
SamYaple | freaks people out | 17:43 |
SamYaple | sdake_: we prit everything docker gives us | 17:44 |
sdake_ | i think the logs all jammed together is going to confuse people | 17:44 |
sdake_ | yes right as docker gives it rather then in order with the build | 17:44 |
jpeeler | i think it'll be obvious, just hard to deal with | 17:44 |
SamYaple | jpeeler: i alwaays redirect to a log file | 17:44 |
SamYaple | i have for a while because of this | 17:45 |
SamYaple | then a simple grep spits out everything i need | 17:45 |
SamYaple | grep nova-base and BAM only nova base stuff | 17:45 |
SamYaple | and all of it too | 17:45 |
sdake_ | i think for the gate that model doesn't work at all samyaple | 17:45 |
jpeeler | well i was just thinking about that | 17:46 |
jpeeler | i mean technically you could wget the logs | 17:46 |
SamYaple | ctrl + f keystone | 17:46 |
SamYaple | thats all teh keystone logs | 17:46 |
jpeeler | ha | 17:46 |
SamYaple | easy to find the error | 17:46 |
SamYaple | its the last enttry ;) | 17:46 |
jpeeler | well, i just thought i would voice it here rather than disagreeing on the review | 17:46 |
SamYaple | i was think we could pump out the full log in order at teh end of the script _it_ it errors | 17:47 |
SamYaple | that should satisfy everyone | 17:47 |
sdake_ | that sounds fantastic | 17:47 |
SamYaple | will do | 17:47 |
jpeeler | yeah i like that | 17:47 |
SamYaple | everyone happy? | 17:47 |
SamYaple | any other thoughts? | 17:47 |
sdake_ | will create double logs | 17:47 |
sdake_ | but thats ok I think ;-) | 17:47 |
SamYaple | eh | 17:47 |
SamYaple | its the gate | 17:47 |
sdake_ | we may have to add a flag later | 17:48 |
* jpeeler wonders how far SamYaple goes to make me happy | 17:48 | |
sdake_ | to select one way or another | 17:48 |
SamYaple | jpeeler: for a fellow south carolina, i think you know | 17:48 |
sdake_ | but i am not keen on adding flags without alot of requests | 17:48 |
jpeeler | SamYaple: :) | 17:48 |
SamYaple | sdake_: ill just make hte extra error output debug info | 17:48 |
sdake_ | that sounds ideal samyaple | 17:48 |
sdake_ | whats going on with that upstream heat bug samyaple | 17:49 |
sdake_ | did any heat cats look at it? | 17:49 |
SamYaple | i gave up | 17:49 |
SamYaple | i pdb'd for 15 hours on it | 17:49 |
SamYaple | i was stepping through sqlalchmey | 17:50 |
SamYaple | it makes no sense | 17:50 |
sdake_ | code used to be easier to understand | 17:50 |
SamYaple | i dont tihnk its heat | 17:50 |
sdake_ | second design syndrome | 17:50 |
SamYaple | full error log print before or after summary prints? | 17:51 |
sdake_ | before | 17:51 |
SamYaple | i would think summary should be bottom | 17:51 |
SamYaple | yea | 17:51 |
sdake_ | that way people can scroll to bottom to see the summary | 17:52 |
sdake_ | make sur the debug output has the data from docker in it ;-) | 17:52 |
SamYaple | duh | 17:52 |
SamYaple | this is why i created the 'logs' key originally | 17:52 |
sdake_ | i haen't looked at or tested your patch ;) | 17:53 |
sdake_ | trying to get multinode in operational state for centos ;-) | 17:53 |
SamYaple | no the 'logs' key was in the initialy build.py patch | 17:53 |
SamYaple | i thought ahead! | 17:53 |
sdake_ | since its busted for everyone i appears that have testedit | 17:53 |
sdake_ | jpeeler re the ansibel_eth0 thing | 17:53 |
sdake_ | jpeeler samyaple has a workaround for the moment | 17:54 |
sdake_ | place *ALL* your nodes in the [network] section | 17:54 |
SamYaple | this is temporary | 17:54 |
sdake_ | duh | 17:54 |
SamYaple | i will have a patch for you guys | 17:54 |
jpeeler | yeah that makes sense, when i start looking at that again i'll probably just look at focusing on all in one rather than multinode | 17:54 |
SamYaple | jpeeler: or multinode running all things | 17:55 |
SamYaple | thats what i do | 17:55 |
jpeeler | SamYaple: i was trying to get multinode working (with vagrant). this was some time ago though | 17:56 |
*** Kennan has quit IRC | 18:03 | |
*** Kennan has joined #kolla | 18:03 | |
SamYaple | sdake_: youll want to stop using -T1000 | 18:07 |
SamYaple | btw | 18:07 |
SamYaple | the threads are in daemon mode now, they stick around for the life of the program | 18:07 |
SamYaple | that actually generates 1000 threads now | 18:07 |
SamYaple | -T50 is more than enough | 18:08 |
SamYaple | the defalut is actually good now | 18:08 |
SamYaple | it wont get much faster i think | 18:08 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Make logging more better https://review.openstack.org/217774 | 18:08 |
*** ftcpops has quit IRC | 18:10 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: DO NOT MERGE - gate haxoring https://review.openstack.org/217831 | 18:11 |
SamYaple | hey matt dorn +1'd the logging patch | 18:12 |
SamYaple | i sit near him | 18:12 |
SamYaple | he builds our training courses | 18:12 |
sdake_ | hey sam can you rebase the heat patch with whatever work is missing | 18:15 |
sdake_ | you seemed to imply the rebase went awry | 18:15 |
SamYaple | im super pissed. i foobar'd that an all my ansible code went away that made heat work right for everyone (not just your machines) | 18:16 |
SamYaple | i rebased it after a 10 hour pdb session | 18:16 |
SamYaple | that was my mistake :( | 18:16 |
sdake_ | bummer | 18:16 |
SamYaple | but yea itll get done | 18:16 |
sdake_ | better them rm -rf /usr/lib/python2.7/site-packages | 18:16 |
sdake_ | I've done that several times :) | 18:16 |
SamYaple | that would be fine on my ubuntu continer | 18:17 |
SamYaple | that is a symlink! | 18:17 |
*** SourabhP has joined #kolla | 18:21 | |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Neutron https://review.openstack.org/217836 | 18:21 |
SamYaple | oops i missed neutron earlier! | 18:21 |
SamYaple | all good now | 18:22 |
rhallisey | shame | 18:22 |
rhallisey | shame | 18:22 |
SamYaple | nooooooooo | 18:22 |
rhallisey | (any GoT fans :p) | 18:22 |
SamYaple | its a reoccuring dream for me | 18:22 |
rhallisey | lol | 18:22 |
SamYaple | so guys | 18:25 |
SamYaple | openvswitch from source | 18:25 |
SamYaple | we should | 18:25 |
SamYaple | any questinos? | 18:25 |
SamYaple | it actually might take to long to compile, but if it doesnt we totally should | 18:26 |
sdake_ | lets focus on librty-3 blueprints that are not finished plz :) | 18:26 |
SamYaple | buzzkill | 18:26 |
SamYaple | btw https://review.openstack.org/#/c/217774/ | 18:27 |
SamYaple | that 'implements' the build-script blueprint | 18:27 |
sdake_ | ok ok i'll revew it | 18:27 |
openstackgerrit | Sam Yaple proposed stackforge/kolla: Unify Centos and Ubuntu Source Install - Neutron https://review.openstack.org/217836 | 18:29 |
SamYaple | that one implements install-from-ubuntu blueprint | 18:30 |
sdake_ | looks good sam | 18:30 |
sdake_ | waiting on gate results | 18:30 |
SamYaple | indeed | 18:30 |
SamYaple | im watching them | 18:30 |
sdake_ | make sure gate still works | 18:30 |
SamYaple | looks good output | 18:30 |
sdake_ | docker cpu = 500% lol | 18:33 |
sdake_ | nice --no-cache-dir didn't know about that | 18:34 |
sdake_ | that should speed up builds alot | 18:34 |
SamYaple | idk maybe. either way it should prevent double writes | 18:35 |
SamYaple | its sequential ~50MB write, so a second or two of disk time | 18:35 |
SamYaple | not that much | 18:35 |
sdake_ | jpeeler or rhallisey can one of you review https://review.openstack.org/#/c/217836/ | 18:36 |
sdake_ | i arrive in tokyo on saturday if anyone wants to hang out ;) | 18:39 |
SamYaple | going a little early arent you | 18:40 |
sdake_ | adjust to tz time | 18:41 |
SamYaple | still.. like a few months early | 18:41 |
sdake_ | samyaple after removing mariadb from base i get this | 18:41 |
sdake_ | TASK: [mariadb | Creating haproxy mysql user] ********************************* | 18:41 |
sdake_ | fatal: [localhost] => Failed to template {{ (haproxy_user.stdout.split('localhost | SUCCESS => ')[1]|from_json).changed }}: an unexpected type error occurred. Error was expected string or buffer | 18:41 |
sdake_ | any idea where to start debug on that one | 18:41 |
SamYaple | i think i should start with a good nights sleep | 18:42 |
SamYaple | i should have been doing that 4 hours ago | 18:42 |
SamYaple | i have to drink heavily tonight | 18:42 |
*** erkules_ is now known as erkules | 18:42 | |
SamYaple | gotta rest up | 18:42 |
*** erkules has quit IRC | 18:42 | |
*** erkules has joined #kolla | 18:42 | |
sdake_ | 150827 18:39:32 [Warning] Hostname 'bigiron' does not resolve to '192.168.1.100'. | 18:43 |
sdake_ | 150827 18:39:32 [Note] Hostname 'bigiron' has the following IP addresses: | 18:43 |
sdake_ | 150827 18:39:32 [Note] - 92.242.140.2 | 18:43 |
sdake_ | groan | 18:43 |
SamYaple | looks like dns problems to me! | 18:43 |
sdake_ | what is haproxy doing when it generates that error | 18:43 |
sdake_ | this is AIO | 18:43 |
sdake_ | which worked previously | 18:43 |
SamYaple | dunno man good thing its a warning | 18:44 |
sdake_ | i am talking about the task faiure above | 18:44 |
SamYaple | thats not haproxy failing | 18:44 |
SamYaple | that the database not being accessible to create an haproxy user | 18:44 |
SamYaple | haproxy has nothing to do with it | 18:44 |
sdake_ | what creates the database? the ansible_kolla container? | 18:45 |
SamYaple | im going to bed | 18:45 |
SamYaple | cant continue at the moment | 18:45 |
sdake_ | enjoy | 18:45 |
SamYaple | i was only here for the okes | 18:45 |
sdake_ | what is an okes | 18:46 |
SamYaple | thats the joke | 18:46 |
SamYaple | i mean i misstypell | 18:46 |
SamYaple | hey if you guys wanna ack this heat bug after you confirm it that would be great | 18:47 |
SamYaple | https://bugs.launchpad.net/heat/+bug/1488779 | 18:47 |
openstack | Launchpad bug 1488779 in heat "stacktrace on heat stack-delete complaining about isotime()" [High,In progress] | 18:47 |
SamYaple | get some more dev love on it | 18:48 |
sdake_ | looks like heat is incompatible with our version of mariadb | 18:53 |
sdake_ | is that accurate? | 18:53 |
SamYaple | no | 18:54 |
SamYaple | i mentioned below there is something more going on | 18:54 |
*** jruano has quit IRC | 18:54 | |
SamYaple | shardy tested against mariadb 10.0.20 no problem (i tested, problem) | 18:54 |
SamYaple | it looks like the problem is outside the heat code so we need to do a full dep compare | 18:55 |
sdake_ | he tested the galera uilt version? | 18:55 |
SamYaple | its not the database trust me | 18:55 |
sdake_ | uil/tbuilt | 18:55 |
SamYaple | database has no problem with fractional seconds | 18:55 |
SamYaple | i tested that by hand | 18:55 |
*** jruano has joined #kolla | 19:00 | |
sdake_ | samyaple where does this module come from: the python mysqldb module is required | 19:02 |
sdake_ | mysql_user seems to be requiring it | 19:02 |
SamYaple | on centos i dont know | 19:02 |
sdake_ | there is a mysql_db in the core directory | 19:02 |
SamYaple | i thinks its MYsql-devel | 19:02 |
sdake_ | does i tmean that or something else? | 19:02 |
SamYaple | on ubuntu the package name is called python-mysqldb | 19:03 |
SamYaple | its a distro package sdake | 19:03 |
sdake_ | can you find the upstream of it in the packaging name on ubuntu samyaple? | 19:03 |
sdake_ | using some apt command | 19:03 |
SamYaple | python-mysqldb | 19:04 |
sdake_ | i am not finding anything | 19:04 |
SamYaple | thats the name | 19:04 |
sdake_ | no i mean the upstream pip repo | 19:04 |
SamYaple | its not in pip | 19:04 |
SamYaple | its a distro package | 19:04 |
sdake_ | i see | 19:04 |
sdake_ | its not called that on centos | 19:04 |
SamYaple | 19:03 < SamYaple> i thinks its MYsql-devel | 19:04 |
sdake_ | can you paste a list of files it installs on ubuntu | 19:05 |
SamYaple | i dont know though | 19:05 |
sdake_ | i can find it that wya | 19:05 |
sdake_ | like rpm -qf works | 19:05 |
SamYaple | not at the moment | 19:05 |
sdake_ | or mayee its -ql | 19:05 |
SamYaple | i is in bed | 19:05 |
sdake_ | groan | 19:05 |
SamYaple | ask the internet | 19:05 |
SamYaple | its a very common package | 19:05 |
SamYaple | sdake: gate says yes https://review.openstack.org/#/c/217774/ | 19:06 |
SamYaple | (20 minute build time woohoo) | 19:06 |
SamYaple | yum install MySQL-python | 19:07 |
SamYaple | literally the first link in google | 19:07 |
openstackgerrit | Merged stackforge/kolla: Make logging more better https://review.openstack.org/217774 | 19:09 |
SamYaple | best commit name ever | 19:09 |
*** chandankumar has joined #kolla | 19:15 | |
*** ftcpops has joined #kolla | 19:16 | |
sdake_ | hey ftcops did you get your system rolling | 19:16 |
ftcpops | No, in fact I came home a little bit ago to get around my corporate proxy and it's still failing the same way | 19:27 |
ftcpops | I set up a Fedora 21 VM and was getting the containers built over there when I came home. I'll try to get it working there next | 19:28 |
openstackgerrit | Major Hayden proposed stackforge/kolla: Fixing CentOS quickstart docs https://review.openstack.org/217860 | 19:35 |
openstackgerrit | Merged stackforge/kolla: Unify Centos and Ubuntu Source Install - Neutron https://review.openstack.org/217836 | 19:37 |
sdake_ | ftcpops could you go into more detail about whatis failing and what command your typing | 19:40 |
sdake_ | ftcpops memory fading :) | 19:40 |
ftcpops | docker_pull_policy: always ==> http://paste.openstack.org/show/428765/ and | 19:43 |
ftcpops | docker_pull_policy: missing ==> http://paste.openstack.org/show/428766/ | 19:43 |
*** jruano has quit IRC | 19:57 | |
*** achanda has quit IRC | 19:58 | |
sdake_ | your not pulling from a registry | 19:59 |
sdake_ | so y9ouo want to use pull policy missing | 19:59 |
sdake_ | that tells kolla to use the locally registered images in the docker image cache | 20:00 |
sdake_ | you see this erro in the missing | 20:00 |
sdake_ | msg: Docker API Error: Cannot start container 3043d21824f95fad1ecdcc2ea8f343030427ab952383e04d7cddeb2ac41290a0: [8] System error: exec format error | 20:00 |
sdake_ | which version of docker arey ou using? | 20:00 |
sdake_ | i suspect you are out of disk space | 20:01 |
sdake_ | how much disk did you give your vm? | 20:01 |
ftcpops | hm taht could be | 20:01 |
ftcpops | I gave the VM 20Gb | 20:01 |
sdake_ | fedora 21 by default does something wierd like give /home 90% of the disk | 20:01 |
sdake_ | you have to mnaully specify your disk ormat | 20:01 |
sdake_ | you need ~100gb of disk space | 20:01 |
sdake_ | maybe more | 20:01 |
ftcpops | then 20Gb won't be enough ;-) | 20:01 |
ftcpops | hehe | 20:01 |
ftcpops | okay, let me do some resizing and I'll try again | 20:02 |
sdake_ | yup that is what the exec format is about | 20:02 |
*** chandankumar has quit IRC | 20:02 | |
sdake_ | make sure /var filesystem has enough diskspace | 20:02 |
sdake_ | again fedora21 will put most storage into /home | 20:02 |
ftcpops | ok, will do | 20:02 |
*** jasonsb has quit IRC | 20:06 | |
*** jasonsb has joined #kolla | 20:06 | |
jpeeler | docker info should give insight into disk space usage | 20:07 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file https://review.openstack.org/216874 | 20:07 |
jpeeler | i definitely didn't think exec format had anything to do with disk space - if so, that error is horrible | 20:07 |
rhallisey | eh that's not completely right.. | 20:08 |
jpeeler | does anybody know of a service that modifies the config file of another service? | 20:08 |
jpeeler | (in kolla) | 20:08 |
sdake_ | jpeeler i dont think that is possible | 20:09 |
jpeeler | uh oh | 20:09 |
jpeeler | i mean i'm sure it's possible somehow... | 20:09 |
sdake_ | tell me what your tyring to do | 20:09 |
sdake_ | is this related to ironic? | 20:09 |
jpeeler | ironic requires nova configuration changes | 20:09 |
sdake_ | ok that is possible | 20:10 |
sdake_ | in the nova/defaults dir under ansible | 20:10 |
sdake_ | /home/sdake/kolla/ansible/roles/nova/templates | 20:10 |
sdake_ | there is a file claled nova.conf.j2 | 20:11 |
sdake_ | use a jinja2 conditional if ironic - configure for ironic else configure for not ironic | 20:11 |
jpeeler | ah ok | 20:11 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file https://review.openstack.org/216874 | 20:13 |
*** akwasnie has joined #kolla | 20:15 | |
sdake_ | jpeeler i thhink you are goin to have to end up with two nova containers | 20:15 |
sdake_ | one for ironic and one for not ironic | 20:15 |
sdake_ | unless one nova server can handle vm and bare metal | 20:16 |
jpeeler | honestly don't know how that would work, but the config conditional would be the same either way at least | 20:16 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file for Galera https://review.openstack.org/217880 | 20:21 |
*** achanda has joined #kolla | 20:22 | |
*** achanda has quit IRC | 20:22 | |
*** achanda has joined #kolla | 20:23 | |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file https://review.openstack.org/216874 | 20:27 |
openstackgerrit | Merged stackforge/kolla: Fixing CentOS quickstart docs https://review.openstack.org/217860 | 20:28 |
sdake_ | rhallisey more comments for you looking better ;) | 20:34 |
rhallisey | sdake_, thanks | 20:35 |
rhallisey | I fixed a bunch of those | 20:35 |
rhallisey | I wrote some c++ in there | 20:35 |
rhallisey | lost my mind | 20:35 |
rhallisey | sdake_, you can test it out with galera | 20:37 |
rhallisey | works | 20:37 |
sdake_ | focused on liberty-3 atm no time to test ;) | 20:37 |
*** jruano has joined #kolla | 20:37 | |
rhallisey | this could make it | 20:37 |
sdake_ | rhallisey <ryansb>sdake_: from pwd import getpwnam | 20:38 |
sdake_ | [13:37:53] <ryansb>getpwnam('ryansb').pw_uid | 20:38 |
sdake_ | [13:38:12] <sdake_>ryansb thanks! | 20:38 |
sdake_ | thats the proper way to do it | 20:38 |
rhallisey | cool thx | 20:38 |
sdake_ | if we are to merge this i want json lists if possible | 20:40 |
rhallisey | that's fine, it will come in following iterations | 20:40 |
sdake_ | rhallisey where is the start.sh change made? | 20:41 |
sdake_ | lets get itin the bse patch | 20:41 |
sdake_ | rther then reworking evrything later | 20:41 |
rhallisey | what change are you referring to? | 20:42 |
sdake_ | the stuff thatruns set_configs.py | 20:42 |
rhallisey | https://review.openstack.org/#/c/217880/1/docker/common/galera/start.sh | 20:42 |
*** jasonsb has quit IRC | 20:43 | |
rhallisey | the 2nd patch I put up | 20:43 |
sdake_ | i see - your dependencies in gerrit are broken | 20:45 |
sdake_ | i believe instead of template there is a copy command | 20:45 |
rhallisey | keystone has template | 20:46 |
*** jasonsb has joined #kolla | 20:46 | |
rhallisey | but it does it differently | 20:46 |
rhallisey | https://github.com/stackforge/kolla/blob/master/ansible/roles/keystone/tasks/config.yml#L19 | 20:46 |
rhallisey | sdake_, ^ that format ok? | 20:46 |
sdake_ | i am reviewing your patch moment | 20:47 |
rhallisey | sdake_, k let me know when you're done got some more additions | 20:48 |
sdake_ | ok done | 20:52 |
sdake_ | get your patches with proper dependencies in gerrit pls :) | 20:52 |
sdake_ | so they are not outdated | 20:53 |
sdake_ | i am pretty sure there is a straight up copy operation | 20:53 |
sdake_ | you are right this coudl make liberty-3 | 20:54 |
sdake_ | if you hussle ;) | 20:54 |
rhallisey | sdake_, once I have a template for this ya | 20:54 |
rhallisey | sdake_, whats the deadline like tmr? | 20:54 |
sdake_ | aug31 is soft deadline | 20:54 |
sdake_ | sept 3rd is hard deadline | 20:54 |
rhallisey | kk | 20:54 |
sdake_ | depends where we are in that range | 20:55 |
sdake_ | i am ok with not autogenerating the json for the moment | 20:55 |
sdake_ | we can tackle that in rc3 | 20:55 |
rhallisey | sdake_, so I looked into that. I didn't find ansible very friendly for that kind of operation | 20:56 |
sdake_ | http://docs.ansible.com/ansible/copy_module.html | 20:57 |
sdake_ | for which operation, genrating the json on the fly? | 20:57 |
rhallisey | it would look like: context: { { "base_file": { "source": "/opt/kolla/config" } }} | 20:57 |
rhallisey | so for neutron it would be 10 bigger | 20:57 |
rhallisey | 10x | 20:57 |
rhallisey | ya on the fly | 20:58 |
sdake_ | ok well lets tackle generation on the fly later | 20:58 |
sdake_ | lets tackle list format now tho | 20:58 |
rhallisey | ya sounds good | 20:58 |
sdake_ | we dont need a dictionary | 20:58 |
rhallisey | testing w/ copy | 20:58 |
sdake_ | what you want is a list of dictionaries | 21:01 |
sdake_ | I think [ {"source": "/optkolla/config", "dest": "destdir", "owner"="rootin", "perms" = "perms"} , { .. }, { .. } ] | 21:02 |
rhallisey | ya looks good I have it on another branch will pull in after this round of testing | 21:03 |
sdake_ | ok well get a patch up so i can beat on the format of the code | 21:03 |
sdake_ | one with the pwname | 21:03 |
rhallisey | how can I get gid? | 21:03 |
*** jtriley has quit IRC | 21:03 | |
sdake_ | try pw_gid | 21:04 |
sdake_ | instead of pw_uid | 21:05 |
*** sdake has joined #kolla | 21:23 | |
*** sdake_ has quit IRC | 21:25 | |
*** akwasnie has quit IRC | 21:26 | |
*** sdake_ has joined #kolla | 21:26 | |
sdake_ | rhallisey close to done with another patch | 21:26 |
sdake_ | ? | 21:26 |
rhallisey | sdake_, I was but I added a few things | 21:27 |
rhallisey | test takes like 5 min | 21:27 |
rhallisey | cause the build | 21:27 |
rhallisey | sometimes | 21:27 |
rhallisey | if I change anything in the base image (set_configs.py) takes a few minutes | 21:28 |
*** sdake has quit IRC | 21:30 | |
sdake_ | you put set_configs.py in the bae image? | 21:32 |
rhallisey | sdake_, ya | 21:41 |
sdake_ | soundsgood | 21:42 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: Replace config-external with a JSON file for Galera https://review.openstack.org/217880 | 21:50 |
sdake_ | why do you need to specify SERVICE? | 21:56 |
sdake_ | rhallisey ^^ | 21:56 |
rhallisey | sdake_, oh my bad I didn't commit my comments | 21:57 |
rhallisey | sdake_, k they're there now | 21:57 |
*** jruano has quit IRC | 22:08 | |
sdake_ | rhallisey you didn't relaly address many of my comments dude | 22:12 |
sdake_ | like the getpwname thing for example | 22:13 |
sdake_ | all the trailing periods | 22:13 |
sdake_ | etc | 22:13 |
sdake_ | i've marked them again please address all comments so we can get down to the nitty gritty | 22:13 |
sdake_ | I realize its painful but this will be used in every container I want to make sure the code is flawless | 22:13 |
rhallisey | sdake_, I'm aware a set behind you | 22:14 |
rhallisey | on comments | 22:14 |
sdake_ | i see | 22:14 |
sdake_ | well let try to get er done today | 22:14 |
rhallisey | testing base changes sucks | 22:14 |
sdake_ | so pepole can have a non wip to review before friday | 22:15 |
rhallisey | 5min a pop | 22:15 |
sdake_ | do this make all the changes | 22:15 |
sdake_ | and when i'm done with the review | 22:15 |
sdake_ | then test ;) | 22:15 |
rhallisey | oh I didn't push a new patch for the first one | 22:15 |
sdake_ | like when I think the code is gtg | 22:15 |
sdake_ | that might explain it | 22:15 |
rhallisey | sdake_, that's why there are no changes addressed | 22:15 |
*** SourabhP has quit IRC | 22:24 | |
*** SourabhP has joined #kolla | 22:26 | |
*** ftcpops has quit IRC | 22:28 | |
*** ccrouch has quit IRC | 22:36 | |
*** unicell1 has joined #kolla | 22:42 | |
*** unicell has quit IRC | 22:42 | |
*** akwasnie has joined #kolla | 22:43 | |
rhallisey | sdake_, I can give you an untested patch if you want to look at it while I head out for a few minutes | 22:44 |
sdake_ | yes push it up i'll review | 22:45 |
sdake_ | push up both your patches please | 22:45 |
sdake_ | and i'll review and make comments | 22:45 |
sdake_ | lets get it iterated on | 22:45 |
sdake_ | then you can test | 22:45 |
sdake_ | make sure to label them Wip | 22:46 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file https://review.openstack.org/216874 | 22:49 |
rhallisey | ok next one.. | 22:49 |
sdake_ | why not do git review on both at same tiem ? :) | 22:49 |
rhallisey | I didn't fix the other | 22:50 |
rhallisey | yet | 22:50 |
rhallisey | only will take a sec | 22:50 |
openstackgerrit | Ryan Hallisey proposed stackforge/kolla: WIP: Replace config-external with a JSON file for Galera https://review.openstack.org/217880 | 22:56 |
rhallisey | sdake_, ok I'll be back in an hour | 22:56 |
rhallisey | sdake_, I'll come back and test it | 22:57 |
rhallisey | thanks for the reviews | 22:57 |
rhallisey | brb | 22:57 |
*** rhallisey has quit IRC | 22:57 | |
sdake_ | more comments | 22:59 |
*** ccrouch has joined #kolla | 22:59 | |
*** jruano has joined #kolla | 23:09 | |
*** dims_ has joined #kolla | 23:18 | |
*** akwasnie has quit IRC | 23:20 | |
*** rhallisey has joined #kolla | 23:20 | |
*** dims has quit IRC | 23:22 | |
*** ccrouch has quit IRC | 23:35 | |
*** dims_ has quit IRC | 23:38 | |
*** SourabhP has quit IRC | 23:43 | |
*** achanda_ has joined #kolla | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!