Monday, 2015-07-06

*** dolpher has joined #kolla00:14
*** jpeeler has quit IRC00:29
*** jpeeler has joined #kolla00:43
*** mandre has joined #kolla00:46
dolpherSeems the "kollaglue/centos-rdo-glance-data" and "kollaglue/centos-rdo-neutron-linuxbridge-agent" don't exist on docker's registry?00:49
*** mandre has quit IRC00:50
*** sdake has quit IRC00:53
*** dims__ has joined #kolla00:55
*** dolpher has quit IRC00:56
*** dims_ has quit IRC00:56
*** dims__ has quit IRC01:06
*** dims_ has joined #kolla01:07
*** dims_ has quit IRC01:15
*** erkules_ has joined #kolla01:26
*** erkules has quit IRC01:29
*** sdake has joined #kolla02:11
*** sdake_ has joined #kolla02:14
sdake_hry golkd02:16
sdake_hey folks02:16
sdake_dolpher the containers have not been pushed to the kollaglue repo02:16
sdake_i am not ssure why I am unable to do it on my end, but its a technical problem02:16
sdake_perhaps someone else can push02:17
sdake_in the meantime, yyou can just build locally02:17
sdake_it takes  about  40 minutes instead of 10 minutes (to pull)02:17
sdake_samyaple can youtry pushign the containers?02:17
*** sdake has quit IRC02:18
*** jasonsb has quit IRC02:31
*** jasonsb has joined #kolla02:34
sdake_jasonsb around?02:36
*** dolpher has joined #kolla02:36
sdake_dolpher02:36
sdake_around?02:36
*** coolsvap|away is now known as coolsvap02:36
jasonsbsdake_: yessir02:37
sdake_jasonsb the midcycle is available for registration - see topic for link02:38
*** dolpher_ has joined #kolla02:38
jasonsbon it02:38
sdake_please take care to register the folks you intend on coming ;)02:38
sdake_thans02:38
dolpher_sorry, my network is bad02:38
jasonsbyou got it.  thanks again02:38
sdake_dolpher_ no problem02:38
sdake_dolpher_ essentially the docker images have not been pushed to the docker registry02:38
sdake_it is a technical issue on my system02:38
sdake_in the meantime, you can workaround by building all imags locally on your system02:39
sdake_it takesabout 45 minutes to do so02:39
dolpher_ok, no problem02:39
sdake_whereas a pull takes 10 minutes02:39
dolpher_I built those images on my own02:39
sdake_we will push the images soon as I can sort out the technical issues02:39
sdake_cool so your all set02:39
sdake_there is already a bug open on this issue02:39
sdake_i'm going to tyr to get one of the other core reviewers to push the images02:39
sdake_tyr to get / will get ;)02:40
dolpher_Good to know that:-)02:40
*** dolpher has quit IRC02:40
*** dolpher_ has quit IRC02:43
*** coolsvap is now known as coolsvap|away02:50
*** walters has joined #kolla03:11
SamYaplesdake_: i can try pushing tonight, but i still haven't setup the RAX servers yet03:16
sdake_samyaple no sweat -whenever you can get  to it - but try to push the tagged version03:17
SamYaplejust --release?03:17
sdake_--releae --push is what i use03:17
SamYapleok03:17
sdake_spelled properly of course ;)03:17
SamYapleso are we going to support things like kilo-1 ... kilo-23 tags?03:17
sdake_huh?03:17
SamYapleevery time we push that push should have a unique tag03:18
SamYapleare we going to do that03:18
SamYapleon top of having the image also tagged as "latest"03:18
sdake_we have taled about it03:19
sdake_but we don't have a way to do it in the gate because of the time limits03:19
sdake_so not sure03:19
SamYapleits instant?03:19
sdake_is the answer thus far03:19
sdake_say gotta eat my salad, bbiaf ok?03:19
sdake_yes but we only push latest 3 times a release03:19
sdake_not on every release03:19
sdake_pushing takes about 5 hours03:19
sdake_rather not on every commit03:20
sdake_let me eat, i'll bb in 20 mins or so ok?03:20
SamYapleoh. i see what you are saying. yea we gotta fix that otherwise well always be woefully out of date for security patches03:20
SamYaplesure03:20
openstackgerritSam Yaple proposed stackforge/kolla: Ansible Glance support  https://review.openstack.org/19849403:23
sdake_samyaple bcak- btw i think its called "ansible-service"03:28
*** nihilifer has joined #kolla03:28
SamYaplenah lets go with server03:28
openstackgerritSam Yaple proposed stackforge/kolla: Ansible Glance support  https://review.openstack.org/19849403:28
sdake_samyaple thanks ;)03:29
SamYapledid you see https://review.openstack.org/#/c/198520/03:29
SamYaplewhy is jenkins making actual comments now03:30
SamYaplenvm03:30
SamYaplewierd formating issue03:30
sdake_i did see, I was waiting until monday for the security cat I asked to look at the bug to taek a look03:30
sdake_but we can pprove as is03:30
SamYaplewho did you ask? someone commented on the bug03:30
SamYaplefrom cisco it looks like03:30
sdake_hmm03:31
sdake_mental block03:31
sdake_sec03:31
*** tobe has joined #kolla03:31
sdake_ya dave03:31
sdake_thats who i asked03:31
SamYapleyea he commented, i responded. It wasn't useful because we dont use python here (ansible does, but thats not us)03:32
sdake_yup03:33
sdake_he didn't quite understand the issue - I line dthe review for the fix03:33
sdake_asked him to take a look with a review03:33
*** nihilifer has quit IRC03:34
SamYaplecool. i played around, we _might_ be able to make the name unique per tempfile, but i didn't get it to work immediately. The 'lookup' part is what kills us03:34
SamYaplei looked at how to fix ansible to avoid this whole mess but it is more complicated than i thought due to the structure03:35
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for cinder-app  https://review.openstack.org/19822303:53
*** coolsvap|away is now known as coolsvap03:53
sdake_coolsvap work looking good ;)03:55
coolsvapsdake_, one update i misssed03:56
coolsvapDockerfile for cinder-base03:56
*** sdake_ is now known as sdake03:56
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for cinder-app  https://review.openstack.org/19822303:57
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external to Designate containers  https://review.openstack.org/19827504:01
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for Ceilometer  https://review.openstack.org/19824104:05
coolsvapsdake, will you review https://review.openstack.org/#/c/181811/, I had done this sometime back04:09
sdakecoolsvap not know sure it does but looks ok to me04:11
*** dims_ has joined #kolla04:12
*** dims__ has joined #kolla04:16
*** dims_ has quit IRC04:18
*** dims__ has quit IRC04:20
*** dims__ has joined #kolla04:21
*** dims___ has joined #kolla04:22
*** dims__ has quit IRC04:26
*** dims___ has quit IRC04:27
*** daneyon has joined #kolla04:28
*** daneyon has quit IRC04:28
*** daneyon has joined #kolla04:28
*** coolsvap is now known as coolsvap|brb04:35
*** tobe has quit IRC04:51
*** nihilifer has joined #kolla04:53
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for cinder-app  https://review.openstack.org/19822305:02
*** coolsvap|brb is now known as coolsvap05:06
nihilifergood morning05:11
SamYaplemorning05:12
vincent_vdkgood morning05:16
vincent_vdki just tried to deploy the kolla containers but it seems that some images are missing..05:17
vincent_vdkcompose.progress_stream.StreamOutputError: Error: image kollaglue/centos-rdo-glance-data:latest not found05:17
sdakebuild your own images vincent_vdk05:17
vincent_vdkah ok05:17
sdakethe images are not pushed at present because of technical problems on my end05:18
vincent_vdki see05:18
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for Ceilometer  https://review.openstack.org/19824105:18
sdakesamyaple said he will get images up in 1-2 days I think05:20
vincent_vdkimages are building :)05:21
sdakemake sure to use --release as a flag vincent_vdk05:24
vincent_vdkok05:24
sdakeotherwise you wont end up with images tagged with :latest05:24
vincent_vdkah05:24
vincent_vdklets restart then05:25
sdakenote if you already have built, and you use --release, it will just tag them05:25
sdakeso the tagging isn't all that slow - probablytakes 1-2 minutes at max after you have already built05:25
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Add config-internal/config-external for horizon  https://review.openstack.org/19860005:26
*** coolsvap has left #kolla05:27
sdakeperhaps next time we should start with the big one ;)05:29
*** bmace has quit IRC05:31
vincent_vdkmust have been quite some work to build the container images05:35
sdakeyou mean to implemenet the code base?05:37
vincent_vdkall the work to get teh openstack services in container images05:37
vincent_vdkand the tooling too05:37
sdakemost of the challenge was sorting out what to do and building the community05:38
SamYaplegetting the community to agree....05:39
sdakewe have gone through about 2-3 major reworks which I'd consider "from scratch" implementations05:39
sdakeor to disagree and commit samyaple ;)05:39
sdakethe really cool thing is each time we did one of these major pieces of rework, our community size doubled :)05:41
sdakeyet it took lots of work to get existing community to agree on change in scope05:41
vincent_vdkis it a goal to use kolla to deploy production workloads05:42
vincent_vdkor is it rather meant for development setups05:42
sdakeproduction workloads05:42
vincent_vdkok, cool05:42
sdakeidea is kolla is an upstream for downstream openstack products05:43
sdakethat way everyone working on making an openstack product doesn't have to have a 40 person team on staff to do the job05:43
sdakeinstead eveyrone can split the cost of the dev05:43
*** bmace has joined #kolla05:43
sdakeand hopefully end up with somehting better then they could have done on their own in a proprietary way05:43
sdakethat said, some folks want to replace devstack with kolla05:44
vincent_vdkwhich makes sense05:44
sdakei personally dont see the appeal, but maye i'm dense ;-)05:44
vincent_vdkeven better to use the same thing for dev/prod05:44
sdakebut folks will do whatever they want, so if the cmmunity ends up wanting to have an alternative to devstack - as long as it meets quality standards of the review process not going to stop it with a -2 vote05:45
vincent_vdki hope i can push this a bit here internally05:45
*** dasm|afk is now known as dasm05:46
sdakenote however I think our prioerity shoudl not be devstack replacement but openstack deployment ;)05:47
dasmhello o/05:47
sdakehey dasm05:47
sdakebut i'm just 1 of our 15-20 person dev community05:47
SamYaplesdake: i agree. it should not be a priority to be a devstack replacement, _however_ we are essentially a devstack alternative by merely existing05:49
SamYapledev a patch and then using that exact same container in testing, and then in production if there is no issues is a major thing05:49
SamYaplethe bit-for-bit dev stuff is now in production. awesome05:50
sdakeyup i see the appeal of that part of things05:50
sdakebut i think the dev cycle is slower with kolla05:50
sdakehave to rebuild containers05:50
sdaketakes more time then stopping/starting a process05:51
SamYaplesdake: the dev cycle is much much faster with kolla05:51
sdakethe big problem with devstac is it wrecks your system after it does its thing05:51
*** bmace has quit IRC05:52
SamYaplerebuilding a container, singular, takes 5 minutes, 10 tops05:52
SamYaplethats with no cache05:52
sdakedid your faster rebuild tool hit the repo yet samyaple05:52
SamYapleno i just added the BP, im working in docker-py libs now05:52
SamYaplewe can merge it when push exists for it05:53
SamYaplethen there would be varioaus todo and imporvements for it05:53
SamYaplebut it can be an alternate script until its ready anyway05:53
sdaketake care to eep it wip then - might put in commit msg title05:53
sdakei was thining for the short term we could eep both05:54
sdakeright05:54
SamYapleyea i was planning on merging it now, but people kept asking for htings05:54
SamYaplenext commit will be merge ready05:54
SamYapleno WIP05:54
sdakelet sjust get it ready for merge05:54
sdakeand merge it in05:54
SamYapleyup05:54
sdakeso we can get some time using it in the real dev cycle05:54
*** bmace has joined #kolla05:54
*** bmace is now known as bmace_away05:54
SamYapleonce it sits for a while we can add the templating to remove our symlink stuff really quickly05:55
SamYapleonly one function needs to change05:55
sdakeyou want to do that prior to l2 then?05:55
sdakeI'd prefer to wait until l305:55
sdakefor templating05:56
sdakewe got alot of changes queued up as is05:56
SamYaplei agree. the build script should be baked in by the start of l305:56
SamYaplethen we can hammer out templating05:56
sdakeya lets get it merged now but wait on templating05:56
sdakeor put the code in but not re-factor our code bae for templating until l305:56
SamYaplethats the plan stan05:56
sdakecool because we only have 3 weeks left to get er all done ;)05:57
SamYapleit should actually greatly reduce our code base size05:57
sdakeand by er, I mean source based installs and ansible05:57
SamYaplei estimate by 50% for the dockerfiles05:57
sdakei just dont want a 2 week slip because of a bunch of changes at l205:57
SamYaplemainly because we can share the common "COPY" lines between all distros now05:57
SamYapleim with you. templating is L3 agreed05:58
sdakewe are already pretty much rewriting the entire code base for l2 ;)05:58
sdakefortunately we only ahve 2-3k lines of code05:58
sdakeso its not like a huge deal05:58
SamYapleIm just happy that we are both adding and removing code. a good project is about reducing lines of code as much as adding it05:58
sdakefortunately our l2 refactor keeps the internal config model mostly the same06:01
sdakeso that model of usage remains mostly unaltered minus the one environment variable we had to add06:01
*** dims_ has joined #kolla06:10
*** dims_ has quit IRC06:15
vincent_vdkany idea what this can be06:19
vincent_vdktools/kolla: line 78: keystone: command not found06:19
vincent_vdki suppose keystone is only available from within the container?06:19
sdakevincent_vdk: https://github.com/stackforge/kolla/blob/master/docs/dev-quickstart.md06:20
sdakeneed to install the clients for various services06:20
vincent_vdkah ok06:21
sdakeNext, install the OpenStack python clients if they are not installed:06:21
sdakesudo yum install python-keystoneclient python-glanceclient \06:22
sdake  python-novaclient python-heatclient python-neutronclient06:22
SamYaplesdake: that should really be switched to python-openstackclient06:22
SamYaplefor most of that at least06:22
sdakeour init-runonce needs to be modified to support osc06:22
SamYapleopenstackclient doesn't support glance image uploads, but openstackclient is _required_ for some keystone bits06:22
vincent_vdki'll have to add some repos it seems06:23
vincent_vdkcan i install from pip?06:23
SamYaplevincent_vdk: i would recommend it06:23
sdakepip should work06:23
vincent_vdkok06:23
sdakevincent_vdk make sure to read that quickstart, another critical thing is you need to turn off the system libvirtd06:28
*** daneyon has quit IRC06:28
sdakeok folks I've hit the wall - bed time :)06:28
vincent_vdkbye06:29
*** tobe has joined #kolla06:38
vincent_vdkis it possible there is a bug in the swift base dockerfile06:49
vincent_vdki see this line06:49
SamYaplevincent_vdk: swift does not work06:49
vincent_vdkconfig-swift.sh /opt/swift/config-swift.sh06:49
vincent_vdkah ok06:49
SamYapleit is the only container series that is broken06:50
vincent_vdkthen there is another isseu i need to figure out06:50
*** Slower has joined #kolla06:53
*** tobe has quit IRC07:08
*** dims_ has joined #kolla07:12
*** dolpher1 has joined #kolla07:13
*** dims_ has quit IRC07:16
*** dolpher has joined #kolla07:17
*** dolpher1 has quit IRC07:19
*** tobe has joined #kolla07:20
*** dolpher1 has joined #kolla07:26
openstackgerritSam Yaple proposed stackforge/kolla: make merge_configs idempotent  https://review.openstack.org/19851907:26
*** dolpher has quit IRC07:27
*** athomas has joined #kolla08:01
*** daneyon has joined #kolla08:17
*** daneyon has quit IRC08:22
*** jmccarthy has joined #kolla08:39
*** gfidente has joined #kolla08:59
*** pbourke has quit IRC09:26
*** pbourke has joined #kolla09:27
*** athomas has quit IRC09:52
*** athomas has joined #kolla10:02
*** mandre has joined #kolla10:03
*** daneyon has joined #kolla10:06
*** daneyon has quit IRC10:11
*** dims_ has joined #kolla10:47
*** dims_ has quit IRC10:52
harmwpbourke: thanks for the review on mariadb, the aim was to get it running with /start.sh and not a thourough cleanup. Ill see if I can put more in commitmsg, and that hint about bash4 is a good reason for another patchset as well10:54
pbourkeharmw: yes, the patch looked like a cleanup, which is why I didn't expect to see any content change on the scripts themselves10:56
pbourkeharmw: if the changes are required to merge the scripts can you note that in the commit msg, if not I would vote to drop them10:57
pbourkeor add them in another commit10:57
*** tobe has quit IRC11:01
*** mstachow has joined #kolla11:04
mstachowo/ all11:04
SamYapleharmw: i commented there, you dont need to adjust the mariadb containers11:07
SamYaplei dont think the mariadb containers iwll ever get config-external love since that container doesn't have any way to HA itself11:10
*** rhallisey has joined #kolla11:15
harmwno probably not, but it sucks having some containers differ from the rest11:28
harmwhence this commit11:29
SamYapleim actually hoping we deprecate that container... for galera11:29
SamYaplesince galera can do single host11:29
harmwSamYaple: didn't I propose something like that already?11:29
harmwduring galera development11:29
SamYaplei said that was a good i dea11:30
SamYaplethre was worry that it didnt work single host11:30
SamYapleit does as i use it single host11:30
harmwweird, since triplo is using it single host11:30
harmwiirc, you objected initially11:30
SamYapleidk im here now11:30
SamYapleno that was because of mariadb 5.5 mariadb 1011:31
harmwand that involved the patches, wsprep11:31
harmw5.5 needed additional patches, 10 doesn't11:31
harmwanyway.. :)11:32
harmwill take another look tonight pbourke11:32
harmwwe shouldn't waste much time on this though, thats not the intent11:32
SamYapleactually its not that 5.5 needed more, its that the version in the repo DOESNT HAVE the package11:32
SamYaplepatches*11:32
*** bmace_away has quit IRC11:33
harmwok11:33
SamYaplebasically whats in the repo is missing the packages, 10 isnt in the repo11:33
*** rhallisey has quit IRC11:36
harmwso SamYaple, instead of aligning mariadb it would be more beneficial to move from maria to single host galera straight away11:38
SamYapleeveryone needs to stop calling it galera :(11:38
SamYapleinstead of mariadb without galera, we should just drop mariadb contaienrs and use the mariadb with glarea containers11:38
harmw:)11:39
harmwill abandon this work, and open up a new bug to get that going11:39
*** bmace_away has joined #kolla11:45
*** mandre has quit IRC11:48
*** mandre has joined #kolla11:50
*** mstachow has quit IRC11:53
SamYapleok sorry about the shortness before. environment down, on the phone with unrelated issues and several irc notifications11:54
SamYapleall cleaned up though11:54
*** daneyon has joined #kolla11:55
*** daneyon has quit IRC12:00
*** bradjones has joined #kolla12:01
*** bradjones has quit IRC12:01
*** bradjones has joined #kolla12:01
*** dims_ has joined #kolla12:03
*** mstachow has joined #kolla12:05
*** rhallisey has joined #kolla12:05
*** dims_ has quit IRC12:08
harmwhehe12:13
openstackgerritRyan Hallisey proposed stackforge/kolla: Install oslo.service in cinder  https://review.openstack.org/19869712:23
harmwrhallisey: how's the cinder work holding up, regarding config-internal?12:24
openstackgerritRyan Hallisey proposed stackforge/kolla: Add oslo package into cinder  https://review.openstack.org/19869712:24
rhalliseyharmw, I think someone already has a patch for that12:24
harmwI know, your name is on the etherpad listed as 2nd :)12:25
SamYaplemorning rhallisey12:25
rhalliseyya I missed the meeting so I just added my name to it12:25
harmwand given the current review scored -1, I was wondering perhaps you've got anything underway12:25
rhalliseySamYaple, morning12:25
*** diga has quit IRC12:26
rhalliseyharmw, unfortunately I don't, the works looks mostly done I'll have to review12:26
harmwok12:26
pbourkeSamYaple: when you say, "Additionally, this will have to implement the openstack-base container rather than just have a placeholder folder." - can you expand?12:27
pbourkewhat implementation is needed?12:27
SamYaplereview number?12:28
pbourkehttps://review.openstack.org/#/c/197897/12:28
pbourkeI understand your first point, just not the second12:29
SamYapleah i see you have a symlink rather than what I thought was happening12:30
SamYapleThis needs to be discussed more. I think both binary AND source can benefit from a 'base -> openstack-base -> glance-base' structure12:30
pbourkeill take a look and see how distruptive it would be to add something like this12:32
SamYapleit would touch all the openstack services, and we would refactor the base image12:34
SamYaplenot to much work to be honest12:34
SamYapledid the same migration in yaodu12:34
SamYaplepbourke: have you actually run the ansible bits?12:35
pbourkeSamYaple: Ive just ran the rabbitmq role to see how it works12:36
SamYaplethoughts?12:36
SamYapledid you do aio, multihost?12:36
SamYaplei need feedback ive been coding in a backhole!12:36
pbourkeaio. just enough to be able to review the related patches really. I can try a multihost one now12:37
SamYapleoh oh scale from aio to multihost12:37
SamYapledo it12:37
SamYapledo it do it do it12:37
SamYapleits slick12:37
pbourkeha12:37
pbourkeok12:37
SamYapleharmw: https://review.openstack.org/#/c/198494/12:38
SamYaplei thought that was interesting too12:38
SamYaplebut it turns out the db code uses the galnce-api config file12:38
pbourkeSamYaple: the first thing I tripped over actually was it the very latest ansible is required (1.9.2)12:40
pbourkethere's some reference to this bug in one of the roles but others suffer from the same problem12:40
SamYapleoh yea... delegate_to12:40
SamYapleit requires 1.912:40
pbourkeit was something around the docker_api_version12:41
pbourkeok, second. why is aio trying to access /usr/share/kolla12:41
SamYapleoh that one too. that was because you were running a to new version of docker-py12:41
SamYaplethats where it dumps template files12:42
pbourkeneed to mention that in the docs12:42
pbourkeI'll file these as I find them12:43
SamYapleits configurable12:43
SamYaplebut please do update the docs12:43
SamYapleI personally dont think it sa good default, but thats what i was told to use a while back12:43
SamYapleprobably owuldnt hold weight today12:44
openstackgerritSam Yaple proposed stackforge/kolla: Add new build script  https://review.openstack.org/19661712:44
pbourkethe main issue is it's not accessible by non-root by default12:44
SamYapleyea we really need to discuss the non-root stuff12:45
SamYaplewe should be able to work the playbooks in such a way they dont REQUIRE AND ABSOLUTELY DEPEND ON ROOT12:45
SamYaplealot of ansible playbooks do12:46
SamYapleincluding some that deploy openstack12:46
SamYaplebut we should be able to work around that12:46
pbourkeCreating haproxy mysql user...12:46
pbourkeOne or more undefined variables: 'keystone_internal_address' is undefined12:46
pbourkebut it is12:46
SamYaplewhat command line are you running it with?12:46
pbourkeansible-playbook -i inventory/all-in-one -e @/etc/kolla/defaults.yml -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml site.yml -vv12:47
SamYaplecheck /etc/kolla/globals.yml to see if keystone_internal_address is set12:47
pbourke$ grep -ri internal_address /etc/kolla/12:47
pbourke/etc/kolla/globals.yml:kolla_internal_address: "192.168.33.10"12:47
pbourke/etc/kolla/globals.yml:database_address: "{{ keystone_internal_address }}"12:47
SamYapleyea thats a typo see12:47
SamYaplei have that fixed ina a bug upstream12:48
SamYaplei thikn12:48
pbourkehmm12:48
pbourkeI'm not seeing it12:48
pbourke(the typo)12:48
SamYaplekolla_internal_address12:49
SamYaplekeystone_internal_address12:49
pbourkeah12:49
SamYapleactually i dont have that fixed, please file a bug for that12:49
SamYapleit is however fixed in my globals file :D12:50
pbourkenow fixed in mine too12:50
SamYapleplz file bug12:51
pbourkewill do12:51
SamYaplemaybe even patch *wink* *wink*12:51
pbourkewant to get through to the end and file all together12:51
pbourkehere's the next one12:51
pbourkesame step (haproxy) - the python mysqldb module is required12:51
SamYapleyea thats a docs bug12:51
SamYapleyou need python-mysqldb package12:52
SamYapleinstalled on the dest host12:52
SamYaplereally it should be taken care of by a "base/common/prep" playbook12:52
pbourkenoted12:53
*** dims_ has joined #kolla12:55
pbourkeis there state I need to clear on localhost other than running containers when trying again from scratch?12:56
pbourkegetting, "unable to connect to database, check login_user and login_password are correct"12:56
SamYaplenope. destroy containers then youre good12:56
SamYaplecheck your configuration files to make sure they were populated wit hthe right ip address12:57
SamYaplemake sure youre listening on the right i paddress12:57
*** dims__ has joined #kolla12:57
SamYaplebrb12:57
pbourke$ docker logs facf88a4766512:58
pbourkeERROR: missing DB_CLUSTER_BIND_ADDRESS12:58
pbourkekk12:58
*** dims___ has joined #kolla13:00
*** dims_ has quit IRC13:00
*** dims_ has joined #kolla13:01
pbourkethat would seem to be coming from config-internal which shouldn't be executing?13:01
pbourkeah I probably need to rebuild the containers to stop them coming from dockerhub13:02
*** dims__ has quit IRC13:03
*** dims___ has quit IRC13:05
*** nihilifer has quit IRC13:05
SamYapleprobably :)13:06
*** dims__ has joined #kolla13:06
*** mstachow has quit IRC13:06
SamYapledriving home. lemme know how it goes and thank you for documenting things pbourke13:07
SamYapleit helps alot13:07
pbourkeno prob13:07
SamYaplei hate building al lthis in a vacuum of feedback13:07
*** dims_ has quit IRC13:09
*** dims__ has quit IRC13:11
pbourkethis is weird I've rebuilt the galera container to make sure I have the right one and it's still complaining about that missing var13:22
*** jtriley has joined #kolla13:24
SamYaplepbourke: you have to rebuild all of them13:35
SamYaplethe base container has common script changes13:35
pbourkeSamYaple: I think the issue was pull_policy13:36
pbourkeit was overwriting my latest13:36
SamYapleyea you need it to be "missing"13:37
SamYaplei thought i put that in the docs13:37
pbourkenope :)13:37
pbourkedont worry I'll add it13:37
pbourkeSamYaple: do you have a sample openrc I can use?13:41
pbourketrying the one as generated by genenv but with OS_PASSWORD set to "password", but not authenticating13:42
*** daneyon has joined #kolla13:44
SamYaplepbourke: yea here is the ones i genereate http://paste.openstack.org/show/348609/13:44
SamYaplepbourke: but remeber, keystone is b0rked without https://review.openstack.org/#/c/198533/13:46
pbourkehmm I think there's other problems13:47
pbourkeException occurred processing WSGI script '/var/www/cgi-13:47
pbourkebin/keystone/main'13:47
SamYaplelook at what that review modifies...13:47
*** daneyon has quit IRC13:48
*** prad has joined #kolla13:53
*** jmccarthy has quit IRC13:59
*** apmelton has joined #kolla14:04
pbourkeis that working for you?14:09
pbourkestill getting logs created in /var/log/keystone owned by root14:09
pbourkeSamYaple: ^14:10
dolpher1Anyone running into this issue? https://github.com/docker/docker/issues/10180, looks like a docker problem.14:32
dolpher1I consistently fail on this while building the kollaglue/centos-rdo-glance-registry14:32
pbourkedolpher1: not personally14:33
dolpher1The error message is "Rpmdb checksum is invalid: pkg checksums:"14:34
pbourkedolpher1: if there's no known workaround mentioned on that thread I doubt you'll get on here14:36
pbourke*one14:36
pbourkenever say never though :)14:36
SamYaplepbourke: the apache process is run as root I believe, so that is what would be creating the logs14:42
SamYaplewhats teh actual issue?14:42
SamYapledolpher1: what is your backend?14:43
pbourkethat's what I was thinking14:43
SamYaplebuilding images with overlayfs is b0rked in a number of ways14:43
dolpher1yeah, it's overlay fs14:43
SamYapledolpher1: yea thats fine for running, but dont build with that14:43
dolpher1I guess I need to switch back to aufs?14:43
pbourkeSamYaple: the issue is when I do a 'keystone token-get' I get 'Authorization Failed: Internal Server Error (HTTP 500)'. If I fix the perms on /var/log/keytone/* I then get 'Authorization Failed: The resource could not be found. (HTTP 404)'14:44
pbourkeSamYaple: not sure if the two are related14:44
SamYapledolpher1: any other backend *should* work14:44
SamYapleI actually use zfs myself so even the newer ones are good14:44
SamYaplepbourke: are you running with the review from above?14:44
SamYaple13:46 < SamYaple> pbourke: but remeber, keystone is b0rked without https://review.openstack.org/#/c/198533/14:44
pbourkeSamYaple: yes14:44
SamYaplewhat is your OS_AUTH_URL?14:45
pbourkedoesn't change anything as the logs aren't present at time of image build14:45
SamYaplewell it does changes things because keyston doesnt work without that patch at ALL14:45
pbourkeOS_AUTH_URL=http://192.168.33.10:3535714:46
pbourkeI meant in regards to the log perms14:46
dolpher1thanks SamYaple and pbourke, I will try other backend14:46
*** vinkman has joined #kolla14:47
*** jruano has joined #kolla14:47
SamYaplepbourke: after you rebuilt the image, did you drop the keystone database?14:48
pbourkeI think so but I'll try again just to be sure14:48
pbourkestopping container == dropping db right?14:48
SamYaplenope14:48
pbourkebah14:48
pbourkerm'ing container?14:49
SamYaplemysql -uroot -ppassword -h192.168.33.10 -Bne 'drop database keystone;'14:49
SamYaplebasically here is what is happening. The KOLLA_BOOTSTRAP code is _only_ being triggered when the databse registers that it had to create the initial keystoen database14:51
SamYapleso when you change the bootstrap code, to get it to rerun you have ot drop the database14:51
SamYaplethe bootstrap code should only ever be run a single time for an environment, so this issue will come up alot for testing, but rarely in production14:51
pbourkeright, but just to be sure, if I stop all containers and rerun it will start those up again rather than re-creating?14:52
*** jtriley has quit IRC14:52
SamYaplepbourke: im not positive. I want to say just start but that depends on a number of factors, including whether it had to pull a new image14:53
SamYaplerecreating wouldn't cause any data loss even in AIO though14:53
SamYaplehave to feed baby and go to bed14:53
pbourkekk14:53
SamYaplequeue the questions if you need14:53
SamYaplefile those bugs! thanks again pbourke14:53
SamYapleyoure the man14:53
pbourkeno worries14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182414:54
openstackLaunchpad bug 1471824 in kolla "/usr/share/kolla needs to be documented in docs/ansible-deployment.md" [Undecided,New]14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182514:54
openstackLaunchpad bug 1471825 in kolla "database_address in etc/kolla/globals.yml should be set to kolla_internal_address" [Undecided,New]14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182714:54
openstackLaunchpad bug 1471827 in kolla "python-mysqldb needs to be installed on the deployment host for ansible" [Undecided,New]14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182814:54
openstackLaunchpad bug 1471828 in kolla "docker_pull_policy should default to "missing" in etc/kolla/globals.yml" [Undecided,New]14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182914:54
openstackLaunchpad bug 1471829 in kolla "Log files in /var/log/keystone are owned by root and not writable by keystone services" [Undecided,New]14:54
pbourkehttps://bugs.launchpad.net/kolla/+bug/147182914:54
pbourkeonce I get keystone working I'll try multinode14:55
*** diga has joined #kolla14:58
*** diga has quit IRC14:58
*** diga has joined #kolla14:59
openstackgerritPaul Bourke proposed stackforge/kolla: Fix incorrect database_address variable in globals.yml  https://review.openstack.org/19875815:00
*** kevsi has quit IRC15:01
*** jmccarthy has joined #kolla15:02
openstackgerritPaul Bourke proposed stackforge/kolla: Set docker_pull_policy to "missing" in etc/globals.yml  https://review.openstack.org/19876015:07
openstackgerritPaul Bourke proposed stackforge/kolla: Set docker_pull_policy to "missing" in globals.yml  https://review.openstack.org/19876015:08
*** dolpher1 has quit IRC15:08
*** dims_ has joined #kolla15:09
openstackgerritMerged stackforge/kolla: Add validation script for min env vars documentation  https://review.openstack.org/19747315:11
openstackgerritMerged stackforge/kolla: Make tools/min-env.sh portable  https://review.openstack.org/19780615:12
openstackgerritMerged stackforge/kolla: Sort variables and remove duplicates for min env doc  https://review.openstack.org/19758615:12
*** jtriley has joined #kolla15:28
*** daneyon has joined #kolla15:33
*** daneyon has quit IRC15:33
*** absubram has joined #kolla16:17
*** jasonsb has quit IRC16:26
*** jasonsb has joined #kolla16:26
*** jasonsb has quit IRC16:31
*** jmccarthy has left #kolla16:33
*** daneyon has joined #kolla17:00
harmwhttps://bugs.launchpad.net/kolla/+bug/147189717:04
openstackLaunchpad bug 1471897 in kolla "move mariadb container to mariadb-galera single-node" [Undecided,New]17:04
harmwthere17:04
harmwhm, no coolswap17:04
harmwwhere's he at17:04
*** jasonsb has joined #kolla17:16
*** nihilifer has joined #kolla17:24
*** nihilifer has quit IRC17:28
*** jtriley has quit IRC17:31
*** absubram has quit IRC17:33
sdakemorning17:36
*** absubram has joined #kolla17:40
*** absubram has quit IRC17:40
*** bmace_away is now known as bmace17:41
*** absubram has joined #kolla17:53
*** jtriley has joined #kolla17:54
*** jtriley has quit IRC17:59
*** daneyon_ has joined #kolla18:15
*** gfidente has quit IRC18:17
*** jtriley has joined #kolla18:17
*** daneyon has quit IRC18:18
*** dims_ has quit IRC18:20
openstackgerritPaul Bourke proposed stackforge/kolla: Set docker_pull_policy to "missing" in globals.yml  https://review.openstack.org/19876018:30
openstackgerritPaul Bourke proposed stackforge/kolla: Fix incorrect database_address variable in globals.yml  https://review.openstack.org/19875818:31
*** jtriley has quit IRC18:37
*** user_ has joined #kolla18:48
*** user_ has quit IRC18:48
*** jtriley has joined #kolla18:50
*** athomas has quit IRC19:05
sdakenote folks, please see subject for registering for magnum midcycle meetup - 2 days food provided 28th19:05
sdakelunch provided 29th19:05
vinkmanmagnum?19:06
sdakewires crossed19:07
sdakesorry - kolla midcycle ;)19:07
*** nihilifer has joined #kolla19:07
sdakei also work on magnum19:07
sdakeand was just talking about the magnum midcycle in our containerschannel19:07
bmace:)19:07
sdakeso basically we are having a KOLLA midcycle 28th, 29th :)19:07
sdakeif your attending rmeotely pls register as well19:07
vinkmanO, you shipping lunch to us then?19:08
vinkman:D19:08
bmaceall the bits you can eat.19:08
vinkmanas long as it is 8 ;)19:08
sdakenot shipping lunch ;)19:14
sdakeif your remote your on your own for food ;)19:15
sdakealso remote participation isn't as ideal as being there locally19:15
sdakehard to get a word in edgewise :)19:15
*** nihilifer has quit IRC19:18
sdakeslower rhallisey ping19:22
sdakecan one of you do that chanserv command I sent you via email plz19:22
rhalliseyya sure19:22
sdakethanks ;)19:23
* rhallisey is still looking for the emial19:24
sdakethe last from me ;)19:25
*** ChanServ sets mode: +o rhallisey19:25
Slowersdake: there you go19:27
sdakethans slower19:27
*** ChanServ sets mode: +o sdake19:28
sdakesweet19:28
sdakethanks ;)19:28
*** sdake changes topic to "Please register for the the Kolla-Palooza midcycle July 28th, July 29th @ https://www.eventbrite.com/e/kolla-palooza-midcycle-event-tickets-17628669826 - Channel is Logged @ http://eavesdrop.openstack.org/irclogs/%23kolla/"19:29
*** ChanServ sets mode: -o sdake19:29
sdakeheads up, I am planning pto 19th-24th or somewhere in that range with less days19:31
sdakei will have high speed access to the internets but plan to go hiking most of the time so ymmv contacting me ;)19:32
*** dims_ has joined #kolla19:36
bmacejust give us your sat phone number ;)19:41
sdakesat phones, do they still make those? :)19:43
sdaketrue story, at my high school iridium directors/vps came and presented during our senior year design thing19:44
sdakeI asked the guy "what happens if one satallite fails"19:44
vinkmanOf course, how on earth are they going to make a new 24 if there is no sat phone Jack Bauer can use in some plane19:44
sdake"we have 1 backup"19:44
sdakewhat happens if the second satellite fails19:44
sdake"the network stops working"19:44
sdakethe look on the guys face, epic :)19:44
sdakewonder how many times he had answered that one before ;)19:44
sdakehe face palmed without face palming19:45
sdakeit was pretty funny :)19:45
bmacewell, when you need to push the hardware into space, it is hard to have a lot of backups :)19:46
vinkmanSo is that true for DirecTV as well?19:47
sdakepossible but unclear19:48
sdakeiridium was doomed from the start ;(19:48
bmacelooks like direcTV has 14 active satellites.19:49
sdakei think iridium is still around - government pays the bills now19:51
sdakefor war and stuff19:51
sdakesamyaple fine sir I need that list of modules to install19:54
sdakesamyaple when you wake up , please file a blueprint with relevant info - thanks :)19:54
sdakeor if you dont want to file blueprint communicate the list via mailing list and I'll file the blueprint19:55
Slowersdake: fyi we're switching to full configs to feed the kolla containers20:01
SlowerI haven't looked at what you guys have yet though20:02
sdakeinteresting, so no need for the internal config method we have now?20:02
sdakewell we can remove it once you are sure that is how you want to proceed, sound good? :)20:02
Slowersdake: yeah20:03
Slowersdake: yeah we have some refactoring to do to support that so don't pull it yet20:03
Slowersdake: but it sounds like the best strategy for us20:03
sdakenah it wont be until after l2 is out20:03
sdakeits actually a tactic, but ok :)20:03
Slowertactic you say.. hmm.. :)20:06
SlowerI prefer eeeeevil plan20:06
Slowerits all about the eeeevil20:06
sdakeslower can you sort out if your coming to midcycle in person or remotely and register appropriately :)20:07
sdakerhallisey same as well :)20:07
sdakewould be nice to get our cores there in person20:08
rhalliseysdake, I haven't heard yet20:08
rhalliseysoon20:08
sdake13th is the 2 week travel window cutoff20:08
sdakeok well if your not there in person, you will be remote at minimum?20:08
Slowersdake: actually I might be coming20:09
Slowersdake: angus is looking into funding20:10
sdakecool well when you find out let me know20:10
sdakemy guess is funding is shot because of the goal to send as many folks to toyo as possible20:10
Slowercould be, I dunno20:10
sdakeI dunno either, but if you can't make in person will you be able to attend remotely?20:10
Sloweronly if I can have beer in my office20:11
sdakeyou can have whatever youw ant in your office afaik :)20:11
SlowerI'll be in my lawn chair drinking beer while you guys work just so you know.. ;-)20:11
vinkmanyou just can not drink it :D20:11
SlowerI may be a bad influence on your meetings :)20:12
sdakethe problem with remote is everyone is sitting around chatting20:12
sdakeand its hard to get a word in edgewise ;)20:12
Slowerya my experience as well20:12
sdakein the future may only do remote midcycles20:13
sdakecheaper and easier on folks20:13
sdakevia google hangouts or webex or the like20:13
bmacewith something like hangouts it would be easy enough to even do smaller  l1 / l2 / l3 type meetings rather than doing some larger mid-cycle20:16
*** dims_ has quit IRC20:17
Slowersdake: a lot fun though imo :)20:19
sdakebmace interestting idea for m cycle20:21
sdakealthough I hate to take up 3% of our output on a sprint meeting every month :(20:22
sdakepart of the midcycle is so folks can meet others20:22
sdakeprior to summit ;)20:22
bmaceif folks have web cams i think you get that, to at least a better degree than irc, on hangouts.  obviously you don't get the after hours / social side.20:25
*** nihilifer has joined #kolla20:32
sdakebmace agree20:38
sdakewell we can discuss during our m sessions if we get any in tokyo :)20:38
sdakejasonsb where you attending the dinner on the 28th?20:40
sdakejasonsb if so, you have to rsvp a ticket for that sepraately20:41
jasonsboh20:43
jasonsbthanks, i missed that20:43
*** bradjones is now known as bradjones|away20:44
jasonsbsdake: thank you for watching out for me :)20:45
sdakedoes anyone know mike nolte (or in channel?)  Also same dinner rsvp if you want to join us for dinner :)20:46
jasonsbmike is one of us20:46
jasonsbme (jason) + umesh + mike20:46
jasonsbi just mentioned rsvp for dinner to him20:46
sdakejasonsb cool could you let him know [pls20:46
sdakejasonsb sounds good20:51
*** dims_ has joined #kolla21:09
*** dims_ has quit IRC21:14
*** dims has joined #kolla21:16
*** rhallisey has quit IRC21:27
*** jruano has quit IRC21:35
*** mfalatic has joined #kolla21:44
*** mfalatic has quit IRC21:56
*** absubram has quit IRC21:59
*** nihilifer has quit IRC21:59
*** jtriley has quit IRC22:08
*** daneyon has joined #kolla22:09
*** daneyon_ has quit IRC22:12
*** prad has quit IRC22:15
sdakedaneyon tooth all settled for the most part - it needs more surgey - a cap or filling or something - not sure on details22:17
sdakewaiting on dentist to call me22:17
sdakeits the step they do after the root canal22:17
sdakebut atlast the pain is gone22:17
bmacehey folks, fyi, looks like our pep8 right now is not pep8 at all :)22:26
bmaceso i would like to change the name of the current tests to something more appropriate, add in a non-voting real pep8 gate, then fix the python issues since what little python we have is not pep8 compliant, then make it voting22:27
*** jruano has joined #kolla22:27
bmaceif people have thoughts one way or another i would love to hear them :)22:27
sdakesounds good, so we don't do flake8 in the current gate then?22:30
sdakeas we are introducing python code we probably should have flake8 in the pep8 gate ;)22:30
bmaceyeah, i was surprised to find it missing :).  it looks like the pep8 gate just verifies the contents of existing .json / .yaml / dockerfiles in the codebase.22:31
bmacenot very peppy22:31
sdakelol22:34
*** Slower has quit IRC22:34
bmacei am also open for good names for the new test that will encompass the json / yaml / dockerfile content validation.22:40
sdakestatic_validate?22:41
bmaceconfig_validate?  they are all pretty much config / property files of some kind or another22:41
sdakesure22:42
sdakei like static because they are static files22:42
sdakemeant not to change i think22:42
sdakebut go with whatever, I'm sure the bikeshedding will occur on your review ;)22:42
bmaceand we actually don't even have any json files right now, but we do have a few yaml, mostly under devenv / demos22:44
bmaceyup, easy enough to change.22:44
bmacesince this also interacts with the project-config tree, i am just going to introduce the new test, that does the same thing as the old one into #kolla, once that is in i can make the change in project-config to actually activate it, then i can make the old pep8 non-voting in project-config, do all the fix stuff in #kolla, then turn it into voting again22:47
sdakethat sounds good that keeps the gate from getting jammed on a bad commit :)22:48
bmaceexactly :)22:48
*** dims has quit IRC22:54
*** jruano has quit IRC22:57
*** daneyon has quit IRC23:05
openstackgerritBorneMace proposed stackforge/kolla: Added additional tox test for json / yaml / link / dockerfile validation.  This currently mimics the existing pep8 test which will be updated to run flake8  https://review.openstack.org/19892223:26
sdakebmace your commit message is busted bro23:37
bmaceoh?  blueprint link seems to work fine.  whats wrong w/ it?23:39
bmaceah, i see your comment23:39
bmaceso 70 chars max?  doesn't let you be terribly descriptive :)23:40
sdakemultiple lines23:40
sdakebut 70 chars max per line23:40
sdakeso23:40
bmacekk23:40
sdaketitle (50 chars)23:40
sdakeenter23:40
sdakelong description wrapped at 70 chars23:40
openstackgerritBorneMace proposed stackforge/kolla: Fixing tox tests - split out validation tests  https://review.openstack.org/19892223:45
sdakebmace enjoy23:46
bmaceif any other core reviewer is around to take a look, i would love to get in the project-config stuff too, which can't be done until this is in.23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!