Tuesday, 2017-01-03

Hallfoncethen I tried another kolla install with openvswitch: ovs-vsctl show would be br-ex with port enp0s31f600:00
Hallfonceno success either00:00
SamYaplei think youre a bit confused as to how this hsould be setup. with a single port the setup is the same, whether linuxbridge or openvsiwtch00:01
HallfonceYou're right, I got totally confused, I've tried so many setup from so many references...00:01
SamYaplelet me see if i can find some link from when i wrote this00:01
SamYaplebut again, i dont recommend doing this for the trouble youre having00:01
HallfonceYou mean setup an AIO kolla with a single port?00:02
SamYapleyea. its ok for testing, but its still a pain00:02
HallfonceSure, I understand00:03
HallfonceI've found this out the hard way...00:03
*** chas__ has quit IRC00:08
*** chas has joined #openstack-kolla00:08
SamYapleHallfonce: score! one of my pastebins from over a year ago00:12
SamYaplehttp://paste.openstack.org/show/478029/00:12
HallfonceEpic !00:12
SamYapleHallfonce: ip l a fake-br10 type veth peer name fake-ovs; brctl addif br10 fake-br10; ip l s fake-br10 up; ip l s fake-ovs up00:12
SamYaplewhere br10 can be replaced with enp0s31f600:13
SamYaplefake-ovs is neutron_external_interface00:13
*** chas has quit IRC00:13
SamYapleso anyway. the deal is it doesnt matter whether you are using ovs or linuxbridge, though i would recommend ovs00:15
SamYaplethe bridge setup on the outside is the same00:15
SamYaplealso, i dont think its talked about, but you can _only_ use flat networks00:15
SamYapleonce you launch a vm, assign a floating ip and add the appropriate secgroup rules and youll be able to hit the floating ip00:16
HallfonceOk, i went a bit fast for me00:18
HallfonceI assume you bridged enp0s31f6 into br10 ?00:19
Hallfonceso i would add brctl addif br10 enp0s31f600:20
HallfonceIf so, I've tried this setup at some point, and I understand well, I was missing the secgroup rules?00:22
*** rhallisey has quit IRC00:25
*** JRobinson__ has joined #openstack-kolla00:26
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: kolla-common value get functions  https://review.openstack.org/41606900:27
*** JRobinson__ has quit IRC00:29
SamYapleHallfonce: let me put it to you this way, i doubt you messed up the veth pair part, i never said it was complicated, just confusing00:30
Hallfoncesure00:30
HallfonceI got confused with the neutron_plugin_agent00:31
Hallfonceas you said it doesnt matter00:31
Hallfonceas long as it's the same on the outside00:31
SamYaplei would stick with openvswitch, only because thats the piece that i wrote and is default because its most tested (its gated against)00:31
Hallfonceok I will stick to it then00:33
Hallfonceabout the secgroup rules, I sticked to  what was in my /usr/share/kolla/init-runonce00:33
HallfonceI only edited 1 line in there about the public subnet00:34
Hallfonceto match my external network subnet00:34
Hallfonceanyway, I'm going to give this another try with your advices00:35
SamYapleuse a cirros image and setup a ping from the console. then find out where the icmp is dropping and youll figure out where your problem is00:36
Hallfonceare there default user/password for the cirros image ?00:38
Hallfonceotherwise I need to access port 22 to use a cert00:38
SamYapleHallfonce: there are indeed default user and password00:38
Hallfoncecool, I'll google it00:40
SamYapleHallfonce: it tells you when you try to log in, but googling will work too00:41
HallfonceOk good to know!00:41
HallfonceThanks a lot for time, it's been very helpful!00:41
Hallfoncehope I won't have to bother you again about this!00:42
SamYapleits no bother, just hard to predict one of the 20 things that could be wrong when doing this. tcpdump is going to be your friend00:43
Hallfonceindeed00:45
HallfonceI'll get there step by step I guess00:45
kfox1111there we go.00:47
*** david-lyle has joined #openstack-kolla00:51
*** jmccarthy has quit IRC00:51
*** jmccarthy has joined #openstack-kolla00:52
*** tovin07 has joined #openstack-kolla00:54
*** david-lyle has quit IRC00:55
*** tovin07 has quit IRC00:57
*** salv-orlando has joined #openstack-kolla01:00
*** salv-orlando has quit IRC01:02
*** salv-orlando has joined #openstack-kolla01:02
*** zhurong has joined #openstack-kolla01:05
*** salv-orlando has quit IRC01:07
*** Hallfonce has quit IRC01:11
*** tovin07 has joined #openstack-kolla01:16
*** yingjun has joined #openstack-kolla01:29
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move trove precheck into its own role  https://review.openstack.org/41603201:36
*** salv-orlando has joined #openstack-kolla02:03
openstackgerritjimmygc proposed openstack/kolla: Add sanity check to install_pip in macros.j2  https://review.openstack.org/41484802:03
*** clayton has quit IRC02:06
*** mdnadeem has joined #openstack-kolla02:07
*** salv-orlando has quit IRC02:08
*** n0isyn0ise has quit IRC02:11
*** sdake has joined #openstack-kolla02:12
*** n0isyn0ise has joined #openstack-kolla02:13
*** sdake has quit IRC02:15
*** sdake has joined #openstack-kolla02:16
sdakeevening folks02:16
openstackgerritWei Cao proposed openstack/kolla-ansible: Add freezer ansible role  https://review.openstack.org/40222402:23
*** zhurong has quit IRC02:25
sdakeoff to more tech support BBL02:25
*** clayton has joined #openstack-kolla02:25
Jeffrey4l__morning02:26
sdakesup Jeffrey4l__02:26
Jeffrey4l__hey sdake, happy new year ;)02:26
sdakesay - might try running if gate==rax_iad then set libvirt type to qemu02:26
sdakethanks you too :)02:26
sdakealthough yours is down  the road a bit :)02:26
sdakeok gotta jet reworking my dad's computer02:27
Jeffrey4l__sdake, libvirty_type=qemu all the time.02:27
Jeffrey4l__in all gate.02:27
Jeffrey4l__so it should not related.02:28
*** Jeffrey4l__ is now known as Jeffrey4l02:28
*** sdake has quit IRC02:31
dupengfeisrwilkers: hello02:34
openstackgerritMerged openstack/kolla-ansible: Move horizon precheck into its own role  https://review.openstack.org/41606802:40
openstackgerritdupengfei proposed openstack/kolla: Not to pull the base images when building.  https://review.openstack.org/40894002:41
Pavoanyone know when the new murano and magma changes are goona get pushed to pip package of kolla?02:42
openstackgerritdupengfei proposed openstack/kolla-kubernetes: Helm charts for keepalived pod  https://review.openstack.org/41398502:43
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Add enable_horizon_* parameter for dashboard  https://review.openstack.org/41348402:50
openstackgerritdupengfei proposed openstack/kolla-kubernetes: Helm charts for heat-create-db and heat-manage-db  https://review.openstack.org/41449602:50
*** david-lyle has joined #openstack-kolla02:52
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move ironic precheck into its own role  https://review.openstack.org/41548102:56
*** david-lyle has quit IRC02:57
openstackgerritdupengfei proposed openstack/kolla-kubernetes: Helm charts for keepalived pod  https://review.openstack.org/41398503:00
*** Pavo has quit IRC03:03
*** salv-orlando has joined #openstack-kolla03:04
*** salv-orlando has quit IRC03:08
*** chas has joined #openstack-kolla03:10
*** chas has quit IRC03:15
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move searchlight precheck into its own role  https://review.openstack.org/41513503:18
*** dave-mccowan has joined #openstack-kolla03:23
*** goldyfruit has joined #openstack-kolla03:29
*** clayton has quit IRC03:34
*** tovin07 has quit IRC03:34
*** jmccarthy has quit IRC03:34
*** yingjun has quit IRC03:34
*** mdnadeem has quit IRC03:34
*** yingjun has joined #openstack-kolla03:35
*** clayton has joined #openstack-kolla03:35
*** tovin07 has joined #openstack-kolla03:35
*** jmccarthy has joined #openstack-kolla03:35
*** mdnadeem has joined #openstack-kolla03:35
*** mdnadeem has quit IRC03:35
*** prameswar has joined #openstack-kolla03:50
*** sdake has joined #openstack-kolla03:51
*** g3ek has quit IRC03:54
*** sdake has quit IRC03:56
*** g3ek has joined #openstack-kolla04:02
*** salv-orlando has joined #openstack-kolla04:05
*** yingjun has quit IRC04:06
*** Pavo has joined #openstack-kolla04:09
*** salv-orlando has quit IRC04:09
*** bmace has quit IRC04:14
*** bmace has joined #openstack-kolla04:14
*** goldyfruit has quit IRC04:15
*** sacharya has quit IRC04:15
*** sacharya has joined #openstack-kolla04:16
*** sacharya has quit IRC04:21
*** dave-mccowan has quit IRC04:22
*** masber has joined #openstack-kolla04:27
openstackgerritjimmygc proposed openstack/kolla: Add sanity check to install_pip in macros.j2  https://review.openstack.org/41484804:27
*** mdnadeem has joined #openstack-kolla04:27
masberhi04:38
masberdoes kolla provide high availability?04:38
masberI assume that with the infrastructure in place (e.g. kubernetes, docker swarm, mesos + marathon, etc) someone could just deploy the controller openstack components on containers and monitor them so options like galera or pacemaker wouldn't be needed. Am I crazy saying that if is it true?04:42
*** Pavo has quit IRC04:51
*** prameswar has quit IRC04:54
*** sdake has joined #openstack-kolla04:56
sdakeJeffrey4l i looked over the code and don't see where qemu is being used04:57
sdakemind offering a pointer for my sanity? :)04:57
Jeffrey4lsdake, kolla project re-user kolla-ansible's deploy_aio.sh script.04:58
Jeffrey4lhttps://github.com/openstack/kolla-ansible/blob/master/tools/deploy_aio.sh#L8604:58
sdakekolla repo04:58
sdake?04:58
sdakethat looks good04:58
Jeffrey4lkolla repo reuse kolla-ansible's deploy_aio.sh script.04:59
Jeffrey4lyep.04:59
sdakei forgot about the refactor04:59
sdakeyou did on that file04:59
sdakeeven though I'm pretty sure i reviewed it :)04:59
Jeffrey4l;)04:59
sdake12 hrs of tech support for parents04:59
sdake= painful04:59
sdakethe yearly reinstall of windows05:00
sdakeactually pristine backup restore :)05:00
Jeffrey4li will try to enable libvirt-debuginfo to catch more info.05:00
Jeffrey4lhaha, buy a mac ;)05:00
sdakei have a mac05:00
Jeffrey4lfor your parent05:00
sdake5 in fact :)05:00
sdakethey dont like laptops05:00
Jeffrey4lwow05:00
sdakethey want a computer they can sit at05:00
Jeffrey4lok..05:01
sdakei have my work 15", my personal 15" my personal 13" and a personal 12"05:01
sdakethe 13" is my wifes, and my wife also has a 12"05:01
*** Pavo has joined #openstack-kolla05:01
Jeffrey4lcool.05:01
Pavosdake you have any idea when the lastest murano and magma patches are gonna get pushed to kolla pip package?05:04
sdakepavo i've been on pto for 2 weeks bro05:05
*** salv-orlando has joined #openstack-kolla05:05
Pavojust wondering an expected time frame05:06
PavoI would just do a git clone of master but then I have to re build all the images05:06
Pavoguess I can do that to test upgrade functionality05:07
*** salv-orlando has quit IRC05:10
*** chas has joined #openstack-kolla05:11
*** prameswar has joined #openstack-kolla05:12
sdakePavo inc0 is probably the person to talk to05:13
sdakepavo i really have no visibility into kolla-ansible at a detailed level atm05:13
sdakepavo focused on kolla-kubernetes atm.05:13
Pavook05:14
*** mdnadeem has quit IRC05:15
sdakepavo and kolla repo of course05:15
*** chas has quit IRC05:16
sdakepavo have hard time reviewing 3 deliverables at high output05:16
Pavo?05:16
sdakekolla = deliverable, kolla-ansible = deliverable, kolla-kubernetes = deliverable05:16
Pavoah05:16
*** mdnadeem has joined #openstack-kolla05:16
sdakewe have a slew of people reviewing kolla-ansible - suspect one of them would know05:17
sdakepavo the real issue is there is no backports team that i am aware of05:17
sdakeas in no stable maintenance team05:17
sdakeinc0 can probably fix that up05:17
sdakesomething to be added to the agenda i think05:17
sdakeportdirect you about05:17
*** sacharya has joined #openstack-kolla05:17
Pavowished I knew what that means lol but ok05:17
sdakepavo a stable maintenance team does backports on a consistent basis and releases z streams consistently05:18
sdakepavo fwiw, I believe Jeffrey4l did release a new z stream05:18
Pavoah ok05:18
sdakePavo although whatyou mention sound like features05:18
sdakepavo you might try pbourke05:18
Pavohow would I know when the features would be added to the pip package install of kolla?05:19
*** sacharya has quit IRC05:22
*** skramaja has joined #openstack-kolla05:22
Jeffrey4lsdake, i didn't release next z stream still now. we need fix the selinux issue, in mitaka branch Or annoucement that mitaka do not support selinux very well.05:23
Pavoanything I can help with that Jeffrey4l05:23
sdakeJeffrey4l bummer - just a heads up - i had indicated iwould be off pto on jan 4th05:24
sdakeJeffrey4l i have to travel 3rd-6th05:24
sdakeand wont make wednesday meeting05:24
Jeffrey4lroger.05:24
sdakei'll also mostly be unavailable on irc as well05:24
Jeffrey4li will talked with inc0 about this.05:25
Jeffrey4lPavo, mitaka stable branch gate failed unless disable selinux. but it is OK in other branch and locally test.05:25
Pavohmmm05:26
Jeffrey4lI have debugged this, but got nothing. really no idea what to do next.05:26
Pavook I guess I am little ocnfused because newton 3.0.1 does the same thing05:26
Jeffrey4l"does the same thing" means?05:27
Pavoinstances do not run, horizon doesn't allow uploading of images in dahboard with selinx enabled05:27
Jeffrey4lPavo, yep. have u  ever gOt the audit logs?05:28
Pavobut selinux disabled works perfectly fine05:28
Pavoyeah I posted them in the bug report05:28
Pavobut didn't see anything interesting05:28
* Jeffrey4l is checking.05:29
Pavohttps://www.dropbox.com/s/bbzj85er47to68a/audit.log?dl=005:30
Jeffrey4lroger.05:30
PavoJeffrey4l if there is any testing you need let me know05:31
PavoI am not good at coding but am good at testing things out05:32
Jeffrey4lPavo, are u pretty sure that the audit.log has the logs when issue occur?05:32
Jeffrey4li do not found any interesting log, too. ;(05:33
Pavoyeah I turned selinux back on, rebooted tried to upload an image in horizon and it failed as usual05:33
Pavouploaded the log to dropbox05:33
*** yingjun has joined #openstack-kolla05:33
Pavoand turned selinux back off05:33
Pavothen rebooted05:33
Pavonow everything is working perfect05:33
Jeffrey4lPavo, hrm. could u try again. and post all /var/lib/docker/volumes/kolla_logs file out?05:34
Jeffrey4lwith the audit.log.05:34
Pavocan tomorrow, already in bed now05:34
Jeffrey4lif possible ;)05:34
Pavobut will do05:34
Jeffrey4lcool. thanks.05:34
Pavono problem05:35
*** sdake has quit IRC05:42
*** satyar has joined #openstack-kolla05:44
*** yingjun has quit IRC05:51
openstackgerritMerged openstack/kolla-ansible: Move sahara precheck into its own role  https://review.openstack.org/41605105:53
*** Pavo has quit IRC06:01
*** salv-orlando has joined #openstack-kolla06:06
*** Pavo has joined #openstack-kolla06:06
openstackgerritMerged openstack/kolla-ansible: Move memcached precheck into its own role  https://review.openstack.org/41605906:09
*** salv-orlando has quit IRC06:11
*** duonghq has joined #openstack-kolla06:14
duonghqhappy new year guys06:14
openstackgerritcaoyuan proposed openstack/kolla-ansible: Update the horizon port into all.yml  https://review.openstack.org/41549506:19
openstackgerritcaoyuan proposed openstack/kolla-ansible: Update the horizon port into all.yml  https://review.openstack.org/41549506:21
openstackgerritcaoyuan proposed openstack/kolla-ansible: Update the horizon port into all.yml  https://review.openstack.org/41549506:24
*** caowei has joined #openstack-kolla06:33
*** Pavo has quit IRC06:41
*** msimonin has joined #openstack-kolla06:41
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move nova precheck into its own role  https://review.openstack.org/41556406:43
*** msimonin1 has joined #openstack-kolla06:44
*** msimonin has quit IRC06:44
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move nova precheck into its own role  https://review.openstack.org/41556406:44
*** sdake has joined #openstack-kolla06:54
*** salv-orlando has joined #openstack-kolla06:55
*** sdake has quit IRC07:06
*** msimonin1 has quit IRC07:10
*** msimonin has joined #openstack-kolla07:10
*** jianyi has joined #openstack-kolla07:10
*** chas has joined #openstack-kolla07:12
*** salv-orl_ has joined #openstack-kolla07:13
*** msimonin has quit IRC07:14
*** salv-orlando has quit IRC07:16
*** chas has quit IRC07:17
*** sacharya has joined #openstack-kolla07:18
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move neutron precheck into its own role  https://review.openstack.org/41555207:19
*** matrohon has quit IRC07:19
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move cinder precheck into its own role  https://review.openstack.org/41549207:22
*** sacharya has quit IRC07:22
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move cinder precheck into its own role  https://review.openstack.org/41549207:24
*** chas has joined #openstack-kolla07:25
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move gnocchi precheck into its own role  https://review.openstack.org/41532607:30
*** salv-orl_ has quit IRC07:37
*** pcaruana has joined #openstack-kolla07:41
*** hawi has quit IRC07:47
*** hawi has joined #openstack-kolla07:47
*** shardy has joined #openstack-kolla07:58
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move murano precheck into its own role  https://review.openstack.org/41511308:05
*** saneax-_-|AFK is now known as saneax08:12
*** msimonin has joined #openstack-kolla08:26
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move senlin precheck into its own role  https://review.openstack.org/41548608:37
*** salv-orlando has joined #openstack-kolla08:38
*** salv-orlando has quit IRC08:42
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move trove precheck into its own role  https://review.openstack.org/41603208:46
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move mistral precheck into its own role  https://review.openstack.org/41551508:51
openstackgerritjianyi proposed openstack/kolla-ansible: Add zun ansible role  https://review.openstack.org/41617908:54
*** david-lyle has joined #openstack-kolla08:57
*** david-lyle has quit IRC09:01
*** prameswar has quit IRC09:04
*** absubram has joined #openstack-kolla09:05
*** zhugaoxiao has joined #openstack-kolla09:17
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move searchlight precheck into its own role  https://review.openstack.org/41513509:18
*** sacharya has joined #openstack-kolla09:19
*** sacharya has quit IRC09:23
*** msimonin has quit IRC09:24
*** mnasiadka has joined #openstack-kolla09:28
*** msimonin has joined #openstack-kolla09:29
*** papacz has joined #openstack-kolla09:30
*** salv-orlando has joined #openstack-kolla09:35
*** Serlex has joined #openstack-kolla09:38
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move magnum precheck into its own role  https://review.openstack.org/41551309:39
openstackgerritcaoyuan proposed openstack/kolla-ansible: Update the place of Watcher HAProxy by alphabet  https://review.openstack.org/41619409:45
*** pbourke has joined #openstack-kolla09:53
*** salv-orlando has quit IRC09:53
*** salv-orl_ has joined #openstack-kolla09:54
openstackgerritjianyi proposed openstack/kolla-ansible: Add zun ansible role  https://review.openstack.org/41617909:57
*** david-lyle has joined #openstack-kolla09:58
*** tovin07 has quit IRC09:59
*** david-lyle has quit IRC10:02
satyarHi Jeffrey4l10:06
satyarHi SamYaple10:06
*** matrohon has joined #openstack-kolla10:07
*** prameswar has joined #openstack-kolla10:08
*** athomas has joined #openstack-kolla10:09
*** wagiel has joined #openstack-kolla10:11
*** gfidente has joined #openstack-kolla10:12
*** gfidente has joined #openstack-kolla10:12
openstackgerritjianyi proposed openstack/kolla-ansible: Add zun ansible role  https://review.openstack.org/41617910:14
*** rmart04 has joined #openstack-kolla10:17
*** hieulq has quit IRC10:17
*** hieulq has joined #openstack-kolla10:18
Jeffrey4lsup satyar10:26
*** JoseMello has joined #openstack-kolla10:28
*** klindgren_ has joined #openstack-kolla10:29
*** gfidente^2nd has joined #openstack-kolla10:30
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes: Updated from global requirements  https://review.openstack.org/41619910:31
*** matrohon has quit IRC10:31
*** klindgren has quit IRC10:32
*** gfidente has quit IRC10:32
*** giulivo^3rd has joined #openstack-kolla10:33
openstackgerritjianyi proposed openstack/kolla-ansible: Add zun ansible role  https://review.openstack.org/41617910:33
*** gfidente^2nd has quit IRC10:35
*** jianyi has quit IRC10:36
satyarjeffrey4l10:38
*** duonghq has quit IRC10:38
satyarthere is an issue with boot from volume10:38
satyarTypeError: Argument must be bytes or unicode, got 'NoneType'10:38
Jeffrey4lsatyar, any more log? where did u get such a error?10:39
satyarhttp://paste.openstack.org/show/593772/10:40
Jeffrey4lsatyar, are u using ceph backedn?10:41
satyaryes10:41
*** caowei has quit IRC10:41
Jeffrey4lsatyar, post the nova.conf file.10:42
satyarfor nova and cinder both10:42
Jeffrey4lcheck whether [libvirt]rbd_secret_uuid has value. if yes, i suppose it should be a bug in nova/cinder ;(10:43
*** giulivo^3rd has quit IRC10:43
satyarrbd_secret_uuid = bbc5b4d5-6fca-407d-807d-06a4f4a7bccb10:45
satyarit have10:45
*** giulivo^3rd has joined #openstack-kolla10:46
satyarjeffrey4l is the secret uuid required in cinder also?10:47
Jeffrey4lsatyar, no. cinder do not need that secret.10:49
Jeffrey4lsatyar, does boot from image work?10:49
satyaryes10:50
satyaronly boot from volume is failing10:50
Jeffrey4lwhich branch are u using?10:50
satyarthe code from 05dec10:55
Jeffrey4lok. i will test this in my env, which is the latest code.10:56
Jeffrey4li guess it should be a bug in nova/cinder.10:57
*** msimonin has quit IRC11:01
*** msimonin has joined #openstack-kolla11:03
*** msimonin has quit IRC11:03
*** msimonin has joined #openstack-kolla11:07
satyarhmm11:07
satyarplease check and let me know how it goes at your end11:07
satyarjeffrey4l11:08
*** mliima has joined #openstack-kolla11:16
*** athomas has quit IRC11:18
*** sacharya has joined #openstack-kolla11:20
mliimamorning guys, i'm back11:21
mliima:)11:21
*** sacharya has quit IRC11:24
*** athomas has joined #openstack-kolla11:26
Jeffrey4lnp11:33
*** wagiel has quit IRC11:35
*** david-lyle has joined #openstack-kolla11:59
*** david-lyle has quit IRC12:04
*** rhallisey has joined #openstack-kolla12:07
*** NachoDuck has quit IRC12:19
*** serverascode has quit IRC12:19
*** kong_ has quit IRC12:19
*** portdirect has quit IRC12:19
*** alanmeadows has quit IRC12:19
*** portdirect has joined #openstack-kolla12:21
*** kong_ has joined #openstack-kolla12:21
*** NachoDuck has joined #openstack-kolla12:23
*** alanmeadows has joined #openstack-kolla12:23
*** serverascode has joined #openstack-kolla12:24
*** giulivo^3rd is now known as gfidente12:31
*** gfidente has quit IRC12:31
*** gfidente has joined #openstack-kolla12:31
*** salv-orl_ has quit IRC12:32
openstackgerritMerged openstack/kolla: Disable networking during bootstrapping  https://review.openstack.org/41590812:34
*** matrohon has joined #openstack-kolla12:39
*** gfidente has quit IRC12:43
*** gfidente has joined #openstack-kolla12:45
*** msimonin has quit IRC12:48
*** saneax is now known as saneax-_-|AFK12:49
*** msimonin has joined #openstack-kolla12:50
openstackgerritMerged openstack/kolla: Remove unneeded oslo-rootwrap in swift-base  https://review.openstack.org/41339212:50
*** msimonin has quit IRC12:50
openstackgerritMerged openstack/kolla: Clean up kolla-ansible related files from Kolla  https://review.openstack.org/40642912:50
openstackgerritMerged openstack/kolla: Quickstart guide: disable libvirt apparmor  https://review.openstack.org/41550112:51
*** srwilkers has joined #openstack-kolla12:51
srwilkersmorning12:51
srwilkershappy new year and such12:51
*** msimonin has joined #openstack-kolla12:52
*** sdake has joined #openstack-kolla13:04
sbezverksrwilkers: morning, same to you :)13:05
sbezverksrwilkers: when you have time please check https://review.openstack.org/#/c/414348/13:06
srwilkerssbezverk, taking a gander now :)13:12
*** mdnadeem has quit IRC13:18
*** sacharya has joined #openstack-kolla13:20
*** sacharya has quit IRC13:25
*** salv-orlando has joined #openstack-kolla13:28
*** dave-mccowan has joined #openstack-kolla13:30
*** lamt has joined #openstack-kolla13:31
portdirecthappy new year kollas :D13:31
srwilkershey mate :)13:33
srwilkershappy new year13:33
*** jheroux has joined #openstack-kolla13:34
sbezverkportdirect: morning!!13:34
portdirecthey man :) just had a look through the lm work you've been doing looks good  - but we should really be mounting cgroups read only if we need to read them in a pod i think?13:36
*** caowei has joined #openstack-kolla13:36
sbezverkportdirect: unfortunately iscsiadm needs it for not just read only13:36
sbezverkportdirect: it stores there targets information13:37
*** salv-orlando has quit IRC13:37
sbezverkbehavior by design (not mine ;) )13:37
portdirectah ok - need to check whats going on with my stuff - but do we need to use the hosts cgroups in that case - I'm using the one in the docker container (I think, unless its mounting the hosts...)13:39
*** apuimedo is now known as apuimedo|away13:39
portdirecthttps://github.com/portdirect/harbor/blob/latest/kubernetes/templates/cinder/controllers.yaml#L603-L61113:39
*** salv-orlando has joined #openstack-kolla13:40
*** sdake has quit IRC13:41
sbezverkportdirect: targetcli tool uses that13:43
sbezverkI think13:43
sbezverkportdirect: and we do mount storage/lvm volumes on the host13:44
portdirectah - whoops see what your doing13:44
*** gfidente has quit IRC13:45
*** yee37913 has joined #openstack-kolla13:46
*** yee379 has quit IRC13:46
*** l4yerffeJ__ has joined #openstack-kolla13:46
*** PyroMani_ has quit IRC13:47
portdirectwas thinking about the way my containers are built - where I use a fat container for this13:47
portdirecthttps://github.com/portdirect/harbor/blob/latest/docker/openstack/openstack-cinder/openstack-cinder-volume-iscsi/assets/etc/supervisord.conf13:47
*** aimeeu has quit IRC13:47
*** gfidente has joined #openstack-kolla13:48
*** skramaja has quit IRC13:48
*** l4yerffeJ_ has quit IRC13:48
*** PyroMani has joined #openstack-kolla13:48
*** skramaja has joined #openstack-kolla13:48
*** Jeffrey4l has quit IRC13:49
*** l4yerffeJ__ has quit IRC13:54
*** l4yerffeJ__ has joined #openstack-kolla13:54
*** aimeeu has joined #openstack-kolla13:54
*** Jeffrey4l has joined #openstack-kolla13:59
*** zhugaoxiao has quit IRC14:00
*** david-lyle has joined #openstack-kolla14:01
*** zhugaoxiao has joined #openstack-kolla14:01
*** david-lyle has quit IRC14:05
openstackgerritMerged openstack/kolla-kubernetes: Adding iscsi support to the gate.  https://review.openstack.org/41471314:08
openstackgerritMerged openstack/kolla-kubernetes: Adds cinder volume lvm microservice  https://review.openstack.org/41434814:08
*** eaguilar has joined #openstack-kolla14:08
*** caowei has quit IRC14:14
*** fguillot has joined #openstack-kolla14:18
*** msimonin has quit IRC14:31
*** caowei has joined #openstack-kolla14:33
*** erhudy has joined #openstack-kolla14:34
*** msimonin has joined #openstack-kolla14:36
*** jtriley has joined #openstack-kolla14:37
*** eaguilar_ has joined #openstack-kolla14:37
*** eaguilar has quit IRC14:39
*** absubram has quit IRC14:43
*** salv-orlando has quit IRC14:45
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes: Updated from global requirements  https://review.openstack.org/41619914:47
*** huikang has joined #openstack-kolla14:49
*** huikang has quit IRC14:49
*** tis has joined #openstack-kolla14:50
*** huikang has joined #openstack-kolla14:50
*** mattmceuen has joined #openstack-kolla14:51
*** goldyfruit has joined #openstack-kolla14:55
*** v1k0d3n has joined #openstack-kolla14:59
*** david-lyle has joined #openstack-kolla15:01
*** Pavo has joined #openstack-kolla15:03
*** Serlex has quit IRC15:05
*** mgiles has joined #openstack-kolla15:05
*** rmart04_ has joined #openstack-kolla15:06
*** rmart04 has quit IRC15:06
*** rmart04_ is now known as rmart0415:06
*** david-lyle has quit IRC15:06
*** jtriley has quit IRC15:10
erhudyis there anyone in here with knowledge of the kolla test gate setup? i'm looking at a build failure on a PR of mine and i think it may be a combination of the gate configuration and an existing bug in kolla15:11
*** rmart04 has quit IRC15:12
erhudyit appears that in master if you supply an override for sources.list, it does not get added to the base container anyway because the dockerfile always copies in sources.list.ubuntu or sources.list.debian15:12
erhudyso i fixed that in my branch, but the gate supplies a custom sources.list, and as a consequence the build now tries to proceed with that sources.list and fails with an unauthenticated package error because it's now trying to draw ubuntu main packages from OSIC mirrors instead of canonical's15:13
satyarHi jeffrey4l15:15
satyarseems nova and cinder changes lot many things15:15
satyarwe need to fix those in kolla now15:15
Jeffrey4lhrm. any details?15:16
*** rmart04 has joined #openstack-kolla15:17
*** eaguilar_ has quit IRC15:17
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move mariadb precheck into its own role  https://review.openstack.org/41558715:18
*** jtriley has joined #openstack-kolla15:19
*** sacharya has joined #openstack-kolla15:21
*** eaguilar has joined #openstack-kolla15:22
openstackgerritLi Yingjun proposed openstack/kolla-kubernetes: Fix different repo issue for documentation  https://review.openstack.org/40505415:23
satyarWhen making connections to Ceph-backed volumes via the Libvirt driver, the auth values (rbd_user, rbd_secret_uuid) are now pulled from the backing cinder.conf rather than nova.conf. The nova.conf values are only used if set and the cinder.conf values are not set, but this fallback support is considered accidental and will be removed in the Nova 16.0.0 Pike release. See the Ceph documentation for configuring Cinder for RBD au15:24
satyarbefore ocata nova would fall back to it's own local rbd auth config for those values if cinder wasn't configured15:24
Jeffrey4lwhere did u get this?15:24
satyarwhich points you here docs.ceph.com/docs/master/rbd/rbd-openstack/#configuring-cinder15:24
satyarfom nova irc15:25
satyari saw you are also there15:25
satyar:)15:25
Jeffrey4lyep. reading.15:25
*** sacharya has quit IRC15:26
satyar:)15:26
*** Pavo has quit IRC15:26
kfox1111sbezverk: I could have swarn someone tsid they tried it, but yeah, its not currently liking having a empty yaml.15:29
*** tovin07 has joined #openstack-kolla15:29
kfox1111we could said15:29
kfox1111one workaround for now would be to load in a 3rd party resource or configmap or something that does nothing.15:30
*** sayantan_ has joined #openstack-kolla15:30
Jeffrey4lsatyar, but it should work, right? even though with some log warning.15:31
satyarnot sure..15:31
Jeffrey4lfrom the code, it says the fall back will be removed in P release.15:32
satyaras jaypipes asked to change rbd_secret_uuid to auth_secret_uuid15:32
Jeffrey4lhave u tried this?15:33
Jeffrey4li can not open docs.ceph.com site right now.15:33
srwilkershey kfox1111. happy new year15:33
sbezverkkfox1111: yeah I was afraid it would come to a workaround :(15:34
sbezverkkfox1111: but it has to be done one way or another..15:34
kfox1111yeah. :/15:36
kfox1111sbezverk: would it be more clear if I renamed kolla_val_get_first to be kolla_val_get_raw?15:36
SamYapleo/15:37
*** absubram has joined #openstack-kolla15:37
sbezverkkfox1111: not sure, from get_first name it is clear what it does, returns first found.. from get_raw I do not get this..15:37
kfox1111its only used if you want to ensure the datatype is retrieved unmollested.15:37
kfox1111otherwise, the _str version is way easier to use.15:38
Jeffrey4lhey SamYaple, about ini parser. what's your approach? copy the code into kolla-ansible?15:38
sbezverkkfox1111: ok so there will be a pair: get_raw and get_str?15:39
kfox1111yeah.15:39
kfox1111that would be the alternate.15:40
satyarjeffrey4l yes tried with auth_secret_uuid still got the same error15:40
sbezverkdoing the same operation but treating returned data a bit differently15:40
SamYapleJeffrey4l: youre call. but you should be talking to other people about it. and perhaps make a note about it. referencing requirements.txt is a week defence15:40
kfox1111sbezverk: yeah. thats really the only difference between the two.15:40
Jeffrey4lsatyar, then i think they are different issue.15:40
satyarhmmm15:41
sbezverkkfox1111: in this case it makes sense I think15:41
kfox1111one returns it as a template snippet (string), the other via dict passback.15:41
kfox1111k. I'll change it in the next ps.15:41
Jeffrey4lSamYaple, all openstack project is referencing requirements.txt file now.15:42
Jeffrey4lsatyar, trying to boot from volume. will tell u the reason later.15:42
SamYapleJeffrey4l: no i mean for kolla-ansible. oslo_config was not there becuse it is needed in kolla-ansible. its there because its needed in kolla15:42
satyarsure jeffrey4l15:42
Jeffrey4lroger SamYaple15:43
SamYapleJeffrey4l: and alot of people dont 'pip install kolla-ansible'15:43
SamYapleme included15:43
Jeffrey4lme included too. but i will run `pip install -r requirements.txt `15:43
satyarwe wrote script for that do that does for us :)15:44
Jeffrey4lsatyar, sorry?15:44
SamYapleJeffrey4l: thats not a choice either of us should make is my point. you need to discuss that in a meeting15:45
satyaroops15:45
*** sdake has joined #openstack-kolla15:45
satyarsorry jeffrey4l wrong window15:45
kfox1111sbezverk: I just changed it to if disbaled make a tiny dummy configmap. seems happy enough with it.15:45
Jeffrey4lroger sam15:45
kfox1111ugly but should work until helm gets dynamic support.15:45
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move tacker precheck into its own role  https://review.openstack.org/41605215:45
*** salv-orlando has joined #openstack-kolla15:46
sbezverkkfox1111: cool for now, but if each service when disabled starts creating these configmaps, we will be in trouble.. I think we need to bring it up to helm community..15:46
kfox1111+1.15:47
kfox1111I think we will only need 3. mariadb/rabbit/memcached.15:47
kfox1111but yeah, lets work with helm upstream to fix it asap.15:47
*** inc0 has joined #openstack-kolla15:47
inc0good morning15:48
sdakesup folks15:48
SamYaplemorning inc015:48
inc0happy new year!:)15:48
sdakesnakes on a plane15:48
sdake:)15:48
SamYapleo/15:48
kfox1111happy new year! :)15:48
sbezverkkfox1111: sounds good.. I think the workaround you suggested will allows up to move forward..15:48
SamYaple\o15:48
SamYaple\o/15:48
sbezverkinc0: morning15:48
sdakeinc0 i wasn't sure if you added the ptg brianstorm or I did to the agenda15:49
srwilkersmorning sdake inc015:49
dmsimardsdake: so I heard you a couple times mention you were moving away from tox to test container builds or something, can you expand on that ?15:49
*** TxGirlGeek has joined #openstack-kolla15:49
sdakeufnortunately I won't be able to make tomorrow's meeting15:49
inc0sdake, I didn't15:49
inc0but good thinking15:49
*** rmart04 has quit IRC15:49
sdakedmsimard Jeffrey4l can expnd15:49
inc0I wanted to do it, but maybe a bit later15:49
sdakedmsimard he is doing the work15:49
sdakeinc0 right - typically takes 2-3 sessions ;)15:49
erhudyif somebody could look at the reason behind the ubuntu build failure on https://review.openstack.org/#/c/414639 with me i would appreciate it, per my previous comments about there being a possible problem with those gates15:49
dmsimardJeffrey4l: so how will containers be built/tested in the gate moving forward ?15:49
inc0we still have almost 2 months till pth15:49
sdakeand I figured after the new year is the time to have to biggest session since poeple haven't been working15:49
sdake:)15:50
sdakedmsimard basically tox -e build is gone - instead you ahve to use environment variables15:50
sdakedmsimard I personally think we want the tox mechanism15:50
Jeffrey4ldmsimard, we still depends on tox, but just move it from project-config to kolla's script.15:50
sdakedmsimard for others to do CI with15:50
sdakeJeffrey4l you removed tox -e build-centos-* right?15:50
Jeffrey4lsdake, no. i don't15:51
Jeffrey4lfor kolla project.15:51
sdakeJeffrey4l ok - that was in the original patches15:51
*** salv-orlando has quit IRC15:51
dmsimardJeffrey4l: okay is there any patches up for review (or a topic) I can follow to look at what's coming ?15:51
sdakemaybe there was a followup15:51
Jeffrey4li removed these from kolla-ansible.15:51
sdakelet me look at code15:51
*** adrian_otto has joined #openstack-kolla15:51
sdakeoh kolla-ansible15:51
Jeffrey4lyep.15:51
sdakeJeffrey4l i think it makes sense to keep these in kolla deliverable15:51
dmsimardJeffrey4l: oh so you're not touching kolla, just kolla-ansible ?15:51
sdakeagree or disagree?15:51
Jeffrey4lnow kolla-ansible will re-user kolla's tox -e build- to build images.15:51
Jeffrey4lre-use*15:52
dmsimardJeffrey4l: that makes sense15:52
sdakei get why you removed from kolla-ansible - because  why test builds in a deployment system? :)15:52
dmsimardso nothing changes in kolla then ?15:52
sdakedmsimard i think i had the wrong deliverable mixed up with the change15:52
Jeffrey4l sdake yep. tox in kolla is useful.15:52
sdakedmsimard which is totally my fault for the miscommunication - apologies for that15:52
inc0Jeffrey4l, on that note, what is the issue with registry?15:52
Jeffrey4ldmsimard, about build image, yes.15:52
inc0I'll pick it up if you hit anythin15:52
inc0g15:53
dmsimardsdake: no worries, I already leveraged tox from kolla, not from kolla-ansible15:53
sdakeinc0 which registry issue?15:53
Jeffrey4linc0, it should work.15:53
inc0Jeffrey4l, registry in gates?15:53
sdakeinc0 i've been on pto since 21st more or less15:53
sdakeso i am super out of the loop15:53
dmsimardFWIW I'd like to bring some attention on the required ansible version that was commented in https://review.openstack.org/#/c/411930/15:53
Jeffrey4linc0, it is not merged. but it work as expected.15:53
*** rmart04 has joined #openstack-kolla15:53
Jeffrey4linc0, check the test Patch15:54
Jeffrey4lhttps://review.openstack.org/413720https://review.openstack.org/41372015:54
dmsimardMy last comment:15:54
*** hieulq_ has joined #openstack-kolla15:54
dmsimardCan we bump that for Ocata ? Ansible 2.0.x is already fairly old, we're not in February yet and Ocata will be supported for a year.15:54
dmsimardThe last release of Ansible in 2.0.x was released in April 2016 and there's been a lot of new releases in both 2.1 and 2.2 since then.It's safe to say that 2.0.x is all but unsupported/EOL.15:54
*** kristian__ has joined #openstack-kolla15:54
sdakedmsimard from my understanding x streams are supported15:54
sdakedmsimard let me look at my review15:54
*** rmart04 has quit IRC15:54
*** strigazi_ is now known as strigazi_AFK15:54
sdakerathe that review15:54
dmsimardI've introduced syntax that is >2.1 specific and it looks like we want to pin <2.115:55
Jeffrey4li have to say: ansible made some big change in y stream, which is bad. ;(15:55
dmsimardwhich I don't particularly agree with15:55
dmsimardJeffrey4l: yeah Ansible has moved very fast in the past year/year and a half15:56
dmsimardboth a blessing and a curse15:56
Jeffrey4linc0, the libvirt issue is very critical now.15:56
inc0have you checked if this is qemu issue?15:56
dmsimardbut there are very nice improvements, especially in Ansible 2.2 the performance improvements are *very* significant -- it's really fast now.15:56
sdakeJeffrey4l ya they broke their ABI15:56
sdakeJeffrey4l adding block for e.g. :)15:56
Jeffrey4lsdake, yep. i also made some workaround to make kolla_docker work on both ansible 2.0 and ansible 2.115:57
dmsimardsdake: not just that, I was personally bitten by other things.. regressions or changes in include behaviors (static includes)15:57
sdakedmsimard we want a wide variety of minimum pin levels since theoretically folks will install these from distro packaging15:57
Jeffrey4ldmsimard, i am agree to bump the ansible dependency.15:57
sdaketo 2.1 or 2.2 Jeffrey4l ?15:58
*** Serlex has joined #openstack-kolla15:58
Jeffrey4linc0, no. i got nothing expect libvirt process segment fault.15:58
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: kolla-common value get functions  https://review.openstack.org/41606915:58
sdakevpn no longer works on gogo internet15:58
* sdake groans15:58
Jeffrey4lsdake, hrm. no idea.15:58
inc0ok it's not qemu issue...15:58
inc0and it's just rdo package?15:59
sdakewhat version is in centos?15:59
Jeffrey4lahh, inc0 got what do u means.15:59
sdakeand what version is in ubuntu 16.04?15:59
*** rmart04_ has joined #openstack-kolla15:59
*** athomas has quit IRC15:59
Jeffrey4lsdake, 2.215:59
inc0it still might be qemu15:59
sdakethis should ideallly be our minimum pin versions - so people don't install ansible via pip15:59
sdakeJeffrey4l centos is 2.2 - what ver is xenail 16.04?15:59
Jeffrey4lchecking~15:59
inc0Jeffrey4l, we use qemu or qemu+kvm in this gate?15:59
inc0I know I had issues on rax vms with kvm16:00
Jeffrey4lwe use qemu all the time. inc016:00
dmsimardinc0: the important piece is to pass the cpu_mode=None with the qemu hypervisor16:00
sdakeinc0 perhaps xen just doesn't work well with qemu16:00
dmsimardbecause the default provided by nova when using qemu is cpu_mode=host-model16:00
inc0sdake, xen doesn't affect qemu in any way16:00
sdakei know how its supposed to work16:00
sdakehow it is working is a different story :)16:00
dmsimardiirc (although may be confusing with openstack-ansible) no values are provided by nova and you're probably therefore using host-model16:01
dmsimards/by/to/16:01
*** sayantan_ has quit IRC16:01
Jeffrey4ldmsimard, this issue only happen on rax-iad, which is xen. and we do not get the same error talked in the ml about cpu_mode.16:01
inc0dmsimard, but it shouldn't end up in libvirt segfault, rather machine misbehaving16:01
sdakedmsimard our gate jobs use virt_type=qemu16:01
sdakewhich is probably host-model16:01
dmsimardsdake: yeah but cpu_mode should also be set to none16:01
Jeffrey4lsdake, re ansible version on xenial:  2.0.0.2 -2  ;(16:02
*** sayantan_ has joined #openstack-kolla16:02
sdakedmsimard didn't know that16:02
inc0Jeffrey4l, and it's only rdo package?16:02
Jeffrey4ldmsimard, i will add cpu_mode in the gate and check what will get.16:02
sdakeJeffrey4l you could try setting cpu_mode to see if that fixes the problem with rax-iad16:02
dmsimardsdake: I would say to at least try that and see if that resolves it16:02
Jeffrey4lsdake, it come from epel repo.16:02
Jeffrey4ldmsimard, yep.16:03
*** zhugaoxiao has quit IRC16:03
sdakeJeffrey4l don't know what to do with that :)16:03
sdakeJeffrey4l if we do make the minimum pin 2.1 (where block is introduced) folks will have to pip install on ubuntu16:04
sdakewhich is not something i'd necessarily do in a production environment16:04
sdakeperhaps inc0 has ideas - he uses ubuntu all the time16:04
dmsimard¯\_(ツ)_/¯16:04
dmsimardhow are you expecting people to install kolla ? from pip ? :P16:04
sdakedmsimard from distro packaging16:04
inc0and I used rax public cloud with no issues apart from qemu instead of  kvm16:04
dmsimardubuntu has kolla packaging ?16:04
Jeffrey4lsdake, there is ansible official ppa for ubuntu now, which provide ansible 2.216:04
Jeffrey4lhttps://launchpad.net/~ansible/+archive/ubuntu/ansible16:04
inc0but it *used to* work for us too16:04
sdakeoh misread sentence16:04
*** zhugaoxiao has joined #openstack-kolla16:04
sdakedmsimard i thought you meant ansible16:05
sdakedmsimard atm we require people to pip install kolla because there is no distro packaging16:05
Jeffrey4linc0, this issue happen since centos 7.3 release ,which upgrade libvirt to 2.0.0.16:05
sdakedmsimard if there was distro packaging, I'd prefer people isntall from that16:05
sdakedmsimard mixing distro packaging with pip in the same workflow ends up with broken ansible for example16:05
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move tacker precheck into its own role  https://review.openstack.org/41605216:06
sdakedmsimard ie. pip install ansible - yum install ansible16:06
sdakebust16:06
sdakeor maybe the order is different16:06
dmsimardyeah16:06
sdakebut you get the idea16:06
kfox1111heading into work. bbiab.16:06
sdakekfox1111 email soon plz :)16:06
sbezverkkfox1111: ttyl16:06
sdakepbourke morning dude16:06
pbourkesdake: o/ morning16:07
sdakepbourke did you see my JJ Abrams photo of my car ;)16:07
sdakewell I know you saw it16:07
pbourkesdake: yeah pretty sweet :)16:07
inc0sdake, show me16:07
inc0;)16:07
sdakebut did you see the light as subject? :)16:07
*** jemcevoy has joined #openstack-kolla16:07
sdakeinc0 its on twitter somewheres16:07
sdakemoment16:07
Jeffrey4lqq: dmsimard add    `cpu_mode=none` into libvirt group, right?16:08
dmsimardJeffrey4l: let me double check16:08
sdakeinc0 https://twitter.com/steven_dake/status/81596492850274304016:09
sdakephone photos bit washed out16:09
inc0nice one16:09
sdakeshame I didn't have my wife's fuji camera with me16:09
sdakethanks - i was actually more impressed with the photo then the car :)16:09
dmsimardJeffrey4l: yes, in libvirt group.16:10
Jeffrey4lroger. thanks.16:10
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST MASTER BRANCH  https://review.openstack.org/39809516:10
openstackgerritMerged openstack/kolla: Not to pull the base images when building.  https://review.openstack.org/40894016:11
inc0I need to take a shot of my gokart16:11
sdakeinc0 lol :)16:11
sdakeI took a college photography class16:12
sdakethere was one young lady in there - her work was absolutely fantastic16:12
inc0I have gokart thats allowed to be on road;)16:12
sdakethe teacher told me the school was a feeder school and she would end up going to asu for professional photography16:12
inc0http://www.diseno-art.com/news_content/wp-content/uploads/2014/09/Mazda-MX-5-ND-front-and-rear.jpg16:12
sdakeI was there just to learn how to photograph stuff :)16:13
sdakewhen I had my headshot taken (literally and figuratively!) the dude had all kinds of interesting tricks16:13
sdakeinc0 i've seen your car iirc16:13
*** awiddersheim has quit IRC16:13
sdakeinc0 i think I stored my jacket in it in austin16:14
sdakeor something - maybe a bag16:14
inc0a yeah16:14
sdakedon't recall exactly16:14
inc0jacket16:14
sdakeyup - I do that often jacket or backpack in trunk :)16:14
dmsimardIs there a prequel to http://docs.openstack.org/developer/kolla/image-building.html ? How do I know what to install/setup prior to building images ? ex: installing docker and stuff.16:14
inc0waiting for spring to get my roof down16:14
sdakeinc0 - i'll be indisposed 4th-6th16:14
inc0thanks for heads up16:15
sdakeyup16:15
inc0dmsimard, quickstart16:15
sdakeinc0 also any chance you can get that etherpad updated with notes + video of meeting?16:15
sdakeinc0 since its the new year, people may start looking at it16:15
dmsimardinc0: so basically kolla-ansible bootstrap-server ?16:15
inc0image building is supposed to go after quickstart16:15
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move grafana precheck into its own role  https://review.openstack.org/41605516:15
*** apuimedo|away is now known as apuimedo16:16
inc0dmsimard, yeah, and pip install kolla16:16
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move grafana precheck into its own role  https://review.openstack.org/41605516:16
inc0sdake, helm guy is silent16:16
inc0I'll ping him on email16:16
sdakeya they are on pto until the 4th i think I saw16:16
inc0probably just went for vacation16:16
sdakein their release notes16:16
dmsimardinc0: the reason I ask is because it looks like the kolla gate jobs seem to do so many hacks that are not covered in quickstart/bootstrap-server16:16
*** athomas has joined #openstack-kolla16:16
sdakedmsimard i wouldn't really use the gate jobs as a reference :)16:17
*** awiddersheim has joined #openstack-kolla16:17
sdakedmsimard but i guess your making your own gating jobs so maybe that makes sense16:17
inc0dmsimard, gates are gates, we need to take into account all the stuff to make it work on all nodepools16:17
*** shananigans has joined #openstack-kolla16:17
inc0most of these hacks are specific to some nodepool issue16:17
sdakedmsimard all that hack stuff we do in the gate jobs = the work hacks is not strong enough :)16:17
sdakework/word16:18
inc0I'd suggest starting fresh and adding hacks as needed:)16:18
sdakeinc0++16:18
dmsimardWell, yes and no, I'd expect the quickstart to be used in the gate and not just as a means to test the actual quickstart16:18
inc0maybe some of these hacks aren't needed any more16:18
dmsimardbootstrap-server isn't tested anywhere right now16:18
sdakedmsimard ya i have wanted that since inception of kolla16:18
dmsimardbut let's pretend that's a kolla-ansible specific issue16:18
sdakedmsimard the qsg tested in the gate16:18
inc0bootstrap servers was designed to provision compute/controller nodes16:19
inc0not aio/deploy node16:19
sdakedmsimard re bootstrap-server in the gate, I think that is something that could be tackled - if someone knew how to work on the gate and was so inclined :)16:19
sdakeinc0 our gate jobs do multinode too :)16:19
inc0but yeah, I agree we can leverage it16:19
sdakeinc0 we actually use ansible in the gate jobs to run against the nodepool inventory that we are passed16:19
dmsimardinc0: a deploy node for you is where the images are built? (making use we use the same vocabulary)16:19
inc0deploying gates with it would be much clearer16:19
sdakeand the ansible we use in our gate jobs is sort of like bootstrap-server16:20
inc0dmsimard, normally thats the case but there is no specific reason for it16:20
inc0it can be any node16:20
*** lamt has quit IRC16:20
dmsimardright16:20
inc0well, reason being that people don't re-ssh into other nodes;)16:20
sdakedmsimard typically the deployment host is different node from the other nodes16:20
sdakedmsimard but it can also be on the controller, compute, etc16:21
dmsimardanyway, I sort of had this discussion with SamYaple before the holidays but it didn't seem like we agreed and I didn't fight it as I was going on PTO :p16:21
sdakedmsimard which discussion - what specific request do you have?16:21
*** lamt has joined #openstack-kolla16:21
dmsimardI was asking about how the gate is setup in general, why not use bootstrap-server, why use a mix of bash and ansible, etc.16:22
dmsimardlet me try and find it16:22
Jeffrey4ldmsimard, it is historical issue.16:23
*** rmart04_ has quit IRC16:23
Jeffrey4lat the beginning, only bash script. later, we want to move to ansible playbook.16:23
Jeffrey4land then we added the bootstrap-server feature.16:23
dmsimardhttp://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2016-12-21.log.html#t2016-12-21T21:16:3816:23
Jeffrey4loptimal, we'd better to mvoe to bootstrap-server.16:24
dmsimardJeffrey4l: then again you're leveraging kolla-ansible inside the kolla gate ?16:24
dmsimardI mean, why not use the bootstrap-server equivalent in kubernetes or now salt16:24
Jeffrey4ldmsimard, yep.16:24
dmsimard(playing devil's advocate)16:24
Jeffrey4lsorry? salt?16:25
dmsimardThere's a new kolla-salt thing that's been in the works apparently16:25
dmsimardsaw it on openstack-dev16:25
Jeffrey4ldmsimard, yep. it it.16:25
Jeffrey4lit is*16:25
Jeffrey4l afaik, bootstrap-server only exist in kolla-ansible.16:26
*** tis has quit IRC16:26
dmsimardbut there can/could/should be something equivalent in other deployment frameworks16:26
Jeffrey4lpossible. i think this kinda of feature will be added in other kolla-* project in the future.16:27
*** adrian_otto has quit IRC16:35
*** adrian_otto has joined #openstack-kolla16:36
kfox1111back16:36
kfox1111sdake: email?16:37
sdakekfox1111 the one about the api and whats needed16:37
kfox1111oh.16:37
kfox1111yeah. I'll try and work on that today.16:37
kfox1111got about a week of email to catch up on this morning. :/16:38
kfox1111and 2 and a half weeks of being out of office. :/16:38
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla: Remove source install type for novnc / spice  https://review.openstack.org/40859916:40
*** klindgren_ has quit IRC16:40
*** sayantan_ has quit IRC16:40
kfox1111sdake: sorry. can you please refresh my memory a bit. what did you want in the email again? is this about what we would like out of helm or what the status of where we're at now, or where we want to be in a week?16:42
*** tovin07 has quit IRC16:42
kfox1111too many things going on right now to keep things streight. :/16:42
sdakekfox1111 ok - here is what Id ideally like16:43
sdakekfox1111 I'd like to understand where the API model will be when we release 0.4.0 next wed16:44
openstackgerritMerged openstack/kolla-ansible: Add Constraints support  https://review.openstack.org/41329116:44
sdakein detail16:44
Jeffrey4lsatyar, around?16:44
openstackgerritMerged openstack/kolla-ansible: Enable horizon deployment in gate  https://review.openstack.org/41005716:44
sdakekfox1111 i think that API model is a crucial thing we have going for us - and I think we want to all understand it in detail16:44
sdakethe fact that I've been on pto for 2 weeks + out of the office starting tomorrow isn't helping :)16:45
sdakekfox1111 normally I'd probably just understand this via reviewing, however I mostly unplugged over the holiday shutdown and did other things with family/etc16:45
*** adrian_otto has quit IRC16:45
*** klindgren has joined #openstack-kolla16:46
openstackgerritMerged openstack/kolla-ansible: Modify the openstack_release in globals.yml  https://review.openstack.org/41447816:46
kfox1111sdake: problem is, its kind of in the middle of a refactor right now. :/ and will take most of the week to nail down.16:47
sdakekfox1111 ok - well can you just give me your end state in the email16:47
sdakeor do you not know it?16:47
kfox1111the helm set function they gave us for christsmas was awesome, but made a bunch of work.16:47
sdakelike what features of helm do we use16:47
kfox1111k. could give you the desired end state.16:47
sdakeas it relates to the API16:47
sdakethis is crucial information i'm after16:47
sdakekfox1111 should go to mailing list16:48
kfox1111k16:48
sdakekfox1111 if you could do today i'd appreciate it16:49
kfox1111k. I'll try and prioritize it.16:49
sdakeas TLDR as possible :)16:50
sdakebut if the TLDRness means you won't do it16:50
sdakethen trim back on the TLDR :)16:50
kfox1111nan, the tldr's much easier for me to try and write then a long thing.16:50
kfox1111nah16:50
sdakekfox1111 i think your irssi is busted again :)16:51
kfox1111no, it just has too many tabs to see when people ping me sometimes. :/16:51
kfox1111especially when users renamethemselves a lot. :/16:51
Jeffrey4lsatyar, got this error when booting from volume. http://paste.openstack.org/show/593799/  will check it in detail tomorrow.16:51
sdakeok peeps gott ajet - airplane landing16:53
satyarHi jeffrey4l yes16:55
satyarin a meeting now...16:55
satyarsorry for late response16:55
Jeffrey4l satyar np. i am  going to sleep. will check this tomorrow ;)16:55
*** sdake has quit IRC16:57
*** vhosakot has joined #openstack-kolla17:00
portdirectany chance a core could give this a look over? https://review.openstack.org/#/c/412205/17:01
portdirect(kolla core that is)17:01
*** david-lyle has joined #openstack-kolla17:03
*** sayantan_ has joined #openstack-kolla17:05
*** inc0 has quit IRC17:06
*** adrian_otto has joined #openstack-kolla17:06
*** matrohon has quit IRC17:07
*** david-lyle has quit IRC17:08
*** chas has quit IRC17:09
*** chas has joined #openstack-kolla17:09
*** chas has quit IRC17:14
*** matrohon has joined #openstack-kolla17:15
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: kolla-common value get functions  https://review.openstack.org/41606917:24
*** mnasiadka has quit IRC17:25
*** l4yerffeJ has joined #openstack-kolla17:25
*** adrian_otto has quit IRC17:33
*** inc0 has joined #openstack-kolla17:35
*** adrian_otto has joined #openstack-kolla17:35
*** msimonin has quit IRC17:35
openstackgerritMerged openstack/kolla: Add zun support in README.rst  https://review.openstack.org/41562817:36
*** apuimedo is now known as apuimed|away17:36
inc0portdirect, out of curiosity, how are we going to deploy k8s images?17:36
inc0https://review.openstack.org/#/c/412205/ <- talking about this one17:37
portdirectinc0, that's an exercise for the reader :) (serious answer to follow...)17:37
portdirectthe images there are drop in compatible with the gcr.io images - but will need retagged to use with k8s deployment tooling17:38
portdirecttaking kubeadm as an example - the current method is to pull the image to the hosts required and then retag to match the gcr.io images before running kubeadm17:39
portdirect^^ horrible - but is the currently reommended way for people beind corb firewalls or in china to use kubeadm17:40
inc0kinda horrible, yeah17:40
portdirectwith other tools eg kargo etc then its a bit simpler as i bleive you can speficy the image name to use - but operaors will still need to retag the version to match the k8s version17:40
inc0on the other hand, keeping images in sync with gcr.io is not trivial17:41
portdirectand some 3rd party tools eg Project Atomic's commissaire should be able to use them directly (eventually)17:42
portdirectI wanted to get the into the repo so that we could have a source of truth for our gate primarity - and have a set of known working images to use for our purposes17:43
inc0yeah makes sense17:43
*** kristian__ has quit IRC17:43
inc0for gates at least - useful17:44
inc0especially if we crack the registry issue17:44
inc0which we're pretty close to do17:44
portdirectas for ceph support we are currently relying  att-comdev's image - thanks srwilkers/v1k0d3n ! :)17:44
inc0att-comdev has different ceph images than kolla?17:44
portdirectno - but we need to use thier controller manager image if we want native pvc support17:45
portdirecthttps://github.com/att-comdev/dockerfiles/blob/master/kube-controller-manager/Dockerfile17:46
*** shardy has quit IRC17:46
portdirectfor actual ceph support, both att-comdev and I are using the images from docker-ceph: https://github.com/portdirect/ceph-helm17:47
portdirect^^ chart for example - theirs is different, though I think about to be updated to use a similar deployment for mons/config17:49
portdirectwhich reminds me, leseb you about?17:49
*** kristian__ has joined #openstack-kolla17:55
*** pcaruana has quit IRC17:58
*** kristian__ has quit IRC17:59
sbezverkportdirect: Are you planning to use native pvc?18:00
sbezverkI do not remember any discussion here about it, other than "it exists" :-)18:00
inc0hmm18:02
inc0so I set up helm stable/mysql18:02
inc0and deleted stable/mysql18:02
inc0and now it's stuck pending:(18:02
inc0ahh here I was, hoping that stuff will work;)18:02
v1k0d3ninc0: did you delete --purge?18:02
sbezverkinc0: mariadb pod is in pending?18:02
inc0v1k0d3n, I did not, let's see18:03
v1k0d3nthat will purge everything, including jobs.18:03
sbezverkv1k0d3n: doubt it will help, as --purge just purges the name from helm18:03
inc0out of curiosity, is stable/mysql really stable?18:03
*** david-lyle has joined #openstack-kolla18:03
inc0sbezverk, nope, it purged it all right18:04
sbezverkinc0: you should use mariadb from kolla-kubernetes repo18:04
inc0sbezverk, my question is18:04
sbezverkit has been proven stable bu many gate runs18:04
inc0if we have good mariadb18:04
v1k0d3nif it's not sbezverk it may be a bug. it's supposed to.18:04
*** eaguilar_ has joined #openstack-kolla18:04
inc0in helm18:04
inc0maybe we should consider using it18:04
portdirectsbezverk: I mush confess I thought that was the intention (native PVC)18:04
portdirect*must18:04
*** eaguilar has quit IRC18:04
inc0sbezverk, our mariadb is still not that great in terms of HA18:04
sbezverkportdirect: possibly, it is just a surprise for me18:05
inc0I'd rather have someone to blame for not great HA;)18:05
inc0narrowing scope of kolla would be good thing tbh18:05
sbezverkinc0: I see your point, I guess we are not there yet18:05
inc0helm might help18:05
v1k0d3nso inc0...18:05
portdirectsbezverk: though there has not been any objection to date whenever I've mentioned it - should we put it on the ajenda for tomorrow to clarify?18:05
inc0so v1k0d3n...18:06
v1k0d3nwe actually recommend using upstream mariabd if the user wants to do so.18:06
v1k0d3n:)18:06
inc0well it does make sense ofc18:06
sbezverkportdirec: sure, if it is a must then why we do not create templates anywhere for it??18:06
inc0and I'd say we should start thinking in this direction18:06
inc0most of crap we deal with is with mariadb/rabbimq18:06
v1k0d3nat the end of the day, mariadb is mariadb (as long as it's a galera quorum). in fact separating out client is really nice.18:06
inc0and if we install stable/mariadb with helm, no reason why we need to support it18:06
sbezverkv1k0d3n: curious how can you do that? or your gate tests upstream?18:07
v1k0d3nmost of the issues you're dealing with is mariadb and not PVC related?18:07
portdirectsbezverk: as it only came into being with 1.5 and we were targetting 1.4 when the services that consumed pvc were created18:07
inc0sbezverk, helm install stable/mariadb will work in gates18:07
portdirectsbezverk: so simply just becasue we havent created them yet i think :)18:07
awiddersheimI am having a weird issue with overriding the api_interface for a specific node18:08
v1k0d3n^ yup18:08
inc0(now I only need to figure out how to use stable/mariadb with ha and stuff)18:08
v1k0d3nthat's all coming though. at least on our side, our jenkins guy has been on nearly a month long vacation.18:08
*** david-lyle has quit IRC18:08
v1k0d3nin fact inc0...18:08
awiddersheimthe docs and examples say you can just do "somehost api_interface=eth0" but what I have configured in my globals.yml file for "network_interface" seems to be taking precedence18:08
sbezverkinc0: our mariadb is not just mariadb, it has bunch of things making it eatble by openstack services18:08
inc0sbezverk, like what?18:09
v1k0d3njust using mysql works perfectly for minikube openstack development :)18:09
* srwilkers catches up18:09
v1k0d3nwe started doing it that way...makes things much, much easier!18:09
inc0v1k0d3n, I'm thinking prod here18:09
sbezverkinc0: svc/pvc/bootstrap18:09
portdirectv1k0d3n: I dont think that upstream mariadb will work with kolla-k8s18:09
sbezverkeach is a microservice under mariadb umbrella18:09
portdirectv1k0d3n: though I may be wrong18:09
inc0sbezverk, boostrap can ofc stay in kolla, pvc, svc should be handled by stable/mariadb maintainers18:10
sbezverkyou can pick and choose what needs to be done18:10
v1k0d3nyeah, not sure about kolla-kube. been out of it for a while.18:10
v1k0d3nthat's what i've been testing on our side, and works.18:10
portdirectv1k0d3n: obviously a database is a database - but the config will need some twaeking18:10
inc0yeah I don't think there is any reason why it wouldnt really18:10
v1k0d3nmake sense. gotcha.18:10
inc0I expect helm to allow us to configure it right?18:11
inc0if it doesn't, helm wouldn't be super useful18:11
*** athomas has quit IRC18:11
*** Pavo has joined #openstack-kolla18:11
v1k0d3nwell...maybe. depends. devil is always in the details.18:11
sbezverkinc0: hmm what about integration with API?18:11
inc0awiddersheim, api_interface defaults to value from network_interface18:11
inc0sbezverk, what do you mean? db is db18:11
inc0all openstack services needs is connections string18:12
sbezverkwhen you said about pvc/pv/svc18:12
sbezverkthese microservices gets info from API18:12
inc0yeah, we need to figure out way to create connection string18:12
inc0I'm not saying it will work right away18:13
inc0but I'm saying with a bit of tweaking we can turn kolla-k8s to use helm/mariadb18:13
SamYaplecan I get a review on my zookeeper role? its been sitting for 2 weeks https://review.openstack.org/#/c/412423/18:13
inc0and stop carying about supporting mariadb, which would be nice18:13
inc0SamYaple, that's what you get if you propose big hard changes in holiday seaon;)18:14
SamYapleits neither big nor hard!18:14
sbezverkinc0: in future possibly yes, but now stop everything until this tweaking is done. I am not sure if we have this luxury18:14
inc0sbezverk, oh I'm not saying we should rewrite it now!18:14
inc0I'm saying, in Pike we can think of rewriting kolla-k8s in a way to support stuff like that18:15
inc0would make our jobs sooo much easier18:15
sbezverkinc0: ok :) I am still recovering from jumping from jinja to helm and now I thought you want another change in direction18:15
inc0nah, in ocata we stick to what we have for good or worse18:15
sbezverkinc0: sounds good :)18:16
inc0for Pike I wanted to take long critical look at it and use all we learned to rearchitect it18:16
inc0we'll have longer release and lots of knowledge18:16
inc0I'm just saying that I agree with v1k0d3n and we should limit scope of kolla as much as we can18:17
srwilkersinc0, i share that sentiment18:17
inc0leave stuff like ceph, mariadb or rabbimq to their communities, we'll end up having better quality18:17
v1k0d3n+1 inc018:18
inc0we kinda have this approach of "batteries included" in kolla-ansible but that's because we didn't have tools that allows us to use different people's expertise to deploy our infra stuff (db and such)18:18
inc0with helm, we kinda do18:18
sbezverkinc0: as long as tweaking to make these consumable by kolla-kubernetes != amount of work to do it within kolla18:18
v1k0d3ni know a lot of work is done around the API for microservices too...took a lot of work, and is required with Docker...but question...is there a more simple solution with a helm version?18:18
v1k0d3nfor instance, is it required?18:19
inc0sbezverk, trust me, it will be *way* less work than having to support these services in the long run18:19
v1k0d3njust discussing only. i really want to avoid controversy. but the more complex, and more custom, i fear some will shy away (large corps).18:19
inc0v1k0d3n, again, let's discuss it for pike18:20
v1k0d3nsure, of course.18:20
inc0as I said on helm meeting18:20
v1k0d3ntabled for now :)18:20
SamYapleyea the "batteries included" was more about making it easier, at least in the beginning18:20
sbezverkinc0 +1 this type of discussions now just unfocusing us18:20
inc0ideally at the end of the day I'd like kolla-k8s community to support helm native stable charts for openstack18:20
inc0SamYaple, agree, what I'm saying, not needed in k8s case really18:21
inc0so we can change this approach I feel18:21
SamYaplei think i would agree with you there18:21
*** kristia__ has joined #openstack-kolla18:21
v1k0d3nsbezverk: it shouldn't unfocus. i can table it. these are good questions for the community to consider and discuss.18:21
SamYaplein kolla-ansible we deploy on an unknown host, k8s is hardly "unknown" in my opinion18:21
inc0SamYaple, also in k8s we use package manager18:21
inc0so we can just type helm install stable/mariad18:22
inc0and that's it18:22
v1k0d3n+1 SamYaple18:22
sbezverkv1k0d3n: it should not but it does unfortunately..18:22
inc0sbezverk, it's natural that it does18:22
inc0that's why I'd suggest starting drafting spec18:22
inc0for Pike18:22
inc0not of what we have, but what we want to focus on after 1.018:23
inc0and *not* bloking our work on that spec18:23
v1k0d3n+1 inc0 that keeps things right on track, and plans in the future as we tried to do in Barcelona18:23
inc0like, work on it in pararell, then in Atlanta we'll just share beer(s) and draft our path for Pike18:24
v1k0d3nbaby steps forward is good.18:24
inc0v1k0d3n, now we know A LOT more than we did in BCN18:24
inc0now we can make educated plans18:24
inc0so bottom line, we don't change our direction in Ocata18:24
inc0but I'd suggest we come to BCN with clean minds and start fresh18:25
portdirectto this end I've been reaching out as much as possible to our up-streams - as for ceph esp - it seemed to make much more sense to help them get to where we need than to build a "propitiatory" implementation, as this way we can capitalise on their expertise and offload ownership to people who *really* care about it working for as many people and use cases as18:25
portdirectpossible. It would be great if somone could do the same for mariadb - to get it offering what we need ootb.18:25
inc0I'd also be ok telling that to people that Ocata kolla-k8s is ready to deploy, but it might change a lot for pike18:25
v1k0d3nagree with you there. the whole exercize has been a learning experience for everyone involved.18:25
inc0we'll provide upgrade path, but don't expect code backward compatibility18:25
*** kristia__ has quit IRC18:25
inc0I'm fairly sure that however we want to change our chart structure, upgrade O->P will be possible18:26
inc0portdirect, also, nothing stops us from helping them18:26
inc0just, share the community, don't silo-ize it18:27
portdirect+2 exactly - though time, and expertese, mean I cant really take that on for mariadb18:27
sbezverkinc0: btw we have already some service level packages in a very native to helm form, mariadb is one of them18:27
portdirectthough alanmeadows work there (mariadb) is pretty awesome, from what I've seen18:28
inc0sbezverk, so my suggestion would be to review helm native mariadb and improve it using our learnings18:29
inc0of if there is no service, and we have it18:29
inc0for example idk...memcache (probably there is but for sake of argument)18:29
inc0we can propose it ourselves18:29
inc0and support it as official helm chart for memcache rather than part of kolla18:30
v1k0d3nportdirect: since you know the kolla parts...what are the main differences between kolla and what we have?18:30
inc0also, again, I'd love for us to provide helm native charts for openstack18:30
v1k0d3ni know the differences between upstream and ours, and honestly...i agree with inc0, having that upstream will be great.18:30
portdirectv1k0d3n: by you, you mean att-comdev?18:31
srwilkersportdirect, yeah18:31
v1k0d3nyes18:31
portdirectin that case the charts you have, taking mariadb as an example - are closer to being consumable by the charts repo18:31
portdirectas they are not so split into subcharts for each component18:32
v1k0d3nah, yes. that's a big one.18:32
portdirect(which has pros and cons)18:32
inc0portdirect, imho this is issue we should resolve with helm18:32
v1k0d3nusing configmaps, seeds, single service, etc? right?18:32
inc0v1k0d3n, or figure out helm way to do it with helm community18:33
portdirectthough once ther service layer is done I expect our charts to be much mor similar in terms of consumption by end users18:33
*** hieulq_ has quit IRC18:33
inc0code in helm feature if needed18:33
*** hieulq_ has joined #openstack-kolla18:34
portdirectbut the unserlying struture will be a bit different - I'm not yet sure if we have decided how the config will be generated yet in kolla18:34
*** hieulq_ has quit IRC18:34
sbezverkportdirect: after API is finilized config gen would be big thing to decide18:34
inc0yeah, we might end up forking ansible configs18:35
*** prameswar has quit IRC18:35
portdirectbut kolla-k8s's mariadb chart will consume a subchart for the pod, svc, pvc and pv - whihc is very different from most helm charts18:35
portdirect^^ again with pros (controll) and cons (lack of uniformity with other charts)18:36
inc0lack of uniformity is an issue in my book tbh18:36
sbezverkportdirect: subcharting is very native for helm, so what we have done by splitting mariadb into a bunch of subcharts/microservices is not a problem imho18:36
*** sayanta__ has joined #openstack-kolla18:37
inc0sbezverk, it is if we'd like to have chart-per-repo whch is requirement for having charts in kubernetes repo18:37
sbezverkI consider it as a more efficient and flexible approach than shoving all into the same chart18:37
inc0and again, I think we should aim to do it18:37
inc0sbezverk, I was thinking that chart per service seems intuitive18:38
inc0nova - a chart18:38
inc0helm install nova18:38
inc0as opposed to helm install openstack18:38
*** adrian_otto has quit IRC18:38
*** sayantan_ has quit IRC18:38
*** adrian_otto has joined #openstack-kolla18:39
sbezverkinc0 what if you need something special for one of nova components?18:39
inc0and keep some "signle source of truth" to connect services with each other18:39
inc0sbezverk, if helm lacks configurability, that's something we should address in helm18:39
portdirectinc0 that's exactly what the service layer should deliver - helm install openstack is madness in my option (and I think we are all in agreement there)18:39
sbezverkif you have everything in the same chart then it becomes less intuitive to deal with cases like that18:39
inc0(I'm talking about end goal, nto what we have now)18:39
sbezverkportdirect: for some cases even madness might be useful ;-) like roadshows18:40
*** sdake has joined #openstack-kolla18:41
sbezverkportdirect: but I agree that service level probably will be used mostly18:41
inc0sbezverk, service level makes much more sense18:42
*** david-lyle has joined #openstack-kolla18:42
inc0we just need to figure out "single source of truth"18:42
*** salv-orlando has joined #openstack-kolla18:42
inc0and get helm to develop featureset for us to customize installations the way you described18:42
inc0I think between me, v1k0d3n's team and other volunteers we could even find people to do it if we define what we need exactly18:43
inc0and Pike is longer, gives us more flexibilty18:44
sdakehey folks18:44
sbezverkinc0: service level is might be the most common but it does not mean microservice level is not usefull.18:44
inc0sbezverk, we should do service level with microservice mechanism18:45
sbezverkinc0 +118:45
inc0although, in kolla-ansible we don't really have that much of microservice management right18:45
inc0?18:45
inc0like, nova will always deploy nova-api18:45
inc0it might not deploy nova-spiceproxy, but there is only so much configurability that makes sens18:46
inc0e18:46
inc0well ok I take that back, you can config it all with inventory modification18:46
inc0but regardless, helm should support that.18:46
sdakeinc0 catching up on the logs - i don't see a strong rationale to rearchitect kolla-kubernetes18:48
*** haplo37 has quit IRC18:48
sdakeinc0 that  would result in second system effect which is to be avoided at all costs18:48
sdakeinc0 instead incrementally changing it to suit is better and doesn't result in a monstrosity :)18:49
inc0sdake, having kolla charts as official charts would need rearchitect18:49
inc0and there is only so much you can do incrementally18:49
sdakei disagree with both statements :)18:50
sdakeheat has been completely incrementally changed atleast 20 times18:50
sdakekolla-ansible 8 times atleast18:50
sdakeand the kolla deliverable has been through 5+ refactors that make it look much different18:50
sdakeanything can be refactored via incremental change to fit the shoe we want it to fit in18:50
inc0but changing compose->ansible was not incrementsl18:51
SamYapleinc0: it was, you just werent around18:51
*** salv-orlando has quit IRC18:51
*** vhosakot has quit IRC18:51
SamYapleinc0: i wrote a docker compose module in ansible originally18:51
SamYaplethen i refactored it out18:51
inc0SamYaple, I was around.18:51
*** salv-orlando has joined #openstack-kolla18:51
inc0but what I'm saying18:51
inc0it's revolution18:52
SamYapleinc0: https://github.com/ansible/ansible-modules-extras/pull/35418:52
sdakestart fresh is terrible idea18:52
SamYaplebut ^ that was incremental?18:52
SamYapleim not sure what your point is18:52
sdakethe community will have state loaded -if you start fresh that will dump state completely18:52
SamYaplecompose->ansible was _acutally_ incremetnal18:52
sdakeSamYaple agreed18:52
inc0sdake, fresh as in "we learned a lot, now let's get to the drawing board"18:53
v1k0d3nwell, this is all very unfortunate.18:53
SamYapleinc0: im not disagreeing with you, i just dont know your point18:53
sdakewe are already at the ddrawing board18:53
inc0however we'll push code18:53
inc0sdake, no, we're not, we're sprinting with development18:53
v1k0d3ngotta run guys. you know where to find me. got other things to do. later.18:53
inc0SamYaple, my point is, at some point you need to drop old approach and start new approach18:53
sdakeright - dev + drawing board happens at same time in my mind :)18:53
sdakeinc0 here is what I get from reading the log18:54
inc0and that's what I'd like to discuss for Pike18:54
sdakeyou intend to delete kolla-kubernetes repo and start over18:54
inc0sdake, no18:54
inc0god no18:54
sdakethats what I get from reading the log18:54
inc0I'm saying we have ocata the way we have today18:54
sdakewhereas incremental change (like the dockerfile refactor you led earlier this year) is the way to go18:54
inc0with microservice arch we have18:54
sdakewe are doing services as well as a meta package18:55
inc0and we will discuss, given learnings we aquired, how to make it better without need of backwards compatibility18:55
inc0but with need of upgrades18:55
sdakekolla-kubernetes is on a release-independent schedule18:55
inc0but we have milestones18:55
sdakeif we release a 1.0.0 it needs to be upgradeable18:55
inc0ofc18:55
sdakeright we have milestones - which lead into april :)18:55
*** vhosakot has joined #openstack-kolla18:55
sdakeour end date is april not jan 2618:55
sdakesummit is may18:56
sdakewe have plenty o time18:56
inc0where did you get april? we submitted for Ocata18:56
inc0Ocata is Jan18:56
inc0we can get to feb18:56
sdakeopenstack summit is in may18:56
inc0but design summit is in Atlanta18:56
sdakewe dont need to release on the schedule18:56
inc0and Ocata is in Feb18:56
inc0but we want to try18:56
sdakewe pick the date we release 1.0.0 on release-independent18:57
inc0when did we pick it?18:57
sdakewe are not beholden to the release schedule according to Doug on the ml18:57
inc0because I'm fairly sure we were talking about ocata18:57
inc0ofc we're not18:57
*** haplo37 has joined #openstack-kolla18:57
sdakethe core team while you were schlepping your stuff across country :)18:57
inc0ok...can I have irc logs please?18:57
sdakemy point is this - march 31 or thereabouts, we need to have kolla-kubernetes 1.0.0 with ready to wrap a bow on it18:57
inc0because I'm pretty sure it wasnt on ML18:57
v1k0d3ncan we hit the pause button for a moment guys?18:58
sdakeand 2.0.0 changed to release:trailing_cycle18:58
sdakeso our 2.0.0 release is actually synced with Pike18:58
inc0guys, this is policy decision18:58
inc0which we agreed we solve on ML18:58
sdakeI talked to doug on the ml about it18:58
inc0with voting18:58
inc0don't overrule your own rules please18:58
sdakei agree changing the release cycle is a policy decision18:58
sdakehowever we are currently release_independt18:58
sdakethat doesn't make any sense for 2.0.018:59
*** v1k0d3n has left #openstack-kolla18:59
sdakewe are unable to change the release cycle model at present18:59
inc0sdake, timeframe when we release 1.0 is a poliy decision18:59
inc0afk for 30min18:59
sdakewhen its ready is the answer18:59
inc0brb18:59
*** inc0 has quit IRC18:59
*** salv-orlando has quit IRC18:59
sdakeportdirect i was pretty sure we agreed to release 1.0.0 beginning of april - is my memory fuzzy?19:00
sdakesbezverk kfox1111 ??19:01
*** salv-orlando has joined #openstack-kolla19:01
sbezverksdake: +119:01
sdake+1 my memory is fuzzy? :)19:01
sbezverkboth ;-)19:01
sbezverksdake: seriously I think april target is correct19:02
sdakewell i want to make sure we have a majority in agreement19:02
*** sdake has quit IRC19:04
* portdirect reading scollback19:09
portdirectjust got back home 2 mins19:10
*** chas has joined #openstack-kolla19:10
Pavoevening gents19:12
*** kristian__ has joined #openstack-kolla19:15
*** chas has quit IRC19:15
*** dave-mccowan has quit IRC19:16
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: kolla-common value get functions  https://review.openstack.org/41606919:17
kfox1111back19:17
kfox1111wow...19:18
* kfox1111 starts reading back19:18
sbezverkkfox1111Ь looks good now, will ack when the gate is green19:19
sbezverkkfox1111: you missed another storm in the glass of tea ;)19:19
*** kristian__ has quit IRC19:20
*** MasterOfBugs has joined #openstack-kolla19:20
*** pramodrj07 has joined #openstack-kolla19:20
*** pramodrj07 has quit IRC19:23
*** MasterOfBugs has quit IRC19:23
openstackgerritMerged openstack/kolla: Add Kubernetes Images for Kolla-Kubernetes  https://review.openstack.org/41220519:24
srwilkersyou say storm, i say the beauty of a community expressing differing opinions19:25
*** MasterOfBugs has joined #openstack-kolla19:27
*** inc0 has joined #openstack-kolla19:28
*** v1k0d3n has joined #openstack-kolla19:29
inc0back19:29
inc0had quick meeting, sorry19:29
srwilkerswb inc019:29
sbezverksrwilkers: I guess it depends on a person ;-)19:30
*** kristian__ has joined #openstack-kolla19:31
*** salv-orlando has quit IRC19:35
*** serverascode has quit IRC19:35
*** alanmeadows has quit IRC19:35
*** NachoDuck has quit IRC19:35
*** kong_ has quit IRC19:35
*** JoseMello has quit IRC19:35
*** peterjenkins_ has quit IRC19:35
*** igordcard has quit IRC19:35
*** markmcclain has quit IRC19:35
*** gmmaha has quit IRC19:35
*** hyakuhei has quit IRC19:35
*** irtermite has quit IRC19:35
*** igordcard has joined #openstack-kolla19:35
*** irtermite has joined #openstack-kolla19:36
*** sdake has joined #openstack-kolla19:36
sdakeyo19:36
sdakedid portdirect or kfox1111 respond to my query?19:36
*** gmmaha has joined #openstack-kolla19:36
*** markmcclain has joined #openstack-kolla19:36
sdakesrwilkers ?19:36
*** gmmaha has quit IRC19:36
*** gmmaha has joined #openstack-kolla19:36
sdake(you weren't on earlier when I asked)19:36
*** hyakuhei has joined #openstack-kolla19:36
*** JoseMello has joined #openstack-kolla19:38
*** harlowja has joined #openstack-kolla19:38
*** gmmaha has quit IRC19:38
*** rmart04 has joined #openstack-kolla19:39
*** kristian__ has quit IRC19:40
*** kristia__ has joined #openstack-kolla19:40
*** gmmaha has joined #openstack-kolla19:40
openstackgerritMerged openstack/kolla-ansible: Update the place of Watcher HAProxy by alphabet  https://review.openstack.org/41619419:41
*** kong_ has joined #openstack-kolla19:41
*** peterjenkins_ has joined #openstack-kolla19:41
*** alanmeadows has joined #openstack-kolla19:44
*** yee37913 has quit IRC19:48
*** yee379 has joined #openstack-kolla19:49
*** rmart04 has quit IRC19:50
*** serverascode has joined #openstack-kolla19:51
*** salv-orlando has joined #openstack-kolla19:53
*** NachoDuck has joined #openstack-kolla19:54
*** sdake has quit IRC19:54
*** msimonin has joined #openstack-kolla19:59
*** dave-mccowan has joined #openstack-kolla20:02
*** TxGirlGeek has quit IRC20:03
*** openstackgerrit has quit IRC20:03
*** haplo37_ has quit IRC20:09
*** openstackgerrit has joined #openstack-kolla20:09
openstackgerritMerged openstack/kolla: Enable zookeeper for CentOS/OracleLinux  https://review.openstack.org/40996520:09
*** adrian_otto has quit IRC20:11
*** haplo37_ has joined #openstack-kolla20:12
kfox1111alot to read. havnet caught up yet.20:14
kfox1111one thing for upstream mariadb vs kolla's.20:14
kfox1111it totally depends on whats tested.20:15
kfox1111and by whom.20:15
kfox1111rdo for example, tests/builds their stuff against their own mariadb, not what upstream mariadb.20:15
kfox1111ideally switching to upstream mariadb would "just work". but the devil's always in the details.20:15
kfox1111I'm guessing it will be an operator level decision. some will want upstream, some will want distro packages to always match.20:16
sbezverkkfox1111: do you have any recollection on kolla-kubernetes target date for 1.0 release?20:16
kfox1111I'm not aware of any date set.20:16
SamYaplesbezverk: hard deadline of april 1st20:16
sbezverkSamYaple: works for me ;-) I was just trying to see if all folks developing for kolla kubernetes are on the same page.20:19
kfox1111I think its something we shuld talk about at the ptg.20:20
kfox1111come up with a list of requirements we think 1.0 should cover.20:20
kfox1111that will inform when we think it might be possible to have it done by.20:20
sbezverkkfox1111: sure it is just I saw Feb mentioned by somebody and it kind of raised an alarm in my head as being not realistic..20:21
kfox1111for finishing 1.0?20:22
kfox1111heh.20:22
kfox1111like i said, depends on what the requirements are. should totally be able to launch an openstack by then.20:23
kfox1111"easily", not so much.20:23
sbezverkkfox1111: yeah 1.0 should cover openstack deployment preferably not using copy and paste command from the notepad ;-)20:27
* kfox1111 nods20:27
kfox1111inc0: chart per service is very corse graned.20:30
kfox1111and doesn't fit helm very well.20:30
kfox1111take nova for example.20:30
kfox1111seems streight foward at first.20:30
kfox1111nova-api, nova-conductor, nova-schedular, nova-compute.20:30
kfox1111sept it starts getting complicated when you realize you might want N different nova-compute instances.20:31
*** satyar has quit IRC20:31
kfox1111one for your cheep nodes, one for those with gpgpu's, etc.20:31
kfox1111for me, I want the flexibilty to upgrade just little bits at a time.20:32
kfox1111one of the hugest benifits of containerization.20:32
kfox1111and very important for openstack upgrades.20:32
kfox1111yes, you can take the big hammer aproach and just delete 'nova' launch some upgrades, and then recreate 'nova'.20:33
kfox1111that hardcodes the unideal world of non live upgrades.20:33
*** kristia__ has quit IRC20:34
*** kristian__ has joined #openstack-kolla20:34
kfox1111on the other end of the spectrum, you should be able to have a ci/cd system detect a new kolla helm package, test it, and push just that one update out to your live system with zero downtime.20:35
kfox1111should be possible today with our microserivice archetecture for those openstack components that support it.20:35
*** TxGirlGeek has joined #openstack-kolla20:37
SamYaplewow. wall of kfox1111 messages20:38
sbezverkSamYaple: but it makes perfect sense :-)20:39
SamYapleyea kfox1111 i agree20:39
SamYaplekfox1111: just to clarify though, nova isnt affecting the instances when it upgrades. in fact it shouldnt affect anything20:39
SamYapleother than api availability20:39
kfox1111SamYaple: yeah. but api availability is an uptime requirement in some places too.20:40
kfox1111just saying. a lot of effort is being put into live upgrades in a lot of projects.20:40
kfox1111if we go and try to hard code assumptions that they don't happen, it will be an even harder thing to solve.20:41
kfox1111I've heard most of compute kit's close at this point to being able to do full major rolling upgrades.20:43
SamYaplekfox1111: oh i agree. i think you may have mis understood me. even _today_ keystone has 100% zero-downtime upgrades20:43
kfox1111ah. nice. :)20:43
SamYaplekfox1111: i wouldnt count on it until Q20:43
kfox1111yeah. I think nova<->neutron might be the trickiest bit.20:44
kfox1111even just rolling upgrade support for minor upgrades is a huge thing though.20:45
inc0kfox1111, what I'm saying is, we should be able to support complex modifications and upgrades on chart level20:45
inc0what I would like at the end of the day is to potentially be able to split each service to different repo20:46
inc0so you can do helm install stable/openstack-nova20:46
SamYaplekfox1111: neutron has 2 more bits to make it full rolling upgrade20:46
SamYaplekfox1111: i think we'll see that in P releaes20:46
kfox1111inc0: with a service level chart, helm would have to do it, which it can't I think.20:47
inc0I hear what you're saying and I agree, but I think our approach with chart nova-bootstap will quickly become messy20:47
kfox1111inc0: with microservices, you have full flexibility there.20:47
* srwilkers reads20:47
inc0kfox1111, yeah, but we might want to draft change that needs to be made to enable it20:47
inc0it is valid use case20:48
srwilkersi agree with kfox1111 -- i wasnt aware of a 1.0 date set either20:48
kfox1111inc0: we can assemble service packages from microservices though.20:48
sbezverkkfox1111: +120:48
kfox1111so helm install stable/openstack-nova works for those with only one nova-compute need.20:48
kfox1111or that kind of thing.20:48
inc0as for dates, I was under impression that we're aiming for ocata, which is this month, potentially nex20:48
inc0t20:49
inc0so we can use PTG to plan for future20:49
kfox1111like  Isaid, it depends on definition of what 1.0 is. if its, you can spawn an openstack, 0.4.0 will be able to.20:49
sbezverkinc0: we are still debating API and then big thing is config management, how do you think it could be possible?20:49
inc0kfox1111, what I'm saying is, we should plan for changes in helm too20:50
kfox1111so if its still stable by end of ocata, we can retag it 1.0.20:50
kfox1111if its heavy orchestration, upgrades, etc, maybe not.20:50
kfox1111inc0: ah. yeah, we should work with helm to see if they can get the features needed in.20:50
kfox1111inc0: but relying on it happening is a seperate issue.20:50
inc0kfox1111, I was thinking 1.0 == just deploy working openstack20:50
inc0that won't hurt you20:50
kfox1111that may be possible in the timeframe.20:51
inc0yeah, no argument there, we do what we have to do20:51
kfox1111we still have service packages, and configmap stuff to figure out.20:51
inc0but nothing will happen unless we specify stuff we need20:51
kfox1111and dep handling.20:51
kfox1111and operators, of those will be needed....20:51
inc0so here's thing20:51
inc0I'd code the thing myself if we agree on what we want20:52
inc0and by we I mean kolla, att and maybe even SAP20:52
inc0and ofc helm20:52
kfox1111and... thats what the ptg kind of thing is for. :/20:52
inc0I'll be frank, I really don't want to have 3 different openstacks on k8s using helm20:52
kfox1111its been realy hard to get everyone on the same page without being in person. :/20:52
inc0yeah, and reason I even mentioned 1.0 date20:53
kfox1111+1 to that.20:53
kfox1111better to cooperate.20:53
inc0is because I would like us all to use PTG to draft this stuff20:53
inc0but not block our current path until we all draft it and agree on it20:53
kfox1111yeah20:54
kfox1111meeting next. bbiab.20:54
*** absubram has quit IRC20:56
*** eaguilar_ has quit IRC20:57
*** athomas has joined #openstack-kolla20:59
*** adrian_otto has joined #openstack-kolla21:02
*** JoseMello has quit IRC21:05
*** chas has joined #openstack-kolla21:11
*** matrohon has quit IRC21:11
*** gfidente has quit IRC21:12
*** jheroux has quit IRC21:14
*** jheroux has joined #openstack-kolla21:15
*** chas has quit IRC21:15
*** matrohon has joined #openstack-kolla21:24
*** sdake has joined #openstack-kolla21:25
*** Serlex has quit IRC21:29
*** athomas has quit IRC21:30
*** l4yerffeJ_ has joined #openstack-kolla21:34
*** l4yerffeJ has quit IRC21:34
*** dave-mccowan has quit IRC21:35
*** l4yerffeJ has joined #openstack-kolla21:35
*** l4yerffeJ__ has quit IRC21:35
*** Jeffrey4l has quit IRC21:35
*** Jeffrey4l has joined #openstack-kolla21:36
*** srwilkers has quit IRC21:41
jemcevoyinc0 Happy New Year21:42
inc0thank you good sir, happy new year to you21:48
*** sdake has quit IRC21:51
*** chas has joined #openstack-kolla21:51
*** msimonin1 has joined #openstack-kolla21:53
*** sdake has joined #openstack-kolla21:53
sdakehey peeps21:53
sdakehey peeps21:53
*** chas has quit IRC21:56
*** msimonin has quit IRC21:56
*** matrohon has quit IRC22:01
jemcevoyI had to shutdown my 4 node cluster... Is there a procedure for restarting it?  and when to run the kolla-ansible mariadb_recovery -i ~/inventory/multinode22:06
jemcevoyinc0:  Do I just ipmitool power on all 4 nodes?22:07
jemcevoyI tried before and this was the result: http://paste.openstack.org/show/593796/22:08
jemcevoyso I resorted to a cobbler kickstart22:08
*** jrobinson has joined #openstack-kolla22:11
*** jtriley has quit IRC22:11
inc0mariadb_recovery is for full failure22:12
inc0if you gracefully shut down mariadb, it should be ok22:12
*** chas has joined #openstack-kolla22:12
inc0but SamYaple can help here - I'm not good at this one22:12
*** huikang has quit IRC22:13
*** matrohon has joined #openstack-kolla22:15
*** chas has quit IRC22:16
kfox1111back.22:16
kfox1111https://review.openstack.org/#/c/416069/ is ready for a second.22:17
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Visual feedback  https://review.openstack.org/41607722:20
sbezverkkfox1111: how do you want to move forward with microservice conversion?22:21
bmacethis may end up being a silly question.  i'm just getting up to speed on the k8s codebase, but is there any rhyme or reason for the file name differences in helm/microservice/<service>/templates/*.. seems like a mix between dashes and underscores in the filenames.22:22
kfox1111sbezverk: I was thinking there were a couple of aproaches.22:22
*** fguillot has quit IRC22:22
kfox1111one would be to do all the templates that shared a single common_X template in a ps.22:23
kfox1111less merge conflicts that way, but potentially less opertunity to generalize some code.22:23
kfox1111the other way would be per variable kind of thing, so like "image_pull_policy" for example, could be made a function, looked up in vals and updated everywhere. and then do the next thing.22:24
kfox1111probably much more merge conflicts that way though.22:24
kfox1111so, maybe the first option might work better?22:25
inc0kfox1111, ad https://review.openstack.org/#/c/416069/15 do we store these values somewhere outside of yaml?22:26
inc0so here's my thingking22:26
inc0in ansible we have multiple layers of config, each will override latter22:26
kfox1111inc0: all values come from values, or the users specified values.22:26
inc0at the end of the day, we have single flat list of all things overriden22:26
inc0here, can we store it in k8s22:27
inc0?22:27
inc0like a configmap?22:27
kfox1111inc0: similar to the way ansible does it, but can be overridden at a per package level too.22:27
kfox1111so more flexible.22:27
kfox1111there's two different things too...22:27
kfox1111configmap data, which this isn't about.22:27
kfox1111or the config of the k8s objects themselves.22:27
kfox1111which this is about.22:27
*** salv-orlando has quit IRC22:28
kfox1111so like, a config option to attach an extra logging container to each pod.22:28
inc0ok, well in ansible case, they both kinda overlap, here it might not22:28
kfox1111what image name to use for the container, etc.22:28
kfox1111yeah. different use case for now.22:28
inc0yeah, it's resource declaration phase22:29
kfox1111right.22:29
inc0so here is why I'm thinking about it22:29
inc0if we'd have signle source of truth available somewhere (configmap probably), we could use it to both configure deployment, pass values between charts and create config files22:30
*** sdake has quit IRC22:30
sbezverkkfox1111: not sure if first approach is scalabale22:30
kfox1111sbezverk: yeah. depends on how fast we can crank through it too.22:30
sbezverkunless you insist to do everything yourself ;-)22:31
kfox1111I won't insist. :) but I am going to try and spend as much time on it as I can for the next few days.22:31
kfox1111really wana get it finished. ;)22:32
SamYaplekfox1111: its easy if you stop caring and start coding!22:32
*** salv-orlando has joined #openstack-kolla22:32
sbezverkkfox1111: ok22:32
kfox1111SamYaple: easier yet if meetings stop. ;)22:32
*** salv-orlando has quit IRC22:32
*** salv-orlando has joined #openstack-kolla22:33
SamYaplekfox1111: i think my previous statement still applies!22:33
sbezverkinc0: could you expand a bit your idea with config map?22:33
kfox1111configmaps arn't readable by helm.22:33
inc0ok, so I'm still thinking on having separate services from each other22:33
openstackgerritMerged openstack/kolla-kubernetes: kolla-common value get functions  https://review.openstack.org/41606922:34
kfox1111I think you get to what your thinking though, with:22:34
inc0kfox1111, helm has no way to access configmaps?22:34
kfox1111 * microservice packages with the new api,22:34
kfox1111 * configmap packages22:34
kfox1111 * service packages22:34
kfox1111user provided config file, ala values.yaml22:35
sbezverkkfox1111: what is configmap pakcages?22:35
kfox1111they simply helm install service-package.tgz --values mycloud.yaml and off they go.22:35
inc0so here is what I was thinking22:35
kfox1111sbezverk: one possible idea. was tryhing to avoid talking about config stuff till after 0.4.0....22:35
inc0you create your config files22:36
sbezverkkfox1111: got it, placeholder for opensatck gen config22:36
inc0then you deploy mariadb22:36
inc0helm install mariadb22:36
inc0mariadb installed will create couple of configmaps/secrets22:36
inc0how to access it and so on22:36
inc0helm install nova22:37
kfox1111how do you reuse existing configmaps/secrets?22:37
inc0it will run bootsrap pod which will access configmaps created by amriadb, bootstrap nova and create config string and put it into configmap22:37
kfox1111what is "it"?22:38
inc0nova db username, nova db password22:38
kfox1111ie, what is orchestrating.22:38
inc0you mean so it wont re-bootsrap later?22:39
kfox1111helm/kubernetes have no ordering garantees.22:39
inc0but init container solves that22:39
SamYaplenot for upgrades22:39
SamYaple(if i partially understand things)22:40
kfox1111an init container is not for bootstrapping a system, but bootstrapping a pod.22:40
inc0yeah, but system is sum of pods as far as I understand22:40
sbezverkkfox1111: I think they want to use init contianer to bootstrap service and including dependecies22:40
kfox1111assertion still holds.22:40
inc0don't start nova-scheduler until nova-bootstrap is done22:40
kfox1111you bootstrap a system once, not n times.22:41
kfox1111on most services, you do n of them.22:41
inc0and nova-bootsrap will wait till mariadb-configmaps it needs exists22:41
kfox1111so.... theres a different way to do it, which we have some prototypes of.22:41
inc0it's a matter of locks22:41
kfox1111inc0: https://review.openstack.org/#/c/412239/22:41
inc0init container is nothing more than a locking mechanism22:41
inc0kfox1111, I'll need some time to understand this blob of code22:42
*** jheroux has quit IRC22:42
kfox1111inc0: k. please do.22:43
inc0what I'm esentially saying is deployments (including upgrades if we're careful) can be done by lock management22:43
kfox1111but, tldr;22:43
inc0and configmaps can act as locks22:43
kfox1111helm service package has deps for all the microservices:22:43
kfox1111https://review.openstack.org/#/c/412239/21/helm/service/neutron/requirements.yaml22:43
kfox1111each microservice has a kubernetes-entrypoint init-contiainer and blocks until its ready:22:44
kfox1111https://review.openstack.org/#/c/412239/21/helm/all_values.yaml22:44
kfox1111you helm install the service package,22:44
kfox1111helm installs all the microservices,22:44
kfox1111and they start up in their own dep order.22:44
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Fix helm charts that are broken behind a proxy  https://review.openstack.org/41636622:44
inc0yeah, that's essentially what I was saying22:45
kfox1111and unfortunately, you can't see the green jenkins test, as the recent merges reverted the tests.... :)22:45
inc0only instead of locking on keystone running, lock on existance of certain configmaps22:45
openstackgerritMarcus Williams proposed openstack/kolla-ansible: Work in Progress: Add ODL to kolla-ansible  https://review.openstack.org/41636722:46
inc0only bootstrap jobs will care about other services22:46
inc0or even not, because they will lock on configmaps these other services make22:46
kfox1111inc0: pods already block if configmaps don't exist.22:46
kfox1111inc0: kubernetes-entrypoint provides blocking on jobs finishing successfully and services existing.22:47
inc0yeah, but if we overhaul using configmaps for all the cross-service communication22:47
inc0we can have services completely outside of openstack22:47
inc0k8s*22:47
openstackgerritMarcus Williams proposed openstack/kolla: Work in Progress: Draft add ODL to kolla  https://review.openstack.org/41636922:47
inc0like totally external mariadb22:47
kfox1111you can now too.22:47
kfox1111svc's an point directly to external things,22:47
kfox1111and so can the configmap data itself.22:47
kfox1111works today. :)22:48
inc0I'm just strugling with idea why we have microservices directory tbh22:48
kfox1111because if you just had nova,22:48
kfox1111it wouldn't work when you have gpgpu nodes, and regular ones,22:48
kfox1111just want to upgrade just nova-api without touching anything else,22:49
kfox1111orchestrate with ansible/chef or heat,22:49
kfox1111etc.22:49
inc0I don't understand why it won't work?22:49
kfox1111why one super package wont work?22:49
inc0one superpackage per service22:49
kfox1111with no microservices?22:49
inc0helm install nova -set nova.enable_spiceproxy=false22:49
inc0microservices as resources rather than charts22:50
kfox1111helm's packages are all or nothing. upgrade them all, or not. delete them all, or not.22:50
kfox11111 package -> 1 release.22:50
kfox1111a release gets manipulated all at once.22:50
inc0ok, but helm guys mentioned people worked around it with templating right?22:50
inc0it's ugly, but maybe we could amke it easier in helm22:51
kfox1111maybe. but I doubht that will be a thing for at least 6 months.22:51
inc0let me guess, this is whole att-comdev vs kolla-k8s difference discussion right?22:52
kfox1111what your asking is?22:52
*** adrian_otto has quit IRC22:52
sbezverkinc0: other than you do not like to have microservice in its own directory, what is the technical reason?22:53
sbezverkI mean like/do not like it not enough really22:53
inc0so I'm trying to understand what would need to be done for helm to accept kolla's charts to official helm repo22:54
kfox1111your suggesting the aproach taken by att-comdev. make it inflexible but easy to deploy for now. then, maybe hopefully some day they will make helm make it easier to do more fine grained things.22:54
kfox1111I'm worried about day 2 operations. how do I upgrade it. how do I kick just one service. etc.22:54
inc0I want configurable and easy to install22:54
kfox1111inc0: so, theres a lot of things to discuss down that road.22:54
inc0totally22:55
inc0I'm just trying to do just this - discuss;)22:55
inc0how to find middle ground - easy and robust22:55
inc0that's the goal22:55
kfox1111inc0: configurable and easy to install's basically every deployment tool. what does kolla-kubernetes bring to the table there?22:55
sbezverkinc0: are you sure that their charts will be accepted as is into official helm chart repo?22:55
inc0well, it's deployment tool which uses specific set of underlying tech22:56
sbezverkthey do funny things as well22:56
kfox1111I do want to get us there, but most distro's focus on making it easy to install on day one, then cause pain on day 2. and make you cry when you go and upgrade it.22:56
inc0sbezverk, I'm not saying we should do what they're doing22:56
kfox1111then the distro gets stuck for years trying to figure out how to do upgrades.22:56
inc0but if our chart has issues in being accepted to helm, then we're doign something helm community doesn't like22:56
inc0and that's bad22:56
kfox1111I've been trying for the other aproach. focus on day 2 life. and upgrades.22:56
kfox1111and day one falls out from there.22:57
inc0kfox1111, upgrades, reconfigure and everything are a must22:57
sbezverkinc0: nobody from helm community ever commented about our charts22:57
inc0I mean if kolla-k8s can't be both, why even use it?22:57
inc0kolla ansible is both easy and day-2 robust22:57
kfox1111sbezverk: +122:57
inc0sbezverk, ok, let's ask them to comment then22:58
kfox1111so, more issues with upstreaming:22:58
inc0guys, I'm not against you here22:58
kfox1111governence: how does that work with openstack?22:58
SamYapleopenstack-ansible pushes to ansible-galaxy22:58
*** sdake has joined #openstack-kolla22:58
kfox1111their repo doesn't handle building multiple packages well at all. needs a lot of work.22:58
kfox1111inc0: please don't feel like we're piling up on you, but we've been down this discussion, a LOT... :/22:59
sbezverkinc0: so far we heard negative comments ONLY from one guy22:59
inc0sbezverk, so I've heard from multiple22:59
SamYapleyea screw that guy22:59
inc0that's why I'm worried22:59
sbezverkbut it makes sense everybody want to sell their shoes ;-)23:00
inc0one guy is particulary vocal about them, but it's not just him:)23:00
sbezverkinc0: subcharts approach is perfectly native for helm23:01
* portdirect back at keyboard - sees loads of scrollback23:01
sbezverkso I do not see any architectural issues23:02
SamYapleportdirect: i made some comments too!23:02
portdirectSamYaple: ;-)23:02
sbezverkhelm is relatively new project, mostly used to deploy simple things23:02
sbezverknow more complex req required more complex solutions23:03
sdakeinc0 kfox1111 portdirect sbezverk srwilker- re schedule - here is the ocata schedule: https://releases.openstack.org/ocata/schedule.html23:03
sdakethat schedule there points out the PTG is feb 20-24th and trailing cycle deadline is March 8th23:04
sdakeone thing to keep in mind is kolla-kubernetes is on a release independent schedule 1.0.0 != necessarily ocata23:04
sdakeor aligned with the ocata schedule23:04
sdakewhat we want out of 1.0.0 is something that is upgradeable in the field with 2.0.0 in 6 months23:05
sdakewhen pike releases23:05
sdakethe minimum viable 1.0.0 includes an upgrade path - which means the implementation needs to be fairly complete23:06
sdakeand of course subject to refactoring23:06
sdakeit doesn't have to upgrade at 1.0.023:06
sdakeit has to be upgradeable from 1.0.023:06
portdirectinc0: (and I'm just answering things as I see them, not voicing an option on right/wrong etc) configmaps can be used as a very effective locking mechanism - if they don;t exist the pod doesn't launch - i used to use secrets for this very purpose.23:06
sdakeI further think helm installing a million microservices works well for big operations but poorly for small/medium shops (100-1k employees)23:07
*** matrohon has quit IRC23:09
sdakeinc0 i find it difficult to believe you ahve heard there is issue with kolla-kubernetes dev since you have been on pto for 5 weeks23:09
sdakeinc0 things are going very well from my POV23:09
sdakein 1 month we have implemented ALL microservices23:09
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Fix helm charts that are broken behind a proxy  https://review.openstack.org/41636623:09
*** kristian__ has quit IRC23:10
SamYaplei mean im not involved in k8s at all and I have seen the issues even in the past two weeks.....23:10
*** kristian__ has joined #openstack-kolla23:10
sdakeSamYaple which issues in particular?  you mean startup issues?23:12
*** srwilkers has joined #openstack-kolla23:12
sdakesrwilkers schedule here: https://releases.openstack.org/ocata/schedule.html23:12
sdakemarch 8th is when kolla-ansible releases23:13
inc0sdake, well, I was on pto but I wasn't totally away23:13
*** openstack has joined #openstack-kolla23:16
inc0you said that's no longer the case and said it as a matter of fact, well, I'd like to see when it was discussed23:17
inc0because *that is not what we agreed on*23:17
sdakeinc0 new decisions can be made - i was not at most of the summit sessions23:18
inc0well, I wasn't during new decision making, does it make it less or more valid decision23:19
inc0?23:19
inc0if you weren't on session, that doesn't mean decision we've done there is invalid23:20
sdakeinc0 i htink we should have a proper vote of the kolla-kubernetes team as it stands now to determine the 1.0.0 target23:20
sdakesince the core reviewers are responsible ultimately for the quality of the work and future success of kolla-kubernetes23:20
inc0so instead of vote on ML, let's make this decision on PTG23:22
inc0as most of core team will be there23:22
sbezverkinc0: i will not but happy to express my opinion now or later via ml23:23
inc0shame sbezverk :(23:24
sdakeinc0 ok well 1.0.0 can't be released before PTG if that is the case - meaning at minimum 1.0.0 is early March23:24
sdakeinc0 imo all votes should be on mailing list23:24
sdakeinc0 votes at ptgs or summits don't include the core review team23:24
sdakeer all of em anyway23:24
sdakei have to say feature freeze by feb 26 is too agressive - pretty sure eveyrone has been operating under a model where we would release when it was ready - because that is what I told them based upon advice from Doug23:25
sdakeDoug posted the mail on the mailing list23:25
inc0ok, so here's my take23:26
inc0we're not release bound23:26
inc0but I want to *aim* for Ocata23:26
inc0not follow it to the letter23:27
inc0and Ocata is Feb 2023:27
*** lamt has quit IRC23:27
*** rhallisey has quit IRC23:28
*** rhallisey has joined #openstack-kolla23:28
sdakeactually march 6-10 for kolla-ansible and kolla deliverables23:29
sdakeptg is feb2023:29
*** saneax-_-|AFK is now known as saneax23:29
sdakethe release traling happens after the PTG23:29
openstackgerritJoshua Harlow proposed openstack/kolla: Allow for image additions archives (as well as plugins)  https://review.openstack.org/41153423:30
portdirectinc0: If possible it would be great to have a vote immediately following ptg, or even during it if notes are on etherpad23:30
openstackgerritMarcus Williams proposed openstack/kolla: Work in Progress: Add OpenDaylight Container to kolla  https://review.openstack.org/41636923:30
inc0oh btw portdirect sbezverk and others who might not be on ptg23:31
inc0we got green light to hold teleconference23:31
inc0so we will23:31
inc0you can all call into sessions23:31
sbezverkinc0: !!!!! Great !!!!!!23:31
portdirectsweet!23:31
srwilkersthats fantastic23:32
portdirect:D no need to put on trousers is always a win23:32
portdirect(though I think others in my office may not be that happy about that...)23:32
*** vhosakot has quit IRC23:32
sdakeportdirect that depends on a whole lotta variables23:33
kfox1111awesome. :)23:33
portdirectsdake: I think it's my turn to say 'I dont wanna know' :)23:33
sdakeportdirect you got it ;)23:34
kfox1111inc0: to be fair, I wasn't at the summit either,23:34
kfox1111nore probably were the majority of the kolla-kubernetes cores now?23:35
kfox1111so what was "agreed" to there, is a guideline. we're doing our best, but not bound to it.23:35
sbezverkkfox1111: I was there but overwhelming amount of info, especially new, made it impossible for me to realize it23:36
sbezverkafter spending few month now, it is much easier to navigate in this info and take right decision..23:37
kfox1111yeah. makes sense.23:37
kfox1111its a jurney the community makes together.23:37
kfox1111and as we're release independent at the moment,23:37
kfox1111my vote would be to reach 1.0 when its ready, not at some arbitrary date.23:38
kfox1111then after that, we can go to date based.23:38
kfox1111that does require setting a definition of "ready" though.23:38
portdirectkfox1111: I think the majority of current cores (at least sdake, sbezverk, srwilkers, inc0, rhallisey and me were) but many of use were not at the time23:38
portdirectkfox1111: +1 to release when ready23:39
portdirectkfox1111: (stayong out of ready discussion for now if pos - it's 11:30pm here..)23:39
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Fix helm charts that are broken behind a proxy  https://review.openstack.org/41636623:39
kfox1111portdirect: ah. didn't think you made it. glad you did. :)23:40
kfox1111portdirect: yeah. I'm staying out of the ready discussion too. that I think is a much bigger conversation. :)23:40
portdirectyeah - my side of the pond :)23:40
srwilkersyeah. last minute approval to go -- glad i was able to23:40
kfox1111inc0: oh.... btw....23:40
kfox1111inc0: the patch Mark Giles just posted?23:40
kfox1111it is needed only because we put in a workaround to some broken containers on the hub.23:41
mgileskfox1111 yes23:41
kfox1111can we get those updated please so the workaroundd can be dropped?23:41
mgilesalthough we may need something else similar in other cases23:41
kfox1111then the patch to fix the workaround won't be needed.23:41
*** msimonin1 has quit IRC23:41
*** msimonin has joined #openstack-kolla23:41
*** msimonin has quit IRC23:41
*** msimonin has joined #openstack-kolla23:41
mgilesbut if we can get the containers updated, I'm happy to abandon the ps23:42
kfox1111mgiles: the docker ideal is that the containers should be fully baked. they should never need to reach out to download stuff.23:42
*** msimonin has quit IRC23:42
kfox1111otherwise, they aren't tested. :/23:42
mgileskfox1111  I hope that's true  :-)23:42
SamYaplekfox1111: hmm im not sure i agree with that "the docker ideal"23:42
SamYapleor wait. we arent talking about configs are we?23:43
kfox1111mgiles: should be able to test/enforce that. :) if we disable internet access in the gate.23:43
portdirectkfox1111: I could keep a rolling set of containers from master up on the docker hub?23:43
kfox1111SamYaple: not talking about configs. just programs.23:43
kfox1111portdirect: would love to see that. hard to do at the moment. :/23:43
portdirectkfox1111: how come?23:44
kfox1111portdirect: alternately, maybe a tarball with them all in there for easy loading.23:44
kfox1111portdirect: short anser is, they take forever to upload, as they upload the same underlayers over and over due to the way the hub is designed. :/23:44
portdirect(I have more bandwidth than normal here...)23:44
openstackgerritMarcus Williams proposed openstack/kolla-ansible: Work in Progress: Add OpenDaylight to kolla-ansible  https://review.openstack.org/41636723:44
kfox1111I think the last number I heard was something like 10 hours to upload the set?23:45
portdirectalso the newer docker client regognises shared layers - takes about 2-3 hours for me to upload23:45
portdirectI have 1.2gbs average upload23:46
*** rhallisey has quit IRC23:46
kfox1111for the whole set?23:46
portdirectyes23:46
kfox1111nice. :)23:46
portdirecteasy for me to set up a cron job and push to a repo fo use to use for devel23:47
kfox1111nightlies wouldn't be too bad then. I wonder if we could replicate that in zuul.23:47
sdakelord knows I've tried kfox1111 :)23:49
sdakethe gate for kolla-kubernetes should be buildilng images fresh rather then pulling from hub imo :)23:49
*** TxGirlGeek has quit IRC23:50
kfox1111I think it shoudl do both. but a cached version, not the hub.23:50
sbezverkmgiles: could you quick explain what will use proxy setting when user gets created?23:50
kfox1111so we can ensure the current release is deployable.23:50
kfox1111sbezverk: its for the pip upgrade. :/23:50
mgilessbezverk The container does a pip install23:50
kfox1111sbezverk: if we can get the hub images upgraded, we can drop the pip install.23:50
mgilesand then I can abandon the ps  :)23:51
sbezverkmgiles: got it23:51
portdirectmgiles: that would be great - i really dont like it23:51
mgilesbut without one or the other, I'm pretty well blocked23:51
srwilkersid like to see this resolved elsewhere. not a fan of seeing proxy stuff thrown in the helm charts23:51
sdakemgiles which image23:51
portdirectmgiles: but like locking you out of development even more...23:51
kfox1111inc0: what do you think about getting the hub image fixed? it has a faulty openstack client.23:51
mgilesportdirect  Have to agree.  I didn't love it either23:51
portdirect*DONT - whoops!23:52
portdirectyou're cool :)23:52
mgilesportdirect Wow - wondered what I had done to you  :)23:52
sdakeok - early knock off day for me - not enough sleep23:52
sdakeand hungree23:52
portdirectmgiles: nothing yet - but i think i just gave you a motive :)23:52
sdakei'll probably be back later23:52
openstackgerritMarcus Williams proposed openstack/kolla-ansible: [WIP] Add OpenDaylight to kolla-ansible  https://review.openstack.org/41636723:53
openstackgerritMarcus Williams proposed openstack/kolla: [WIP] Add OpenDaylight Container to kolla  https://review.openstack.org/41636923:53
portdirectkfox1111: who has push access to docker hub?23:54
kfox1111not sure who all does. sbezverk, sdake, and I do for sure.  Ithink inc0 too.23:54
kfox1111I'm relucant to change any existing container though, as I don't know the rules.23:55
sdakekfox1111 typically someone builds new images when a new tag is created23:55
portdirectroger - who does Jeffrey4l ?23:55
sdakekfox1111 other then that, they are poorly maintained23:55
sdakeportdirect coolsvap was maintaining them23:55
sdakein a very limited way23:56
kfox1111what about when the tag doesn't need updating, but just a rebuild is needed?23:56
kfox1111can we just do that?23:56
kfox1111or alternately, when will 3.0.2 show up?23:56
portdirectwhat do you think the best way to get this fixed - it was a bit of an anyoance before - but now its blocking devs its getting a bit ugly23:56
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes: Updated from global requirements  https://review.openstack.org/41619923:57
kfox1111portdirect: yeah. :/23:57
sdakekfox1111 3.0.2 - no idea when that will show up - the last tag of the z stream for kolla was Nov 1723:57
kfox1111can we cut one at any time?23:58
sdakewe are due for another zstream shortly23:58
sbezverkwe can easily substitute existing image with the same tag but new content, it is a bit dangerous but maybe less dangerous that puyshing proxy setting everywhere23:58
sdakekfox1111 depends when the release liason wants to pull the trigger on the release (who is Jeffrey4l )23:58
sdakebut then someone has to build them and push them to the hub23:58
sdakei spoke to Jeffrey4l prior to break about releasing new images23:58
sdakehe said its in progress23:58
kfox1111in progress? cool.23:59
sdakei think something with the gates is blocking23:59
kfox1111so, within the next day or two?23:59
sdakekfox1111 no idea23:59
sdakekfox1111 given the pace of z streams i find it unlikely to happen on that time scale23:59
kfox1111:/23:59
kfox1111hmm....23:59
sdakethis is a problem for inc0 to sort out unfortunately23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!