Thursday, 2014-10-16

*** cody-somerville has quit IRC00:01
*** ramishra has joined #tripleo00:10
*** sdake_ has quit IRC00:11
openstackgerritSteven Dake proposed a change to stackforge/kolla: Move nova.conf script to nova-api directory  https://review.openstack.org/12879400:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Fix log settings for nova.conf  https://review.openstack.org/12879500:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Make nova.conf copies in each nova-controller container  https://review.openstack.org/12879600:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Remove unnecessary nova-controller which has no associated binary  https://review.openstack.org/12879700:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Have Dockerfile add nova.conf and have start.sh run nova.conf  https://review.openstack.org/12879800:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Make nova-controller/*/start.sh and nova-controller/*/nova.conf +x  https://review.openstack.org/12879900:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Import from fedora-rdo-base rather then fedora-rdo-nova-controller  https://review.openstack.org/12880000:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Use name nova-config.sh instead of nova.conf  https://review.openstack.org/12880100:28
openstackgerritSteven Dake proposed a change to stackforge/kolla: Nova has an undeclared dependency on python-cinderclient  https://review.openstack.org/12880200:29
*** rainya has quit IRC00:29
*** otoolee has quit IRC00:31
*** otoolee has joined #tripleo00:42
*** cody-somerville has joined #tripleo00:43
*** cody-somerville has quit IRC00:44
*** ramishra has quit IRC00:51
*** ramishra has joined #tripleo00:52
*** ramishra has quit IRC01:02
*** cwolferh_ has quit IRC01:07
*** ramishra has joined #tripleo01:32
*** ramishra has quit IRC01:42
*** ramishra has joined #tripleo02:02
*** pradeep has joined #tripleo02:07
*** nosnos has joined #tripleo02:20
*** pradeep1 has joined #tripleo02:23
*** pradeep has quit IRC02:23
SpamapShm02:50
SpamapSno_proxy=127.0.0.1 break=after-error PYPI_MIRROR_URL=http://127.0.0.1/mirrors/pypi/ SEED_DIB_EXTRA_ARGS="pypi" DIB_NO_PYPI_PIP=1 scripts/devtest_seed.sh02:51
SpamapSI'd expect this to use my local mirror02:51
SpamapSwhich does have pip... but I get02:51
SpamapS+ python /tmp/get-pip.py02:51
SpamapSDownloading/unpacking pip http://127.0.0.1/mirrors/pypi/pip/ uses an insecure transport scheme (http). Consider using https if 127.0.0.1 has it available Could not find any downloads that satisfy the requirement pip02:51
SpamapSCleaning up...02:51
StevenKexport PYPI_MIRROR_URL=http://silver/pypi/simple02:53
StevenKexport DIB_COMMON_ELEMENTS="pypi common-venv stackuser"02:53
StevenKIs what I use02:53
StevenKSpamapS: Ah ha, are you missing a simple ?02:53
SpamapSdoohh02:54
*** cwolferh_ has joined #tripleo02:54
StevenKSpamapS: That tripped me up too :-)02:55
*** ramishra_ has joined #tripleo02:58
*** jerryz has joined #tripleo02:59
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/138140803:00
*** untriaged-bot has quit IRC03:00
uvirtbotLaunchpad bug 1381408 in diskimage-builder ""visudo" always got error when building overcloud image" [Undecided,New]03:00
*** ramishra has quit IRC03:00
*** spzala has quit IRC03:07
*** rhallisey has quit IRC03:09
*** ramishra_ has quit IRC03:10
*** noslzzp has quit IRC03:11
SpamapSStevenK: ok, I had the wrong thing in my mirror's Alias. :-P03:14
SpamapSStevenK: and I was missing simple. :-P03:14
*** pradeep has joined #tripleo03:21
*** pradeep1 has quit IRC03:22
*** pradeep1 has joined #tripleo03:24
*** pradeep has quit IRC03:24
*** nosnos has quit IRC03:26
*** nosnos has joined #tripleo03:27
*** pradeep1 has quit IRC03:29
*** nosnos has quit IRC03:31
*** ramishra has joined #tripleo03:32
tchaypoSo hp's abuse team have just /away03:38
tchaypowell done jaames03:38
tchaypolicenced to operate a car, cannot operate irc03:38
tchayposo as i was saying, hp's abuse team have notified us that the undercloud control node is resolving DNS for randoms on the internet, and this makes them sad03:39
* greghaynes wonders if rh is also doing this03:41
*** pradeep has joined #tripleo03:44
StevenKtchaypo: Speaking of, I need VPN to talk to the bastions, don't I?03:45
greghaynesnope03:45
tchaypoStevenK: nope03:47
*** cwolferh_ has quit IRC03:47
StevenKHmm03:47
tchaypogreghaynes: II'm *guessing* that RH has iptables rules that block the inbound requests03:47
greghaynesyea, hopefully03:47
StevenKHave you tried?03:47
StevenKSo, uh, how do I log into these things?03:48
tchaypowhich is what I've dropped in place for now, as it's a lot faster than tweaking dnsmasq to listen on all the other interfaces03:48
*** yamahata has quit IRC03:48
greghaynesssh stack@bastion.hp2.tripleo.org03:48
tchaypoStevenK: but only after someone deploys your keys on the box03:48
tchaypoI've got a to-do to fix the deployment script on hp203:49
tchayposo it's not likely to have dropped your key03:49
StevenKRight, so it is stack. I get permission denied, so I'll just wait :-)03:49
tchaypono, you'll need to point me at your key so i can add it by hand, since I haven't fixed the script yet03:49
* tchaypo fetches from gerrit03:50
StevenKtchaypo: https://launchpad.net/~stevenk/+sshkeys03:50
SpamapSssh-import-id stevenk03:50
SpamapSOh I forgot to follow-up and do the steps to get your keys on03:51
StevenKHere I was being naive and thinking since my branch has landed, everything would magically work :-P03:51
tchaypotry now03:52
greghaynesStevenK: You think were the types to automate that sort of thing?03:52
StevenKHaha03:52
tchaypoif someone else can give you access to the gSheet that'd be handy, I have to run to the supermarket03:52
StevenKtchaypo: Works for hp2, not hp103:52
* tchaypo is now nursing a wisdom teeth extractee03:52
tchaypoi *think* hp1 is fixed and it should be applied there soon. If it's not I can handle it when i come back.03:53
tchaypoif you're looking for something to do03:53
tchaypothe next thing I was planning to do was run your new register-nodes over /home/shared/nodes.json into the undercloud, then deploy-ci-overcloud again03:54
tchaypogive me time to run to the shops and back and i can walk through that with you, if you don't mind03:55
*** rbrady has quit IRC04:08
StevenKgreghaynes: You said in https://review.openstack.org/126456 LGTM, but then you -1'd?04:13
greghaynesStevenK: yes, id like to deprecate the original behavior04:14
greghaynesso print a warning or somesuch04:14
greghaynesbecause then we can remove it04:15
StevenKgreghaynes: Sure. In a different branch, since that change is about PKI setup only04:15
StevenKdevtest_overcloud still calls -s on init-keystone04:15
greghaynesactually, I should comment on the commit msg too. Currently we init the pki using another method so there isnt an incoming change that effects this04:16
greghaynesit currently just checks if the certs already exist so its essentially a no-op04:16
jerryztchaypo: if i have a bunch of real baremetals, can i still make use of devtest script to deploy them? and how? change testenv.json?04:17
StevenKgreghaynes: --public for init-keystone sets http://04:17
StevenKSo we need a Better Way04:18
greghaynesStevenK: Yes, im referring to the --pki-setup bits04:19
StevenKgreghaynes: Oh. I'm only touching --ssl/--public in that patch because I add another mutually exclusive group04:20
StevenKgreghaynes: And the PKI setup init-keystone does just calls keystone-manage pki_setup, is that clever enough to check for existing certs?04:21
greghaynesStevenK: http://logs.openstack.org/56/126456/1/check-tripleo/check-tripleo-ironic-undercloud-precise-nonha/5d9fa00/console.html.gz#_2014-10-07_03_02_46_37604:21
greghaynesStevenK: Yep04:21
*** nosnos has joined #tripleo04:23
StevenKjerryz: Use --nodes for devtest.sh04:23
SpamapSI'm just doing that right now actually. :)04:24
SpamapSand just discovered the mac's I have are all wrong04:24
* SpamapS ponders trying to use ilo pm driver04:24
*** ramishra has quit IRC04:24
greghaynesStevenK: basically, the --pki-setup bits id like to remove because they really dont make sense there, but we have to wait a cycle after deprecation so deprecating ASAP seems ideal :)04:25
jerryzStevenK: thanks04:29
StevenKgreghaynes: Huh. So we don't need the command line options04:30
StevenKgreghaynes: Or we do, if we flip the default to False04:31
greghaynesYea, we could just remove it alltogether if we didnt care about backwards compat04:31
StevenKExcept we decided we do care about it04:32
greghaynesyep, so deprecate!04:32
*** rushiagr_away is now known as rushiagr04:37
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Teach init-keystone about PKI setup  https://review.openstack.org/12645604:38
StevenKgreghaynes: Like ^?04:38
StevenKgreghaynes: And then we can set --no-pki-setup for devtest once that lands and is released04:38
greghaynesyaya, deprecate all the things!04:38
*** yamahata has joined #tripleo04:40
*** dshulyak_ has joined #tripleo04:48
* StevenK fixes the commit message.04:49
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Deprecate PKI setup in init-keystone  https://review.openstack.org/12645604:56
tchaypojerryz: yeah, as other people said, nodes.json05:03
tchaypospamaps: where are your thingies?05:04
tchaypoI'm doing the same thing in hp2, except that I've just fixed 70% of my mac addresses05:04
tchaypojerryz: if you can't figure out the format of nodes.json i have a sample i can give you05:04
tchaypojerryz: but I'm pretty sure the docs have a sample05:04
tchaypoStevenK: greghaynes: lifeless: it seems t me that the issue with dnsmasq being available on the public interface is a special case of https://bugs.launchpad.net/tripleo/+bug/118806705:07
uvirtbotLaunchpad bug 1188067 in tripleo "* listening services available on all addresses" [High,Triaged]05:07
tchayposo I'm thinking of glomming onto that bug and raising the priority05:07
jerryztchaypo: similar to testenv.json?05:07
jerryztchaypo: please also send me the sample just for reference.05:08
tchaypomaking a note of the work we've already done to allow the tripleo services to be available on a subset of interfaces, and then tackling dnsmasq. I'm not sure if we can handle dnsmasq the same way though - I don't think we can put it behind haproxy..05:08
*** otoolee has quit IRC05:09
tchaypojerryz: I'd really really like to just point you at the bit in the docs where there's a handy sample05:12
tchaypojerryz: and indeed, devtest_testenv.sh says that "The schema for nodes can be found in the devtest_setup documentation."05:12
tchaypojerryz: as far as I can tell that's a lie.05:12
StevenKIt isn't05:12
StevenKA list of node metadata. Each node has "memory" in K, "cpu" in05:13
StevenK##    threads, "arch" (one of i386/amd64/etc), "disk" in GB, mac, a list of05:13
StevenK##    MAC addresses for the node and "pm_type", "pm_user", "pm_addr", and05:13
StevenK##    "pm_password" fields.05:13
tchaypothat's not a schema.05:13
tchayponor is it a useful sample.05:13
tchaypoit's a set of instructions for creating a sample05:13
StevenKhttp://docs.openstack.org/developer/os-cloud-config/usage.html#registering-nodes-with-a-baremetal-service05:14
tchaypojerryz: there you go, stevenk found a sample :)05:16
StevenKSure, because I *wrote* it :-)05:16
jerryzStevenK, tchaypo: the format is indeed the same as testenv.json, but a question: if it is a bare metal, what does a disk size and number of cpu mean?05:19
jerryzthe disk size going to be formatted for OS partition? or just a number when registering the node05:20
*** ramishra has joined #tripleo05:21
tchaypothe former - if you set it to something bigger than the actual disk you'll get an error when it's trying to write the image05:23
tchaypobefore a patch that stevenk is working in, that means you need to set all nodes to the size of the smallest node; after his patch it has the ability to generate custom-sized images for each size of disk you have05:24
StevenKtchaypo: Er?05:24
tchaypoStevenK: don't tell me i dreamed that patch that does baremetal flavors?05:25
tchaypoi even thought  it might have landed already05:25
StevenKtchaypo: It's up for os-cloud-config05:26
openstackgerritSteven Dake proposed a change to stackforge/kolla: Import nova-ctr-base into every image to get kolla-common.sh  https://review.openstack.org/12883105:28
*** k4n0 has joined #tripleo05:29
*** otoolee has joined #tripleo05:33
jerryztchaypo: gotcha. so right now, i can just use a common flavor for all the nodes despite of the physical spec difference as long as the disk size is sufficient? the nodes and flavor registered will use the numbers in my node.json?05:35
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Add pointers to a sample nodes.json  https://review.openstack.org/12883205:37
*** pensu has joined #tripleo05:38
tchaypojerryz: hang on, let me check something05:38
StevenKThe flavor will use the memory, disk and cpu from the first node05:38
tchaypofor a flaver with an NGb root image, and MMMGb specifed in nodes.json, NGb will be partitioned for / and MMM-N for /mnt05:39
jerryzStevenK: what if the seed and under cloud are libvirt vms and the rest are real bare metals for the overcloud05:39
tchaypoif the value in nodes.json is smaller than the actual disk on the node you have some unused space; if it's bigger you have an error building05:40
*** pcrews has quit IRC05:40
StevenKjerryz: The setup-baremetal script takes a number of nodes -- it's the first one passed to setup-baremetal05:40
StevenKIf it's split like that, you may need to do things by hand05:41
StevenKYou might be able to get away with having the first node (or the first three for HA) be VMs, and then have the remainder as actual physical machines05:42
jerryzStevenK: i remember the nodes for setup-baremetal are the ones left after seed /under are built.05:42
StevenKjerryz: Actually, setup-baremetal runs during devtest_seed and devtest_undercloud to register nodes with a baremetal service and create the flavor05:43
jerryzStevenK: i mean the setup-baremetal in devtest_overcloud05:44
StevenKdevtest_overcloud does not run setup-baremetal05:44
*** bvivek has joined #tripleo05:51
tchaypoStevenK: so. register-nodes..05:57
tchaypohow to i figure out what to provide as for the SERVICE_HOST ?05:58
*** rushiagr is now known as rushiagr_away06:00
tchaypooh, it's optional06:03
*** mrunge has joined #tripleo06:08
* tchaypo discovers he can't use the new register-nodes goodness until he rebases06:11
* tchaypo sees the rebase06:11
* tchaypo cries06:11
StevenKtchaypo: It's optional for ironic06:11
StevenKI'll be doing the same for -u in init-keystone once no-pki-setup lands06:11
*** dshulyak_ has quit IRC06:16
tchaypohttp://paste.openstack.org/show/121398/06:26
tchaypooh, i see who's to blame for this mess.06:27
tchaypothe only thing my change has added there is NeutronPublicInterfaceTag06:31
tchaypoand https://review.openstack.org/#/c/103757/36/scripts/devtest_overcloud.sh seems to say taht that should be in the heat template06:31
tchayposo presumably if I understood how to put it in heat I could avoid this merge..06:32
*** dsneddon has quit IRC06:36
*** dsneddon has joined #tripleo06:37
openstackgerritA change was merged to openstack/tripleo-image-elements: Add svc-map to cinder-api  https://review.openstack.org/10419806:44
*** rbrady has joined #tripleo06:45
lifelesstchaypo: ECONTEXT06:52
*** ramishra has quit IRC06:56
*** dsneddon has quit IRC06:57
*** ramishra has joined #tripleo06:57
*** dsneddon has joined #tripleo06:58
*** jtomasek has joined #tripleo07:03
*** ifarkas has joined #tripleo07:11
*** jcoufal has joined #tripleo07:16
*** ramishra has quit IRC07:16
*** ramishra has joined #tripleo07:17
*** dsneddon has quit IRC07:18
*** dsneddon has joined #tripleo07:18
*** ramishra has quit IRC07:22
jerryzanybody?07:28
jerryzlifeless: around?07:29
lifelessjerryz: somewhat, how can I help?07:36
*** jprovazn has joined #tripleo07:36
jerryzlifeless: where is neutron worker element located? is there a separate role for that?07:37
jerryzlifeless: or just inside controller?07:37
*** ramishra has joined #tripleo07:38
*** dtantsur|afk is now known as dtantsur07:46
*** ramishra has quit IRC07:53
*** gfidente has joined #tripleo07:55
*** jp_at_hp has joined #tripleo07:57
*** jml has joined #tripleo08:01
*** bvivek has quit IRC08:01
*** jistr has joined #tripleo08:09
*** otoolee has quit IRC08:14
*** yuanying_ has quit IRC08:17
*** yuanying has joined #tripleo08:17
*** otoolee has joined #tripleo08:21
*** yuanying has quit IRC08:21
*** akrivoka has joined #tripleo08:22
*** lucasagomes has joined #tripleo08:27
*** yamahata has quit IRC08:36
*** bvivek has joined #tripleo08:37
*** kbyrne has quit IRC08:42
*** ramishra has joined #tripleo08:43
*** bashok has joined #tripleo08:46
*** killer_prince is now known as lazy_prince08:53
*** yuanying has joined #tripleo08:54
*** jang2 has joined #tripleo08:59
*** jang has quit IRC09:00
*** jang1 has quit IRC09:00
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/138140809:00
uvirtbotLaunchpad bug 1381408 in diskimage-builder ""visudo" always got error when building overcloud image" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
*** ramishra has quit IRC09:01
*** lazy_prince has quit IRC09:05
lifelessjerryz: worker element?09:06
lifelessjerryz: do you mean the neutron agent?09:06
jerryzlifeless: yes09:09
tchaypolifeless: about dnsmasq?09:11
*** pensu has quit IRC09:11
jerryzlifeless: tchaypo: also yes09:14
jerryzlifeless, tchaypo: i mean where the external network is09:16
*** lazy_prince has joined #tripleo09:17
jerryzthe baremetals that need internet access09:18
openstackgerritPeter Belanyi proposed a change to openstack/tripleo-heat-templates: Add converted version of block and object storage  https://review.openstack.org/12371309:25
*** pensu has joined #tripleo09:26
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Add a big hammer  https://review.openstack.org/9833609:28
*** funzo has quit IRC09:28
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Force empty $TMPDIR inside the chroot  https://review.openstack.org/12869609:28
openstackgerritA change was merged to openstack-infra/tripleo-ci: Support debug logging on seed  https://review.openstack.org/12870409:37
*** derekh has joined #tripleo09:39
openstackgerritA change was merged to openstack-infra/tripleo-ci: Revert "Set tripleo ci control variables based on job name"  https://review.openstack.org/12378809:39
*** pblaho has joined #tripleo09:43
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Remove first-boot.d support  https://review.openstack.org/12683609:48
GheRiveromorning all09:57
*** killer_prince has joined #tripleo10:03
openstackgerritTom Cammann proposed a change to openstack/tripleo-image-elements: Disable SSLv3 in Stunnel  https://review.openstack.org/12886810:05
*** jerryz has quit IRC10:13
*** pelix has joined #tripleo10:16
openstackgerritLoganathan Parthipan proposed a change to openstack/tripleo-image-elements: sets novnc endpoint uri to https if tls is enabled  https://review.openstack.org/12887110:16
*** openstackgerrit has quit IRC10:19
*** openstackgerrit has joined #tripleo10:20
*** akrivoka has quit IRC10:21
*** lazy_prince has quit IRC10:24
*** killer_prince is now known as lazy_prince10:24
*** ramishra has joined #tripleo10:28
tchaypojerryz: I'm not sure how that works. I think maybe it's as simple as them getting a route to the seed by default, withthe ability to add in a different gateway IP if needed10:30
tchaypobut I don't actually know anything about it so I could be wildly wrong10:30
*** ramishra has quit IRC10:32
*** yamahata has joined #tripleo10:36
*** jp_at_hp has quit IRC10:38
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Fix RabbitMQ element clustering start and stop  https://review.openstack.org/11552410:48
*** pensu has quit IRC10:57
*** ramishra has joined #tripleo10:57
*** akrivoka has joined #tripleo10:59
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Avoid overwritting of hooks  https://review.openstack.org/12686911:06
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Make safe int cast  https://review.openstack.org/12888311:06
*** otoolee has quit IRC11:28
*** otoolee has joined #tripleo11:34
*** rhallisey has joined #tripleo11:35
*** ramishra has quit IRC11:35
*** julim has quit IRC11:46
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu and Debian  https://review.openstack.org/9013411:48
*** bvivek has quit IRC12:00
*** radez_g0n3 is now known as radez12:04
*** ramishra has joined #tripleo12:16
*** jehb has joined #tripleo12:19
openstackgerritAna Krivokapic proposed a change to openstack/tuskar-ui: Style the Deployment Confirmation dialog  https://review.openstack.org/12890312:19
*** funzo has joined #tripleo12:23
*** morazi has quit IRC12:25
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Disable all interfaces on eni systems  https://review.openstack.org/12821312:27
*** ramishra has quit IRC12:28
*** dprince has joined #tripleo12:32
*** jdob has joined #tripleo12:32
openstackgerritA change was merged to openstack/os-collect-config: Modify gitignore to ignore cover  https://review.openstack.org/12099412:48
*** pensu has joined #tripleo12:53
*** ramishra has joined #tripleo12:54
*** pradk has quit IRC12:55
*** shadower39 has joined #tripleo12:57
*** noslzzp has joined #tripleo12:59
*** shadower39 has quit IRC13:01
*** shadower78 has joined #tripleo13:02
*** shadower has quit IRC13:03
*** shadower78 is now known as shadower13:03
*** jang2 has quit IRC13:05
*** tzumainn has joined #tripleo13:06
*** pblaho has quit IRC13:07
*** tsedovic has joined #tripleo13:08
*** tsedovic has quit IRC13:09
*** jang has joined #tripleo13:10
*** jang1 has joined #tripleo13:10
openstackgerritA change was merged to openstack/tripleo-image-elements: ensure that port is added to bridge after a reboot  https://review.openstack.org/12353413:10
openstackgerritA change was merged to openstack/os-cloud-config: Switch to oslo.i18n  https://review.openstack.org/12801713:11
*** morazi has joined #tripleo13:15
*** ramishra has quit IRC13:20
*** ramishra has joined #tripleo13:21
*** julim has joined #tripleo13:21
*** pensu has quit IRC13:25
*** bashok has quit IRC13:28
openstackgerritErik Colnick proposed a change to openstack/tripleo-specs: TripleO DVR enablement  https://review.openstack.org/11398313:34
SpamapStchaypo: my thingies are buried deep inside the corporate HP network unfortunately. :-P13:37
*** jprovazn has quit IRC13:37
*** rbrady_ has joined #tripleo13:46
*** rbrady has quit IRC13:47
*** rbrady_ is now known as rbrady13:48
sdakemorning13:49
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Create 'neutron-router' image element  https://review.openstack.org/12754113:49
*** cody-somerville has joined #tripleo13:50
*** cody-somerville has joined #tripleo13:50
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Remove redundancies between neutron-router and neutron-network-node  https://review.openstack.org/12761813:56
*** jp_at_hp has joined #tripleo13:57
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Add Neutron DVR configuration variables  https://review.openstack.org/12798813:58
*** pradk has joined #tripleo13:59
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Fix race between apache2 starting and o-r-c configuring apache2  https://review.openstack.org/11203914:00
*** ramishra has quit IRC14:00
*** noslzzp has quit IRC14:00
sdakelarsks rhaillisey can you ahe a look at  the nova-conductor changes14:03
sdakenova-conductor is almost working with these14:03
larskssdake: sure, will look in just a few.14:03
*** TheJulia has joined #tripleo14:05
*** ramishra has joined #tripleo14:09
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-image-elements: Ensure neutron rootwrap.d symlink is not nested  https://review.openstack.org/12893514:11
*** jprovazn has joined #tripleo14:12
*** spzala has joined #tripleo14:14
sdakenova-conductor/nova-controller above14:17
*** pradeep has quit IRC14:17
*** spzala has quit IRC14:18
*** spzala has joined #tripleo14:18
*** weshay has quit IRC14:20
*** athomas has quit IRC14:23
rhalliseysdake, were you able to see nova.conf when you ran the api?14:23
*** morazi has quit IRC14:23
greghaynesSpamapS: TheJulia Heres the pattern that im thinking for bootstrap_host group https://review.openstack.org/#/c/128938/14:23
*** jang1 has quit IRC14:24
*** jang has quit IRC14:24
*** jang has joined #tripleo14:25
SpamapSgreghaynes: yes that is fantastic14:26
SpamapSgreghaynes: so what you're saying is.. add another layer of abstraction.. ;)14:26
greghaynesheh14:26
greghaynesbasically14:26
greghaynesor at least use the already existing one14:27
*** marun has joined #tripleo14:27
*** jang has quit IRC14:28
*** athomas has joined #tripleo14:30
*** jang has joined #tripleo14:30
*** morazi has joined #tripleo14:31
*** jang has quit IRC14:32
*** jang has joined #tripleo14:33
*** jang has quit IRC14:33
*** jang has joined #tripleo14:34
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Fix race between apache2 starting and o-r-c configuring apache2  https://review.openstack.org/11203914:34
*** jang has quit IRC14:36
*** jang has joined #tripleo14:37
*** jang has quit IRC14:37
*** weshay has joined #tripleo14:38
*** jang has joined #tripleo14:38
*** noslzzp has joined #tripleo14:38
larskssdake: question about some of your changes in e.g. https://review.openstack.org/#/c/128800/: for other things (like glance), we are creating a glance-base image that installs the common stuff required by all the other glance images.  We were doing the same for nova; any reason not to?14:40
*** jang has quit IRC14:40
larsksIt'll work either way, of  course. Just curious.14:40
*** k4n0 has quit IRC14:40
*** jang has joined #tripleo14:41
*** jang has quit IRC14:41
rhalliseysdake, so for the sake of getting it working we can do it the way you have it14:42
larsksYup.14:42
rhalliseysdake, but after  I might go back and change it to make it similar to the others14:42
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Fix race between apache2 starting and o-r-c configuring apache2  https://review.openstack.org/11203914:44
*** jang has joined #tripleo14:44
*** cody-somerville has quit IRC14:44
*** akrivoka has quit IRC14:47
*** openstackgerrit has quit IRC14:48
*** openstackgerrit has joined #tripleo14:49
*** dmellado has joined #tripleo14:50
*** akrivoka has joined #tripleo14:50
larskssdake: actually, I think we need to rework those changes.14:53
larsks I think we need to keep the nova-controller base image (we can rename it if you like)...because in https://review.openstack.org/#/c/128796/, we ended up with three different shell scripts all doing the same thing.14:54
larsksSo for every change we need to make, we need to make it in three places.14:54
rhalliseylarsks, https://review.openstack.org/#/c/128796/ we can just do the config in the controller14:57
larsksrhallisey: I think that is what I was just saying, or are you saying something else?14:57
larsksI am not sure I followed that comment.14:57
rhalliseylarsks, yes I agree but that's what it was before14:58
rhalliseyI was wondering why it was changed14:58
rhalliseyor didn't work14:58
larsksrhallisey: I am still not following you! Are you agreeing that we should go back to having a base image from which the other images inherit?14:58
rhalliseyyes14:58
larsksYay! :)14:58
rhallisey:)14:58
larsksThe reason it was changed is that the start script was trying to start the nova-controller image, which was an error.  The correct fix would have been to fix the start script.14:59
larsksI am working on that, and trying to integrate changes from this series of patches.14:59
rhalliseylarsks, ok, that answers my question14:59
*** ramishra has quit IRC15:00
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/138140815:00
uvirtbotLaunchpad bug 1381408 in diskimage-builder ""visudo" always got error when building overcloud image" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
larskswe do need to remove start.sh from the nova-controller image (that will just get tromped on by start scripts in other images).15:00
larskswe can handle common setup as in glance (where the base image installs a configure-glance.sh script that is called by start.sh in other images)15:00
*** jprovazn is now known as jprovazn_afk15:00
*** lazy_prince is now known as killer_prince15:01
*** sdake_ has joined #tripleo15:03
*** sdake_ has quit IRC15:03
*** sdake_ has joined #tripleo15:03
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Fix RabbitMQ element clustering start and stop  https://review.openstack.org/11552415:04
*** noslzzp has quit IRC15:07
*** noslzzp has joined #tripleo15:09
*** ramishra has joined #tripleo15:10
*** pcrews has joined #tripleo15:12
*** ifarkas has quit IRC15:14
sdakelarsks I ended up making that change later15:14
sdakebut happy to rework the change set if it matters15:14
*** daneyon has quit IRC15:14
sdakepersonally I dont trhink it  does, as we don't have any real ci :(15:14
*** daneyon has joined #tripleo15:15
*** noslzzp has quit IRC15:15
*** ifarkas has joined #tripleo15:16
larsksThis isn't a question of CI.15:16
larsksThis is a maintenance issue.  Someone reports a bug, you fix it in nova-api/start.sh or something, but not in nova-conductor/start.sh, and then someone reports the same bug in that image...15:16
*** noslzzp has joined #tripleo15:17
greghaynesTheJulia: SpamapS For some reason I end up with a "All hosts have already failed" but I dont see any failing tasks and the ply recap says failed=0 for every host....15:18
greghaynesyouall seen this before?15:19
*** pensu has joined #tripleo15:19
*** pradeep has joined #tripleo15:21
*** ramishra has quit IRC15:22
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Add logic to ensure vlan configuration is persisted  https://review.openstack.org/12511815:27
*** killer_prince has quit IRC15:28
*** bvivek has joined #tripleo15:30
*** avozza has joined #tripleo15:33
*** pradeep has quit IRC15:36
*** killer_prince has joined #tripleo15:37
*** killer_prince is now known as lazy_prince15:37
lxsligreghaynes: at your convenience please would you see my response on https://review.openstack.org/#/c/113546 ? Cheers!15:38
*** jistr has quit IRC15:40
*** bvivek has quit IRC15:42
openstackgerritA change was merged to openstack/tripleo-heat-templates: Refactor Heat dependencies for nova compute  https://review.openstack.org/12238415:42
*** bvivek has joined #tripleo15:43
greghayneslxsli: will do15:43
sdakelarsks look later in the patch stream15:45
sdakeif we had ci, this would probably bust15:45
sdakebut later in the patch stream I do exactly as you mention15:46
sdakeI'd rather not have to rewrite all the patches but can if necessary...15:46
larskssdake: I see https://review.openstack.org/#/c/128831/15:46
larsks...which seems to introduce even more duplication.  errr, triplication.15:46
larsksI am in the midst of reworking these patches.15:46
sdakeit copies the same pattern of glance15:46
sdakeexact same pattern15:47
sdakenova-ctr-base is like glance-base15:47
sdakenotice the D next t o nova-config.sh in each of the subdirs15:47
SpamapSgreghaynes: no that should not happen15:48
SpamapSgreghaynes: you should have at least one failed15:48
SpamapSgreghaynes: it is a misnomer "all" .. it really means "the max fail percentage has been exceeded"15:49
larsksYup.  But it's not clear why you discarded the nova-controller-base  only to re-introduce the same thing.15:49
larsksIt's kind of a conusing patch series.15:49
larsks*confusing15:49
sdakeok I'll rework the patch set then15:50
larsksLike my typing.15:50
greghaynesSpamapS: hrm, well it looks like the command at the top of galera_status is skipping the hosts because the removed=, which is fine, but then it stops running the playbook there and reports fail15:50
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: changes to base nova configuration  https://review.openstack.org/12896615:52
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: fix nova-controller pod  https://review.openstack.org/12896715:52
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: have start.sh source in common configuration script  https://review.openstack.org/12896815:52
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: Nova has an undeclared dependency on python-cinderclient  https://review.openstack.org/12896915:52
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: introduce conductor to common configuration  https://review.openstack.org/12897015:52
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: introduce scheduler to common configuration  https://review.openstack.org/12897115:52
larskssdake: see what you think of that series ^^^^^15:52
greghaynesSpamapS: Heres an end portion of my log http://paste.ubuntu.com/8574580/15:52
sdakelarsks you need MY_IP15:53
sdakehence you need kolla-common in the start scripts15:53
larsksTake a closer look.15:53
larsksThey are all getting kolla-common.15:53
larsksVia config-nova-controller.15:53
larsks(.sh)15:53
sdakei see15:53
sdakediffeerente then glance15:53
sdakeI think you need to have separate ops because you want to check between common import and nova-config15:54
sdakecheck for variables that is15:54
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Set tripleo ci control variables based on job type  https://review.openstack.org/12897215:54
larsksThere are start-specific checks where appropriate (e.g., we only check DB_ROOT_PASSWORD in the nova-conductor start script)15:55
larsks(via check_for_db)15:55
sdakeI think we want nova-conductor creating the db/keystone users15:55
sdakerather then nova-api15:55
larsksOkay.  Why for?15:56
sdakeapi could horizontally scale15:56
sdakeconductor not so15:56
larsksIt does create the database and database users.  But why keystone?15:56
sdakeit is something we can fix later I guess15:56
SpamapSgreghaynes: I'd share that in #ansible .. it looks weird to me.15:56
TheJuliagreghaynes: would seem like it thinks it can't proceed but no single host failed... possibly a bad when conditional15:56
larsksI think the fix for that and elsewhere is going to be gus' suggestion re: moving one-time init completely outside of the service images.15:57
*** rhallisey has quit IRC15:57
greghayneshrm15:57
larsksE.g., keystone and glance have the same problem right now.15:57
TheJuliagreghaynes: that or it doesn't know about the host you've referenced on the hosts line15:57
greghaynesTheJulia: Yea, my thinking is its some wierd interaction between serial: 1 and when15:57
SpamapSsdake: have you considered having an ephemeral container that is just brought up to bootstrap things, and then removed.15:57
greghayneswell, it skipped one host which seems correct15:57
SpamapSlarsks: ahh I see you have considered that :)15:58
TheJuliagreghaynes: I'd put name tags in between each step to give you more visibility15:58
larsksSpamapS: Yeah, exactly.15:58
sdakeSpamapS yes we have - enounicorns :)15:58
larsksWe can use kubernetes restart policies to create run-once containers.15:58
SpamapSSeparate orchestration of end goal from workflow :)15:59
larsksSpamapS: That is totally the direction in which I would like to move.15:59
sdakeyup but keep in mind wehave 2 days to finish getting nova working15:59
greghaynesTheJulia: ah, good call15:59
SpamapSyou have a bring-up workflow which is [put things in a pre-bootstrapped state][bootstrap][put things in ready state]15:59
sdakeso lets not complicate things in the short term for our milestone 1 deadline15:59
larskssdake: and that is why I am not suggesting we make those changes right now :)15:59
SpamapSthat [bootstrap] state is the reason os-cloud-config exists. ;)16:00
larsksSpamapS: well, this seems like a completely separate issue. Regardless of how we bootstrap things, we need to move that process outside of the service containers.16:00
*** jistr has joined #tripleo16:00
SpamapS'kube-magic-thing run os-cloud-config'16:00
SpamapSlarsks: yeah ignore my how .. the where and when are more interesting. :)16:01
SpamapSthough I'm still pretty sure we shouldn't have two ways to bootstrap OpenStack clouds if we can avoid it.16:01
sdakelarsks I've reviewed the patch set with what I think would make it proper - if yuou can rework quickly we can merge16:03
*** rameshg87-afk has joined #tripleo16:03
larsksI'm on it.  I would like to test before we merge.16:03
rameshg87-afkdprince, SpamapS, hi16:03
sdakelarsks have to keep in mind I spent all day debugging that stuff yesterday so that is why my patch set looks disorganized16:03
larsksThat is what rebase is for :)16:03
rameshg87-afkdprince, SpamapS, request you to have a look at the review https://review.openstack.org/123642 .16:04
sdakeyes, but ~9pm time for bed rather then rebase ;)16:04
larsksYou are obviously better at time management that I am.16:04
sdakealso you want to fix the busted crudini as a dependency somewhere in that chain16:04
* larsks REBASES ALL THE THINGS16:04
sdakebecause set -e will cause the script to error immediately durnig setup16:04
larsksThat is the idea, yes.16:05
sdakewhat I mean is there is a broken crudini operation16:05
sdakewhich will not return 016:05
larsksIf you have noted it in comments, I will fix it.16:05
sdakeI'm not sure where it s hould go sec16:05
larsksThere were some typos in the existing configs (a --del with a value, and some operations that were referring to /etc/nova/nova instead of /etc/nova/nova.conf).  I fixed in these existing patches.16:06
greghaynesSpamapS: TheJulia So in wierder news, if I do --limit to one host it fails in the step after the last one it prints out16:07
greghaynesmakes me think its actually running the next step and just not printing it16:07
sdakelarsks - ok then that should be fine16:07
*** pcrews has quit IRC16:08
larskssdake: re: 31 character limit on docker image names, where is that?  I can successfully build "kollaglue/fedora-rdo-nova-controller", which has 36 characters.16:08
sdaketry nova-controller-base16:08
larsksOr is that a limitation in the docker hub?16:08
sdakedocker hub i guess16:08
sdakeit spits out an error on push16:08
larsksThat works also.16:09
larsksAh, okay.16:09
*** pcrews has joined #tripleo16:09
sdakei'm not sure what the limit encompasses16:09
sdakeI just know nova-controller-base no worky16:09
*** rainya has joined #tripleo16:09
larsksYup, I see the error from the hub now.  Seems like a silly limit, but what can ya do.16:09
*** nosnos has quit IRC16:12
*** nosnos has joined #tripleo16:12
larsksHmmm, kollaglue/fedora-rdo-nova-ctr-base is 34 characters, but seems to push anyway.  I guess kollaglue/fedora-rdo-n-c-base is safer.16:13
*** rhallisey has joined #tripleo16:14
*** openstackstatus has joined #tripleo16:14
*** ChanServ sets mode: +v openstackstatus16:14
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: renamed nova-controller -> n-c-base  https://review.openstack.org/12898516:15
*** nosnos has quit IRC16:17
*** rameshg87-afk is now known as rameshg8716:17
*** noslzzp has quit IRC16:19
*** dtantsur is now known as dtantsur|afk16:19
rameshg87derekh: can you please have a look at this review https://review.openstack.org/#/c/123642/ . this is required for an ironic driver landed in juno.16:19
*** akrivoka has quit IRC16:19
-openstackstatus- NOTICE: An error in a configuration change to mitigate the poodle vulnerability caused a brief outage of git.openstack.org from 16:06-16:12. The problem has been corrected and git.openstack.org is working again.16:20
*** ramishra has joined #tripleo16:22
*** jcoufal has quit IRC16:26
*** pradeep has joined #tripleo16:28
*** lsmola has quit IRC16:29
*** lazy_prince has quit IRC16:30
*** dmellado has quit IRC16:31
sdakelarsks I went aheaed and abanonded my change set, so if you can power out a new set, tha twouldrock so I can get some dev done today :)16:33
*** dshulyak_ has joined #tripleo16:33
larskssdake: I responded to several of your comments.  I think things are in good shape.16:33
larsksLet me know.  I am building images for testing right now.16:34
rhalliseylarsks, oh you're the culprit :P.  I'm waiting for you to finish16:34
larsksrhallisey: Huh? I'm building into my larsks/ namespace.16:34
*** ramishra has quit IRC16:34
rhalliseyRepository kollaglue/fedora-rdo-base already being pulled by another client. Waiting16:35
rhalliseyhow is this.. I'm building from my namespace..16:35
openstackgerritJeff Peeler proposed a change to stackforge/kolla: WIP: Current heat service work  https://review.openstack.org/12899416:36
*** rameshg87 has quit IRC16:36
larsksrhallisey: that error is referring to operations happening on your system.16:36
larskss/error/"informative message"16:37
larsksThat means your local docker daemon is engaged in a pull operation.16:37
rhalliseylarsks, I just rebuilt these minions.. weird..16:37
* rhallisey builds again16:38
greghaynesSpamapS: TheJulia So, it seems like setting max_fail_percentage: 0 is a no-nop16:39
*** jp_at_hp has quit IRC16:40
sdakelarsks I went back through, check the -1's out16:41
larskschecking16:41
sdakepretty minor stuff16:41
sdakedaneyon your changes rock :) - I hada fit nitpicks about spaces vs tabs16:41
sdakeI prefer tabs, but everyone else loves spaces for whatever reason16:41
jpeelerpeople have their editors still setup for python16:42
larskssdake: the comment on https://review.openstack.org/#/c/128968/ really belongs to https://review.openstack.org/#/c/128966/, since this is the patch that introduces the crudini operations.16:42
larskssdake: I love tabs for shell scripts, especially because some operations (indented "here" documents) can't be used with spaces.16:43
openstackgerritA change was merged to openstack/tripleo-image-elements: Ensure neutron rootwrap.d symlink is not nested  https://review.openstack.org/12893516:43
daneyonsdake: thanks. now i'm working on the difficult part... neutron agents and plugins. u cool with focusing on ml2/ovs/gre to align with the os docs or mls/ovs/vxlan or something else?16:44
*** lsmola has joined #tripleo16:44
larsksdaneyon: I think vxlan is the preferred tunneling solution these days, no?16:46
larsksOr am I imagining?16:46
greghaynesSpamapS: TheJulia https://review.openstack.org/#/c/128997/16:47
*** lucasagomes has quit IRC16:47
* larsks needs lunch. back in a bit.16:48
sdakedaneyon I can barely get neutron working typically so I'd go with your judgement :)16:49
*** derekh has quit IRC16:50
sdakelarsks I thought you might just introduce a patch between the two16:50
sdakerather then ham it iinto one or the other16:51
*** yamahata has quit IRC16:52
*** dshulyak_ has quit IRC16:53
*** pradeep1 has joined #tripleo16:53
*** pradeep has quit IRC16:56
*** bvivek has quit IRC16:57
daneyonsdake: I agree vxlan is preferred. I will shoot for mls/ovs/vxlan and use gre as a backup plan. ok?16:59
*** avozza is now known as zz_avozza17:03
*** killer_prince has joined #tripleo17:04
*** killer_prince is now known as lazy_prince17:05
SpamapSgreghaynes: nice work. :)17:06
*** yamahata has joined #tripleo17:08
*** rainya has quit IRC17:08
*** rainya has joined #tripleo17:09
sdakedaneyon speaking french but wfm ;-)17:09
*** pradeep has joined #tripleo17:09
*** pradeep1 has quit IRC17:09
*** noslzzp has joined #tripleo17:09
*** rainya has quit IRC17:14
sdakelarsks I notice with mion count of 1 the heat template works great but minion count of 2 thiungs don't seem to work17:14
larsksI run with 2 minions all the time, no problems.17:15
sdakeyup17:15
sdakemaybe you can investigate my setup after we release17:15
larsksIs the overlay network up?17:15
*** rainya has joined #tripleo17:15
sdakeso I can get with 2 minions17:15
sdake1 minion is working fine17:15
larsksYou should be able to ping the address of kbr0 on one minion from the other minion.17:15
sdakeI am not sure, I stopped trying with 2 minions a couple days ago17:15
larsksOkay.  Well, when you have the chance, give that a shot.17:15
sdakeI think we would be wise to recommend 1 minion in the docs17:15
sdakeafter release I should be able to get into it17:15
*** ifarkas has quit IRC17:18
sdakedocs/release info17:18
*** dshulyak_ has joined #tripleo17:19
larskssdake: I haven't seen a problem with multiple minions elsewhere. I don't want to discourage testing with multiple minions unless we have a real problem, because I think it helps flush out configuration issues that might "work by accident" on a single minion.17:21
larsksI guess I am happy to make the default number_of_minions=1, without otherwise discouraging people from trying more.17:23
larsksDoes that work for you?17:23
sdakewfm17:25
*** cody-somerville has joined #tripleo17:25
sdakethat sounds great17:25
sdakethatway we dont have to mention in our directions about that point17:25
sdakelarsks I think generally all the devs except you are havingtrouble with multiple minions17:27
sdakeso maybe its environmental?17:27
larsksI don't know.  That's the first I've heard about problems.17:27
larsksNo reports == no chance to diagnose.17:27
sdakeI have been complainign for a  week ;-)17:27
*** yamahata has quit IRC17:27
sdakebut its all good17:27
sdakeI'm satisified in single minion setup the code we have minus nova is in good working order17:27
larsksI'm satisfied that it's in good working order on a multi-minion system.17:28
sdakeonce the nova changes land, we can fixed it17:28
sdakecool17:28
larsksBut again, if someone can actually show me a problem, I would love to take a look.17:28
sdakei'll show you tuesday/wed of next week17:28
sdakemaybe you can debug17:28
larsksAwesome.17:28
sdakeI'm not quite clear on how lanmanager fits into the equation17:28
sdakeor how to debug all the things17:28
sdakei've managed to learn to debug k8s and that is about it17:29
sdakeand docker17:29
larskslinkamanager sets up the overlay network that permits any pod to be accessible from any minion.  That's pretty much all it does.17:29
sdakebut neutron+linkmanager nope17:29
sdakeyup I got that17:29
sdakenot sure if I'm hitting a problem with that or neutron or something else17:29
sdakebut I don't want to distrub my env for t hat atm17:29
larskshttps://github.com/larsks/heat-kubernetes/commit/40f943cc4a1aac98ae3e54f1153c6b984ed1045817:30
sdakecool thanks!17:30
sdakeI think goal of announce email is just to get people to try openstack is the simplest possible way17:31
sdake(on k8s+docker that is)17:31
larsksRemember that the k8s folks provide instructions for setting up a vagrant-based test environment.  I'm only using Heat because I already had an appropriate openstack environment up and running.17:32
sdakeyup heat rocks for this17:32
sdakeand it fits in with the target audience17:32
larsksWell, my point was that for folks who do not already have openstack, it might be easier to follow the k8s vagrant instructions.17:32
sdakeoh - right I guess we need a faq for that then :)17:32
larsksBecause then they don't need to learn about nova and neutron and heat just to learn about k8s and the kolla images.17:32
sdakeor a link to theier setup docs17:33
sdakeI find launching a heat template with a git pull and heat create easier then following the directions from k8s upstream17:33
larsksEven so, you've had some neutron issues to overcome.  The templates are great for people who already have openstack up and running, but not necessarily the best starting point for other folks.17:35
sdakeright17:35
larsksI haven't looked closely at the upstream docs though, so I don't know how hard/easy is that process.17:35
sdakeagree / agree :)17:35
sdakeso where are you wiht fixing the -1s17:35
sdakeso we can get this merged and start fixing nova-controller to actually work17:36
sdakebecause right now, it doesn't even after my change set (which ends i nthe same state as yours)17:36
*** bkero has quit IRC17:36
larsksThere are only two -1s left.17:36
larsksOn https://review.openstack.org/#/c/128968/, I don't know which, if any, crudini op is failing.17:37
larsksBut if there is a failure I should be able to replicate it real soon now.17:37
sdakesomething to do with a log operation I think17:37
larsksOn https://review.openstack.org/#/c/128985/, I like "nova-ctr-base" also, but the resulting image name is 34 characters.17:37
sdakeand the logs change /etc/nova/nova vs /etc/nova/nova.conf17:38
larskssdake: I think that is already fixed.17:38
sdakenova-ctr-base works I've pushed it17:38
larsksI realize that it works, but it violates the 31 character limit, which means it might stop working without warning.17:38
sdakeI don't know what the 31 character limit is17:38
larsksIf docker hub says "limit your image names to 31 characters", I think we should do that.17:38
sdakeperhaps its after the "user"( kolla)17:38
larsksHmmm, maybe.17:38
larsksLet me experiment.17:38
openstackgerritA change was merged to openstack/diskimage-builder: Don't re-install cloud-init for centos7 images  https://review.openstack.org/12741117:39
sdakedaneyon if you want perms to push images, sign up on docker.io hub and let me know your id17:40
*** zz_avozza is now known as avozza17:40
larsksThe error suggests that it is the entire thing: "2014/10/16 13:40:13 Error: Status 400 trying to push repository larsks/this-is-a-long-name-so-stop-whi: "this-is-a-long-name-so-stop-whi is too long, needs to be < 31 characters"17:40
larsks(and len("this-is-a-long-name-so-stop-whi") == 31)17:41
daneyonsdake: i have an acct: danehans17:41
sdakethats 38 chars larsks17:41
sdake38 -5 = 32 not 3117:41
openstackgerritA change was merged to stackforge/kolla: changes to base nova configuration  https://review.openstack.org/12896617:41
openstackgerritA change was merged to stackforge/kolla: fix nova-controller pod  https://review.openstack.org/12896717:41
sdakeya maybe it needs to be 30 chars17:41
larsksRight. You were suggesting "perhaps its after the "user"".17:41
larsksI was testing that theory.17:41
sdakemaybe it includes the / ;)17:42
larsksAnd it appears to be not the case.17:42
sdaketry 1 less character17:42
larsksThat seems to work.  I almost suspect that their length check is off.  But given that, I'll go change https://review.openstack.org/#/c/128985/17:42
*** ccrouch has quit IRC17:43
*** ccrouch has joined #tripleo17:43
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: renamed nova-controller -> nova-ctr-base  https://review.openstack.org/12898517:44
larskssdake: that is the last -1; the one on https://review.openstack.org/#/c/128968/ it's not clear to me that there's actually anything there to fix.17:45
*** athomas has quit IRC17:46
sdakedaneyon your all set17:46
sdakelarsks ok I'll +2 an we can get it merged and work from there17:47
sdakeya off by 1 ftw :)17:47
daneyonsdake: great. would you like me to push my neutron-base and neutron-server images to kollaglue and update my patch accordingly?17:47
sdakethey probably count the /17:48
sdakedaneyon ya go ahead and push em17:48
openstackgerritA change was merged to stackforge/kolla: have start.sh source in common configuration script  https://review.openstack.org/12896817:48
sdakesudo ./build --release --push17:48
openstackgerritA change was merged to stackforge/kolla: Nova has an undeclared dependency on python-cinderclient  https://review.openstack.org/12896917:48
daneyonsdake: will do17:48
sdakedaneyon not sure we have enough time to sort out neutron integration before release17:48
sdakecan SUID processes run in containers?17:49
sdakeI am getting a permission error on something that should run using rootwrap17:49
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: renamed nova-controller -> nova-ctr-base  https://review.openstack.org/12898517:49
openstackgerritA change was merged to stackforge/kolla: introduce conductor to common configuration  https://review.openstack.org/12897017:50
openstackgerritA change was merged to stackforge/kolla: introduce scheduler to common configuration  https://review.openstack.org/12897117:50
sdakecool all merged17:51
sdakeI'll do a  build for kolla now and push it17:51
larskssdake: https://review.openstack.org/#/c/128985/ doesn't seem to have merged yet.17:53
sdakemaybe it needs a rebase?17:53
*** wuhg has quit IRC17:53
larsksNope, gerrit would flag that.17:53
sdakeit takes a moment for gerrit to flag - v isn't +2 yet17:54
larsksTry adding another +2+1 and see what happens, maybe?17:54
sdakethe gate is still running I suspect17:54
larsksOh, right, waiting on jenkins.17:54
larsksYup.17:54
sdakealthough the gate should be immediate17:54
sdakesince we hae very little gating17:54
sdakeyou code reviewed your own change :)17:55
sdakenaughty17:55
sdakejenkins takes forever to sort out rebases17:56
sdakei'll bbi 1517:56
*** cody-somerville has quit IRC17:57
openstackgerritA change was merged to stackforge/kolla: renamed nova-controller -> nova-ctr-base  https://review.openstack.org/12898517:57
daneyonsdake: should i use any specific tag for the neutron images or just leave it as CID?18:04
*** marun has quit IRC18:05
larsksdaneyon: Right now you are welcome to push as :latest, although at some point (I hope) that will only be available via an automated workflow.18:07
*** marun has joined #tripleo18:07
*** noslzzp has quit IRC18:07
daneyonlarsks: thx.18:07
*** akrivoka has joined #tripleo18:07
*** pradeep1 has joined #tripleo18:08
*** pradeep has quit IRC18:08
*** dshulyak_ has quit IRC18:08
*** akrivoka has quit IRC18:14
daneyonI'm doing a "docker push kollaglue/fedora-rdo-neutron-base" and I still receive "Access Denied: Not allowed to create Repo at given location"18:15
sdakelarsks +118:19
sdakedaneyon are you using sudo ./build --push --release?18:19
larsksdanehans was not a member of kollaglue.18:20
larsksdaneyon: I just fixed that.  Try now.18:20
sdakewierd I added it a minute ago18:20
sdakemaybe buggy docker.io18:20
larsksThere is a "save" button that I almost missed.18:20
sdakeit put your mug shot up and everything18:20
sdakeoh maybe that was the problem18:20
larsksBut you can see it here now: https://hub.docker.com/account/organizations/kollaglue/18:21
sdakeapologies daneyon18:21
sdakerebuilding nova-controller now18:22
sdakeso we can have a joint debug session if possible :)18:22
*** pelix has quit IRC18:25
*** noslzzp has joined #tripleo18:28
*** noslzzp has quit IRC18:28
daneyonpushing to kollaglue works now. thx!18:31
*** ramishra has joined #tripleo18:35
openstackgerritA change was merged to openstack/diskimage-builder: Remove first-boot.d support  https://review.openstack.org/12683618:37
*** spzala has quit IRC18:37
*** jdob has quit IRC18:38
*** jdob has joined #tripleo18:38
*** ramishra has quit IRC18:40
openstackgerritSteven Dake proposed a change to stackforge/kolla: Make shell scripts executible  https://review.openstack.org/12901718:40
sdakecurrent images are busted larsks :(18:40
larsksNeed details sdake :(18:41
sdakesee last review18:41
sdakethat should get em booting18:41
larsksLooks good.18:42
sdakeI don't know why docker doesn't just fix that18:42
sdakeseems easy enough18:42
sdakeor atleast spit out a warning18:43
larsksHard to spit out a warning.  You may have "CMD /does/not/exist", and expect that to be provided by images based on yours.18:43
sdakeI see18:43
*** saurabhs has joined #tripleo18:43
sdakewell that makes sense then18:43
sdakeout of pain meds18:44
sdakefkn dentists and their  tight ass pain med practicies18:44
sdake3 crowns = suck18:44
sdakeI havet o drive over there if I want more - not sure if I'm motivated enough18:45
sdakeguess we will see how lunch goes18:45
larsksGood luck.18:45
openstackgerritA change was merged to stackforge/kolla: Make shell scripts executible  https://review.openstack.org/12901718:45
sdakedoes crux get installed in fedora-rdo-base?18:46
larsksYes.18:47
larskshttps://github.com/stackforge/kolla/blob/master/docker/fedora-rdo-base/Dockerfile#L1418:47
sdakewho sorted out that dependency lsit?18:47
*** jdob has quit IRC18:49
*** jdob has joined #tripleo18:49
*** derekh has joined #tripleo18:50
larskshttps://github.com/stackforge/kolla/blame/master/docker/fedora-rdo-base/Dockerfile#L1418:50
larsks"sorted out" in what sense?18:51
*** bnemec has quit IRC18:52
*** bnemec has joined #tripleo18:53
*** ramishra has joined #tripleo18:53
*** pradeep has joined #tripleo18:55
*** pradeep1 has quit IRC18:55
*** ramishra has quit IRC18:58
openstackgerritSteven Dake proposed a change to stackforge/kolla: Wrong shell script specified in start.sh  https://review.openstack.org/12901918:58
*** rainya has quit IRC18:58
*** jistr has quit IRC18:59
sdakei sapmmed the logs and docker seems to keep the whole thing19:00
sdakeand doesn't crash on my end19:00
sdakemight oom at some point tho19:00
larsksI guess you could spam the logs and watch disk usage on /var/lib/docker.  Possibly it's keeping it on disk and not in memory.19:00
sdakeyup possible19:01
*** spzala has joined #tripleo19:01
*** boris-42 has quit IRC19:02
*** rainya has joined #tripleo19:04
*** boris-42 has joined #tripleo19:04
sdake/opt/kolla/config-nova-controller.sh: line 14: NOVA_DB_PASSWORD: command not found19:04
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: correct missing backslash in config-nova-controller.sh  https://review.openstack.org/12902219:05
*** pradeep1 has joined #tripleo19:06
sdakethat patch is in merge conflict whichever that means :)19:06
larsksmeans it needs a rebase on master.  just  a sec.19:07
sdakeoh right, thought you were working on master19:08
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: correct missing backslash in config-nova-controller.sh  https://review.openstack.org/12902219:08
larsksI thought so too :)  Apparently I needed a19:09
larsks'git pull'19:09
*** otoolee has quit IRC19:09
*** pradeep has quit IRC19:09
*** cody-somerville has joined #tripleo19:09
sdakeneed SSD for my minidell19:12
sdakethese builds take forever19:12
sdakei have 4 raid 0 ssd on my main workstation19:12
*** jprovazn_afk has quit IRC19:12
openstackgerritA change was merged to openstack/os-cloud-config: Add support for creating baremetal flavors  https://review.openstack.org/11587219:19
*** otoolee has joined #tripleo19:22
*** mrunge has quit IRC19:23
openstackgerritDaneyon Hansen proposed a change to stackforge/kolla: Adds Neutron Server Support  https://review.openstack.org/12855819:29
*** dprince has quit IRC19:32
openstackgerritA change was merged to openstack/tripleo-image-elements: Migrate ceilometer elements to svc-map  https://review.openstack.org/12868919:39
openstackgerritA change was merged to openstack/tripleo-image-elements: Add ironic to enable-packages-install  https://review.openstack.org/12739019:39
jpeelerlarsks: so are we using spaces or tabs?19:41
larskssdake wants to use "bashate", which seems to like spaces.19:42
larsksI prefer tabs, but not enough to whinge it.  So spaces, I guess.19:42
jpeeleri'm going to approve this change, but it has tabs19:43
larsksGo ahead.  At some point sdake will fix the gate and we will have a massive fix-all-the-bashate-warnings day.19:43
openstackgerritA change was merged to stackforge/kolla: correct missing backslash in config-nova-controller.sh  https://review.openstack.org/12902219:44
jpeelerlarsks: do you know if i delete a pod and docker is still downloading the image, if it can be resumed upon relaunching the pod?19:45
larsksjpeeler: I am having trouble parsing that question. Here is my best answer: the "docker pull" operation is done by the daemon, and will generally continue even if you cancel the client that initiated the request.19:46
*** cody-somerville has quit IRC19:46
larsksSo a "docker pull" will probably continue even if you stop the pod that triggered it.19:46
larsksDoes that help?19:47
sdakeI like tabs as well19:47
sdakeopenstack is anti tab for some reason :(19:47
jpeeleryes, i figured that was the case19:49
openstackgerritDaneyon Hansen proposed a change to stackforge/kolla: Adds Neutron Server Support  https://review.openstack.org/12855819:50
*** ramishra has joined #tripleo19:54
*** derekh has quit IRC19:54
*** cody-somerville has joined #tripleo19:58
*** cody-somerville has joined #tripleo19:58
*** ramishra has quit IRC19:59
*** gfidente has quit IRC20:00
openstackgerritA change was merged to openstack/diskimage-builder: Force empty $TMPDIR inside the chroot  https://review.openstack.org/12869620:04
*** noslzzp has joined #tripleo20:05
SpamapSsdake: https://etherpad.openstack.org/p/kilo-tripleo-summit-topics wants Kolla topics. :)20:06
*** daneyon has quit IRC20:11
*** pensu has quit IRC20:12
*** daneyon has joined #tripleo20:12
*** pradeep has joined #tripleo20:14
*** pradeep1 has quit IRC20:14
*** jehb has quit IRC20:23
greghaynesSpamapS: I notice there is no name by the ansible for updates session...20:24
greghayneshint hint hint20:24
SpamapSgreghaynes: Do we need names?20:25
SpamapSgreghaynes: it sprang forth from the mountain like a rainbow of truth20:25
greghaynes*shrug* all the otheres have names. Last time there had to be a person who actually 'proposed' each one20:25
greghaynesAh, the session of immaculate conception20:25
* SpamapS credits the flying spaghetti monster20:26
slaglethat guy is busy20:34
greghaynesSpamapS: re: https://review.openstack.org/#/c/113358/ - wha?20:35
greghaynesdoes code exist to do that?20:36
*** noslzzp has quit IRC20:37
*** noslzzp has joined #tripleo20:40
openstackgerritSteve Baker proposed a change to openstack/os-collect-config: Collector which does a simple HTTP GET  https://review.openstack.org/12847920:40
sdakeSpamapS I won't be at kilo summit, but larsks will be20:41
*** ChuckC has quit IRC20:42
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: fix typo in crudini command  https://review.openstack.org/12904220:43
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: add /check.sh script to nova-api image  https://review.openstack.org/12904320:44
*** cody-somerville has quit IRC20:46
*** noslzzp has quit IRC20:46
sdakelarsks can you ack this https://review.openstack.org/#/c/129019/20:46
openstackgerritA change was merged to stackforge/kolla: Wrong shell script specified in start.sh  https://review.openstack.org/12901920:47
*** noslzzp has joined #tripleo20:52
openstackgerritA change was merged to stackforge/kolla: fix typo in crudini command  https://review.openstack.org/12904220:53
openstackgerritA change was merged to stackforge/kolla: add /check.sh script to nova-api image  https://review.openstack.org/12904320:53
*** rainya has quit IRC20:54
*** ramishra has joined #tripleo20:55
*** ramishra has quit IRC20:59
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/138140821:00
uvirtbotLaunchpad bug 1381408 in diskimage-builder ""visudo" always got error when building overcloud image" [Undecided,New]21:00
*** untriaged-bot has quit IRC21:00
*** jdob has quit IRC21:01
*** rainya has joined #tripleo21:03
openstackgerritgreghaynes proposed a change to openstack/tripleo-image-elements: Add SSL PKI properties for keystone  https://review.openstack.org/12904921:05
openstackgerritBen Nemec proposed a change to openstack/tripleo-incubator: Check for firewalld before starting seed  https://review.openstack.org/12905121:09
openstackgerritgreghaynes proposed a change to openstack/tripleo-heat-templates: Add SSL PKI properties for keystone  https://review.openstack.org/12905221:11
*** ChuckC has joined #tripleo21:13
openstackgerritA change was merged to openstack/diskimage-builder: Consolidate lsb_release source-repositories  https://review.openstack.org/12843421:14
*** noslzzp has quit IRC21:16
sdakenova-api working !!21:17
openstackgerritSteven Dake proposed a change to stackforge/kolla: Run nova-api as privileged so it can muck with iptables  https://review.openstack.org/12905421:19
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: fix sql typo in nova-conductor start script  https://review.openstack.org/12905621:22
*** andreaf has joined #tripleo21:23
*** rainya has quit IRC21:24
*** cody-somerville has joined #tripleo21:25
*** cody-somerville has joined #tripleo21:25
*** rainya has joined #tripleo21:25
tchaypoStevenK: it came to me in a dream21:25
tchaypothat your register-nodes work is in os-cloud-config21:26
tchaypoI'm hoping my dream is accurate21:26
openstackgerritLars Kellogg-Stedman proposed a change to stackforge/kolla: configure nova database connection  https://review.openstack.org/12905921:26
openstackgerritA change was merged to stackforge/kolla: fix sql typo in nova-conductor start script  https://review.openstack.org/12905621:27
openstackgerritgreghaynes proposed a change to openstack/os-cloud-config: Make keystone_pki less keystone specific  https://review.openstack.org/12906221:28
greghaynesStevenK: ^ Your favorite patch!21:28
*** rainya has quit IRC21:31
*** rainya has joined #tripleo21:31
*** cody-somerville has quit IRC21:35
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/tripleo-image-elements: Ensure password is set correctly on existing users  https://review.openstack.org/9220721:39
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/tripleo-image-elements: Add an element to restore ssh keys from /mnt/state  https://review.openstack.org/11436021:42
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Migrates from map-services to svc-map  https://review.openstack.org/12906521:45
*** julim has quit IRC21:47
*** spzala has quit IRC21:47
*** spzala has joined #tripleo21:48
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Migrate from map-services to svc-map  https://review.openstack.org/12906621:49
*** ramishra has joined #tripleo21:55
openstackgerritA change was merged to openstack/diskimage-builder: Allow for multiple image outputs from raw source  https://review.openstack.org/12547521:56
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Migrates corosync element to svc-map  https://review.openstack.org/12907221:56
*** cody-somerville has joined #tripleo22:00
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Migrate geard to svc-map  https://review.openstack.org/12907322:00
*** cody-somerville has quit IRC22:00
*** cody-somerville has joined #tripleo22:00
*** ramishra has quit IRC22:00
sdakenova                kollaglue/fedora-rdo-nova-conductor,kollaglue/fedora-rdo-nova-scheduler,kollaglue/fedora-rdo-nova-api   10.0.0.4/           name=nova-controller   Running22:00
sdakelarsks ^^22:00
sdakeRUNNING ftw22:00
sdakenow for compute22:01
*** rhallisey has quit IRC22:02
*** radez is now known as radez_g0n322:04
sdakejpeeler radez_g0ne can you g uys take a quick look at https://review.openstack.org/#/c/129054/22:04
sdakeor if larsks is around as well22:04
*** rainya has quit IRC22:05
*** pradeep1 has joined #tripleo22:05
*** rainya has joined #tripleo22:06
*** pradeep has quit IRC22:07
*** rwsu has quit IRC22:07
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Create RabbitMQ users on controller  https://review.openstack.org/9220422:08
larskssdake: not actually running, because it needs at least https://review.openstack.org/#/c/129059/22:11
larsksI will take a look at that review...22:12
*** funzo has quit IRC22:15
*** funzo has joined #tripleo22:15
*** pradk has quit IRC22:18
*** sdake has quit IRC22:21
*** larstobi has quit IRC22:23
*** pcrews has quit IRC22:24
*** larstobi has joined #tripleo22:29
*** pcrews has joined #tripleo22:30
*** rhallisey has joined #tripleo22:32
openstackgerritIan Wienand proposed a change to openstack/diskimage-builder: Standarise tracing for scripts  https://review.openstack.org/11902322:33
SpamapShttp://logs.openstack.org/60/114360/9/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/d5638f0/console.html#_2014-10-16_22_20_56_97422:37
SpamapShas anybody seen this fail?22:37
*** sdake has joined #tripleo22:37
*** sdake has quit IRC22:37
*** sdake has joined #tripleo22:37
greghaynesSpamapS: hrm, its annoying we dont have haproxy logs22:40
sdakelarsks oh yes, I had that locally, I guess I missed your review22:42
sdakecan you set max_retries to something like 5, so the container exits if the db isn't available22:43
sdakeotherwise the container will spin forever and the db may never come up22:43
sdakewhich would make diagnostics a bit difficult I think22:43
sdake\o/ yay for controller22:48
sdakenow to beat compute into submission22:48
SpamapSgreghaynes: very22:49
greghaynesSpamapS: that does really look like a haproxy doesnt have a backend error22:49
greghaynesSpamapS: since theres not other errors in any of the logs22:49
SpamapSI like our little mini-signal implementation with nc22:52
SpamapShad forgotten somebody got that working. :-P22:52
greghaynesyea, thats impressive. My big gripe is that it doesnt respect ctrl+c22:52
*** ramishra has joined #tripleo22:56
greghaynesSpamapS: So there is a really interesting issue ive run into with our update playbook. We have a lot of sets of two tasks that both register a variable foo, one if helion defined, one if !defined22:58
SpamapSgreghaynes: also notice in that error series, os-collect-config log ends 7 minutes earlier, with an error... that is weird22:58
greghaynesturns out whatever runs last overwites the value of the variable22:58
SpamapSgreghaynes: yeah a lot of that was because we used to have two distinct sets of services22:59
SpamapSwell we do still actually22:59
greghaynesyep, so for registering variables that doesnt work so much22:59
SpamapSI think the end goal is to actually not have a 'helion' mode, but a helion branch that has roles that override this stuff.22:59
greghaynesyea, If only include had a when: property...23:00
SpamapSit does sort of23:00
SpamapSuse roles :)23:00
SpamapSI'm learning about them now and I have a restructuring plan23:00
SpamapSbut I want to have CI first23:00
greghayneshaha, ok, I should also read up on that23:00
*** ramishra has quit IRC23:01
SpamapSSo the idea is to have the role be 'tripleo' and then helion can override that with 'helion' and provide a helion role that diverges.23:01
SpamapSI think23:01
SpamapSanyway... brain is pretty much cooked today23:02
SpamapSand still no CI.. I think I need to give up trying to deploy a cloud on these blades and just setup a bunch of identical jenkins slaves. :-P23:02
SpamapStoo hard when it's not my network23:02
greghaynesyea, sounds like what they call environments in salt23:03
SpamapSit's more contained than that23:04
SpamapSgreghaynes: actually let me back up. The service list thing is a little wonky IMO. I think we might want to define that ordering in the elements.23:05
SpamapSgreghaynes: what we want to use roles for is to encapsulate the steps each class of servers need to go through. That way Helion with it's special groupings can just write a new workflow that fits in vsa and whatever else where it needs to be, but all of the "magic" is happening in roles underneath.23:07
SpamapSgreghaynes: so the idea is, let them patch the high level logical workflow, instead of having all this low level stuff in update_cloud.yml23:08
*** yamahata has joined #tripleo23:10
*** ChuckC___ has joined #tripleo23:13
*** marun has quit IRC23:26
*** ChuckC___ has quit IRC23:34
*** morazi has quit IRC23:35
*** marun has joined #tripleo23:37
*** cody-somerville has quit IRC23:39
*** saurabhs has quit IRC23:42
*** juice has quit IRC23:46
*** yamahata has quit IRC23:53
*** juice has joined #tripleo23:53
*** ramishra has joined #tripleo23:57
tchayporeboot undercloud control node, come up without /mnt/state, yay23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!