| greghaynes | oh, this different code path for seed vs everything else when we gen ssl certs is pretty gross | 00:01 |
|---|---|---|
| tchaypo | is this the keystone ssl cert generation dooby? | 00:01 |
| greghaynes | I have a funny idea - what if we emulated heat parameters to the seed using $templating, like load up a parameters environment then to a templating pass on teh seed config.json | 00:02 |
| tchaypo | and is the different code path the generate-files-in-a-directory vs put-certs-in-json? | 00:02 |
| tchaypo | because that's where I'm at right now.. | 00:02 |
| greghaynes | yep | 00:02 |
| greghaynes | Im tring to generalise it for more than just keystone | 00:02 |
| tchaypo | right now I've confirmed that I can replicate the behaviour I'm seeing in my build | 00:04 |
| tchaypo | ie it generates ca.pem and signing_cert.pem which just have a newline character | 00:04 |
| tchaypo | which is not super handy | 00:04 |
| greghaynes | tchaypo: does your seed config.json have a keystone.signing_key parameter? | 00:04 |
| greghaynes | (very likely a no) | 00:05 |
| tchaypo | nope | 00:05 |
| tchaypo | *checks out mater and pulls* | 00:06 |
| greghaynes | So youre probably running the t-i-e which templates out from those parameters, but os-c-c isnt supplying it | 00:06 |
| tchaypo | woah that's had some changes lately | 00:06 |
| greghaynes | the other os-c-c, os-cloud-config | 00:06 |
| tchaypo | mayhap i should rebase... | 00:06 |
| tchaypo | okay, rebased, still no signing_key | 00:08 |
| greghaynes | what is your os-cloud-config version? | 00:09 |
| *** yamahata has quit IRC | 00:09 | |
| tchaypo | b98434 (HEAD, origin/master, origin/HEAD, master) Merge "Add public endpoint support to init-keystone" | 00:09 |
| greghaynes | we install that from pip | 00:09 |
| tchaypo | ah. | 00:09 |
| lifeless | is your mirror up to date? | 00:10 |
| greghaynes | so $TRIPLEO_ROOT/tripleo-incubator/openstack-tool/bin/pip freeze | 00:10 |
| greghaynes | will tell you | 00:10 |
| greghaynes | er, openstack-tools | 00:10 |
| lifeless | yeah, we ain't a tool | 00:10 |
| lifeless | we're tools | 00:10 |
| lifeless | greghaynes: so thoughts on the ssl thing ? | 00:10 |
| tchaypo | 0.1.2 | 00:11 |
| greghaynes | tchaypo: thats what I have, odd | 00:11 |
| greghaynes | lifeless: Is it happening now in CI? I thought I was holding off on you to overcloud | 00:11 |
| lifeless | greghaynes: well, if you need input data | 00:11 |
| *** ci-overcloud has joined #tripleo | 00:11 | |
| ci-overcloud | ************** ci-overcloud complete status=127 ************ | 00:11 |
| *** ci-overcloud has quit IRC | 00:11 | |
| lifeless | greghaynes: but to see whats going on - the script to deploy is in tie in tripleo-cd deploy-ci-overcloud | 00:12 |
| lifeless | wat | 00:12 |
| lifeless | + register-state-path --leave-symlink /etc/libvirt/qemu | 00:12 |
| lifeless | /tmp/in_target.d/install.d/81-nova-kvm: line 10: register-state-path: command not found | 00:12 |
| greghaynes | oh, so just run that locally with an undercloud sourced and should repro? | 00:12 |
| lifeless | there's a heat environment file in configs/ for it | 00:12 |
| greghaynes | kk | 00:12 |
| * greghaynes reads codes | 00:13 | |
| lifeless | and you'll need a vlan enabled undercloud | 00:13 |
| greghaynes | oh joy | 00:13 |
| greghaynes | :) | 00:13 |
| lifeless | see the tripleo-end-to-end refs I posted above | 00:13 |
| greghaynes | Has anyone mentioned getting that tool added into -incubator? | 00:13 |
| lifeless | greghaynes: see review 96659 | 00:14 |
| greghaynes | tchaypo: maybe youre not running the right -incubator version for it to actually call generate-keystone-pki? | 00:15 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Revert "Move libvirt's qemu configuration dir to state fs" https://review.openstack.org/107209 | 00:19 |
| lifeless | can I have a second eyeball on https://review.openstack.org/#/c/107209/ ? | 00:19 |
| * tchaypo rebases all the things | 00:19 | |
| lifeless | greghaynes: ^ | 00:19 |
| lifeless | adam_g: ^ | 00:20 |
| lifeless | StevenK: I thought you nuked the bakup-of-images-thing ? | 00:21 |
| adam_g | lifeless, hmm lookin | 00:21 |
| greghaynes | today is revert day | 00:21 |
| adam_g | lifeless, would a one-liner fix be easier than the revert at this point? | 00:23 |
| lifeless | adam_g: never roll forward :) | 00:24 |
| adam_g | lifeless, sure, in that case revert away. ill push another libvirt fix with the proper dep added | 00:24 |
| lifeless | adam_g: always assume there will be some surprise, and rollback unless its actually really hard to do so | 00:24 |
| lifeless | greghaynes: so +2 from you? | 00:29 |
| lifeless | SpamapS: https://review.openstack.org/#/c/107209/ | 00:29 |
| lifeless | who else might be around | 00:30 |
| lifeless | bnemec: https://review.openstack.org/#/c/107209/ | 00:30 |
| greghaynes | The issue is jsut missing the dep? | 00:31 |
| greghaynes | and youre building on something without preserve-ephemeral? | 00:32 |
| greghaynes | er, use-ephemeral | 00:32 |
| lifeless | greghaynes: I'm running deploy-ci-overcloud | 00:34 |
| lifeless | export DIB_COMMON_ELEMENTS="stackuser hosts" | 00:34 |
| lifeless | I'm guessing that what we've just discovered is that using environment variables is a bad idea | 00:34 |
| lifeless | ah yes | 00:34 |
| greghaynes | yep, we define use_ephemeral in _variables | 00:35 |
| lifeless | DIB_COMMON_ELEMENTS MUST NOT be set by users because we aren't paranoid about it. | 00:35 |
| lifeless | anyhow, yes the dependency is missing | 00:35 |
| *** Penick has joined #tripleo | 00:35 | |
| greghaynes | +2 | 00:35 |
| tchaypo | rebased t-i-e, incubator, and t-h-t. I should be running tip for everything else. Still getting the same issue. | 00:36 |
| tchaypo | have to run to an optometrist appointment | 00:36 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Add a hp1 region configs. https://review.openstack.org/96666 | 00:36 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: HP1 region deploy config fixups. https://review.openstack.org/99799 | 00:36 |
| greghaynes | tchaypo: fun | 00:37 |
| *** tzumainn has quit IRC | 00:38 | |
| *** xuhaiwei has joined #tripleo | 00:40 | |
| StevenK | lifeless: But DIB_COMMON_ELEMENTS is used by everything. :-( Perhaps we need a DIB_EXTRA_ELEMENTS or something | 00:40 |
| lifeless | StevenK: if users set it to a value, when we change it, we break users. | 00:42 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: HP1 region deploy config fixups. https://review.openstack.org/99799 | 00:45 |
| *** Penick has quit IRC | 00:48 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Deploy cloud hypervisor type https://review.openstack.org/94586 | 00:52 |
| * StevenK stabs bandersnatch in the face | 00:54 | |
| StevenK | It seems it either wants to log its output to stdout/stderr or nowhere | 00:55 |
| *** ci-overcloud has joined #tripleo | 01:01 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 01:01 |
| *** ci-overcloud has quit IRC | 01:01 | |
| *** chuckC has quit IRC | 01:03 | |
| *** yamahata has joined #tripleo | 01:09 | |
| *** cinerama has quit IRC | 01:09 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: HP1 region deploy config fixups. https://review.openstack.org/99799 | 01:11 |
| *** ci-overcloud has joined #tripleo | 01:14 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 01:14 |
| *** ci-overcloud has quit IRC | 01:14 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Define and bind a public virtual IP https://review.openstack.org/101438 | 01:18 |
| *** ci-overcloud has joined #tripleo | 01:23 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 01:23 |
| *** ci-overcloud has quit IRC | 01:23 | |
| lifeless | oh arggghhh | 01:24 |
| lifeless | hmm | 01:24 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Define and bind a public virtual IP https://review.openstack.org/101438 | 01:24 |
| *** pmyers has quit IRC | 01:27 | |
| openstackgerrit | Ian Wienand proposed a change to openstack/diskimage-builder: Initial centos7 support https://review.openstack.org/104449 | 01:37 |
| lifeless | hmmoh great | 01:39 |
| lifeless | locked up IPMI | 01:39 |
| lifeless | adam_g: did Ironic land the rate limiting tstuff ? | 01:39 |
| lifeless | Server Power: Reset | 01:40 |
| *** nosnos has joined #tripleo | 01:58 | |
| lifeless | \n', "InternalServerError: RPC do_node_tear_down failed to validate deploy info. Error: Can not validate PXE bootloader. The following parameters were not passed to ironic: ['ro | 02:05 |
| lifeless | 02:05 | |
| lifeless | grumble | 02:06 |
| lifeless | manual cleanup time | 02:06 |
| lifeless | adam_g: (unless you happen to have a patch stashed somewhere for that ?) | 02:06 |
| *** Yanyanhu has joined #tripleo | 02:08 | |
| *** eghobo has quit IRC | 02:13 | |
| *** ci-overcloud has joined #tripleo | 02:13 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 02:13 |
| *** ci-overcloud has quit IRC | 02:13 | |
| *** chuckC has joined #tripleo | 02:17 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: HP1 region deploy config fixups. https://review.openstack.org/99799 | 02:44 |
| lifeless | so this will be fun | 02:50 |
| lifeless | 3 node ctl plane + VLAN + public VIP | 02:50 |
| *** nati_ueno has quit IRC | 02:57 | |
| *** ramishra has joined #tripleo | 02:59 | |
| *** weshay has quit IRC | 03:00 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 03:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1341936 | 03:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 03:00 |
| uvirtbot | Launchpad bug 1341936 in os-collect-config "Require keystone v3 auth_url in heat collector" [Undecided,New] | 03:00 |
| *** weshay has joined #tripleo | 03:05 | |
| lifeless | ulp, another locked up ilo. sadface | 03:08 |
| lifeless | when ilos attack | 03:08 |
| *** Yanyan has joined #tripleo | 03:29 | |
| *** Yanyanhu has quit IRC | 03:29 | |
| lifeless | ahha | 03:39 |
| lifeless | http://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdHome/?sp4ts.oid=4154847&spf_p.tpst=swdMain&spf_p.prp_swdMain=wsrp-navigationalState%3DswEnvOID%253D4155%257CswLang%253D%257Caction%253DlistDriver&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken may be useful | 03:39 |
| adam_g | lifeless, there were a couple commits to address that https://bugs.launchpad.net/ironic/+bug/1320513 | 03:40 |
| uvirtbot | Launchpad bug 1320513 in nova "IPMI commands are sent / queried too fast" [Low,Confirmed] | 03:40 |
| adam_g | gotta run | 03:41 |
| lifeless | ciao | 03:42 |
| *** nosnos has quit IRC | 03:43 | |
| *** ci-overcloud has joined #tripleo | 03:53 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 03:53 |
| *** ci-overcloud has quit IRC | 03:53 | |
| *** bmahalakshmi has joined #tripleo | 04:01 | |
| *** eghobo has joined #tripleo | 04:09 | |
| *** kevinbenton has joined #tripleo | 04:13 | |
| *** weshay has quit IRC | 04:14 | |
| *** eghobo has quit IRC | 04:14 | |
| *** eghobo has joined #tripleo | 04:15 | |
| *** Yanyan has quit IRC | 04:18 | |
| tchaypo | greghaynes: what version of keystone do you have in your seed? | 04:25 |
| tchaypo | if if activate the /opt/stack/venves/keystone venv, pip show keystone tells me I have 2014.2.dev158.gbbfd58a | 04:25 |
| lifeless | tchaypo: surely its the bad certs in the file ? | 04:27 |
| *** eghobo has quit IRC | 04:27 | |
| tchaypo | If by file you mean the seed's config.json, there are no certs | 04:27 |
| tchaypo | yeah, that would explain it, but why aren't they there? | 04:28 |
| tchaypo | and to answer my own question | 04:28 |
| tchaypo | it's either because they're static and in the repo and I'm missing a commit | 04:28 |
| *** Yanyanhu has joined #tripleo | 04:28 | |
| tchaypo | or because they haven't been generated because my existing env hasn't been updated, either because I was using -c or possibly just because it's one of those things that doesn't touch existing files | 04:28 |
| lifeless | no, generate-keystone-pki is meant to be called from the seed sccript | 04:29 |
| tchaypo | but neither of the things I just said make sense because the seed generates the keys and writes them into files, as far as I can tell it's not reading them from heat | 04:29 |
| lifeless | it doesn't have heat | 04:30 |
| lifeless | but it gets them from a json file | 04:30 |
| tchaypo | *nods* | 04:30 |
| lifeless | in /var/lib/heat-cfntools/cfn-init-data | 04:30 |
| lifeless | which devtest_seed creates | 04:30 |
| *** killer_prince is now known as lazy_prince | 04:31 | |
| tchaypo | no certs there. Under the ekystone key there's a host and a db but that's all | 04:31 |
| tchaypo | that looks a lot like config.json? | 04:31 |
| tchaypo | yep, that seems to be config.json | 04:32 |
| *** nosnos has joined #tripleo | 04:32 | |
| tchaypo | which git tells me has a history, so it's in the repo | 04:33 |
| *** slok184 has joined #tripleo | 04:33 | |
| tchaypo | the t-i-e repo | 04:33 |
| tchaypo | I'm not seeing any changes to the file since april 9 though | 04:33 |
| tchaypo | so devtest_seed seems to combine that file and $TE_DATAFILE to create tmp_local.json | 04:35 |
| tchaypo | let me remove -c and see if that helps - i think it should blow away $TE_DATAFILE | 04:36 |
| tchaypo | f it doesn't I'll remove it by hand | 04:36 |
| StevenK | | 0dfade63-0d2f-45b0-a25a-5a07b311eba7 | overcloud-controller2-7e2d7dp37yhc | ERROR | - | NOSTATE | | | 04:39 |
| StevenK | Grrr | 04:40 |
| *** Yanyan has joined #tripleo | 04:43 | |
| *** ramishra has quit IRC | 04:43 | |
| *** ramishra_ has joined #tripleo | 04:44 | |
| *** Yanyanhu has quit IRC | 04:46 | |
| tchaypo | hrm. just had lunch but light-headed and sleepy. | 04:57 |
| tchaypo | *checks sleep logger* Okay, getting just 4:20 sleep last night may not have been optimal | 04:57 |
| tchaypo | that's 4h 20m, not any other interpretatation of "4:20 sleep" | 04:58 |
| tchaypo | *sigh* now I can't build because my apt-mirror has a hash sum mismatch on the security repo | 05:03 |
| *** bnemec has quit IRC | 05:03 | |
| * tchaypo trys again with upstream repos | 05:04 | |
| tchaypo | yep, that seems to work for some reason | 05:04 |
| tchaypo | that seems to be going better | 05:06 |
| *** tchaypo has quit IRC | 05:09 | |
| *** tchaypo has joined #tripleo | 05:09 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Compile and install custom SELinux policies https://review.openstack.org/106909 | 05:19 |
| *** rakesh_hs has joined #tripleo | 05:20 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Allow mysql access to port 4567 https://review.openstack.org/107232 | 05:24 |
| tchaypo | success! | 05:25 |
| tchaypo | i get a completely different error now | 05:26 |
| tchaypo | http://paste.openstack.org/show/86651/ | 05:26 |
| *** shausy has joined #tripleo | 05:26 | |
| *** marios_ has quit IRC | 05:27 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Custom policy for ssh-keygen failure https://review.openstack.org/107233 | 05:28 |
| *** marios has joined #tripleo | 05:28 | |
| StevenK | tchaypo: Huh | 05:29 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Enable haproxy_connect_any port access https://review.openstack.org/107234 | 05:32 |
| tchaypo | so looking at the subnet-create line | 05:34 |
| tchaypo | neutron subnet-create's help tells me it expects just to positional parameters - NETWORK and CIDR | 05:35 |
| tchaypo | but that commandline passes 3, which seem to be the gateway IP, the network ID, and the network CIDR | 05:35 |
| *** sseago has quit IRC | 05:36 | |
| *** rushiagr has joined #tripleo | 05:37 | |
| *** andreaf has quit IRC | 05:37 | |
| tchaypo | looking at setup-neutron, lines 81-87, looks like that is meant to be $NET_ID, $NETWORK_CIDR, $SUBNET_EXTRA | 05:38 |
| lifeless | gateway is duplicated | 05:39 |
| *** shausy2 has joined #tripleo | 05:46 | |
| *** akuznetsov has quit IRC | 05:47 | |
| *** shausy has quit IRC | 05:47 | |
| *** sseago has joined #tripleo | 05:48 | |
| *** chuckC has quit IRC | 05:51 | |
| *** chuckC has joined #tripleo | 05:52 | |
| tchaypo | yep. found the cause. see comment on https://review.openstack.org/#/c/103757/16/scripts/setup-neutron | 05:53 |
| tchaypo | the fix is clear so I'll push it momentarily. | 05:53 |
| *** dshulyak_ has joined #tripleo | 05:53 | |
| *** slok184 has quit IRC | 05:54 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Add public VLAN support. https://review.openstack.org/103757 | 05:56 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Add --public to setup-endpoints. https://review.openstack.org/105045 | 05:56 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Fix setup-endpoints handling of public URLs. https://review.openstack.org/105044 | 05:56 |
| tchaypo | (if anyone wants to tell me that momentarily means "for" rather than "in" a moment, I have citations of the former use going back to 1899 to show that it's long been used that way) | 05:59 |
| * tchaypo kicks off yet another build, and goes away to study german | 06:00 | |
| lifeless | tchaypo: I think you're wrong | 06:03 |
| lifeless | tchaypo: aesthetically, anyhow. | 06:06 |
| tchaypo | tis possible | 06:07 |
| tchaypo | http://paste.openstack.org/show/86656/ shows what I did to check the syntax | 06:07 |
| *** akuznetsov has joined #tripleo | 06:07 | |
| tchaypo | on the other hand | 06:07 |
| lifeless | what we've done elsewhere for ternary is | 06:07 |
| * lifeless pages it in | 06:07 | |
| tchaypo | http://logs.openstack.org/57/103757/16/check-tripleo/check-tripleo-undercloud-precise/ef8dec4/console.html doesn't show the double gateway | 06:08 |
| lifeless | if set, new value, else if not set notset-value | 06:08 |
| lifeless | which looks like | 06:08 |
| lifeless | ${${FOO:+newvalue:-notsetvalue} | 06:08 |
| lifeless | from memory | 06:08 |
| *** slok184 has joined #tripleo | 06:09 | |
| tchaypo | I think you're missing a closing } | 06:09 |
| lifeless | right so probably we aren't always passing the gateway in and neutrons default-to-.1 was making things work | 06:09 |
| lifeless | now that we want no gateway somtimes, that will break. probably. | 06:09 |
| tchaypo | do you know where I could look for an example of that? | 06:12 |
| *** slok184 has quit IRC | 06:12 | |
| lifeless | which that to that do you refer when you say that? | 06:15 |
| tchaypo | ${${FOO:+.... | 06:15 |
| tchaypo | I'm playing with it but I can't get it to work | 06:16 |
| tchaypo | ... grep should find it for me, one momen | 06:16 |
| tchaypo | t | 06:16 |
| lifeless | IIRC the SSL stuff in setup-endpoints or whatever used to use it | 06:17 |
| tchaypo | grep doesn't find me any lines containing both :+ and :- in all of ~/.cache/tripleo, so if it's there it's spread across multiple lines | 06:19 |
| tchaypo | I can find | 06:19 |
| tchaypo | SSL_PORT=${SSL:+13777} | 06:20 |
| tchaypo | SSL_PORT={SSL_PORT:-$NORMAL_PORT} | 06:20 |
| *** regebro has joined #tripleo | 06:21 | |
| *** pcrews has quit IRC | 06:21 | |
| tchaypo | in case it wasn't clear, I agree that what I've done is ugly and I'd like to get something more standard or better-looking in | 06:22 |
| *** nati_ueno has joined #tripleo | 06:26 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Add public VLAN support. https://review.openstack.org/103757 | 06:29 |
| tchaypo | but for now - I seem to have a working seed and my undercloud is being built | 06:30 |
| tchaypo | no wait, i have a working undercloud and my overcloud is being built | 06:30 |
| * tchaypo really goes away to german this time | 06:30 | |
| *** markmc has joined #tripleo | 06:31 | |
| *** jcoufal has joined #tripleo | 06:38 | |
| proffalken | Hey all you lovely cores out there... Whilst I work out how to appease some of you over my abuse of o-a-c for installing check_mk server, I'd really welcome a final +2/+A on https://review.openstack.org/#/c/91324/ so I can at least allow people to start writing checks :) | 06:49 |
| *** mrunge has joined #tripleo | 06:51 | |
| *** mrunge has quit IRC | 06:52 | |
| *** mrunge has joined #tripleo | 06:57 | |
| xuhaiwei | lifeless: Sorry to trouble again, I am using seedcloud to deploy a baremetal undercloud with nova-baremetal manually, and the deployment succeeded. Now I want to use heat to do the same job, but stack-create failed | 07:02 |
| greghaynes | tchaypo: I like that youve solved all the errors im runing into in backscroll | 07:02 |
| greghaynes | tchaypo: including mirror hash mismatch | 07:03 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Auto-collect local checks https://review.openstack.org/91324 | 07:03 |
| greghaynes | tchaypo: FYI - I fixed my mirror by rm-ing my dists/trusty-updates/universe/ and /main dirs | 07:04 |
| tchaypo | Well, I didn't solve that one | 07:04 |
| greghaynes | on the mirror | 07:04 |
| tchaypo | Just ignored it | 07:04 |
| tchaypo | Thanks. I'll try that tomorrow. | 07:04 |
| xuhaiwei | lifeless: I just got the same result with "nova boot --flavor baremtal ....", and openstack is not started in undercloud. | 07:06 |
| *** jtomasek has joined #tripleo | 07:06 | |
| *** rdopieralski has joined #tripleo | 07:06 | |
| lifeless | xuhaiwei: nova show should tell you whats wrong | 07:06 |
| lifeless | xuhaiwei: at least some of it | 07:06 |
| *** d0ugal has quit IRC | 07:07 | |
| proffalken | lifeless: thanks for the +2 on check_mk auto-collect checks, slightly confused by the "please don't mix patches in future" - is that because of the iptables stuff? | 07:07 |
| lifeless | greghaynes: tchaypo: you should file a bug on apt-mirror | 07:07 |
| lifeless | proffalken: yup | 07:07 |
| proffalken | lifeless: fair enough, message received and understood... ;) | 07:07 |
| *** d0ugal has joined #tripleo | 07:07 | |
| xuhaiwei | lifeless: I mean for nova list I can see the instance's state is 'ACTIVE', so no error | 07:07 |
| lifeless | proffalken: https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes | 07:07 |
| xuhaiwei | lifeless: but stack-list shows the stack failed | 07:08 |
| lifeless | xuhaiwei: stack-show $stack will show you the reason it failed | 07:08 |
| proffalken | lifeless: yeah, I understand, think I might have done a "git add ." at some point and not realised | 07:09 |
| lifeless | proffalken: cool | 07:10 |
| xuhaiwei | lifeless: no error reason at all in stack-show, form resource-list I can see 99_undercloudDeployment failed | 07:10 |
| lifeless | proffalken: I'm training my fingerse to only ever add -u | 07:10 |
| proffalken | lol, yeah, muscle memory can be a real bastard some times :D | 07:11 |
| lifeless | xuhaiwei: that means the code in the instance failed; log into the machine and look in /var/log/upstart/os-collect-config.log | 07:11 |
| xuhaiwei | lifeless: thank you | 07:11 |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install the "classic" icinga interface https://review.openstack.org/87223 | 07:14 |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install the "classic" icinga interface https://review.openstack.org/87223 | 07:15 |
| *** marun is now known as marun_afk | 07:16 | |
| lifeless | ok we have a massive missing trap in the dib code :/ trivially leaving mounted fs's in post-install.d errors. | 07:17 |
| * proffalken really hopes that it was the above patch that made lifeless aware of that one... | 07:18 | |
| *** pblaho has joined #tripleo | 07:18 | |
| lifeless | nope | 07:18 |
| proffalken | s/was/wasn't/g | 07:18 |
| proffalken | :) | 07:18 |
| proffalken | so whilst I'm busy trying to rectify my misappropriation of o-r-c and o-a-c, where should crontabs live? /etc/cron.d or /mnt/state/etc/cron.d? | 07:22 |
| lifeless | /etc/cron.d | 07:22 |
| lifeless | they are software, not state | 07:22 |
| proffalken | ok | 07:22 |
| lifeless | (if they are state, then thats a whole new problem to solve ;P) | 07:22 |
| proffalken | do we have a clear-cut list of what is software and what is state anywhere? | 07:22 |
| proffalken | or do I get to find out with each new review I propose? ;) | 07:22 |
| lifeless | if you run tests after changing it before deploying it, its software :P | 07:23 |
| proffalken | (I'm happy to work on writing one up btw if it doesn't exist) | 07:23 |
| *** dshulyak_ has quit IRC | 07:23 | |
| lifeless | I don't think its super clearly defined | 07:23 |
| proffalken | lifeless: where tests == (unit tests || functional tests || monitoring checks) ? | 07:24 |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Add a ramdisk-install.d hook path. https://review.openstack.org/107246 | 07:24 |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Add support for flashing ILO BMC's from ramdisks https://review.openstack.org/107247 | 07:24 |
| lifeless | proffalken: ahhha I saw what you did there :) - its a slippery slope right :) | 07:24 |
| proffalken | indeed... | 07:24 |
| proffalken | :D | 07:24 |
| proffalken | so I'm coming at this from a sysadmin background, not a dev one (although technically I'm a developer these days) | 07:25 |
| proffalken | "tests" is a very fluffy word to me as I consider a system "tested" if my monitoring is all green | 07:25 |
| proffalken | however I'm also getting used to writing unit tests | 07:26 |
| proffalken | although most unit tests appear to also be functional in nature if having to deal with data stores etc | 07:26 |
| tchaypo | lifeless: I'll chase that tomorrow if greghaynes doesn't beat me too it | 07:26 |
| proffalken | I would argue that cronjobs are part of system *state* in the sense that they are generally applied post-installation and are specific to a task | 07:27 |
| proffalken | however given that I just opened up a whole new world of pain, I'm happy to put them into /etc/cron.d for now ;) | 07:27 |
| lifeless | proffalken: they are generally coupled to a piece of software though, right? | 07:28 |
| *** jistr has joined #tripleo | 07:33 | |
| *** cody-somerville has joined #tripleo | 07:36 | |
| *** rameshg87 has joined #tripleo | 07:36 | |
| proffalken | they are, but are they not configuration files for cron.d that we want to keep between reboots/rebuilds? | 07:41 |
| proffalken | I guess that there are times where you want to configure cron at boot/image build, and other times when you want to add them as part of o-r-c/o-a-c | 07:41 |
| *** ramishra_ has quit IRC | 07:42 | |
| proffalken | I'll leave it in /etc/cron.d for now, but I think we may want to look at this again in future as we get a clearer split between software and state | 07:42 |
| proffalken | :) | 07:42 |
| *** ci-overcloud has joined #tripleo | 07:42 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 07:42 |
| *** ci-overcloud has quit IRC | 07:42 | |
| lifeless | I wouldn't expect orc/oac to be creating cron jobs | 07:42 |
| lifeless | I'd expect cron jobs that read configuration | 07:42 |
| lifeless | that said, yes you can create cron jobs out of oac; if you're doing that they should be in /mnt/state because / is going to be readonly one of these days. | 07:43 |
| lifeless | so fingers crossed, I'm about to automatically update firmware on these machines. | 07:43 |
| lifeless | or its going to crash and burn | 07:46 |
| lifeless | I hope for !burn | 07:46 |
| * proffalken hands lifeless a fire extinguisher just in case | 07:47 | |
| lifeless | then since I don't know how to automate setting the iLO advanced licence | 07:47 |
| lifeless | I probably have 44 machines to manually add it to | 07:47 |
| proffalken | lifeless: HP hardware? | 07:48 |
| lifeless | proffalken: the fishbowl | 07:48 |
| lifeless | proffalken: in US East IIRC | 07:48 |
| proffalken | lifeless: I'll PM you a contact who may be able to help | 07:48 |
| lifeless | oh, thanks | 07:48 |
| lifeless | proffalken: thats what this patch: https://review.openstack.org/107247 is about | 07:49 |
| *** jang has joined #tripleo | 07:51 | |
| proffalken | lifeless: oooh, shiney... :) | 07:51 |
| lifeless | yah | 07:53 |
| lifeless | now of course the node I decided to look at to assess this working has a hung iLO | 07:53 |
| lifeless | which btw is why I'm doing this in the first place, to get past that :) | 07:53 |
| proffalken | :( | 07:53 |
| lifeless | its not meant to sit at | 07:53 |
| lifeless | </>hpiLO-> textcons | 07:53 |
| lifeless | indefinitely :/ | 07:53 |
| proffalken | I'd log that as a bug with HP and see if they fix it... ;) | 07:54 |
| proffalken | </trollbait> | 07:54 |
| lifeless | I have | 07:54 |
| lifeless | I'm in contact with the iLO team | 07:54 |
| lifeless | having found out that we have hugely old fw, fixing that systematically is an obvious first step | 07:54 |
| proffalken | +1 | 07:55 |
| lifeless | good news, I didn't brick anything | 07:55 |
| *** ChanServ changes topic to "Using OpenStack to deploy OpenStack;meetings Tuesday 1900//0700 UTC in #openstack-meeting-alt" | 07:55 | |
| lifeless | bad news, I didn't flash anything either | 07:55 |
| *** nati_ueno has quit IRC | 07:56 | |
| *** pblaho has quit IRC | 08:04 | |
| *** dtantsur|afk is now known as dtantsur | 08:06 | |
| *** derekh_ has joined #tripleo | 08:10 | |
| lifeless | derekh_: hi :) | 08:21 |
| lifeless | derekh_: I saw you had trouble with br-untagged? Its not needed at all anymore -see the deltas in 99799 and predecessor | 08:22 |
| *** akrivoka has joined #tripleo | 08:22 | |
| derekh_ | lifeless: looking | 08:23 |
| lifeless | derekh_: I'm just working through a fw update | 08:28 |
| lifeless | derekh_: we're locking up machines like crazy | 08:28 |
| lifeless | ilo 1.7 will still lockup, but much less often | 08:29 |
| *** akrivoka has quit IRC | 08:29 | |
| derekh_ | ok | 08:29 |
| *** akrivoka has joined #tripleo | 08:30 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Add a "driver" field to the Register Nodes dialog https://review.openstack.org/107052 | 08:34 |
| lifeless | bbiab, I still have a lock on the hp1 region sorry | 08:34 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: export HOSTIP https://review.openstack.org/106390 | 08:37 |
| *** lucasagomes has joined #tripleo | 08:39 | |
| derekh_ | Anybody else wanna add a second +2 too https://review.openstack.org/#/c/106852/ , StevenK I see your point for the -1 but really just wanna get this in and not go through another day of pointless CI runs | 08:45 |
| rdopieralski | jcoufal: so I'm looking at the "create deployment plan" wireframes | 08:46 |
| rdopieralski | jcoufal: and I wonder if it really should be a popup? | 08:46 |
| rdopieralski | jcoufal: in particular, what view should it pop up in? | 08:46 |
| rdopieralski | jcoufal: it looks more like a full view to me | 08:47 |
| StevenK | derekh_: Feel free to ignore my -1 if you just want to land it. | 08:48 |
| *** martyntaylor has joined #tripleo | 08:49 | |
| derekh_ | StevenK: thanks, | 08:50 |
| derekh_ | btw, zuul has just been restarted the queues are clear so now would be a excelant time to merge it | 08:50 |
| rameshg87 | derekh_: request your review on https://review.openstack.org/#/c/97143/ and https://review.openstack.org/#/c/97301/ | 08:50 |
| rameshg87 | derekh_: sorry for bugging constantly :-) | 08:50 |
| derekh_ | rameshg87: no prob will get to them as soon as I can | 08:51 |
| rameshg87 | derekh_: thanks :-) | 08:51 |
| jcoufal | rdopieralski: well you need to keep in mind that we will have index page for deployments (once we support multiple deployments) and we should be consistent with other "creation" workflows | 08:54 |
| rdopieralski | jcoufal: that doesn't answer my question | 08:55 |
| jcoufal | rdopieralski: when there is no deployment plan and you click on "Deployment" in navigation, you should get an empty page with a button to create one | 08:55 |
| rdopieralski | jcoufal: on what page should that popup appear? | 08:55 |
| jcoufal | rdopieralski: and there should the pop-up appear | 08:55 |
| rdopieralski | jcoufal: ok, thank you | 08:55 |
| jcoufal | rdopieralski: np | 08:55 |
| jcoufal | rdopieralski: I am trying to be consistent with other views and with respect to the future direction | 08:56 |
| *** cody-somerville has quit IRC | 08:59 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-specs: Targeted node deployment https://review.openstack.org/106996 | 09:01 |
| *** untriaged-bot has joined #tripleo | 09:01 | |
| untriaged-bot | Untriaged bugs so far: | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 09:01 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 09:01 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 09:01 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1341936 | 09:01 |
| uvirtbot | Launchpad bug 1341936 in os-collect-config "Require keystone v3 auth_url in heat collector" [Undecided,New] | 09:01 |
| *** untriaged-bot has quit IRC | 09:01 | |
| *** regebro has quit IRC | 09:01 | |
| *** yamahata has quit IRC | 09:04 | |
| *** jp_at_hp has joined #tripleo | 09:08 | |
| *** pelix has joined #tripleo | 09:10 | |
| *** regebro has joined #tripleo | 09:17 | |
| *** andreaf has joined #tripleo | 09:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Revert "Set default OVERCLOUD_CONTROLSCALE to 3" https://review.openstack.org/106852 | 09:21 |
| *** IvanBerezovskiy has joined #tripleo | 09:23 | |
| *** andreaf has quit IRC | 09:27 | |
| *** andreaf has joined #tripleo | 09:28 | |
| *** Yanyan has quit IRC | 09:30 | |
| *** rushiagr_ has joined #tripleo | 09:33 | |
| *** rushiagr has quit IRC | 09:33 | |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Add support for flashing ILO BMC's from ramdisks https://review.openstack.org/107247 | 09:40 |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Correct statement about ramdisk and tmpfs. https://review.openstack.org/107280 | 09:40 |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Ceilometer graphs for nodes https://review.openstack.org/104188 | 09:42 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Improve readability of long JQ expression https://review.openstack.org/99681 | 09:45 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for os-collect-config to complete on the seed https://review.openstack.org/96515 | 09:45 |
| *** cody-somerville has joined #tripleo | 09:46 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Implement the Create Plan dialog https://review.openstack.org/107285 | 09:49 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-specs: Remove merge.py from TripleO Heat Templates https://review.openstack.org/97939 | 09:50 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-image-elements: Move masquerade host_ip to config.json https://review.openstack.org/101280 | 09:52 |
| *** rushiagr_ has quit IRC | 09:58 | |
| lifeless | hah | 10:03 |
| lifeless | nova really doesn't like having an instance deleted when the flavor was deleted first | 10:03 |
| lazy_prince | need +A for https://review.openstack.org/#/c/105966/. it already has 2 +2 and is a simple change.. | 10:06 |
| *** athomas has quit IRC | 10:08 | |
| *** martyntaylor has left #tripleo | 10:10 | |
| d0ugal | shadower: You around? | 10:18 |
| shadower | d0ugal: yep | 10:18 |
| *** dkehn_ has joined #tripleo | 10:18 | |
| d0ugal | shadower: I'm trying to get my head around changing to Heat storing the parameters | 10:19 |
| d0ugal | shadower: Tuskar API will never peak to Heat directly AFAIK | 10:19 |
| shadower | d0ugal: yea that's the plan as I understand it | 10:19 |
| d0ugal | shadower: However, jays code has lots of references to parameters - so, am I missing something or will that all need to change? | 10:19 |
| d0ugal | You may not know the answer, I can wait for jdob if its easier. | 10:20 |
| *** lazy_prince is now known as killer_prince | 10:20 | |
| shadower | d0ugal: so afaiui, tuskar will want to store some parameters for the planning phase -- things like the initial number of compute and controller nodes for example | 10:20 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/diskimage-builder: Don't try to install if packages is empty https://review.openstack.org/107008 | 10:20 |
| shadower | d0ugal: these will be stored in the environment file (which can do that already) | 10:21 |
| d0ugal | ah | 10:21 |
| *** dkehnx has quit IRC | 10:22 | |
| shadower | killer_prince: the undercloud tests failed so we can't merge it. Can't investigate the failure right now (sorry) but if it's not caused by the patch, you should do a recheck | 10:23 |
| *** rlandy has joined #tripleo | 10:25 | |
| *** athomas has joined #tripleo | 10:26 | |
| *** tserong_ has joined #tripleo | 10:29 | |
| *** tserong has quit IRC | 10:32 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow capabilites to be registered to ironic nodes https://review.openstack.org/107308 | 10:32 |
| *** killer_prince is now known as lazy_prince | 10:32 | |
| *** dsneddon has quit IRC | 10:32 | |
| *** andreaf has quit IRC | 10:32 | |
| *** cody-somerville has quit IRC | 10:33 | |
| *** andreaf has joined #tripleo | 10:34 | |
| *** tserong_ is now known as tserong | 10:36 | |
| openstackgerrit | Eamonn O'Toole proposed a change to openstack/tripleo-image-elements: Set venv ownership for services https://review.openstack.org/106380 | 10:38 |
| *** akuznetsov has quit IRC | 10:40 | |
| *** akrivoka has quit IRC | 10:41 | |
| *** xuhaiwei has quit IRC | 10:43 | |
| *** pmyers has joined #tripleo | 10:47 | |
| *** akrivoka has joined #tripleo | 10:48 | |
| TheJulia | good morning | 10:49 |
| lifeless | success | 10:52 |
| lifeless | Running /ilo/fw_current.scexe/flash_ilo3 | 10:52 |
| lifeless | morning TheJulia | 10:55 |
| *** akuznetsov has joined #tripleo | 10:56 | |
| lifeless | oath | 11:05 |
| lifeless | derekh_: we have a winner | 11:05 |
| lifeless | iLO 3 Standard 1.70 at Jan 09 2014 | 11:05 |
| lifeless | automaticallty updated | 11:05 |
| lifeless | now, of course, this is going to brick stuff left and right as we find and uncover race conditions :) | 11:05 |
| *** ci-overcloud has joined #tripleo | 11:07 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 11:07 |
| *** ci-overcloud has quit IRC | 11:07 | |
| lifeless | derekh_: now, in principle my next trick will bring up a ci-overcloud such that ... nuts | 11:07 |
| derekh_ | lifeless: sounds like we're getting close | 11:08 |
| *** bmahalakshmi has quit IRC | 11:08 | |
| lifeless | derekh_: yeah, I've integrated in tchaypos public VIP stuff | 11:08 |
| lifeless | derekh_: but we've found an SSL issue with haproxy -> stunnel -> boom | 11:09 |
| lifeless | derekh_: then I side tracked off when well, we kept running into locked up iLOs and so forth | 11:09 |
| lifeless | derekh_: since I was not going to do 45 machine updates by hand :) | 11:10 |
| derekh_ | lifeless: ya, manually updating 45 machines doesn't sound good | 11:12 |
| lifeless | my workaround for the race seems pretty stable here | 11:13 |
| lifeless | I know it wont scale to e.g. 500 machine clusters though | 11:13 |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Prevent multiple MySQL masters https://review.openstack.org/104414 | 11:13 |
| *** akuznetsov has quit IRC | 11:20 | |
| *** akuznetsov has joined #tripleo | 11:23 | |
| openstackgerrit | Lennart Regebro proposed a change to openstack/tuskar-ui: Added Management Dashboard https://review.openstack.org/107321 | 11:24 |
| *** robsparker has quit IRC | 11:27 | |
| *** akrivoka has quit IRC | 11:27 | |
| *** cody-somerville has joined #tripleo | 11:28 | |
| *** cody-somerville has joined #tripleo | 11:28 | |
| *** pblaho has joined #tripleo | 11:28 | |
| *** rameshg87 has left #tripleo | 11:35 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Don't try to install if packages is empty https://review.openstack.org/107171 | 11:36 |
| lifeless | Ng: can I get you to file some jira tickets? its way past bedtime ;) | 11:36 |
| *** akrivoka has joined #tripleo | 11:40 | |
| *** rushiagr_ has joined #tripleo | 11:43 | |
| *** robsparker has joined #tripleo | 11:43 | |
| lifeless | Ng: they need the hung ilo treatment - physically remove power, wait, re-insert | 11:43 |
| lifeless | Ng: (see hp1's nodes tabular list | 11:48 |
| *** rushiagr_ has quit IRC | 11:48 | |
| *** yamahata has joined #tripleo | 11:49 | |
| *** rushiagr_ has joined #tripleo | 11:52 | |
| *** dprince has joined #tripleo | 11:57 | |
| *** Yanyan has joined #tripleo | 11:59 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-specs: Targeted node deployment https://review.openstack.org/106996 | 12:01 |
| *** jdob has joined #tripleo | 12:09 | |
| *** tzumainn has joined #tripleo | 12:12 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow capabilites to be registered to ironic nodes https://review.openstack.org/107308 | 12:12 |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow capabilites to be registered to ironic nodes https://review.openstack.org/107308 | 12:13 |
| *** ci-overcloud has joined #tripleo | 12:16 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 12:16 |
| *** ci-overcloud has quit IRC | 12:16 | |
| *** rushiagr__ has joined #tripleo | 12:18 | |
| *** rushiagr_ has quit IRC | 12:18 | |
| *** lucasagomes is now known as lucas-lunch | 12:21 | |
| *** lblanchard has joined #tripleo | 12:21 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow capabilites to be registered to ironic nodes https://review.openstack.org/107308 | 12:23 |
| *** rushiagr has joined #tripleo | 12:24 | |
| *** rushiagr__ has quit IRC | 12:25 | |
| *** yamahata has quit IRC | 12:25 | |
| *** ramishra has joined #tripleo | 12:27 | |
| *** yamahata has joined #tripleo | 12:30 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Implement the Create Plan dialog https://review.openstack.org/107285 | 12:30 |
| *** sballe has joined #tripleo | 12:44 | |
| *** bashok has joined #tripleo | 12:46 | |
| *** nosnos has quit IRC | 12:49 | |
| *** julim has joined #tripleo | 12:49 | |
| *** weshay has joined #tripleo | 12:52 | |
| lifeless | derekh_: night, another node bad, | 12:58 |
| lifeless | derekh_: please don't try anothe rdeploy, I need to wait for nova to error, delete the stack then put a9d71ef3-3a7f-4932-a952-534c1128a1e6 into maintenance mode | 12:59 |
| lifeless | derekh_: (or if you want to try a deloy, do those things first :)) | 12:59 |
| *** rushiagr has quit IRC | 12:59 | |
| *** rushiagr has joined #tripleo | 12:59 | |
| derekh_ | lifeless: ok, night, I'll leave it got plenty to do anyways :-) | 13:04 |
| d0ugal | jdob: FWIW, I was just trying to figure out if you needed the hidden parameters since you refered to them - your reply answers that, so thanks! | 13:11 |
| *** lucas-lunch is now known as lucasagomes | 13:12 | |
| *** dtantsur is now known as dtantsur|lunch | 13:12 | |
| jdob | cool, np. | 13:12 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Protect from a unset TRIPLEO_OS_FAMILY. https://review.openstack.org/106343 | 13:13 |
| d0ugal | jdob: For the rationale about splitting it out, who would ask us to do that? | 13:14 |
| d0ugal | jdob: I'm not sure if you had something in mind, or if it was just a precaution? | 13:14 |
| *** rushiagr_ has joined #tripleo | 13:16 | |
| *** rushiagr has quit IRC | 13:20 | |
| jdob | kinda loosely had something in mind | 13:23 |
| jdob | for CI purposes | 13:23 |
| jdob | wanting the combination stuff without the full storage services | 13:24 |
| *** vipul has quit IRC | 13:28 | |
| *** NobodyCam has quit IRC | 13:29 | |
| *** gothicmindfood has quit IRC | 13:30 | |
| *** BadCub_ has quit IRC | 13:30 | |
| *** zaro has quit IRC | 13:30 | |
| *** dtantsur|lunch is now known as dtantsur | 13:31 | |
| *** vipul has joined #tripleo | 13:35 | |
| *** BadCub has joined #tripleo | 13:36 | |
| *** gothicmindfood has joined #tripleo | 13:36 | |
| *** zaro has joined #tripleo | 13:37 | |
| *** lazy_prince is now known as killer_prince | 13:40 | |
| *** ci-overcloud has joined #tripleo | 13:40 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 13:40 |
| *** ci-overcloud has quit IRC | 13:40 | |
| *** NobodyCam has joined #tripleo | 13:42 | |
| *** killer_prince is now known as lazy_prince | 13:43 | |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Adding keepalived restart check https://review.openstack.org/107147 | 13:45 |
| *** marun_afk is now known as marun | 13:48 | |
| *** jcoufal has quit IRC | 13:50 | |
| *** jcoufal has joined #tripleo | 13:51 | |
| *** yamahata has quit IRC | 13:54 | |
| *** yamahata has joined #tripleo | 13:54 | |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Ceilometer graphs for nodes https://review.openstack.org/104188 | 13:59 |
| *** nati_ueno has joined #tripleo | 14:00 | |
| *** jistr has quit IRC | 14:02 | |
| *** jistr has joined #tripleo | 14:03 | |
| *** bnemec has joined #tripleo | 14:03 | |
| *** imcsk8 has quit IRC | 14:07 | |
| *** imcsk8 has joined #tripleo | 14:07 | |
| *** Yanyan has quit IRC | 14:08 | |
| *** lazy_prince is now known as killer_prince | 14:09 | |
| openstackgerrit | Jonathan Brownell proposed a change to openstack/diskimage-builder: Increase source-repositories support for tarballs https://review.openstack.org/104694 | 14:11 |
| *** ramishra has quit IRC | 14:18 | |
| *** ramishra has joined #tripleo | 14:28 | |
| *** jistr has quit IRC | 14:32 | |
| *** jcoufal has quit IRC | 14:35 | |
| *** rdopieralski has quit IRC | 14:36 | |
| *** shausy2 has quit IRC | 14:39 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Revert "Move libvirt's qemu configuration dir to state fs" https://review.openstack.org/107209 | 14:39 |
| *** dkehn_ is now known as dkehnx | 14:39 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Move python-netaddr install to network-utils https://review.openstack.org/103898 | 14:39 |
| *** mrunge has quit IRC | 14:43 | |
| *** nati_ueno has quit IRC | 14:45 | |
| *** jang1 has joined #tripleo | 14:45 | |
| *** akrivoka has quit IRC | 14:47 | |
| ekarlso | https://review.openstack.org/#/c/107128/ < anyone know why that breaks ? | 14:49 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Enable haproxy_connect_any port access https://review.openstack.org/107417 | 14:57 |
| *** ramishra has quit IRC | 14:58 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Enable haproxy_connect_any port access https://review.openstack.org/107234 | 14:59 |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 15:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 15:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 15:00 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1341936 | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1341936 in os-collect-config "Require keystone v3 auth_url in heat collector" [Undecided,New] | 15:00 |
| Ng | lifeless: k | 15:02 |
| Ng | lifeless: derekh_: I've just flipped over tripleo.org to new DNS servers. I believe everything should continue to work as-is, but please let me know if you see something resolving wrong. I removed some records that I think are unused (git/demo/poc/jenkins), please let me know if that's not the case | 15:03 |
| Ng | lifeless: derekh_: assuming this beds down nicely, we will have API on our DNS, so you can a) do fancy stuff, b) not block on me for DNS changes :) | 15:03 |
| *** jmoffitt has joined #tripleo | 15:04 | |
| derekh_ | Ng: nice, willdo | 15:04 |
| lxsli | Please could I get some eyes on: https://review.openstack.org/#/c/86316 ? | 15:05 |
| *** akuznetsov has quit IRC | 15:07 | |
| *** athomas has quit IRC | 15:08 | |
| *** akuznetsov has joined #tripleo | 15:08 | |
| *** ifarkas has quit IRC | 15:15 | |
| *** cinerama has joined #tripleo | 15:15 | |
| *** amitpp has joined #tripleo | 15:16 | |
| *** dshurtle has joined #tripleo | 15:20 | |
| *** jistr has joined #tripleo | 15:21 | |
| *** dmellado has joined #tripleo | 15:22 | |
| *** bashok has quit IRC | 15:23 | |
| *** nati_ueno has joined #tripleo | 15:28 | |
| *** pcrews has joined #tripleo | 15:32 | |
| *** nati_ueno has quit IRC | 15:35 | |
| *** ramishra has joined #tripleo | 15:36 | |
| lxsli | Do we have a Juno spec freeze date for TripleO? | 15:36 |
| *** cinerama has quit IRC | 15:36 | |
| *** nati_ueno has joined #tripleo | 15:38 | |
| *** pcrews has quit IRC | 15:38 | |
| *** rakesh_hs has quit IRC | 15:42 | |
| *** pcrews has joined #tripleo | 15:42 | |
| *** cinerama has joined #tripleo | 15:42 | |
| cinerama | morning guys | 15:42 |
| *** nati_ueno has quit IRC | 15:43 | |
| *** nati_ueno has joined #tripleo | 15:43 | |
| *** dtantsur is now known as dtantsur|afk | 15:44 | |
| lxsli | greghaynes slagle: please would you check https://review.openstack.org/#/c/86316 ? | 15:45 |
| *** yamahata has quit IRC | 15:47 | |
| *** cody-somerville has quit IRC | 15:52 | |
| *** ramishra has quit IRC | 15:53 | |
| slagle | lxsli: hi, per your response to my earlier comment, the only way to use that element is to host a private apt/yum repository, then configure the image build to use that repo? | 15:57 |
| lxsli | slagle: Yes | 15:57 |
| slagle | or, i guess drop the custom apt/yum config into the image and pull from elasitcsearch.org | 15:58 |
| lxsli | yeah you could do that too | 15:58 |
| *** cody-somerville has joined #tripleo | 15:59 | |
| lxsli | It's far from ideal but I don't want to complect repo management with the element | 15:59 |
| *** yamahata has joined #tripleo | 16:08 | |
| *** nati_ueno has quit IRC | 16:09 | |
| *** chuckC has quit IRC | 16:10 | |
| dprince | slagle: So re https://review.openstack.org/#/c/106231/, would it make it more sense to hard code it with regards to the /mnt/state cleanup effort? | 16:13 |
| dprince | slagle: configurable /mnt/state that is | 16:14 |
| dprince | slagle: I made it configurable, and that is largely why gfidente abandoned his patch and +1'd mine | 16:15 |
| * dprince wonders if we should have just kept gfidente's patch now | 16:16 | |
| *** cody-somerville has quit IRC | 16:17 | |
| *** andreaf has quit IRC | 16:17 | |
| slagle | dprince: even if/when /mnt/state is configurable it just changes the default value there in that oac call to $STATEFUL_PATH/var/lib/cinder/conversion | 16:17 |
| slagle | dprince: so that doesn't really matter | 16:17 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Create/configure cinder's image_conversion_dir https://review.openstack.org/106231 | 16:17 |
| dprince | slagle: okay, well ^^ I rebased it. Thanks for the feedback | 16:18 |
| slagle | dprince: sure, np. i was mostly just asking to being with if there was something special about this path where it *needed* to be configurable | 16:18 |
| dprince | slagle: well, it is a temp conversions dir. It certainly doesn't need to be in the state dir (even though that is the default) | 16:19 |
| *** dmellado has quit IRC | 16:19 | |
| slagle | right, maybe some people would want to configure it to be a tmpfs path or something | 16:20 |
| dprince | slagle: all boils down to I don't like the hard coding if things | 16:20 |
| *** markmc has quit IRC | 16:20 | |
| * dprince thinks TripleO has gotten a bit too comfortable w/ hard coding | 16:20 | |
| dprince | mind you people can call hard coding other things like production defaults, or whatever... It is still hard coding | 16:21 |
| *** yamahata has quit IRC | 16:21 | |
| derekh_ | hardcoding is like having lots of different config files :-) | 16:21 |
| dprince | derekh_: that too | 16:22 |
| *** regebro has quit IRC | 16:22 | |
| *** derekh_ has quit IRC | 16:27 | |
| *** dshulyak_ has joined #tripleo | 16:29 | |
| *** rakesh_hs has joined #tripleo | 16:29 | |
| *** jistr has quit IRC | 16:30 | |
| *** dshurtle has quit IRC | 16:30 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-incubator: devtest_overcloud: boot an instance from a volume https://review.openstack.org/107437 | 16:35 |
| *** dkehn_ has joined #tripleo | 16:45 | |
| *** martyntaylor has joined #tripleo | 16:46 | |
| *** rdopieralski has joined #tripleo | 16:48 | |
| *** dkehnx has quit IRC | 16:49 | |
| *** dkehn_ is now known as dkehnx | 16:51 | |
| *** cody-somerville has joined #tripleo | 17:00 | |
| *** andreaf has joined #tripleo | 17:01 | |
| *** cinerama has quit IRC | 17:05 | |
| *** andreaf has quit IRC | 17:10 | |
| *** nati_ueno has joined #tripleo | 17:11 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Update node index view to match Juno wireframes https://review.openstack.org/106889 | 17:13 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-image-elements: openvswitch and openvswitch-datapath element https://review.openstack.org/95151 | 17:14 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-image-elements: Make use of openvswitch elements https://review.openstack.org/104115 | 17:14 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Adds driver and network card details to node detail page https://review.openstack.org/107102 | 17:15 |
| *** openstackgerrit has quit IRC | 17:16 | |
| *** openstackgerrit has joined #tripleo | 17:17 | |
| *** andreaf has joined #tripleo | 17:25 | |
| *** lucasagomes is now known as lucas-dinner | 17:32 | |
| *** cinerama has joined #tripleo | 17:37 | |
| *** Lexis has quit IRC | 17:38 | |
| *** marun is now known as marun_afk | 17:40 | |
| *** rushiagr_ has quit IRC | 17:42 | |
| *** ifarkas has joined #tripleo | 17:42 | |
| *** saurabhs has joined #tripleo | 17:45 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Custom policy for nova-api tmpfs errors https://review.openstack.org/107447 | 17:45 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Allow keepalived to load kernel modules https://review.openstack.org/107448 | 17:47 |
| *** amitpp has quit IRC | 17:47 | |
| *** weshay has quit IRC | 17:49 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-image-elements: Move masquerade host_ip to config.json https://review.openstack.org/101280 | 17:49 |
| *** weshay has joined #tripleo | 17:55 | |
| *** Penick has joined #tripleo | 17:55 | |
| *** rdopieralski has quit IRC | 17:56 | |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/plans/<plan-uuid> endpoint https://review.openstack.org/106779 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds DELETE /v2/plans/<plan-uuid> endpoint https://review.openstack.org/106780 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds POST /v2/plans endpoint to API https://review.openstack.org/106781 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/plans endpoint to API https://review.openstack.org/106388 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/roles endpoint to API https://review.openstack.org/107450 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds POST /v2/plans/:plan_uuid/roles endpoint https://review.openstack.org/107451 | 17:57 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/plans/:plan_uuid/templates endpoint https://review.openstack.org/107452 | 17:57 |
| *** Penick has quit IRC | 17:59 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: WIP: Custom policy for ssh-keygen failure https://review.openstack.org/107233 | 18:00 |
| *** Penick has joined #tripleo | 18:02 | |
| *** dshurtle has joined #tripleo | 18:04 | |
| *** pblaho has quit IRC | 18:12 | |
| *** killer_prince is now known as lazy_prince | 18:12 | |
| *** martyntaylor1 has joined #tripleo | 18:14 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Move libvirt's qemu configuration dir to state fs https://review.openstack.org/107461 | 18:15 |
| *** martyntaylor has quit IRC | 18:15 | |
| *** martyntaylor1 has quit IRC | 18:21 | |
| *** martyntaylor has joined #tripleo | 18:21 | |
| *** dprince has quit IRC | 18:34 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-specs: SSL PKI https://review.openstack.org/103692 | 18:34 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-specs: SSL PKI https://review.openstack.org/103692 | 18:35 |
| *** andreaf has quit IRC | 18:38 | |
| *** pelix has quit IRC | 18:43 | |
| *** pblaho has joined #tripleo | 18:50 | |
| *** pblaho has quit IRC | 18:50 | |
| *** jang has quit IRC | 18:53 | |
| *** lazy_prince is now known as killer_prince | 18:56 | |
| *** jp_at_hp has quit IRC | 19:00 | |
| *** dsneddon has joined #tripleo | 19:02 | |
| *** Penick has quit IRC | 19:02 | |
| *** rlandy has quit IRC | 19:03 | |
| *** jang has joined #tripleo | 19:04 | |
| *** panda has quit IRC | 19:06 | |
| openstackgerrit | Jordan OMara proposed a change to openstack/tuskar-ui: Updating Baremetal Node UUIDs with realistic values for display https://review.openstack.org/107474 | 19:06 |
| *** panda has joined #tripleo | 19:07 | |
| lifeless | morning | 19:09 |
| *** martyntaylor1 has joined #tripleo | 19:09 | |
| greghaynes | O/ | 19:09 |
| slagle | you only slept 8 hours | 19:10 |
| lifeless | not even | 19:10 |
| greghaynes | creepy slagle is creepy | 19:10 |
| slagle | assuming you slept the whole time you've been gone :) | 19:10 |
| greghaynes | ;) | 19:10 |
| lifeless | slagle: so I reviewed the docker spec again | 19:10 |
| lifeless | slagle: there was just one thing left that I can see to adress | 19:11 |
| slagle | lifeless: i pushed up a patchset to address that | 19:11 |
| *** martyntaylor has quit IRC | 19:11 | |
| lifeless | wicked | 19:11 |
| *** Penick has joined #tripleo | 19:12 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix argparse issue for RHEL 6.5. https://review.openstack.org/105966 | 19:20 |
| *** bashok has joined #tripleo | 19:32 | |
| *** chuckC has joined #tripleo | 19:38 | |
| *** ci-overcloud has joined #tripleo | 19:40 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 19:40 |
| *** ci-overcloud has quit IRC | 19:40 | |
| *** chuckC has quit IRC | 19:44 | |
| *** martyntaylor1 has left #tripleo | 19:50 | |
| openstackgerrit | Jay Dobies proposed a change to openstack/tuskar: Added template combination functionality https://review.openstack.org/106853 | 20:00 |
| *** Penick has quit IRC | 20:05 | |
| *** Penick has joined #tripleo | 20:07 | |
| *** rakesh_hs has quit IRC | 20:13 | |
| *** ifarkas has quit IRC | 20:15 | |
| *** akuznetsov has quit IRC | 20:20 | |
| *** weshay has quit IRC | 20:21 | |
| openstackgerrit | Jay Dobies proposed a change to openstack/tuskar: Added template combination functionality https://review.openstack.org/106853 | 20:21 |
| *** yamahata has joined #tripleo | 20:23 | |
| *** jml has quit IRC | 20:25 | |
| *** marun_afk is now known as marun | 20:27 | |
| *** jml has joined #tripleo | 20:28 | |
| *** jml has quit IRC | 20:28 | |
| *** jml has joined #tripleo | 20:28 | |
| jdob | dumb question, why is the image element called "use-ephemeral" when it's about persistent state? | 20:33 |
| *** weshay has joined #tripleo | 20:34 | |
| *** dshulyak_ has quit IRC | 20:34 | |
| *** chuckC has joined #tripleo | 20:35 | |
| tchaypo | morgens | 20:38 |
| tchaypo | because it's about storing state on ephemeral images, or not? | 20:39 |
| * tchaypo is guessing, doesn't actually understand | 20:39 | |
| tchaypo | greghaynes: my scrollback is disappointingly free of you solving new bugs for me *fakes up some tears* | 20:41 |
| greghaynes | haha, sorry! | 20:41 |
| greghaynes | I got to learn about job builder | 20:41 |
| tchaypo | that sounds like fun | 20:41 |
| tchaypo | i see lifeless was busy blowing things up as well | 20:42 |
| tchaypo | i seem to have missed dprince | 20:43 |
| *** Penick has quit IRC | 20:44 | |
| greghaynes | tchaypo: what was your patch for the subnet being defined incorrectly error? | 20:44 |
| adam_g | jdob, its not a dumb question. the language is confusing. typical virt nova instances, when booted /w flavor that provide it, end up booted with an ephemeral drive usually mounted @ /mnt. in the ironic/baremetal case, this is actually the persistent partition while the root fs is ephemeral | 20:44 |
| greghaynes | yea, its a pretty annoying naming | 20:45 |
| jdob | ok, that's kinda what I was thinking, that it referred to the drive itself and not the functionality | 20:45 |
| adam_g | jdob, so on instance rebuild you end up with a new image being deployed to the / while nova's 'ephemeral partition' actually keeps its data | 20:45 |
| greghaynes | use-thing-that-lets-me-have-ephemeral | 20:45 |
| *** Penick has joined #tripleo | 20:45 | |
| greghaynes | we should just rename the element to that ^ | 20:45 |
| jdob | threw me off when I saw a patch that added usage of /mnt/state that needed to include use-ephemeral | 20:45 |
| tchaypo | greghaynes: that would be https://review.openstack.org/#/c/103757/16..17/scripts/setup-neutron | 20:46 |
| jdob | cool, thanks for the explanation. glad i'm not the only one a bit thrown off by it | 20:46 |
| *** cadenzajon has joined #tripleo | 20:50 | |
| *** bashok has quit IRC | 20:51 | |
| *** andreaf has joined #tripleo | 20:54 | |
| tchaypo | Does anyone know what dprince means when he mentions "we need 2 bridges and our templates onlysupport 1"? | 20:55 |
| tchaypo | that sounds to me a lot like the work lifeless has been leading to get our templates to support 2 bridges, but maybe I don't understand the context | 20:55 |
| *** chuckC has quit IRC | 20:56 | |
| *** ccrouch has quit IRC | 20:57 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 21:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 21:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 21:00 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1341936 | 21:00 |
| uvirtbot | Launchpad bug 1341936 in os-collect-config "Require keystone v3 auth_url in heat collector" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| lifeless | tchaypo: where does dprince say that? | 21:01 |
| tchaypo | http://lists.openstack.org/pipermail/openstack-dev/2014-July/040464.html | 21:02 |
| tchaypo | I'm not sure if he's talkinga botu things we've already done, or if he's talking about a thing we're missing (or doing by hand) | 21:04 |
| tchaypo | s/a obotu/ about/ | 21:04 |
| tchaypo | in other news, my latest attempt to build the stack got me as far as wait_for_stack_ready overcloud | 21:05 |
| tchaypo | so that's progress | 21:05 |
| *** lblanchard has quit IRC | 21:07 | |
| *** jdob has quit IRC | 21:07 | |
| tchaypo | ah. it seems that controller0 and controller2 are ACTIVE - Running | 21:07 |
| tchaypo | but controller1 is ERROR - NOSTATE | 21:07 |
| tchaypo | to me this is a successful build of the controller cluster | 21:08 |
| tchaypo | it's running, it has redundancy, I'd like to move forward and let the usual repair mechanisms handle spanking the third node into shape later | 21:08 |
| greghaynes | heh | 21:08 |
| tchaypo | but wait_for_stack_ready sees ERROR and decides to abort | 21:08 |
| greghaynes | tchaypo: Have you seen SpamapS's convergence spec's | 21:08 |
| *** ccrouch has joined #tripleo | 21:09 | |
| *** imcsk8 has quit IRC | 21:10 | |
| tchaypo | so. broader question is how do we update wait_for_stack_ready to be more clueful about scale? maybe a new param to tell it how many lines matching the node name regex need to be failed before the whole thing is a failure? | 21:10 |
| tchaypo | no I haven't, what's the number? | 21:10 |
| lifeless | he's talking about the previous state | 21:12 |
| lifeless | tchaypo: look in the heat-specs repo | 21:12 |
| lifeless | tchaypo: but anyway, we can't fix wait_for_stack_ready | 21:12 |
| lifeless | tchaypo: because it heat that decides the things has failed, not our wrapper. | 21:13 |
| tchaypo | lifeless: so if I respond saying "we've already got most of that particular issue sorted, please provide a different example" I'm not off track? | 21:13 |
| tchaypo | I do thing os-net-config sounds useful, I just think that's a particularly poor example to show why it might be useful | 21:13 |
| tchaypo | lifeless: i think you mean that heat decides whether stack creation failed, and wait_for_stack_ready has always used that to determine if it's safe to proceed? | 21:14 |
| tchaypo | now that we have a HA cluster I think we're going to see cases where stack create failed, but it failed close enough to the end that it's safe for us to keep going | 21:15 |
| lifeless | more than 'has always used' | 21:15 |
| lifeless | 'has to use today' | 21:15 |
| *** ccrouch1 has joined #tripleo | 21:16 | |
| lifeless | we can't further evolve the stack once its in ERROR | 21:16 |
| tchaypo | what would we need to get away from that sate? | 21:16 |
| tchaypo | *state | 21:16 |
| lifeless | we need heast to be different | 21:16 |
| tchaypo | ah. | 21:16 |
| lifeless | this is why I've been saying 'don't worry about quorum in XXXX' where it was something that a failure would also mean the stack fails | 21:17 |
| lifeless | we need quorum stuff where the failure can occur and not make the stack ERROR | 21:17 |
| lifeless | which is still the majority of things :/ | 21:17 |
| lifeless | tchaypo: I have replied to dan | 21:17 |
| tchaypo | my next question would be "do we need to evolve the stack? at that point we're basically at the end of the build anywaay, could we just call it 'complete with some errors'" and hand the cloud over to the user to drive via tuskar etc - but when it's handed over they'll be relying on heat to do anything further to the cloud, and they can't because it's in an error state, so that won't fly. | 21:18 |
| *** ccrouch has quit IRC | 21:18 | |
| lifeless | if you want to fix that broken ctlplane node | 21:18 |
| lifeless | thats evolving the stack | 21:18 |
| tchaypo | yeah. | 21:18 |
| lifeless | if you want to change config parameters, same | 21:19 |
| tchaypo | hence what you said a few lines back | 21:19 |
| tchaypo | everythign that heat runs has to be cluster-aware and decide if a failure is a stack-breaking failure-to-achieve-quorum or just a local failure that shouldn't affect stack state | 21:20 |
| lifeless | yes | 21:20 |
| *** shardy is now known as shardy_afk | 21:20 | |
| * tchaypo has subscribed to heat-specs | 21:20 | |
| *** jml has quit IRC | 21:21 | |
| *** jml has joined #tripleo | 21:24 | |
| *** Penick has quit IRC | 21:27 | |
| *** Penick has joined #tripleo | 21:32 | |
| *** jmoffitt has left #tripleo | 21:35 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Add Discovered nodes tab https://review.openstack.org/107509 | 21:40 |
| *** julim has quit IRC | 21:46 | |
| lifeless | ok another mind bending race fixed | 21:49 |
| lifeless | Ng: the fail-to-boots turn out to be a bug in nova | 21:59 |
| greghaynes | \O/ hash mismatch again | 22:00 |
| greghaynes | for apt mirror | 22:00 |
| * greghaynes debugs properly again | 22:00 | |
| lifeless | Ng: https://bugs.launchpad.net/nova/+bug/1342919 | 22:01 |
| uvirtbot | Launchpad bug 1342919 in nova "instances rescheduled after building network info do not update the MAC" [Undecided,In progress] | 22:01 |
| lifeless | Ng: so don't jira things, cause I think its all going to be that | 22:01 |
| lifeless | Ng: except the genuine ILO hung ones | 22:01 |
| greghaynes | lifeless: Hey, I found a bug of yours! https://bugs.launchpad.net/ubuntu/+source/apt/+bug/972077 | 22:03 |
| uvirtbot | Launchpad bug 972077 in apt "apt repository disk format has race conditions" [Medium,Confirmed] | 22:04 |
| lifeless | greghaynes: yup | 22:05 |
| lifeless | greghaynes: thats the root cause :) | 22:05 |
| lifeless | greghaynes: however a mirroring tool has the opportunity to minimize the race by the order it copies in and by checking the hashes and not moving stuff into place when they don't match | 22:06 |
| lifeless | greghaynes: for instance | 22:06 |
| *** cody-somerville has quit IRC | 22:15 | |
| greghaynes | So, I think my upstream mirror may be more at fault than my mirror | 22:27 |
| greghaynes | I agree, mine could be smarter | 22:27 |
| greghaynes | but... /me shakes fist at upstream | 22:27 |
| SpamapS | lifeless: also apache can be configured to syncronize the expire times to the last file that is usually synced | 22:28 |
| greghaynes | http://i.imgur.com/5I8QQPe.jpg | 22:28 |
| SpamapS | lifeless: which is what archive.ubuntu.com did | 22:28 |
| pcrews | Hi - so for our current ci over/undercloud node names - overcloud-controller0-57qrrf3w7fwa, for example. How important is the random string portion of those node names? | 22:29 |
| * pcrews asks as the indexing code does not appear to play well w/ wildcard file names at first look. | 22:29 | |
| pcrews | indexing code = log indexing code - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/logstash/jenkins-log-client.yaml | 22:30 |
| lifeless | SpamapS: it doesn't fix the race | 22:32 |
| lifeless | SpamapS: I was in on those discussions with elmo etc, repeatedly, as we tried to fix countries (like NZ) with intercepting caches. | 22:32 |
| SpamapS | lifeless: I imagine expires isn't 100% respected | 22:33 |
| lifeless | SpamapS: its not that, its still racy | 22:33 |
| lifeless | SpamapS: as I explain in great detail in that bug. | 22:34 |
| SpamapS | Really unfortunate, rbasak had consensus on a plan to fix it | 22:34 |
| lifeless | SpamapS: there is literally no HTTP header that can fix it. | 22:34 |
| lifeless | SpamapS: and it can turn up on file:// too. | 22:34 |
| SpamapS | lifeless: it's like the ET Atari game. It just has to be buried in the desert. | 22:34 |
| *** Penick has quit IRC | 22:34 | |
| SpamapS | lifeless: I wonder if it could be done with A/B dirs storing rsync --link-dest copies of the meta-data | 22:35 |
| pcrews | http://www.virtualatari.org/soft.php?soft=ET_Fixed | 22:35 |
| lifeless | SpamapS: nope | 22:40 |
| lifeless | SpamapS: its not about atomic update on the server | 22:40 |
| lifeless | SpamapS: its about read atomicity | 22:40 |
| SpamapS | pcrews: jesus it is as nightmarish as I remember | 22:40 |
| *** dobson has joined #tripleo | 22:40 | |
| *** ivan has left #tripleo | 22:40 | |
| lifeless | SpamapS: you have to read two different files together, either A1 and A2 or B1 and B2 | 22:40 |
| SpamapS | lifeless: yeah, that's the thing rbasak was going to correct | 22:40 |
| lifeless | SpamapS: I know, I designed it :) | 22:40 |
| pcrews | SpamapS: lol, indeed :D | 22:41 |
| SpamapS | lifeless: and yet he's doing MariaDB packaging instead. :-/ | 22:41 |
| lifeless | it may have been independently designed a few times by now | 22:41 |
| SpamapS | FYI I shut down the nagios I had running on the hp1 bastion. I don't think it was in use, but it was spamming me | 22:42 |
| lifeless | SpamapS: ack | 22:42 |
| SpamapS | lifeless: IIRC Debian doesn't care because they just use pdiff | 22:42 |
| SpamapS | which "sort of" solves it | 22:42 |
| lifeless | SpamapS: huh, it affects debian too | 22:42 |
| SpamapS | lifeless: I had thought that it affects far fewer debian users because pdiffs mean most of the time you're not refreshing Packages.gz via HTTP, but via pdiff, which are all accurate in the hashsums file | 22:45 |
| SpamapS | so then you d/l the pdiffs, and your new packages.gz == hashsum of the downloaded sums list | 22:46 |
| lifeless | SpamapS: packages is not the only race pair | 22:46 |
| lifeless | everything with detached sigs is racy | 22:46 |
| lifeless | such as Releases | 22:46 |
| lifeless | hmm, I might need to open an archive at this point to refresh memory | 22:46 |
| SpamapS | Sounds right to me. | 22:47 |
| SpamapS | It's more that Debian's pile of sand over their head is a bit larger because they have mitigations. | 22:47 |
| SpamapS | anyway.. | 22:47 |
| * SpamapS goes back to revising heat specs | 22:47 | |
| Ng | lifeless: ok, that's good because I was feeling bad that I'd bumped the jira stuff to the end of my day. Now, less so ;) | 22:57 |
| lifeless | Ng: well still 2 to file | 22:57 |
| lifeless | Ng: :) | 22:57 |
| Ng | lifeless: I said less so, not zero so ;) | 22:57 |
| lifeless | :> | 22:57 |
| *** yamahata has quit IRC | 23:03 | |
| *** cody-somerville has joined #tripleo | 23:07 | |
| lifeless | someone needs to get devananda a rack to just run lots and lots and lots of deploys on | 23:07 |
| SpamapS | lifeless: maybe we can all chip in and give him an orange box | 23:10 |
| *** Penick has joined #tripleo | 23:10 | |
| *** hewbrocca has quit IRC | 23:10 | |
| *** lucas-dinner has quit IRC | 23:20 | |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: Convert Fn::Select to extended get_attr https://review.openstack.org/105347 | 23:26 |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: merge.py use a class to represent template syntax https://review.openstack.org/105326 | 23:26 |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: merge.py --hot to process and generate HOT https://review.openstack.org/105327 | 23:26 |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: WIP Make target to port to HOT https://review.openstack.org/105329 | 23:26 |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: Port all templates to HOT https://review.openstack.org/105328 | 23:26 |
| *** ccrouch1 has quit IRC | 23:29 | |
| lifeless | SpamapS: how many nodes in that ? | 23:31 |
| *** Penick has quit IRC | 23:33 | |
| * lifeless tries batches of 5 nodes | 23:48 | |
| lifeless | I want to know that all these nodes are bootable | 23:48 |
| lifeless | also that they all have modern ilo f/w | 23:48 |
| *** cinerama has quit IRC | 23:51 | |
| stevebaker | hey, has check-tripleo replaced experimental? | 23:57 |
| lifeless | no | 23:59 |
| lifeless | there is a tripleo check queue | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
