| greghaynes | SpamapS: yea, mirrors suck | 00:06 |
|---|---|---|
| greghaynes | SpamapS: I am afraid to merge the debian test | 00:06 |
| greghaynes | SpamapS: I wonder if we could convince infra to host a mirror... | 00:07 |
| *** chuckC_ has quit IRC | 00:17 | |
| *** cody-somerville has quit IRC | 00:23 | |
| *** jeh has joined #tripleo | 00:36 | |
| *** sdake_ has quit IRC | 00:38 | |
| *** sdake has joined #tripleo | 00:38 | |
| *** yamahata has joined #tripleo | 00:39 | |
| *** penick has quit IRC | 00:45 | |
| *** sdake_ has joined #tripleo | 00:52 | |
| *** sdake has quit IRC | 00:57 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Update default Fedora to 22 https://review.openstack.org/185843 | 01:03 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Work around yum/dnf differences https://review.openstack.org/185842 | 01:03 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add YUM variable to for Fedora >= 22 https://review.openstack.org/185841 | 01:03 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Cleanup yum downloading https://review.openstack.org/186620 | 01:03 |
| *** cody-somerville has joined #tripleo | 01:17 | |
| *** sdake_ has quit IRC | 01:17 | |
| *** sdake has joined #tripleo | 01:18 | |
| *** GheRivero has joined #tripleo | 01:22 | |
| *** GheRivero has left #tripleo | 01:23 | |
| *** yamahata has quit IRC | 01:36 | |
| *** Slower has quit IRC | 01:40 | |
| *** alop has quit IRC | 01:43 | |
| *** Slower has joined #tripleo | 01:47 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add virtual IPs for split out networks https://review.openstack.org/186238 | 01:47 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Combine Heat API networks into single net https://review.openstack.org/186625 | 01:47 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Make all-nodes Ip networks configurable https://review.openstack.org/185581 | 01:57 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add virtual IPs for split out networks https://review.openstack.org/186238 | 01:57 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Combine Heat API networks into single net https://review.openstack.org/186625 | 01:57 |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Merge deployments data for collectors heat, request https://review.openstack.org/183356 | 02:01 |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Refactor merging out of cfn collector https://review.openstack.org/183355 | 02:01 |
| *** rwsu has quit IRC | 02:25 | |
| *** sdake_ has joined #tripleo | 02:40 | |
| *** sdake has quit IRC | 02:43 | |
| *** sdake has joined #tripleo | 02:43 | |
| *** sdake_ has quit IRC | 02:47 | |
| *** barra204 has quit IRC | 03:11 | |
| *** barra204 has joined #tripleo | 03:13 | |
| *** thrash is now known as thrash|g0ne | 03:57 | |
| *** tzumainn has quit IRC | 04:29 | |
| *** sdake_ has joined #tripleo | 04:29 | |
| *** sdake has quit IRC | 04:33 | |
| *** lazy_prince has joined #tripleo | 04:39 | |
| *** yog__ has joined #tripleo | 05:14 | |
| *** jeh has quit IRC | 05:17 | |
| *** masco has joined #tripleo | 05:21 | |
| *** ishant has joined #tripleo | 05:21 | |
| *** cody-somerville has quit IRC | 05:23 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Update default Fedora to 22 https://review.openstack.org/185843 | 05:29 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Work around yum/dnf differences https://review.openstack.org/185842 | 05:29 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Cleanup yum downloading https://review.openstack.org/186620 | 05:29 |
| *** AzherKhan has joined #tripleo | 05:34 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Skip backups and other non-relevant files for dib-lint https://review.openstack.org/186647 | 05:41 |
| *** sdake has joined #tripleo | 05:46 | |
| *** sdake_ has quit IRC | 05:50 | |
| dasm | good morning p/ | 06:03 |
| *** cody-somerville has joined #tripleo | 06:08 | |
| *** radek__ has joined #tripleo | 06:09 | |
| *** jtomasek has joined #tripleo | 06:13 | |
| openstackgerrit | Yuriy Zveryanskyy proposed openstack/diskimage-builder: Add Ironic API version to passthru URL for deploy-ironic https://review.openstack.org/186655 | 06:20 |
| *** jprovazn has joined #tripleo | 06:25 | |
| *** bvandenh has quit IRC | 06:32 | |
| *** aufi has joined #tripleo | 06:36 | |
| *** gfidente has joined #tripleo | 06:40 | |
| *** dtantsur|afk is now known as dtantsur | 06:55 | |
| *** ifarkas has joined #tripleo | 06:59 | |
| *** al has quit IRC | 07:00 | |
| *** al has joined #tripleo | 07:01 | |
| *** tzumainn has joined #tripleo | 07:10 | |
| *** yog__ has quit IRC | 07:12 | |
| *** tzumainn has quit IRC | 07:29 | |
| *** paramite has joined #tripleo | 07:31 | |
| *** dtantsur is now known as dtantsur|brb | 07:35 | |
| *** Goneri has joined #tripleo | 07:35 | |
| *** jistr has joined #tripleo | 07:38 | |
| *** yog__ has joined #tripleo | 07:41 | |
| *** pino|work_ has joined #tripleo | 07:47 | |
| *** pino|work_ is now known as pino|work | 07:47 | |
| *** shardy has joined #tripleo | 07:50 | |
| *** sdake_ has joined #tripleo | 07:55 | |
| *** sdake has quit IRC | 07:59 | |
| gfidente | jistr++ | 08:02 |
| gfidente | ^_^ | 08:02 |
| jistr | so one interesting thing, on the *compute* the puppet hook does run but it seems like puppet doesn't even finish compiling the catalog for some reason | 08:06 |
| gfidente | yeah apparently timing out | 08:06 |
| gfidente | but yet it is the very same behaviour we get with ps#4 | 08:06 |
| gfidente | marios, trying to investigate the nova failures | 08:07 |
| gfidente | marios, did you try adding the comma on your neutron change? | 08:08 |
| jistr | another interesting thing is, that puppet actually did run there apparently | 08:09 |
| jistr | if i look at nova.conf, e.g. rabbit_hosts is populated | 08:09 |
| jistr | so WUT, how come the puppet run is not logged in os-collect-config logs | 08:10 |
| jistr | huh maybe the puppet run got stuck, and maybe the puppet hook only prints its output after the puppet run finishes | 08:11 |
| jistr | hmm do we have any "wait for"-like bits in compute manifests | 08:12 |
| * jistr looks | 08:12 | |
| gfidente | in compute logs | 08:12 |
| gfidente | if you extract the tarball | 08:12 |
| gfidente | it looks like nova-compute is started but never leaves the | 08:13 |
| gfidente | Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor? Reattempting establishment of nova-conductor connection... | 08:13 |
| gfidente | stage | 08:13 |
| gfidente | jistr, did your change to make compute depend on controller ever land? | 08:13 |
| jistr | gfidente: i don't think so, as we've managed to fix our problem without doing that | 08:14 |
| gfidente | yeah and I think it still applies | 08:14 |
| gfidente | I wanted to try it out though to see if that helps anyhow | 08:14 |
| jistr | i can revive it | 08:14 |
| gfidente | yeah set a depend on this one so we can see if that passes CI/ | 08:14 |
| gfidente | ? | 08:14 |
| jistr | yeah | 08:15 |
| gfidente | tks | 08:15 |
| gfidente | other than that, controller 2 remains a mistery | 08:15 |
| jistr | do you remember what was the cause the last time when nova compute couldn't recover? | 08:17 |
| *** sdake_ has quit IRC | 08:17 | |
| gfidente | rabbitmq isolated nodes behind haproxy | 08:18 |
| jistr | ah yes | 08:18 |
| gfidente | not sure why compute wouldn't attach to the isolated conductor running on its same rabbitmq | 08:19 |
| gfidente | but the config changed quite a bit now | 08:19 |
| jistr | yeah that should be ok now | 08:19 |
| gfidente | or at least, diferent | 08:19 |
| jistr | i'm still puzzled by the missing logs from controller3 though | 08:19 |
| gfidente | yeah | 08:20 |
| jistr | that shouldn't have anything to do with nova-compute starting too early | 08:20 |
| gfidente | it doesn't seem to be same issue | 08:20 |
| gfidente | so to be honest, patchset #4 had same issue on compute, but not on controller 2 | 08:21 |
| gfidente | while previous run of patchset #5 does have same issue on controller 2 | 08:21 |
| gfidente | and also same issue with compute node | 08:22 |
| gfidente | are there other HA jobs which succeeded after memcache/keystone merging? | 08:23 |
| *** derekh has joined #tripleo | 08:24 | |
| marios | gfidente: no, just poking, good news is i hit it also on my beaker box so trying sthing out in a sec | 08:25 |
| *** lucasagomes has joined #tripleo | 08:25 | |
| jistr | the last green one is 186416,2 tripleo-heat-templates | 08:25 |
| *** regebro has joined #tripleo | 08:27 | |
| jistr | and this one is green too, has no deps, started 9:33 PM https://review.openstack.org/#/c/186415/2 | 08:27 |
| gfidente | but I also see same error happening | 08:28 |
| gfidente | in unrelated changes | 08:28 |
| gfidente | https://review.openstack.org/#/c/185581/ | 08:28 |
| gfidente | that is in relation to compute config timing out | 08:29 |
| gfidente | but controller 2 does have logs | 08:29 |
| jistr | ok i'll just revive the dependency patch then and let CI run on it | 08:30 |
| *** pblaho has joined #tripleo | 08:30 | |
| *** mcornea has joined #tripleo | 08:36 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Configure computes after controllers https://review.openstack.org/184064 | 08:37 |
| jistr | gfidente: i'm a bit sceptical of that though... i don't think that nova-compute not running correctly would cause the puppet run to hang... I don't think there's any logic in puppet (be it ours or puppet-nova) to say "wait till nova-compute connects to conductor" | 08:41 |
| gfidente | yeah | 08:42 |
| marios | think i got it missing , possibly | 08:42 |
| jistr | but both problems (puppet time out + nova compute stranded) might have the same cause | 08:42 |
| hewbrocca | Hey, who knows whether we are putting ceph MONs on our controllers? | 08:47 |
| *** regebro has quit IRC | 08:48 | |
| gfidente | hewbrocca, yes we are | 08:50 |
| gfidente | and we can optionally put ceph OSDs on controllers too | 08:50 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 08:50 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add metering agent as neutron-server plugin https://review.openstack.org/179370 | 08:53 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add neutron-lbaas as neutron-server plugin & neutron-lbaas-agent https://review.openstack.org/180535 | 08:53 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 08:53 |
| hewbrocca | gfidente: cool. Thanks. | 08:53 |
| jistr | gfidente: so i unpacked the logs from one of the controllers, looking at host_info.txt -- only nova-cert and nova-consoleauth are running there. No sign of api/conductor... | 08:55 |
| jistr | i thought they'd get killed off because of RAM or something | 08:57 |
| jistr | but the node has 4GB which should be enough | 08:57 |
| jistr | (i ran into problems locally with 3GB though) | 08:57 |
| jistr | and there are also no logs for nova-api / nova-conductor on the controllers | 08:57 |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 09:00 |
| openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora 21 due to Yum multilib error" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| gfidente | cert is the only one which doesn't have constraints | 09:01 |
| gfidente | consoleauth is the root of the constraints tree | 09:01 |
| jistr | ah ha, starts to get interesting | 09:08 |
| marios | guys was the last tripleo meeting really on may 05th? | 09:09 |
| marios | http://eavesdrop.openstack.org/meetings/tripleo/2015/ | 09:09 |
| jistr | i think it's fairly possible | 09:10 |
| jistr | it happens bi-weekly nowadays afaik | 09:10 |
| jistr | and the one after may 05 collided with summit | 09:10 |
| marios | jistr: yeah kinda suspected/inferred that from last few (thanks for confirming), the biweekly thing | 09:11 |
| marios | jistr: cool ta | 09:11 |
| *** akrivoka has quit IRC | 09:12 | |
| *** akrivoka has joined #tripleo | 09:13 | |
| *** paramite has quit IRC | 09:13 | |
| *** paramite has joined #tripleo | 09:14 | |
| shardy | https://review.openstack.org/#/c/162109/ is ready to land I think, CI is green and all the other pieces have now landed | 09:15 |
| shardy | It enables stepped deployment via devtest using heat hooks | 09:15 |
| shardy | any reviews would be most appreciated :) | 09:15 |
| gfidente | shardy, I'd merge it actually | 09:20 |
| gfidente | I think I reviewed but forgot to vote the other day | 09:20 |
| shardy | gfidente: great, thanks! | 09:21 |
| gfidente | I had to do it to understand why you downvoted the step name/number matching :P | 09:22 |
| openstackgerrit | Merged openstack/tripleo-incubator: Adding stepped deployment to devtest_overcloud.sh https://review.openstack.org/162109 | 09:22 |
| *** paramite has quit IRC | 09:22 | |
| jistr | yeah lgtm as well. I thought 30 secs might be a bit long time to wait between checks by default but with "-n 5" it's understandable | 09:22 |
| gfidente | jistr, I am running locally with 4G to see what happens | 09:23 |
| shardy | gfidente: Yeah, we may want to introduce a little test for the templates to help enforce that naming convention, or it'll inevitably keep getting broken | 09:27 |
| shardy | assuming folks actually find the stepped deployment useful that is, it was flagged as a "must have" feature, but I get the impression nobody is really using it :\ | 09:27 |
| *** bvandenh has joined #tripleo | 09:28 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci: Add pacemaker status to host_info.txt https://review.openstack.org/186690 | 09:28 |
| jistr | gfidente: ^ that could help us see what is going on in there | 09:28 |
| gfidente | jistr, yeah I remember telling derekh I wanted to do it ... :) | 09:29 |
| gfidente | derekh, ^^ pcs in host_info that is | 09:29 |
| gfidente | I even wanted to add ceph status :) | 09:29 |
| *** regebro has joined #tripleo | 09:31 | |
| *** paramite has joined #tripleo | 09:34 | |
| *** bvandenh has quit IRC | 09:36 | |
| gfidente | jistr, your turn for OPM, to rebase tht I think we need to update at least memcached and mongodb modules | 09:38 |
| *** pdb has quit IRC | 09:40 | |
| openstackgerrit | Merged openstack/os-collect-config: Drop use of 'oslo' namespace package https://review.openstack.org/180671 | 09:40 |
| *** pdb has joined #tripleo | 09:40 | |
| gfidente | marios, in https://review.openstack.org/#/c/181015/16/puppet/manifests/overcloud_controller_pacemaker.pp,cm as local_ip I think you want hiera('conroller_host') not the list of controllers | 09:41 |
| *** jprovazn has quit IRC | 09:56 | |
| gfidente | anyone got seed with disk full? | 10:02 |
| gfidente | mysql taking 20G for me there | 10:11 |
| gfidente | of -bin files | 10:12 |
| openstackgerrit | Merged openstack/diskimage-builder: rhel-common element should not use attach with activation key https://review.openstack.org/184253 | 10:13 |
| marios | gfidente: thanks, been trying to parse like $neutron_ovs_ml2_agent_local_ips = $controller_node_ips.filter |$x| { $x != '127.0.0.1' } :) will try that next | 10:14 |
| gfidente | :) | 10:15 |
| *** paramite is now known as paramite|afk | 10:16 | |
| gfidente | that is despite all database summing ~100mb together | 10:17 |
| *** u_glide has joined #tripleo | 10:20 | |
| u_glide | Hello folks! Thanks for great project! Is it possible to isolate somehow code for different distros in diskimage-builder element? | 10:22 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 10:32 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Switch CinderEnableIscsiBackend to CinderEnableLvmBackend https://review.openstack.org/186704 | 10:33 |
| *** dtantsur|brb is now known as dtantsur | 10:38 | |
| jistr | re OPM, ack | 10:43 |
| gfidente | so for mysql/seed issue I had to do a 'mysql reset master' to delete old binglogs | 10:47 |
| gfidente | not sure if we should set some expiry there | 10:47 |
| jistr | gfidente: mongodb should be ok i think https://review.gerrithub.io/#/c/234434/ | 10:47 |
| jistr | gfidente: that's 2nd latest commit | 10:47 |
| *** yog__ has quit IRC | 10:48 | |
| jistr | the latest commit in mongodb is related to puppet 4 unit tests so i doubt we need that | 10:48 |
| gfidente | jistr, ack, so I can only remember of memcached | 10:48 |
| jistr | and i'll also check if the keystone / glance / cinder updates landed | 10:49 |
| jistr | (db sync ones) | 10:49 |
| jistr | i saw lbezdick submitted them | 10:50 |
| jistr | but some of his patches are in merge conflict now | 10:50 |
| hewbrocca | derekh: you have an answer for u_glide ^^^ | 10:54 |
| hewbrocca | ? | 10:54 |
| derekh | hewbrocca: sorry missed it, reading up | 10:54 |
| *** jprovazn has joined #tripleo | 10:55 | |
| *** ishant has quit IRC | 10:59 | |
| derekh | u_glide: when you say isolate code for different distros what is it your trying to achieve? Most of the distro specific elements are in their own element e.g. http://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/fedora | 10:59 |
| derekh | gfidente: RE: pcs commands in host info, iirc I had no problem with doing it did I? | 11:00 |
| gfidente | derekh, you didn't I just never pushed the change :) | 11:00 |
| gfidente | hoped you to vote, not to comment :P | 11:00 |
| gfidente | merge merge merge | 11:01 |
| derekh | gfidente: cool, just making sure you weren't blocked on me or something +1 | 11:01 |
| derekh | ahh is their a review up there | 11:01 |
| gfidente | oh no no yeah I wanted to merge it | 11:01 |
| gfidente | and I remember we discussed so was asking for + | 11:01 |
| derekh | gfidente: found it, looking | 11:02 |
| *** bvandenh has joined #tripleo | 11:03 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add pacemaker status to host_info.txt https://review.openstack.org/186690 | 11:06 |
| *** AzherKhan has quit IRC | 11:09 | |
| *** kodiakfff has quit IRC | 11:12 | |
| *** kodiakfff has joined #tripleo | 11:14 | |
| gfidente | jistr, think I reproduced the nova issue locally | 11:14 |
| gfidente | pacemaker blocking nova chain on novncproxy not starting | 11:15 |
| gfidente | nova-novncproxy[2385]: AttributeError: 'module' object has no attribute 'ProxyRequestHandler' | 11:15 |
| jistr | that's actually great news! i mean certainly it's better than just hitting that on CI | 11:15 |
| gfidente | why should that happen only in the HA job though :!>!_ >!KOPK!) ? | 11:16 |
| *** bvandenh has quit IRC | 11:16 | |
| *** masco has quit IRC | 11:18 | |
| * jistr looking further | 11:18 | |
| * gfidente too | 11:19 | |
| jistr | got it | 11:19 |
| jistr | it's this http://lists.openstack.org/pipermail/openstack-dev/2014-October/048535.html | 11:20 |
| jistr | [root@ov-i7rc3g7pal4-0-zb74lymu7bdd-controller-bt22c2vran5g ~]# rpm -q python-websockify | 11:20 |
| jistr | python-websockify-0.5.1-2.fc21.noarch | 11:20 |
| gfidente | yeah same which made this fail | 11:20 |
| gfidente | to me | 11:20 |
| gfidente | I am still unsure why HA only? | 11:20 |
| jistr | yeah i dunno... i just looked into non-HA and it has the same old websockify | 11:21 |
| *** panda has quit IRC | 11:21 | |
| *** panda has joined #tripleo | 11:21 | |
| gfidente | which asking yum it seems we're getting from the official fedora reppo | 11:22 |
| openstackgerrit | Jan Provaznik proposed openstack/tripleo-heat-templates: Config & deployments to update overcloud packages https://review.openstack.org/179974 | 11:22 |
| hewbrocca | woooo jprovazn and the overcloud updates! | 11:22 |
| jprovazn | hewbrocca: this is steve's patch ;) | 11:23 |
| jprovazn | hewbrocca: I'm just trying to make CI happy | 11:23 |
| jprovazn | which is pretty tough job | 11:23 |
| hewbrocca | LOL OK | 11:23 |
| hewbrocca | yes | 11:23 |
| gfidente | jistr, it's not only HA job | 11:24 |
| jprovazn | maybe some other tripleoers more familiar with recent changes might help me to find out why http://logs.openstack.org/74/179974/6/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/0de5026/ is failing? | 11:24 |
| gfidente | jistr, HA job is the only one noticing the failure thanks to pacemaker I think | 11:24 |
| jprovazn | it's apparently because mysql on controller node failed to start because it didn't get heat metadata | 11:25 |
| jprovazn | but I wonder why? | 11:25 |
| gfidente | stack | 11:25 |
| gfidente | for the first time ever we have HA job doing something useful! | 11:25 |
| gfidente | with thanks to pacemaker! | 11:26 |
| gfidente | I like this | 11:26 |
| * hewbrocca applauds | 11:27 | |
| gfidente | jprovazn, you mean this https://review.gerrithub.io/#/c/233990/ | 11:27 |
| gfidente | jprovazn, sorry, this No hosts in Heat, nothing written. | 11:27 |
| jprovazn | gfidente: no hosts in heat? | 11:28 |
| jprovazn | not sure what you mean? | 11:28 |
| gfidente | that seems to be it http://logs.openstack.org/74/179974/6/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/0de5026/logs/ov-xp4mfu77trd-0-xcst4xoic5oy-Controller_logs/os-collect-config.txt.gz#_May_28_18_26_57 | 11:29 |
| *** kodiakfff has quit IRC | 11:29 | |
| jprovazn | gfidente: ah, yes - I focused on the 255 code | 11:30 |
| jprovazn | gfidente: so it's a known bug? | 11:30 |
| gfidente | I dunno | 11:30 |
| * gfidente checking other jobs | 11:30 | |
| jistr | jprovazn: see http://logs.openstack.org/74/179974/6/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/0de5026/console.html#_2015-05-28_18_33_15_118 | 11:30 |
| jprovazn | gfidente: I already did - other jobs recently CId are green | 11:31 |
| jistr | jprovazn: the controller/compute resources never got created | 11:31 |
| jistr | jprovazn: puppet didn't even start running anywhere | 11:31 |
| *** thrash|g0ne is now known as thrash | 11:31 | |
| jistr | jprovazn: could be that there weren't enough resources available | 11:31 |
| gfidente | jprovazn, this is recent and doesn't log that http://logs.openstack.org/16/186416/2/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/2289c07/logs/ov-d5ccxmavngw-0-pghkcncjewfq-Controller_logs/os-collect-config.txt.gz | 11:31 |
| *** kodiakfff has joined #tripleo | 11:33 | |
| jprovazn | jistr: wait, controller/compute resources - you mean nova instance resources or Controller/Compute ResourceGroup resources? | 11:35 |
| jistr | jprovazn: oh actually, it might not be not enough resources... i meant overloaded CI | 11:36 |
| jistr | jprovazn: sometimes it takes a looong time for the VMs to get created (can be 1 hour no problem), that's why we had to bump up heat timeout on HA job to 90 minutes | 11:36 |
| jistr | puppet runs for ~ 13 minutes but machine reservation takes 1 hour | 11:37 |
| jistr | but | 11:37 |
| jprovazn | jistr: well, I can see that VMs spawned properly, I see logs from both - here is the log from controller where is the reason why it timedout - http://logs.openstack.org/74/179974/6/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/0de5026/logs/ov-xp4mfu77trd-0-xcst4xoic5oy-Controller_logs/os-collect-config.txt.gz | 11:37 |
| jistr | i think i was lead astray by a false assumption that the compute/controller resources weren't changed in the patch, which is not true | 11:37 |
| jistr | yeah | 11:37 |
| jprovazn | jistr: but I don't know where to find in logs the reason it didn't find for what it was looking for :( | 11:38 |
| * gfidente recheck everything failing on non-puppet :) | 11:38 | |
| jprovazn | and because it failed twice with same error and other recent CI jobs pass, I almost tend to think it might be caused by the patch | 11:39 |
| jprovazn | which would really break my Friday mood | 11:39 |
| jistr | yeah that's probable actually | 11:39 |
| * hewbrocca sends jprovazn a beer to restore his friday mood | 11:43 | |
| jprovazn | hehe, thanks | 11:43 |
| *** rhallisey has joined #tripleo | 11:43 | |
| *** paramite|afk is now known as paramite | 11:43 | |
| *** lucasagomes is now known as lucas-hungry | 11:44 | |
| jprovazn | actually that is a good idea - I don't think this is a problem which couldn't be solved by increased amount of beer | 11:44 |
| jistr | [2015-05-28 18:01:49,402] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1] | 11:44 |
| jistr | this looks like a problem right? | 11:44 |
| jprovazn | jistr: I suppose this is a first run when not metadata were avail yet, if you search for "ERROR" you can see that later it got further to the mysql setup script | 11:47 |
| jistr | ah ok | 11:48 |
| jistr | 2 questions/observations | 11:52 |
| jistr | 1) should we run first puppet and then yum update? | 11:52 |
| gfidente | jistr, failing job is non-puppet | 11:53 |
| jistr | ok, but the question was a bit more general in the sense -- does that patch implement what was intended? | 11:54 |
| hewbrocca | There are not many problems which fail to yield to beer | 11:54 |
| jistr | i saw a mailing list discussion where i think the outcome was that puppet should be run first, so that it knows which service packages got updated and which services need restarting | 11:55 |
| jistr | (not sure if i read all the latest messages but at some point at least this seemed like a consensus) | 11:55 |
| *** trown|outttypeww is now known as trown | 11:56 | |
| jistr | if that's the target, then i think the yum update needs to be part of controller-post(-puppet) instead of controller(-puppet) | 11:56 |
| jistr | 2) does the overcloud in CI have internet connectivity? if not, could it be the cause of the issues with the yum update part? | 11:57 |
| hewbrocca | yum update, in CI | 11:58 |
| hewbrocca | WHAT HAVE WE DONE | 11:58 |
| * hewbrocca sends more beer | 11:58 | |
| hewbrocca | hey, does anybody know what we do as far as deploying keystone with httpd, or not? | 12:00 |
| *** dprince has joined #tripleo | 12:01 | |
| hewbrocca | IOW are we using Keystone v3 API with httpd and WSGI | 12:01 |
| jistr | i don't think we deploy it with httpd atm | 12:01 |
| hewbrocca | or earlier keystone API with eventlet? | 12:01 |
| openstackgerrit | Marek Aufart proposed openstack/python-tuskarclient: Filter and format parameters for plan role in OSC https://review.openstack.org/186728 | 12:01 |
| jistr | hewbrocca: i think it's v2 with eventlet | 12:02 |
| hewbrocca | fabbione: ^^^ | 12:03 |
| hewbrocca | you had asked about keystone... | 12:03 |
| gfidente | directxman12, what do we do with websockify for nova? | 12:05 |
| gfidente | directxman12, it looks like f22 will get 0.6 but f21 will stay with 0.5.1 ? | 12:05 |
| gfidente | directxman12, I can see 0.6 on rhel 7.1 as well though | 12:07 |
| jistr | gfidente: yeah i wanted to get back to this (too many issues for one friday). I have a deployment predating the Nova-pcmk patch, and nova-novncproxy is down there too, except nothing checks it so nothing cares, you were right that nothing new is broken, it just doesn't deploy because pacemaker cares about the breakage :) | 12:08 |
| gfidente | jistr, yeah I was thinking to comment with a FIXME in the .pp as wel | 12:08 |
| gfidente | jistr, or maybe a TODO | 12:08 |
| gfidente | jistr, I have yet to decide | 12:09 |
| gfidente | :P | 12:09 |
| jistr | gfidente: yeah i think the best we can do now is just disable novncproxy for right now and proceed | 12:09 |
| gfidente | ack will update | 12:09 |
| *** jayg|g0n3 is now known as jayg | 12:09 | |
| gfidente | I am concerned though about f21 not getting 0.6 though | 12:09 |
| jistr | yeah... | 12:10 |
| gfidente | directxman12, is owner, we'll see when he is back https://apps.fedoraproject.org/packages/python-websockify | 12:10 |
| fabbione | hewbrocca: ack | 12:14 |
| hewbrocca | fabbione: I responded to your email as well | 12:16 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Nova as Pacemaker resource https://review.openstack.org/185766 | 12:18 |
| *** weshay has quit IRC | 12:24 | |
| marios | \o/ https://review.openstack.org/#/c/181015/17 green ha job | 12:30 |
| marios | hewbrocca: this means i am relinquished of my obligation to perform ritual harakiri by sundown local | 12:30 |
| marios | win win | 12:30 |
| hewbrocca | hot damn son | 12:31 |
| hewbrocca | nice job | 12:31 |
| gfidente | marios++ :) | 12:31 |
| * hewbrocca thwacks marios on the... back | 12:31 | |
| marios | actually it was mostly gfidente so, for now, i'll call the hit off on his head | 12:31 |
| gfidente | <hewbrocca> everybody shares the credit | 12:32 |
| *** morazi has joined #tripleo | 12:32 | |
| *** weshay has joined #tripleo | 12:34 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add metering agent as neutron-server plugin https://review.openstack.org/179370 | 12:41 |
| u_glide | derekh: I want to achieve something similar to separation for different install types like here https://github.com/openstack/diskimage-builder/tree/master/elements/simple-init/install.d but for distros | 12:41 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add neutron-lbaas as neutron-server plugin & neutron-lbaas-agent https://review.openstack.org/180535 | 12:42 |
| hewbrocca | holy christ marios lbaas? you really are going for broke aren't you | 12:42 |
| jistr | marios: pls let me know when you open the bug discussed on the neutron review (the line there overrides the networking code, it's a bit weird that it didn't work for you locally, we should try to remove it in a follow up patch) | 12:43 |
| jistr | marios: cause i'm ready to +a | 12:43 |
| marios | jistr: sure, sec | 12:44 |
| marios | jistr: where does it belong though? puppet-neutron? tht? | 12:44 |
| gfidente | marios, yeah I think it is obvious that for other classes we inherit the params | 12:44 |
| gfidente | so it's something with that which seems broken | 12:45 |
| jistr | mandre: tht | 12:45 |
| jistr | marios: tht | 12:45 |
| gfidente | puppet-neutron | 12:45 |
| marios | lol | 12:45 |
| jistr | mandre: sorry again :) | 12:45 |
| gfidente | jistr, tht? | 12:45 |
| marios | 'what does mine say' | 12:45 |
| marios | 'duuuuude' | 12:45 |
| u_glide | ok, another question - folks, Is it possible to specify custom parameter in package-installs.yaml/json ? like --auto-remove for apt-get ? | 12:45 |
| jistr | marios, gfidente: are we talking about this thing, right? https://review.openstack.org/#/c/181015/17/puppet/manifests/overcloud_controller_pacemaker.pp | 12:46 |
| jistr | marios, gfidente: that line needs to be removed from tht, so the bug belongs to tht | 12:46 |
| marios | jistr: ok, will file there for now anyway | 12:46 |
| marios | ack | 12:46 |
| gfidente | jistr, marios sec | 12:46 |
| gfidente | I think the line in tht is fine, module does not seem to be using it when defining clas { ... } | 12:47 |
| jistr | oooh | 12:47 |
| gfidente | but that shouldn't be the case, so I think the bug goes to puppet-neutron | 12:47 |
| jistr | ok sorry folks :) | 12:47 |
| gfidente | but actually, we need to track this issue in the yaml/pp couple as well | 12:48 |
| gfidente | so probably best to add both components to same bug | 12:48 |
| jistr | yeah so both ideally | 12:48 |
| jistr | ack | 12:48 |
| *** lucas-hungry is now known as lucasagomes | 12:48 | |
| *** regebro has quit IRC | 12:50 | |
| marios | jistr: https://bugs.launchpad.net/tripleo/+bug/1460058 | 12:53 |
| openstack | Launchpad bug 1460058 in tripleo "tripleo-heat-templates hiera neutron::agents::ml2::ovs::local_ip defined and passed but not consumed during puppet pacemaker run" [Undecided,New] | 12:53 |
| marios | gfidente: jistr yeah wherever as long as we have a placeholder (incidentally, tht bugs go against tripleo right? can't see tht specific bugs_ | 12:54 |
| *** radez_g0n3 is now known as radez | 12:54 | |
| marios | gfidente: feel free to change it to whatever | 12:54 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 12:55 |
| *** lblanchard has joined #tripleo | 12:56 | |
| jistr | marios: thanks! merged | 12:57 |
| marios | \o/ | 12:59 |
| marios | hewbrocca: can we all retire now? | 12:59 |
| hewbrocca | Yep | 13:00 |
| hewbrocca | We are DONE | 13:00 |
| marios | excellent | 13:00 |
| marios | gfidente: how can i help with the nova one. is it still useful to have a run with that? (will you rebase :( sorry it might be a tricky one, i can do it for you? /me checks) | 13:01 |
| gfidente | no worries, rebasing that and a couple of others on top so we don't conflict | 13:02 |
| gfidente | we'll have to wait for CI results anyway | 13:02 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Nova as Pacemaker resource https://review.openstack.org/185766 | 13:02 |
| marios | showoff | 13:03 |
| hewbrocca | LOL | 13:03 |
| marios | gfidente: will still hit you. just for fun, not because it is your fault. | 13:03 |
| gfidente | I will never ever make any useful comment anymore | 13:04 |
| * gfidente wasn't this already the case? | 13:04 | |
| gfidente | ^_^ | 13:04 |
| marios | gfidente: well if you're gonna be like that about it i'm going to go pull master now that _my_ patch merged and then try out your patch. and there is nothing you can do about it. | 13:05 |
| marios | nothing | 13:05 |
| gfidente | marios, empathy was on 'useful' not on 'comment' | 13:06 |
| gfidente | which was already the case | 13:06 |
| marios | y u hate me? | 13:06 |
| marios | :( | 13:06 |
| gfidente | not if we merge the things | 13:07 |
| derekh | u_glide: for code seperation based on distro you can use $DISTRO_NAME e.g. the kind for thing you'll see in https://github.com/openstack/diskimage-builder/blob/master/elements/iso/post-install.d/01-copy-bootloaders | 13:07 |
| u_glide | derekh: awesome! thanks! | 13:08 |
| marios | gfidente: i rebuilt images like an hour ago (puppet modules from source) should be good yes? | 13:08 |
| gfidente | yes! | 13:09 |
| gfidente | thanks1 | 13:09 |
| dprince | shardy: just discovered that parameter_defaults doesn't apply to the top level stack. It only applies to nested stacks... https://bugs.launchpad.net/heat/+bug/1460063 | 13:09 |
| openstack | Launchpad bug 1460063 in heat "parameter_defaults only apply to nested stacks" [Undecided,New] | 13:09 |
| gfidente | :) | 13:09 |
| dprince | shardy: can work around it for now. Any thoughts on what I'm suggesting? | 13:09 |
| dprince | shardy: my choices are to duplicate things into both parameter_defaults and parameters... or I can be sneaky and return ServiceNetMap to the top level stack from one of the nested stacks. | 13:11 |
| dprince | shardy: I'm leaning towards the latter so as not to force us to duplicate things in our env files for now... | 13:12 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Heat as a Pacemaker resource https://review.openstack.org/185975 | 13:18 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Nova as Pacemaker resource https://review.openstack.org/185766 | 13:18 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Ceilometer as a Pacemaker resource https://review.openstack.org/185916 | 13:18 |
| gfidente | marios, look what I rebased for you | 13:18 |
| gfidente | :P | 13:18 |
| shardy | dprince: We should just fix it in heat, I've taken the bug and will aim to get a patch to you by EOD | 13:22 |
| shardy | whatever works in the meantime so it doesn't hold you up :) | 13:22 |
| *** noslzzp_ has joined #tripleo | 13:22 | |
| shardy | dprince: you could probably use a yaml reference between parameter_defaults and parameters as a workaround? | 13:23 |
| dprince | shardy: could try that too | 13:23 |
| *** noslzzp has quit IRC | 13:25 | |
| *** jeh has joined #tripleo | 13:29 | |
| *** subscope has quit IRC | 13:35 | |
| *** pblaho has quit IRC | 13:38 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Make all-nodes Ip networks configurable https://review.openstack.org/185581 | 13:44 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add virtual IPs for split out networks https://review.openstack.org/186238 | 13:44 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Combine Heat API networks into single net https://review.openstack.org/186625 | 13:44 |
| *** gfidente is now known as gfidente|afk | 13:49 | |
| *** paramite has quit IRC | 13:50 | |
| *** subscope has joined #tripleo | 13:51 | |
| *** tzumainn has joined #tripleo | 13:57 | |
| *** mcornea has quit IRC | 14:00 | |
| dasm | guys, quick question about HA for controllers. Does Tripleo have it? | 14:00 |
| dasm | i'm a little confused, because afair i've seen somewhere info, that tripleo allows to deploy cluster of controllers in ha. but now cannot find it anymore. | 14:01 |
| dasm | and i see ongoing work in ha of services, by pacemaker. | 14:01 |
| dasm | so, in short words: what is the status of ha ;) | 14:01 |
| hewbrocca | dasm: you're asking at exactly the right time! | 14:02 |
| *** mcornea has joined #tripleo | 14:02 | |
| hewbrocca | I believe we have merged just about all of the pacemaker resource support | 14:02 |
| dasm | hewbrocca: i see heat, ceilometer and nova in review, so probably some bits are still to be merged. | 14:03 |
| dasm | ;) | 14:03 |
| hewbrocca | which means that we, as of this afternoon, support three-controller HA with Pacemaker, Galera, Rabbit, Neutron, and so on | 14:03 |
| hewbrocca | heat, ceilo, and nova are the last bits waiting to be merged | 14:03 |
| hewbrocca | Ceph, too! | 14:03 |
| dasm | so, may i assume that at Morning/Tuesday we'll have full HA support in Tripleo? Is it correct? | 14:04 |
| dasm | (so what am i doing here? :P) | 14:04 |
| hewbrocca | dasm: everything but automated config of fence agents | 14:06 |
| hewbrocca | which we will be working on next week | 14:06 |
| dasm | hewbrocca: thanks, you helped me very much. | 14:07 |
| hewbrocca | The other big thing we're trying to wrap up is network isolation | 14:09 |
| hewbrocca | we're getting there, but there is still quite a bit to do | 14:09 |
| hewbrocca | dprince: is leading that work | 14:09 |
| *** cody-somerville has quit IRC | 14:13 | |
| *** subscope has quit IRC | 14:15 | |
| *** sseago has left #tripleo | 14:22 | |
| jprovazn | shardy: slagle: CI is finally happy with https://review.openstack.org/#/c/179974/ , could you please take a look when you have a minute? | 14:23 |
| *** mcornea has quit IRC | 14:24 | |
| *** mcornea has joined #tripleo | 14:25 | |
| * jistr reloc | 14:26 | |
| *** jistr has quit IRC | 14:26 | |
| *** olaph has quit IRC | 14:29 | |
| *** cody-somerville has joined #tripleo | 14:30 | |
| *** subscope has joined #tripleo | 14:30 | |
| *** olaph has joined #tripleo | 14:30 | |
| dasm | hewbrocca: mhm.. good to know about dprince work. | 14:35 |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Configure virtual IPs for split out networks https://review.openstack.org/186240 | 14:37 |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Use node IP lists for HA Proxy ipaddresses https://review.openstack.org/186775 | 14:37 |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Add interface_for_ip function https://review.openstack.org/186776 | 14:38 |
| hewbrocca | holy hell dprince, VIPs? | 14:38 |
| hewbrocca | you are a machine | 14:38 |
| dprince | hewbrocca: well, I haven't touched the pacemaker bits yet. but this is a start | 14:40 |
| dprince | hewbrocca: this is coming together | 14:40 |
| *** dasm is now known as dasm|afk | 14:42 | |
| *** rameshg87 has joined #tripleo | 14:42 | |
| rameshg87 | hello tripleo-cores | 14:43 |
| rameshg87 | can someone please have a look at https://review.openstack.org/#/c/178512/ and https://review.openstack.org/#/c/179992/ | 14:43 |
| rameshg87 | it has been with one +2 for more than 20 days | 14:44 |
| *** rameshg87 is now known as rameshg87-away | 14:45 | |
| *** sdake has joined #tripleo | 14:46 | |
| *** jistr has joined #tripleo | 14:47 | |
| hewbrocca | dprince: Yes... yes it is... | 14:49 |
| *** rameshg87-away has quit IRC | 14:52 | |
| *** gfidente|afk is now known as gfidente | 15:03 | |
| *** jprovazn has quit IRC | 15:07 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Heat as a Pacemaker resource https://review.openstack.org/185975 | 15:15 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Nova as Pacemaker resource https://review.openstack.org/185766 | 15:15 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Ceilometer as a Pacemaker resource https://review.openstack.org/185916 | 15:15 |
| marios | gfidente: why you -1 your nova | 15:15 |
| marios | ah looking again | 15:15 |
| gfidente | to remove nova-cert | 15:15 |
| gfidente | there was some conversation about it not being in the refarch | 15:16 |
| gfidente | so I removed it and removed the dependency on metering and lbaas | 15:16 |
| *** rameshg87 has joined #tripleo | 15:17 | |
| *** rameshg87 has left #tripleo | 15:18 | |
| *** panda has quit IRC | 15:21 | |
| *** alop has joined #tripleo | 15:21 | |
| *** panda has joined #tripleo | 15:21 | |
| gfidente | jistr, L3HA in CI https://review.openstack.org/#/c/185461/ | 15:23 |
| *** lazy_prince has quit IRC | 15:24 | |
| *** sdake_ has joined #tripleo | 15:26 | |
| gfidente | marios, ^^ | 15:26 |
| *** alop has quit IRC | 15:27 | |
| *** ifarkas has quit IRC | 15:28 | |
| *** sdake has quit IRC | 15:30 | |
| *** bvandenh has joined #tripleo | 15:30 | |
| *** subscope has quit IRC | 15:35 | |
| jistr | gfidente: merged | 15:41 |
| gfidente | ^_^ | 15:41 |
| jistr | gfidente: later i realized that we could even put that yaml to t-h-t but it's not worth a resubmit and re-CI | 15:42 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Enable NeutronL3HA for the HA job https://review.openstack.org/185461 | 15:42 |
| gfidente | jistr, you mean using a specific environment file for CI? | 15:43 |
| marios | HAPPY FRIDAY | 15:44 |
| marios | bai | 15:44 |
| jistr | marios: o/ enjoy the weekend | 15:44 |
| hewbrocca | obai | 15:44 |
| marios | u2all | 15:44 |
| *** mcornea has quit IRC | 15:45 | |
| jistr | gfidente: not really, i meant just instead of cat <<EOF we can just put the l3ha file in t-h-t proper, and then folks can use it even with their dev envs by just pointing to it (in combination with the HA file, exactly as TOCI has it) | 15:45 |
| jistr | just too many "just" in one sentence | 15:46 |
| gfidente | jistr, I see, so not a specific file for CI but one with L3HA settings only to be combined with HA? | 15:46 |
| jistr | gfidente: yup | 15:46 |
| gfidente | I promise if you don't do, I might do it sometime next week :) | 15:47 |
| jistr | yeah it's a nice thing to have but low prio | 15:47 |
| *** alop has joined #tripleo | 15:48 | |
| marios | gfidente: i was forced to reveal the truth | 15:49 |
| marios | about how you treat me | 15:49 |
| marios | https://review.openstack.org/#/c/185766/ | 15:49 |
| *** daneyon has joined #tripleo | 15:50 | |
| gfidente | I think this has to be sorted somewhere not on irc | 15:50 |
| *** cwolferh has joined #tripleo | 15:51 | |
| marios | gfidente: i am going to leave now. but i'll leave my nick around to keep an eye on you. and i won't respond. i will ignore you. until monday. and there is nothing you can do about it | 15:51 |
| marios | nothing | 15:51 |
| gfidente | thanks | 15:51 |
| *** daneyon_ has joined #tripleo | 15:53 | |
| *** daneyon has quit IRC | 15:55 | |
| shardy | Is there any info about the ceph implementation around other than the templates? | 15:56 |
| jistr | marios, gfidente: LOL you two | 15:56 |
| jistr | i'm off, have a nice weekend everybody | 15:57 |
| shardy | I'm looking for details of how it all hangs together with the Block/ObjectStorage nodes | 15:57 |
| hewbrocca | mmm morazi ^^^ | 15:57 |
| hewbrocca | where do we find that | 15:57 |
| hewbrocca | shardy: are you blogging, or writing patches? | 15:58 |
| *** jistr has quit IRC | 15:58 | |
| shardy | hewbrocca: slide preparation, looking for some cut/pasteable resources ;) | 15:58 |
| hewbrocca | ahhh -- noslzzp_ ^^^ ? | 15:59 |
| noslzzp_ | (FBI call starting) | 15:59 |
| gfidente | shardy, objectstorage is not using ceph yet | 15:59 |
| shardy | It's entirely possible no such resources exist, I just wanted to avoid wheel reinvention :) | 15:59 |
| shardy | gfidente: Ok, thanks what I wanted to know | 16:00 |
| gfidente | to be honest the documentation we had was in the cinder/ha blueprint | 16:00 |
| shardy | but the cinder backend gets automagically switched to ceph right? | 16:00 |
| gfidente | automagically no | 16:00 |
| gfidente | I think best documentation we have is this env file | 16:00 |
| gfidente | https://github.com/openstack/tripleo-heat-templates/blob/master/environments/puppet-ceph-devel.yaml | 16:01 |
| shardy | gfidente: Aha, thanks I'd forgotten that | 16:02 |
| gfidente | so you can optionally enable it for glance/cinder/nova | 16:02 |
| gfidente | and optionally deploy osd on controller | 16:02 |
| gfidente | instack is enabling ceph for all three when cephstoragescale > 1 | 16:03 |
| shardy | gfidente: perfect, thanks - that's exactly the info I needed | 16:03 |
| gfidente | yeah so instack is automagically ... | 16:04 |
| gfidente | shardy, spare a vote on CI! https://review.openstack.org/#/c/183473/ :) | 16:05 |
| gfidente | (ceph) | 16:05 |
| *** jistr has joined #tripleo | 16:06 | |
| shardy | gfidente: done | 16:08 |
| gfidente | tks, it's there from a while now | 16:08 |
| gfidente | would be nice to have it | 16:08 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add virtual IPs for split out networks https://review.openstack.org/186238 | 16:09 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add PortName to ports stacks https://review.openstack.org/186818 | 16:09 |
| *** jistr has quit IRC | 16:09 | |
| *** tteggel has quit IRC | 16:09 | |
| shardy | gfidente: one last dumb ceph question, with CinderEnableRbdBackend do you still deploy BlockStorage nodes, or do the cinder services on the controller interact directly with ceph? | 16:10 |
| * shardy needs more hardware to try this stuff.. | 16:10 | |
| gfidente | shardy, BlockStorage nodes are meant to run cinder-volume only and they only get configured with the LVM backend | 16:10 |
| gfidente | shardy, CephStorage nodes run ceph-osd | 16:11 |
| *** tteggel has joined #tripleo | 16:11 | |
| gfidente | so the two scale independently and you can have 0 or more of each | 16:11 |
| gfidente | the rbd driver talks to both ceph-mon (running on controller) and ceph-osd (running on cephstorage nodes or on controller) | 16:12 |
| *** dtantsur is now known as dtantsur|afk | 16:12 | |
| shardy | gfidente: Ok, thanks - that clarifies things, I wasn't sure exactly how the services were wired up without digging in templates | 16:13 |
| morazi | shardy, I think gfidente has got you mostly covered there and agree re: object storage. that would come in when we do the rgw work, but it isn't there yet. | 16:13 |
| *** rwsu has joined #tripleo | 16:15 | |
| *** Goneri has quit IRC | 16:20 | |
| *** spzala has quit IRC | 16:24 | |
| *** spzala has joined #tripleo | 16:25 | |
| *** rhallisey has quit IRC | 16:34 | |
| *** rhallisey has joined #tripleo | 16:39 | |
| dprince | dsneddon: How would you feel about naming KeystonePublicApiNetwork just KeystoneApiNetwork. | 16:39 |
| dprince | dsneddon: it will have VIPs (configurable of course) bound to 2 networks, public, and say internal_api by default. | 16:40 |
| dprince | dsneddon: the public network is sort of a given however, we only need to specify the "internal_api" network via this parameter... | 16:41 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements https://review.openstack.org/178419 | 16:41 |
| dprince | dsneddon: in other words, I'm just saying I find the 'Public' in that setting confusing... because we probably want it set to internal_api by default. | 16:41 |
| * gfidente salute everyone | 16:42 | |
| *** gfidente has quit IRC | 16:43 | |
| *** derekh has quit IRC | 16:48 | |
| *** weshay has quit IRC | 16:53 | |
| *** weshay has joined #tripleo | 16:53 | |
| openstackgerrit | Merged openstack/diskimage-builder: Fix disk image create errors behind proxy https://review.openstack.org/178512 | 16:53 |
| *** spzala has quit IRC | 16:55 | |
| *** spzala has joined #tripleo | 16:55 | |
| *** jeh has quit IRC | 16:56 | |
| *** daneyon_ has quit IRC | 17:05 | |
| dsneddon | dprince, Keystone Public Network should be on the External network, not internal_api. | 17:08 |
| *** sdake has joined #tripleo | 17:09 | |
| dprince | dsneddon: we use the keystone API both internally and externally. http://git.openstack.org/cgit/stackforge/puppet-tripleo/tree/manifests/loadbalancer.pp#n296 | 17:10 |
| dprince | dsneddon: this setting will effectively control the private side (i.e. swap in for $controller_virtual_ip in that puppet manifest) | 17:10 |
| dprince | dsneddon: Most of the OpenStack API's already have VIPs on the "public" network... | 17:11 |
| *** sdake_ has quit IRC | 17:12 | |
| *** noslzzp has joined #tripleo | 17:12 | |
| *** noslzzp_ has quit IRC | 17:13 | |
| dsneddon | dprince, We aren't making use of internal URLs at this time, only Public. The OpenStack Security Guide is confusing on this matter and recommends Internal URLs to be "forward-compatible": http://docs.openstack.org/security-guide/content/api-endpoint-configuration-recommendations.html | 17:19 |
| *** sdake_ has joined #tripleo | 17:20 | |
| dprince | dsneddon: right, so that is what we do via the catalog. But within the cluster we are using some "private" IPs directly | 17:20 |
| dsneddon | dprince, We definitely want HAProxy listening on both the Public network and the Internal API network. | 17:21 |
| dprince | dsneddon: it is for these cases that I find the KeystonePublicApiNetwork confusing | 17:21 |
| dprince | dsneddon: I think we are on the same page... all I'm dealing with here is a naming/confusion issue | 17:21 |
| dprince | dsneddon: I want the templates to read clearly.... | 17:21 |
| dsneddon | dprince, I don't see how we can combine KeystoneApiNetwork and KeystonePublicApiNetwork and still be able to control the placement onto two places. Remember, in the end we want Keystone listening on port 5000 in two places. | 17:23 |
| *** sdake has quit IRC | 17:23 | |
| dprince | dsneddon: it will always listen on the public one. The other is what I'm suggesting is currently configurable. | 17:23 |
| dsneddon | dprince, OK, that's fine. | 17:24 |
| dprince | dsneddon: so given that would you buy calling it just KeystoneApiNetwork then (drop the Public) | 17:24 |
| dprince | dsneddon: and we'd still have the separate setting for KeystoneAdminApiNetwork too | 17:25 |
| dprince | dsneddon: perhaps KeystoneAdminApiNetwork is the one we should potentially not have listening on public BTW... | 17:25 |
| *** cody-somerville has quit IRC | 17:25 | |
| dprince | dsneddon: or at least an option to do that if someone wants it | 17:25 |
| *** sdake has joined #tripleo | 17:26 | |
| dsneddon | dprince, I could see making the KeystoneAdminApiNetwork configurable, but most people are not going to want the Keystone Admin API to be public-facing, because that's where you add users/passwords. | 17:26 |
| *** barra204 has quit IRC | 17:26 | |
| dsneddon | dprince, Also, the Admin API doesn't seem to be wired up, or at least I didn't find where it was. I've been meaning to talk to you about that. | 17:27 |
| dprince | dsneddon: well, it currently would do both public (always) and the other is configurable: http://git.openstack.org/cgit/stackforge/puppet-tripleo/tree/manifests/loadbalancer.pp#n278 | 17:27 |
| *** sdake_ has quit IRC | 17:29 | |
| dsneddon | dprince, We should change this, but we can do a follow-up patch. The Admin API shouldn't be enabled on the Public by default. | 17:30 |
| dprince | dsneddon: okay, sounds reasonable | 17:30 |
| *** trown is now known as trown|lunch | 17:31 | |
| *** penick has joined #tripleo | 17:41 | |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Configure virtual IPs for split out networks https://review.openstack.org/186240 | 17:41 |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Add interface_for_ip function https://review.openstack.org/186776 | 17:41 |
| dsneddon | dprince, Hoo boy, MySQL should definitely not be on the Public API network. How about a follow-up security-related patch that cleans up several of these things? | 17:44 |
| dhellmann | bnemec: thanks | 17:44 |
| dprince | dsneddon: mysql is only on the "private" network currently. So I think we are good there: http://git.openstack.org/cgit/stackforge/puppet-tripleo/tree/manifests/loadbalancer.pp#n563 | 17:45 |
| dhellmann | slagle: I would like to release os-collect-config 0.1.35 on Monday to include the change that unblocks the work we're doing to drop the use of the oslo namespace package. I can tag the release, but wanted to coordinate with you to ensure that it wouldn't cause you any trouble. | 17:45 |
| dprince | dsneddon: so once these patches land essentially it will only bind to its VIP on the configured MysqlNetwork | 17:45 |
| dprince | dsneddon: a security audit would be in order when we are done this stuff though... | 17:46 |
| dsneddon | dprince, If that's the case, then what is happening in L545 of this file: https://review.openstack.org/#/c/186240/3/manifests/loadbalancer.pp | 17:46 |
| dsneddon | dprince, Oops I mean L563 | 17:47 |
| dprince | dsneddon: controller_virtual_ip is the provisioning network | 17:47 |
| dsneddon | dprince, Actually, I think you made a typo. You turned [controller_virtual_ip] into unique([hiera('mysql_vip', $controller_virtual_ip), $public_virtual_ip]) | 17:48 |
| *** noslzzp_ has joined #tripleo | 17:49 | |
| dprince | dsneddon: Oh, Yeah. That is nasty. -1 me for that | 17:49 |
| dprince | dsneddon: copy paste error | 17:49 |
| bnemec | dhellmann: slagle is out today. I don't think that will be a problem though since that's the only change in the project since the last release. | 17:51 |
| *** noslzzp has quit IRC | 17:51 | |
| dhellmann | bnemec: so it's ok for me to release? | 17:52 |
| bnemec | dhellmann: Yeah, should be fine. I don't think there's anything significantly different in our release process. https://wiki.openstack.org/wiki/TripleO/ReleaseManagement | 17:55 |
| bnemec | We should probably get rid of that and just use the new release tools. | 17:55 |
| dhellmann | bnemec: yeah, I was planning to use the library release scripts, but I'll check and make sure there are no big differences in process | 17:55 |
| dhellmann | oh, no, cool, that's just the manual process that has been automated now | 17:56 |
| bnemec | Yeah, it's not doing anything too crazy. | 17:56 |
| *** cwolferh has quit IRC | 17:56 | |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Configure virtual IPs for split out networks https://review.openstack.org/186240 | 17:58 |
| *** aufi has quit IRC | 18:00 | |
| *** spzala has quit IRC | 18:03 | |
| *** spzala has joined #tripleo | 18:03 | |
| *** penick has quit IRC | 18:05 | |
| *** jrist has quit IRC | 18:06 | |
| *** sdake_ has joined #tripleo | 18:07 | |
| *** barra204 has joined #tripleo | 18:08 | |
| *** sdake has quit IRC | 18:12 | |
| *** jrist has joined #tripleo | 18:12 | |
| *** barra204 has quit IRC | 18:15 | |
| *** pradk has joined #tripleo | 18:15 | |
| *** penick has joined #tripleo | 18:17 | |
| *** spzala has quit IRC | 18:18 | |
| *** spzala has joined #tripleo | 18:18 | |
| *** barra204 has joined #tripleo | 18:19 | |
| *** spzala has left #tripleo | 18:19 | |
| *** barra204_ has joined #tripleo | 18:27 | |
| *** pradk has quit IRC | 18:27 | |
| *** pino|work has quit IRC | 18:30 | |
| *** pradk has joined #tripleo | 18:30 | |
| *** barra204 has quit IRC | 18:31 | |
| *** trown|lunch is now known as trown | 18:36 | |
| *** dsneddon is now known as dsneddon_lunch | 18:37 | |
| *** barra204_ is now known as shakamunyi | 18:52 | |
| *** jcoufal has joined #tripleo | 18:54 | |
| *** jcoufal has quit IRC | 18:55 | |
| *** penick has quit IRC | 19:00 | |
| *** panda has quit IRC | 19:21 | |
| *** panda has joined #tripleo | 19:22 | |
| *** lucasagomes has quit IRC | 19:24 | |
| *** penick has joined #tripleo | 19:33 | |
| *** yamahata has joined #tripleo | 19:35 | |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Configure virtual IPs for split out networks https://review.openstack.org/186240 | 19:38 |
| openstackgerrit | Dan Prince proposed stackforge/puppet-tripleo: Add interface_for_ip function https://review.openstack.org/186776 | 19:38 |
| *** yamahata has quit IRC | 19:43 | |
| *** sdake_ is now known as sdake | 19:43 | |
| *** yamahata has joined #tripleo | 19:43 | |
| *** dprince has quit IRC | 19:44 | |
| *** sdake_ has joined #tripleo | 19:46 | |
| *** sdake has quit IRC | 19:49 | |
| *** sdake has joined #tripleo | 19:49 | |
| *** penick has quit IRC | 19:50 | |
| *** sdake_ has quit IRC | 19:53 | |
| *** cwolferh has joined #tripleo | 19:55 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add Nova as Pacemaker resource https://review.openstack.org/185766 | 20:01 |
| *** openstack has joined #tripleo | 20:06 | |
| *** openstack has joined #tripleo | 20:31 | |
| *** sdake_ has joined #tripleo | 20:36 | |
| *** lblanchard has quit IRC | 20:39 | |
| *** sdake has quit IRC | 20:40 | |
| *** spzala has joined #tripleo | 20:49 | |
| *** sdake has joined #tripleo | 20:50 | |
| *** dsneddon_lunch is now known as dsneddon | 20:53 | |
| *** sdake_ has quit IRC | 20:54 | |
| *** openstackgerrit has quit IRC | 20:59 | |
| *** openstackgerrit has joined #tripleo | 20:59 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 21:00 |
| openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora 21 due to Yum multilib error" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** pradk has quit IRC | 21:02 | |
| *** julim has quit IRC | 21:07 | |
| *** trown is now known as trown|outttypeww | 21:09 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Use heat inputs for network port settings https://review.openstack.org/186415 | 21:19 |
| openstackgerrit | imain proposed openstack/tripleo-heat-templates: Containerized TripleO Overcloud. https://review.openstack.org/178840 | 21:20 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Map Mysql to isolated networks https://review.openstack.org/186416 | 21:21 |
| *** rhallisey has quit IRC | 21:23 | |
| *** sdake is now known as steakache | 21:23 | |
| *** steakache is now known as sdake | 21:24 | |
| *** shakamunyi has quit IRC | 21:26 | |
| *** openstack has joined #tripleo | 21:32 | |
| -cameron.freenode.net- [freenode-info] channel trolls and no channel staff around to help? please check with freenode support: http://freenode.net/faq.shtml#gettinghelp | 21:32 | |
| *** openstackstatus has joined #tripleo | 21:32 | |
| *** ChanServ sets mode: +v openstackstatus | 21:32 | |
| *** Goneri has joined #tripleo | 21:33 | |
| *** stendulker has quit IRC | 22:03 | |
| *** Goneri has quit IRC | 22:03 | |
| *** jayg is now known as jayg|g0n3 | 22:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tuskar: Updated from global requirements https://review.openstack.org/186939 | 22:28 |
| *** cody-somerville has joined #tripleo | 22:32 | |
| *** cody-somerville has quit IRC | 22:36 | |
| *** sdake_ has joined #tripleo | 22:39 | |
| *** sdake has quit IRC | 22:43 | |
| *** sdake has joined #tripleo | 22:49 | |
| *** sdake has quit IRC | 22:49 | |
| *** sdake has joined #tripleo | 22:49 | |
| *** sdake_ has quit IRC | 22:52 | |
| *** radek__ has quit IRC | 22:54 | |
| *** penick has joined #tripleo | 22:57 | |
| *** penick has quit IRC | 22:58 | |
| *** Goneri has joined #tripleo | 23:09 | |
| *** Marga_ has joined #tripleo | 23:10 | |
| *** Marga_ has quit IRC | 23:16 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-incubator: Document how to use network isolation/static IPs https://review.openstack.org/186954 | 23:16 |
| *** penick has joined #tripleo | 23:24 | |
| *** Goneri has quit IRC | 23:29 | |
| *** penick has quit IRC | 23:31 | |
| *** spzala has quit IRC | 23:50 | |
| *** AzherKhan has joined #tripleo | 23:54 | |
| *** alop has quit IRC | 23:55 | |
| *** sdake_ has joined #tripleo | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!