Monday, 2016-06-13

*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177800:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]00:10
*** xinwu has quit IRC00:12
*** rcernin has quit IRC00:13
*** tzumainn has joined #tripleo00:27
*** chlong has joined #tripleo00:36
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177801:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]01:10
*** tzumainn has quit IRC01:38
*** jefrite has quit IRC01:43
*** jefrite has joined #tripleo01:50
*** dixiaoli has joined #tripleo02:01
*** rhallisey has quit IRC02:06
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177802:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]02:10
*** r-mibu has quit IRC02:12
*** r-mibu has joined #tripleo02:17
*** ayoung has quit IRC02:48
*** ramishra has joined #tripleo03:06
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177803:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]03:10
*** coolsvap has joined #tripleo03:20
*** xinwu has joined #tripleo03:25
*** xinwu has quit IRC03:25
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177804:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]04:10
*** links has joined #tripleo04:27
*** masco has joined #tripleo04:46
*** liverpooler has joined #tripleo05:05
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177805:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]05:10
*** ooolpbot has quit IRC05:10
*** dixiaoli has quit IRC05:10
*** jaosorior has joined #tripleo05:11
*** numans has joined #tripleo05:15
openstackgerritNuman Siddique proposed openstack/tripleo-puppet-elements: FOR TESTING ONLY... PLZ DONT MERGE  https://review.openstack.org/32883905:15
*** skramaja has joined #tripleo05:26
*** saneax_AFK is now known as saneax05:32
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Set debugging flag to testenv-client script  https://review.openstack.org/32884305:32
*** saneax is now known as saneax_AFK05:32
*** florianf has joined #tripleo05:38
*** dbecker has joined #tripleo05:42
*** xinwu has joined #tripleo05:44
*** bandini has joined #tripleo05:45
*** saneax_AFK is now known as saneax05:48
*** chlong has quit IRC06:03
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177806:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]06:10
*** rcernin has joined #tripleo06:13
*** chlong has joined #tripleo06:15
*** ccamacho has joined #tripleo06:16
*** rcernin has quit IRC06:18
*** rcernin has joined #tripleo06:19
*** xinwu has quit IRC06:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for keystone  https://review.openstack.org/32702906:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for heat  https://review.openstack.org/32706906:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for glance API and registry  https://review.openstack.org/32747306:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for RabbitMQ  https://review.openstack.org/32748206:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for cinder-api  https://review.openstack.org/32885906:22
*** dsariel has joined #tripleo06:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for glance API and registry  https://review.openstack.org/32747306:36
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for RabbitMQ  https://review.openstack.org/32748206:36
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for cinder-api  https://review.openstack.org/32885906:36
*** xinwu has joined #tripleo06:38
*** akrivoka has joined #tripleo06:39
*** itamarl has joined #tripleo06:40
ccamachojaosorior, hey man!!! Good morning!!!! Dude quick question, from one of your comments for the MongoDB submission, (https://review.openstack.org/#/c/304780/24/manifests/profile/base/database/mongodbcommon.pp) I tried to add that block as a resource "define instead a class" but when scoping the variables from here "https://review.openstack.org/#/c/304780/26/manifests/profile/base/database/mongodb.pp" im getting them all empty...06:42
jaosoriorthat's strange06:43
jaosoriorgotta research a bit then06:44
jaosoriorccamcaho: Well, it doesn't need to be a define. it can be a class, as long as it's passed without parameters it should be fine06:45
jaosoriorif the define didn't work06:45
*** tremble has joined #tripleo06:46
*** tremble has joined #tripleo06:46
*** bvandenh has joined #tripleo06:47
*** jpena|off is now known as jpena06:51
ccamachojaosorior, yeahp is called without parameters as they are defaulted06:53
*** anshul has joined #tripleo06:54
*** anshul is now known as Guest1372006:54
jaosoriorfair enough06:55
jaosoriorwell, I guess the only thing to address is the comment by marios06:55
jaosoriorif those sections that create the replset string can be removed from the overcloud_controller.pp and overcloud_controller_pacemaker.pp06:55
*** george_goh has quit IRC06:57
ccamachojaosorior, from the comments, it should be deleted after, right, when landing the composable ceilometer?06:58
*** george_goh has joined #tripleo06:58
jaosoriormaaaybe, not sure if we could actually remove it now, and just use those resources in the same manner as we do in the puppet-tripleo patch06:58
*** Tobias_ has joined #tripleo06:59
Tobias_Hi,06:59
Tobias_is this guide uptodate?06:59
Tobias_http://docs.openstack.org/developer/tripleo-docs/environments/environments.html06:59
*** zephcom has joined #tripleo06:59
Tobias_zephcom:06:59
jaosoriorTobias_ I'm not sure, but this one should be http://tripleo.org/index.html07:00
ccamachoYeahp, but in that case we should land the puppet-tripleo submission first to be able to use it in the ceilometer one07:00
ccamachojaosorior, I think that is a really good idea as we will be sure that the code is removed07:00
jaosoriorccamacho: Not use it in the ceilometer patch. But use it in the ceilometer parts in t-h-t07:00
*** tesseract has joined #tripleo07:00
jaosoriorthat, or make the ceilo patch depend on this07:01
Tobias_jaosorior: Great thanks!07:01
ccamachojaosorior, agreed, I will make the changes then Ill check the ceilo patch07:01
jaosoriorccamacho: Awesome07:02
jaosoriorbandini: Online yet?07:08
bandinijaosorior: yessir. que onda?07:08
*** jprovazn has joined #tripleo07:08
jaosoriorbandini: How's it going dude?07:08
jaosoriorHey, probably this is a dumb question07:08
bandinijaosorior: as good as it can be for a Monday ;) yourself?07:09
bandinijaosorior: shoot07:09
jaosoriorbut in a pacemaker-managed overcloud, I need to reload a specific service's configuration in some event07:09
jaosoriorI guess I can't do it via systemctl, right?07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177807:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]07:10
jaosoriorIf I can't do systemctl reload; what do I use for that?07:10
bandiniso if you do it via systemctl the following can happen: pacemaker monitors the state *exactly* when it is stopped and will mark it as failed. and at that point it will actually try and stop it and then start it again07:10
*** paramite has joined #tripleo07:10
*** ebarrera has joined #tripleo07:11
bandinijaosorior: does your service support the "reload" command?07:11
jaosoriorbandini: Well, what depends on the openstack service; I guess07:11
jaosoriorat least for keystone (and other services under httpd) it shouldn't be a problem07:12
bandiniyeah07:12
jaosoriornova uses oslo.service. So it should support sighup for that07:12
jaosoriorbut I haven't done proper research for other services07:12
bandiniit is a very tricky problem07:14
jaosoriorbandini: In the worst case I gotta restart the service07:15
bandinialso because if you decide to restart the service via pcs, you a) need to do it on one node only and b) it will restart all the dependent services07:15
*** fzdarsky has joined #tripleo07:15
bandinithat is another bonus point of the next ha arch07:15
jaosoriorbandini: Any idea when that will come?07:15
*** chlong has quit IRC07:15
cmysterwhen its ready07:16
bandinijaosorior: so, right now I am 100% tasked on L->M upgrades, as soon as I unblock that I will work on it. plan is still newton07:16
jaosoriorwell07:19
jaosoriorcrap07:19
jaosoriornova-api doesn't support reload apparently07:19
*** hewbrocca-afk has joined #tripleo07:19
jaosoriordocumentation was bogus07:19
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Ceilometer composable roles  https://review.openstack.org/32411707:19
*** hewbrocca-afk is now known as hewbrocca07:19
hewbroccaMorning folks!07:20
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add ceilometer profiles  https://review.openstack.org/30514307:20
jaosoriormorning07:20
ccamachoo/07:22
coolsvapo/07:22
Tobias_a general question, if we deploy one controllernode with TripleO, is it possible to add additional controllers afterwards? this is though only for lab for now. We are going to deploy a full HA-environment this autumn.07:24
Tobias_all this with tripleO ofc.07:24
*** athomas has joined #tripleo07:25
*** zoli_gone-proxy is now known as zoliXXL07:25
Tobias_Guess no1 wants to answer my question. Ah well.07:27
Tobias_We'll try and redeploy if we do not get it to work.07:27
*** zoliXXL is now known as zoli|wfh07:29
matbubandini: morning07:29
jaosorior Tobias_: theoretically it should be possible to scale it. However, I have not done that (I work with other stuff). One guy I know has worked on that is not online.07:32
zoli|wfhgood morning07:32
Tobias_jaosorior: i understand. i know that redhat does not recommend to do that though. but as this is a lab, it would be cool to actually try it.07:33
jaosoriorTobias_: on the other hand, this is IRC. People sometimes take a while to answer; So patience is good here :) people usually are working and/or in several other channels07:34
jaosoriorTobias_ I can try asking around once the dude I'm talking about goes online07:34
Tobias_jaosorior: what we do want to achieve is basically get a 2controllernode setup and one compute, which is also not recommended07:34
jaosoriorTobias_: HA won't work there IIRC. you need 3 controllers there07:35
Tobias_jaosorior: ah okey, then we probably need to start up wit h1 controller and then when we get the hardware for 3 controllers, (missing raidcard in one of the machines, old vmware-servers) we'll just redeploy i guess :P07:36
jaosoriorTobias_: That sounds like a reasonable approach07:36
Tobias_jaosorior: quiscing and stuff i guess is the problem when not using 3 controllers.07:36
*** xinwu has quit IRC07:37
*** jaosorior is now known as jaosorior_brb07:38
*** mikelk has joined #tripleo07:38
*** Tobias_ is now known as Tobias_Fiberdata07:38
*** mikelk has quit IRC07:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for keystone  https://review.openstack.org/32702907:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for heat  https://review.openstack.org/32706907:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for glance API and registry  https://review.openstack.org/32747307:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for RabbitMQ  https://review.openstack.org/32748207:42
*** apetrich has quit IRC07:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for cinder-api  https://review.openstack.org/32885907:42
*** dbecker has quit IRC07:43
*** dtantsur|afk is now known as dtantsur07:43
*** bandini has quit IRC07:44
*** mikelk has joined #tripleo07:47
*** mcornea has joined #tripleo07:49
*** bandini has joined #tripleo07:50
bandinimatbu: bon jour ;)07:52
ccamachojaosorior, man the mongo conn string is used in ceilo, then reused in aodh (havent seen any aodh submission as they are not part of the ceilo one)... https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_controller_pacemaker.pp#L574  mmm for the sake of simplicity.. I think we should not remove that block yet..07:52
ccamachobandini morning!07:52
*** pino|work has joined #tripleo07:52
bandinihola ccamacho07:52
ccamachoEmpezando bien el Lunes?!? ^07:53
*** hjensas has joined #tripleo07:53
matbubandini: bonjour comment vas tu  ?07:54
bandinimatbu: am alright thanks ;) still working on upgrades07:55
bandinimatbu: yourself?07:55
matbubandini: yep mee too07:55
matbubandini: the step 1 passed successfully07:55
matbubandini: but i don't understand why heat is not able to execute the step_2.sh07:56
matbubandini: the cluster is still down and heat wait forever07:56
bandinimatbu: I can explain07:56
matbubandini: wow cool :)07:56
bandinimatbu: so the upgrade in tripleo upstream needs three patches:07:56
bandini1) aodh -> https://review.openstack.org/#/c/320150/19 (almost done, there is one issue still left but should not be anything major)07:57
bandini2) keystone -> https://review.openstack.org/#/c/302235/7 (still needs work, port listening is wrong for some reason, will investigate today)07:57
bandini3) mariadb -> https://review.openstack.org/#/c/325205/ (this kind of works but is still very WIP)07:58
bandiniwithout a working 2) the cluster will have all kinds of problems starting up (it even has problems shutting down for me)07:58
dciabrinmatbu, hey hi, i'm trying to assist bandini for #3 btw07:58
matbubandini: k but, on my deployment i can start the cluster manually, but not with heat07:59
matbudciabrin: k cool07:59
bandinimatbu: what errors do you get with heat? I would imagine that without 2) it cannot possibly work08:00
openstackgerritmathieu bultel proposed openstack/tripleo-heat-templates: WIP Fix mariadb upgrade and add sanity checks for pcs  https://review.openstack.org/32795908:00
matbubandini: i will try the #1 and #208:00
matbubandini: i got no error, heat just hang on step_208:00
matbubandini: like if heat wasn't able to trigger the bash script08:01
matbubandini: btw i used that for the issue #3 (mariadb) https://review.openstack.org/32795908:01
bandinimatbu: so the cluster is not even partially started?08:01
* bandini looks08:02
matbubandini: not started at all08:02
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Gnocchi composable roles  https://review.openstack.org/31841308:02
matbubandini: it's a dirty WIP review :D but the steps works fine on my env08:02
bandinimatbu: I am genuinely surprised those steps work :)08:02
*** cllewellyn_ has joined #tripleo08:02
matbubandini: hehe, but i don't care about the backup & cie08:03
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add gnocchi profiles  https://review.openstack.org/31552708:03
matbubandini: the sed line is mandatory for galera to start08:03
*** ifarkas has joined #tripleo08:03
bandinimatbu: yes those tweaks are needed for galera indeed. I mean I guess it is okay as a quick hack if it works ;)08:04
*** electrofelix has joined #tripleo08:06
*** hjensas_ has joined #tripleo08:06
*** cllewellyn_ has quit IRC08:07
matbubandini: and so the review #2 is in WIP ?08:08
matbubandini: i'm deploying a new env, i'll test the #1 and #208:08
bandinimatbu: it should work but it doesn't08:08
bandinimatbu: basically the vhosts and the ports have a wrong IP address for the admin part of keystone08:09
bandinineed to find out why08:09
*** hjensas has quit IRC08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177808:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]08:10
*** devvesa has joined #tripleo08:12
*** jtomasek_ has joined #tripleo08:13
*** ohamada has joined #tripleo08:14
matbubandini: k, keep me in touch08:16
matbuplz08:16
bandinimatbu: will do absolutely. once I unblock the keystone stuff I will send an update to everyone involved08:16
bandinijistr: need another of your l33t puppet moves on https://review.openstack.org/#/c/320150/ ;)08:17
*** shardy has joined #tripleo08:18
*** dbecker has joined #tripleo08:18
*** jtomasek_ has quit IRC08:19
*** cllewellyn_ has joined #tripleo08:19
*** milan has joined #tripleo08:21
*** shardy has quit IRC08:21
jistrbandini: did the httpd config work as expected now? (not overriden?)08:21
* jistr looks at the patch08:21
bandinijistr: it did, the only issue is that the ports.conf file gets overwritten (which is fine for the aodh step, but when I run the keystone step it will remove the aodh ports)08:22
*** cllewellyn_ has quit IRC08:24
*** aufi has joined #tripleo08:26
*** oshvartz has joined #tripleo08:29
*** chem has joined #tripleo08:30
jistrbandini: would it work if we just leave the ports.conf alone? We probably need to write something in, don't we?08:34
jistre.g. the aodh migration should add the aodh port to listen at, and the WSGI should add the WSGI port?08:34
dtantsurhey folks! does anyone has an easy explanation what these "steps" in puppet-tripleo profiles are?08:35
dtantsure.g. what's the difference between steps 3 and 4?08:35
openstackgerritDmitry Tantsur proposed openstack/puppet-tripleo: Add base ironic profiles  https://review.openstack.org/31929708:37
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: [WIP] Basic support for deploying Ironic in overcloud  https://review.openstack.org/31612808:37
bandinijistr: yeah we need to add ports in there. If we can't find a clean solution, I thought of forcibly adding all the keystone and aodh ports in the keystone step08:37
*** derekh has joined #tripleo08:38
*** jaosorior_brb has quit IRC08:39
*** jaosorior_brb has joined #tripleo08:39
openstackgerritDmitry Tantsur proposed openstack/puppet-tripleo: Add base ironic profiles  https://review.openstack.org/31929708:39
jistrbandini: yea i'm afraid we might have to do it. This time there's probably not an easy way around. The concat resource works by "put together all fragments defined during a puppet run and write them into the file". We could probably disable the writing altogether, but if we need to write a fragment, we need to define all the other fragments too, if we don't want to lose that content. It looks like it's all08:41
jistror nothing.08:41
*** apetrich has joined #tripleo08:41
bandinijistr: that was my fear as well, oh well. I will try that and report back08:42
bandinijistr: thanks for confirming!08:42
jistrbandini: another option would be to disable the management of that file from puppet, and tweak the file externally via a single purpose bash script crafted specifically for the migrations we're about to do08:43
jistrboth options are kinda hackish08:43
*** dmk0202 has joined #tripleo08:44
bandinijistr: I will try to force set the ports in the keystone step, if that does not work I will ping you to try this disabling of file you just mentioned08:45
bandinibut yeah both are non-ideal08:45
social__jistr: create array of ports and define to create listen and iterate over the array every time apache class is used08:45
social__jistr: :(08:45
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: [WIP] Basic support for deploying Ironic in overcloud  https://review.openstack.org/31612808:47
jistrsocial__: our problem is that we're applying partial manifest, because if we apply everything, things will break. So we don't have the complete list of the 'listens' we need.08:47
jistrwe could hard code a bunch of calls to this https://github.com/puppetlabs/puppetlabs-apache/blob/master/manifests/listen.pp08:50
*** dmacpher has joined #tripleo08:51
jistrbut that needs the IPs and ports08:51
jistrbandini: so maybe we could just pull in ceilometer::wsgi::apache08:51
jistrhttps://github.com/openstack/puppet-ceilometer/blob/master/manifests/wsgi/apache.pp08:51
jistrand hopefully all would get fed from Hiera08:51
*** fragatin_ has quit IRC08:52
bandinijistr: will try, this might be actually the cleanest way if it works08:52
jaosorior_brbccamacho: Well, seems that the mongo patches only need the CI now08:54
*** jaosorior_brb is now known as jaosorior08:54
jaosorioraaaand the CI is broken08:54
jaosorioranybody has an idea what's up?08:54
ccamachojaosorior yeeeeahhh!! :) have no clue about the CI issue "bash: line 1: 16044 Killed bash -xe /opt/stack/new/tripleo-ci/toci_gate_test.sh"08:56
jaosoriorit seems to be an issue with gearman08:56
jaosorioror at least the script that uses it: testenv-client08:56
ccamacho:( no idea man08:56
jaosoriorI tried debugging it https://review.openstack.org/#/c/328843/08:57
jaosoriorbut have no idea where those logs go to08:57
matbubandini: so the keystone part should be run after the aodh step ?08:58
matbu(it's written in the commit message, but i just want to confirm :))08:58
bandinimatbu: yep :)08:58
Tobias_FiberdataWhat is the general opinion on running storage-nodes on the same nodes as compute?08:58
Tobias_Fiberdatajaosorior: do you have any tip?08:59
jaosoriorTobias_Fiberdata: Well, for a non-production environment it should be fine. for production I think storage nodes should be separate09:01
jaosoriorbut that's what the aim of composable services is. To give deployers more freedom of choice for their architecture09:01
jaosoriormcornea: ping09:01
mcorneajaosorior: hello09:01
jaosoriormcornea: You've been testing scaling for the overcloud, right?09:01
mcorneajaosorior: yes, I generally do scale out operations during my testing09:02
jaosoriormcornea: Tobias_Fibertdata would like to know if it's possible for them to start with a one or two-node deployment, and later add more controllers09:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for keystone  https://review.openstack.org/32702909:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for heat  https://review.openstack.org/32706909:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for glance API and registry  https://review.openstack.org/32747309:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for RabbitMQ  https://review.openstack.org/32748209:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for cinder-api  https://review.openstack.org/32885909:03
jaosoriormarios: hey dude can you give a quick look to the commits above? ^^ It's still WIP but that's the approach I'm taking for TLS for the internal netwrok09:03
jaosorior*network09:03
mcorneajaosorior: adding more controllers doesn't work but it works for computes, storage09:04
jaosoriorTobias_Fiberdata ^^09:04
jaosoriormcornea: Thanks dude09:05
*** pino|work has quit IRC09:05
openstackgerritJiri Stransky proposed openstack/tripleo-common: Fix interactive update race condition  https://review.openstack.org/32891509:05
mcorneajaosorior: np, this is the BZ for controllers scale out: https://bugzilla.redhat.com/show_bug.cgi?id=124331209:05
openstackbugzilla.redhat.com bug 1243312 in rhel-osp-director "RFE: Support Scaling controllers" [High,New] - Assigned to hbrock09:05
mariosjaosorior: ack added to queue - i'll probably get to them tomorrow mate09:05
*** zaneb has joined #tripleo09:06
*** zaneb has quit IRC09:06
jaosoriormarios: No worries; still need to sort out some things for that work anyway09:06
*** pino|work has joined #tripleo09:07
*** leanderthal|afk is now known as leanderthal09:08
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177809:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]09:10
openstackgerritMerged openstack/tripleo-validations: Fix requirements  https://review.openstack.org/32749909:12
social__jistr: we pulled in puppet to use it like shellscript :(09:16
* derekh starts looking into bug #159177809:18
openstackbug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed] https://launchpad.net/bugs/159177809:18
jistrsocial__: yea, unfortunately the non-shell-script usage wouldn't work at all in this case, or we'd have to backport it to liberty, which is undesirable from release management perspective09:19
jaosoriorderekh: Any idea where the testenv-client logs go to?09:19
jaosoriorI tried adding the debug flag here https://review.openstack.org/#/c/328843/ to see why it takes more than 20 mins (and eventually times out and gets killed)09:20
social__jistr: yeah but this is hmm unmaneageable fo future09:20
social__*for the09:20
derekhjaosorior: the 20 minutes is because we kill the job if its waiting for a env more then 20 minutes09:20
derekhjaosorior: line 149 http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh#n14909:21
*** olap has joined #tripleo09:21
derekhjaosorior: not sure where the logs would be,09:21
jaosoriorderekh: Yep, I understand why those 20 minutes happen. That's why I tried adding those debug logs... to see if that gave more info on what's going on there09:21
*** jpich has joined #tripleo09:22
derekhjaosorior: but I guess there is a probably a problem with the geard setver, looking now09:22
jaosorioralright09:22
jaosoriorderekh: Is that an internal heard server or is it from the openstack infra09:22
derekhjaosorior: internal09:22
derekhjaosorior: 2 of our compute nodes died on friday, one of them had our geard server on it, bring them back up now, everything should be running again soon09:26
jaosoriorderekh: Thanks dude09:27
*** pcaruana has joined #tripleo09:27
*** Guest13720 has quit IRC09:38
bandinimatbu: I *think* I know why the keystone upgrade is not working. the liberty puppet modules do not have the admin_bind_host parameter and so keystone ends up binding to the wrong network09:38
jistrsocial__: yea. In general Puppet seems to be quite bad at doing complex migrations like what we attempt. Perhaps it could be argued that the Puppet modules should be written to handle such situations better, but the reality is what it is :) I think it would be much safer to do migrations after upgrade, during the last step of the upgrade where we can actually run full puppet manifest on the whole cloud.09:39
jistrUnfortunately that can be done only while still within the deprecation period, which is not the case at least for AODH.09:39
jaosoriorbandini: You could open a bug and do the backport09:39
bandinijistr: I guess a yum update -q -y openstack-puppet-modules before applying the keystone-upgrade manifest is the way to go?09:39
bandinijistr: or the route jaosorior suggests ?09:40
bandinisomehow the backport seems more work and not sure how long it will take to trickle down all the way downstream?09:40
bandinidciabrin: ^09:40
jistrbandini: yea it would be more work... OTOH it's probably the more correct way to do it (we still run liberty RPMs, we should use liberty puppet modules to configure them, otherwise we risk some breakage?)09:41
social__jistr: if you use liberty puppet you should backport to liberty09:42
*** tosky has joined #tripleo09:42
jistrbandini: so just to clarify -- i'm not against updating OPM, merely just bringing up that we may hit a problem when going that way.09:42
bandinijistr: understood. I agree having it in liberty will be cleaner.09:42
bandiniLet me try that route and see how it goes09:43
bandinithis upgrade stuff is like the neverending story :)09:43
jistrcool, thanks09:43
jistryea :))09:43
*** paramite is now known as paramite|afk09:48
*** dsneddon has quit IRC09:48
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: add ssh config to collect-logs  https://review.openstack.org/32829409:49
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient: Don't update whole system on undercloud upgrade  https://review.openstack.org/32717609:49
*** Guest13720 has joined #tripleo09:53
*** numans has quit IRC09:58
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Remove stray ";" from purge env command  https://review.openstack.org/32893309:59
derekhjaosorior: we should be back on business now10:00
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828910:00
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add baremetal workflows  https://review.openstack.org/30020010:04
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add RegisterNodesAction action  https://review.openstack.org/31958710:04
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles  https://review.openstack.org/32603710:08
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Keystone liberty mitaka upgrade step  https://review.openstack.org/30223510:08
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177810:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]10:10
*** athomas has quit IRC10:10
*** athomas has joined #tripleo10:17
*** hjensas__ has joined #tripleo10:20
*** bfournie has joined #tripleo10:21
*** hjensas_ has quit IRC10:23
*** cmyster has quit IRC10:25
jaosoriorderekh: You da man10:30
derekhjaosorior: wait to see if its working first ;-)10:30
*** tosc_fiberdata has joined #tripleo10:31
*** cmyster has joined #tripleo10:33
jaosoriorderekh: Well, some patches have been running for more than 51 minutes. around 30 was the point were it used to fail due to the gearman problem10:36
jaosoriorso yeah, that seems to be fixed :D10:36
jaosoriornow lets see if there's other issues O_O10:36
derekhits a good start10:37
*** mikelk has quit IRC10:41
*** zoli|wfh is now known as zoli|lunch10:45
*** adarazs is now known as adarazs_lunch10:47
*** pkovar has joined #tripleo10:51
*** tosc_fiberdata has quit IRC10:53
*** Guest13720 has quit IRC10:54
*** Tobias_Fiberdata has quit IRC10:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add flags to enable internal TLS  https://review.openstack.org/32821611:01
*** Guest13720 has joined #tripleo11:03
*** mikelk has joined #tripleo11:04
*** Guest13720 has quit IRC11:04
*** anshul has joined #tripleo11:05
*** anshul is now known as Guest5913811:05
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177811:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]11:10
*** olap has quit IRC11:12
*** olap has joined #tripleo11:14
matbubandini: ack (sorry /me was at lunch)11:16
matbubandini: so you are doing the backport to opm ?11:17
bandinimatbu: yes will do, once I make sure which patches are all needed for a successful keystone upgrade11:19
bandinilunchtime for me too now ;)11:19
*** hjensas__ has quit IRC11:22
*** rhallisey has joined #tripleo11:25
*** ccamacho is now known as ccamacho|lunch11:26
*** adarazs_lunch is now known as adarazs11:28
*** tosc_fiberdata has joined #tripleo11:39
*** olap has quit IRC11:40
*** olap has joined #tripleo11:42
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828911:43
EmilienMgood morning11:44
*** jpena is now known as jpena|lunch11:44
EmilienMderekh: hi, I noticed CI broke EOD Friday, ~5pm11:44
EmilienMderekh: ok I know why HA job brea11:46
EmilienMjaosorior: ^11:46
EmilienMwe need https://review.openstack.org/#/c/328446/11:46
jaosoriorEmilienM: CI is working again11:47
jaosoriorI rechecked those commits and will merge them once they passed CIO11:47
jaosorior* CI11:47
EmilienMjaosorior: well, we need https://review.openstack.org/#/c/328933/ and https://review.openstack.org/#/c/328446/ to have CI working again11:48
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart: Allow passing extra arguments to overcloud deploy script  https://review.openstack.org/32897011:48
EmilienMjaosorior: https://review.openstack.org/#/c/328446/ won't pass until https://review.openstack.org/#/c/328933/ is merged11:48
jaosoriorEmilienM https://review.openstack.org/#/c/328933/ is not needed really11:48
EmilienMjaosorior: why?11:49
jaosoriorwhy is it?11:49
sshnaidmmatbu, hi11:50
jaosoriorEmilienM: For instance, this just finished running https://review.openstack.org/#/c/315051/ We see there that the other jobs are passing, while HA is failing11:50
jaosoriorEmilienM: so it's yours and dan's commits that are needed11:50
EmilienMjaosorior: yours? what commit except https://review.openstack.org/#/c/328446/ ?11:51
matbusshnaidm: hi11:51
*** apetrich has quit IRC11:51
*** panda|Zz is now known as panda11:51
jaosoriorEmilienM: Ah, thought this was needed too https://review.openstack.org/#/c/328347/211:51
EmilienMjaosorior: so what did you do for the timeout thing?11:51
jaosoriorEmilienM: I did nothing. derekh rebooted the gearman server11:52
sshnaidmmatbu, regarding https://trello.com/c/oc3eFG3l/442-local-tripleo-ci-liberty-mitaka-major-upgrade-run11:52
EmilienMjaosorior: yeah we'll need https://review.openstack.org/#/c/328347/ but not blocking11:52
jaosoriorok11:52
EmilienMjaosorior: ack for gearman11:52
jaosoriorgood to know11:52
sshnaidmmatbu, can you tell what is the goal there?11:52
EmilienMjaosorior: using Nova { } was a bad idea11:52
derekhEmilienM: the fix for the stray ";" isn't needed to unbreak anything, just removes a error message11:52
jaosoriorlets wait for CI and merge as soon as possible11:52
EmilienMjaosorior: it took me hours to figure it's really not working as we expect11:52
EmilienMderekh: ok, thx11:52
jaosoriorEmilienM: Why is it not working as we expect?11:53
matbusshnaidm: yep, one second11:53
EmilienMjaosorior: I had some issues on the HA job when dropping ::nova from THT11:53
matbusshnaidm: actually i pushed a review for enabling the major upgrade in tripleo-ci https://review.openstack.org/32375011:54
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles  https://review.openstack.org/32603711:54
*** bfournie has quit IRC11:54
jaosoriorwas it an issue with the way the parameters were being set? which is what you changes here https://review.openstack.org/#/c/328347/2 ?11:54
matbushadower: and i wanted to run this tripleo.sh script locally (on my local box) , so the question/goal is how to run it ?:)11:54
*** paramite|afk is now known as paramite11:55
jaosoriorEmilienM ^^11:56
*** weshay has joined #tripleo11:56
EmilienMjaosorior: no, it was something with catalog, the class was not included... really weird bug11:56
jaosoriorwhaaa11:56
jaosoriorEmilienM: Did you figure out a solution?11:57
jaosoriorin case we stumble with something like that again11:58
EmilienMjaosorior: yes, i'll make the puppet-tripleo not passing CI but THT depends on this one that will pass CI11:58
EmilienMand we'll land them together11:58
sshnaidmmatbu, so my help is not required anymore?11:58
matbusshnaidm: well your help is for guiding me on how should I deploy and test the tripleo-ci script on a local machine11:59
*** jdob has joined #tripleo11:59
*** paramite is now known as paramite|afk12:01
*** jayg|g0n3 is now known as jayg12:01
sshnaidmmatbu, I think the better sequence will be: installing undercloud with quickstart, then cloning there tripleo-ci repo, then running there tripleo.sh wih --all or as you need.12:02
matbusshnaidm: on the undercloud ?12:02
sshnaidmmatbu, yes12:02
sshnaidmmatbu, this script is for undercloud12:03
matbusshnaidm: ok thanks i'll try that12:03
sshnaidmmatbu, np, ask if something is going unwell, I also try to make such job now that will run tripleo.sh over undercloud of quickstart on centos ci12:04
*** paramite|afk is now known as paramite12:04
*** zoli|lunch is now known as zoli|wfh12:05
matbusshnaidm: ack thx12:05
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828912:05
*** pradk has joined #tripleo12:06
*** MaxPC has joined #tripleo12:10
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/159177812:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1591778 in tripleo "Broken CI: toci_gate_test.sh failed - exit code: 137" [Critical,Confirmed]12:10
*** liverpooler has quit IRC12:11
EmilienMderekh, jaosorior: can we stop the alert now? ^12:15
EmilienMI closed the bug as CI should be back12:16
jaosoriorEmilienM: Hey dude, do you have any idea where these two facts are being used? https://github.com/openstack/puppet-tripleo/tree/master/lib/facter12:19
EmilienMjaosorior: did you check THT?12:20
jaosoriorI grepped tht, yeah12:20
*** Guest59138 has quit IRC12:20
EmilienMgalera_bootstrapped can be dropped I think12:21
EmilienMthough I think we still need https://github.com/openstack/puppet-tripleo/blob/master/lib/facter/netmask_ipv6.rb for ipv612:22
EmilienMnot sure, maybe chem knows ^12:22
*** fultonj has joined #tripleo12:22
jaosoriorI would have thought so too, but I didn't see any usage of it12:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Drop galera_bootstrapped fact  https://review.openstack.org/32897912:22
jaosoriorEmilienM: Thing is, I intend to add a new fact, and was checking how those were being used... and to my surprise I didn't find their usage12:23
chemEmilienM: jaosorior looking12:24
*** hjensas__ has joined #tripleo12:24
*** cllewellyn_ has joined #tripleo12:24
*** coolsvap has quit IRC12:25
chemEmilienM: jaosorior for the fix I've done is now included upstream facture 2.4.7 which is unrealesed at the moment12:25
chemEmilienM: jaosorior for the entire fact, it's necessary for haproxy configuration to get the ipv6 address associated with the interface as the netmask6_* facts is not available for facter < 312:27
*** ramishra has quit IRC12:27
EmilienMah yeah now I remember12:27
jaosoriorI see12:28
*** ramishra has joined #tripleo12:29
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828912:29
*** morazi has joined #tripleo12:30
*** jcoufal has joined #tripleo12:32
*** apetrich has joined #tripleo12:32
*** Guest59138 has joined #tripleo12:32
openstackgerritMerged openstack/tripleo-heat-templates: Update .sh references from openstack-keystone to openstack-core  https://review.openstack.org/32663112:34
*** egafford has quit IRC12:36
*** pkovar has quit IRC12:36
*** pkovar has joined #tripleo12:41
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: ensure polkit is installed  https://review.openstack.org/32898412:41
*** julim has joined #tripleo12:42
*** tzumainn has joined #tripleo12:43
*** jpena|lunch is now known as jpena12:44
*** ccamacho|lunch is now known as ccamacho12:50
*** rbrady has joined #tripleo12:56
*** chlong has joined #tripleo12:57
*** cllewellyn_ has quit IRC12:58
EmilienMjaosorior: good news, looking at zuul, https://review.openstack.org/#/c/328446/ is passing HA job12:58
EmilienMI'll land it once it pass upgrade job too12:58
jaosoriornice!12:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828913:01
*** cllewellyn_ has joined #tripleo13:02
*** cllewellyn_ has quit IRC13:03
*** cllewellyn__ has joined #tripleo13:04
*** pradk has quit IRC13:04
*** cllewellyn__ has quit IRC13:05
*** cllewellyn__ has joined #tripleo13:05
*** cllewellyn_ has joined #tripleo13:06
*** cllewellyn__ has quit IRC13:06
*** ayoung has joined #tripleo13:09
jaosoriorccamacho: HA is broken13:10
jaosoriorand is being fixed by https://review.openstack.org/#/c/328446/13:11
EmilienMI'm waiting for gate-tripleo-ci-centos-7-upgrades job to land it13:11
ccamachoI though was already landed that fix, sorry for rechecking13:11
*** noslzzp has joined #tripleo13:12
jaosoriorccamacho: No worries13:12
jaosoriorEmilienM: gave a +A13:13
*** noslzzp has quit IRC13:13
*** noslzzp has joined #tripleo13:14
*** jcoufal has quit IRC13:14
EmilienMjaosorior: excellent!13:15
EmilienMour CI is back on stage13:15
ccamachonice!13:15
openstackgerritYuriy Zveryanskyy proposed openstack/diskimage-builder: [EXPERIMENTAL] Ansible deploy ramdisk for Ironic  https://review.openstack.org/23944713:16
matbubandini: so #1 looks good, i'm testing now your latest patchset ( 9 ) on the keystone review13:16
bandinimatbu: ack. dciabrin and I have found another bug with httpd restarting, investigating13:17
*** pblaho has joined #tripleo13:19
*** masco has quit IRC13:19
*** sshnaidm has quit IRC13:21
jaosoriorEmilienM: I uploaded a patch to add the new reference for puppet-certmonger https://review.openstack.org/#/c/327654/ is that the way to do it? Do we have to wait for the new puppet-certmonger to be in RDO before merging that?13:21
*** sshnaidm has joined #tripleo13:21
*** sshnaidm has quit IRC13:21
*** sshnaidm has joined #tripleo13:22
*** oneswig has joined #tripleo13:22
EmilienMjaosorior: I'll look13:23
*** liverpooler has joined #tripleo13:25
EmilienMjaosorior: this git is going to be the stable one?13:25
jaosoriorEmilienM: stable one?13:26
EmilienMjaosorior: git url is going to change later?13:26
EmilienMor is it the final one?13:26
openstackgerritMerged openstack/puppet-tripleo: Pacemaker: only create DB on pacemaker master  https://review.openstack.org/32844613:26
jaosoriorEmilienM: I don't think it will change later. It's the one where the puppet-forge is receiving the code from. And the main one13:27
EmilienMcool13:27
EmilienM+2 then13:27
jaosoriorthanks man13:27
jaosoriorjistr: Are you around?13:28
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles  https://review.openstack.org/30478013:28
jistrjaosorior: yea13:29
*** mcornea has quit IRC13:29
openstackgerritMerged openstack/tripleo-quickstart: add ssh config to collect-logs  https://review.openstack.org/32829413:29
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: profile/base/nova: declare nova class properly  https://review.openstack.org/32834713:29
jaosoriorjistr: Hey dude, I've been working on enabling internal TLS on the services in the internal network. And this is more or less the pattern that I'm taking https://review.openstack.org/#/c/327029/ can you give it a quick check and tell me what you think? Specially to how I'm adding the postsave command for certmonger13:30
*** lblanchard has joined #tripleo13:30
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove usage of ::nova class in THT  https://review.openstack.org/32598313:30
*** mcornea has joined #tripleo13:30
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Implement Nova API profile  https://review.openstack.org/32593713:32
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Deploy nova::db classes in api profile  https://review.openstack.org/32602713:33
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-api as a composable service  https://review.openstack.org/32602013:33
*** jprovazn has quit IRC13:33
jaosoriorjistr: Currently the postsave command for pacemaker is just set to /bin/true cause I'm yet to figure out how to do that part. Of course, if you have any ideas for that one itw ould be great :D13:33
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-api as a composable service  https://review.openstack.org/32602013:35
*** jcoufal has joined #tripleo13:39
jistrjaosorior: i'm not a certmonger expert but the approach to postsave looks ok to me. Do we have certmonger managing the haproxy key/cert too or not yet?13:40
jaosoriorjistr: I haven't pushed the commit for that yet13:40
jaosoriorjistr: That's gonna be more challenging actually, since I need to somehow figure out a way to make only one of the nodes try the cert resubmit13:40
jaosoriorjistr, bandini: Is there a way to know from within a node if it's the pacemaker master?13:41
*** rodrigods has quit IRC13:43
*** rodrigods has joined #tripleo13:43
jistrjaosorior: if [ "$(hiera -c /etc/puppet/hiera.yaml bootstrap_nodeid)" = "$(facter hostname)" ]13:43
jaosoriorI guess I could rely on that13:44
*** thrash|wknd is now known as thrash13:45
jaosoriorso the pacemaker master node would do the resubmit first, and then once that's done (probably gonna put a sleep there or something of the sort). The other nodes request the new cert13:45
jistrjaosorior: but yea i was wondering about this. Can certmonger help with the sync of the certs/keys? Or do we have to request a new cert from one node and then distribute the key/cert to other nodes via some other means?13:46
*** tosky has quit IRC13:48
jaosoriorjistr: Well, it could help with the sync... if we did a custom CA... so basically one node would be a proxy, and the other nodes would ping it for the new cert13:48
*** egafford has joined #tripleo13:48
*** dtantsur is now known as dtantsur|brb13:48
jaosoriorjistr: though, if we're using FreeIPA, we probably wouldn't need that. One node would request the cert, and the other ones would just fetch the new one13:49
jaosoriorof course, the nodes need to have a proper keytab, and permissions need to be set in FreeIPA so the other nodes can fetch that service's cert13:49
jistrjaosorior: ah right so some other components of FreeIPA would come into play too, gotcha13:49
jaosoriorI still need to get something concrete and formalize that13:50
jaosoriorso for now I'm taking the easy path and doing the individual services in the ndoes13:50
jaosorior*nodes13:50
*** tosky has joined #tripleo13:51
jistrjaosorior: regarding restarting pacemaker services, i think `pcs resource restart <resource name> <node name>` should do it, so e.g. `pcs resource restart httpd overcloud-controller-0`. Didn't try to run it yet tho.13:52
jistrs/pacemaker services/pacemaker-managed services/13:53
jistror maybe the resource name would be `httpd-clone` rather than `httpd`13:53
*** pkovar has quit IRC13:54
*** links has quit IRC13:54
*** mcornea has quit IRC13:55
*** myoung has joined #tripleo13:55
jistrjaosorior: though in httpd case, you could probably just SIGHUP it to pick up the new config without restarting13:55
jaosoriorjistr: Yeah, that should work.13:58
jaosoriorjistr: wouldn't systemctl reload do a SIGHUP?13:58
jaosoriorthat's the reason I used reload there13:58
openstackgerritBoris Kreitchman proposed openstack/puppet-tripleo: Add a Cinder backup and a Cinder backup Ceph profiles.  https://review.openstack.org/31121813:58
openstackgerritBoris Kreitchman proposed openstack/puppet-tripleo: Add a Cinder backup and a Cinder backup Ceph profiles.  https://review.openstack.org/31121813:59
*** pblaho has quit IRC14:01
openstackgerritBoris Kreitchman proposed openstack/puppet-tripleo: Add a Cinder backup and a Cinder backup Ceph profiles.  https://review.openstack.org/31121814:03
*** pblaho has joined #tripleo14:03
openstackgerritMartin André proposed openstack/instack-undercloud: Add tripleo deployment config dir to hiera  https://review.openstack.org/32901614:04
jistrjaosorior: right it probably would. And i *think* that should work even when the service is managed by pacemaker.14:06
jaosoriorjistr: Would it? From what I've seen for several services in pacemaker, the systemctl service file is removed14:07
jistrjaosorior: i think that depends on a particular service, i.e. what is the resource "provider" that pacemaker uses. For openstack services the provider is systemd, and systemd commands do affect the service managed by pacemaker. For other services like Galera etc. there's a specific resource agent, and systemd commands won't affect the service in this case AFAIK.14:09
*** cllewellyn_ has quit IRC14:10
jaosoriorthe plot thickens14:11
*** pkovar has joined #tripleo14:11
*** jprovazn has joined #tripleo14:11
hewbroccajistr: that's right, I believe we noop them for the resource agent ones14:11
*** tosc_fiberdata has quit IRC14:11
*** tobias_fiberdata has joined #tripleo14:11
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: [WIP] Update puppet dependancies before running puppet  https://review.openstack.org/32826414:12
jaosoriorjistr: Is there any way to know which are managed by pacemaker and which aren't?14:12
jaosoriorwell14:12
*** pradk has joined #tripleo14:12
jistrjaosorior: haha yea plot thickens. One can tell which service is which by the way we instantiate it from puppet pacemaker::resource::service is systemd, and pacemaker::resource::ocf is resource agent14:13
jaosoriorhopefully the lightweight HA architecture comes soon14:13
EmilienMjaosorior: can you review again https://review.openstack.org/#/c/325983/ & https://review.openstack.org/#/c/328347/ please?14:13
jistrjaosorior: and if you want to find out if it's managed by pacemaker at all, you could `pcs resource show --full | grep ...`14:13
EmilienMjaosorior: if they pass CI today, I'll land them14:13
jaosoriorEmilienM: On it14:14
EmilienMthx14:14
jaosoriorjistr: well, either way, if I can tell by the code. I can just set the specific command in the code anyway14:14
jaosoriorbut yeah, that stuff is really good to know. Thanks a lot jistr14:14
jistrnp :)14:14
d0ugalrbrady: ping14:15
jaosoriorEmilienM: Why was the Nova { .. } definition not working correctly again?14:15
jaosoriorreferring to this patch https://review.openstack.org/#/c/328347/314:15
* rbrady is summoned from the lamp14:15
rbradyd0ugal: ack14:15
jaosoriorit looks good but I would like to know for future reference14:16
d0ugalrbrady: when running the registration workflow, I get this: http://paste.openstack.org/show/515734/14:16
EmilienMjaosorior: I haven't found all details, but it was failing on https://review.openstack.org/#/c/325983/14:16
EmilienMjaosorior: puppet catalog was not finding ::nova class14:16
d0ugalrbrady: Is that related to dans comment on PS10? https://review.openstack.org/#/c/319587/14:16
*** tosc_fiberdata has joined #tripleo14:17
EmilienMjaosorior: honestly, I haven't found exactly why, didn't spent time on it, to focus on nova services14:17
jaosoriorok14:17
jaosoriorit's the way it goes14:17
jaosoriorhopefully it doesn't happen to us again14:17
jaosorior+2 given14:17
*** tobias_fiberdata has quit IRC14:19
EmilienMjaosorior: thx man14:19
rbradyd0ugal: I don't think the zaqar issue is related to the puppet-glance comment.  I know he mentioned a zaqar issue on friday14:19
d0ugalrbrady: okay, just guessing.14:20
d0ugalrbrady: I'll like to my paste on the review then14:20
*** Goneri has joined #tripleo14:21
*** Goneri has quit IRC14:21
*** tobias_fiberdata has joined #tripleo14:23
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Drop deprecated nova network options  https://review.openstack.org/32709714:25
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Drop broken keystone pki setup  https://review.openstack.org/32717114:25
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Upgrade only necessary packages  https://review.openstack.org/32721514:25
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Move class options to hiera  https://review.openstack.org/32643614:25
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Update puppet dependancies before running puppet  https://review.openstack.org/32826414:25
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Call systemctl-daemon-reload only if necessary  https://review.openstack.org/32722714:25
*** tosc_fiberdata has quit IRC14:26
*** Larion has joined #tripleo14:28
*** ebalduf_ has joined #tripleo14:31
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles  https://review.openstack.org/32603714:31
*** tobias_fiberdata has quit IRC14:31
*** tobias_fiberdata has joined #tripleo14:32
*** oshvartz has quit IRC14:32
bnemecjaosorior: I approved https://review.openstack.org/#/c/327654 but make sure you take a look at my comment there.  I think there's a follow-up needed before it will be usable.14:32
jaosoriorbnemec: Thanks man, will do14:33
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list  https://review.openstack.org/32902514:35
jaosoriorbnemec: Done ^14:35
*** Larion has quit IRC14:35
jaosoriorbnemec: Do you know if the .git is mandatory?14:35
openstackgerritMerged openstack/tripleo-puppet-elements: Add new puppet-certmonger reference  https://review.openstack.org/32765414:36
jaosoriordamn14:36
jaosoriorit is14:36
bnemecReally?14:36
bnemecI never actually add .git when pulling things from github.14:36
*** itamarl has quit IRC14:37
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list  https://review.openstack.org/32902514:37
jaosoriorbenemc: uhm... could be that it just works because of the port that's used. Only thing I did was check what URL did github give me when I selected clone from HTTPS14:37
jaosoriorand it had the .git in the end14:37
ccamachopradk are you around?14:37
bnemecjaosorior: Yeah, it's fine to have for consistency, but I'd be shocked if it actually didn't work without the .git extension.14:38
bnemecjaosorior: It might also be good to leave a reference in the commit message to the t-p-e change too.14:39
jaosoriorbnemec: good idea. Will do14:39
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list  https://review.openstack.org/32902514:40
jaosoriorbnemec: There is a strange issue happening sporadically, where the overcloud deploy fails in around 20 minutes14:41
jaosoriorit's been going on pretty randomly14:41
jaosoriorand at least I haven't figured out why14:41
bnemecjaosorior: I was just looking at a patch where that happened.14:41
jaosoriorit's pretty random :/14:42
bnemecVery strange.14:42
jaosoriorccamacho had a theory ^^14:42
*** mcornea has joined #tripleo14:43
ccamachoyeahp, does the deployment hangs for ever?? (Until killed?) I recorded a screencast how happened to me14:43
ccamachocheck it at the end of https://bugs.launchpad.net/tripleo/+bug/1546749  The vms when booting are not able to "boot" but if you re-execute again the deployment.. They can..14:44
openstackLaunchpad bug 1546749 in tripleo "CI testenv vm stuck in boot loop trying to boot rescue kernel" [Critical,Invalid] - Assigned to James Slagle (james-slagle)14:44
ccamachoNow you can reproduce it in Liberty following the steps in (https://gist.github.com/ccamacho/aa136fd87a23b4898185b8c95081905c), but I wasnt able to locally reproduce it on mitaka or master14:46
*** mgould has joined #tripleo14:47
bnemecThis is a different problem.  The timeout is legitimately being set to 20: http://logs.openstack.org/72/320072/2/check-tripleo/gate-tripleo-ci-centos-7-ha/c544cfb/console.html.gz#_2016-06-10_06_34_25_96163314:47
*** Goneri has joined #tripleo14:47
mgouldhi everyone14:48
bnemecWhich tells me infra is passing us a bogus timeout value because that's calculated based on the gate timeout.14:48
mgouldcould someone please review https://review.openstack.org/#/c/321116/ ? Trivial docs-only patch, already has one +214:48
jaosoriormgould: done14:49
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add gnocchi profiles  https://review.openstack.org/31552714:49
*** ebalduf_ has quit IRC14:50
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add gnocchi profiles  https://review.openstack.org/31552714:50
openstackgerritMerged openstack-infra/tripleo-ci: Use dlrn instead of delorean for delorean-build command  https://review.openstack.org/32582314:50
openstackgerritMartin André proposed openstack/tripleo-common: Allow running validation against different plans  https://review.openstack.org/31819414:54
openstackgerritMartin André proposed openstack/tripleo-common: Validations actions and workbook  https://review.openstack.org/31363214:54
openstackgerritMartin André proposed openstack/tripleo-common: Deployment workflows for remote execution, etc.  https://review.openstack.org/31395314:54
openstackgerritMartin André proposed openstack/tripleo-common: Mistral action to drive Heat software deployments  https://review.openstack.org/31096614:54
openstackgerritMartin André proposed openstack/tripleo-common: Disable retry files for ansible validations  https://review.openstack.org/32903914:54
openstackgerritMerged openstack/python-tripleoclient: Document --initial-state param on baremetal import  https://review.openstack.org/32111614:55
openstackgerritBoris Kreitchman proposed openstack/tripleo-heat-templates: Create Cinder backup pool in Ceph  https://review.openstack.org/32904114:57
mgouldjaosorior: thanks!14:57
mgould\o/14:57
* mgould rechecks the backport to liberty again14:58
*** andrearosa has joined #tripleo14:58
*** julim has quit IRC14:59
*** andrearosa has left #tripleo15:00
*** julim has joined #tripleo15:00
*** dsneddon has joined #tripleo15:02
*** coolsvap has joined #tripleo15:05
*** zoli|wfh is now known as zoli|brb15:07
*** dsariel has quit IRC15:07
*** fzdarsky is now known as fzdarsky|afk15:07
*** cllewellyn_ has joined #tripleo15:07
*** cllewellyn_ has quit IRC15:08
*** cllewellyn_ has joined #tripleo15:08
*** paramite is now known as paramite|afk15:09
*** cllewellyn_ has quit IRC15:10
*** cllewellyn_ has joined #tripleo15:10
*** fragatina has joined #tripleo15:11
*** leanderthal is now known as leanderthal|afk15:13
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Move ironic config to post install  https://review.openstack.org/32830015:13
openstackgerritSven Anderson proposed openstack/tripleo-heat-templates: Enable nova-compute as a composable service  https://review.openstack.org/32836115:17
*** devvesa has quit IRC15:20
*** dtantsur|brb is now known as dtantsur15:21
*** weshay is now known as weshay_lunch_brb15:26
*** paramite|afk is now known as paramite15:29
*** ebarrera has quit IRC15:32
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: profile/base/nova: declare nova class properly  https://review.openstack.org/32834715:33
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove usage of ::nova class in THT  https://review.openstack.org/32598315:34
*** paramite is now known as paramite|afk15:34
EmilienMjaosorior: I had to adjust to how neutron works now ^15:35
*** mikelk has quit IRC15:37
*** zoli|brb is now known as zoli|wfh15:38
jaosoriorEmilienM: Why was the nova declaration moved to step 4 (or step 3 if it's db_sync)15:39
*** tesseract has quit IRC15:39
EmilienMjaosorior: because it was like this in THT15:39
jaosorioris it very problematic to have it in step 3 always?15:40
jaosoriorI think neutron is nowadays always included in step 315:40
EmilienMmhhh15:40
jaosoriornevermind15:41
jaosoriorit isn't15:41
EmilienMthe thing is the catalog failed (ha job)15:41
jaosoriorI just saw15:41
EmilienMit could not lookup Nova::Db15:41
jistrmarios: heya saw your poke re that you need it asap? https://review.openstack.org/#/c/308985/2/doc/source/post_deployment/upgrade.rst -- what did you have in mind?15:41
EmilienMjaosorior: i haven't figured why15:41
jaosoriorlets do it that way then15:41
jaosoriorit seems to be the same for neutron and the other services anyway15:41
EmilienMjaosorior: I'm having hard time with this role to move15:41
jaosoriorit is definitely not a trivial one15:42
EmilienMyeah15:42
mariosjistr: hey man, no emergency, just saying we should land that since we don't have upstream upgrades docs at present15:43
mariosjistr: was going to update it with the comments addressed possibly tomorrow unless you wanted to do so (didn't know how much time you have to spend there)15:43
jaosoriorEmilienM, marios: Do you guys know if we have keystone on httpd in stable/mitaka?15:43
*** pblaho has quit IRC15:43
jaosoriorjistr ^^15:43
EmilienMno15:43
EmilienMon undercloud only AFIK15:44
*** Guest59138 has quit IRC15:44
jaosoriorwell, crap15:44
jaosoriorthought it was there already15:44
EmilienMwait15:44
EmilienMhttps://github.com/openstack/instack-undercloud/blob/stable/mitaka/elements/puppet-stack-config/puppet-stack-config.pp#L13915:44
mariosjaosorior: i'd say we have aodh, but not sure of keystone (am deploying momentarily so i can tell you in a bit) and i thought we had keystone too15:44
*** xinwu has joined #tripleo15:44
mariosbut i think jistr was looking at that with pradk15:45
jistrmarios: if you can update it yourself that would be awesome. We can land it but maybe we should put a comment somewhere that the AODH/Keystone bits are being worked out?15:45
EmilienMhttps://github.com/openstack/tripleo-heat-templates/blob/stable/mitaka/puppet/manifests/overcloud_controller_pacemaker.pp#L118115:45
jistrmarios: or should we document just Mitaka->Newton upgrades at this point...15:45
EmilienMactually yes it's on the overcloud too15:45
*** weshay_lunch_brb is now known as weshay15:45
mariosjistr: so here's the thing. we can make it 'about liberty' for now and iterate. i mean the main bits are the example repo setup.15:45
mariosjistr: OR15:45
jaosoriorexcellent!15:46
pradkjistr, i think aodh upgrade is in a good shape to land?15:46
EmilienMhttps://github.com/openstack/tripleo-heat-templates/blob/stable/mitaka/puppet/hieradata/controller.yaml#L7115:46
mariosjistr: we make it about 'newton' so we have the things bandini points out too (like the mariadb stop&mysqldump before upgrade)15:46
EmilienMjaosorior: ^15:46
jaosoriorEmilienM: Awesome!15:46
jaosoriorso that's good15:46
EmilienMcool you're happy :P15:47
mariosjistr: ok i will try get a pass out tomorrow and we take it from there. mainly my point is whatever it is, it's better and more plentiful than the current upgrades docs :/15:47
mariosjistr: i think the aodh + keystone will be needed for liberty --> mitaka upgrades right? is that somethign bandini has mentioned working on i think?15:49
jistrpradk: +2'd that one. We'll need another core +2.15:49
mariosjistr: so then perhaps his comments are about l-->M and not M-->N anyway i'll try clarify15:49
jaosoriorjistr: Which patch?15:49
mariosjistr: gimme patch and i can look tomorrow reviews time guys too fwiw pradk15:50
jistrmarios: yea i think any keystone wsgi or aodh related comments wrt upgrades would be L->M15:50
jistrjaosorior: https://review.openstack.org/#/c/32015015:50
jistrmarios: thanks!15:50
mariosack thx added to queue after jaosorior request :)15:50
marios(the tls on internal)15:51
*** dmk0202 has quit IRC15:51
pradkthx jistr and marios15:51
marioso/ pradk15:51
dtantsurfolks, could you please merge a trivial docs-only change https://review.openstack.org/#/c/321117/ ?15:51
dtantsurit's sad to see people rechecking tripleo CI on it ;)15:51
*** dtrainor has joined #tripleo15:52
trownwe have versioned docs?15:52
jaosoriorjistr, pradk: So... running that, disables the usual puppet run and only runs this upgrade path? Did I understand correctly?15:53
jistrjaosorior: yes15:54
pradkjaosorior, yea if you run that with -e only aodh will be upgraded and rest is a noop15:54
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Use quickstart.sh to collect logs  https://review.openstack.org/32828915:54
jaosoriorok15:55
*** derekh has quit IRC15:56
*** pblaho has joined #tripleo15:57
*** mcornea has quit IRC15:58
jaosoriorpradk, jistr:  liberty_to_mitaka_aodh_upgrade_1.pp seems to be a subset of liberty_to_mitaka_aodh_upgrade_2.pp. is this intended? Why not just use steps and have one file there?15:58
pradkjaosorior, we needed to split the upgrade process into two steps other wise we ran into case where pacemaker config was running before packages were installed15:59
openstackgerritMerged openstack/python-tripleoclient: Document --initial-state param on baremetal import  https://review.openstack.org/32111715:59
*** rcernin has quit IRC15:59
pradkjaosorior, this ensure everything is in place before pcmk kicked in15:59
jaosorioryep, but why not just use an explicit step parameter coming from hiera like we do in other manifests?15:59
pradkjaosorior, this is a stand alone manifest .. i'm not sure how and where the steps get passed in .. we just took this approach and it worked16:00
jaosoriorwell16:01
jaosoriorother than that it looks well16:01
jaosoriorif jistr is OK with it I could merge it16:01
*** tremble has quit IRC16:02
jaosorioror should I wait for marios to take a look?16:02
* bandini back16:03
jistrjaosorior, pradk: yea we could do steps similarly as we do in the usual puppet config, but i don't feel like we'd necessarily need to block on that (at least that's my 2 cents)16:03
*** xinwu has quit IRC16:03
jistrit's a thing that will never live in master IIUC, just stable/mitaka one-time migration16:03
mariosjaosorior: jistr up to you guys if you think is ready go for it... /me calling it a day momentarily i'll check it in the morning either way16:03
pradkthats correct.. this is mitaka only16:04
*** ayoung has quit IRC16:04
* jistr gotta run too16:04
jaosoriorok, I'll merge it then16:04
jistrack16:04
bandinipradk, jistr, marios, jaosorior: I think the aodh stuff is good to go. I have tested it pretty intensely16:04
jistrjaosorior: thanks16:04
pradkcool, thx guys16:05
*** apetrich has quit IRC16:05
openstackgerritMerged openstack/tripleo-heat-templates: Aodh upgrade heat template and manifests to replace ceilometer alarms  https://review.openstack.org/32015016:07
hewbroccasos16:07
hewbroccaer16:07
hewbroccawow16:07
jaosoriorany +A for this? https://review.openstack.org/#/c/329025/16:10
EmilienMjaosorior: wait for CI or?16:11
jaosoriorah16:11
jaosoriormessed up16:11
jaosoriorthought it had passed CI already16:11
jaosoriorsorry about that16:11
EmilienMjaosorior: no worries, I'll +A once it's green16:11
*** electrofelix has quit IRC16:11
*** cllewellyn_ has quit IRC16:12
dtantsurfolks, do yoiu know why https://github.com/openstack/puppet-ironic/blob/master/manifests/init.pp#L389 does not end up with a package installed? Oo16:12
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Support various tempest configurations  https://review.openstack.org/32908216:12
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles  https://review.openstack.org/32603716:13
dtantsurso I actually see in logs: Notice: /Stage[main]/Ironic/Package[ironic-common]/ensure: created16:15
EmilienMdtantsur: what?16:15
EmilienMshow me logs16:15
dtantsurbut the files are not there16:15
EmilienMit's maybe an orchestration thing16:15
dtantsurEmilienM, it's from http://logs.openstack.org/28/316128/19/check-tripleo/gate-tripleo-ci-centos-7-nonha/ad4d293/logs/overcloud-controller-0/var/log/messages16:15
dtantsurI see the package installed, but then everything else fails as if it was not installed16:15
EmilienMJun 13 13:42:19 localhost os-collect-config: #033[1;31mError: Could not set 'present' on ensure: No such file or directory - /etc/ironic/ironic.conf at 428:/etc/puppet/modules/ironic/manifests/init.pp#033[0m16:15
EmilienMlet me dig the code, 1s16:16
*** rwsu has joined #tripleo16:16
EmilienMdtantsur: wait, ironic-common does not create /etc/ironic and /etc/ironic/ironic.conf?16:17
*** cllewellyn_ has joined #tripleo16:17
dtantsurEmilienM, it's supposed to. at least on my system /etc/ironic/ironic.conf belongs to it16:17
EmilienMI found a bug in init.pp though16:18
EmilienMlet me dig packaging16:18
*** ayoung has joined #tripleo16:19
*** cllewellyn_ has quit IRC16:22
EmilienMso IMHO we should not manage /etc/ironic anymore, we did it in the past to workaround packaging issues16:22
EmilienMbut now, let's drop it.16:22
EmilienMhttps://review.openstack.org/32908716:22
pradkjaosorior, can you review https://review.openstack.org/#/c/326136/ and +A https://review.openstack.org/#/c/327396/16:24
*** yolanda has joined #tripleo16:24
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: [WIP] Basic support for deploying Ironic in overcloud  https://review.openstack.org/31612816:24
dtantsurEmilienM, we'll see ^^^16:24
dtantsurthanks for your help anyway16:24
*** panda has quit IRC16:24
EmilienMdtantsur: excellent, I'll monitor it16:24
EmilienMlet me know otheriwz16:24
EmilienMotherwise*16:24
*** panda has joined #tripleo16:25
panda[apologies for double post] do overcloud nodes usually have to access internet during deploy ?16:26
*** cllewellyn_ has joined #tripleo16:26
dtantsurpanda, well, HA deployment seems to require NTP (might be intranet though). otherwise I'd be surprised16:28
dtantsurbut I'm not the best person to ask :)16:29
pandadtantsur: does the deploy involve installing packages ?16:29
dtantsurpanda, I *assume* we usually have packages preinstalled. maybe not all of them.16:30
*** rcernin has joined #tripleo16:30
dtantsurif some packages are missing, they'll get installed during deployment, yeah16:30
openstackgerritMerged openstack/tripleo-heat-templates: Add coordination url to gnocchi  https://review.openstack.org/32613616:31
bnemecpanda: dtantsur: We avoid installing packages at deploy time.  The intent is for everything to be installed on the image ahead of time.16:32
EmilienMjaosorior: https://review.openstack.org/#/c/328487/116:32
bnemecI'm not sure it's safe to assume overcloud nodes have external connectivity either.16:33
EmilienMplease review this one^16:33
EmilienMbnemec: if you can look also, it's a really quick one16:33
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Keystone liberty mitaka upgrade step  https://review.openstack.org/30223516:33
*** akrivoka has quit IRC16:34
*** rcernin has quit IRC16:35
*** rcernin has joined #tripleo16:35
*** trown is now known as trown|lunch16:35
pandabnemec: so is it normal that I have the dnsmasq that listens to the overcloud ctlplane not configured for dns forwarding ?16:36
*** adarazs is now known as adarazs_afk16:36
*** jpich has quit IRC16:36
*** olap has quit IRC16:37
bnemecpanda: I believe so.  The dnsmasq on the provisioning network is only there to provide DHCP for deployment.16:38
*** pcaruana has quit IRC16:38
*** jaosorior has quit IRC16:39
pandabnemec: understood, thanks.16:39
*** jaosorior has joined #tripleo16:39
*** athomas has quit IRC16:43
*** pino|work has quit IRC16:43
pandaso I'm back to square one ... I have a deploy that hangs and I don't understand which task is blocking the progress :( not a network problem, nothing on overcloud logs, heat-engine log looping over task completion checks. Is there a way to understand what's the heat resource currently in progress ?16:43
openstackgerritMerged openstack/tripleo-heat-templates: neutron: stop include ::neutron & ::neutron::config  https://review.openstack.org/32848716:44
*** tosky has quit IRC16:51
sshnaidmbnemec, hi, what do you mean by duplicate copies in your mail?16:51
*** aufi has quit IRC16:52
*** numans has joined #tripleo16:53
bnemecsshnaidm: You said you were bringing in the latest changes from the RDO scripts.  That makes it sound like there are two copies of this that we're having to keep in sync manually.16:53
*** numans has quit IRC16:53
*** numans has joined #tripleo16:53
sshnaidmbnemec, ok, I mean that we have an exact copy of tempest configurator ("rdo script") in our tripleo-ci repo, but since tempest changed -> tempest configurator in rdo changed -> we need to change it in our repo16:55
sshnaidmbnemec, and this script - yes, we do sync it manually16:55
*** mcornea has joined #tripleo16:56
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING  https://review.openstack.org/31643616:56
*** dobson has quit IRC16:56
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Fix tempest configuration  https://review.openstack.org/32605516:57
bnemecsshnaidm: Right, and that's bad.  Why are we doing it?16:57
sshnaidmbnemec, because of limited choice to configure tempest16:57
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Support various tempest configurations  https://review.openstack.org/32908217:01
*** zoli|wfh is now known as aoli|gone17:01
*** aoli|gone is now known as zoli|gone17:01
*** jaosorior has quit IRC17:02
openstackgerritMerged openstack/tripleo-image-elements: Fix is_local_ip matching  https://review.openstack.org/32651117:02
*** ayoung has quit IRC17:04
*** fragatina has quit IRC17:06
*** fragatina has joined #tripleo17:06
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Don't try to get stack details if the stack doesn't exist  https://review.openstack.org/31933717:07
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: TEST: Delete the overcloud when finished  https://review.openstack.org/29732817:07
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Don't pipe yes to a function  https://review.openstack.org/31773017:07
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test overcloud deletion in periodic job  https://review.openstack.org/29676517:07
*** zoli|gone is now known as zoli_gone-proxy17:08
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add gnocchi profiles  https://review.openstack.org/31552717:09
*** mcornea has quit IRC17:09
*** ifarkas has quit IRC17:09
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add gnocchi profiles  https://review.openstack.org/31552717:11
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker.  https://review.openstack.org/30906917:13
*** yamahata has joined #tripleo17:15
*** adarazs_afk is now known as adarazs17:16
*** dbecker has quit IRC17:17
*** ayoung has joined #tripleo17:17
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: integration of the new puppet pacemaker.  https://review.openstack.org/30240917:21
*** julim has quit IRC17:28
*** yamahata has quit IRC17:28
*** cllewellyn_ has quit IRC17:30
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: use environmental variables for ansible configuration  https://review.openstack.org/32912417:35
*** ohamada has quit IRC17:38
*** pcaruana has joined #tripleo17:39
*** sambetts is now known as sambetts|afk17:42
*** xinwu has joined #tripleo17:46
*** xinwu has quit IRC17:46
*** numans has quit IRC17:46
*** xinwu has joined #tripleo17:51
*** mgould is now known as mgould|afk17:52
*** jpena is now known as jpena|off17:53
*** fragatina has quit IRC17:53
*** yamahata has joined #tripleo17:54
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Change Aodh to use own backend  https://review.openstack.org/32912617:55
*** mbound has joined #tripleo17:55
*** coolsvap has quit IRC17:56
*** coolsvap has joined #tripleo17:56
*** dobson has joined #tripleo17:58
*** fragatina has joined #tripleo18:00
*** saneax is now known as saneax_AFK18:01
*** trown|lunch is now known as trown18:01
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Run collect logs only once for ooo-usb gate jobs  https://review.openstack.org/32913118:01
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Keystone liberty mitaka upgrade step  https://review.openstack.org/30223518:04
*** florianf has quit IRC18:05
*** michchap has quit IRC18:05
*** florianf has joined #tripleo18:05
*** fragatina has quit IRC18:06
*** ebalduf has quit IRC18:06
*** dbecker has joined #tripleo18:07
*** mcornea has joined #tripleo18:07
*** dtantsur is now known as dtantsur|afk18:11
*** michchap has joined #tripleo18:13
*** coolsvap has quit IRC18:13
*** olap has joined #tripleo18:16
*** pkovar has quit IRC18:16
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Keystone liberty mitaka upgrade step  https://review.openstack.org/30223518:17
*** mbound has quit IRC18:19
*** chem` has joined #tripleo18:22
*** mcornea has quit IRC18:22
*** mbound has joined #tripleo18:22
*** chem has quit IRC18:23
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: use environmental variables for ansible configuration  https://review.openstack.org/32912418:24
*** mbound has quit IRC18:25
*** mbound has joined #tripleo18:26
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: [WIP] Initial work to dump and restore galera db during major upgrades  https://review.openstack.org/32520518:37
*** fragatina has joined #tripleo18:39
*** cdearborn has joined #tripleo18:45
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: use environmental variables for ansible ssh configuration  https://review.openstack.org/32912418:56
*** jcoufal_ has joined #tripleo18:56
*** apetrich has joined #tripleo18:56
*** ibravo has joined #tripleo18:56
*** TSCHAK_ has quit IRC18:58
*** jcoufal has quit IRC18:59
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job  https://review.openstack.org/22978919:09
*** xinwu has quit IRC19:13
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Implement Nova Scheduler profile  https://review.openstack.org/32609219:22
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Implement Nova Consoleauth profile  https://review.openstack.org/32611219:22
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Implement Nova vncproxy profile  https://review.openstack.org/32613019:22
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-api as a composable service  https://review.openstack.org/32602019:23
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove nova::db classes, moved to puppet-tripleo  https://review.openstack.org/32602819:30
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-scheduler as a composable service  https://review.openstack.org/32609619:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-consoleauth as a composable service  https://review.openstack.org/32611819:33
*** xinwu has joined #tripleo19:33
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-vncproxy as a composable service  https://review.openstack.org/32613219:34
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-compute as a composable service  https://review.openstack.org/32836119:34
*** cllewellyn_ has joined #tripleo19:35
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: make the config dir outside of src. virt_env/config  https://review.openstack.org/32728819:37
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Implement Nova Compute profile  https://review.openstack.org/32723619:37
*** cllewellyn_ has quit IRC19:40
openstackgerritJeff Peeler proposed openstack/tripleo-common: Add client caching for Mistral actions  https://review.openstack.org/32916319:40
*** paramite|afk is now known as paramite19:43
*** florianf has quit IRC19:50
*** fzdarsky|afk has quit IRC19:55
*** jefrite has quit IRC19:59
*** dbecker has quit IRC20:07
*** jcoufal_ has quit IRC20:09
*** fzdarsky|afk has joined #tripleo20:11
*** chlong has quit IRC20:12
*** jprovazn has quit IRC20:12
*** paramite has quit IRC20:13
*** qasims has joined #tripleo20:15
*** xinwu has quit IRC20:15
*** qasims has quit IRC20:18
*** panda has quit IRC20:24
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Deploy and Upgrade Ironic to run in mod_wsgi  https://review.openstack.org/30914020:24
*** panda has joined #tripleo20:25
*** jayg is now known as jayg|g0n320:27
openstackgerritJeff Peeler proposed openstack/tripleo-common: Add client caching for Mistral actions  https://review.openstack.org/32916320:40
*** cllewellyn_ has joined #tripleo20:44
*** cllewellyn_ has quit IRC20:50
*** julim has joined #tripleo20:50
*** thrash is now known as thrash|g0ne20:53
*** Goneri has quit IRC20:58
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add undercloud idempotency test  https://review.openstack.org/27921821:00
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove usage of ::nova class in THT  https://review.openstack.org/32598321:04
*** fzdarsky|afk has quit IRC21:04
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-api as a composable service  https://review.openstack.org/32602021:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove nova::db classes, moved to puppet-tripleo  https://review.openstack.org/32602821:06
*** trown is now known as trown|outtypewww21:06
*** lblanchard has quit IRC21:12
*** noslzzp_ has joined #tripleo21:25
*** noslzzp has quit IRC21:27
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add undercloud idempotency test  https://review.openstack.org/27921821:33
*** julim has quit IRC21:33
bnemecNow with 100% less leftover merge conflicts21:33
*** rhallisey has quit IRC21:39
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer composable roles  https://review.openstack.org/32411721:42
*** dmk0202 has joined #tripleo21:47
*** rcernin has quit IRC21:48
*** cdearborn has quit IRC21:51
*** fzdarsky|afk has joined #tripleo21:52
*** MaxPC has quit IRC22:01
openstackgerritDan Sneddon proposed openstack/tripleo-specs: Specification for tripleo-lldp-validation blueprint  https://review.openstack.org/32920322:07
*** fzdarsky|afk has quit IRC22:13
*** rhallisey has joined #tripleo22:14
*** chlong has joined #tripleo22:20
*** saneax_AFK is now known as saneax22:21
openstackgerritDan Sneddon proposed openstack/tripleo-specs: Specification for tripleo-lldp-validation blueprint  https://review.openstack.org/32920322:23
*** ayoung has quit IRC22:27
*** dmacpher has quit IRC22:28
*** akshai has joined #tripleo22:36
*** saneax is now known as saneax_AFK22:40
*** dmacpher has joined #tripleo22:41
*** akshai has quit IRC22:45
*** dmk0202 has quit IRC22:52
*** egafford has quit IRC23:04
*** saneax_AFK is now known as saneax23:06
*** pradk has quit IRC23:07
*** yamahata has quit IRC23:11
*** akshai has joined #tripleo23:13
*** TSCHAK has joined #tripleo23:14
*** dsariel has joined #tripleo23:14
*** jonbcraig has joined #tripleo23:21
*** devananda has joined #tripleo23:21
*** mbound has quit IRC23:23
*** akshai_ has joined #tripleo23:24
*** akshai has quit IRC23:24
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: debug - do not merge  https://review.openstack.org/32922423:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: debug - do not merge  https://review.openstack.org/32922523:32
jonbcraigI'm having trouble introspecting some ibm 3650's.  It seems the latest centos7 generic-cloud image is missing the bnx2 firmware files.  Anyone have suggestions?23:32
*** ayoung has joined #tripleo23:33
jonbcraigAnyone ... Anyone ... Bueller?23:35
*** jonbcraig has left #tripleo23:38
*** akshai_ has quit IRC23:42
*** yamahata has joined #tripleo23:51
*** weshay has quit IRC23:54
*** numans has joined #tripleo23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!