Tuesday, 2017-04-25

« Monday, 2017-04-24 Index Wednesday, 2017-04-26 »
*** milan has quit IRC00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025900:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427200:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168541800:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581400:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]00:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]00:10
openstackLaunchpad bug 1685418 in tripleo "upgrade jobs failing with openstack-heat-templates vs openstack-heat-agents dependencies" [Critical,In progress] - Assigned to Alfredo Moralejo (amoralej)00:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)00:10
openstackgerritMerged openstack/tripleo-common master: Add etcd image to overcloud containers  https://review.openstack.org/44588300:18
*** bfournie has joined #tripleo00:31
*** bfournie has quit IRC00:32
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add DeploymentSwiftDataMap parameter  https://review.openstack.org/45944000:32
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add DeploymentSwiftDataMap parameter  https://review.openstack.org/45944000:34
*** ansmith has quit IRC00:49
*** dixiaoli has joined #tripleo01:00
*** limao has joined #tripleo01:04
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025901:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427201:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168541801:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581401:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]01:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]01:10
openstackLaunchpad bug 1685418 in tripleo "upgrade jobs failing with openstack-heat-templates vs openstack-heat-agents dependencies" [Critical,In progress] - Assigned to Alfredo Moralejo (amoralej)01:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)01:10
openstackgerritMerged openstack/puppet-tripleo master: Dell SC: Add secondary DSM support  https://review.openstack.org/45538601:58
*** dixiaoli_ has joined #tripleo02:01
*** brault has joined #tripleo02:04
*** dixiaoli has quit IRC02:05
*** japestinho has joined #tripleo02:07
*** brault has quit IRC02:08
*** bkopilov has quit IRC02:09
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025902:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427202:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581402:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]02:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]02:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)02:10
*** yamahata_ has quit IRC02:24
*** thrash is now known as thrash|g0ne02:35
*** fragatin_ has joined #tripleo02:44
*** fragatina has quit IRC02:48
*** fragatin_ has quit IRC02:49
*** rlandy has quit IRC03:04
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025903:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427203:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581403:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]03:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]03:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)03:10
*** dixiaoli_ has quit IRC03:13
*** dixiaoli has joined #tripleo03:15
*** psahoo has joined #tripleo03:30
*** bkopilov has joined #tripleo03:37
*** jaganathan has joined #tripleo03:43
*** morazi has joined #tripleo03:54
*** ykarel has joined #tripleo04:01
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025904:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427204:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581404:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]04:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]04:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)04:10
openstackgerritMerged openstack/instack-undercloud stable/ocata: Update Gemfile to pull spec_helper from stable/ocata  https://review.openstack.org/45938504:11
openstackgerritMerged openstack/tripleo-heat-templates master: Add initial support for NSX plugin  https://review.openstack.org/44166804:11
*** dsariel has joined #tripleo04:13
*** fragatina has joined #tripleo04:25
*** fragatina has quit IRC04:26
*** fragatina has joined #tripleo04:27
*** fzdarsky has joined #tripleo04:36
*** yamahata_ has joined #tripleo04:36
*** udesale has joined #tripleo04:40
*** jaganathan has quit IRC04:44
*** jaganathan has joined #tripleo04:45
*** ramishra has joined #tripleo04:51
*** ramishra has quit IRC04:51
*** garyk has joined #tripleo04:51
*** dixiaoli has quit IRC04:51
*** ramishra has joined #tripleo04:53
*** gkadam has joined #tripleo05:00
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025905:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427205:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581405:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]05:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]05:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)05:10
*** masco has joined #tripleo05:14
*** ratailor has joined #tripleo05:15
*** dixiaoli has joined #tripleo05:16
*** mdnadeem has joined #tripleo05:17
*** agurenko has joined #tripleo05:17
*** janki has joined #tripleo05:27
*** yolanda has quit IRC05:31
*** fzdarsky has quit IRC05:34
*** saibarspeis has joined #tripleo05:36
*** prateek has joined #tripleo05:39
*** yprokule has joined #tripleo05:49
*** lmiccini has joined #tripleo05:57
*** anshul has joined #tripleo05:58
*** iranzo has joined #tripleo05:59
*** iranzo has joined #tripleo05:59
*** florianf has joined #tripleo06:02
*** nyechiel has joined #tripleo06:07
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025906:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427206:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581406:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]06:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]06:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)06:10
*** tzumainn has quit IRC06:13
openstackgerritMerged openstack/tripleo-heat-templates master: Disable Manila CephFS snapshots by default  https://review.openstack.org/43641106:14
openstackgerritMerged openstack/os-net-config master: Removed vlan port on ovs_user_bridge in dpdk bond samples  https://review.openstack.org/44744606:14
*** jaosorior_away is now known as jaosorior06:16
*** anshul has quit IRC06:17
*** anshul has joined #tripleo06:17
*** dparkes has joined #tripleo06:20
*** axisys has quit IRC06:20
*** fpan has joined #tripleo06:20
*** dixiaoli_ has joined #tripleo06:22
*** dixiaoli has quit IRC06:24
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere for etcd  https://review.openstack.org/45530906:31
*** fpan has quit IRC06:32
*** jaganathan has quit IRC06:32
*** ratailor_ has joined #tripleo06:32
*** ratailor has quit IRC06:32
*** axisys has joined #tripleo06:42
*** chem_gone has quit IRC06:54
jaosoriorbandini: Hey dude, I need an RPM with pymysql, is it you that I can poke about it? :)06:55
bandinijaosorior: that would be dciabrin most likely :)06:55
jaosoriordciabrin: you around?06:56
dciabrinjaosorior, heya06:56
jaosoriordciabrin: hey dude, is it possible for you to pass me an rpm with the latest pymysql? Would like to try it out.06:56
dciabrinjaosorior, a test rpm?06:56
jaosorioryep06:56
dciabrinjaosorior, sure. any specific version you're interested in?06:57
jaosoriorI had tried it out before, but got pretty stuck with some neutron related issues and eventually started doing something else. Now I gotta get back to this06:57
jaosoriordciabrin: anything that contains this https://github.com/PyMySQL/PyMySQL/commit/557d0d90a5305bb7b3f49aafb83425ef5d1b449006:58
jaosorioroh, well, I think this one does https://github.com/PyMySQL/PyMySQL/commit/f6d28536493596e4ff7b87cf4d5201d657dae44d06:58
dciabrinjaosorior, oh ok looking06:58
dciabrinjaosorior, checking if fedora already has something for us06:59
dciabrinjaosorior, seems like f25 and f26 already have python-PyMySQL 0.7.1107:02
jaosorioryay07:03
*** yolanda has joined #tripleo07:04
dciabrinjaosorior, so can you test with fedora pkg if it that works I can ask on #rdo to bump PyMySQL to 0.7.11 in CBS07:06
dciabrinjaosorior, I can't build it myself in CBS07:06
jaosoriortrying it out07:07
jaosoriordciabrin: thanks dude!07:07
*** khrystoph has quit IRC07:07
*** khrystoph has joined #tripleo07:08
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025907:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427207:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581407:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]07:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]07:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)07:10
*** aufi has joined #tripleo07:11
*** jlinkes has joined #tripleo07:16
*** khrystoph has quit IRC07:16
*** khrystoph has joined #tripleo07:16
*** bogdando has joined #tripleo07:17
*** ratailor_ has quit IRC07:17
*** ccamacho has quit IRC07:19
*** amoralej|off is now known as amoralej07:21
*** cylopez has joined #tripleo07:21
*** g0d355__ has quit IRC07:26
*** jaganathan has joined #tripleo07:28
*** pcaruana has joined #tripleo07:32
*** jpich has joined #tripleo07:33
*** jpena|away is now known as jpena07:36
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart-extras master: Generate the list of required containers from t-h-t  https://review.openstack.org/45857507:36
*** yprokule_ has joined #tripleo07:36
mandrejistr, sshnaidm: this fixes the containers CI failure https://review.openstack.org/#/c/458575/07:37
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart-extras master: Generate the list of required containers from t-h-t  https://review.openstack.org/45857507:38
*** mcornea has joined #tripleo07:38
mandrejaosorior: oops, I fixed a typo in the commit message, can you restore your +2? ;)07:39
mandrejaosorior: thx07:39
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use list_concat for metadata_settings for haproxy  https://review.openstack.org/45956207:39
jaosoriormandre: no biggie, let me know if you need any other reviews07:39
*** beekneemech has joined #tripleo07:40
*** khrystoph has quit IRC07:40
*** milan has joined #tripleo07:40
*** fultonj_ has joined #tripleo07:41
*** fultonj has quit IRC07:41
*** bnemec has quit IRC07:41
*** yprokule has quit IRC07:41
*** yprokule_ is now known as yprokule07:41
*** khrystoph has joined #tripleo07:41
mandrejaosorior: yep, we need the parent patch at https://review.openstack.org/#/c/458574/07:42
*** brault has joined #tripleo07:43
*** yolanda has quit IRC07:44
*** bkopilov_ has joined #tripleo07:45
*** brault has quit IRC07:48
*** beekneemech has quit IRC07:50
*** milan has quit IRC07:54
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Merge "Merge pre|post puppet resources into pre|post config."  https://review.openstack.org/45957207:54
*** ccamacho has joined #tripleo07:54
*** milan has joined #tripleo07:55
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Move containers common volumes from yaql to list_concat  https://review.openstack.org/45957307:56
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Merge "Merge pre|post puppet resources into pre|post config."  https://review.openstack.org/45957207:56
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45957207:57
*** zoli|gone is now known as zoli07:59
*** zzzeek has quit IRC08:00
*** yolanda has joined #tripleo08:00
*** zzzeek has joined #tripleo08:01
*** dsariel has quit IRC08:02
rnoriegaccamacho, pliu dropped you an email about the L2GW integration08:02
rnoriegaccamacho, what do you think about it?08:02
*** aufi has quit IRC08:03
ccamachohey rnoriega, mmm I was out let me dig in my email to get it08:05
*** florianf has quit IRC08:05
ccamachornoriega do you know +- the dates?08:06
*** nyechiel has quit IRC08:06
rnoriegaccamacho, last week I think08:06
ccamachocool let me check it08:06
*** nyechiel has joined #tripleo08:07
pliuccamacho, on 17th08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025908:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427208:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581408:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]08:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]08:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)08:10
pliuccamacho, just want to know if you insist the idea of putting the l2gw env in single file.08:10
ccamachopliu rnoriega reading it right now.08:11
rnoriegaccamacho, thanks buddy!08:12
*** ratailor_ has joined #tripleo08:13
*** lucas-afk is now known as lucasagomes08:13
*** paramite has joined #tripleo08:13
jaosoriorbogdando: hey dude08:14
jaosoriorbogdando: regarding https://review.openstack.org/#/c/458501/08:14
*** karimb has joined #tripleo08:15
bogdandojaosorior: hey08:15
jaosoriorbogdando: so the deal is that, there are two undercloud install methods, one is the "undercloud deploy" which is based on t-h-t and was introduced recently to deploy a container based undercloud; and the other one is "undercloud install" which is the classic method that deploys the undercloud.08:15
*** nyechiel has quit IRC08:16
jaosoriorbogdando: the hieradata overrides are used for the "undercloud install" and give extra hieradata for the underlying puppet manifests to use, for the undercloud08:16
*** trozet has joined #tripleo08:16
*** nyechiel has joined #tripleo08:17
jaosoriorbogdando: the overcloud can take extra hieradata, but it's done through a heat environment and through a key called ExtraConfig. So that file (the hieradata overrides) is not used for the overcloud.08:17
*** yolanda has quit IRC08:17
*** aufi has joined #tripleo08:17
bogdandojaosorior: that works, thank you. I commented with the hope to see this explained in some docs :)08:17
bogdandojust to complete the change request08:18
ccamachornoriega, pliu, yeah I don't have a strong opinion about it, but like I replied I do like to have the env files as simple as possible, do you know how many users might use them independently (splitting the env files)?08:19
ccamachobut in any case you can go for it08:19
ccamachoIll remove my -108:19
bogdandojaosorior: so I was agree with the change, just wanted other users to see it explained like you did08:20
ccamachomaybe beagles can give us also some feedback08:20
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Include Deployed Nodes in NodesAssignment counts  https://review.openstack.org/45533608:20
*** karimb has quit IRC08:21
bogdandojaosorior: I'm wondering how could I apply that knowledge to the https://review.openstack.org/#/c/450792/ to unify both ways a little bit?..08:21
rnoriegaccamacho, AFAIK, I know more advance users choosing Opendaylight as backend than the agent. pliu what about you?08:22
bogdandoI mean the undercloud_extra_args https://review.openstack.org/#/c/450792/13/roles/undercloud-deploy/templates/undercloud-deploy.sh.j2@8908:22
*** florianf has joined #tripleo08:23
bogdandofolks, could you help me to figure out, why there was no containers jobs for the https://review.openstack.org/#/c/450792/13 ?..08:23
bogdandoshouldn't the 'check experimental' to do the trick? and where is ovb-containers? :(08:24
*** milan has quit IRC08:24
bogdandoshall I submit some patch to tht with depends on put to the extras patch in order to test it as appropriate?08:25
*** milan has joined #tripleo08:25
bogdandomandre: ^^ ?08:25
pliuccamacho rnoriega,  no matter how, we will have more than one env file. Just 2 vs 3, for me I prefer 3 independent files. Anyway, if anyone insist 2 files, I'm ok with that.08:26
bogdandomwhahaha: EmilienM: hi, could we merge https://review.openstack.org/#/c/459414/ , updated https://review.openstack.org/#/c/459395/1 also08:28
*** aufi has quit IRC08:29
ccamachornoriega pliu naa wait for beagles to see his opinion, in any case we can merge it soon, anyway.. are we testing it anywhere?08:29
ccamachoin a test scenario?08:29
ccamachornoriega mind to activate it in a test scenario?08:30
bogdandoplease merge https://review.openstack.org/#/q/topic:rfe1676373 for quickstart extras08:32
*** yolanda has joined #tripleo08:33
*** hewbrocca_afk is now known as hewbrocca08:33
bogdandoand please finish that one https://review.openstack.org/#/c/449552/ :)08:35
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui stable/ocata: Imported Translations from Zanata  https://review.openstack.org/45958208:35
*** anshul has quit IRC08:36
*** dtantsur|afk is now known as dtantsur08:37
rnoriegaccamacho, sure! I'll enable this in puppet modules and then TripleO08:37
*** karimb has joined #tripleo08:37
ccamachowill be nice at least to install it (just install the packages check the config files) as we had to fix things before have working the bgpvpn, currently the CI is not doing anything with it08:39
*** derekh has joined #tripleo08:39
jaosoriorbogdando: I guess you could if you have a separate template for both approaches08:40
jaosoriorbogdando: like, the actual hieradata could be the same, both that would be rendered on another template.08:40
*** aufi has joined #tripleo08:41
mandrebogdando: for a reason I don't understand the containers job doesn't appear in the job summary on https://review.openstack.org/#/c/450792/, but it's there if you look at the jenkins messages08:41
mandrebogdando: http://logs.openstack.org/92/450792/13/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/fa7724a/08:41
bogdandomandre: hm, I couldn't find it for build 13 in comments from jenkins... sigh08:42
*** chem has joined #tripleo08:42
mandrebogdando: FYI, the containerized overcloud job was green on PS12, http://logs.openstack.org/92/450792/12/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/fedadae/08:43
bogdandomandre: found ovb in comments, indeed! but where is undercloud multinode in experimental?..08:43
openstackgerritDamien Ciabrini proposed openstack/puppet-tripleo master: WIP containerized pacemaker haproxy  https://review.openstack.org/45883208:45
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Cleanup cinder paste configuration for ssl.  https://review.openstack.org/45927108:47
*** brault has joined #tripleo08:52
*** anshul has joined #tripleo08:52
*** jprovazn has joined #tripleo08:55
openstackgerritMerged openstack-infra/tripleo-ci master: Remove centos-release-openstack-* in nodes preparation  https://review.openstack.org/45942108:55
openstackgerritMerged openstack-infra/tripleo-ci master: Include experimental jobs in cistatus page  https://review.openstack.org/45940808:56
*** brault has quit IRC08:56
mandrebogdando: iiuc the undercloud container job doesn't run with quickstart yet, there is no reason to add it to oooq and oooq-extra CI jobs08:57
bogdandomandre: ok, so then I can only test it as a dependency to some stub tht patch perhaps08:58
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: WIP THT part of HAproxy HA docker container  https://review.openstack.org/45959709:00
*** salmankhan has joined #tripleo09:00
*** dbecker has joined #tripleo09:01
*** fzdarsky has joined #tripleo09:04
*** salmankhan has quit IRC09:05
*** salmankhan has joined #tripleo09:06
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025909:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427209:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581409:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]09:10
*** athomas has quit IRC09:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]09:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)09:10
*** brault has joined #tripleo09:13
jpichjistr: Hello! Looking at the review you linked to yesterday, I think the undercloud upgrade job is now clear from that error message, is that right?09:15
jistrjpich: we didn't get far enough to see that yet: http://logs.openstack.org/84/450784/18/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/8bb05f1/logs/undercloud/home/jenkins/overcloud_upgrade_console.log.txt.gz09:17
*** athomas has joined #tripleo09:18
jistrlooks like that error should be fixed now though -- i can see the image here https://hub.docker.com/r/tripleoupstream/centos-binary-mistral-engine/09:19
jpichjistr: Damn, the error happened during the "deploy" step, not the undercloud migration step, I got mixed up. Ok! I'll keep an eye on that job too then09:19
*** dsariel has joined #tripleo09:20
jistrjpich: re-ran check experimental09:20
jistrwe'll see09:21
jpichAck09:21
*** karimb has quit IRC09:21
*** yolanda has quit IRC09:22
*** milan has quit IRC09:25
*** milan has joined #tripleo09:25
*** zoli is now known as zoli|lunch09:26
*** karimb has joined #tripleo09:26
*** dsariel has quit IRC09:27
*** limao has quit IRC09:28
*** limao has joined #tripleo09:29
*** tosky has joined #tripleo09:30
*** limao has quit IRC09:33
*** dsariel has joined #tripleo09:35
tbarronjprovazn: Hi Jan: do you think we can backport https://review.openstack.org/#/c/436411/ ?09:36
jprovazntbarron, hi, yes, I think. I'll send a patch soon09:37
tbarronjprovazn: cool, can probably just rebase to stable/ocata in gerrit then09:37
*** yolanda has joined #tripleo09:37
*** dixiaoli_ has quit IRC09:38
openstackgerritMerged openstack/tripleo-puppet-elements master: Add puppet-etcd module  https://review.openstack.org/45941409:40
*** shardy has joined #tripleo09:41
openstackgerritJan Provaznik proposed openstack/tripleo-heat-templates stable/ocata: Disable Manila CephFS snapshots by default  https://review.openstack.org/45961209:43
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Enable no-unneeded-ternary eslint rule  https://review.openstack.org/45931509:44
tbarronjprovazn: while I'm bothering you, do you know the typical overcloud deploy command for integrated ceph for a HCI environment?09:46
tbarronjprovazn: in particular, HA and networking aspects?09:46
tbarronjprovazn: product managment is interested in providing support for native cephfs in such an09:46
tbarronjprovazn: environment ahead of general availability for certain customers09:47
tbarronjprovazn: on the theory that their nova vms are under admin control, running NFV apps, so that09:47
tbarronjprovazn: the security aspect of allowing nova vms direct access to the ceph storage is not a big concern.09:48
tbarronjprovazn: so the question of whether the storage network is isolated, bridging, etc.09:48
tbarronjprovazn: needs to be investigated for these topologies09:49
tbarronjprovazn: all I know at this point is that they are not putting their OSDs on separate storage nodes09:49
openstackgerritMerged openstack/puppet-tripleo master: Include zaqar apache module  https://review.openstack.org/44795709:53
jaosoriorbandini, dciabrin: Is there a reason why we're using skip-name-resolve in the galera configuration?09:53
*** milan has quit IRC09:54
*** milan has joined #tripleo09:55
jprovazntbarron, not off hand (never looked closer at HCI)09:58
jprovazntbarron, but I can take look09:58
tbarronjprovazn: I'm happy to look as well, now that you've taught me about isolated networks and bridges :D10:00
*** karimb_ has joined #tripleo10:00
*** karimb has quit IRC10:01
*** saibarspeis has quit IRC10:01
jprovazntbarron, taught... lol10:02
openstackgerritThomas Herve proposed openstack/tripleo-heat-templates master: Pass httpd service_name to Zaqar  https://review.openstack.org/45961910:02
tbarronjprovazn: well, everything is relative :)10:02
*** fzdarsky has quit IRC10:05
*** anshul has quit IRC10:06
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025910:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427210:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581410:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]10:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]10:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)10:10
*** dsariel has quit IRC10:14
paramiteEmilienM, Hey Emilien, so just to make sure, this is not a problem in path itself, right: http://logs.openstack.org/24/459224/1/check/gate-tripleo-ci-centos-7-multinode-upgrades/45bee83/console.html#_2017-04-25_08_52_17_642074 ?10:17
*** anshul has joined #tripleo10:17
florianfdtantsur: Hi! Can I bother you with an ironic question?10:18
dtantsurflorianf, hi, sure10:20
florianfdtantsur: Is there a way to get some ssh connection data out of ironic after node registration?10:20
*** fzdarsky has joined #tripleo10:21
dtantsurflorianf, I don't think so (putting aside that we're removing ssh support completely)10:22
florianfdtantsur: oh right, understood. that makes the question kind of obsolete... Just curiuos: Why are you removing ssh support?10:23
dtantsurflorianf, it's for VMs, and we are the bare metal service :)10:24
dtantsurpreviously we used SSH to simulate BMs on VMs, now we have virtualbmc for that10:25
dtantsursee http://tripleo.org/environments/virtualbmc.html10:25
*** agurenko has quit IRC10:25
florianfdtantsur: :-) ok, thanks!10:26
florianfdtantsur: will have a look at that.10:26
*** thrash|g0ne is now known as thrash10:27
jankiHi. I am working on this bug https://bugs.launchpad.net/tripleo/+bug/161492710:27
openstackLaunchpad bug 1614927 in tripleo "openstack overcloud node delete should be powered by a mistral workflow" [Medium,Confirmed] - Assigned to Janki Chhatbar (jankihchhatbar)10:27
jankiBut it seems the bug is already solved and implemented10:27
*** brault has quit IRC10:29
*** ebarrera has joined #tripleo10:29
*** agurenko has joined #tripleo10:30
*** owalsh_ has joined #tripleo10:30
*** ebarrera has joined #tripleo10:31
jankiEmilienM, Hi. Can you please confirm confusion regarding the above mentioned bug10:31
openstackgerritMerged openstack/tripleo-ui stable/ocata: Imported Translations from Zanata  https://review.openstack.org/45958210:32
*** ratailor__ has joined #tripleo10:33
*** nyechiel_ has joined #tripleo10:33
*** owalsh has quit IRC10:33
*** nyechiel has quit IRC10:33
*** ratailor_ has quit IRC10:34
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: DO NOT MERGE testing hostpath logs  https://review.openstack.org/45962510:34
*** ramishra has quit IRC10:34
*** zoli|lunch is now known as zoli10:36
*** yamahata_ has quit IRC10:36
*** ramishra has joined #tripleo10:40
*** fragatina has quit IRC10:40
*** morazi has quit IRC10:41
*** fragatina has joined #tripleo10:41
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Simplify Node Ports fetching  https://review.openstack.org/45962810:44
*** jkilpatr has quit IRC10:49
*** brault_ has joined #tripleo10:52
*** ramishra has quit IRC10:53
jistrmatbu: hi, have you had any success with the mixed version deployment?10:55
jistrmatbu: i can't even get the downloaded undercloud qcow2 running10:55
matbujistr: hello, more or less, i first deployed a nett-iso env10:55
matbujistr: i got some issue with the mysql connection10:56
jistrmatbu: and which undercloud qcow2 did you use?10:56
matbujistr: so to get something more simple, im deploying w/o net-iso and i will try to get a full working upgrade today10:56
matbujistr: hmm secs.. (it's weird that you are not able to create the undercloud)10:56
matbujistr: im grabing this one: https://images.rdoproject.org/master/delorean/current-tripleo/stable/undercloud.qcow210:57
matbujistr: well i mirror it on my local box, cause my network connection is too slow to donwload it each time10:57
jistrmatbu: hm weird. I tried the same one and OOOQ fails with this https://paste.fedoraproject.org/paste/kHKDpt2~t9IY-cWVHL-mP15M1UNdIGYhyRLivL9gydE=10:58
matbujistr: erf paste not found, idk why this happen to my on fedora paste10:59
matbus/my/me/10:59
*** chandankumar is now known as chkumar|intervie10:59
jistrFAILED - RETRYING: TASK: setup/undercloud : Get undercloud vm ip address (1 retries left).10:59
matbui guesss my irc client should make some weird char transformation on the url10:59
jistrthe VM starts but doesn't get an IP address11:00
matbujistr: always on the same hardware?11:00
jistryea11:00
matbujistr: did you try to boot it manually ? and check the virsh console11:01
jpichjanki: Hi! Thank you for looking into this bug. I think you are right, it was implement in Newton in https://review.openstack.org/#/c/382707/ as far as I can tell11:01
matbujistr: maybe the vm failed to boot,11:01
jistrmatbu: yes, virsh console is blank, nothing printing even after restart11:01
jpichjanki: I'll update the bug11:01
*** bkopilov has quit IRC11:03
*** nyechiel_ has quit IRC11:03
*** karimb_ has quit IRC11:03
*** bkopilov_ has quit IRC11:03
*** ebarrera has quit IRC11:04
*** dsariel has joined #tripleo11:04
*** fzdarsky has quit IRC11:04
matbujistr: weird, idk if quickstart has a undercloud.xml file11:04
matbujistr: and otherwise, it's working with ocata undercloud or newton ?11:04
openstackgerritDamien Ciabrini proposed openstack/tripleo-common master: WIP kolla template overrides for pacemaker HA containers  https://review.openstack.org/45617411:05
*** yolanda has quit IRC11:05
jistrmatbu: i think all-ocata deployment worked, yes. But there i wasn't downloading undercloud.qcow2. It downloaded overcloud-full.tar from Ocata, and from that it produced Ocata undercloud.qcow2 via libguesfs tools, IIRC.11:06
*** karimb has joined #tripleo11:06
*** ebarrera has joined #tripleo11:07
*** karimb has quit IRC11:07
*** brault_ has quit IRC11:07
*** pkovar has joined #tripleo11:08
*** saibarspeis has joined #tripleo11:10
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025911:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427211:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581411:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]11:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)11:10
*** saibarspeis has quit IRC11:10
*** jkilpatr has joined #tripleo11:11
jaosoriorzzzeek: Hey, I'm having a pretty strange issue when enabling TLS on mariadb, and have been told you might have some insights on this. Let me know when you're around.11:14
*** brault has joined #tripleo11:14
matbujistr: btw the upload of the container, takes a really huge time11:18
jistrmatbu: yea, it's about 2 GB11:18
*** ebarrera has quit IRC11:18
*** pgadiya has joined #tripleo11:18
jistrmatbu: CI should be using infra mirror now, so it should be fast, < 10 minutes for all images11:19
*** owalsh has joined #tripleo11:19
matbujistr: k, yep 10 minutes looks better :)11:20
*** owalsh_ has quit IRC11:22
ccamachohey bandini! Do you mind if I rebase the submissions for https://review.openstack.org/#/q/topic:change-ha-default11:22
ccamacho??11:22
jankijpich, thanks :)11:24
bandiniccamacho: that'd be great, please do. I am just too swamped atm11:25
ccamachobandini sure thing, thanks!11:26
*** diana_clarke has joined #tripleo11:27
jistrmandre: please see https://review.openstack.org/#/c/458575/5//COMMIT_MSG -- if we merge the patch we stop testing the yaml file in tripleo-common, and we might break production but CI wouldn't catch it11:28
*** dsariel has quit IRC11:28
openstackgerritMerged openstack/puppet-tripleo master: Add support for Redfish hardware in Ironic  https://review.openstack.org/45927411:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Deploy ceilometer_auth_enabled to node containing keystone  https://review.openstack.org/45938811:31
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Upgrade to containerized overcloud  https://review.openstack.org/44857611:32
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart master: add featureset config for a containerised overcloud  https://review.openstack.org/44615111:33
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart master: Add featureset011 - upgrade from BM to containerized  https://review.openstack.org/45029511:33
openstackgerritDougal Matthews proposed openstack/instack-undercloud master: Gate test  https://review.openstack.org/45964111:34
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Support container upgrades in multinode OOOQ CI  https://review.openstack.org/45078411:34
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: DO NOT MERGE testing hostpath logs/custom undercloud  https://review.openstack.org/45962511:35
*** dtantsur is now known as dtantsur|brb11:35
*** anton has quit IRC11:37
EmilienMbogdando: done11:37
EmilienMparamite: the link you showed is a different issue that is part of the alerts ^11:38
bogdandoEmilienM: thank you11:38
*** anton has joined #tripleo11:40
*** pradk has joined #tripleo11:44
*** ckyriakidou has joined #tripleo11:47
bogdandoahem, oooq folks, could we please merge https://review.openstack.org/#/q/topic:rfe1676373 ?11:47
*** jayg|g0n3 is now known as jayg11:48
bogdandoa very minor change to enable a very nice to have use case11:48
EmilienMbogdando: this change is useless: https://review.openstack.org/#/c/453056/4/roles/libvirt/setup/overcloud/tasks/main.yml11:49
bogdandoEmilienM: just an empty line, or do you mean the patch? this part is important https://review.openstack.org/#/c/453056/4/roles/libvirt/setup/undercloud/tasks/main.yml11:50
EmilienMbogdando: the empty line in a file11:50
bogdandoEmilienM: just a cosmetic fix :(11:51
cmyster.211:51
*** pcaruana has quit IRC11:51
*** lucasagomes is now known as lucas-hungry11:52
*** ratailor__ has quit IRC11:52
mandrejistr: FWIW, the overcloud_images.yaml file was in a contrib directory not so long ago11:54
*** owalsh_ has joined #tripleo11:54
EmilienMbogdando: please avoid it next time... it's not helping git history11:55
mandrejistr: I'm thinking a better approach would be to generate the list of container images to pull from 'openstack overcloud container image upload' when you don't give it a config file to parse11:55
jistrmatbu: hm i rebooted the host, rm -rf /home/stack and /var/cache/tripleo-quickstart, still the same issue. Can you please share how you run quickstart? I must be doing something wrong...11:55
*** pcaruana has joined #tripleo11:56
*** owalsh has quit IRC11:57
*** salmankhan has quit IRC11:57
jistrmandre: yea that might be better, as we could keep CIing what end users would use11:57
matbujistr: yep basicaly, i have a workspace on my laptop, and i run that: bash $WORKSPACE/tripleo-quickstart/ci-scripts/upgrade.sh newton delorean composable_upgrade periodic master-tripleo True True ocata11:57
matbujistr: i just export VIRTHOST=(my_local_box) and WORKSPACE=$PWD11:57
jistrhm newton? i thought we were dealing just with ocata and pike...11:58
*** karimb has joined #tripleo11:58
* jistr looks at the script11:59
*** morazi has joined #tripleo11:59
*** weshay_PTO is now known as weshay11:59
matbujistr: yep don't cara about the version12:00
matbus/cara/care/12:00
matbujistr: its just the basic command that i used to run, but i change the value depending on what i want to deploy12:00
jistrok thanks12:01
matbujistr: for example, for O to P i replace newton by master-undercloud-ocata-overcloud and ocata by master12:01
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Enable the HA deployment by default  https://review.openstack.org/35906012:02
*** atoth has quit IRC12:02
*** dprince has joined #tripleo12:02
sshnaidmEmilienM, bogdando wrt https://review.openstack.org/#/c/453056/ - it was reverted here https://review.openstack.org/#/c/452998/, because it broke baremetal deployments12:07
bogdandosshnaidm, EmilienM: that was it's previous version, this is a new patch and it doesn't modify that role anymore...12:08
bogdandothe breaking change https://review.openstack.org/#/c/448294/2/roles/libvirt/setup/overcloud/tasks/main.yml was removed from the new patch12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025912:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427212:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581412:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]12:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]12:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)12:10
*** pblaho has quit IRC12:10
sshnaidmbogdando, ok, thanks for clarifying12:11
EmilienMsshnaidm: hey, welcome back12:12
sshnaidmEmilienM, hi12:12
sshnaidmEmilienM, how is going everything?12:12
*** jcoufal has joined #tripleo12:12
EmilienMsshnaidm: well, routine12:13
EmilienMsshnaidm: :D12:13
EmilienMsshnaidm: but... we survived !12:13
sshnaidmEmilienM, glad to see it :)12:13
*** atoth has joined #tripleo12:14
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: WIP Fix idempotency and ordering of db sync  https://review.openstack.org/45940412:19
*** chlong has quit IRC12:20
*** yolanda has joined #tripleo12:21
*** dsariel has joined #tripleo12:22
*** trozet has quit IRC12:26
*** rhallisey has joined #tripleo12:27
*** dtantsur|brb is now known as dtantsur12:28
*** arxcruz has quit IRC12:28
*** ramishra has joined #tripleo12:28
*** liverpooler has joined #tripleo12:29
*** salmankhan has joined #tripleo12:29
weshayjpich, honza greetings. Do you have a few minutes to review https://review.openstack.org/#/c/407733/12:29
honzaweshay: yes!12:29
weshayrelated change https://review.openstack.org/#/c/459061/12:30
weshayhonza, thanks :)12:30
honzaweshay: I saw lots of activity so I was putting it off12:30
*** jpena is now known as jpena|lunch12:30
weshayhonza, it should be ready.. I can see improvements to the doc but the code is good, ssl overcloud may not be right.12:32
*** rlandy has joined #tripleo12:34
honzaweshay: do you happen to know if oooq now has instructions on how to run it on ovb?  i tried a couple months ago and it was full of sadness12:35
*** ckyriakidou has quit IRC12:35
weshayhonza, rlandy is putting finishing touches on such a script12:35
honzaweshay: very good12:35
weshayhonza, I was pto yesterday, not sure of the status. rlandy could comment. https://review.openstack.org/#/c/452249/12:36
shardyWhat's the status of gate-tripleo-ci-centos-7-ovb-containers-oooq-nv - is it worth rechecking atm?12:37
rlandyweshay: honza: I asked members of the ci team to try out the script yesterday. I fixed some comments. waiting for more feedback/ core reviews/merge12:37
shardyit failed for me but the reason doesn't appear related to my patch12:37
honzarlandy: excellent, very happy to see this patch12:38
*** nyechiel_ has joined #tripleo12:40
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add option to build delorean changes and include the repo  https://review.openstack.org/43400412:40
openstackgerritDamien Ciabrini proposed openstack/puppet-tripleo master: WIP containerized pacemaker haproxy  https://review.openstack.org/45883212:40
*** ansmith has joined #tripleo12:43
*** trown|outtypewww is now known as trown12:46
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci master: Enable the HA deployment by default  https://review.openstack.org/41517912:47
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add option to build delorean changes and include the repo  https://review.openstack.org/43400412:48
matbujistr: Stack overcloud UPDATE_COMPLETE12:49
matbujistr: so with network isolation, there is probably some fixes to do, i'll debug that today12:49
*** pcaruana has quit IRC12:50
matbujistr: and btw i'll add a follow up review for getting the mixed upgrade12:50
jistrmatbu: great. Btw i'm going through the release files trying to decrypt what it does12:50
*** amoralej is now known as amoralej|lunch12:50
jistri'm not sure this is correct https://github.com/openstack/tripleo-quickstart/blob/master/config/release/master-undercloud-ocata-overcloud.yml12:50
dr_gogeta86good morning all12:50
*** lucas-hungry is now known as lucasagomes12:50
jistrthe overcloud image files are non-existent, but even when i corrected them, in the undercloud image i still get this12:51
dmsimardsshnaidm: hey, if you're around could you approve https://review.openstack.org/#/c/448108/ please, it's been in review for so long12:51
jistr-rw-rw-r--. 1 stack stack 1279655936 Apr 21 09:23 overcloud-full.qcow212:51
*** arxcruz has joined #tripleo12:51
dmsimardI don't want to end up breaking you because the deprecation still hasn't been removed12:51
*** owalsh has joined #tripleo12:51
jistrmatbu: ^ in other words, i think the overcloud-full.qcow2 is just taken from what already was within undercloud.qcow212:51
jistrmatbu: as we don't specify inject_images anwhere12:52
*** zoli is now known as pkralikc12:52
matbujistr: yep, the  image is downloaded somewhere else12:52
*** pkralikc is now known as zoli12:52
matbujistr: in quickstart12:52
dmsimardsshnaidm: actually wait ...12:52
matbujistr: but you're right, i need to fix the overcloud-full path12:52
jistrmatbu: ok, so the overcloud-full.qcow2 image from within undercloud.qcow2 is not used?12:53
* jistr confused12:53
*** pcaruana has joined #tripleo12:53
openstackgerritJuan Badia Payno proposed openstack/tripleo-heat-templates stable/newton: Increase documentation about parameters  https://review.openstack.org/45967012:53
*** owalsh_ has quit IRC12:53
*** rhallisey has quit IRC12:54
matbujistr: nop, because the undercloud.qcow2 has already an OC image on it, but it's a master OC image12:54
openstackgerritDavid Moreau Simard proposed openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html", move report to ara_oooq  https://review.openstack.org/44810812:54
*** ramishra_ has joined #tripleo12:54
matbujistr: so we need to drop it and download an Ocata image, then upload it into glance12:54
jistrmatbu: yea that's why i thought it was broken. But if we use a different one then all's fine i think :)12:54
matbujistr: actually, having a mixed undercloud images would be better, it would avoid extra complicated settings12:56
mandrejistr: hmm, there is no clean way that I can see to add the logic to build containers list on the fly to tripleo-common12:56
*** chkumar|intervie is now known as chandankumar12:56
mandrejistr: it requires a lot of refactoring12:56
*** rhallisey has joined #tripleo12:57
*** dsavineau has joined #tripleo12:58
matbujistr: so, i saw a converge file in tht for container upgrade, do we have to run a converge step ?12:58
*** ffiore has joined #tripleo12:58
jistrmandre: can't we keep using the old way then? just correct the path in the shell script?12:59
*** ffiore has quit IRC12:59
jistrmandre: i don't think it's a big problem that we need to add the images to tripleo-common first, and then land t-h-t patches12:59
*** anshul has quit IRC12:59
*** zoli is now known as pkralikc12:59
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Fix depends-on for tipleo-quickstart  https://review.openstack.org/45222512:59
jistrmandre: it's been like this with tripleo-puppet-elements, and (at least from my point of view) it's never been a problem13:00
*** pkralikc is now known as zoli13:00
*** anshul has joined #tripleo13:00
*** garyk has quit IRC13:00
mandrejistr: yeah... maybe, it doesn't feel right though13:01
mandrejistr: i'll push a change to fix the path so we have CI again13:01
*** garyk has joined #tripleo13:02
jistrmandre: +1, thanks13:02
*** ffiore has joined #tripleo13:03
bogdandofolks, AFAICT from CI logs, the logs patch is ok https://review.openstack.org/#/c/44260313:03
*** ffiore has quit IRC13:03
jistrmandre: having to submit a patch specifically when introducing a new container image is a good thing imo... it keeps the "TripleO dependencies snowball" under closer scrutiny13:03
bogdandojistr, mandre, dprince, shardy: ^^13:03
*** ffiore has joined #tripleo13:03
jistrbogdando: cool, we should get the containers CI green on it though, when we have a fix for the issue discussed above13:05
* bogdando reading13:06
openstackgerritMartin AndrĂ© proposed openstack/tripleo-quickstart-extras master: Fix path to overcloud_containers.yaml  https://review.openstack.org/45967513:06
mandrejistr, bogdando: we do now :) ^^13:07
bogdandoyay13:07
bogdandoso fast13:07
jistr+213:08
jistri did the same in the upgrade workflow patch13:08
*** ffiore_ has joined #tripleo13:08
*** ffiore has quit IRC13:08
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart master: Fix overcloud image path and repo url  https://review.openstack.org/45967713:08
matbujistr: https://review.openstack.org/45967713:08
*** psahoo has quit IRC13:08
matbujistr: for having this mixed upgrade we need also few reviews pending13:09
*** japestinho has quit IRC13:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025913:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427213:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581413:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]13:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]13:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)13:10
*** jmelvin has joined #tripleo13:11
jistrccamacho: jinx!! :)13:11
jistrsame review13:11
ccamachoajaj13:11
ccamacho:)13:11
ccamachojistr and at the same time :P13:11
*** owalsh has quit IRC13:11
*** ffiore_ has quit IRC13:12
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh  https://review.openstack.org/45224913:12
*** owalsh has joined #tripleo13:12
*** bkopilov has joined #tripleo13:13
*** bfournie has joined #tripleo13:13
*** bkopilov_ has joined #tripleo13:14
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart master: Fix overcloud image path and repo url  https://review.openstack.org/45967713:14
*** gbarros has joined #tripleo13:14
*** ffiore_ has joined #tripleo13:15
*** ffiore_ has quit IRC13:16
*** tzumainn has joined #tripleo13:16
*** ffiore_ has joined #tripleo13:16
*** owalsh has quit IRC13:17
*** abishop has joined #tripleo13:18
*** ffiore_ has quit IRC13:18
*** owalsh has joined #tripleo13:19
*** ramishra_ has quit IRC13:19
*** limao has joined #tripleo13:20
*** jaganathan has quit IRC13:21
*** ffiore_ has joined #tripleo13:21
*** ffiore_ has quit IRC13:22
*** ffiore_ has joined #tripleo13:22
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh  https://review.openstack.org/45224913:22
*** limao_ has joined #tripleo13:26
jpichDoes the error "Evaluation Error: Resource type not found: LimitNOFILE" on deploy ring a bell for anyone?13:26
*** Goneri has joined #tripleo13:26
*** ffiore_ has quit IRC13:26
*** limao has quit IRC13:26
jistrmatbu: btw i think we may want a slightly different set of repos than what's in master-undercloud-ocata-overcloud.yaml13:27
*** ibravo has joined #tripleo13:27
jistrmatbu: i started having some success with the undercloud finally, i'm presently testing this release file https://paste.fedoraproject.org/paste/K777Ntua9NmQY44kYCDSz15M1UNdIGYhyRLivL9gydE=13:27
*** masco has quit IRC13:28
jistrsorry i'll try different pastebin13:28
jistrmatbu: https://pastebin.com/qk20Uzr913:28
bogdandojpich: it stands for an obsolete puppet-tripleo package. I'm used to fix it with updating from current delorian instead of current-tripleo13:28
*** yolanda has quit IRC13:28
jpichbogdando: Ah, thank you for the pointer! I'll try that13:29
jistrmatbu: the files directly under /releases/ directory (not in subdirs), what are they for? RDO CI maybe?13:29
bogdandojpich: I normally deploy only undercloud yet though, with this patch https://review.openstack.org/#/c/450792/13/roles/undercloud-deploy/templates/undercloud-deploy.sh.j2@41 in place (but I use "current")13:29
*** ffiore_ has joined #tripleo13:30
jistrmatbu: because the repo set there (at least for master) doesn't match what TripleO CI uses13:30
bogdandocan't comment on other ways to deploys, overcloud etc...13:30
bogdandoto deploy*13:30
*** rhallisey has quit IRC13:31
*** rhallisey has joined #tripleo13:32
jpichCheers13:32
*** ffiore_ has joined #tripleo13:33
*** ffiore_ has quit IRC13:33
*** toure|gone is now known as toure13:33
*** trozet has joined #tripleo13:33
*** ffiore_ has joined #tripleo13:34
jistrjpich: seems like we got into a different error now... http://logs.openstack.org/84/450784/19/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/4f742f9/logs/undercloud/home/jenkins/overcloud_upgrade_console.log.txt.gz#_2017-04-25_12_53_5113:34
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Move gnocchi wsgi configuration to step 3  https://review.openstack.org/45725813:34
openstackgerritMerged openstack/instack-undercloud master: Replace mistral command with OpenStack CLI  https://review.openstack.org/45880813:34
openstackgerritMerged openstack/puppet-tripleo master: Update puppet-etcd version  https://review.openstack.org/45939513:35
openstackgerritMerged openstack/tripleo-quickstart master: Define libvirt pool as a common meta role  https://review.openstack.org/44854313:35
*** owalsh has quit IRC13:35
openstackgerritMerged openstack/tripleo-quickstart master: Allow custom undercloud initrd/vmlinuz  https://review.openstack.org/44740913:35
openstackgerritMerged openstack/tripleo-quickstart master: Instack.json for undercloud VM image  https://review.openstack.org/45305613:35
*** ffiore_ has quit IRC13:35
jpichjistr: Hm... http://logs.openstack.org/84/450784/19/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/4f742f9/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-04-25_12_53_50_80513:35
openstackgerritMerged openstack/python-tripleoclient master: Don't create tempest-deployer-input.conf in tests  https://review.openstack.org/44766513:35
jistrjpich: maybe something wrong with t-h-t for real?13:36
jpichjistr: Maybe? At least it got a bit further since the plan was updated correctly this time, I'm not sure at all about the circular dependency stuff though13:37
*** owalsh has joined #tripleo13:38
jpichMaybe it could also be the migration stuff causing separate issues13:38
matbujistr: yep13:38
matbujistr: idk tbh, and it's probably useless in this file13:38
*** chlong has joined #tripleo13:39
*** ffiore_ has joined #tripleo13:39
*** ffiore_ has quit IRC13:39
*** jpena|lunch is now known as jpena13:39
shardyThe circular dependency error looks real, but I'm not sure why the patch has triggered ot13:39
shardyit13:39
*** amoralej|lunch is now known as amoralej13:41
shardyhttps://github.com/openstack/tripleo-heat-templates/commit/258c6ce52d0c8467f34693722a883d96345802b213:42
shardyI wonder if that caused a regression in the docker-steps?13:42
*** owalsh has quit IRC13:42
jistrshardy: strange that we don't hit on a fresh deployment though...13:42
shardyjistr: yeah, but is this switching from puppet to containers during the upgrade?13:43
*** pcaruana has quit IRC13:43
jistrshardy: yea it is13:43
shardyso maybe deploy is broken for containers, we didn't check experimental https://review.openstack.org/#/c/452734/13:43
shardyI guess that should be easy to prove local - jistr you have an env to try, or do you want me to?13:44
shardylocally I mean13:44
*** yolanda has joined #tripleo13:44
*** owalsh has joined #tripleo13:44
jistrshardy: i'm trying an e2e upgrade right now with OOOQ, not sure yet how will it go (i'm attempting the mixed version UC/OC setup). If you have an env of yours where you can test this test and a couple of cycles for it, then it would be helpful.13:46
jistrs/test this test/test this/13:46
shardyjistr: Ok let me try it with the undercloud installer, that should show the same error if the patch broke us13:46
jistrshardy: ok thanks!13:46
bogdandofolks, it looks like https://review.openstack.org/#/c/447627/ is ready for merge at last13:47
openstackgerritMark Hamzy proposed openstack/python-tripleoclient master: Display error if no bare metal nodes  https://review.openstack.org/45969913:48
jistrjpich: i'll report this as a separate bug. I think it's more likely that it's some real issue with t-h-t, rather than mistral workflows being the issue.13:48
jistrjpich: thanks for the fix on the Mistral issue!13:48
jpichjistr: Sounds reasonable to me, I'm kind of out of my depth here :) I'll follow the bug, good luck with this new issue!! And no problems13:49
d0ugaljpich, jistr - which issue is this? :)13:49
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Change the relase name for the templates to pike.  https://review.openstack.org/43534513:49
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Bump ansible to 2.3  https://review.openstack.org/45819313:49
jistrd0ugal: which one :) fixed mistral issue or current (likely t-h-t) issue?13:50
* jistr looks for link on the earlier one13:50
jistrd0ugal: https://bugs.launchpad.net/tripleo/+bug/168517113:51
openstackLaunchpad bug 1685171 in tripleo "updating overcloud after undercloud upgrade: Unable to create plan. A container with the name overcloud already exists." [High,Fix released] - Assigned to Julie Pichon (jpichon)13:51
shardyjistr: the undercloud installer doesn't reproduce it, but any deploy with more than one role does13:51
shardyjistr: trying a revert now13:51
d0ugaljistr: the mistral issue mostly, but the other might be interesting too!13:51
*** gbarros has quit IRC13:51
jpichd0ugal: The possible tht one is http://logs.openstack.org/84/450784/19/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/4f742f9/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-04-25_12_53_50_80513:51
*** janki has quit IRC13:52
*** janki has joined #tripleo13:52
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Simplify Node Ports fetching  https://review.openstack.org/45962813:52
openstackgerritMark Hamzy proposed openstack/tripleo-common master: Add error message if no bare metal nodes available  https://review.openstack.org/45970113:54
*** ebarrera has joined #tripleo13:54
*** ibravo has quit IRC13:54
*** pcaruana has joined #tripleo13:55
EmilienMmeeting in 5 min, feel free to bring topics: https://etherpad.openstack.org/p/tripleo-meeting-items13:56
jistrd0ugal: https://bugs.launchpad.net/tripleo/+bug/168609813:56
openstackLaunchpad bug 1686098 in tripleo "upgrade from puppet to containers - circular dependency" [High,Triaged]13:56
mandredprince, shardy, jaosorior: quick fix for containers CI at https://review.openstack.org/#/c/459675/13:57
zzzeekjaosorior: sure have any mysql.logs?  is this Red Hat OSP or just generic ?13:57
dprincemandre: looking13:57
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Include Deployed Nodes in NodesAssignment counts  https://review.openstack.org/45533613:58
mandredprince: I had another approach at first at https://review.openstack.org/#/c/458575/ but jistr and bogdando didn't like it :'(13:59
EmilienMtripleo meeting is about to start on #openstack-meeting-alt, feel free to join us!13:59
*** ibravo has joined #tripleo13:59
bogdandomandre: actually that looked fine to me as I wasn't aware of the impact :)14:00
*** ebarrera has quit IRC14:00
*** garyk has quit IRC14:00
*** lmiccini has quit IRC14:00
jistrmandre: haha sorry :)14:01
d0ugaljistr, jpich - thanks.14:01
beaglesccamacho: I wanna sync up on that review - seems I'm back to back mtgs these days and when there aren't meetings there are appointments ;) I'll ping you when it looks like there's  a window14:01
ccamachosure thing beagles! thanks man14:02
mandrebogdando: hehe, I should have had a less detailed commit message14:02
*** dtantsur is now known as dtantsur|mtg14:03
*** tzumainn has quit IRC14:03
*** japestinho has joined #tripleo14:03
*** tzumainn has joined #tripleo14:03
jaosoriorzzzeek: so, it's the one coming from CentOS upstream.14:04
jistrmandre, bogdando: i gave it a +2 at first too, and i wanted to do the same edit in the upgrade patch (as it was broken by the same thing), and only then i realized the problem14:04
jaosoriorzzzeek: I traced down the issue down to pymysql. And it seems to be getting stuck reading packages coming from a select query14:04
zzzeekjaosorior: OK so...this is not galera tls, you're trying to do client/server via tls14:05
openstackgerritBrad P. Crochet proposed openstack/tripleo-common master: Move the gathering of flavors into the CollectFlavors action  https://review.openstack.org/45970414:05
thrashd0ugal: ^^^14:05
thrashbandini: ^^^14:05
jaosoriorzzzeek: right14:06
thrashbandini: that should fix the bug you reported about the validations14:06
zzzeekjaosorior: does this SELECT likely return  a lot of rows ?14:06
bandinithrash: thanks *so much* :)14:06
jaosoriorzzzeek: around 400014:06
jaosoriorin this case14:06
thrashbandini: if you have time to test it out, that would be super duper awesome14:07
zzzeekjaosorior: so....have a stack trace?  err message ?  is it "commands out of sync" or something?14:07
jaosoriorzzzeek: so... no, it just gets stuck trying to read packages for some reason.14:08
zzzeekjaosorior: so.... galera or mariadb plain (or mysql)? what specific version?  what version of pymysql?  SQL logs, what is the SELECT query?  going through haproxy / vip / direct / etc (config for all that) ?14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025914:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427214:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168581414:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]14:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]14:10
openstackLaunchpad bug 1685814 in tripleo "centos-release-* repos conflicting with dlrn" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)14:10
*** ibravo has quit IRC14:10
weshayadarazs, tripleo mtg?14:11
weshayplease join14:11
*** pblaho has joined #tripleo14:11
jaosoriorzzzeek: Galera, 10.1.20-MariaDB, pymysql 0.7.11; the query was merely: SELECT ml2_gre_allocations.gre_id AS ml2_gre_allocations_gre_id, ml2_gre_allocations.allocated AS ml2_gre_allocations_allocated FROM ml2_gre_allocations;14:12
*** lmiccini has joined #tripleo14:12
*** prateek has quit IRC14:12
jaosoriorzzzeek: if you need I can also give you access to the environment.14:12
zzzeekjaosorior: what app / DB is that ?14:12
zzzeekjaosorior: sure, is there an RH BZ ?14:13
jaosoriorzzzeek: neutron. and the db is ovs_neutron;14:13
*** yamahata_ has joined #tripleo14:13
jaosoriorzzzeek: no BZ yet.14:13
*** bnemec has joined #tripleo14:14
adarazsweshay: joined. my 3d print just messed itself up so had to run to the other building to stop it right at the start of the meeting o_o :)14:15
weshayheh14:16
thrashadarazs: octoprint my man14:17
openstackgerritMerged openstack/tripleo-specs master: Add network configuration spec  https://review.openstack.org/39638314:18
adarazsthrash: I'll check it out! we have a lulzbot taz in the office, we have lower tech monitoring interface with 3 webcams: http://3dprinter.brq.redhat.com/ :D14:18
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras master: Add role to deploy FreeIPA  https://review.openstack.org/43619814:19
EmilienMbeagles: taking it here14:19
EmilienM(no need to block people on it)14:19
beaglesEmilienM: ack14:19
rlandydtantsur|mtg: ping when you are out of meeting. re: introspection and starting instances14:20
adarazswow, that was a short meeting :)14:20
beaglesEmilienM: so yeah, afaict this is basically a composable service that wants the option of being deployed as a composable14:20
beaglesEmilienM: argh14:20
dtantsur|mtgrlandy, I'm not going to be out of meetings today, I may try to just answer in some break14:20
beaglesEmilienM: "as a pacemaker managed resource"14:20
dtantsur|mtgrlandy, what's the question?14:21
*** yolanda has quit IRC14:21
openstackgerritMark Hamzy proposed openstack/python-tripleoclient master: Display error if no bare metal nodes  https://review.openstack.org/45969914:21
rlandydtantsur|mtg: if you run introspection on an instance that is powered off, it should be powered on when introspection starts, correct?14:21
dtantsur|mtgrlandy, yes14:21
rlandydtantsur|mtg: so I have debugging a case where I have two instances powered off and introspection never gets going because the instances are never powered on14:22
rlandyi can ironic node-set-power-state on14:22
dtantsur|mtgrlandy, probably power management failure, worth checking ironic-conductor logs14:22
rlandyand then introspection will start14:22
beaglesEmilienM: I'm not sure what the right approach is in this particular case ... it's probably something obvious and I'm just overthinking it14:22
jristEmilienM: quick meeting. thanks14:23
dtantsur|mtgrlandy, hmm, strange, inspector uses the same command essentially14:23
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart master: Add supplemental node provisioner  https://review.openstack.org/45152314:23
* beagles has to go pick up son brb14:23
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart master: Update quickstart-extras to deploy FreeIPA  https://review.openstack.org/45322314:23
*** trown is now known as trown|brb14:23
jkilpatrd0ugal, what's the status of this patch? https://review.openstack.org/#/c/451228/14:24
d0ugaljkilpatr: well, it depends on https://review.openstack.org/#/c/439568/14:24
d0ugaljkilpatr: but as for that one...14:25
d0ugaljkilpatr: I am trying to find somebody with more hardware that can help me test it14:25
rlandydtantsur|mtg: I see a 'IPMI power off timed out after 4 retries on node x' error but never a power on one14:25
EmilienMbeagles: I'll take time and look at it14:25
d0ugaljkilpatr: I'm unable to do anything other than a trivial test - which doesn't seem enough for a change designed for large deploys14:25
bnemecrlandy: I had a thought about this last night.  Are you shutting down the baremetal vms after you create them by any chance?14:26
EmilienMjrist: always a please to make it fast.14:26
*** gbarros has joined #tripleo14:26
rlandybnemec: I am14:26
*** fzdarsky has joined #tripleo14:26
jristEmilienM: +114:26
bnemecrlandy: Don't do that. :-)14:26
rlandybnemec: why not?14:26
bnemecThe bmc assume it owns the vm, and if you change the power state out from under it it may be confused about the state of the vm.14:27
dtantsur|mtgrlandy, right, because inspector issues reboot, not on. good catch14:27
jkilpatrd0ugal, agreed, is there any failure tolernece in the batching one? aka failures get pushed to the last batch to retry at least once?14:27
dtantsur|mtgbnemec, inspector calls reboot, which calls off, then on14:27
rlandybnemec: that's problematic ...14:27
rlandyif a user can't power off an instance14:27
bnemecThat also explains why you don't see the same behavior during deploy.14:27
hamzyis there a recommended way to restart the undercloud vm so that everything works when it is brought back up? I am seeing "openstack overcloud deploy" hang now in "Waiting for messages on queue"14:28
rlandyI test IPMP connection14:28
d0ugaljkilpatr: no, there isn't anything like that - it would make sense tho'14:28
rlandyso I power off the instances14:28
rlandyif there is no connection, we can stop the test then14:28
jkilpatrd0ugal, please add it when you get a chance, I'll work on getting you hardware to test it.14:28
rlandybnemec: also - this is a new problem.14:28
bnemecrlandy: Well, you could bounce the bmc vm after shutting down the instances.  That would force it to sync up.14:29
*** trown|brb is now known as trown14:29
bnemecrlandy: This was done to prevent excessive load on the ci cloud controllers from 70 jobs pinging for vm status once a minute.14:29
rlandybnemec: 'bounce the bmc vm' as in turn it on and off?14:30
d0ugaljkilpatr: that would be great if you could help there - I'll have time to look at it this week.14:30
bnemecrlandy: It might be worth making the caching behavior configurable though.14:30
bnemecrlandy: Yes14:30
rlandybnemec: so, if I only interacted with the nodes using ironic, I'd be ok14:32
bnemecrlandy: Right14:32
openstackgerritBrad P. Crochet proposed openstack/instack-undercloud master: Enable mistral mod_wsgi in undercloud  https://review.openstack.org/45848214:32
rlandyit's the fact that I issue a command directly, that I mess up the sync state14:32
rlandythis seems like a possible cause of problems14:33
*** anshul has quit IRC14:35
*** yolanda has joined #tripleo14:36
jkilpatrd0ugal, we'll see what the schedule is like for getting ~50 bm nodes, I'm going to try and work out a scale ci job with weshay sooner than that though.14:40
jkilpatrweshay, we never did get together last friday did we?14:40
weshayjkilpatr, I spoke to rlandy, the tenant is up and it looks like we're ready to try it.14:40
weshayjkilpatr, we need a topology file w/ the  nodes you want14:41
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Add ssh tunneling for TripleO-UI  https://review.openstack.org/40773314:41
weshayjkilpatr, oh sorry I forgot.. there are suprise.. introspection errors in rdo-cloud atm14:42
rlandyweshay: I am addressing that now14:42
d0ugaljkilpatr: great.14:42
d0ugaljkilpatr: I'll try and do some prep and testing so I am ready for it then :)14:42
jkilpatrweshay, funny that sounds like what d0ugal's patch is supposed to fix :P14:43
weshayjkilpatr, https://review.openstack.org/#/c/448805/20/config/general_config/ovb-minimal-pacemaker-public-bond.yml  line 2214:44
weshayjkilpatr, submit a review w/ the same but w/ more nodes.. we may need to adjust other settings so better to have a config for it14:45
weshayimho14:45
*** dtantsur|mtg is now known as dtantsur14:46
jkilpatrweshay, sounds good14:49
*** limao_ has quit IRC14:50
*** zoli is now known as zoli|brb14:52
openstackgerritMerged openstack/tripleo-ui master: Enable no-unneeded-ternary eslint rule  https://review.openstack.org/45931514:53
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors  https://review.openstack.org/44516314:54
*** rhallisey has quit IRC14:55
*** rhallisey has joined #tripleo14:56
*** jaganathan has joined #tripleo14:58
weshaysshnaidm, can you please spend some time and see if we have any outstanding reviews for phase2 https://review.openstack.org/#/q/status:merged+project:openstack-infra/tripleo-ci+branch:master+topic:oooq-t-phase214:59
weshaylooks all merged to me14:59
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time  https://review.openstack.org/43956814:59
openstackgerritDougal Matthews proposed openstack/tripleo-common master: TESTING small batch  https://review.openstack.org/44921515:00
rlandyweshay: jkilpatr should be able to start testing15:00
rlandyI can adjust the introspection change for him on the fly15:00
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Allow users to specify the Introspection batch size  https://review.openstack.org/45122815:00
weshayrlandy, and may he use the tripleo-ci tenant?15:00
sshnaidmweshay, it depends on what is included in phase-215:01
rlandyweshay: jkilpatr: yes - let's just chat to coordinate. I am using that tenant as well. is there no individual tenant for  browbeat usage?15:01
weshaysshnaidm, ok.. have a sec to chat about it?15:02
sshnaidmweshay, yep15:02
jkilpatrrlandy, not yet there's no browbeat stuff on rdo cloud yet15:02
jkilpatrdo we want one like that? can we give a tenant lower job priority or whatever15:02
rlandyjkilpatr: what do you need to run?15:03
*** dmacpher has quit IRC15:03
rlandydo you have creds?15:03
weshaysshnaidm, I'm on15:04
jkilpatrrlandy, for the cloud? No15:04
jkilpatrif you mean like cyberpunk currency, I've got some dogecoin15:04
weshayrlandy, he's looking to try ( I think ) a 20 node deployment15:04
openstackgerritChris Jones proposed openstack/tripleo-heat-templates master: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/45974315:05
rlandyjkilpatr: ok - you can work copy my poc jobs. when your job is ready, I'll give you the tenant creds15:06
openstackgerritChris Jones proposed openstack/puppet-tripleo master: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/45974415:06
rlandywe will need to coordinate. I am working in two tenants - including that one15:06
adarazsrlandy: can you move the TripleO CI Squad Weekly Mtg on this Thursday from 14:30 UTC to 15:30 UTC (so +1 hour) and make it last only 30 minutes? (this is only for this week as panda|pto is going to do a deep dive for Quickstart on the usual timeslot)15:07
jkilpatrrlandy, https://review.openstack.org/#/c/448805/20/config/environments/rdocloud.yml bmc is controller right?15:07
openstackgerritPhilip Choi proposed openstack/diskimage-builder master: Use python2 with RHEL6 in 50-dib-python-version  https://review.openstack.org/45974515:07
rlandyadarazs: don't you own that meeting?15:07
*** ansmith has quit IRC15:07
adarazsrlandy: nope, it seems not. Calendar: Ronelle Landy, Created by: Harry Rybacki15:08
rlandyadarazs: hmm ... ok - though Harry transferred that one to you ... moving meeting15:09
*** gkadam has quit IRC15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025915:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427215:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]15:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]15:10
rlandyadarazs: in fact this meeting should move to the RHOS meeting calendar and be on UTC. changing that first - then will update Thursday's meeting15:10
adarazsokay.15:10
*** dmarlin has joined #tripleo15:11
*** aufi has quit IRC15:13
rlandyadarazs: ugh - here we go again with moving meetings and timezone confusion15:15
*** yolanda has quit IRC15:15
adarazsrlandy: can you just recreate it completely?15:15
adarazsmaybe it's better :)15:15
rlandyadarazs: I am going to have to  - this is what happened last time15:15
rlandysomehow it allocates the time 1 hour off15:15
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Fix dependencies for PreConfig/PostConfig resources  https://review.openstack.org/45975115:15
adarazsyeah, looks like a bug to me.15:15
rlandyjkilpatr: will be with you in a bit - just reconfiguring calendar stuff15:16
shardyjistr: ^^ I think that fixes the circular dependency error15:20
shardyccamacho: ^^ FYI, review welcome15:20
jistrshardy: cool, thanks a lot! i'll pull it into the upgrade CI patch as a depends-on15:21
ccamachoshardy ack, sorry for breaking the docker steps15:22
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates stable/ocata: Deploy ceilometer_auth_enabled to node containing keystone  https://review.openstack.org/45975615:23
*** ansmith has joined #tripleo15:23
rlandyadarazs: can you check that you have the squad meeting showing up on your calendar (rhos meetings) at the right general time15:24
rlandyif so, I will move the specific meeting for this week15:24
adarazsrlandy: yes, looks correct.15:24
shardyccamacho: no worries, hopefully that fixes it up15:24
shardywe really, really need CI on it15:25
*** shardy is now known as shardy_afk15:25
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Support container upgrades in multinode OOOQ CI  https://review.openstack.org/45078415:26
*** zoli|brb is now known as zoli15:26
rlandyadarazs: ok - this week's meeting should be +1 hr - only 30 mins15:26
adarazsrlandy: looks correct. impossible. :)15:27
rlandywoohoo  - finally conquered google calendar15:27
jkilpatrrlandy, have you really? or have you just beaten it into submission for a day15:28
karimbhello, do we have some doc somewhere on how to configure a switch for a tripleo deployment ?15:28
*** vpickard is now known as vpickard_15:29
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Add ssh tunneling for TripleO-UI  https://review.openstack.org/40773315:29
rlandyjkilpatr: probably the latter - but I live to try another day!15:29
*** yolanda has joined #tripleo15:30
*** karimb has quit IRC15:30
*** limao has joined #tripleo15:31
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Add ssh tunneling for TripleO-UI  https://review.openstack.org/40773315:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - CI testing - Upgrades from Ocata to Pike  https://review.openstack.org/45760315:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - CI testing - Upgrades from Ocata to Pike  https://review.openstack.org/45760315:31
*** jmelvin has quit IRC15:31
*** jmelvin has joined #tripleo15:32
*** yprokule has quit IRC15:33
*** fzdarsky has quit IRC15:34
Ngcould I get some folk to look at https://review.openstack.org/#/c/459743/ and https://review.openstack.org/#/c/459744/ ? would be very thankful, this is going to be a downstream blocker I think :)15:35
jkilpatrrlandy, how much of this commit should I replicate? just put a new config in general_config maybe called "scale ci"15:35
jkilpatrhttps://review.openstack.org/#/c/448805/2015:35
jkilpatrthe flavors seem ok for now. might need to beef up the undercloud or controllers but I'm not sure yet.15:36
*** pblaho has quit IRC15:36
*** croelandt has joined #tripleo15:37
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: First draft of TripleO Release Management document  https://review.openstack.org/45976415:37
EmilienMmwhahaha, jrist: any feedback would be welcome ^15:37
EmilienMwhen you have time15:37
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: First draft of TripleO Release Management document  https://review.openstack.org/45976415:38
rlandyjkilpatr:one moment - getting introspection  set - then you'll have a working base to go off - will ping you in a few15:39
*** udesale has quit IRC15:39
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing oooq  https://review.openstack.org/45976615:43
croelandtIs there a CI for containerized deployments? I'm wondering whether Glance works well with all the backends defined in the tripleo templates (file, swift and rbd).15:46
jistrmatbu: i finally got all the way to overcloud upgrade started, i suspect it will fail because of the bug in docker-steps. Do you mind if i post the code as patches on top of the current container CI patches? We can either continue hashing it out and updating those patches then, or if they're bad (i'm not sure how much i'm following OOOQ expectaions) we can abandon them and merge different ones, but at least15:46
jistrwe'll have something posted, to discuss about the approach etc.15:46
*** nyechiel_ has quit IRC15:47
*** shardy_afk is now known as shardy15:49
*** fzdarsky has joined #tripleo15:50
*** saibarspeis has joined #tripleo15:50
*** agurenko has quit IRC15:50
jistrmatbu: e.g. i still can't find how master-undercloud-ocata-overcloud.yaml does the mixed version deploy. From my tracing through the OOOQ code, it looks like it uses the images that came with undercloud.qcow2... And my overcloud failed to deploy with those.15:50
EmilienMmwhahaha: fyi i'm EOLing tripleo mitaka right now15:53
mwhahahasounds fun15:53
mwhahaha:D15:53
EmilienMmwhahaha: it means we need to remove the jobs in zuul layout for puppet modules15:53
* bnemec has a sad that we're eoling a branch that's broken15:53
EmilienMbnemec: what is broken?15:54
mwhahahashould we fix it before EOLing then?15:54
bnemecEmilienM: The mitaka nonha jobs are all failing on a ceph issue right now.15:54
EmilienMgate-tripleo-ci-centos-7-ovb-ha-mitaka passed yesterday15:54
bnemecI was hoping it would be fixed by the centos-release change, but it doesn't seem to have been.15:54
EmilienMbnemec: what is the reason?15:54
EmilienMif it's tripleo, yeah let's fix it before15:54
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors  https://review.openstack.org/44516315:54
bnemecEmilienM: http://logs.openstack.org/84/450784/19/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/a7607fc/logs/postci.txt.gz#_2017-04-25_12_39_21_00015:55
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Add libvirt_cpu_model  https://review.openstack.org/45976815:55
bnemecI'm not sure what the problem is though.15:55
bnemecCould be that ceph changed something and broke us.15:56
mwhahahaare we getting the wrong ceph packages?15:56
mwhahahacauseit can't find the ceph service15:56
EmilienMto me, it's not something in tripleo itself15:56
EmilienMbut I'll dig into it15:56
EmilienMit sounds like ceph-10.2.3-0.el7.x86_64 is deployed15:57
*** brault has quit IRC15:57
rlandyjkilpatr: something is turning the baremetal instances off. I am watching for that. ipmi validations have changed. I disabled them for the moment15:57
EmilienMmwhahaha: in puppet ci, we run ceph-0.94.9-0.el7.x86_64 (in stable/mitaka)15:58
EmilienMmwhahaha: probably another source15:58
jkilpatrrlandy, whats new in the ipmi validations?15:59
EmilienMceph-10.2.3-0.el7.x86_64 is jewel, isn't?15:59
mwhahahayea15:59
*** trown is now known as trown|lunch15:59
EmilienMah that's why15:59
EmilienMprobably something in tripleo-ci, let me look16:00
rlandyjkilpatr: https://github.com/rthallisey/clapper - we're calling it before instacknev. json is imported16:00
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Use mixed upgrade in upgrade_to_containers  https://review.openstack.org/45977016:00
*** ykarel has quit IRC16:00
EmilienMah found it16:01
EmilienMit's even my fault16:01
*** janki has quit IRC16:01
* mwhahaha blames EmilienM 16:01
*** trozet has quit IRC16:01
EmilienMyou can16:01
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: repo: allow to download hammer on mitaka  https://review.openstack.org/45977316:02
EmilienMbnemec, mwhahaha  ^ let's see16:02
*** karimb has joined #tripleo16:02
mwhahahaoh you16:03
EmilienMyes...16:03
*** pblaho has joined #tripleo16:03
bnemecFunny, I'm pretty sure I complained about the custom repo files on that review... ;-)16:04
*** mdnadeem has quit IRC16:05
*** jpich has quit IRC16:06
bnemecHmm, did we merge that with failing ci then?  Naughty.16:06
*** jaganathan has quit IRC16:06
*** limao has quit IRC16:06
bnemecNope, it passed: https://review.openstack.org/#/c/458474/1816:07
bnemecWeird.16:07
* bnemec will miss the 78 minute mitaka nonha jobs16:07
mwhahahahmm weird ineed16:08
mwhahahasince that job had the hammer ceph16:08
mwhahahamaybe that's not the fix16:08
EmilienMmwhahaha, bnemec : my patch worked because mitaka images are built nightly IIUC16:09
EmilienMso I don't even think my patch will be tested16:09
bnemecOh, right.16:09
EmilienMbut I'm not sure16:09
openstackgerritMerged openstack/instack-undercloud master: Rely on puppet-ironic to install required packages  https://review.openstack.org/45918216:09
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix path to overcloud_containers.yaml  https://review.openstack.org/45967516:09
mwhahahaoh death by images16:09
bnemecSo we had a bogus ceph repo on the undercloud, but since we don't use ceph there it didn't matter.16:09
EmilienMsshnaidm: can you confirm what I said? /me not sure16:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025916:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427216:10
*** ooolpbot has quit IRC16:10
*** limao has joined #tripleo16:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]16:10
bnemecWe actually should build images in jobs against tripleo-ci.16:10
EmilienManyway, I'll continue the work to EOL mitaka and wait until we see all mitaka jobs green16:10
*** chlong has quit IRC16:10
EmilienMbnemec: yes, I agree16:10
bnemecBut I think that would completely kill our ci capacity. :-/16:10
EmilienMhow much time does it take?16:10
*** zoli is now known as zoli|gone16:11
bnemecI think ~25 minutes in rh1.16:11
bnemecI haven't looked recently though.16:11
EmilienMit's not that often that this situation happens, tbh and we catch it pretty quickly16:11
EmilienMyeah 25 min is a lot :/16:11
bnemecYeah, unless it's a stable branch nobody cares about. :-)16:11
sshnaidmEmilienM, not sure I understand, what to confirm..?16:12
bnemecBut I agree it's a ci hole we just have to live with.  In an ideal world we would build images though.16:12
EmilienMsshnaidm: that patches against tripleo-ci doesn't build images for mitaka jobs, so if we break something, it will be detected the day after, since we build mitaka images nightly16:12
*** limao_ has joined #tripleo16:12
sshnaidmbnemec, images building take about 40 min16:13
*** snecklifter has joined #tripleo16:13
snecklifterHello, would appreciate a review on https://review.openstack.org/#/c/313933/ if anyone has a moment, thanks16:13
sshnaidmEmilienM, yes, we build them nightly. But what kind of breaking are we talking..?16:14
*** diana_clarke has left #tripleo16:14
*** jcoufal has quit IRC16:14
*** limao has quit IRC16:15
*** yolanda has quit IRC16:15
EmilienMsshnaidm: https://review.openstack.org/45977316:16
*** jcoufal has joined #tripleo16:16
EmilienMsshnaidm: last week, we switched tripleo-ci to use AFS mirrors and I did a mistake ^16:16
*** hewbrocca is now known as hewbrocca_afk16:17
*** owalsh_ has joined #tripleo16:17
*** dspano has joined #tripleo16:19
*** dparkes has quit IRC16:19
*** lmiccini has quit IRC16:20
*** owalsh has quit IRC16:20
openstackgerritMerged openstack/puppet-tripleo stable/ocata: SSHD Service extensions  https://review.openstack.org/45882716:20
*** snecklifter has quit IRC16:22
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Stop SSHD profile clobbering SSH client config  https://review.openstack.org/45882816:22
*** chlong has joined #tripleo16:23
*** lucasagomes is now known as lucas-afk16:27
*** nyechiel_ has joined #tripleo16:31
*** ebarrera has joined #tripleo16:31
bnemecsshnaidm: Graphite says our image builds are taking around 1600 seconds, or a little less than 27 minutes.  The slow jobs are around 30.16:31
*** limao_ has quit IRC16:31
bnemecIt does vary somewhat based on the age of the base centos image.16:31
bnemecThe older it is, the more packages have to be updated during the build.16:32
*** yolanda has joined #tripleo16:32
*** cylopez has quit IRC16:32
sshnaidmbnemec, what is the address of graphite?16:35
bnemecsshnaidm: https://66.187.229.172/16:35
*** fzdarsky has quit IRC16:37
*** pcaruana has quit IRC16:39
*** japestinho has quit IRC16:40
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: DONT REVIEW: used for testing only  https://review.openstack.org/44727616:43
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test image building  https://review.openstack.org/45978616:43
*** yolanda has quit IRC16:44
openstackgerritChris Jones proposed openstack/puppet-pacemaker master: Add support for Pacemaker Remote to stonith drivers.  https://review.openstack.org/45978716:45
*** mhenkel_ has joined #tripleo16:46
openstackgerritChris Jones proposed openstack/puppet-tripleo master: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/45974416:47
openstackgerritChris Jones proposed openstack/tripleo-heat-templates master: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/45974316:48
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Update Gemfile to pull spec_helper from stable/ocata  https://review.openstack.org/45938316:48
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: SSH known_hosts config  https://review.openstack.org/45867216:48
openstackgerritMerged openstack/tripleo-heat-templates master: Deploy ceilometer_auth_enabled to node containing keystone  https://review.openstack.org/45938816:49
openstackgerritMerged openstack/tripleo-heat-templates master: Containerize etcd service  https://review.openstack.org/44762716:49
Ngshardy: thanks for the reviews, sorry I jumped the gun with the description on one of them ;)16:50
Ngshardy: I fixed the Depends-On order (and there's now a third patch)16:50
EmilienMbnemec: maybe we could give it some love and create a DNS entry, like graph-ci.tripleo.org or something cc dprince16:51
rlandydtantsur: bnemec: I watched the baremetal instances from undercloud install through introspection. The state of the instances is in sync. Introspection starts - powers off the instances and then should reboot, but the instances never start again. if I then start the instances with ironic node-set-power state, introspection continues. So the issue is that reboot during introspection does not complete16:51
*** jaganathan has joined #tripleo16:52
*** jlinkes has quit IRC16:52
* dtantsur is chairing a huge meeting, sorry16:53
*** dtantsur is now known as dtantsur|mtg16:53
*** ebarrera has quit IRC16:55
dprinceEmilienM: let me know what you need and I can add it16:56
*** bogdando has quit IRC16:57
EmilienMdprince: something for 66.187.229.172, something we can easily remember :)16:58
EmilienMmaybe just graphite.tripleo.org16:58
*** rhallisey has quit IRC16:59
*** rhallisey has joined #tripleo17:00
rlandybnemec: I can reproduce that. if a baremetal instance is running and I issue ''ironic node-set-power-state <ID> reboot', the instance will shut down and not start up again.17:02
dprinceEmilienM: BOOM: graphite.tripleo.org17:03
bnemecrlandy: Weird.  Do you see a log message in the bmc that it got a request to start it back up?17:04
*** tosky has quit IRC17:04
EmilienMnice17:04
EmilienMdprince: thx!17:04
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart master: TripleO CI mixed release master UC / ocata OC  https://review.openstack.org/45978917:05
Slowerlarsks: let us know what you find with the logging stuff17:06
Slowerlarsks: we definitely need to deal with this asap17:06
rlandybnemec: I see the error about power off timing out. where is the bmc-specific log?17:07
Slowerlarsks: and I think it's probably good enough to document where they are for a given service even17:07
rlandyoh - on the instance17:08
*** tdasilva has quit IRC17:08
bnemecrlandy: The bmc console log, or the journal on the bmc.  It should be logging to both places.17:08
pradkcan i get one more review and merge this please https://review.openstack.org/#/c/442151/17:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025917:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427217:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]17:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]17:10
jaosoriorhrybacki: commented https://review.openstack.org/#/c/436198/1517:10
rlandybnemec: looking at the console log, I see some traces of ''[ 4109.810075] openstackbmc[2392]: 4fa474d3-d50f-4eb4-aa4c-2a7f74067884 is already off. - running reboot again to follow exact traces for this action17:12
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Use mixed upgrade in upgrade_to_containers  https://review.openstack.org/45977017:12
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart master: TripleO CI mixed release master UC / ocata OC  https://review.openstack.org/45978917:12
*** nyechiel_ has quit IRC17:13
*** agurenko has joined #tripleo17:14
*** tellesnobrega has joined #tripleo17:15
tellesnobregadprince, hey, I saw that you picked up the upstream sahara bug about containerization... we (good folks from sahara) are curious on how is that going and if you have done any progress into that or if you need us to do anything on this matter17:16
*** pkovar has quit IRC17:16
dprincetellesnobrega: no progress yet. I emailed egafford and got stalled out on it. Want to take it over?17:17
tellesnobregadprince, I can help out, egafford is out of medical leave, I don't have much experience with that, but if can give me some tips I can try to take over17:18
*** saibarspeis has quit IRC17:18
*** yamahata_ has quit IRC17:18
*** tdasilva has joined #tripleo17:19
*** jaosorior is now known as jaosorior_away17:19
dprincetellesnobrega: I can help too. Want to setup a meeting to go over things17:20
tellesnobregadprince, that would be great17:20
*** salmankhan has quit IRC17:23
*** mhenkel__ has joined #tripleo17:23
*** mhenkel_ has quit IRC17:26
*** mhenkel__ has quit IRC17:27
*** trown|lunch is now known as trown17:28
rlandybnemec: later reboots do work - I see some traces of Ignoring exception: "Cannot 'stop' instance while it is in task_state powering-off (HTTP 409)". The ipxe-boot image was uploaded with ' --property os_shutdown_timeout=5 '.17:29
bnemecrlandy: That's a little surprising, but not necessarily a problem.17:32
jristEmilienM: was planning on it, thanks17:32
*** mhenkel_ has joined #tripleo17:33
*** mhenkel_ has quit IRC17:33
openstackgerritMerged openstack/puppet-tripleo master: Enable internal network TLS for etcd  https://review.openstack.org/45488217:33
openstackgerritMerged openstack/os-apply-config stable/ocata: Updated from global requirements  https://review.openstack.org/44385117:34
openstackgerritMerged openstack/os-collect-config stable/ocata: Updated from global requirements  https://review.openstack.org/44385417:34
*** mhenkel_ has joined #tripleo17:34
openstackgerritMerged openstack/os-refresh-config stable/ocata: Updated from global requirements  https://review.openstack.org/44385517:34
*** owalsh_ is now known as owalsh17:35
rlandybnemec: I don't know why this is an issue now - we were pinning jobs last week without any problems. The new images were uploaded then but they don't look different17:36
rlandyspinning17:36
honzaI'm getting cors rejections from the ironic api --- is that a new thing?17:36
*** chlong has quit IRC17:37
*** fragatina has quit IRC17:37
*** yamahata_ has joined #tripleo17:39
rlandyjkilpatr: ping - we can work on your jobs now if youwish17:40
jkilpatrrlandy, sounds good. There isn't really a job definition yet, what's managing jobs on rdo cloud jenkins I presume or is it zuul yet?17:41
bnemecrlandy: I'm not sure either.  I haven't made any significant changes to ovb in at least two weeks.17:43
rlandybnemec: weird - and the failure is consistent.17:43
rlandyjkilpatr: right now - jenkins17:44
rlandyzuul in the future when we put upstream ci on rdocloud17:44
jkilpatrrlandy, cool where are we keeping job definitions?17:44
rlandyjkilpatr: there is a devmode equivalent17:44
rlandywhich will become what jenkins runs17:45
honzajtomasek: florianf: jrist: ^^^17:45
jkilpatrrlandy, how is devmode being handled anyways? I mean what are the rules17:45
rlandyjkilpatr: https://review.openstack.org/#/c/452249/17/ci-scripts/full-deploy-ovb.sh17:46
rlandythat is the basic workflow17:46
rlandyyou can run devmode --ovb17:46
rlandyor call full-deploy-ovb.sh from jenkins17:47
rlandysome deal17:47
rlandyjkilpatr: what do you want to set up now? a poc to test?17:47
jkilpatrrlandy, yup, just want a poc job with like 30 nodes, provided that's not going to be a problem to do now17:47
rlandyjkilpatr; ok - I set that up for you17:49
rlandyjkilpatr: I am going to send you some links in pvt (internal jenkins) and how to access the slaves set up on the tenant17:50
*** brault has joined #tripleo17:50
*** jpena is now known as jpena|off17:50
jkilpatrrlandy, thanks!17:50
rlandywhen we get one consistently passing run, you can trigger the job every couple minutes  - will get you 30 concurrently running jobs17:51
rlandypls see pvt17:51
*** chlong has joined #tripleo17:52
*** brault has quit IRC17:54
*** morazi has quit IRC17:56
*** mhenkel__ has joined #tripleo17:56
*** mhenkel__ has quit IRC17:57
*** mhenkel_ has quit IRC17:57
*** mhenkel_ has joined #tripleo17:57
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: add steps to enable tripleo_ui  https://review.openstack.org/45906117:58
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Add ssh tunneling for TripleO-UI  https://review.openstack.org/40773317:59
*** mhenkel_ has quit IRC18:00
*** mhenkel_ has joined #tripleo18:01
*** mhenkel_ has quit IRC18:02
*** paramite has quit IRC18:02
*** mhenkel_ has joined #tripleo18:03
*** dprince has quit IRC18:03
*** gbarros has quit IRC18:04
*** gbarros has joined #tripleo18:04
tbarronfultonj_: when you have a moment, are you a good person to ask about HCI deployments for NFV customers?  Specifically,18:05
tbarronfultonj_: I'm trying to determine what sort of 'openstack deploy ...' command would be used so I can understand the node and network topology18:06
tbarronfultonj_: as we look at deploying manila with CephFS backend ...18:06
dtrainorhonza, i've not seen that.  how are you trying to talk to ironic api?18:07
honzadtrainor: From the GUI, normal ajax requests18:07
*** gbarros has quit IRC18:07
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/mitaka: mitaka: 1.1.0 EOL  https://review.openstack.org/45982418:08
dtrainorhuh18:08
fultonj_hi tbarron i think i can help. i'm not NFV expert but I have done HCI with people doing NFV work.18:09
*** fultonj_ is now known as fultonj18:09
fultonjtbarron: i'll pm you18:09
tbarronfultonj: excellent.  thanks.18:10
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025918:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427218:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]18:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]18:10
*** ebarrera has joined #tripleo18:10
*** amoralej is now known as amoralej|off18:11
dtrainorhonza, is there anything atypical about the cors errors themselves?  maybe something to indicate something was changed?18:13
honzadtrainor: It's missing all of the Access-* headers.18:13
honzadtrainor: I checked /etc/ironic/ironic.conf for any weirdness but didn't see any18:14
dtrainorweird, default is allowed_origin = *18:15
dtrainorcan you grep allowed_origin /etc/ironic/ironic.conf18:15
*** athomas has quit IRC18:16
honzadtrainor: it's *, all other occurrences are commented out18:17
dtrainorjust for kicks... can you: sudo systemctl restart openstack-ironic-api.service18:17
honza... yeah, I've been doing that18:18
honzano change18:18
dtrainoris there anything in /var/log/ironic/ironic-api.log that coincides with the failed request?18:19
honzaLet me try to curl it from the undercloud18:19
*** fragatina has joined #tripleo18:19
honzasame issue18:20
*** gbarros has joined #tripleo18:22
honzadtrainor: There is a keystone rejection right before the OPTIONS request18:22
dtrainorcan you paste the part in context of that log?18:22
honzadtrainor: http://paste.openstack.org/show/607906/18:22
dtrainorone step ahead of me, thx18:22
honzadtrainor: I cleared the log, reloaded the page, and this is what appeared18:22
jtomasekhonza: I was seeint it too with quickstart setup18:23
jtomasekhonza: I am actually still seeing it...18:24
dtrainorwonder if it has to do with keystone versioning on the endpoint....18:24
dtrainorcan you grep auth_url /etc/ironic/ironic.conf18:24
*** pkovar has joined #tripleo18:24
dtrainoroops:   auth_uri /etc/ironic/ironic.conf18:25
dtrainorsigh.  this:  grep auth_uri /etc/ironic/ironic.conf18:25
honzadtrainor: auth_uri = http://192.168.24.1:5000/v318:25
honzajtomasek: I started happening right I commented out all of the endpoints in tripleo_ui_config like you suggested so I figured that it was related...18:26
honzabut doesn't seem to be18:26
dtrainoranything fun under /var/log/keystone/keystone.log to indicate why keystone rejected the request?18:27
honzanope, don't think so18:27
*** gbarros has quit IRC18:27
honzaonly a bunch of INFOs mapping to requests18:28
honzamay not be specific to oooq --- i installed mine via ansible-ovb18:28
dtrainoranything suspicious in /var/log/httpd/keystone*.log18:29
*** dcritch has joined #tripleo18:29
honzadtrainor: empty18:30
dcritchhey gang, has anyone tried to get tripleo-quickstart to work w/ official red hat packages instead of rdo?18:30
*** mhenkel__ has joined #tripleo18:34
*** gbarros has joined #tripleo18:34
EmilienMdcritch: yes, and it has been documented, let me find the link18:36
*** shardy has quit IRC18:36
*** gbarros has quit IRC18:37
*** mhenkel_ has quit IRC18:37
*** gbarros has joined #tripleo18:37
*** mhenkel__ has quit IRC18:39
*** ebarrera has quit IRC18:39
*** dprince has joined #tripleo18:39
honzaJust for fun, I tries it with our new keystone v3 patch, and the same thing happens there18:39
dtrainori'm out of ideas honza18:42
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: Add config for many node RDO cloud job  https://review.openstack.org/45983018:42
honzadtrainor: thanks for your help, much appreciated18:42
dtrainormy pleasure18:42
*** rbowen has quit IRC18:42
*** tellesnobrega has quit IRC18:43
openstackgerritMerged openstack/tripleo-common stable/ocata: Add MigrationSshKey to generated passwords  https://review.openstack.org/45867118:45
*** jaganathan has quit IRC18:45
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Add migration SSH tunneling support  https://review.openstack.org/45867318:45
openstackgerritMerged openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html", move report to ara_oooq  https://review.openstack.org/44810818:45
openstackgerritMerged openstack/os-collect-config stable/ocata: Update .gitreview for stable/ocata  https://review.openstack.org/42634418:46
openstackgerritMerged openstack/os-net-config stable/ocata: Update .gitreview for stable/ocata  https://review.openstack.org/42634518:46
*** jprovazn has quit IRC18:47
*** agurenko has quit IRC18:47
*** mhenkel_ has joined #tripleo18:49
*** derekh has quit IRC18:54
*** dtantsur|mtg is now known as dtantsur|afk18:58
openstackgerritAndy Smith proposed openstack/puppet-tripleo master: WIP qpid-dispatch-router mesh  https://review.openstack.org/45983518:58
*** mhenkel_ has quit IRC19:00
*** mhenkel_ has joined #tripleo19:01
*** tellesnobrega has joined #tripleo19:01
jristhonza: just got done with meetings/calls19:03
jristhonza: you all squared away now?19:03
jristhonza: what build?19:04
honzajrist: nope :(19:04
jristoh19:04
jristansible-vb?19:04
jristsorry19:04
jristansible-ovb?19:04
honzayes19:05
*** dbecker has quit IRC19:05
honzaI'm trying to rebuild my undercloud but I'm hitting an OVB bug it seems.19:05
*** mhenkel_ has quit IRC19:05
jrist:( :( :(19:05
jristok19:05
jristso19:05
jristI don't think I can help at this time,m is that correct?19:05
jristjtomasek: you are hitting the same thing with quickstart?19:06
honzaI'm taking a break from it all, and will try again in a bit.  The brain might work then.19:06
*** tellesnobrega has quit IRC19:08
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: Show all roles in inventory  https://review.openstack.org/45023319:08
jristnoted19:08
*** ebarrera has joined #tripleo19:08
dtrainormwhahaha, didn't you work on ansible-ovb?  does this smell familiar?19:08
mwhahahaque?19:09
mwhahahaansible-ovb is mine19:09
EmilienMwow19:09
jristso smelly19:09
EmilienMit's YOURS?19:09
jristNOW YOU KNOW19:09
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Remove no longer used environment files - older upgrade workflows  https://review.openstack.org/45931919:10
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025919:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427219:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]19:10
mwhahahaಠ_ಠ19:11
mwhahahawhat ovb bug, scrolling fails to give me context19:12
jristmwhahaha: appears to be CORS issue with Ironic19:15
jrist<honza> I'm getting cors rejections from the ironic api --- is that a new thing?19:15
jrist<honza> dtrainor: From the GUI, normal ajax requests19:15
jrist<honza> dtrainor: It's missing all of the Access-* headers.19:16
honzamwhahaha: http://paste.openstack.org/show/607913/19:16
mwhahahaso i assume this is  from the UI?19:16
jristmwhahaha: context starting at 12:12 our time19:16
jristmwhahaha: yeah precisely19:16
mwhahahai have never successfully used the ui19:16
mwhahahaso i am no help :D19:16
*** florianf has quit IRC19:17
mwhahahabut that seems weird that baremetal would be a client object19:17
jristthat's false!19:18
jristyou will be immensely helpful19:18
jristI Contend19:18
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: Add config for many node RDO cloud job  https://review.openstack.org/45983019:18
mwhahahahttps://github.com/cybertron/openstack-virtual-baremetal/blob/master/openstack_virtual_baremetal/build_nodes_json.py#L16319:21
mwhahahafinally found where that code comes f rom19:21
mwhahahaso i guess nova.servers.get isn't returning image correctly with it19:21
*** ebarrera has quit IRC19:22
mwhahahastill not my world of experiance, maybe bnemec might have an idea19:22
*** milan is now known as milan|zzZZZzzz19:22
jristcommendable effort mwhahaha :)19:22
bnemecWait, what?  That doesn't call ironic at all.19:22
bnemecOh, http://paste.openstack.org/show/607913/19:23
openstackgerritMerged openstack/os-apply-config stable/ocata: Update .gitreview for stable/ocata  https://review.openstack.org/42634219:24
openstackgerritMerged openstack/python-tripleoclient master: Remove log translations  https://review.openstack.org/44907119:24
*** morazi has joined #tripleo19:24
dtrainormwhahaha, sorry hah had a call19:24
dtrainorsorry to make you wait19:24
dtrainorat the end of the day, if all my other attempts fail, at least i'm a good networker :)19:24
bnemecOh, I bet novaclient removed the images proxy.19:24
bnemecjrist: mwhahaha: ^19:24
bnemecThat should be using glanceclient.19:25
* bnemec looks at a fix19:25
mwhahahaif only there was a unified sdk19:25
* mwhahaha trolls his way out19:25
honzato clarify: I had an ironic issue with CORS.  In an attempt to fix it, I'm rebuilding my OVB stack.  When building nodes for the stack, I see that images error in the past.19:26
dtrainormwhahaha++19:26
dtrainorcool, honza, confident in what we're seeing?19:26
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart-extras master: Allow complex upgrade deployment for N to O  https://review.openstack.org/43959819:27
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart-extras master: Allow complex upgrade deployment for N to O  https://review.openstack.org/43959819:27
*** tellesnobrega has joined #tripleo19:27
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart-extras master: Download rpm tht package for mixed upgrade  https://review.openstack.org/44935019:28
honzapaste*19:28
honzadtrainor: I don't know what you're implying :)19:28
bnemecOh, I didn't write that code.  Not my fault. \o/ :-)19:28
bnemecExcept not catching it in review...19:28
bnemechonza: Okay, pull the latest ovb and try again.  You should be unblocked now: https://github.com/cybertron/openstack-virtual-baremetal/commit/2fe2fd0569a8ae2e90802d7a1d43a1166eeb74b119:33
bnemecI'll fix it properly as a followup.19:34
*** dspano has quit IRC19:37
*** yolanda has joined #tripleo19:38
EmilienMjrist: please look https://review.openstack.org/#/c/459061/ when you can, and give me feedback on my -1 (cc weshay)19:43
weshayEmilienM, I'm only changing it for the libvirt tunnel19:44
weshayEmilienM, libvirt ssh tunnel19:44
weshayEmilienM, this doesn't have anything to do w/ the project/product19:44
weshaythe config.js is managed by puppet19:45
EmilienMok19:45
EmilienMgood then, +219:45
weshaylolz :)19:45
EmilienMI just feel bad everytime I +2 weshay's patches :P19:45
weshayit's dirty right19:45
*** chlong has quit IRC19:46
weshayEmilienM, go take a shower, you will feel better19:46
weshayEmilienM, mwhahaha the majority of that patch is in https://review.openstack.org/#/c/407733/19:46
mwhahahaisn't there proxy support in the ui?19:47
EmilienMif I would take a shower everytime I +2 a dirty patch in OpenStack, we wouldn't have water anymore in Canada19:47
mwhahahai thought dtrainor did something19:47
EmilienMand trust me, we have millions of lakes here19:47
weshaylol19:47
dtrainorya was it one of mine?  that prob broke it19:47
mwhahahathat seems like an weird hack to implement something that was already  in there19:47
mwhahahaproxying via apache19:47
mwhahahadtrainor: didn't you implement an improvement on service proxying for the ui with apache?19:48
dtrainori did yes - sorry i'm in a meeting, trying to read backlog19:48
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Deploy ceilometer_auth_enabled to node containing keystone  https://review.openstack.org/45975619:48
openstackgerritMerged openstack/os-refresh-config stable/ocata: Update .gitreview for stable/ocata  https://review.openstack.org/42634619:48
EmilienMmwhahaha: I can hold this one19:48
weshayso this is enabling tunneling via one or two hops19:48
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add option to build delorean changes and include the repo  https://review.openstack.org/43400419:48
mwhahahaweshay: right the thing is there's already something to reduce that complexity19:49
dtrainorsorry, i need a few minutes before i can look at this19:49
mwhahahawe may still need the tunnel, but it might be a simplier configuration19:50
openstackgerritMerged openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/45603019:50
mwhahahai'd wait for dtrainor to review19:50
openstackgerritMerged openstack/puppet-tripleo stable/mitaka: mitaka: 1.1.0 EOL  https://review.openstack.org/45982419:51
openstackgerritMerged openstack/tripleo-ui master: Include Deployed Nodes in NodesAssignment counts  https://review.openstack.org/45533619:51
weshaydtrainor, I added you to both patches19:54
dtrainorappreciate it thank you19:54
*** chlong has joined #tripleo19:56
jristmwhahaha: I meant to ask the basis for your freenode nick19:57
jristEmilienM: will do19:57
honzabnemec: \o/19:57
honzabnemec: thank you19:57
jristnice19:57
jristprogress19:57
mwhahahajrist: oh its just what i've used for like 15+ years19:57
*** iranzo has quit IRC19:58
*** dsariel has quit IRC19:59
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Add overcloud support report command  https://review.openstack.org/40500120:00
openstackgerritAlex Schultz proposed openstack-infra/tripleo-ci master: Use support report command to collect node logs  https://review.openstack.org/41856220:00
*** jkilpatr has quit IRC20:10
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025920:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427220:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]20:10
*** tellesnobrega has quit IRC20:15
*** gbarros has quit IRC20:23
*** dsariel has joined #tripleo20:28
openstackgerritBen Nemec proposed openstack/python-tripleoclient stable/mitaka: Test mitaka with image build  https://review.openstack.org/45985220:31
*** mhenkel_ has joined #tripleo20:32
*** jayg is now known as jayg|g0n320:33
dtrainorOk, so, re: https://review.openstack.org/#/c/459061/20:37
dtrainoroh this is part of tripleo-quickstart-extras20:38
mwhahahayea so it's for deploying it via quickstart20:39
dtrainorthinking20:39
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Deprecate ceilometer collector  https://review.openstack.org/45088520:39
mwhahahaso it's to add support for a tunneled UI, my question is the support you added for teh apache proxy, does that change the way this should be implemented20:39
*** gbarros has joined #tripleo20:40
weshaydtrainor, other half of that https://review.openstack.org/#/c/407733/20:40
dtrainortripleo itself doesn't care, the only part that cares is the browser, so those endpoint URLs can be whatever is appropriate for your environment.  if it's a standard ssh tunnel then 'localhost' is sufficient, but how do you tell quickstart that it's going to be using tunneling or not?  wouldn't it need to know that in order to change the contents of the config?20:40
weshaydtrainor, the tunnel is turned on if you are using libvirt undercloud only20:40
dtrainorok then that's the magical parameter we were looking for a whilea go20:41
weshayit works if you are installing directly from the virt host or remotely20:41
dtrainorperfect20:41
*** ebarrera has joined #tripleo20:42
*** karimb has quit IRC20:42
dtrainorbackground on the "apache tunnel" - haproxy accepts all https connections for the endpoints in their whole request forms and forwards them to an Apache host listening on the admin network:3000 (non-ssl), so that the request is made to https://undercloud:443/zaqar for example and then mod_proxy via ProxyPass/ProxyPassReverse directive then forwards the request to the admin IP of the service being requested, as filtered by the /zaqar par20:43
dtrainort of the uri20:43
weshayah ok20:43
weshayI saw that in the backup file20:43
dtrainoras if to say, anything non-ssl that connects to any service on the admin network can make a direct connection to that service and is unaffected in any sense by haproxy20:44
dtrainorclear as mud?20:44
dtrainorno jk is there any other part of that i can help explain?20:44
weshayso question.. I tried this five or six times testing w/ and w/o ssl.  deploying through the ui and cli to see if everything worked properly..20:46
*** ansmith has quit IRC20:46
weshaydoes the apache tunnel only get setup on ssl deployments or any?20:46
weshaydtrainor, can you also review https://review.openstack.org/#/c/407733/20:47
*** jkilpatr has joined #tripleo20:47
*** trown is now known as trown|outtypewww20:53
*** rhallisey has quit IRC20:54
*** ebarrera has quit IRC20:54
dtrainorany20:56
dtrainorcool, i like it, good job20:58
dtrainori need to run for an appointment20:58
*** brault has joined #tripleo20:58
*** gbarros has quit IRC20:59
*** jcoufal has quit IRC21:02
*** brault has quit IRC21:03
*** dprince has quit IRC21:08
*** mhenkel__ has joined #tripleo21:09
*** mhenkel__ has quit IRC21:09
*** mhenkel__ has joined #tripleo21:09
*** mhenkel_ has quit IRC21:10
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025921:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427221:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]21:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]21:10
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing oooq  https://review.openstack.org/45976621:18
*** jmelvin has quit IRC21:23
rlandybnemec: here is the bmc console output through the introspection reboot: https://paste.fedoraproject.org/paste/L~2U~wRp-JIOCPnCeRxitF5M1UNdIGYhyRLivL9gydE= .. you will see the power off - but no power on21:23
rlandyweshay: ^^ fyi21:24
honzaSo, rebuilding my undercloud didn't change anything in terms of my ironic CORS issue :(21:25
bnemecrlandy: Hmm, so I believe that means it didn't get a request from Ironic to turn it back on.21:25
bnemecYet ironic can see that the nodes are off, so it shouldn't be waiting for anything.21:26
rlandybut this should be a reboot - power off, power on21:26
rlandyhttps://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/introspect.py#L16321:27
rlandybmc console does keep showing new lines of:21:28
rlandy[ 2316.235815] openstackbmc[2395]: Getting power state for 6132d8ff-0842-4def-939f-84b8a3a2e49b21:28
rlandy[ 2316.268485] openstackbmc[2425]: Getting power state for 6371bfed-0fab-4411-8490-35194ee3df9021:28
rlandyso it is checking the power state21:28
*** chlong has quit IRC21:28
rlandydoen't think it needs to do anything more21:29
*** abishop has quit IRC21:29
*** Goneri has quit IRC21:31
openstackgerritEmilien Macchi proposed openstack/instack-undercloud master: undercloud/upgrade: change ownership for ironic-dbsync.log  https://review.openstack.org/45987121:34
bnemecrlandy: Yeah, ironic polls periodically for power status, so it's normal to see a lot of repeated Getting power state messages.21:36
EmilienMhttps://review.openstack.org/459871 should fix undercloud upgrade job, I don't know if we have a launchpad for that21:37
bnemecrlandy: Hmm, maybe it's calling the not implemented power_reset function now? https://github.com/cybertron/openstack-virtual-baremetal/blob/master/openstack_virtual_baremetal/openstackbmc.py#L15621:38
*** milan|zzZZZzzz has quit IRC21:38
openstackgerritEmilien Macchi proposed openstack/instack-undercloud master: undercloud/upgrade: change ownership for ironic-dbsync.log  https://review.openstack.org/45987121:40
*** brault has joined #tripleo21:41
rlandybnemec: hmmm ... if I run ironic node-set-power-state <ID> reboot, I can reproduce it - power off in the log, no power on. iow, the log does not show 'reboot" (which I thought it might), it just hangs out after power off21:42
bnemecrlandy: In the ironic logs or the bmc logs?21:45
rlandybnemec: in bmc logs21:45
*** brault has quit IRC21:45
bnemecrlandy: There's nothing in the bmc to handle a reboot command though.21:46
bnemecAlthough it's strange that power_off would get called if it's using a reboot command.21:46
bnemecrlandy: You could maybe try adding a log call to that power_reset function on your bmc and see if that's getting called.21:48
rlandybnemec: since this is only showing up on rdocloud, is there anything that could be environment specific?21:49
rlandyseems weird to show up in one env all of a sudden - when we had things running nicely last week21:49
bnemecrlandy: I don't know what it would be.  Ironic can clearly talk to the bmc, which is usually the problem you run into if the environment is wrong somehow.21:50
rlandyno changes to OVB21:50
bnemecrlandy: What repos are you using for the openstack bits?  Is it possible Ironic made a change that is incompatible with ovb and we haven't hit it yet in tripleo-ci because it won't promote?21:51
openstackgerritDan Sneddon proposed openstack/os-net-config stable/ocata: os_net_config should map nics that are down if nic is in mapping file  https://review.openstack.org/45987421:51
openstackgerritDan Sneddon proposed openstack/os-net-config stable/newton: os_net_config should map nics that are down if nic is in mapping file  https://review.openstack.org/45987521:51
rlandybnemec: release master (https://github.com/openstack/tripleo-quickstart/blob/master/config/release/master.yml) - I should try new to check - will do that now21:52
rlandynewton21:52
bfourniedsneddon: thanks for merging to stable branches21:54
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Remove existing Ceph repo on host  https://review.openstack.org/45987621:54
dsneddonbfournie, no problem, thanks for the patch21:54
dsneddonbfournie, Can you add a few reviewers?21:55
bfourniedsneddon: sure21:55
*** owalsh_ has joined #tripleo21:55
*** oneswig has joined #tripleo21:56
*** owalsh has quit IRC21:58
rlandyand now we lost rdocloud :(22:01
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: sensu: fix upgrade case when service is added  https://review.openstack.org/45922422:04
openstackgerritMerged openstack/tripleo-heat-templates master: Enable internal network TLS for etcd  https://review.openstack.org/45510422:05
*** owalsh_ has quit IRC22:05
openstackgerritMerged openstack/tripleo-heat-templates master: Fix dependencies for PreConfig/PostConfig resources  https://review.openstack.org/45975122:05
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix path to overcloud_containers.yaml  https://review.openstack.org/45865022:05
openstackgerritMerged openstack/tripleo-quickstart-extras master: Make docker_registry_namespace a variable  https://review.openstack.org/45857422:05
*** bfournie has quit IRC22:05
openstackgerritMerged openstack/tripleo-quickstart-extras master: Increase default novajoin read timeout to 20 seconds  https://review.openstack.org/45813022:05
openstackgerritMerged openstack/tripleo-quickstart-extras master: Handle UPDATE_COMPLETE in overcloud-deploy.sh script  https://review.openstack.org/45524122:05
openstackgerritMerged openstack/tripleo-quickstart-extras master: Use subjectAltName in self-generated SSL certs  https://review.openstack.org/44958822:05
*** trozet has joined #tripleo22:07
*** owalsh has joined #tripleo22:07
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix path of working dir for Jinja linters  https://review.openstack.org/44732522:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025922:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427222:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]22:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]22:10
*** dsariel has quit IRC22:13
*** dsariel has joined #tripleo22:14
*** rhallisey has joined #tripleo22:16
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Refactor SSHD config to allow both SSHD options and banner/motd to be set  https://review.openstack.org/45883022:16
*** mcornea has quit IRC22:22
*** tellesnobrega has joined #tripleo22:27
*** fragatin_ has joined #tripleo22:35
*** owalsh has quit IRC22:37
*** fragatina has quit IRC22:38
*** owalsh has joined #tripleo22:38
*** brault has joined #tripleo22:41
*** brault has quit IRC22:45
*** dmarlin has left #tripleo22:56
*** trozet has quit IRC22:57
*** mhenkel_ has joined #tripleo22:57
*** mhenkel_ has joined #tripleo22:58
*** mhenkel__ has quit IRC22:58
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: SSHD Service extensions  https://review.openstack.org/45883622:58
openstackgerritMerged openstack-infra/tripleo-ci master: repo: allow to download hammer on mitaka  https://review.openstack.org/45977322:59
*** tellesnobrega has quit IRC23:01
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Add pre-selected Control Plane IP support.  https://review.openstack.org/41327823:04
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168025923:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/168427223:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1680259 in tripleo "Upgrades jobs timing out regularly" [Critical,In progress]23:10
openstackLaunchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress]23:10
*** openstack has joined #tripleo23:27
*** tdasilva has quit IRC23:34
honzaWhat is os_service_default in puppet?  I'm trying to determine where the header options are coming from (when doing :ironic:cors)23:36
honzahttps://github.com/openstack/puppet-ironic/blob/master/manifests/cors.pp23:36
honzawhich appears to be triggered from https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/ironic.pp#L8523:37
honzaMy puppet fu is weak.23:37
*** limao has joined #tripleo23:38
honzaHmmm, it seems like that is coming from hiera23:42
*** tellesnobrega has joined #tripleo23:42
*** limao has quit IRC23:43
*** dsneddon has quit IRC23:45
*** tdasilva has joined #tripleo23:48
*** bfournie has joined #tripleo23:55
*** myoung has quit IRC23:59
« Monday, 2017-04-24 Index Wednesday, 2017-04-26 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!