Friday, 2017-12-08

*** threestrands has joined #tripleo00:01
*** itlinux has joined #tripleo00:03
*** liverpooler has joined #tripleo00:04
*** dtrainor has quit IRC00:05
*** dtrainor has joined #tripleo00:08
*** liverpooler has quit IRC00:09
openstackgerritwes hayutin proposed openstack/python-tripleoclient master: testing DLRN do not merge  https://review.openstack.org/52655000:10
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647500:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663400:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704200:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)00:10
*** bfournie has quit IRC00:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)00:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)00:10
*** bfournie has joined #tripleo00:11
*** bfournie has quit IRC00:12
*** bfournie has joined #tripleo00:12
*** tosky has quit IRC00:13
*** bfournie has quit IRC00:14
*** bfournie has joined #tripleo00:16
*** matbu has quit IRC00:24
*** rcernin has quit IRC00:25
*** rcernin has joined #tripleo00:25
*** tongl has quit IRC00:27
*** matbu has joined #tripleo00:30
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add glance config for barbican  https://review.openstack.org/52406300:52
openstackgerritJames Slagle proposed openstack/tripleo-common master: Break up messages to avoid max message size  https://review.openstack.org/52649200:59
openstackgerritJames Slagle proposed openstack/tripleo-common master: Break up messages to avoid max message size  https://review.openstack.org/52649201:01
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647501:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663401:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704201:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)01:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)01:10
*** gvrangan has joined #tripleo01:12
*** bfournie has quit IRC01:14
rookis there a known issue with the puppet-unit-centos ci?01:15
*** gvrangan has quit IRC01:16
EmilienMrook: show logs01:23
EmilienMrook: we had an issue yesterday01:23
openstackgerritMerged openstack/tripleo-heat-templates master: Add multiple secret store backends for barbican  https://review.openstack.org/52353901:23
EmilienMbut it should be solved01:23
EmilienMit was an issue when cloning puppet opendaylight01:23
rookWhen i looked through the logs i didn't see anything obvious01:23
EmilienMrook: url?01:23
rookhttp://logs.openstack.org/24/497524/10/check/puppet-openstack-unit-4.8-centos-7/0d61f04/job-output.json.gz01:24
EmilienMrook: http://logs.openstack.org/24/497524/10/check/puppet-openstack-unit-4.8-centos-7/0d61f04/job-output.txt.gz#_2017-12-07_23_14_18_49859701:25
EmilienMtrozet: ^ can you help on this one?01:25
rookEmilienM: thanks, I was looking at the ansibe output01:26
* rook should of opened the other file01:26
EmilienMit happens randomly01:28
EmilienMjust do recheck01:28
* rook hopes Paul didn't see that :P01:29
EmilienMweshay|ruck: new error in IPv6 OVB01:30
EmilienMweshay|ruck: https://logs.rdoproject.org/52/526152/2/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/Zc82f7d56a5fd452192f6d6426c191787/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-12-07_23_36_2001:30
EmilienMit's a pacemaker timeout01:30
EmilienMI'll file a bug01:30
*** dtrainor has quit IRC01:31
EmilienMweshay|ruck: not sure if we want a alert: https://bugs.launchpad.net/tripleo/+bug/173706601:33
openstackLaunchpad bug 1737066 in tripleo "ovb/ipv6 timeout: Error: unable to push cib" [High,Triaged]01:33
*** fragatina has quit IRC01:33
EmilienMbandini: ^01:33
EmilienMit's bank holiday in Italy on Friday, so it'll probably wait Monday01:33
*** lblanchard has joined #tripleo01:36
*** lblanchard has quit IRC01:38
*** dtrainor has joined #tripleo01:40
*** rhallisey has quit IRC01:45
EmilienMdpeacock: nice first patches ;-)01:46
*** fragatina has joined #tripleo01:46
*** fragatina has quit IRC01:51
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647502:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663402:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704202:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)02:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)02:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)02:10
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574302:10
*** gvrangan has joined #tripleo02:12
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574302:16
*** gvrangan has quit IRC02:21
*** bfournie has joined #tripleo02:24
openstackgerritDan Prince proposed openstack/puppet-tripleo master: Remove INSECURE_REGISTRY from docker_registry.pp  https://review.openstack.org/52576102:30
*** ebarrera has joined #tripleo02:36
*** Goneri has quit IRC02:36
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add docker-registry service  https://review.openstack.org/52613202:39
*** dmacpher has quit IRC02:49
Tenguhello!02:54
Tengumwhahaha: are you here ?02:54
mwhahahaTengu: hey what's up02:55
Tengumwhahaha: just wanted to talk a bit about my review for the "::" instead of "." :)02:55
Tenguif you have a minute02:55
openstackgerritMerged openstack/tripleo-common master: Reduce the calls to heat when downloading config  https://review.openstack.org/52523602:55
mwhahahaTengu: yea I got a minute02:56
Tengumwhahaha: basically I'd like to make a move toward the "::" also for the firewall thingy, with a backward compatibility of course.02:56
Tengumwhahaha: I'll amend my current patch in order to also add that backward compat for the haproxy service_endpoints - even if it's not necessary for now.02:56
mwhahahaOk we can try it02:57
Tenguthe main goal being: allow to have real and working unit tests so that we can check for regressions in case we have to modifiy something in those two definitions.02:57
Tengubasically, a hiera('foo::bar::blah', hiera('foo.bar.blah',{})) would be fine02:57
TenguI'll push an update on the current review for that, and create a second one in order to add unit tests for the firewall service_rules def.02:58
Tengujust wanted to check with you if those reviews would meet the requirements and be allowed.02:58
Tenguthink it will, especially since I add backward compat :).02:59
*** chlong has joined #tripleo02:59
mwhahahaTengu: so the one thing is that we are late in Queens so we'll have to see what the actual impact is and make sure we don't add any regressions03:00
Tengumwhahaha: :) welll... without unit tests, it's kind of hard ;)03:00
mwhahahaTengu: but we won't really know until we get it posted03:00
mwhahahaYea I know03:00
Tenguand we can't have unit due to the weird syntax currently in use03:01
*** fragatina has joined #tripleo03:01
mwhahahaTengu: you created a bug for it right?03:01
Tenguhmm nope. but I can do that, of course.03:02
Tengu2s03:02
mwhahahaPlease do that. The bug is that we can't properly unit test due to the dot notation03:03
mwhahahaSo long as we maintain and test the backwards compatibility it should be ok03:03
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: WIP move more prepare logic into kolla_builder  https://review.openstack.org/52657603:06
openstackgerritDan Prince proposed openstack/python-tripleoclient master: Configure undercloud docker registry/mirror  https://review.openstack.org/52614703:08
Tengumwhahaha: https://bugs.launchpad.net/tripleo/+bug/173708603:09
openstackLaunchpad bug 1737086 in tripleo "We can't do unit tests due to "dot" notation" [Undecided,New]03:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647503:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663403:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704203:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)03:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)03:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)03:10
*** tongl has joined #tripleo03:10
openstackgerritDan Prince proposed openstack/python-tripleoclient master: Drop docker_name_space option  https://review.openstack.org/52657703:14
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP move more prepare logic into kolla_builder  https://review.openstack.org/52657903:14
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP prepare action extra arguments  https://review.openstack.org/52658003:14
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: WIP move more prepare logic into kolla_builder  https://review.openstack.org/52657603:14
*** ykarel has joined #tripleo03:15
*** fragatina has quit IRC03:19
Tengumwhahaha: running the unit tests on my side in order to ensure my patch is OK, then I'll update the review. and the commit message in order to link to the issue I just opened.03:20
mwhahahaOk sounds good. I'll check it in the morning03:20
Tengunp :)03:21
TenguI can't sleep, sooo....03:21
Tengudoing some tripleo unit testing is the right thing to do03:22
*** rlandy has quit IRC03:23
Tenguaaand rebased against master as well.03:24
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: Add unit test for tripleo::haproxy::service_endpoints  https://review.openstack.org/52629203:24
Tengutadaaa03:24
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add NovaSchedulerMaxAttempts parameter  https://review.openstack.org/52658203:24
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Check source-repository-* files for trailing newline  https://review.openstack.org/52658303:25
openstackgerritDan Prince proposed openstack/python-tripleoclient master: Undercloud: wire in scheduler_max_attempts  https://review.openstack.org/52658403:26
*** gvrangan has joined #tripleo03:34
*** gvrangan has quit IRC03:34
*** gvrangan has joined #tripleo03:35
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: Added unit tests for tripleo::firewall::service_rules  https://review.openstack.org/52658903:35
Tengumwhahaha: -^03:35
Tengujaosorior: -^^  :)03:36
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: Add unit tests for tripleo::firewall::service_rules  https://review.openstack.org/52658903:37
Tengubetter.03:37
*** gvrangan has quit IRC03:39
*** links has joined #tripleo03:41
*** ramishra has joined #tripleo03:42
openstackgerritzenghui.shi proposed openstack/tripleo-heat-templates master: Add PTP composable service  https://review.openstack.org/49131703:42
Tenguah. we can't reference the same bug in two reviews… at least launchpad doesn't like it :D03:43
Tengumeaning: either I should merge my two reviews in one, or open two bugs. both aren't cool. well. duh.03:44
*** psahoo has joined #tripleo03:46
*** gvrangan has joined #tripleo03:47
*** udesale has joined #tripleo03:54
*** iranzo has joined #tripleo03:55
*** gvrangan has quit IRC03:55
*** BryanS68 has joined #tripleo04:01
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647504:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663404:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704204:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)04:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)04:10
*** pdeore has joined #tripleo04:13
*** gbarros has quit IRC04:17
*** shreshtha has joined #tripleo04:21
*** gbarros has joined #tripleo04:32
*** pgadiya has joined #tripleo04:33
*** gbarros has quit IRC04:39
*** tzumainn has quit IRC04:55
*** karthiks has joined #tripleo04:56
jaosoriorTengu: I'll check it out04:58
jaosoriorTengu: daaaamn you're up early04:58
*** BryanS68 has quit IRC05:06
*** dpawar has joined #tripleo05:06
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647505:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663405:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704205:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)05:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)05:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)05:10
*** rcernin has quit IRC05:11
*** pgadiya has quit IRC05:11
*** BryanS68 has joined #tripleo05:12
*** ratailor has joined #tripleo05:13
*** ebarrera has quit IRC05:14
*** ebarrera has joined #tripleo05:15
*** threestrands has quit IRC05:24
*** threestrands has joined #tripleo05:39
*** threestrands has quit IRC05:39
*** threestrands has joined #tripleo05:39
*** BryanS68 has quit IRC05:39
jaosoriorEmilienM: so... why is there a commit comenting the barbican tests?05:46
jaosoriorah, nevermind, they're not being commented.05:47
EmilienMjaosorior: woot?05:47
EmilienM:)05:47
jaosoriorEmilienM: the commit message was misleading05:47
jaosoriorit's all good05:47
*** jaganathan has joined #tripleo05:48
EmilienMok05:49
EmilienMjaosorior: I go sleep05:49
jaosoriorenjoy05:50
*** fragatina has joined #tripleo05:52
*** fpan has quit IRC05:54
*** fragatina has quit IRC05:57
*** janki has joined #tripleo05:59
*** ykarel_ has joined #tripleo06:03
*** ykarel has quit IRC06:05
*** ykarel_ is now known as ykarel06:08
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647506:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663406:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704206:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)06:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)06:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)06:10
*** rcernin has joined #tripleo06:11
*** cshastri has joined #tripleo06:13
*** ludovic has joined #tripleo06:18
ludovicHello everybody06:19
jaosoriorhello06:24
*** s_b_34343 has joined #tripleo06:26
*** threestrands has quit IRC06:28
*** fragatina has joined #tripleo06:29
*** ebarrera has quit IRC06:29
*** ebarrera has joined #tripleo06:30
*** fragatina has quit IRC06:30
*** fragatina has joined #tripleo06:30
*** chlong has quit IRC06:36
*** radeks has joined #tripleo06:46
*** skramaja has joined #tripleo06:48
*** marios has joined #tripleo06:52
*** pmannidi has quit IRC06:56
*** tongl has quit IRC07:02
-openstackstatus- NOTICE: Due to some unforseen Zuul issues the gate is under very high load and extremely unstable at the moment. This is likely to persist until PST morning07:06
*** ChanServ changes topic to "Due to some unforseen Zuul issues the gate is under very high load and extremely unstable at the moment. This is likely to persist until PST morning"07:06
*** Ng has quit IRC07:07
*** Ng has joined #tripleo07:07
*** ChanServ sets mode: +v Ng07:07
*** mugsie has quit IRC07:07
*** tdasilva has quit IRC07:08
*** vkmc has quit IRC07:08
*** melwitt has quit IRC07:08
*** mugsie has joined #tripleo07:08
*** mugsie has quit IRC07:08
*** mugsie has joined #tripleo07:08
*** nguyentrihai has joined #tripleo07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647507:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663407:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704207:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)07:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)07:10
*** melwitt has joined #tripleo07:10
*** vkmc has joined #tripleo07:10
*** vkmc has quit IRC07:10
*** vkmc has joined #tripleo07:10
*** melwitt is now known as Guest905407:11
*** cshastri has quit IRC07:11
*** nguyentrihai has quit IRC07:11
*** tdasilva has joined #tripleo07:13
*** abregman|afk has quit IRC07:14
*** ebarrera has quit IRC07:14
*** bandini has quit IRC07:17
*** bandini has joined #tripleo07:19
openstackgerritzenghui.shi proposed openstack/tripleo-heat-templates master: Add PTP composable service  https://review.openstack.org/49131707:20
*** aufi has joined #tripleo07:23
*** holser has joined #tripleo07:24
*** mdnadeem has joined #tripleo07:25
*** dtrainor has quit IRC07:27
*** moshele has joined #tripleo07:27
*** paramite has joined #tripleo07:30
*** bogdando has joined #tripleo07:41
*** pmannidi has joined #tripleo07:46
*** yprokule has joined #tripleo07:47
*** pmannidi has quit IRC07:51
ludovicHello ,  Maybe somebody can help ...  I'm trying nova host-evacuate but many of VM rebuilds terminated with Gateway Timeout Error 504 . In nova-compute.log Connection pool is full, discarding connection I see a lot of07:51
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: Add Instance HA support  https://review.openstack.org/50635007:55
*** shardy has joined #tripleo07:58
*** cylopez has joined #tripleo08:07
*** cylopez has quit IRC08:08
*** cylopez has joined #tripleo08:09
*** yprokule_ has joined #tripleo08:09
*** yprokule has quit IRC08:10
*** yprokule_ is now known as yprokule08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647508:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663408:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704208:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)08:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)08:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)08:10
*** florianf has joined #tripleo08:12
*** chem` has joined #tripleo08:12
*** ykarel has quit IRC08:13
*** stendulker has joined #tripleo08:14
*** ykarel has joined #tripleo08:14
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: In compute IHA make no_shared_storage a class parameter  https://review.openstack.org/52661908:14
*** moshele has quit IRC08:15
s_b_34343Hi all08:16
s_b_34343wanted to know that how many controller ha triplo is going to support ?08:16
ludovicHi all , I have a question for nova experts ... Maybe somebody can help me to resolve a trouble with a full host evacuation ?08:17
shardys_b_34343: 3 controllers is the default HA setup, but you can scale services beyond that using composable roles08:18
shardyludovic: Hi, if you can provide details then perhaps lyarwood or owalsh may be able to help08:19
*** mrch has joined #tripleo08:19
s_b_34343<shardy> :so i just need to set profiles and it will work ?08:19
*** tesseract has joined #tripleo08:20
ludovicok08:21
ludovicthanks08:21
shardys_b_34343: well you need to decide what architecture you're trying to deploy, then create a roles_data.yaml file that describes it08:21
shardys_b_34343: see https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/custom_roles.html08:21
shardythe 3 controller setup is, by far, the most well tested, but in theory you have total flexibility on what is deployed with composable roles08:22
shardyincluding things managed via pacemaker08:22
s_b_34343have any body tested for more that 3 controller ?08:22
shardys_b_34343: yes, we found that some things (galera was one IIRC) didn't scale well beyond 3, thus 3 is the reccomended setup08:23
shardys_b_34343: with composable roles you have flexibility to deploy the controlplane over multiple roles, but you'll have to do your own testing to decide what architecture makes sense for your environment08:23
*** yprokule has quit IRC08:24
s_b_34343yeah and one more query . why can't we add one more controller after deploying 2 controller08:24
s_b_34343can we add controller any time ?08:24
*** cylopez has left #tripleo08:24
shardys_b_34343: That isn't tested at all, it's best to deploy with 3 controllers08:25
shardyit might work, but 2 is a bad number for any HA setup08:25
*** yprokule has joined #tripleo08:25
s_b_34343what if 1 controller is there and wanted to add 2 more08:25
s_b_34343will it wor08:25
s_b_34343*work08:26
shardys_b_34343: we have zero test coverage of that so I can't really say if it will work08:26
ludovicI just posted an instance status  http://paste.openstack.org/show/628428/08:26
shardythe reccomendation is to deploy with 1 for poc/testing then deploy with 3 for production use08:26
s_b_34343<shardy> :oh ok  . what is the last date if i want to opensource any code ?08:27
shardys_b_34343: we're moving into feature freeze for Queens now, so if you want to post features for tripleo, do it ASAP08:27
shardyor it will be deferred until Rocky08:28
s_b_34343<shardy> : may i know the date ?08:28
*** pmannidi has joined #tripleo08:28
shardys_b_34343: well in theory we're already in feature freeze, see the minutes from our most recent meeting08:28
shardycertain exceptions may be made, but really it's getting too late now for any major features, we're aiming to focus on bugfixes for the final milestone08:29
shardys_b_34343: on the mailing list the freeze was communicates as milestone 2, which happened this week08:29
s_b_34343<shardy> : oh ok . can i get that mail too08:30
s_b_34343so i can keep track of everything ?08:30
shardys_b_34343: my advice is just post the code as soon as you have it, and raise blueprints/specs as soon as you start any work08:30
shardyeven if it's WIP it's much better to give the team visibility vs a late code-drop08:31
*** shardy is now known as shardy_afk08:31
s_b_34343shardy_afk : ok thanks :)08:31
*** ludovic has quit IRC08:33
*** chem` is now known as chem08:33
*** etingof has quit IRC08:34
*** jpich has joined #tripleo08:36
Tengujaosorior: well, I just couldn't sleep in fact ;)08:38
Tengujaosorior: I'm now in a meeting for the whole day...08:38
jaosoriorTengu: bummer dude08:39
jaosoriorTengu: today's my last day working before my vacations :D08:39
Tengujaosorior: happy guy ;)08:40
*** pmannidi has quit IRC08:40
*** ludovic has joined #tripleo08:40
jaosoriorawwww yeeaaah08:41
Tenguduh, ok. just read the topic in here, CI loaded.08:41
openstackgerritzenghui.shi proposed openstack/tripleo-heat-templates master: Add PTP composable service  https://review.openstack.org/49131708:43
*** mcornea has joined #tripleo08:43
*** ykarel has quit IRC08:47
*** ykarel has joined #tripleo08:47
*** shardy_afk is now known as shardy08:48
*** abregman|afk has joined #tripleo08:50
*** milan has joined #tripleo08:50
Tengushardy: thank you for your answer on the ML regarding the "." notation in hiera :)08:53
Tenguduh08:54
TenguI think I found the issue :D08:54
shardyTengu: np, I'm not certain of the history there, it was a while ago :)08:55
holsershardy can we merge https://review.openstack.org/#/c/526440/4 ?08:55
Tengushardy, jaosorior : hiera backend on the nodes is set to json. apparently....08:55
holserMorning!08:55
*** yprokule has quit IRC08:55
Tengushardy jaosorior : and it's set to YAML for the unit tests.08:55
*** yprokule has joined #tripleo08:56
Tengushardy jaosorior : so, if I move the hirea config to json in the unit tests to match what's deployed on the prod, we should be able to converge nicely.08:56
shardyholser: sec, will review shortly08:56
Tengushould have checked the hiera backend on the nodes earlier.08:56
*** lucas-afk is now known as lucasagomes08:57
openstackgerritzenghui.shi proposed openstack/tripleo-puppet-elements master: add puppet-ptp into image  https://review.openstack.org/52663008:57
shardys_b_34343: links FYI https://releases.openstack.org/queens/schedule.html08:57
shardys_b_34343: http://lists.openstack.org/pipermail/openstack-dev/2017-December/125216.html08:57
*** aditya_r has joined #tripleo08:58
shardys_b_34343: As mentioned, you open your code for review at any time (earlier is better, even if it's WIP), but we may not be able to merge it into the main repos until January if it's a large feature08:58
shardyTengu: aha, good news :)08:58
s_b_34343<shardy> what if the code is related to specific organisation and they want to opensource their code so for that also date will be same ?08:59
s_b_34343<shardy> thanks its really useful09:00
*** itlinux has quit IRC09:01
shardys_b_34343: you mean it's a vendor integration?09:02
*** yprokule has quit IRC09:02
*** yprokule_ has joined #tripleo09:02
s_b_34343<shardy> yes right09:03
shardys_b_34343: really regardless of where the code comes from, the rules/reccomendations are the same, it's best to get the code under review as early in the cycle as possible, or there is a risk it'll have to wait until the next release09:03
shardys_b_34343: that said, we've sometimes made exceptions for vendor integrations that are disabled by default09:04
*** yprokule_ has quit IRC09:04
shardys_b_34343: best thing is just push the code, then we can review and discuss the best way forward09:04
*** yprokule_ has joined #tripleo09:04
*** yprokule_ is now known as yprokule09:05
s_b_34343<shardy> :ok shardy will do that as soon as possible .09:07
*** yprokule has quit IRC09:07
shardys_b_34343: ack thanks, you could also send a message to the ML tagged [tripleo] explaining your plans, as Alex mentions the process is to discuss and agree if a feature freeze exception is reasonable for any late-landing things09:08
*** yprokule has joined #tripleo09:08
shardythe general idea is just to reduce the chance of regressions late in the cycle, so disabled by default integrations may be OK09:08
*** yprokule has quit IRC09:09
*** yprokule has joined #tripleo09:09
*** owalsh has quit IRC09:10
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647509:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663409:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704209:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)09:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)09:10
*** owalsh has joined #tripleo09:12
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart-extras master: [WIP] Support running tripleo-common-tempest-plugin on an undercloud  https://review.openstack.org/52230009:12
*** yprokule has quit IRC09:12
*** yprokule has joined #tripleo09:13
s_b_34343<shardy> ok will discuss with team09:14
*** jbadiapa has joined #tripleo09:14
*** athomas has joined #tripleo09:15
*** etingof has joined #tripleo09:19
*** karthiks is now known as karthiks_afk09:20
*** jbadiapa has quit IRC09:21
*** ykarel_ has joined #tripleo09:21
*** moshele has joined #tripleo09:22
*** ykarel has quit IRC09:24
*** dbecker has joined #tripleo09:24
*** aditya_r has quit IRC09:25
*** cshastri has joined #tripleo09:26
*** thrash|biab is now known as thrash|g0ne09:27
*** nyechiel has joined #tripleo09:35
*** ykarel_ is now known as ykarel|away09:35
*** ykarel|away has quit IRC09:40
* Tengu banging his head on the walls and desk09:41
*** nyechiel has quit IRC09:41
Tengushardy: moved the hiera stuff to yaml in the unit tests, but apparently it's still not working09:41
Tenguduh.... ok, even with the JSON backend, it wants "::"09:43
Tenguhow the hell is it working o__O09:44
shardyTengu: maybe push the failing tests and we can help figure it out, I'm still not exactly clear what the problem is, as e.g "yaml.foo: xyz09:45
shardy"09:45
shardyparses just fine as yaml for me09:45
*** numans has quit IRC09:45
*** moshele has quit IRC09:46
Tengushardy: in a unit test ?09:48
shardyTengu: no only tried it in python, happy to look tho if you can push a failing test09:49
*** numans has joined #tripleo09:50
*** mcornea has quit IRC09:50
shardyd0ugal: hey as discussed yesterday I think we do need to fix the container name in check_container09:50
shardyd0ugal: testing a follow-up to your patch now09:50
shardyI'm a bit unclear how this ever worked, e.g why it broke just recently09:51
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/pike: Add modulepath option when applying puppet inside docker.  https://review.openstack.org/52647309:51
* shardy grumbles about lack of ci coverage09:51
Tengushardy: 2s, gisting stuff09:52
d0ugalshardy: did you see my patch?09:53
d0ugalshardy: yeah, I don't understand how this ever worked.09:54
d0ugalshardy: https://review.openstack.org/52646209:54
*** iranzo has quit IRC09:54
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/pike: Add modulepath option when applying puppet inside docker.  https://review.openstack.org/52647309:54
s_b_34343<shardy> which launguage we use to write puppet ?09:55
Tengushardy: https://gist.github.com/cjeanneret/2199d08626de3fcc36954f8787bb3c1e09:55
Tengus_b_34343: it's the puppet language :)09:56
Tengus_b_34343: you can check on puppet.com for their doc and basis09:56
s_b_34343Tengu : oh cool will try09:57
*** hewbrocca_afk is now known as hewbrocca09:58
shardyd0ugal: ack thanks, trying your patch now10:03
d0ugalshardy: I am just setting up a new env to test it too - so you'll probably beat me :)10:03
s_b_34343if in puppet $p="list"   $pp="dict"    now using some if else condition want to replace $p=$pp10:06
*** salmankhan has joined #tripleo10:06
s_b_34343this is how we can reassign one value to another variable ?10:06
s_b_34343<shardy> <Tengu> :?10:08
Tengus_b_34343: we can't re-assign10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647510:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663410:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704210:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)10:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)10:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)10:10
*** brault has quit IRC10:11
lyarwoodfinal FFU rebase from me, sorry about the spam folks!10:11
s_b_34343<Tengu> why not ?10:11
*** brault has joined #tripleo10:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: ffu: Add fast-forward upgrade outputs to RoleConfig  https://review.openstack.org/49922110:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: ffu: tripleo-packages repo management  https://review.openstack.org/51542910:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: ffu: Allow FASTFORWARDUPGRADE as a StackUpdateType  https://review.openstack.org/52254710:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: ffu: Add fast-forward-upgrade env  https://review.openstack.org/52220310:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Keystone fast-forward upgrade tasks  https://review.openstack.org/51462110:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Neutron fast-forward upgrade tasks  https://review.openstack.org/52154310:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Glance fast-forward upgrade tasks  https://review.openstack.org/52154410:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Cinder fast-forward upgrade tasks  https://review.openstack.org/52154510:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Nova fast-forward upgrade tasks  https://review.openstack.org/52292110:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Pacemaker fast-forward upgrade tasks  https://review.openstack.org/52339910:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP Add missing become to upgrade_steps_playbook  https://review.openstack.org/52292210:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP Add missing become to Host prep steps play  https://review.openstack.org/52292310:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: DNM ffu: ignore_errors during upgrade_tasks  https://review.openstack.org/52340010:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: DNM ffu: Workaround missing repo control when moving to Queens  https://review.openstack.org/51871910:11
openstackgerritLee Yarwood proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Swift fast-forward upgrade tasks  https://review.openstack.org/52664210:11
Tengus_b_34343: language limitation10:12
social:))10:12
Tengus_b_34343: you might want to read the introduction for the language ;)10:12
*** brault has quit IRC10:12
s_b_34343oh ok .10:12
*** dtantsur|afk is now known as dtantsur10:16
*** mcornea has joined #tripleo10:20
ludovicHi , Maybe some Nova experts can help me with a Host-evacuate troubleshooting  ? I can post errors encountered10:21
shardyd0ugal: http://paste.openstack.org/show/628440/ seems like we're still trying to create the container when it already exists10:23
ludovicFor information, all prerequites are respected, few VMs are well rebuilded but another one not, so i would like to understand10:23
ludovicanother information i use Hitachi Block storage driver for volumes10:24
d0ugalshardy: odd.10:25
d0ugalshardy: just to be clear, that error happens on a second deploy? Are any of the envs you passed requried?10:25
shardyI'm really confused why this has suddenly broken - any kind of update is totally broken now10:25
shardyd0ugal: yeah I'm just re-running the same deploy command with no changes10:25
d0ugalk, thanks. I am not even at the undercloud install yet :(10:26
*** lebauce_ has quit IRC10:28
*** tosky has joined #tripleo10:28
shardyd0ugal: how can I trace the workflow that failed, e.g that error output isn't useful as it just dumps the workflow definition?10:30
shardywe need something that shows each task run, the status and the data?10:31
d0ugalshardy: You have the execution ID in that error10:31
d0ugalshardy: mistral task-list 606c69c5-69b2-4c7d-b4e9-ae2b6d77d35110:31
d0ugalthat'll show the tasks for the execution10:31
d0ugalshardy: I am curious to know why the head_container didn't work.10:32
shardyd0ugal: aha, thanks - it'd be cool to wire that into the tripleoclient output10:32
shardycheck_container did fail, looking at why10:32
d0ugalshardy: $ mistral run-action swift.head_container '{"container":"overcloud-swift-rings"}'10:33
d0ugalthat is another way to quickly see what that actions is returning.10:33
shardyd0ugal: http://paste.openstack.org/show/628445/10:34
shardyI think we're using the wrong endpoint?10:34
shardyit's trying to HEAD the container with the keystone API10:34
d0ugalshardy: oh10:35
d0ugalshardy: thrash|g0ne said something about this the other day, I thought it had been fixed.10:35
d0ugalshardy: maybe you are missing this patch: https://github.com/openstack/mistral/commit/43e453d8b648f46267f973e5806eb638b2e8c86710:36
shardyd0ugal: yeah the promoted mistral is still at f1e238510:37
* shardy pulls trunk mistral10:37
*** daidv has quit IRC10:38
*** akrivoka has joined #tripleo10:40
d0ugalshardy: sorry about that, we had some rushed refactoring in Mistral when oslo.context broke us (but had hadn't fully understood the issue yet)10:40
shardyd0ugal: No worries, at least that explains the mystery breakage, upgrading mistral seems to fix it10:40
*** derekh has joined #tripleo10:40
shardythanks :)10:41
d0ugalGreat10:41
d0ugalI don't understand why the tripleo CI job didn't fail without the mistral fix...10:41
shardyd0ugal: because we no longer have any update or upgrade coverage10:42
d0ugalshardy: right, but all the swift actions should have been broken - we use swift in other places.10:42
shardyyeah that's a good point10:42
shardythe initial deploy worked10:42
d0ugal¯\_(ツ)_/¯10:42
shardyActually we do have one upgrade job on t-h-t, looking at results10:43
*** lebauce has joined #tripleo10:45
milanshardy, hey,  apparently my cargo-fu sucks: https://review.openstack.org/#/c/523909/3/docker/services/ironic-inspector.yaml@195 doesn't pass validation because:10:48
milanERROR: upgrade task ({'get_attr': ['IronicInspectorBase', 'role_data', 'upgrade_tasks']}) is missing its tags keyword.10:48
milanwhat do I do? :)10:48
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Add a report of the Workflow execution on failure  https://review.openstack.org/52665310:52
d0ugalshardy: ^ 100% untested, ugly and hacked together - but is that the sort of thing you imagined?10:53
d0ugallol, it even has a syntax error I see :)10:54
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud master: Out-of-box configuration for Ironic Ansible deploy method  https://review.openstack.org/52643910:55
*** rfolco|off is now known as rfolco11:01
openstackgerritMarios Andreou proposed openstack/tripleo-docs master: WIP: FFU Developer docs  https://review.openstack.org/52665611:01
*** stendulker has quit IRC11:01
shardymilan: try upgrade_tasks: {get_attr ...11:02
shardymilan: you're creating a list with a list in it11:03
milanshardy, k, /me tries, thx11:03
*** karthiks_afk is now known as karthiks11:08
Tengushardy: btw... apparently there's another way to do things with the firewall: I added the following entry in the extraconfig.json on my deployed openstack: "tripleo::firewall::firewall_rules": {...}  and it apparently works. Sooooo a nice way would be to converge, maybe :).11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647511:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663411:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704211:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)11:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)11:10
shardyTengu: ack, sorry not got to trying your test yet11:10
Tengushardy: no problem11:10
Tengushardy: it's not an urgent issue ;). just a cosmetic (and, well, tech-debt-squashing)11:10
*** iranzo has joined #tripleo11:11
shardymilan: I actually think the validation in tools/yaml-validate.py may be too strict11:12
shardyyou do need to make the change I mentioned I think, so the tasks will run, but we may also have to tweak the validation to ignore get_attr lookups11:12
shardywill add a comment on the patch11:13
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/tools/yaml-validate.py#L60611:14
shardymilan: nvm, we already special-case it :)11:14
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Add a report of the Workflow execution on failure  https://review.openstack.org/52665311:14
*** openstackgerrit has quit IRC11:17
jpichAny hint what can cause a stack to go into "create aborted"? I'm not sure how to debug from there - http://paste.openstack.org/show/628450/ . All the nodes seem to be up and fine, can ssh into the couple I tried11:21
*** ccamacho has joined #tripleo11:22
jpichI didn't cancel anything. Could conflicting templates cause something like that, maybe?11:22
shardyjpich: a stack operation gets aborted when a timeout is hit, and nested stacks are sometimes aborted due to a failure somewhere else in the graph11:23
shardyjpich: try openstack stack failures list overcloud --long as a starting point11:24
jpichshardy: That was the full long output, in that paste :/11:24
jpichOh, I see NodeTLSCAData seems to have started the problem?11:24
jpichI guess I can poke at the SSL config then11:24
jpichshardy: Thanks for making me look closer! I usually look for reasons, not task names :-)11:25
shardyjpich: ah sorry missed that, yea overcloud.Controller.2.NetworkDeployment failed then the other IN_PROGRESS stuff got aborted11:25
shardyjpich: that deployment runs os-net-config, so I'd ssh onto the controller and check the nic setup and try running it manually11:26
jpichshardy: Oh, so the errors starts from the bottom? "Network Deployment" seems kinda large as a space to debug11:26
jpichshardy: Ok, I'll start there! Thanks for the tip11:26
*** links has quit IRC11:28
shardyjpich: you can get the full output via openstack software deployment output show 65f6bf72-bf06-46bf-8097-5c1bca807be8 --all --long11:28
*** jkilpatr has quit IRC11:28
shardyprobably stack failures list should have a --reallylong option that does that ;)11:28
*** udesale has quit IRC11:29
*** pdeore_ has joined #tripleo11:29
shardyjpich: the stack failures list output isn't really ordered - we should maybe hide create aborted output by default though11:30
*** openstackgerrit has joined #tripleo11:31
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Correct the container name confusion in the swift rings workflow  https://review.openstack.org/52646211:31
*** pdeore has quit IRC11:32
jpichshardy: Haha, I like the new proposed option name! So that's a command to use with the resource_id for the resource that shows the actual error message... Perfect, thank you.11:33
*** gfidente has joined #tripleo11:33
jpich"[/etc/sysconfig/network-scripts/ifup-eth] Error, some other host (CA:3F:1A:57:34:A7) already uses address 172.16.0.10." - so I guess I messed things up while poking at network isolation and the like11:33
*** salmankhan has quit IRC11:33
*** salmankhan has joined #tripleo11:37
*** brault has joined #tripleo11:38
*** shreshtha has quit IRC11:38
*** artom_ has joined #tripleo11:38
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud master: Out-of-box configuration for Ironic Ansible deploy method  https://review.openstack.org/52643911:38
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-common stable/pike: [DO NOT MERGE] Testing upgrade.  https://review.openstack.org/52340811:38
*** artom_ has quit IRC11:39
*** artom_ has joined #tripleo11:39
*** artom has quit IRC11:41
*** links has joined #tripleo11:41
*** pkovar has joined #tripleo11:47
openstackgerritGabriel Szasz proposed openstack/tripleo-heat-templates master: Add Novacontrol role  https://review.openstack.org/52650111:47
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Unit testing for ini  https://review.openstack.org/47352211:49
*** tesseract has quit IRC11:50
*** tesseract has joined #tripleo11:51
lyarwoodchem: did you want to move your ffu/ci patches onto the same topic as the rest of the blueprint? bp/fast-forward-upgrades11:53
lyarwoodchem: just writing up a progress report for the ML and noticed you had used a different topic :)11:54
chemlyarwood: well it's because it's part of the ci, but yeah why not but ... I've already written to the ml  with those links ..11:55
chemoh boy, there's a link(s) nick ..11:55
chemlyarwood: I guess it's kind of too late now11:55
lyarwoodchem: ah of course, np11:56
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: [WIP] Minor update testing for Pike on.  https://review.openstack.org/52640811:57
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: [WIP] Minor update testing for Pike on.  https://review.openstack.org/52640811:57
*** pdeore_ has quit IRC11:58
*** raildo has joined #tripleo12:00
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs master: [WIP] Document using Ironic Ansible deploy interface  https://review.openstack.org/52666312:00
dtantsurmwhahaha: hey! should a formally request an FFE for https://blueprints.launchpad.net/tripleo/+spec/ansible-deploy ? it's really close, but needs a few minor fixes still12:02
dtantsurthis feature won't be used by default in Queens12:03
*** artom_ has quit IRC12:06
*** ansmith has joined #tripleo12:07
*** janki has quit IRC12:07
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647512:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663412:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704212:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)12:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)12:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)12:10
*** rhallisey has joined #tripleo12:12
*** rhallisey has quit IRC12:16
*** rhallisey has joined #tripleo12:17
*** ecerquei has joined #tripleo12:19
*** lvdombrkr has joined #tripleo12:20
openstackgerritMerged openstack/tripleo-heat-templates master: Add DockerRegistryMirror parameter  https://review.openstack.org/52576412:21
*** jkilpatr has joined #tripleo12:23
openstackgerritDougal Matthews proposed openstack/tripleo-quickstart-extras master: Always use overcloudrc, it is now v3 by default  https://review.openstack.org/52339312:25
*** ecerquei has quit IRC12:25
*** ecerquei has joined #tripleo12:26
*** rfolco is now known as rfolco_brb12:30
*** ratailor has quit IRC12:32
*** ecerquei has quit IRC12:34
*** cshastri has quit IRC12:34
openstackgerritMerged openstack/instack-undercloud master: Use the docker profile to set INSECURE_REGISTRY  https://review.openstack.org/52575512:36
*** bfournie has quit IRC12:36
*** bfournie has joined #tripleo12:36
openstackgerritJames Slagle proposed openstack/tripleo-common master: Install openstack-heat-agents instead of python-heat-agent*  https://review.openstack.org/52666912:40
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533112:40
*** bfournie has quit IRC12:41
openstackgerritMerged openstack/tripleo-common master: Correct a task name in create_swift_rings_backup_container_plan  https://review.openstack.org/52635712:43
*** pchavva has joined #tripleo12:46
openstackgerritMerged openstack/tripleo-quickstart master: Use cachable facts to allow multiple runs  https://review.openstack.org/52523712:47
*** psahoo has quit IRC12:48
*** iranzo has quit IRC12:56
*** ecerquei has joined #tripleo13:01
*** rhallisey has quit IRC13:04
*** rhallisey has joined #tripleo13:04
*** liverpooler has joined #tripleo13:05
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647513:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663413:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704213:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)13:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)13:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)13:10
*** jaosorior has quit IRC13:12
*** jaosorior has joined #tripleo13:13
*** eck`gone is now known as eck`13:17
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add incoming storage driver param  https://review.openstack.org/52424813:17
*** bfournie has joined #tripleo13:17
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add S3 storage driver as an option and its config  https://review.openstack.org/52109313:18
*** nguyentrihai has joined #tripleo13:26
*** rlandy has joined #tripleo13:28
*** trown|outtypewww is now known as trown13:29
milanshardy, thanks, I tried with upgrade_tasks: {get_attr....  with this error; http://paste.openstack.org/show/628449/ so I guess some of the merge magic might be missing?13:29
shardymilan: can you push the revised template please?13:31
*** trozet has quit IRC13:31
milanshardy, sure, the way it is13:32
openstackgerritMilan Kováčik proposed openstack/tripleo-heat-templates master: Enable inspector dnsmasq dhcp filter  https://review.openstack.org/52390913:33
milanshardy, ^13:33
* milan reads the review comments13:33
shardymilan: how are you triggering that error?  The environment generator works for me locally with your patch applied13:37
* milan runs tox -epep813:38
slagleEmilienM: mwhahaha : i could use some puppet expertise help with: https://bugs.launchpad.net/tripleo/+bug/173717413:38
openstackLaunchpad bug 1737174 in tripleo "docker-puppet.py: rabbitmq -q list_users run during stack update" [High,In progress] - Assigned to James Slagle (james-slagle)13:38
slagleapparently our "noop" does not actually noop the rabbitmq_user resource13:38
*** toure|gone is now known as toure13:38
milanshardy, I've also had to hack the validator a bit as it was TypeErroring all the time with args.debug < 1: TypeError can't compare int with None13:38
milanmaybe I touched something extra13:38
shardymilan: I wonder if you have some files locally modified but not added to the commit, as tox -e pep8 works for me13:39
shardybut it's not generating an environment for ironic-inspector13:39
shardydo you have a locally updated capabilities map perhaps?13:39
* milan checks13:40
milanshardy, I guess this is just the validation output http://paste.openstack.org/show/628454/13:42
shardymilan: one difference is I cherry picked to current head, it may be worth rebasing13:42
shardyyour parent commit is a few days old13:42
milanshardy, ack, will try13:42
*** tesseract has quit IRC13:47
openstackgerritOliver Walsh proposed openstack/instack-undercloud master: Run nova online data migration before updating packages  https://review.openstack.org/52667513:48
*** rfolco_brb has quit IRC13:49
openstackgerritGabriel Szasz proposed openstack/tripleo-heat-templates master: Add Novacontrol role  https://review.openstack.org/52650113:49
*** abishop has quit IRC13:50
*** tesseract has joined #tripleo13:50
milanshardy, http://paste.openstack.org/show/628457/ after the rebase13:51
* milan pushes update13:52
openstackgerritMilan Kováčik proposed openstack/tripleo-heat-templates master: Enable inspector dnsmasq dhcp filter  https://review.openstack.org/52390913:52
*** jkilpatr has quit IRC13:53
*** s_b_34343 has quit IRC13:54
*** rfolco has joined #tripleo13:55
*** yamahata has joined #tripleo13:55
*** lblanchard has joined #tripleo13:56
*** morazi has joined #tripleo13:56
dmsimardmandre: https://review.rdoproject.org/r/#/c/9328/4/tasks/kolla.yml@10613:58
dmsimardmandre: so the command should properly parse the file then ? Doing a curl | awk is not very friendly versus a command :p13:59
mandredmsimard: oh, yeah :) we need to adapt the script, I wrote it to test something without tripleo-common package installed13:59
*** ramishra has quit IRC13:59
dmsimardmandre: ok, just making sure I wasn't missing something obvious14:00
mandredmsimard: the awk is the interesting part14:00
dmsimard"interesting" :p14:00
mandredmsimard: yeah I guess it all depends on what you enjoy in life :)14:01
*** tzumainn has joined #tripleo14:01
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: update zuul cloned repo directory for zuulv3  https://review.openstack.org/52654614:02
*** ChanServ changes topic to "CI status: GREEEEEEEEN! see alerts and read ML. | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"14:02
-openstackstatus- NOTICE: The issues have been fixed, Zuul is operating fine again but has a large backlog. You can recheck jobs that failed.14:02
*** jmelvin has joined #tripleo14:03
*** Goneri has joined #tripleo14:08
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647514:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663414:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704214:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)14:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)14:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)14:10
*** atarlov has joined #tripleo14:10
*** trozet has joined #tripleo14:11
shardymilan: sorry I still can't reproduce, but it does look like the env generator is failing because it expects a list in the upgrade_tasks14:12
shardyhttp://paste.openstack.org/show/628458/14:12
*** ecerquei has quit IRC14:12
shardyit works for me, so not sure what to suggest14:12
milanshardy, never mind, let's see how zuul likes it14:12
milanthanks anyway!14:13
Tengushardy: re: unit tests. apparently, puppet-rspec doesn't call "hiera" per se, and there might be some "cast" or conversions that breaks with the "dot" separator. Hence, my proposal to support both notation (as in the reviews) is nice, allowing to get unit tests AND without breaking the existing stuff.14:13
shardyTengu: ack, yeah we could support both, or make the delimiter configurable via a parameter?14:14
Tengushardy: supporting both is probably the best (and easiest) one14:14
Tengushardy: https://review.openstack.org/#/c/526292/3/manifests/haproxy/service_endpoints.pp here's how to support both notation :)14:14
Tengufar better that way, and doesn't require anything more. Beware, order *does* matter.14:15
Tenguthat order might be discussed, of course.14:15
*** ecerquei has joined #tripleo14:16
milanshardy, actually, if I run bash -c ./tools/check-up-to-date.sh no exception shown but tox -epep8 gives the traceback14:17
shardymilan: tried rm -fr .tox/pep8 ?14:18
* milan tried tox -repep814:18
* milan tries rm14:18
milansame traceback, shardy14:20
* milan would almost copy&paste the same upgrade tasks into both the files ;)14:21
*** abregman|afk is now known as abregman14:23
*** gbarros has joined #tripleo14:23
shardymilan: yea you could, but it'd be good to follow the traceback and add a special-case like in the yaml-validate example I linked14:24
milanshardy, I wonder whether it will actually do what we expect it to, the get_attr14:24
shardymilan: it should get the tasks from the base puppet template14:25
*** dansmith is now known as superdan14:25
milanbut lemme check the special-case14:25
*** ecerquei has quit IRC14:25
shardybut you can test it by deploying then looking at the output of openstack overcloud config download14:25
milanshardy, right, gonna take me couple of days to figure out how to do that14:26
milan* deploy14:26
milanbut seems inevitable14:26
shardymilan: do you have a working test environment atm?14:26
*** ecerquei has joined #tripleo14:27
shardypep8: commands succeeded14:27
shardystrange14:27
milanshardy, not really, I don't14:27
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574314:32
*** salmankhan has quit IRC14:33
*** jprovazn has joined #tripleo14:33
jprovazntbarron, ping14:33
tbarronjprovazn: hey!14:34
openstackgerritMartin André proposed openstack/tripleo-docs master: Drop docker_name_space option  https://review.openstack.org/52668914:34
jprovazntbarron, hi, how is it going?14:34
tbarronjprovazn: Pretty good.  How are *you* ?  I'm more or less the same, you're the one with a new context.14:35
weshay|ruckSlower, yo yo ping me today. I miss you14:36
jprovaznjprovazn, not bad, little bit rush, thanks :). I was wondering if any help wit patches is needed?14:36
tbarronjprovazn: I'm starting to deploy with dsneddon's 5000+ line composable network patch :D14:36
jprovazntbarron, ah, cool14:36
tbarronjprovazn: last deploy I hit some issue with the ganesha VIP though ceph-nfs and manila-share came up OK14:37
tbarronjprovazn: I think we'll have a bit of instability until the composable net patch settles and merges14:37
jprovazntbarron, well, 5000+ line patch? it might be it doesn't pass the first time, but second time I'm sure it will be ok14:37
*** links has quit IRC14:37
tbarronjprovazn: it's on PS#4 now, I"m sure PS#5 will be perfect14:38
*** trozet has quit IRC14:38
*** ramishra has joined #tripleo14:38
jprovazntbarron, yes - ganesha vip depends on proper networking14:38
tbarronjprovazn: Thank you for checking though.  If I get too stuck it will probably be evident from comments in the reviews.14:38
jprovazntbarron, you may at least quickly check if you see something passed to puppet14:39
tbarronjprovazn: like when I forgot about the validation changes you were making (that's all ironed out now)14:39
jprovazntbarron, cool, I didn't have much time this week to keep an eye on them, but it should be better next week14:39
tbarronjprovazn: yeah, I'll look at what we're passing for the VIP when I next re-deploy14:39
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common master: SRIOV derive parameters workflows  https://review.openstack.org/52226514:40
tbarronjprovazn: well I'm surprised you are here at all and haven't just filtered away the emails from those reviews!14:40
tbarronjprovazn: Any advice you have is way above the call of duty.14:40
tbarronjprovazn: I guess you are just missing pacemaker and tripleo too much.14:40
jprovazntbarron, hehe, well I just didn't read them (which is basically quite close to filtering)14:41
jprovazntbarron, of course I d14:41
jprovazno14:41
*** salmankhan has joined #tripleo14:42
ansiwenmandre: did you tell me once, that the dirs under /var/log/containers and /var/lib/config-data should be named after the actual container name? this is obviously not the case currently, so I wonder how I should name the log directory now...14:43
mandreansiwen: well, both /var/log/containers/<service> and /var/lib/config-data/<service> can apply to a group of containers, so they likely do not match the container names 1 to 114:45
mandreansiwen: is it still the ec2-api vs ec2_api vs ec2api?14:45
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Move step 1 preparation to deploy-steps-tasks.yaml  https://review.openstack.org/52526014:46
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Convert bind IPs to hiera interpolation  https://review.openstack.org/52669214:46
ansiwenmandre: ok, I remembered you wanted to keep them separately by container, maybe I remembered wrong14:46
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574314:47
ansiwenmandre: no, in this case it's nova_libvirt container, I used /var/log/containers/nova_libvirt first, but owalsh correctly mentioned, that we don't do that in other cases14:47
*** yamahata has quit IRC14:47
owalshansiwen: why not add mandre as a reviewer?14:48
*** gbarros has quit IRC14:48
mandreansiwen: yes, add me to reviewer please14:49
ansiwenmandre: in the case of ec2_api we also used two directories, ec2_api and ec2_api_metadata14:49
ansiwenowalsh: yup, done14:49
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Convert bind IPs to hiera interpolation  https://review.openstack.org/52669214:51
*** artom has joined #tripleo14:51
*** Guest9054 is now known as melwitt14:52
*** fzdarsky has joined #tripleo14:53
*** dhill_ has quit IRC14:53
*** eck` is now known as eck`gone14:53
*** jprovazn has quit IRC14:54
*** ihrachys has joined #tripleo14:54
*** rcernin has quit IRC14:55
EmilienMhello14:56
EmilienMslagle: ok14:56
EmilienMslagle: I think it's in puppet-tripleo/manifests/profile/base/rabbitmq.pp14:58
EmilienMslagle: see what lyarwood is doing for the ffu, https://review.openstack.org/#/c/522550/9/manifests/profile/base/rabbitmq.pp14:58
derekhI'm running a command to download images to my local repo "openstack overcloud container image upload " , they appear to download then get removed, why do they get removed again ?? Removing local copy of trunk.registry.rdoproject.org/master/centos-binary-neutron-l3-agent:tripleo-ci-testing14:58
*** eck`gone is now known as eck`14:58
*** dhill_ has joined #tripleo14:59
shardyderekh: does it just remove the local download after it's been pushed to the local registry, e.g to save duplicate images using disk space?15:01
derekhshardy: I'm left with nothing in the registry15:01
Tenguwow. the queue in the CI is impressive.15:01
dtantsurfolks, especially interested in containerized undercloud, looking for reviews on https://review.openstack.org/#/c/524577/15:02
derekhshardy: I can see images appear and then vanish15:02
dtantsurthis is needed for switch to hardware types, already has 1x +215:02
*** trown is now known as trown|brb15:02
*** chlong has joined #tripleo15:04
shardyderekh: ouch, that sounds wrong - this is what I ran yesterday and it worked OK15:04
shardyhttp://paste.openstack.org/show/628466/15:04
shardythe script is from quickstart, I ran it and ended up with a bunch of images in the local registry15:05
shardyhttp://paste.openstack.org/show/628467/ derekh that's the catalog15:06
*** trown|brb is now known as trown15:06
mwhahahaEmilienM: slagle: could that be related to the fix that bandini had to do in puppet-rabbitmq to allow better nooping15:07
slaglemwhahaha: possibly. i'm not familiar with that one15:08
EmilienMI don't know what we did but looking at http://tripleo.org/cistatus.html - we don't have much OVB timeouts15:08
mwhahahadtantsur: are they just fixes or is it further implementation items?15:08
mwhahahaslagle: https://github.com/voxpupuli/puppet-rabbitmq/pull/67215:09
*** BryanS68 has joined #tripleo15:09
slagleah, no wonder i could not find it15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647515:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173663415:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704215:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1736634 in tripleo "CI/oooq: some Ansible tasks run multiple times (causing job timeouts)" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)15:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)15:10
*** ramishra has quit IRC15:11
slagleyea, and it seems lyarwood is trying to fix the same issue15:11
derekhshardy: maybe its because I pulled them from trunk.registry.rdoproject.org I'll try docker.io15:11
*** jaganathan has quit IRC15:12
lyarwoodslagle: the rabbitctl use while rabbitmq-server is down issue?15:12
mwhahahalyarwood: it's the rabbitmq_user provider not properly being nooped. it is calling rabbitctl15:13
slaglelyarwood: yes15:13
mwhahahaso in theory bandini's patch to puppet-rabbitmq might fix it. it's only in master right now15:13
slaglelyarwood: i hit it a slightly different way: https://bugs.launchpad.net/tripleo/+bug/173717415:13
openstackLaunchpad bug 1737174 in tripleo "docker-puppet.py: rabbitmq -q list_users run during stack update" [High,In progress] - Assigned to James Slagle (james-slagle)15:13
lyarwoodmwhahaha / slagle ; cool, I'll try to retest with that, would be nice to avoid the workaround I added tbh15:14
mwhahahalyarwood: we were looking to backport that to pike but i wanted to make sure it didn't break master first15:14
lyarwoodmwhahaha: kk, I only need it on master FWIW15:15
mwhahahathat should be available as of yesterday15:15
lyarwoodmwhahaha: ack thanks15:15
dtantsurmwhahaha: something like "oh, I forget to set this option"15:20
mwhahahadtantsur: no that's ok15:20
*** trozet has joined #tripleo15:20
*** abregman is now known as abregman|afk15:20
mwhahahadtantsur: that would be a bug on the implementation :D15:20
mwhahahawhich is perfectly acceptable to fix in m215:20
mwhahahaer m315:20
mwhahaha"stablization"15:21
dtantsurI see, yeah :) I'm bumping the SSH madness to Rocky.. I need to think it through15:21
dtantsurheh15:21
mwhahahadmsimard: dns failures looking up the mirror.dfw.rax.openstack.org in CI http://logs.openstack.org/90/526090/9/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/29459a4/job-output.txt.gz#_2017-12-07_23_08_36_07222815:21
mwhahahadtantsur: good call15:21
dmsimardmwhahaha: looking15:21
mwhahahadmsimard: it was in the tempest run meaning it had been working for some time15:22
dmsimardyeah I can tell15:23
dmsimardlooks like it's in cache, too http://logs.openstack.org/90/526090/9/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/29459a4/logs/dns_cache.txt.gz15:23
*** fzdarsky has quit IRC15:23
*** gfidente has quit IRC15:25
*** tongl has joined #tripleo15:25
jaosoriormwhahaha, slagle: is there a gate that uses network isolation and the config-download method?15:26
mwhahahajaosorior: i don't think so, don't we only do network isolation in ovb?15:26
jaosoriorand ovb doesn't do config-download?15:26
mwhahahanot yet i don't beleive15:27
*** radeks has quit IRC15:27
mwhahahaor better english version: i don't believe it does yet15:27
jaosoriorok15:27
jaosoriorweshay|ruck, shardy: Is there a plan to move one of the ovb jobs to run with config-download?15:28
mwhahahamaybe we can create a new one on rdo cloud?15:28
mwhahaharight now they are ipv4 and ipv615:29
*** milan has quit IRC15:29
mwhahahaor we could move the ipv4 one to config download15:29
jaosoriormwhahaha: I wanna start testing the IPSEC stuff somewhere :D15:29
mwhahahapffft ci coverage15:29
mwhahahawhat a novel concept15:29
jaosoriorand basically the two requirements are: net-iso and config-download15:29
mwhahahawhy don't we create a new job for that15:29
jaosoriorI would sure dig that15:29
mwhahahato test all those bits15:30
mwhahaharather than mess with teh ha ones15:30
jaosoriormwhahaha: I would prefer if pacemaker was running on the job (since there is a resource agent that uses pacemaker as part of the IPSEC ansible role)15:31
slaglejaosorior: i've been working on ovb-ha with config-download: https://review.openstack.org/#/c/525331/15:31
jaosoriorslagle: nice!15:31
mwhahahajaosorior: if it's an ovb job it would make sense to have an ha job, just wondering if we should have more of an experimental version15:32
mwhahahasince it's the bleeding edge stuff15:32
jaosoriormwhahaha: I'm fine with anything as long as we constantly test it somewhere15:32
slagleyea, right now i'm just hijacking featureset001. once passing, we can setup an experimental15:32
openstackgerritGabriel Szasz proposed openstack/tripleo-heat-templates master: Add Novacontrol role  https://review.openstack.org/52650115:34
*** Goneri has quit IRC15:34
openstackgerritJames Slagle proposed openstack/os-net-config master: Only log that mapping file is in use if it's actually true  https://review.openstack.org/52670915:35
*** Goneri has joined #tripleo15:36
*** fzdarsky has joined #tripleo15:39
openstackgerritDan Prince proposed openstack-infra/tripleo-ci master: Update reviewday project list  https://review.openstack.org/52671215:40
openstackgerritJames Slagle proposed openstack/os-net-config master: Use INFO log level for getting nics  https://review.openstack.org/52671315:41
openstackgerritJames Slagle proposed openstack/os-net-config master: Use INFO log level for getting nics  https://review.openstack.org/52671315:42
*** paramite has quit IRC15:45
*** shardy has quit IRC15:47
*** mrch has quit IRC15:49
*** paramite has joined #tripleo15:51
openstackgerritMartin André proposed openstack/python-tripleoclient master: Switch to tripleomaster registry  https://review.openstack.org/52671415:55
*** salmankhan has quit IRC15:57
*** rfolco is now known as rfolco|brb15:57
openstackgerritGabriel Szasz proposed openstack/tripleo-heat-templates master: Add Novacontrol role  https://review.openstack.org/52650115:58
*** aufi has quit IRC16:02
EmilienMdo we still need alert on https://bugs.launchpad.net/tripleo/+bug/1737042 ?16:02
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)16:02
*** salmankhan has joined #tripleo16:02
*** artom has quit IRC16:03
jaosorioralright folks! Imma be off 2 weeks and a half cause of vacations :D16:04
jaosoriorhappy holidays everyone!16:05
jaosoriorbest wishes16:05
slaglejaosorior: see you in 2018 bro16:05
jaosoriorlol16:05
jaosoriortrue that16:05
jaosoriorsee you next year!16:05
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Merge undercloud service env files into undercloud env files  https://review.openstack.org/52672316:06
mwhahahajaosorior: what who said you could take a vacation16:06
mwhahahajaosorior: did you properly apply for an FFE16:07
* mwhahaha -2's jaosorior's vacation16:07
jaosoriorlol16:07
*** atarlov has quit IRC16:08
*** paramite has quit IRC16:10
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Don't pass service env files to undercloud deploy  https://review.openstack.org/52672416:10
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647516:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704216:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)16:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)16:10
*** tongl has quit IRC16:11
*** tongl has joined #tripleo16:12
*** jaosorior has quit IRC16:13
*** itlinux has joined #tripleo16:14
*** weshay|ruck is now known as weshay|ruck|MOD16:15
openstackgerritMartin André proposed openstack/tripleo-common master: Switch to tripleomaster registry  https://review.openstack.org/52672616:15
openstackgerritMartin André proposed openstack/tripleo-docs master: Switch to tripleomaster registry  https://review.openstack.org/52672916:20
*** yprokule has quit IRC16:20
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Merge undercloud service env files into undercloud env files  https://review.openstack.org/52672316:21
*** mcornea has quit IRC16:26
Slowerweshay|ruck|MOD: haha :)16:29
*** rfolco|brb is now known as rfolco16:32
openstackgerritGabriel Szasz proposed openstack/tripleo-heat-templates master: Add Novacontrol role  https://review.openstack.org/52650116:33
dtantsurfolks, can someone explain me how additional packages work in the containers world?16:35
dtantsurlike, previously we just installed everything on overcloud-full via DIB16:35
dtantsurand now? specifically, if I need to add a package or two to ironic-conductor container, what should I do?16:36
*** s_b_34343 has joined #tripleo16:36
mwhahahayou have to add it to the kolla config if it's not already there16:36
mandredtantsur: depends if your package is specific to tripleo or is useful in the general case16:36
mwhahaha(is my understanding)16:36
*** fzdarsky has quit IRC16:36
dtantsurmandre: assuming it should not go to RPM dependencies16:36
dtantsurlike, we have an optional deploy method in ironic that uses ansible16:36
dtantsurwhere should I put the ansible dependency, given that RDO is not keen on making ironic-conductor depend on it?16:37
dtantsurmandre, mwhahaha ^^^16:37
mwhahahait's an optional feature for ironic sounds like kolla might like support for it16:37
mwhahahait's not specific to tripleo so IMHO kolla is the spot to add it16:37
mandredtantsur: sooo... how likely is it that people will use ansible in that image?16:37
mandredtantsur: if that's specific to tripleo, we can easily add that to our kolla customization16:38
dtantsurmandre: very likely for containerized undercloud. not very likely for overcloud.16:38
*** hewbrocca is now known as hewbrocca_afk16:38
dtantsurbut we should support in in the overcloud eventually as well16:38
mandrebut in general we prefer to submit patches to kolla directly so everyone can benefit16:38
dtantsurmandre: this raises a question: what if people using kolla don't want a particular driver?16:39
dtantsurthis is not a new problem, to be fair.. just ironic gets more and more drivers :)16:39
mandredtantsur: kolla's meant to be customized for your own usage, that's what we do in tripleo16:40
*** ludovic has quit IRC16:40
mandredtantsur: you'll need to add your package to https://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_kolla_template_overrides.j216:40
*** itlinux has quit IRC16:42
mandredtantsur: some more docs at https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/3rd_party.html#building-new-containers-with-kolla-build16:42
mwhahahaovercloud prep containers failed in the gate with a glorious None: None return 1 http://logs.openstack.org/61/525761/5/gate/tripleo-ci-centos-7-containers-multinode/ac5497b/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2017-12-08_15_50_5316:43
* mwhahaha would like more logging in overcloud container * commands16:43
*** itlinux has joined #tripleo16:43
*** dpawar has quit IRC16:44
mandremwhahaha: wow it's beautiful16:45
dtantsurthanks mandre, I'll check it16:46
*** dciabrin_ has joined #tripleo16:51
*** dciabrin__ has joined #tripleo16:51
*** dciabrin_ has quit IRC16:51
*** d0ugal has quit IRC16:52
mwhahahaweshay|ruck|MOD, Slower: just fyi looks liek some of the post_config items from instack-undercloud are missing in the containerization of the undercloud https://github.com/openstack/instack-undercloud/blob/5565e25bedd59026a567e890edd3400010226edb/instack_undercloud/undercloud.py#L174816:52
mandremwhahaha: I think we can get more logs by repeating the --verbose16:52
*** dciabrin has quit IRC16:52
mwhahahamandre: hmm ok16:53
*** d0ugal has joined #tripleo16:54
*** d0ugal has quit IRC16:54
*** d0ugal has joined #tripleo16:54
*** fragatina has quit IRC16:57
*** dtrainor has joined #tripleo16:57
*** marios has quit IRC16:58
dtantsurmandre, mwhahaha https://review.openstack.org/#/c/526734/ is some CI covering it?16:59
*** fragatina has joined #tripleo17:00
mwhahahain kolla? not sure about their ci17:00
*** itlinux has quit IRC17:00
*** holser has quit IRC17:00
dtantsuranyway, the driver list is outdated, so it was good to update it :)17:01
mwhahahayet another place to update :D17:02
mwhahahayapu17:02
dtantsur\o/17:02
*** bogdando has quit IRC17:03
*** gvrangan has joined #tripleo17:03
*** fragatina has quit IRC17:05
*** itlinux has joined #tripleo17:05
*** dtrainor has quit IRC17:06
*** chlong has quit IRC17:07
*** ecerquei_ has joined #tripleo17:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647517:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704217:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)17:10
*** ecerquei has quit IRC17:12
Slowermwhahaha: ah yes was trying to figure out what was going on there17:12
*** derekh has quit IRC17:12
*** lucasagomes is now known as lucas-afk17:12
*** jpich has quit IRC17:12
*** fpan has joined #tripleo17:12
*** tongl has quit IRC17:17
dsneddontbarron, I've run into an issue with the composable networking patch that renders the NIC configs. Right now we render all networks into the role, even if the network is disabled.17:18
*** panda is now known as panda|off17:18
dsneddontbarron, This allows for continuity with older versions that had a Management IP defined even if the deployment didn't use that network.17:19
tbarrondsneddon: ack17:19
dsneddontbarron, I'm thinking of changing it so that only enabled networks are included in the role, but changing Management to be enabled by default but not included in any roles.17:19
dsneddontbarron, So I'm going to test that out shortly.17:19
*** karthiks has quit IRC17:19
tbarrondsneddon: ok, I'll look for the patch and re-deploy with it17:20
dsneddontbarron, What I mean by "included in the role" is that the role will get an actual Management IP, rather than the network just existing with no ports on the role.17:20
tbarrondsneddon: right now I'm running with PS#3 just so I can check out other stuff in the deploy (not really networking related)17:20
tbarronthough everything is related to neworking :D17:21
dsneddontbarron, Sounds good. PS#3 didn't check to see if a network was disabled or not.17:21
tbarrondsneddon: I *think* I understand, will study the delta to your next patch and might ask dumb questions then17:21
dsneddontbarron, Thanks17:22
*** dtantsur is now known as dtantsur|afk17:24
*** mdnadeem has quit IRC17:26
*** tesseract has quit IRC17:29
*** raildo has quit IRC17:29
*** dhill_ has quit IRC17:29
*** jkilpatr has joined #tripleo17:34
*** dhill_ has joined #tripleo17:35
*** jkilpatr has quit IRC17:39
*** fpan has quit IRC17:42
*** fragatina has joined #tripleo17:43
mwhahahajrist: you around?17:44
*** dhill_ has quit IRC17:45
*** gvrangan has quit IRC17:46
*** gbarros has joined #tripleo17:47
*** dhill_ has joined #tripleo17:47
*** pblaho has quit IRC17:48
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/ocata: Prepare 6.5.6 (ocata)  https://review.openstack.org/52674217:49
EmilienMjrist: afik he's on pto17:49
mwhahahaslacker17:49
mwhahahano biggie17:49
EmilienMbut I might be wrong17:49
*** jkilpatr has joined #tripleo17:53
*** fpan has joined #tripleo17:53
*** raildo has joined #tripleo17:53
*** dhill_ has quit IRC17:56
*** dhill_ has joined #tripleo17:56
*** morazi has quit IRC17:59
openstackgerritJames Slagle proposed openstack/tripleo-common master: Break up messages to avoid max message size  https://review.openstack.org/52649217:59
EmilienMweshay|ruck|MOD: have we figured the issue with jenkins URL for RDO CI jobs?18:00
EmilienMI guess it's not a huge deal, since logs can be found in the jenkins job18:01
EmilienMbut we'll probably need to announce and document that new thing18:01
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Increase logging for overcloud image uploads  https://review.openstack.org/52674618:01
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Increase logging for overcloud container uploads  https://review.openstack.org/52674618:01
*** openstackgerrit has quit IRC18:03
*** fpan has quit IRC18:05
*** jkilpatr has quit IRC18:06
*** ebarrera has joined #tripleo18:08
*** tongl has joined #tripleo18:10
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647518:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704218:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)18:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)18:10
*** openstackgerrit has joined #tripleo18:11
openstackgerritJames Slagle proposed openstack/tripleo-common master: Break up messages to avoid max message size  https://review.openstack.org/52649218:11
*** raildo has quit IRC18:12
*** raildo has joined #tripleo18:13
ansiwenmy quickstart fails in repo_setup.sh with virt-customize: error: libguestfs error: lvs: lvm lvs --help:   Invalid units specification18:13
mwhahahaansiwen: what version of lvm? There's https://bugzilla.redhat.com/show_bug.cgi?id=147501818:16
openstackbugzilla.redhat.com bug 1475018 in lvm2 "Downgrading LVM2 from 7.4 to 7.3 causes "Invalid units specification" in every command" [Unspecified,Closed: nextrelease] - Assigned to lvm-team18:16
mwhahahaas we don't see this in CI, i assume it's something specific to your env18:16
ansiwenmwhahaha: which version _where_ :-) on my VIRTHOST?18:16
mwhahahanot sure would need to know where the task that ran that failed, it could possibly be the virthost18:17
mwhahahai think that's the part that injects the repos into the overcloud images which might be done on the undercloud actually18:17
ansiwenmwhahaha: fatal: [127.0.0.2]: FAILED! => ... so if 127.0.0.2 is my VIRTHOST, it's probably there, right?18:19
ansiwenmwhahaha: would make sense, I didn't upgrade any packages there since ages...18:20
mwhahahaansiwen: it's localhost of whereever your running quickstart from. so if you're running that on the virthost yes18:20
ansiwenmwhahaha: I'm starting quickstart.sh with 127.0.0.2 as VIRTHOST, because I read that somewhere (was a localhost issue I think)... so yes, that's my VIRTHOST18:21
ansiwenmwhahaha: I will just upgrade my packages there and see if it helps, thanks!18:21
*** jkilpatr has joined #tripleo18:22
*** ebarrera has quit IRC18:23
EmilienMslagle: looking at https://review.openstack.org/#/c/525331/ - looks like it failed on "/etc/ssh/ssh_host_rsa_key.pub\\Overcloud configuration failed"18:28
*** tosky has quit IRC18:30
*** salmankhan has quit IRC18:31
slagleEmilienM: no. it was the zaqar max message size again18:33
slagleEmilienM: you can see the error in the mistral executor log18:33
EmilienMyeah I was wondering18:33
slagleEmilienM: i'm trying to fix it in one of the depends-on18:33
EmilienMok18:33
openstackgerritIan Main proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers  https://review.openstack.org/51897218:34
slagleand I guess I need to propagate this error to the client somehow. although I don't really see how right off18:34
slaglewe'd need to monitor both the message queue and the status of the workflow18:34
slaglewhich we can really only do with mulitple threads18:35
*** pkovar has quit IRC18:48
*** tosky has joined #tripleo18:54
*** zaneb has quit IRC19:05
*** dhill_ has quit IRC19:07
*** dhill_ has joined #tripleo19:08
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647519:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704219:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)19:10
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: install python-neutron-tests for stable releases  https://review.openstack.org/52675719:10
*** Goneri has quit IRC19:10
*** BryanS68 has quit IRC19:13
*** ebarrera has joined #tripleo19:13
*** BryanS68 has joined #tripleo19:14
*** s_b_34343 has quit IRC19:18
*** zaneb has joined #tripleo19:19
*** tongl has quit IRC19:21
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart-extras master: install python-neutron-tests for stable releases  https://review.openstack.org/52675719:23
*** moshele has joined #tripleo19:24
*** raildo has quit IRC19:25
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart-extras master: install python-neutron-tests for stable releases  https://review.openstack.org/52675719:25
*** morazi has joined #tripleo19:28
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: Merge tripleo-upgrade repo from redhat-openstack namespace  https://review.openstack.org/52414119:31
*** gbarros has quit IRC19:34
openstackgerritJames Slagle proposed openstack/os-net-config master: Use INFO log level for getting nics  https://review.openstack.org/52671319:35
*** moshele has quit IRC19:36
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: Merge tripleo-upgrade repo from redhat-openstack namespace  https://review.openstack.org/52414119:36
*** moshele has joined #tripleo19:37
mwhahahaEmilienM: did you add a linters job to tripleo-upgrades19:46
mwhahahaEmilienM: or will you if one isn't already there. the tox task is there19:47
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Generate a default keypair for nova.  https://review.openstack.org/52676619:51
openstackgerritIan Main proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744519:54
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574319:55
*** raildo has joined #tripleo19:55
*** BryanS68 has quit IRC20:08
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647520:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704220:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)20:10
*** BryanS68 has joined #tripleo20:13
*** ChanServ has quit IRC20:17
EmilienMmwhahaha: I'm on my phone now, lunch. See project config, I sent a patch20:22
mwhahahaEmilienM: sure thanks20:22
*** ChanServ has joined #tripleo20:24
*** barjavel.freenode.net sets mode: +o ChanServ20:24
*** Goneri has joined #tripleo20:26
*** ecerquei_ has quit IRC20:27
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: install python-neutron-tests for stable releases  https://review.openstack.org/52675720:42
weshay|ruck|MODEmilienM, mwhahaha need review on https://review.openstack.org/#/c/526757/20:44
weshay|ruck|MODthanks20:44
weshay|ruck|MODbrb20:45
*** jkilpatr has quit IRC20:47
*** dparkes has quit IRC20:49
*** dparkes has joined #tripleo20:50
*** itlinux has quit IRC20:55
*** itlinux has joined #tripleo20:59
*** moshele has quit IRC21:01
*** pchavva has quit IRC21:02
*** holser has joined #tripleo21:03
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647521:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704221:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)21:10
*** rlandy has quit IRC21:10
ihrachyswhen I reboot controller, I no longer can ssh into it. is it known behavior?21:10
mwhahahano21:14
mwhahahawhat do you mean you can no longer ssh to it21:14
mwhahahaas heat-admin?21:14
ihrachysyeah21:15
ihrachysI can ping the node but not ssh21:15
*** jmelvin has quit IRC21:16
ihrachysit's btw deployed with infrared21:16
ihrachyson libvirt21:16
openstackgerritMerged openstack/tripleo-quickstart master: Make repo-setup script name as a parameter  https://review.openstack.org/50309021:16
mwhahahaonly thing i could think is that it's still booting21:16
mwhahahaso network is up but ssh isn't21:16
mwhahahabut no, that's not normal behavior. console show anything?21:16
ihrachysyeah right. I am more asking, why it's not up. seems weird.21:16
ihrachysmwhahaha, no console type I tried is enabled21:18
ihrachysnot serial, not novnc, not console-log21:18
ihrachysI guess I can try virsh..21:18
mwhahahait's libvirt so virsh21:18
mwhahahaor vnc to libvirt21:18
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: install python-neutron-tests for stable releases  https://review.openstack.org/52675721:19
ihrachysmwhahaha, console controller-0 just hangs there, as if there is no getty there or smth21:20
ihrachysif I do same for a node I haven't rebooted it gives getty21:20
* mwhahaha shrugs21:20
*** etingof has quit IRC21:22
ihrachyssigh. it takes a while to deploy this thing :(21:22
mwhahahapretty much21:23
ihrachysbefore I go with it, I noticed that I don't have a mariadb unit on controller-0 though I see traces of the process in syslog21:23
ihrachysI was wondering, is it running in a container now?21:23
mwhahahayes it should be21:23
mwhahahayou could always try virsh restarting the node21:23
*** holser has quit IRC21:25
aleeEmilienM, well - so this is interesting.  https://review.openstack.org/#/c/524064/  glance image signing test pass , but the volume encryption tests fail21:26
openstackgerritwes hayutin proposed openstack/tripleo-ui master: DO NOT MERGE, used in CI ONLY  https://review.openstack.org/42202521:27
aleeEmilienM, this is because nova is configured to check the signatures of images, and is coded to fail if it canot verify an image - even if no signature is provided21:27
aleeEmilienM, so the volume encryption tests fail because the images used there are not signed21:27
*** holser has joined #tripleo21:28
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: update the mock change to test dlrn patches  https://review.openstack.org/52678021:29
*** akrivoka has quit IRC21:29
aleeEmilienM, ah - actually - there is a solution ..21:30
ihrachysmwhahaha, total reset of the node helped. apparently it was stuck on some service to reboot (I rebooted with just calling 'reboot' from its console)21:30
mwhahahaihrachys: ah so it was hung on something shutting down or something21:31
ihrachysyeah21:31
* mwhahaha blames pacemaker because why not21:31
*** etingof has joined #tripleo21:37
EmilienMmwhahaha: back online, see https://review.openstack.org/#/c/526763/21:37
EmilienMmwhahaha: is that what we want, right?21:37
EmilienMweshay|ruck|MOD: ack21:37
mwhahahamaybe?21:37
mwhahahawe have a linters task21:37
mwhahahadoes ansible-lint-jobs run tox linters21:37
EmilienMmwhahaha: which one?21:37
mwhahahaEmilienM: https://github.com/openstack/tripleo-upgrade/blob/master/tox.ini#L5821:38
EmilienMI guess so21:38
EmilienMah21:38
EmilienMok so we don't need that job21:38
EmilienMI'll abandon it21:38
mwhahahawe just need tox -e linter21:38
mwhahahato cover bash/pep8/ansible21:38
EmilienMmwhahaha: where do we do that again?21:39
mwhahahaisn't there an openstack-linters job21:39
mwhahahai think that's the one we want21:39
mwhahahaopesntack-tox-linters21:40
mwhahahawe can drop pep8 and just run openstack-tox-linters21:40
EmilienMok21:40
EmilienMmwhahaha: and keep python-27 right?21:40
mwhahahai guess21:41
EmilienMk21:41
* mwhahaha goes back to updating blueprints21:42
EmilienMmwhahaha: https://review.openstack.org/52676321:43
*** lblanchard has quit IRC21:43
mwhahahaEmilienM: should we just v3 the jobs in tree?21:44
*** trown is now known as trown|outtypewww21:44
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Change volume encryption test to use barbican_tempest test  https://review.openstack.org/52678421:45
*** rcernin has joined #tripleo21:45
EmilienMmwhahaha: not these ones21:45
EmilienMmwhahaha: infra doesn't want21:45
mwhahahak21:45
EmilienMcan't remember why21:45
mwhahahareasons probably21:46
*** gbarros has joined #tripleo21:46
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia  https://review.openstack.org/48745021:47
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Add image signing tests with barbican  https://review.openstack.org/52406421:48
mwhahahaEmilienM, is https://blueprints.launchpad.net/tripleo/+spec/tripleo-composable-upgrade-job implemented?21:50
EmilienMyes, we can close it imho21:51
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper  https://review.openstack.org/51540221:51
mwhahahaEmilienM: https://blueprints.launchpad.net/tripleo/+spec/tripleo-deploy-guide21:53
openstackgerritJames Slagle proposed openstack/tripleo-common master: DO NOT MERGE - add some logging  https://review.openstack.org/52678821:53
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533121:54
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run Octavia configuration on the overcloud  https://review.openstack.org/50819521:54
mwhahahaEmilienM: do we want to actuallyw rit ethe deploy/install guide in m3?21:55
mwhahahaEmilienM: or should we just close this out21:55
EmilienMmwhahaha: close this out21:55
EmilienMwe have some content now...21:55
EmilienMhttps://docs.openstack.org/tripleo-docs/latest/21:55
EmilienMI think it's not in bad shape21:55
EmilienMI found the doc really growing over the last months21:56
mwhahahak21:56
EmilienMhuge kudos to Slower and weshay|ruck|MOD for the contributor content21:57
weshay|ruck|MODit's going to be huuuuuge21:58
weshay|ruck|MODreally bigly21:58
EmilienMmwhahaha: https://review.openstack.org/526746 can you explain why having two --verbose?22:00
mwhahahaEmilienM: check the logs22:00
mwhahahaEmilienM: double verbose basically switches it to debug22:00
EmilienMlol22:00
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add glance config for barbican  https://review.openstack.org/52406322:00
EmilienMmwhahaha: is triple verbose show strace and processor instructions?22:00
mwhahahaprobably22:01
*** itlinux has quit IRC22:01
EmilienMthe log file is HUGEEEEE like weshay|ruck|MOD would say22:01
mwhahahaso current http://logs.openstack.org/61/525761/5/gate/tripleo-ci-centos-7-containers-multinode/ac5497b/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2017-12-08_15_50_5322:01
mwhahahawith --verbose --verbose http://logs.openstack.org/46/526746/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/41f1ce2/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz22:01
EmilienMfine by me22:02
weshay|ruck|MODugh22:02
weshay|ruck|MODgate22:02
mwhahahahmm?22:02
EmilienMa bit busy yeah22:02
EmilienMthat?22:02
weshay|ruck|MODwhat actually happened there?22:02
mwhahahadsneddon: is there anything more to do on https://blueprints.launchpad.net/tripleo/+spec/tripleo-lldp-validation ? all the stuff seems merged back in pike22:02
weshay|ruck|MODNone: none22:03
mwhahahaweshay|ruck|MOD: probably just one failed job22:03
mwhahahaoh that22:03
mwhahahayea22:03
*** rcernin has quit IRC22:03
mwhahahano idea, hence why i want --verbose --verbose22:03
dsneddonmwhahaha, That's a good question for bfournie22:03
*** holser has quit IRC22:03
weshay|ruck|MODah22:03
weshay|ruck|MODroger22:03
*** rcernin has joined #tripleo22:03
weshay|ruck|MODmwhahaha, you really specify the --verbose --verbose22:04
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: DNM: test all linters  https://review.openstack.org/52678922:04
EmilienMmwhahaha: ^ this patch should tell us how linter works^22:04
mwhahahabfournie: https://blueprints.launchpad.net/tripleo/+spec/tripleo-lldp-validation i'm going to close this as implemented since there doesnt' appear to be any patches open. There was the one item to add validation, let me know if we need to reopen it and move it to rocky22:04
mwhahahaEmilienM: it should pass fine, it was working fine when i added it :D22:05
mwhahahaEmilienM: now that giant patch, probably won't pass it22:05
EmilienM  The project template "openstack-tox-linters" was not found.22:05
bfourniemwhahaha: there is a patch one sec22:05
mwhahahabfournie: ok22:05
EmilienMwat22:05
EmilienMyou lied to me :(22:06
EmilienMopenstack-tox-linters is a job not a template22:06
EmilienM:'(22:06
mwhahahaEmilienM: oh thought you knew22:07
EmilienMhe lied to me22:07
*** raildo has quit IRC22:07
mwhahahai never said it was a template :D22:07
mwhahahayou just assumed22:07
EmilienMright22:07
bfourniemwhahaha: for that blueprint the reporting landed in Pike, the validation patch is here and should land in queens 2 - https://review.openstack.org/#/c/512375/22:08
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: DNM: test all linters  https://review.openstack.org/52678922:08
mwhahahabfournie: today is the last day for queens2 :D22:09
mwhahahabut i shall review and mark as implemented22:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647522:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704222:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)22:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)22:10
bfourniemwhahaha: oh, I thought from stuart's email we had a couple weeks for patches in progress, but thanks for marking as implemented!22:10
mwhahahanot really22:10
mwhahahacause my email said this week/next :D22:10
mwhahahaother wise you would have FFE'd22:10
bfourniemwhahaha: ok, I'll figure out FFE, I had been going by this - "M2 is Dec. 8.  The soft freeze requirement is that RFEs must be on22:12
bfournieON_DEV, have patches posted _and_ be under active review. The22:12
bfournieexpectation is that these well-progressed patches should be landed22:12
bfourniewithin "weeks" of M2. We are setting this window to end on Jan 1st."22:12
mwhahahabfournie: yea so since it was already posted it's not that big of a deal22:13
mwhahahabfournie: but if there was outstanding work that needed to be done then it would have needed more scrutiny22:13
bfourniemwhahaha: ok, thanks22:13
bfourniemwhahaha: just waiting on some reviews since 10/3122:14
mwhahahabfournie: that's kinda what the irc meetings are for :) poking people22:14
bfourniemwhahaha: yeah agreed, I have to be better with that22:15
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add parameters for Barbican worker image  https://review.openstack.org/52574222:16
bfourniemwhahaha: thanks for +2!22:16
*** itlinux has joined #tripleo22:17
*** itlinux has quit IRC22:18
weshay|ruck|MODmwhahaha, do you have an undercloud running?22:19
mwhahahaweshay|ruck|MOD: that's a very personal question22:19
mwhahahaweshay|ruck|MOD: but yes i've got one around22:19
weshay|ruck|MODor you could go the 8yr old route.. you better catch it then22:19
weshay|ruck|MODmwhahaha, is there a --debug flag on the  openstack overcloud container image upload22:19
EmilienMlol22:20
EmilienM"very personal question"22:20
mwhahahaweshay|ruck|MOD: yea but it's the same output as --verbose --verbose22:20
mwhahahaweshay|ruck|MOD: so we could switch to --debug if you want22:20
EmilienMimho a very personal question would be: do you have a non-containerized overcloud deployment running?22:20
* mwhahaha puts EmilienM in a container22:20
EmilienMopenstack-tox-linters failed on tripleo-upgrade22:21
mwhahahalink?22:21
weshay|ruck|MODmwhahaha, I srlsy thought you were kidding w/ --verbose --verbose22:21
weshay|ruck|MODfucking hard to tell22:21
dpeacockQ: "Do you have an undercloud?" A: "No, it's just the way I am sitting."22:21
mwhahahaweshay|ruck|MOD: was not kidding22:21
weshay|ruck|MODlolz22:21
weshay|ruck|MODjebus22:21
EmilienMhttp://logs.openstack.org/89/526789/2/check/openstack-tox-linters/ac6133f/job-output.txt.gz22:21
weshay|ruck|MODk.. /me fires one up22:21
* weshay|ruck|MOD fires up my undercloud22:21
weshay|ruck|MODOUCH22:21
EmilienM"my undercloud is usally down on Fridays"22:21
EmilienMhttp://logs.openstack.org/89/526789/2/check/openstack-tox-linters/ac6133f/job-output.txt.gz#_2017-12-08_22_17_26_22268622:22
dpeacockOh my.22:22
EmilienMhey we catch errors!22:22
mwhahahaEmilienM: hmm interesting22:22
openstackgerritJames Slagle proposed openstack/tripleo-common master: DO NOT MERGE - add some logging  https://review.openstack.org/52678822:22
EmilienM48 bashate error(s) found22:22
EmilienMlol I quit22:22
mwhahahai didn't get that when i ran locally22:22
EmilienMmwhahaha: probably in  new code?22:22
mwhahahano nothing's merged22:22
EmilienMI commented on https://review.openstack.org/#/c/52414122:23
weshay|ruck|MODEmilienM, don't expose your undercloud to dpeacock, just seems wrong to me22:23
EmilienMwe probably want to fix errors before merging this one22:23
mwhahahayea we need to22:23
dpeacocklol22:23
EmilienMweshay|ruck|MOD: are you jealous if I do?22:24
weshay|ruck|MODa little.. so what?22:24
* EmilienM containerizes weshay as well22:25
weshay|ruck|MODmwhahaha, ok.. going to give it a go on my environment.. fyi https://bugs.launchpad.net/tripleo/+bug/173725822:26
openstackLaunchpad bug 1737258 in tripleo " openstack overcloud container image upload --verbose is not verbose enough" [Critical,Triaged]22:26
mwhahahaweshay|ruck|MOD: i have a patch22:26
mwhahahaweshay|ruck|MOD: https://review.openstack.org/#/c/526746/22:26
mwhahahaweshay|ruck|MOD: you can just look at those logs if you want22:26
mwhahaha:D22:26
slagleanybody want to approve https://review.openstack.org/#/c/524437/ ?22:26
slaglei'd like to reference some docs in a blog post i'm working on22:27
mwhahahapffft documentation22:27
weshay|ruck|MODoh.. that's where the comment came from22:27
slaglei know right ;)22:27
weshay|ruck|MODjebus22:27
*** bfournie has quit IRC22:30
openstackgerritMerged openstack/tripleo-docs master: Document manual config-download  https://review.openstack.org/52443722:32
*** BryanS68 has quit IRC22:33
*** tongl has joined #tripleo22:34
*** skramaja has quit IRC22:39
slaglemwhahaha: thanks!22:41
mwhahahakeep documenting stuff :D22:41
openstackgerritMerged openstack/tripleo-common master: Added ability to generate kek for barbican simple crypto backend  https://review.openstack.org/52533723:07
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173647523:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173704223:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1736475 in tripleo "Telemetry tempest tests fail due to Max retries exceeded with url: /nodata-gnocchi-threshold-resource" [Critical,In progress] - Assigned to wes hayutin (weshayutin)23:10
openstackLaunchpad bug 1737042 in tripleo "ERROR! the role 'tripleo-bootstrap' was not found in /var/lib/mistral roles path" [Critical,Triaged] - Assigned to wes hayutin (weshayutin)23:10
*** fpan has joined #tripleo23:19
openstackgerritMerged openstack/tripleo-specs master: Add blueprint for Logging to stdout and rsyslog  https://review.openstack.org/51000123:22
openstackgerritMerged openstack/instack-undercloud stable/newton: Fix the expirer cron task to run correctly  https://review.openstack.org/51982423:23
openstackgerritMerged openstack/tripleo-quickstart master: remove tempest plugin install from convert image  https://review.openstack.org/52565923:23
*** japestinho has quit IRC23:35
*** yamahata has joined #tripleo23:37
openstackgerritMerged openstack/tripleo-quickstart-extras master: Use cachable facts to allow multiple runs  https://review.openstack.org/52567123:53
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add missing steps from tripleo-ci ovb playbook  https://review.openstack.org/52645123:53
openstackgerritMerged openstack-infra/tripleo-ci master: update zuul cloned repo directory for zuulv3  https://review.openstack.org/52654623:53
openstackgerritMerged openstack/python-tripleoclient master: Support more hardware types and deprecate enabled_drivers  https://review.openstack.org/52457723:53
openstackgerritMerged openstack/tripleo-quickstart master: Clarify SSH server requirements prior to tripleo-quickstart deployment  https://review.openstack.org/52647123:53
mwhahahaweshay|ruck|MOD: EmilienM, was lurking in the infra channel -> http://zuulv3.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario001-multinode-oooq-container23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!