Thursday, 2016-10-06

*** mbozhenko has joined #tripleo00:02
*** mbozhenko has quit IRC00:07
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000400:16
*** bana_k has quit IRC00:18
*** pradk has joined #tripleo00:19
*** bana_k has joined #tripleo00:31
*** thrash is now known as thrash|g0ne00:38
openstackgerritMerged openstack/tripleo-image-elements: Open tripleo-image-elements for Ocata  https://review.openstack.org/38243100:39
openstackgerritMerged openstack/python-tripleoclient: Open python-tripleoclient for Ocata  https://review.openstack.org/38241800:42
*** tobias-fiberdata has quit IRC00:46
*** tobias-fiberdata has joined #tripleo00:46
*** dtrainor has quit IRC00:47
*** dtrainor has joined #tripleo01:02
*** kberger has joined #tripleo01:09
openstackgerritMerged openstack-infra/tripleo-ci: Increase disk_allocation_ratio to 4  https://review.openstack.org/37961801:18
*** bana_k has quit IRC01:18
openstackgerritMerged openstack-infra/tripleo-ci: Add Mistral to the CI status  https://review.openstack.org/37933501:19
openstackgerritMerged openstack-infra/tripleo-ci: Add TripleO scenarios jobs in status dashboard  https://review.openstack.org/36732401:19
openstackgerritDan Sneddon proposed openstack/os-net-config: Modify os-net-config to make changes without bouncing interfaces.  https://review.openstack.org/15273201:28
openstackgerritDan Sneddon proposed openstack/os-net-config: Modify os-net-config to make changes without bouncing interfaces.  https://review.openstack.org/15273201:29
*** fultonj has quit IRC01:30
openstackgerritMerged openstack/diskimage-builder: Create (md5|sha256) checksum files for images  https://review.openstack.org/38057701:45
*** fzdarsky_ has joined #tripleo01:45
*** fzdarsky|afk has quit IRC01:49
*** mbozhenko has joined #tripleo02:03
*** mbozhenko has quit IRC02:08
EmilienMccamacho: ok02:12
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Add generic template for custom roles.  https://review.openstack.org/38158702:13
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: enable undercloud/ssl on multinode jobs  https://review.openstack.org/38208202:15
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120902:15
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000402:15
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: DO NOT MERGE testing switch to Ocata DLRN instance  https://review.openstack.org/38252102:25
*** ramishra has quit IRC02:28
*** ramishra has joined #tripleo02:28
*** tiswanso has joined #tripleo02:40
*** maticue has quit IRC02:55
*** bana_k has joined #tripleo02:59
openstackgerritMerged openstack/tripleo-heat-templates: j2 template role config templates  https://review.openstack.org/38227603:02
*** mbozhenko has joined #tripleo03:02
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000403:04
*** mbozhenko has quit IRC03:07
openstackgerritMerged openstack/puppet-tripleo: Use service-specific servernames for haproxy  https://review.openstack.org/38253403:08
openstackgerritMerged openstack/tripleo-heat-templates: Move the main template files for defalut services to new syntax generation  https://review.openstack.org/38213703:08
*** yamahata has quit IRC03:12
*** bana_k has quit IRC03:16
*** tiswanso has quit IRC03:30
openstackgerritOpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements  https://review.openstack.org/35090503:31
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/37599303:33
*** dtrainor has quit IRC03:33
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements  https://review.openstack.org/37599703:34
openstackgerritRyan Brady proposed openstack/tripleo-common: Port Scale Down Functionality into Workflow  https://review.openstack.org/38270703:35
openstackgerritMerged openstack/python-tripleoclient: Add optional overcloud deploy roles_data.yaml override  https://review.openstack.org/38222403:37
openstackgerritMerged openstack/tripleo-common: Remove references to overcloud-without-mergepy  https://review.openstack.org/38248603:37
openstackgerritMerged openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367403:37
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:39
*** links has joined #tripleo03:47
*** pmannidi has quit IRC03:48
*** pmannidi has joined #tripleo03:50
*** dsariel_ has quit IRC03:59
openstackgerritMerged openstack/tripleo-quickstart: Add swap to the undercloud when using an overcloud image  https://review.openstack.org/37480904:09
*** jaosorior has joined #tripleo04:25
*** pgadiya has joined #tripleo04:28
*** rajinir has quit IRC04:35
*** oshvartz has quit IRC04:36
*** sudswas__ has joined #tripleo04:36
*** sudipto has joined #tripleo04:36
*** sudipto has quit IRC04:42
*** sudswas__ has quit IRC04:42
*** liverpooler has quit IRC04:45
*** yamahata has joined #tripleo04:52
*** pmannidi has quit IRC05:19
*** jprovazn has joined #tripleo05:25
*** mbozhenko has joined #tripleo05:30
*** pmannidi has joined #tripleo05:35
openstackgerritMerged openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/37599305:36
openstackgerritMerged openstack/tripleo-common: Updated from global requirements  https://review.openstack.org/37599705:36
*** mbozhenk1 has joined #tripleo05:42
*** dsariel_ has joined #tripleo05:43
jaosoriorccamacho: hey dude, can you check this out? https://review.openstack.org/#/c/382183/05:43
*** bana_k has joined #tripleo05:44
openstackgerritJulie Pichon proposed openstack/puppet-tripleo: Explicitly use Keystone v2 endpoint in the UI  https://review.openstack.org/38272405:44
*** mbozhenko has quit IRC05:44
openstackgerritMerged openstack-infra/tripleo-ci: Only ask for the overcloud-full image  https://review.openstack.org/36359205:47
openstackgerritMerged openstack-infra/tripleo-ci: Always configure ipv6 address with net-iso  https://review.openstack.org/37492205:53
*** mbozhenk1 has quit IRC05:55
openstackgerritMerged openstack/instack-undercloud: correctly spell yaql_limit_iterators  https://review.openstack.org/38205605:55
*** jaosorior has quit IRC05:55
*** oshvartz has joined #tripleo05:56
*** bana_k has quit IRC05:56
*** jaosorior has joined #tripleo05:57
ccamachojaosorior morning man :)06:01
ccamachodone06:01
jaosoriorccamacho: thanks dude, how's it going?06:01
*** liverpooler has joined #tripleo06:02
ccamachoahahah with this ga stuff.f..06:02
jaosoriorhaha know the feel06:02
*** bana_k has joined #tripleo06:02
jaosoriordaaamn I need coffee06:02
ccamachoall night trying to pass06:02
ccamachoci06:02
jaosoriorand it's getting pretty cold here in Finland06:02
ccamacho:P06:02
ccamachodude you are ir the north pole06:02
ccamachowhat you want :P06:02
*** tzumainn has quit IRC06:02
jaosoriorhahaha it's the south of Finland, it shouldn't be thaaaat bad06:03
jaosoriorccamacho: hey dude, can you re-visit this patch https://review.openstack.org/#/c/366548/ ? There was a merge conflict and I lost all the votes there :/06:03
ccamachojaosorior can you check https://review.openstack.org/#/c/381587/ and https://review.openstack.org/#/c/381593/ I need to finish to land them in order to push an additional fix06:04
ccamachosure dude06:04
jaosoriorccamacho: of course, let me take a look06:05
ccamachojaosorior sorry for not having time for the tests on the TLS changes, Ill do them asap (I have already checked the blog post)06:06
jaosoriorccamacho: is used anywhere yet? https://review.openstack.org/#/c/381587/14/puppet/role.role.j2.yaml06:06
ccamachonot yet06:06
ccamachobut depending on what lands first06:06
jaosoriorcause the output for role.role.j2.yaml will need an extra value06:07
ccamachowill need a fix for https://review.openstack.org/#/c/378764/06:07
ccamachoyeahp06:07
ccamachoI know :)06:07
*** liverpooler has quit IRC06:07
jaosoriorccamacho: ok, lets merge it (since it's not used yet) and that extra value can be added in a subsequent patch.06:07
*** liverpooler has joined #tripleo06:07
ccamachocool06:08
ccamachoHopefully we will merge the backport and have them into ga06:08
ccamachojaosorior btw I already have created the patch here https://review.openstack.org/#/c/382217/06:10
*** numans has joined #tripleo06:11
ccamachoI did that yesterday in the morning for not forgetting about it.06:11
jaosoriorI seee06:11
jaosoriorccamacho: why not just put a depends-on?06:12
ccamachodude, I dont want to disturb the forces as was really painful to pass CI :P06:12
ccamachoand because its not used yet06:13
ccamachoyesterday stable/newton was broken06:13
ccamachobut it will get merged today for sure :)06:14
ccamachoyeeeeahh!06:14
* ccamacho going for coffee06:14
*** rcernin has joined #tripleo06:15
*** bana_k has quit IRC06:15
jaosoriorhaha nice06:16
jaosoriorccamacho: why was stable/newton broken?06:16
ccamacholet me get the fix06:18
ccamachojaosorior this https://review.openstack.org/#/c/382658/06:18
ccamachoafter it got merged I have rechecked lot of stuck submissions06:18
*** bana_k has joined #tripleo06:18
ccamachoand they are all landed by now :)06:19
ccamachoccamacho06:20
*** mbozhenko has joined #tripleo06:20
ccamachojaosorior and this was the error from the jobs 2016-10-05 19:34:52.681 30093 ERROR ironic_inspector.main Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-aa14d9ab-4154-4ed6-a478-3a833a29b263)06:20
*** rasca has joined #tripleo06:20
openstackgerritMerged openstack/python-tripleoclient: Remove auth_required=False from upload image command  https://review.openstack.org/38265606:23
jaosoriorccamacho: wow, that's funky06:25
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Add generic template for custom roles.  https://review.openstack.org/38273706:27
*** bana_k has quit IRC06:27
openstackgerritCarlos Camacho proposed openstack/tripleo-common: Add support to create role main template file based in role.role.j2.yaml  https://review.openstack.org/38273806:27
ccamachoyeahh.. the weird stuff for me was that It was working before man..06:28
ccamachoand mast was working fine...06:28
ccamachos/mast/master/06:29
*** ohamada has joined #tripleo06:29
jaosoriorwhat the hell06:29
jaosoriorccamacho: hey dude, any idea on why this failed to merge? https://review.openstack.org/#/c/381587/06:30
ccamachommm06:31
ccamachocan we recheck it?06:31
ccamachothe jobs were not failing06:32
*** pmannidi_ has joined #tripleo06:32
jaosoriorlets try that06:32
*** pmannidi has quit IRC06:33
ccamachobrb06:34
*** ccamacho has quit IRC06:34
*** kaslcrof has joined #tripleo06:40
jaosoriorgfidente, shadower: Can you guys revisit https://review.openstack.org/#/c/366548/ ? There was a merge conflict and the commit lost all the votes :(06:40
*** dciabrin has joined #tripleo06:41
*** athomas has joined #tripleo06:44
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756206:45
*** masco has joined #tripleo06:46
*** masco has quit IRC06:47
*** masco has joined #tripleo06:48
*** zoli_gone-proxy is now known as zoliXXL06:53
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756206:55
*** cylopez has joined #tripleo07:01
*** b00tcat has joined #tripleo07:04
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Adds Environment File for Removing Sahara during M/N upgrade  https://review.openstack.org/38274807:05
*** zoliXXL is now known as zoli_gone-proxy07:05
*** jlinkes has joined #tripleo07:05
*** aufi has joined #tripleo07:07
*** abehl has joined #tripleo07:09
*** amoralej|off is now known as amoralej07:10
jtomasekd0ugal: looking at https://review.openstack.org/#/c/382277/2 I still see the wrong behavior of the action - acting as a workflow: this is the response I am getting: http://paste.openstack.org/show/584617/07:11
*** tesseract- has joined #tripleo07:11
d0ugaljtomasek: You probably don't have the depends-on :)07:11
d0ugal(and the depends-on for the depends-on)07:11
jtomasekd0ugal: aaha, cool, I probably did not wake up yet...07:12
d0ugaljtomasek: one merged an hour ago, the other had +W, so should soon07:13
jtomasekd0ugal: do you expect it to get to newton mistral?07:13
*** zoli_gone-proxy is now known as zoliXXL07:13
d0ugaljtomasek: no, I think it is too late to be backported :(07:14
jtomasekd0ugal: ok07:14
*** jidar has quit IRC07:16
*** jidar_ has joined #tripleo07:16
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756207:17
*** jidar_ is now known as jidar07:17
*** ebarrera has joined #tripleo07:22
*** jpena|off is now known as jpena07:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756207:24
*** pcaruana has joined #tripleo07:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109407:29
*** ccamacho_ has joined #tripleo07:32
*** shardy_afk is now known as shardy07:34
shardymorning all!07:34
bandinimorning ;)07:34
ccamacho_morning!07:34
shardyCan anyone check https://review.openstack.org/#/c/382480 please, passing CI, one line fix :)07:35
ccamacho_stable/newtow got unstuck yesterday about 1:30am CET+207:35
jaosorioris something wrong in the gate? Several commits have passed all tests and haven't merged. They just seem stalled...07:35
ccamacho_really?07:35
ccamacho_;(07:36
shardyjaosorior: got a link?07:36
jaosoriorshardy: all the commits for the per-network hostnames07:36
jaosoriorshardy: http://status.openstack.org/zuul/07:36
jaosoriorzuul has kept them there, all green, for a while now07:36
*** abregman_ has joined #tripleo07:37
shardyhmm07:37
jaosorioryeah, pretty weird07:40
jaosorioryolanda: you were working in infra, right?07:42
shardyThe top of the gate queue seems to be https://review.openstack.org/#/c/382273/ which has been gating for 1hr4007:42
* shardy checks if we run the ovb jobs in the gate queue07:42
jaosoriorshardy: I thought we weren't07:43
yolandahi jaosorior , i'm on infra yes07:44
*** jpich has joined #tripleo07:44
jaosorioryolanda: hi, I see several tripleo commits here http://status.openstack.org/zuul/ that have passed all tests in the gate, however, it's taking unusually long for them to get merged by zuul. Any idea what could be the issue?07:46
*** masco_ has joined #tripleo07:48
shardyjaosorior: I don't think they have passed all tests in the gate pipeline07:48
shardythe top of the tripleo-ci gate queue is still running the nonha-multinode job07:48
shardyit's just either stalled or taking much longer than normal07:48
jaosoriorshardy: but that is a stable/newton patch, unrelated to these ones07:48
*** zoliXXL is now known as zoli|afk07:49
yolandaso they way it works, is that these patches are queued. The latest in the queue won't be merged until the first one of the queue does07:50
yolandaas soon as non-ha-multinode job finishes, the next ones will be merged in cascade07:51
jaosorioryolanda: I see, so that happens regardless if they're on different branches?07:51
yolandayes, zuul is not considering branches at this point, it only looks for changes with similar jobs07:51
jaosorioraah, I see07:51
*** masco has quit IRC07:51
jaosorioryolanda: thanks for the explanation and sorry for the spam07:51
shardythat makes sense, given that e.g Depends-On isn't branch aware07:51
shardythanks yolanda ! :)07:52
yolandajaosorior, no problems, your concern was logic, and it's not the first time that people ask about it. Actually that should be a nice feature07:52
jaosoriorit sure would be :)07:52
ccamacho_good to hear this :P07:53
gfidentetbarron hey thanks for trying out the manila change08:02
jaosoriorgfidente: thanks for vote dude08:05
*** hewbrocca-afk is now known as hewbrocca08:12
d0ugaljtomasek: it is going to be in Newton :-D08:15
jtomasekd0ugal: cool!08:16
hewbroccaMorning folks08:17
d0ugalMorning!08:17
*** jaosorior is now known as jaosorior_lunch08:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add cloud names to hosts output  https://review.openstack.org/38276908:17
*** ccamacho_ has quit IRC08:19
*** tremble has joined #tripleo08:20
*** tremble has joined #tripleo08:20
*** abregman__ has joined #tripleo08:21
*** abregman_ has quit IRC08:25
*** limao has joined #tripleo08:26
*** dbecker has joined #tripleo08:27
*** limao_ has joined #tripleo08:29
snecklifterHello, basic doc change to help out people running tempest, thanks https://review.openstack.org/#/c/380954/08:31
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Save the result of direct action calls in Mistral  https://review.openstack.org/38277508:32
openstackgerritDougal Matthews proposed openstack/tripleo-ui: Save the result of direct action calls in Mistral  https://review.openstack.org/38277608:32
*** limao has quit IRC08:32
*** fzdarsky_ is now known as fzdarsky08:34
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] Add IPv6 coverage using existing jobs  https://review.openstack.org/38251508:35
*** derekh has joined #tripleo08:38
*** zoli|afk is now known as zoli|wfh08:42
tbarrongfidente: no problem, not sure what's going on there yet08:42
* tbarron starts his first coffee today08:43
shardysnecklifter: lgtm, thanks!08:44
sneckliftershardy: thanks08:44
tbarrongfidente: trying changes is **so** much less painful with the puppet artifacts upload!08:45
shardyGah, all the multinode jobs in the gate queue are timing out :(08:47
*** lhinds is now known as lhinds|away08:47
*** masco__ has joined #tripleo08:48
sshnaidmderekh, hi08:48
derekhsshnaidm: hi ya08:49
*** dtantsur|afk is now known as dtantsur08:49
sshnaidmderekh, maybe you know, what could I adjust to solve such errors? http://logs.openstack.org/94/381094/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c004046/logs/undercloud/var/log/ironic/ironic-conductor.txt.gz#_2016-10-06_08_21_27_69908:49
sshnaidmderekh, is it flavor issue..?08:49
*** limao has joined #tripleo08:50
derekhsshnaidm: looking08:50
sshnaidmderekh, it's my patch issue, not ci08:51
sshnaidmderekh, trying to run quickstart there..08:51
openstackgerritMerged openstack/puppet-tripleo: Enable usage of "short names" for Galera cluster  https://review.openstack.org/38218308:52
*** masco_ has quit IRC08:52
shardyAnyone know how I can connect to the ipv6 telnet console exposed via the zuul queue?08:52
*** limao_ has quit IRC08:52
openstackgerritMerged openstack/tripleo-ui: Add image names to Nodes registration workflow  https://review.openstack.org/38185508:56
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Save the result of direct action calls in Mistral  https://review.openstack.org/38277508:56
therveshardy, Well, I know that I can connect to it08:57
therveshardy, I bet you don't have v6 connectivity?08:57
d0ugalshardy: Yeah, you'll need to connect via something with ipv608:57
d0ugalI also don't have ipv6 :(08:57
d0ugalshardy: http://ipv6-test.com/08:58
*** _milan_ has joined #tripleo08:59
derekhsshnaidm: I don't understand, the error message say "only 4865 MiB " but the overcloud nodes have 41G disks08:59
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add image names to Nodes registration workflow  https://review.openstack.org/38278508:59
openstackgerritJulie Pichon proposed openstack/tripleo-ui: Add image names to Nodes registration workflow  https://review.openstack.org/38278508:59
jpichjtomasek: Sorry!!09:00
jtomasekjpich: lol!09:00
sshnaidmderekh, yeah, it's confusing mee too09:00
jpichjtomasek: :)09:00
jtomasekjpich: what is the chance of doing that at totally same time :D09:01
jtomasekshardy or anyone: I see only OvercloudControlFlavor and OvercloudComputeFlavor in heat validate output, I can't figure out why I don't see other role flavor parameters there or what I need to do to make that happen09:02
jpichjtomasek: On RC day I guess slightly higher than usual! :)09:03
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109409:03
*** chem has joined #tripleo09:04
dtantsurhi owlers, could you please merge a trivial documentation addition? https://review.openstack.org/#/c/382524/09:04
dtantsurI'm really afraid of people getting hit by it09:04
jtomasekjpich: it should be possible to cherry pick on top of other pending patch for subsequent patches (to maintain the order and avoid conflicts) using gerrit, right?09:05
sshnaidmderekh, I'll run it again with saving instackenv.json. It's weird that it's not reproducing on local den env on rh1.09:05
*** abehl has quit IRC09:06
shardytherve, d0ugal: yeah ipv6 is the problem, my isp supports it, but I don't have it enabled on my local network09:06
derekhsshnaidm: you have instackenv.json here http://logs.openstack.org/94/381094/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c004046/console.html#_2016-10-06_07_38_45_86892709:06
shardywas wondering if folks had some other approach to access those consoles09:06
d0ugalshardy: aha, my isp doesn't support it :(09:06
jpichjtomasek: I think the automatic cherry-picking isn't that smart... Or maybe if you specify the commit number instead of a branch name?09:07
*** chem has quit IRC09:07
sshnaidmderekh, oh, right, thanks.. looks correct09:07
shardyjtomasek: it's probably because the count is zero for all other roles09:07
*** chem has joined #tripleo09:07
jpichjtomasek: If you're busy finishing stuff I can do manual cherry-picking for stuff that needs to be backported in a series, just let me know which patches09:08
* derekh is now curious if he can get to IPv609:09
shardyjtomasek: https://review.openstack.org/#/c/374226/ would probably "fix" it for you, but the alternative is to pass *Count parameters to the validation so all roles are non-zero09:09
*** limao_ has joined #tripleo09:11
openstackgerritmathieu bultel proposed openstack/python-tripleoclient: Download templates from swift before processing with heatclient  https://review.openstack.org/37954709:12
jtomasekshardy: interesting, thanks, and also quite cool:) since it technically allow user to configure role parameters only when that role is enabled09:12
jtomasek(in GUI)09:13
snecklifterhi folks, so cinder-backup is not configurable in Mitaka, am I correct?09:13
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375009:13
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs: Recommend using only metadata cleaning with Ironic in virtual overcloud  https://review.openstack.org/38252409:13
*** karthiks has joined #tripleo09:13
dtantsurfixed a typo ^^^09:13
*** limao has quit IRC09:14
shardyjtomasek: Yup - if that's desirable behavior tho please comment on the heat patch I just linked09:15
shardyas that would change it to assume count=1 by default09:15
shardythere are advantages to both approaches, but if folks will rely on the current behavior we probably can't change it09:15
shardythe main disadvantage is we don't fully validate the tree of templates09:16
shardytherve: ^^ FYI ref https://review.openstack.org/#/c/374226/09:16
jtomasekshardy: I don't have a strong opinion and probably favor the one which that patch brings09:16
jtomasekshardy: because in general the best approach is allow user to add/enable roles as he wants (which means manipulate roles_data.yaml somehow from GUI) and if the role is enabled, he can configure it regardless of nodes count being set09:18
shardyjtomasek: Yeah, I think for TripleO that's the right approach09:18
jtomasekjpich: those are GUI patches that need to get reviewed and merged asap https://review.openstack.org/367562 (dependent tripleo-common one is almost merged), https://review.openstack.org/38184509:20
*** egafford has quit IRC09:20
*** tosky has joined #tripleo09:21
*** abehl has joined #tripleo09:21
*** pgadiya has quit IRC09:22
openstackgerritMerged openstack/tripleo-common: Add support to create role main template file based in role.role.j2.yaml  https://review.openstack.org/38159309:26
openstackgerritMerged openstack/tripleo-heat-templates: Fix OpendaylightApiNetwork key naming  https://review.openstack.org/38248009:26
derekhsshnaidm: http://logs.openstack.org/94/381094/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c004046/logs/undercloud/var/log/host_info.txt.gz09:28
derekhsshnaidm: search for "df -h"09:28
jpichjtomasek: Ok, will look and test a bit09:29
derekhthe undercloud node has only 6.5G available, possibly less when that failed, I'd see if you can free more space up09:29
jtomasekjpich: thanks!09:30
derekhsshnaidm: we free up some space here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh#n1909:30
derekhsshnaidm: you could try removing all of /opt/stack/cache I don't think we use anything in there after the zuul setup is done09:31
*** limao_ has quit IRC09:32
*** pgadiya has joined #tripleo09:34
*** jaosorior_lunch has quit IRC09:36
sshnaidmderekh, it doesn't exist when I start install undercloud09:36
*** jaosorior_lunch has joined #tripleo09:36
*** limao has joined #tripleo09:37
derekhsshnaidm: you mean in your manual test?09:37
openstackgerritMerged openstack/tripleo-common: Support node untagging  https://review.openstack.org/37262809:38
sshnaidmderekh, uh, yeah09:38
sshnaidmderekh, forgot that zull installs it there09:38
sshnaidmderekh, but ironic errors are related to ironic node, isn't it?09:39
*** saneax-_-|AFK is now known as saneax09:39
*** mah has joined #tripleo09:40
derekhsshnaidm: I thought that at first but I'm starting to think it might be the undercloud its talking about09:40
mahHi all, I have shutdown my tripleo cluster. then powered them on again. but I can not connect to the undercloud VM and the instances do not get an IP. So what I you should do after rebooting the cluster ?09:41
*** jaosorior_lunch is now known as jaosorior09:44
dtantsurfolks, shardy, EmilienM, I've got a potentially flame war blueprint for you: https://blueprints.launchpad.net/tripleo/+spec/re-enable-cleaning :)09:46
dtantsurlucas-afk, FYI ^^09:46
jaosoriorHey guys, I'm seeing the following errors while trying to install the undercloud:09:46
jaosorior2016-10-06 08:25:04 - ESC[1;31mError: /Stage[main]/Tripleo::Profile::Base::Ui/Package[openstack-tripleo-ui]/ensure: change from absent to present failed: Execution of '/bin/yum -d 0 -e 0 -y install openstack-tripleo-ui' returned 1: Error downloading packages:09:46
jaosorior2016-10-06 08:25:04 -   openstack-tripleo-ui-1.0.3-0.20160929175454.f7297c3.el7.centos.noarch: [Errno 256] No more mirrors to try.ESC[0m09:46
mahHi all, I have shutdown my tripleo cluster. then powered them on again. but I can not connect to the undercloud VM and the instances do not get an IP. So what I you should do after rebooting the cluster ?09:47
jpichjaosorior: Only for that package?09:48
*** masco_ has joined #tripleo09:48
jaosoriorjpich: that's the first one I saw09:49
jaosoriorjpich: nevermind, seems to be more packages09:49
jaosoriorjpich: same with mistral, same with heat packages09:49
*** mbozhenko has quit IRC09:49
jpichjaosorior: Beh. Hopefully a transient error?09:50
*** mbozhenko has joined #tripleo09:50
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart: Stop using deprecated network range  https://review.openstack.org/34344309:50
jaosoriorjpich: I'll try the deployment again, but it has happened twice in the last couple of hours09:50
jpichjaosorior: That's concerning...09:51
*** masco__ has quit IRC09:52
openstackgerritMerged openstack/tripleo-heat-templates: j2 template per-role ServiceNetMapDefaults  https://review.openstack.org/38227309:52
jaosoriorjpich: tried again, same issue :(09:54
*** limao has quit IRC09:54
*** limao has joined #tripleo09:54
jpichjaosorior: I'm not sure if this will help but, just in case it's actually that package causing a problem, can you try adding openstack-tripleo-ui to the list of included packages in /etc/yum.repos.d/delorean-current.repo? I've been meaning to update the docs for a while, CI already does that. I doubt it's causing the issue for peace of mind09:56
jpichjaosorior: I'm setting up a new virt setup at the moment so I'll be trying soon as well09:56
jaosoriorjpich: well, I'm having issues with openstack-mistral-engine and even heat09:56
jaosoriorso I think it might be an issue with the RDO mirrors, asked there09:56
jpichjaosorior: I'll be watching for replies too... can you access other URLs just fine from that env?09:58
jaosoriorjpich: I can09:58
*** mbozhenko has quit IRC10:06
*** limao_ has joined #tripleo10:16
*** ccamacho has joined #tripleo10:17
mariosgfidente: are we goign to target https://bugs.launchpad.net/tripleo/+bug/1630777 for rc3?10:18
openstackLaunchpad bug 1630777 in tripleo "T-H-T manila cepfhs backend sets wrong cephfs_conf_path" [Undecided,In progress] - Assigned to Tom Barron (tpb)10:18
mariosgfidente: https://review.openstack.org/#/c/382671/1 lgtm10:18
*** akrivoka has joined #tripleo10:19
*** mbozhenko has joined #tripleo10:19
sshnaidmdoes anybody know, what is default for heat::api::workers: ?10:19
*** limao has quit IRC10:20
tbarronmarios: gfidente thanks for the reviews on https://review.openstack.org/#/c/382671/110:21
jpichjaosorior: fwiw undercloud install worked for me, but I think you got your solution from the #rdo folks already anyway :)10:21
shardysshnaidm: https://github.com/openstack/heat/blob/master/heat/common/config.py#L8810:21
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Add ability to deploy an overcloud with ssl  https://review.openstack.org/38283010:21
shardythe engine workers defaults to either 4 or the number of CPUs10:21
shardyI think the API workers are just the number of CPUs10:21
tbarronmarios: gfidente if we can avoid having users complain/ask about that unconventional ceph config file path that would be great10:22
mariostbarron: o/ sorry i was away for a few days - other that this review, are things OK with netapp/generic/ceph manila now?10:22
tbarronmarios: we have an outstanding issue for ceph deploy, see https://review.openstack.org/#/c/38238610:23
sshnaidmshardy, and we limit them to 1 here, right? https://github.com/openstack/tripleo-heat-templates/blob/master/environments/low-memory-usage.yaml10:23
mariostbarron: thanks looking10:24
tbarronmarios: thank you10:24
shardysshnaidm: yup, we override the defaults for CI10:24
* tbarron doesn't understand puppet arrays and string joins10:25
jaosoriorgchamoul: ping10:26
sshnaidmshardy, each worker is a separate process, right? trying to understand why I have 6 processes of heat-engine on undercloud10:28
shardysshnaidm: there's a parent process then $workers forked workers10:28
shardyso I'd expect to see a minimum of 5 heat-engine processes in ps aux | grep heat-engine10:29
shardydepending on the hardware10:29
*** thrash|g0ne is now known as thrash10:29
openstackgerritJiri Tomasek proposed openstack/tripleo-common: Support node untagging  https://review.openstack.org/38283210:29
tbarronmarios: gfidente I can set topic on  https://review.openstack.org/382671 to tripleo/rc3 if you guys agree10:30
shardysshnaidm: http://paste.openstack.org/show/584643/10:30
sshnaidmshardy, yeah, I have those: http://paste.openstack.org/show/584644/10:31
gchamouljaosorior: pong10:31
sshnaidmshardy, seems like settings didn't work?10:31
shardysshnaidm: those settings only control the overcloud deployment, not the undercloud10:32
mariostbarron: sure i wanted just to get a feel for what is outstanding before proposing we rush it into rc3... if indeed it is those two (the path one and the enable_share_protocols, i think we can make the case.. thought that last one needs at least one more revision but almost there i think)10:32
sshnaidmshardy, uh, sorry, confused it10:33
sshnaidmshardy, so for limit them on undercloud I need configure it in undercloud.conf?10:34
shardysshnaidm: yeah, add a hieradata_override = foo.yaml to undercloud.conf10:34
shardythe add the hieradata to control the workers in foo.yaml10:34
sshnaidmshardy, it's possible also to configure them in the undercloud.conf itself, right?10:35
shardysshnaidm: Not as far as I'm aware10:35
tbarronmarios: ack, i'll wait on the label then but will propose stable/newton backports if I see these land in master10:35
tbarrons/label/topic/10:36
mariostbarron: gonna quickly fixup the puppet-tripleo for the nits you point at there10:36
mariosgfidente: ^10:36
*** limao_ has quit IRC10:36
shardyNote that for bugfixes we can backport them after rc3/GA and tag another newton release10:37
sshnaidmshardy, do you know if we have any performance tests of tripleo deployments? like dependencies of memory, workers count, cpu, deploy time, etc ?10:37
*** limao has joined #tripleo10:37
tbarronmarios: gfidente i've started a deploy with local changes that attempt that, can test again if you push a new patch too of course10:37
EmilienMhi10:37
shardysshnaidm: Yes, rook has provided such data for scale testing in the past10:37
shardysshnaidm: I've also done some basic profiling of heat memory usage locally on a few occasions10:38
shardyhttps://bugs.launchpad.net/heat/+bug/1626675 covers the most recent10:38
openstackLaunchpad bug 1626675 in heat "Further memory usage issues with big stacks" [Critical,In progress] - Assigned to Zane Bitter (zaneb)10:38
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/38283910:40
sshnaidmshardy, it'd be cool to have it running regularly with nice test matrix..10:40
shardysshnaidm: Yeah I agree10:40
* cmyster clicks10:40
sshnaidmshardy, maybe on rdo infra10:41
shardywe've been bitten repeatedly by memory usage regressions, and it would be great to catch them earlier10:41
openstackgerritMarios Andreou proposed openstack/puppet-tripleo: Set enabled_share_protocols based on enabled backends  https://review.openstack.org/38238610:41
openstackgerritRyan Brady proposed openstack/python-tripleoclient: Use Password Generation in the Mistral workflows  https://review.openstack.org/37554410:41
cmystershardy: thing is, you need a system that runs for days to really check for those.10:41
mariostbarron: ack thanks10:41
shardycmyster: for long term leaks, yes10:42
shardybut right now we don't even catch when e.g some change in heat means we use an extra gigabyte of ram10:42
*** mburned_out is now known as mburned10:42
cmysterwhats an extra gb...10:42
cmysteranywho10:42
shardyhttp://people.redhat.com/~shardy/heat/plots/heat_before_after_end_newton.png10:44
EmilienMshardy: I'm afk ~3h from now fyi keep "recheck" on the backports10:44
shardycmyster: it's much worse than that, our heat memory usage quadrupled during Newton10:44
EmilienMshardy: I'll prepare the release patch tonight and we'll proceed tomorrow I guess, i'll sync with doug10:44
shardySome fixes are landing which fix it tho10:44
cmysterseen ^ bug10:44
cmystershardy: did anyone see it on rhel as well? is this rdo?10:45
shardyEmilienM: ack, yup I'll try to get the remaining patches landed today so we can cut the release late today or early tomorrow10:45
EmilienMchem, bandini, ccamacho, jaosorior, gfidente, marios, jpich, jtomasek: please make sure having tripleo/rc3 gerrit topic if needed and also do the cherry-picks10:45
ccamachoEmilienM sure man thanks10:45
chemEmilienM: ack10:45
*** dsariel_ has quit IRC10:46
jaosoriorEmilienM: will do10:46
jpichEmilienM: Yes, thank you10:47
jtomasekEmilienM: ack10:47
rookshardy sshnaidm we are working on something to do better tracking of this... However it still takes human eyes to identify the regression for what we are using, ie collectd->graphite->grafana.10:47
mariosEmilienM: ack ... tbarron i think the issues you report at https://review.openstack.org/#/c/382386/ are because indeed of the comments  you made about the wrong hiera lookup. so they were in fact nill/undef. Once we see/hear that looks OK (v5) then I'd be happy to back that and https://review.openstack.org/#/c/382671/ for rc3 (the last one can go i think idependently right?)10:47
mariosgfidente: ^10:47
rookshardy sshnaidm - we are working to get this work added to CI, so they can use our playbooks to install collectd and point their agents at our hosts so we can monitor the system utilization.10:48
sshnaidmrook, sure, do you have these statistics online that we can see it?10:48
rookshardy sshnaidm this was a conversation I had this week, we move fast, but not that fast.10:48
tbarronmarios: gfidente EmilienM yes, those two reviews are independent of one another10:48
rooksshnaidm shardy right now CI has a dstat running where you can get some information.10:48
sshnaidmrook, thanks, good to know it will be :)10:48
sshnaidmrook, yeah, although it requires an additional parsing10:49
rooksshnaidm shardy however that just gets us system stats... I am also interestd in the time it takes for different deployments.10:49
shardyrook: ack, yeah would be good to keep discussing so we can hopefully reuse a common approach10:49
rooksshnaidm: correct.10:49
shardythere's been similar discussion around how we might wire in tests to the heat/tripleo gates10:49
*** masco__ has joined #tripleo10:49
rooksshnaidm shardy so there is another effort within browbeat to add a lib for tripleo to allow us to get better metrics on time for deployements, time for scale along with tracking system utilization.10:49
shardye.g actually do something with the dstat data and/or have a load test for heat which fails if memory usage goes up10:49
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Change README.rst to point to the full docs  https://review.openstack.org/38284210:50
rookthe idea is two fold shardy sshnaidm --- how does scaling impact tripleo, and how does adding addtional resources to the overcloud impact the overcloud...10:50
sshnaidmrook, we already have metrics inside ci scripts, it sends them to dprince grafana10:50
rooksweet, where?10:50
shardyThat's only time though I think10:50
shardyhttps://23.253.94.71/10:51
shardyrook: ^^10:51
rookoh, that is a graphite host10:51
sshnaidmrook, https://23.253.94.71/dashboard/#temporary-310:51
sshnaidmrook, yeah, graphite, not grafana10:52
rooktotal in seconds?10:52
rooksshnaidm: so what deployment is this showing? 3 control N number of computes?10:52
mariosEmilienM: tbarron i added the https://review.openstack.org/#/c/382671/1 to rc3 easy enough fix imo10:52
*** masco_ has quit IRC10:53
tbarronmarios: thx, yeah can't get any easier :)10:53
EmilienMmarios: why don't you approve it?10:53
sshnaidmrook, I suspect it mix of ha and nonha jobs10:53
mariosEmilienM: ack i was just checking in case we needed to discuss an exception etc10:54
rooksshnaidm ok -- might want to have different graphs per the deployment.10:54
rooksshnaidm I am interested in not just control/computes, how does ceph impact the deployment,etc.10:54
rookall things we are looking into... but need to automate better.10:54
mariosEmilienM: tbarron thanks +a that one so now just https://review.openstack.org/#/c/382386/5 ... if you get clean run with that maybe we can land today tbarron10:54
rookshardy speaking of things to look into :)10:54
rookshardy did that blog post ever see the light of day? :P10:55
shardyrook: the custom roles one?10:55
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add devmode.sh to wrap full-deploy.sh  https://review.openstack.org/38284510:55
rookyessir10:55
tbarronmarios: yup, will test10:55
EmilienMmarios: can you cherry-pick it now the patch you just +A? so I can approve it too10:55
shardyI'm writing it right now - we had a bunch of bugs that needed fixing (last couple of patches are gating now)10:55
mariosEmilienM: ack10:55
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Set proper ceph config path for manila  https://review.openstack.org/38284610:55
rookall good shardy just giving you a hard time.10:55
shardyrook: should be posted later today, I just didn't want to post something with a bunch of workarounds or pending patches10:55
mariosEmilienM: https://review.openstack.org/#/c/382846/10:55
shardyrook: hehe, no worries10:56
rookshardy tip of the spear.10:56
EmilienMmarios: thx, approved10:56
shardytripleo-docs patch should be ready next week10:56
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Document Third Party CI and devmode.sh  https://review.openstack.org/36000710:56
mariosEmilienM: thanks10:56
rooksshnaidm are you developing anything for testing OOO?10:56
rooksshnaidm if so, we should look to converge on what each of us are doing10:57
rookor planning to do :)10:57
sshnaidmrook, hmm.. I'm usually suffering from perf. issues in testing ooo :))10:57
rooklol10:57
rookin what regard?10:57
rooktesting in VMs10:57
*** limao has quit IRC10:58
sshnaidmrook, yeah, memory shortage there10:58
openstackgerritMartin Mágr proposed openstack/puppet-tripleo: Deploy monitoring/logging agents sooner  https://review.openstack.org/38160410:58
sshnaidmrook, I think we can look where can I help with that10:58
*** limao has joined #tripleo10:58
sshnaidmrook, if you will tell about what is planned and so10:58
rooksshnaidm it was my understanding that Newton was going to help with the memory consumption, i haven't looked in much detail here yet.10:59
rookI am going to do a scale deployment with newton today10:59
shardyrook: we fixed a bunch of issues, then reintroduced new ones late in the cycle :(10:59
sshnaidmrook, I think it became worse, correct me if I'm wrong10:59
shardyrook: currently under investigation ref https://bugs.launchpad.net/heat/+bug/162667511:00
openstackLaunchpad bug 1626675 in heat "Further memory usage issues with big stacks" [Critical,In progress] - Assigned to Zane Bitter (zaneb)11:00
rooksshnaidm yeah, I know CI saw some increased mem usage11:00
openstackgerritMerged openstack/tripleo-heat-templates: Adds Environment File for Removing Sahara during M/N upgrade  https://review.openstack.org/38274811:00
rookit is much easier to see the mem issue in VMs then baremetal11:00
rookas you have a much lower ceiling11:00
*** shardy is now known as shardy_lunch11:01
*** pkovar has joined #tripleo11:01
sshnaidmrook, which infra do you use for tests?11:02
rook:)11:02
rookall baremetal11:02
rooksshnaidm: are you a RH Employee?11:02
sshnaidmrook, yep11:02
rooksshnaidm so i lead the team doing Performance and Scale within Red Hat.11:03
rooksshnaidm so we have a bunch of gear :)11:03
rooksshnaidm we are in RDU if you are local :)11:03
sshnaidmrook, cool! I know rdu, but not local :) I think this question could be discussed in internal net channels then11:04
*** d0ugal has quit IRC11:05
*** d0ugal has joined #tripleo11:06
*** Lokesh_Jain has quit IRC11:14
*** jpena is now known as jpena|lunch11:14
*** rhallisey has joined #tripleo11:16
*** Lokesh_Jain has joined #tripleo11:17
*** limao has quit IRC11:19
*** limao has joined #tripleo11:19
openstackgerritRyan Brady proposed openstack/tripleo-common: Port Scale Down Functionality into Workflow  https://review.openstack.org/38270711:24
*** shardy_lunch is now known as shardy11:31
shardyhttps://review.openstack.org/#/c/382430/ needs another reviewer if anyone has a moment11:33
*** dprince has joined #tripleo11:39
*** limao has quit IRC11:40
*** limao has joined #tripleo11:41
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109411:41
openstackgerritJulie Pichon proposed openstack/puppet-tripleo: Update websocket service name in config template  https://review.openstack.org/38287511:49
*** masco_ has joined #tripleo11:50
*** masco__ has quit IRC11:54
*** bfournie has quit IRC11:55
openstackgerritMerged openstack/puppet-tripleo: Enable usage of "short names" for Ceph cluster  https://review.openstack.org/38240811:55
openstackgerritMerged openstack/tripleo-heat-templates: Select per-network hostnames for service_node_names  https://review.openstack.org/37876411:56
mariospradk: hey https://review.openstack.org/#/c/360004/22 added another comment... do we need to redeclare apache here at all?11:57
marioschem: ^11:57
pradkmarios, thats what i'm trying to confirm11:59
pradkmarios, rerunning the update without it to see11:59
pandajaosorior: do you have a few minutes to mentor me over SSL + net-iso ?12:00
jaosoriorpanda: what's up?12:00
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable usage of "short names" for Galera cluster  https://review.openstack.org/38288312:01
pandajaosorior: apparently I have to create the new certificate specifying (probably) the IP we use in net-iso as CN12:01
*** limao has quit IRC12:01
jaosoriorpanda: that's correct.12:01
pandajaosorior: http://logs.openstack.org/15/382515/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/954a5d5/console.html#_2016-10-06_10_09_30_89691312:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable usage of "short names" for Ceph cluster  https://review.openstack.org/38288512:02
*** limao has joined #tripleo12:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Select per-network hostnames for service_node_names  https://review.openstack.org/38288612:02
jaosoriorpanda: yeah, that is expected.12:03
*** trown|outtypewww is now known as trown12:03
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109412:03
jaosoriorso you need to do two things: You need to set up the IP address to be predictable (we do that for the nonha job), and you need a new cert, with that IP as the CN12:03
mariospradk: thanks... i think we should be good with just the include ::ceilometer::wsgi::apache like at https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/ceilometer/api.pp#L33 for example... and all the wsgi apache services are the same afaics and we just include apache in one place at https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/ui.pp#L2212:04
*** rbrady-afk is now known as rbrady12:05
*** jayg|g0n3 is now known as jayg12:05
pandajaosorior: so I have to create a certificate request with tha CN, and sign it with the CA key in inject-trust-anchor.yaml ?12:06
pandajaosorior: I think net-iso is using fixed IPs in the environment12:06
pradkmarios, just include wont work due to servername issue12:06
pandajaosorior: also, I may need to change the ControlFixedIp on enable-tls.yaml, right ?12:06
jaosoriorpanda: so, inject-trust-anchor.yaml doesn't have a CA key. It's only the cert12:07
*** limao has quit IRC12:07
jaosoriorwe don't need to pass around keys for the CA12:07
*** lucas-afk is now known as lucasagomes12:07
pradkjaosorior, may be you have some thoughts on https://review.openstack.org/#/c/365585/ .. this is causing issues during upgrades  as the fqdn_inertnal_ip return nil in hiera12:07
*** limao has joined #tripleo12:07
mariospradk: yeah even just declaring class { '::ceilometer::wsgi::apache then12:07
jaosoriorpradk: any idea why it's returning nil? we should have that available12:08
*** mburned is now known as mburned_out12:09
pandajaosorior: I wanted to add a enable-tls-netiso.yaml with the keyfor the netiso IP, dont' I need to sign that cert with CA key ?12:09
jaosoriorpanda: yes, if you don't, the undercloud won't trust the certificate that the overcloud offers and will return SSL errors12:10
pandajaosorior: ok, so where can I find the CA key ?12:11
*** jpena|lunch is now known as jpena12:11
pandajaosorior: or should I request someone to make a certificate for me ?12:11
*** tzumainn has joined #tripleo12:11
jaosoriorpanda: the CA is just a self signed certificate. We don't use anything fancy for CI12:12
*** mburned_out is now known as mburned12:12
*** limao_ has joined #tripleo12:13
*** abregman__ is now known as abregman12:13
jaosoriorpanda: you can use this if you need http://galeracluster.com/documentation-webpages/sslcert.html12:13
pradkjaosorior, not sure, does this need corresponding puppet-tripleo change to get that value? or is this directly from facter12:14
jaosoriorpradk: there is a puppet-tripleo change. I thought it had been there for a while though12:14
jaosoriorlet me check12:14
pradkjaosorior, in newton, but not mitaka.. during pre upgrade step the puppet-tripleo is still mitaka12:14
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756212:14
jaosoriorpradk: you are correct :/12:15
pradkjaosorior, this causes apache to fail with syntax error.. as servername is nil and httpd conf file for service is missing that :(12:15
jaosoriorpradk: well, if you're doing pre-upgrade, you don't necessarily need the correct values for the servername. As I mentioned the other day, those are not used yet since we are using IPs for everything12:15
jaosoriorpradk: this is why I told you we could hardcode $::hostname in that manifest12:16
*** limao has quit IRC12:16
pradkjaosorior, right but we need to do that for a aodh,gnocchi and keystone and repopulate with all the releavnt params again12:16
pandajaosorior: so you're suggesting to not use the same CA cert and key used for the ssl/nonha certificate we're currently using, just make another CA cert and key, create the ssl/nonha/netiso certificate with that and add the inject-trust-anchor-netiso.yaml  and enable-tls-netiso.yaml to test-environments12:16
openstackgerritmathieu bultel proposed openstack/python-tripleoclient: Download templates from swift before processing with heatclient  https://review.openstack.org/37954712:17
pradkjaosorior, just doing include *::wsgi::apache would leverage existing hiera ..12:17
pradkwhich we cant now12:17
jaosoriorpanda: pretty much. Unfortunately the key wasn't included for the CA that's used in the nonha job... so we can't sign another certificate with that one :/12:19
pandajaosorior: ok, I'll proceed then, maybe I'll ask bnemc if he saved the CA key somewhere, so we don't have to distribute two CA eventually12:20
pandajaosorior: thanks.12:20
*** dsariel_ has joined #tripleo12:21
*** mbozhenk1 has joined #tripleo12:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment states optimization  https://review.openstack.org/38184512:25
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient: Add ReNo support  https://review.openstack.org/38204612:25
*** maticue has joined #tripleo12:25
*** mbozhenko has quit IRC12:26
*** mbozhenk1 has quit IRC12:27
*** mbozhenko has joined #tripleo12:28
openstackgerritMerged openstack/tripleo-heat-templates: Set proper ceph config path for manila  https://review.openstack.org/38267112:31
*** bfournie has joined #tripleo12:32
*** limao_ has quit IRC12:33
openstackgerritMerged openstack/python-tripleoclient: Do not use selinux-permissive for the CentOS image  https://review.openstack.org/36009712:34
*** mbozhenk1 has joined #tripleo12:34
*** limao has joined #tripleo12:34
openstackgerritMerged openstack/tripleo-common: Add support to create role main template file based in role.role.j2.yaml  https://review.openstack.org/38273812:34
openstackgerritMerged openstack/tripleo-heat-templates: Add generic template for custom roles.  https://review.openstack.org/38158712:34
openstackgerritMerged openstack/tripleo-heat-templates: restore missing fluentd client functionality  https://review.openstack.org/38243012:34
*** mbozhenko has quit IRC12:35
EmilienMtbarron: can you cherry-pick https://review.openstack.org/#/c/382671/ please?12:36
EmilienMgfidente: do we need https://review.openstack.org/#/c/360097/ into newton?12:36
openstackgerritMerged openstack/tripleo-heat-templates: Set proper ceph config path for manila  https://review.openstack.org/38284612:36
EmilienMccamacho: can you backport https://review.openstack.org/#/c/381587/ please?12:37
openstackgerritCarlos Camacho proposed openstack/tripleo-common: Add the J2 exclude file exception (When file not found)  https://review.openstack.org/38291112:37
EmilienMlarsks: can you backport https://review.openstack.org/#/c/382430/ please?12:37
*** fultonj has joined #tripleo12:37
ccamachoEmilienM sure https://review.openstack.org/#/c/382737/ waiting for the merge12:37
EmilienMccamacho: excellent thanks12:38
ccamachoEmilienM im keeping the log for all the submissions related to the pending work for the composable roles (with the backports)12:38
*** jcoufal has joined #tripleo12:40
openstackgerritMerged openstack/tripleo-quickstart: Change README.rst to point to the full docs  https://review.openstack.org/38284212:41
*** jaosorior has quit IRC12:41
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Add Select per-network hostnames for service_node_names to role.role.j2.yaml  https://review.openstack.org/38221712:42
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Add Select per-network hostnames for service_node_names to role.role.j2.yaml  https://review.openstack.org/38221712:43
*** jeckersb_gone is now known as jeckersb12:44
*** dsavineau has joined #tripleo12:47
EmilienMccamacho: ok12:48
*** morazi has joined #tripleo12:48
gfidentemarios https://review.openstack.org/#/c/382386/4..5/manifests/profile/base/manila/api.pp12:49
gfidenteI wanted the heat hiera there12:50
gfidentebuilt after the service_name parameter12:50
*** links has quit IRC12:50
*** masco__ has joined #tripleo12:50
*** tiswanso has joined #tripleo12:50
*** tiswanso has quit IRC12:50
*** tiswanso has joined #tripleo12:51
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add the KeystoneCredential0 and KeystoneCredential1 parameters  https://review.openstack.org/38116912:54
*** masco_ has quit IRC12:54
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] Add IPv6 coverage using existing jobs  https://review.openstack.org/38251512:54
*** limao has quit IRC12:54
*** morazi has quit IRC12:54
mariosgfidente: yes like https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/manila-backend-netapp.yaml#L9112:55
mariosgfidente: not sure what the questino is tho12:55
*** limao has joined #tripleo12:55
*** liverpooler has quit IRC12:56
EmilienMgfidente: can you reply to my question please?12:56
tbarronmarios: gfidente patch set 5 worked deploying ceph + netapp http://paste.fedoraproject.org/444820/14757585/12:57
EmilienMgfidente: same question for https://review.openstack.org/#/c/382671/12:57
EmilienMtbarron: ^12:57
larsksEmilienM, sure; backport to stable/newton, yes?12:57
tbarronmarios: gfidente i'll try the singular cases yest12:57
EmilienMlarsks: yes;12:57
EmilienMI'm about to release rc3 very shortly I suggest people to submit the backports asap12:58
mariostbarron: cool12:58
gfidentetbarron so can you see the output of12:58
gfidentehiera manila_backend_generic_enabled12:58
*** Goneri has joined #tripleo12:58
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109412:58
gfidenteheat is supposed to generate ${service_name}_enabled for each role enabled12:59
tbarrongfidente: where do i look?  heat logs?12:59
gfidenteno on a controller13:00
gfidenteas root13:00
gfidentejust give that command13:00
*** jtomasek_ has joined #tripleo13:00
tbarrongfidente: sec13:00
openstackgerritMerged openstack/instack-undercloud: Pin beaker due to ruby requirement  https://review.openstack.org/38268613:01
*** pkovar has quit IRC13:01
jristjtomasek: https://blueprints.launchpad.net/tripleo-ui/+spec/deployment-progress is this taken care of by your progress/deployment detail work?13:01
*** akuznetsov has joined #tripleo13:01
jtomasekjrist: nope13:02
tbarrongfidente: output is: nil with the current deployment, which just deployed cephfs and netapp13:02
gfidenteEmilienM https://review.openstack.org/#/c/382671/ went in newton already13:02
*** oshvartz has quit IRC13:03
*** [1]cdearborn has joined #tripleo13:03
tbarrongfidente: output for: hiera manila_backend_netapp_enabled is true13:03
EmilienMgfidente: ok, I missed it. And https://review.openstack.org/#/c/360097/ ?13:03
gfidentetbarron yet you don't have CEPHFS amongst the protocols13:03
mariosgfidente: ah i see so you mean you expect them to be available from some other mechanism not what we pass ?( y u no speak to me faidentee just cos i told you to shutup a few times i means srsly some poeople!)13:03
tbarrongfidente: oh, you are right!!!13:04
gfidentetbarron yeah you don't have CEPHFS anyway13:04
tbarrongfidente: wtf!!13:04
gfidentesec guys13:04
tbarrongfidente: same issue13:04
gfidenteI think we had to fix the delete_undef_below13:04
*** ayoung has joined #tripleo13:04
gfidenteyeah give me a sec to update13:04
sshnaidmdo we still need "ironic::drivers::deploy::http_port: 3816" in hiera override? I don't see it's applied, do I miss something?13:04
mariosgfidente: i.e. you expect hiera('manila_backend_generic_enabled', false)  as well as  hiera('manila_generic_enable_backend', false) to be a thing?13:04
mariosgfidente: also, shutup13:04
* tbarron suffers from wishful thinking sometimes13:04
gfidentemarios it's just a generic thing in the templates13:05
gfidenteworks for any role like that13:05
*** links has joined #tripleo13:05
EmilienMayoung: now https://review.openstack.org/#/c/371247/ has landed, do you plan to do the tripleoclient/tripleo-common/tht work to enable fernets?13:05
gfidenteEmilienM the puppet-tripleoclient one13:05
ayoungEmilienM, yep13:05
gfidenteI think it'd be a nice to have yes, but it wasn't a bug13:05
ayoungEmilienM, was on the task list for today13:05
EmilienMayoung: cool :)13:05
EmilienMgfidente: ok, so no13:05
mariosgfidente: right i see... well why is it still nil on tbarron env perhaps didn't land into his env tht yet?13:06
mariosgfidente: but we _dO_ definitely set the ones on v5 in the tht right now, so we could leave as is and revise later?13:06
gfidentemarios it only sets to :true13:06
gfidentewhen the role is enabled13:06
gfidenteso generic was nil because he was using netapp/cephfs as intended13:07
tbarrongfidente: marios and for completeness: hiera manila_backend_cephfs_enabled is indeed true in that deploy13:07
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment states optimization  https://review.openstack.org/38184513:07
gfidenteyeah let me revert that so we refer to the generic mechanism13:07
gfidenteI think the issue is in the delete_undef13:07
mariostbarron: ah great so it *does* work there then?13:07
*** morazi has joined #tripleo13:08
tbarronmarios: no, in manila.conf: 'enabled_share_protocols = NFS,CIFS'13:08
tbarronmarios: 'CEPHFS' is missing despite being enabled13:08
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Set enabled_share_protocols based on enabled backends  https://review.openstack.org/38238613:09
tbarronmarios: i guess i was wrong though about the tests for what is enabled, gfidente's way for that works13:09
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates: Fix OpendaylightApiNetwork key naming  https://review.openstack.org/38292613:09
gfidentemarios tbarron ^^ unless of syntax errors13:09
gfidenteI think the problem is puppet module expects a string with comma13:09
tbarrongfidente: marios i'll pick it up and see13:10
*** panda is now known as panda|afk13:10
EmilienMlarsks: do you have merge conflict?13:10
larsksEmilienM, doesn't look like it.13:10
openstackgerritDan Prince proposed openstack/python-tripleoclient: Revert "Clear the old environment in rc file"  https://review.openstack.org/38292813:11
gfidenteEmilienM this guy worries me a little https://review.openstack.org/#/c/382131/13:11
EmilienMlarsks: ok, I was wondering if you had it, because I don't see the backport13:11
gfidentewhich we want in newton13:11
EmilienMlarsks: you just need to click on Gerrit 'Cherry-pick' fyi13:11
EmilienMgfidente: looking13:11
larsksEmilienM, that is because you just asked me two minutes ago, and I have not yet gotten to it :)13:11
EmilienMah ok np13:11
larsksJust catching up on a few other things first.13:12
gfidenteEmilienM this is the tht part https://review.openstack.org/#/c/382132/13:12
EmilienMlarsks: ok. Sorry for pushing, trying to release rc3 on time13:12
dprincegfidente: next time I guess I'll use a -2 :/ https://review.openstack.org/#/c/347553/13:12
larsksEmilienM, let me go do that right now, then.13:12
dprincegfidente: https://bugs.launchpad.net/tripleo/+bug/163099313:12
openstackLaunchpad bug 1630993 in tripleo "source'ing overcloudrc unsets any external OS_ variables and breaks some environments" [Critical,In progress] - Assigned to Dan Prince (dan-prince)13:12
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-heat-templates: restore missing fluentd client functionality  https://review.openstack.org/38293013:12
dprincegfidente: unsetting OS_ variables is just bad practice I think13:12
*** b00tcat has quit IRC13:12
EmilienMlarsks: thx. Approving now.13:13
mariosgfidente: ack thanks13:13
gfidentedprince: (also) ack thanks13:13
EmilienMgfidente: do you need me to resolve a conflict?13:13
gfidentedprince I think I didn't see the implications there13:13
EmilienMgfidente: wrt puppet ceph13:13
*** tiswanso has quit IRC13:14
dprincegfidente: its okay. I had a -1 on it, sometimes people take -2's the wrong way....13:14
dprincegfidente: when actually I just wanted a sticky -1 that doesn't go away13:14
gfidentedprince yeah I only saw that now13:14
EmilienMdprince: I don't think -2 is wrong when discussion is required before a patch to land13:15
gfidenteI think the patch was updated and the -1 went away13:15
dprinceEmilienM: people often perceive the -2 as harsh, that is all I think13:15
EmilienMyes, which is the main diff with a -213:15
EmilienMthe -2 doesn't go away13:15
EmilienMdprince: yeah, I agree. Though it's not harshing13:15
*** limao has quit IRC13:16
EmilienMit's just a strong opinion of making sure the patch doesn't land before we find an agreement13:16
gfidenteEmilienM which conflict13:16
gfidente?13:16
*** nyechiel_ has quit IRC13:16
gfidentehttps://review.openstack.org/#/c/382131/ looks good to me13:16
EmilienMgfidente: puppet-ceph patch, you're worried. Is it fine?13:16
shardyI think -2 is often percieved as hostile, so FWIW I always prefer to use -1 when there's some possibility a patch could be reworked into something acceptable13:16
*** limao has joined #tripleo13:16
EmilienMgfidente: ok I'll -2 it :P13:16
shardybut I agree a sticky -1 would be useful, -1.5 :)13:16
dprincegfidente: do you mind if I fast track revert that and we can re-propose to debate it13:17
EmilienMI agree it's hard to -2, you feel like the bad guy13:17
*** nyechiel_ has joined #tripleo13:17
gfidentefwiw, not to me guys13:18
EmilienMgfidente: why do you pass an array here ? https://review.openstack.org/#/c/382131/7/manifests/params.pp13:18
EmilienMI don't get it13:18
gfidenteI always said I need the -1s and -2s13:18
gfidentewhat's gerrit for otherwise13:18
EmilienMgfidente: you can still give an array in hiera, no need to set default to an array, right?13:18
gfidentedprince yes revert is good13:19
*** lblanchard has joined #tripleo13:19
gfidentenot sure if there is a mechanism to re-use the pre-existing change-id to refresh the old submission then13:19
*** tiswanso has joined #tripleo13:20
gfidenteEmilienM I think it needs to be an array for Package { $packages } to parse it correctly, not comma separated13:20
gfidenteso I kinda 'suggested' that in the default13:20
EmilienMok but it's not required13:20
EmilienMI'm fine with it fwiw13:21
gfidenteit's not required in the default value which is only 1 item no13:21
EmilienMgfidente: +213:21
gfidentetks!13:21
EmilienMgfidente: please add tripleo/rc3 gerrit topic to puppet-ceph patches13:21
EmilienMgfidente: I need to know when to release puppet-ceph again13:21
EmilienMfor our newton release13:21
openstackgerritMerged openstack/tripleo-ui: Add image names to Nodes registration workflow  https://review.openstack.org/38278513:21
*** pkovar has joined #tripleo13:23
*** b00tcat has joined #tripleo13:26
*** jtomasek_ has quit IRC13:26
*** dsavineau has quit IRC13:27
*** dsavineau has joined #tripleo13:27
EmilienMMitaka CI looks broken, anyone already looked at it?13:30
d0ugaldprince: ping13:31
EmilienMok I found the Mitaka CI issue, it's the same packaging thing we had this week end with cisco networking. Will fix it13:32
*** masco__ has quit IRC13:32
dprinced0ugal: hey, whats up?13:36
*** bnemec has joined #tripleo13:36
d0ugaldprince: Do you remember discussing getting the snmpd_readonly_user_password from hiera?13:37
*** limao has quit IRC13:37
d0ugaldprince: That doesn't work, the Mistral user can't access it. Any other ideas?13:37
*** limao has joined #tripleo13:37
dprinced0ugal: can we specifically store/stash it somewhere on deployment (create or update)13:38
d0ugaldprince: This is the undercloud password (IIUC), so maybe instack could do that?13:38
ccamachohey shardy, finally https://paste.fedoraproject.org/444864/47576058/  custom role (WakaWaka) deployed in an additional node, overriding roles_data.yaml with --roles-file, so just waiting to have all merged and backported  T-15:22:00 for Newton GA I think the basic interface for custom roles/services is finished13:38
dprinced0ugal: sure, but it is also supplied to the overcloud via a Heat parameter so we should have access to it either way right?13:39
*** tiswanso has quit IRC13:39
d0ugaldprince: Right, but how do we get it to Heat?13:39
d0ugaldprince: it's easy for the CLI to pass it to a workflow, that is what it does now13:39
d0ugaldprince: but not so easy for the UI13:40
d0ugaldprince: it is stored in hiera and the undercloud conf file in the stack home directory.13:40
dprinced0ugal: Can't we write a custom Mistral action that essentially looks it up for us via the same python function (utils.get_config_value)13:42
dprinced0ugal: kind of an evil thing to expose via the API so perhaps guard it to just the passwords we need13:42
d0ugaldprince: Mistral is running as the wrong user, so it wont have the permissions13:42
dprinced0ugal: so, then I think this is a 2 part fix. Update the instack installer to provide access to this for the Mistral user13:43
d0ugal(wrong user for that file, it is running as the correct user for Mistral)13:43
d0ugaldprince: okay, how would you provide access?13:43
dprinced0ugal: we can have multiple hiera files.... they don't all have to have extra permissions I think13:43
maticueHi! I hope someone can help me. On which stage of an "openstack overcloud deploy" endpoints are configured? Because I have an environment with TLS (enable-tls.yaml) miss configured (undercloud reports CERTIFICATE_VERIFY_FAILED after Stack is created) and when I see the endpoints table on the database is incomplete (just Keystone entries)13:43
dprinced0ugal: just read perms would be required, perhaps we create a simple unix group that can read this extra file13:44
*** panda|afk is now known as panda13:44
dprinced0ugal: share_passwords.yaml or something13:44
dprinced0ugal: shared_passwords?13:44
d0ugaldprince: okay, that sounds good, I'll start digging.13:44
dprinced0ugal: I don't have a strong oppinion on the name yet13:44
d0ugal /cc rbrady ^13:45
dprinced0ugal: thanks for chasing this. The last few params are always the funnest13:45
rbradydprince: thanks13:46
*** akshai has joined #tripleo13:49
EmilienMfor the record, https://bugs.launchpad.net/tripleo/+bug/163101213:49
openstackLaunchpad bug 1631012 in tripleo "neutron/mitaka is broken at dbsync" [Critical,Triaged]13:49
rookheya lucasagomes - openstack baremetal import for newton... any reason we cannot have all the added registration as a post operation13:50
rookso the CLI isn't just sitting there?13:50
openstackgerritLeif Madsen proposed openstack/tripleo-validations: Port the network environment validation  https://review.openstack.org/34158613:50
openstackgerritLeif Madsen proposed openstack/tripleo-validations: Add jsonschema validation for os-net-config data  https://review.openstack.org/35134513:50
rooklucasagomes w/ ~150 hosts it is quite slow.13:50
*** akshai_ has joined #tripleo13:50
lucasagomesrook, that's a TripleO command actually13:51
lucasagomesAFAICT it's totally possible to break the steps of import into more steps13:51
rooklucasagomes: oh, who should I bother then?13:52
rook:D13:53
lucasagomesrook, you are in the right channel! The import command is part of the python-tripleoclient13:53
d0ugalrook: You can bother me :)13:53
lucasagomesrook, https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/baremetal.py#L11613:54
*** akshai has quit IRC13:54
*** akuznetsov has quit IRC13:55
d0ugallucasagomes: most of the "logic" is now in a mistral workflow.13:55
rookd0ugal: cool, so it is my understanding with Newton there is this added level of steps for import -- due to mistral13:55
d0ugalrook: Yeah, so the difference now is that you don't see any output until they are all finished13:56
d0ugalrook: is that what you are asking about?13:56
rookd0ugal well output and time13:56
rookwith 150 nodes it is close to more than 10 minutes now.13:56
rookwell 15 now.13:56
rookand it still hasn't completed.13:56
*** tiswanso has joined #tripleo13:56
d0ugalrook: dang, out of interest, how long did it take before?13:57
rookis there a test upstream to keep a eye on how long this takes?13:57
rookd0ugal < 1 minute13:57
d0ugalNo, I don't think there is a test.13:57
rookthis is a bit excessive.13:57
d0ugaloh, wow13:57
d0ugalYeah.13:57
rookhonestly, with this amount of time, i would expect introspection to happen too :D13:58
d0ugalrook: I wonder if there is an issue.13:58
d0ugalrook: Can you look at "mistral execution-list" for any errors?13:58
*** limao has quit IRC13:58
d0ugale.g. mistral execution-list | grep "ERROR";13:58
*** limao has joined #tripleo13:58
rookhttps://gist.github.com/jtaleric/7ea802c7b98d735223c382aeb986578213:59
openstackgerritMerged openstack/tripleo-heat-templates: Add generic template for custom roles.  https://review.openstack.org/38273713:59
d0ugalrook: okay, next: "mistral action-execution-list"13:59
d0ugalrook: This one will be quite a bit bigger :)13:59
*** akuznetsov has joined #tripleo14:00
openstackgerritCarlos Camacho proposed openstack/tripleo-common: Add the J2 exclude file exception (When file not found)  https://review.openstack.org/38297314:00
*** ramishra_ has joined #tripleo14:01
rookhttps://gist.github.com/jtaleric/195921495a5627e4d2673ba56e0db8ad14:01
d0ugalrook: the formatting is a bit wonky there, but it looks like you have hit an error.14:02
*** oshvartz has joined #tripleo14:02
d0ugalrook: I would probably dive into the mistral logs to track it down.14:03
socialEmilienM:  why this isn't going to newton ?https://bugs.launchpad.net/tripleo/+bug/161492814:03
openstackLaunchpad bug 1614928 in tripleo "openstack overcloud update stack should be powered by a mistral workflow" [High,In progress] - Assigned to Brad P. Crochet (brad-9)14:03
rookyeah d0ugal that could be from a prior import?14:03
d0ugalsocial: because we are doing a workaround in the CLI for Newton14:03
d0ugalsocial: the full move to Mistral will happen in Ocata.14:03
*** ramishra has quit IRC14:03
EmilienMsocial: have you seen https://review.openstack.org/#/c/381351/ ?14:04
d0ugalsocial: https://bugs.launchpad.net/bugs/162612814:04
openstackLaunchpad bug 1626128 in tripleo "openstack overcloud update stack is broken" [Critical,In progress] - Assigned to Brad P. Crochet (brad-9)14:04
EmilienMsocial: maybe ask thrash14:04
sociald0ugal: thx14:04
d0ugalrook: oh, maybe, sorry it is quite hard to read that paste.14:04
openstackgerritLeif Madsen proposed openstack/tripleo-validations: Port the network environment validation  https://review.openstack.org/34158614:05
d0ugalrook: How many nodes are registered in ironic now?14:05
*** dtrainor has joined #tripleo14:05
d0ugalrook: I'm trying to figure out if it is still progressing.14:05
openstackgerritMerged openstack/tripleo-common: Support node untagging  https://review.openstack.org/38283214:06
rookdoes it wait for enroll to complete?14:06
rookd0ugal ^14:06
rook14114:06
d0ugalrook: Good question, I'm looking.14:07
rookwhere does this code live?14:07
rookso many layers of indirection14:07
d0ugalrook: Yeah, so the workflow is defined here: https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L2814:07
d0ugalrook: but this is the actual registration code: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/utils/nodes.py#L47414:08
openstackgerritMerged openstack/tripleo-ui: Deployment states optimization  https://review.openstack.org/38184514:08
d0ugalrook: I suspect we might be doing some extra setting of the node states14:10
bnemecEmilienM: So the updates job now appears to be failing correctly on that missing file bug.  How would you feel about just merging the ci patch so we can use the experimental job to test the fixes?14:10
EmilienMbnemec: looking14:10
bnemecOh, I forgot a link to the patch.14:11
bnemecEmilienM: https://review.openstack.org/#/c/374406/14:11
EmilienMbnemec: we could indeed14:12
EmilienMwfm14:12
EmilienMI'll let you change the commit msg14:12
EmilienMpanda: any progress on testing ipv6 for ovb-ha only?14:12
d0ugalrook: Would you mind opening a bug for this?14:12
bnemecEmilienM: Oh, I should probably recheck it anyway to make sure the stable jobs still pass on it.14:12
EmilienMpanda: oh I just saw https://review.openstack.org/#/c/382515/14:13
openstackgerritLeif Madsen proposed openstack/tripleo-validations: Add jsonschema validation for os-net-config data  https://review.openstack.org/35134514:13
EmilienMbnemec: I think it will, it only affects updates14:13
EmilienMbut yeah, let's remove depends-on and see if it pass. I'll +214:13
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Make the ovb-updates job work again  https://review.openstack.org/37440614:13
pandaEmilienM: I think I'm making good progress, but there's a big problem, nonha configuration with net-iso is loaded for multinode jobs too, so I may need to refactor that part too.14:14
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Do Not Merge - Test Undercloud upgrade mitaka -> newton  https://review.openstack.org/38130914:15
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Undercloud upgrade for mitaka and newton  https://review.openstack.org/38128614:15
*** snecklifter has quit IRC14:15
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment states optimization  https://review.openstack.org/38298214:16
*** sshnaidm has quit IRC14:17
*** athomas has quit IRC14:19
*** limao has quit IRC14:19
*** limao has joined #tripleo14:20
jristslagle: https://review.openstack.org/#/c/382707/14:20
jristslagle: did you see that?14:20
shardymatbu: Hey, I added a question on https://review.openstack.org/#/c/379547 when you get a moment14:21
EmilienMdprince: I have some spare time now, are you looking at firewall problem already?14:21
shardymatbu: I'm testing it locally trying to make the patch resolve bug #1625783 and it's working a little differently than how I expected14:22
openstackbug 1625783 in tripleo "unable to leverage swap-partition extraconfig" [High,Triaged] https://launchpad.net/bugs/1625783 - Assigned to Steven Hardy (shardy)14:22
dprinceEmilienM: yes, I was locally, but the python-tripleoclient has wrecked my env. I think I'm just going to revert it locally for now and move ahead14:22
EmilienMdprince: ok14:22
rookyup d0ugal so it is waiting for hosts to leave enroll14:22
rookwhy!!! :P14:22
*** liverpooler has joined #tripleo14:23
matbushardy: ack thanks for testing it, feel free to comment amend or whatever .. :) i change the temp dir the user home dir, according to Ben comment14:23
shardymatbu: yup, it's the "get tht_root file" section I'm not sure about14:24
shardyprobably missing something, just unclear what that walking the tree is doing14:24
matbushardy: yep i answer to your comment14:24
*** athomas has joined #tripleo14:25
jistrhi, has anyone seen on overcloud deploy:14:25
jistrObject GET failed: http://192.0.2.1:8080/v1/AUTH_a78762b707954f42b618b97715c4c288/overcloud/ 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<14:25
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: DO NOT MERGE fake periodic job to update cache to Ocata  https://review.openstack.org/38252114:25
jistri wonder if there's something i need to clean up?14:25
*** sshnaidm has joined #tripleo14:25
EmilienMjistr: where do you see it, in CI?14:25
shardyjistr: aha, yeah I see now, thanks14:25
jistrEmilienM: no, locally14:26
EmilienMjistr: newton/ocata?14:26
shardysorry matbu I see now14:26
jistrEmilienM: master14:26
EmilienMjistr: sounds like you might need to update your packages14:26
jistri deployed about 3 hrs ago...14:26
EmilienMwe merged a bunch of stuffs lately, i'm not sure aobut this error but it's not in CI14:26
EmilienMjistr: what version of tripleoclient do you have?14:26
*** akuznetsov has quit IRC14:27
jistri tried master and also pulling matbu's patch to fix stack updates, both behave the same14:27
jistri've used to hit weird issues when i didn't delete the overcloud plan between deploys14:27
jistrso actually to add more context14:28
jistrfirst deploy went fine14:28
jistrthen i deleted the stack and the plan14:28
jistrand now 2nd deploy hits that error ^^14:28
jistrd0ugal: does it ring a bell perhaps?14:28
jistri'll try to restore from the package just to verify it's not some desync14:29
matbujistr: hmm i think i got it also, when i delete the overcloud and clean swift and mistral14:29
shardyjistr: I'd check the container is there (swift list), and assuming it's not then check the mistral logs for errors creating the plan14:31
shardyjistr: you could also try openstack overcloud plan create to debug the plan create independent of the deploy/update14:31
jistroh i got it. Indeed it's probably some desync, when i use everything from packages, it works. So i messed up when i used tripleoclient from source, but not tripleo-common. I deployed from scratch very recently so i thought i'd be in the clear even with packages, but it's not the case...14:32
jistrthanks all :)14:32
jistrEmilienM was right ;)14:32
sshnaidmpabelanger, hi14:33
pabelangermorning14:33
EmilienMjistr: you're welcome :-P14:34
jistrbtw is there something i need to do to refresh mistral workflows/actions when using a new tripleo-common from source? i guess restart mistral at least, but maybe there's more?14:34
shardyjistr: http://paste.fedoraproject.org/444933/5764540114:35
shardyprobably don't need all the sleeps14:35
jistrshardy: awesome, thanks14:36
*** rhallisey has quit IRC14:36
*** dhill_ has quit IRC14:38
openstackgerritMartin André proposed openstack/tripleo-heat-templates: Have docker start script honor configuration  https://review.openstack.org/36613814:39
openstackgerritMartin André proposed openstack/tripleo-heat-templates: Add steps to containerized compute deployment  https://review.openstack.org/34692714:39
openstackgerritMartin André proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles  https://review.openstack.org/33065914:39
openstackgerritMartin André proposed openstack/tripleo-heat-templates: Bind mount files to run DiD in latest atomic host  https://review.openstack.org/34721814:39
openstackgerritMartin André proposed openstack/tripleo-heat-templates: Be more inclusive in insecure registry regex  https://review.openstack.org/36611114:39
*** dhill_ has joined #tripleo14:39
mandreSlower: ^^14:40
pradkmarios, confirmed the vhosts files get overridden without apache init14:41
*** limao has quit IRC14:44
EmilienMccamacho: you should not backport a patch that is not merged upstream.14:45
*** limao has joined #tripleo14:45
*** mbozhenk1 has quit IRC14:45
EmilienMbecause if the master patch change, you need to update the cherry pick again, pointless.14:45
dprinceEmilienM: can you diff the old and latest again?14:45
dprinceEmilienM: I've checked them manually too but I don't have that old file14:45
dprincehandy14:45
matbubnemec: hey :) regarding your comment, you mean 'working into a '/tmp/tmpZv7yrt' ' ? actually, in my latest update i tried to address both your and shardy comment, who was pointing the fact that we could leak the /tmp dir14:46
EmilienMdprince: ok a sec14:46
ccamachoEmilien, i did that to gain some time (passing CI) both patchsets should not be updated14:46
matbubnemec: i was originaly in mind to create a mkdtemp() dir14:46
ccamachoEmilienM good idea to have it blocked until the other gets merged14:47
EmilienMccamacho: but don't do it please14:47
EmilienMit's very risky14:47
EmilienMto merge something in stable before master and code is not same at the end14:47
ccamachoEmilienM ack14:47
EmilienMthx14:47
bnemecmatbu: I explained how to do that in my previous comment.  tmp_dir = tempfile.mkdtemp(); try: [all of the processing code]...finally: shutil.rmtree(tmp_dir)14:49
bnemecNo leaked files, no security problems, no chance of overwriting user files.14:49
EmilienMmarios, gfidente: lint issue on https://review.openstack.org/#/c/382386 - like CI actually shows it :)14:49
*** pkovar has quit IRC14:51
shardymatbu: If it helps I'm hacking on a reworked version locally which resolves the ordering issue I mention14:51
shardyI'm happy to rework the tmpdir stuff too and push a revision for consideration, unless you're already on it?14:51
matbushardy: as you want, i'm on a mtg right now, so if you have a bit of time that fine to me14:52
shardymatbu: ack, thanks, just trying to help out as this is one of the final blockers for Newton14:52
matbushardy: yep thnks very much :)14:53
*** pkovar has joined #tripleo14:53
EmilienMdprince: doing it now14:55
*** mwhahaha has quit IRC14:55
*** CaptTofu has quit IRC14:55
*** coolsvap has quit IRC14:55
*** patrickeast has quit IRC14:55
*** hrybacki has quit IRC14:55
EmilienMdprince: https://www.diffchecker.com/txfANLJw14:57
*** fultonj_ has joined #tripleo14:58
dprinceEmilienM: looks good to me. I'm wondering if there is a missing rule on the compute node?14:59
d0ugalrook: dang, thanks for tracking that down - we don't need that obviously. Did you open a bug? Otherwise I can.14:59
EmilienMdprince: excellent idea15:00
EmilienMdprince:  let me check15:00
*** mbozhenko has joined #tripleo15:00
*** ccamacho has quit IRC15:01
openstackgerritTom Barron proposed openstack/puppet-tripleo: Set enabled_share_protocols based on enabled backends  https://review.openstack.org/38238615:01
EmilienMdprince: oh wait15:01
dprinceEmilienM: also, when I read the console.log it looks like ssh is initiated15:02
EmilienMdprince: http://paste.openstack.org/show/FkhckBkIyYh3nh0sipqB/15:02
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Mistral services  https://review.openstack.org/32343615:02
*** limao has quit IRC15:02
EmilienMlooks like no iptables rules on the compute (back in August, when things worked)15:02
gfidentetbarron thanks for updating it, is it working as intended now?15:02
EmilienMlet's see now15:02
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Zaqar services  https://review.openstack.org/33168215:02
*** jlinkes has quit IRC15:02
*** limao has joined #tripleo15:02
*** mwhahaha has joined #tripleo15:02
tbarrongfidente: I knew you guys were busy.  Test deploy is still running.15:03
dprinceEmilienM: hmmm, that is almost certainly the issue. We are likely missing rules on compute then. Perhaps it never really worked then/15:03
EmilienMdprince: http://paste.openstack.org/show/EiUAtsi3G9klN9J8uMBo/15:03
dprinceEmilienM: I can disable it on compute really quick via the roles_data.yaml by removing Firewall entirely there15:03
EmilienMdprince: that's now ^15:03
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Add the workflow name to the starting workflow message  https://review.openstack.org/37533115:04
*** r-mibu has quit IRC15:04
dprinceEmilienM: at a minimum I'd say we need the vxlan one15:04
EmilienMdprince: the problem is that we never tested iptables rules on compute until now15:04
dprinceEmilienM: perhaps more15:04
EmilienMyes15:04
EmilienMdprince: ok let's iterate: please disable rules on compute now15:04
shardydprince, EmilienM: according to https://bugs.launchpad.net/tripleo/+bug/1630761 no non-controller roles get the rules configured15:04
openstackLaunchpad bug 1630761 in tripleo "Custom roles nodes don't get iptables rules configured" [High,Triaged]15:04
EmilienMdprince: and I'm working on compute iptables rules15:04
dprinceEmilienM: ack, yes. Step at a time15:04
shardynot had time to look into why yet15:04
EmilienMshardy: /me looking now. I think it's a security problem15:05
EmilienMI took the bug15:05
EmilienMdprince: ack, excellent15:05
*** numans has quit IRC15:05
*** mbozhenko has quit IRC15:05
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Re-enable ManageFirewall by default.  https://review.openstack.org/38186415:06
dprinceEmilienM: ^^ lets see what happens ther15:06
EmilienMdprince: ok I'll propose a patch on top of it for compute.15:06
EmilienMdprince: I found out I think15:07
EmilienM18 neutron vxlan networks rule is on neutron-server nodes15:07
EmilienMit must be where l2 agent is running15:07
EmilienMI'm proposing a fix15:07
dprinceEmilienM: agree that looks suspicious15:08
EmilienMdprince: let me rebase your patch15:08
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Re-enable ManageFirewall by default.  https://review.openstack.org/38186415:08
EmilienMdone15:08
*** r-mibu has joined #tripleo15:09
*** rajinir has joined #tripleo15:10
*** tremble has quit IRC15:11
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109415:12
EmilienMbandini, jistr, gfidente: can you confirm pacemaker needs vrrp open in iptables?15:14
*** _milan_ has quit IRC15:14
*** milan has joined #tripleo15:15
*** lucasagomes is now known as lucas-hungry15:15
beaglesEmilienM: it is needed for neutron L3 HA15:15
EmilienMbeagles: yeah but for pacemaker itself?15:15
beaglesEmilienM: ah... yeah, no idea :)15:16
EmilienMfor pacemaker heartbeat I mean15:16
bandiniEmilienM: no it is not needed by pacemaker. only by neutron (under certain conditions)15:16
EmilienMbandini: ok thanks15:17
*** CaptTofu has joined #tripleo15:18
*** ccamacho has joined #tripleo15:18
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38302915:18
EmilienMdprince: let's try ^15:18
openstackgerritMerged openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756215:19
rookd0ugal: one sec, I have something else to try15:20
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/38303015:21
*** mwhahaha is now known as mwhahaha_15:21
dprinceEmilienM: +2 on that one. given compute wasn't covered before there may be other roles not working as well. Like object, and ceph, or volume15:21
EmilienMdprince: good proof multinode job was not enough here, we need to increase the # of nodes15:22
EmilienMto have at least a controller/compute15:22
dprinceEmilienM: that is a start, we'd want more nodes that that to cover important cases too15:23
*** limao_ has joined #tripleo15:23
dprinceEmilienM: specifically separate rabbit, and or mysql roles. And a separate networking role15:24
bnemecI suspect this may have been at least somewhat intentional.  The firewall was protecting the nodes with a public interface.15:24
dprinceEmilienM: without upstream tests on these they will constantly broken15:24
bnemecAnd given that it didn't even work until about halfway through this cycle I imagine nobody got around to implementing firewall rules for the other nodes.15:24
dprincebnemec: agree, that hidden meaning slipped through the cracks perhaps when we decomposed the services in newton. It was just assumed Firewalls were on everywhere...15:25
EmilienMdprince: I agree15:26
*** abregman is now known as abregman|afk15:26
*** limao has quit IRC15:27
gfidenteEmilienM bandini jistr no I don't think pcmk uses vrrp15:28
bandinigfidente: nope it most definitely does not15:28
EmilienMok good15:28
EmilienMI didn't create the rule for it15:28
bandiniunless they snuck it in when I was drunk15:28
*** saneax is now known as saneax-_-|AFK15:29
*** hrybacki has joined #tripleo15:29
bnemecI feel like something used vrrp though.  keepalived maybe?15:30
*** bana_k has joined #tripleo15:30
EmilienMah yes indeed15:30
EmilienMbnemec: thx :)15:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38302915:31
gfidentebnemec though we don't use keepalived in the pcmk scenario15:33
gfidenteEmilienM ^15:33
bandinibnemec: yes keepalived and neutron l3 ha15:33
*** abregman|afk is now known as abregman15:33
EmilienMgfidente: I thought we used it for the vip15:34
gfidenteEmilienM in pcmk scenario is pcmk owning the VIPs15:34
gfidentein non-pcmk is keepalived15:34
EmilienMmaybe on nonha jobs?15:34
gfidentenonha exactly15:35
EmilienMok cool15:35
pandabnemec: is it possible to have a predicatble endpoint VIP when using network isolation ?15:36
*** patrickeast has joined #tripleo15:37
openstackgerritMerged openstack/tripleo-ui: Deployment states optimization  https://review.openstack.org/38298215:37
openstackgerritMerged openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/38303015:38
bnemecpanda: Yes, that's what is in the docs now.15:38
bnemecparameter_defaults:15:38
bnemec    PublicVirtualFixedIPs: [{'ip_address':'10.0.0.1'}]15:38
*** [1]cdearborn has quit IRC15:39
b00tcathey, I'm in a new laptop so I forgot to add the commit-id thing to my last commits15:40
b00tcat`git-review` suggestion only applies to the last commit15:40
b00tcatdoes someone know how to add the commitId to the latest `n` commits?15:41
b00tcator is it even possible?15:41
shardyb00tcat: you can create a new branch, then cherry pick the commits one at a time, then do git commit --amend to add the ChangeId15:41
pandabnemec: mh, thanks15:41
shardyI'm not aware of a way to batch-amend the commits15:42
b00tcatshardy: that's smart, I'll try that thanks!15:42
pandabnemec: the "mh" is because I'm not sure where to put this in current jobs configuration15:42
*** bana_k has quit IRC15:44
bnemecb00tcat: git rebase -i HEAD~#, where # is the number of commits you need to amend.15:45
bnemecChange the "pick" on each commit to "edit" and it will stop and let you amend each one as it rebases.15:45
bnemecDo git rebase --continue after each amend.15:46
*** coolsvap has joined #tripleo15:47
shardybnemec: ah, nice, I'd forgotten about edit15:48
*** mwhahaha has joined #tripleo15:49
bnemecSome day I'm going to sit down and record a video of all these git/gerrit common tasks.  I've been meaning to for about two years. :-)15:50
rbradybnemec++15:50
jpichd0ugal: Hey, I wonder if I may have found a belated issue with https://review.openstack.org/#/c/381169/15:52
openstackgerritAlejandro Andreu proposed openstack/tripleo-puppet-elements: Installs more packages with the Midonet element  https://review.openstack.org/38305615:53
openstackgerritAlejandro Andreu proposed openstack/tripleo-puppet-elements: Separate Datastax repository from the Midonet one  https://review.openstack.org/38305715:53
openstackgerritAlejandro Andreu proposed openstack/tripleo-puppet-elements: Update git commit sources for Midonet element  https://review.openstack.org/38305815:53
shadowerjtomasek: hey so do I understand it correctly, that the GUI calls the tripleo.baremetal.v1.register_or_update workflow to register nodes?15:53
*** ohamada has quit IRC15:53
d0ugaljpich: oh, fun.15:53
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] Add IPv6 coverage using existing jobs  https://review.openstack.org/38251515:54
jpichshadower: Should be correct15:54
rbradyjpich: should be a quick patch.  do you want to file a bug?15:54
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000415:54
shadowerjpich: awesome, thanks!15:55
jpichrbrady: It's not merged yet so maybe we can fix it as part of it? I'd like to file a bug anyway so it's easier to backport (the missing param is breaking deploys in the UI)15:55
beaglesugh... if someone decides to enable GRE tunnelling they are going to get bitten by the firewall rule issue as well15:55
beaglesEmilienM: ^ do we care?15:55
d0ugaljpich: Your UI deploys will break missing another param when these are fixed :(15:55
*** akshai_ has quit IRC15:55
d0ugaljpich: The other one is proving much harder to resolve15:55
EmilienMbeagles: we might need to support both15:55
rbradyjpich: ack15:56
EmilienMbeagles: GRE is port 47 ?15:56
jpichd0ugal: Oh :( Is there a bug for that one? I'm trying to document workarounds + bugs/patches needed for the UI at https://etherpad.openstack.org/p/tripleo-ui-newton-rc during this rough period :)15:56
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform a booting test for our images  https://review.openstack.org/20463915:56
d0ugalEmilienM: It looks like you added this, is this how keystone credentials must be created? https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/utils.py#L57215:57
EmilienMd0ugal: yes Sir15:57
EmilienMd0ugal: I stole it from keystone istelf15:57
EmilienMitsel15:57
jpichhaha15:57
EmilienMerr.. /me goes sleep15:57
d0ugallol15:57
EmilienMwell you got it15:57
beaglesEmilienM: yup15:57
thrashd0ugal: I'm still at a loss on this update thing.15:57
d0ugalEmilienM: Indeed, thanks, I'll steal it also15:57
d0ugaljpich: no, there isn't a but - I was just tackling these under the overall password generation bug thing15:58
d0ugalthrash: damn :(15:58
* d0ugal juggles15:58
EmilienMbeagles: indeed then, we need to add it too. I'm curious how can we make it flexible but I have an idea ;-)15:58
d0ugalthrash: What happens if (when?) we miss RC3?15:58
thrashd0ugal: since we have a bug for it, we can still backport.15:58
d0ugalthrash: okay, good. Just trying to figure out where I should focus my attention right now :)15:59
thrashd0ugal: ack. Don't pull away from anything just yet. I'm still messing with it.15:59
thrashshardy: is there anything recently in heat that you know of that might mess with breakpoints?15:59
EmilienMbeagles: wait no, it's 'gre' proto in iptables, no port. Right?16:00
d0ugaljpich: The other issue you will hit is that SnmpdReadonlyUserPassword isn't set.16:00
beaglesEmilienM: just thinking that... dbl checking the extact rule16:00
d0ugaljpich: well, it is set as an empty string16:00
*** ebarrera has quit IRC16:00
shardythrash: Nothing that I'm aware of, but honestly I've not tested any breakpoint stuff in a while16:00
*** rcernin has quit IRC16:01
thrashshardy: ack.16:01
thrashd0ugal: question for ya... once someone does a deployment and the plan is created, it contains all of the environment files too right?16:02
d0ugalthrash: Yeah, in a slightly strange way, but yeah16:02
thrashmatbu: take a look at bnemec's comment on https://review.openstack.org/#/c/381899/7/tripleoclient/v1/overcloud_update.py16:02
thrashmatbu: if he hasn't commented on your patch yet.16:02
thrashd0ugal: so, the update command should only need to just pull down the files from the plan, and work with those, right?16:03
*** weshay is now known as weshay_bbiab16:03
thrashand add in the environment that update needs...16:03
jpichd0ugal: Ok, thank you for the information. The workaround would be to check the hiera data and set it manually before the deploy, for now?16:03
thrashd0ugal: it doesn't make sense to me to pass additional environment files to update.16:04
beaglesEmilienM: yup it's just gre16:04
EmilienMbeagles: ok16:04
beaglesbeen forever since I deployed with GRE16:04
*** kberger has quit IRC16:05
pandaEmilienM: bnemec: something I'm missing for the ipv6 gate is why we don't just add the ha-ipv6 gate job to the list of gates. Resources problem ?16:05
*** kberger has joined #tripleo16:05
*** cdearborn has joined #tripleo16:05
bnemecpanda: Right.16:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38302916:06
EmilienMbeagles: done ^16:06
EmilienMpanda: yep16:06
bnemecWe've only got capacity to run so many jobs, and we need the nonha, ha, and updates jobs.  Adding ha-ipv6 too would ~double our current load.16:06
bnemec(because we're not running updates yet either)16:06
beaglesEmilienM: we can ditch VRRP on the compute dvr btw16:06
dprincebnemec: I think just running it on one of the jobs woudl be a start16:07
EmilienMbeagles: why?16:07
beaglesEmilienM: HA routers don't get scheduled to L3 agents running on compute nodes16:07
EmilienMah k16:07
*** rhallisey has joined #tripleo16:07
bnemecdprince: Yeah, it's on the updates job, but the updates job wasn't working until like yesterday. :-)16:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38302916:07
dprincegotcha16:08
*** hewbrocca is now known as hewbrocca-afk16:08
EmilienMbnemec, dprince, panda: why not deploy hajob with ipv6 and nonha with ipv4?16:08
bnemecEmilienM: Because then we have no CI coverage without net-iso.16:08
tbarrongfidente: marios looks like it worked! - http://paste.fedoraproject.org/444996/77004314/16:08
gfidentetbarron cool, thanks16:08
dprinceEmilienM: IPv4 is still probably the most common case. I'd hate to see us not cover that for the ha job16:08
pandaEmilienM: there is a discussion on this on the change comments16:08
bnemecAnd a lot of devs deploy that way, so breaking it would be bad.16:08
* beagles makes note on checklist "do firewall rules make sense everywhere"16:09
pandaaslo, I'm tryiong to point out that with the current script, it's difficult to separate nonha and multinode configuration16:09
dprincebeagles: if we don't have firewalls everywhere I fear that we'll leave holes open with all the incoming composability16:10
beaglesdprince: indeed!16:10
beaglesdprince: I meant "make sure the firewall rules are properly configured" :)16:11
dprincebeagles: noting that I don't even run the firewalls in my dev environment. There are users who will roll this way too16:11
*** akshai has joined #tripleo16:11
beaglesack16:11
dprincebeagles: I can only care soo much about things. But maybe I'll turn them on16:12
jpichd0ugal, rbrady: I opened https://bugs.launchpad.net/tripleo/+bug/1631049 for that KeystoneProperty0 issue, if you want to reference it in the next version of the patch16:12
openstackLaunchpad bug 1631049 in tripleo "Property KeystoneCredential0 is not assigned" [High,In progress] - Assigned to Dougal Matthews (d0ugal)16:12
*** lucas-hungry is now known as lucasagomes16:12
trownEmilienM: I set https://bugs.launchpad.net/tripleo/+bug/1631052 to ocata-1 because I do not have a patch for it and we are trying to put RC3 today, but I think it is a bit of a blocker (at least for anyone not using tripleo.sh)16:13
openstackLaunchpad bug 1631052 in tripleo "openstack overcloud image upload command is broken" [Critical,Confirmed]16:13
EmilienMtrown: right it is16:13
trownEmilienM: revert of https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=b2e5648fda8815dd448777339e6a24d95e0d48bb would be a very quick fix16:14
d0ugaljpich: will do, thanks!16:14
trownI can put that up if we are ok with that16:14
EmilienMJuan is not around16:14
EmilienMtrown: go for it16:14
EmilienMtrown: at least in stable/newton16:14
*** jistr is now known as jistr|afk16:14
*** aufi has quit IRC16:15
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add the KeystoneCredential0 and KeystoneCredential1 parameters  https://review.openstack.org/38116916:15
d0ugaljpich: ^16:15
trownEmilienM: oh, actually I dont think it was backported and landed after we branched, so we should be fine wrt blocker16:16
tbarrongfidente: marios EmilienM so https://review.openstack.org/#/c/382386/6 looks good from a functional standpoint; i will follow with tests on openstack.org/#/c/382386/7 but I can't imagine a functional difference.16:16
trownEmilienM: so maybe worth figuring out how to fix it properly in tripleoclient since glance V1 API is going away16:16
jpichd0ugal: Thanks! Looking now16:16
*** jcoufal has quit IRC16:16
d0ugaljpich: cheers, just testing it manually16:17
EmilienMtrown: wait, are we sure the commit isn't in stable/newton?16:17
trownEmilienM: I dont see it https://git.openstack.org/cgit/openstack/instack-undercloud/log/?h=stable/newton16:17
EmilienMtrown: ok same. Good :-D16:17
EmilienMno more bugs please!16:18
trownI hit the issue on master16:18
d0ugaljpich: actually, I wont be able to do that right now, my clouds are reforming.16:18
EmilienMgfidente: can you write commit messages when you send a patch? example: https://review.openstack.org/#/c/382386/16:18
d0ugalrbrady, EmilienM: can you re-review https://review.openstack.org/#/c/381169 now than I'm doing it properly?16:19
*** jcoufal has joined #tripleo16:20
EmilienMd0ugal: done16:20
d0ugalThanks!16:20
thrashbnemec: I totally get your concerns about the tmp directory... However, there is no reliable way to know when everyone is done with those templates.16:21
thrashbnemec: re: https://review.openstack.org/#/c/379547/20 and my update patch16:21
bnemecthrash: Why not?  They only need to persist through the running of the command, right?  After that we're done with the local copies.16:22
thrashbnemec: I guess as long as the heat api call has been made, we can get rid of them?16:22
bnemecthrash: I would think so, but we poll the status in update anyway, don't we?  We could just wait until everything is done.16:23
*** penick has joined #tripleo16:24
openstackgerritJohn Trowbridge proposed openstack/instack-undercloud: Revert "Stop pinning Glance API version on stackrc"  https://review.openstack.org/38309116:24
thrashbnemec: not necessarily. If you don't do an interactive update, then the answer to your question is no.16:24
trownI put up a revert anyways ^ because CI did not legitimately pass on the original change16:24
*** mwhahaha_ has quit IRC16:24
thrashbnemec: but i think you're right... once we've made the call to either stacks.create or stacks.update, they are no longer needed.16:24
*** fultonj_ has quit IRC16:25
thrashall the other calls just refer to a stack id16:25
*** limao has joined #tripleo16:25
*** ccamacho1 has joined #tripleo16:25
bnemecthrash: Yeah, I would think the api call sends whatever local data Heat needs as part of the call.16:25
*** ccamacho has quit IRC16:26
gfidentetbarron see Emilien's comment on https://review.openstack.org/#/c/382386/16:26
gfidentedo you want to update the commit message?16:26
gfidente(explaining why)16:26
openstackgerritJohn Trowbridge proposed openstack/instack-undercloud: Revert "Stop pinning Glance API version on stackrc"  https://review.openstack.org/38309116:26
EmilienMgfidente: it helps reviewers to understand what is happenning16:26
EmilienMtrown: maybe shoot an email to Juan16:27
thrashbnemec: I will try it on the update. It's broken as it is. :P16:27
openstackgerritLucas Alvares Gomes proposed openstack/instack-undercloud: Add option to set a custom dnsmasq configuration file  https://review.openstack.org/36439116:27
bnemecthrash: Yeah, I noticed.  It didn't fix the updates job when I added the depends-on. :-)16:28
bnemecthrash: Hopefully we can get the ci patch merged soon so we can test the client patches more easily though.16:28
gfidenteEmilienM yeah again it was in the LP bug but having it in the commit is also good16:28
trownEmilienM: I added him to the review16:28
*** limao_ has quit IRC16:28
EmilienMcool16:28
* EmilienM afk lunch16:28
tbarrongfidente: ok, i'll update the commit msg16:29
gfidentetbarron ack16:29
thrashbnemec: definitely. But there is something fundamentally broken in the update code. What I am seeing is the breakpoint hits, and I clear it, but nothing happens. It just spins until it times out16:29
bnemecthrash: Good times. :-)16:31
*** tesseract- has quit IRC16:31
*** limao has quit IRC16:34
*** b00tcat has quit IRC16:38
*** limao has joined #tripleo16:39
*** cylopez has quit IRC16:40
lucasagomeshi, can I get some eyes on https://review.openstack.org/#/c/370773/  ? It's tiny review already +2 fixing the enrollment of nodes using the UCS driver. Thanks16:41
rookd0ugal so now I am curious if a single host fails to go from enroll->managable if that causes the hiccup :/16:41
*** dsariel_ has quit IRC16:43
rooklucasagomes dtantsur what is the best medium to start the discussion on making oc deploy more resilient?16:44
*** ebarrera has joined #tripleo16:44
dtantsurrook, define resilient please :)16:45
lucasagomesrook, hmm we need to map what parts needs to be fixed and probably open bugs for them16:45
dtantsurprobably the ML in any case, with [ironic] tag16:45
rookdtantsur: "able to withstand or recover quickly from difficult conditions."16:45
dtantsurrook, I think it would be cool to collect real life examples and bring them to the ML. we were discussing it last summit, but we didn't have enough examples.16:46
rookso, i should use ironic or tripleo?16:48
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Set enabled_share_protocols based on enabled backends  https://review.openstack.org/38238616:48
*** rbrady is now known as rbrady-afk16:49
bnemeclucasagomes: Was that a typo being fixed, or did the name actually change?16:49
lucasagomesbnemec, the name changed16:49
lucasagomesbnemec, I put the link to the patch changing it in the commit message16:50
bnemeclucasagomes: Because we don't have stable branches for os-cloud-config, so this is going to break with older Ironics.16:50
lucasagomesbnemec, hmmm16:50
bnemeclucasagomes: Is it safe to just set both, or will Ironic freak out if we pass something it doesn't expect?16:50
lucasagomesbnemec, it's safe, yes16:51
d0ugalrook: use tripleo I think16:51
lucasagomesbnemec, looks odd when you do a node-show because the informaiton will be duplicated16:52
lucasagomesbut in ironic it's safe16:52
jtomasekshadower: yes16:53
lucasagomesbnemec, you wanna go that route then ?16:53
bnemeclucasagomes: That seems like the simplest solution.  If they don't want it duplicated, then they shouldn't break their driver interface like this. :-)16:53
dtantsurrook, it depends on whose attention you want to get :) you pinged Lucas and me, so I assumed you want something from Ironic :)16:54
rookdtantsur: lol16:55
rookdtantsur: ill just email you guys directly then16:55
*** rasca has quit IRC16:55
bnemeclucasagomes: Come to think of it, how does that work on the Ironic side?  If someone registered a node with the old name, then upgrades Ironic, is there something to map the old name to the new name?16:55
shadowerjtomasek: thanks16:55
*** derekh has quit IRC16:56
dtantsurrook, well, big changes in Ironic won't happen after private emails ;) but again, I probably miss the context of the problems you're trying to solve16:56
lucasagomesbnemec, we would keep both options working for at least 1 cycle and logigng a warm when the old one is used16:56
lucasagomesand then after the deprecation period we would remove the old one16:56
lucasagomessimilar to what we do with config options16:56
*** jtomasek_ has joined #tripleo16:56
lucasagomesand now we have release notes as well to notify deployers that a deprecation option was removed and they have to update as part of the upgrade16:57
bnemeclucasagomes: Okay, makes sense.  Was just wondering.16:57
*** jistr|afk is now known as jistr16:57
lucasagomesbnemec, but yeah, since cloud-config does not cut releases. I guess the best way to keep it backward compatible is to register both16:58
lucasagomesbnemec, I will change the patch16:58
bnemeclucasagomes: Yeah, maybe leave a comment there that the old name can be removed once Mitaka (?) Ironic goes out of support.16:58
*** jcoufal_ has joined #tripleo16:59
lucasagomesbnemec, ack will do16:59
*** limao_ has joined #tripleo16:59
*** yamahata has quit IRC17:01
*** yamahata has joined #tripleo17:01
*** jcoufal has quit IRC17:01
*** dtantsur is now known as dtantsur|afk17:02
*** cylopez has joined #tripleo17:02
*** cylopez has quit IRC17:02
*** limao has quit IRC17:02
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Downloads templates from swift before processing update  https://review.openstack.org/38189917:04
*** fzdarsky is now known as fzdarsky|afk17:06
*** saneax-_-|AFK has quit IRC17:07
*** chem has quit IRC17:07
rookshardy: heya so I am doing a large deployment now, watching rss memory w/ heat-engine17:08
rookshardy: if you are interested in watching, snooping in17:08
rooksshnaidm: heya17:09
dprinceshardy: the templates are in so much flux every time I rebase my hiera patch I have to start over again :/17:09
rooksame as ^^^17:09
sshnaidmrook, yeah, sure17:09
dprinceshardy: it hasn't been top of list mind you, but proving to be a significant rebase time sync17:10
rookwhat might be interesting is to monitor things going on in heat, and try to associate the memory growth with actions in heat17:11
rookbut I dunno what/where to look17:11
rookshardy ^ maybe you can help with that.17:11
rookwe could add some sort of collection of information?17:11
*** pkovar has quit IRC17:12
shardydprince: I'm sorry about that - we made the decision to go ahead and get the composability features landed for Newton, and a bunch of stuff had to change to make that possible17:12
shardyI hope it'll mean much less churn during ocata, but I agree it's been a rough process with the custom roles stuff landing so late17:13
*** jpich has quit IRC17:14
dprinceshardy no need to appoligies man. I'm turning my head towards ocata w/ this anyways. But it is really wild :)17:14
*** weshay_bbiab is now known as weshay17:15
*** fzdarsky|afk has quit IRC17:17
*** saneax-_-|AFK has joined #tripleo17:18
*** tosky has quit IRC17:18
*** cdearborn has quit IRC17:18
*** abregman has quit IRC17:18
lucasagomesbnemec, I'm looking at the commit history in ironic, apparently it was never called ucs_hostname. It was a mistake in the documentation only17:19
lucasagomesbnemec, https://review.openstack.org/#/c/159734/28/ironic/drivers/modules/ucs/helper.py17:19
lucasagomesthat's the patch introducing the driver17:19
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000417:19
lucasagomesbnemec, and this patch https://review.openstack.org/#/c/232834/ is only fixing the docs17:20
openstackgerritLucas Alvares Gomes proposed openstack/os-cloud-config: Replace ucs_hostname with ucs_address  https://review.openstack.org/37077317:23
* lucasagomes updated the commit message ^17:23
*** jpena is now known as jpena|off17:24
*** bnemec has quit IRC17:25
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Specify the Ceph packages to be installed  https://review.openstack.org/38213217:25
*** cdearborn has joined #tripleo17:34
*** athomas has quit IRC17:38
*** bnemec has joined #tripleo17:38
*** panda is now known as panda|off17:40
*** trown is now known as trown|lunch17:41
*** milan has quit IRC17:45
*** bnemec has quit IRC17:47
*** jprovazn has quit IRC17:48
*** bana_k has joined #tripleo17:48
openstackgerritPaul Belanger proposed openstack/python-tripleoclient: Do not merge  https://review.openstack.org/38316417:48
*** rbrady-afk is now known as rbrady17:50
EmilienMdprince: your patch is passing ovb ha, congrats17:52
EmilienMdprince: firewall17:52
*** bnemec has joined #tripleo17:57
EmilienMbnemec: can you review https://review.openstack.org/#/c/381864/ please?17:59
*** ebarrera has quit IRC18:00
*** bnemec has quit IRC18:03
ccamacho1EmilienM mind to check https://review.openstack.org/#/c/382911/ already passing CI18:05
*** cdearborn has quit IRC18:06
EmilienMccamacho1: looking18:06
*** bnemec has joined #tripleo18:08
ccamacho1EmilienM thanks! Ill be around until things get merged. we still have 10 hours left :)18:08
*** cdearborn has joined #tripleo18:10
openstackgerritAlex Schultz proposed openstack/puppet-pacemaker: Pin beaker to < 3.0.0  https://review.openstack.org/38318418:12
openstackgerritAlex Schultz proposed openstack/puppet-pacemaker: Changed the home-page to point Openstack Puppet Homepage  https://review.openstack.org/38158218:13
*** shardy is now known as shardy_afk18:14
openstackgerritPaul Belanger proposed openstack/python-tripleoclient: Do not merge  https://review.openstack.org/38316418:14
*** flepied1 has joined #tripleo18:15
*** yamahata has quit IRC18:15
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Serialize AllNodesDeployment and UpdateWorkflow  https://review.openstack.org/38318818:16
*** mcornea has joined #tripleo18:16
*** bnemec has quit IRC18:17
*** flepied has quit IRC18:18
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000418:20
*** amoralej is now known as amoralej|off18:22
*** zoli|wfh is now known as zoli|gone18:23
*** zoli|gone is now known as zoli_gone-proxy18:24
openstackgerritAndreas Jaeger proposed openstack/diskimage-builder: Enable release notes translation  https://review.openstack.org/38321018:26
*** bnemec has joined #tripleo18:27
EmilienMfirewall/compute: pingtest OK :)18:31
EmilienMsounds like enabling VXLAN was good :)18:31
openstackgerritMerged openstack/puppet-tripleo: Explicitly use Keystone v2 endpoint in the UI  https://review.openstack.org/38272418:31
*** bnemec has quit IRC18:33
*** bnemec has joined #tripleo18:34
ccamacho1dprince around? mind to check https://review.openstack.org/#/c/382911/  +2ed and passing CI, need to land it to land the backport18:36
mcorneaEmilienM: hi! quick question regarding the firewall rules: they have an implicit accept policy and there's no deny/reject rule at the end so I guess we allow all traffic even if we set specific services rules. Do I get it right?18:36
ccamacho1gfidente around?18:37
EmilienMmcornea: I think we're dropping in last rule by default AFIK18:37
EmilienMlet me check18:37
EmilienMmcornea: https://github.com/openstack/puppet-tripleo/blob/master/manifests/firewall/post.pp#L43-L4718:38
EmilienMlet's see if we enable debug18:39
*** trown|lunch is now known as trown18:39
mcorneaEmilienM: I don't have an overcloud deployed atm to check it but noticed this in the output in https://bugs.launchpad.net/tripleo/+bug/163076118:40
openstackLaunchpad bug 1630761 in tripleo "Custom roles nodes don't get iptables rules configured" [High,In progress] - Assigned to Emilien Macchi (emilienm)18:40
*** bnemec has quit IRC18:41
EmilienM-A INPUT -m comment --comment "999 drop all" -m state --state NEW -j DROP18:41
*** saneax-_-|AFK has quit IRC18:41
EmilienMthat's the last INPUT rule18:41
EmilienMsee in:18:41
EmilienMhttp://logs.openstack.org/64/381864/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/fa37860/logs/overcloud-controller-0/var/log/host_info.txt.gz18:41
EmilienMthough we don't block in OUTPUT but less critical I would say18:42
mcorneaok, I see it there. I'll check it on my env as well once I get something deployed. thanks18:42
EmilienMcool18:43
*** saneax-_-|AFK has joined #tripleo18:44
openstackgerritPaul Belanger proposed openstack/python-tripleoclient: Do not merge  https://review.openstack.org/38316418:46
ccamacho1beagles around?18:47
beaglesccamacho1: hey, s'up?18:47
openstackgerritAndreas Jaeger proposed openstack/puppet-pacemaker: Enable release notes translation  https://review.openstack.org/38335418:47
ccamacho1supa!  man mind to check https://review.openstack.org/#/c/382911/ already passed CI +2ed and waiting for merge, I need that for rc318:48
beaglesccamacho1: ack.. looking18:48
ccamacho1beagles thanks!!!18:48
*** yamahata has joined #tripleo18:53
*** bnemec has joined #tripleo18:54
beaglesccamacho1: so if getting the excludes fails, we don't run the jinja render?18:55
tbarrongfidente: when you get a chance, pls. see alex schulz's -1 on https://review.openstack.org/#/c/382386/18:55
ccamacho1it skips the missing file and defaults the list to {'name'=[]} that list is only for those templates that we dont want to render18:56
*** jcoufal_ has quit IRC18:56
ccamacho1then if {'name'=[]} will render all18:56
ccamacho1beagles ^18:56
ccamacho1i.e. we stil have controller-role.yaml18:56
ccamacho1so the list will have {'name'=[puppet/controller-role.yaml]}, this validation is in case of not having that file18:57
beaglesccamacho1: I'm thinking of the 'return' in the except block... won't that just exit the function without running the code blow the try/except?18:58
ccamacho1nope18:58
ccamacho1:)18:58
ccamacho1check the logs from the comments18:58
ccamacho1this is what you will have18:59
ccamacho12016-10-06 13:05:05.801 567 INFO tripleo_common.actions.templates [-] No J2 exclude file found, defaulted to: {'name': []}18:59
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Include ceilometer in swift proxy pipeline  https://review.openstack.org/37195018:59
*** jtomasek_ has quit IRC19:00
ccamacho1beagles end to end test https://paste.fedoraproject.org/444864/47576058/19:00
gfidentetbarron ack mwhahaha commented on https://review.openstack.org/#/c/382386/19:00
mwhahahagfidente: the point is they are  new lookups to the class19:01
mwhahahawe need to be doing hiera lookups as parameters so that when we write tests we can properly test the logic19:01
ccamacho1beagles dude also this one https://review.openstack.org/#/c/382217/ which is missing from https://review.openstack.org/#/c/378764/ as we didnt had the template merged19:02
gfidentemwhahaha yeah but those are somewhat special19:02
ccamacho1please :)19:02
gfidenteheat generates "glance_api_enabled"19:02
gfidenteor "ceph_mon_enabled"19:02
gfidenteand so on, for each role19:02
gfidentewe don't want those to be class-specific19:02
gfidente(for each service)19:02
mwhahahagfidente: the point is for testing19:03
mwhahahaand puppet structure19:03
mwhahahanot tripleo functionality19:03
mwhahahause the hiera lookup as the default value for parameters to the class19:03
beaglesccamacho1: I'm either misunderstanding something or not asking the right question... 1s19:03
gfidentemwhahaha ok19:03
ccamacho1thanks19:04
mwhahahagfidente: for example, https://review.openstack.org/#/c/380406/2/manifests/profile/base/ceilometer/collector.pp19:04
mwhahahaallows us to do https://review.openstack.org/#/c/380406/2/spec/classes/tripleo_profile_base_ceilometer_collector_spec.rb19:04
mwhahahawhere if you bury the hiera lookups in the class there is not a good way to exercise the logic you're writting19:04
mwhahahaany new hiera lookups should go as class parameters and not be buried in the puppet class itself19:05
*** yamahata has quit IRC19:05
mwhahahait also allows us to see what we're actually using in each class in terms of hieradata that we query and not using databindings for19:05
*** jcoufal has joined #tripleo19:06
beaglesccamacho1: I guess this is what I'm getting at: http://paste.fedoraproject.org/445129/75780764/19:06
gfidentemwhahaha agreed, working on the update19:07
mwhahahathanks19:07
*** yamahata has joined #tripleo19:08
matbushardy_afk: do you plan to update the review or do you want me to update regarding the comments from you and ben ?19:08
*** absubram has joined #tripleo19:08
dprinceccamacho1: +2 +A19:10
EmilienMgfidente: can we cut puppet-ceph?19:10
*** bnemec has quit IRC19:10
EmilienMgfidente: for newton19:10
EmilienMgfidente: or is there still WIP in there?19:11
ccamacho1beagles I was getting the info from the python docs, let me get it again19:11
*** bnemec has joined #tripleo19:11
ccamacho1dprince also mind to review https://review.openstack.org/#/c/382217/ this is to add the per-network hostnames in the generic role.  (coming from https://review.openstack.org/#/c/378764)19:13
ccamacho1??19:13
ccamacho1thanks!!!19:13
*** coolsvap has quit IRC19:16
trozetslagle: hi19:16
openstackgerritMerged openstack/puppet-pacemaker: Enable release notes translation  https://review.openstack.org/38335419:18
beaglesccamacho1: done with https://review.openstack.org/#/c/382217/19:19
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Set enabled_share_protocols based on enabled backends  https://review.openstack.org/38238619:21
tbarrongfidente: thanks ^^, will do a deploy with that patch set19:22
trozetdoes anyone know how to fix " install failed with error Running install-packages install. Package list: PyYAML" when building overcloud images? is there some ENV variable I'm missing?19:23
* beagles reviews what mwhahaha was saying.. learning ensues19:23
tbarronmwhahaha: if you can take a look at https://review.openstack.org/382386 before I start the next deploy that would be awesome19:24
mwhahahatbarron: on it19:24
mwhahahatbarron gfidente: looks good, thanks19:25
tbarronmwhahaha: gfidente thanks, uploading the latest puppet modules now ...19:26
gfidenteEmilienM puppet-ceph seems good to me, was waiting for https://review.openstack.org/#/c/382131/ to merge19:26
gfidentethanks mwhahaha for checking19:26
beaglesccamacho1: so is it that you expect the 'return' statement in the catch to have no affect (i.e. the rest of the function is run) or is it supposed to return early...19:26
EmilienMgfidente: ok I'm cutting it. Thoughts mwhahaha ?19:27
mwhahahaEmilienM: should be ok19:27
* beagles may need a dose of caffeine19:27
*** Goneri has quit IRC19:28
beaglesccamacho1: actually in the except block just above it, the return comes after a log msg saying it is skipping the templating19:28
beaglesccamacho1: if that's what is intended, my apologies :)19:29
ccamacho1should fail early yeahp19:29
ccamacho1yeahp19:29
beaglesaaaahhhh okay19:29
ccamacho1we need to agree what to do19:29
EmilienMgfidente: if that change please let me know. We're releasing puppet-ceph now.19:29
ccamacho1but I needed to remove that defaulted list19:29
gfidenteEmilienM so up to and including https://review.openstack.org/#/c/382131/ but19:30
gfidenteworks for me19:30
beaglesccamacho1: apologies19:30
gfidente*thanks* EmilienM19:30
ccamacho1if we decide that the file is not mandatory (or if can be empty) we need to change it19:30
beaglesack19:30
ccamacho1but after rc319:30
ccamacho1yeahp19:30
ccamacho1:)19:30
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: enable undercloud/ssl on multinode jobs  https://review.openstack.org/38208219:32
*** penick has quit IRC19:34
slagletrozet: hello19:34
trozetslagle: hey your name pops up for multiple hits on google about install failed with error Running install-packages install. Package list: PyYAML19:35
slagledang. am I getting ad revenue from that?19:35
trozetslagle: i see you closed a bug that was filed on it, and looks liek wrote some element in DIB19:35
trozetslagle: haha I'll paypal you a dollar if you fix this for me, how about that? ;)19:35
EmilienMslagle: lol19:35
EmilienMyour name is on google when I type "tripleo bug"19:36
trozetEmilienM: i think all i have to do is search puppet error to find you :)19:36
slagleEmilienM: i'm sure i'm the #1 hit19:36
EmilienM:)19:36
trozetcongrats you're famous19:36
trozeton par with taylor swift19:37
slagletrozet: how are you building images? can i see the command and log somewhere?19:37
trozetslagle: well I was trying to build using python-tripleoclient19:37
trozetslagle: openstack overcloud image build --type overcloud-full --overcloud-full-name overcloud-odl --base-image /home/stack/rhel-guest-image-7.2-20160302.0.x86_64.qcow219:37
trozetslagle: so one thing here is --base-image in tripleoclient doesnt actually work so i had to export DIB_LOCAL_IMAGE19:38
slagleok yea i was going to suggest that19:38
slagleyou also probably need to inject some yum repo files via DIB_YUM_REPO_CONF19:38
trozetslagle: yeah that was what i was wondering if I need to set that19:39
ccamacho1LOL this sounds like https://www.youtube.com/watch?v=y4uTwTxefbE  (too bad its in Spanish)19:39
trozetslagle: so my goal here is to build the overcloud-odl image, which uses a different tripleo-common image-yaml19:39
trozetslagle: so is it correct that they --type should still be overcloud-full, and hten --overcloud-full-name will search that image-yaml for the right image name?  I'm a bit confused on how that works19:40
slagleif you want to use the image-yaml approach, you can't use python-tripleoclient19:40
slagleyou'd want to use tripleo-build-images from tripleo-common19:40
slaglethat should still honor the env vars I think19:41
*** mbozhenko has joined #tripleo19:41
trozetslagle: yeah so I tried that way too heh.  But no matter what image-yaml I use: https://paste.fedoraproject.org/445152/raw/19:41
slaglewhose name is on google for that one?19:43
slaglei don't think you want --config-file19:43
slagletry --image-config-file19:43
slagleand pass it overcloud-odl-rhel7.yaml instead of overcloud-images.yaml19:43
*** jbennett_ has joined #tripleo19:44
slagleand be sure also have the DIB_* vars set19:44
jbennett_\join19:44
jbennett_hello19:44
jbennett_during swift introspection I received the following error: Swift failed to create container ironic-inspector. Error was: Container PUT failed: http://10.1.82.1:8080/v1/AUTH_88cd3098b3d94f33bc1e4f03a384cbf4/ironic-ins pector 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.< |19:45
trozetslagle: haha well i asked trown first about that, but he didnt come on google19:45
pabelangeranybody mind +3 on https://review.openstack.org/#/c/376469/ adds ssl development heads for bindep, so buildimage jobs works again19:46
*** mbozhenko has quit IRC19:46
pradkEmilienM, i think this is in pretty good shape now https://review.openstack.org/#/c/360004/25 .. see my last comment19:46
trozetslagle: this is my first time building with RHEL, do i need all those portal registration vars?19:47
EmilienMpradk: nice work19:47
slagletrozet: not if you pass in the right repo files via DIB_YUM_REPO_CONF19:47
*** penick has joined #tripleo19:47
pradkthx19:47
slagletrozet: if those repos are just unauthenticated yum repos, then the image build will use those19:48
slagletrozet: and that is much easier than trying to register to the portal during the image build19:48
trownI am confued... who is on google?19:48
pradkEmilienM, we need to get shardy's patch in before this  .. https://review.openstack.org/#/c/383188/119:48
trozettrown: slagle is famous19:48
trownwell ya, obviously19:48
trozetslagle: so then I can just point the DIB_YUM_REPO_CONF to centos7 repos? it will work?19:48
slagletrozet: if those packages install on rhel, you could19:49
trozetslagle, trown: btw newton links are missing on trunk.rdoproject.org19:49
slagleor use internal repos (if there are any), and you're image build has access to them19:50
trowntrozet: indeed they are... we just cut master over to ocata this morning, will ask on #rdo where the source for that page comes from19:52
*** penick has quit IRC19:53
*** akuznetsov has joined #tripleo19:53
*** akuznetsov has quit IRC19:53
*** penick has joined #tripleo19:54
*** jcoufal_ has joined #tripleo20:00
*** jbennett_ has quit IRC20:02
*** jcoufal has quit IRC20:03
*** lblanchard has quit IRC20:04
*** jcoufal_ has quit IRC20:05
beaglesEmilienM: I was looking at the CI failures in https://review.openstack.org/#/c/382082/  and besides the bogosity with the testresources being included in neutron (wt.), I noticed messages like ct  5 15:48:48 localhost os-collect-config: #033[1;31mError: /Stage[main]/Neutron/Resources[neutron_config]: Failed to generate additional resources using 'generate': OpenStackConfig only support collecting instances20:06
beagleswhen a file path is hard coded#033[0m20:07
beaglesEmilienM: which is weird, because i think we fixed the puppet modules for mitaka20:07
beaglesat least I hope we did.. I remember doing it :)20:08
EmilienMbeagles: i'll check20:08
* beagles checks other CI runs.. maybe something particular to that job20:09
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/38343320:12
beaglesargh.20:16
beaglesEmilienM: that fix wasn't pulled into openstack-puppet-modules package20:16
EmilienMbeagles: what fix?20:17
beaglesEmilienM: it's a fix to puppet-openstacklib If27152863b9df5a28bf45e23ca989019666a958c20:17
* beagles goes to work the program to get it updated20:22
*** gfidente has quit IRC20:25
*** pkovar has joined #tripleo20:27
*** TSCHAK_ has joined #tripleo20:28
*** TSCHAK has joined #tripleo20:28
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform a booting test for our images  https://review.openstack.org/20463920:29
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail  https://review.openstack.org/38344520:30
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail  https://review.openstack.org/38344620:31
*** jayg is now known as jayg|g0n320:34
openstackgerritMerged openstack/tripleo-heat-templates: Re-enable ManageFirewall by default.  https://review.openstack.org/38186420:35
openstackgerritMerged openstack/instack-undercloud: Switch to use Puppet Openstack spec helper  https://review.openstack.org/38266220:35
openstackgerritMerged openstack/tripleo-heat-templates: Fix OpendaylightApiNetwork key naming  https://review.openstack.org/38292620:35
*** gfidente has joined #tripleo20:39
*** gfidente has quit IRC20:39
*** gfidente has joined #tripleo20:39
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Re-enable ManageFirewall by default.  https://review.openstack.org/38344820:40
EmilienMdprince: backport ^20:40
tbarrongfidente: marios mwhahaha beagles updated https://review.openstack.org/#/c/382386/9 with the latest deploy results - looks good20:41
tbarrongfidente: marios mwhahaha beagles EmilienM ^^^ I don't plan to do more deploys of this one unless it's requested for some reason20:42
*** bfournie has quit IRC20:42
tbarrongfidente: marios mwhahaha beagles EmilienM IMO when this passes jenkins it should merge and we should do the stable/newton backport20:43
beaglesdprince: more worker count bugs.. so nice20:43
EmilienMagain manilla?20:43
EmilienMit never ends :(20:43
tbarronEmilienM: :(20:43
*** abregman has joined #tripleo20:44
tbarronEmilienM: I don't have any more up my sleeve, and I want to get CI or at least scenario going for this (with cephfs backend) in O20:44
*** mcornea has quit IRC20:44
EmilienMtbarron: have you start to push on it?20:44
tbarronEmilienM: there's nothing like being a human impersonating CI to make one a believer in CI20:44
tbarronEmilienM: well, I'm dev, not QE: I just push a little bit :)20:45
EmilienMmwhahaha: see https://bugs.launchpad.net/tripleo/+bug/163113320:45
openstackLaunchpad bug 1631133 in OpenStack Compute (nova) "nova's osapi_compute_workers is not valid (must be greater than 1)" [High,In progress] - Assigned to Dan Prince (dan-prince)20:45
tbarronEmilienM: or do you mean, push *for* CI20:45
tbarronEmilienM: that I am doing20:46
EmilienMmwhahaha: isn't it something we're fixing or fixed?20:46
mwhahahakinda20:46
EmilienMmwhahaha: the worker thing :D20:46
mwhahahathe problem is there that we're defauling to 0 in tht20:46
mwhahahawhich completely by passes any puppet work we do around this20:46
*** nyechiel_ has quit IRC20:46
mwhahahaso we still would like to allow an end user to override it i would assume, but we shouldn't be using 0 in tht20:47
EmilienMtbarron: feel free to add manila to one of our CI scenarios.20:47
EmilienMmwhahaha: we should default to '' right?20:47
beaglesnova used to do something specific when set to 0, so we left it alone20:47
beaglesthey changed it, breaking all20:47
mwhahahaEmilienM: yea but it's a number in THT20:47
openstackgerritgreghaynes proposed openstack/diskimage-builder: Fix developer guide TOC  https://review.openstack.org/38345420:48
*** paramite has quit IRC20:48
mwhahahaI looked, techincally i think we should default to $::os_workers but not sure how to do that in THT20:48
beaglesEmilienM, mwhahaha, in retrospect, I should've changed all of them, not just the ones that werent' working at the time20:48
tbarronEmilienM: ok, first step will be to get the ceph deploy to deploy MDS on controllers (or whereever composable roles put them).  Shouldn't be a big deal but was not a newton deliverable.20:48
mwhahahabeagles: it's all good, the defaults for all the services are so inconsistent20:49
*** lucasagomes is now known as lucas-afk20:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Stop forcing oslosphinx for docs  https://review.openstack.org/38345620:54
openstackgerritgreghaynes proposed openstack/diskimage-builder: Use globaltoc in docs for sidebar  https://review.openstack.org/38345720:55
*** pkovar has quit IRC20:55
openstackgerritgreghaynes proposed openstack/diskimage-builder: Remove copyright from docs / toc  https://review.openstack.org/38345920:59
mwhahahain tht is there a way to  pass through to leverage a puppet fact or something20:59
mwhahahacause it looks like all of our workers are set to 0 which is probably going to be problematic21:00
openstackgerritgreghaynes proposed openstack/diskimage-builder: Rename devloper documentation to developer guide  https://review.openstack.org/38346121:02
dprincebeagles: I pushed a fix to nova for that. I always love how projects allow default config settings to get changed, and then they shoot the guy in the foot who points out the breakage https://review.openstack.org/#/c/383440/21:02
*** ayoung has quit IRC21:02
openstackgerritMerged openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38302921:05
*** shardy_afk is now known as shardy21:05
*** penick has quit IRC21:05
shardymatbu: Hey I've been working on a revised patch but it's not quite complete21:05
shardyI can either push what I have or finish it first thing tomorrow21:05
openstackgerritgreghaynes proposed openstack/diskimage-builder: Fix formatting for supported distros in docs  https://review.openstack.org/38346421:06
*** penick has joined #tripleo21:07
*** trown is now known as trown|outtypewww21:11
*** fultonj_ has joined #tripleo21:12
openstackgerritSteven Hardy proposed openstack/python-tripleoclient: Download templates from swift before processing with heatclient  https://review.openstack.org/37954721:13
*** maticue has quit IRC21:14
EmilienMshardy: I'll request rc3 Friday 9pm for you21:16
shardymatbu: ^^ pushed my WIP patch, we can chat again tomorrow about it21:16
EmilienMwe really need to get this bug fixed before21:16
shardyEmilienM: ack, thanks21:16
shardyEmilienM: Yeah, I'll prioritize it tomorrow morning21:16
EmilienMthx21:16
beaglesdprince:  were you going to do the nova workers patches for tht? If not, I'll take it. I'll also go through the rest of the workers-as-a-number templates21:17
fultonj_shardy: EmilienM : you had +2'd https://review.openstack.org/#/c/382132/ before (only the commit message changed). CI just had slight issue. do you need a recheck before re-reviewing?21:18
*** tiswanso has quit IRC21:18
dprincebeagles: the NovaWorkers patch is already a number though right?21:18
beaglesdprince: yup21:18
EmilienMfultonj_: doing recheck21:19
beaglesdprince: umm.. actually not the patch, the current template21:19
dprincebeagles: so do we need to change t-h-t?21:19
fultonj_EmilienM: thanks21:19
beaglesdprince: fwict, with your fix to nova we don't need it right now. We might to consider changing tht to just not set anything for ocata21:20
beagles"might to consider" = "might want to consider"21:20
dprincebeagles: Right. I suppose I like setting it to 0. That way we keep the integer validation in Heat templates21:20
dprincebeagles: we can debate this I suppose. For now though I was just planning on pushing to fix the Nova regression. This is a regression IMO21:21
mwhahahaI was just looking at teh THT and thinking of switching it to a string with validation for a number or ''21:21
beaglesdprince: agreed21:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable firewalling by default on compute nodes  https://review.openstack.org/38347721:21
EmilienMdprince: last one ^21:21
beaglesmwhahaha: yeah..that'd be cool21:21
*** dprince has quit IRC21:21
*** shardy has quit IRC21:22
beaglesmwhahaha: too bad there isn't a way to make a parameter truly optional in heat but I presume that was a decision made for the best of reasons21:23
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Switch Nova workers to '' by default  https://review.openstack.org/38347821:24
mwhahahagoing to see if -^ works21:25
mwhahahanova isn't the only one we set 0 for the default for so if this works we'll probably need to update the other ones as well21:26
openstackgerritgreghaynes proposed openstack/diskimage-builder: Generate ssh-hostkeys on boot for ironic agent  https://review.openstack.org/38348121:30
beaglesmwhahaha: yup21:30
beaglesmwhahaha: although for some services '0' means don't spawn any workers, i.e. do everything in a single process. Saves memory etc.21:31
mwhahahaof course it does because why would we want a consistent experience across services21:31
beaglesmwhahaha: but I guess 0 is a number ;)21:31
beaglesmwhahaha: heh21:32
mwhahahathe installer life21:32
beaglesmwhahaha: if we want it 0 in the gate, we can always set it that way as long as the validation allows21:32
beagless/gate/ci/21:32
mwhahahahttp://i.imgur.com/F6b0h2v.gif <- the installer life21:33
beagleslol21:34
ccamacho1mwhahaha++21:35
EmilienMmwhahaha: lol21:35
openstackgerritIan Main proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles  https://review.openstack.org/33065921:38
*** cdearborn has quit IRC21:38
*** pradk has quit IRC21:39
*** mbozhenko has joined #tripleo21:42
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Switch Nova workers to '' by default  https://review.openstack.org/38347821:43
tbarronEmilienM: gfidente mwhahaha marios beagles Jenkins likes https://review.openstack.org/#/c/382386/ - workflow?21:44
tbarronEmilienM: also, do we wait for merge in master in this project before cherry-pick to stable/newton ?21:45
EmilienMyes21:46
EmilienMwe wait21:46
tbarronEmilienM: ack21:46
*** mbozhenko has quit IRC21:47
*** rcernin has joined #tripleo21:47
*** bnemec has quit IRC21:47
openstackgerritgreghaynes proposed openstack/diskimage-builder: Add bugs link to docs  https://review.openstack.org/38348821:49
openstackgerritgreghaynes proposed openstack/diskimage-builder: Add low-hanging-fruit bug tag to docs  https://review.openstack.org/38348921:51
ccamacho1EmilienM man quick question .. Im going to sleep, I have:21:58
ccamacho1to be merged: (https://review.openstack.org/#/c/382911/ https://review.openstack.org/#/c/382217/)21:58
ccamacho1pending backport after master is merged: (https://review.openstack.org/#/c/382973/)21:58
ccamacho1to be backported: (https://review.openstack.org/#/c/382217/)21:58
ccamacho1Do you think you can check them, before GA? I'm falling asleep21:58
*** penick has quit IRC21:59
*** rcernin has quit IRC21:59
*** rcernin has joined #tripleo21:59
*** absubram has quit IRC22:01
slagleccamacho1: i can check back on them later tonight22:01
ccamacho1hey slagle! thanks man :)22:02
ccamacho1Its that im like super asleep right now22:02
ccamacho1:)22:02
ccamacho1ttl :)22:02
*** penick has joined #tripleo22:04
openstackgerritMerged openstack/tripleo-common: Add the J2 exclude file exception (When file not found)  https://review.openstack.org/38291122:04
*** assassin has quit IRC22:07
*** akshai has quit IRC22:07
*** ubijtsa has joined #tripleo22:08
*** ubijtsa is now known as Guest2133122:08
openstackgerritBrent Eagles proposed openstack/tripleo-common: Add the J2 exclude file exception (When file not found)  https://review.openstack.org/38297322:11
beaglesslagle: updated one of ccamacho1's backports.. though it looks like he still might be around ...22:12
ccamacho1beagles just shutting down the laptop22:13
ccamacho1:)22:13
*** ccamacho1 has quit IRC22:13
beagles15423922:14
beaglesugh22:14
*** tiswanso has joined #tripleo22:16
*** fultonj_ has quit IRC22:17
*** abregman has quit IRC22:19
*** ayoung has joined #tripleo22:21
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates: Update OpenContrail loadbalancer plugin value  https://review.openstack.org/38349722:22
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates: Update OpenContrail loadbalancer plugin value  https://review.openstack.org/38349822:23
*** morazi has quit IRC22:24
mwhahahabeagles: nope looks like you can't mix numbers and strings in heat templates22:27
beaglesmwhahaha: crap22:27
mwhahahau'message': u"Failed to run action [action_ex_id=be268ff5-e0e1-4a21-9287-444d9e6a83cf, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'container': u'overcloud', u'timeout': 80}']\n ERROR: Failed to validate: : resources.ControllerServiceChain: : Failed to validate nested template: Failed to validate:22:27
mwhahaharesources[24]: Parameter 'NovaWorkers' is invalid: Invalid default 1 (expected string or buffer)",22:27
mwhahahayay artificial complexities22:28
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Switch Nova workers to '' by default  https://review.openstack.org/38347822:30
*** rhallisey has quit IRC22:40
*** rhallisey has joined #tripleo22:40
*** rhallisey has joined #tripleo22:41
*** rhallisey has quit IRC22:41
*** apetrich has quit IRC23:03
*** apetrich has joined #tripleo23:04
*** tiswanso has quit IRC23:04
*** ayoung has quit IRC23:05
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: Full quickstart gate run on OVB  https://review.openstack.org/38109423:06
*** saneax-_-|AFK is now known as saneax23:12
*** jcoufal has joined #tripleo23:13
*** abehl has quit IRC23:18
*** anshul has joined #tripleo23:19
*** anshul has quit IRC23:23
openstackgerritMerged openstack/puppet-tripleo: Enable usage of "short names" for Galera cluster  https://review.openstack.org/38288323:24
openstackgerritMerged openstack/tripleo-heat-templates: Add Select per-network hostnames for service_node_names to role.role.j2.yaml  https://review.openstack.org/38221723:24
openstackgerritMerged openstack/tripleo-common: Add the KeystoneCredential0 and KeystoneCredential1 parameters  https://review.openstack.org/38116923:24
openstackgerritMerged openstack/instack-undercloud: Revert "Stop pinning Glance API version on stackrc"  https://review.openstack.org/38309123:24
*** thrash is now known as thrash|g0ne23:33
*** rcernin has quit IRC23:36
*** mbozhenko has joined #tripleo23:42
*** mbozhenko has quit IRC23:47
*** ayoung has joined #tripleo23:49
*** dtantsur|afk has quit IRC23:53
*** jcoufal has quit IRC23:54
*** pcaruana has quit IRC23:55
*** pblaho has quit IRC23:55
*** jcoufal has joined #tripleo23:57
*** bana_k has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!