Monday, 2016-12-19

« Sunday, 2016-12-18 Index Tuesday, 2016-12-20 »
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968800:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974200:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]00:10
*** dmacpher has joined #tripleo00:21
*** fragatina has joined #tripleo00:40
*** Hazelesque has quit IRC00:42
*** limao has joined #tripleo00:44
*** bana_k has joined #tripleo00:48
*** bana_k has quit IRC00:48
*** bana_k has joined #tripleo00:50
*** Hazelesque has joined #tripleo00:51
openstackgerritTristan Cacqueray proposed openstack/diskimage-builder: Add squashfs output image format  https://review.openstack.org/41224300:55
*** bana_k has quit IRC01:08
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968801:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974201:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)01:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]01:10
*** saneax-_-|AFK is now known as saneax01:18
*** nyechiel has quit IRC01:37
*** bana_k has joined #tripleo01:57
*** bana_k has quit IRC02:06
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968802:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974202:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)02:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]02:10
*** bana_k has joined #tripleo02:12
*** fragatina has quit IRC02:17
*** flepied has quit IRC02:24
*** bfournie has quit IRC02:27
*** jeckersb_gone is now known as jeckersb02:29
*** saneax is now known as saneax-_-|AFK02:45
*** saneax-_-|AFK is now known as saneax02:58
openstackgerritMerged openstack/diskimage-builder: Install dracut-generic-config package  https://review.openstack.org/41154002:58
*** michapma_alt has quit IRC03:05
*** michchap_alt_ has joined #tripleo03:05
*** arxcruz has quit IRC03:06
*** arxcruz has joined #tripleo03:06
*** ctayal has joined #tripleo03:07
*** saneax is now known as saneax-_-|AFK03:09
*** CaptTofu has quit IRC03:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968803:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974203:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]03:10
*** Nakato has quit IRC03:10
*** Nakato has joined #tripleo03:10
*** CaptTofu has joined #tripleo03:12
*** akuznetsov has joined #tripleo03:16
*** ctayal has quit IRC03:21
*** trozet_ has quit IRC03:22
*** trozet_ has joined #tripleo03:22
*** matbu has quit IRC03:22
*** udesale has joined #tripleo03:22
*** matbu has joined #tripleo03:22
*** mwhahaha has quit IRC03:26
*** mwhahaha has joined #tripleo03:29
*** bana_k has quit IRC03:53
*** bana_k has joined #tripleo03:54
*** sudipto has joined #tripleo03:56
*** coolsvap has joined #tripleo03:59
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968804:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974204:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)04:10
openstackgerritMerged openstack/diskimage-builder: FIx the DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH variable name in README file  https://review.openstack.org/41204004:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]04:10
*** Goneri has quit IRC04:20
*** limao has quit IRC04:28
*** prateek has joined #tripleo04:49
*** limao has joined #tripleo04:51
openstackgerritMerged openstack/diskimage-builder: Fix bootloader element on ppc  https://review.openstack.org/41154104:55
*** akuznetsov has quit IRC04:56
*** noama has joined #tripleo05:01
openstackgerritMerged openstack/instack: Changed author and author-email  https://review.openstack.org/40348905:02
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968805:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974205:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)05:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]05:10
*** akuznetsov has joined #tripleo05:12
*** newmember has joined #tripleo05:13
*** newmember has quit IRC05:14
*** newmember has joined #tripleo05:14
*** akuznetsov has quit IRC05:14
*** akuznetsov has joined #tripleo05:16
*** akuznetsov has quit IRC05:16
*** sudipto has quit IRC05:18
*** akuznetsov has joined #tripleo05:24
*** Vijayendra has joined #tripleo05:30
openstackgerritMerged openstack/instack-undercloud: Switch mistral to use authtoken configuration  https://review.openstack.org/40947805:32
*** akuznetsov has quit IRC05:33
*** saneax-_-|AFK is now known as saneax05:35
*** sudipto has joined #tripleo05:47
*** masco has joined #tripleo05:47
*** bkopilov has joined #tripleo06:04
*** anshul has joined #tripleo06:04
*** apetrich has quit IRC06:05
*** jaosorior has joined #tripleo06:06
*** anshul has quit IRC06:09
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968806:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974206:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]06:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]06:10
*** abregman has joined #tripleo06:12
*** apetrich has joined #tripleo06:13
*** bkopilov has quit IRC06:17
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: WIP: Configure Kernel Args and Tuned and then reboot for Compute  https://review.openstack.org/41179706:20
*** dmacpher has quit IRC06:25
*** abregman has quit IRC06:31
*** pmannidi has quit IRC06:32
*** bana_k has quit IRC06:33
*** dsneddon has quit IRC06:34
*** dsneddon has joined #tripleo06:35
*** abregman has joined #tripleo06:36
*** dsneddon has quit IRC06:39
*** dsneddon has joined #tripleo06:39
*** iranzo has joined #tripleo06:47
*** iranzo has joined #tripleo06:47
*** pmannidi has joined #tripleo06:49
*** oshvartz has quit IRC06:49
jaosoriord0ugal: ping06:53
jaosoriord0ugal: any idea what's up with this error http://logs.openstack.org/92/408992/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/b55fdf1/console.html#_2016-12-19_06_27_31_502513 ? this seems to be happening on nonha ovb jobs :/06:53
jaosoriord0ugal: I remember we used to have that same error before when the db-sync was failing for some actions. So it wouldn't populate all the actions and it would silently fail. so the baremetal actions were empty06:55
jaosorioruhm...actually, that was the issue with getting the python-ironic-inspector client to load, which is what was used to populate those actions06:56
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates: Split OVN northd and ml2 plugin  https://review.openstack.org/38794007:00
*** pmannidi has quit IRC07:03
*** ealcaniz has joined #tripleo07:04
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates: OVN plugin configuration fixes  https://review.openstack.org/39767407:04
*** tesseract has joined #tripleo07:04
*** tesseract is now known as Guest3325407:05
*** pmannidi has joined #tripleo07:05
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates: OVN bridge mappings for tripleo  https://review.openstack.org/40229307:06
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968807:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974207:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]07:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]07:10
*** fragatina has joined #tripleo07:18
openstackgerritTristan Cacqueray proposed openstack/diskimage-builder: Add squashfs output image format  https://review.openstack.org/41224307:21
openstackgerritNoam Angel proposed openstack/diskimage-builder: set default DIB_PYTHON_VERSION=2 for rhel7  https://review.openstack.org/41232407:27
*** apetrich has quit IRC07:27
*** iranzo has quit IRC07:28
*** iranzo has joined #tripleo07:28
*** iranzo has joined #tripleo07:28
*** apetrich has joined #tripleo07:30
openstackgerritNoam Angel proposed openstack/diskimage-builder: set default DIB_PYTHON_VERSION=2 for rhel7  https://review.openstack.org/41232407:31
*** pcaruana has joined #tripleo07:33
*** shinobu_ has quit IRC07:34
*** shinobu|afk has joined #tripleo07:34
openstackgerritMerged openstack/tripleo-heat-templates: Enable SECURE_PROXY_SSL_HEADER option for horizon  https://review.openstack.org/40899207:41
*** lmiccini has joined #tripleo07:41
openstackgerritMerged openstack/tripleo-heat-templates: Introduce role-specific nova-server-metadata  https://review.openstack.org/41054507:45
*** b00tcat has joined #tripleo07:46
*** pgadiya has joined #tripleo07:48
*** tobias_fiberdata has joined #tripleo07:50
openstackgerritAlejandro Andreu proposed openstack/tripleo-puppet-elements: Install Curator from RPM in MidoNet element  https://review.openstack.org/41030107:52
openstackgerritAlejandro Andreu proposed openstack/tripleo-puppet-elements: Install Curator from RPM in MidoNet element  https://review.openstack.org/41030107:56
*** jaosorior has quit IRC07:58
*** shardy has joined #tripleo08:02
b00tcatHi, can someone move this tripleo-puppet-elements patch forward? https://review.openstack.org/#/c/39497808:08
shardyb00tcat: done08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968808:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974208:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]08:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]08:10
shardyand g'morning all :)08:10
*** bvandenh has joined #tripleo08:10
b00tcatthanks shardy08:10
shardywoop, upgrades CI job is green :) https://review.openstack.org/#/c/404831/08:11
*** flepied has joined #tripleo08:12
*** oshvartz has joined #tripleo08:12
*** yprokule has joined #tripleo08:14
*** rasca has joined #tripleo08:16
matbushardy: morning08:17
shardyhey matbu08:17
matbushardy: yep green, but i'm a bit worried about it :)08:17
matbushardy: i was looking at the http://logs.openstack.org/31/404831/37/experimental/gate-tripleo-ci-centos-7-multinode-upgrades-nv/e6c47df/logs/var/log/bootstrap-subnodes-upgrade.txt.gz08:17
matbushardy: and it looks like we still use ocata repo on the nodes08:18
*** mcornea has joined #tripleo08:18
shardymatbu: ouch, yeah there must still be something wrong with the bootstrap08:19
shardyugh08:19
shardyOk I'll try to run that locally on a VM to debug08:19
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata  https://review.openstack.org/40483108:19
shardyunless you're already doing that08:19
matbushardy: nop, but i'll, but i just add some debug info in the repo-setup08:20
matbushardy: i think this is just a matter of passing this STABLE_RELEASE08:20
*** tremble has joined #tripleo08:23
*** dsariel has joined #tripleo08:24
*** jaosorior has joined #tripleo08:24
jaosoriorshardy: hey, could you take another look at this https://review.openstack.org/#/c/411339/ ?08:26
*** remix_tj has joined #tripleo08:29
*** jpena|off is now known as jpena08:32
*** anton has quit IRC08:36
shardyjaosorior: looks good, just one minor comment added08:36
*** anton has joined #tripleo08:39
jaosoriorshardy: hey! That's a really good idea08:40
openstackgerritMerged openstack/diskimage-builder: Switch to openSUSE Leap 42.2 release by default  https://review.openstack.org/40907108:40
skramajashardy: i have added the scripts on top of your example. i have modified it, so that the boot parameter are customizable per role like - ComputeOvsDpdkKernelArgs, ComputeSriovKernelArgs - https://review.openstack.org/#/c/411797/08:41
*** mrunge has quit IRC08:41
skramajashardy: is the approch OK?08:42
shardyjaosorior: cool, it's a minor thing, but makes the hook really pretty much zero cost08:42
jaosoriorshardy: only one thing08:43
jaosoriorshardy: when we pass the metadata_settings coming from the role_data08:43
jaosoriorshardy: if I set the parameter as json, wouldn't it be an issue with the value we pass is null?08:43
*** dsneddon has quit IRC08:44
*** dsneddon has joined #tripleo08:45
*** jaosorior has quit IRC08:45
*** mrunge has joined #tripleo08:46
shardyjaosorior: Hrm, good point - I think it should work because we don't do value validation of parameters until the stack is created, but if it proves a problem then I guess leave it where it is08:46
*** jaosorior has joined #tripleo08:46
shardyjaosorior: Hrm, good point - I think it should work because we don't do value validation of parameters until the stack is created, but if it proves a problem then I guess leave it where it is08:46
shardy(repeated since I think you just dropped)08:47
jaosoriorshardy: I'll try it out anyway08:47
jaosoriorshardy: the other option is that I pass the role_data itself08:47
jaosoriorthat shouldn't be null08:47
shardyjaosorior: yeah that should work - if it proves problematic then I'm fine leaving it, just that I know yaql queries are relatively expensive08:48
shardythis is only one per-role tho, so not too bad08:48
*** noama has quit IRC08:48
shardyhttp://docs.openstack.org/developer/heat/template_guide/hot_spec.html#filter08:49
shardyI've been meaning to try that, which therve added as a less expensive way for us to filter lists08:49
*** jpich has joined #tripleo08:50
*** flepied has quit IRC08:50
yolandahi, anyone has seen error 2016-12-18 21:59:46.952 15325 ERROR paramiko.transport [-] AttributeError: 'module' object has no attribute 'Cryptography_EVP_MD_CTX_new' on ironic recently?08:52
yolandai get08:52
yolandaFailed to validate power driver interface for node 02f1ea4c-16b7-4e09-80af-8535bf00ede0. Error: SSH connection cannot be established: Failed to establish SSH connection to host 192.168.23.1.08:52
yolandaand that error08:52
*** sshnaidm|afk is now known as sshnaidm08:52
yolandaseems something to do with latest cryptography version and paramiko...08:52
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133908:53
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134008:53
jaosoriorshardy: that's my current try ^^ is that what you were talking about?08:53
jaosoriorshardy: gonna try that in my deployment in a bit08:53
openstackgerritMerged openstack/tripleo-heat-templates: Set rabbitmq's port and IP via the config file and not the env file  https://review.openstack.org/41072208:54
shardyjaosorior: +1 lgtm08:54
shardyskramaja: That looks OK, I made a few minor comments08:55
jaosoriorshardy: great, let me try it out08:55
*** fzdarsky has joined #tripleo08:55
sshnaidmyolanda, https://bugs.launchpad.net/tripleo/+bug/1650374 - is it?08:55
openstackLaunchpad bug 1650374 in tripleo "[instack-undercloud] package upgrade should happen before service configuration" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)08:55
*** ealcaniz has quit IRC08:55
shardyskramaja: the only thing to be aware of is we've got a kernel profile in puppet, so while using ansible is fine I guess, it's not getting captured as part of the global system state08:55
*** jtomasek has joined #tripleo08:57
skramajashardy: can you decode "global system state"?08:57
sshnaidmyolanda, try to restart ironic-conductor08:58
jaosoriorshardy: awww it breaks: The Referenced Attribute (ServiceServerMetadataHook metadata) is incorrect.09:01
shardyskramaja: we apply all configuration via puppet, which builds a catalog that defines the system state09:03
shardyskramaja: by using ansible, there is a risk that puppet then later reconfigures things differently, but the same is true if you use scripts etc09:04
*** yamahata has quit IRC09:04
shardyskramaja: if you wanted to have it managed by puppet, it would instead go here: https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/kernel.pp09:05
*** ohamada has joined #tripleo09:06
*** hewbrocca_afk is now known as hewbrocca09:07
*** rasca has quit IRC09:07
shardyskramaja: you'd do a puppet apply in your template of e.g ::tripleo::profile::base::kernel, then later we'd re-apply the same manifest with updated hieradata from https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/kernel.yaml#L5909:07
shardyskramaja: I'm not blocking on it, just pointing out that you'd get some additional validation for free with that approach09:08
shardyand things might work more consistently e.g when upgrades or other reconfigurations happen09:08
hewbroccaGood morning everyone!09:09
shardy'morning hewbrocca!09:09
yolandasshnaidm, may be that yes09:09
yolandalet me try a restart09:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968809:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974209:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]09:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]09:10
d0ugalooh, mistral error09:10
b00tcatshardy: is this something that can be solved with a recheck? or is there something in CI? http://logs.openstack.org/78/394978/3/gate/gate-tripleo-ci-centos-7-nonha-multinode/fe9c7dd/console.html09:10
skramajashardy: i am not very comfortable in realizing https://review.openstack.org/#/c/411797/3/extraconfig/pre_network/ansible-kernel-args.ansible@47 block in puppet, with my puppet knowledge.. which left with bash/python/ansible09:11
skramajashardy: i felt ansible was cleaner that bash.. thats the only reason for it.. if it has drawbacks, i will go back to scripts..09:11
jaosoriorb00tcat: what's the patch where you saw that?09:12
shardyskramaja: why would you go back to scripts?  I just said that has the same problem09:12
b00tcatjaosorior: https://review.openstack.org/#/c/394978/ it's a very trivial change09:12
skramajashardy: oopss.. sorry missed that..09:13
shardyskramaja: if you're uncomfortable doing it in puppet then I guess stick with ansible, but at least you are aware that you might get broken on upgrades etc now09:14
jaosoriorb00tcat: try rechecking, the error seems very unrelated... and it's pretty strange, at some point it fails cause it can't contact keystone09:14
shardyI suspect what you're doing would be pretty simple to do in puppet, but I've not got time to rewrite it for you atm09:14
shardyb00tcat: Yeah there were issues with the undercloud install which seem resolved since last week09:15
shardyso I'd recheck in this case09:15
yolandahi, can i get reviews for https://review.openstack.org/406895 ?09:16
*** flepied has joined #tripleo09:17
*** b00tcat has quit IRC09:18
*** gfidente has joined #tripleo09:18
*** gfidente has quit IRC09:18
*** gfidente has joined #tripleo09:18
skramajashardy: thanks.. i will work on puppet as second step..09:19
*** jtomasek has quit IRC09:19
*** pgadiya has quit IRC09:19
*** lucas-afk is now known as lucasagomes09:26
*** dbecker has joined #tripleo09:27
*** pgadiya has joined #tripleo09:31
*** shinobu_ has joined #tripleo09:38
*** akrivoka has joined #tripleo09:44
*** athomas has joined #tripleo09:46
jaosoriorshardy: do you know what this means  Unknown function "#property#type" as a yaql error09:48
shardyjaosorior: what does the query look like?09:51
*** bkopilov has joined #tripleo09:52
jaosoriorshardy: oh, I think I got it09:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134009:56
jaosoriorshardy: that commit had an error ^09:56
*** sshnaidm has quit IRC09:56
jaosoriorshardy: so, yeah! seems to work! ended up using a Heat::Value in between where I filtered the nulls and duplicates09:58
jaosoriorshardy: let me just add it the hook to the README and that's done09:59
*** chem has joined #tripleo10:00
*** zoliXXL is now known as zoli10:01
*** flepied has left #tripleo10:02
*** flepied has joined #tripleo10:02
*** chem has quit IRC10:03
*** chem has joined #tripleo10:03
*** milan has joined #tripleo10:03
zoligood morning10:05
jaosoriorshardy: only thing I'm not sure is if it would be more useful for people to just receive the whole role_data instead of just the metadata_settings10:06
shardyjaosorior: Yeah maybe it would10:08
*** bkopilov_ has joined #tripleo10:08
jaosoriorshardy: I can do the change10:08
jaosoriorshould be pretty trivial now that the yaql is only within the stack10:09
shardyI guess you might want to e.g modify the metadata based on the enabled services or something, which would be easier if the roles_data containing the service_names was available10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968810:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974210:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]10:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]10:10
jaosoriorshardy: I'll do that10:18
*** paramite_ has joined #tripleo10:19
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133910:19
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134010:19
jaosoriorshardy: what do you think? ^^10:20
*** paramite has joined #tripleo10:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134010:21
shinobu_whom should i talk to regarding to ceph cluster deployment?10:32
*** limao has quit IRC10:33
shardyshinobu_: Hi, what is the question re ceph?10:34
*** limao has joined #tripleo10:34
shardyshinobu_: gfidente may be able to help, or direct you to a good contact10:34
*** leanderthal|afk is now known as leanderthal10:38
*** limao has quit IRC10:38
*** sshnaidm has joined #tripleo10:41
*** pblaho has quit IRC10:41
shinobu_shardy: my original question is how we separately build / scale ceph cluster after deploying overcloud with ceph-storage 3 or similar10:43
*** derekh has joined #tripleo10:43
shardyshinobu_: you should be able to re-run the deploy command with an increased CephStorageCount to add more storage nodes10:46
shardyshinobu_: there are some workarounds required when adding Swift nodes, but I'm not aware of workarounds required for ceph10:47
shardygfidente: ^^ got any more details on ceph scale-out?10:47
openstackgerritSaravanan KR proposed openstack/puppet-tripleo: Add support to changing the Rabbitmq password on update  https://review.openstack.org/40206110:47
openstackgerritMerged openstack/puppet-tripleo: Fix a typo in haproxy.pp  https://review.openstack.org/37995810:49
*** panda|off is now known as panda10:49
jaosoriorshardy: how is the ouput of role_data formed? the input I'm getting is something like https://paste.fedoraproject.org/509617/14821448/10:54
shardyjaosorior: it's just a list of maps, one per service template10:55
jaosoriorshardy: any idea where those None are coming from?10:55
shardyjaosorior: yes, the services which are mapped to OS::Heat::None10:56
shardythat's why the yaql queries filter null values10:56
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Return proper exit code on plan deletion failure  https://review.openstack.org/41069710:56
*** udesale has quit IRC10:58
derekhHave we any idea whats up with introspection on the nonha job?10:59
jaosoriord0ugal: did you get the messages I sent you earlier?10:59
d0ugaljaosorior: no, I don't think so... sorry /me scrolls up11:00
*** sshnaidm has quit IRC11:00
d0ugaljaosorior: no, I don't really understand it.11:01
d0ugaljaosorior: rlandy was hitting something similar in the downsteam CI11:01
jaosoriord0ugal: anybody has been able to reproduce it?11:01
d0ugaljaosorior: it was worked around by adding an extra call to the Mistral db populate command11:01
jaosoriord0ugal: we really need the logs of that command in CI :/11:02
d0ugaljaosorior: our only theory was that the actions were being loaded before inspector client was installed properly11:02
d0ugaljaosorior: what command?11:02
jaosoriordb-populate11:02
gfidenteshinobu_ you can only scale up storage nodes though, not monitors11:02
d0ugaljaosorior: yeah, that would be useful.11:02
*** pblaho has joined #tripleo11:04
*** amoralej is now known as amoralej|brb11:05
*** sshnaidm has joined #tripleo11:06
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968811:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974211:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]11:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]11:10
*** sudipto has quit IRC11:10
*** pblaho has quit IRC11:13
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Run upgrade steps before post-deploy config  https://review.openstack.org/40985711:16
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Move UpgradeInitCommand to role templates  https://review.openstack.org/41131011:16
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Add glance service support for composable upgrades  https://review.openstack.org/40524011:16
*** jkilpatr has quit IRC11:17
shardyhttps://review.openstack.org/#/c/408511 can probably land if someone wants to +A it11:20
*** ramishra has quit IRC11:22
*** ramishra_ has joined #tripleo11:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133911:23
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134011:23
jaosoriorshardy: is this better? ^^11:23
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970411:24
*** zoli is now known as zoli|lunch11:25
*** nyechiel has joined #tripleo11:30
*** jefrite has joined #tripleo11:32
EmilienMhello11:33
*** b00tcat has joined #tripleo11:34
*** b00tcat has quit IRC11:34
*** b00tcat has joined #tripleo11:34
jaosoriorEmilienM: hey dude, regarding the issues on CI. I think it's an issue with mistral's db-populate, for the ironic-inspector actions. Problem is, if there's an issue with an action, db-populate keeps running11:39
jaosoriorI think that's on purpose though, so one can optionally disable the actions of some client just by not having it installed or available.11:39
jaosorioranyway, I think we need to set this to true https://github.com/openstack/puppet-mistral/blob/master/manifests/db/sync.pp#L2811:40
d0ugalEmilienM: this looks like the same issue rlandy had that I pinged you about during bootcamp11:40
jaosoriorso we can get the logs always, and debug easier11:40
jaosoriorsince now we don't get the logs11:40
EmilienMjaosorior: you can override it in our hiera11:40
EmilienMjaosorior: https://github.com/openstack/puppet-mistral/blob/master/manifests/db/sync.pp#L28 is now the default in all puppet modules11:41
EmilienMI don't think we'll change it11:41
jaosoriorwell, in order to override it in hiera, we need to have it parametrized, don't we?11:41
EmilienMno11:41
EmilienMah11:41
jaosorioryep11:41
EmilienMwell, yeah it's better but not required11:42
EmilienMyou can collect the resource in the manifest and override the parameter11:42
EmilienMjaosorior: can you review https://review.openstack.org/#/c/411957/ please? also important to have11:42
jaosoriorEmilienM: I did review it, did you see the comment?11:42
EmilienMjaosorior: if you read the history in comments, we already replied to that11:42
jaosoriorah11:42
jaosoriorlet me check11:43
jaosoriorEmilienM: makes sense, done11:43
EmilienMjaosorior: cool. So what do you propose for the mistral problem (not the logs thing)11:44
EmilienMdo we need to have logs first? or have you tracked the issue?11:44
d0ugalThe logs might help.11:45
d0ugalThe issue is strange.11:45
jaosoriorEmilienM: we need logs first I think and I haven't properly tracked the issue. It's prettys trange, at least from the timings in the puppet logs, the resources seemt o be created in the correct order... so I'm quite confused11:45
EmilienMd0ugal: the thing with db-populate, is that mistral should log it in it's own logfile11:46
EmilienMd0ugal: outputing in the puppet catalog outside an error is not optimum imho11:46
d0ugalEmilienM: agreed11:46
EmilienMI can propose a patch in puppet-mistral and depends-on on instack-undercloud to have ovb-nonha job11:47
EmilienMbut we won't merge it11:47
*** hjensas has quit IRC11:47
EmilienMlet me do that11:47
*** sshnaidm is now known as sshnaidm|lnch11:47
jaosoriorthat makes sense11:47
openstackgerritLucas Alvares Gomes proposed openstack/python-tripleoclient: Install VirtualBMC in the overcloud-full image  https://review.openstack.org/41241611:50
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: debug -- do not merge  https://review.openstack.org/41241811:51
EmilienMd0ugal, jaosorior ^ let's see11:51
d0ugalEmilienM: Thanks.11:51
d0ugalI am trying to add logging in Mistral for this too11:52
*** pblaho has joined #tripleo11:53
*** lucasagomes is now known as lucas-hungry11:55
*** masco has quit IRC11:56
*** yolanda has quit IRC11:59
*** yolanda has joined #tripleo12:00
mariosdbecker: sry let me know when12:01
dbeckermarios, rdy sry12:03
mariosdbecker: coming12:04
*** jkilpatr has joined #tripleo12:04
*** fultonj has joined #tripleo12:05
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968812:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974212:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]12:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]12:10
*** jkilpatr has quit IRC12:11
*** dbecker has quit IRC12:15
*** shardy is now known as shardy_lunch12:15
*** dbecker has joined #tripleo12:15
*** amoralej|brb is now known as amoralej12:15
*** pkovar has joined #tripleo12:22
*** dprince has joined #tripleo12:22
dprincegood morning tripleo12:24
*** prateek has quit IRC12:24
hewbrocca<  good morning >12:25
hewbrocca ---------------12:25
hewbrocca    \   ,   .   ,12:25
hewbrocca     \  )-_"""_-(12:25
hewbrocca       ./ o\ /o \.12:25
hewbrocca      . \__/ \__/ .12:25
hewbrocca      ...   V   ...12:25
hewbrocca      ... - - - ...12:25
hewbrocca       .   - -   .12:25
hewbrocca        `-.....-´12:25
hewbrocca12:25
*** jkilpatr has joined #tripleo12:25
*** sshnaidm|lnch is now known as sshnaidm12:25
*** dmacpher has joined #tripleo12:27
dprincehewbrocca: :)12:30
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add TLS proxy resource  https://review.openstack.org/40978212:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Use TLS proxy for Glance API's internal TLS  https://review.openstack.org/40853612:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Clean TLS proxy-related setup for glance api profile  https://review.openstack.org/40856512:31
*** bfournie has joined #tripleo12:34
*** pgadiya has quit IRC12:38
honzahewbrocca: this might be relevant to your interests https://github.com/honza/owlsay12:39
jaosoriorsshnaidm: hey dude, could you review this https://review.openstack.org/#/c/391738/ ?12:41
sshnaidmjaosorior, sure12:42
*** bkopilov has quit IRC12:42
*** bkopilov_ has quit IRC12:42
EmilienMjaosorior: https://review.openstack.org/#/c/391738/70/heat-templates/freeipa_server.yaml12:43
EmilienMI would rather try to see it in THT12:43
EmilienMso we can version it12:43
EmilienMlike we do for scenarios12:43
jaosoriorEmilienM: sure12:44
*** flaper87 has joined #tripleo12:45
*** panda is now known as panda|afk12:48
gfidenteEmilienM can you check this https://review.openstack.org/#/c/411983/ ?12:48
gfidenteit's puppet-nova for the qemu limits12:48
EmilienMgfidente: sure thing, right now12:49
gfidenteEmilienM :)12:49
gfidentethanks12:49
gfidente:)12:49
gfidentehonza https://github.com/honza/owlsay :)12:50
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras: Build role: support using sources from directory  https://review.openstack.org/40536912:50
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add FreeIPA server installation template  https://review.openstack.org/41244412:52
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction  https://review.openstack.org/39173812:53
jaosoriorEmilienM: done ^^12:53
EmilienMjaosorior: excellent12:53
*** pgadiya has joined #tripleo12:53
EmilienMjaosorior: would you see a scenario for that? or do you prefer to re-use ovb?12:53
EmilienMjaosorior: I see you're creating a new jobname?12:54
matbushardy_lunch: srly, i see why this stable_release is unset for the --repo-setup here : http://logs.openstack.org/31/404831/38/experimental/gate-tripleo-ci-centos-7-multinode-upgrades-nv/d86b491/logs/var/log/bootstrap-subnodes-upgrade.txt.gz12:54
matbushardy_lunch: i tried the ssh opt it should works12:54
EmilienMjaosorior: why having ovb job? is it really worth it to use baremetal resources just to test SSL stuff?12:54
jaosoriorEmilienM: I'm planning on doing a scenario specifically for this12:54
matbuEmilienM: maybe you have an idea on the link ^12:55
jaosoriorwell, at the time I pushed the new job we only had 2 node multinode jobs (undercloud and all-in-one overcloud)12:55
jaosoriorsince I needed a dedicated node for FreeIPA I used ovb12:55
EmilienMjaosorior: right, but we can adjust now12:55
EmilienMjaosorior: oh, you need a dedicated node?12:55
EmilienMcan't it be on subnode-1 (undercloud)?12:56
jaosoriorunfortunately no12:56
jaosoriorunless I remove swift for the undercloud12:56
jaosoriorthen maybe we could use the undercloud for FreeIPA as well12:56
*** jtomasek has joined #tripleo12:56
jaosoriorI think there was another port conflict... I don't really remember12:56
jaosoriorbut we did try, and it didn't work out12:56
*** oshvartz has quit IRC12:57
*** hjensas has joined #tripleo12:57
*** hjensas has joined #tripleo12:57
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Set the default event pipeline publisher  https://review.openstack.org/40993112:58
EmilienMjaosorior: I see12:58
EmilienMjaosorior: let's keep ovb for now12:58
*** skramaja_ has joined #tripleo13:00
*** zoli|lunch is now known as zoli13:00
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata  https://review.openstack.org/40483113:00
*** skramaja has quit IRC13:00
*** lucas-hungry is now known as lucasagomes13:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras: Build role: support using sources from directory  https://review.openstack.org/40536913:01
*** jcoufal has joined #tripleo13:04
jaosoriorsshnaidm: why the -f in the rm?13:05
sshnaidmjaosorior, for case it's "-i" by default or something like this, just usual to do it such way always :)13:06
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add FreeIPA server installation template  https://review.openstack.org/41244413:07
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction  https://review.openstack.org/39173813:07
jaosoriorsshnaidm: ok, addressed your comments ^^13:07
jaosoriorsshnaidm: thanks dude13:07
sshnaidmjaosorior, np13:08
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968813:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]13:10
*** ooolpbot has quit IRC13:10
jaosoriorEmilienM: where are we supposed to see the logs? Your test commit is ready http://logs.openstack.org/18/412418/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/08ebf8d/ but I haven't found them13:11
EmilienMjaosorior: in the undercloud logs I guess13:11
EmilienMhttp://logs.openstack.org/18/412418/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/08ebf8d/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-12-19_12_47_20_00013:12
EmilienMd0ugal, jaosorior ^13:12
EmilienMit sounds like db-populate needs endpoints in place?13:12
EmilienMthat's terrible :D13:12
*** pblaho has quit IRC13:13
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Move Mistral service to authtoken  https://review.openstack.org/41245013:13
*** pblaho has joined #tripleo13:13
jaosoriorEmilienM: There was an error trying to create the ironic-inspector client using a session: Invalid input for field 'identity/password/user/password': None is not of type 'string'13:13
*** milan is now known as milan|afk13:14
jaosoriorthat's a new one13:14
*** trown|outtypewww is now known as trown13:14
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970413:14
*** limao has joined #tripleo13:14
EmilienMjaosorior: it sounds like it tries to connect to keystone API13:15
jaosoriorEmilienM: that's because ironic-inspector requires a valid session to even be able to instantiate the client13:15
jaosoriorhowever, that validation is a new thing13:15
d0ugalEmilienM: oh, thanks for getting that!13:15
jaosoriord0ugal: we really need your patch so we can get where this validation is taking place x_x13:16
d0ugaljaosorior: :-D13:16
jaosoriorand we might even need a mock session or auth plugin just to work around this13:16
d0ugalEmilienM: Can you depend on this? https://review.openstack.org/41243013:16
*** bfournie has quit IRC13:17
d0ugaljaosorior: so many broken actions lol, kinda amazing we have not hit this before.13:17
EmilienMd0ugal: yes we could depends on that13:17
jaosoriord0ugal: actually, I a lot of them have been broken for a VERY long time, it's just that we don't use them13:19
jpichd0ugal: Shouldn't this be LOG.exception rather than LOG.error?13:19
jaosoriorfor instance, the keystone ones13:19
d0ugaljpich: yes, good point.13:19
yolandahi jaosorior , do you remember the fix you did for capturing /home/stack logs?13:20
yolandai have same issue in http://logs.openstack.org/86/407386/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/8929b5c/console.html13:20
yolandadiskimage-builder fails with returned non-zero exit status 113:20
openstackgerritMerged openstack/tripleo-heat-templates: Correction to SRIOV THT Examples  https://review.openstack.org/40851113:20
yolandaand no clue on the build image, /home/stack logs are not pushed13:20
jaosorioryolanda: that fix should ahve merged already AFAIK13:21
yolandado you remember the fix you did? can we do it as well for this kind of changes?13:21
jaosoriorlet me check13:21
jaosoriorright13:21
jaosoriorso13:21
jaosorioryolanda: here, http://logs.openstack.org/86/407386/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/8929b5c/logs/ and download undercloud.tar.xz13:21
jaosoriorthere you'll see the files in /home/jenkins13:22
yolandaah sorry, i forgot it was on the gzipped file13:23
yolandai was looking at some /home/stack dir :(13:23
yolandaah, hitting /usr/local/bin/os-svc-enable: line 56: map-services: command not found13:25
*** morazi has quit IRC13:25
yolandajaosorior, EmilienM , can you review https://review.openstack.org/406895 ?13:25
yolandai need that fix to be on master branch as well, or the CI is not passing13:25
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configure conr parameters for Cinder Heat Keystone and Nova  https://review.openstack.org/41245413:26
jristjtomasek: on your patch 409825, akrivoka is getting an error trying to save a parameter13:26
jristjtomasek: Edit Configurations -> Parameters -> change any parameter -> Save Changes -> Error: Cannot read property 'map' of undefined13:26
jristjtomasek: but I am not...13:27
jaosorioryolanda: what patch needs that?13:27
jristjtomasek: is it possible one of us haven't updated something properly?13:27
jaosorioryolanda: have you tried using depends-on?13:27
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configure cron parameters for Cinder Heat Keystone and Nova  https://review.openstack.org/41245413:29
*** ipsecguy_ is now known as ipsecguy13:29
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Add a per service bootstrap node variable  https://review.openstack.org/41245513:30
jtomasekjrist: I'll try to reproduce13:30
openstackgerritMerged openstack/instack-undercloud: Run `yum update -y` before Puppet run  https://review.openstack.org/41195713:31
*** pradk has joined #tripleo13:31
*** prateek has joined #tripleo13:31
*** weshay_lata is now known as weshay13:31
EmilienMyolanda: looking13:31
jtomasekakrivoka: that patch has dependency, did you apply both?13:32
EmilienMyolanda: I'm afraid to approve something that doesn't pass CI13:32
jaosoriorbandini: are network-based hostnames needed for the bootstrap nodes?13:33
*** limao has quit IRC13:34
bandinijaosorior: yeah, I think so. See the related partial-bug. I will be tweaking all the if $::hostname == bootstrap_node checks in a subsequent patch13:34
*** limao has joined #tripleo13:35
*** zaneb has quit IRC13:36
*** skramaja has joined #tripleo13:36
*** skramaja_ has quit IRC13:36
jaosoriorbandini: I see, but still, all you want to know if, for that specific role, it's the first node, so the information on what network the service is using, or the network-based FQDN is not really needed there, I think.13:36
bandinijaosorior: yeah, I added just in case it might be needed. I can probably trim it out13:37
bandinilet's see how the related puppet-tripleo patch shapes up13:38
jaosoriorsure13:38
*** noama has joined #tripleo13:38
*** rlandy has joined #tripleo13:39
*** zaneb has joined #tripleo13:39
shinobu_gfidente: after building overcloud ceph cluster using undercloud, the additional nodes added to overcloud ceph cluster will not affected by os-***-confit. Is it correct? meaning that those additional nodes are completely stand-alone.13:39
bandinijaosorior: btw. this is the one we briefly mentioned on friday https://review.openstack.org/#/c/412214/ (cleaning up the unused pacemaker profiles from tht)13:39
*** noama has left #tripleo13:39
jaosoriorbandini: that looks good! :D13:40
bandinieheh code removal is always good ;)13:40
jaosoriorindeedo13:40
shinobu_*os-***-config*13:42
shinobu_sorry for typos...13:42
yolandaEmilienM, i mean, the changes in diskimage-builder don't pass CI13:42
yolandathis change was passing, now fails but i think is unrelated13:42
jaosoriorslagle: I saw you added the DeployedServer resource to heat, is it something we'll take into use in TripleO?13:43
*** fultonj has quit IRC13:43
EmilienMyolanda: ok,i'll look13:43
jristjtomasek: I wonder at the very least if we should protect the user from being exposed to that possible error13:45
*** fultonj has joined #tripleo13:45
*** jpena is now known as jpena|lunch13:46
*** pgadiya has quit IRC13:46
jtomasekjrist: It depends on the reason why akrivoka is hitting it. If there is a possibility to hit that error, then, we should fix it13:47
openstackgerritLucas Alvares Gomes proposed openstack/python-tripleoclient: Install VirtualBMC in the overcloud-full image  https://review.openstack.org/41241613:50
*** ansmith has joined #tripleo13:52
*** limao_ has joined #tripleo13:52
*** limao has quit IRC13:56
*** tzumainn has joined #tripleo13:57
*** bfournie has joined #tripleo13:57
*** panda|afk is now known as panda13:58
akrivokajrist: jtomasek: I didn't have the dependent patch applied - apologies. I'll try again with the dependency applied.13:58
jtomasekakrivoka: cool :)13:58
akrivokais it just me or was it easier to spot dependencies in previous gerrit versions...13:59
jpichakrivoka: For the "same repo" dependencies, I agree but mostly 'cause I've been making similar mistakes :-)14:00
akrivokajpich: :)14:00
slaglejaosorior: yea, see my patches14:02
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo: WIP DO NOT MERGE Make bootstrap_node checks more robust  https://review.openstack.org/40872614:04
*** Goneri has joined #tripleo14:06
*** sudipto has joined #tripleo14:08
*** amoralej is now known as amoralej|lunch14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968814:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]14:10
openstackgerritMerged openstack/tripleo-quickstart: Update author and author-email  https://review.openstack.org/40845214:10
*** flepied has quit IRC14:10
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: WIP DO NOT MERGE pacemaker remote profile support  https://review.openstack.org/40097214:13
*** limao has joined #tripleo14:13
openstackgerritMerged openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970414:14
*** chlong has joined #tripleo14:14
*** lblanchard has joined #tripleo14:15
*** bkopilov has joined #tripleo14:15
*** shardy_lunch is now known as shardy14:16
*** limao_ has quit IRC14:16
*** bkopilov_ has joined #tripleo14:16
jaosoriorshardy: hey dude, what do you think about https://review.openstack.org/#/c/411340/13 should I add or change anything?14:16
openstackgerritMerged openstack/tripleo-quickstart: Update list of packages to include from delorean-current for devmode  https://review.openstack.org/41095514:17
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo: WIP DO NOT MERGE Make bootstrap_node checks more robust  https://review.openstack.org/40872614:18
openstackgerritMerged openstack/tripleo-quickstart: Add option to set DNS name servers on the undercloud  https://review.openstack.org/40145214:19
*** jmelvin has joined #tripleo14:19
*** almondjoy has joined #tripleo14:19
*** yprokule has quit IRC14:21
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras: Create tasks to install, run stackviz and collect logs  https://review.openstack.org/40078214:22
*** pradk has quit IRC14:25
*** morazi has joined #tripleo14:25
*** ohamada has quit IRC14:26
*** ohamada has joined #tripleo14:26
*** almondjoy has quit IRC14:27
*** flepied has joined #tripleo14:27
*** social has quit IRC14:29
*** morazi has quit IRC14:29
openstackgerritMerged openstack/tripleo-ui: Disallow underscores in plan names  https://review.openstack.org/41133014:33
openstackgerritHonza Pokorny proposed openstack/tripleo-quickstart: Add ssh tunneling for GUI  https://review.openstack.org/40773314:34
*** kjw3 has joined #tripleo14:35
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Add basic pre/post upgrade sanity checks  https://review.openstack.org/41184614:37
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Add glance to major upgrade CI  https://review.openstack.org/41247514:37
*** rbowen has joined #tripleo14:37
*** kaslcrof has joined #tripleo14:38
*** zoli is now known as zoli|FBC114:39
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo: WIP DO NOT MERGE Make bootstrap_node checks more robust  https://review.openstack.org/40872614:41
*** almondjoy has joined #tripleo14:41
*** morazi has joined #tripleo14:43
*** ohamada has quit IRC14:44
*** ohamada has joined #tripleo14:44
*** liverpooler has joined #tripleo14:44
*** prateek has quit IRC14:46
trozet_dsneddon: hi14:49
*** trozet_ is now known as trozet14:49
*** liverpooler has quit IRC14:49
*** jtomasek has quit IRC14:50
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras: Repositories setup role  https://review.openstack.org/40816214:51
*** cdearborn has joined #tripleo14:51
*** liverpooler has joined #tripleo14:51
*** prateek has joined #tripleo14:51
*** milan|afk is now known as milan14:51
*** noslzzp has joined #tripleo14:55
*** amoralej|lunch is now known as amoralej14:56
*** noslzzp has quit IRC14:56
*** noslzzp has joined #tripleo14:57
*** sudipto has quit IRC14:59
*** jtomasek has joined #tripleo14:59
*** sudipto has joined #tripleo14:59
*** sudipto has quit IRC14:59
openstackgerritMerged openstack/tripleo-quickstart-extras: Repositories setup role  https://review.openstack.org/40816214:59
*** sudipto has joined #tripleo14:59
*** sudipto_ has joined #tripleo15:00
*** sudipto_ has quit IRC15:00
*** sudipto has quit IRC15:00
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: move the undercloud deploy role from quickstart to quickstart-extras  https://review.openstack.org/41035315:00
*** sudipto has joined #tripleo15:00
*** sudipto has quit IRC15:00
*** sudipto has joined #tripleo15:01
*** sudipto_ has joined #tripleo15:01
*** dmarlin has joined #tripleo15:01
*** dmarlin is now known as dmarlin_15:01
*** jpena|lunch is now known as jpena15:03
*** pradk has joined #tripleo15:04
*** limao has quit IRC15:04
openstackgerritMerged openstack/tripleo-heat-templates: Add a type for the ControlVirtualIP resource  https://review.openstack.org/40954515:05
*** limao has joined #tripleo15:06
*** zoli|FBC1 is now known as zoli15:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968815:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]15:10
*** Vijayendra has quit IRC15:12
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-heat-templates: Configure DPDK options to isolate PMD cores and ovs process cores  https://review.openstack.org/41249215:12
*** jtomasek has quit IRC15:13
EmilienMd0ugal, jaosorior: any update on the introspection issue? could you reproduce? d0ugal, could you have more logs with your patch in mistral?15:15
*** liverpooler has quit IRC15:15
*** liverpooler has joined #tripleo15:15
*** flepied has quit IRC15:18
pandaEmilienM: you changed postci bug to fix released, what's the fix ? https://review.openstack.org/#/c/411514/ ?15:19
openstackgerritSven Anderson proposed openstack/tripleo-heat-templates: Add ec2-api service  https://review.openstack.org/39863415:19
openstackgerritSven Anderson proposed openstack/tripleo-heat-templates: DON'T MERGE - tester for I2510fd4ed935d8423216fff9ce3adf2d69c9c804  https://review.openstack.org/40616715:19
*** cdearborn has quit IRC15:20
EmilienMpanda: I thought we killed the process15:21
EmilienMpanda: was it temporary? feel free to open it15:21
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: WIP -- Add pacemaker ansible module of composable upgrade  https://review.openstack.org/40339715:21
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Adds a step0 for pre upgrade-init checks  https://review.openstack.org/40863115:21
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: WIP - Add the pacemaker ansible modules during upgrade init  https://review.openstack.org/41249415:21
*** limao_ has joined #tripleo15:22
pandaEmilienM: bnemec fix with SSL should be temporary, but it's good enough. No killing was doing any good since it's definitely a firewall problem: https://review.openstack.org/411189, using firewall debug makes things work (disables drop-all-packets fallback rule), now we'll probably have to find the right combination of rules. Mitaka still fails probably because debug flag wasn't present there.15:24
pandaEmilienM: I'll add these infos to the bug15:24
*** limao has quit IRC15:25
thrashd0ugal: any ideas for unit testing actual workflows?15:25
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Try temprevert of puppet-mistral authtoken change  https://review.openstack.org/41249515:26
*** scorcoran_mtg has joined #tripleo15:26
EmilienMbnemec: oh damn, nice catch! ^15:27
bnemecEmilienM: I don't know if it will help, but I figured it's an easy thing to try.15:27
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: make quickstart-extras-requirements.txt a default requirements file  https://review.openstack.org/41075715:28
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: move the undercloud deploy role to quickstart-extras for composability  https://review.openstack.org/41042715:28
EmilienMbnemec: I'm currently investigating mitaka gate issues15:28
EmilienMbnemec: they happen during pingtest15:28
bnemecEmilienM: Oh, that's probably not fun.  mitaka ovb jobs are still hanging, aren't they?15:29
d0ugalEmilienM: I am working out how to make Mistral always log the CLI commands.15:30
d0ugalthrash: I have pondered it, there is no easy way. Mistral runs workflows in it the internal unit tests - we could do that if we are happy to use private APIs... :)15:31
d0ugalthrash: they are unlikely to change (as it would break many Mistral tests, but it could happen)15:31
EmilienMbnemec: no, pingtest fails, boot from volume15:32
EmilienMd0ugal: please let me know before your EOD, maybe I can follow up15:32
EmilienMshould we stop testing introspecting in the meantime?15:32
thrashbnemec: fyi... https://review.openstack.org/#/c/412450/ not sure if that helps.15:33
* bnemec dislikes shutting off features that are actually broken15:33
*** flepied has joined #tripleo15:33
bnemecThat said, I did it to get the updates job working, so I can't really throw stones here. :-)15:33
EmilienMbnemec: ha15:34
bnemecthrash: It's undercloud mistral that's failing though.  A similar patch there might do the trick though.15:34
thrashbnemec: ack.15:34
thrashbnemec: I'll take a look15:35
*** cdearborn has joined #tripleo15:37
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: move the undercloud deploy role from quickstart to quickstart-extras  https://review.openstack.org/41035315:37
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add cell_v2 simple_cell_setup  https://review.openstack.org/41200615:38
*** milan is now known as milan|afk15:38
*** weshay is now known as weshay_afk15:41
*** limao has joined #tripleo15:43
thrashbnemec: looks to already be done? https://review.openstack.org/#/c/409478/15:43
*** saneax is now known as saneax-_-|AFK15:43
thrashbnemec: not sure if an 'include mistral::keystone::authtoken' is necessary in puppet-stack-config.pp15:44
*** prateek has quit IRC15:44
bnemecthrash: Could be.  The merged patch obviously didn't fix the problem.15:45
*** weshay_afk is now known as weshay15:45
*** limao_ has quit IRC15:45
thrashbnemec: nope... should just work.15:46
thrashbnemec: https://github.com/openstack/puppet-mistral/blob/master/manifests/init.pp#L300-L302 and auth_strategy defaults to 'keystone'15:47
*** cdearborn has quit IRC15:48
*** cdearborn has joined #tripleo15:51
yolandaagh, the tripleo CI keeps failing with unrelated changes to me15:52
yolandaunrelated failures i mean15:53
*** yamahata has joined #tripleo15:53
*** cdearborn has quit IRC15:55
*** scorcoran_mtg is now known as scorcoran15:55
*** yamahata has quit IRC15:56
*** yamahata has joined #tripleo15:57
*** Guest33254 has quit IRC16:02
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: Add generic undercloud setup role  https://review.openstack.org/40632416:03
*** limao_ has joined #tripleo16:04
*** nyechiel has quit IRC16:04
*** nyechiel has joined #tripleo16:04
openstackgerritAde Lee proposed openstack/tripleo-quickstart: changes to add novajoin to undercloud  https://review.openstack.org/39877216:05
*** ebarrera has joined #tripleo16:05
*** limao has quit IRC16:06
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632416:06
*** scorcoran is now known as scorcoran_afk16:08
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968816:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]16:10
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489216:10
*** ebarrera has quit IRC16:11
*** jtomasek has joined #tripleo16:12
*** pcaruana has quit IRC16:15
*** jtomasek has quit IRC16:16
mwhahahahey people, questions for you on the best way to pass the nova rabbitmq connection information over to the mysql setup, https://review.openstack.org/#/c/412007/2/manifests/profile/base/database/mysql.pp16:17
mwhahahahow would on go about getting that information over to the mysql THT16:17
EmilienMmwhahaha: service_config_settings16:18
*** akrivoka has quit IRC16:18
EmilienMa sec16:18
*** cdearborn has joined #tripleo16:18
jaosoriormwhahaha: would sure use the ssl option there :D16:18
EmilienMmwhahaha: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/nova-api.yaml#L13916:18
openstackgerritAde Lee proposed openstack/tripleo-quickstart: changes to add novajoin to undercloud  https://review.openstack.org/39877216:19
mwhahahaEmilienM: so i add that to the mysql THT?16:19
EmilienMmwhahaha: in nova-base16:19
EmilienMit will send the info over to the mysql16:19
mwhahahaah ok16:19
EmilienMmwhahaha: see that https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/nova-api.yaml#L147-L15016:20
mwhahaha*magic*16:20
shardymwhahaha: why does mysql need the nova rabbit connection information, out of interest?16:20
mwhahahashardy: that stupid cell v2 setup16:20
EmilienMshardy: cells v2 :-)16:20
shardymwhahaha: oh yeah, ugh :(16:20
EmilienMthat's terrible, right?16:20
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras: [WIP] - Add basic documentation for overcloud-upgrade role  https://review.openstack.org/40932816:21
*** jtomasek has joined #tripleo16:22
*** hewbrocca is now known as hewbrocca_afk16:22
*** tremble has quit IRC16:24
*** pblaho has quit IRC16:25
*** limao has joined #tripleo16:25
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Pass nova rabbit information to mysql  https://review.openstack.org/41252016:25
*** limao_ has quit IRC16:29
*** fzdarsky has quit IRC16:31
*** weshay is now known as weshay_bbiab16:34
*** abregman has quit IRC16:35
*** jtomasek has quit IRC16:35
mnaser"Since there can be no one-size-fits-all procedure for upgrading an existing cloud to use TripleO, it is recommended that a new cloud be deployed with TripleO and any workloads running on an existing cloud be migrated off."16:36
mnaserdoes this really mean that it cannot happen at all? or that there is no documented procedure16:37
mnaserbecause we have a few ideas in mind to accomplish this.. we want to move a deployment to run over triplo16:37
*** snecklifter has joined #tripleo16:37
EmilienMmwhahaha: don't we need the transport_url thing?16:38
EmilienMmwhahaha: or nova::rabbit* are enough?16:38
mwhahahaEmilienM: transport_url would but the rabbit stuff is sufficient16:38
mwhahahawould be ideal16:38
EmilienMmwhahaha: fair enough, +216:39
*** akrivoka has joined #tripleo16:39
shardymnaser: it's definitely completely untested and undocumented, and I think to pull it off you'd need to either use heat's adopt functionality, or the deployed server functionality (still under development and only partially supported)16:39
mnaseri was thinking to do something like a full deploy, then slowly start moving services and their db stuff over16:39
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras: Create tasks to install, run stackviz and collect logs  https://review.openstack.org/40078216:40
shardymnaser: Yeah some kind of dump/restore migration might also be possible I guess16:40
openstackgerritMerged openstack/tripleo-quickstart: make quickstart-extras-requirements.txt a default requirements file  https://review.openstack.org/41075716:40
mnaserand so doing that stage by stage would keep both environments running together16:40
mnaserthe thing here is we don't mind redeploying machines.. but we don't want to shut the whole thing down either16:41
shardymnaser: the tricky think is how you have both clouds co-exist, like which keystone etc - essentially the reccomendation is to stand up a new cloud then migrate the data/workloads -16:42
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Add a per service bootstrap node variable  https://review.openstack.org/41245516:42
mnasershardy: well if both keystone-api services are pointing to the same db, essentially, whichever one you'll hit will work properly16:43
*** cinerama has joined #tripleo16:43
mnaserthe only thing i guess is the per-service passwords need to be matching16:43
mnaserso that services (ex: nova, etc) auth info don't fail16:43
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Add cell_v2 setup for nova  https://review.openstack.org/41200716:43
*** cinerama has quit IRC16:44
*** cinerama has joined #tripleo16:44
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Add a 'manage' workflow  https://review.openstack.org/41252516:46
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Optionally deploy undercloud node with OVB  https://review.openstack.org/41252616:47
jaosoriorbnemec: something like this? ^^16:47
*** jaosorior has quit IRC16:48
snecklifterhello, on OSP 10 I'm getting "Could not find data item memcached_node_ips in any Hiera data file"16:48
*** limao has quit IRC16:48
snecklifterI note that overcloud.j2.yaml is missing the d in memcached16:49
*** jaosorior has joined #tripleo16:49
snecklifteram I barking up a wrong tree?16:49
*** b00tcat has quit IRC16:49
snecklifter memcache_node_ips: {get_attr: [ .....16:49
*** limao has joined #tripleo16:50
snecklifterI'm using predictable IPs and hostnames16:50
*** sshnaidm is now known as sshnaidm|afk16:52
shardysnecklifter: that get_attr probably isn't related as it's only used for the memcached_node_ips_v6 in puppet/all-nodes-config.yaml16:54
shardythe memcached_node_ips should be automatically generated based on this service name16:55
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/memcached.yaml#L2916:55
shardysnecklifter: you can confirm by grep node_ips in /etc/puppet/hieradata on the failing node16:55
shardysnecklifter: are you using any custom roles or service lists?16:55
openstackgerritMerged openstack/tripleo-quickstart-extras: move the undercloud deploy role from quickstart to quickstart-extras  https://review.openstack.org/41035316:56
sneckliftershardy: using custom servicenetmap16:56
sneckliftershardy: have updated to Newton, I believe16:56
snecklifterusing custom network names too16:56
shardysnecklifter: Ok but you're not using a customized ControllerServices parameter, or any special roles_data.yaml?16:57
shardyanother reason you might see that error is customizing the services list, and forgetting to add OS::TripleO::Services::Memcached16:57
snecklifterno, not customizing services16:58
snecklifterno controllerservices or roles_data either16:58
*** leanderthal is now known as leanderthal|afk16:59
EmilienMbnemec, shardy: introspection looks broken for us now, and it's blocking packaging promotions to happen. I think we might want to stop testing it now until we find out what was wrong16:59
snecklifteras for grepping, it returns /etc/puppet/hieradata/all_nodes.yaml:controller_node_ips: 10.0.10.22,10.0.10.14,10.0.10.1216:59
bnemecjaosorior: Commented16:59
bnemecEmilienM: Do we really want to promote a broken repo?17:00
sneckliftershardy: and the ipv6 line17:00
EmilienMbnemec: I know17:00
snecklifterI will try backing out the custom network names17:00
EmilienMbnemec: do we really want to lag so much?17:00
EmilienMbnemec: we all know that the more we lag, higest is the number of bugs we might have17:01
EmilienMhighest*17:01
*** limao has quit IRC17:02
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Block the scale down workflow until the stack is COMPLETE or FAILED  https://review.openstack.org/40774117:02
shardysnecklifter: The generation of the per-service hieradata does depend on the ServiceNetMap (or ServiceNetMapDefaults) entries :17:03
EmilienMbnemec, let's see how goes your patch and we'll discuss17:03
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/network/service_net_map.j2.yaml17:03
EmilienMmaybe https://review.openstack.org/#/c/412495 will pass17:03
shardysnecklifter: did you use ServiceNetMap, or ServiceNetMapDefaults?17:03
EmilienMbnemec: since mwhahaha did that https://review.openstack.org/#/c/409478/ it could be unpin now17:04
sneckliftershardy: ServiceNetMap17:04
*** jpich has quit IRC17:04
EmilienMbnemec: https://review.openstack.org/#/c/409477/ this one is ready for review also17:05
bnemecEmilienM: Note that https://review.openstack.org/#/c/409478/ still failed introspection, so I'm skeptical that it fixed the problem.17:06
*** nyechiel has quit IRC17:07
EmilienMbnemec: well, if we've pin the module no?17:07
sneckliftershardy: hieradata indicates its selecting the right custom network name... odd17:08
bnemecEmilienM: We haven't, have we?  That's why the temprevert patch was needed.17:08
shardysnecklifter: Ok, that should work - something is definitely wrong though as after upgrading to newton, you should see *node_ips for every service listening on a network in the all_nodes hieradata file17:08
EmilienMbnemec: ok I need a second coffee.17:09
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632417:09
sneckliftershardy: right, it looks like most of them are present, just missing memcached ips17:09
sneckliftershardy: the node names are there17:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968817:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]17:10
*** scorcoran_afk has quit IRC17:12
bnemecEmilienM: The ovb jobs don't deploy mistral, right?  So https://review.openstack.org/#/c/409477/1 can merge without them since they don't use that file anyway?17:12
EmilienMbnemec: not on overcloud. Only scenarios.17:13
EmilienMbnemec: yes we can merge it17:13
bnemecOkay17:13
EmilienMbnemec: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/pingtests/scenario003-multinode.yaml#L129-L14317:14
EmilienMbnemec: scenario003 actually tested mistral API17:14
bnemecEmilienM: I don't think that temprevert patch is going to pass.  We already merged the instack-undercloud patch to use the parameters that were added in it. :-(17:14
EmilienMbnemec: it's only hiera17:15
EmilienMbnemec: it will be ignored, so no worries17:15
sneckliftershardy: am opening case for it, thanks for answering, much appreciated17:15
bnemecI think we need to revert both for it to work.17:15
bnemecEmilienM: Yeah, but we removed the old config, didn't we?17:15
* bnemec looks17:15
EmilienMah, maybe17:15
EmilienMwe could try to 1) revert mwhahaha's patch and 2) submit a temprevert in tripleo-ci with depends-on the revert17:16
EmilienMand see17:16
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Also temprevert instack-undercloud authtoken change  https://review.openstack.org/41254017:18
bnemecEmilienM: Or I can abuse temprevert^ :-)17:18
bnemecYeah, the first temprevert failed. :-(17:19
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632417:19
bnemecAlthough actually, testing this in instack-undercloud would be better.17:19
EmilienMbnemec: will it just revert and rebase? or checkout?17:19
bnemecLess jobs wasted.17:19
EmilienMyeah17:20
*** scorcoran_afk has joined #tripleo17:20
bnemectemprevert will just revert the single patch.17:20
*** scorcoran_afk is now known as scorcoran17:20
bnemecThere's a separate pin function that will freeze it at a single commit.17:20
openstackgerritBen Nemec proposed openstack/instack-undercloud: Revert "Switch mistral to use authtoken configuration"  https://review.openstack.org/41254117:22
*** rwsu has joined #tripleo17:22
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Try temprevert of puppet-mistral authtoken change  https://review.openstack.org/41249517:22
*** rwsu has quit IRC17:23
*** mcornea has quit IRC17:23
bnemecHeh, actually that wasn't any better, was it?17:23
*** milan|afk is now known as milan17:23
bnemecI had to change the ci patch anyway so we still have to run all those jobs.17:23
* bnemec blames Monday17:23
openstackgerritAde Lee proposed openstack/tripleo-puppet-elements: Add ipa-admintools package to the overcloud.  https://review.openstack.org/41254217:24
*** sudipto has quit IRC17:25
*** sudipto_ has quit IRC17:25
*** sudipto_ has joined #tripleo17:25
*** sudipto has joined #tripleo17:25
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configure cron parameters for Cinder Heat Keystone and Nova  https://review.openstack.org/41245417:26
bnemecBleh, I shouldn't have approved the overcloud patch either yet.17:28
bnemecOne more thing to revert if we end up having to.17:29
* bnemec thinks about calling it a day and trying again tomorrow17:29
bnemecUgh, so many jobs queued and all doomed to fail17:31
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Convert "convert_image.yml" task list to role  https://review.openstack.org/41254617:32
*** x220_ has joined #tripleo17:35
*** rwsu has joined #tripleo17:36
*** zoli is now known as zoli|gone17:37
*** newmember has quit IRC17:38
*** panda is now known as panda|bbl17:38
*** egafford has quit IRC17:38
*** iranzo has quit IRC17:40
*** chem has quit IRC17:40
*** sshnaidm|afk has joined #tripleo17:40
*** chem has joined #tripleo17:41
*** derekh has quit IRC17:44
shinobu_i'm going to add existing ceph osd nodes to overcloud which are defined in crush map uniquely.17:47
EmilienMbnemec: didn't you miss Depends-On on https://review.openstack.org/#/c/412541/ ?17:47
*** leanderthal|afk has quit IRC17:47
shinobu_those particular nodes are expected to be maintained manually.17:47
shinobu_i hope those particular nodes won't be impacted by os-*-config...17:48
larsksHey, tripleo folks. Is there any way to make a template parameter optional? I would like my puppet modules to receive "undef" from the tripleo, but parameters without a default get flagged as unset and cause the validation to fail.17:48
*** trown is now known as trown|lunch17:48
bnemecEmilienM: No, the depends-on goes the other way: https://review.openstack.org/#/c/412495/17:48
bnemecFor testing I guess it doesn't matter, but if we decide to merge any of this then it needs to happen in that order.17:49
*** nyechiel has joined #tripleo17:49
EmilienMbnemec: excellent, sorry I missed it17:50
*** lucasagomes is now known as lucas-afk17:50
*** weshay_bbiab is now known as weshay17:51
*** sudipto_ has quit IRC17:51
*** sudipto has quit IRC17:51
*** liverpooler has quit IRC17:53
*** liverpooler has joined #tripleo17:55
jristEmilienM, shardy, dprince, slagle - can one of you please add honza to https://launchpad.net/~tripleo-drivers/+members17:55
jristso he can triage bugs and the like17:56
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Run extras playbook by default  https://review.openstack.org/40755517:56
EmilienMdone17:56
openstackgerritMerged openstack/puppet-tripleo: Drop vip_hosts  https://review.openstack.org/40344917:58
openstackgerritMerged openstack-infra/tripleo-ci: Add jpich to Planet  https://review.openstack.org/41125517:58
shinobu_but those nodes should be happy with nodes deployed via undercloud.17:58
jaosoriorbnemec: hey dude, saw your comments on the patch17:59
*** leanderthal|afk has joined #tripleo17:59
jaosoriorbnemec: I actually set the value to nothing here https://review.openstack.org/#/c/412526/1/testenv-client , cause I wanted to keep an empty default here https://review.openstack.org/#/c/412526/1/scripts/te-broker/create-env18:00
*** yamahata has quit IRC18:01
*** flepied has quit IRC18:03
mandrejaosorior: I'm trying to debug a mysql failure on my overcloud control node, can I borrow your brain for a minute or two?18:04
bnemecjaosorior: Right, but exposing that in the testenv-client interface is going to require the caller to pass in a meaningless arbitrary string when they call it.18:04
bnemec(I think)18:04
bnemecI'd rather make that a boolean and translate it to a string at my second comment there.18:04
jaosoriorbnemec: I can do that18:04
jaosoriormandre: I was about to log off :/18:04
mandrejaosorior: darn!18:05
jaosoriormandre: sorry dude, been here 13 hours18:05
jaosoriormy brain quality is questionable18:05
mandrejaosorior: this can wait until tomorrow, no worries :)18:05
jaosoriorsure! ping me tomorrow18:05
mandrejaosorior: thanks, enjoy your evening18:05
jaosoriorwill do :D18:06
openstackgerritMerged openstack/diskimage-builder: set default DIB_PYTHON_VERSION=2 for rhel7  https://review.openstack.org/41232418:07
*** shardy has quit IRC18:08
*** jpena is now known as jpena|away18:09
bkeroEmilienM: did the latest patch set on https://review.openstack.org/#/c/410391/ fix what you were referring to?18:09
*** nyechiel has quit IRC18:09
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Close testenv-client before zuul finishes  https://review.openstack.org/41047018:09
*** ohamada has quit IRC18:10
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968818:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]18:10
*** almondjoy has quit IRC18:11
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Close testenv-client before zuul finishes  https://review.openstack.org/41047018:16
*** athomas has quit IRC18:17
openstackgerritAlex Schultz proposed openstack/tripleo-docs: Remove Liberty install instructions  https://review.openstack.org/41256518:21
*** gfidente is now known as gfidente|afk18:23
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256718:25
openstackgerritBen Nemec proposed openstack/instack-undercloud: Add a generic failure message on exception  https://review.openstack.org/41256818:26
shinobu_afaik os-collect-config will be triggered by heat metadata modification. is there any documentation regarding to what heat metadata means?18:27
EmilienMbkero: I proposed to have nonha in the name18:31
EmilienMwe have nonha-multinode18:31
EmilienMwe could have non-ha-multinode-oooq18:32
*** yamahata has joined #tripleo18:40
*** bana_k has joined #tripleo18:41
openstackgerritBen Nemec proposed openstack/instack-undercloud: Set fernet max_active_keys to 2  https://review.openstack.org/41257718:45
openstackgerritMerged openstack/os-cloud-config: Skip 0.13.1 version of docutils  https://review.openstack.org/41094418:45
openstackgerritMerged openstack/tripleo-heat-templates: Switch mistral to use authtoken configuration  https://review.openstack.org/40947718:46
*** flepied has joined #tripleo18:46
*** fultonj has quit IRC18:46
openstackgerritMerged openstack-infra/tripleo-ci: Test plan delete with stack delete  https://review.openstack.org/40830118:48
EmilienMmwhahaha: so iiuc, blockers to promote tripleo CI is 1) nonha job failing on introspection (see alert) and 2) nova cell v2 stuff you're finishing18:52
*** lmiccini has quit IRC18:52
*** pkovar has quit IRC18:52
mwhahahadid we figure out how introspection broke because of the auth change?18:52
*** almondjoy has joined #tripleo18:53
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Revert "Enable undercloud ssl on all OVB jobs"  https://review.openstack.org/41258318:54
EmilienMmwhahaha: not yet it seems ^18:54
bnemecmwhahaha: We're still not sure that's the culprit.18:54
mwhahahai figured it wasn't but just wanted to check :D18:54
* mwhahaha goes back to bashing head against wall aka cell v218:55
bnemecDepending on the outcome of https://review.openstack.org/#/c/412495/ we'll know whether it is or not.18:55
openstackgerritgreghaynes proposed openstack/diskimage-builder: Update our package classifiers  https://review.openstack.org/41258418:55
bnemecJobs are running on that patch now so hopefully we'll know soon.18:55
EmilienMyeah, it's deploying undercloud now18:55
EmilienMbnemec: thanks for https://review.openstack.org/#/c/412577/ - we might want to do the same for THT18:56
bnemecEmilienM: Yeah, I would guess the overcloud has the same messages, but I haven't gotten that far in my local test.18:57
bnemecI can't even upload images to undercloud glance right now for some reason. :-(18:57
EmilienMoops18:57
EmilienMbnemec: ocata?18:57
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Add cell_v2 setup for nova  https://review.openstack.org/41200718:58
bnemecEmilienM: Yeah, I'm just trying to do a standard deployment of current-tripleo to test a few things.18:58
bnemecSo far not so good.18:58
*** fultonj has joined #tripleo18:58
EmilienMI've also noticed a fair amount of timeouts in overcloud deployments for multinode-non ha jobs18:59
EmilienMbnemec: oops, maybe $newbug18:59
mwhahahai've been getting a timeout when I try to deploy myself18:59
bnemecEmilienM: Those have been going on for a while.  I can't remember if I opened a bug for it or not.18:59
mwhahahalike heat never runs on anything19:00
bnemecI noticed it when the multinode jobs were all dying on something else.19:00
bnemecOh goody, we've got an undercloud idempotency bug on master too.19:00
* mwhahaha gives up and goes to lunch19:01
EmilienMI thought we gated on that19:01
bnemecEmilienM: We do.  I assume it's blocking promotion.19:01
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632419:01
bnemecAh, we also have a glance promotion-blocker: https://bugs.launchpad.net/tripleo/+bugs?field.tag=promotion-blocker19:03
bnemecETOOMANYBUGS19:03
bnemec(this appears to be different from the glance problem I'm having)19:03
EmilienMwe miss the cell v2 stuff19:03
EmilienMI'll add the tag to it19:03
bnemecOh, my glance problem is the cidr change.19:05
bnemecI don't have it in my no_proxy list.19:05
bnemec\o/ one not-actually-a-bug solved19:06
*** b00tcat has joined #tripleo19:06
*** scorcoran is now known as scorcoran_afk19:06
*** jkilpatr_ has joined #tripleo19:06
* bnemec breaks for lunch on that happy-ish note19:07
numansEmilienM, Hi, could you please add this in your review queue - https://review.openstack.org/#/c/387940/19:07
* EmilienM process the information19:08
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256719:13
*** fragatina has quit IRC19:13
*** ooolpbot has joined #tripleo19:13
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:13
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968819:13
*** ooolpbot has quit IRC19:13
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]19:13
*** david-lyle_ has joined #tripleo19:13
EmilienMpanda|bbl: have you progress on https://bugs.launchpad.net/tripleo/+bug/1646756 ?19:13
*** openstackstatus has quit IRC19:13
*** jkilpatr has quit IRC19:13
*** david-lyle has quit IRC19:13
*** openstack has joined #tripleo19:14
*** x220__ has joined #tripleo19:16
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras: Build role: support using sources from directory  https://review.openstack.org/40536919:17
EmilienMnumans: would you be interested to test OVN in tripleo gate?19:19
*** x220_ has quit IRC19:19
EmilienMnumans: if yes, add OVN services and configure them in one of those scenarios https://github.com/openstack/tripleo-heat-templates/tree/master/ci/environments19:19
EmilienMnumans: when we do that, we'll add tripleo jobs to puppet-ovn and ovn profile in puppet-tripleo19:20
numansEmilienM, Yes. thanks. i will look into it19:20
EmilienMnumans: cool. let me know if any help is needed19:20
numanssure.19:20
EmilienMnumans: the soonest the better, we have zero testing for OVN today.19:20
numansEmilienM, agree. that would be my next task tomorrow morning :)19:21
EmilienMnumans: keep in mind we can't pass a new env file. All services & params need to go in this env file19:21
numansok19:21
EmilienMnumans: which means, you might need to copy/paste environments/neutron-ml2-ovn.yaml content into the scenario env19:22
EmilienMor include the env19:22
numansEmilienM, sure.19:23
*** trown|lunch is now known as trown19:25
*** flepied has quit IRC19:25
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632419:25
EmilienMpanda|bbl: same question for https://bugs.launchpad.net/tripleo/+bug/1646750 ? is it still the case?19:26
openstackLaunchpad bug 1646750 in tripleo "CI: periodic jobs fail to upload test image to overcloud glance" [Critical,Triaged]19:26
EmilienMI'll close it, pretty old19:26
EmilienM(closing the promotion-blocker tag, not the bug)19:26
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256719:29
*** sshnaidm|afk is now known as sshnaidm19:29
*** ipsecguy_ has joined #tripleo19:30
*** scorcoran_afk is now known as scorcoran19:34
*** chlong has quit IRC19:34
*** ipsecguy has quit IRC19:35
*** chlong has joined #tripleo19:36
EmilienMbnemec: I have a doubt, which jobs are testing undercloud idempotency?19:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: add the containers prep role to the quickstart-extras  https://review.openstack.org/40098319:37
*** panda|bbl is now known as panda19:38
bnemecEmilienM: Only in periodic jobs.  It takes too long to run in every regular job.19:38
*** scorcoran_ has joined #tripleo19:39
EmilienMbnemec: nonha/introspcetion in progress19:39
EmilienMIT WORKED???19:39
EmilienMah not yet19:40
bnemecLooking promising though.  I don't think it got this far before.19:40
*** scorcoran has quit IRC19:40
EmilienMpanda, sshnaidm: I updated launchpad with current promotion blockers https://bugs.launchpad.net/tripleo/+bugs?field.tag=promotion-blocker19:42
EmilienMpanda: I removed some tags from some bugs you created, because I don't think they are relevant anymore but I might be wrong. Please re-add them if I was wrong.19:42
*** x220__ has quit IRC19:43
EmilienMbnemec: ok it works. How do we proceed? we need to revert 2 things?19:43
EmilienMwe have 2 options: revert in the upstream repos or temprevert in tripleo-ci19:43
EmilienMmwhahaha: ^ fyi mistral thing19:44
EmilienMmwhahaha: it sounds like reverting your puppet-mistral / i-u patches, bring back the ovb-nonha19:44
*** dsariel has quit IRC19:44
mwhahahaK19:44
mwhahahaSounds like we need tripleo CI on puppet-mistral19:45
EmilienMbnemec: let's wait to see if ovb-nonha is fully green and then merge the tmp revert in tripleo-ci and the revert in instack-undercloud19:45
pandaEmilienM: closing them19:45
bnemecEmilienM: We could merge those two patches as-is.  Or just the instack-undercloud one and revert puppet-mistral itself.19:45
EmilienMmwhahaha: we have, tripleo scenarios.19:45
EmilienMmwhahaha: though we don't cover introspection19:45
EmilienMmwhahaha: as multinode don't use ironic19:45
bnemecWe need to run the nonha job on puppet-mistral.19:46
sshnaidmEmilienM, is https://bugs.launchpad.net/tripleo/+bug/1649341 a blocker? I don't  think we check undercloud_upgrade on periodic jobs19:46
openstackLaunchpad bug 1649341 in tripleo "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)19:46
bnemecBecause that's the only one that runs introspection, which seems to be what breaks most of the time.19:46
bnemecAlthough it may just look that way because it's the first major workflow we run.19:46
EmilienMsshnaidm: you're right, I'll remove the tag19:46
EmilienMbnemec: i'll take an action on that19:46
EmilienMbnemec: today19:46
mwhahahaSo going back to how does an auth change break the mistral/ironic interaction? Are we using the old values19:47
bnemecNo idea19:47
mwhahahaSounds like we need an RCA on this one :D19:47
bnemecThis was purely looking at what changed recently and then reverting the most likely thing.19:48
* bnemec can't remember what RCA means19:48
mwhahahaRoot cause analysis19:48
mwhahahaCause we need that auth change to support keystone v3 and everything else already has it19:49
mwhahahaI bet mistral isn't doing v3 against ironic19:49
mwhahahaOr we don't config ironic right19:49
EmilienMsshnaidm: can I use your recheck patch later in the evening to try promotion? or too riskyt?19:49
sshnaidmEmilienM, sure19:50
sshnaidmEmilienM, in worst case it'll just fail, no risks19:50
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Convert "convert_image.yml" task list to role  https://review.openstack.org/41254619:50
sshnaidmEmilienM, but do "rebase" there, please, so it'll pick up all last fixes19:51
*** gfidente|afk has quit IRC19:51
*** sai is now known as sai-pto19:51
EmilienMsshnaidm: ack19:52
*** jaosorior has quit IRC19:52
EmilienMbnemec, mwhahaha: https://review.openstack.org/41260119:52
EmilienMbnemec: do you prefer to revert things in git repos? or maintain your current patches and merge it?19:53
EmilienMbnemec: I'm fine either way.19:53
bnemecEmilienM: I'd prefer to revert in repo.  The ci temprevert only helps with ci.19:53
bnemecBut I also understand that it was a significant change to puppet-mistral, so if we think we can come up with a fix reasonably fast I could be convinced to carry the temprevert.19:55
EmilienMmwhahaha, bnemec: https://review.openstack.org/41260219:55
EmilienMbnemec: +2 https://review.openstack.org/#/c/412541/ -- feel free to remove the -119:56
*** rcernin has joined #tripleo19:56
openstackgerritMerged openstack/instack-undercloud: Add gnocchi support on undercloud  https://review.openstack.org/39299219:59
*** fultonj has quit IRC19:59
*** fultonj has joined #tripleo20:01
*** scorcoran_ is now known as scorcoran20:02
EmilienMsshnaidm: wdyt about adding scenarios into the promotion pipeline?20:04
EmilienMsshnaidm: it would increase the coverage wrt the number of services we test during a promotion20:04
sshnaidmEmilienM, agree, but I need to investigate it, have no strong opinion right now.20:05
EmilienMsshnaidm: they should be pretty stable, and add value20:06
sshnaidmEmilienM, let's start from trying to add them as periodic jobs?20:07
EmilienMsshnaidm: yeah, +120:07
sshnaidmEmilienM, ok20:07
EmilienMsshnaidm: and see their rate of success over 1 week or 220:07
EmilienMmaybe wait for the break20:07
sshnaidmsure20:08
*** sai-pto is now known as sai-air20:08
*** jkilpatr_ has quit IRC20:09
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968820:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]20:10
*** scorcoran is now known as scorcoran_food20:10
*** mhenkel has joined #tripleo20:12
mhenkelhey alll20:12
EmilienMmhenkel: o/20:13
mhenkelquick question: is there any infrastructure in tripleo to enable heat plugins in the overcloud?20:13
EmilienMmhenkel: can you be more specific please? what do you want to achieve?20:13
mhenkelso I have a heat plugin for the Contrail SDN controller20:14
mhenkelbasically what I need to do is to 1. install a package 2. add some lines to /etc/heat/heat.conf20:14
mhenkeland that needs to happen before heat-engine is started or it needs to trigger a restart of heat-engine20:15
mhenkelwe have that kind of infrastructure already for neutron. here I can add a plugin20:15
EmilienMwhat does this plugin?20:15
EmilienMis the code upstream?20:15
mhenkelit provides additional resources20:15
*** thrash is now known as thrash|biab20:16
mhenkelhttps://github.com/Juniper/contrail-heat20:16
*** dbecker has quit IRC20:17
*** amoralej is now known as amoralej|off20:17
mhenkelthis is basically the sequence I need to execute in puppet:20:18
mhenkelhttp://pastebin.com/T6uLvFPc20:18
mhenkelI can hack it into my neutron plugin (and I did it as a proof of concept) but it doesn't feel right that way;)20:18
*** jkilpatr_ has joined #tripleo20:22
EmilienMmhenkel: you might want to contact shardy about this thing20:25
mhenkel;)20:25
EmilienMmhenkel: I'm not sure where this could should live now20:25
mhenkelsure, I will. Thanks!20:26
*** ccamacho has quit IRC20:30
*** newmember has joined #tripleo20:30
*** sshnaidm is now known as sshnaidm|afk20:33
EmilienMbnemec:  Overcloud pingtest SUCCEEDED20:33
EmilienMbnemec: nonha is not so broken :D20:33
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Revert "Switch mistral to use authtoken configuration"  https://review.openstack.org/41261020:34
bnemecEmilienM: Nice.  Just realized we also need ^20:35
bnemecOtherwise we're going to break when we revert the puppet-mistral change.20:35
*** scorcoran_food has quit IRC20:35
EmilienMbnemec: not sure20:36
EmilienMbnemec: because it passed CI20:36
EmilienMbnemec: because it's a ironic/mistral problem20:37
*** scorcoran_food has joined #tripleo20:37
EmilienMbnemec: we don't support ironic on overcloud iirc20:37
bnemecEmilienM: That patch hadn't merged yet when the revert started ci.20:37
bnemecIt's the same problem as the undercloud revert though - once we revert puppet-mistral we lose all mistral config.20:37
bnemecI guess it will only break the scenario job that uses mistral though.20:38
EmilienMbnemec: which is voting :D20:38
*** shinobu_ has quit IRC20:38
EmilienMwe need to merge THT first20:39
EmilienMthen undercloud20:39
EmilienMand then the puppet-mistral20:39
EmilienMiiuc20:39
bnemecOrder doesn't matter much for the first two, but yeah that's basically my understanding too.20:39
*** nyechiel has joined #tripleo20:39
EmilienMyeah ok20:40
EmilienMbnemec: I blocked the puppet-mistral, just in case :D20:40
bnemecYeah20:40
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart-extras: Add modify-images role  https://review.openstack.org/41199520:40
bnemecWorst case scenario, we break all the jobs for a little while until all the reverts land.20:40
bnemecSince nonha is broken until that happens anyway, it's not a huge deal.20:41
*** weshay is now known as weshay_relocatin20:42
*** ccamacho has joined #tripleo20:46
*** bana_k has quit IRC20:48
*** jkilpatr_ has quit IRC20:48
*** chem has quit IRC20:50
*** chem has joined #tripleo20:50
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata  https://review.openstack.org/40483120:50
*** jkilpatr has joined #tripleo20:51
*** social has joined #tripleo20:56
*** jpena|away is now known as jpena20:57
*** fultonj has quit IRC21:07
*** flepied has joined #tripleo21:07
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968821:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]21:10
*** ooolpbot has quit IRC21:10
*** scorcoran_food is now known as scorcoran21:13
*** morazi has quit IRC21:14
*** ccamacho has quit IRC21:15
*** egafford has joined #tripleo21:19
*** abregman has joined #tripleo21:23
openstackgerritAde Lee proposed openstack/instack-undercloud: Add code to support novajoin in the undercloud  https://review.openstack.org/39922021:26
*** fultonj has joined #tripleo21:33
*** scorcoran has quit IRC21:33
*** scorcoran has joined #tripleo21:35
*** nyechiel has quit IRC21:40
*** scorcoran has quit IRC21:41
*** liverpooler has quit IRC21:41
*** flepied has quit IRC21:45
*** jcoufal has quit IRC21:47
*** shinobu|afk has quit IRC21:48
*** fragatina has joined #tripleo21:48
*** shinobu_ has joined #tripleo21:48
*** thrash|biab is now known as thrash21:49
*** saneax-_-|AFK is now known as saneax21:51
*** scorcoran has joined #tripleo21:54
*** fragatina has quit IRC21:54
*** fragatina has joined #tripleo21:55
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: mysql: remove 'test' default database  https://review.openstack.org/41262521:56
openstackgerritSteve Baker proposed openstack/tripleo-common: Use packaged docker-cmd hook in heat-agents  https://review.openstack.org/40782721:56
openstackgerritSteve Baker proposed openstack/tripleo-common: Install python-heat-agent-docker-cmd in overcloud-full  https://review.openstack.org/41262621:56
*** ansmith has quit IRC21:56
*** fragatina has quit IRC21:58
*** fragatina has joined #tripleo21:59
*** Goneri has quit IRC21:59
*** trown is now known as trown|outtypewww21:59
*** dprince has quit IRC22:02
*** panda is now known as panda|Zz22:03
*** milan has quit IRC22:05
*** abregman has quit IRC22:05
*** fragatina has quit IRC22:06
EmilienMbnemec: in fact, puppet-mistral gate was solid enough to block the revert before we first revert in tripleo https://review.openstack.org/#/c/412602/22:07
openstackgerritMerged openstack/instack-undercloud: Revert "Switch mistral to use authtoken configuration"  https://review.openstack.org/41254122:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968822:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]22:10
*** lblanchard has quit IRC22:10
*** mhenkel has quit IRC22:10
bnemecYeah, I saw that.22:10
EmilienMbnemec: I'll debug the mitaka failures tomorrow :D22:11
*** bana_k has joined #tripleo22:11
*** chem has quit IRC22:12
*** chem has joined #tripleo22:12
*** tzumainn has quit IRC22:15
*** jpena is now known as jpena|off22:26
*** fultonj has quit IRC22:27
*** almondjoy has quit IRC22:27
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632422:29
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256722:30
*** jmelvin has quit IRC22:33
*** scorcoran has quit IRC22:35
*** paramite has quit IRC22:36
*** paramite_ has quit IRC22:36
*** jkilpatr has quit IRC22:36
openstackgerritMerged openstack/puppet-tripleo: Disable legacy ceilometer api by default  https://review.openstack.org/40921822:39
*** cdearborn has quit IRC22:46
*** b00tcat has quit IRC22:46
*** dsneddon_ has joined #tripleo22:48
openstackgerritMerged openstack/diskimage-builder: Generate ssh-hostkeys on boot for ironic agent  https://review.openstack.org/38348122:55
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256722:57
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras: Modify OVB role to work with non-ready undercloud images  https://review.openstack.org/41256722:58
bnemecEmilienM: Opened a bug for the multinode timeouts: https://bugs.launchpad.net/tripleo/+bug/165126723:01
openstackLaunchpad bug 1651267 in tripleo "Timeouts in multinode jobs" [Critical,Triaged]23:01
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Release 5.5.0 (newton)  https://review.openstack.org/41264123:01
EmilienMbnemec: thanks23:01
bnemecI'm suspecting this might be related to the firewall issues we're having in ovb too.23:01
bnemecI think the firewall might be dropping packets in some of those jobs for some reason.23:01
EmilienMbnemec: should we enable ssl?23:01
*** egafford has quit IRC23:01
bnemecI don't know, but I suspect it wouldn't help in this case.  I think it was dumb luck that it helps with ovb.23:01
*** pradk has quit IRC23:01
EmilienMlol23:02
bnemecI think it's blocking ovs packets this time instead of geard packets.23:02
EmilienMmwhahaha: please review https://review.openstack.org/#/c/412641/ when you can (quick)23:02
EmilienMbnemec: interesting. Would be nice to log iptables errors somewhere if not done already23:02
EmilienMat least for debug in CI23:02
bnemecIt's telling us what it's dropping: iptables dropped: IN=eth0 OUT= MAC=fa:16:3e:45:3d:89:fa:16:3e:7b:cb:4f:08:00 SRC=10.16.8.189 DST=10.16.8.201 LEN=78 TOS=0x00 PREC=0x00 TTL=64 ID=1696 DF PROTO=UDP SPT=51940 DPT=4789 LEN=5823:03
bnemecBut I don't think it should be because I thought we opened that port in the multinode jobs.23:03
*** bfournie has quit IRC23:03
bnemecMaybe there's a race between when we configure the firewall and something else messing with our config?23:03
* bnemec is totally speculating here23:03
*** egafford has joined #tripleo23:04
openstackgerritEmilien Macchi proposed openstack/tripleo-ui: Release 1.1.0 (Newton)  https://review.openstack.org/41264223:05
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632423:06
EmilienMbkero: have you seen my comment on https://review.openstack.org/#/c/410391/ ?23:07
bkeroEmilienM: Yes, thank you for such a quick response. I'll fix that now.23:10
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968823:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed]23:10
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras: WIP: Add generic undercloud setup role  https://review.openstack.org/40632423:12
*** rlandy is now known as rlandy|bbl23:13
EmilienMbkero: thx23:13
bkeroEmilienM: updated, was this more what you were thinking?23:16
*** jkilpatr has joined #tripleo23:16
EmilienMbkero: commented. Still not ok23:21
openstackgerritgreghaynes proposed openstack/diskimage-builder: Switch py34 tests to py35  https://review.openstack.org/41264823:22
bkeroEmilienM: Bah, sorry. I was going off of an ovb review for adding. Do you think check-tripleo-jobs:experimental is the right place?23:24
EmilienMbkero: in which pipeline does it go?23:24
bkeroEmilienM: I would estimate the experimental pipeline now and check/gate later23:27
EmilienMbkero: yeah check-tripleo-jobs:experimental sounds good.23:27
bkeroEmilienM: ok, new review up23:28
EmilienMcool23:30
openstackgerritMerged openstack/tripleo-heat-templates: Revert "Switch mistral to use authtoken configuration"  https://review.openstack.org/41261023:32
openstackgerritMerged openstack/tripleo-heat-templates: Remove unused attr from templates  https://review.openstack.org/40777623:32
*** bana_k has quit IRC23:34
*** bana_k has joined #tripleo23:35
sshnaidm|afkEmilienM, bkero I'm afraid it'll make some mess with options if you use nonha and multindoe together in job name23:35
sshnaidm|afkEmilienM, bkero nonha and ha mostly are for ovb jobs..23:35
EmilienMsshnaidm|afk: we already do that23:35
EmilienMgate-tripleo-ci-centos-7-nonha-multinode23:36
sshnaidm|afkEmilienM, oh, cool, didn't see it23:36
EmilienMsshnaidm|afk: when https://review.openstack.org/#/c/412602/ and is packaged in RDO, we could promote I think23:39
EmilienMsshnaidm|afk: I'll take care of it tonight23:39
EmilienMsshnaidm|afk: I'll email you if I have a blocker23:39
EmilienMbut we should have a promotion tonight23:39
sshnaidm|afkEmilienM, ok, but it'll run automatically soon23:40
sshnaidm|afkI think in 6.00 UTC or near this23:41
*** bfournie has joined #tripleo23:47
*** scorcoran has joined #tripleo23:49
openstackgerritMerged openstack/os-collect-config: Updated from global requirements  https://review.openstack.org/40090723:51
*** mhenkel has joined #tripleo23:58
« Sunday, 2016-12-18 Index Tuesday, 2016-12-20 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!