Wednesday, 2017-07-12

*** pliu_ has quit IRC00:00
*** samdoran has joined #tripleo00:00
*** ecerquei has joined #tripleo00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339100:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170359900:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1703599 in tripleo "Containers multinode job deploying an empty overcloud" [Critical,In progress] - Assigned to Jiří Stránský (jistr)00:10
*** jbadiapa_ has quit IRC00:17
*** jbadiapa_ has joined #tripleo00:32
*** fragatin_ has joined #tripleo00:43
*** fragatina has quit IRC00:46
*** fragatin_ has quit IRC00:47
*** fragatina has joined #tripleo00:48
*** dmacpher has quit IRC00:50
*** samdoran has quit IRC01:00
*** dixiaoli has joined #tripleo01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339101:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170359901:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1703599 in tripleo "Containers multinode job deploying an empty overcloud" [Critical,In progress] - Assigned to Jiří Stránský (jistr)01:10
*** itlinux has joined #tripleo01:15
*** samdoran has joined #tripleo01:20
*** cnf has quit IRC01:21
*** thrash is now known as thrash|g0ne01:26
openstackgerritMerged openstack/tripleo-quickstart-extras master: Make test_regex simpler for tempest  https://review.openstack.org/48193301:28
itlinuxhello do we have pcs as default when we are doing 3 controllers.. or I need to add an option for the deploy overcloud? TY01:30
*** atoth has quit IRC01:32
*** dmacpher has joined #tripleo01:35
*** limao has joined #tripleo01:39
openstackgerritMerged openstack/tripleo-docs master: Update No Valid Host doc link  https://review.openstack.org/48170901:40
*** fzdarsky_ has joined #tripleo01:51
*** fzdarsky|afk has quit IRC01:53
*** yamahata has quit IRC02:00
*** ecerquei has quit IRC02:01
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339102:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170359902:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)02:10
openstackLaunchpad bug 1703599 in tripleo "Containers multinode job deploying an empty overcloud" [Critical,In progress] - Assigned to Jiří Stránský (jistr)02:10
*** jschlueter has quit IRC02:11
*** jschlueter has joined #tripleo02:12
*** jmelvin_ has joined #tripleo02:15
*** pliu_ has joined #tripleo02:18
*** fragatin_ has joined #tripleo02:18
*** jmelvin_ has quit IRC02:20
*** fragatina has quit IRC02:22
openstackgerritMerged openstack/tripleo-quickstart master: Escapes username and password in repo_setup script template  https://review.openstack.org/47895902:28
*** achadha_ has quit IRC02:28
*** achadha has joined #tripleo02:29
*** itlinux has quit IRC02:31
*** achadha_ has joined #tripleo02:33
*** achadha has quit IRC02:33
*** fragatin_ has quit IRC02:33
*** fragatina has joined #tripleo02:34
*** fragatina has quit IRC02:34
*** fragatina has joined #tripleo02:35
*** achadha_ has quit IRC02:37
*** ramishra has joined #tripleo02:45
openstackgerritSteve Baker proposed openstack/instack-undercloud master: Disable undercloud nova metadata server  https://review.openstack.org/40040702:46
*** yamahata has joined #tripleo02:49
*** zzzeek_ has quit IRC02:52
*** samdoran has quit IRC02:58
*** achadha has joined #tripleo03:00
*** achadha has quit IRC03:06
*** achadha has joined #tripleo03:08
*** itlinux has joined #tripleo03:09
*** ramishra has quit IRC03:10
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339103:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170359903:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)03:10
openstackLaunchpad bug 1703599 in tripleo "Containers multinode job deploying an empty overcloud" [Critical,In progress] - Assigned to Jiří Stránský (jistr)03:10
openstackgerritMerged openstack/tripleo-heat-templates master: Don't confuse Heat with empty parameter_defaults  https://review.openstack.org/48254503:13
openstackgerritMerged openstack/tripleo-common master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47815203:14
openstackgerritMerged openstack/instack-undercloud master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47826303:14
openstackgerritMerged openstack/tripleo-image-elements master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47826103:14
openstackgerritMerged openstack/tripleo-puppet-elements master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47826203:14
openstackgerritMerged openstack/tripleo-heat-templates master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47815303:14
openstackgerritMerged openstack/tripleo-quickstart-extras master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47826703:15
openstackgerritMerged openstack/tripleo-quickstart master: Switch from oslosphinx to openstackdocstheme  https://review.openstack.org/47826603:15
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: Generate docker-centos-tripleoupstream.yaml with prepare  https://review.openstack.org/48273903:16
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: Use a single configuration file for specifying docker containers.  https://review.openstack.org/47862803:16
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: Remove DockerNamespace references  https://review.openstack.org/47939803:16
*** achadha has quit IRC03:23
*** itlinux has quit IRC03:23
*** achadha has joined #tripleo03:23
*** rlandy has quit IRC03:23
*** gbarros has quit IRC03:28
openstackgerritMerged openstack/puppet-tripleo master: Do not fail if PCI device is missing  https://review.openstack.org/47850303:28
*** udesale has joined #tripleo03:44
*** gkadam has joined #tripleo03:47
*** itlinux has joined #tripleo03:54
openstackgerritOpenStack Proposal Bot proposed openstack/instack-undercloud master: Updated from global requirements  https://review.openstack.org/48279103:58
openstackgerritAmrith Kumar proposed openstack/diskimage-builder master: the current 08-debootstrap doesn't allow cross platform development  https://review.openstack.org/48279204:00
itlinuxhello all04:03
itlinuxdoes the HA version come with pcs.. or do I need to add a different template for it?04:03
itlinuxthanks04:03
*** tonyb has quit IRC04:05
*** sshnaidm|afk has joined #tripleo04:08
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339104:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)04:10
*** ooolpbot has quit IRC04:10
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/48279304:10
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48279404:10
*** ykarel has joined #tripleo04:12
*** mdnadeem has joined #tripleo04:13
*** achadha has quit IRC04:17
*** achadha has joined #tripleo04:18
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy kubernetes using TripleO on the overcloud  https://review.openstack.org/47175904:26
*** tonyb has joined #tripleo04:28
*** fragatin_ has joined #tripleo04:31
*** psachin has joined #tripleo04:31
*** fragatina has quit IRC04:34
*** nyechiel_ has joined #tripleo04:34
*** fragatin_ has quit IRC04:35
*** dsariel has joined #tripleo04:41
*** pcaruana has joined #tripleo04:41
*** fragatina has joined #tripleo04:44
*** ramishra has joined #tripleo04:47
*** fragatina has quit IRC04:48
*** coolsvap has joined #tripleo04:56
*** pcaruana has quit IRC04:58
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339105:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)05:10
*** ooolpbot has quit IRC05:10
*** fragatina has joined #tripleo05:14
*** dpawar has joined #tripleo05:18
*** fragatina has quit IRC05:18
*** cylopez has joined #tripleo05:28
*** rcernin has joined #tripleo05:33
*** ratailor has joined #tripleo05:36
*** pgadiya has joined #tripleo05:39
*** janki has joined #tripleo05:40
*** lmiccini has joined #tripleo05:48
*** itlinux has quit IRC05:52
*** nyechiel_ has quit IRC05:55
*** prateek has joined #tripleo05:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260706:00
*** aufi has joined #tripleo06:01
*** iranzo has joined #tripleo06:02
*** iranzo has joined #tripleo06:02
*** jprovazn has joined #tripleo06:04
*** ramishra has quit IRC06:04
*** ramishra has joined #tripleo06:06
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339106:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)06:10
*** ooolpbot has quit IRC06:10
*** florianf has joined #tripleo06:11
*** masco has joined #tripleo06:12
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Send ARA statistics to Graphite server  https://review.openstack.org/47988206:14
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: send graphite stats from ARA  https://review.openstack.org/48012106:14
*** anshul has joined #tripleo06:15
*** hjensas has quit IRC06:15
*** tzumainn has quit IRC06:19
*** nyechiel has joined #tripleo06:22
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Use ara generation from collect-logs role  https://review.openstack.org/47854906:26
*** dsneddon has joined #tripleo06:31
*** marios has joined #tripleo06:35
*** dsariel has quit IRC06:37
*** amoralej|off is now known as amoralej06:38
*** jfrancoa has joined #tripleo06:39
*** jfrancoa has joined #tripleo06:40
*** dixiaoli has quit IRC06:41
*** yprokule has joined #tripleo06:43
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove journal binary log  https://review.openstack.org/48281806:44
*** d0ugal has quit IRC06:44
*** openstackgerrit has quit IRC06:48
*** jlinkes has joined #tripleo06:50
-openstackstatus- NOTICE: FYI, ask.openstack.org is down, review.o.o is slow - please have patience until this is fixed06:53
*** jaganathan has joined #tripleo06:56
*** ebarrera has joined #tripleo07:02
*** openstackgerrit has joined #tripleo07:02
openstackgerritMerged openstack/tripleo-quickstart-extras master: remove the tripleo-ui exception for localhost installs  https://review.openstack.org/47622007:02
*** cschwede has joined #tripleo07:02
openstackgerritMerged openstack/tripleo-quickstart-extras master: Ensure the tripleo-ui-config.js file exits prior to modifying  https://review.openstack.org/47661307:02
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Run rsync for Swift without xinetd  https://review.openstack.org/48118307:02
*** suuuper has joined #tripleo07:03
*** udesale has quit IRC07:03
*** ykarel_ has joined #tripleo07:04
*** cylopez has quit IRC07:04
*** limao has quit IRC07:05
*** limao has joined #tripleo07:05
*** ykarel has quit IRC07:07
*** limao has quit IRC07:10
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339107:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)07:10
*** ooolpbot has quit IRC07:10
*** tesseract has joined #tripleo07:11
*** bogdando has joined #tripleo07:12
*** shardy has joined #tripleo07:13
*** skramaja has joined #tripleo07:15
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Fix stackviz collect data  https://review.openstack.org/47455907:20
*** agurenko has joined #tripleo07:24
*** bfournie has joined #tripleo07:27
*** bfournie has quit IRC07:28
*** bfournie has joined #tripleo07:28
*** pcaruana has joined #tripleo07:31
*** jayg is now known as jayg|g0n307:32
*** eck` is now known as eck`gone07:32
*** ffiore has joined #tripleo07:32
*** fzdarsky_ is now known as fzdarsky07:34
*** egonzalez has joined #tripleo07:34
*** radez has quit IRC07:35
*** eck`gone has quit IRC07:36
*** jayg|g0n3 has quit IRC07:36
*** hjensas has joined #tripleo07:36
*** ccamacho has joined #tripleo07:38
openstackgerritMerged openstack/tripleo-heat-templates master: Remove ceilometer apache files on upgrade  https://review.openstack.org/46771307:39
*** pcaruana has quit IRC07:41
*** pcaruana has joined #tripleo07:45
-openstackstatus- NOTICE: Gerrit is going to be restarted, due to low performance07:51
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Convert objectstorage-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262407:53
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Convert blockstorage-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262507:53
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Convert cephstorage-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262607:53
*** jpich has joined #tripleo07:54
*** limao has joined #tripleo07:56
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Refactor setup_docker_host.sh as host_prep_tasks  https://review.openstack.org/46916307:59
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Containerize virtlogd  https://review.openstack.org/46911607:59
-openstackstatus- NOTICE: Gerrit has been successfully restarted07:59
mandrejistr: I rebased your virtlogd change, hope you don't mind07:59
*** zzzeek has quit IRC08:00
jistrmandre: sure, thanks :) i have the rebase done locally too, was just waiting for the CI fix to merge, otherwise i'd have to recheck the chain anyway08:00
*** zzzeek has joined #tripleo08:00
jistrwill submit a rebase of the cinder chain too08:00
owalshmandre: good morning, I'm hitting this kolla bug in my local dev env https://bugs.launchpad.net/kolla/+bug/1701876, do you know if we have the fix in the CI images?08:01
openstackLaunchpad bug 1701876 in kolla "kolla_set_configs doesn't handle permissions and ownerships of config_files" [Undecided,Fix released] - Assigned to Chulmin Kang (chamankang)08:01
*** mcornea has joined #tripleo08:01
mandrejistr: oops sorry I should have checked with you before08:01
jistrmandre: no worries :) it was a very short task anyway08:02
mandreowalsh: let me check what's in the image08:02
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Allow modprobing from cinder-volume container  https://review.openstack.org/47923808:02
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Reintroduce containerized Cinder  https://review.openstack.org/47900108:02
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: LVM in cinder-volume container without udev  https://review.openstack.org/47816408:02
prateekpanda|off, tripleo quickstart is failing for me since the past couple of days because of the following error http://paste.openstack.org/show/615118/08:02
*** sshnaidm|afk has quit IRC08:03
prateektrown, ^^08:04
*** sshnaidm|afk has joined #tripleo08:05
*** achadha has quit IRC08:06
*** achadha has joined #tripleo08:07
mandreowalsh: yeah, the bug is present in the images. I'm going to build new images08:07
owalshmandre: great, thanks08:08
mandreowalsh: just curious, any reason not to use the 'preserve_properties' option?08:08
owalshmandre: yea, it's for the ssh host keys, I need to change them to root:root 060008:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339108:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)08:10
*** ooolpbot has quit IRC08:10
owalshmandre: or I could possible change the gid to the container ssh_keys group (if it exists) and leave them at 064008:11
owalshpossibly08:11
mandreowalsh: gotcha, you can still workaround by using the 'ownership' structure to set permissions after copying the key to the right place08:11
*** achadha has quit IRC08:11
owalshmandre: yea, mounting /etc/ssh to /host-ssh then copying the keys and setting the perms08:12
*** dr_gogeta86 has joined #tripleo08:13
*** dr_gogeta86 has quit IRC08:13
*** dr_gogeta86 has joined #tripleo08:13
owalshmandre: https://review.openstack.org/#/c/471956/19/docker/services/nova-migration-target.yaml@9808:13
panda|offprateek: update to at least https://review.openstack.org/48218708:17
*** ykarel__ has joined #tripleo08:17
*** panda|off is now known as panda08:17
mandreowalsh: hmm, I was thinking we could workaround like http://paste.openstack.org/show/615120/ but the 'permissions' structure doesn't accept a glob08:17
mandreowalsh: I'm rebuilding the images, I'll let you know when it's done08:18
owalshmandre: yea, I tried, then gave up and pushed a new image to my local registry :-)08:18
*** dmacpher has quit IRC08:19
mandreowalsh: great, so at least you have everything you need to test locally08:20
*** jayg|g0n3 has joined #tripleo08:20
*** eck`gone has joined #tripleo08:20
*** ykarel_ has quit IRC08:20
owalshlyarwood: good morning.... got a second to look over the nova container config? I expect we are missing some mounts08:30
*** dbecker has joined #tripleo08:30
openstackgerritDamien Ciabrini proposed openstack/puppet-tripleo master: Fix mysql client config generation with containerized environment  https://review.openstack.org/48155408:31
*** mcornea_ has joined #tripleo08:31
*** lucas-afk is now known as lucasagomes08:32
*** pcaruana has quit IRC08:33
*** snecklifter has joined #tripleo08:34
*** mcornea has quit IRC08:34
*** cnf has joined #tripleo08:36
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Add settings for ovb in rdocloud  https://review.openstack.org/48095008:36
*** limao has quit IRC08:40
*** limao has joined #tripleo08:41
*** limao has quit IRC08:44
*** limao has joined #tripleo08:44
*** dsariel has joined #tripleo08:46
*** pcaruana has joined #tripleo08:46
*** d0ugal has joined #tripleo08:46
*** limao has quit IRC08:48
lyarwoodowalsh: sure, hitting an issue somewhere?08:48
*** limao has joined #tripleo08:48
*** raissa_ has joined #tripleo08:49
owalshlyarwood: no, but I expect we will :-) e.g we don't currently mount /sys/block08:49
jaganathand0ugal, hi, please look into patch https://review.openstack.org/#/c/468989/08:49
openstackgerritMerged openstack/tripleo-validations master: Fixup the undercloud service list used in the validation  https://review.openstack.org/48261508:49
owalshlyarwood: https://review.openstack.org/478504 adds some of the mounts required for SRIOV, do you know of any others we may need e.g for different storage08:50
*** jbadiapa_ has quit IRC08:50
*** katkapilatova has joined #tripleo08:51
lyarwoodowalsh: /sys/class and /sys/block , did we remove a /sys mount or something? I was sure that was there tbh.08:51
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Add settings for ovb in rdocloud  https://review.openstack.org/48095008:51
lyarwoodowalsh: FC also uses /sys/devices08:52
owalshlyarwood: not sure... might be best to map all of /sys08:52
lyarwoodowalsh: yeah I was sure we did tbh, odd.08:52
*** jbadiapa_ has joined #tripleo08:52
*** limao has quit IRC08:53
*** d0ugal has quit IRC08:54
*** d0ugal has joined #tripleo08:55
*** d0ugal has quit IRC08:55
*** d0ugal has joined #tripleo08:55
owalshlyarwood: might not be needed if we mount /dev08:55
*** salmankhan has joined #tripleo08:55
lyarwoodowalsh: so FC and multipath iSCSI both use https://github.com/openstack/os-brick/blob/master/os_brick/initiator/linuxscsi.py#L63 to remove devices08:56
*** tvignaud has quit IRC08:58
lyarwoodowalsh: there are also loads of lookups against sysfs in various other places09:00
*** udesale has joined #tripleo09:02
*** jlinkes_ has joined #tripleo09:02
*** jlinkes has quit IRC09:03
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Track the tempest resources before and after tempest run  https://review.openstack.org/48193909:04
owalshlyarwood: yea, but sysfs is a virtual fs. It seems *most* of the paths exists within the nova_compute container so I think we should be ok09:05
*** snecklifter has quit IRC09:07
owalshlyarwood: I think it may be a non-issue, unless there are some special cases that require an explicit mapping of the host sysfs path09:08
shardyjistr, mandre: hey what's the status of the containers CI jobs today?  Did we land the fixes needed yesterday or are there still some pending?09:09
shardyI see a couple of passes but there's still a lot of red today..09:09
jistrshardy: the deploy job (voting one) is landed, and looking at changes still in zuul, they're all green. It just hasn't made it to cistatus.html yet.09:09
*** jpich has quit IRC09:09
shardyjistr: ack, so it's reasonable to recheck at this point then?09:10
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339109:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)09:10
jistrshardy: the upgrade job fix (non-voting, but should be voting, and proposed to be voting) isn't landed yet. It didn't pass some non-container ocata jobs, it's now on recheck as the failures are surely unrelated.09:10
*** akrivoka has joined #tripleo09:10
jistrshardy: yes it's reasonable to recheck i think09:10
shardyjistr: ack, OK thanks, let me know if I can help at all09:11
*** tvignaud has joined #tripleo09:11
*** jpich has joined #tripleo09:11
openstackgerritFrank A. Zdarsky proposed openstack/os-net-config master: Add schema-based config data validation  https://review.openstack.org/42323009:12
shardyHrm the ovb updates job has been replaced with the intuitively named gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset02409:12
* shardy tries to figure out if that still covers ipv609:12
lyarwoodowalsh: ah cool, so yeah as I said I was sure we had access to this stuff already tbh09:12
mandrejistr: a couple of changes I rechecked after your fix landed still came back with an empty hieradata, did I hit recheck too soon?09:12
mandrejistr: for example https://review.openstack.org/#/c/471157/09:12
* mandre brb09:14
jistrmandre: right, i don't know, it's confusing :) it looks like that change should be green...09:14
jistrmandre: but all recent in zuul looks green so i'm hoping we did fix it indeed, and the previous patches just didn't pick up the fix09:15
jistrhttps://review.openstack.org/476602 will return green, so will https://review.openstack.org/481183, https://review.openstack.org/482102, ...09:17
jistrwe'll see..09:17
owalshmandre, jistr: in https://review.openstack.org/#/c/471157/ jenkins commented ~40 minutes after the recheck but the failing job took over 1 hour... am I misisng sometihng here?09:18
owalshmissing09:18
owalshsomething09:18
*** tosky has joined #tripleo09:19
* owalsh needs a new keyboard09:19
*** salmankhan has quit IRC09:20
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: Generate MySQL client config if service requires database  https://review.openstack.org/48196509:21
*** limao has joined #tripleo09:22
*** salmankhan has joined #tripleo09:23
*** limao has quit IRC09:23
*** limao has joined #tripleo09:24
jistrowalsh: looking at log timestamps, it seems that it was mandre's +A that triggered the check09:24
owalshjistr: ah, missed that...09:25
jistri rechecked it again..09:25
prateekpanda, thanks09:28
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Enhance completion message when upgrading non controller nodes  https://review.openstack.org/48286809:28
*** limao has quit IRC09:28
openstackgerritOr Idgar proposed openstack/tripleo-validations master: adapt neutron sanity check to containerized environment  https://review.openstack.org/47517309:29
*** dciabrin has quit IRC09:31
*** pcaruana has quit IRC09:32
*** pcaruana has joined #tripleo09:32
sshnaidm|afkjistr, seems like containers-multinode continues to fail?09:33
jistrsshnaidm|afk: see recent tripleo-heat-templates patches at http://status.openstack.org/zuul/09:33
*** yamahata has quit IRC09:34
sshnaidm|afkjistr, weird.. it passed in tht, but fails in tripleo-ci09:35
*** athomas has joined #tripleo09:36
jistrsshnaidm|afk: right, weird. Also in t-h-t and puppet-tripleo jobs there seems to have been quite a time window between the fix merging and jobs going green.09:37
jistrsshnaidm|afk: looks like we still need to keep an eye on this09:38
openstackgerritAmrith Kumar proposed openstack/diskimage-builder master: the current 08-debootstrap doesn't allow cross platform development  https://review.openstack.org/48279209:39
*** dciabrin has joined #tripleo09:46
*** limao has joined #tripleo09:48
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: WIP: enable containers periodic job  https://review.openstack.org/47574509:49
*** dpawar has quit IRC09:49
skramajashardy: i have fixed the comments in https://review.openstack.org/#/c/478163/ while fixing an issue for supporting openvswitch 2.7 version. ptal09:49
shardyskramaja: ack looking09:50
skramajathanks09:50
*** snecklifter has joined #tripleo09:51
*** akane has joined #tripleo09:55
*** mcornea has joined #tripleo09:56
openstackgerritTuan Luong-Anh proposed openstack/puppet-tripleo master: Leverage kolla config_files to copy config into containers  https://review.openstack.org/47753509:56
chandankumararxcruz: sshnaidm|afk panda https://review.openstack.org/#/c/481939/09:57
*** mcornea_ has quit IRC09:58
openstackgerritAmrith Kumar proposed openstack/diskimage-builder master: the current 08-debootstrap doesn't allow cross platform development  https://review.openstack.org/48279209:59
*** dpawar has joined #tripleo10:00
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Add support for running crontabs in containers  https://review.openstack.org/47891910:01
*** pliu_ has quit IRC10:03
akaneshardy: Hi. Yesterday we discussed about passing parameters to vendor specific puppet modules using environment files.10:06
akaneshardy: In https://review.openstack.org/#/c/475765/9/roles_data.yaml, instead of calling the composable service "OS::TripleO::Services::CinderBackendVRTSHyperScale" in controller role, I called it in blockstorage role.10:06
akaneshardy: With this I was able to add a check “hiera('cinder_backend_veritas_hyperscale_enabled’)” in https://review.openstack.org/#/c/475762/14/manifests/profile/base/rabbitmq.pp which worked, but I was not able to access any new parameters (veritas_rabbitmy_passwd) I defined in “config_settings” of https://review.openstack.org/#/c/475765/9/puppet/services/cinder-backend-veritas-hyperscale.yaml.10:06
akaneshardy: It looks like the parameters in “config_settings” are accessible only on the role where the composable service is added. Is this understanding correct?10:06
*** ckyriakidou has joined #tripleo10:06
openstackgerritJulie Pichon proposed openstack/instack-undercloud master: Add UI config endpoints for Ironic Inspector  https://review.openstack.org/48288410:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339110:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)10:10
shardyakane: yes, the config_settings are added only to the roles where the service is enabled10:10
shardyakane: there is also service_config_settings and global_config_settings10:10
shardyservice_config_settings wires in config values from any role to e.g the one running keystone (or any service)10:11
shardyglobal_config_settings distributes the data to all roles10:11
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP Make network-isolation-v6 environment rendered for all roles  https://review.openstack.org/47448610:11
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Enhance completion message when upgrading non controller nodes  https://review.openstack.org/48286810:12
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/README.rst#config-settings10:12
shardyakane: ^^10:12
shardyadarazs: Hey do you know if there's a matrix showing the features covered by CI now?  I thought we had one in tripleo-ci which showed e.g ha/ipv6/ssl coverage, but I'm not sure that data is now we moved some ovb jobs to oooq featuresets?10:13
openstackgerritMerged openstack/tripleo-ui master: Plans routing refactor  https://review.openstack.org/47690410:13
openstackgerritMerged openstack/tripleo-ui master: Remove CurrentPlanReducer  https://review.openstack.org/47693910:13
shardyadarazs: it looks like we replaced the ovb updates job with featureset026, but AFAICS that means we no longer test ipv610:14
adarazsshardy: this is the featureset matrix: https://docs.openstack.org/tripleo-quickstart/latest/feature-configuration.html -- it's a bit complicated to match up the jobs with the featuresets for now.10:14
shardywanted to confirm if that is actually the case10:14
shardyadarazs: yeah, tbh I'm not super happy with how complex this has become10:14
shardyadarazs: why can't we just have a quickstart config per job?10:15
adarazsshardy: we sort of do. except for the node configuration.10:15
adarazsshardy: but most featuresets should work with most node configs.10:15
shardyOh wait, we replaced -upates with 024, which does cover ipv610:16
adarazsshardy: so you remember we talked about renaming jobs to contain the node config + featureset, this way you can reproduce them.10:16
shardyand https://review.openstack.org/#/c/481083/ passed it :)10:16
adarazsshardy: yes, the only thing we don't cover is updates.10:16
shardyadarazs: Ok, is there a plan to reinstate that coverage?10:16
adarazsshardy: it's somewhat out of our hand, the lifecycle team is responsible for making those updates and upgrades work, we simply don't have enough people and time on our hand for it and nothing happened in the area for months so we had to go forward somehow.10:17
akaneshardy: thanks!10:18
shardyadarazs: that's not really fair - we've replaced a working updates job with one that doesn't test updates?10:18
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Refactor iscsi initiator-name reset into separate service  https://review.openstack.org/48217010:18
shardyit didn't test rolling minor updates, but it tested stack updates didn't replace resources10:18
shardywhich has been a source of bugs in the past10:18
adarazshmm, I'm not sure if we do that or not.10:19
*** limao has quit IRC10:19
shardyadarazs: FWIW I also think it's not super great to just punt critical release blocker problems to other teams - we need to work together to get coverage of upgrades, updates etc10:20
shardye.g I've repeatedly asked for help with the 3nodes job, because without it we can't test rolling updates10:20
* shardy will try to raise that in the CI squad meeting tomorrow10:20
adarazsshardy: I agree that this is not ideal, but there's really not enough people in our team that can do it.10:22
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Add support for nova live/cold-migration with containers  https://review.openstack.org/47195610:24
*** udesale__ has joined #tripleo10:24
adarazssame with the 3node multinode, we are trying to get somebody to do it since January but there was always something more pressing. :(10:26
adarazsshardy: but I'm happy to discuss priorities on the squad meeting, maybe we really have them wrong.10:27
*** udesale has quit IRC10:27
shardyadarazs: ack, lets discuss it, I appreciate there's always conflicting priorities, but we do need to figure out getting HA and updates coverage w/containers10:29
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add deployment configuration IDs  https://review.openstack.org/47786410:31
adarazsshardy: I think why we're concentrating on getting rdocloud working is so that we can test HA better/more with OVB.10:33
adarazsapart from other things like having promotion more often :)10:33
*** udesale has joined #tripleo10:37
*** dbckz has joined #tripleo10:38
*** jlinkes_ has quit IRC10:38
*** dbckz has quit IRC10:38
*** jlinkes_ has joined #tripleo10:38
*** udesale__ has quit IRC10:38
*** dbckz has joined #tripleo10:38
shardyadarazs: ack, understood - my main concern is that the current CI oooq implementation appears to have assumptions about an all-in-one deployment10:39
shardyand to get good coverage of HA or updates, we really need multi-controller deployments10:39
shardyso it'd be good to get a quickstart expert to take a look and at least understand the work required to fix that10:40
shardyor, we proceed with the existing 3nodes script based approach, which is fine, but I don'd personally want to implement the same coverage twice10:40
adarazsshardy: you're right, we should probably do some investigation at least, because I have no clue either how much work it would be.10:41
shardyadarazs: yeah, both EmilienM and myself looked at it, and it seemed like significant work due to assumptions baked into the current integration with tripleo-ci10:42
shardymaybe I'm wrong though, so would be good to get some other eyes on it10:42
openstackgerritHarry Rybacki proposed openstack/tripleo-heat-templates stable/newton: Use comma_delimited_list for token flush cron time settings  https://review.openstack.org/48290010:43
*** snecklifter has quit IRC10:44
*** udesale has quit IRC10:45
skramajadsneddon: marios: shardy: i have a query regarding upgrade support for ovs2.7 & DPDK. we need to configure the dpdk ports with a additional option for ovs2.7, related os-net-config change - https://review.openstack.org/#/c/480540/10:46
skramajait is not backward comptabile. so during the upgrade we have to set this additional option for exiting dpdk ports10:46
skramajaNetworkDeployment (os-net-config) is run only for create and update10:47
skramaja*not for upgrade10:47
skramajais there any history behind it?10:47
shardyskramaja: yes, at one point there was a bug where running os-net-config caused nics to get restared10:48
skramajai look at how to add this support for upgrade - via os-net-config or upgrade ansible steps10:48
shardywhich may well still be the case10:48
shardyand it caused packemaker to fence nodes on update10:48
skramajakarthiks: ^ fyi10:48
shardyskramaja: there is NetworkDeploymentActions which enables passing e.g [CREATE, UPDATE]10:48
shardybut you'll need to test that with a full HA environment to ensure it works OK10:49
*** d0ugal has quit IRC10:49
shardyI suspect it will on upgrade, since we stop the pacemaker cluster10:49
* karthiks following10:49
shardybut you may need some additional environments or docs to add to the upgrade docs10:49
rooktripleo ci hates me10:50
shardyskramaja: you'd do something like add NetworkDeploymentActions to https://github.com/openstack/tripleo-heat-templates/blob/master/environments/major-upgrade-composable-steps-docker.yaml10:50
shardyskramaja: then set it back to only CREATE in the converge environment10:51
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/environments/major-upgrade-converge.yaml10:51
shardythat may be reasonable for all upgrades, or maybe safer to enable only for dpdk10:51
skramajashardy: i prefer it only for dpdk.10:51
shardyskramaja: Ok10:52
skramajai remember there are some upgrade based workflows under review. i will check if we can include this based on dpdk.10:52
skramajashardy: apart from ha, there is a restart problem also which you mentioned is still there. is is specific to any type of deployment or nic configs?10:54
skramajathere are lot of work done to avoid network being restarted when upgradeing ovs.10:55
skramajadont want to hit the same problem. i will add all those cases for validating.10:55
shardyskramaja: it's probably best to chat with dsneddon - he pushed https://review.openstack.org/#/c/152732/ in 2015 but it never landed10:55
skramajashardy: thanks. i will follow up.10:55
shardyAFAIK there is still the risk of bouncing interfaces on update, particularly if there are any changes at all to the config10:55
skramajathanks shardy10:57
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy kubernetes using TripleO on the overcloud  https://review.openstack.org/47175910:57
mariosskramaja: shardy yeah os net config will run every time but th epoint is we don't support changes to network config so the interfaces stay up (well with the ovs workarounds too)10:57
mariosshardy: so even if skramaja passes the new templates with the xtra parameter, and overrides it, i think that would cause the interface to bounce10:57
mariosshardy: i mean" overrides " so that the network deployment is done on update too10:58
*** pkovar has joined #tripleo11:03
*** dciabrin has quit IRC11:05
*** jkilpatr has quit IRC11:06
skramajamarios: but we are giving only "create" action for Network Deployment. will it run os-net-config during upgrade too?11:07
*** rhallisey has joined #tripleo11:07
skramajamarios: there is no additional parameters. during upgrade, the ifcfg file will be update with an new command for OVS_EXTRA11:07
skramaja*for DPDK requirement of ovs2.711:07
*** snecklifter has joined #tripleo11:09
mariosskramaja: OK (yeah i was referring to overriding that with the UPDATE), we don't explicilty run os-net-config during the upgrade process no - the only thing we've had to carry so far was the ovs upgrade workaround11:09
*** Vijayendra has quit IRC11:09
openstackgerritCarlos Camacho proposed openstack/tripleo-quickstart-extras master: Fix task name in tripleo-ui.yaml  https://review.openstack.org/48290711:09
*** dciabrin has joined #tripleo11:09
mariosskramaja: right, not parameters, but i assume you will need to add 'pci_address' or sthing? (from https://review.openstack.org/#/c/480540/3/os_net_config/utils.py ) into your network templates?11:09
*** lucasagomes is now known as lucas-hungry11:10
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339111:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)11:10
*** ooolpbot has quit IRC11:10
skramajamarios: its internally done in the os-net-config. as of now nic config will have only dpdk0 associated with interface name. additionally we will find the pci address internally and write it to ifcfg file11:11
*** pkovar has quit IRC11:11
shardymarios: ack, but during a major upgrade, you could theoretically bounce nics, because we already stopped the pcmk cluster?11:11
*** ckyriakidou has quit IRC11:11
shardye.g overriding NetworkDeploymentActions for this specific case, only during the upgrade, might work?11:11
mariosshardy: yeah assuming it doesn't break the upgrade (the ovs special case was needed otherwise you lost connectivity and the upgrade hung for example)11:12
shardymarios: yeah, needs testing for sure11:13
mariosshardy: ok cool, so all you need then is the new version of os net config?11:13
marioserr skramaja  i mean11:13
skramajamarios: yes.11:13
mariosskramaja: as in, you aren't changing you templates/nic config at all11:13
skramajamarios: exactly no nic changes.11:13
skramajamarios: ovs special case is needed from ovs2.5 to ovs2.6 version.. it has been solved in the ovs package for ovs2.6 to ovs2.7..11:14
mariosskramaja: well, it was needed for 2.4 to 2.5 as well (just slightly different :) )11:14
skramaja:) ok11:14
mariosskramaja: and yeah in theory we shouldn't need it if you are already on 2.6 going to 2.711:14
mariosskramaja: but that is still tbd/tested/we are cautious ;)11:15
skramajaagreed.11:15
*** jlinkes__ has joined #tripleo11:15
*** ckyriakidou has joined #tripleo11:16
shardydtantsur: Hey, just wondering, are you aware of anyone else seeing problems with the vbmc processes randomly dying?11:16
skramajamarios: shardy thanks for your inputs. we will start validating the upgrade in quite a bit. these information are really user ful.11:16
shardydtantsur: it's always on delete, and there's no logs giving clues as to why11:17
shardyrestarting the vbmc systemctl service fixes the issue but it's kind of annoying11:17
*** jlinkes_ has quit IRC11:18
*** atoth has joined #tripleo11:18
*** ansmith_ has quit IRC11:19
*** ansmith has quit IRC11:19
jistrsshnaidm|afk: the container multinode job does look much greener now, it just took a surprisingly long time for the change to take effect..11:19
*** dpawar has quit IRC11:21
*** jlinkes__ is now known as jlinkes11:22
*** amoralej is now known as amoralej|brb11:22
*** dpawar has joined #tripleo11:23
*** brandor5 has joined #tripleo11:23
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: DNM - hijack 3nodes job to test docker+HA  https://review.openstack.org/48291311:23
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Remove non-containerized pacemaker resources on upgrade  https://review.openstack.org/48020211:24
shardydciabrin: ^^ FYI due to uncertainty around the oooq/3nodes plan I'll iterate on that and we can hopefully prove the docker+HA pieces11:24
shardywhen it's working we can decide if we add another job, or wait for quickstart integration w/3nodes11:24
shardydciabrin: I was wondering what the status is of https://review.openstack.org/#/c/469372/ - could you rebase it so the DependsOn works?11:25
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: PlanList and PlanCard updates  https://review.openstack.org/47694211:27
*** ratailor has quit IRC11:27
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Fix Logout and authentication loop  https://review.openstack.org/47842411:27
*** liverpooler has quit IRC11:31
*** salmankhan has quit IRC11:32
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Containerize Manila Share service  https://review.openstack.org/47152711:32
*** shardy is now known as shardy_lunch11:32
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: PlanList and PlanCard updates  https://review.openstack.org/47694211:35
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Fix Logout and authentication loop  https://review.openstack.org/47842411:35
*** agurenko has quit IRC11:36
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove legacy and unused code from tripleo-ci  https://review.openstack.org/48292911:39
*** agurenko has joined #tripleo11:41
*** jkilpatr has joined #tripleo11:42
dciabrinshardy_lunch, hey. I'm out for lunch. the review you're mentioning is going to be replaced with https://review.openstack.org/#/c/48196511:44
*** florianf has quit IRC11:45
dciabrinshardy_lunch, that review also takes care of non-containerized services (cinder-sheduler?) on containerized overclouds. and it's supposed to work with minor upgrades11:45
*** ecerquei has joined #tripleo11:48
*** morazi has joined #tripleo11:48
dtantsurshardy_lunch: I've heard about it, yes.. never had time to play myself11:53
*** pradk has joined #tripleo11:55
*** florianf has joined #tripleo11:59
*** pkovar has joined #tripleo12:00
*** milan has joined #tripleo12:01
*** rbowen has joined #tripleo12:01
*** oidgar has joined #tripleo12:03
*** dprince has joined #tripleo12:05
*** salmankhan has joined #tripleo12:06
pradkmwhahaha, not sure if you saw my comment in the patch, goneri tried the patch and did a deploy and still failed the same way12:06
*** sbrzozow has joined #tripleo12:06
mwhahahapradk: do we have logs?12:06
pradknot sure how he applied, still need to chat with him today12:06
pradknot yet, waiting for him to get online12:06
*** jtomasek_ has joined #tripleo12:07
*** lucas-hungry is now known as lucasagomes12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339112:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)12:10
*** ooolpbot has quit IRC12:10
*** shardy_lunch is now known as shardy12:10
*** dpawar has quit IRC12:10
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates master: Revert "Don't confuse Heat with empty parameter_defaults"  https://review.openstack.org/48294012:10
shardydciabrin: ack thanks, I'll give https://review.openstack.org/#/c/481965 a try12:10
shardydtantsur: ack, OK well I seem to be reprodcing it quite regularly, so let me know if there's any debug I can try to help find the issue12:11
*** ecerquei has quit IRC12:11
*** pchavva has joined #tripleo12:13
*** jayg|g0n3 is now known as jayg12:14
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Revert "Add a new docker centos Heat environment"  https://review.openstack.org/48294312:14
*** ffiore has quit IRC12:14
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Revert "Add a new docker centos Heat environment"  https://review.openstack.org/48294312:14
openstackgerritPaul Belanger proposed openstack/diskimage-builder master: Bump fedora-minimal DIB_RELEASE 26  https://review.openstack.org/48257012:15
atothshardy, beagles, another newbe question, I finally have a stable generic overcloud deployed, can I just use the "openstack overcloud update" command and pass it the neutron-lbaasv2.yaml env file to enable the service on the existing deployment? or do I have to tear down the overcloud and redeploy it?12:15
*** udesale has joined #tripleo12:16
shardyatoth: you just re-run the overcloud deploy command again12:17
shardyatoth: don't use the overcloud update command, as it's only really intended for package minor updates atm12:17
shardyatoth: you can delete before re-running the deploy, but if you don't then the existing deployment is updated12:17
*** ansmith has joined #tripleo12:18
atothshardy, thanks, great info :-)12:18
shardyit's generally a good idea to test that both situations do what you expect12:18
*** MVenesio has joined #tripleo12:18
atothshardy, by both you mean clean install and update install?12:18
*** assassin has quit IRC12:19
shardyatoth: yeah, as you'll need it to work in both scenarios12:19
atothshardy, ack, will do, thanks12:19
shardyin theory they work the same, but occasionally there are issues on update which aren't evident on initial deploy12:19
atothshardy, good to know12:19
*** rlandy has joined #tripleo12:20
openstackgerritMatthias Runge proposed openstack/tripleo-heat-templates master: Make collectd run as root inside the container  https://review.openstack.org/48294512:22
*** Vijayendra has joined #tripleo12:22
*** jayg has left #tripleo12:22
*** jtomasek_ has quit IRC12:25
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Send ARA statistics to Graphite server  https://review.openstack.org/47988212:25
openstackgerritMartin Mágr proposed openstack/tripleo-common master: [WIP] Add HEALTHCHECK directive to Kolla override  https://review.openstack.org/48294812:25
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Implement Zaqar logger adapter  https://review.openstack.org/45929412:26
pkovaris this doc published anywhere? ttps://github.com/openstack/python-tripleoclient/tree/master/doc/source/command-objects12:26
*** kbyrne has quit IRC12:28
*** kbyrne has joined #tripleo12:30
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: [WIP] Add health check for sensu-client container  https://review.openstack.org/48295012:31
dtantsurshardy: a bug and a reproducer would be nice12:31
*** zzzeek_ has joined #tripleo12:32
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Support configurable Zaqar backends  https://review.openstack.org/47755912:33
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop MongoDB from the undercloud  https://review.openstack.org/47756212:33
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Download logs interface  https://review.openstack.org/47393312:33
*** dciabrin has quit IRC12:34
*** pgadiya has quit IRC12:35
shardydtantsur: yeah, I don't have a reproducer other than to say a process dies randomly every few overcloud deletes12:36
dtantsur:(12:36
shardydtantsur: I'll see if I can perhaps run the vbmc processes in a shell and catch any output when they die12:36
dtantsurcool12:36
dtantsurI thought we introduced logs recently..12:36
shardydtantsur: yea there are logs, just no clues why it exitted12:37
shardyso maybe running on the shell will expose an exciting backtrace with clues ;)12:37
*** kbyrne has quit IRC12:38
*** thrash|g0ne is now known as thrash12:38
akaneshardy: Hi. If I put config parameters like http://paste.openstack.org/show/615147/, I am able to access them in vendor specific modules which get called from profiles in puppet-tripleo https://review.openstack.org/#/c/475762/14/manifests/profile/base/rabbitmq.pp.12:38
akaneshardy: I am wondering whether I need to check in the changes mentioned in http://paste.openstack.org/show/615147/. It looks like user has flexibility to add any parameters to an environment file.12:39
*** kbyrne has joined #tripleo12:39
*** dk_ has joined #tripleo12:39
*** milan is now known as milan|mtg12:39
sshnaidm|afkshardy, can you please vote? https://review.openstack.org/#/c/479615/12:40
*** v1k0d3n_ has joined #tripleo12:40
*** hrybacki_ has joined #tripleo12:40
*** suuuper has quit IRC12:40
*** nyechiel has quit IRC12:40
*** chandankumar has quit IRC12:41
*** dk__ has quit IRC12:41
*** tbarron_ has joined #tripleo12:41
*** suuuper has joined #tripleo12:41
*** nyechiel has joined #tripleo12:41
*** zbitter has joined #tripleo12:41
*** v1k0d3n has quit IRC12:41
*** hrybacki has quit IRC12:41
*** dprince has quit IRC12:41
*** zaneb has quit IRC12:41
*** numans has quit IRC12:41
*** tbarron has quit IRC12:41
*** melwitt has quit IRC12:41
*** vkmc has quit IRC12:41
*** v1k0d3n_ is now known as v1k0d3n12:41
*** hrybacki_ is now known as hrybacki12:41
*** chandankumar has joined #tripleo12:41
*** melwitt has joined #tripleo12:41
*** mrunge has quit IRC12:41
*** tbarron_ is now known as tbarron12:41
*** numans has joined #tripleo12:41
*** melwitt is now known as Guest8620812:41
*** mrunge has joined #tripleo12:42
*** vkmc has joined #tripleo12:42
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove legacy and unused code from tripleo-ci  https://review.openstack.org/48292912:42
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372812:43
*** gkadam has quit IRC12:43
*** kbyrne has quit IRC12:44
*** ecerquei has joined #tripleo12:44
*** achadha has joined #tripleo12:45
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: WIP: Adjust MTUs in multinode workflow on RDO Cloud  https://review.openstack.org/48122212:45
*** kbyrne has joined #tripleo12:46
*** dciabrin has joined #tripleo12:46
*** dprince has joined #tripleo12:47
*** ecerquei_ has joined #tripleo12:48
*** liverpooler has joined #tripleo12:49
*** samdoran has joined #tripleo12:49
*** ecerquei has quit IRC12:50
*** achadha has quit IRC12:50
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Fix ironic-pxe startup issues  https://review.openstack.org/48134312:54
*** chem has quit IRC12:55
*** samdoran has quit IRC12:56
*** tzumainn has joined #tripleo12:56
*** chem has joined #tripleo12:56
*** samdoran has joined #tripleo12:57
*** d0ugal has joined #tripleo12:57
*** jmelvin has joined #tripleo12:58
atothshardy, ok I've found a couple bugs in the patch, as we know the patch set was already merged as I've been taking a few liberties with the processes as I'm figuring it all out, so should I just --amend the same patch or open a new patch for the fixes?12:58
*** Goneri has joined #tripleo12:59
shardyatoth: well if it already merged you'll have to post a new patch :)13:00
atothshardy, that's what I thought but was hoping otherwise :-)13:02
*** dpawar has joined #tripleo13:02
shardyatoth: yeah just post a follow-up with the fixes needed, unless we revert it there's no way to rewrite history ref the patch that landed13:03
*** oidgar has quit IRC13:04
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop ComputeServices from environments/docker.yaml  https://review.openstack.org/48174713:04
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: Remove package if service stopped and disabled  https://review.openstack.org/47988613:04
skramajaEmilienM: could you take a look at this patch - https://review.openstack.org/#/c/478163/?13:04
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372813:04
adarazspblaho: you got this merged, congrats: https://review.openstack.org/478959 :)13:05
pblahoadarazs: haha, thanks13:05
*** lblanchard has joined #tripleo13:05
*** chem has quit IRC13:05
shardyakane: yes you'll need to check in changes like http://paste.openstack.org/show/615147/ as the user environment file can't directly influence service_config_settings13:06
*** lblanchard has quit IRC13:06
*** lblanchard has joined #tripleo13:06
shardyakane: if you want the user to change the username/password which I guess is likely, you'll need to add parameters to the services/*.yaml template and change the hard-coded strings to {get_param: NameOfParameter}13:06
shardythen users can put NameOfParameter: something into the environment file13:07
openstackgerritDan Prince proposed openstack/puppet-tripleo master: Add insecure_registry_address to docker profile  https://review.openstack.org/47941213:08
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372813:08
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339113:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)13:10
*** ooolpbot has quit IRC13:10
openstackgerritmathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow  https://review.openstack.org/46376513:12
shardymwhahaha: Hey when you get a sec could you pls check https://review.openstack.org/#/c/481083/ - I'm a little confused why the two unit test results are different, as despite the unit-latest test failures looking related, I'm not clear why the unit-4.5 test passed13:12
*** zzzeek_ has quit IRC13:12
adarazspanda, trown: can we merge this? https://review.openstack.org/47548013:13
*** zzzeek_ has joined #tripleo13:13
*** ffiore has joined #tripleo13:13
sshnaidm|afkadarazs, can you approve this please? https://review.openstack.org/#/c/479615/13:14
openstackgerritMerged openstack/tripleo-validations master: Remove the pre-update group and add a post-upgrade group  https://review.openstack.org/48166713:14
* adarazs looks13:15
fultonjdougbtv13:15
*** mdnadeem has quit IRC13:16
fultonj(sorry typo)13:16
brandor5Hello everyone: After having a deploy almost complete yesterday (stupid error in postconfig killed it) I've been getting the following errors during deployment http://sprunge.us/SUaA I think it may be this bug? https://bugzilla.redhat.com/show_bug.cgi?id=1437499 Any hopes of this being backported? or is there another workaround I'm missing?13:17
openstackbugzilla.redhat.com bug 1437499 in puppet-swift "TripleO/Director deployment fails if Swift rings rebalanced within within min_part_hours" [Unspecified,Closed: errata] - Assigned to cschwede13:17
dougbtvnp :)13:17
*** amoralej|brb is now known as amoralej13:17
pandaI loathe doc configuration in featuresets :/13:17
adarazssshnaidm|afk: reviewed, needs doc line for the featureset, but otherwise it's okay.13:17
mwhahahashardy: the lastest test are broken, https://bugs.launchpad.net/puppet-openstacklib/+bug/170296413:17
openstackLaunchpad bug 1702964 in puppet-openstacklib "latest puppet expects catalog to contain resources to for requires and subscribes" [Medium,In progress] - Assigned to zhongshengping (chdzsp)13:18
mwhahahashardy: just ignore them for now in puppet-tripleo since we don't deploy puppet513:18
shardymwhahaha: ack thanks - I wasn't sure as the failures looked related to the code I'd touched changing ipv6 list normalization13:19
*** d0ugal has quit IRC13:19
*** oidgar has joined #tripleo13:20
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add featureset for Heat convergence job  https://review.openstack.org/47961513:24
sshnaidm|afkadarazs, fixed13:25
*** skramaja has quit IRC13:25
openstackgerritHarry Rybacki proposed openstack/tripleo-heat-templates stable/newton: Run token flush cron job hourly by default  https://review.openstack.org/48258513:26
shardybrandor5: Hi!  AFAICS that fix has been backported already to ocata/newton - what release are you using?13:27
shardybrandor5: https://review.openstack.org/#/q/I03dd075c590344a920099c57b6a7530f103e0ede,n,z13:27
*** achadha has joined #tripleo13:27
brandor5shardy: I'm using newton13:27
brandor5shardy: working on verifying if the fix is in place now13:27
brandor5(i don't think it is)13:27
shardybrandor5: ack, the fix is https://review.openstack.org/#/c/472253/ so I guess you can confirm by inspection if it's applied13:28
brandor5yea13:28
*** zzzeek_ has quit IRC13:29
*** eck`gone is now known as eck`13:29
*** achadha has quit IRC13:31
*** fragatina has joined #tripleo13:31
*** oidgar has quit IRC13:32
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share service  https://review.openstack.org/47152713:33
brandor5shardy: yeah, it's not there13:34
*** ansmith has quit IRC13:34
*** zzzeek_ has joined #tripleo13:34
*** fragatina has quit IRC13:36
*** agopi has joined #tripleo13:37
brandor5shardy: I've got puppet-swift-9.5.0-1.el7ost installed13:41
*** trown is now known as trown|brb13:42
brandor5it may be more appropriate for me to ask over in RDO13:43
adarazsshardy: can you re-review this please? https://review.openstack.org/47961513:44
*** oidgar has joined #tripleo13:45
*** coolsvap has quit IRC13:45
*** trown|brb is now known as trown13:47
*** social has quit IRC13:47
*** ansmith has joined #tripleo13:50
pradkGoneri, Hi, so that patch dint fix the issue? how did you apply the patch to puppet-tripleo?13:55
Goneripradk, manually: http://paste.openstack.org/show/615165/13:56
pradkGoneri, thats on undercloud?13:56
Goneriyes it is13:57
pradkGoneri, hmm dont you have to do upload-puppet-modules -d puppet-modules for yhis change to reflect?13:58
*** deadnull has joined #tripleo13:58
*** ckyriakidou has quit IRC13:58
brandor5just to close the loop here, the upstream work for the patch I need has been completed and just hasnt landed in rhosp yet. I've opened a support request with red hat to try and nudge that along. Thanks to everyone for the help!13:59
shardysamdoran: Hey, I have a question re dynamic includes if you have a moment13:59
pradkGoneri, I usually follow shardy's blog .. http://hardysteven.blogspot.com/2016/08/tripleo-deploy-artifacts-and-puppet.html .. see example 213:59
samdoranshardy: I'd love to clarify that. It's kinda tricky.13:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260714:00
shardysamdoran: hehe - so I looked at https://docs.ansible.com/ansible/include_module.html because we have some tasks which vary depending on user inputs14:00
pradkGoneri, i dont think manually patching undercloud file will apply the change to overcloud deployment14:00
shardysamdoran: I'm wondering, can I write a file with tasks on the target system, then include them at runtime?14:00
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy kubernetes using TripleO on the overcloud  https://review.openstack.org/47175914:00
shardysamdoran: in particular, it'd also be useful to loop over the includes, because blocks don't support loops yet14:00
*** dparkes has joined #tripleo14:00
*** deadnull has quit IRC14:01
shardysamdoran: but also because it'd enable us to keep the main playbook generic and just include the dynamically generated bits14:01
*** dbecker has quit IRC14:01
*** jcoufal has joined #tripleo14:02
shardysamdoran: FYI this is the generic playbook https://github.com/openstack/tripleo-heat-templates/blob/master/docker/deploy-steps-playbook.yaml14:02
shardysamdoran: but we also generate one here https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-steps.j2#L16214:02
shardyI'd like the latter to run the former14:02
shardyso we can do the configuration in one ansible-playbook run14:02
*** dbecker has joined #tripleo14:02
samdoranSo you want to have a task file exist on the remote system, then pick them up and run them?14:03
*** janki has quit IRC14:03
shardybut the dynamic stuff doesn't exist on the host that runs ansible-playbook (unless we stick to the current model, where heat runs ansible against localhost, but I'm trying to make this work from the undercloud instead)14:03
samdoranThis sounds somewhat dangerous.14:03
shardysamdoran: yes exactly14:03
samdoranWorried about code injection?14:03
shardysamdoran: the tasks would be assembled by heat as part of the tripleo deployment14:04
shardysamdoran: saying this is an anti-pattern is totally valid, I'm just wondering how else we might do it :)14:04
shardysamdoran: basically we assemble a list of tasks based on which services (encapsulated by heat templates) are enabled14:04
*** ykarel__ is now known as ykarel14:05
shardyand we don't know that until the deploy starts14:05
shardysamdoran: this is one of the tricky areas where our current composition model doesn't fit super well with ansible14:05
shardye.g roles etc14:05
shardysamdoran: I suppose we could write e.g a tripleo_controller role on the host system, and include that?14:06
shardysince roles can be baked into the host filesystem depending on the path?14:06
*** jlinkes has quit IRC14:06
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-steps.j2#L18014:07
samdoranIn general, sprinkling task files on remote systems, then ingesting them and running them seems overly complex.14:07
*** panda is now known as panda|bbl14:07
shardysamdoran: ^^ is the tricky part because that list of tasks isn't know until runtime14:07
samdoranI think it'd be fine to generate the task files on the control node, then pick them up.14:08
shardysimilar lists exist for e.g upgrade tasks etc14:08
samdoranThat would work fine.14:08
Goneripradk, so I've just to do a: upload-puppet-modules -d /usr/share/openstack-puppet ?14:08
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Test the example nic configs we actually ship  https://review.openstack.org/47670414:08
shardysamdoran: heh, I was afraid you'd say that ;)14:08
shardysamdoran: the issue is the heat model doesn't control the state of the control node14:08
samdoranIs the list of tasks generated for each deployment highly variable? Or is it a pretty regular group of tasks?14:09
shardysamdoran: it's highly variable as operators can select any arbitrary list of services on each group of nodes (role)14:09
shardysamdoran: we basically take the list of services, and assemble a list of tasks (same with puppet configuration data, it's just merged together then we run it)14:10
*** ykarel is now known as ykarel|afk14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339114:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)14:10
samdoranshardy: So we generate a list of tasks based on configured services on a target system?14:10
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/aodh-api.yaml#L15014:11
shardysamdoran: yes, on deployment we take a list of services, e.g above perhaps the operator said aodh-api will run on the Controller role14:11
*** akrivoka has quit IRC14:11
*** lmiccini has quit IRC14:11
shardysamdoran: in that case we run the tasks from the aodh-api.yaml host_prep_tasks during the deployment on the nodes in the Controller group14:11
samdoranWondering if we could just set custom facts on the managed node, then have the playbook run from the control node make decisions based on local facts.14:12
*** akrivoka has joined #tripleo14:12
shardysamdoran: you mean assemble a list of all possible tasks, then conditionally run them?14:12
shardywe do have flags that say if each service is enabled, but right now we don't have a way to generate a list of all possible tasks14:13
samdoranOr conditionally run roles based on local facts.14:13
*** hjensas has quit IRC14:13
shardysamdoran: ack, so we'd need to somehow translate the heat template with tasks into roles on each node14:13
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: WIP: Adjust MTUs in multinode workflow on RDO Cloud  https://review.openstack.org/48122214:14
pradkGoneri, yea so you need to check out the patch into a directory puppet-modules/tripleo14:14
*** ykarel|afk has quit IRC14:14
pradkGoneri, where tripleo == puppet-triplo checkout with patch14:14
*** paramite has quit IRC14:15
pradkGoneri, then you upload that with upload-puppet-modules to update the artifacts in swift14:15
samdoranThis still seems a bit convoluted, though. Because the end goal is a playbook that runs a custom list of roles on a node.14:15
GoneriI would prefer to just patch the existing /usr/share/openstack-puppet directory14:15
pradkGoneri, once thats done, your povercloud deploy will grab that tar and untar and use those modules for overcloud deployment14:15
*** dbckz has quit IRC14:15
shardysamdoran: yeah, right now the requirement is just to run a custom list of tasks14:15
pradkGoneri, may be that works, i havent tried that.. shardy ^^ ?14:16
samdoranWhat's variable is the list of roles to run on each host.14:16
shardysamdoran: it exists within the heat model, so we'd need to write the joined list of tasks somewhere if they aren't run via heat14:16
*** masco has quit IRC14:16
shardysamdoran: today heat creates a playbook dynamically, then runs it on each node in localhost mode14:17
samdoranAh.14:17
shardybut clearly that's not a super great pattern from a pure ansible perspective14:17
samdoranNo control node.14:17
*** arxcruz has quit IRC14:17
samdoranDoes Heat run locally on each node as well? I think I'm imaging a Heat control node of some sort.14:17
shardysamdoran: yeah, well there kind of is, it's the undercloud, but right now we don't run ansible from it for this case14:17
shardysamdoran: yeah there's a heat control node (which could run ansible, we just don't do it like that atm), it's the undercloud14:18
shardythat runs heat, which then passes data to each node in the deployed "overcloud"14:18
shardyso e.g overcloud-controller-0 polls heat running on the undercloud14:18
shardyit gets some metadata, which includes a generated playbook14:18
*** arxcruz has joined #tripleo14:18
pradkGoneri, the key is we need to update the artifacts in swift so its using the updated patched repo14:18
shardyand there's some scripts on the node that knows how to run ansible-playbook14:18
shardyand report back if it worked14:19
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: Fix `when` statement in deployment image validation  https://review.openstack.org/48297914:19
Goneripradk, yep, I got it. A new deployment is ongoing.14:19
samdoranSo this is running Ansible in an inverted fashion.14:19
pradkGoneri, cool14:19
* samdoran starting to understand a bit14:19
shardysamdoran: yeah, it's kind of a pull model14:19
shardysamdoran: which I know isn't really the normal way to do ansible things14:20
shardysamdoran: but it fits well with the Heat model for node configuration14:20
shardysame with puppet, we run puppet apply on each node, with no puppet master14:20
shardyheat on the undercloud acts as the master essentially14:20
samdoran🤔14:21
samdoranSo essentially you want Heat to generate a playbook, custom to each node, that will be run directly on each node by Ansible?14:22
shardysamdoran: yes, or at least I'm wondering if it's possible14:22
shardyas it would be a stepping stone to enabling running ansible-playbook outside heat more easily14:23
shardysamdoran: basically I just want an include that does late binding and looks on the target system instead of the control node14:24
shardysamdoran: alternatively, can task includes reference an URL?14:24
*** zzzeek has quit IRC14:24
*** lmiccini has joined #tripleo14:24
samdoranYes, that's possible. I'm wondering if it would be better to have the same set of roles or each node, i.e., checkout a git repo, and just generate a playbook custom to that node.14:24
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Make featureset numbers links to the definition  https://review.openstack.org/48248014:25
*** gbarros has joined #tripleo14:25
shardysamdoran: yeah long term that probably would be better, but right now we're missing a way to translate the heat templates into roles on the nodes14:25
shardye.g the git checkout is heat templates not roles we can consume14:26
shardyhence me looking for a workable albeit hacky interim step14:26
shardy:)14:26
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Make featureset numbers links to the definition  https://review.openstack.org/48248014:26
*** ansmith has quit IRC14:27
samdoranshardy: This is starting to make sense.14:29
shardysamdoran: great! :)14:30
samdoranshardy: So you're asking if you can grab a task file from a URL rather than generate it directly on the client?14:30
shardysamdoran: yes if that's possible it'd be a way to solve this I think14:30
shardywriting it to the remote system would be easier in the TripleO model, but I posted https://review.openstack.org/#/c/469039/ which would enable Heat to write a task file to swift14:31
shardywhich can expose it via a URL14:31
*** fultonj has quit IRC14:32
shardyI'll try a few things though, maybe it'd be cleaner to just add some way to write the data to the control node14:32
samdoranshardy: Include must point to a file. You would use `get_url` to download the file, then `include` it.14:32
shardyjust trying to understand the options at this point really14:32
shardysamdoran: ack, thanks, I'll give it a try, thanks for the help!14:32
*** dparkes has quit IRC14:33
*** fultonj has joined #tripleo14:33
samdoranSo something generates the task file and makes it available on a URL, then your playbook can be get_url and include.14:33
shardysamdoran: Ok nice, if I revive that heat patch that should work14:34
samdoranAnd you can capture the output of get_url and pass that to the include if you want.14:34
samdoranFrom there, you just have to have ansible installed on all the nodes and have something actually run the playbook.14:34
samdoranYou could have a whole bunch of files using the system hostname in the file.14:35
shardysamdoran: yeah we have ansible installed, and we can run it from the undercloud14:35
shardyand we have a dynamic inventory where we can put per role/node data14:35
shardye.g ansible-playbook -vvv -b -i /usr/bin/tripleo-ansible-inventory tripleo-heat-templates/docker/deploy-steps-playbook.yaml --limit overcloud-controller-0 --extra-vars "step=2 docker_puppet_debug=''"14:35
shardythat allows us to run the static playbook I linked14:35
shardyjust not the other dynamically generated part14:35
samdoranThen you get_url task could specify url: http://someservec.com/{{ ansible_hostname }}-playbook.yml14:35
shardyideally I'd like to combine them14:36
shardysamdoran: yup, I think it only needs to be per role actually, so e.g Controller-playbook.yaml14:36
*** itlinux has joined #tripleo14:37
shardywe can use vars to control any per-node differences, the main differences in tasks comes from the services enabled per role (group of nodes)14:37
dprinceshardy, dtantsur, jistr: Ironic PXE has been functionally broken for weeks https://review.openstack.org/#/c/481343/14:37
samdoranshardy: And Controller would have a certain set of tasks/Ansible roles in its playbook.14:37
*** zbitter is now known as zaneb14:37
dtantsurdprince: :(14:37
shardysamdoran: yup exactly - derived from those host_prep_tasks (and some other things contained in the heat model) I linked14:37
dtantsurdprince: we need to merge derekh's CI job ASAP14:37
dtantsurotherwise it will keep getting broken14:38
dprincedtantsur: its a simple fix, but we have no CI on this yet14:38
*** udesale has quit IRC14:38
*** ykarel has joined #tripleo14:38
dprincedtantsur: switching the Undercloud to containers will fix this too fwiw. I like undercloud testing for Ironic as it fails faster14:38
dtantsurtotally14:38
dtantsurdprince: the change LGTM, but I don't have +2 on THT14:38
dprincedtantsur: ack, thanks14:39
shardydprince: thanks for the heads up, +214:39
dprinceshardy: thanks14:39
shardysounds like we don't need to wait for CI if this isn't tested?14:39
*** ansmith has joined #tripleo14:40
*** suuuper has quit IRC14:40
dtantsurdprince, shardy: can we get https://review.openstack.org/#/c/474802/ moving please?14:40
shardyor do we have a job that at least starts the service?14:40
dtantsurironic is porbably the only service that is not covered by even scenarious...14:40
dtantsurshardy: undercloud-containers14:40
*** ansmith_ has joined #tripleo14:41
shardydtantsur: ah yeah of course we run it there, just don't pxe anything14:41
dtantsurright14:41
dprincedtantsur: I noticed it locally as I'm using a containers undercloud to bring up a containers overcloud14:42
dprincedtantsur: but I only re-spin my UC every few days....14:42
samdoranshardy: But to your original question on includes, given the decentralized approach and wanting to keep the "main playbook generic", you could put all the arbitrary task files in a directory on each node in a specific directory, and you can pick them all up using with_fileglob.14:43
dtantsurthat's why people make CI jobs :)14:43
dprincedtantsur: right, step at a time. Sometimes it takes time to move things forward... not because its hard but people need to warm up to the new approach14:45
shardysamdoran: Aha! That's actually exactly what I was looking for I think, thanks! :)14:45
dtantsuryeah, sure.. so many things to do :)14:45
dtantsurstill, please review derek's stuff :)14:45
dprincedtantsur: do you have a link?14:45
dtantsurdprince: https://review.openstack.org/#/c/474802/ is one of them14:45
samdoranshardy: with_fileglob is relative to the control node (node running Ansible), which is why it's not always useful. But in this case, it would be since you're running Ansible in a decentralized fashion.14:45
dtantsurah, already approved14:45
samdoranSo if you have Heat drop all the files to run in a standard dir, the playbook could simply be a with_fileglob to run what's in the folder.14:46
shardysamdoran: oh, wait - we're running it decentralized right now, but I'm trying to enable a centralized model14:46
shardywhere we run ansible-playbook from one control node14:46
samdoranBut just make sure those files are locked down to prevent unauthorized editing to prevent code injection.14:46
samdoranSo you want to use Ansible in push mode to pick up arbitrary task files on the managed nodes?14:47
shardysamdoran: yes14:47
shardyor have some other way to inject a variable list of tasks14:48
samdoranIn that case, I would use find and an include with_items.14:48
* samdoran writing quick example14:49
*** itlinux has quit IRC14:49
openstackgerritBrent Eagles proposed openstack/puppet-tripleo stable/ocata: Do not fail if PCI device is missing  https://review.openstack.org/48298914:51
*** psachin has quit IRC14:54
*** dtantsur is now known as dtantsur|afk14:55
*** pliu_ has joined #tripleo14:55
openstackgerritThomas Herve proposed openstack/tripleo-heat-templates master: Stop Heat WSGI services on docker upgrade  https://review.openstack.org/48299114:55
*** milan|mtg has quit IRC14:55
*** bfournie has quit IRC14:58
samdoranshardy: https://gist.github.com/samdoran/a6e75762b64e7ddaaa8c84f99796a9cc14:59
*** prateek has quit IRC14:59
brandor5what steps do i need to take after I've updated my overcloud-full.qcow2 image?14:59
brandor5do i need to delete out the old images and then reupload?15:00
samdoranshardy: This will include task files, not playbooks.15:00
shardybrandor5: you can do openstack overcloud image upload --update-existing15:01
shardysamdoran: thanks!  Looking15:01
*** rcernin has quit IRC15:01
*** pliu_ has quit IRC15:02
brandor5shardy: thanks, I'll try that15:02
brandor5will I need to rerun "configure boot"?15:02
shardysamdoran: Ok, so just to confirm, if the include wasn't a variable, ansible would try to resolve the path on the control node, but because it's a variable it'll try to resolve it on the target host?15:02
*** zzzeek_ has quit IRC15:02
shardybrandor5: only if the kernel or initrd changed I think15:02
brandor5ok, thanks!15:02
samdoranshardy: If you need to run roles, you would then use include_role in your task file.15:03
* samdoran doing Ansible triage.15:03
shardysamdoran: ack, thanks for the info15:04
*** zzzeek has joined #tripleo15:04
*** abishop has joined #tripleo15:06
*** jfrancoa has quit IRC15:06
trowndoes anyone know what changes the 3nodes job runs on? or is it just in check-experimental?15:07
*** atheurer has quit IRC15:08
*** ebarrera has quit IRC15:09
weshayshardy, ^15:09
*** ckyriakidou has joined #tripleo15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339115:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)15:10
*** ooolpbot has quit IRC15:10
EmilienMtrown: in tht15:10
dmsimardsshnaidm|afk: added a comment in https://review.openstack.org/#/c/479882/15:10
trownEmilienM: just all tht changes?15:10
EmilienMtrown: and probably more, you need to check in zuul/layout.yaml15:10
EmilienMyes15:10
trownEmilienM: k thanks15:10
EmilienMit's a -nv job15:10
*** fzdarsky is now known as fzdarsky|afk15:10
trownjust looking for logs from a job15:10
EmilienMtrown: http://tripleo.org/cistatus.html15:11
EmilienMtrown: http://logs.openstack.org/70/482170/5/check/gate-tripleo-ci-centos-7-3nodes-multinode-nv/3d7a828/15:11
trownthanks15:11
dtrainorI see a traceback in my ironic-inspector logs that happens with some regular frequency (every 2s).  I can't help but think it's related to the reason why introspection won't ever put nodes in an 'available' state after 'openstack overcloud node import --introspect --provide nodes.json' (it eventually just times out.  Anyone got any pointers?  http://paste.openstack.org/show/615176/15:12
weshayrasca, have time now15:12
jpichdtrainor: Hi! About https://bugs.launchpad.net/tripleo/+bug/1703596 - neither Ironic nor Ironic-inspector has a proxy endpoint set up at the moment. Do they both need it? Neither? Keystone doesn't either but that might be a special case?15:12
openstackLaunchpad bug 1703596 in tripleo "No nodes visible in the UI after undercloud upgrade" [High,In progress] - Assigned to Julie Pichon (jpichon)15:12
rascaweshay, I ALWAYS have time for you15:13
dtrainorhi.  sorry.  i was just reading your bz response and got side-tracked a bit.  one moment.15:13
shardytrown: It's in the check queue for at least t-h-t, sec let me find it in project-config15:14
gbarrosshardy at your convenience might you be able to take a look at the changes mhenkel did to his patches per your recommendations? https://review.openstack.org/#/c/475045/ https://review.openstack.org/#/c/475048/15:14
shardytrown: it's not experimental, although it is non-voting15:14
shardygbarros: ack will do15:14
gbarrosty shardy. i just kicked off a rechek to have it run through the fixed containers-multinode.15:15
shardygbarros: yup thanks, hopefully those will pass now15:15
jpichdtrainor: No worries, just not sure how to poke at this part since it seems to work anyway (have had no luck setting up a second SSL env so far). If the proxy part isn't set up properly, does that mean the regular ports are used and that might be why it's working for me locally?15:16
jpichAlso, sorry I have no idea about your own inspector issue :/15:17
weshayrasca, I'm on my bluejeans15:17
dtrainorjpich, yes, if you're using non-ssl endpoints then the request is going to go straight to the service endpoint port (non-ssl)15:17
jpichI see15:18
dtrainorsome of the services required both a *-ui-config and a *-ui-proxy, some did not, and I'm trying to remember why15:18
dtrainorjpich, note that in that array, each element has different specifications for the internal and public ssl and non-ssl ports15:19
*** fragatina has joined #tripleo15:19
*** gbarros has quit IRC15:19
*** fragatina has quit IRC15:20
*** prateek has joined #tripleo15:20
*** fragatina has joined #tripleo15:20
*** oidgar has quit IRC15:21
*** gbarros has joined #tripleo15:21
*** anshul has quit IRC15:23
*** athomas has quit IRC15:24
jpichdtrainor: Maybe when the URL was similar/simple enough to not need to change?15:24
*** jaganathan has quit IRC15:25
*** jfrancoa has joined #tripleo15:25
dtrainorThey *all* need *-ui-config, since that's what adjusts tripleo_ui_config.js for you.  Since this is true, the omission of an ironic_inspector-ui-config would explain the actual bug15:26
dtrainoras for other potential missing pieces, that probably warrants more investigation15:26
Goneripradk, I just realized that I should actually patch the openstack-full image15:26
*** atheurer has joined #tripleo15:26
*** atheurer has quit IRC15:27
*** dbecker has quit IRC15:28
*** atheurer has joined #tripleo15:28
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates master: Add Mistral event engine  https://review.openstack.org/46297515:33
*** coolsvap has joined #tripleo15:34
fultonjjistr: aplogies, i misunderstood. thanks for the clarification. i will read that example15:34
fultonjvkmc: ^15:34
*** agurenko has quit IRC15:34
fultonjregarding https://review.openstack.org/#/c/471527/7/docker/services/manila-share.yaml@9715:34
*** yamahata has joined #tripleo15:35
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: PS Cinder: Added support for password less login  https://review.openstack.org/47762815:38
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Ps Cinder: Added support for password less login  https://review.openstack.org/47764115:39
jpichdtrainor: Ack - let's fix one bug at a time then!15:41
*** marios has quit IRC15:42
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: WIP: Adjust MTUs in multinode workflow on RDO Cloud  https://review.openstack.org/48122215:43
*** jfrancoa has quit IRC15:44
*** achadha has joined #tripleo15:45
*** jfrancoa has joined #tripleo15:45
*** ccamacho has quit IRC15:46
*** fragatina has quit IRC15:47
openstackgerritBen Nemec proposed openstack/tripleo-docs master: Split the ha env file and command parameters block  https://review.openstack.org/48301415:47
*** jtomasek_ has joined #tripleo15:47
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share service  https://review.openstack.org/47152715:48
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share service  https://review.openstack.org/47152715:49
*** jcoufal has quit IRC15:49
*** achadha has quit IRC15:49
*** lucasagomes is now known as lucas-afk15:52
fultonjvkmc: nice update on ps9 for https://review.openstack.org/#/c/471527/7..9/docker/services/manila-share.yaml15:52
*** jtomasek_ has quit IRC15:53
EmilienMshardy: it seems that we need to release instack asap15:53
EmilienMslagle: what is our plan about instack? do we still use it?15:54
*** itlinux has joined #tripleo15:54
*** yprokule has quit IRC15:56
*** thrash is now known as thrash|biab15:57
*** egonzalez has quit IRC15:57
*** ffiore has quit IRC15:59
*** bogdando has quit IRC16:01
*** jcoufal has joined #tripleo16:04
*** tesseract has quit IRC16:05
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: allow output for ara logs  https://review.openstack.org/48266016:06
*** achadha has joined #tripleo16:06
bnemecEmilienM: We do still use it.16:06
*** jfrancoa has quit IRC16:06
EmilienMbnemec: ok, thx. I proposed its release for pike16:07
dtrainorworks for me, jpich.  can you fix the ironic-inspector bug?  i'll make time next week to give this a thorough ssl vs non-ssl test to make sure the work i did works as intended.  i did test, but i think having an opportunity to do more testing is warranted.16:07
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share service  https://review.openstack.org/47152716:07
jpichdtrainor: Did you see https://review.openstack.org/#/c/482884/ ? Are you talking about something else?16:07
openstackgerritKarthik S proposed openstack/os-net-config master: OvS 2.7 support - dpdk-devargs needs to provided for DPDK devices  https://review.openstack.org/48054016:08
*** atheurer has quit IRC16:08
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339116:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)16:10
dtrainorOops.  I had not, sorry.  Too many context switches.  Excellent work.16:10
Arilotanhello, i'm trying to follow the Network Isolation documentation but when I am using the deploy command from the doc I have this error :16:10
Arilotanopenstack overcloud deploy: error: unrecognized arguments: --neutron-network-type vlan16:10
jpichdtrainor: No worries! And no problems, thanks for the review :)16:11
dtrainormy pleasure16:11
bnemecArilotan: The cli parameters for those settings were deprecated and removed.  It's a doc bug if they're still referenced.16:11
ArilotanI am following hhttps://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html#deploying-the-overcloud-with-network-isolation16:12
bnemecArilotan: They should be set with Heat parameters now: https://github.com/openstack/tripleo-heat-templates/blob/ccb0655db4cb0df125a52262e67b39609f78703b/environments/network-environment.yaml#L5216:12
Arilotanthank you :)16:13
*** trown is now known as trown|lunch16:13
karthiksdsneddon,   Can you please review https://review.openstack.org/#/c/480540/16:13
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Backport container CI environments to Ocata  https://review.openstack.org/48216716:17
dpawarEmilienM: any idea what could be error here http://paste.openstack.org/show/615184/ ?16:18
openstackgerritMerged openstack/tripleo-heat-templates master: Bind mount needed cert for haproxy for HA too  https://review.openstack.org/48210216:19
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: [WIP] Implements AIDE Intrusion Detection System  https://review.openstack.org/48302516:19
dsneddonkarthiks, Checking16:21
karthiksdsneddon,  Thanks16:21
*** fragatina has joined #tripleo16:21
*** shardy has quit IRC16:25
EmilienMdpawar: you better ask on #rdo - also have you registered on softwarefactory?16:27
EmilienMdpawar: maybe you need to login first on review.rdoproject.org16:27
*** jpich has quit IRC16:27
openstackgerritLuke Hinds proposed openstack/tripleo-heat-templates master: [WIP] Implements AIDE Intrusion Detection System  https://review.openstack.org/48302816:28
*** atheurer has joined #tripleo16:29
dpawarEmilienM: i am able to login to review.rdoproject.org , using my github credentials16:29
*** atheurer has quit IRC16:29
*** atheurer has joined #tripleo16:30
EmilienMdpawar: I would check on #rdo with admins16:32
dpawarEmilienM: it would be really helpfull if admin can reset my passoword.16:33
EmilienMdpawar: I'm pretty sure they can, just go on #rdo and ask16:33
*** Guest86208 is now known as melwitt16:36
weshayjistr, should this -e {{ overcloud_templates_path }}/environments/docker-centos-tripleoupstream.yaml be in all the invocations for containers now?16:36
weshayjistr, the patch only added it when network isolation is not on16:36
weshaymandre, ^16:37
*** raissa_ has quit IRC16:39
weshayre: https://review.openstack.org/#/c/479400/2/roles/overcloud-deploy/tasks/pre-deploy.yml16:39
*** dsariel has quit IRC16:42
*** fragatina has quit IRC16:43
*** fragatina has joined #tripleo16:44
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Adjust MTU on br-ctlplane and make adjustment survive a reboot  https://review.openstack.org/48303216:44
*** tosky_ has joined #tripleo16:53
*** jcoufal has quit IRC16:53
*** tosky is now known as Guest4517716:53
*** tosky_ is now known as tosky16:53
*** Guest45177 has quit IRC16:55
*** pcaruana has quit IRC16:56
*** jtomasek_ has joined #tripleo16:58
*** fragatin_ has joined #tripleo17:01
*** mcornea has quit IRC17:04
*** fragatina has quit IRC17:04
*** aufi has quit IRC17:05
*** jaganathan has joined #tripleo17:05
*** gbarros has quit IRC17:08
*** ebarrera has joined #tripleo17:08
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339117:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)17:10
*** cschwede has quit IRC17:11
*** oidgar has joined #tripleo17:12
*** thrash|biab is now known as thrash17:13
*** ramishra has quit IRC17:13
*** dbecker has joined #tripleo17:16
*** trown|lunch is now known as trown17:18
openstackgerritAna Krivokapic proposed openstack/tripleo-ui master: Add IDs to Nodes page  https://review.openstack.org/48303917:18
*** achadha has quit IRC17:19
*** weshay is now known as weshay|lunch17:20
*** weshay|lunch is now known as weshay17:20
*** bogdando has joined #tripleo17:22
*** yamahata has quit IRC17:25
*** agopi has quit IRC17:27
*** oidgar has quit IRC17:28
*** ykarel has quit IRC17:30
mandreweshay: I suppose the intention was to add the new environment all the time, can you fix it?17:34
weshaymandre, ya..17:35
mandrethx17:36
openstackgerritBen Nemec proposed openstack/tripleo-common master: Add error message if no bare metal nodes available  https://review.openstack.org/45970117:36
weshaymandre, so qq though.. so w/ that change is the overcloud taking containers directly from the registry17:36
weshayand not the undercloud?17:36
*** agopi has joined #tripleo17:37
*** bogdando has quit IRC17:37
*** bogdando has joined #tripleo17:38
pradkGoneri, Hi, any update?17:38
mandreweshay: yeah, unless environments/docker-centos-tripleoupstream.yaml point to tripleoupstream images pulled into a local registry17:38
*** tosky has quit IRC17:38
Goneripradk, I patched the overcloud-full image and restarted a deployment. I should finish soon.17:39
pradkcool17:39
*** iranzo has quit IRC17:40
*** iranzo has joined #tripleo17:40
*** iranzo has joined #tripleo17:40
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add deployment configuration IDs  https://review.openstack.org/47786417:41
*** achadha has joined #tripleo17:44
*** gbarros has joined #tripleo17:46
*** fragatin_ has quit IRC17:47
*** fragatina has joined #tripleo17:47
*** achadha has quit IRC17:48
dmsimardmandre, dprince, Slower: are we using docker (1.12) or docker-latest (1.13) in tripleo ?17:49
dmsimardBoth are available out of CentOS base extras repo17:49
*** achadha_ has joined #tripleo17:49
*** achadha_ has quit IRC17:50
dprincedmsimard: 1.12.617:50
dprincedmsimard: we don't have 1.13 yet17:50
*** achadha_ has joined #tripleo17:50
dmsimarddprince: 1.13 is available, though17:50
dprincedmsimard: in Centos?17:51
dmsimarddprince: yup: http://paste.openstack.org/raw/615195/17:51
dprinceNg: ^^^, fyi17:52
dmsimarddprince: although it's kind of weird, because it allows side-by-side installation with docker, everything is suffixed with -latest17:52
dmsimarddprince: so, config is in /etc/docker-latest, data in /var/lib/docker-latest, etc.17:52
dprincedmsimard: I don't think we want to rework for this at this time17:53
dprincedmsimard: if docker-latest is isolated it doesn't sound like it is ready17:53
dmsimarddprince: I don't know the story17:53
dprincedmsimard: I dunno either17:53
dmsimarddprince: do we know what version downstream is going to ship with ?17:53
*** yamahata has joined #tripleo17:53
*** jaosorior has joined #tripleo17:54
dprincedmsimard: 1.12... something is what RH would use ATM I think17:54
dprincedmsimard: trying 1.13 would be a fun experiment17:54
dprincedmsimard: would require separate puppet modules to configure it, and swap it in17:54
dprincedmsimard: I could ninja it into the undercloud installer pretty easily and see if it worked17:55
dprincedmsimard: is /usr/bin/docker-latest who you run it17:56
dprincehow17:56
dmsimarddprince: so long as the knowledge of "1.13.1 is available" and we decide to go with something else as a conscious decision that's fair17:56
dmsimarddprince: which docker returns /bin/docker which returns 1.13.1 version17:57
openstackgerritBen Nemec proposed openstack/instack-undercloud master: Load Mistral workbooks via Python instead of Bash  https://review.openstack.org/48272317:57
*** Lokesh_Jain__ has joined #tripleo17:57
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Test the example nic configs we actually ship  https://review.openstack.org/47670417:57
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: WIP: Adjust MTUs in multinode workflow on RDO Cloud  https://review.openstack.org/48122217:58
dmsimarddprince: when both "docker" and "docker-latest" are installed, /bin/docker is 1.12.617:58
dprincedmsimard: I'd need to know more about docker-latest. If it is meant to be a faster moving target than docker it may or may not be a good fit17:58
*** salmankhan has quit IRC17:58
Slowerare there any features we need from 1.13?17:58
Slowerseems like 1.12 is working fine for us now..17:59
dprinceSlower: yes. Ng meantioned a feature around "real time" guarantees17:59
Slowerchanging the docker binary name could cause issues17:59
dmsimardSlower: changelog between 1.13.1 and 1.12.6 is available here17:59
*** pkovar has quit IRC17:59
dprinceSlower: that was why I copied him17:59
dmsimardSlower: https://github.com/moby/moby/blob/master/CHANGELOG.md#1131-2017-02-0817:59
Slowerdprince: ah17:59
dmsimardhttps://github.com/moby/moby/blob/master/CHANGELOG.md#1126-2017-01-1017:59
dprinceSlower: apparently corosync might benefit from this featre18:00
dmsimardSlower: if only docker-latest is installed, the binary doesn't change18:00
dmsimardbut some stuff does change.. :/18:00
*** tosky has joined #tripleo18:01
*** jcoufal has joined #tripleo18:01
dmsimarddprince, Slower: there's stuff about overlay2 in 1.13.118:01
Slowerya I see that.. that could be good18:01
dmsimardSlower: but yeah, when using the docker-latest package, directories and systemd units are suffixed with -latest18:02
dmsimardSlower: so systemctl start docker-latest, /etc/docker-latest, etc.18:02
Slowermmm18:02
dmsimardThe binary "docker" is available, as is "docker-latest"18:02
Slowerthat's wierd18:03
dmsimardSlower: I am told it's to allow side-by-side installation with DOcker18:03
Slowerodd packaging choice it seems18:03
Slowerya18:03
dmsimarddprince: lsm5 would be a good person to talk about this I would guess from looking at https://cbs.centos.org/koji/packageinfo?packageID=329118:03
dmsimardI've seen the name around but never personally talked to him18:04
dmsimard(or her?)18:04
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Adjust MTU on br-ctlplane and make adjustment survive a reboot  https://review.openstack.org/48303218:04
dmsimardlsm5 is in #centos-devel18:04
*** ykarel has joined #tripleo18:06
Ngdprince: fwiw, the RT stuff in 1.13.1 isn't anything that matters for HA deployments18:06
Ngdprince: but I believe there is stuff either in, or heading towards 1.13, which helps with RT inside containers18:06
Ngdprince: (and it's corosync that is having a problem with being RT while docker is running on the host)18:07
Ngdmsimard: fwiw, you can edit /etc/sysconfig/docker to change the DOCKERBINARY entry and /usr/bin/docker will be docker-latest :)18:09
dmsimardNg: hah, but you need to edit /etc/sysconfig/docker-latest for docker-latest :P18:09
Ngdmsimard: not to change the client18:09
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339118:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)18:10
*** ooolpbot has quit IRC18:10
Ng /usr/bin/docker is just a script that executes whatever DOCKERBINARY is in /etc/sysconfig/docker18:10
dmsimardNg: the path to the binary is a non-issue when only using docker-latest anyway, it's only a problem if you install both docker and docker-latest18:10
dmsimardNg: because "docker" in $PATH points to the proper 1.13.1 binary when only installing docker-latest18:10
Ngdmsimard: ah interesting, I've only run into this sysconfig file when both are installed18:11
dmsimardNg: oh, you're right, docker-latest does ship a /etc/sysconfig/docker file18:12
*** thrash is now known as thrash|brb18:13
thrash|brbquit18:13
*** jprovazn has quit IRC18:14
*** dpawar has quit IRC18:15
dmsimarddprince: well, just wanted to point that out -- you guys do what you gotta do :)18:15
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48279418:17
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-validations master: Updated from global requirements  https://review.openstack.org/48261918:17
*** ykarel has quit IRC18:19
dprincedmsimard: ack, thanks for the heads up18:19
mwhahahaGoneri: can you provide logs for your reproduction with that patch?18:20
mwhahahaGoneri: specifically i need the messages log from the node that it happened on, trying to see why the ordering didn't get updated18:20
mwhahahaGoneri: this patch https://review.openstack.org/#/c/482707/18:20
Goneriok, I attach the messages18:20
Goneriwhat tracker should I use, the BZ or LP?18:21
mwhahahaeither as long as yout tell me where you put it :D18:21
*** gbarros has quit IRC18:22
Gonerimwhahaha, if you prefer, I can also give you a ssh access18:22
*** gbarros has joined #tripleo18:22
mwhahahaGoneri: that would also work18:22
*** rlandy is now known as rlandy|brb18:24
*** nyechiel has quit IRC18:26
*** gbarros has quit IRC18:27
*** akane has quit IRC18:29
*** jaganathan has quit IRC18:29
*** MVenesio has quit IRC18:32
*** agopi has quit IRC18:39
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci master: TEST: Use v2 net-iso templates in ci  https://review.openstack.org/48306418:41
brandor5is there a better way to run a command during pre_deploy than using str_replace to stamp in a script?18:43
brandor5i literally want to run a single command18:43
mwhahahapradk: so this ceilometer-upgrade thing, it's not keystone returning the 503 it's gnocchi isn't it?18:45
bnemecbrandor5: You can just insert the script directly in the template too, like https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/all-nodes-validation-disabled.yaml#L3618:46
brandor5bnemec: ah, ok let me try that18:46
brandor5I've got a sed command that has lots of special characters and it's failing out during parse... i'm pretty sure because of those characters18:47
mwhahahapradk: nm i found it, it's gnocchi returning a 503 because keystone middleware got a 503 from something18:48
brandor5bnemec: that seems to have done the trick, thank you sir18:53
bnemecbrandor5: Cool18:53
itlinuxhello all.. I see that the networkdeployment takes a long time is that normal? TY18:54
itlinuxdoes anyone know at what stage the bonds are created?18:54
*** gbarros has joined #tripleo18:55
*** agopi has joined #tripleo18:56
*** rlandy|brb is now known as rlandy18:57
*** thrash|brb is now known as thrash18:58
itlinuxis there a way to see what this task is doing ? .NetworkDeployment]: CREATE_IN_PROGRESS  state changed...18:58
*** atoth has quit IRC18:59
*** dbecker has quit IRC19:02
*** ecerquei has joined #tripleo19:02
*** ecerquei_ has quit IRC19:03
*** coolsvap has quit IRC19:03
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170339119:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1703391 in tripleo "Containers upgrade job broken by missing all-nodes-validation-disabled.yaml" [Critical,In progress] - Assigned to Jiří Stránský (jistr)19:10
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates master: Ensure yum cache is ready before update  https://review.openstack.org/48307419:17
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: WIP Testing changes required to move 3nodes job to quickstart  https://review.openstack.org/48307719:23
openstackgerritJohn Trowbridge proposed openstack/tripleo-heat-templates master: DNM testing https://review.openstack.org/483077  https://review.openstack.org/48307819:26
itlinuxhello guys.. I look at teh ovs_bridge in the yaml file and I have an old deployment where I use linux_bond but does not look like it working.. has been stuck on the same spot for a while now.. I see the doc shows  ovs_bond .. any suggestions on that.. I am using cumulus switches. TY19:28
openstackgerritAlex Schultz proposed openstack/puppet-tripleo master: Retry ceilometer-upgrade  https://review.openstack.org/48270719:30
EmilienMbnemec: I think you're right: https://github.com/openstack/networking-bigswitch/blob/9a88aa29b6c849c2d92c24049b5b031b1228fc6e/networking_bigswitch/plugins/bigswitch/plugin.py#L10619:31
*** amoralej is now known as amoralej|off19:35
*** dsariel has joined #tripleo19:36
*** eck` is now known as eck`gone19:37
openstackgerritMerged openstack/tripleo-heat-templates master: Add dependency relationship between nested get_attr targets  https://review.openstack.org/48217219:38
openstackgerritAna Krivokapic proposed openstack/tripleo-ui master: Add IDs to Nodes page  https://review.openstack.org/48303919:38
*** lmiccini has quit IRC19:38
*** ebarrera_ has joined #tripleo19:38
openstackgerritMerged openstack/tripleo-heat-templates master: Remove controller specific bootstack_nodeid  https://review.openstack.org/48114519:39
*** ebarrera has quit IRC19:40
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates master: Start using nic-configs from THT repo instead of tripleo-ci  https://review.openstack.org/47670819:43
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Create undercloud interfaces with multi-nic net-iso  https://review.openstack.org/47582419:44
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Change default network isolation type to multi-nic  https://review.openstack.org/47536319:44
*** akrivoka has quit IRC19:45
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-common master: healthchecks: start to implement container healthchecks  https://review.openstack.org/48308119:47
*** ecerquei_ has joined #tripleo19:47
larsksdprince: https://review.openstack.org/#/c/483081/ as discussed, including a modification to the pep8 check.19:48
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-common master: healthchecks: start to implement container healthchecks  https://review.openstack.org/48308119:49
EmilienMlarsks: doesn't it overlap with https://review.openstack.org/#/c/478228/ ?19:49
EmilienMlarsks: or it works together?19:49
larsksEmilienM: it does not overlap.  The patch I just submitted doesn't require paunch.19:49
EmilienMso why doing that in paunch?19:50
EmilienMI'm confused19:50
*** ecerquei has quit IRC19:50
larsksEmilienM: providing paunch with the ability to modify things at runtime still seems like it might be useful. I don't see a conflict or overlap here.19:50
dprincelarsks: nice, just commented19:52
dprincelarsks: I think we maybe should split this one into two patches so we can do the RPM work before trying to consume the actual healthcheck19:52
larsksdprince: I am happy to split,19:52
larsksJust got copy-and-paste happy with that patch from our call.19:53
dprincelarsks: I dunno19:53
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-common master: healthchecks: start to implement container healthchecks  https://review.openstack.org/48308119:53
dprincelarsks: if it passes CI we could land it, but it does depend on a patch to the RPM I think otherwise it wouldn't actually build a working container19:53
dprincelarsks: also coauthors :)19:54
larsksdprince: if that bothers you I will modify it, but I figure we will all be submitting additional patches.19:54
dprincelarsks: nice on that pep8 test though19:54
dprincelarsks: leave it all contained for now, it demonstrates how it works quite nicely19:54
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-common master: healthchecks: start to implement container healthchecks  https://review.openstack.org/48308119:55
*** florianf has quit IRC19:55
larsksdprince: added co-authored-by...19:55
dprincelarsks: cool, thanks :)19:56
pradkmwhahaha, hmm shouldnt the dependency ensure we ran upgrade before apache restart.. is there something else restarting apache as well?19:58
mwhahahapradk: that only covers the local node, the other controllers are getting restarted19:58
mwhahahapradk: the problem is that the ceilometer-upgrade script is communicating to the haproxy endpoints so any host is in play19:58
pradkmwhahaha, so in the latest run, it passed but otehr controllers are failing?19:58
pradklocal node passed*19:59
mwhahahapradk: no the problem was that it ran before the config but failed because httpd restarted on controller1 (script ran on controller0)19:59
openstackgerritMerged openstack/puppet-tripleo master: Let pacemaker bind-mount needed cert for haproxy bundle  https://review.openstack.org/48213519:59
pradkinteresting19:59
mwhahahapradk: what we should be doing (but i don't think tripleo does this) is setting up the bootstrap controller first and then extending the additional controllers later19:59
mwhahahapradk: the problem is that we're configuring all controllers simulatenously but we have actions that have to occur on a single place and expects everything to be functional the entire time20:00
pradkyeah20:00
mwhahahapradk: so i added retries to ceilometer-upgrade because that's just about the only spot to do it20:00
pradkyea saw that20:00
pradkok20:00
mwhahahaeither keystone middleware has to retry (so gnocchi doesn't 503) or the ceilometer-upgrade script should retry internally or we retry in the deployment20:01
mwhahaharealistically the deployment is the spot we can try first20:01
mwhahahait would be a good idea to raise a bug against ceilometer upgrade to be a bit more resiliant20:01
mwhahahadat cloud native app20:01
pradkmwhahaha, yea we do retries in other upgrades.. since ceilo-upgrade talks to mainly gnocchi, retry could be useful20:02
pradki'll see if we can fix it upstream20:02
*** sbrzozow has quit IRC20:04
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-specs master: add container-healthcheck spec  https://review.openstack.org/46563320:06
*** nyechiel has joined #tripleo20:08
*** liverpooler has quit IRC20:10
*** pkovar has joined #tripleo20:17
EmilienMlarsks: /me back from lunch20:17
larsksEmilienM: ack20:17
EmilienMlarsks: ok I thought both tools were designed for monitoring health for containers20:18
*** ansmith_ has quit IRC20:18
*** ansmith has quit IRC20:18
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add environment file for multinode tests on RDO Cloud  https://review.openstack.org/48122220:18
larsksEmilienM: paunch is a tool for launching containers.  It has (with that patch) options for setting the various --healthcheck* command line options to 'docker run'.  But that's all it does.20:18
*** nyechiel has quit IRC20:19
larsksThat is, health check logic still needs to happen somewhere, whether or not we make use of that feature or use the HEALTHCHECK command in dockerfiles (which is what my patch proposes).20:19
*** jlinkes has joined #tripleo20:19
*** ansmith has joined #tripleo20:20
EmilienMlarsks: makes sense, thx20:21
EmilienMbnemec: please see my comment on https://review.openstack.org/#/c/481726/20:21
*** salmankhan has joined #tripleo20:23
*** jkilpatr has quit IRC20:25
*** eck`gone is now known as eck`20:25
EmilienMthrash: hey, any progress on https://review.openstack.org/#/c/481151/ failures? it sounds like it still doesn't work fine with latest mistral20:25
EmilienMthrash: I wonder if it's still the same issue now, in the latest logs ^ we haven't reached ironic yet20:27
EmilienMthrash: in fact yes, we're failing at introspection like before: http://logs.openstack.org/51/481151/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/22fbfef/logs/undercloud/home/jenkins/overcloud_prep_images.log.txt.gz#_2017-07-11_19_21_4320:28
itlinuxhello all.. is there a way to specify the order of the machine in the rack? Thanks..20:29
bnemecEmilienM: If they can confirm that they just need notifications in _some_ format then that's fine.20:29
bnemecI'm just uncomfortable changing a setting in a third-party driver template that we can't test.20:29
EmilienMbnemec: right. Me too. But from what i've understand, messaging format is abstracted in oslo.messaging20:30
EmilienMbnemec: i've sent an email, let's see what they say20:30
EmilienMbnemec: I'm hoping we can skip messaging v1 which is afik deprecated20:31
EmilienMbnemec: maybe not deprecated but kind of obsolete20:31
bnemecEmilienM: Not according to the docstring: https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/notify/messaging.py#L4820:31
stevebakermorning20:32
bnemecEmilienM: Yeah, I guess I'm not an expert in messaging.20:32
bnemecIt's the one olso project I pretty much stayed away from because it was too complex to pick up in my spare time.20:33
EmilienMfrom what I've seen in http://lists.openstack.org/pipermail/openstack-dev/2014-June/037860.html - it seems like v2 messages are the same, with an optional signature20:33
EmilienMstevebaker: hey yo20:33
bnemecEmilienM: I believe it wraps the payload in an envelope, so if your code is expecting bare data then it could break you.20:34
bnemecAnyway, we'll see what bigswitch says.20:34
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: WIP Testing changes required to move 3nodes job to quickstart  https://review.openstack.org/48307720:34
EmilienMbnemec: I bet they have no idea :D20:34
bnemecEmilienM: Heh20:35
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Support configurable Zaqar backends  https://review.openstack.org/47755920:35
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Generate docker-centos-tripleoupstream.yaml with prepare  https://review.openstack.org/48273920:36
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Use a single configuration file for specifying docker containers.  https://review.openstack.org/47862820:36
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop MongoDB from the undercloud  https://review.openstack.org/47756220:36
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Fix ironic-pxe startup issues  https://review.openstack.org/48134320:36
bnemecEmilienM: I just don't want to get called in at 1 AM because somebody upgraded their bigswitch-based deployment to Pike and it's horribly broken. :-)20:36
mwhahahabigswitch is still fixing newton20:36
mwhahaha:D20:36
mwhahahawe've got a while before they would hit that20:36
bnemeclol20:36
itlinuxstupid question I have all my vlans on the same IP that does not look right am I missing something?20:36
bnemecI think I reviewed that patch.20:37
EmilienMmwhahaha: lol, fair point.20:37
bnemecitlinux: Sounds like you're missing some settings from the network-isolation.yaml environment.20:37
itlinuxwell I have the env yaml file with the info there..20:38
itlinuxshould I add the network-isolation into deploy command?20:38
stevebakerdprince: you're right, since you're there could you squash https://review.openstack.org/#/c/482739 into the next change? CI won't actually pass until it is20:38
dprincestevebaker: on it20:39
*** jkilpatr has joined #tripleo20:41
stevebakerdprince: thanks20:44
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Support configurable Zaqar backends  https://review.openstack.org/47755920:44
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Use a single configuration file for specifying docker containers.  https://review.openstack.org/47862820:44
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop MongoDB from the undercloud  https://review.openstack.org/47756220:44
*** pchavva has quit IRC20:44
*** ansmith has quit IRC20:46
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Use updated multi-nic configs in THT  https://review.openstack.org/48069720:46
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Test the example nic configs we actually ship  https://review.openstack.org/47670420:48
itlinuxI have added the network isolation and use this deploy patter but it fails saying cannot create a flat network.. can someone direct me where I am wrong? Thanks http://paste.openstack.org/show/615210/20:49
itlinuxmuch appreciated the help20:49
*** lblanchard has quit IRC20:50
*** Goneri has quit IRC20:50
*** trown is now known as trown|outtypewww20:54
openstackgerritAlex Schultz proposed openstack/puppet-tripleo stable/ocata: Use correct manage_firewall hieradata  https://review.openstack.org/48309520:54
EmilienMbnemec, mwhahaha : check your inbox, I think we can move forward with messagingv2 :D20:54
weshayanyone know why this has not merged? https://review.openstack.org/#/c/481731/20:55
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: WIP Testing changes required to move 3nodes job to quickstart  https://review.openstack.org/48307720:56
EmilienMweshay: https://review.openstack.org/#/c/481727 hasn't merged20:56
EmilienMweshay: and https://review.openstack.org/#/c/481731/ is on top20:57
bnemecEmilienM: I still think we're making an assumption that bigswitch is actually using oslo.messaging to consume the notifications, but I guess I don't want to block this useful work either.20:58
weshaybah.. thanks20:58
*** cschwede has joined #tripleo20:58
EmilienMbnemec: oh you mean, by an external software?20:59
bnemecEmilienM: Yeah.  I think they have a separate service that runs, which is why they use notifications instead of just hooking directly into the OpenStack services.21:00
bnemecI don't know how that separate service was implemented.21:00
openstackgerritSteve Baker proposed openstack/instack-undercloud master: Disable undercloud nova metadata server  https://review.openstack.org/40040721:00
bnemec(I could be completely wrong about the architecture of their software too - it's been a long time since I had any involvement with them)21:00
openstackgerritFrank A. Zdarsky proposed openstack/os-net-config master: Add schema-based config data validation  https://review.openstack.org/42323021:01
EmilienMbnemec: it's a valid point - they might have some SDN tools which consume these notifications.21:01
*** jtomasek_ has quit IRC21:02
*** jcoufal has quit IRC21:03
openstackgerritDan Prince proposed openstack/tripleo-common master: Add DockerIronicApiConfigImage  https://review.openstack.org/48309621:03
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Support configurable Zaqar backends  https://review.openstack.org/47755921:04
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Use a single configuration file for specifying docker containers.  https://review.openstack.org/47862821:04
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop MongoDB from the undercloud  https://review.openstack.org/47756221:04
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Remove DockerNamespace references  https://review.openstack.org/47939821:04
*** morazi has quit IRC21:06
EmilienMbnemec: sounds like we're good to go (see inbox)21:07
bnemecEmilienM: Cool, so they are using oslo.messaging.  +2A then.21:08
*** abishop has quit IRC21:08
EmilienMbnemec: I'm wondering if the way I did is correct, like defaulting to messagingv2 and override to noop in disable-telemetry21:09
EmilienMbnemec: or if we should disable notifications by default and activate only when telemetry is enabled21:09
EmilienMI'm afraid this ^ wouldn't be backward compatible so we can't do that21:09
EmilienMbut double checking here21:09
bnemecEmilienM: We deploy telemety by default, so we should enable notifications by default too.  IMHO.21:09
EmilienMgood21:09
bnemec*telemetry21:09
EmilienMwfm21:09
bnemecTyping is hard. :-)21:09
EmilienManyway, we provide a way to disable notifications, it's a bonus21:10
bnemecIf we stop deploying telemetry by default then we'll need to change this default too, of course.21:10
bnemecBut currently I think this is the right thing to do.21:10
bnemecWe actually should have fixed this like a year ago when we first hit it in rh1.  I recall us being a little frazzled at the time though because it basically took the cloud down for like a week.21:11
EmilienMok, thx21:12
itlinuxdo we need to add pacemaker or does it come with the HA mode?21:14
EmilienMitlinux: it should be deployed with HA mode21:14
openstackgerritMerged openstack/puppet-tripleo master: Refactor nova migration config into client & target profiles  https://review.openstack.org/47195021:14
openstackgerritMerged openstack/tripleo-quickstart master: Generate docs for periodic jobs  https://review.openstack.org/47548021:14
openstackgerritMerged openstack/tripleo-heat-templates master: Implement scenario006 with Ironic in overcloud  https://review.openstack.org/47480221:15
*** agopi|away has joined #tripleo21:16
itlinuxthanks EmilienM..21:16
*** jlinkes has quit IRC21:17
*** ebarrera_ has quit IRC21:18
*** agopi has quit IRC21:19
*** agopi|away has quit IRC21:19
*** ckyriakidou has quit IRC21:19
itlinuxcan anyone glance through this  http://paste.openstack.org/show/615210/ it fails on me any suggestions? TY21:19
*** dprince has quit IRC21:20
*** pkovar has quit IRC21:22
*** trozet has quit IRC21:26
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-common master: healthchecks: implement service-specific checks  https://review.openstack.org/48310421:28
*** bogdando has quit IRC21:28
bnemecPing to 15.184.64.1 failed. Retrying...21:30
* bnemec rage quits21:30
EmilienMbnemec: where? in master?21:31
bnemecEmilienM: undercloud-containers.  It must have a different location for its environment file.21:31
EmilienMstevebaker: do you know the status of gate-tripleo-ci-centos-7-scenario00X-multinode-oooq-container-upgrades jobs?21:31
*** trozet has joined #tripleo21:32
stevebakerEmilienM: was that the one that needed a backport to get working again?21:33
EmilienMstevebaker: maybe21:33
EmilienMI'm checking the last time we ran it now21:34
bnemecEmilienM: I think the fix for that just merged today.21:34
bnemecOr at least the patch that proposed to fix them.21:34
bnemecYeah: https://review.openstack.org/#/c/482167/21:35
stevebakerEmilienM: this one landed 5 hours ago21:35
bnemecThat's why the alert bug went away.21:35
EmilienMmhh ok21:35
stevebakerhttps://review.openstack.org/#/c/482167/21:35
EmilienMI'll give it a try21:35
EmilienMgate-tripleo-ci-centos-7-scenario001-multinode-upgrades is different though21:35
EmilienMbut I'll check gate-tripleo-ci-centos-7-scenario00X-multinode-oooq-container-upgrades21:36
*** ecerquei_ has quit IRC21:36
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - Test all CI scenarios jobs  https://review.openstack.org/48310521:38
*** zzzeek_ has joined #tripleo21:42
*** Goneri has joined #tripleo21:46
fultonjhas anyone deploying a containerized overcloud seen a timeout during puppet apply of /var/lib/tripleo-config/puppet_step_config.pp ?21:49
fultonjhttp://sprunge.us/LJDh21:49
*** tzumainn has quit IRC21:50
EmilienMfultonj: I haven't seen it before but it sounds like your mariadb isn't up21:52
fultonjThanks EmilienM , docker ps shows that container to be running http://sprunge.us/LZTg21:53
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372821:54
*** cschwede has quit IRC21:54
EmilienMfultonj: can you do mysql queries?21:54
* fultonj testing a mysql query21:55
EmilienMfultonj: I have to go afk now, I'm back later22:00
fultonjack, thanks EmilienM22:00
*** brandor5 has quit IRC22:03
*** jaosorior has quit IRC22:03
*** Vijayendra_ has joined #tripleo22:05
*** Vijayendra has quit IRC22:07
*** jmelvin has quit IRC22:08
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Add a new docker centos Heat Environment  https://review.openstack.org/48311122:10
*** agopi has joined #tripleo22:12
*** salmankhan has quit IRC22:15
*** zzzeek_ has quit IRC22:15
*** dprince has joined #tripleo22:16
*** zzzeek_ has joined #tripleo22:17
*** zzzeek has quit IRC22:18
*** dprince has quit IRC22:22
*** mmethot_ has joined #tripleo22:31
*** mmethot has quit IRC22:31
*** Goneri has quit IRC22:32
*** dk_ has quit IRC22:32
*** tosky has quit IRC22:32
*** artom has quit IRC22:32
*** dhill_ has quit IRC22:32
*** dk_ has joined #tripleo22:32
*** tosky has joined #tripleo22:32
*** artom has joined #tripleo22:33
*** ansmith has joined #tripleo22:33
*** tzumainn has joined #tripleo22:35
*** slagle has quit IRC22:35
*** slagle has joined #tripleo22:35
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Render isolated network templates using jinja2  https://review.openstack.org/47561222:38
itlinuxhello I need some help on this issue can someone guide me what needs to be changed here is the files I used22:41
itlinuxhttp://paste.openstack.org/show/615221/22:41
*** rlandy is now known as rlandy|bbl22:46
*** limao has joined #tripleo22:46
*** dciabrin has quit IRC22:47
openstackgerritDan Sneddon proposed openstack/tripleo-common master: Pass all network data to render network/port templates  https://review.openstack.org/47668222:47
*** limao_ has joined #tripleo22:49
*** limao has quit IRC22:51
fultonjmysql did answer a query http://paste.openstack.org/show/615223/22:56
*** tosky has quit IRC22:57
itlinux!vi23:03
openstackitlinux: Error: "vi" is not a valid command.23:03
*** itlinux has quit IRC23:06
*** limao_ has quit IRC23:08
*** limao has joined #tripleo23:09
*** dciabrin has joined #tripleo23:10
*** d0ugal has joined #tripleo23:12
*** atheurer has quit IRC23:14
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: Test the example nic configs we actually ship  https://review.openstack.org/47670423:14
*** limao has quit IRC23:14
*** masco has joined #tripleo23:54
*** bfournie has joined #tripleo23:56
*** bfournie has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!