Thursday, 2019-01-10

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429300:00
*** slaweq has quit IRC00:05
*** moguimar has quit IRC00:12
*** itlinux has joined #tripleo00:17
openstackgerritMerged openstack/diskimage-builder master: Only cap pylint for python2  https://review.openstack.org/61995500:17
openstackgerritMerged openstack/diskimage-builder master: Delete the duplicate words in  50-zipl  https://review.openstack.org/62881500:17
*** tosky has quit IRC00:21
openstackgerritMerged openstack/diskimage-builder master: Add policycoreutils-python-utils to bindep  https://review.openstack.org/62777500:22
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: Gracefully shutdown Mysql before upgrade.  https://review.openstack.org/62816100:40
*** lblanchard has quit IRC00:44
*** stee has quit IRC00:57
*** mjturek has quit IRC01:20
*** cfontain has joined #tripleo01:24
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: Don't include port numbers in insecure registries  https://review.openstack.org/62974001:39
openstackgerritSteve Baker proposed openstack/tripleo-common master: Make python uploader the default  https://review.openstack.org/61601901:40
openstackgerritSteve Baker proposed openstack/tripleo-common master: Export images to be served by httpd  https://review.openstack.org/62262801:40
openstackgerritSteve Baker proposed openstack/tripleo-common master: Role to install apache to serve container images  https://review.openstack.org/62482901:40
*** cfontain has quit IRC01:56
*** cfontain has joined #tripleo02:25
*** tzumainn has quit IRC02:27
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: Make sure $TMP_BUILD_DIR/mnt is owned by root  https://review.openstack.org/62974402:35
*** mschuppert|afk has quit IRC02:37
*** ykarel|away has joined #tripleo02:45
openstackgerritMerged openstack/puppet-tripleo master: Configure switchedv while there is no vfs bounded  https://review.openstack.org/62498402:47
openstackgerritMerged openstack/python-tripleoclient stable/rocky: Fix defaults to templates, roles-file and plan-environment  https://review.openstack.org/62591702:47
openstackgerritMerged openstack/python-tripleoclient stable/rocky: Add version to python execution  https://review.openstack.org/62592502:47
openstackgerritMerged openstack/python-tripleoclient stable/rocky: Remove unused services in tripleo deploy  https://review.openstack.org/62592602:47
*** raildo has quit IRC02:51
*** wolverineav has quit IRC02:58
*** cfontain has quit IRC02:58
*** nkinder has quit IRC02:59
*** wolverineav has joined #tripleo02:59
*** ykarel|away has quit IRC03:05
*** wolverineav has quit IRC03:06
openstackgerritRabi Mishra proposed openstack/tripleo-heat-templates stable/queens: Fix typo in octavia upgrade_tasks  https://review.openstack.org/62975003:07
*** wolverineav has joined #tripleo03:12
*** wolverineav has quit IRC03:17
*** d0ugal has quit IRC03:19
*** xarlos has quit IRC03:28
*** d0ugal has joined #tripleo03:33
*** cfontain has joined #tripleo03:34
*** skramaja has joined #tripleo03:42
*** ykarel|away has joined #tripleo03:42
*** cfontain has quit IRC04:07
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: Make sure $TMP_BUILD_DIR/mnt is owned by root  https://review.openstack.org/62974404:08
*** cfontain has joined #tripleo04:36
*** ykarel|away has quit IRC04:44
*** udesale has joined #tripleo04:50
*** jrist has quit IRC04:54
*** ykarel|away has joined #tripleo05:00
*** jrist has joined #tripleo05:06
*** cfontain has quit IRC05:10
*** xek_ has joined #tripleo05:11
*** xek has quit IRC05:12
*** ratailor has joined #tripleo05:18
*** itlinux has quit IRC05:18
*** pdeore has joined #tripleo05:23
*** rh-jelabarre has quit IRC05:25
jaosoriorGood morning!05:33
openstackgerritMerged openstack/tripleo-heat-templates master: Flatten Keystone service configuration  https://review.openstack.org/61555205:37
*** cfontain has joined #tripleo05:39
Tenguhello there :)05:39
Tengujaosorior: hey :)05:39
*** shyamb has joined #tripleo05:42
*** radeks_ has joined #tripleo05:42
jaosoriorTengu: how's it going?05:44
Tenguhad a "nice" wake today...05:44
jaosorior?05:44
Tengufirefighter all around with policemen and medics... apparently something bad has happened in the neightbourhoud05:44
jaosoriorwtf05:44
jaosoriordamn, that sucks05:44
Tenguerf... how do we write than name now... neighborhood? anyway.05:45
*** radeks__ has joined #tripleo05:45
jaosoriorneighbourhood I think05:45
Tengupossible. too early for me ;)05:46
Tengujaosorior: btw, care to juste have a quick look in https://bugs.launchpad.net/tripleo/+bug/1810512 ? As it will probably affect selinux.05:47
openstackLaunchpad bug 1810512 in tripleo "AVC regarding "ss" command with latest master deploy" [Medium,Triaged] - Assigned to Cédric Jeanneret (cjeanner)05:47
*** radeks_ has quit IRC05:47
*** rascasoft has quit IRC05:48
jaosoriorTengu: what is netlink_tcpdiag_socket for?05:52
*** radeks__ has quit IRC05:55
*** rascasoft has joined #tripleo05:55
Tengujaosorior: so it's related to netlink directly, but I don't have more info. its name suggests "tcp diagnostics", so it's in order to get tcp socket diagnostics via netlink. maybe `ss' man can provide more info.05:56
Tenguallowing container_t to access that socket shouldn't harm the system05:56
Tenguespecially if we can only "read" it05:57
Tenguthat said, I suspect we might need to add some more perms on that one, jillr will test it shortly05:58
jaosoriorTengu: are we sure that adding permission to using netlink sockets will fix the issue?05:58
Tengujaosorior: apparently there was the same thing with collectd: https://bugzilla.redhat.com/show_bug.cgi?id=91727905:59
openstackbugzilla.redhat.com bug 917279 in selinux-policy "SELinux is preventing /usr/sbin/collectd from create access on the netlink_tcpdiag_socket" [Medium,Closed: errata] - Assigned to mgrepl05:59
Tengujaosorior: as said, it will be tested shortly - I didn't push the PR yet on openstack-selinux :)05:59
jaosoriorgot it05:59
jaosoriorlet me know how that goes05:59
Tenguof course05:59
Tenguspeaking of test..... going back to "deploy osp14 on my lab using proxies for some resources".06:00
Tengu-.-'06:00
Tengujaosorior: https://selinuxproject.org/page/ObjectClassesPerms#netlink_tcpdiag_socket  that's all I found...06:04
*** hakhande_ has joined #tripleo06:07
jaosoriorhjensas: let me know when you're around06:12
*** cfontain has quit IRC06:12
*** rascasoft has quit IRC06:20
*** saneax has joined #tripleo06:21
*** rascasoft has joined #tripleo06:26
Tenguhmmm. can anyone point me where this handler is defined? RUNNING HANDLER [container-registry : Docker | wait for registry]06:32
Tenguit apparently has some troubles taking the no_proxy env var -.-.06:32
Tenguand of course it makes all my deploy crash…06:32
*** radeks__ has joined #tripleo06:33
Tengufound it.06:36
jaosoriorTengu: where was it?06:36
jaosoriorI was looking for it too06:36
Tengu/usr/share/ansible/roles/container-registry/handlers06:36
jaosoriorah, got it06:37
Tenguok. so the "uri" module from ansible is borked.06:37
Tenguwhy am I not even surprised?06:37
*** maufart__ has joined #tripleo06:38
Tenguhttps://github.com/ansible/ansible/issues/4082706:40
openstackgerritwaleed mousa proposed openstack/puppet-tripleo stable/rocky: Configure switchedv while there is no vfs bounded  https://review.openstack.org/62977406:41
*** rcernin has quit IRC06:44
*** rcernin has joined #tripleo06:44
*** agurenko has joined #tripleo06:45
openstackgerritKarthik S proposed openstack/os-net-config stable/queens: Remove mox usage  https://review.openstack.org/62977506:45
*** cfontain has joined #tripleo06:49
*** ykarel|away is now known as ykarel06:55
openstackgerritKarthik S proposed openstack/os-net-config stable/rocky: NIC Partitioning works with NM enabled  https://review.openstack.org/62977606:57
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Fix files: for scenario003 standalone - pointing to wrong env  https://review.openstack.org/62977707:01
*** rcernin has quit IRC07:03
*** shyamb has quit IRC07:09
*** kopecmartin has joined #tripleo07:10
*** kopecmartin has quit IRC07:10
skramajacould i get reviews on https://review.openstack.org/#/c/621133/?07:11
*** kopecmartin has joined #tripleo07:14
hjensasjaosorior: Good morning, I'm caffinating. :)07:14
jaosoriorenjoy07:15
hjensasjaosorior: I added needinfo on that bug last night for your awareness.07:15
jaosoriorcan you pass the link?07:16
hjensasjaosorior: https://bugzilla.redhat.com/show_bug.cgi?id=1643423 <- It's related to the novajoin issue we talked about yesterday afternoon/evening.07:16
openstackbugzilla.redhat.com bug 1643423 in openstack-tripleo-heat-templates "openstack stack output show overcloud HostsEntry return wrong entries" [Medium,Modified] - Assigned to ramishra07:16
*** quiquell|off is now known as quiquell07:19
jaosoriorhjensas: I have an environment that reproduces the novajoin issue07:19
jaosoriorjust digging a little bit to have more informed questions07:19
jaosoriorafter that, would you have time to jump in the session with me to debug?07:19
hjensasjaosorior: yes I can.07:19
*** ccamacho has joined #tripleo07:21
*** hakhande_ has quit IRC07:21
ramishrajaosorior, hjensas Is there abug for that tls/cermonger issue? I mean if apache_certificates_specs contains a nil hostbame/principal where does it fail?07:22
*** gfidente has joined #tripleo07:22
*** factor has quit IRC07:23
openstackgerritcaoyuan proposed openstack/tripleo-validations master: Update the bugs link to storyboard  https://review.openstack.org/62783807:23
hjensasramishra: I just have a suspicion due to the discussion on IRC - See http://eavesdrop.openstack.org/irclogs/%23tripleo/%23tripleo.2019-01-09.log.html#t2019-01-09T16:43:29-207:26
ramishramay be we can use a default in that hiera lookup as a workaround in https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/apache.j2.yaml#L119, though we should have role specific apache_certificates_specs07:26
hjensasramishra: mschuppert's comment there.07:27
hjensasramishra: I wonder if what iterated over networks should be moved to the node. We could pass the networks as hiera value for each role or something?07:28
*** cfontain has quit IRC07:28
ramishrahmm that's not clear, if the network is not used why would the certificate needed by a service?07:28
jaosoriorramishra: it's not needed.07:29
jaosoriorit was probably an oversight from my side07:29
jaosorioryeah07:29
jaosoriorso, the code wrongly iterates through all the networks07:29
jaosoriormy initial thought was that that only contained the networks for that specific role07:30
hjensasramishra: right, ^^ not needed. But we create them anyway. So fixing this is good. Your change does the right thing imo, it just opened up the can of worms that surface now. :)07:30
jaosoriorso, before, it would just get those certs, and not use them at all07:30
Tengupfff. the no_proxy directive is apparently ignored in many cases when we install tripleo..... bad.07:30
jaosorioryeah, we now just need to figure out how to get the apache bit to do the right thing07:30
jaosoriorramishra: was your code backported?07:31
jaosorior(just want to know to what extent do we need to backport this TLS fix)07:31
ramishrahjensas: yeah, it's good to expose the issues, better late than never07:31
*** cfontain has joined #tripleo07:32
*** boazel_ has joined #tripleo07:32
jaosoriorhjensas: do you have any ideas on how to fix this? Do you want to pair working this out?07:33
*** agurenko has quit IRC07:33
ramishrajaosorior: yeah, it's backported downstream to stable/queens, that's what hjensas was talking about in that bz comment:)07:33
hjensasramishra: jaosorior: it's the downstream backport that worries me, we can't ship this if it breaks TLS everywhere.07:34
jaosoriorunderstood07:34
ramishrahjensas: I still don't know which service needs the cert, I guess having nil values in that map would not be an issue07:35
hjensasjaosorior: I have an idea. We can pass/may already pass the networks for the role to the node.07:35
*** boazel has quit IRC07:35
hjensasjaosorior: then the puppet? code use that to decide which networks to generate certs for?07:36
jaosoriorhjensas: it's done in t-h-t currently.07:36
jaosoriorramishra: no service needs the cert, it was an oversight from the beginning.07:36
hjensasjaosorior: ramishra: Or, if it passes nil then whatever uses this could skip those entries with nil values?07:36
openstackgerritMerged openstack/python-tripleoclient master: Force lower-case for hostname lookup  https://review.openstack.org/62661307:36
jaosoriorramishra: what we need to do is fix puppet/services/apache.j2.yaml to only try to request the certs that actually belong to the role where apache is installed.07:37
ramishrahjensas: yeah, the hash  can be filtered in the puppet-tripleo I guess07:37
hjensasramishra: yes, it may be a trivial fix there.07:37
jaosoriorramishra, hjensas: Do we have anywhere a variable that defines what role is currently being rendered?07:38
ramishrajaosorior: that would be good, but I guess that service yamls are not rendered per role07:39
hjensasjaosorior: that template is rendered once for the whole stack, not per role.07:39
jaosoriorOK07:39
ramishraso I would go with the puppet-tripleo fix to filter out the nil values as a workaround, but would still need to know the exact issue with those nil values, as there may be some other bug07:40
jaosoriorramishra: which nil value?07:40
*** jfrancoa has joined #tripleo07:41
jaosoriorhjensas: I have an environment up and running with the bug, if you want to try out something on that env07:41
hjensashttps://github.com/openstack/puppet-tripleo/blob/3da5ea3f4a48b5453b3ba8ffbec59507c206b568/manifests/profile/base/certmonger_user.pp#L183-L18507:41
ramishrajaosorior: hostname and prinipal in apache_certificate_specs https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/apache.j2.yaml#L11907:41
hjensasramishra: if we pass nil we could check for it there maby?07:41
jaosoriorI se07:42
jaosorior* I see07:42
jaosoriorso... what's the status currently of that management network?07:42
jaosorioris it disabled?07:42
ramishrahjensas: we don't know it's nil till puppet tries to run them07:42
hjensasjaosorior: If you use this role: https://github.com/openstack/tripleo-heat-templates/blob/master/roles/Novacontrol.yaml07:42
hjensasjaosorior: You would get the issue for all networks except internal_api.07:42
ramishraunless we make those specs role specific somehow07:43
jaosoriorthey gotta be role specific07:43
jaosoriorthat would be the ideal case07:43
*** gkadam has joined #tripleo07:43
*** jfrancoa has quit IRC07:45
hjensasramishra: it'd be good to log in and do a shared tmux on jaosorior's system?07:45
ramishrahjensas: after I'm back from lunch?07:46
hjensasramishra: sure, thanks!07:46
*** janki has joined #tripleo07:46
*** tosky has joined #tripleo07:49
*** pcaruana has joined #tripleo07:51
hjensasjaosorior: [root@overcloud-controller-0 hieradata]# hiera -c /etc/puppet/hiera.yaml fqdn_management                                                                                                     │·····················07:53
hjensasnil                                                                                                                                                                                          │·····················07:53
hjensas[root@overcloud-controller-0 hieradata]# hiera -c /etc/puppet/hiera.yaml fqdn_tenant                                                                                                         │·····················07:53
hjensasovercloud-controller-0.tenant.ooo.test07:53
jaosoriorright07:54
hjensaswe get 'nil' when eveluating that hiera for networks that are not in the role.07:54
hjensasif we just filter entries with nil in puppet-tripleo and skip generating certs for those we'd be good?07:54
jaosoriorhjensas: that's one option07:55
jaosoriorit would be a workaround07:56
jaosoriorultimately the best solution would be to not even populate wrong entries in the specs07:56
jaosoriorbut that seems non-trivial07:56
hjensasjaosorior: yeah, maby we should remove the apache_certificates_specs and metadata_settings from tht, and do the figuring out which networks entirely on the puppet side.07:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:00
hjensasjaosorior: maby we could use the fqdn entries in /etc/puppet/hieradata/%ROLE%.json somehow in puppet.08:00
jaosoriorwe an do explicit hiera calls from puppet08:00
jaosoriorhjensas: do we not have any network information on the service template itself?08:01
*** jfrancoa has joined #tripleo08:02
*** brault has joined #tripleo08:08
*** jpena|off is now known as jpena08:08
*** ykarel is now known as ykarel|lunch08:09
hjensasjaosorior: we have the ServiceNetMap, but it's not Role specific. So if we try to filter on that, for example the roles/Novacontrol.yaml in THT would'nt work.08:09
jaosoriorcrap08:10
jaosorioryeah08:10
jaosoriorI guess the only way is to filter in puppet-tripleo08:10
*** paramite has joined #tripleo08:11
*** hakhande_ has joined #tripleo08:12
hjensasjaosorior: is there a bug already in LP?08:14
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Rework the generated openshift-ansible playbook  https://review.openstack.org/61971308:14
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Restart openshift master services after stack update  https://review.openstack.org/62401108:14
jaosoriorhjensas: I don't think so.08:14
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix address for glusterfs container images  https://review.openstack.org/62055708:17
jaosoriorhjensas: I'll open one08:17
hjensasjaosorior: thanks, I will be afk for about 15 minutes.08:19
*** hjensas is now known as hjensas|afk08:19
*** avivgt has quit IRC08:24
jaosoriorhjensas|afk: LP created https://bugs.launchpad.net/tripleo/+bug/181120708:24
openstackLaunchpad bug 1811207 in tripleo "TLS everywhere: the apache service wrongly requests certificates for all networks" [Critical,Confirmed]08:24
*** hberaud has joined #tripleo08:24
*** radeks_ has joined #tripleo08:27
*** radeks__ has quit IRC08:29
openstackgerritMerged openstack/tripleo-heat-templates master: Only add internal_api_virtual_ip if InternalApi in network_data  https://review.openstack.org/62815108:31
openstackgerritMerged openstack/tripleo-heat-templates master: Fall back public_virtual_ip to ctlplane if External net not present  https://review.openstack.org/62815208:32
Tenguwow. ok. so.... http_proxy and https_proxy are taken as is. But the "no_proxy" ? must be NO_PROXY08:37
Tenguthat's.....08:37
Tengu*headshot*08:37
*** cylopez has joined #tripleo08:40
*** limao has joined #tripleo08:41
*** limao has quit IRC08:42
*** sai_p has quit IRC08:43
*** cylopez has left #tripleo08:44
*** shyamb has joined #tripleo08:45
Tengugreat. skopeo doesn't take no_proxy into account.08:51
*** hjensas|afk is now known as hjensas08:51
*** shyam89 has joined #tripleo08:54
hjensasjaosorior: thanks. What about puppet/services/apache.yaml:123 - It also iterates all the networks, is there a potential problem as well?08:55
jaosoriorhjensas: puppet/services/apache.yaml gets generated from puppet/services/apache.j2.yaml08:55
hjensasjaosorior: ^^ metadata_settings: ...08:55
*** jpich has joined #tripleo08:56
hjensasjaosorior: oh, sorry. :)08:56
jaosoriorhjensas: but yeah, the metadata settings have that same issue08:56
jaosoriornot sure if that'll be a problem08:56
jaosoriorbut it would be nice to fix08:56
hjensasjaosorior: the correct reference puppet/services/apache.j2.yaml:12308:56
jaosoriorproblem is, that doesn't get used in puppet at all08:56
jaosoriorthat's why I was hoping for a fix on the t-h-t level08:56
*** ykarel|lunch is now known as ykarel08:57
*** shyamb has quit IRC08:57
jaosoriorbrb, lunch08:58
* hjensas wonder why we need cert for other networks than get_param: [ServiceNetMap, ApacheNetwork] ...09:01
jaosoriorhjensas: keystone, for instances, is on apache, and listens on both the internal API network and the ctlplane network09:03
jaosoriorhjensas: ultimately, services are configured individually, and ApacheNetwork is not that relevant compared to what services using apache actually listen on09:04
hjensasjaosorior: ok, and other services as well ... yeah got it. We need certs for all the networks used by services behind apache wsgi something.09:04
*** shyam89 has quit IRC09:05
*** shyam89 has joined #tripleo09:05
jaosoriorhjensas: yep09:06
*** shardy has joined #tripleo09:10
openstackgerritTobias Urdin proposed openstack/tripleo-heat-templates master: DNM: Test setting keystone admin_bind_host  https://review.openstack.org/62979809:11
openstackgerritMerged openstack/tripleo-quickstart master: Lint fixes for playbooks and roles  https://review.openstack.org/62762109:12
*** janki has quit IRC09:13
*** shardy has quit IRC09:13
*** shardy has joined #tripleo09:13
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: WIP: Split upgrade_steps_playbook into different plays.  https://review.openstack.org/62589009:15
*** holser_ has joined #tripleo09:15
openstackgerritKarthik S proposed openstack/os-net-config stable/queens: Adding VLAN support and other configs for sriov_vf  https://review.openstack.org/62980409:23
openstackgerritAnkit Kumar proposed openstack/diskimage-builder master: Modifying installation of proliantutils  https://review.openstack.org/62980609:24
*** shyam89 has quit IRC09:29
*** janki has joined #tripleo09:31
*** wolverineav has joined #tripleo09:36
*** shyamb has joined #tripleo09:38
*** derekh has joined #tripleo09:39
*** derekh has quit IRC09:40
*** suuuper has joined #tripleo09:40
*** derekh has joined #tripleo09:40
*** wolverineav has quit IRC09:41
*** apetrich has quit IRC09:42
*** apetrich has joined #tripleo09:43
openstackgerritKarthik S proposed openstack/os-net-config stable/rocky: Sample templates for linux_bond, ovs_bond, dpdkbond using VFs  https://review.openstack.org/62981409:43
*** ramishra_ has joined #tripleo09:43
openstackgerritKarthik S proposed openstack/os-net-config stable/rocky: NIC-Partitioning: DPDK devices comes up properly after reboot  https://review.openstack.org/62981609:45
ramishra_hjensas, jaosorior I guess http://paste.openstack.org/show/741560/ would work as a workaround too?09:45
*** ramishra has quit IRC09:45
jaosoriorramishra_: it would would, but I rather filter them so we don't even try to ask for those certs.09:46
hjensasramishra_: yes. I looking into something like this: http://paste.openstack.org/show/741561/09:47
ramishra_jaosorior: yeah, but that would require quite a bit of change , this kind of falls with 'fallback to ctlplane' approach for everything09:48
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates master: Enable image inject metadata properties & user roles to be ignored  https://review.openstack.org/62613109:48
ramishra_quite bit of change in puppet-tripleo as we use that hash in all services09:48
hjensasramishra_: ack, maby your idea is a good candidate for a workaround to backport. Then we can push a change to filter them in master?09:50
hjensaswe get some time to fix potential other issues with changing puppet-tripleo that way.09:50
ramishra_hjensas: ok, can we try it somewhere? I can then push that change09:51
hjensasramishra_: jaosorior have a reproducer, if he can share the tmate session, you could test it there?09:51
openstackgerritKarthik S proposed openstack/os-net-config stable/queens: Add sriov_vf type as member of ovs_dpdk_port  https://review.openstack.org/62981709:52
openstackgerritKarthik S proposed openstack/os-net-config stable/queens: Add default VF configs for ovs and ovs_user bridges  https://review.openstack.org/62982009:55
*** mmethot has quit IRC09:55
*** mmethot has joined #tripleo09:56
jaosoriorbrb10:00
*** jaosorior has quit IRC10:00
*** itlinux has joined #tripleo10:01
*** shyamb has quit IRC10:02
*** itlinux has quit IRC10:05
*** shyamb has joined #tripleo10:06
*** jpich has quit IRC10:08
*** jpich has joined #tripleo10:09
Tenguhmmm, who's "responsible" for the rabbitmq things? PIDONE ?10:10
Tengubandini: do you know a bit about rabbitmq? if so, care to have a look here? https://review.openstack.org/#/c/629120/ :)10:10
*** dtantsur|afk is now known as dtantsur10:13
arxcruz|ruckany keystone guy around to help me with https://bugs.launchpad.net/tripleo/+bug/1811185 ?10:15
openstackLaunchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] - Assigned to Tobias Urdin (tobias-urdin)10:15
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Allow overlay tunnel endpoints on IPv6 address  https://review.openstack.org/60353210:26
openstackgerritJanki Chhatbar proposed openstack/tripleo-quickstart master: fs35: move to network-isolation-v6-all.yaml  https://review.openstack.org/62760010:27
*** sshnaidm|afk is now known as sshnaidm10:31
*** hberaud has quit IRC10:32
*** hberaud has joined #tripleo10:33
jankiHI. Can I get some reviews for https://review.openstack.org/#/c/603532/ and https://review.openstack.org/#/c/620013/10:35
*** bogdando has joined #tripleo10:38
*** agurenko has joined #tripleo10:42
*** avivgt has joined #tripleo10:43
*** skramaja has quit IRC10:45
*** brault has quit IRC10:52
*** agurenko has quit IRC10:54
*** jaosorior has joined #tripleo11:01
jaosoriorramishra_: I'm back11:03
*** openstackgerrit has quit IRC11:05
*** udesale has quit IRC11:14
*** shardy has quit IRC11:14
*** udesale has joined #tripleo11:14
*** morazi has quit IRC11:14
*** openstackgerrit has joined #tripleo11:29
openstackgerritHarald Jensås proposed openstack/puppet-tripleo master: WIP :: Filter apache_certificates_specs if hostname is empty  https://review.openstack.org/62983511:29
*** shyamb has quit IRC11:29
openstackgerritHarald Jensås proposed openstack/puppet-tripleo master: WIP :: Filter apache_certificates_specs if hostname is empty  https://review.openstack.org/62983511:31
*** pdeore has quit IRC11:31
*** psachin has joined #tripleo11:36
*** shyamb has joined #tripleo11:42
*** shyamb has quit IRC11:44
*** shyamb has joined #tripleo11:44
*** psachin has quit IRC11:44
*** bogdando has quit IRC11:45
*** brault has joined #tripleo11:46
*** shardy has joined #tripleo11:46
*** bogdando has joined #tripleo11:48
*** psachin has joined #tripleo11:49
*** brault has quit IRC11:50
openstackgerritDavid Peacock proposed openstack/tripleo-heat-templates master: flatten sahara service configuration  https://review.openstack.org/62527011:57
*** brault has joined #tripleo11:57
*** slaweq has joined #tripleo11:58
*** ratailor has quit IRC12:00
*** holser_ is now known as holser|lunch12:01
*** itlinux has joined #tripleo12:01
*** abishop has joined #tripleo12:02
*** itlinux has quit IRC12:06
*** thrash|g0ne is now known as thrash12:07
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add some fixes for libvirt  https://review.openstack.org/62983912:07
*** shyamb has quit IRC12:12
*** shyamb has joined #tripleo12:12
hjensasjaosorior: no CI actually tests the novajoin TLS everywhere stuff?12:15
hjensasjaosorior: It would be good to test the puppet-tripleo change in your env as well, after verifying ramishra_'s fix.12:16
openstackgerritJohn Fulton proposed openstack/tripleo-quickstart-extras master: WIP/DNM: Add split-controlplane-standalone.yml playbook  https://review.openstack.org/61736812:20
*** srini_ has joined #tripleo12:20
*** sri_ has quit IRC12:20
jaosoriorhjensas: ramishra_'s fix didn't work12:20
jaosoriorhjensas: cause of some funky syntax issue :/12:20
jaosoriordidn't figure it out12:20
jaosoriorjust put your fix in my system and am gonna test it out12:20
jaosoriorhjensas: no CI for TLS everywhere yet. We still don't figure out how to actually make all the nodes use FreeIPA as the DNS server, since the undercloud (as it's setup by zuul) runs unbound and it conflicts... we got stuck at that.12:21
*** raildo has joined #tripleo12:29
ssbarnea|roverEmilienM: can you help with https://bugs.launchpad.net/tripleo/+bug/1811185 ? see yatin comment regarding recent change, which seems close by.12:29
openstackLaunchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] - Assigned to Tobias Urdin (tobias-urdin)12:29
EmilienMssbarnea|rover: yeah I think we talked about in on #puppet-openstack12:33
EmilienMunfortunately I'm in a meeting right now and can't help12:34
EmilienMI think ykarel is working on it12:34
ssbarnea|roverEmilienM: good to hear that, thanks.12:34
ykarelnope /me not working on a patch:)12:34
EmilienMotherwise i'll take a look after my call12:34
ykarelbut what i said on #puppet-openstack is worth try12:35
ykareli need to leave for home, will join from there12:35
*** slaweq has quit IRC12:38
EmilienMykarel: I'm not sure changing KeystoneAdminApiNetwork to internal_api is something jaosorior will accept12:40
EmilienMI can send a patch, however I'm not sure it's the right fix12:40
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Revert "Add keystone::wsgi::apache::api_port"  https://review.openstack.org/62984512:41
jaosoriorEmilienM: from a security stand point it's all fine really. And in the long run we even want to get rid of the admin endpoint entirely (it's really not very useful now that we're keystone v3 only)12:41
jaosoriormy main concern is that folks might have integration with that endpoint, and might assume it to be in the ctlplane network12:41
jaosoriorthat may or may not be the case, I'm honestly not sure12:41
jaosoriorI do know folks rely on that port, just not sure what the state of their hardcoding is12:41
EmilienMwell we can try12:42
EmilienMuntil we have a bug report ...12:42
ykarelEmilienM, jaosorior ack12:42
jaosoriorykarel, EmilienM: I'm still confused on why that fixes the issue though.12:42
*** ykarel is now known as ykarel|away12:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Revert "Add keystone::wsgi::apache::api_port"  https://review.openstack.org/62984512:43
*** jpena is now known as jpena|lunch12:43
ykarel|awayEmilienM, u meant revert ^^ or puppet-keystone one?12:45
ykarel|awaythe other parameters i think are ignore now12:45
EmilienMykarel|away: did I revert the wrong patch?12:47
ykarel|awayEmilienM, u reverted tht patch12:47
ykarel|awayme don't think that fixes12:47
EmilienMykarel|away: https://review.openstack.org/#/c/629846/12:48
ykarel|awayEmilienM, ack12:48
ykarel|awaygood to test internal_api too in parallel12:49
*** mcornea has joined #tripleo12:49
openstackgerritDerek Higgins proposed openstack/tripleo-docs master: Add documentation on using net-ansible in overcloud  https://review.openstack.org/62010112:51
openstackgerritTobias Urdin proposed openstack/tripleo-heat-templates master: DNM: Set keystone bind_host to both public and admin  https://review.openstack.org/62979812:53
*** dciabrin has joined #tripleo12:53
*** lblanchard has joined #tripleo12:55
*** shyamb has quit IRC12:57
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Allow container healthchecks to access netlink data  https://review.openstack.org/62985012:59
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: Make scenario004-standalone voting too  https://review.openstack.org/62985112:59
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Add scenario002-standalone to gates as we make it voting  https://review.openstack.org/62985212:59
openstackgerritMarios Andreou proposed openstack/tripleo-common master: Add scenario002-standalone and 4 to gates as they are voting  https://review.openstack.org/62985312:59
openstackgerritMarios Andreou proposed openstack/puppet-tripleo master: Add scenario002 standalone to gate as well as it is voting  https://review.openstack.org/62985413:00
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Add scenario002 standalone to gate as well as it is voting  https://review.openstack.org/62985513:00
*** ykarel|away has quit IRC13:00
mariosquiquell: mwhahaha weshay when you have some time please https://review.openstack.org/#/q/topic:scen2-standalone-vote+(status:open+OR+status:merged) - keeps 3 as non voting (until https://bugs.launchpad.net/tripleo/+bug/1811004 ) but adds 2 into gates too13:02
openstackLaunchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,Triaged]13:02
mariosquiquell: mwhahaha weshay i made a mistake so we have 3/4 as non voting currently, so in that topic branch i make 4 vote too with https://review.openstack.org/#/c/629851/ and the rest is for adding 2 to gates)13:03
mariosrfolco: ^^13:03
*** trown|outtypewww is now known as trown13:06
*** thrash is now known as thrash|biab13:09
*** hakhande_ has quit IRC13:11
*** raildo has quit IRC13:13
*** holser|lunch is now known as holser_13:17
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add some fixes for libvirt  https://review.openstack.org/62983913:17
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add some fixes for libvirt  https://review.openstack.org/62983913:20
*** rh-jelabarre has joined #tripleo13:20
*** dprince has joined #tripleo13:21
*** jfrancoa has quit IRC13:21
*** morazi has joined #tripleo13:23
*** ykarel|away has joined #tripleo13:23
*** raildo has joined #tripleo13:24
*** trozet has quit IRC13:24
openstackgerritBogdan Dobrelya proposed openstack-infra/tripleo-ci master: Install Docutils as gear requires it  https://review.openstack.org/62985913:25
*** derekh has quit IRC13:25
*** ade_lee has quit IRC13:25
quiquelldprince: Have just remember the cockpit-ci.tripleo.org subdomain13:26
quiquelldprince: Have you being able to create it ?13:26
quiquellsshnaidm: ^13:26
*** tzumainn has joined #tripleo13:27
dprincequiquell: hi, sorry I haven't done it yet. Give me a few minutes this morning and I can set it up. Whats the IP again?13:27
weshayanyone seeing tripleo_iscsid_healthcheck.service failed because the control process exited with error code. See \"systemctl status tripleo_iscsid_healthcheck.service13:27
weshayssbarnea|rover, arxcruz|ruck ^13:27
sshnaidmdprince, 38.145.35.97 exactly as dashboard-ci.tripleo.org13:27
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add some fixes for libvirt  https://review.openstack.org/62983913:29
*** ykarel|away is now known as ykarel13:30
*** rh-jelabarre has quit IRC13:33
*** saneax has quit IRC13:34
dprincequiquell, sshnaidm the change is made. may take a few minutes to propagate13:36
openstackgerritMerged openstack/tripleo-heat-templates master: Rework the generated openshift-ansible playbook  https://review.openstack.org/61971313:37
sshnaidmdprince, thanks!13:38
quiquelldprince: cool cool13:38
*** ansmith_ has joined #tripleo13:41
openstackgerritMartin André proposed openstack/tripleo-heat-templates stable/rocky: Rework the generated openshift-ansible playbook  https://review.openstack.org/62986213:41
*** rlandy has joined #tripleo13:42
*** jpena|lunch is now known as jpena13:45
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Re-enable idempotent check for releases before master  https://review.openstack.org/62918613:46
Tengujaosorior: hey :). Coming back with the small selinux thingy we talked about this morning: I have a better solution: https://review.openstack.org/#/c/629850/  care to have a look when you have a minute?13:50
jaosoriorTengu: that is a better solution! nice!13:51
Tengu:)13:51
Tenguso that will solve the actual AVC - but I stumbled on some other, and for those we will need an actual selinux policy update.13:52
TenguI'll take care of it later.13:52
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add some fixes for libvirt  https://review.openstack.org/62983913:52
openstackgerritMerged openstack/tripleo-upgrade stable/rocky: Migrate linting to pre-commit  https://review.openstack.org/62717813:52
*** thrash|biab is now known as thrash13:54
slagleall: i'll be a few minutes late starting the edge squad meeting14:01
slagle#startmeeting TripleO Edge Squad Meeting14:07
openstackMeeting started Thu Jan 10 14:07:23 2019 UTC and is due to finish in 60 minutes.  The chair is slagle. Information about MeetBot at http://wiki.debian.org/MeetBot.14:07
slagleping slagle, csatari, jaosorior, owalsh, fultonj, gfidente, hjensas, jtomasek, bogdando, dtantsur, rbrady, d0ugal, toure, abishop14:07
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:07
slagle#info remove or update your nick from the Meeting Template on the etherpad if you want (or don't want) to be ping'd for the start of the meeting14:07
*** openstack changes topic to " (Meeting topic: TripleO Edge Squad Meeting)"14:07
openstackThe meeting name has been set to 'tripleo_edge_squad_meeting'14:07
dtantsur\o14:07
jaosorioro/14:07
slagle#link https://etherpad.openstack.org/p/tripleo-edge-squad-status14:07
gfidenteo/14:07
d0ugalo/14:07
fultonjo/14:07
slaglehello!14:07
bogdandoo/14:07
slagleAnyone can use the #link, #action, #help, #idea, #agreed, and #info commands, not just the moderatorǃ14:08
slagle#topic Agenda14:08
*** openstack changes topic to "Agenda (Meeting topic: TripleO Edge Squad Meeting)"14:08
slagle* Review past action items14:08
slagle* Goals/action items for the week14:08
csatario/14:08
abishopo/14:08
slagle#topic Review past action items14:08
*** openstack changes topic to "Review past action items (Meeting topic: TripleO Edge Squad Meeting)"14:08
slagle(sorry for the late start btw)14:08
d0ugalnp14:09
slagleACTION: fultonj/slagle get help (perhaps from ci team) for ci inventory for second node for https://review.openstack.org/#/q/topic:bp/split-controlplane-ci (slagle, 14:13:30)14:09
fultonjprogress!14:09
fultonjwaiting for results of recheck of...14:09
fultonj#link https://review.openstack.org/#/c/615988/14:09
fultonj(15 hours!)14:09
fultonjthanks to bogdando slagle and panda for help with improvements of...14:10
fultonj#link https://review.openstack.org/#/c/617368/14:10
owalsho/14:10
slaglenp. it's looking pretty good14:10
fultonjwork should continue but i'll be in brno next week14:10
toureo/14:10
slaglefultonj: did you double check that 615988 is in the zuul queue?14:11
fultonjso multinode job is triggering both standalone deployments but they're failing and coordination (including sharing of control plane data) needs improvement14:11
fultonjslagle: i don't see it in zuul queue14:11
fultonjsuppose i could recheck again14:11
slaglefultonj: i think it needs to be rebased to clear the zuul merge conflict14:11
slagleshould just be able to do it from the gerrit ui14:12
openstackgerritJohn Fulton proposed openstack-infra/tripleo-ci master: WIP/DNM: Add split-controlplane standalone non-voting job definition  https://review.openstack.org/61598814:12
fultonjrebased14:12
slagleok i see it in check now14:12
slaglethanks!14:12
slaglewill continue to keep an eye on it14:13
fultonjthanks14:13
slagleACTION: slagle send scaling doc to ML14:13
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova  https://review.openstack.org/57685614:13
slaglei don't think i did that...14:14
fultonji read scaling doc, looked good. many many parts.14:14
slagleok i'll send it out. i don't see any further comments on it14:14
slagle#action slagle send scaling doc to ML14:15
slagleACTION: owalsh continue investigating nova AZ config14:15
*** derekh has joined #tripleo14:15
owalshslagle: working on it right now14:15
slaglecool thanks!14:16
fultonjowalsh nice work on #link https://review.gerrithub.io/c/redhat-openstack/infrared/+/43550414:16
*** ade_lee has joined #tripleo14:16
*** mjturek has joined #tripleo14:16
owalshfultonj: thanks, not sure how accurate the traffic shaping is but it gets us something resembling a DCN deployment14:17
owalshmust look at routed networks next14:17
slagleACTION: slagle document the split-controlplane deployment from a central undercloud (not using standalone)14:19
fultonjhas anyone tried split control plane deployments outside of standalone and gotten multi-ceph?14:19
slaglei didn't get to the docs but will try again this week14:19
slaglefultonj: i could try that in my environment. multiple ceph clusters you mean? 1 per DCN?14:19
fultonjslagle: yes14:19
fultonje.g. do two DCNs and put a ceph on one of each14:20
openstackgerritMerged openstack/tripleo-validations master: Fix ansible lint errors  https://review.openstack.org/62895314:20
slagleok, i was running through my environment for the docs, so i can try that14:20
fultonji'd like to try it too14:20
vkmcEmilienM, hey o/ can we merge this? https://review.openstack.org/#/c/629485/1 and https://review.openstack.org/#/c/629484/14:20
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations stable/rocky: Fix ansible lint errors  https://review.openstack.org/62987114:20
slagle#action slagle document the split-controlplane deployment from a central undercloud (not using standalone)14:20
fultonjslagle: if you have a wip of your doc i could try with 3 VMs14:20
slagle#action slagle/fultonj try multi-ceph deployments with DCN14:20
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Rewrite nodepool-setup for the zuul repro  https://review.openstack.org/62987214:20
slaglefultonj: will hopefully have something soon14:21
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Update the bugs link to storyboard  https://review.openstack.org/62783814:21
slagleACTION: slagle update roles patch https://review.openstack.org/#/c/626563/14:21
EmilienMvkmc: maybe :P14:21
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix the misspelling of "interface"  https://review.openstack.org/62884514:21
fultonjany thoughts on putting glance api into roles patch?14:21
* vkmc sends alfajores to EmilienM 14:21
vkmcwhat about now?14:21
* EmilienM +A14:21
fultonjslagle abishop gfidente ^ ?14:21
fultonjlooks like omit for now14:22
fultonjand add later14:22
fultonji guess that's all there is to say14:22
fultonjfor now14:22
abishopfultonj: my comments are noted in https://review.openstack.org/62656314:22
gfidentefultonj abishop I realized we might have a problem there where we need CephExternal role for GlanceApi14:23
slagleyea, was just catching up on the comments about GlanceAPI on the review from this morning14:23
gfidentefor glanceapi only I mean14:23
gfidentebecause cinder or nova instead won't use CephExternal but the other Ceph services as they need to be pointed to the local Ceph cluster14:23
slagleabishop: do you want to document somewhere how you got this to work?14:23
abishopgfidente: yeah, this needs more thought, and probably input from glance folks14:23
slaglejust your poc i mean. so other folks can try it14:24
owalshwe don't want this on all distributed computes though, *maybe* on the HCI nodes14:24
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Rewrite nodepool-setup for the zuul repro  https://review.openstack.org/62987214:25
fultonjso ceph is there but doesn't get configured unless you -e environments/ceph-ansible/...14:25
abishopslagle: can do that, but I'm pondering the best place/method for describing what I did (could just add doc notes to my PoC?)14:25
slagleabishop: sure, just etherpad or whereever14:25
owalshbut for larger sites probably need an "edge controller" role that includes glance api cache, nova-conductor, mq, and db14:26
vkmcEmilienM++14:26
abishopslagle: sure, will put together something, which we can move to a better spot as necessary14:26
slaglethanks14:26
fultonjowalsh: ack, "edge controller" role would be a new thing14:27
gfidenteslagle abishop fultonj basically what is concerning me is that we need both Ceph and CephExternal services enabled on the same role14:27
slagle#action abishop document GlanceAPI at the edge for DCN14:27
abishopand I don't think we want to introduce another db14:27
owalshabishop: that's for multicell14:27
abishopowalsh: ah14:27
owalshcould have another role too with just the glance api cache, e.g for sites with limited bandwith (otherwise it's kinda pointless, not going to save much by caching the images for 3 nodes)14:28
fultonjgfidente: can you elaborate on ceph-extrnal and ceph-internal?14:28
*** mjturek has quit IRC14:29
abishopfultonj: I'm guessing "ceph" for edge site, and "ceph-external" would be ceph at central site14:29
slagleowalsh: does nova-compute not cache images on its own?14:29
owalshslagle: yea14:29
gfidentefultonj so might end up in a situation where, if we colocate glance api with cinder-volume or nova-compute14:29
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix the containerized location of ceilometer ceilometer.conf  https://review.openstack.org/62508314:29
gfidentefultonj in which case glance needs to be pointed to the central ceph cluster, while cinder and nova do not14:29
abishopslagle, owalsh: yeah, that would mitigate need for glance doing caching at edge14:29
owalshslagle: and if we bump the TCP windows then nova-compute *should* be able to download the images reasonable fast over hight latency14:29
fultonjglance -> swift | nova/cinder -> ceph14:30
gfidenteowalsh is nova compute keeping the images cache after guests using such an image are deleted?14:30
fultonjglance caches to ephemeral local disk on edge14:30
gfidentefultonj yeah if glance in the edge goes to swift, then yes it all works14:30
owalshgfidente: not sure why it cleans up the images, I'll check14:30
gfidentefultonj the problem is if in central site we use ceph as well14:30
*** jfrancoa has joined #tripleo14:31
abishopfultonj: ack, that's what I do in my PoC (glance uses swift, which is enabled in central site)14:31
owalshs/why/when14:31
gfidenteabishop that explains it then14:31
gfidenteabishop glance/swift in central14:31
fultonjin addition to it being in Alan's the POC that set up is documented14:31
fultonj#link https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html#example-2-nodes-2-nic-using-remote-compute-with-tenant-and-provider-networks14:31
fultonj(which was one of the AIs)14:31
fultonje.g. "GlanceBackend: swift | GlanceCacheEnabled: true"14:32
slagleowalsh: can't you also pre-cache images for nova-compute?14:33
slagleand tell it to keep them around14:33
owalshslagle: not sure... saw something about that but I couldn't see an implementation14:33
fultonjslagle: so you're saying, why have glance cache if nova can cache?14:33
slaglefultonj: kind of wondering :)14:33
fultonjgood question14:33
owalshfultonj: and why cache if downloading is fast (enough) and bandwidth isn't constrained14:33
slaglehttps://blueprints.launchpad.net/nova/+spec/nova-image-cache-management-214:34
slagle2012 :). probably done and since been deprecated by now14:34
slaglebut we should look into it14:34
csatariHere is a spec for the image chaching in Glance: https://review.openstack.org/#/c/619638/14:34
fultonjVNF images are big14:34
fultonjtend to be14:34
*** hakhande_ has joined #tripleo14:35
owalshfultonj: cache still has to fetch it so it depends on how many computes nodes would be using it14:35
fultonjif they break you want to bring them up quickly, so i heard people asking for a solution to that problem14:35
gfidenteslagle well first because every nova node has its own cache, second because I think it deletes cached images when all guests using it are gone14:36
fultonjyes, our assumption is that one of them would be running14:36
fultonjonly the second one is fast14:36
fultonj(to boot)14:36
slaglegfidente: this spec/bp seems to indicate otherwise. but investigation is needed14:36
owalshnot necessarily faster - e.g if the center->edge link is 10gbit/s it might be the disk on the glance cache that limits it14:37
slagle#action continue investigating glance and nova image caching status/options14:37
gfidenteslagle doesn't look like it ever got implemented14:37
owalshalthough assuming the central glance is busier, it gives us a way to scale out glance14:37
gfidenteowalsh yeah scales as well14:38
gfidenteowalsh slagle but we could consider it optional14:38
owalshack, similar with nova cells which is why I'd combine the two in an edge controller role14:38
openstackgerritMerged openstack/tripleo-heat-templates master: Ensure /var/lib/iscsi actually exists before mounting it  https://review.openstack.org/62913214:38
slaglegfidente: there is also remove_unused_base_images which is a boolean14:39
gfidenteslagle yeah that I saw14:39
slagleok, well if anyone has time to dig into this a bit, i added an action item above14:39
owalshand maybe an edge controller + compute role, and maybe just and edge glance cache + compute role, plus all of these with ceph too... but that's getting a bit ridiculous14:39
fultonjowalsh: i don't think pulling from central will work in all cases so we should have some kind of localization answer for those who ask for it14:40
*** haleyb has joined #tripleo14:40
owalshfultonj: ack, definitely has some use cases14:41
*** haleyb is now known as haleyb|away14:41
*** jcoufal has joined #tripleo14:41
slagle#topic Goals/action items for the week14:42
*** openstack changes topic to "Goals/action items for the week (Meeting topic: TripleO Edge Squad Meeting)"14:42
* fultonj will be out next week14:43
slagleone thing i wanted to mention is that the CFP for the summit ends January 2314:43
slagle~2 weeks14:43
slagleif we want to submit anything tripleo/edge related14:43
slaglewe could brainstorm some ideas on the squad etherpad14:44
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: WIP: flatten the neutron service configurations  https://review.openstack.org/62966414:44
fultonjslagle: split control plane deployment demo w/ multiceph for summit?14:45
slaglefultonj: sure :)14:46
slagleanything else?14:47
slagle#endmeeting14:48
*** openstack changes topic to "CI Status: Green | https://docs.openstack.org/tripleo-docs/latest/"14:48
openstackMeeting ended Thu Jan 10 14:48:26 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:48
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-10-14.07.html14:48
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-10-14.07.txt14:48
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-10-14.07.log.html14:48
*** jchhatbar has joined #tripleo14:51
*** trown is now known as trown|outtypewww14:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - disable concurrency in docker-puppet.py  https://review.openstack.org/62954614:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - Add debug to docker-puppet  https://review.openstack.org/62925814:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62837914:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62838014:52
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62838114:52
sshnaidmarxcruz|ruck, ssbarnea|rover you know all ovb jobs fail, right?14:53
*** janki has quit IRC14:54
*** rh-jelabarre has joined #tripleo14:54
*** ykarel is now known as ykarel|away14:55
*** shardy_ has joined #tripleo14:57
arxcruz|rucksshnaidm: yes14:57
ssbarnea|roversshnaidm: in call now with arxcruz|ruck :D14:57
arxcruz|rucksshnaidm: there are a bug already opened https://bugs.launchpad.net/tripleo/+bug/181118514:58
openstackLaunchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] - Assigned to Tobias Urdin (tobias-urdin)14:58
sshnaidmarxcruz|ruck, cool, thanks14:58
*** shardy has quit IRC15:00
*** jchhatbar is now known as janki15:00
jaosoriorhjensas: your patch did the trick. we no longer have the cert issue15:00
*** itlinux has joined #tripleo15:01
hjensasjaosorior: cool, I think it needs some testing. What about ramishra_'s change? Did it work?15:03
jaosoriorhjensas: it didn't15:03
*** moguimar has joined #tripleo15:03
jaosoriorhjensas: we ran into some syntax issues. I don't think that syntax was possible when called through hiera15:03
*** dalvarez has joined #tripleo15:04
hjensasjaosorior: ok, maby I should drop the WIP on my patch and add puppet spec tests in a follow up?15:04
*** moguimar has quit IRC15:05
hjensasjaosorior: by, needs some testing I meant rspec tests in puppet.15:05
hjensas:)15:05
*** moguimar has joined #tripleo15:05
jaosoriorhjensas: yes! that sounds great15:05
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: DNM: test change  https://review.openstack.org/62568015:06
*** itlinux has quit IRC15:06
openstackgerritHarald Jensås proposed openstack/puppet-tripleo master: Filter apache_certificates_specs if hostname is empty  https://review.openstack.org/62983515:06
jaosoriorAnyway, let me know when the patch is ready so I can review it asap. Thanks for working on this.15:06
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: DNM: test fix for LP 1811185 broken admin url  https://review.openstack.org/62568015:06
openstackLaunchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] https://launchpad.net/bugs/1811185 - Assigned to Tobias Urdin (tobias-urdin)15:07
hjensasjaosorior: I think it's ready. Always good if some proper puppet coders like mwhahaha and EmilienM looks at it. ;)15:08
pandamwhahaha: is there user documentation on how to use the new method of building containers in TripleO ? That is, if a developers wants to contribute to the effort of fixing container building process, where does it start ?15:08
mwhahahapanda: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/3rd_party.html#building-new-containers-with-kolla-build15:09
mwhahahapanda: fixing container building?15:09
EmilienMhjensas: https://review.openstack.org/629835 .15:10
EmilienM?15:10
hjensasEmilienM: yes.15:10
mwhahahapanda: if by fixing container builds you mean for like python3, it's already in progress https://review.openstack.org/#/c/629679/15:10
jaosoriorhjensas: are you planning to further modify that patch? Or do you think it's ready?15:13
pandamwhahaha: I would expect other problems in this phase. So this command just needs an undercloud installed ?15:14
hjensasI will look into unit tests later + tomorrow. I can do that as a follow up if we want to merge that and get a backport downstream.15:14
mwhahahapanda: no just needs docker+tripleo-common+python-tripleoclient+kolla15:15
mwhahahapanda: though if you have an undercloud, that has everything15:15
pandamwhahaha: ok: setting up repos, then installing docker+tripleo-common+python-tripleoclient+kolla, then those docs.15:16
mwhahahapanda: yea, you'll need to configure a docker repo, but the ansible-role-container-registry will do that for you.15:16
mwhahahaso you can use that15:16
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Flatten rabbitmq service - step 1  https://review.openstack.org/62912015:18
mwhahahapanda: if you are looking to try out that python3 patch w/ a patched kolla, this is the playbook for ansible-role-container-registry and the kolla-build.conf http://paste.openstack.org/show/741582/15:19
mwhahahaadjust paths/users as necessary15:20
mwhahahapanda: note, we're currently missing fulentd, skydive and sensu packages so you have to exclude those from the build http://paste.openstack.org/show/741584/15:23
mwhahahai got them built yesterday install all of the items via source in a venv (rather than packages+patching)15:24
* mwhahaha wanders off15:24
Tenguhey folks, PTAL? https://review.openstack.org/#/c/629850/  thank you :)15:24
pandamwhahaha: mind if I create a etherpad with all the informations you unloaded here ?15:24
mwhahahapanda: go for it.15:30
openstackgerritJose Luis Franco proposed openstack/tripleo-common master: WIP: Split upgrade_tasks.yaml per step tasks.  https://review.openstack.org/62852215:31
*** jcoufal_ has joined #tripleo15:34
*** Vorrtex has joined #tripleo15:34
*** pbandark has joined #tripleo15:35
*** jcoufal has quit IRC15:36
openstackgerritTobias Urdin proposed openstack/tripleo-heat-templates master: Set keystone bind_host to both public and admin  https://review.openstack.org/62979815:44
*** ccamacho has quit IRC15:45
*** ccamacho has joined #tripleo15:46
openstackgerritMerged openstack/tripleo-heat-templates master: Add Distributed Compute roles  https://review.openstack.org/62656315:48
*** sudeepbatra has joined #tripleo15:48
EmilienMjaosorior: who is doing stein-2 release?15:50
EmilienMdeadline is tomorrow15:50
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Prepare stein-2  https://review.openstack.org/62988315:51
openstackgerritEmilien Macchi proposed openstack/tripleo-ui master: Prepare stein-2  https://review.openstack.org/62988415:51
pandaweshay: mwhahaha gathered information on how to locally build fedora28 containers https://etherpad.openstack.org/p/fedora28-tripleo-containers-build15:52
pandaweshay: user story updated and closed.15:52
weshaysweeet :)15:52
*** slaweq has joined #tripleo15:53
openstackgerritMerged openstack/paunch master: Default log location failes for non-root users  https://review.openstack.org/61801415:53
*** quiquell is now known as quiquell|off15:54
jaosoriorEmilienM: I will15:57
EmilienMjaosorior: I'm working on it no worries15:57
jaosorioroh15:57
jaosoriorthanks!15:57
EmilienMI'll send the patch in the next hour or so15:57
*** hakhande_ has quit IRC15:57
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Add Swift container sharder service  https://review.openstack.org/57415815:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429816:00
*** jcoufal_ has quit IRC16:00
*** jcoufal has joined #tripleo16:01
*** ksambor_ has quit IRC16:02
*** ratailor has joined #tripleo16:05
sudeepbatraused the below steps to deploy TripleO using Rocky >16:05
sudeepbatraexport RELEASE=rocky export VIRTHOST=127.0.0.2 time bash quickstart.sh      \      --teardown all  \  --release $RELEASE  \  --nodes /home/user/tripleo-quickstart/nodes.yaml  \  --config /home/user/tripleo-quickstart/config.yaml \  -e undercloud_undercloud_ntp_servers=147.117.251.28 \  --clean   \  $VIRTHOST16:05
sudeepbatraBut it does not install the overcloud as well16:05
*** udesale has quit IRC16:06
jaosoriorsudeepbatra: you gotta use --tags all16:09
fultonjsudeepbatra: or just ssh into your undercloud and deploy your overcloud with 'openstack overcloud deploy ...'16:10
*** sshnaidm has quit IRC16:10
fultonji personally use oooq to get me a working undercloud but once I have that undercloud I use it to my tripleo work16:10
*** sshnaidm has joined #tripleo16:11
TenguEmilienM or mwhahaha - care to make that one merge? That would make the audit.log really happy (and healthchecks for podman as well) :) https://review.openstack.org/#/c/629850/16:14
EmilienMTengu: looking now16:14
Tengu(at least healthchecks that are checking ports)16:15
TenguEmilienM: thanks :)16:15
Tengutomorrow I'll try to tackle the other selinux issues we have for healthchecks...16:16
*** slaweq has quit IRC16:19
ssbarnea|roverEmilienM: do you know if there is a way to avoid the "ulimit -n -1" warning/error we see with mysql?16:19
EmilienMssbarnea|rover: any logs handy?16:20
ssbarnea|roverEmilienM: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky/70f1c9b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-01-10_12_59_3416:20
ssbarnea|roveri think is. only a warning but is annoying / confusing.16:20
ssbarnea|roveri would prefer to eradicate it if possible.16:21
sudeepbatrafultonj, so if I want to do openstack overcloud deploy --templates /home/stack/openstack-tripleo-heat-templates , do I need to clone and use https://github.com/openstack/tripleo-heat-templates ?16:21
fultonjsudeepbatra: no, the appropriate version of THT will already be present on the undercloud in /usr/openstack/tripleo-heat-templates16:22
fultonjnow that you have a working undercloud you can use it16:22
EmilienMssbarnea|rover: it sounds like a bug in mysqld_safe16:22
fultonjhttps://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html16:22
fultonjsudeepbatra: ^16:22
fultonjsudeepbatra: you should already have your images too16:22
ssbarnea|roverEmilienM: exactly, but I checked the code and is likely a param we give.16:22
fultonjsudeepbatra: 'source stackrc' and 'openstack baremetal node list'16:23
fultonjso intropsection should be done16:23
fultonjsudeepbatra: you should be able to just start here...16:23
fultonjhttps://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html#configure-a-nameserver-for-the-overcloud16:23
sudeepbatraok let me check, by the way I used your config.yaml and nodes.yaml from oooq (https://github.com/fultonj/oooq/blob/master/under/nodes.yaml ) but still needed to specify the  " -e undercloud_undercloud_ntp_servers=147.117.251.28"16:24
fultonjsudeepbatra: yep16:24
sudeepbatraotherwise was getting error. ok :)16:24
fultonjsudeepbatra: i recommend a simple overcloud one compute and one controller to start with16:24
fultonjonce that's working grow it as you like16:24
*** ccamacho has quit IRC16:33
fultonjslagle: good news regarding the CI job it's geting closer. latest comments in https://review.openstack.org/#/c/617368/1616:34
fultonjabishop gfidente ^ fyi16:34
slaglefultonj: looking16:34
*** itlinux has joined #tripleo16:34
fultonjslagle: i have an idea on how to change it so it doesn't rely and those booleans and instead just checks if /home/zuul/export_control_plane.tar.gz exists and then if it does it uses it for a compute deploy16:36
*** slaweq has joined #tripleo16:36
fultonjthose booleans --> split_control_plane_compute in https://review.openstack.org/#/c/617368/16/roles/standalone/templates/standalone.sh.j216:37
openstackgerritMerged openstack/python-tripleoclient master: Remove DhcpStart and DhcpEnd - Replaced by AllocationPools  https://review.openstack.org/62618816:39
*** hberaud has quit IRC16:41
*** suuuper has quit IRC16:42
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Update auth parameters  https://review.openstack.org/62948416:45
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Update auth parameters  https://review.openstack.org/62948516:45
openstackgerritMerged openstack/tripleo-validations master: Update the bugs link to storyboard  https://review.openstack.org/62783816:45
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: update toci_gate_test for no tebroker  https://review.openstack.org/62675816:46
openstackgerritSorin Sbarnea proposed openstack/puppet-tripleo master: Replace mysql invalid open_files_limit of -1 with 65536  https://review.openstack.org/62989716:46
ssbarnea|roverEmilienM: ^^ not sure if this is the correct fix, but it should be. The only place where I was able to spot the invalid -1 value.16:47
jankiEmilienM, can you take a look at https://review.openstack.org/#/c/603532/ plz. needs w+ probably16:48
*** slaweq has quit IRC16:48
EmilienMssbarnea|rover, janki : looking16:52
*** jcoufal has quit IRC16:52
jankiEmilienM, thanks :)16:54
*** ekultails has joined #tripleo16:57
*** maufart__ has quit IRC17:00
*** jcoufal has joined #tripleo17:02
*** avivgt has quit IRC17:07
jankiEmilienM, also https://review.openstack.org/#/c/602488/ if you have bandwidth17:07
hjensascores, can we land the last patches for this bp before stein-2: https://review.openstack.org/#/q/topic:bp/tripleo-routed-networks-templates+(status:open)17:07
openstackgerritIlya Etingof proposed openstack/python-tripleoclient master: Undercloud: support inspect interface for redfish  https://review.openstack.org/62990517:10
*** dtantsur is now known as dtantsur|afk17:12
*** ratailor has quit IRC17:12
*** paramite has quit IRC17:13
*** paramite has joined #tripleo17:19
*** gkadam has quit IRC17:21
*** paramite has quit IRC17:23
*** paramite has joined #tripleo17:23
*** rh-jelabarre has quit IRC17:24
*** mjturek has joined #tripleo17:27
*** fragatina has joined #tripleo17:27
*** psachin has quit IRC17:28
*** jpich has quit IRC17:32
*** jpena is now known as jpena|off17:32
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Make ODL deployment compatible with podman  https://review.openstack.org/60248817:33
*** paramite has quit IRC17:34
*** thrash is now known as thrash|biab17:35
NobodyCamGood morning TripleO folks :)17:36
jaosoriorGood evening!17:37
NobodyCam:)17:41
*** ykarel|away has quit IRC17:43
NobodyCamI tend to use UGT time :p17:43
*** shardy_ has quit IRC17:53
*** panda is now known as panda|off17:53
*** janki has quit IRC17:54
*** ayoung has joined #tripleo17:54
*** janki has joined #tripleo17:54
*** derekh has quit IRC18:00
*** bogdando has quit IRC18:00
*** thrash|biab is now known as thrash18:07
NobodyCamWhere is the best place to file bug's for tripleO-ui?18:08
*** kopecmartin is now known as kopecmartin|off18:13
*** morazi has quit IRC18:16
*** jrist is now known as jrist__18:16
*** gfidente is now known as gfidente|afk18:17
*** jrist has joined #tripleo18:17
*** jrist__ has quit IRC18:18
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Switch scenario001-standalone to Ceph bluestore  https://review.openstack.org/54768218:19
sudeepbatrathanks fultonj, started the overcloud deployment , sfsg >18:20
sudeepbatraopenstack overcloud deploy --templates  --libvirt-type qemu -e environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml --ntp-server 147.117.251.28 --ceph-storage-scale 3 -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml18:20
*** janki has quit IRC18:20
sudeepbatrais there a way to install TripleO quickstart on any public cloud like Google Cloud ?18:21
jaosoriorsudeepbatra: I think someone has done it on vexxhost18:23
sudeepbatraright now I am doing on a single baremetal node as Host machine, but both undercloud and overcloud as VMs but when I tried todo on all-in-one-VM - it gave error18:25
*** brault has quit IRC18:25
weshayjaosorior, gate failures seem to be.. podman and pulling from docker.io18:25
jaosoriorweshay: what's failing from podman?18:26
jaosoriorweshay: and are we getting rate-limited by docker.io?18:26
weshayjaosorior, http://status.openstack.org/elastic-recheck/#180443418:26
jaosorioroho18:26
weshayI have no idea re: the rate limiting but I don't think so18:27
*** brault has joined #tripleo18:27
sudeepbatrajaosorior , any links ?18:29
jaosoriorno, sorry18:29
jaosorioruhm18:29
weshayjaosorior, docker issue is here.. http://status.openstack.org/elastic-recheck/#1805186  however it's not reporting all times this occurs afaict18:29
fultonjsudeepbatra: i use an all in one host (a machine w/ 64G RAM) to host my undercloud and my overcloud nodes for development18:29
jaosoriorbnemec: did yuo have a blog post or something about trying out quickstart in vexxhost?18:29
weshay0 fails in 24 hrs / 8 fails in 10 days is wrong18:30
fultonjsudeepbatra: if you want openstack with an all-in-one VM then try the tripleo standalone installer18:31
bnemecjaosorior: http://blog.nemebean.com/content/openstack-virtual-baremetal-public-cloud18:31
jaosoriorsudeepbatra: ^^18:31
*** brault has quit IRC18:31
fultonjsudeepbatra: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html18:31
sudeepbatrafultonj, thanks so this steps can be done on a all-in-one VM (lets say 32GB RAM ) ?18:33
fultonjsudeepbatra: yes18:33
fultonjsudeepbatra: e.g i did it on an 8 G VM on my laptop https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html18:34
fultonjsudeepbatra: in that case the "undercloud"  and overcloud are all on one node18:34
fultonjyou can do it with ceph too18:34
fultonjit just puts the ceph containers there also18:34
sudeepbatraok , let me go thru18:37
fultonjsudeepbatra: quickstart lets you more closely simulate a production deployment because it's using Ironic to booth VMs as if they were baremetal18:39
fultonjit's a matter of picking the right way to use tripleo for your goal18:39
fultonjfor baremetal you could just follow the docs directly and install the undercloud and then install the overcloud18:40
*** wolverineav has joined #tripleo18:48
*** wolverineav has quit IRC18:48
*** wolverineav has joined #tripleo18:48
*** stee has joined #tripleo18:49
sudeepbatrafultonj , thanks - the NTP is troubling again in my deployment > fatal: [overcloud-controller-2]: FAILED! => {"changed": true, "cmd": ["ntpdate", "-u", "147.117.251.28"]18:50
sudeepbatraits working on (undercloud) [stack@undercloud ~]$ sudo ntpdate 147.117.251.28 10 Jan 18:45:50 ntpdate[528048]: the NTP socket is in use, exiting18:50
fultonjsudeepbatra: i suspect that overcloud-controller-2 is unable to reach 147.117.251.28 you need to configure networking so that it can18:51
fultonjsudeepbatra: my back for this is to run this on the undercloud https://github.com/fultonj/oooq/blob/master/over/route_hack.sh18:51
sudeepbatraok18:52
fultonjif you run that and hop on overcloud-controller-2 try to ping the ntp server18:52
fultonjif it responds, then resume 'openstack overcloud deploy ...'18:52
fultonjif not, then troubleshoot the network18:52
fultonjtraceroute to the ntp server from overcloud-controller-218:53
sudeepbatraok let me try18:53
*** mcornea has quit IRC18:54
*** rh-jelabarre has joined #tripleo18:57
*** mcornea has joined #tripleo18:59
*** pbandark has quit IRC19:03
bnemecsudeepbatra: fultonj: Not sure if this is relevant to you, but when I deployed recently with network isolation enabled I found I needed another firewall rule on the undercloud:19:03
bnemecsudo iptables -I FORWARD -s 10.0.0.0/24 -d 10.0.0.0/24 -j ACCEPT19:03
fultonjbnemec: did you have to start doing this in Rocky?19:06
slaglefultonj: https://review.openstack.org/#/c/617368/16/roles/standalone/templates/standalone.sh.j219:08
bnemecfultonj: Not sure. I only noticed it recently, but I don't do as many net-iso deployments these days as I used to so I'm not sure when it started.19:08
*** holser_ has quit IRC19:08
*** abishop has quit IRC19:10
*** brault has joined #tripleo19:10
*** rascasoft has quit IRC19:12
*** morazi has joined #tripleo19:13
fultonj{% if bool(split_control_plane_compute) %}19:13
fultonjslagle: thanks this is what you had in mind ^19:13
fultonj?19:13
*** brault has quit IRC19:14
slaglefultonj: yes, although the 2nd comment i added shows the convention used elsewhere in the oooq roles19:15
sudeepbatra[heat-admin@overcloud-controller-2 ~]$ sudo ntpdate 147.117.251.28 10 Jan 19:14:53 ntpdate[17325]: step time server 147.117.251.28 offset -87.445023 sec19:15
sudeepbatraresuming the overcloud deployment19:15
slaglefultonj:  so it would be {% if split_control_plane_compute|bool %}19:15
fultonjack19:15
slaglefultonj: i'm still unsure why these plays produced no output: http://logs.openstack.org/88/615988/7/check/tripleo-ci-split-controlplane-standalone/f101235/job-output.txt.gz#_2019-01-10_15_00_30_02654019:16
slaglebut they seemed to do the tasks19:16
fultonjslagle: i'm wondering if they didn't do the tasks19:17
fultonjbecause control plane produced them on undercloud and the second control plane produced them on subnode219:17
slaglefultonj: the tarball is there though19:17
slaglefultonj: oh, hmm. suppose so19:17
fultonjshould be created on uc rsync'd to subnode219:18
fultonjbut we got created and created19:18
fultonjlet's try the bool filters19:18
openstackgerritJohn Fulton proposed openstack/tripleo-quickstart-extras master: WIP/DNM: Add split-controlplane-standalone.yml playbook  https://review.openstack.org/61736819:18
fultonjthanks slagle19:19
slaglefultonj: maybe b/c line 51 is not indented?19:20
slaglethat might be it19:20
openstackgerritJohn Fulton proposed openstack/tripleo-quickstart-extras master: WIP/DNM: Add split-controlplane-standalone.yml playbook  https://review.openstack.org/61736819:24
fultonjslagle: indented 51 ^19:24
*** moguimar has quit IRC19:26
*** moguimar has joined #tripleo19:27
*** brault has joined #tripleo19:31
*** moguimar has quit IRC19:32
*** brault has quit IRC19:36
*** moguimar has joined #tripleo19:36
*** moguimar has quit IRC19:41
*** moguimar has joined #tripleo19:43
*** moguimar has quit IRC19:48
sudeepbatrafultonj , got error > mistral |  fatal: [undercloud]: FAILED! =>19:59
sudeepbatra"ERROR! the playbook: /usr/share/ceph-ansible/site-docker.yml.sample could not be found"20:00
fultonjsudeepbatra: 'yum install ceph-ansible' on the undercloud20:00
sudeepbatraok :)20:00
*** dprince has quit IRC20:03
*** jfrancoa has quit IRC20:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - disable concurrency in docker-puppet.py  https://review.openstack.org/62954620:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - Add debug to docker-puppet  https://review.openstack.org/62925820:06
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62837920:07
openstackgerritMerged openstack/tripleo-ui master: Prepare stein-2  https://review.openstack.org/62988420:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62838020:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - podman testing  https://review.openstack.org/62838120:07
*** gfidente|afk has quit IRC20:26
*** mcornea has quit IRC20:28
*** zul has quit IRC20:35
openstackgerritMerged openstack-infra/tripleo-ci master: Make scenario004-standalone voting too  https://review.openstack.org/62985120:41
openstackgerritMerged openstack/puppet-tripleo master: Add scenario002 standalone to gate as well as it is voting  https://review.openstack.org/62985420:42
openstackgerritMerged openstack/tripleo-common master: Add scenario002-standalone and 4 to gates as they are voting  https://review.openstack.org/62985320:42
openstackgerritMerged openstack/tripleo-quickstart master: Add scenario002 standalone to gate as well as it is voting  https://review.openstack.org/62985520:42
*** dprince has joined #tripleo20:43
*** mcornea has joined #tripleo20:44
openstackgerritMerged openstack/tripleo-heat-templates master: Add scenario002-standalone to gates as we make it voting  https://review.openstack.org/62985220:46
openstackgerritMerged openstack/puppet-tripleo master: Prepare stein-2  https://review.openstack.org/62988320:46
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Mount the public TLS certificate for HAProxy on up(date|grade) on pacemaker  https://review.openstack.org/61770120:46
openstackgerritMerged openstack/tripleo-heat-templates master: Allow container healthchecks to access netlink data  https://review.openstack.org/62985020:47
*** openstackgerrit has quit IRC20:50
sudeepbatragot the error on ceph-0 > failed: [overcloud-cephstorage-0] (item=/dev/vdb) => {\"changed\": false, \"err\": \"Error: Could not stat device /dev/vdb - No such file or directory.\\n\", \"item\": \"/dev/vdb\", \"msg\": \"Error while getting device information with parted script: '/sbin/parted -s -m /dev/vdb -- unit 'MiB' print'\", \"out\": \"\", \"rc\": 1}",20:53
sudeepbatrafultonj ^20:53
fultonjsudeepbatra: does /dev/vdb exist on your overcloud ceph-0 node?20:54
fultonjto pass parameters defining which devices you want to as OSDs see https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ceph_config.html#configure-osd-settings-with-ceph-ansible20:54
*** abishop has joined #tripleo20:57
*** abishop has quit IRC20:58
*** abishop has joined #tripleo20:59
*** jcoufal has quit IRC21:05
*** openstackgerrit has joined #tripleo21:07
openstackgerritSteve Baker proposed openstack/tripleo-common master: Make python uploader the default  https://review.openstack.org/61601921:07
openstackgerritSteve Baker proposed openstack/tripleo-common master: Export images to be served by httpd  https://review.openstack.org/62262821:07
openstackgerritSteve Baker proposed openstack/tripleo-common master: Role to install apache to serve container images  https://review.openstack.org/62482921:07
*** radeks_ has quit IRC21:12
openstackgerritMerged openstack/tripleo-heat-templates master: Allow overlay tunnel endpoints on IPv6 address  https://review.openstack.org/60353221:13
openstackgerritMerged openstack/tripleo-upgrade stable/rocky: Autodiscover SSL cert after uc upgrade  https://review.openstack.org/62579921:13
openstackgerritMerged openstack/tripleo-heat-templates master: Apache - Use net_cidr_map for proxy_ips  https://review.openstack.org/61964621:13
openstackgerritMerged openstack/tripleo-heat-templates master: Designate - Use net_cidr_map for rndc_allowed_addresses  https://review.openstack.org/61964721:13
openstackgerritMerged openstack/tripleo-heat-templates master: Memcached - Use net_cidr_map for firewall rules  https://review.openstack.org/61964821:13
openstackgerritMerged openstack/tripleo-heat-templates master: Snmp - Use net_cidr_map for firewall rules  https://review.openstack.org/61964921:13
*** chem has quit IRC21:13
openstackgerritMerged openstack/puppet-tripleo master: Filter apache_certificates_specs if hostname is empty  https://review.openstack.org/62983521:13
*** radeks has joined #tripleo21:15
sudeepbatrafultonj , do I need to update the ceph-osd.yaml ? > http://paste.openstack.org/show/741621/21:17
fultonjsudeepbatra: if you have ceph-ansible 3.2 it should work with the settings from the doc21:18
fultonjsudeepbatra: what disks do you have on ceph-node2 ?21:18
fultonj /dev/vdb, /dev/vdc, etc?21:19
fultonjyou should only put in disks that are actually on the system21:19
fultonjlsblk will show you21:19
sudeepbatrawill ssh into ceph node and check, its shd be default since all are VM21:19
sudeepbatrahmm, vda only so need to update the ceph-osd.yaml21:21
sudeepbatralet me do that an rerun21:21
fultonjvda is your OS drive21:22
fultonjyou can't use vda21:22
fultonjyou can install ceph in your only have 1 disk21:22
fultonjyou can not install ceph in your only have 1 disk21:22
fultonji mean21:22
sudeepbatraI have another Disk on the Baremetal Host21:22
fultonjif you used that nodes.yml file, then one of your other VMs has more disks21:23
fultonjyou need to make sure the node with the disks is your ceph node21:24
fultonjthe docs cover how to do this21:24
fultonjhere https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/profile_matching.html21:24
*** rlandy is now known as rlandy|biab21:25
fultonjsudeepbatra: i think ther shoul already by an oercloud_deploy.sh or similar21:25
fultonjset up by oooq21:25
fultonjin /home/stack on undercloud21:25
fultonjhave a look at it and see that it's using a flavor like oooq_ceph already21:26
fultonjyou might just want to run the overcloud_deploy script that oooq made21:26
*** Vorrtex has quit IRC21:27
sudeepbatrano overcloud_deploy.sh on undercloud21:29
fultonjls /home/stack/21:30
sudeepbatraah its overcloud-deploy.sh :)21:31
sudeepbatrait has it > --ceph-storage-flavor oooq_ceph21:32
sudeepbatralet me run it21:32
fultonjyep21:32
*** ramishra_ has quit IRC21:37
sudeepbatraError: The following files were not found: /home/stack/containers-default-parameters.yaml > http://paste.openstack.org/show/741625/21:37
fultonjrun the containers prep script21:39
*** ansmith_ has quit IRC21:40
fultonjfyi ceph-ansible 3.2 for the undercloud is at https://cbs.centos.org/koji/buildinfo?buildID=2472621:40
*** radeks has quit IRC21:42
openstackgerritDouglas Mendizábal proposed openstack/tripleo-heat-templates master: Update Barbican HSM ansible roles  https://review.openstack.org/62993721:43
*** pcaruana has quit IRC21:46
*** holser_ has joined #tripleo21:50
*** toure is now known as toure|gone21:50
*** dprince has quit IRC21:51
openstackgerritDouglas Mendizábal proposed openstack/tripleo-heat-templates master: Update Barbican HSM ansible roles  https://review.openstack.org/62993721:51
*** mjturek has quit IRC21:55
*** ekultails has quit IRC22:03
sudeepbatrafultonj , NotSupported: resources.VipPort: Update to properties network of VipPort (OS::TripleO::Network::Ports::ControlPlaneVipPort) is not supported > http://paste.openstack.org/show/741626/22:11
sudeepbatrawill try again tomorrow - using --tag all22:11
fultonji don't know a quick fix for ^22:11
fultonjyou'll have to read the tripleo docs on network isoation and debug22:11
sudeepbatraok ,np - thanks for great help22:11
openstackgerritDouglas Mendizábal proposed openstack/tripleo-heat-templates master: Update Barbican HSM ansible roles  https://review.openstack.org/62993722:13
openstackgerritMerged openstack-infra/tripleo-ci master: update toci_gate_test for no tebroker  https://review.openstack.org/62675822:14
*** holser_ has quit IRC22:20
*** sudeepbatra has quit IRC22:27
*** ade_lee has quit IRC22:34
*** rlandy|biab is now known as rlandy22:36
*** rcernin has joined #tripleo22:38
*** lblanchard has quit IRC22:53
*** itlinux has quit IRC22:55
*** mcornea has quit IRC23:04
*** jbadiapa has quit IRC23:10
*** ansmith_ has joined #tripleo23:11
*** ade_lee has joined #tripleo23:11
*** ade_lee has quit IRC23:21
*** ade_lee has joined #tripleo23:21
*** lblanchard has joined #tripleo23:26
*** lblanchard has quit IRC23:26
openstackgerritMerged openstack/tripleo-heat-templates master: Set keystone bind_host to both public and admin  https://review.openstack.org/62979823:29
openstackgerritMerged openstack/tripleo-heat-templates master: Restart openshift master services after stack update  https://review.openstack.org/62401123:31
openstackgerritMerged openstack/tripleo-heat-templates master: Move docker into deployment directory  https://review.openstack.org/62417723:35
*** lblanchard has joined #tripleo23:42
*** lblanchard has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!