Tuesday, 2019-02-12

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722400:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/pike: GATE CHECK for TripleO  https://review.openstack.org/60224800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429300:00
*** cfontain has joined #tripleo00:04
*** wolverineav has quit IRC00:04
*** wolverineav has joined #tripleo00:05
*** wolverineav has quit IRC00:10
*** rh-jelabarre has quit IRC00:11
*** rh-jelabarre has joined #tripleo00:13
*** wolverineav has joined #tripleo00:14
*** tosky has quit IRC00:18
*** rlandy is now known as rlandy|bbl00:36
*** mcornea has quit IRC00:37
*** wolverineav has quit IRC00:37
*** rascasoft has joined #tripleo00:38
*** wolverineav has joined #tripleo00:39
*** wolverineav has quit IRC00:43
*** wolverineav has joined #tripleo00:44
*** cfontain has quit IRC00:44
*** rascasoft has quit IRC00:45
*** hrybacki has joined #tripleo00:59
*** mnasiadka has joined #tripleo00:59
*** csatari has joined #tripleo00:59
*** TheJulia has joined #tripleo00:59
*** rh-jelabarre has quit IRC00:59
*** preece has joined #tripleo00:59
*** sdake has joined #tripleo01:01
*** kplant has joined #tripleo01:03
*** wolverineav has quit IRC01:09
*** wolverineav has joined #tripleo01:09
kplantis there a method of having tripleo perform some customization post-deployment? for instance some changes in local_settings.py for horizon or changes in nova.conf?01:18
kplantsomething like OS::TripleO::ControllerExtraConfigPre s/Pre/Post would be awesome but i don't see that documented anywhere i've looked01:19
*** _fragatina has quit IRC01:37
*** sdake has quit IRC01:37
*** sdake has joined #tripleo01:41
*** sdake has quit IRC01:43
*** wolverineav has quit IRC01:49
*** sdake has joined #tripleo01:50
*** sdake has quit IRC01:51
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: warnings should be first and in red  https://review.openstack.org/63602801:51
*** wolverineav has joined #tripleo01:52
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431701:52
*** wolverineav has quit IRC01:56
*** kplant has quit IRC01:59
*** kplant_ has joined #tripleo02:01
*** kplant_ is now known as kplant02:02
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: [DNM] Test tripleo-buildcontainers on py3/f28  https://review.openstack.org/63618802:09
*** rascasoft has joined #tripleo02:20
*** rascasoft has quit IRC02:26
*** mschuppert has quit IRC02:30
openstackgerritMerged openstack/tripleo-puppet-elements master: Use external centos7-rt if present  https://review.openstack.org/63513502:34
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431702:36
*** cfontain has joined #tripleo02:42
*** saneax has joined #tripleo02:50
*** saneax has quit IRC02:52
*** saneax has joined #tripleo02:53
*** rascasoft has joined #tripleo02:57
*** kplant has quit IRC03:03
*** rascasoft has quit IRC03:07
*** rlandy|bbl is now known as rlandy03:12
*** cfontain has quit IRC03:15
*** rlandy has quit IRC03:19
*** ykarel has joined #tripleo03:26
ykarelstevebaker, the ceph tasks still failing03:33
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431703:33
ykarelstevebaker, looks like it needs become: true for ceph_base_external_deploy_task03:33
ykarelfor current failure03:34
ykarelhttp://logs.openstack.org/86/636086/7/check/tripleo-ci-centos-7-scenario010-multinode-oooq-container/d53d091/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-12_01_38_0103:34
*** dmacpher_ has joined #tripleo03:34
*** dmacpher has quit IRC03:34
ykarelstevebaker, ok /me pushing changes to make some progress before quiquell|off or you takes over03:39
*** sdake has joined #tripleo03:40
openstackgerrityatin proposed openstack/tripleo-heat-templates master: Make ceph-ansible working dir owned by tripleo-admin  https://review.openstack.org/63608603:43
*** homeski has joined #tripleo03:44
*** thrash is now known as thrash|g0ne03:44
*** cfontain has joined #tripleo03:45
*** homeski has quit IRC03:47
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WIP - wes, Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63599103:52
openstackgerritwes hayutin proposed openstack/tripleo-common master: Add python3 packages to overrides  https://review.openstack.org/62967903:58
*** sdake has quit IRC03:59
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63599104:01
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WES WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63626304:02
*** udesale has joined #tripleo04:02
*** ramishra has joined #tripleo04:06
*** vinaykns has quit IRC04:07
*** cfontain has quit IRC04:17
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WES WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63626304:19
*** rascasoft has joined #tripleo04:23
openstackgerritwes hayutin proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431704:23
*** gbarros has quit IRC04:28
*** janki has joined #tripleo04:30
*** rascasoft has quit IRC04:32
*** weshay has quit IRC04:34
*** saneax has quit IRC04:42
*** ykarel has quit IRC04:47
*** homeski has joined #tripleo04:56
*** shyamb has joined #tripleo05:03
shyambHi05:03
shyambjaosorior: How you doing?05:03
*** dmacpher_ has quit IRC05:03
shyambWe have started integrating triliovault deployment with osp1405:04
shyamb+ director05:04
shyambWe already have integration scripts with osp1305:05
shyambI read, tripleo in osp14 has heavily used ansible instead of puppet05:05
*** ykarel has joined #tripleo05:05
shyambIs puppet scripts deprecated then05:06
shyamb?05:06
*** homeski has quit IRC05:06
shyambTengu: ?05:06
*** ratailor has joined #tripleo05:08
openstackgerritjacky06 proposed openstack/diskimage-builder master: Update hacking version to latest  https://review.openstack.org/62846605:09
openstackgerritjacky06 proposed openstack/diskimage-builder master: Update hacking version to latest  https://review.openstack.org/62846605:10
*** shyamb has quit IRC05:15
*** pdeore has joined #tripleo05:19
*** dmacpher has joined #tripleo05:32
jaosoriorGood morning folks!05:36
*** udesale has quit IRC05:37
*** chandankumar is now known as chkumar|ruck05:47
*** wolverineav has joined #tripleo05:55
*** skramaja has joined #tripleo06:05
*** cfontain has joined #tripleo06:15
openstackgerritmelissaml proposed openstack/tripleo-specs master: fix a typo in specs/policy/expedited-approvals.rst  https://review.openstack.org/62522006:16
*** rascasoft has joined #tripleo06:18
*** rascasoft has quit IRC06:24
openstackgerritRabi Mishra proposed openstack/paunch master: Keep the old paunch behaviour  https://review.openstack.org/63438306:33
*** wolverineav has quit IRC06:38
*** sshnaidm is now known as sshnaidm|afk06:39
*** hakhande_ has joined #tripleo06:42
*** yprokule has joined #tripleo06:43
*** saneax has joined #tripleo06:45
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add supplemental node to inventory  https://review.openstack.org/63609406:46
openstackgerritRabi Mishra proposed openstack/paunch master: Keep the old paunch behaviour  https://review.openstack.org/63438306:47
*** ccamacho has quit IRC06:49
*** cfontain has quit IRC06:50
*** zaneb has quit IRC06:53
*** ksambor has joined #tripleo06:57
*** bandini has joined #tripleo06:58
quiquell|offchkumar|ruck: hi07:01
Tenguhello there07:01
quiquell|offykarel: o/07:01
quiquell|offTengu: hello07:01
*** cfontain has joined #tripleo07:01
Tengu«o/ quiquell|off07:01
*** quiquell|off is now known as quiquell|rover07:02
ykarelquiquell|rover, o/07:02
chkumar|ruckquiquell|rover: \o/07:02
quiquell|roverchkumar|ruck: found the issue with the modify image07:02
chkumar|ruckquiquell|rover: yes, thanks for looking into it07:03
quiquell|roverNo problem07:03
quiquell|roverAt least you can advance now07:03
quiquell|roverThe error was super counterintuitive07:03
chkumar|ruckquiquell|rover: as per stevebaker comment https://review.openstack.org/634904 will I move the yum_update.sh script from template to previous script?07:04
quiquell|roverWell not sure since we need the PKG detection in ansible too07:04
quiquell|roverMoving to a file means duplicate it07:04
Tengugrmbl. CI playing bad games on me.07:05
quiquell|roverTengu: whats the issue?07:05
chkumar|ruckquiquell|rover: I will comment on it and update it07:05
quiquell|roverAck07:05
Tenguquiquell|rover: one of my changes can't merge due to POST_FAILURE. each time.07:05
Tenguhttps://review.openstack.org/#/c/635785/507:05
*** kopecmartin|off is now known as kopecmartin07:05
quiquell|roverchkumar|ruck: btw you can use ansible ternary to simplify it07:06
chkumar|ruckquiquell|rover: yes on it07:06
quiquell|roverchkumar|ruck: but he is right about the Dockerfile07:06
quiquell|roverAbout complexity and entrypoint07:06
*** rascasoft has joined #tripleo07:07
chkumar|ruckquiquell|rover: I am going to remove entrypoint that is not needed07:07
chkumar|ruckquiquell|rover: we added it based on error we were getting from builadh07:08
*** udesale has joined #tripleo07:09
quiquell|roverchkumar|ruck: ack about complexity maybe is not a bad option to put that in the scri0t07:09
chkumar|ruckquiquell|rover: dnf/yum detection should be taken care in the dockerfile itself07:09
quiquell|roverTengu: we have same failures at gates07:09
ykarellooks like collect-logs is running twice07:10
Tenguquiquell|rover: yeah, I know, it failed there 2 recheck earlier.07:11
chkumar|ruckykarel: sshnaidm|afk has a patch for that07:11
ykarelchkumar|ruck, that was for ovb i think07:12
chkumar|rucksshnaidm|afk: yes07:12
chkumar|rucksorry07:12
chkumar|ruckykarel: yes07:12
quiquell|roverchkumar|ruck: the post failure?07:14
chkumar|ruckykarel: quiquell|rover https://review.openstack.org/#/c/635510/07:14
ykarelquiquell|rover, yup the logs which Tengu shared07:14
ykarelin the review which Tengu shared07:14
ykareli checked logs,07:15
ykarelthe POST is timing out07:15
ykarelpost playbook07:15
quiquell|roverAck give me a min and I will look07:16
quiquell|roverIt's going to be the today's issue07:16
*** aufi has joined #tripleo07:16
Tengu:D07:17
TenguI'm sure you're happy07:17
quiquell|roverhttps://review.openstack.org/#/c/63515707:18
quiquell|roversshnaidm|afk, ykarel: can we merge this? Depends-On has merge07:18
ykarellooking07:18
quiquell|roverSo we close the bug07:18
quiquell|roverI will do some follow up reviews at puppet-elements branches07:20
ykarel+1 from me07:20
quiquell|roverykarel: thanks07:20
*** jfrancoa has joined #tripleo07:21
ykarelquiquell|rover, i pushed a patch morning for ceph thing07:21
ykarelto make some progress, seems atleast fs016,019 should be good, checking here:- https://review.openstack.org/#/c/636088/07:22
ykarelwaiting for results07:22
ykarelshould be available in some time, so don't push update to fix octavia until then07:22
ykarelscenario010 current issue07:23
*** agurenko has joined #tripleo07:23
*** cfontain has quit IRC07:24
*** cfontain has joined #tripleo07:25
quiquell|roverykarel: then we activate also the master.ini criteria ?07:26
quiquell|roverykarel: do we need to do the same at rocky ?07:26
quiquell|roverykarel: humm no we have rocky promotions07:26
*** mschuppert has joined #tripleo07:26
ykarelquiquell|rover, for which job?07:26
quiquell|roverykarel: we have still scenario010 failure :-/07:27
bandinihjensas: https://bugzilla.redhat.com/show_bug.cgi?id=1321681 fyi07:27
openstackbugzilla.redhat.com bug 1321681 in rhosp-director "RFE: Enable rabbitmq management plugin" [Medium,Assigned] - Assigned to michele07:28
bandiniaka I have not spent time in ages to look at that07:28
quiquell|roverykarel: will look into scenario010 issue to see what's the next stuff to do07:28
ykarelquiquell|rover, i don't know about that job's plan, so can't say anything about it's inclusion in promotion criteria, The general criteria i am aware is:- if jobs are running in upstream gate and are good then should be running in promotion pipeline and07:28
ykarelquiquell|rover, but before pushing update to patch wait for results of scenario001/00407:29
quiquell|roverykarel: but I think they are going to fail at same point07:29
ykarelquiquell|rover, at what point?07:29
ykarelthose should be good or atleast progress with latest PS07:30
*** holser_ has joined #tripleo07:31
*** ykarel is now known as ykarel|lunch07:33
bandinihjensas: I am thinking that it might be simpler to enable it now that we use containers07:34
openstackgerritRabi Mishra proposed openstack/paunch master: Keep the old paunch behaviour  https://review.openstack.org/63438307:34
quiquell|roverykarel|lunch: 001 and 004 does not have octavia ?07:37
*** ramishra_ has joined #tripleo07:37
*** ramishra has quit IRC07:38
openstackgerritQuique Llorente proposed openstack/tripleo-puppet-elements stable/rocky: Use external centos7-rt if present  https://review.openstack.org/63627707:40
openstackgerritQuique Llorente proposed openstack/tripleo-puppet-elements stable/queens: Use external centos7-rt if present  https://review.openstack.org/63627807:40
*** ykarel|lunch has quit IRC07:41
quiquell|roverowalsh: looks like pike/ocata does not use centos7-rt07:43
*** ykarel has joined #tripleo07:43
Tenguquiquell|rover: do you want some help with the post_failure?07:43
*** slaweq has joined #tripleo07:44
quiquell|roverTengu: let me reproduce it07:47
quiquell|roverTengu: we don't have needed stuff to find issue07:47
Tenguok. my change seems to be pretty steady in the reproduction :/07:47
quiquell|roverTengu: ack let's try the zuul based one, it's more realistic07:48
Tengu:)07:48
quiquell|roverTengu: humm noop change works fine https://review.openstack.org/#/c/560445/07:49
quiquell|roverThat's weird...07:49
*** ksambor has quit IRC07:50
Tenguhmm. my patch doesn't really add new logs. so it's not a question of timing in the collect or related things.07:52
quiquell|roverTengu: let's see if we can reproduce the issue07:52
quiquell|roverTengu: have to drop for a few07:53
*** quiquell|rover is now known as quique|rover|brb07:53
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Install pyliblzma with pip instead of distro packages  https://review.openstack.org/63589207:55
*** ramishra_ is now known as ramishra07:56
*** ccamacho has joined #tripleo07:58
*** ksambor has joined #tripleo07:59
*** ccamacho has quit IRC07:59
*** ccamacho has joined #tripleo07:59
openstackgerritChandan Kumar proposed openstack/ansible-role-tripleo-modify-image master: Ported tripleo-modify-image to py3  https://review.openstack.org/63490408:08
*** gkadam has joined #tripleo08:08
*** hberaud|gone is now known as hberaud08:09
*** quique|rover|brb is now known as quiquell|rover08:12
chkumar|ruckquique|rover|brb: regarding this https://review.openstack.org/#/c/634904/ tripleo modify image py3 port we donot need to change anything in ansible tasks as all operatin is going to happen with in container itself so pkg mgr and pip should be determined from there08:12
quiquell|roverDetection logic is more complex08:15
quiquell|roverFor example at f28 you have pip and pip3 but you have to use pip308:15
*** gkadam_ has joined #tripleo08:15
quiquell|roverAt least for tq was like that08:16
*** gkadam has quit IRC08:17
*** paramite has joined #tripleo08:18
*** paramite has quit IRC08:23
*** agurenko has quit IRC08:23
quiquell|roverykarel: scenario004 at tempest \o/ :-)08:23
quiquell|roverykarel: if scenario001 passes I will merge the thing08:23
quiquell|roverykarel: scenario010 is not part of the criteria right now08:23
*** xek_ has joined #tripleo08:28
*** tkajinam has quit IRC08:29
*** gfidente has joined #tripleo08:29
*** cylopez has joined #tripleo08:30
*** paramite has joined #tripleo08:34
ykarelquiquell|rover, just checked, tempest passed in both scenario001,00408:39
quiquell|rover|O/08:39
ykarelcurrently collecting logs08:39
quiquell|roverlet's merge if they pass08:39
ykarelquiquell|rover, and good to fix scearnio010, another patch is fine though08:39
quiquell|roverykarel: I think let's unblock first08:40
*** tosky has joined #tripleo08:40
ykarelquiquell|rover, okk for me, just want scenario010 is not left behind08:40
ykareland is not forgotten08:40
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Allow Update and Upgrade Ansible to run without mistral  https://review.openstack.org/63628208:49
quiquell|roverchkumar|ruck: commented https://review.openstack.org/#/c/63490408:49
chkumar|ruckquiquell|rover: so basically first think I need to check if dnf is pkg manager then look for python3-pip from rpm -qa if not then install python3-pip and just set to pip3?08:51
openstackgerritCédric Jeanneret proposed openstack/tripleo-specs master: Amend Validation Framework Spec  https://review.openstack.org/63628308:52
Tengugchamoul: -^^08:53
quiquell|roverchkumar|ruck: you can express that at bindep I think08:53
*** jpena|off is now known as jpena08:53
quiquell|roverchkumar|ruck: also you can try to install without check08:53
quiquell|roverchkumar|ruck: if is already installed will pass I think08:53
gchamoulTengu: thx08:54
quiquell|roverchkumar|ruck: maybe you can put a binde.txt there08:54
quiquell|roverbut don't know08:54
quiquell|roverlet's just go for pkg_mgr stuff08:54
quiquell|roverif dnf install python3-pip08:55
quiquell|roverand use pip308:56
quiquell|roverchkumar|ruck: what about runing this thing at virtualenv ?08:57
quiquell|roverchkumar|ruck: so you  just use python2 -m virtualenv or python3 -m virtualenv08:57
quiquell|roverpip name is going to be the same08:57
quiquell|roverDon't know08:57
quiquell|roverLet's just fix it08:57
chkumar|ruckquiquell|rover: but all stuff going to install as root going to be system wide08:58
chkumar|rucklet me put the login based on pkg_mgr08:58
quiquell|roverack08:58
*** jpich has joined #tripleo08:58
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Switch scenario001-standalone to Ceph bluestore  https://review.openstack.org/54768209:00
*** agurenko has joined #tripleo09:03
*** bogdando has joined #tripleo09:03
*** panda|off is now known as panda09:04
quiquell|roverTengu, ykarel: Also the ceph-ansible review failed at post time out :-/09:07
openstackgerritChandan Kumar proposed openstack/ansible-role-tripleo-modify-image master: Ported tripleo-modify-image to py3  https://review.openstack.org/63490409:07
quiquell|roverLet's open a bug09:07
Tenguquiquell|rover: "yoohoooo"09:07
Tengu:)09:07
quiquell|roverYes :-/09:07
ykarelquiquell|rover, ack09:09
*** jistr_ is now known as jistr09:09
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates stable/rocky: Adding support of glance cinder store settings  https://review.openstack.org/63542909:11
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add centos7-rt repository  https://review.openstack.org/63515709:18
quiquell|rovermarios: ^ fixed comment09:18
quiquell|roverpanda: can you help merge this to close the bug ?09:18
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add centos7-rt repository  https://review.openstack.org/63515709:19
*** pkopec has joined #tripleo09:20
*** ykarel has quit IRC09:20
*** pkopec has quit IRC09:21
*** shardy has joined #tripleo09:21
*** ramishra has quit IRC09:22
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo master: Extra variables to reprovision pacemaker cluster one node at a time.  https://review.openstack.org/63095609:22
pandaquiquell|rover: same repos for all releases ?09:22
mariosquiquell|rover: ack09:23
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to add/manage an additional nova cellv2  https://review.openstack.org/63628609:23
quiquell|rovermarios: Âthanks09:24
*** avivgt has joined #tripleo09:24
quiquell|roverpanda: The package tuned_profiles_cpu_partitioning_package depends on centos version09:25
quiquell|roverpanda: and not openstack version09:25
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to add/manage an additional nova cellv2  https://review.openstack.org/63628609:26
*** ykarel has joined #tripleo09:27
quiquell|roverpanda: Have found differences at pike/octata at tripleo-puppet-elements :-/09:29
*** ramishra has joined #tripleo09:29
quiquell|roverThey do includepkgs=tuned-profiles-cpu-partitioning09:29
quiquell|roverskramaja: ping09:30
quiquell|roverowalsh: ping09:30
owalshquiquell|rover: hey09:30
quiquell|roverowalsh: Have found a difference at pike/ocata centos7-rt repo at puppet-elements09:31
quiquell|roverowalsh: looks like they include includepkgs=tuned-profiles-cpu-partitioning09:31
owalshquiquell|rover:  yea, we mentioned that09:31
quiquell|roverowalsh: I will do the same at release files for pike ocata then09:31
quiquell|roverowalsh: Sorry I forgot09:31
owalshquiquell|rover: originally skramaja added that for DPDK, later I enabled all pkgs for realtime kernel09:31
*** pkopec has joined #tripleo09:32
quiquell|roverowalsh: so we can remove the line from pike/ocata or we add the line to release files tripleo-quickstart ?09:32
skramajaas realtime kernel was implemented/supported from queens onwards, there is a difference.09:32
quiquell|roverskramaja: ack, let's add the line to release files09:33
skramajado we still support pike and ocata?09:33
quiquell|roverowalsh: ^ Â?09:33
owalshI guess adding the line to the releases file is the more correct option, but I doubt it matters09:34
owalshe.g the packages in the rt repo will not conflict with pkgs in any other repo09:34
*** jpich has quit IRC09:37
*** jpich has joined #tripleo09:38
openstackgerritQuique Llorente proposed openstack/tripleo-puppet-elements stable/pike: Use external centos7-rt if present  https://review.openstack.org/63628909:38
quiquell|roverowalsh: pike ^09:38
openstackgerritQuique Llorente proposed openstack/tripleo-puppet-elements stable/ocata: Use external centos7-rt if present  https://review.openstack.org/63629009:39
quiquell|roverand ocata09:39
quiquell|roverowalsh: so this is good ? https://review.openstack.org/#/c/635157/09:40
hjensasbandini:09:41
*** skramaja has quit IRC09:41
hjensashttps://github.com/voxpupuli/puppet-rabbitmq/issues/77509:41
*** skramaja has joined #tripleo09:41
hjensasbandini: ^^ I'm working on a pull request for puppet-rabbitmq.09:42
bandiniack, nice!09:42
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Enable objects versioning for Ceph RadosGW  https://review.openstack.org/63629109:43
*** aedc has joined #tripleo09:44
hjensasbandini: I think we should make the puppet module support writing the /etc/rabbitmq/enabled_plugins if service_manage is false, or if flag - manage_plugins_via_config_file = true or something.09:44
bandiniaye09:44
hjensasbandini: or, it might work to just decouple the download of the client from enabling the management_interface.09:45
bandinihjensas: do you want me to prepare an env to test this stuff?09:45
hjensasbandini: I have a env, undercloud only, but there are nodes to deploy overcloud as well once I get it working on the undercloud.09:46
owalshquiquell|rover: I guess there is a very very slight risk that tuned-profiles-cpu-partitioning could add a dep on another package in the rt repo, which would work in CI but fail if building images with the regular repos09:47
owalshquiquell|rover: I don't think it's worth worrying about though, so yea, LGTM09:47
quiquell|roverpanda: ^09:47
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to manage additional nova cells v2  https://review.openstack.org/63628609:48
sri_hjensas, the issues was resolved, Thank you :)09:48
quiquell|roverowalsh: let's also merge the stable branches reviews from puppet-element09:48
hjensassri_: Cool.09:48
hjensasbandini: does the undercloud run things under pacemaker as well? Or do I need the overcloud with 3 controllers to verify that?09:50
bandinihjensas: the latter09:50
hjensasbandini: ok, and a single controller OC is'nt enough either?09:50
owalshquiquell|rover: ack, for https://review.openstack.org/635157 WDYT? Perhaps it would be better to add the includepkgs for the repos, just in case...09:50
bandinihjensas: one controller should be fine for this test09:51
hjensasbandini: ok, cool. Thanks.09:51
quiquell|roverowalsh: yep let's do it09:51
quiquell|roverowalsh: thanks09:51
owalshquiquell|rover: thank you!09:51
*** sshnaidm|afk is now known as sshnaidm09:52
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: Ensure all upgrade tasks depend on a step.  https://review.openstack.org/62815409:55
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: WIP: Split upgrade_steps_playbook into different plays.  https://review.openstack.org/62589009:55
openstackgerritGabriele Cerami proposed openstack/tripleo-heat-templates master: [WIP] Translating scenario012 to standalone  https://review.openstack.org/63472309:55
pandaquiquell|rover: so good to merge ?09:57
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add centos7-rt repository  https://review.openstack.org/63515709:57
quiquell|roverpanda, marios: Updated with owalsh comments ^, let's give it another try before +w09:57
quiquell|roverpanda, marios: you can +2 it though09:58
pandaquiquell|rover: also, I'd like to wait for checks09:58
pandaquiquell|rover: but yeash +209:58
quiquell|roverpanda: yep that's what I mean09:58
*** hberaud is now known as hberaud|afk09:58
owalshquiquell|rover: does it need to be a list?09:59
quiquell|rover owalsh what do you mean ?09:59
owalshquiquell|rover: includepkgs, does it need to be a yaml list?10:00
*** cfontain has quit IRC10:00
quiquell|roverowalsh: it's how we represent the repositories we install10:00
*** cfontain has joined #tripleo10:00
quiquell|roverowalsh: they get transformed into proper .repo files10:00
owalshquiquell|rover: e.g it's a list here https://review.openstack.org/#/c/635157/10..11/config/release/tripleo-ci/CentOS-7/pike.yml@5410:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory to freeipa playbook  https://review.openstack.org/63609710:02
quiquell|roverowalsh: no this is different10:02
quiquell|roverowalsh: humm wait depending on how it's implemented10:03
quiquell|roverowalsh: it can work as list and as string10:03
quiquell|roverLet me check10:03
owalshquiquell|rover: yea, that's what I'm asking10:03
quiquell|roverowalsh: good catch though10:03
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Add collecting files for freeipa job  https://review.openstack.org/63629410:03
*** shyamb has joined #tripleo10:03
*** pkopec has quit IRC10:03
shyambHi10:04
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory to freeipa playbook  https://review.openstack.org/63609710:04
*** hberaud|afk is now known as hberaud10:04
mariosquiquell|rover: nit? https://review.openstack.org/#/c/635157/10..11/config/release/tripleo-ci/CentOS-7/pike-undercloud-ocata-overcloud.yml10:05
*** cfontain has quit IRC10:05
shyambhi10:05
quiquell|roveropenstackgerrit: tq do includepkgs={{ repo.includepkgs|join(",") }10:06
shyambFor osp14, release notes shows ansible is heavily used10:06
quiquell|roverowalsh: ^10:06
owalshquiquell|rover: ack, cool10:07
quiquell|roverowalsh: https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/templates/repo_setup.sh.j2#L7210:07
shyambowalsh: I am from Trilio, wanted to extend deployment integration with osp1410:07
shyambany suggestions?10:07
quiquell|rovermarios: yep!10:08
*** gfidente has quit IRC10:09
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Add centos7-rt repository  https://review.openstack.org/63515710:09
quiquell|rovermarios, panda: fixed nit `10:10
owalshshyamb: you've osp13 working with containers yea? Have you tried the same changes with 14?10:11
shyambowalsh: Yes, we have osp13 scripts10:13
shyambnot tried with osp1410:13
shyambShould I go ahead and try, you think they should work with some minor changes?10:13
shyambI read about ansible in release notes10:14
shyambSo, was curious about if we need to convert puppet module into ansible10:14
openstackgerritKamil Sambor proposed openstack/puppet-tripleo master: Fix generating connections to OVN db  https://review.openstack.org/63587210:16
quiquell|roverchkumar|ruck: commented https://review.openstack.org/#/c/63611410:16
owalshshyamb: the templates didn't change that much AFAIK, so I would test and see if there are any regressions10:17
owalshshyamb: don't have to convert puppet into ansible10:18
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add supplemental node to inventory  https://review.openstack.org/63609410:18
shyambowalsh: Great10:19
shyambthanks10:19
shyambI will test the scripts first10:19
quiquell|roverchkumar|ruck, ykarel: Something is not right at tripleo-container-image-prepare10:19
quiquell|roverIt's starting to fail more and more10:19
ykarelwhat error?10:19
chkumar|ruckquiquell|rover: with scenario standalone jobs na?10:20
ykarelmetadata?10:20
ykarelblob10:20
chkumar|ruckquiquell|rover: just saying failed at image prepare job part with no more info in tripleo image prepare log10:20
quiquell|roverchkumar|ruck, ykarel: http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_07_37_0310:20
quiquell|roverykarel: I remember we found and issue like in the middle of the log fail regarding metadata or manifest10:21
ykarelquiquell|rover, hmm but can't see something in this job10:22
ykarellooking at http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz10:22
quiquell|roveryep me neither10:23
*** pkopec has joined #tripleo10:24
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: WIP: Split upgrade_steps_playbook into different plays.  https://review.openstack.org/62589010:25
*** social has joined #tripleo10:26
*** derekh has joined #tripleo10:27
*** shyamb has quit IRC10:34
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix tempest init path in tempest-venv  https://review.openstack.org/63321510:37
quiquell|roverykarel: {"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}10:40
quiquell|roverIt's like in the middle of th efile10:40
ykarelquiquell|rover, yup this seems to be the issue10:42
ykareland possibly related to buildah10:42
quiquell|roverykarel: could be related with promotion pipeline ?10:42
*** gfidente has joined #tripleo10:42
ykarelmeans?10:43
quiquell|roverykarel: I mean like gate job trying to use a container with is in the middle of a push elsewhere10:43
ykarelquiquell|rover, propotion is already blocked with scenario001/00410:43
quiquell|roverykarel: and maybe buildah push is not very atomic yet10:43
ykarelquiquell|rover, possibly10:43
quiquell|roverBut we don't have master promotions10:44
quiquell|rover:-/10:44
quiquell|roverMaybe somewhere else is pushing stuff ?10:44
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add supplemental node to inventory  https://review.openstack.org/63609410:45
*** jtomasek has quit IRC10:46
*** florianf has joined #tripleo10:46
*** udesale has quit IRC10:49
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory to freeipa playbook  https://review.openstack.org/63609710:49
quiquell|roverykarel: welll the tripleo-container-image-prepare script runs in parallel10:50
quiquell|roverdepending on the machine's number of cores10:51
*** suuuper has joined #tripleo10:52
*** hberaud is now known as hberaud|school-r10:54
*** jtomasek has joined #tripleo10:54
*** suuuper has quit IRC10:56
*** suuuper has joined #tripleo10:57
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: adding container push after build in the container buid job  https://review.openstack.org/63522211:02
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: [WIP] Install systemd unit files for safe side containers  https://review.openstack.org/63630311:04
openstackgerritMartin André proposed openstack/tripleo-common master: [WIP] Switch to podman for tripleo-deploy-openshift  https://review.openstack.org/62849811:05
*** hberaud|school-r is now known as hberaud|lunch11:08
*** morazi has quit IRC11:10
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: adding container push after build in the container buid job  https://review.openstack.org/63522211:17
openstackgerritGabriele Cerami proposed openstack/tripleo-heat-templates master: [WIP] Translating scenario012 to standalone  https://review.openstack.org/63472311:26
quiquell|roverpanda: Do you know if current work at promotion pipeline could be affecting jobs upstream ?11:31
quiquell|roverpanda: http://logs.openstack.org/45/560445/247/check/tripleo-ci-centos-7-scenario004-standalone/6bbc324/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz11:31
quiquell|roverhttps://bugs.launchpad.net/tripleo/+bug/181557611:31
openstackLaunchpad bug 1815576 in tripleo "Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)11:31
pandaquiquell|rover: not until things are merged ...11:32
quiquell|roverpanda: ack, yep docker images are 4 days old :-/11:33
pandaquiquell|rover: also I don't thinkg there's any work items with tripleo-container-image-prepare, it's not in scope, the scope is the build, the prepare comes after11:33
*** pdeore has quit IRC11:33
quiquell|roverpanda: I was thinking that maybe we are running something at "check" that breaks containers at dockerhub11:34
openstackgerritGabriele Cerami proposed openstack/tripleo-heat-templates master: [WIP] Translating scenario012 to standalone  https://review.openstack.org/63472311:35
quiquell|rovermarios: Do you remember why we have this ? standalone_container_cli: docker11:36
quiquell|rovermarios: for standalone scenarios11:36
mariosquiquell|rover: yah cos it was defaulting to podman11:36
mariosbut we don't have it in standalone11:36
quiquell|roverbut non scenario job is using podman I think11:37
mariosquiquell|rover: it was one of the early fails we had it was trying to use pod and failing so we switched to that11:37
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory to freeipa playbook  https://review.openstack.org/63609711:37
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates stable/queens: NFV: Support for config-download to deploy node with kernel args  https://review.openstack.org/63630911:40
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates stable/queens: [stable/queens] Disable default config-download method for PreNetworkConfig  https://review.openstack.org/63631011:40
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates stable/queens: [stable/queens] Add a check for kernels args update to avoid unnecessary reboot  https://review.openstack.org/63631111:40
*** holser_ is now known as holser|lunch11:42
pandaquiquell|rover: jobs in check do not have credentials to modify containters in dockerhub. Also if job in the harmless check pipeline are breaking something we are doing something deeply wrong :)11:43
pandawhich may be the case ... you never know ...11:43
quiquell|roverpanda: I meaning running periodic jobs at RDO check pipeline11:43
quiquell|roverpanda: like pushing jobs and the like11:44
quiquell|roverpanda: not upsream check jobs11:44
pandaquiquell|rover: I understand, not the case. periodic jobs running at check still can't use the credentials to dockerhub. No job can use the credentials until it's merged in rdoproject.org/config11:45
quiquell|roverpanda: ack nice nice11:46
quiquell|roverpanda: btw11:46
quiquell|roverpanda: I see docker using overlay2 and podman just overlay11:47
quiquell|rover:-/11:47
quiquell|roverpanda: http://logs.openstack.org/45/560445/247/check/tripleo-ci-centos-7-scenario004-standalone/6bbc324/logs/undercloud/var/log/extra/podman/podman_allinfo.log.txt.gz11:47
quiquell|roverpanda: http://logs.openstack.org/45/560445/247/check/tripleo-ci-centos-7-scenario004-standalone/6bbc324/logs/undercloud/var/log/extra/docker/docker_allinfo.log.txt.gz11:47
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: [DNM] Test tripleo-buildcontainers on py3/f28  https://review.openstack.org/63618811:52
quiquell|roverTengu, ykarel: Do you know what's the best way to configure container.conf at undercloud ?11:53
quiquell|rovertht deployment/podman/podman-baremetal-ansible.yaml  ?11:54
openstackgerritMerged openstack/tripleo-quickstart-extras master: Remove double quote from tempestconf  https://review.openstack.org/63324711:55
*** _fragatina has joined #tripleo11:56
openstackgerritMerged openstack/tripleo-quickstart-extras master: Change headless chrome port to 4444  https://review.openstack.org/63348911:58
openstackgerritMerged openstack/tripleo-common stable/queens: Publish a better failure message for Update Ansible  https://review.openstack.org/63419511:58
*** _fragatina has quit IRC11:59
*** _fragatina has joined #tripleo12:00
*** dciabrin has quit IRC12:01
*** dciabrin has joined #tripleo12:02
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates master: WIP: Use overlay2 storage driver for podman  https://review.openstack.org/63631712:02
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates master: WIP: Use overlay2 storage driver for podman  https://review.openstack.org/63631712:03
*** avivgt has quit IRC12:05
quiquell|roverykarel: Do we have to fix this ? https://review.openstack.org/#/c/636086/8/docker/services/ceph-ansible/ceph-base.yaml@44712:07
quiquell|roverEmilienM: This make any sense ? https://review.openstack.org/#/c/636317/12:07
ykarelquiquell|rover, i think you can do cleanup with octavia fixes12:08
ykareland clear jobs now12:08
*** raildo has joined #tripleo12:08
quiquell|roverykarel: that's right we will do that later on12:10
quiquell|roverykarel: welll check is almost finished :-) want to merge the thing12:11
quiquell|roverykarel: Are we going to break stuff if we merege like that ?12:11
ykarelquiquell|rover, don't think so12:12
quiquell|roverykarel: let's just merge this12:12
openstackgerritJakub Libosvar proposed openstack/tripleo-heat-templates stable/rocky: Set Ironic default interface when using networking-ansible  https://review.openstack.org/63632112:13
quiquell|roverykarel:  I am not going to close the bug12:13
quiquell|roverykarel: so we don't forget about scnario01012:13
ykarelquiquell|rover, ack12:14
ykareladd comment12:14
zbrEmilienM: mwhahaha : check my comment on https://review.openstack.org/#/c/633527/15/playbooks/tripleo-buildcontainers/pre.yaml -- am i right/wrong?12:17
quiquell|roverbogdando: ping12:18
Tenguzbr: I'd say you're right, the 172.17 is a /1612:18
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: adding container push after build in the container buid job  https://review.openstack.org/63522212:18
quiquell|roverbogdando: this is still a thing ? https://bugs.launchpad.net/tripleo/+bug/179971712:19
openstackLaunchpad bug 1799717 in tripleo "Deployment fails with :/usr/bin/docker-current: Error response from daemon: grpc: the connection is unavailable" [High,Triaged]12:19
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory and repos to freeipa playbook  https://review.openstack.org/63609712:19
openstackgerritMerged openstack/python-tripleoclient master: Add overcloud admin ssh authorize command.  https://review.openstack.org/63357412:23
openstackgerritMerged openstack/tripleo-common-tempest-plugin master: Update hacking version to latest  https://review.openstack.org/62936212:23
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Run standalone scenario jobs at tht podman changes  https://review.openstack.org/63632312:23
*** hakhande_ has quit IRC12:24
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Run standalone scenario jobs at tht podman changes  https://review.openstack.org/63632312:24
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Disable stack check and cancel update for undercloud  https://review.openstack.org/63610812:24
openstackgerritMerged openstack/tripleo-heat-templates master: Do not mount ceph-ansible and octavia playbook within mistral container  https://review.openstack.org/63610712:24
openstackgerritMerged openstack/tripleo-heat-templates master: flatten the horizon service configurations  https://review.openstack.org/63475212:25
openstackgerritMerged openstack/tripleo-heat-templates master: Fix with_items indentation in logs readme ec2-api.  https://review.openstack.org/63573712:25
openstackgerritMerged openstack/tripleo-heat-templates master: Enable glance image cache's cleaner and pruner  https://review.openstack.org/63618412:25
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: mysql: sync credentials in running container on password change  https://review.openstack.org/63597612:25
openstackgerritMerged openstack/os-net-config master: Update hacking version to latest  https://review.openstack.org/62860512:25
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates master: Use overlay2 storage driver for podman  https://review.openstack.org/63631712:26
*** jpena is now known as jpena|lunch12:26
*** hberaud|lunch is now known as hberaud12:27
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63599112:27
sri_hjensas, if you have min,  can you please take a look into this issue openstack stack failures list iosstack --long  https://pastebin.com/0qeE0vfN,   openstack software deployment show --long   https://pastebin.com/KvnPUyN112:30
*** panda is now known as panda|lunch12:32
EmilienMquiquell|rover: hey, looking at https://bugs.launchpad.net/tripleo/+bug/181557612:33
openstackLaunchpad bug 1815576 in tripleo "Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)12:33
EmilienMdid you find something new since your last comment?12:33
quiquell|roverEmilienM: well, standalone scenarios are overriding container_cli with docker12:34
quiquell|roverEmilienM: is all in the bug12:34
quiquell|roverEmilienM: docker uses overlay2 and podman overlay12:34
quiquell|roverEmilienM: I am chaing podman to overlay2 to check it out12:34
*** ratailor has quit IRC12:34
openstackgerritJuan Antonio Osorio Robles proposed openstack/os-net-config master: Enable setting domain parameter for interfaces  https://review.openstack.org/63632512:34
quiquell|roverEmilienM: https://review.openstack.org/#/c/636317/12:35
EmilienMI suspect overlay2 doesn't work on centos712:35
EmilienMAFIK it requires kernel 4.18 at least12:35
quiquell|roverEmilienM: well or promoter is using it at centos7 so it works at least for docker12:35
quiquell|roverEmilienM: humm12:35
quiquell|roverEmilienM: how well works podman with overlay2 ?12:36
quiquell|roverEmilienM: but docker is configured with overlay2 there12:36
openstackgerritMerged openstack/tripleo-heat-templates master: Add support for transferring MariaDB data between nodes  https://review.openstack.org/63147812:36
*** florianf has quit IRC12:36
quiquell|roverEmilienM: maybe we have to "downgrade" docker to overlay12:36
EmilienMnot 100% sure12:36
EmilienMalso there is a new runc, we'll have to try it12:37
hjensassri_: parsing [/etc/haproxy/haproxy.cfg20190212-12-1feytb5:164] : 'bind' : invalid address: 'internal_api' in 'internal_api:8778'12:37
sri_hjensas, yep,  If I have different vlans for InternalAPI can I use the same Name for the two stacks ?12:37
*** hakhande_ has joined #tripleo12:37
*** bfournie has quit IRC12:38
hjensassri_: Wrong argument given internal_api is not an ip address. at /etc/puppet/modules/apache/manifests/vhost.pp:341:2212:38
quiquell|roverEmilienM: we can play with options in the review12:38
hjensassri_: it looks like you get the string 'internal_api' instead of an ip address for some reason.12:39
quiquell|roverEmilienM: but it's starting to be more frequent12:39
quiquell|roverykarel, zbr: Is journal indexed at kibana ?12:39
quiquell|roverI mean logstash12:39
EmilienMquiquell|rover: https://review.rdoproject.org/r/1881512:39
*** florianf has joined #tripleo12:40
EmilienMquiquell|rover: I could even test your THT patch in my rdo patch ^12:40
quiquell|roverEmilienM: that would be nice12:40
hjensassri_: I think using different vlan id's you don't want to allocate IPs from the same pool? (It may work, if the two never need to be able to talk to eachother. But it's confusing to have ip's in the same ip range in two vlans?12:40
zbrquiquell|rover: kibana/logstash receives whatever we sent to it. its a push process not a pull12:40
zbrquiquell|rover: so, if we configure journald to send data to it, it should have it.12:40
quiquell|roverzbr: so do we push journal ?12:41
quiquell|roverzbr: don't know where do we do that12:41
sri_hjensas, hmmm, It two diffrent Vlans with two diffrent subnets12:41
sri_hjensas, I am sure this issue is coming up because I don't have internal api, I've created a stack before but at the time I've using internalAPI12:42
zbrnot as far as I know, but I wanted to ask the same last week. for both: build nodes and even provisioned machine. somehow I like the idea to send the logs outside the tested machines instead of collecting them at the end.12:42
*** dprince has joined #tripleo12:43
TenguEmilienM quiquell|rover : hmm, iirc, the registry should tell the client what's the next blob ID - so maybe the issue is more related to the registry?12:43
EmilienMit could also be related to the python uploader12:44
EmilienMwe merged it very recently12:44
zbrzbr: but we have some issues implementing that with containers. docker has support for log-handlers but if I remember well podman does not, only json output.12:44
Tenguhmmm.12:44
*** janki has quit IRC12:44
EmilienMquiquell|rover: do you have a logstash query alraedy?12:44
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: adding container push after build in the container buid job  https://review.openstack.org/63522212:44
*** janki has joined #tripleo12:44
quiquell|roverEmilienM: nop12:44
EmilienMit should be the first thing to do when filling these bugs12:45
EmilienMso we know when it started12:45
quiquell|roverEmilienM: Don't know if we index journal or tripleo-prepare-foobar.log12:45
EmilienMand easier to track down12:45
EmilienMlet me search12:45
quiquell|roverEmilienM: cool thanks man I have to learn that too12:45
quiquell|roverEmilienM: in the bug is enough stuff to compose it12:45
hjensassri_: yeah, it may be something hardcoded expecting the InternalAPI network to always be there.12:46
*** moguimar has joined #tripleo12:46
hjensassri_: I'd look at the hieradata files under /etc/puppet/hieradata for clues about which map/resource in THT is putting internal_api12:47
hjensassri_: instead of an IP.12:47
*** holser|lunch is now known as holser_12:47
*** ykarel is now known as ykarel|away12:48
hjensassri_: If you need to two vlans, and two different ip subnets you I don't think you can use the same name for two networks. :(12:48
sri_hjensas, Oh, I will look into it. and keep you posted. :)12:50
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Improve virt-resize UX  https://review.openstack.org/63632712:54
*** ykarel|away has quit IRC12:54
*** morazi has joined #tripleo12:54
*** thrash|g0ne is now known as thrash12:55
EmilienMquiquell|rover: for now i'm using this simple query12:56
EmilienMhttp://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%20%5C%22blob%20unknown%20to%20registry%5C%2212:56
*** _fragatina has quit IRC12:56
sri_hjensas, I've found this in /etc/puppet/hieradata/all_nodes.json http://paste.openstack.org/show/744929/12:56
EmilienMit sounds like really new12:56
quiquell|roverEmilienM: can we see if it only happend at scenarios ?12:56
quiquell|roverEmilienM: it returns nothing here12:56
*** weshay has joined #tripleo12:57
quiquell|roverEmilienM: I think we don't push journal12:58
EmilienMquiquell|rover: I'm going to compare packages to check something12:59
EmilienMquiquell|rover: have you seen that bug before eysterday?12:59
EmilienMquiquell|rover: mhh nevermind, my query is wrong (it shows kolla logs)13:00
quiquell|roverEmilienM: yep, but no so often13:00
quiquell|roverEmilienM: like last week or so13:00
EmilienMI highly suspect the python uploader then13:01
TenguEmilienM: so it started today, more or less?13:01
* EmilienM continues to poke at it13:01
Tenguat least the only mentions of that message, according to your simple query, are for today - even with a 7-days span13:01
*** rh-jelabarre has joined #tripleo13:01
jaosoriorHey folks! I'll start going through LP bugs; in case someone wants to join: https://bluejeans.com/252741440913:02
*** aedc has quit IRC13:03
*** aedc has joined #tripleo13:04
Tengujaosorior: as a moral support? :)13:05
jaosoriorTengu: lol, just in case you're curious and wanna help out. If you're busy it's cool. Gonna be doing that the next hour anyway13:06
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates master: Introduce memcached_servers and use it  https://review.openstack.org/63450513:06
EmilienMquiquell|rover: don't we index the journal logs?13:08
quiquell|roverEmilienM: don't know man13:08
*** bfournie has joined #tripleo13:08
*** mcornea has joined #tripleo13:09
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to manage additional nova cells v2  https://review.openstack.org/63628613:09
*** udesale has joined #tripleo13:10
*** chkumar|ruck is now known as chkumar|pto13:10
*** trown|outtypewww is now known as trown13:11
*** nawar has joined #tripleo13:12
nawarhi13:12
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Improve virt-resize UX  https://review.openstack.org/63632713:12
nawarif i want to deploy a hyper-converged overcloud is this guide is good https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/hyper-converged_infrastructure_guide/index13:12
weshayzbr++13:13
EmilienMquiquell|rover: /var/log/tripleo-container-image-prepare.log is in the logstash files13:13
weshayI like this register: result13:13
weshay  no_log: result is success13:13
sri_hjensas, It looks like I miss some ServiceNetMap and it's defaults to internalapi13:13
EmilienMat least I see it in roles/collect-logs/defaults/main.yml13:13
*** panda|lunch is now known as panda13:13
quiquell|roverEmilienM: then we have to look MANIFEST_BLOB_UNKNOWN13:13
EmilienMyea I find nothing13:17
*** quiquell|rover is now known as quique|rover|eat13:17
quique|rover|eatI am going to eat my sadness13:17
*** rlandy has joined #tripleo13:17
EmilienM:) come back!13:18
weshaylolz13:20
EmilienMhttp://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%20*tripleo-ci*%20AND%20build_status%3A%20FAILURE%20AND%20build_branch%3A%20master%20AND%20build_queue%3A%20gate13:21
EmilienMif we look at this query, it's pretty clear that the gate started to have failures over yesterday or so13:22
weshaywhat did you break?13:22
EmilienMlet's first compare rpms with, let's say 3 days ago13:22
EmilienMweshay: I suspect the python uplaoder13:22
EmilienMweshay: https://bugs.launchpad.net/tripleo/+bug/181557613:22
openstackLaunchpad bug 1815576 in tripleo "Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)13:22
EmilienMweshay: I can't figure out why /var/log/tripleo-container-image-prepare.log output isn't in logstash index13:23
EmilienMany idea?13:23
EmilienMI see it in tripleo-quickstart-extras/roles/collect-logs/defaults/main.yml13:23
* weshay checks settings.. 13:23
weshaynot everything goes into log stash13:23
weshayaye.. k.. checking a bit more13:25
*** agopi has joined #tripleo13:26
*** suuuper has quit IRC13:27
jaosoriorgchamoul: is this still an issue https://bugs.launchpad.net/tripleo/+bug/1792113 ?13:28
openstackLaunchpad bug 1792113 in tripleo "All tripleo-validations fail while running through Mistral with a stderr "which: no tripleo-ansible-inventory in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)"" [High,New]13:28
weshayhrm.. EmilienM I see some modify-image logging there, but ya.. ur right13:28
weshaycontainer-image-prepare is not stashed13:29
EmilienManyway, it's the python uplader13:30
EmilienMhttps://git.openstack.org/cgit/openstack/tripleo-common/tree/tripleo_common/image/image_uploader.py#n60713:30
*** jpena|lunch is now known as jpena13:30
*** ade_lee_ has quit IRC13:30
EmilienMmaybe we need a retry? again13:30
*** sdake has joined #tripleo13:31
*** pkopec has left #tripleo13:32
*** agopi has quit IRC13:34
weshayEmilienM, quique|rover|eat it's trying to add that log to logstash http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/quickstart_collect_logs.log13:36
weshaylook for tripleo-container-image-prepare.log13:36
*** openstackgerrit has quit IRC13:37
*** quique|rover|eat is now known as quiquell|rover13:39
quiquell|roverweshay: what do you mean ?13:39
weshayquiquell|rover, collect-logs is trying add that log to logstash TASK [collect-logs : Create a index file for logstash] *************************13:42
weshaylook for that in the collect-logs... log13:42
*** ykarel|away has joined #tripleo13:43
weshayhttps://goo.gl/images/3yrTei13:43
weshayquiquell|rover, maybe we can improve how that works so we can better determine when things like this happen13:44
EmilienMweshay, quiquell|rover : I'm currently looking how we can add a retry to the code, so we try again to post the layer13:45
EmilienMhowever there is already a retry to the function13:46
*** nawar has left #tripleo13:46
*** openstackgerrit has joined #tripleo13:46
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates master: Use overlay2 storage driver for podman  https://review.openstack.org/63631713:46
quiquell|roverEmilienM: ^ fixed overlay2 config13:47
EmilienMI'm not sure this will help but ok13:47
quiquell|roverweshay: we were also facing collect logs timeouts today13:47
weshayquiquell|rover, ok.. we own that setting13:48
weshaybut dang it13:48
weshayquiquell|rover, should we sync since chkumar|pto is out?13:48
weshayI only see a few failed gate jobs13:49
quiquell|roverack13:49
quiquell|roverweshay: going to your blue13:50
quiquell|roverEmilienM: what retry ?13:50
EmilienMquiquell|rover: https://git.openstack.org/cgit/openstack/tripleo-common/tree/tripleo_common/image/image_uploader.py#n57913:50
weshayk13:51
EmilienMactually I'm not even sure it's the failure13:51
*** ykarel|away is now known as ykarel13:51
EmilienMin the logs, we don't see the blob error more than 2 times13:52
EmilienMand we have a retry of 513:52
EmilienMwhat is confusing is the return code of tripleo container image prepare13:53
EmilienMhttp://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz13:53
EmilienMansible didn't show failed tasks13:53
pandaderekh: can you take a look at http://logs.openstack.org/25/634725/3/check/tripleo-ci-centos-7-scenario012-standalone/f2221b1/logs/undercloud/home/zuul/tempest.log.txt.gz ? We can discuss what's missing after tripleO meeting here.13:57
weshayEmilienM, podman is configured w/ the overlay storage driver, and docker is overlay v2.  is that by design?13:58
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to manage additional nova cells v2  https://review.openstack.org/63628613:59
EmilienMweshay: I don't have the answer13:59
weshayEmilienM, any reason why we shouldn't use overlay2 w/ podman?13:59
weshaymaybe we can see if that works13:59
EmilienMthe kernel version maybe13:59
Tenguif docker is using overlay2, it should work with podman... ?14:00
EmilienMlast week I had some issue with buildah and 4.1614:00
EmilienMand we needed 4.1814:00
EmilienMTengu: it depends for what14:00
EmilienMfor mounts? etc14:00
Tenguhmm ok. well. it's worth a try I guess?14:00
jaosorior#startmeeting tripleo14:01
openstackMeeting started Tue Feb 12 14:01:16 2019 UTC and is due to finish in 60 minutes.  The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot.14:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:01
*** openstack changes topic to " (Meeting topic: tripleo)"14:01
openstackThe meeting name has been set to 'tripleo'14:01
TenguEmilienM: that said, wasn't the podman version updated lately?14:01
bogdandoquiquell|rover: I think it never was14:01
jaosorior#topic agenda14:01
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:01
jaosorior* Review past action items14:01
ccamachoo/ \o14:01
jaosorior* One off agenda items14:01
EmilienMthe only thing that is sure, is that overlay isn't supported over xfs14:01
jaosorior* Squad status14:01
jaosorior* Bugs & Blueprints14:01
jaosorior* Projects releases or stable backports14:01
jaosorior* Specs14:01
jaosorior* open discussion14:01
jaosoriorAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:01
jaosoriorHey folks! who's around?14:01
TenguEmilienM: yeah and it crashes I think14:01
Tengu«o/14:01
weshayo/14:01
rfolcoo/14:01
marioso/14:01
ksamboro/14:01
jaosoriorbig thanks to weshay for running last week's meeting!14:02
weshaygolf clap14:02
EmilienMit was terrible14:02
weshay:)14:02
EmilienMlol14:02
weshaylolz14:02
*** agopi has joined #tripleo14:02
quiquell|rovero/14:03
panda\o14:03
*** sdake has quit IRC14:03
*** leanderthal has joined #tripleo14:04
openstackgerritRonelle Landy proposed openstack/tripleo-heat-templates master: [WIP] [DNM] translate scenario009 to standalone  https://review.openstack.org/63520814:05
jistr\o14:05
jaosorior#topic review past action items14:05
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:05
chemo/14:05
jaosoriorNone.14:05
jaosorior#topic one off agenda items14:05
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:05
jaosorior#link https://etherpad.openstack.org/p/tripleo-meeting-items14:05
openstackgerritRonelle Landy proposed openstack/tripleo-heat-templates master: [WIP] [DNM] translate scenario009 to standalone  https://review.openstack.org/63520814:05
jaosoriorFirst topic: (rfolco): CI updates on Fedora 28 containers build: Reviewing containers build process for CentOS and Fedora to use the same workflow. Adding support for f28/python3 to tripleo-buildcontainers playbook. Check jobs to be added in this sprint and promotion jobs to be consolidated with the same workflow. Epic: https://tree.taiga.io/project/tripleo-ci-board/epic/44514:06
rfolcothis a quick update on our current work for f28 containers14:06
fultonjo/14:06
rfolcowe'll update here as we get progress on this14:07
rfolcothat's it jaosorior thanks14:07
jaosoriorrfolco: thanks for the update!14:07
openstackgerritMerged openstack/tripleo-heat-templates master: FFWD: Introduce workaround for neutron cisco plugin  https://review.openstack.org/61948214:08
jaosorior#topic Squad status14:08
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:08
jaosoriorci14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:08
jaosoriorupgrade14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:08
jaosoriorcontainers14:08
jaosorior#link https://trello.com/b/S8TmOU0u/tripleo-podman14:08
jaosoriorintegration14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:08
jaosoriorui/cli14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:08
jaosoriorvalidations14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:08
jaosoriornetworking14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:08
dpeacocko/14:08
jaosoriorsecurity14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-security-squad14:08
jaosorioredge14:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-edge-squad-status14:08
jaosoriorAny squad wanting to bring up their status, or a topic for the general public?14:09
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Add collecting files for hosts without zuul user  https://review.openstack.org/63629414:09
sshnaidmquiquell|rover, marios, weshay  take a look please ^^14:09
openstackgerritNoam Angel proposed openstack/tripleo-common master: fix if statement compare strings with -gt  https://review.openstack.org/63634214:10
* redrobot sneaks in through the back14:10
openstackgerritNoam Angel proposed openstack/tripleo-common master: fix if statement compare strings with -gt  https://review.openstack.org/63634214:11
jaosorior#topic bugs & blueprints14:11
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:11
jaosorior#link https://launchpad.net/tripleo/+milestone/stein-314:11
jaosoriorFor Stein we currently have 21 (for stein-3) blueprints open in Launchpad.14:11
*** tzumainn has joined #tripleo14:11
jaosoriorBugs: 582 (-11) stein-3 in Launchpad. 104 (0) open Storyboard bugs.14:11
openstackbug 582 in glabels (Ubuntu) "Cannot edit text field" [Medium,Invalid] https://launchpad.net/bugs/58214:11
jaosorior#link https://storyboard.openstack.org/#!/project_group/7614:11
jaosorior#topic projects releases or stable backports14:12
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:12
jaosorior#topic specs14:14
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:14
jaosorior#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:14
jaosoriorAny specs folks wanna bring attention to?14:14
fultonjlooks like https://review.openstack.org/#/c/622324 is ready to merge14:15
fultonjMajor Upgrades Including Operating System Upgrade14:15
jaosoriorgreat! I'll give it a read in a bit and merge it.14:15
jaosoriorthanks for bringing it up14:15
fultonjjistr++ chem++14:15
jaosorior#topic open discussion14:17
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:17
jaosoriorAnything else that folks want to bring up to the meeting?14:17
jistrthanks for the feedback there folks (re upgrade spec)14:17
*** moguimar has quit IRC14:20
*** sdake has joined #tripleo14:20
*** sdake has quit IRC14:21
*** moguimar has joined #tripleo14:21
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: WIP: Split upgrade_steps_playbook into different plays.  https://review.openstack.org/62589014:23
*** agurenko has quit IRC14:23
*** skramaja has quit IRC14:23
jaosoriorAlright folks! thanks for joining!14:24
jaosorior#endmeeting14:24
*** openstack changes topic to "CI Status: GREENish | https://docs.openstack.org/tripleo-docs/latest/"14:24
rfolcoCI community meeting (office hours) starts now at https://bluejeans.com/4113567798 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:24
openstackMeeting ended Tue Feb 12 14:24:23 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:24
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-12-14.01.html14:24
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-12-14.01.txt14:24
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-12-14.01.log.html14:24
weshaymwhahaha, jaosorior so f28-standalone the percentage pass rate on f28-standalone is much closer to centos7-standalone, however only in the last few days14:28
openstackgerritMerged openstack/tripleo-specs master: Major Upgrades Including Operating System Upgrade  https://review.openstack.org/62232414:29
*** sshnaidm has quit IRC14:30
jaosoriorweshay: lets let it run for a week. If the pass rate continues to be this consistent. I'm for making it voting again14:30
*** sshnaidm has joined #tripleo14:30
weshaythanks!14:31
weshayquiquell|rover, FYI ^14:31
*** ade_lee_ has joined #tripleo14:31
quiquell|roverjaosorior, weshay: ack noted14:31
weshayquiquell|rover, any other questions on those bugs?14:32
quiquell|roverweshay: nope14:32
*** sdake has joined #tripleo14:35
*** sdake has quit IRC14:37
*** sdake_ has joined #tripleo14:37
*** rainleander has joined #tripleo14:37
openstackgerritCarlos Camacho proposed openstack/python-tripleoclient master: Allow run_update_ansible_action run with Mistral or Ansible  https://review.openstack.org/63628214:39
Tenguquiquell|rover, weshay, EmilienM : apparently podman already uses "overlay2", they just renamed it to "overlay"........14:39
Tengufresh news from #podman14:39
EmilienMyes, see https://www.redhat.com/en/blog/working-container-storage-library-and-tools-red-hat-enterprise-linux14:39
quiquell|roverTengu, EmilienM: Then why it's failing ?14:40
EmilienMthe error isn't related to overlayfs again14:40
quiquell|roverhttp://logs.openstack.org/17/636317/4/check/tripleo-ci-centos-7-standalone/f710cfc/14:40
EmilienMit's the python uploader, it became the default a few days ago14:40
quiquell|roverEmilienM: ack14:40
quiquell|roverEmilienM: big red herring I was following14:40
EmilienMi'm still debugging14:40
quiquell|roverEmilienM: but error is different14:41
quiquell|roverthere is no unknown blob stufff14:41
quiquell|roverEmilienM: can we see the retries in the logs ?14:42
quiquell|roverAhh yeah14:42
*** bfournie has quit IRC14:43
openstackgerritAlex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration  https://review.openstack.org/63352714:43
pandaderekh:  ?14:46
derekhpanda: sorry, I vener saw your ping, looking now14:47
sshnaidmjaosorior, need your +2 here: https://review.openstack.org/#/c/636294/14:47
pandaderekh: thanks!14:47
*** janki has quit IRC14:47
jaosoriorsshnaidm: done14:48
quiquell|roverEmilienM: I don't see the retries here http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/journal.txt.gz14:49
quiquell|roverstatus is 404014:49
quiquell|rover40414:49
EmilienMsee https://bugs.launchpad.net/tripleo/+bug/1815576/comments/314:49
openstackLaunchpad bug 1815576 in tripleo "Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)14:50
EmilienMI'm digging more14:50
quiquell|roverEmilienM: let me put a unit test on that and fix14:50
quiquell|roverEmilienM: was smelling like that14:50
EmilienMa unit test on what? fix what?14:50
quiquell|roverEmilienM: we have to change retry_if_exception_type and add also this one14:51
quiquell|roverEmilienM: so it will retry14:51
*** udesale has quit IRC14:52
*** gkadam_ has quit IRC14:54
*** gkadam has joined #tripleo14:54
*** rbrady has joined #tripleo15:00
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add supplemental node to inventory  https://review.openstack.org/63609415:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add inventory and repos to freeipa playbook  https://review.openstack.org/63609715:02
*** udesale has joined #tripleo15:02
weshayjaosorior, do you know if http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/undercloud-deploy/templates/undercloud.conf.j2#n155 has been deprecated15:02
weshay?15:02
weshayhttps://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-8444/undercloud/home/stack/install-undercloud.log.gz15:03
Tenguo_O15:03
jaosoriorweshay: I don't know15:03
jaosoriorI thought it wasn't15:04
Tenguwasn't it replaced?15:04
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Refactor passing libvirt environment variables  https://review.openstack.org/63634715:05
openstackgerritMerged openstack/python-tripleoclient master: Support '--tags' for 'overcloud upgrade run'  https://review.openstack.org/63096715:05
Tenguweshay: at least it seems to be applied only for m-n-o-p - no more for newer version15:05
*** holser_ has quit IRC15:05
*** kplant has joined #tripleo15:05
Tenguso master shouldn't get it.15:06
weshayk15:06
weshaythanks Tengu15:06
openstackgerritAlex Schultz proposed openstack-infra/tripleo-ci master: Make container build job voting  https://review.openstack.org/63406015:06
Tenguline 15215:06
Tengu{% if release in ['mitaka', 'newton', 'ocata', 'pike'] %}15:06
derekhpanda: Looks like its not creating the ironic VM's, need to set "ironic_overcloud" here15:12
derekhhttp://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/validate-tempest/templates/cleanup-network.sh.j2#n5115:12
*** moguimar has quit IRC15:13
*** moguimar has joined #tripleo15:13
pandaderekh: ah, yeah, I'm not passing that new flag you just implemented. Thanks.15:13
derekhpanda: np15:14
quiquell|roverEmilienM: So I don't see the retries at the journal.log15:17
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431715:18
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to manage additional nova cells v2  https://review.openstack.org/63628615:18
EmilienMquiquell|rover: me neither, I see 2 attempts15:18
EmilienMactually no, I see one sorry15:19
quiquell|roverEmilienM: ok15:19
EmilienMI was confuses because the same missing blob was used by other images15:19
EmilienMand obviously it failed too15:19
Tengubeagles: heya! could you give a try to octavia deploy with the selinux change?15:22
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Restrict use of become to minimum necessary for Ceph deployment  https://review.openstack.org/63635315:24
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: Add featureset for scenario012 standalone  https://review.openstack.org/63635515:26
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Adding scenario012 standalone job  https://review.openstack.org/63472515:27
*** gbarros has joined #tripleo15:30
*** bfournie has joined #tripleo15:30
*** moguimar has quit IRC15:31
*** moguimar has joined #tripleo15:32
*** moguimar has quit IRC15:38
openstackgerritMerged openstack/tripleo-heat-templates master: Make ceph-ansible working dir owned by tripleo-admin  https://review.openstack.org/63608615:39
*** holser_ has joined #tripleo15:40
*** gbarros has quit IRC15:40
*** moguimar has joined #tripleo15:40
*** saneax has quit IRC15:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - run container-image-prepare with --debug  https://review.openstack.org/63635815:42
EmilienMquiquell|rover: ^ I'm using that to debug further in CI for now15:42
EmilienMI'm still looking at the code15:42
*** ykarel is now known as ykarel|away15:43
EmilienMalso I couldn't reproduce it in my env, probably happens with flacky networks or something15:43
*** vinaykns has joined #tripleo15:43
sshnaidmjaosorior, hrybacki those are ready for review: https://review.openstack.org/#/c/636094/ https://review.openstack.org/#/c/636097/15:46
jaosoriorsshnaidm: ack.15:47
openstackgerritMerged openstack-infra/tripleo-ci master: add 1.1.1.1 to multinode dns  https://review.openstack.org/63607315:48
*** Vorrtex has joined #tripleo15:49
quiquell|roverEmilienM:I am trying to put in place a little unit test for _cross_repo_mount15:50
quiquell|roverEmilienM: to test retries and whatnot15:50
*** aedc has quit IRC15:53
*** moguimar has quit IRC15:55
*** leanderthal has quit IRC15:56
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates master: Switch default neutron ML2 mechanism driver to OVN  https://review.openstack.org/59305615:57
*** rainleander has quit IRC15:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044516:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429816:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429316:00
*** mjturek has joined #tripleo16:00
sshnaidmbnemec, hi16:01
bnemecsshnaidm: o/16:02
sshnaidmbnemec, I'm working now with OVB stacks with "extra" node, and seems like there is no connections to extra node except 22. Also no ping to public-* network (only to private). I suspect something is wrong with port security of security groups..16:03
sshnaidmbnemec, for example I try to connect by 389 ldap port from undercloud node to extra node, and no connection16:03
sshnaidmbnemec, security groups: http://paste.openstack.org/show/744948/16:05
sshnaidmjaosorior, hrybacki which ports should be available on freeipa node? 389 of ldap and what else?16:06
hrybackisshnaidm: let me take that as a research item16:06
sshnaidmhrybacki, ack16:07
jaosoriorsshnaidm: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/scripts/freeipa_setup.sh16:07
sshnaidmbnemec, I think it's better to allow all communication between "regular" stack and "extra" stack..16:07
jaosoriorspecifically https://github.com/openstack/tripleo-heat-templates/blob/master/ci/scripts/freeipa_setup.sh#L5416:07
hrybackinevermind sshnaidm ^^16:08
sshnaidmI see, thanks16:08
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: wip: remove other-requirements.txt  https://review.openstack.org/63636616:08
bnemecsshnaidm: I think you want to use https://github.com/openstack/openstack-virtual-baremetal/blob/master/templates/baremetal-ports-extra-node-all.yaml16:09
bnemecIt disabled port-security on all the extra node ports.16:09
*** gkadam has quit IRC16:09
bnemec*disables16:09
bnemecI think we actually added that template in Denver for exactly this purpose. :-)16:10
bnemecThere should maybe be a new environment that uses it though, since the existing extra-node one uses the security group version.16:11
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: add push after build in tripleo-build-containers-centos-7  https://review.openstack.org/63522216:11
*** hberaud is now known as hberaud|school-r16:12
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates master: Switch default neutron ML2 mechanism driver to OVN  https://review.openstack.org/59305616:13
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/rocky: WIP: Split upgrade_steps_playbook into different plays.  https://review.openstack.org/63246916:13
*** gkadam has joined #tripleo16:14
sshnaidmbnemec, I think I use it: http://paste.openstack.org/show/744953/16:15
sshnaidmbnemec, and I remember it worked :)16:15
bnemecsshnaidm: No: OS::OVB::BaremetalPorts: templates/baremetal-ports-extra-node-port-security.yaml16:16
bnemecThat should be: OS::OVB::BaremetalPorts: templates/baremetal-ports-extra-node-all-port-security.yaml16:16
*** hberaud|school-r is now known as hberaud16:17
openstackgerritQuique Llorente proposed openstack/tripleo-common master: WIP: Add unit test for upload with 404 http code  https://review.openstack.org/63636916:17
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge  https://review.openstack.org/63637016:17
*** quiquell|rover is now known as quiquell|off16:17
*** jfrancoa has quit IRC16:18
sshnaidmbnemec, oh, I see.. will test it, thanks!16:18
bnemecsshnaidm: If you want to add a new sample environment for that it would be good.16:19
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten tripleo-ui service configuration  https://review.openstack.org/63620316:19
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637116:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637216:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637316:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637416:20
ksamborHello, this error is something that occured periodicly for rdo-ci http://pastebin.test.redhat.com/713054 ?16:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637516:20
*** abishop_ has joined #tripleo16:20
openstackgerritMerged openstack/tripleo-quickstart-extras master: warnings should be first and in red  https://review.openstack.org/63602816:21
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten sensu service configuration  https://review.openstack.org/63620616:22
openstackgerritBrent Eagles proposed openstack/puppet-tripleo master: Change haproxy mode to http for octavia  https://review.openstack.org/63637616:22
*** abishop has quit IRC16:22
*** ccamacho has quit IRC16:23
*** ccamacho has joined #tripleo16:24
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten collectd service configuration  https://review.openstack.org/63623216:25
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten ceilometer service configuration  https://review.openstack.org/63361716:27
*** mcornea has quit IRC16:28
*** mcornea has joined #tripleo16:29
*** udesale has quit IRC16:37
*** paramite has quit IRC16:41
weshayEmilienM, what are you testing?16:46
EmilienMweshay: I want to see the failure with --debug16:47
EmilienMin tripleo container image prepare16:47
weshayk16:47
EmilienMI managed to reproduce weird things locally, I suspect the number of workers is too high for some methods16:47
*** cylopez has quit IRC16:48
EmilienMalso it's hard to know since we are alone using centos7 & podman, at that scale16:48
EmilienMit could be our kernel or overlayfs16:48
*** hakhande_ has quit IRC16:48
*** ykarel|away has quit IRC16:49
openstackgerritBrent Eagles proposed openstack/puppet-tripleo master: Run octavia-api under apache  https://review.openstack.org/63637916:50
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run octavia-api under httpd  https://review.openstack.org/63638016:51
*** ccamacho has quit IRC16:53
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: add push after build in tripleo-build-containers-centos-7  https://review.openstack.org/63522216:53
hjensasbandini: still around?16:54
hjensasbandini: any reason a provider such as https://github.com/voxpupuli/puppet-rabbitmq/blob/master/lib/puppet/provider/rabbitmq_plugin/rabbitmqplugins.rb would'nt work with our containers?16:55
*** mrunge_ is now known as mrunge16:56
*** ramishra has quit IRC16:57
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: wip: remove other-requirements.txt  https://review.openstack.org/63636616:59
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: WIP: bindep can be called w/o -f file  https://review.openstack.org/63638516:59
*** sdake_ has quit IRC17:01
*** hakhande_ has joined #tripleo17:01
*** sdake has joined #tripleo17:02
*** hamzy has quit IRC17:06
*** yprokule has quit IRC17:07
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: WIP: add push after build in tripleo-build-containers-centos-7  https://review.openstack.org/63522217:09
*** gkadam has quit IRC17:19
*** jpich has quit IRC17:24
*** social has quit IRC17:28
*** aedc has joined #tripleo17:33
*** gfidente is now known as gfidente|afk17:34
bogdandoslagle: hi. could you please remove WIP https://review.openstack.org/#/c/631489/1?17:35
bogdandoI really want that as a placeholder for the patch sitting on top... We could update it later in follow-ups17:35
*** social has joined #tripleo17:35
*** spsurya has quit IRC17:40
slaglebogdando: i'll need to update it first17:41
bogdandoslagle: ok, thanks17:41
*** gbarros has joined #tripleo17:44
*** sdake has quit IRC17:44
openstackgerritMerged openstack/tripleo-common master: Fixes OVN docker image prep with Neutron  https://review.openstack.org/59687117:45
*** dtrainor has quit IRC17:46
*** sdake has joined #tripleo17:46
*** dtrainor has joined #tripleo17:48
EmilienMquiquell|off: I reproduced17:48
EmilienMhttp://logs.openstack.org/70/636370/1/check/tripleo-ci-centos-7-standalone/635f0ea/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz17:48
*** hakhande_ has quit IRC17:49
sshnaidmbnemec, thanks, worked like a charm17:51
*** bogdando has quit IRC17:52
sshnaidmbnemec, another question though :) when extra node is created it has IPs in 2 networks, but I see only one (private) on the host: http://paste.openstack.org/show/744959/17:52
sshnaidmbnemec, actually in 3 networks: provision-4213=192.0.2.2; public-4213=10.0.0.3; private=192.168.100.1517:52
bnemecsshnaidm: That's because there's no DHCP on provision or public.17:52
bnemecAlso, I believe cloud-init only DHCP's the first interface.17:53
bnemecThe others have to be configured manually.17:53
sshnaidmbnemec, I see, so to configure them manually using info from stack?17:53
sshnaidmbnemec, well, undercloud node got all IPs17:54
bnemecsshnaidm: Not necessarily. I would recommend just assigning a static IP like we do for the undercloud.17:54
bnemecObviously make sure it's outside the DHCP range configured on the undercloud though.17:54
EmilienMweshay, quiquell|off : found the root cause: https://bugs.launchpad.net/tripleo/+bug/1815576/comments/717:54
openstackLaunchpad bug 1815576 in tripleo "Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)17:54
openstackgerritMerged openstack/tripleo-heat-templates master: Remove RoleConfig  https://review.openstack.org/63297417:54
* weshay looks17:54
weshayEmilienM, nice man17:55
sshnaidmbnemec, where are we assigning static IP to undercloud?17:56
*** derekh has quit IRC17:56
bnemecsshnaidm: undercloud.conf17:56
bnemecAlso, I assume an IP is getting assigned to the public network too.17:56
sshnaidmbnemec, well, so in our case it's manually17:57
sshnaidmbnemec, in undercloud host I have all IPs are assigned, yet before undercloud is installed17:58
*** dtantsur is now known as dtantsur|afk17:58
*** kopecmartin is now known as kopecmartin|off17:58
sshnaidmbnemec, oh, no.. I have 10.0.0.1 in undercloud17:59
sshnaidmwhich is public network17:59
bnemecsshnaidm: That's possible. There's no reason you can't configure IPs before installing the undercloud, but _something_ is doing it.17:59
sshnaidmjaosorior, hrybacki is it important which freeipa IP to use in job? from private/public/other network?18:00
sshnaidmjaosorior, hrybacki it uses network isolation18:00
sshnaidmweshay, https://review.openstack.org/#/c/636094/18:03
openstackgerritMerged openstack/tripleo-puppet-elements stable/rocky: Use external centos7-rt if present  https://review.openstack.org/63627718:03
openstackgerritMerged openstack/tripleo-puppet-elements stable/queens: Use external centos7-rt if present  https://review.openstack.org/63627818:03
openstackgerritMerged openstack/tripleo-puppet-elements stable/pike: Use external centos7-rt if present  https://review.openstack.org/63628918:03
*** artom has quit IRC18:04
*** agopi is now known as agopi|food18:06
*** panda is now known as panda|off18:06
*** paramite has joined #tripleo18:10
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: image_uplaoder: add retry to _layer_stream_registry  https://review.openstack.org/63640518:11
EmilienMweshay, quiquell|off : this might help ^18:11
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - run container-image-prepare with --debug  https://review.openstack.org/63635818:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge  https://review.openstack.org/63637018:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637118:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637218:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637318:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637418:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test $î - never merge  https://review.openstack.org/63637518:13
EmilienMweshay: did we figure out why the tripleo container image prepare logs aren't indexed by logstash?18:14
EmilienMit's pretty critical to have it18:14
*** aufi has quit IRC18:19
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640618:20
*** homeski has joined #tripleo18:22
*** jpena is now known as jpena|off18:23
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63599118:24
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Add fedora-28 support to container-build job on fedora-28 with Buildah  https://review.openstack.org/63599118:24
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add settings to launcher-playbook for OVB  https://review.openstack.org/63640818:25
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Configure FreeIPA vars  https://review.openstack.org/63640918:27
*** agopi|food is now known as agopi18:30
hrybackisshnaidm: it needs to be on the ctlplane18:32
hrybackiat least for the time being18:34
sshnaidmhrybacki, it means on "private" network?18:34
*** panda|off has quit IRC18:34
hrybackisshnaidm: IIRC yes -- too many names for the same thing18:34
sshnaidmhrybacki, yeah, ok18:35
*** gcerami_ has joined #tripleo18:37
*** khyr0n has joined #tripleo18:38
*** gloriapg has joined #tripleo18:39
*** trown is now known as trown|lunch18:41
weshayEmilienM, not yet18:42
*** hamzy has joined #tripleo18:43
*** sdake has quit IRC18:44
*** hberaud is now known as hberaud|gone18:44
*** wolverineav has joined #tripleo18:45
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Add python3/fedora28 support to tripleo-repos  https://review.openstack.org/63616018:50
*** paramite has quit IRC18:50
*** sdake has joined #tripleo18:51
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Use net=none for gnocchi_init_lib  https://review.openstack.org/63642018:54
*** _fragatina has joined #tripleo18:55
*** slaweq has quit IRC18:59
*** wolverineav has quit IRC19:00
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Add python3/fedora28 support to tripleo-repos  https://review.openstack.org/63616019:02
*** wolverineav has joined #tripleo19:04
*** adrianreza has quit IRC19:04
*** dtantsur|afk has quit IRC19:04
*** mcornea has quit IRC19:04
*** bandini has quit IRC19:04
*** dpeacock has quit IRC19:04
*** jroll has quit IRC19:04
*** mugsie has quit IRC19:04
*** jschlueter has quit IRC19:04
*** tzumainn has quit IRC19:04
*** dr_gogeta86 has quit IRC19:04
*** jschlueter has joined #tripleo19:05
*** mugsie has joined #tripleo19:05
*** dr_gogeta86 has joined #tripleo19:05
*** mcornea has joined #tripleo19:06
*** jroll has joined #tripleo19:07
*** tzumainn has joined #tripleo19:07
*** dtantsur has joined #tripleo19:08
*** dpeacock has joined #tripleo19:08
sshnaidmbnemec, btw, can we merge it? https://review.openstack.org/#/c/633681/19:11
*** dprince has quit IRC19:12
*** gcerami_ is now known as panda19:14
bnemecsshnaidm: Based on Sorin's comment, can you open a bug with details about the problem? It would be good to reference that from the TODO comment.19:18
sshnaidmbnemec, bug for which project?19:19
zbrbnemec: i think that import openstack_virtual_baremetal.auth should work well in both.19:19
bnemecsshnaidm: Just against tripleo. Maybe we should add an official ovb tag.19:20
sshnaidmbnemec, ok19:20
bnemeczbr: It doesn't because we don't actually install it as a package in real world use.19:20
bnemecSo there's no openstack_virtual_baremetal package to import from.19:20
zbrbnemec: why not?19:21
bnemecThat's why https://github.com/openstack/openstack-virtual-baremetal/commit/9e7ee5dd5c399fb6edeee771d9027846c4bdd1b6 broke it functionally.19:21
zbrif I remember well, relative imports are no longer allowed in newer python, right19:22
bnemeczbr: They're allowed, but not the implicit form that this was using.19:23
openstackgerritMerged openstack/tripleo-quickstart master: Add centos7-rt repository  https://review.openstack.org/63515719:23
openstackgerritMerged openstack/tripleo-quickstart master: Ensure the latest rpms versions are installed  https://review.openstack.org/63623019:23
sshnaidmbnemec, https://bugs.launchpad.net/tripleo/+bug/181566019:23
openstackLaunchpad bug 1815660 in tripleo "OVB import auth fails in python3" [Undecided,Triaged] - Assigned to Ben Nemec (bnemec)19:23
bnemeczbr: I also tried this: https://review.openstack.org/#/c/633550/2/openstack_virtual_baremetal/deploy.py19:23
bnemecWhich works in unit tests, but fails when you run it for real because . isn't a package or something.19:24
bnemecI forget the exact error.19:24
zbrbrb after dinner, i know some ugly code involving inspect that can do it but is need to find where I used, it was very long time ago. there is a reason why they changed the relative imports....19:25
bnemecYeah, most likely the right fix is just to pip install it before use, but all of the code assumes it's being run from the git repo right now so that may be non-trivial.19:27
bnemecAlthough...maybe not if we just keep the requirement that the commands get run with the git repo as working directory.19:27
sshnaidmbnemec, we can try to use "pip install ." from git dir, right?19:28
bnemecWe don't really care where the code lives. All that would change is you'd need to run ovb-deploy or something instead of bin/deploy.py.19:28
bnemecsshnaidm: Right.19:28
bnemecMy previous concerns were around RPM packaging, but we don't necessarily need to go that far.19:29
sshnaidmbnemec, well, to have it part of out pipeline is fine I think..19:29
sshnaidms/out/our19:29
sshnaidmbnemec, I'll try with pip install, let's see how it works..19:30
bnemecsshnaidm: Sounds good.19:31
openstackgerritBen Nemec proposed openstack/openstack-virtual-baremetal master: Import auth with try/except  https://review.openstack.org/63368119:33
*** artom has joined #tripleo19:34
*** artom has quit IRC19:35
*** artom has joined #tripleo19:35
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect IPA install log  https://review.openstack.org/63643019:39
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640619:39
*** sdake has quit IRC19:42
*** wolverineav has quit IRC19:43
*** wolverineav has joined #tripleo19:44
*** trown|lunch is now known as trown19:44
*** wolverineav has quit IRC19:46
*** wolverineav has joined #tripleo19:47
*** wolverineav has quit IRC19:49
*** wolverineav has joined #tripleo19:50
*** jfrancoa has joined #tripleo19:52
*** dprince has joined #tripleo19:54
*** wolverineav has quit IRC19:55
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add settings to launcher-playbook for OVB  https://review.openstack.org/63640819:55
*** jfrancoa has quit IRC19:56
*** wolverineav has joined #tripleo19:57
*** morazi has quit IRC19:58
*** dprince has quit IRC20:02
openstackgerritTrevor Vardeman proposed openstack/tripleo-common master: Add manifest list image support to the kolla builder  https://review.openstack.org/62039720:15
openstackgerritMerged openstack/openstack-virtual-baremetal master: Import auth with try/except  https://review.openstack.org/63368120:17
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640620:18
*** slaweq has joined #tripleo20:19
*** wolverineav has quit IRC20:20
*** wolverineav has joined #tripleo20:21
*** wolverineav has quit IRC20:26
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Move ipa enrollment to host_prep_tasks  https://review.openstack.org/63643420:27
EmilienMweshay: https://review.openstack.org/#/c/636405/ looks promising, I don't see any failure in all the 7 depends-On20:32
EmilienMyou can check 636358 in the check pipeline in zuul20:32
*** xek__ has joined #tripleo20:35
openstackgerritMerged openstack-infra/tripleo-ci master: Add collecting files for hosts without zuul user  https://review.openstack.org/63629420:36
weshaycool20:37
*** xek_ has quit IRC20:37
*** dciabrin has quit IRC20:38
*** dciabrin has joined #tripleo20:39
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640620:41
*** galaxyyahoo has quit IRC20:45
*** galaxyyahoo has joined #tripleo20:46
*** mjturek has quit IRC20:47
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten tripleo-ui service configuration  https://review.openstack.org/63620320:49
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten sensu service configuration  https://review.openstack.org/63620620:57
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: flatten collectd service configuration  https://review.openstack.org/63623220:59
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640620:59
*** wolverineav has joined #tripleo21:00
*** morazi has joined #tripleo21:01
*** slaweq_ has joined #tripleo21:04
*** slaweq has quit IRC21:06
*** dtantsur has quit IRC21:06
*** matbu has quit IRC21:06
*** tzumainn has quit IRC21:06
*** mugsie has quit IRC21:06
*** dtantsur has joined #tripleo21:06
*** mugsie has joined #tripleo21:06
*** matbu has joined #tripleo21:07
*** mjturek has joined #tripleo21:08
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Configure FreeIPA vars  https://review.openstack.org/63640921:13
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Set DNS for jobs dynamically in OVB  https://review.openstack.org/63643821:15
*** holser_ has quit IRC21:15
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Unset dns variables for OVB  https://review.openstack.org/63643921:15
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: wip: remove other-requirements.txt  https://review.openstack.org/63636621:15
*** raildo has quit IRC21:18
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Support for Buildah in kolla_builder  https://review.openstack.org/63431721:21
*** dsneddon has quit IRC21:22
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640621:24
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Add python3/fedora28 support to tripleo-buildcontainers  https://review.openstack.org/63616021:26
*** sdake has joined #tripleo21:26
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: [DNM] Test tripleo-buildcontainers on py3/f28  https://review.openstack.org/63618821:26
*** wolverineav has quit IRC21:30
*** xek__ has quit IRC21:31
EmilienMweshay, quiquell|off : FYI new podman + new runc incoming to tripleo ci - we landed https://review.rdoproject.org/r/#/c/18815/21:31
*** dprince has joined #tripleo21:31
*** slaweq_ is now known as slaweq21:32
*** wolverineav has joined #tripleo21:34
fultonjEmilienM: do you think will that help with https://bugs.launchpad.net/tripleo/+bug/1814860 ?21:34
openstackLaunchpad bug 1814860 in tripleo "overcloud deploy with podman fails on step4 with systemctl failed error" [High,Triaged]21:34
EmilienMfultonj: looking21:34
weshayEmilienM got a sec?21:34
*** jtomasek has quit IRC21:35
EmilienMweshay: wait plz21:35
weshayaye21:35
EmilienMfultonj: yes this problem should be gone21:35
EmilienMlet me tripleo check21:35
EmilienMyes I confirm fultonj21:36
*** trown is now known as trown|outtypewww21:36
EmilienMweshay: ok what's up21:36
weshayjust reviewing those jobs and our comfort level w/ introducing the new version across all jobs21:36
EmilienMweshay: yeah we should be good21:37
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop Docker services from Overcloud roles  https://review.openstack.org/63644121:37
*** agopi_ has joined #tripleo21:37
fultonjEmilienM: thanks, i've been waiting for that to be gone so i can keep moving on https://review.openstack.org/#/c/617368/21:37
EmilienMfultonj: good21:37
EmilienMdprince: ^ FYI see my comment21:38
dprinceEmilienM: ack, just queing them up. It works for me locally21:39
EmilienMdprince: because you didn't have pacemaker21:39
EmilienMbut yeah we need this soon21:39
*** agopi has quit IRC21:39
EmilienMnot sure what to do with ovb jobs, need to talk with bandinir21:39
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop Docker service from Undercloud roles  https://review.openstack.org/63644221:39
EmilienMweshay: https://github.com/containers/libpod/commit/921f98f8795eb9fcb19ce581020cfdeff6dee09f#diff-e8800259c4985f4a73b0936128034c5321:40
EmilienMFYI21:40
EmilienMlooks pretty safe to me21:40
dprinceEmilienM: I don't use pacemaker (usually). correct21:40
dprincethis was a blocker for me though: https://review.openstack.org/#/c/636420/21:41
*** agopi_ is now known as agopi21:41
weshayEmilienM, aight.. thanks. I would still rather push the rdo packagers to squash dlrn and dlrn deps into a single repo21:41
dprincegnocchi won't work w/ podman ATM21:41
dprincewithout that patch21:41
*** wolverineav has quit IRC21:41
EmilienMdprince: also commented this patch ^ but good to have it in queue21:41
weshayso these changes can have the same level of assurance as a change from nova21:41
*** wolverineav has joined #tripleo21:41
EmilienMweshay: you mean automatic updates? I think Alfredo was working on it21:41
weshayEmilienM, no21:42
dprinceEmilienM: I didn't remove DockerRegistry fwiw. Just docker so it should be fine I think21:42
*** artom has quit IRC21:42
weshayEmilienM, I mean dep changes are treated the same way as openstack changes21:42
*** Vorrtex has quit IRC21:42
weshayEmilienM, no more dlrn-deps21:42
weshayjust dlrn21:42
weshayone repo21:42
EmilienMdprince: if you remove this service, docker engine won't run, so all docker commands will fail because no socket21:42
EmilienMweshay: i don't have strong opinion on that, except I want to control when we update these things21:43
EmilienMand have automatic testing21:43
dprinceEmilienM: nah, it already works without it I think. skopeo man21:43
weshayright.. keep what we have but after https://review.rdoproject.org/r/#/c/18815/ merges.. it would go into the pipeline to be tested w/ all the latest changes21:43
* dprince just tried it today and it worked for him21:43
EmilienMdprince: ok good then21:44
EmilienMI'm fine removing docker everywhere, if that works :D21:45
*** hamzy has quit IRC21:46
openstackgerritDan Prince proposed openstack/tripleo-puppet-elements master: Drop docker packages from overcloud-base  https://review.openstack.org/63644421:47
*** mjturek has quit IRC21:48
*** sdake has quit IRC21:50
weshayoh shit.. I think both upstream and rdo are down21:50
fultonjsomeone typed recheck too many times21:51
weshay2019-02-12 21:40:50.348996 | primary | The error appears to have been in '/home/zuul/workspace/logs/zuul-variables.yaml': line 142, column 1, but may21:52
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640621:53
weshayrlandy, ^21:53
EmilienMfor once it's not podman21:54
* EmilienM runs21:54
rlandyweshay: is that from my patch adding to zuul-variables?21:55
rlandyif so, yeah - I know it's not working21:55
*** zaneb has joined #tripleo21:58
*** pv2b has joined #tripleo22:06
*** gloriapg has quit IRC22:08
*** gloriapg has joined #tripleo22:10
*** NobodyCam has quit IRC22:12
*** NobodyCam has joined #tripleo22:12
openstackgerritHarald Jensås proposed openstack/instack-undercloud stable/rocky: Fix - delete ironic-neutorn-agent queue to allow auto_delete  https://review.openstack.org/63644922:13
openstackgerritHarald Jensås proposed openstack/instack-undercloud stable/queens: Fix - delete ironic-neutorn-agent queue to allow auto_delete  https://review.openstack.org/63645122:15
weshayrfolco, you chat w/ EmilienM yet?22:17
rfolcono, I got some progress, trying to fix errors22:18
weshayrfolco, /me compares https://review.openstack.org/#/c/635991/9/zuul.d/build-containers.yaml w/ https://review.openstack.org/#/c/636188/5/zuul.d/build-containers.yaml22:18
openstackgerritMerged openstack/python-tripleoclient master: Use cliff autodoc generation  https://review.openstack.org/52351022:18
openstackgerritMerged openstack/tripleo-quickstart master: Add supplemental node to inventory  https://review.openstack.org/63609422:18
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add inventory and repos to freeipa playbook  https://review.openstack.org/63609722:18
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640622:19
rfolcoweshay, EmilienM so you think its conflicting or waste of time in making f28 buildcontainers as-is without buildah ?22:20
mwhahahait's not exactly a waste of time22:21
weshayrfolco, ya.. we should continue down the path ur on.. but we could at least have single format for the job def :)22:22
mwhahahatechincally if you get it working with the existing stuff, it should be a quick switch to the buildha version22:22
mwhahahabecause we're just going to try and add a cli cmdline to switch in tripleoclient22:22
weshayhttps://review.openstack.org/#/c/636188/5/zuul.d/build-containers.yaml vs https://review.openstack.org/#/c/635991/9/zuul.d/build-containers.yaml22:22
rfolcomwhahaha, I am trying to add f28/py3 support to tripleo-buildcontainers playbook... https://review.openstack.org/#/c/636160/, which is tested in https://review.openstack.org/#/c/636188 as weshay pointed out22:26
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: wip: remove other-requirements.txt  https://review.openstack.org/63636622:26
mwhahahayea i'm not sure we can add that just yet22:27
*** kplant has quit IRC22:27
mwhahahaas the kolla bits are still up for discussion22:27
mwhahahaso it's probably not a good idea to try and build the job just yet22:27
weshayrfolco, /me wonders if we can get all the installs done by adding a bindep file in tripleo-ci22:27
weshayrfolco, can I show you something for a sec?22:28
rfolcoweshay, like we do for tripleoclient22:28
rfolcoweshay, sure22:29
rfolcoI just wanted to ask mwhahaha question about this error while starting the containers he built22:29
weshayk.. in my blue22:29
rfolcoERROR:__main__:MissingRequiredSource: /var/lib/kolla/config_files/src/etc/httpd/conf.d file is not found22:29
weshayk k22:29
rfolcosrc files are not there22:30
*** jjoyce|gone is now known as jjoyce22:30
mwhahahai've not run into that one22:30
mwhahahathere was an issue around httpd where we were getting some extra files22:30
rfolcook mwhahaha thx, will try to figure it out22:32
rfolcoweshay, joining sorry22:33
weshayrfolco, https://github.com/openstack/tripleo-quickstart/blob/master/bindep.txt22:34
*** slaweq has quit IRC22:34
*** mcornea has quit IRC22:34
*** agopi has quit IRC22:43
rfolcoEmilienM, reparenting to tripleo-build-containers-base on https://review.openstack.org/#/c/635991/9/zuul.d/build-containers.yaml looks much cleaner. I'll copy your idea on my patch so we're aligned in case of a merge.22:50
*** ekultails has quit IRC22:50
*** dprince has quit IRC22:51
*** wolverineav has quit IRC22:54
*** wolverineav has joined #tripleo22:55
*** wolverineav has quit IRC22:58
*** wolverineav has joined #tripleo22:59
openstackgerritSteve Baker proposed openstack/tripleo-common master: Handle uncompressed layers on image export  https://review.openstack.org/63646023:00
openstackgerritBen Nemec proposed openstack/openstack-virtual-baremetal master: Enable pip install  https://review.openstack.org/63646123:01
openstackgerritBen Nemec proposed openstack/openstack-virtual-baremetal master: Fix environment-generator  https://review.openstack.org/63646223:01
openstackgerritBen Nemec proposed openstack/openstack-virtual-baremetal master: Update baremetal-ports-extra-node-all  https://review.openstack.org/63646323:01
openstackgerritBen Nemec proposed openstack/openstack-virtual-baremetal master: Add base-extra-node-all sample environment  https://review.openstack.org/63646423:01
bnemecsshnaidm: ^23:02
*** gfidente|afk has quit IRC23:04
openstackgerritSteve Baker proposed openstack/tripleo-common master: Handle uncompressed layers on image export  https://review.openstack.org/63646023:05
openstackgerritSteve Baker proposed openstack/tripleo-common master: Role to install apache to serve container images  https://review.openstack.org/62482923:05
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add job variables to be available in zuul-variables  https://review.openstack.org/63640623:05
openstackgerritMerged openstack/paunch master: systemd: fix a typo in comments  https://review.openstack.org/63612323:13
*** rascasoft has quit IRC23:13
*** tkajinam has joined #tripleo23:16
*** kplant has joined #tripleo23:16
*** rascasoft has joined #tripleo23:16
*** cfontain has joined #tripleo23:16
*** artom has joined #tripleo23:18
*** florianf has quit IRC23:20
*** cfontain has quit IRC23:21
*** cfontain has joined #tripleo23:23
*** vinaykns has quit IRC23:26
*** rascasoft has quit IRC23:27
*** cfontain has quit IRC23:30
*** agopi has joined #tripleo23:33
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add python3 packages to overrides (package-replacement)  https://review.openstack.org/63647223:34
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add python3 packages to overrides (package-replacement)  https://review.openstack.org/63647223:35
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add python3 packages to overrides  https://review.openstack.org/62967923:35
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - WIP - Add job variables to be available in zuul-variables  https://review.openstack.org/63640623:36
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add python3 packages to overrides  https://review.openstack.org/62967923:40
*** rascasoft has joined #tripleo23:42
*** sdake has joined #tripleo23:43
*** ade_lee_ has quit IRC23:47
*** rh-jelabarre has quit IRC23:48
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Add python3 packages to overrides (package-replacement)  https://review.openstack.org/63647223:49
*** wolverineav has quit IRC23:49
*** wolverineav has joined #tripleo23:50
*** rascasoft has quit IRC23:55
*** wolverineav has quit IRC23:56
*** cfontain has joined #tripleo23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!