Friday, 2017-10-27

*** itlinux has joined #tripleo00:10
*** catintheroof has joined #tripleo00:11
*** rcernin has joined #tripleo00:21
*** dtrainor has quit IRC00:22
*** bobh has joined #tripleo00:22
*** gkadam has joined #tripleo00:23
*** gkadam_ has quit IRC00:25
*** catintheroof has quit IRC00:32
*** catintheroof has joined #tripleo00:38
*** catintheroof has quit IRC00:42
*** catintheroof has joined #tripleo00:44
itlinuxhello all.. from my UC when I do the ipmitool -I lanplus  it work when I do it with ipmitool -I lan it does not and my ironic node-show do show error.. what's the best way to configure it to use the right command. TY00:46
*** hjensas has quit IRC00:55
*** achadha has joined #tripleo00:55
*** achadha has quit IRC00:55
*** catintheroof has quit IRC00:55
*** achadha has joined #tripleo00:55
*** catintheroof has joined #tripleo00:56
*** zzzeek has quit IRC00:58
*** zzzeek has joined #tripleo01:00
*** gkadam has quit IRC01:07
*** amodi has joined #tripleo01:20
*** catintheroof has quit IRC01:20
*** fzdarsky_ has joined #tripleo01:21
*** catintheroof has joined #tripleo01:21
*** catintheroof has quit IRC01:24
*** fzdarsky|afk has quit IRC01:24
*** psachin has joined #tripleo01:36
*** rcernin has quit IRC01:43
*** rcernin has joined #tripleo01:44
*** catintheroof has joined #tripleo01:55
*** catintheroof has quit IRC02:00
*** achadha has quit IRC02:00
*** achadha has joined #tripleo02:01
*** achadha has quit IRC02:05
*** bobh has quit IRC02:11
*** achadha has joined #tripleo02:18
*** achadha has quit IRC02:18
*** achadha has joined #tripleo02:19
*** achadha_ has joined #tripleo02:23
*** achadha has quit IRC02:24
*** dparkes has quit IRC02:24
*** catintheroof has joined #tripleo02:28
*** yamahata has quit IRC02:29
*** catintheroof has quit IRC02:33
*** dparkes has joined #tripleo02:36
*** fragatina has quit IRC02:40
*** dparkes has quit IRC02:44
*** dparkes has joined #tripleo02:45
*** fragatina has joined #tripleo02:45
*** trozet has joined #tripleo02:47
*** fragatina has quit IRC02:50
*** amodi has quit IRC02:53
openstackgerritSai Ram Peesapati proposed openstack/tripleo-heat-templates stable/newton: Include Lbaas Agent in stable newton  https://review.openstack.org/51557402:55
openstackgerritSai Ram Peesapati proposed openstack/puppet-tripleo stable/newton: Include Lbaas agent in stable newton  https://review.openstack.org/51557502:58
tonybitlinux: That implies that when you did the introspection you got a protocol_verion other than 2.0 .. Can you validate (or update) the driver_info associated with your node(s)02:59
*** pmannidi is now known as pmannidi|brb03:00
itlinuxhow would I change that.. any tips.. that will be great... thanks tonyb:03:00
*** achadha has joined #tripleo03:01
*** achadha has quit IRC03:01
*** achadha has joined #tripleo03:01
*** achadha has quit IRC03:01
*** achadha_ has quit IRC03:01
itlinuxhttp://paste.openstack.org/show/624773/03:01
itlinuxtonyb: here is what is shows..03:01
tonybitlinux: I'm trying to re-create it here03:03
itlinuxthanks03:03
*** dparkes has quit IRC03:06
*** shreshtha has joined #tripleo03:07
tonybitlinux: what version of ironic do you have?03:07
itlinuxone sec I lost vpn connection..03:08
tonybnp03:08
*** gvrangan has joined #tripleo03:08
itlinuxironic --version03:09
itlinux1.11.103:09
openstackgerritTim Rozet proposed openstack/puppet-tripleo master: Enables websocket based port status for OpenDaylight  https://review.openstack.org/51557603:11
*** trozet has quit IRC03:11
openstackgerritTim Rozet proposed openstack/puppet-tripleo master: Enables websocket based port status for OpenDaylight  https://review.openstack.org/51557603:13
tonybitlinux: for reasons, that's the client version not the server version.  How'd you install it?03:14
itlinuxtripleo03:15
itlinuxpart of the undercloud03:15
itlinuxcentos03:15
openstackgerritTim Rozet proposed openstack/puppet-tripleo master: Enables websocket based port status for OpenDaylight  https://review.openstack.org/51557603:15
itlinuxpuppet-ironic-10.4.1-1.el7.noarch03:16
tonybpike?03:16
itlinuxocata03:16
tonybokay thanks03:16
itlinuxthank you!03:16
*** dparkes has joined #tripleo03:18
itlinuxI also found that if I do install the same version with UUID LDAP groups work.. if I install it with Fernet openstack group list --domain xxxxxx.com do not show anything..03:19
itlinuxso not sure if that'a bug!03:19
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51163303:20
tonybitlinux: Try ironic node-update $NODE_UUID add driver_info/ipmi_protocol_version=2.003:22
itlinuxok...03:22
tonybitlinux: sorry I don't know the state of support for Fernet in ocata03:22
itlinuxok03:23
itlinuxhow do I update the info now..03:23
itlinuxsince it still shows the error..03:24
*** dparkes has quit IRC03:24
tonybthat command didn't update it?03:26
itlinuxhttp://paste.openstack.org/show/624776/03:27
itlinuxdoes not look to me..03:27
tonybYeah it's there. I guess we need to force ironic to revalidate the data in the database ...03:30
tonybYou could try reestarting the service? I admit I don't know how to do force a refresh03:31
itlinuxok..03:31
itlinuxok thanks tonyb.03:31
itlinuxwhich services is that..03:31
itlinuxhave it on top of your head?03:31
itlinuxI can search for it..03:31
tonybsystemctl restart ironic* ?03:32
tonybsorry systemctl restart openstack-ironic*03:32
itlinuxok03:33
*** achadha has joined #tripleo03:35
*** yamahata has joined #tripleo03:39
*** achadha has quit IRC03:43
*** achadha has joined #tripleo03:43
itlinuxtonyb: do you know anything about dual LDAP servers..03:46
itlinuxI have a config but for some reasons it fails..if I have two of them..03:46
*** dparkes has joined #tripleo03:47
tonybitlinux: sorry no03:47
itlinuxok thanks tonyb:03:48
*** achadha has quit IRC03:48
tonybitlinux: did the node work?03:49
itlinuxno updates..03:50
itlinuxbut the nodes is active..03:50
itlinuxa8033033-f884-4a26-8905-e815f7f4effb | None | 5dad9929-4975-43eb-af56-a25046f69144 | None        | active             | True03:50
itlinuxit's active but does not show anything..03:50
tonyb:(03:52
* tonyb is logging off now03:52
tonybsorry I couldn't help more03:52
*** fragatina has joined #tripleo03:57
*** fragatina has quit IRC03:58
*** udesale has joined #tripleo03:58
*** fragatina has joined #tripleo03:58
*** catintheroof has joined #tripleo04:01
*** psachin has quit IRC04:03
*** dsneddon has quit IRC04:03
*** catintheroof has quit IRC04:05
*** pdeore has joined #tripleo04:08
*** mdnadeem has joined #tripleo04:08
*** ykarel has joined #tripleo04:10
*** tzumainn has quit IRC04:15
*** fragatina has quit IRC04:16
*** dparkes has quit IRC04:18
*** zoli has quit IRC04:19
*** gkadam has joined #tripleo04:19
*** dbecker has quit IRC04:20
*** psachin has joined #tripleo04:21
*** zoli has joined #tripleo04:21
*** masco has joined #tripleo04:27
*** dparkes has joined #tripleo04:30
*** janki has joined #tripleo04:36
*** dbecker has joined #tripleo04:36
*** skramaja has joined #tripleo04:37
*** pmannidi|brb has quit IRC04:38
*** pmannidi|brb has joined #tripleo04:39
*** pmannidi|brb is now known as pmannidi04:39
*** pmannidi has quit IRC04:39
*** pmannidi has joined #tripleo04:40
EmilienMadarazs|rover, panda|ruck|off: when you're back online, periodic jobs fail on REPORTING FAILURE TO DLRN API04:42
EmilienMjpena|off, amoralej|off ^04:42
EmilienMhttps://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-master/264/console04:42
EmilienMI have no clue tbh04:42
EmilienMplease let me know if there is more blockers / bugs04:42
*** ramishra has quit IRC04:42
*** ramishra has joined #tripleo04:44
*** achadha has joined #tripleo04:51
*** achadha has quit IRC04:51
*** gvrangan has quit IRC04:52
*** achadha has joined #tripleo04:52
*** gvrangan has joined #tripleo04:52
*** dparkes has quit IRC04:54
*** nguyentrihai has joined #tripleo04:55
Tenguhumpf… the CI is under water. That's unbearable, all those checks failing due to timeout :/.04:57
*** dparkes has joined #tripleo05:00
*** dparkes has quit IRC05:05
*** jaosorior has joined #tripleo05:05
*** dparkes has joined #tripleo05:06
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs  https://review.openstack.org/51245005:10
EmilienMTengu: again? which one? send link05:11
jaosoriorgood morning everyone!05:11
chandankumarEmilienM: hello05:13
EmilienMI was about to go to bed05:13
EmilienMwhat's up?05:13
chandankumarEmilienM: https://review.openstack.org/#/c/504899/ i am working on packaging zaqar tempest plugin when you wake up i hope it will be available05:13
jaosoriorholy crap05:13
jaosoriorthat's a big patch05:13
chandankumarhehe05:14
EmilienMchandankumar: yeah PTL pinged me about that05:14
jaosorioroh05:14
EmilienMhttps://review.openstack.org/504899 is going to break us05:14
jaosoriormostly deleting code05:14
jaosoriorthat's nice05:14
chandankumarEmilienM: /me is the culprit, i am going to fix it05:14
*** psachin has quit IRC05:14
chandankumarEmilienM: sleep tight, see ya tomorrow in your time zone :-)05:14
jaosoriorhave a good one EmilienM05:15
TenguEmilienM: my two reviews - https://review.openstack.org/#/c/515329/1 and https://review.openstack.org/#/c/513274/1105:15
TenguEmilienM: I've launched a recheck for both. again, and again, and again, and again05:15
Tengujaosorior: hello :)05:16
jaosoriorTengu: sup dude05:16
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Move keystone init log container to files logging template  https://review.openstack.org/51559005:16
*** psachin has joined #tripleo05:16
jaosoriorTengu: I feel like we're in a silent competition for who starts working the earliest :P05:16
EmilienMlol05:16
Tengujaosorior: uhuhuh :)05:16
EmilienMI beat you guys05:16
EmilienMI start at 6 :P05:16
EmilienMTengu: ok this is ovb jobs, timeouting05:17
EmilienMsorry for that, we're still investigating05:17
*** dparkes has quit IRC05:17
Tenguhmm. I wake up at 5:30, and go to job at 6, arrival at 6:45. I won't be able to start earlier for now ;)05:17
TenguEmilienM: ah, still that one. oook. Guess it's not easy to pinpoint that kind of issue seeing how many things are involved in the CI. maybe it's "just" a lack of resources?05:18
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Move keystone init log container to files logging template  https://review.openstack.org/51559005:19
EmilienMTengu: ok you beat me.05:19
EmilienMTengu: yeah, bnemec was looking at it05:19
EmilienMtbh I've lost track o nthat05:20
Tengu:)05:20
Tengucan't blame you05:20
*** jaganathan has joined #tripleo05:25
bandinimorning05:25
Tengumorning bandini05:27
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: glance logging: Fix description and use yaml anchor  https://review.openstack.org/51559405:28
Tenguthanks jaosorior for the +2 :)05:28
jaosoriorTengu: no biggie05:28
Tenguoh, btw, I suppose redhat people can't vote for their own review right?05:28
*** achadha has quit IRC05:29
bandinihey jaosorior Tengu05:29
bandiniTengu: nope :)05:29
Tengu:)05:29
*** achadha has joined #tripleo05:29
bandiniI mean you can, but it makes little sense (unless it is to add a -1 to signal some needed changes)05:29
Tenguah, ok.05:30
jaosoriorbandini: sup bandini05:30
*** newmember has joined #tripleo05:31
Tenguanyone can tell me what the 137 error code is?05:32
TenguI suppose it has some specific meaning as the 14305:33
*** achadha has quit IRC05:33
Tengu"testenv-client - ERROR - Failed creating OVB stack"  ooook.05:34
Tengudear OVB: stop that already! :'(05:34
jaosoriorTengu: right, means that it wasn't able to get an environment at all.05:34
jaosoriorMight be for several reasons05:34
jaosoriorthe quota has been filled for CI (so it's not accepting any more jobs)05:34
jaosorioror there might be something wrong with the openstack instance05:34
*** suuuper has joined #tripleo05:36
Tenguso… re-re-recheck «o/05:36
Tenguguess I'll know by the end all the specific error codes :D05:36
EmilienMI would refrain to do recheck until we fix the actual issue05:40
EmilienMdoing a recheck burns our CI resources and prevent code to land05:40
Tenguisn't there a queue ?05:41
Tengui.e. checks are pushed to the queue, and they are consumed one by one according the the ci capacities?05:41
*** yolanda has joined #tripleo05:42
EmilienMyeah there is a qyeye05:46
EmilienMqueue sorry05:46
EmilienMbut if you do recheck, you're increasing the queue size05:46
EmilienMfor ex, if I'm fixing a bug that needs to land asap, because of the large queue it won't land today05:46
EmilienMthat happenned this week05:46
EmilienMwe fixed a bunch of bugs05:46
EmilienMbut everyone was doing recheck and we couldn't merge anything05:46
EmilienMso I blocked the gate by abandoning all patches and myself to select what we want to merge05:47
EmilienMI don' want to do it again05:47
Tenguok.05:47
Tengumight be interresting to get some passthrough way I think, especially for core-maintainers05:47
Tenguso I'll wait next week in order to recheck both patches - one is running now but I think it won't do, the other just failed with 13705:48
*** cshastri has joined #tripleo05:51
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Rename python-zaqar-tests to python-zaqar-tests-tempest  https://review.openstack.org/51560205:52
*** jprovazn has joined #tripleo05:53
*** dsariel has joined #tripleo05:56
*** iranzo has joined #tripleo05:56
*** agurenko has quit IRC05:58
*** yolanda__ has joined #tripleo06:02
*** jfrancoa has joined #tripleo06:02
*** aufi has joined #tripleo06:02
*** ratailor has joined #tripleo06:03
*** jbadiapa has joined #tripleo06:10
*** nguyentrihai has quit IRC06:11
*** anilvenkata has joined #tripleo06:15
*** spectr has joined #tripleo06:22
Tengubingo, failed.06:23
Tenguanyway.06:23
Tenguhave to do some other stuff, I'll let the CI breath ;)06:24
*** ratailor has quit IRC06:24
openstackgerritMerged openstack/tripleo-common stable/pike: Derive Params OvsPmdCoreList compute issue  https://review.openstack.org/51141106:25
jaosoriorsshnaidm|off: around yet?06:25
*** dparkes has joined #tripleo06:26
*** gvrangan has quit IRC06:26
openstackgerritMerged openstack/tripleo-common master: Fix nova_vnc_proxy, ceilometer_agent_central and aodh_evaluator health check  https://review.openstack.org/51347106:29
*** gvrangan has joined #tripleo06:32
*** ratailor has joined #tripleo06:37
*** shreshtha has quit IRC06:44
*** gvrangan has quit IRC06:44
openstackgerritMartin André proposed openstack/tripleo-common stable/pike: Fix nova_vnc_proxy, ceilometer_agent_central and aodh_evaluator health check  https://review.openstack.org/51561206:45
*** ccamacho has quit IRC06:47
*** pgadiya has joined #tripleo06:48
*** jlinkes has joined #tripleo06:50
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: Make kolla builder test not depend on image order  https://review.openstack.org/51561306:56
*** ratailor has quit IRC06:56
*** ratailor has joined #tripleo06:57
openstackgerritMerged openstack/tripleo-heat-templates master: Set metric procssing delay for metricd  https://review.openstack.org/51123607:00
openstackgerritMerged openstack/instack-undercloud stable/pike: Set notification_bind on zaqar websocket  https://review.openstack.org/51529007:00
openstackgerritMerged openstack/tripleo-common master: Remove references to the current task from task() in baremetal.yaml  https://review.openstack.org/51369307:00
*** pblaho has joined #tripleo07:00
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for nova containers to log to stdout/stderr  https://review.openstack.org/50915707:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove log-dir option from neutron-dhcp execution  https://review.openstack.org/51046307:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for Neutron containers to log to stdout/stderr  https://review.openstack.org/51046407:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for nova compute container to log to stdout/stderr  https://review.openstack.org/51053707:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: libvirt: Remove unnecessary binding of /var/log/containers/nova  https://review.openstack.org/51056107:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for nova-libvirt container to log to stdout/stderr  https://review.openstack.org/51056207:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for heat containers to log to stdout/stderr  https://review.openstack.org/51503807:03
jaosoriormandre: just tested it out ^^ added all the "cleanup" tips from the previous commits :D (using anchors, removing the init_logs from the main template and such)07:04
*** rbowen has joined #tripleo07:05
*** ebarrera has joined #tripleo07:05
openstackgerritRajat Sharma proposed openstack/tripleo-ui master: In the Exporting Plan modal, “Download Export” should read “Download”,  “Cancel” should read “Close”  https://review.openstack.org/51561607:05
*** mrch has joined #tripleo07:06
jaosoriordoes anybody know from what puppet module do we configure memcached?07:09
jaosorioroh.. found it. It's https://github.com/saz/puppet-memcached07:10
*** florianf has joined #tripleo07:11
jaosoriorjistr|mtgs, mandre: are you acquainted with our memcached configuration?07:17
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs  https://review.openstack.org/51245007:17
jaosoriorjistr|mtgs, mandre: I think we're not configuring memcached at all for containers :/07:19
mandrejaosorior: all I know about memcached is that we do not configure it to log, https://bugs.launchpad.net/tripleo/+bug/172018307:19
openstackLaunchpad bug 1720183 in tripleo "memcached container logs are missing" [Low,Triaged]07:19
jaosoriormandre: we don't figure it at all as far as I can see07:19
jaosoriormandre: oh wait07:20
*** anilvenkata has quit IRC07:20
jaosoriormandre: we do, but we don't use memcached.conf for some strange reason07:20
mandrejaosorior: the config goes in /etc/sysconfig IIRC07:20
jaosoriormandre: right07:20
*** anilvenkata has joined #tripleo07:21
*** tesseract has joined #tripleo07:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: persist memcached logs in /var/log/containers/memcached/memcached.log  https://review.openstack.org/51561807:23
jaosoriormandre: that should do it ^^07:23
jaosoriordidn't do it on top of my big ass patch chain so we can backport it to pike07:25
mandrejaosorior: if you look at mwhahaha's comment on the bug it seems we do not configure memcached to log at all07:26
jaosoriormandre: this is what's in sysconfig http://paste.openstack.org/show/624786/07:26
jaosoriormandre: it would seem that we actually do07:26
d0ugaljaosorior: Could you take a look at https://review.openstack.org/#/c/515351/?07:26
d0ugaljaosorior: and note EmilienM's comment.07:27
*** yolanda__ has quit IRC07:27
*** anshul has joined #tripleo07:29
jaosoriormandre: oh, right07:30
jaosorioruhm07:30
jaosoriorfunky07:30
mandrejaosorior: the options passed to memcached don't include -v so nothing actually ends up in that file07:31
*** ffiore has joined #tripleo07:31
jaosoriormandre: ok, let me fix that07:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: persist memcached logs in /var/log/containers/memcached/memcached.log  https://review.openstack.org/51561807:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Set verbosity by default for memcached  https://review.openstack.org/51562407:33
jaosoriormandre: done ^^ :D07:33
*** cylopez has joined #tripleo07:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud master: Add one level of verbosity for memcached  https://review.openstack.org/51562707:35
*** hewbrocca_afk is now known as hewbrocca07:35
jaosoriorand undercloud ^^07:35
mandreyay!07:35
*** rbowen has quit IRC07:36
*** pblaho has quit IRC07:37
*** oidgar has joined #tripleo07:38
*** rbowen has joined #tripleo07:38
jaosoriormandre: by the way, were you able to get past the networking errors with the containerized undercloud installation?07:39
openstackgerritJan Provaznik proposed openstack/puppet-tripleo master: Add support for cephfs+ganesha backend in manila  https://review.openstack.org/47124707:41
*** ratailor has quit IRC07:41
*** pblaho has joined #tripleo07:42
mandrejaosorior: I didn't get a chance to try again yet07:43
*** itlinux has quit IRC07:43
d0ugaljaosorior: thanks.07:44
*** jpena|off is now known as jpena07:49
*** amoralej|off is now known as amoralej07:51
jpenaEmilienM, adarazs|rover panda|ruck|off: that looks normal. The job has failed, and it is reporting that failure to the DLRN API07:51
jaosorioradarazs|rover, sshnaidm|off: the ovb jobs seem to be constantly failing. Jobs can't get an OVB environment. is there an outage or something?07:53
lvdombrkrfolks, someone tried implement 2 two-factor authentication in dashboard?07:53
*** rbowen has quit IRC07:53
jaosoriorlvdombrkr: not that I know of07:53
*** ratailor has joined #tripleo07:54
lvdombrkrjaosorior: this only what i found: https://ask.openstack.org/en/question/53047/how-to-implement-2-factor-authentication-in-horizon/07:55
adarazs|roverjaosorior: we did increase the available nodepool nodes on rdocloud yesterday, so we might have some problem. I just got online and that happened when I left so I will have to figure out what's up :)07:56
*** bogdando has joined #tripleo07:57
*** rcernin has quit IRC07:58
*** rbowen has joined #tripleo07:58
*** nguyentrihai has joined #tripleo08:05
*** oidgar has quit IRC08:08
*** ccamacho has joined #tripleo08:10
*** yamahata has quit IRC08:14
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart master: WIP: Use tripleo-upgrade role in CI  https://review.openstack.org/51338208:16
*** paramite has joined #tripleo08:18
jaosoriorlarsks: seems iscsid also logs to syslog. Just to take it into account.08:19
*** lucas-afk is now known as lucasagomes08:26
yolandaEmilienM, mwhahaha , can i get a second +2 on https://review.openstack.org/499588 ?08:27
*** panda|ruck|off is now known as panda|ruck08:29
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564308:29
*** udesale has quit IRC08:35
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common stable/pike: Default DockerCephDaemonImage to stable, not master, tag  https://review.openstack.org/51141808:35
*** iranzo has quit IRC08:36
*** dmacpher has joined #tripleo08:37
*** iranzo has joined #tripleo08:37
*** iranzo has joined #tripleo08:37
jaosoriorbandini: could you check this out https://review.openstack.org/#/c/515594/ https://review.openstack.org/#/c/515590/ ?08:40
*** colonwq has quit IRC08:42
*** fzdarsky_ is now known as fzdarsky08:43
*** derekh has joined #tripleo08:45
*** ykarel is now known as ykarel|lunch08:51
*** pblaho has quit IRC08:51
*** udesale has joined #tripleo08:53
*** pmannidi has quit IRC08:55
bcafareljanki: hi, sfc and friends were dropped from https://review.openstack.org/#/c/512501/ ? (looking at latest patchset)08:58
amoralejhi, i've reported a new bug that is blocking promotions in master https://bugs.launchpad.net/tripleo/+bug/172798508:59
openstackLaunchpad bug 1727985 in tripleo "yum-plugins-update is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]08:59
amoralejwe are thinking in a workaround from RDO side08:59
jankibcafarel, ya. thought to not duplicate the same file in 2 folders. Hope that is fine08:59
*** pdeore has quit IRC08:59
*** shreshtha has joined #tripleo09:00
jankibcafarel, so the env files would be at ../environments/ rather than at ../environments/services-docker/. It also sends out the message that those services are not containarised yet09:00
*** stendulker has joined #tripleo09:01
jankibcafarel, to deploy use -e  ../environments/services-docker/neutron-opendaylight.yaml -e  ../environments/neutron-opendaylight-sfc.yaml09:01
*** salmankhan has joined #tripleo09:02
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564309:02
bcafareljanki: ack if it works fine I am not complaining to only have 1 file :)09:05
jankibcafarel, once those services are containarised, it would make more sense to relocate them09:07
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798509:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugins-update is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]09:10
*** ffiore has quit IRC09:12
*** ffiore_ has joined #tripleo09:12
jaosoriormandre: should I merge https://review.openstack.org/#/c/515612/ ?09:15
*** masco has quit IRC09:15
mandrejaosorior: yes please, the failures are irrelevant to the patch09:16
*** tosky has joined #tripleo09:17
*** ffiore has joined #tripleo09:20
*** ykarel|lunch is now known as ykarel09:20
*** ffiore_ has quit IRC09:20
*** pdeore has joined #tripleo09:22
*** gfidente has joined #tripleo09:26
*** gfidente has quit IRC09:26
*** gfidente has joined #tripleo09:26
mandrejaosorior: I eventually reached the end of your stdout patch series \o/09:36
mandrejaosorior: see my comment on the heat patch, https://review.openstack.org/#/c/515038/... do we need to pass the heat params around?09:37
*** abregman has joined #tripleo09:42
*** links has quit IRC09:42
*** ratailor is now known as rtailor09:45
jaosoriormandre: those come parameter defaults. so we don't need to pass the param around.09:45
jaosoriormandre: thanks a LOT for the reviews! I know it was a lot of work :/09:47
*** dsariel has quit IRC09:47
*** fragatina has joined #tripleo09:48
*** abregman is now known as abregman|afk09:49
mandrejaosorior: ah ok, thanks for the clarification09:52
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564310:04
*** psachin has quit IRC10:05
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798510:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]10:10
*** ooolpbot has quit IRC10:10
jaosoriorbrb, lunch10:11
*** dsariel has joined #tripleo10:11
*** daidv has quit IRC10:12
rtailorCan anyone point out, why I get timed out while deploying overcloud with oooq ?10:17
rtailorI have setup FreeIPA setup for enabling TLS, and nodes 1ctrl_1comp_1supp node config for deployment.10:18
rtailorWhen I apply my custom changes and enable ec2api in overcloud-deploy.sh and try to deploy overcloud again after deleting the previous stack. I get timeout as mentioned below.10:20
rtailor2017-10-27 07:13:29Z [overcloud.AllNodesDeploySteps.ControllerDeployment_Step4]: CREATE_COMPLETE  state changed10:20
rtailor2017-10-27 09:26:20Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED  CREATE aborted10:20
rtailor2017-10-27 09:26:20Z [overcloud]: CREATE_FAILED  Create timed out10:20
rtailor2017-10-27 09:26:20Z [overcloud.AllNodesDeploySteps.ComputeDeployment_Step4]: CREATE_FAILED  CREATE aborted10:20
rtailor2017-10-27 09:26:20Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED  Resource CREATE failed: Operation cancelled10:20
*** Goneri has joined #tripleo10:20
rtailorMy changes are here.10:21
rtailorhttps://review.openstack.org/#/c/509392/10:21
rtailorhttps://review.openstack.org/#/c/509393/10:21
rtailorhttps://review.openstack.org/#/c/511772/10:21
rtailorhttps://review.openstack.org/#/c/511773/10:21
rtailorjaosorior, shardy ^^10:21
rtailorany help ?10:21
shardyrtailor: can you see in /var/log/messages on the node why it's getting stuck on Step 4?10:23
shardymost likely there's a puppet apply hanging, which you can kill then re-run with --debug to see where it's getting stuck10:23
shardyps ax | grep puppet should show it if that is the issue10:23
shardyyou can kill the pid then re-run puppet apply with --debug added10:24
rtailorshardy, let me check it and will try that.10:25
*** gkadam has quit IRC10:34
*** jaganathan has quit IRC10:37
rookhey shardy - did I ever send you the heat logs -- which had convergence enabled?10:41
rookand I couldn't scale past the initial deployment.10:41
shardyrook: Hey no I don't think I saw them10:41
shardyit'd be good to see the also nova/neutron logs as IIRC those were the services that actually errored?10:41
rookoh, i think i see why10:42
*** pblaho has joined #tripleo10:44
*** gkadam has joined #tripleo10:47
*** rtailor has quit IRC10:49
*** ansmith has quit IRC10:52
*** jprovazn has quit IRC10:53
*** Goneri has quit IRC10:53
*** Goneri has joined #tripleo10:55
*** rtailor has joined #tripleo11:02
*** lucasagomes is now known as lucas-hungry11:07
*** dsariel has quit IRC11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798511:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]11:10
*** pkovar has joined #tripleo11:10
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates stable/pike: Set metric procssing delay for metricd  https://review.openstack.org/51568411:12
*** rtailor has quit IRC11:15
openstackgerritMerged openstack/tripleo-heat-templates master: Update CephPools format in the docker templates to fit ceph-ansible  https://review.openstack.org/50885911:15
*** dparkes has quit IRC11:16
*** rbowen has quit IRC11:17
*** bogdando has quit IRC11:18
*** bogdando has joined #tripleo11:19
*** salmankhan has quit IRC11:25
*** atoth_ has joined #tripleo11:30
*** Goneri has quit IRC11:30
*** hamzy has joined #tripleo11:30
therveAnyone knows what that failure means: http://logs.openstack.org/71/506071/4/check/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq-container/4ace0c8/job-output.txt.gz#_2017-10-27_08_24_50_876641 ?11:31
therve"File contains no section headers" ?!11:31
*** atoth has quit IRC11:33
* adarazs|rover looks11:33
*** rtailor has joined #tripleo11:34
*** ffiore has quit IRC11:34
adarazs|roverhmm, that's interesting.11:35
*** Goneri has joined #tripleo11:36
adarazs|roverpanda|ruck, amoralej: have you seen this error before? here's the full log from repo-setup: http://logs.openstack.org/71/506071/4/check/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq-container/4ace0c8/logs/subnode-2/home/zuul/repo_setup.log.txt.gz11:36
*** mburned is now known as mburned_out11:36
adarazs|rovernot much more enlightening.11:36
*** colonwq has joined #tripleo11:37
*** rtailor has quit IRC11:37
*** ratailor has joined #tripleo11:37
therveadarazs|rover, http://logs.openstack.org/71/506071/4/check/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq-container/4ace0c8/logs/subnode-2/etc/yum.repos.d/delorean-current.repo.txt.gz is weird11:38
amoralejadarazs|rover, nop11:38
amoralejbut that's the first command in the repo-setup11:38
amoralejit looks like there is an incorrect delorean-current.trepo file before running repo-setup11:38
*** akrivoka has joined #tripleo11:39
amoralejyah11:39
amoralejin subnode2 repos are configured in node preparation phase11:39
amoralejlet me check11:39
amoralejadarazs|rover, look at http://logs.openstack.org/71/506071/4/check/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq-container/4ace0c8/logs/undercloud/var/log/bootstrap-subnodes.log.txt.gz11:40
amoralejYou don't have permission to access /centos7/current/delorean.repo11:41
amoralejmmmm11:41
adarazs|roveryeah. o_O11:41
*** mburned_out is now known as mburned11:42
therveamoralej, A race condition when the "current" repo is updated?11:43
ratailorshardy, I have checked compute /var/log/messages and getting this error ==>[/usr/lib/systemd/system/ip6tables.service:3] Failed to add dependency on syslog.target,iptables.service, ignoring: Invalid argument11:44
panda|ruckknock knock11:44
panda|ruckthe race condition11:44
panda|ruckwho's there ?11:44
ratailorshardy, I think, the solution provided here should work https://bugzilla.redhat.com/show_bug.cgi?id=150504311:44
openstackbugzilla.redhat.com bug 1505043 in iptables "[/usr/lib/systemd/system/ip6tables.service:3] Failed to add dependency on syslog.target,iptables.service, ignoring: Invalid argument" [Medium,Closed: duplicate] - Assigned to psutter11:44
amoralejtherve, yeah something like that11:45
adarazs|roverheh11:45
amoralejwe've had 4 403 on centos7/current/delorean.repo in the last 4 days11:45
amoralejjpena, ^11:46
*** abishop has joined #tripleo11:47
therveThe script should fail so that the error is a bit more obvious11:47
adarazs|roveryep11:49
*** anilvenkata has quit IRC11:51
*** rbowen has joined #tripleo11:53
*** dparkes has joined #tripleo11:55
openstackgerritThomas Herve proposed openstack-infra/tripleo-ci master: Fail on curl error when setting up repo  https://review.openstack.org/51569511:57
therveadarazs|rover, amoralej  ^^ Not that it fixes the issue, but at least it may be a bit more clear11:57
amoralejtherve, adarazs|rover it may be a race related to the sync from trunk-primary.r.o to trunk.r.o, jpena is investigating11:57
therveYeah that would make sense11:57
*** jpena is now known as jpena|lunch11:58
*** dtantsur|afk is now known as dtantsur11:58
*** ratailor has quit IRC11:58
*** ratailor has joined #tripleo11:58
adarazs|roverpanda|ruck: ^12:00
*** athomas has joined #tripleo12:01
panda|ruck4 in 3 days is not that much, but every single error at the moment is making us lose 4 hours at least, and we are craving for promotions12:02
*** anilvenkata has joined #tripleo12:04
*** liverpooler has joined #tripleo12:05
*** dprince has joined #tripleo12:05
*** stendulker has quit IRC12:06
*** raildo has joined #tripleo12:07
chandankumarHello Tripleo, Tempest-17.1.0 has just landed for Pike, please let us know if you find any breakage12:07
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798512:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]12:10
*** ooolpbot has quit IRC12:10
*** pdeore has quit IRC12:11
*** salmankhan has joined #tripleo12:13
*** ffiore has joined #tripleo12:13
*** lucas-hungry is now known as lucasagomes12:14
*** salmankhan has quit IRC12:17
*** ratailor has quit IRC12:17
*** trown|outtypewww is now known as trown12:18
*** morazi has joined #tripleo12:24
*** rhallisey has quit IRC12:25
*** pgadiya has quit IRC12:28
*** StevenK has quit IRC12:29
Tenguhello! small question: I activated the cephfsnative option using environments/manila-cephfsnative-config.yaml - but apparently, it's either not sufficient, or it requires some variables edition. I've spoted the following error in the manila/scheduler.log: https://gist.github.com/cjeanneret/4b391db3c3bdedf8671e43b6fb3eb61812:29
Tengucare to either send me to the right people or help me a bit with that? :)12:29
Tengu(yes, I know, Friday afternoon, hard time ;))12:29
*** jcoufal has joined #tripleo12:30
tbarronTengu: that's a very small pastebin12:30
tbarronTengu: that doesn't have the error12:31
Tenguwell, yes12:31
Tenguactually, first line, and following. that's what I get when I want to create a share with, for example, manila create --share-type cephfsnativetype --name cjt-test cephfs 112:32
Tenguapparently, it doesn't find a "host"12:32
*** rlandy has joined #tripleo12:32
Tenguhmm, actually, I can paste it on paste.openstack, the page layout might be better12:32
tbarronTengu: so you need to look at the manila share service log and see why it is down12:32
Tenguor you can get https://gist.githubusercontent.com/cjeanneret/4b391db3c3bdedf8671e43b6fb3eb618/raw/cb0527b0131145d37e1cfa23ef3bd53dde789d81/CephFS%2520native%2520error12:32
tbarronTengu: apologies for not scrolling far enough right on the first line12:33
tbarronTengu: but still, the manila share service log should have a clue12:34
Tengutbarron: no problem, the display wasn't convenient, my fault as well.12:34
Tenguhmm.12:34
tbarronTengu: note that it runs active-passive, so if you have multiple controllers get the right one12:34
Tenguweird: tail -f doesn't show any move in that log. but I probably need to open the three of them as we're in a three-controller setup.12:34
Tengu:) bingo12:34
* Tengu invoke tmux powers12:34
Tenguhmm.12:36
Tengutbarron: so it's normal I get only one "share.log"? apparently it's not present on the two other controllers.12:36
Tenguis it what you intended to say with the "active-passive" thing?12:37
*** gkadam has quit IRC12:37
Tenguah. doing a `grep -i error' in that log shows up many things. among them, python tracebacks.12:38
*** abregman|afk is now known as abregman12:38
*** toure_biab is now known as toure12:38
tbarronTengu: yes, that's normal (one share log, not the tracebacks) look for the early tracebacks after the service started12:39
Tenguyup, digging12:39
tbarronTengu: note that this configuration requires that cephfs be set up, including ceph mds service, etc.12:40
Tengutbarron: http://paste.openstack.org/show/624823/12:40
Tengutbarron: we actually have a ceph, managed by tripleO12:41
Tenguand it seems to be working as expected12:41
*** StevenK has joined #tripleo12:41
Tenguall is integrated with tripleo - the only external thing is our freeIPA for user authentication12:41
tbarronTengu: do you have cephfs?  MDS service?  or just ceph mons and osds12:41
Tengutbarron: good question :/. wait, reviewing the included env files for the deploy.12:42
*** abregman is now known as abregman|afk12:42
Tengutbarron: I think you pointed out the issue. apparently ./openstack-tripleo-heat-templates/environments/storage/enable-ceph.yaml includes only mon and osds12:42
tbarronTengu: also check ceph health and 'systemctl status'12:42
Tengunow I have to find out how to enable cephfs at ceph level I guess12:43
tbarronTengu: right, we may have some doc gaps, I have a meetinig for 15 min but let met check down the doc references that we have and see if they are sufficient12:43
*** ecerquei has joined #tripleo12:43
tbarronTengu: would you send me a mail at 'tbarron@redhat.com' and I'll reply with doc references?12:44
*** dsariel has joined #tripleo12:45
Tengutbarron: doing that, thanks a lot!12:45
*** jpena|lunch is now known as jpena12:50
*** pchavva has joined #tripleo12:50
*** dsariel has quit IRC12:51
*** lblanchard has joined #tripleo12:51
*** dsariel has joined #tripleo12:55
*** amoralej is now known as amoralej|lunch12:55
*** ansmith has joined #tripleo12:57
*** ansiwen has quit IRC12:58
Tengutbarron: ah, it's maybe with https://docs.openstack.org/manila/pike/contributor/cephfs_driver.html#authorizing-the-driver-to-communicate-with-ceph - although I thought that part would be done with the deploy, in fact :/13:01
*** pblaho has quit IRC13:01
*** pblaho has joined #tripleo13:02
Tenguok, apparently the keyring is present. next step.13:02
*** dmarlin has joined #tripleo13:03
*** shreshtha has quit IRC13:04
*** catintheroof has joined #tripleo13:04
*** shardy has quit IRC13:07
Tenguhmm. weird. apparently some options in manila.conf are missing, if we follow the mentioned doc…13:08
*** udesale has quit IRC13:08
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798513:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172803313:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]13:10
openstackLaunchpad bug 1728033 in tripleo "testenv-client is failing with error "Couldn't retrieve env" on rdo cloud jobs" [Critical,Triaged]13:10
tbarronTengu: my meeting went long and now I have another coming up but I got your email and will reply later today (it's morning for me). If13:12
tbarronTengu: you can send me the actual overcloud deploy command you used (for manila and for cephfs) that will be a help.13:12
Tengutbarron: ah, it's the afternoon for me ;). thanks for the hint. I'm digging in the doc I found.13:12
Tenguyup13:12
*** lblanchard has quit IRC13:16
*** anilvenkata_ has joined #tripleo13:18
*** tzumainn has joined #tripleo13:19
*** cdearborn has joined #tripleo13:20
Tengutbarron: you have almost everything in your mailbox :).13:21
*** ansiwen has joined #tripleo13:21
tbarronTengu: you don't know how true that statement is ^^^ but ack I see your latest.13:22
Tengu:)13:22
*** anilvenkata has quit IRC13:22
*** dsariel has quit IRC13:23
*** skramaja has quit IRC13:24
openstackgerritOpenStack Proposal Bot proposed openstack/instack-undercloud stable/pike: Updated from global requirements  https://review.openstack.org/51570613:28
*** jmelvin has joined #tripleo13:28
*** artom has quit IRC13:28
*** artom has joined #tripleo13:28
*** jlinkes_ has joined #tripleo13:28
dtantsurd0ugal: hey! is there a simple way to make a workflow retry itself several times on failure?13:29
dtantsurd0ugal: context: https://bugzilla.redhat.com/show_bug.cgi?id=1506038#c313:29
openstackbugzilla.redhat.com bug 1506038 in openstack-tripleo-common "openstack-ironic: errors: Only 14 nodes are exposed to Nova of 15 requests." [Unspecified,Assigned] - Assigned to dtantsur13:29
*** rbowen has quit IRC13:30
d0ugaldtantsur: it is easy to have it retry from the other workflow that calls it - is that enough?13:30
d0ugalor is it ever called directly?13:30
*** dansmith is now known as superdan13:30
d0ugalhttps://github.com/openstack/tripleo-common/blob/6d3ce6329db353e848e453656798d1d1c0c5d387/workbooks/validations.yaml#L735-L73613:30
dtantsurd0ugal: it's called directly. I can probably rename it and create a wrapper though13:30
d0ugalah13:30
d0ugaldtantsur: yeah, that is probably the easiest approach then. Basically you can easily retry a task13:30
dtantsuroh, wait, maybe you're right13:30
dtantsurI don't know if it's called directly13:31
*** jlinkes has quit IRC13:31
thrashdtantsur: https://github.com/openstack/tripleo-common/blob/6d3ce6329db353e848e453656798d1d1c0c5d387/workbooks/validations.yaml#L73613:31
d0ugaldtantsur: https://github.com/openstack/mistral/blob/f2ed583540cd3429df975a368ee5be55af0eb537/mistral/tests/unit/engine/test_policies.py#L52-L5513:31
dtantsurd0ugal: anyway, how do I do it? do you have a reference handy?13:31
dtantsurthrash: was about to ping you :) thanks13:31
thrashdtantsur: that's where it's called. you can retry it there.13:31
dtantsurnice! thanks thrash, d0ugal13:32
d0ugaldtantsur: oh, it is actually documented too: https://docs.openstack.org/mistral/latest/main_features.html#yaml-example13:32
tbarrondo we have a list of containers that need to run privileged and why?  in master? in pike?13:32
openstackgerritDan Prince proposed openstack/python-tripleoclient master: Support to drive undercloud deploy via undercloud.conf  https://review.openstack.org/51135013:33
*** salmankhan has joined #tripleo13:33
tbarrondprince: ^^^ is this the right place to ask that question? ^^^13:33
*** catintheroof has quit IRC13:34
dtantsurd0ugal: do I get it right that I don't need break-on or continue-on? will it understand that it has to retry on errors and continue on success?13:34
d0ugaldtantsur: yes, I believe that is correct13:34
dprincetbarron: you could grep for 'privaleged' in t-h-t13:34
dtantsursweet13:34
tbarrondprince: sure, that'll get me started13:35
d0ugaldtantsur: on all the zaqar posts we only use count and delay. https://github.com/openstack/tripleo-common/blob/25750c9b920a71f3c5627097787015e0caa9c32e/workbooks/baremetal.yaml#L9013:35
dtantsurdprince: hey man, have you noticed that the whole nature, as well as the gate gods, are against including ironic-inspector docker templates in THT? :D13:35
dtantsurI don't remember a patch that failed so many times in the CI13:35
*** Goneri has quit IRC13:35
dtantsurd0ugal: good, thanks again!13:35
tbarrondprince: as we containerize nfs-ganesha it looks like we have to run privileged atm because it interacts with host dbus13:36
tbarrondprince: so we're looking for anyone who has solved that issue, short of using dbus policy that just lets anyone mess with the service13:36
*** janki has quit IRC13:37
dprincedtantsur: I have. and it kind of makes me sad as I have tried so hard to get this in :/13:37
dtantsurheh, yeah13:37
dtantsurI don't get it, the patch itself cannot affect the CI..13:38
tbarrondprince: i guess it's permissible to run privileged for pike and queens but we want to eliminate doing this "real soon now"?13:38
dtantsurbad karma of mkovacik13:38
jaosoriorjistr|mtgs, bogdando: could you check this otu https://review.openstack.org/#/c/509157/24 ?13:39
*** Goneri has joined #tripleo13:41
*** catintheroof has joined #tripleo13:42
EmilienMhi13:43
Tenguwelcome back EmilienM :)13:43
EmilienMadarazs|rover: what's up13:43
EmilienMTengu: thanks13:43
TenguEmilienM: small question: how's the CI load during the weekend?13:43
adarazs|roverEmilienM: so... there's something wrong with the te-broker on rdocloud.13:43
Tengui.e. may I recheck my stuff tomorrow without impacting the "prod" pipeline?13:44
EmilienMTengu: low13:44
adarazs|roverEmilienM: https://bugs.launchpad.net/tripleo/+bug/1728033 -- not sure if panda|ruck managed to fix it yet or not.13:44
openstackLaunchpad bug 1728033 in tripleo "testenv-client is failing with error "Couldn't retrieve env" on rdo cloud jobs" [Critical,Triaged]13:44
EmilienMadarazs|rover: how did that happen?13:44
panda|ruckupdating13:44
TenguEmilienM: cool, so I'll try to recheck my stuff tomorrow or on Sunday then :)13:44
adarazs|roverEmilienM: one of the compute nodes on RDO cloud went down yesterday evening. could have been caused by that.13:45
EmilienMah13:45
EmilienMadarazs|rover: ok so nothing I can help for now, right?13:45
adarazs|roverEmilienM: in other news, the 0800UTC jobs didn't trigger so we shuffled around the trigger times a bit to allow a bit more time for the runs in the busy periods: https://review.rdoproject.org/r/1031513:45
adarazs|roverEmilienM: that means we will have another run in 15 minutes.13:45
adarazs|roverEmilienM: nope, it's again just a few unforseen infra errors knocking us off before a promotion. -_-13:46
panda|ruckupdated13:47
jaosoriorbogdando: if you have some time, could you check these out as well https://review.openstack.org/#/c/510463/21 https://review.openstack.org/#/c/510464/22 ?13:47
adarazs|roverpanda|ruck: cool, thanks!13:48
adarazs|roverEmilienM, panda|ruck: so OVB jobs should be back in business for the next run soon.13:48
bogdandojaosorior: will do13:48
bogdandojaosorior: we really need to test all services on a box13:48
bogdandoI doubt docker will make it to run so much side cars for each wsgi service13:49
panda|ruckadarazs|rover: EmilienM if nothing else happened in the meantime ... :(13:49
*** spectr has quit IRC13:49
jaosoriorbogdando: right. I'm rebuilding the whole environment right now. But so far with the containers that I've been testing (basically the standard configuration with telemetry) it seems alright. Maybe as we include more services we start seeing issues? I'm not sure.13:51
EmilienMadarazs|rover, panda|ruck: ok, thx13:51
EmilienMadarazs|rover, panda|ruck: just a pure fyi - I'm afk all weekend (and really afk)13:51
*** morazi has quit IRC13:52
EmilienMyolanda: ack13:53
panda|ruckEmilienM: you're never really afk.13:54
panda|ruckEmilienM: you replied me once while skiing. You're anseer is invalid.13:55
EmilienMpanda|ruck: well, this time I'll be. We go 2 days in wild nature without network13:55
EmilienMahah lol13:55
panda|ruckyour*13:55
EmilienMyolanda: hey, is there a quick way to test https://review.openstack.org/#/c/499588/5/elements/overcloud-secure/block-device-default.yaml before merging?13:55
panda|ruckdammit13:55
*** owalsh has quit IRC13:56
*** amoralej|lunch is now known as amoralej13:56
yolandaEmilienM, i tested on my side. An image needs to be built and then, it needs to be used for a TripleO deploy13:56
*** arxcruz is now known as arxcruz|pto13:56
EmilienMyolanda: ok, approved13:56
yolandait will not affect the standard tripleo deploy, as this image needs to be generated on demand13:57
EmilienMadarazs|rover: do we actually need the alert on https://bugs.launchpad.net/tripleo/+bug/1728033 ?13:58
openstackLaunchpad bug 1728033 in tripleo "testenv-client is failing with error "Couldn't retrieve env" on rdo cloud jobs" [Critical,Triaged]13:58
adarazs|roverEmilienM: hehe, enjoy your trip :)13:58
EmilienMadarazs|rover: it sounds like something on RDO Cloud side, nothing we can really do here13:58
adarazs|roverEmilienM: well, not anymore. I think panda|ruck fixed it.13:58
panda|ruckadarazs|rover: shoosh, don't let the servers hear you.13:59
adarazs|rover:I13:59
* adarazs|rover zips his mouth.13:59
*** owalsh has joined #tripleo13:59
EmilienMI'm removing alert13:59
adarazs|roverEmilienM: ack.14:00
adarazs|roverpanda|ruck: the new pipeline triggered at least :)14:00
EmilienMalso assigning to panda|ruck14:00
EmilienMsince he worked on it14:00
bnemecEmilienM: https://bugs.launchpad.net/tripleo/+bug/172805814:00
openstackLaunchpad bug 1728058 in tripleo "Nova DB archiving not working in rh1" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)14:00
bnemec^Something I noticed last night when I was cleaing up after the zuul restart.14:00
bnemecProbably has a lot to do with why ovb jobs are slow in rh1.14:00
EmilienMbnemec: can we ssh/tmate an ddebug together?14:02
yolandathx EmilienM14:02
EmilienMbnemec: or you fixed it?14:02
bnemecEmilienM: I'm pretty sure I know the problem, I just need to do some db hackery to fix it.14:02
EmilienMbah, ok14:03
*** Dinesh_Bhor has quit IRC14:04
*** d0ugal_ has joined #tripleo14:04
*** d0ugal has quit IRC14:05
fultonjcan DockerInsecureRegistryAddress: be passed a list of insecure registries?14:05
fultonjmandre: ^ ?14:06
*** tosky has quit IRC14:06
*** bogdando has quit IRC14:07
*** bogdando has joined #tripleo14:07
*** cylopez has quit IRC14:07
*** tosky has joined #tripleo14:08
*** spectr has joined #tripleo14:09
adarazs|roverpanda|ruck: sooo, OVB jobs are dead again :( could you take a look at the te-broker again?14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798514:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]14:10
dprincetbarron: you might could look at the nova services. They had some issues with PAM modules to resolve, owalsh did those patches. I don't think there were dbus issues there though14:12
*** amodi has joined #tripleo14:12
tbarrondprince: i heard that in k8s there were dbus issues with iscsiadm in container and iscsid on host14:13
tbarrons/were/are/14:13
dprincetbarron: we run iscsid with 'privileged: true' fwiw.14:14
dprincetbarron: it functionally works for me. was able to use it w/ Ironic for example14:15
tbarrondprince: ack, but we'll want to figure a way to do it unprivileged in the future, right?14:15
*** ffiore has quit IRC14:15
dprincetbarron: if possible yes. Unclear that for some of the services this would be an option though14:16
*** gfidente has quit IRC14:16
tbarrondprince: i have the impression that if we want to run services under k8s then this will be a stronger requirement?  or no?14:17
*** ratailor has joined #tripleo14:17
*** dsariel has joined #tripleo14:17
dprincetbarron: perhaps it is desired... depending on how fast we want to move to k8s we may have to adjust our desires a bit though :)14:18
tbarrondprince: ack, thanks14:18
panda|ruckadarazs|rover: starting mirror-server again. it's useed as proxy too14:19
adarazs|roverpanda|ruck: damn :/14:20
*** gfidente has joined #tripleo14:20
*** leitan has joined #tripleo14:20
adarazs|roverpanda|ruck: we have some problem with the pike promoter script btw: http://38.145.33.13/pike.log14:20
adarazs|roverit doesn't seem to check stuff.14:20
adarazs|rover2017-10-27 14:02:01,772 27478 INFO     promoter Trying to promote tripleo-ci-testing to current-tripleo 2017-10-27 14:12:01,212 27569 INFO     promoter STARTED promotion process with config /home/centos/ci-config/ci-scripts/dlrnapi_promoter/config/pike.ini14:21
bnemecadarazs|rover: It's not the te-broker: https://bugs.launchpad.net/tripleo/+bug/172805814:22
openstackLaunchpad bug 1728058 in tripleo "Nova DB archiving not working in rh1" [Critical,Triaged] - Assigned to Ben Nemec (bnemec)14:22
owalshdprince: had issues with libvirt + polkit, which used dbus to communicate, but dbus wasn't the problem14:22
adarazs|roverbnemec: that is something else, we're having problems on rdocloud with the periodic jobs: https://bugs.launchpad.net/tripleo/+bug/172803314:22
openstackLaunchpad bug 1728033 in tripleo "testenv-client is failing with error "Couldn't retrieve env" on rdo cloud jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami)14:22
bnemecadarazs|rover: Ah, okay.14:23
*** suuuper has quit IRC14:24
*** jidar has quit IRC14:29
*** ffiore has joined #tripleo14:30
dprincetbarron: see comment from owalsh above. Thanks owalsh14:31
*** dsariel has quit IRC14:31
tbarrondprince: owalsh thanks14:31
*** jidar has joined #tripleo14:32
*** dsariel has joined #tripleo14:33
*** dbecker has quit IRC14:35
*** dbecker has joined #tripleo14:35
*** bogdando has quit IRC14:36
ratailorowalsh, you around ?14:37
owalshratailor: yea14:37
ratailorowalsh, I was trying to deploy TLS setup using oooq with my custom changes for ec2api.14:38
ratailorowalsh, my overcloud deployment failed with timedout, after increasing the timeout also it gets failed.14:38
ratailorowalsh, I debugged into /var/log/messages on compute node and found that it fails with this error.14:39
ratailorOct 27 12:25:09 localhost systemd: [/usr/lib/systemd/system/ip6tables.service:3] Failed to add dependency on syslog.target,iptables.service, ignoring: Invalid argument14:39
ratailorOct 27 12:25:09 localhost systemd: Starting OpenStack Nova Compute Server...14:39
ratailorOct 27 12:25:14 localhost libvirtd: 2017-10-27 12:25:14.989+0000: 20037: info : libvirt version: 3.2.0, package: 14.el7_4.3 (CentOS BuildSystem <http://bugs.centos.org>, 2017-09-07-11:27:44, c1bm.rdu2.centos.org)14:39
ratailorOct 27 12:25:14 localhost libvirtd: 2017-10-27 12:25:14.989+0000: 20037: info : hostname: overcloud-novacompute-0.tripleodomain.example.com14:39
ratailorOct 27 12:25:14 localhost libvirtd: 2017-10-27 12:25:14.989+0000: 20037: error : virNetSocketReadWire:1808 : End of file while reading data: Input/output error14:39
ratailorowalsh, ^^14:39
*** spectr has quit IRC14:39
*** dbecker has quit IRC14:40
*** dbecker has joined #tripleo14:40
ratailorowalsh, there is already a BZ reported for this https://bugzilla.redhat.com/show_bug.cgi?id=150504314:40
openstackbugzilla.redhat.com bug 1505043 in iptables "[/usr/lib/systemd/system/ip6tables.service:3] Failed to add dependency on syslog.target,iptables.service, ignoring: Invalid argument" [Medium,Closed: duplicate] - Assigned to psutter14:40
*** bogdando has joined #tripleo14:41
owalshratailor: this is with containers?14:41
ratailorwithout containers.14:41
owalshratailor: ok. looks alright to me14:41
ratailorowalsh, Then why the overcloud fails with timeout every time.14:42
owalshratailor: could be an actual timeout if the hardware is slow14:43
ratailorI have already increased the timeout to 150 in overcloud-deploy.sh14:43
owalshratailor: then need to figure out which heat resources were still in progress14:44
ratailorThe suspect this is the culprit because the timing of messages with timeout matches the timing of these log messages in /var/log/messages.14:44
openstackgerritOpenStack Proposal Bot proposed openstack/instack-undercloud stable/pike: Updated from global requirements  https://review.openstack.org/51570614:45
*** adarazs|rover is now known as adarazs|rover|af14:46
*** adarazs|rover|af is now known as adarazs_afk14:47
ratailorowalsh, its overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.014:49
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient master: Support to drive undercloud deploy via undercloud.conf  https://review.openstack.org/51135014:49
jaosoriorpradk: is this acurate https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/panko-api.yaml#L5 ?14:50
*** abregman_ has joined #tripleo14:50
owalshratailor: what time did that step start?14:51
pradkjaosorior, yes, but it still remains enabled as of now14:51
pradkwe dont have a replacement yet14:51
pradkbut its deprecated14:51
jaosoriorpradk: what's the reason for deprecating panko without replacement?14:51
pradkno one to support it upstream14:51
jaosoriorpradk: thought it would be the telemetry team14:52
jaosorioroh, wait, usptream, right14:52
jaosoriorpradk: so, no community?14:52
jaosoriorpradk: so... what do folks use in production then?14:52
pradkjaosorior, right, its meant to die eventually.. but evidently we still have people using it14:52
ratailorowalsh, 2017-10-27 12:24:17Z [overcloud.AllNodesDeploySteps.ComputeDeployment_Step4]: CREATE_IN_PROGRESS  Stack CREATE started14:53
ratailor2017-10-27 12:24:17Z [overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.0]: CREATE_IN_PROGRESS  state changed14:53
pradkjaosorior, we cant kill it until people stop using it.. and the way to do it is to announce its deprecated :)14:53
ratailorowalsh, it failed at 2017-10-27 14:32:50Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED  CREATE aborted14:54
ratailor2017-10-27 14:32:50Z [overcloud]: CREATE_FAILED  Create timed out14:54
ratailor2017-10-27 14:32:51Z [overcloud.AllNodesDeploySteps.ComputeDeployment_Step4]: CREATE_FAILED  CREATE aborted14:54
ratailor2017-10-27 14:32:51Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED  Resource CREATE failed: Operation cancelled14:54
ratailor2017-10-27 14:32:51Z [overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.0]: CREATE_FAILED  CREATE aborted14:54
*** abregman|afk has quit IRC14:54
owalshratailor: anything in /var/log/nova/nova-compute.log?14:54
*** bobh has joined #tripleo14:55
fultonjis it possible to have my deployment use >1 insecurity docker registry by passing a list to  DockerInsecureRegistryAddress ?14:56
fultonjdprince ^ ?14:57
*** toure is now known as toure_biab14:57
ratailorowalsh, 2017-10-27 12:25:13.496 20172 ERROR oslo.messaging._drivers.impl_rabbit [req-07abaece-547c-4ee7-b2d0-27b63b0ba19c - - - - -] [ca254f1e-8fc8-46d5-a0ea-1dfe60bcbe26] AMQP server on overcloud-controller-0.internalapi.tripleodomain.example.com:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. Client port: None: error: [Errno 111] ECONNREFUSED14:58
*** pblaho has quit IRC14:58
ratailorowalsh, its rabbitmq issue.14:58
*** bobh has quit IRC14:58
dprincefultonj: yes, that is an array14:59
fultonjdprince: thanks!14:59
dprincefultonj: well, a comma_delimited_list in Heat.14:59
*** toure_biab is now known as toure15:00
dprincefultonj: also, there is a related patch from stevebaker here https://review.openstack.org/#/c/514473/15:00
dprincefultonj: where we are trying to autodetect this for you15:00
*** anilvenkata_ has quit IRC15:00
fultonjnice15:00
*** abregman_ has quit IRC15:02
*** bogdando has quit IRC15:05
owalshratailor: ok, is rabbitmq down or something else (e.g network)15:05
openstackgerritMerged openstack/os-net-config master: Allow dns_servers to be an empty array  https://review.openstack.org/51020715:05
*** bogdando has joined #tripleo15:05
*** iranzo has quit IRC15:06
*** chlong has quit IRC15:06
*** mdnadeem has quit IRC15:06
*** thrash is now known as thrash|biab15:06
*** dsariel has quit IRC15:07
*** ebarrera has quit IRC15:07
ratailorowalsh, rabbitmq is down15:07
*** cylopez has joined #tripleo15:08
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564315:09
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798515:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172807015:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]15:10
openstackLaunchpad bug 1728070 in tripleo "CI Promotion. Pike and master periodic jobs are failing" [Critical,In progress]15:10
owalshratailor: figured out why?15:10
*** mdnadeem has joined #tripleo15:10
ratailorowalsh, didn't get enough time. Just got to know its rabbitmq.15:11
*** tosky has quit IRC15:12
*** rbowen has joined #tripleo15:13
*** yamahata has joined #tripleo15:13
jaosoriormwhahaha: could you check out these ones https://review.openstack.org/#/q/topic:bug/1720183+status:open ?15:15
*** bogdando_ has joined #tripleo15:15
*** newmember has quit IRC15:15
EmilienMjaosorior: he's on PTO all wekk15:17
EmilienMI'm looking at it now15:17
panda|ruckadarazs_afk: the proxy server is up, running and replying to requests.15:17
panda|ruckadarazs_afk: at this point I think it was a transient problem on the rax mirror15:18
panda|ruckbecause all other packages were installed15:18
EmilienMpanda|ruck: why don't we run our own mirror?15:18
*** links has joined #tripleo15:19
pabelangerright15:20
panda|ruckEmilienM: AFAIK it's planned, but for now we rely on a mirror from infra15:20
panda|ruckor at least in discussion15:20
EmilienMand it's not working15:20
EmilienM:(15:20
EmilienMusing a local mirror should be critical priority15:20
panda|ruckit worked until yesteday ...15:20
EmilienMit has proved to be very useful in tripleo ci15:20
panda|ruckand for all pother requests on the same job worked15:20
EmilienMmirrors helped us to reduce the failures, a lot15:20
pabelangerRDO should stand up a local mirror using openstack-infra/system-config, would be straightforward to do, but currently is a show stopper because it ubuntu based15:20
*** bogdando_ has quit IRC15:21
pabelangerso, somebody needs to re-write all the manifests to work on centos15:21
EmilienMouch15:21
panda|ruck/o\15:21
EmilienMok, it sounds a lot of work15:21
pabelangerI don't see it being a big deal honestly15:21
pabelangermirror.regionone.tripleo-test-cloud-rh1.openstack.org is ubuntu15:21
panda|ruckanyway, I'm not sure the uptime would be more than what we have now ...15:21
pabelangerEmilienM: yah, it would be less then 30mins to stand up new mirror in rdo cloud, by consuming openstack-infra/system-config15:22
pabelangernow, you'll need to add the mirror to sprint and have somebody replicate it15:22
pabelangerYou even stand up our mirror now, stop depending on rackspace, then work on porting to centos and push back upstream15:23
pabelangerthat's best patch forword IMO15:23
pabelangerpanda|ruck: you'll need to get sign off from apevec, dmsimard and team.15:23
panda|ruckpabelanger: so step one, find which group should do the work. step 2 use the existing to create a ubuntu mirror. step 3 convert the manifest for a centos server,, step 4 replace, step 5 profit ?15:25
pabelangeranother option, would be donate some capacity to openstack-infra, for jobs to run on, then we'd have to stand up a local mirror anyways :)15:25
*** cshastri has quit IRC15:25
pabelangerpanda|ruck: yah, but so far, step 2 is a no-go15:25
pabelangerbecause ubuntu15:25
pabelangerpanda|ruck: EmilienM lets move this to #rdo15:26
EmilienMjaosorior: question on https://review.openstack.org/#/c/515618/15:26
EmilienMpabelanger: sure15:26
*** ramishra has quit IRC15:27
jaosoriorEmilienM: answered15:29
EmilienMjaosorior: all approved, good to go15:30
*** itlinux has joined #tripleo15:30
*** mdnadeem has quit IRC15:31
jaosoriorEmilienM: thanks15:31
*** gfidente is now known as gfidente|warhw15:34
*** ykarel has quit IRC15:34
*** hewbrocca is now known as hewbrocca_afk15:34
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart master: WIP: Use tripleo-upgrade role in CI  https://review.openstack.org/51338215:36
*** Goneri has quit IRC15:39
*** anshul has quit IRC15:41
*** jlinkes_ has quit IRC15:42
*** thrash|biab is now known as thrash15:44
*** adarazs_afk is now known as adarazs15:46
*** adarazs is now known as adarazs|rover15:46
*** jaosorior has quit IRC15:48
*** rhallisey has joined #tripleo15:52
*** derekh has quit IRC15:52
*** rbowen has quit IRC15:52
*** anshul has joined #tripleo15:54
*** Goneri has joined #tripleo15:55
*** itlinux has joined #tripleo15:57
*** artom has quit IRC15:58
*** anshul has quit IRC15:58
*** ffiore has quit IRC16:05
*** ratailor has quit IRC16:06
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798516:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172807016:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]16:10
openstackLaunchpad bug 1728070 in tripleo "CI Promotion. Pike and master periodic jobs are failing" [Critical,In progress]16:10
openstackgerritMerged openstack/tripleo-image-elements master: Use volumes for security hardened images  https://review.openstack.org/49958816:12
*** lucasagomes is now known as lucas-afk16:13
*** anshul has joined #tripleo16:15
*** jfrancoa has quit IRC16:19
*** tosky has joined #tripleo16:19
*** artom has joined #tripleo16:21
*** suuuper has joined #tripleo16:21
*** catintheroof has quit IRC16:23
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Prepare 8.0.1  https://review.openstack.org/51576916:25
*** cylopez has quit IRC16:26
*** catintheroof has joined #tripleo16:27
*** rhallisey has quit IRC16:27
*** ccamacho has quit IRC16:29
*** gvrangan has joined #tripleo16:32
*** gvrangan has quit IRC16:33
*** gvrangan has joined #tripleo16:33
*** Goneri has quit IRC16:34
*** catintheroof has quit IRC16:35
*** salmankhan has quit IRC16:35
*** ccamacho has joined #tripleo16:35
*** dtantsur is now known as dtantsur|afk16:36
*** catintheroof has joined #tripleo16:36
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Use rdoproject-based settings for containers and dlrn_hash for images  https://review.openstack.org/51466816:38
*** links has quit IRC16:38
*** catintheroof has quit IRC16:39
*** catintheroof has joined #tripleo16:39
*** gvrangan has quit IRC16:39
owalshgfidente|warhw, ping16:41
*** ccamacho has quit IRC16:41
*** gvrangan has joined #tripleo16:42
*** gvrangan has quit IRC16:43
*** bogdando_ has joined #tripleo16:43
*** gvrangan has joined #tripleo16:46
*** bogdando_ has quit IRC16:46
*** bogdando_ has joined #tripleo16:47
*** bogdando has quit IRC16:47
*** bogdando_ is now known as bogdando16:47
*** bogdando has quit IRC16:48
*** gvrangan has quit IRC16:51
*** achadha has joined #tripleo16:51
*** sshnaidm|off has quit IRC16:54
*** trown is now known as trown|lunch16:54
*** florianf has quit IRC16:54
*** achadha has quit IRC16:55
*** mrch has quit IRC16:56
*** dmacpher has quit IRC17:01
*** fzdarsky is now known as fzdarsky|afk17:03
*** fragatina has quit IRC17:03
*** ykarel has joined #tripleo17:06
*** ykarel is now known as ykarel|away17:06
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798517:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172807017:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]17:10
openstackLaunchpad bug 1728070 in tripleo "CI Promotion. Pike and master periodic jobs are failing" [Critical,In progress]17:10
*** ebarrera has joined #tripleo17:12
*** leitan has quit IRC17:12
EmilienMpanda|ruck: removed alert and renamed title on https://bugs.launchpad.net/tripleo/+bug/172807017:12
openstackLaunchpad bug 1728070 in tripleo "Placeholder for RDO Cloud issues" [Critical,In progress]17:12
*** leitan has joined #tripleo17:12
EmilienMand assigned yourself17:12
panda|ruckEmilienM: :( I need to create an escalation card for that bug17:14
EmilienMpanda|ruck: please avoid titles like "Pike promotion fails"17:15
EmilienMthat doesn't help much17:15
EmilienMbut rather "RDO Cloud fails to give nova nodes"17:15
EmilienMsomething that we can understand17:15
EmilienMwe know Pike promotion fails17:15
EmilienMbut we need to break down the issues ok?17:15
panda|ruckit's a tracker bug for the promotion ....17:15
panda|ruckWehn I escalate the bug, I need a bug to attach it to17:16
panda|ruckIf none of the bugs survises more than 4 hours, I cannot create the escalation ... and the escalation is about missing promotion I thinkg17:16
EmilienMpanda|ruck: fair enough, I guess I'm just asking for bug reports with good titles that's all17:16
EmilienMpanda|ruck: also, assignment17:17
panda|ruckThat's why I created a generic bug17:17
*** Goneri has joined #tripleo17:17
EmilienMwe don't want bugs like "Pike promotion is failing" or "CI is failing" or things like this17:17
*** leitan has quit IRC17:17
EmilienMI'll rename them, to make them more understandable17:17
EmilienMpanda|ruck: because it ends up in dozen of untriaged and open bugs in our backlog17:18
EmilienMpanda|ruck: alex and I spent hours to close them all recently17:18
EmilienMpanda|ruck: we found bugs like you created without any update since 6 months17:18
EmilienMso to avoid that, create targeted bugs17:18
EmilienMand assign someone, and make proper triage17:18
EmilienMso we close them at some point. That's how we make bug triage in tripleo17:18
EmilienMif that doesn't work for your need, then use another tracker17:19
EmilienMeverything about ^ is documented and has been discussed as a team17:19
*** jpena is now known as jpena|off17:19
panda|ruckfine, I'll create another tracker, I'll call it launchpand.17:20
*** fragatina has joined #tripleo17:21
*** thrash is now known as thrash|bbl17:21
*** sshnaidm has joined #tripleo17:24
*** leitan has joined #tripleo17:24
*** anshul has quit IRC17:26
*** itlinux has quit IRC17:28
*** rmascena has joined #tripleo17:28
*** sshnaidm has quit IRC17:30
EmilienMpanda|ruck: :-)17:30
*** raildo has quit IRC17:31
*** ykarel|away has quit IRC17:32
*** dsneddon has joined #tripleo17:39
*** leitan has quit IRC17:41
*** leitan has joined #tripleo17:41
*** ffiore has joined #tripleo17:44
*** chlong has joined #tripleo17:44
*** rhallisey has joined #tripleo17:45
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Use dynamic inventory  https://review.openstack.org/51542517:47
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Fix extra newline in deploy command  https://review.openstack.org/48517217:50
*** yamahata has quit IRC17:52
*** itlinux has joined #tripleo17:53
*** newmember has joined #tripleo17:58
*** jbadiapa has quit IRC18:02
*** pkovar has quit IRC18:03
*** ffiore has quit IRC18:08
*** aufi has quit IRC18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798518:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]18:10
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo master: Drop the old skip-metering-database flag  https://review.openstack.org/51579618:10
*** fultonj has quit IRC18:21
openstackgerritAndy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends  https://review.openstack.org/50796318:23
*** trown|lunch is now known as trown18:44
*** fragatina has quit IRC18:48
*** dprince has quit IRC18:48
*** amoralej is now known as amoralej|off18:48
*** ebarrera has quit IRC18:49
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798519:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172813519:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]19:10
openstackLaunchpad bug 1728135 in tripleo "CI rdocloud 295 orphaned nodes have exhausted tenant resources" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami)19:10
*** dprince has joined #tripleo19:13
*** athomas has quit IRC19:14
panda|ruckpromotion failed again for the mirror problem ...19:21
panda|ruckit's not transient but it doesn't show whenm I test it manually19:21
*** Goneri has quit IRC19:25
*** catintheroof has quit IRC19:29
*** tesseract has quit IRC19:33
*** catintheroof has joined #tripleo19:41
*** catintheroof has quit IRC19:45
openstackgerritMerged openstack/tripleo-common stable/pike: chown fernet keys to match container's keystone user and group  https://review.openstack.org/51536919:46
*** pchavva has quit IRC19:51
*** suuuper has quit IRC19:55
*** suuuper has joined #tripleo19:55
*** rcernin has joined #tripleo19:57
*** suuuper has quit IRC19:59
*** itlinux has quit IRC20:01
*** bobh has joined #tripleo20:02
*** bobh has quit IRC20:04
*** bobh has joined #tripleo20:05
*** dpeacock_ has joined #tripleo20:09
*** dpeacock has quit IRC20:09
*** dpeacock_ is now known as dpeacock20:09
*** bobh has quit IRC20:09
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798520:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172813520:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]20:10
openstackLaunchpad bug 1728135 in tripleo "CI rdocloud 295 orphaned nodes have exhausted tenant resources" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami)20:10
*** yamahata has joined #tripleo20:24
*** dprince has quit IRC20:27
*** rhallisey has quit IRC20:35
*** ansiwen has quit IRC20:37
*** itlinux has joined #tripleo20:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Disable MongoDB in scenario002  https://review.openstack.org/51472720:42
*** ansiwen has joined #tripleo20:44
*** yamahata has quit IRC20:44
bnemecEmilienM: So it turns out rh1 had a number of issues.20:45
bnemecI think they're all resolved now.20:45
bnemecI'll send an email to the list with a post-mortem.20:45
*** pchavva has joined #tripleo20:45
*** ansmith has quit IRC20:49
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/newton: [newton] Add Puppet package to bindep, for module build  https://review.openstack.org/51594520:50
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Add Puppet package to bindep, for module build  https://review.openstack.org/51594620:50
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/ocata: [ocata] Add Puppet package to bindep, for module build  https://review.openstack.org/51594720:50
EmilienMbnemec: ok, thanks20:50
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/pike: [pike] Add Puppet package to bindep, for module build  https://review.openstack.org/51594820:50
*** pchavva has quit IRC20:52
*** rlandy has quit IRC20:54
*** abishop has quit IRC21:00
*** pchavva has joined #tripleo21:04
*** fragatina has joined #tripleo21:04
*** achadha has joined #tripleo21:05
*** trown is now known as trown|outtypewww21:06
*** itlinux has quit IRC21:07
*** pchavva has quit IRC21:09
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798521:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]21:10
*** itlinux has joined #tripleo21:15
EmilienMbnemec: thx for the summary21:18
bnemecnp21:18
*** gfidente|warhw has quit IRC21:23
*** yamahata has joined #tripleo21:24
*** rcernin has quit IRC21:28
*** rmascena has quit IRC21:31
*** newmember has quit IRC21:33
*** amodi has quit IRC21:38
*** ansmith has joined #tripleo21:40
*** itlinux has quit IRC21:44
openstackgerritOpenStack Proposal Bot proposed openstack/instack-undercloud stable/pike: Updated from global requirements  https://review.openstack.org/51570621:45
*** dougbtv_ has joined #tripleo21:45
*** pradk has quit IRC21:55
*** dciabrin has quit IRC21:55
*** dciabrin has joined #tripleo21:56
openstackgerritBen Nemec proposed openstack/tripleo-common master: Revert "Remove references to the current task from task() in baremetal.yaml"  https://review.openstack.org/51595521:56
bnemecYay, a new crit to end the week.21:56
panda|ruckoh joy.21:58
bnemecEmilienM: FYI^22:01
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51163322:09
*** dougbtv_ has quit IRC22:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798522:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172815822:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1728158 in tripleo "Can not evaluate YAQL expression when registering nodes" [Critical,In progress] - Assigned to Ben Nemec (bnemec)22:10
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51163322:11
EmilienMbnemec: approved22:12
EmilienMd0ugal_, thrash|bbl, rbrady : please be more careful on the patch approved without passing CI22:12
EmilienMthe failures were legit :(22:12
*** jtomasek has quit IRC22:14
EmilienMI'm out22:19
*** artom has quit IRC22:19
*** achadha has quit IRC22:20
*** achadha has joined #tripleo22:20
honzaanyone else having issues with pysqlite2 in oooq?22:23
*** panda|ruck is now known as panda|ruck|off22:24
*** achadha_ has joined #tripleo22:26
*** achadha has quit IRC22:26
*** achadha_ has quit IRC22:30
*** itlinux has joined #tripleo22:33
*** achadha has joined #tripleo22:33
*** cdearborn has quit IRC22:34
*** sshnaidm has joined #tripleo22:38
*** achadha has quit IRC22:38
*** jcoufal has quit IRC22:44
openstackgerritMerged openstack/tripleo-common master: Remove references to the current task from task() in ceph-ansible.yaml  https://review.openstack.org/51369222:45
*** itlinux has quit IRC22:46
openstackgerritIan Main proposed openstack/python-tripleoclient master: Use ansible to deploy undercloud.  https://review.openstack.org/50958622:47
openstackgerritIan Main proposed openstack/python-tripleoclient master: Remove fake_keystone  https://review.openstack.org/51028822:47
openstackgerritIan Main proposed openstack/python-tripleoclient master: Don't install packages as part of undercloud deploy.  https://review.openstack.org/51023922:47
openstackgerritIan Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy.  https://review.openstack.org/51596122:47
Slowera little friday game of chubby bunny for CI.. ;)22:49
*** leitan has quit IRC22:50
*** leitan has joined #tripleo22:50
*** leitan has quit IRC22:51
*** leitan has joined #tripleo22:51
*** leitan has quit IRC22:51
openstackgerritIan Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy.  https://review.openstack.org/51596122:58
*** tosky has quit IRC23:00
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: WIP: Only mount selinux sysfs in nova_libvirt container  https://review.openstack.org/51596623:08
*** jmelvin has quit IRC23:09
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798523:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172815823:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]23:10
openstackLaunchpad bug 1728158 in tripleo "Can not evaluate YAQL expression when registering nodes" [Critical,In progress] - Assigned to Ben Nemec (bnemec)23:10
*** ansiwen[q] has quit IRC23:10
*** yamahata has quit IRC23:10
*** ansiwen[q] has joined #tripleo23:15
openstackgerritIan Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy to use Ansible.  https://review.openstack.org/51596723:17
openstackgerritIan Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy to use Ansible.  https://review.openstack.org/51596723:17
*** ansiwen[q] has quit IRC23:22
openstackgerritIan Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy to use Ansible.  https://review.openstack.org/51596723:24
*** owalsh is now known as owalsh_pto23:29
*** oanson has quit IRC23:54
*** oanson has joined #tripleo23:55
*** leitan has joined #tripleo23:55
*** achadha has joined #tripleo23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!