Tuesday, 2017-12-05

EmilienMhttp://status.openstack.org/openstack-health/#/g/build_name/tripleo-ci-centos-7-scenario003-multinode-oooq-container-mistral00:03
EmilienMthere are some failures: ^00:03
EmilienMmwhahaha, weshay ^00:03
EmilienMhttp://logs.openstack.org/69/511169/4/gate/tripleo-ci-centos-7-scenario003-multinode-oooq-container/475633f/logs/tempest.html.gz00:03
EmilienMthe jobname is weird though00:03
mwhahahathats on mistral itself00:03
*** ebarrera has quit IRC00:04
mwhahahahence the -mistral00:04
EmilienMopenstack-health adds the project name at the end of the job00:04
EmilienMI asked on infra channel00:04
EmilienMyeah00:05
EmilienMalso, we need to keep in mind this isn't check pipeline00:05
EmilienMbut gate / periodic00:06
EmilienMand we don't have periodic jobs anymore00:06
EmilienMso these are failures in gate00:06
EmilienMwhich should be lower than check00:06
mwhahaharight00:06
mwhahahawe want them to be 100%00:06
mwhahahaor as close to as possible00:06
EmilienMalso maybe we're missing some data00:08
EmilienMalso-bis I'm wondering if a fake subunit is created before the deployment, because the deployment can fail before running tempest00:08
EmilienMwhich is usually the case00:09
EmilienMI know Sagi has a patch for that00:09
EmilienMhttps://review.openstack.org/#/c/522955/00:09
*** rakhmerov has quit IRC00:16
*** etingof has quit IRC00:18
*** dhill__ has joined #tripleo00:18
*** dhill_ has quit IRC00:18
*** rakhmerov has joined #tripleo00:19
*** etingof has joined #tripleo00:19
*** dmacpher has quit IRC00:22
*** athomas has quit IRC00:27
*** hjensas has joined #tripleo00:28
*** hjensas has joined #tripleo00:28
*** tosky has quit IRC00:32
weshayEmilienM, I see them, thanks00:36
*** hjensas has quit IRC00:44
*** bfournie has joined #tripleo00:52
*** bfournie has quit IRC00:53
*** tvignaud has quit IRC00:55
*** sai_ has quit IRC00:59
*** bfournie has joined #tripleo00:59
openstackgerritIan Main proposed openstack/tripleo-quickstart-extras master: DNM: Update quickstart extras with undercloud install for containers  https://review.openstack.org/51744401:00
*** matbu has quit IRC01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628401:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]01:10
*** psachin has joined #tripleo01:11
*** dmacpher has joined #tripleo01:12
*** tvignaud has joined #tripleo01:13
*** fragatina has quit IRC01:14
*** fragatina has joined #tripleo01:15
*** fragatina has quit IRC01:16
*** matbu has joined #tripleo01:16
Sloweranyone know which part of the CI configures the undercloud insecure docker registry?01:19
Slowerit's not happening in my job.. (a CI testing job)01:19
*** cshastri has joined #tripleo01:26
*** matbu has quit IRC01:27
openstackgerritSteve Baker proposed openstack/tripleo-common master: Implement post-upload cleanup of docker images  https://review.openstack.org/51920101:37
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP Concurrent upload container images  https://review.openstack.org/52205501:37
*** fragatina has joined #tripleo01:42
*** fragatina has quit IRC01:44
*** mrunge has quit IRC01:56
*** matbu has joined #tripleo01:57
*** mrunge has joined #tripleo01:59
*** matbu has quit IRC02:08
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628402:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]02:10
*** matbu has joined #tripleo02:15
*** thrash is now known as thrash|g0ne02:20
*** dparkes has quit IRC02:39
EmilienMSlower: in quickstart probably?02:49
*** moshele has joined #tripleo02:53
openstackgerritMerged openstack/tripleo-common master: Memory channels parameter is not derivable  https://review.openstack.org/52331503:03
*** ramishra has joined #tripleo03:05
*** links has joined #tripleo03:07
mwhahahaOh noes is the ovsdbapp issue back? I thought that was the same problem we had with scenario001/303:07
mwhahahaSlower: puppet-tripleo I think as part of the docker setup03:07
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - zuul/tripleo-ci-dsvm: remove required-projects  https://review.openstack.org/52526503:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628403:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]03:10
*** ooolpbot has quit IRC03:10
openstackgerritMerged openstack/tripleo-common master: Add DesignatePassword  https://review.openstack.org/51899303:14
*** noslzzp has joined #tripleo03:16
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Add noop's for docker pacemaker for rabbitmq  https://review.openstack.org/52475103:20
weshayhey Slower03:22
*** threestrands has joined #tripleo03:27
openstackgerritMerged openstack-infra/tripleo-ci master: Default $NODEPOOL_PROVIDER  https://review.openstack.org/49003703:31
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-validations master: Updated from global requirements  https://review.openstack.org/52541303:39
Slowerweshay: heya!03:41
weshaySlower, hey o03:41
Slowerweshay: so if you look at the setup in the undercloud, the docker registry isn't configured properly03:42
weshayhrm.. ok let's take a look03:42
Slowerdocker-distribution still has port 500003:42
weshayhrm03:42
weshayk03:42
weshayneed a sec to get the check outs03:42
Slowerand there's no setting for insecure registry03:42
Slowersure03:42
Slowerweshay: http://logs.openstack.org/11/523511/10/check/tripleo-ci-centos-7-containers-multinode/e2f12e4/logs/undercloud/etc/sysconfig/docker.gz03:43
Slowerhttp://logs.openstack.org/11/523511/10/check/tripleo-ci-centos-7-containers-multinode/e2f12e4/logs/undercloud/etc/docker-distribution/registry/config.yml.txt.gz03:43
Slowervs ours03:43
Slowerwhich has port 5000 and no insecure registry setting03:43
weshayso we need03:45
Sloweris it because we're installing docker rpm later?03:45
Slowermaybe it's updating.. shouldn't overwrite that stuff tho eh?03:45
weshayya.. so wait a sec..03:46
weshayyou are referring to errors in the overcloud deployment at this point03:46
Slowercorrect03:46
weshayso nothing has changed there03:46
Slowercorrect03:46
weshayhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L15-L2203:47
Slowerwell I added something but we can puill it03:47
weshaypull it where though.. move it from overcloud deployment to undercloud?03:47
Slowerweshay: right, so I set that03:47
Slowerto try it03:47
weshayya.. but it already should be done..03:48
Slowerbut no where else is that set03:48
weshayhrm.. /me looks at our review again03:48
Sloweralso03:48
Slowerit's still set to port 500003:48
weshaySlower, can you chat?03:48
weshaySlower, the overcloud-prep-containers should still be run as is.. w/ the same variables03:49
weshaywe've only touched the undercloud03:50
weshayso I'm confused03:50
openstackgerritIan Main proposed openstack/instack-undercloud master: DNM, undercloud containers TESTING ONLY  https://review.openstack.org/51811803:50
Sloweryeah sure03:51
weshayin your channel03:52
Slowerbe there in a sec :)03:52
*** psahoo has joined #tripleo03:55
*** fragatina has joined #tripleo03:56
*** psahoo is now known as psahoo|BF04:01
*** fragatina has quit IRC04:01
*** psahoo|BF is now known as psahoo04:06
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628404:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]04:10
*** ooolpbot has quit IRC04:10
*** pdeore has joined #tripleo04:10
*** shreshtha has joined #tripleo04:11
*** ykarel|away has joined #tripleo04:13
*** psachin has quit IRC04:15
*** fragatina has joined #tripleo04:17
*** skramaja has joined #tripleo04:18
*** pgadiya has joined #tripleo04:24
*** psachin has joined #tripleo04:30
*** psachin has quit IRC04:32
*** fpan has quit IRC04:33
*** moshele has quit IRC04:37
*** dpawar has joined #tripleo04:43
*** jaganathan has joined #tripleo04:44
*** udesale has joined #tripleo04:53
*** jaosorior has joined #tripleo04:55
*** anilvenkata has joined #tripleo04:55
*** ratailor has joined #tripleo04:56
jaosoriorstevebaker: around?04:58
jaosoriorEmilienM: around?04:59
*** pdeore_ has joined #tripleo05:01
*** pdeore has quit IRC05:03
*** gfidente has quit IRC05:07
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628405:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]05:10
*** psachin has joined #tripleo05:10
*** tzumainn has quit IRC05:11
*** moshele has joined #tripleo05:18
*** threestrands has quit IRC05:25
openstackgerritMerged openstack/python-tripleoclient stable/pike: Fix for timeouts on scale down  https://review.openstack.org/52353205:26
*** gfidente has joined #tripleo05:34
EmilienMjaosorior: yes05:38
jaosoriorEmilienM: Hey, do you know the status the issue of not being able to add new images?05:38
jaosoriorEmilienM: For several weeks we haven't been able to add new images (e.g. rsyslog, barbican) cause they fail when trying to add them to tripleo-common, cause they're not in RDO. and we can't add them to RDO cause there's work on-going to add the ability to make that liste of autogenerated images flexible05:39
jaosoriorEmilienM: but this is effectively blocking adding any new containterized services :/05:40
EmilienMjaosorior: I'm not aware about that tbh05:40
jaosoriorEmilienM: it's blocking https://review.openstack.org/#/c/521529/ and it would block barbican if we didn't have some REALLY old testing images in the tripleomaster dockerhub repo (which we're still testing if they work)05:42
EmilienMjaosorior: I think you need to get in touch with amoralej|off / mandre on your morning05:44
*** fragatina has quit IRC05:44
jaosoriorEmilienM: will do05:45
jaosoriorEmilienM: this has been blocking adding any new images for new services :/ I don't get why other folks aren't having ths issue.05:45
EmilienMjaosorior: I understand how bad is it05:46
EmilienMjaosorior: I just don't know how to help05:46
jaosoriorEmilienM: it's alright, I'll keep poking people. thanks05:46
EmilienMjaosorior: ping panda as well05:47
EmilienMall these people have been working on that part05:47
EmilienMjaosorior: let me know if things don't move05:49
EmilienMI'm out now05:49
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: zuul/tripleo-ci-dsvm: remove required-projects  https://review.openstack.org/52526505:50
EmilienMdmsimard: it passed CI ^05:50
jaosoriorEmilienM: oh, it's blocking barbican now as well :(05:53
jaosoriorhttp://logs.openstack.org/39/523539/5/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/6c4fd7c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz05:53
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: DNM - test dependencies  https://review.openstack.org/52546905:53
EmilienMdmsimard: I'm testing it https://review.openstack.org/#/q/topic:zuul/required-projects/cleanup05:53
EmilienMjaosorior: dude like I said I can't do anything05:53
EmilienMjaosorior: if it's blocking barbican's gate, disable voting05:54
jaosoriorEmilienM: not saying that you do; just keeping (someone) updated.05:54
jaosoriorapparently I'm the only one that's hitting stuff with it, so at least somebody else knows05:54
EmilienMsorry for that05:54
* EmilienM out05:54
EmilienMttyl05:54
jaosorior~have a good one05:54
*** rlandy has quit IRC05:57
*** janki has joined #tripleo05:58
*** pdeore has joined #tripleo05:59
*** pdeore_ has quit IRC06:01
*** masco has joined #tripleo06:03
*** gfidente has quit IRC06:03
*** masco has quit IRC06:05
Tenguhello there06:05
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628406:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]06:10
jaosoriorTengu: sup dude06:15
Tengujaosorior: fine and you? a bit frozen with the home->work travel with the motorbike… it was about -1°C this morning :]06:17
*** pdeore has quit IRC06:18
jaosoriorTengu: haha shiiit motorbike at -1?06:19
Tenguyup06:19
*** oanson has quit IRC06:20
Tengunot a big deal. Especially since I got heated handles :D06:20
*** oanson has joined #tripleo06:20
bandinimorning06:21
Tenguhello bandini06:21
bandiniyo Tengu06:22
*** pgadiya has quit IRC06:25
*** nyechiel has joined #tripleo06:26
*** dmacpher has quit IRC06:30
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common stable/pike: Memory channels parameter is not derivable  https://review.openstack.org/52547606:34
*** masco has joined #tripleo06:35
*** pdeore has joined #tripleo06:38
*** skramaja is now known as skramaja|afk06:38
*** karthiks has joined #tripleo06:39
*** pgadiya has joined #tripleo06:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add deploy_steps_tasks interface  https://review.openstack.org/52234406:51
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186306:51
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Move step 1 preparation to deploy-steps-tasks.yaml  https://review.openstack.org/52526006:52
*** marios has joined #tripleo06:53
Tengujaosorior: IPSEC like in "you're able to direct-connect your office network to the openstack over IPSEC tunnels"?06:54
jaosoriorTengu: nah, IPSEC like, set up tunnels for each network and make all nodes communicate with encryption.06:55
jaosoriorTengu: an alternative to TLS everywhere06:55
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci master: WIP, test CI with experimental undercloud upgrade  https://review.openstack.org/51536106:56
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Set simple crypto plugin as global default for Barbican  https://review.openstack.org/52548006:56
openstackgerritmathieu bultel proposed openstack/tripleo-common master: Fix step_config output which is str not list  https://review.openstack.org/51013106:57
*** udesale__ has joined #tripleo06:58
*** udesale has quit IRC06:58
jaosoriorTengu: although, at some point we could make it configurable for such a thing :D06:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260707:00
*** udesale has joined #tripleo07:00
*** shreshtha has quit IRC07:01
*** udesale__ has quit IRC07:02
*** abregman|afk has quit IRC07:03
*** bogdando has joined #tripleo07:07
*** dparkes has joined #tripleo07:08
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628407:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]07:10
*** ooolpbot has quit IRC07:10
*** pgadiya has quit IRC07:10
*** jfrancoa has joined #tripleo07:11
Tengujaosorior: ok :)07:17
Tengujaosorior: there was a VPNaaS, but it was dropped in either ocata or pike, because no real interest, and no maintainer :/07:18
Tenguso. haproxy basic auth support. let's go :D07:18
*** agurenko has joined #tripleo07:20
*** pgadiya has joined #tripleo07:23
Tengu\o/ haproxy::userlist does already exists, so it should be really easy to play.07:24
Tengualthough… duh… it's an ugly implementation :/.07:25
*** iranzo has joined #tripleo07:25
*** oidgar has joined #tripleo07:26
stevebakerjaosorior: hey, I'll take a look tomorrow, if mandre doesn't figure it out first07:27
jaosoriorstevebaker: that would be great :D07:27
openstackgerritAde Lee proposed openstack/puppet-tripleo master: Add multiple backends for barbican  https://review.openstack.org/52353807:28
*** pmannidi has quit IRC07:31
*** pcaruana has joined #tripleo07:32
stevebakerjaosorior: so the issue is that tripleomaster hasn't had images promoted for 4 days, so its missing new ones like centos-binary-barbican-keystone-listener?07:33
jaosoriorstevebaker: the issue is that there is currently no way to get new images07:36
jaosoriorstevebaker: which is something you were working on, IIRC07:36
jaosoriorbut your work was blocked for some reason?07:36
jaosorioror -1ed07:36
jaosoriorI don't recall07:36
jaosoriorstevebaker: yeah, this https://review.rdoproject.org/r/#/c/9328/07:37
jaosoriorstevebaker: so, for instance, this adds the rsyslog image to tripleo-common https://review.openstack.org/#/c/521529/ but it fails CI cause we don't have the rsyslog image in RDO. So... I can't add a new image to tripleo cause we don't have that image (which I have no way of adding yet)07:37
*** aditya_r has joined #tripleo07:38
jaosoriorstevebaker: currently there is a txt file that maintains that list of containers, which we can't modify because it's the same list as pike containers07:39
jaosoriorstevebaker: so, that effectively blocks barbican and rsyslog now07:39
stevebakerjaosorior: centos-binary-barbican-keystone-listener is being built, it just hasn't been promoted to docker.io yet https://console.registry.rdoproject.org/registry#/images/master/centos-binary-barbican-keystone-listener07:41
*** hjensas has joined #tripleo07:41
jaosoriorstevebaker: is it?07:42
jaosoriorstevebaker: as far as I had understood, stuff is being tracked here https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/parsed_containers.txt which is where we can't add images right now07:42
stevebakerjaosorior: this one was built 3 hours ago https://console.registry.rdoproject.org/registry#/images/master/centos-binary-barbican-keystone-listener:tripleo-ci-testing07:42
jaosoriorstevebaker: how do we add extra images? we also need centos-binary-barbican-worker07:43
jaosoriorstevebaker: and rsyslog07:43
*** pmannidi has joined #tripleo07:44
pmannidi'll own it as it's australian military case07:45
*** pmannidi has quit IRC07:45
jaosoriorlol dafuq07:45
stevebakerhttps://console.registry.rdoproject.org/registry#/images/master/centos-binary-barbican-worker and https://console.registry.rdoproject.org/registry#/images/master/centos-binary-rsyslog-base are both being built and pushed to the RDO registry. Maybe there is a backlog due to the weekend zuul outage07:45
*** pmannidi has joined #tripleo07:45
jaosoriorstevebaker: how do we add centos-binary-rsyslog ? (need it besides the -base image)07:46
stevebakerjaosorior: but you could ping weshay or dmsimard about when a promotion to docker.io/tripleomaster might happen07:46
jaosoriorstevebaker: ok ok, so apparently the issue I had has been fixed. How does the adding of images work? if I need a new one for instance07:47
*** pmannidi has quit IRC07:48
jaosoriorstevebaker: soo... this should be fixed once we get a docker.io/tripleomaster promotion, right? http://logs.openstack.org/29/521529/2/check/tripleo-ci-centos-7-containers-multinode/5bb3c76/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2017-12-04_18_51_5307:48
stevebakerAll you need to do is get the change landed to container-images/overcloud_containers.yaml.j2, then wait for the eventual promotion to docker.io/tripleoupstream07:48
*** rcernin has quit IRC07:48
jaosoriorstevebaker: ok! well, then it's all sorted out then :D I'll poke weshay and dmismard when they're around.07:49
jaosoriorstevebaker: thanks!07:49
stevebakerjaosorior: yeah, I believe so. I mean that is my expectation. No promotions for 4 days looks like the root cause of your issue07:50
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186307:52
stevebakerjaosorior: did you see my comment about deleting container-images/overcloud_containers.yaml ?07:52
jaosoriorstevebaker: eh... no07:52
*** dbecker has joined #tripleo07:52
jaosoriorstevebaker: where?07:52
stevebakerjaosorior: https://review.openstack.org/#/c/515613/07:53
*** oidgar has quit IRC07:53
jaosorioraaah alright07:53
jaosorioryeah07:53
jaosoriorstevebaker: I'll abandon that change then07:53
*** athomas has joined #tripleo07:54
stevebakerok, thanks07:54
jaosoriorstevebaker: thanks for the help and the reviews07:54
*** dsariel has joined #tripleo07:54
stevebakerjaosorior: no problem, good evening!07:54
jaosoriorhave a good one!07:54
*** morazi has joined #tripleo07:56
*** radeks has joined #tripleo07:58
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: WIP: haproxy basic auth support  https://review.openstack.org/52549207:58
Tengujaosorior: -^ started.07:58
jaosoriornice nice07:58
Tenguwill now integrate that new thing in the current endpoints.07:58
*** tvignaud has quit IRC07:58
Tenguwell, endpoint definition and so on.07:59
*** tvignaud has joined #tripleo07:59
*** skramaja|afk is now known as skramaja08:00
*** paramite has joined #tripleo08:01
*** shreshtha has joined #tripleo08:03
*** artom_ has joined #tripleo08:04
*** aufi has joined #tripleo08:04
*** ykarel_ has joined #tripleo08:06
*** artom has quit IRC08:07
*** ykarel|away has quit IRC08:08
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628408:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]08:10
*** ooolpbot has quit IRC08:10
*** jaosorior_ has joined #tripleo08:10
*** yprokule has joined #tripleo08:11
*** ffiore has joined #tripleo08:11
*** jaosorior has quit IRC08:13
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/pike: Reintroduce TripleoPackages in the default Controller service.  https://review.openstack.org/52425908:14
*** pdeore has quit IRC08:15
*** pdeore has joined #tripleo08:16
*** jtomasek has joined #tripleo08:16
*** abregman has joined #tripleo08:16
*** brault has quit IRC08:17
*** dpawar has quit IRC08:21
*** masco has quit IRC08:22
*** florianf has joined #tripleo08:22
*** masco has joined #tripleo08:23
*** shardy has joined #tripleo08:24
*** tesseract has joined #tripleo08:24
*** mdnadeem has joined #tripleo08:27
*** mcornea has joined #tripleo08:28
*** dpawar has joined #tripleo08:31
*** ccamacho has joined #tripleo08:32
jtomasekflorianf: hi, could you please check these again, I had to rebase due to merge conflict https://review.openstack.org/#/q/project:openstack/tripleo-ui+branch:master+topic:4modalPanel08:32
*** rcernin has joined #tripleo08:33
*** cylopez has joined #tripleo08:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186308:35
*** brault has joined #tripleo08:37
*** gkadam has joined #tripleo08:38
openstackgerritMerged openstack/tripleo-ui master: Layout components updates  https://review.openstack.org/51636108:38
*** ykarel_ is now known as ykarel08:38
*** ffiore has quit IRC08:39
*** dsariel has quit IRC08:40
*** pcaruana has quit IRC08:40
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom overcloud user/key for deployed server prepare  https://review.openstack.org/52277608:41
*** mrch has joined #tripleo08:43
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: WIP: haproxy basic auth support  https://review.openstack.org/52549208:45
Tengupfiou.08:46
Tengujaosorior_: -^^ some more stuff. not yet finished.08:46
jaosorior_ack08:47
*** jaosorior_ is now known as jaosorior08:47
florianfjtomasek: sure, just a sec08:50
Tengujaosorior: it's a bit painful, but I think I'm pretty close to a nice resolution :). unit tests are passing on my laptop, sooo… should be ok shortly.08:50
*** ffiore has joined #tripleo08:53
*** jpena|off is now known as jpena08:55
*** skramaja is now known as skramaja|lunch08:55
Tengujaosorior: I take the opportunity to add unit tests for your new tripleo::haproxy::service_endpoints definition :). I'll need them in order to ensure nothing is broken.08:56
*** jpich has joined #tripleo08:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186308:58
jaosoriorTengu: hey! great!08:58
jaosoriorappreciated08:58
*** sshnaidm|off is now known as sshnaidm|rover08:58
Tengu:)08:58
Tenguplaying a bit with hiera in the fixture.08:58
* Tengu puts some starwars stuff in the tests08:59
*** oidgar has joined #tripleo09:00
jaosoriorshardy: when running config-download, have you ran into this issue?09:01
jaosoriorfailed: [192.168.24.6] (item=/etc/libvirt/secrets) => {"changed": false, "failed": true, "item": "/etc/libvirt/secrets", "msg": "There was an issue creating /etc/libvirt/secrets as requested: [Errno 13] Permission denied: '/etc/libvirt/secrets'", "path": "/etc/libvirt/secrets", "state": "absent"}09:01
jaosoriorfailed: [192.168.24.6] (item=/etc/libvirt/qemu) => {"changed": false, "failed": true, "item": "/etc/libvirt/qemu", "msg": "There was an issue creating /etc/libvirt/qemu as requested: [Errno 13] Permission denied: '/etc/libvirt/qemu'", "path": "/etc/libvirt/qemu", "state": "absent"}09:01
shardyjaosorior: No, perhaps you're missing become: true somewhere?09:02
jaosoriorshardy: I didn't modify that section at all09:02
jaosoriorshardy: only thing I did was pull the deploy_steps_tasks patch and add the ipsec composable service09:03
jaosoriorshardy: actually, checking this out https://review.openstack.org/#/c/522344/3/common/deploy-steps.j2 there is no become: true  in the host prep steps09:04
shardyjaosorior: yeah I think we expect the service templates to set it when needed09:05
shardyjaosorior: I wasn't testing containerized libvirt yesterday so maybe this is an issue specific to that09:06
jaosorioryeah09:06
shardynot sure why we wouldn't see it in CI tho09:06
jaosoriorit does seem like an issue there though09:06
jaosoriorshardy: I run the playbook like this:09:06
jaosorioransible-playbook \09:06
jaosorior    -v \09:06
jaosorior    -i /usr/bin/tripleo-ansible-inventory \09:06
jaosorior    ~/config-download/tripleo-*/deploy_steps_playbook.yaml09:06
jaosoriorshardy: I'm trying now by specifically adding the "become" as part of the command line. But I didn't expect it was needed there.09:06
shardyjaosorior: yeah at one point it was, but then we added become: true in various places09:07
shardyI suspect this way of deploying w/containers isn't yet covered in CI09:07
radeksI have sort of generic question, I am looking for the largest overcloud installation with TripleO Ocata/Pike? I have 150 phy computes with Newton (classic architecture, 3 controllers) and I wonder how far I can go with Pike and containers?09:08
*** etingof has quit IRC09:09
shardyradeks: I think rook tested to ~300 nodes but some tuning of the undercloud was needed09:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628409:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]09:10
shardyradeks: not sure if that was ocata or pike tho tbh09:10
radeks<shardy> do you know by any chance if that was with containers or classic architecture ?09:10
shardyradeks: Not sure on the status of scale testing w/containers, IIRC that was with the classic architecture09:11
shardyrook should be able to confirm later when he is around09:11
radeksshardy: thanks !09:11
radeksI will ask him09:11
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job  https://review.openstack.org/52390309:15
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job  https://review.openstack.org/52390309:15
radeksshardy: I also tweaked few bits and pieces on UC, mostly MariaDB, heat and keystone (httpd).09:15
shardyradeks: ack - FYI from Queens onwards it's possible to decouple deploying the nodes/networks from the configuration (via ansible)09:16
shardyI suspect that will help with very large deployments, but there's discussion ongoing about further improvements09:17
radeksshardy: I just wonder how this new architecture will influence these large scale deployments, I didn't have chance to check it myself yet :)09:19
radeksshardy: it's hard to find spare 100+ servers to play with :)09:20
shardyradeks: I think the changes to enable ansible for the configuration workflow will help, but probably the switch to containers won't have a large impact provided there is a local image registry to cache the images09:24
Tengujaosorior: small question for your dynamic endpoints definition: I should add a "tripleo.<service_name>.haproxy_endpoints:" hash entry in hiera, right?09:24
shardyTengu: you're working on making the LB configuration pluggable in puppet-tripleo?09:25
shardythat has been on the TODO list for some time now :)09:25
Tengushardy: nope, well, partly, I'm working on top of jaosorior patches in order to add support for basic authentication for selected haproxy endpoint (user managed)09:26
Tengushardy: if you want the details: https://bugs.launchpad.net/tripleo/+bug/173613209:26
openstackLaunchpad bug 1736132 in tripleo "Allow to add Basic Authentication on HAProxy endpoint" [Wishlist,In progress] - Assigned to Cédric Jeanneret (cjeanneret-c2c)09:26
shardyTengu: Ah, I see09:27
*** abregman has quit IRC09:27
shardyTengu: the reason I asked is we've got a historical non-pluggable interface with conditionals, e.g:09:27
shardyhttps://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L167109:27
shardythat would be nicer if it was pluggable like firewall rules, but it's not been done yet09:28
Tengushardy: yep, jaosorior has already done part of the jobs, creating a pluggable interface.09:28
shardyTengu: Aha, nice, will check out the patches :)09:30
*** dtantsur|afk has quit IRC09:30
jaosoriorshardy: it's pluggable09:30
Tengushardy: it's already merged, at least the tripleo::haproxy::service_endpoints definition :).09:30
Tenguhttps://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy/service_endpoints.pp  now it has to be called ;)09:30
jaosoriorshardy: https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L88309:30
Tengu:D09:31
shardyjaosorior: nice!  So we just have to port the services in tht over to use that then09:31
jaosoriorshardy: it's VERY manual right now though, so there's a lot of manual stuff you need to add about your endpoint; but yeah, it's there.09:31
*** lucas-afk is now known as lucasagomes09:31
shardyjaosorior: cool, well it's a first step at least, thanks for doing it09:32
jaosoriorshardy: I was starting to do that https://review.openstack.org/#/c/474107/  https://review.openstack.org/#/c/474110/ but never actually got it working. Then again, I barely dedicated any time to it.09:32
*** aditya_r has quit IRC09:32
Tengujaosorior: on my side, I'm unable to actually use that, at lest unable to make unit tests working -.-09:33
*** dtantsur has joined #tripleo09:33
*** dtantsur has joined #tripleo09:33
jaosoriorTengu: what's up with the unit tests?09:34
*** skramaja|lunch is now known as skramaja09:34
Tengujaosorior: apparently, adding in the "default.yaml" hiera file in fixtures/hieradata an entry isn't working.09:34
Tengujaosorior: wait, gisting the current state.09:34
* jaosorior sucks at puppet unit testing09:35
* jaosorior dislikes the interface as well09:35
*** etingof has joined #tripleo09:35
Tengujaosorior: https://gist.github.com/cjeanneret/b5f8cc093261eeb620f941f38dc96e0109:36
Tengu:D09:36
Tenguof course it's nasty and unfriendly.09:36
Tengubut, at least, it exists, and should be used ;).09:36
jaosoriorI don't disagree09:36
Tenguwill check if I have to point to hiera in some way.09:36
Tengumuahahahahahahaaaaa   classes/tripleo_profile_base_cinder_volume_spec.rb:    # hiera. (╯°□°)╯︵ ┻━┻09:37
Tenguok, made my day XD09:37
*** gfidente has joined #tripleo09:37
*** gfidente has quit IRC09:37
*** gfidente has joined #tripleo09:37
*** dtantsur has joined #tripleo09:38
*** derekh has joined #tripleo09:39
*** aufi has quit IRC09:39
*** etingof has quit IRC09:42
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add subnet property to ctlplane network for server resources  https://review.openstack.org/47381709:47
Tengujaosorior: apparently, the syntaxe with "dots" in the index isn't correctly understood.09:48
*** stendulker has joined #tripleo09:48
Tengujaosorior: in order to test, I entered "tripleo::foo::haproxy_endpoints" in the hiera file, updated your code, and tadaaaa, working.09:48
Tenguat least it finds the entry.09:48
jaosoriornice!09:49
Tengusooo... may I correct your code in my own patch, or should I first update it ?09:50
jaosoriorTengu: what do you need to correct?09:50
Tengujaosorior: I think I'll first need a patch in order to add unit tests to your code (and correct it), then only shall I push my basic auth…09:50
jaosorioroh09:50
jaosoriorthe . syntax09:50
Tengujaosorior: hiera index, replacing dots by semi-colons09:50
Tenguyep09:50
Tengunasty one to spot.09:50
*** cylopez has left #tripleo09:51
jaosorioruhm... sure, although I don't understand why. That's exactly the same syntax we use to set the firewall rules09:51
Tengufunny thing: I'm pretty sure I already saw that one.09:51
Tenguhmmmm.09:51
Tengumaybe something fishy with my ruby libs version?09:51
Tenguwait, rolling back to your syntaxe, and will push the code to my review in order to check what Zuul thinks of it.09:51
jaosoriorok09:52
*** owalsh_ is now known as owalsh09:52
Tengujaosorior: I think zuul doesn't run tests when we prefix the commit message with a "WIP" right?09:52
*** salmankhan has joined #tripleo09:53
Tenguoh. it actually does. ook.09:53
*** dbellant has joined #tripleo09:53
Tengujaosorior: small remark: in order to check what was going on, I added a check that makes your definition fail when there isn't any endpoint defined for the service. I'm not sure if I can let it (i.e. will it break anything?)09:54
Tenguhmm. it will break. because no service uses hiera for now.09:54
*** etingof has joined #tripleo09:55
*** iranzo has quit IRC09:55
*** aufi has joined #tripleo09:55
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: WIP: haproxy basic auth support  https://review.openstack.org/52549209:58
*** dsariel has joined #tripleo09:58
Tengujaosorior: -^ I'll let that one run the tests until the end of zuul in order to ensure nothing breaks.09:58
*** gfidente has quit IRC10:00
shardydtantsur: Hey quick question, what's the osc plugin equivalent of ironic node-set-provision-state control-0 deleted ?10:00
shardyprobably obvious but I can't see what it is atm10:01
shardyhttps://bugs.launchpad.net/tripleo/+bug/1734831 means I still need that a few times a day :(10:01
openstackLaunchpad bug 1734831 in tripleo "virtualbmc processes silently die over time" [Medium,Triaged]10:01
shardynot been able to find the root cause yet, may try running the vbmc processes in a terminal to figure it out10:02
*** dbellant has quit IRC10:02
*** dbellant has joined #tripleo10:03
dtantsurshardy: openstack baremetal node unprovision IIRC10:04
dtantsurshardy: etingof is working on making vbmc a proper daemon, that may help10:04
shardydtantsur: ack, thanks - if you've got any other ideas how to debug I'm happy to try10:04
*** SlickNik- has joined #tripleo10:04
shardythe processes just die and AFAICS there's nothing in the logs10:04
*** SlickNik has quit IRC10:05
*** SlickNik- is now known as SlickNik10:05
Tengunice one.10:05
jistrflaper87: are any of these errors the ones that stopped your OpenShift deployment? Inability to start origin-node.service because of:10:05
jistrDec 05 10:03:27 overcloud-controller-2 origin-node[849132]: W1205 10:03:27.530436  849176 cni.go:157] Unable to update cni config: No networks found in /etc/cni/net.d10:05
jistrDec 05 10:03:27 overcloud-controller-2 origin-node[849132]: F1205 10:03:27.538985  849176 start_node.go:140] could not start DNS, unable to read config file: open /etc/origin/node/resolv.conf: no... or directory10:05
* jistr may try redoing the OC from scratch to see if the error is reproducible\10:06
flaper87jistr: that's exactly the error I'm hitting10:09
flaper87it's basically because the 99-something dispatch for network-manager is not running10:09
flaper87or it is running but not for the right IFACE10:09
*** gfidente has joined #tripleo10:09
*** gfidente has quit IRC10:09
*** gfidente has joined #tripleo10:09
jistrack, thanks10:10
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628410:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]10:10
flaper87if you run the sript manually, it'll generate the resolv.conf and you can start object-node10:10
flaper87jistr: ^10:10
*** tosky has joined #tripleo10:11
jistrthanks. I'm redoing the OC to clean it up, i finished the install after a few unsuccessful attempts so i want to make sure i have a "clean expected broken state" :))10:11
jistrerr i mean i didn't finish it, but in previous attempts it failed on different issues even earlier, hence the cleanup10:12
*** dbellant has quit IRC10:13
*** dbellant has joined #tripleo10:15
*** SlickNik- has joined #tripleo10:17
*** SlickNik has quit IRC10:18
*** SlickNik- is now known as SlickNik10:18
*** sshnaidm|rover is now known as sshnaidm|rover|a10:18
*** sshnaidm|rover|a is now known as sshnaidm|afk10:18
*** hewbrocca_afk is now known as hewbrocca10:19
Tengujaosorior: hmmm. weird. apparently, it's not only in my run of the tests that it fails. I'll try to check this afternoon or tomorrow.10:21
flaper87jistr: mmh, what did it fail on?10:23
jaganathanplease review tripleo-heat-templates patch https://review.openstack.org/#/c/523358/10:24
jistrflaper87: pebkac and environment setup issues, not really fault of the installer. Btw i use the 3.6 branch of openshift-ansible from github, do you use the same?10:24
flaper87jistr: I use the RPM10:25
jistrflaper87: and where do you get it from? (not CentOS i guess?)10:25
flaper87jistr: sudo yum install -y centos-release-openshift-origin10:25
flaper87jistr: openshift-ansible10:25
flaper87jistr: it's 3.6, IIRC10:25
jistrhuh i tried that and i got some old openshift-ansible without the byo playbook directory10:26
*** jwb has quit IRC10:26
flaper87jistr: there are multiple packages. Make sure openshift-ansible-playbooks is installed10:26
flaper87jistr: yum search openshift-ansible should give you several packages10:26
jistri installed just openshift-ansible IIRC, so if it doesn't pull all the necessary stuff then i was probably missing some RPM. Will try that again, thanks :)10:27
openstackgerritMerged openstack/tripleo-ui master: Prevent filter duplicates  https://review.openstack.org/50853610:28
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/pike: Make CephPools override properties of other pools  https://review.openstack.org/52552010:28
*** rcernin has quit IRC10:30
*** dsariel has quit IRC10:32
*** abregman has joined #tripleo10:33
*** aufi has quit IRC10:37
*** sshnaidm|afk is now known as sshnaidm|rover10:43
*** ffiore has quit IRC10:45
*** ffiore has joined #tripleo10:45
shardyjistr: Hey re https://review.openstack.org/#/c/523436/ I am missing something as that variable is undefined, where does it come from?10:46
dtantsurfolks, is tripleo-ci-centos-7-scenario002-multinode-oooq-container down?10:46
* shardy reads inventory docs10:47
jistrshardy: ah ha! And it works fine for you without the --private-key param altogether? I guess the variable may be only defined if --private-key is explicitly passed on the command line to the outer playbook... That's what happens when executing via tripleoclient --config-download, but i guess i broke the manual run by doing this. I'll post a patch to make the --private-key optional.10:49
shardyjistr: ah yeah, perhaps we can make it optional10:49
shardyjistr: I'm running via config download && ansible-playbook manually10:49
*** aufi has joined #tripleo10:50
*** daidv has quit IRC10:50
*** dbellant has quit IRC10:51
*** dbellant has joined #tripleo10:52
openstackgerritDaniel Alvarez proposed openstack/tripleo-heat-templates master: Add support for containerized OVN Metadata Agent  https://review.openstack.org/52516410:52
*** salmankhan has quit IRC10:52
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Make Kubespray install work without --private-key too  https://review.openstack.org/52552410:53
jistrshardy: ^^10:54
jistri'll mark it wip for now as i didn't test yet10:54
shardyjistr: thanks, testing now10:55
openstackgerritLee Yarwood proposed openstack/tripleo-quickstart master: Allow the confgiuration of virtualbmc on Newton  https://review.openstack.org/50921910:55
*** salmankhan has joined #tripleo10:57
*** dsariel has joined #tripleo11:02
*** daidv has joined #tripleo11:05
*** cshastri has quit IRC11:05
sshnaidm|rovermyoung|ruck, weshay fyi https://bugs.launchpad.net/tripleo/+bug/173638311:05
openstackLaunchpad bug 1736383 in tripleo "RDO project registry: container push fails for a few containers" [Critical,Triaged]11:05
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628411:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]11:10
*** chem has quit IRC11:11
*** etingof has quit IRC11:12
*** stendulker_ has joined #tripleo11:14
*** marios has quit IRC11:16
*** stendulker has quit IRC11:18
*** chem has joined #tripleo11:21
*** stendulker_ has quit IRC11:21
shardyjistr: now that we don't set kubectl_localhost, what's the preferred method to get kubectl on the undercloud, do we have packages already or is it a manual install?11:24
*** etingof has joined #tripleo11:26
*** udesale has quit IRC11:29
*** adarazs is now known as adarazs_lunch11:35
*** jkilpatr has quit IRC11:36
*** janki has quit IRC11:39
*** shreshtha has quit IRC11:40
*** cshastri has joined #tripleo11:47
*** ramishra has quit IRC11:50
shardyOuch, actually turning it back on fails due to https://github.com/ansible/ansible/issues/1170211:50
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove workaround for periodic jobs  https://review.openstack.org/52554111:52
sshnaidm|roverpanda, adarazs_lunch  ^^11:53
*** akrivoka has joined #tripleo11:55
Tenguhmmm, is it an known issue that doesn't break the CI checks? http://logs.openstack.org/92/525492/3/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/6e8c26c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2017-12-05_11_31_2111:55
*** jkilpatr has joined #tripleo11:55
openstackgerritOr Idgar proposed openstack/puppet-tripleo stable/pike: Remove erroneous octavia code related to sync_db  https://review.openstack.org/51337611:57
*** oidgar is now known as oidgar__11:58
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372811:58
jistrshardy: centos ships kubernetes-client RPM11:58
*** oidgar__ has quit IRC12:02
*** ramishra has joined #tripleo12:02
jaosoriorTengu: yeah, we need promotion in docker.io to happen before that gets fixed.12:02
*** oidgar__ has joined #tripleo12:02
*** oidgar__ has quit IRC12:03
*** oidgar has joined #tripleo12:03
*** cshastri has quit IRC12:03
*** raildo has joined #tripleo12:05
*** anilvenkata has quit IRC12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628412:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]12:10
*** ooolpbot has quit IRC12:10
*** fzdarsky has joined #tripleo12:10
openstackgerritMerged openstack/tripleo-common stable/pike: Memory channels parameter is not derivable  https://review.openstack.org/52547612:11
adarazs_lunchsshnaidm|rover: +212:12
*** adarazs_lunch is now known as adarazs12:13
*** lucasagomes is now known as lucas-hungry12:13
*** pkovar has joined #tripleo12:14
*** marios has joined #tripleo12:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186312:17
jistrflaper87: so i got successfully further by deploying without br-ex, but not much further :) The origin-node.service started fine but it got to12:17
jistrRUNNING HANDLER [openshift_node : restart openvswitch]12:18
jistrwhere it got stuck12:18
jistrand i can't ssh to the overcloud node...12:18
jistrprobably the ovs restart broke something w/ networking12:18
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Instance HA support  https://review.openstack.org/50692212:19
*** pgadiya has quit IRC12:23
*** bfournie has quit IRC12:24
*** bfournie has joined #tripleo12:24
openstackgerritNuman Siddique proposed openstack/tripleo-quickstart master: Enable OVN HA profile in fset30  https://review.openstack.org/52380412:24
*** pkovar has quit IRC12:28
*** bfournie has quit IRC12:28
*** pkovar has joined #tripleo12:28
*** iranzo has joined #tripleo12:32
*** ykarel_ has joined #tripleo12:34
*** masco has quit IRC12:35
*** skramaja has quit IRC12:36
*** ykarel has quit IRC12:37
*** psahoo has quit IRC12:37
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add subnet property to ctlplane network for server resources  https://review.openstack.org/47381712:37
*** fpan has joined #tripleo12:40
jaosoriorshardy: just tested your patch out https://review.openstack.org/#/c/522344/3 worked awesomely :D12:42
jaosoriortested it with my IPSEC patch https://review.openstack.org/#/c/521863/12:42
*** ratailor has quit IRC12:43
jaosoriorAND... the IPSEC stuff works! (I knew it worked, but it now works as a composable service)12:45
*** abregman is now known as abregman|mtg12:45
*** Goneri has quit IRC12:45
jaosoriorjistr: could you check this out? https://review.openstack.org/#/c/521863 could it be an acceptable iteration? The next step would be to stop cloning the IPSEC repo and have that available already. But other than that, it looks quite solid.12:46
jaosoriorshardy: ^^12:46
openstackgerritHarald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754412:48
*** jpena is now known as jpena|lunch12:48
*** mrch has quit IRC12:48
*** fzdarsky has quit IRC12:49
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559412:50
*** abishop has joined #tripleo12:51
*** gfidente has quit IRC12:54
*** BryanS68 has joined #tripleo12:55
openstackgerritMerged openstack/tripleo-ui master: Modals with react-bootstrap  https://review.openstack.org/51636212:55
openstackgerritMerged openstack/tripleo-ui master: Rework ModalPanel to be based on react-overlays  https://review.openstack.org/51636412:55
openstackgerritMerged openstack/tripleo-ui master: ModalPanel vs Validations sidebar positioning  https://review.openstack.org/51636512:55
*** gfidente has joined #tripleo12:56
*** gfidente has quit IRC12:56
*** gfidente has joined #tripleo12:56
*** paramite has quit IRC12:58
*** dprince has joined #tripleo12:58
jistrjaosorior: yea lgtm. Gave just +1 for now but i can circle back later, will also look at the parent patch.12:59
*** mrch has joined #tripleo12:59
openstackgerritHarald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754413:01
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186313:01
jaosoriorjistr: fixed. Thanks!13:01
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add modal closing components  https://review.openstack.org/52036613:01
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add release notes for Modals and Validations sidebar changes  https://review.openstack.org/52410113:01
jistrjaosorior: i didn't notice this at first -- i think the paths are out of sync there https://review.openstack.org/#/c/521863/12/extraconfig/services/ipsec.yaml13:05
*** jcoufal has joined #tripleo13:05
*** cshastri has joined #tripleo13:06
*** ramishra has quit IRC13:08
jaosoriorjistr: oh wait, in the playbook directory there's supposed to be a "roles" directory that contains any custom roles that you add13:08
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: Implements AIDE Intrusion Detection System  https://review.openstack.org/48302513:09
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628413:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]13:10
*** ooolpbot has quit IRC13:10
*** bfournie has joined #tripleo13:10
*** paramite has joined #tripleo13:10
*** rhallisey has joined #tripleo13:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add IPSEC composable service  https://review.openstack.org/52186313:10
*** lucas-hungry is now known as lucasagomes13:14
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Fix stackviz data  https://review.openstack.org/52560313:14
jaosoriorjistr: changed it and trying it out again.13:15
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: WIP: Add disruption detector for Overcloud endpoint monitoring  https://review.openstack.org/52012613:17
jaosoriorjkilpatr: seems that folks are uploading roles to tripleo-common now. that would make it easier to integrate the rsyslog role: https://review.openstack.org/#/c/487450/13:18
jkilpatrjaosorior, nice! I was just working on updating my spec13:19
jaosoriorjkilpatr: that way, you won't need to worry about packaging n stuff :D13:20
dtantsurattempt #2: should I recheck Could not pull image docker.io/tripleomaster/centos-binary-barbican-keystone-listener in sc002?13:20
dtantsuris there a bug number for it?13:21
jaosoriordtantsur: no bug yet13:21
jaosoriordtantsur: promotion is needed for docker images on docker.io13:21
jaosoriorsshnaidm|rover: ^^13:21
dtantsurjaosorior: fine, but why do we try getting a non-existing image? shouldn't we use an older one?13:21
jaosoriorno idea13:22
*** links has quit IRC13:22
dtantsursigh.. is everything affected or is it just my bad luck (i.e. is it worth retrying)?13:22
jaosoriordtantsur: it will fail scenario002 every time.13:22
sshnaidm|roverdtantsur, I'm working on this right now..13:23
dtantsurack, thanks sshnaidm|rover13:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update roles listing  https://review.openstack.org/50941513:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Refactor ImportPlanCard into ActionCard  https://review.openstack.org/51226513:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Select Roles feature  https://review.openstack.org/51226613:23
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: [WIP] Submit select roles  https://review.openstack.org/52151013:23
*** s_b_34343 has joined #tripleo13:25
weshaymyoung|ruck, ping13:25
dpawarIs there a know issue for Zuul gate is timed_out for legacy-puppet-unit-4.8-centos-7 ?13:25
dpawarhttp://logs.openstack.org/05/525105/2/gate/legacy-puppet-unit-4.8-centos-7/79bb988/job-output.txt.gz#_2017-12-05_07_01_00_93836113:26
*** liverpooler has joined #tripleo13:26
*** thrash|g0ne is now known as thrash13:26
*** jwb_ has joined #tripleo13:26
dpawarabishop: ^^13:26
*** jwb_ is now known as jwb13:27
weshaysshnaidm|rover, myoung|ruck looks like the promoter is currently promoting testing to current-tripleo13:27
weshayafaict13:27
*** cshastri has quit IRC13:27
dpawardprince: ^^13:27
sshnaidm|roverweshay, yes, it is13:27
weshaysshnaidm|rover, rock on13:27
*** ffiore has quit IRC13:28
*** pchavva has joined #tripleo13:28
*** tzumainn has joined #tripleo13:28
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559413:29
weshayarxcruz, is this associated w/ a bug? https://review.openstack.org/#/c/525603/13:30
arxcruzweshay: yes, I haven't add the bug because i'm still testing13:30
*** trown|outtypewww is now known as trown13:30
weshayarxcruz, k13:30
*** udesale has joined #tripleo13:32
abishopdpawar: sorry, I'm not familiar with that test13:32
sshnaidm|roverjaosorior, dtantsur https://bugs.launchpad.net/tripleo/+bug/173641113:32
openstackLaunchpad bug 1736411 in tripleo "CI: scenario002 job fails when not finding centos-binary-barbican-keystone-listener container" [Critical,Triaged]13:32
dtantsurthanks!13:32
sshnaidm|roverI'll try to upload it manually right now13:32
*** ffiore has joined #tripleo13:32
*** rlandy has joined #tripleo13:33
jistrjaosorior: will the generated_psk work fine w/ stack updates too?13:33
*** psachin has quit IRC13:33
*** oidgar has quit IRC13:34
jaosoriorjistr: it should, it should only rekey (which is nice for security). I'll try it right now :D13:36
*** moshele has quit IRC13:37
jaosoriorjistr: a stack update would be a matter of running ansible again, right?13:37
*** moshele has joined #tripleo13:39
*** janki has joined #tripleo13:40
*** jaganathan has quit IRC13:42
openstackgerritJustin Kilpatrick proposed openstack/tripleo-specs master: rsyslog remote logging  https://review.openstack.org/52349313:43
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci master: Need to add openstack-selinux with OVS  https://review.openstack.org/52560813:45
*** jkilpatr has quit IRC13:47
*** jkilpatr has joined #tripleo13:48
*** liverpooler has quit IRC13:49
*** liverpooler has joined #tripleo13:49
*** fpan has quit IRC13:50
*** artom_ has quit IRC13:52
*** jangutter has joined #tripleo13:52
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Specify docker.io defaults for upstream config release files.  https://review.openstack.org/52063913:53
jistrjaosorior: yea for the purpose of the generated_psk thing, yes13:53
*** jpena|lunch is now known as jpena13:57
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559413:59
EmilienMhi13:59
mwhahahahi2u13:59
EmilienMhi2u213:59
jistrhi4all14:00
mwhahaha#startmeeting tripleo14:00
mwhahaha#topic agenda14:00
mwhahaha* Review past action items14:00
mwhahaha* One off agenda items14:00
mwhahaha* Squad status14:00
mwhahaha* Bugs & Blueprints14:00
openstackMeeting started Tue Dec  5 14:00:05 2017 UTC and is due to finish in 60 minutes.  The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
mwhahaha* Projects releases or stable backports14:00
mwhahaha* Specs14:00
mwhahaha* open discussion14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
mwhahahaAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:00
mwhahahaHi everyone! who is around today?14:00
*** openstack changes topic to " (Meeting topic: tripleo)"14:00
openstackThe meeting name has been set to 'tripleo'14:00
weshay0/14:00
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:00
EmilienMo/14:00
jtomaseko/14:00
fultonjo/14:00
abishopo/14:00
gfidenteo/14:00
marioso/14:00
akrivokao/14:00
beagleso/14:00
jaosorioro/14:00
florianfgchamoul: Hi! akrivoka and I have created an etherpad with patches we'd like to give priority among the many which haven't been reviewed yet. feel free to add some of your patches you consider most important: http://etherpad.corp.redhat.com/validations-reviews14:00
jfrancoao/14:01
florianfo/14:01
slaglehi14:01
*** vpickard_ is now known as vpickard14:01
*** pkovar has quit IRC14:01
pandao/14:01
sshnaidm|rovero/14:01
akrivokaflorianf: gchamoul: let's use this one instead :) https://etherpad.openstack.org/p/validations-reviews14:01
d0ugalo/14:01
ccamachoo/14:02
shardyo/14:02
mwhahaha#topic review past action items14:02
mwhahahanone14:02
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:02
mwhahahamoving on14:02
mwhahaha#topic one off agenda items14:02
mwhahaha#link https://etherpad.openstack.org/p/tripleo-meeting-items14:02
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:02
*** mmethot|away is now known as mmethot14:02
dpeacocko/14:02
mwhahaha(mwhahaha) Queens-m2 is here and gone. Queens-m3 is next (Jan 22 - Jan 26).14:02
mwhahahaMilestone 2 is this week. We've already cut the release so at this point it's now more book keeping to understand what was released and where we're are for the final milestone. It should be noted that we were going to stop allowing the starting of new features at milestone 2 this release so if you're feature isn't started then now is the time to considering moving it to Rocky. I believe we can still make some14:02
mwhahahaprogress on some of the work if we can ensure that the feature is off by default and does not impact the existing features.14:02
mwhahahaIf you have an open feature, please review the outstanding work and be able to explain why we should allow your feature to continue forward for Queens.  There are some features which we will most likely still need allow new work. For example the upgrades workflows and things related to fast forward upgrades. Beyond that, we should be focusing on bugs as we have nearly 540 open bugs.14:03
mwhahahaI'm going to also send an ML note about this as well.14:03
*** aputtur_ has joined #tripleo14:03
*** oidgar has joined #tripleo14:03
mwhahahaIt's time to start stablizing14:03
EmilienMtl;dr no new things from now14:03
oidgaro/14:03
*** artom has joined #tripleo14:04
florianfakrivoka: :-)14:04
mwhahahaany questions?14:04
gchamoulflorianf, akrivoka: ack14:04
EmilienMI agree with Alex, we should stabilize what we started in queens, stop starting, start stopping, with exceptions for FFU and upgrades14:04
gfidenteI think integration squad has a couple of things in progress, which landed only partially14:04
gfidentethat ceph rbd mirror and nfs-ganesha14:04
EmilienMgfidente: I think that's fine, since it's a feature in progress I guess14:05
*** ramishra has joined #tripleo14:05
EmilienMit can be finished in m314:05
mwhahahagfidente: which is fine but we'll need to raise those and explain the impact and make sure we're not adding regressions14:05
matbuo/14:05
EmilienMiiuc the message is: from now, don't expect patches to land if they do something really new14:05
shardyI was wondering if we planed to swith to the --config-download workflow for default or if we'll have to wait for rocky14:05
mwhahahagfidente: it would also be beneficial to land those sooner than later, so let us know if you need reviews14:05
gfidentemwhahaha EmilienM ok thanks guys14:06
EmilienMshardy: really good question14:06
gfidenterbdmirror should be good, just needs an update14:06
shardyIMO it'd be good to align on one deploy workflow, as it's a bit of a headache trying to keep the two approaches in sync14:06
gfidentenfs-ganesha news some more iterations14:06
* social o/14:06
EmilienMshardy: AFIK it doesn't work yet with OVB (we tried yesterday) - maybe a simple thing, but we haven't tested it in CI14:06
shardyunless there are blocker issues, I'd like to switch to it by default, with a flag to re-enable the "classic" heat based workflow for Queens14:07
EmilienMshardy: I agree14:07
shardythen remove the heat software deployment bits potentially in Rocky14:07
dprince++ for config-download14:07
mwhahahaif we can, let's do it like ASAP14:07
mwhahahaIt's not something we should do in january14:07
shardyEmilienM: Ok, maybe we could have a WIP patch switching the default to flush out any remaining issues?14:07
mwhahahai'd rather we have time to flush out all the issues14:07
EmilienMslagle: thoughts?14:08
shardymwhahaha: yup agreed, if we're doing it we need to land it asap14:08
arxcruzo/14:08
jpichWill we be tagging releases early on milestone week now, in general? They used to be done on Thursdays during milestones, from what I recall14:08
gfidenteshardy is config-download currently interpolating with the ExternalResource ?14:09
shardygfidente: yes it works with the external_deploy_tasks, but not the mistral workflow resources14:09
gfidentesorry not interpolating, interleaving14:09
shardygfidente: what's the status of reworking the ceph integration to external_deploy_tasks?14:09
gfidenteshardy auch, like zero14:09
mwhahahajpich: the release team got really approve happy, but we shouldn't expect the end of the week. that being said we'll allow a few things to land this week while we do the updates to BPs and such14:09
shardygfidente: :(14:09
shardyI thought it was discussed a while ago14:10
gfidentediscussed yes, at ptg14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628414:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173641114:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]14:10
openstackLaunchpad bug 1736411 in tripleo "CI: scenario002 job fails when not finding centos-binary-barbican-keystone-listener container" [Critical,Triaged]14:10
jpichmwhahaha: Noted14:10
jpichThanks14:10
shardyin theory it should be fairly easy as it's just moving the ansible playbook run from mistral to t-h-t14:10
gfidenteshardy did not do the actual work though14:10
shardygfidente: that's true, I did not ;)14:10
gfidenteI think jistr did some investigation?14:10
gfidenteshardy yeah I did not even, just bringing up needs to be done14:10
shardygfidente: maybe we can take a look and figure out how much work it will be14:10
gfidenteshardy yeah sounds like we should14:11
EmilienMwe probably need to take some actions14:11
mwhahaha#action EmilienM and slagle look at switching config-download as default14:11
EmilienM#action EmilienM to talk with slagle about switching default deployments to use config-download14:12
EmilienMargggg14:12
beaglesgfidente, shardy, if you guys were going to do this soon (like today) we could kill two birds with one stone if I could look-on14:12
EmilienM#undo14:12
* mwhahaha is faster14:12
EmilienMmwhahaha: mine was longer14:12
EmilienMundo didn't work?14:12
shardybeagles: ah yeah we need the same for octavia14:12
weshaylol14:12
s_b_34343HI all , i'm new to triplo controller HA.14:12
mwhahaha#action gfidente and shardy look at ceph-ansible and config-download14:12
beaglesI think I've got the gist of it and am in the process of trying it out but I think the general ideas are the same14:12
EmilienMs_b_34343: in a meeting right now, we'll answer all your questions shortly14:12
slagleEmilienM: sorry got distracted with something else14:13
slaglewe can discuss post-meeting14:13
shardybeagles: yeah I think it's logically the same as the mistral approach, but generating the inventory may differ a bit14:13
s_b_34343how shoul i start will be needing doc14:13
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/services/kubernetes-master.yaml14:13
beaglesshardy, ack14:13
EmilienMslagle: perfect14:13
shardythat is a working example for kubespray14:13
s_b_34343to understand controller ha working for triplo14:13
mwhahahaalso14:13
mwhahaha#link https://releases.openstack.org/queens/schedule.html14:13
gfidenteshardy ack14:13
mwhahahaok moving on14:13
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations stable/pike: Remove yum warning in ceph-ansible validation  https://review.openstack.org/52561814:13
mwhahaha(akrivoka) Validations patches in need of reviews14:13
mwhahaha#link https://etherpad.openstack.org/p/validations-reviews14:13
akrivokahi14:13
akrivokait's been a bit of a challange lately getting validations patches merged, as we have very few cores reviewing regularly14:13
akrivokawe have some patches that have been up for a long time, so just wanted to ask people to review them so we can get them merged14:14
akrivoka(see link above)14:14
EmilienMakrivoka: nice work on validations, I'm doing some reviews sometimes, it's really good14:14
akrivokathat's pretty much it14:14
mwhahahaakrivoka: thanks14:14
s_b_34343<EmilienM>: ok will wait14:14
akrivokaEmilienM: thanks, much appreciated!14:14
mwhahaha#action team to review open validation patches14:14
gchamoul+1 for EmilienM14:14
mwhahaha(gcerami) getting a list of containers built by kolla14:14
EmilienMquick question14:14
*** aputtur_ has quit IRC14:14
EmilienMbefore we switch to containers14:14
*** ykarel_ is now known as ykarel|afk14:14
EmilienMakrivoka: where these validations are tested?14:15
EmilienMI know tripleo-ci-centos-7-containers-multinode is supposed to run them?14:15
akrivokaEmilienM: not sure tbh14:15
EmilienMakrivoka: can you show me logs on where validations are run?14:15
EmilienMhttp://logs.openstack.org/79/525179/1/check/tripleo-ci-centos-7-containers-multinode/a72e345/logs/14:15
akrivokamaybe florianf knows?14:16
EmilienMakrivoka: your team needs to figure out how your code is tested, imho14:16
EmilienMakrivoka: and if you need help from CI team, please ask14:16
gchamoulEmilienM: validations runs are enabled only on a few list of scenarios14:16
akrivokaok, I will look into it14:16
EmilienMgchamoul: which ones? can I see logs?14:17
EmilienMwe can take it offline though14:17
gchamoulEmilienM: let me check the list for you14:17
EmilienM#action EmilienM, akrivoka, gchamoul to check where validations are tested in tripleo ci14:17
EmilienMgchamoul: ok, post-meeting14:17
EmilienMmwhahaha: go ahead14:17
florianfEmilienM: only manually afaik14:17
mwhahahawell gcerami is not here14:17
EmilienMpanda: ^14:17
sshnaidm|rovermwhahaha, he's panda14:17
gchamoulEmilienM: I planned to add the right set of jobs into tripleo-validations14:18
pandahi, right now we are using a single static file as a list of containers to upload after promotion. WE were planning to make it pe release and get the list from kolla/parsed_containers.txt on each build, but thre are some reviews that make that list obsolete, What is now the best way to get a list of built containers so we know what to upload after a promotion ?14:18
mwhahahaah brain not working yet14:18
mwhahahaI guess that's a question for dprince or mandre -^14:18
*** anilvenkata has joined #tripleo14:19
sshnaidm|roverit's root cause of current https://bugs.launchpad.net/tripleo/+bug/173641114:19
openstackLaunchpad bug 1736411 in tripleo "CI: scenario002 job fails when not finding centos-binary-barbican-keystone-listener container" [Critical,Triaged]14:19
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodesListView  https://review.openstack.org/50561114:20
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodeDrives list  https://review.openstack.org/50561214:20
dprincepanda: I think you can use the new 'container image prepare' commands to generate an upload yaml file dynamically14:20
dprincepanda: in python-tripleoclient. Would this help solve your case?14:20
pandadprince: do we need to install undercloud first ?14:20
openstackgerritAdriano Petrich proposed openstack/tripleo-common master: Validate parameters before updating  https://review.openstack.org/51124914:20
dprincepanda: no, the command just requires python-tripleoclient14:21
apetrichjtomasek, I've updated ^^ now it returns the flattened params14:21
dprincepanda: you could even install it in a .venv14:21
sshnaidm|roverdprince, does it generate per release?14:21
mandrepanda: the overcloud_containers.yaml.j2 file in tripleo-common has the list of all the images used in tripleo deployments14:21
jtomasekapetrich: sweet14:21
apetrich:)14:21
dprincesshnaidm|rover: it an. It has parameters for the container names (prefix/suffix). Also the tag, and namespace can be fully controlled14:22
jaosoriorjistr: the PSK change worked with updates.14:22
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Do not merge - Check job multinode-upgrades-master  https://review.openstack.org/52562514:22
pandamandre: taht would require tripleo-common to be fetched too ?14:22
jistrjaosorior: ack great :) (mtg here...)14:22
jaosoriorsorry14:23
dprincepanda,sshnaidm|rover: openstack overcloud container image prepare --namespace=trunk.registry.rdoproject.org/master --images-file=rdo-images.yaml --push-destination=172.19.0.2:8787 --tag tripleo-ci-testing14:23
sshnaidm|roverdprince, I mean in pike set of containers could be different then in queens, isn't it?14:23
dprincesshnaidm|rover: then you'd use the Pike version of tripleoclient. It has the same command and could manage the differences via the template in python-tripleoclient14:23
mandrepanda: right, or a curl since it's just a single file14:24
jistr( gfidente, shardy: sorry i missed the mention earlier, let's sync after the meeting. We have Kubespray running in CI and green so that's at least somewhat of a "proof" that porting ceph-ansible to external_deploy_tasks should be achievable.)14:24
jaosoriorsshnaidm|rover: they already are different.14:24
sshnaidm|roverdprince, great14:24
pandayeah i think we can try the command solution14:24
pandathanks all!14:25
weshaythanks panda14:25
mwhahahacool thanks14:25
mwhahahamoving on14:26
mwhahaha#topic Squad status14:26
mwhahahaci14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:26
mwhahahaupgrade14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:26
mwhahahacontainers14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-containers-squad-status14:26
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:26
mwhahahaintegration14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:26
mwhahahaui/cli14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:26
mwhahahavalidations14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:26
mwhahahanetworking14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:26
mwhahahaworkflows14:26
mwhahaha#link https://etherpad.openstack.org/p/tripleo-workflows-squad-status14:26
*** jmelvin has joined #tripleo14:26
mwhahahaworkflows missing updates14:26
mwhahahaI want to thank beagles for starting the networking squad meeting yesterday. very informative14:27
mwhahahaeveryone updated their status, thanks!14:27
mwhahahaany other status related items folks would like to raise14:27
EmilienMI was wondering if a weekly small summary on the ML would help14:28
EmilienMsome other teams like Keystone, Nova, do it14:28
*** Goneri has joined #tripleo14:28
EmilienMbut it would take someone willing to write-up something14:28
d0ugalOne for tripleo or per squad?14:28
EmilienMfor tripleo maybe? I'm not sure, it's a proposal14:29
EmilienMlike "a week in tripleo", that you can read in less than 5 min14:29
d0ugalI had considered doing this for the workflow squad, but there hasn't been much to share. (We have only had two meetings so far)14:29
mwhahahaEmilienM: are you volunteering to put it together? :D14:29
mwhahahai mean it's a good idea but it's yet more work14:29
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart master: Use cachable facts to allow multiple runs  https://review.openstack.org/52523714:29
EmilienMok... got it... I'll start something then :D14:29
d0ugallol14:29
EmilienMbut in french14:29
*** jcoufal has quit IRC14:29
mwhahahasure no one reads the ML anyway14:30
trownthere is also chicken egg of... will people read it14:30
EmilienMtrown: I'll give it a first try and ask who read it14:30
mwhahahaI guess it's more for folks not already knee deep in tripleo14:30
mwhahahato get the highlights14:30
trownor lowlights :p14:30
mwhahahalet's try it and it could just be a basic summary out of the status etherpads14:30
EmilienM#action EmilienM starts to send a weekly newsletter about tripleo14:30
mwhahahacool, moving on then14:31
EmilienMor a summary of weshay's bad jokes14:31
mwhahaha#topic bugs & blueprints14:31
mwhahaha#link https://launchpad.net/tripleo/+milestone/queens-314:31
mwhahahaFor Queens we currently have 71 (+1) blueprints and about 540 (-1) open bugs. 540 queens-3.14:31
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:31
trownhaha14:31
EmilienMbut we need more than 5 min reading14:31
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Do not merge - Check job multinode-upgrades-master  https://review.openstack.org/52562514:31
weshayEmilienM, you had a good one today.. I see forward progress14:31
mwhahahai'm probably going to move the <= medium bugs out to a rocky-1 milestone just so we can see what high/critical bugs we have for milestone 314:32
mwhahahaplease take a look at the high/critical bugs as we've previously mentioned now is a time for stablization14:32
EmilienMmwhahaha: do you need help in creating milestones and moving things around?14:32
mwhahahaalso i see some alert bugs poping up, let's make sure to take a look at them14:32
mwhahahaEmilienM: no i'll get it today. i moved all the m2 bugs to m3 last night14:32
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Do not merge - Check job multinode-upgrades-master  https://review.openstack.org/52562514:32
EmilienMok14:32
EmilienMthe 2 alerts are unassigned14:33
EmilienMhttps://bugs.launchpad.net/tripleo/+bug/173628414:33
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]14:33
EmilienMwho is working on this one?14:33
mwhahahathat one looks like the one we just fixed14:33
mwhahahai hadn't looked at the logs yet14:33
*** dpawar has quit IRC14:33
mwhahahabut we'd see the same stuff with the ovsdbapp thing14:33
EmilienMI pinged Ihar14:33
mwhahahak14:33
EmilienMsame for https://bugs.launchpad.net/tripleo/+bug/173641114:33
openstackLaunchpad bug 1736411 in tripleo "CI: scenario002 job fails when not finding centos-binary-barbican-keystone-listener container" [Critical,Triaged]14:33
mwhahahathat one i assume is a missing container from promotion?14:34
EmilienMjaosorior: can you take this one? or find someone actively working on it?14:34
mwhahaharelated to panda's request previously?14:34
EmilienMmost probably - but we need an owner14:34
sshnaidm|roverEmilienM, I'm actually working on it14:34
jaosoriorEmilienM: it's the image issue I was complaining about.14:34
jaosoriorEmilienM: sshnaidm|rover is working on it AFAIK.14:34
EmilienMsshnaidm|rover: so if you're working on it, please assign it to you14:35
EmilienMfolks, please keep launchpad updated14:35
sshnaidm|roverEmilienM, assigned14:35
EmilienMgood.14:35
mwhahahacool thanks, any other bug related items?14:35
mwhahaha#topic projects releases or stable backports14:37
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:37
mwhahahaso m2 went out yesterday14:37
mwhahahaEmilienM: when is the next round of stable releases? next week?14:37
EmilienMyeah I think so14:38
EmilienMor I can do it end of this week14:38
*** pdeore has quit IRC14:39
mwhahahathe release team would probably prefer next week14:39
EmilienMsounds good for me14:39
mwhahahaok14:39
mwhahahaany other backports people need reviews on?14:39
EmilienMwe said we keep stable/newton until January and then we see14:40
EmilienM(just a heads-up)14:40
*** udesale has quit IRC14:40
mwhahahak14:40
EmilienMthe # of backports into that branch is decreasing a lot14:41
*** BryanS68 has quit IRC14:41
EmilienM(good sign)14:41
EmilienMI think we can move on14:41
mwhahaha#topic specs14:41
mwhahaha#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:41
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:41
mwhahahasince it's m2, we should be moving any open specs to rocky14:42
EmilienMmaybe we can give an extra-very-last week14:42
mwhahahathat's fine14:42
EmilienMand warn people by IRC, email, etc14:42
mwhahahai'll include that in my email note about m2/m314:43
EmilienMI see 5 potential specs that has +2 already14:43
EmilienMgood14:43
mwhahahaAlso i'll be going through the blueprints and pinging folks and moving to future releases14:44
mwhahahaanyway on to bonus time14:44
mwhahaha#topic open discussion14:44
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:44
mwhahahaany random topics that people want to chat about?14:44
*** moshele has quit IRC14:44
EmilienMdid we update the list of cores?14:45
EmilienMI've seen nominations lately14:45
EmilienMrlandy was the latest iirc14:45
EmilienMtrown: ^14:45
mwhahahai believe the rest have been included, rlandy should be added if not already14:45
*** jaosorior has quit IRC14:46
EmilienMI went ahead and added her to https://review.openstack.org/#/admin/groups/190,members14:46
mwhahahak14:47
EmilienMsounds like we're done14:47
mwhahahaok thanks everyone14:47
mwhahaha#endmeeting14:47
*** openstack changes topic to "CI status: GREEEEEEEEN! see alerts and read ML. | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"14:47
openstackMeeting ended Tue Dec  5 14:47:33 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:47
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-12-05-14.00.html14:47
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-12-05-14.00.txt14:47
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-12-05-14.00.log.html14:47
EmilienMs_b_34343: don't hesitate to ask your question here, please add details of what you're doing, and which doc you're following.14:47
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add version and git sha to debug modal  https://review.openstack.org/51983514:48
EmilienMgchamoul: I propose to enable validations on tripleo-ci-centos-7-containers-multinode which already runs in tripleo-validations14:48
gchamoulEmilienM, akrivoka, florianf: well, one example and the only one I found https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-tripleo-ci-delorean-full-minimal-76/undercloud/home/stack/14:48
EmilienMgchamoul: this is not the gate14:49
gchamoulEmilienM: yes that was tested before zuulv314:49
gchamoulI think14:49
EmilienMgchamoul: no, I checked the featuresets, and it was tested on multinode-scenarios jobs (non containerized)14:49
*** udesale has joined #tripleo14:50
EmilienMbut since we stopped to run these jobs in master (and we only run them in stable/pike and before), we don't run validations anymore14:50
gchamoulEmilienM: yes14:50
EmilienMso I propose to use tripleo-ci-centos-7-containers-multinode14:50
EmilienMwhich is featureset010.yml14:50
EmilienMso if that's fine for you, you can send a patch to featureset010.yml to activate the validations and we'll see how that works14:50
gchamoulEmilienM: that's fine to me14:51
EmilienMgchamoul: cool, let me know when you have a patch, i'll review it14:51
gchamoulEmilienM: so nothing to change in the tripleo-validations zuul.d/project.yaml?14:51
EmilienMgchamoul: nope14:52
EmilienMgchamoul: for now14:52
gchamoulEmilienM: ack got it! thanks, I am doing the patch right now14:52
*** pkovar has joined #tripleo14:52
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Refactor lodash imports  https://review.openstack.org/50977814:52
EmilienMgchamoul: if there are some validations specific to some services (e.g. ceilometer), we'll have to run scenario001 (and patch the featureset)14:52
EmilienMgchamoul: but let's start small14:52
EmilienMgchamoul: thanks for the help14:52
slagleshardy: what is it that you have to do in 2 places with regards to config-download?14:53
florianfgchamoul: thanks for creating a patch!14:53
EmilienMslagle: during the meeting we discussed about whether or not we want to enable config-download by default in queens - and we said "yes, why not, but we need to make sure it works everywhere first (in all CI jobs I guess) and fix all found issues as soon as possible - we also said we would allow to disable it via the client14:54
EmilienMwhat are your thoughts?14:54
*** lblanchard has joined #tripleo14:54
*** BryanS68 has joined #tripleo14:54
gchamoulEmilienM: ok let's start small, but yes in the future we should test validations against ceilomter, ceph and maybe others services we (florianf & akrivoka) would like to see running14:54
slagleEmilienM: 1) given that we have insufficient api abstraction behind our workflows, it requires client changes, thus there are UI implications. We'd need to make sure they can take on that work14:55
slagleEmilienM: 2) it does not exactly fit with the themes of "stabilization"14:56
EmilienMgchamoul: sure, I can even help there14:56
slagle3) there is still some work to do to make it work for the SSHKnownHosts deployments for nova live migration since those deployments rely on Heat deployment outputs14:56
gchamoulEmilienM: perfect!14:56
slagleEmilienM: those are the 3 big items why I've not been pushing to make it default in queens14:57
EmilienMslagle: that's good feedback, indeed.14:57
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart master: Enable tripleo-validations runs in featureset010  https://review.openstack.org/52563714:57
florianfgchamoul, EmilienM: For now it would be a big step forward if we would test the inventory (beyond unit tests) and *some* of the undercloud validations. But I agree, in the long run we might want to have as many validations as possible covered by ci.14:57
mandrepanda: something like this should get you all the required images for a given release of tripleo, http://paste.openstack.org/show/628173/14:57
shardyslagle: https://review.openstack.org/#/c/522344/3/common/deploy-steps.j2 is an example, e.g if we're not switching the default I probably have to update that to add the per-service deploy_steps_tasks into the heat driven playbook14:58
gchamoulEmilienM, akrivoka, florianf: https://review.openstack.org/#/c/525637/14:58
*** lblanchard has quit IRC14:58
shardyit's not a huge issue but at some point it'd be really great to switch to one workflow14:58
EmilienMgchamoul: commented14:58
shardymaybe it's too soon tho given the issues mentioned above14:58
shardythanks for the update14:58
pandamandre: thanks, adding among the possible solutions14:59
*** udesale has quit IRC14:59
slagleshardy: i thought that was the heat driven playbook? that's the same playbook that config-download uses14:59
d0ugalAnyone got a moment for a small quickstart change? https://review.openstack.org/52339314:59
slagleEmilienM: shardy : i'm certainly not opposed to making it the default. just given where we are, i'm not sure it's feasibly at this point15:00
EmilienMslagle, shardy : I guess a good compromise is to expose the feature without enabling it by default15:00
shardyslagle: yes that's my point, it's what config download uses, but the heat driven one is in the RoleConfig resource at line 81-10315:00
shardyslagle: and we can't support stuff like external_deploy_tasks via the heat driven workflow15:00
EmilienMslagle: no I understand totally, no need to justify - it makes sense. I personally didn't realize about 1) and 3)15:00
sshnaidm|roverEmilienM, wrt https://review.openstack.org/#/c/523903/ - should we change tripleo-ci-centos-7-ovb-containers-oooq to run on branches only? If so, I'll update the patch15:01
gchamoulEmilienM: answered15:02
shardyEmilienM: yes we can leave it disabled by default, that just means we'll have to delay moving any commonly used features like ceph to external_deploy_tasks I think15:02
slagleshardy: ok, i think i follow now.15:03
gchamoulflorianf: running those tests could be done quickly through the openstack-tox-* IMHO15:03
shardybeagles: ^^ FYI15:04
beaglesshardy, what are your thoughts with respect to octavia? use the external_post_deploy_taks or ...15:04
beaglesshardy: :)15:04
beaglesIRC collision ;)15:04
shardyslagle: we'd been looking at moving the recent octavia bits to external_deploy_tasks, but it sounds like we may have to stick with the mistral based approach for Queens, or document use of --config-download in the case of octavia being enabled15:04
slagleshardy: well as I said, it sounds like it depends on what other folks can take on honestly. UI, ceph, octavia15:05
slaglei'm not saying it shouldnt be the default :)15:05
shardybeagles: IMHO the external_deploy_tasks interface is much cleaner but if we have to stick with the mistral approach for Queens then so be it15:05
beaglesshardy, I think we still want that asap so we can transition nicely15:05
beaglesshardy, ack15:05
shardyslagle: ack sure, I wasn't aware of some of the issues you mentioned, I'd wrongly assumed we could just hide the default in the existing deploy workflow15:06
shardye.g make it pretty transparent15:06
EmilienMpanda, weshay: how do I know on which hash containers have been built in a job? for example http://logs.openstack.org/01/511701/7/gate/tripleo-ci-centos-7-scenario003-multinode-oooq-container/86a2e14/logs15:06
EmilienMpanda, weshay: delorean hash15:06
*** dsariel has quit IRC15:06
EmilienMpanda, weshay: I see the tag: 3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d but how do I know the hash?15:06
slagleshardy: yes, i had assumed (hoped) as well when I first started. but it's not possible afaict given the existing deploy workflow has an immediate exit after Heat has accepted the stack create request15:06
slaglethat's an insufficient abstraction to make any change to our deployment, other than just call heat stack create15:07
weshaypanda, can you help EmilienM please.. I'm in a mtg15:07
pandaEmilienM: that's the hash ?15:07
shardyslagle: Ok, so I guess we need a v2 deploy workflow so that clients know to listen for a different kind of output from zaqar15:07
EmilienMpanda: yeah?15:08
EmilienMpanda: I fail to find the URL15:08
EmilienMpanda: I want to check packages version15:08
shardyslagle: thanks for the update, I guess we can iterate on that early next cycle to align with the various things in progress expecting to use external_deploy_tasks15:08
shardybrb15:08
*** shardy is now known as shardy_afk15:08
slagleshardy: that is along the lines of what I've done with config_download_deploy workflow. you call the workflow -> client polls zaqar queue until workflow completes -> deployment complete15:08
pandaEmilienM: it may have been purged , let me check15:09
EmilienMpanda: https://trunk.rdoproject.org/centos7-master/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d/15:09
EmilienMpanda: this one?15:09
*** agurenko has quit IRC15:09
pandaEmilienM: yes15:09
EmilienMpanda: ok...15:09
EmilienMpanda: it's a bit complex to find, but ok15:09
EmilienMdalvarez: python-ovsdbapp-0.6.0-0.20171024151214.0b86d12.el7.centos.src.rpm15:09
EmilienMhttps://trunk.rdoproject.org/centos7-master/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d/15:09
EmilienMdalvarez: http://logs.openstack.org/01/511701/7/gate/tripleo-ci-centos-7-scenario003-multinode-oooq-container/86a2e14/logs/undercloud/home/zuul/overcloud_containers.yaml.txt.gz15:09
EmilienMthat's how I found it15:09
dalvarezEmilienM, thanks a lot! yeah we may need 0.9.015:10
EmilienMdalvarez: I have to switch context now, but I need your help on this bug (and your team)15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173628415:10
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Triaged]15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173641115:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1736411 in tripleo "CI: scenario002 job fails when not finding centos-binary-barbican-keystone-listener container" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)15:10
EmilienMdalvarez: yeah I'm not sure why latest promotion gives us old repo - it's a question for panda maybe.15:10
jistrgfidente: re the ceph-ansible port to external_deploy_tasks, inspiration can be drawn from https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/services/kubernetes-master.yaml15:10
gfidentejistr ack, sec15:10
gfidenteI think I'll get back to you soon15:10
jistrgfidente: AFAIK only thing that we're missing to add is https://review.openstack.org/#/c/525524/15:10
gfidentewith questions :D15:10
*** ykarel_ has joined #tripleo15:11
pandaif the promotion ran close to the job time, then the job may have picked the old hash still15:11
*** aditya_r has joined #tripleo15:11
jistrotherwise it all works in CI (we've seen it green twice and not red since then) and AFAIK also locally for folks, so it's no sand castle :P15:12
jistrack re questions :)15:12
florianfgchamoul: How about we pick a bunch of specific validations for fs10 we know will succeed in the limited ci environment and only run those?15:12
akrivokagchamoul: thanks for the patch!15:13
akrivokaflorianf: good idea, that sounds good to me15:13
*** janki has quit IRC15:13
*** janki has joined #tripleo15:14
EmilienMI see we had a promotion on https://dashboards.rdoproject.org/rdo-dev15:14
gchamoulflorianf: well by default, the run-tripleo-validations will run all the validations for one group, and we know that undercloud-ram/cpu are always due to some default values from oooq15:14
*** ykarel|afk has quit IRC15:14
*** mrch has quit IRC15:14
EmilienMso the latest hash should be good, let me verify15:14
*** janki has quit IRC15:14
gchamoulflorianf: always failing...15:15
EmilienMdalvarez: ^ rechecking for you15:15
*** janki has joined #tripleo15:15
EmilienMlatest hash is ac82ea9271a4ae3860528eaf8a813da7209e62a6_28eeb6c7, let's see the version15:15
dalvarezEmilienM, thanks, that should do it15:15
florianfgchamoul: and that default would be hard to change?15:16
dalvarezEmilienM, just one more question, if i have the image (from the link you sent) whats the quickest way to check which packages are installed?15:16
gchamoulflorianf: we need to find an agreement with the CI team15:16
EmilienMdalvarez: I just showed you ^15:17
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: Convert tags to when statements for Q major upgrade workflow  https://review.openstack.org/51090215:17
gchamoulflorianf: when I added the tripleo-validations patches in OOOQ, we've had some issues with the resources15:17
EmilienMdalvarez: you look at logs/undercloud/home/zuul/overcloud_containers.yaml.txt.gz15:17
EmilienMdalvarez: that gives you the delorean hash15:17
florianfgchamoul: would it be easier with the negative tests?15:17
florianfgchamoul: "it" = only run specific validations15:18
dalvarezEmilienM, ack got it! thanks15:18
dalvarezEmilienM, didn't know that it was the dlrn hash15:18
EmilienMdalvarez: https://trunk.rdoproject.org/centos7-master/ac/82/ac82ea9271a4ae3860528eaf8a813da7209e62a6_28eeb6c7/15:18
EmilienM python-ovsdbapp-0.9.0-0.20171130203756.7d4b4b6.el7.centos.src.rpm15:18
gchamoulflorianf: it's easier with the negative tests because I run a small list of validations and tricking the validations default15:18
EmilienMso this time we're good!15:18
EmilienMdalvarez: so it seems like the bug is invalid then15:19
EmilienMdalvarez: i'll comment and close15:19
dalvarezEmilienM, cool :) thanks15:19
gchamoulflorianf: for example, https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/tripleo-validations/templates/run-preintro-validations-negative-tests.sh.j215:19
EmilienMweshay, dalvarez: FYI https://bugs.launchpad.net/tripleo/+bug/1736284/comments/215:20
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Invalid]15:20
*** dsariel has joined #tripleo15:21
dalvarezEmilienM, ack this one was easy :)15:21
EmilienMdalvarez: thanks for being responsive, as usual15:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Make Kubespray install work without --private-key too  https://review.openstack.org/52552415:23
dalvarezEmilienM, happy to help :)15:23
dalvarezthx15:23
EmilienMdalvarez: i'm going to document how to find the URL15:23
EmilienMif I found it hard, I guess it's not easy for newcomers15:23
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Use ansible-playbook in place of quickstart.sh  https://review.openstack.org/52529415:23
gchamoulflorianf, akrivoka: the negative tests for pre-deployment and post-deployment (in oooq-extras)  are still in review though https://review.openstack.org/#/q/status:open+project:openstack/tripleo-quickstart-extras+branch:master+topic:tripleo-validations15:23
openstackgerritMartin André proposed openstack/tripleo-quickstart-extras master: Do not create roles unless keystone was deployed  https://review.openstack.org/52564815:24
dalvarezEmilienM++ that one is useful for us not very familiar with it... also i added to my todo list to write a tiny guide about how to modify code within a container (rebuild the image and so) which is also useful for debugging15:24
dalvarezEmilienM, maybe there's already something in place?15:25
gchamoulflorianf, akrivoka: do you think it would be better to have those negative tests in tripleo-validations instead of oooq-extras?15:25
EmilienMdalvarez: not afik15:25
dalvarezack15:26
sshnaidm|roverdtantsur, containers should work now15:26
sshnaidm|roverdtantsur, tell me please if you see any problem again15:27
dtantsurneat! will recheck15:27
florianfgchamoul: If we had them in tripleo-validations it would be easier to ad tests for new validations within the same patch that actually adds the validation15:27
florianfgchamoul: So, yes, I think it would have some advantages15:27
EmilienMmwhahaha: promotion OK - one alert: assigned & WIP15:27
gchamoulflorianf: ok I need to think about how we can integrate them into the current way of the validations are working15:32
s_b_34343<EmilienM> : cuurently i'm not refereing anything doc . i have deployed controler ha for 3 controller . currently we are deploying our services on physical machine . now i wanted to deploy them on container like other services15:33
s_b_34343i'm very new to docker and container15:33
gchamoulflorianf: because that something we don't want to see through the tripleo-ui, I guess15:33
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: add nova and neutron tempest plugins to default config  https://review.openstack.org/52565715:33
*** anilvenkata has quit IRC15:34
*** anilvenkata has joined #tripleo15:34
s_b_34343<EmilienM>: even i wanted to know that how controller failover happens and the architecture also15:34
*** oidgar has quit IRC15:34
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: remove tempest plugin install from convert image  https://review.openstack.org/52565915:34
gchamoulflorianf, akrivoka: do you have time this week to talk a little bit about this?15:34
*** paramite has quit IRC15:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: add nova and neutron tempest plugins to default config  https://review.openstack.org/52565715:36
*** udesale has joined #tripleo15:36
*** dsariel has quit IRC15:40
*** ramishra has quit IRC15:40
s_b_34343<EmilienM> : can you point out to any document which i can refere15:41
*** udesale has quit IRC15:41
*** shardy_afk is now known as shardy15:42
*** udesale has joined #tripleo15:42
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: Document containet tags / delorean  https://review.openstack.org/52566715:45
EmilienMdalvarez: ^ draft15:45
dalvarezEmilienM, wow that was quick, thansks a lot  (s/containet/container) :)15:45
*** ihrachys has joined #tripleo15:45
ihrachysEmilienM, I saw you pinged me15:45
EmilienMyeah15:45
EmilienMihrachys: too late, we' closed the bug15:46
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: Document container tags / delorean  https://review.openstack.org/52566715:46
ihrachys:)15:46
ihrachysI am so sad about it /s15:46
ihrachysgood job :)15:46
EmilienMihrachys: don't worry I know where to find you :P15:47
sshnaidm|rovercores, please review/merge dns issues patch: https://review.openstack.org/#/c/525228/15:47
*** wojdec1 has joined #tripleo15:47
ihrachysI added the channel to auto-join now, so you can ping me here15:47
*** openstackgerrit has quit IRC15:48
*** morazi has quit IRC15:48
wojdec1Hey Folks… I'm trying to deploy tripleo using quickstart (master), in a proxied lab environment. Am consistently running into the following error:15:49
*** openstackgerrit has joined #tripleo15:49
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Fix stackviz data  https://review.openstack.org/52560315:49
wojdec1from the ansible log: FAILED! => {"changed": true, "cmd": "virt-customize  --run /home/stack/repo_setup.sh -a /home/stack/overcloud-full.qcow2 > /home/stack/_home_stack_repo_setup.sh.log 2>&1"15:50
florianfgchamoul: sure!15:51
wojdec1running that command from the shell on the machine, works ok. so it would appear to be something in the way the env is setup, and likely the http_proxy isn't used… Any hints? Is there perhaps some other channel for quickstart issues like these?15:52
*** ffiore has quit IRC15:53
*** dsariel has joined #tripleo15:53
*** nyechiel has quit IRC15:53
*** thrash is now known as thrash|biab15:54
mwhahahawojdec1: this is the right channel, i don't know if http_proxy support is fully developed for quickstart15:55
*** bogdando has quit IRC15:57
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: Use cachable facts to allow multiple runs  https://review.openstack.org/52567115:58
mwhahahawojdec1: you can file a bug against tripleo https://bugs.launchpad.net/tripleo/+filebug15:58
wojdec1mwhahaha: Ok. will do. tx15:58
*** ffiore has joined #tripleo15:58
EmilienMihrachys: thanks15:58
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update parameters updating  https://review.openstack.org/52567216:02
*** hamzy has quit IRC16:02
EmilienMmandre: please look https://review.openstack.org/#/c/525667/ when you can16:02
*** salmankhan has quit IRC16:03
*** gvrangan has joined #tripleo16:05
*** aufi has quit IRC16:05
*** ccamacho has quit IRC16:06
*** salmankhan has joined #tripleo16:06
s_b_34343<EmilienM> : can you help me?16:08
*** yprokule has quit IRC16:09
*** dbecker has quit IRC16:09
*** ffiore has quit IRC16:12
*** athomas has quit IRC16:12
EmilienMs_b_34343: sure, sorry I was afk a little16:12
EmilienMs_b_34343: https://docs.openstack.org/tripleo-docs/latest/ is an excellent start16:12
*** gbarros has quit IRC16:13
*** gbarros has joined #tripleo16:13
s_b_34343<EmilienM> : thanks EmilienM but i didnt find anything related controller ha16:15
openstackgerritMerged openstack/tripleo-quickstart master: Enable and document tags for all tasks  https://review.openstack.org/47611416:15
openstackgerritMerged openstack/tripleo-heat-templates master: Add NovaMigrationTarget service to SR-IOV Compute role  https://review.openstack.org/52034216:15
openstackgerritMerged openstack-infra/tripleo-ci master: Fix yum command for TLS job  https://review.openstack.org/52501816:15
openstackgerritMerged openstack/tripleo-validations master: Updated from global requirements  https://review.openstack.org/52541316:15
openstackgerritMerged openstack/tripleo-validations master: Change HAProxy validation queue timeout to match the default  https://review.openstack.org/52517316:15
openstackgerritMerged openstack/tripleo-validations master: Change controller ulimit to match the default  https://review.openstack.org/52517916:15
EmilienMs_b_34343: controllers are deployed with HA by default, if you follow the recommended architecture16:16
EmilienMdmsimard: FYI https://review.openstack.org/#/c/525265 worked fine16:16
openstackgerritAlex Schultz proposed openstack/tripleo-upgrade master: Setup tox  https://review.openstack.org/52567616:16
dmsimardEmilienM: you tested in different cases ?16:17
EmilienMdmsimard: yes16:17
EmilienMdmsimard: where in ara can I see the runtime for gitclones?16:18
EmilienMcouldn't find it http://logs.openstack.org/65/525265/5/check/tripleo-ci-centos-7-undercloud-oooq/448749e/logs/ara/16:18
mwhahahaEmilienM: isn't it this part http://logs.openstack.org/65/525265/5/check/tripleo-ci-centos-7-undercloud-oooq/448749e/job-output.txt.gz#_2017-12-05_05_57_01_14180916:19
s_b_34343<EmilienM> : what if opensatck is deployed with 3 controller ha but currenly we are deploying are services on physcical machine . but now we are having requiremnt that our services should be containerize ?16:19
s_b_34343how shoild i proceed for this ?16:19
EmilienMs_b_34343: yes, in queens we expect our users to deploy with containers16:19
EmilienMs_b_34343: I think the procedure is documented, isn't?16:19
*** hewbrocca is now known as hewbrocca_afk16:20
EmilienMmwhahaha: nice, it takes less than 1 min now16:20
EmilienMs_b_34343: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html16:20
s_b_34343<EmilienM> : really sorry for asking stuppid question but i really dont know havent read yet16:20
sri_s_b_34343, http://tripleo.org/install/containers_deployment/overcloud.html16:21
dmsimardEmilienM: that's the ara from devstack-gate16:22
dmsimardEmilienM: not the one from zuul16:22
EmilienMsshnaidm|rover: https://review.openstack.org/#/c/523903/17/zuul.d/ovb-jobs.yaml - does my comment make sense?16:22
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job  https://review.openstack.org/52390316:23
EmilienMsshnaidm|rover: I re-ran CI jobs since we got promotion, let's see how it works now16:23
dmsimardEmilienM: the one from zuul is at the top level16:23
EmilienMdmsimard: link?16:23
*** udesale has quit IRC16:23
dmsimardEmilienM: oh, but we don't generate ara reports for successful jobs right now http://logs.openstack.org/65/525265/5/check/tripleo-ci-centos-7-undercloud-oooq/448749e/16:23
s_b_34343thanks EmilienM , and sri_ . just wanted to know that in ha envirnment if any controller goes down then failover happens . but what about failback is triplo handing ?16:23
EmilienMdmsimard: why?16:23
dmsimardEmilienM: ara static reports are not big in size but they're a lot of small files and it's not very healthy for the log server (inodes) so we're transitioning to another way of providing reports that is much more lightweight https://ara.readthedocs.io/en/latest/advanced.html16:24
dmsimardwe'll be able to test it on logs-dev.o.o soon: https://review.openstack.org/#/q/topic:ara-sqlite-middleware16:25
EmilienMok16:25
EmilienMcool16:25
dmsimardso we will be able to provide reports all the time16:25
s_b_34343hi  EmilienM , and sri_ going offline16:25
*** ffiore has joined #tripleo16:25
sri_s_b_34343, using pacemaker and haprxoy , this hasn't change in container environment, the only thing changes we put servers in containers16:25
*** suuuper has joined #tripleo16:26
s_b_34343<sri_> are we using both pacemaker and haprxoy . i thought only pacemaker16:26
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update parameters updating  https://review.openstack.org/52567216:27
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Remove the name from task() functions that refer to themselves  https://review.openstack.org/52425016:27
beaglesmwhahaha, re: comment on https://review.openstack.org/#/c/487450 with respect to gnocchi integration - I wasn't aware that it did that. Can you point me in the general direction of what you are referring to?16:27
s_b_34343<sri_>: wiil connect aftr 1 hr will ask this Q again16:27
weshaymwhahaha, removed alert on https://bugs.launchpad.net/tripleo/+bug/173628416:27
openstackLaunchpad bug 1736284 in tripleo "Cannot commit transaction AddPortCommand exceptions cause tempest network_basic_ops to incur ssh timeouts" [Critical,Invalid]16:27
mwhahahabeagles: it doesn't pull existing ids but ceilometer does require that the services be up before it can complete.16:28
mwhahahabeagles: the point being is that durring the regular deployment there should be a point at which info can be pulled from other openstack services16:28
beaglesmwhahaha, ahhh16:29
sshnaidm|roverEmilienM, we got promotion right now16:29
beaglesmwhahaha, so like - we create the ports and then ask neutron for those ports?16:29
sri_s_b_34343, you have to use haproxy and pacemaker to do ha16:30
EmilienMsshnaidm|rover: I know, that's why I rebased it to re-kick jobs16:30
EmilienMsshnaidm|rover: but your -1, see my comment please.16:30
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: [WIP] Early release for fast forward upgrade testing.  https://review.openstack.org/52568016:31
mwhahahabeagles: we should be able to do that in step516:31
mwhahahabeagles: i think the issue was the ovs being up which we moved to step4 in containerized world16:31
beaglesmwhahaha, right with the ovs agent in step 4 that should do16:31
*** s_b_34343 has quit IRC16:31
*** ykarel__ has joined #tripleo16:32
beaglesmwhahaha, yup, I think the problem initially was that we had trouble running calls against the cloud from within puppet early on. The last I attempted was July I think and by then Or was off and running doing most of this in ansible16:32
*** marios has quit IRC16:33
beaglesmwhahaha, if the problem of running openstack calls against the cloud from within puppet during deployment is solid then this should be less of an issue16:33
sshnaidm|roverEmilienM, removed it16:34
flaper87shardy: jistr I hate my ISP! I'm sorry16:34
EmilienMsshnaidm|rover: thx, let's see how CI pass now16:34
mwhahahabeagles: it should be and if isn't then that's  a bug we need to fix and not try and workaround it16:34
*** ykarel_ has quit IRC16:34
beaglesmwhahaha, k I'll give it a go16:35
shardyflaper87: np! :)16:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: remove tempest plugin install from convert image  https://review.openstack.org/52565916:37
*** aditya_r has quit IRC16:37
*** hamzy_ has joined #tripleo16:37
*** oidgar has joined #tripleo16:38
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Migrate to the new Mistral context class  https://review.openstack.org/50618616:40
*** thrash|biab is now known as thrash16:40
openstackgerritLiz Blanchard proposed openstack/tripleo-ui master: Extending contribution docs with more detail  https://review.openstack.org/52568316:41
*** hamzy has joined #tripleo16:42
openstackgerritJames Slagle proposed openstack/tripleo-quickstart-extras master: Only create config-download.yaml for multinode  https://review.openstack.org/52568416:44
*** iranzo has quit IRC16:44
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: [WIP] Early release for fast forward upgrade testing.  https://review.openstack.org/52568016:44
slagleEmilienM: ^^16:45
*** dparkes has quit IRC16:45
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533116:45
*** hamzy_ has quit IRC16:46
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533116:46
sshnaidm|roverpabelanger, is it forbidden for you too? http://mirror.ord.rax.openstack.org:8080/rdo/centos7/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d/python2-oslo-service-1.26.0-0.20170921123949.1a973c4.el7.centos.noarch.rpm16:48
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart master: [WIP] New featureset for fast forward upgrade testing.  https://review.openstack.org/52568616:48
openstackgerritJames Slagle proposed openstack/tripleo-quickstart-extras master: Only create config-download.yaml for multinode  https://review.openstack.org/52568416:49
flaper87slagle: got 5 mins?16:50
flaper87slagle: wanted to chat real quick about the OOO patch16:50
EmilienMslagle: any reason to hold https://review.openstack.org/#/c/522344/ or should we move forward?16:52
flaper87slagle: fwiw, I fully agree with you that NM is a big red flag. As far as installing and configuring it goes, the steps that I currently have in my patch exist in the openshift-ansible master branch16:52
*** abregman|mtg has quit IRC16:52
flaper87slagle: we have two options to move this patch forward: 1) I remove the code and use openshift-ansible master or 2) We land it and wait for the release of openshift-ansible before changing that code16:53
flaper87slagle: ultimately, I'm also working on removing NM from the picture entirely16:53
flaper87it's currently a requirement for openshift but I think we might be able to do away with it16:53
flaper87thing is, it's currently not possible to disable NM in openshift-ansible and that's something that I'll bring up to the upstream community16:54
flaper87slagle: pls, let me know if any of the above makes sense :)16:54
*** hamzy has quit IRC16:54
slagleflaper87: it makes sense. what is the urgency to land it given where we are with NM?16:55
*** aputtur_ has joined #tripleo16:55
flaper87slagle: allow for working in parallel on other things like multi-node deployment16:55
flaper87slagle: more than an urgency it's a convenience16:55
flaper87jistr: is also helping with this and it's easier to iterate if we have it in-tree16:56
pabelangersshnaidm|rover: 404, does it exist on trunk.rdoproject.org?16:56
slagleflaper87: did you see my comments on the patch?16:56
flaper87slagle: mmh, I saw one from yday. Lemme check again16:56
flaper87:D16:56
slagleflaper87: it's teh same comment i left previously fwiw16:56
pabelangersshnaidm|rover: it is a reverse proxy cache, so any issues you see, are most likely from trunk.rdoproject.org. Unless we run out of HDDs16:57
slagleflaper87: a path forward could be a ci only template16:57
*** yprokule has joined #tripleo16:57
slagleflaper87: i'm 100% on board with workarounds / first steps in CI. and those templates are in tree16:57
slagleflaper87: the problem i have with this is living under extraconfig/services16:57
flaper87slagle: this can totally wait until after q-216:58
flaper87makes sense to me, I don't think there's any urgency other than convenience to work on the feature.16:58
EmilienMgchamoul: I replied on https://review.openstack.org/#/c/525637/ - maybe we can find a technical solution here.16:58
flaper87slagle: We can hold-off until after Q-2 and work on the NM in the meantime16:58
slagleflaper87: if we could somehow isolate the NM bits to a test or ci template, i'd be fine to have that land16:58
EmilienMgchamoul: how can we ignore the RAM test but validate others16:58
flaper87slagle: ++16:59
EmilienMgchamoul: I don't want to run tests if we ignore results, what's the point?16:59
flaper87slagle: I can do that, yes16:59
*** eck`gone is now known as eck`16:59
flaper87those bits should go away anyways so, I might as well take them out now and move them into a ci template16:59
slagleflaper87: i do wonder some about how what you're doing with installing openshift fits with the shiftonstack work17:00
slaglei thought they were also looking at openshift on undercloud managed nodes17:00
slagleas well as on overcloud managed nodes17:00
flaper87slagle: it integrates with that. The bits that I'm would like to use are the ones from shiftonstack which landed in the master branch recently17:01
flaper87the difference is that we're creating the nodes ourselves17:01
EmilienMshardy, slagle: any concensus was reached on next step for https://review.openstack.org/#/c/525637/1 ?17:01
slagleright17:01
EmilienMsorry https://review.openstack.org/#/c/522344/17:01
*** fragatina has joined #tripleo17:01
*** janki has quit IRC17:02
*** dbellant has quit IRC17:02
*** fragatin_ has joined #tripleo17:02
shardyEmilienM: I think if we're not moving to --config-download by default I'll need to rework it17:02
*** aditya_r has joined #tripleo17:02
shardyI'll mark it WIP and take a look tomorrow17:02
slagleshardy: your latest comment is fine with me17:03
*** derekh has quit IRC17:03
shardyOr we can land that and I'll do a follow-up17:03
slagleshardy: whichever you want :)17:03
slaglei added +2. you can land it if you want to17:03
shardyCI looks pretty green and I think jaosorior was waiting on this so perhaps we land it and I'll make the non-config-download path work as a follow up17:04
EmilienMshardy: ok, thx17:04
EmilienMshardy: let's go ahead then and land it17:04
shardyEmilienM: Ok sounds good, I'll work on a follow-up patch tomorrow17:05
shardythanks!17:05
EmilienMcool, good night!17:05
*** fragatina has quit IRC17:06
*** jpich has quit IRC17:06
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: remove tempest plugin install from convert image  https://review.openstack.org/52565917:09
beaglesEmilienM, can you point me at the types of stuff mwhahaha is referencing above with respect to gnocchi/ceilometer?17:10
beaglesEmilienM, I'm guessing there isn't a case where puppet making REST calls on other openstack components?17:10
mwhahahabeagles: talk to pradk.17:10
EmilienMbeagles: sure, give me a min17:10
beaglespradk ^17:10
mwhahahabeagles: ceilometer upgrade talks to gnocchi which talks to keystone over rest17:11
*** moshele has joined #tripleo17:11
*** gvrangan has quit IRC17:11
mwhahahabeagles: a better solution for Octavia would be to have a python app/script that does all the setup17:12
* beagles ponders that for a sec17:13
mwhahahaLike ceilometer-upgrade17:13
beaglesmwhahaha, that's packaged with ceilometer?17:14
mwhahahaYea17:14
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Add a new UndercloudHomeDir parameter  https://review.openstack.org/52440817:15
mwhahahaThe install/setup would be more user-friendly that way17:15
mwhahahaToo many manual steps17:15
*** jfrancoa has quit IRC17:16
openstackgerritIan Main proposed openstack/tripleo-quickstart-extras master: DNM: Update quickstart extras with undercloud install for containers  https://review.openstack.org/51744417:16
beaglesmwhahaha, no argument there - the octavia crowd are a little ... funny with some of this stuff I think17:16
*** moshele has quit IRC17:16
beaglesmwhahaha, like the whole port creation thing should really be handled by some kind of in-app layer the same way dhcp and l3 agent do in neutroy17:17
beaglesneutron17:17
mwhahahaYes17:17
beaglesmwhahaha, but I think nmagnezi's been fighting an uphill battle even on that17:17
beagles(cuz frankly that part is particularly insane)17:17
*** nyechiel has joined #tripleo17:18
*** mdnadeem has quit IRC17:20
openstackgerritIan Main proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744517:21
* beagles wonders if there is some kind of secret prize for services that are "most awkward or difficult" to configure/integrate that he doesn't know about 17:21
*** pcaruana has joined #tripleo17:22
openstackgerritIan Main proposed openstack/tripleo-quickstart-extras master: DNM: Update quickstart extras with undercloud install for containers  https://review.openstack.org/51744417:22
*** radeks has quit IRC17:23
*** BryanS68 has quit IRC17:23
weshaySlower, morning17:24
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Get rid of Undercloud* specific service entries in resource registry  https://review.openstack.org/52464417:25
*** pcaruana has quit IRC17:26
*** lucasagomes is now known as lucas-afk17:27
*** pcaruana has joined #tripleo17:27
*** dtantsur is now known as dtantsur|afk17:27
*** ffiore has quit IRC17:30
*** gkadam has quit IRC17:30
EmilienMSlower, weshay : what's the progress on idempotency failures?17:34
EmilienMon undercloud-container17:34
SlowerEmilienM: nothing at the moment17:35
EmilienMgchamoul: let me know what you think on https://review.openstack.org/#/c/525637/17:35
SlowerEmilienM: I'm working on other issues17:35
SlowerEmilienM: love it if youc ould look at this.  I did check that teh passwords were being saved/restored properly17:35
EmilienMSlower: ok, it sounds like it fails to access to mysql17:35
SlowerEmilienM: and that seems to work, so I'm not sure why we get a perm failed on mysql17:35
EmilienMSlower: could you reproduce locally?17:36
SlowerEmilienM: yep17:36
SlowerEmilienM: even doit will reproduce17:36
weshayahoy17:36
*** florianf is now known as florianf|afk17:37
Slowerweshay: !!17:38
weshaySlower, need anything from me buddy?17:38
Slowerweshay: good for now17:38
EmilienMSlower, weshay: do you have logs of the failure again? please17:40
EmilienMthe mysql thing17:40
*** yprokule_ has joined #tripleo17:40
*** yprokule has quit IRC17:40
*** yprokule_ is now known as yprokule17:40
weshayEmilienM, http://logs.openstack.org/18/518118/7/check/tripleo-ci-centos-7-undercloud-oooq/97451ab/logs/undercloud/home/zuul/undercloud_reinstall.log.txt.gz#_2017-12-04_22_40_4617:41
EmilienMthx17:42
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559417:44
*** fragatin_ has quit IRC17:44
openstackgerritMerged openstack/tripleo-heat-templates master: Update capabilities-map.yaml to include latest storage environments  https://review.openstack.org/52392717:45
*** BryanS68 has joined #tripleo17:45
*** sshnaidm|rover is now known as sshnaidm|off17:47
Slowerdoes anyone know where the docker registry is configured on the undercloud, for the overcloud?17:48
Slowerit looks like puppet tripleo is doing it, but I can'17:48
EmilienMyeah17:48
Slowercan't figure out where it's being called from17:48
EmilienMin quickstart I think17:48
EmilienMa sec17:48
weshayEmilienM, naw.. don't think so17:48
SlowerEmilienM: ya, where? :)17:48
weshaymaybe I'm wrong.. EmilienM is usually right17:49
weshayI had to tweet about it17:49
EmilienMhttps://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/config/release/tripleo-ci/consistent-master.yml#n417:49
EmilienMhere?17:49
EmilienMor https://review.openstack.org/#/c/520639/8/config/release/tripleo-ci/master.yml17:50
*** wojdec1 has left #tripleo17:50
EmilienMrlandy: just did that ^17:50
*** s_b_34343 has joined #tripleo17:50
EmilienMSlower: when you test idempotency, can you very that you have /root/.my.cnf in the mysql container? if yes, can you paste it please? or make sure creds are good?17:51
SlowerEmilienM: we need the one that sets up the local registry, eg configures /etc/docker-distribution/config.yml17:52
*** s_b_34343 has quit IRC17:55
*** gbarros has quit IRC17:56
shardySlower: https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/docker_registry.pp is the manifest which does it17:57
*** pcaruana has quit IRC17:58
*** itlinux has joined #tripleo17:58
itlinuxhello all.. I wonder if we have any docs or direction on how to turn all the sercurity group off.. I have a case where I need to do some testing. Thanks17:59
shardySlower: AFAICS we don't yet have a composable service template for it17:59
*** radeks has joined #tripleo17:59
Slowershardy: where is that called from when instack is the undercloud?18:00
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp#L55418:01
shardySlower:18:01
shardy^^18:01
*** trown is now known as trown|lunch18:02
dprinceshardy: yes, we've established that the profile is included by default. More perplexes is what is setting the Hiera variables for CI18:03
dprincefor the remote_registry, and the insecure_registries options18:04
dprinceyou can set those via hiera overrides18:04
dprincein the Overcloud we have proper parameters we can use for them in puppet/services/docker.yaml18:04
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L718:05
*** gfidente is now known as gfidente|afk18:06
shardydprince: I think we just default to hiera('controller_host') in the manifest18:06
shardyhttps://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/docker_registry.pp#L3918:06
shardys/manifest/class18:06
dprinceshardy: right. that is for controller host. But I don't see any references to the hiera settings for 'insecure_registries' or 'remote_mirror'18:07
*** mcornea has quit IRC18:07
shardydprince: do we need those with the prepare/upload workflow?18:07
openstackgerritMerged openstack/tripleo-heat-templates master: Add name property where missing  https://review.openstack.org/52199218:07
shardye.g it's not being configured as a pull through cache, we upload to it directly?18:07
*** dsariel has quit IRC18:08
*** morazi has joined #tripleo18:08
dprinceshardy: I see the defaults now.18:08
dprinceshardy: so I think we'll need a mix of these manifests on the Undercloud now. And the issue is probably that these manifests have duplicate resources18:08
dprinceshardy: so we need to resolve this18:09
dprincefor ecample both or configuring /etc/sysconfig/docker. That isn't good I think18:09
shardydprince: yea we'll need to fix that to allow a composable service template for sure18:09
Slowershardy: thanks!18:10
*** oidgar has quit IRC18:10
dprinceshardy: I'll work with Slower to try and decouple these things18:10
dprinceSlower: we need to use docker_registry.pp in addition to docker.pp18:10
shardydprince: ack sounds good, happy to help test/review etc18:11
*** s_b_34343 has joined #tripleo18:11
dprinceSlower: *but* before we can do this I think we have to untangle a few thanks18:11
Slowerdprince: ok18:11
dprincefew things18:11
*** ykarel__ has quit IRC18:11
SlowerI'll get back on bj18:11
dprinceSlower: so, I think we need to change instack. This will probably be at least 3 patches in a series18:11
*** paramite has joined #tripleo18:12
*** yprokule has quit IRC18:14
*** moshele has joined #tripleo18:14
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add multiple secret store backends for barbican  https://review.openstack.org/52353918:15
openstackgerritMatt Young proposed openstack/tripleo-quickstart master: Create an artifact for convert-image log  https://review.openstack.org/52572518:17
*** tesseract has quit IRC18:21
*** paramite has quit IRC18:22
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559418:23
trozetEmilienM, mwhahaha: can one of you take a look at this blueprint please? https://blueprints.launchpad.net/tripleo/+spec/opendaylight-ssl-support18:23
*** salmankhan has quit IRC18:24
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Introduce tripleo-bootstrap Ansible role  https://review.openstack.org/52528518:24
EmilienMslagle, sdoran: first step, https://review.openstack.org/525285 (see setup.cfg)18:25
EmilienMslagle, sdoran: second step: https://review.rdoproject.org/r/1080918:26
EmilienMat this stage, is won't be for Queens but for Rocky, ok?18:26
EmilienMtrozet: we don't create blueprints in second milestone for current cycle, at this stage it's for the next cycle.18:27
trozetEmilienM: need it for queens18:28
EmilienMtrozet: why didn't you create the blueprint on time?18:28
EmilienMtrozet: we sent so much emails about milestones and blueprints18:29
EmilienMtrozet: right now, we're in milestone 3, which means we're stabilizing what we started in milestone 1 and 2 and work on releasing Queens18:29
EmilienMtrozet: opendaylight-ssl-support is a new feature, that didn't start in milestone 1 and 2, so it will move to Rocky milestone 1.18:29
*** vpickard is now known as vpickard_18:30
EmilienMtrozet: unless we make an exception, but mwhahaha needs to give thought on that18:30
trozetEmilienM: ok, what is the MS3 deadline? january something?18:30
EmilienMreleases.openstack.org/queens/schedule.html18:30
EmilienMJan 22 - Jan 2618:31
trozetEmilienM: yeah so let me see if i can have all the patches posted soon18:31
EmilienMtrozet: we're trying to get better at planning18:31
trozetEmilienM: then might make an exception easier18:31
EmilienMtrozet: and posting blueprints in the middle of a cycle doesn't help in this effort18:31
EmilienMif we let everyone pushing blueprints in middle or end of cycles, it does nowhere18:32
*** tosky has quit IRC18:32
EmilienMgoes*18:32
EmilienMand we end up wtih broken CI at end of cycles like we had before18:32
rookshardy: hey18:32
trozetEmilienM: understood18:32
shardyrook: Hi!18:32
rookshardy: did you ping me earlier?18:32
EmilienMtrozet: next time, please schedule the blueprint before...18:32
rookI don't have enough scroll back18:32
shardyrook: I mentioned you as someone was asking about scale testing, sec18:33
shardyrook: it was radeks, asking about scale testing and if we've yet got results for pike based builds w/containers18:33
openstackgerritLiz Blanchard proposed openstack/tripleo-ui master: Extending contribution docs with more detail  https://review.openstack.org/52568318:35
dmsimardmwhahaha: can we merge https://review.openstack.org/#/c/525228/ ?18:35
dmsimardit'll likely fix a bunch of dns errors.18:35
*** jpena is now known as jpena|off18:35
rookshardy: yeah we do! but obviously neutron isn't containreized.18:36
rookcontainerized.18:36
rookThe scale is nothing impressive18:37
rookall ~100 nodes18:37
*** tongl has joined #tripleo18:37
rookshardy i shared results to rhos-dev like ~2 months ago18:37
shardyrook: Ok thanks for confirming!18:38
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update parameters updating  https://review.openstack.org/52567218:38
EmilienMcan someone approve https://review.openstack.org/#/c/525228/ please18:40
*** BryanS68 has quit IRC18:41
openstackgerritMerged openstack/tripleo-common master: Add the RootStackName parameter to the plan  https://review.openstack.org/52225218:41
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Make Kubespray install work without --private-key too  https://review.openstack.org/52552418:41
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: WIP Enable loadbalancer for kubernetes masters  https://review.openstack.org/51689418:41
*** dhill__ has quit IRC18:43
*** akrivoka has quit IRC18:43
*** dhill_ has joined #tripleo18:43
*** BryanS68 has joined #tripleo18:48
*** gvrangan has joined #tripleo18:49
*** eck` is now known as eck`gone18:49
openstackgerritAde Lee proposed openstack/tripleo-common master: Added ability to generate kek for barbican simple crypto backend  https://review.openstack.org/52533718:49
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Introduce tripleo-bootstrap Ansible role  https://review.openstack.org/52528518:49
*** pkovar has quit IRC18:52
*** suuuper has quit IRC18:57
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Introduce tripleo-bootstrap Ansible role  https://review.openstack.org/52528518:57
*** gvrangan has quit IRC18:57
*** shardy has quit IRC18:58
mwhahahaEmilienM, dmsimard approved19:00
dmsimardthanks19:01
openstackgerritMerged openstack/puppet-tripleo stable/pike: Remove erroneous octavia code related to sync_db  https://review.openstack.org/51337619:02
openstackgerritMerged openstack/tripleo-common master: Fix step_config output which is str not list  https://review.openstack.org/51013119:02
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Make CephPools override properties of other pools  https://review.openstack.org/52552019:02
*** iranzo has joined #tripleo19:03
*** strattao__ has joined #tripleo19:06
*** fragatina has joined #tripleo19:06
*** gbarros has joined #tripleo19:07
strattao__anyone in there that can help with a question about trying tripleo quickstart with a satellite server?19:09
*** Goneri has quit IRC19:10
*** hamzy has joined #tripleo19:10
*** abregman has joined #tripleo19:12
openstackgerritMerged openstack/tripleo-docs master: Document container tags / delorean  https://review.openstack.org/52566719:12
*** aditya_r has quit IRC19:14
*** trown|lunch is now known as trown19:14
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Reintroduce TripleoPackages in the default Controller service.  https://review.openstack.org/52425919:14
weshaymwhahaha, toure picked up https://bugs.launchpad.net/tripleo/+bug/173071219:14
openstackLaunchpad bug 1730712 in tripleo "tripleo should not query overcloud if never deployed to prevent extraneous error messages in swift/mistral" [Medium,Triaged] - Assigned to Toure Dunnon (toure)19:14
mwhahahak19:14
*** moshele has quit IRC19:15
openstackgerritMerged openstack/tripleo-quickstart master: Specify docker.io defaults for upstream config release files.  https://review.openstack.org/52063919:15
openstackgerritIan Main proposed openstack/tripleo-quickstart-extras master: DNM: Update quickstart extras with undercloud install for containers  https://review.openstack.org/51744419:16
openstackgerritIan Main proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers  https://review.openstack.org/51897219:16
*** eck`gone is now known as eck`19:16
openstackgerritIan Main proposed openstack/instack-undercloud master: DNM, undercloud containers TESTING ONLY  https://review.openstack.org/51811819:20
*** moshele has joined #tripleo19:22
*** tosky has joined #tripleo19:28
*** gbarros has quit IRC19:28
*** iranzo has quit IRC19:38
*** mcornea has joined #tripleo19:53
*** salmankhan has joined #tripleo19:54
*** Vijayendra has quit IRC19:56
*** salmankhan has quit IRC19:58
*** florianf|afk has quit IRC20:04
*** gkadam has joined #tripleo20:05
*** jrist has quit IRC20:05
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add parameters for Barbican keystone listener  https://review.openstack.org/52574120:06
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add parameters for Barbican worker image  https://review.openstack.org/52574220:06
*** florianf|afk has joined #tripleo20:08
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: WIP: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574320:10
openstackgerritHarald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754420:10
*** vpickard_ is now known as vpickard20:10
*** BryanS68 has quit IRC20:11
*** moshele has quit IRC20:11
*** BryanS68 has joined #tripleo20:15
*** strattao__ has quit IRC20:16
openstackgerritMerged openstack/tripleo-ui master: Extending contribution docs with more detail  https://review.openstack.org/52568320:26
*** rwsu has quit IRC20:31
*** rwsu has joined #tripleo20:31
EmilienMmwhahaha: ovb-ipv6 is ready https://review.openstack.org/#/c/523903/20:40
*** nyechiel has quit IRC20:40
*** Vijayendra has joined #tripleo20:41
*** gbarros has joined #tripleo20:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: zuul: update OVB jobs  https://review.openstack.org/52574620:42
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: zuul: update OVB jobs  https://review.openstack.org/52574720:44
openstackgerritEmilien Macchi proposed openstack/paunch master: zuul: add OVB ipv6 job  https://review.openstack.org/52574920:45
openstackgerritKeith Schincke proposed openstack/puppet-tripleo master: Update cephx keys with ACLs for openstack services.  https://review.openstack.org/51953120:45
openstackgerritEmilien Macchi proposed openstack/puppet-pacemaker master: zuul: add OVB ipv6 job  https://review.openstack.org/52575020:46
openstackgerritKeith Schincke proposed openstack/tripleo-heat-templates master: Add ceph-rbdmirror ansible container service  https://review.openstack.org/52024420:48
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: cistatus.html: update OVB jobs  https://review.openstack.org/52575120:50
*** mcornea has quit IRC20:50
*** dparkes has joined #tripleo20:51
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add -c for clean_templates  https://review.openstack.org/52575320:57
*** Goneri has joined #tripleo20:59
openstackgerritAlex Schultz proposed openstack/tripleo-upgrade master: Setup tox  https://review.openstack.org/52567621:01
mwhahahaEmilienM: so tripleo-upgrade has the docs job running on it but no docs/ folder. should we create some docs or disable the job21:05
EmilienMmwhahaha: it's exactly what I told them last week21:05
EmilienMbut nothing changed I guess21:05
mwhahahano i'm trying to fix it21:05
mwhahahaso which should we do?21:05
EmilienMdisable job21:06
EmilienMthey don't have doc21:06
mwhahahak21:07
mwhahahawhat about release notes21:07
mwhahahasame deal?21:07
openstackgerritDan Prince proposed openstack/instack-undercloud master: Use the docker profile to set INSECURE_REGISTRY  https://review.openstack.org/52575521:07
mwhahahai assume we might want to drop in the reno support21:07
EmilienMmwhahaha: for this repo?21:08
mwhahahayea21:08
EmilienMwell no21:08
EmilienMif we start something new, let's have something clean21:08
mwhahahaok so disable that job too21:08
EmilienMif we don't do it now21:08
EmilienMwe'll never do it.21:08
mwhahahaso add in reno support21:09
EmilienMcatching up release notes isn't fun, I did it for tripleo21:09
EmilienMyes21:09
mwhahahak21:09
*** hamzy has quit IRC21:12
openstackgerritDan Prince proposed openstack/puppet-tripleo master: Remove INSECURE_REGISTRY from docker_registry.pp  https://review.openstack.org/52576121:17
*** jtomasek has quit IRC21:18
openstackgerritDan Prince proposed openstack/instack-undercloud master: Use the docker profile to set INSECURE_REGISTRY  https://review.openstack.org/52575521:24
openstackgerritDan Prince proposed openstack/instack-undercloud master: Use the docker profile to set INSECURE_REGISTRY  https://review.openstack.org/52575521:25
openstackgerritDan Prince proposed openstack/puppet-tripleo master: Remove INSECURE_REGISTRY from docker_registry.pp  https://review.openstack.org/52576121:26
*** noslzzp has quit IRC21:28
*** gfidente|afk has quit IRC21:29
*** jkilpatr has quit IRC21:35
openstackgerritAlex Schultz proposed openstack/tripleo-upgrade master: Setup tox  https://review.openstack.org/52567621:36
*** gkadam has quit IRC21:38
weshaySlower, dprince should I run through and rebase all the deps for the container undercloud work?21:38
*** itlinux has quit IRC21:39
openstackgerritSteve Baker proposed openstack/tripleo-common master: Concurrent upload container images  https://review.openstack.org/52205521:39
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add DockerRegistryMirror parameter  https://review.openstack.org/52576421:39
dprinceweshay: sure21:39
dprinceweshay: I've got 2 more patches to subit21:40
dprincesubmit21:40
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744521:42
weshaydprince, Slower k.. it's all up2date21:43
*** itlinux has joined #tripleo21:44
*** jrist has joined #tripleo21:46
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Use ansible-playbook in place of quickstart.sh  https://review.openstack.org/52529421:46
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: WIP Allow for multiple playbooks to be run  https://review.openstack.org/52576521:46
*** ihrachys has quit IRC21:49
*** ihrachys has joined #tripleo21:49
openstackgerritMerged openstack/tripleo-validations stable/pike: Remove yum warning in ceph-ansible validation  https://review.openstack.org/52561821:49
openstackgerritMerged openstack/tripleo-heat-templates master: Add deploy_steps_tasks interface  https://review.openstack.org/52234421:49
openstackgerritMerged openstack-infra/tripleo-ci master: Remove debug config of unbound since it's enabled  https://review.openstack.org/52522821:49
*** radeks has quit IRC21:50
openstackgerritJames Slagle proposed openstack/tripleo-docs master: Document manual config-download  https://review.openstack.org/52443721:50
*** trown is now known as trown|outtypewww21:50
openstackgerritMerged openstack-infra/tripleo-ci master: zuul/tripleo-ci-dsvm: remove required-projects  https://review.openstack.org/52526521:50
openstackgerritMerged openstack/tripleo-heat-templates master: Add noop's for docker pacemaker for rabbitmq  https://review.openstack.org/52475121:51
*** pchavva has quit IRC21:51
*** ihrachys has quit IRC21:54
*** abregman has quit IRC21:54
*** ihrachys has joined #tripleo21:54
openstackgerritDan Prince proposed openstack/puppet-tripleo master: Remove INSECURE_REGISTRY from docker_registry.pp  https://review.openstack.org/52576121:55
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Migrate to the new Mistral context class  https://review.openstack.org/50618621:58
*** rhallisey has quit IRC21:59
*** BryanS68 has quit IRC22:01
*** rbowen has quit IRC22:03
*** zshi has quit IRC22:03
*** threestrands has joined #tripleo22:05
*** threestrands has quit IRC22:05
*** threestrands has joined #tripleo22:05
*** rcernin has joined #tripleo22:05
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Add step config to OPTIONAL_DOCKER_SECTIONS  https://review.openstack.org/52577622:08
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Add step config to OPTIONAL_DOCKER_SECTIONS  https://review.openstack.org/52577622:10
*** iranzo|AFK is now known as iranzo22:11
*** iranzo is now known as iranzo|AFK22:12
*** fragatina has quit IRC22:13
*** fragatina has joined #tripleo22:17
*** fragatin_ has joined #tripleo22:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: WIP: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574322:18
*** dprince has quit IRC22:18
*** yamahata has joined #tripleo22:19
*** fragatina has quit IRC22:21
*** raildo has quit IRC22:21
*** nyechiel has joined #tripleo22:22
openstackgerritAlex Schultz proposed openstack/tripleo-upgrade master: Setup tox  https://review.openstack.org/52567622:23
*** eck` is now known as eck`gone22:24
openstackgerritAlex Schultz proposed openstack/tripleo-upgrade master: Setup tox  https://review.openstack.org/52567622:29
*** jmelvin has quit IRC22:41
openstackgerritMerged openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job  https://review.openstack.org/52390322:43
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: Remove default for --overcloud-ssh-key  https://review.openstack.org/52577922:46
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533122:46
*** yamahata has quit IRC22:49
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Include tripleo-bootstrap role  https://review.openstack.org/52578022:57
openstackgerritJames Slagle proposed openstack/tripleo-common master: Remove jq package install  https://review.openstack.org/52578122:58
slagleEmilienM: ^^ those are the next steps to consuming the role.22:59
openstackgerritK Jonathan Harker proposed openstack/diskimage-builder master: zypper: fix package removal  https://review.openstack.org/51765822:59
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Include tripleo-bootstrap role  https://review.openstack.org/52578023:00
*** yamahata has joined #tripleo23:03
*** bfournie has quit IRC23:12
*** yamahata has quit IRC23:13
*** abishop has quit IRC23:17
*** aputtur_ has quit IRC23:27
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559423:31
*** tongl has quit IRC23:41
weshaymyoung|ruck, fyi.. it worked the second time.. and it promoted23:41
*** yamahata has joined #tripleo23:43
*** tongl has joined #tripleo23:49

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!