Thursday, 2019-03-14

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722400:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429300:00
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309500:01
*** wolverineav has quit IRC00:02
EmilienMweshay: weird00:03
weshaynot sure if that's avail in rdo00:03
EmilienMhttps://logs.rdoproject.org/95/643095/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/520592b/logs/undercloud/var/log/00:03
weshayI looked at the ovb jobs in the review and they did not trigger w/ the change00:03
EmilienMwhere is tripleo image prepare log00:03
weshay2019-03-13 09:43:48.045776 | primary | Looking in indexes: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/pypi/simple, http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/wheel/centos-7.6.1810-x86_6400:04
weshayhttp://logs.rdoproject.org/08/642808/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/9f07d25/job-output.txt.gz00:04
* weshay looks at your log00:04
EmilienMwait00:04
EmilienM2019-03-13 23:21:02 | HTTPError: 500 Server Error: Server Error for url: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/00:05
EmilienMit's wrong00:05
EmilienMthe v2 should be there00:05
weshay2019-03-13 23:21:02.039 16233 ERROR tripleoclient.v1.tripleo_deploy.Deploy [  ] Exception: 500 Server Error: Server Error for url: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/00:05
weshay2019-03-13 23:21:02.042 16233 ERROR tripleoclient.v1.tripleo_deploy.Deploy [  ] None00:05
weshayhttps://logs.rdoproject.org/95/643095/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/520592b/logs/undercloud/home/zuul/install-undercloud.log.txt.gz00:05
EmilienMit's without /v200:05
weshayso ovb failed in undercloud install00:06
EmilienMdocker_registry_mirror = http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/00:06
weshayso there is no log00:06
EmilienMwhat the00:06
EmilienMlet me dig00:06
weshayhttps://review.openstack.org/#/c/642808/3/scripts/common_vars.bash00:06
EmilienMstevebaker: ^ if you're around00:06
weshayEmilienM so I need one more change00:06
EmilienMweshay: no no00:06
EmilienMI'm talking about a v200:06
weshayexport NODEPOOL_DOCKER_REGISTRY_V2_PROXY=${NODEPOOL_DOCKER_REGISTRY_V2_PROXY:-""}00:07
weshayif it's ovb we should set that back00:07
EmilienMno...00:07
weshayk00:07
EmilienMthe bug is in tripleo-common00:07
weshayk00:07
EmilienMsomething adds v200:07
EmilienMlet me find what00:07
EmilienMmhh sounds like we force v2 in _build_url method00:08
EmilienMand it works on multinode?00:08
weshayseems to00:09
* weshay looks again at the job00:09
EmilienMI think I know00:09
EmilienMwe might need to force the mirror to be in docker_insecure_registries00:10
weshay2019-03-13 12:39:03.462 20786 DEBUG tripleo_common.image.image_uploader [  ] http://mirror.ord.rax.openstack.org:8082/v2/ status code 40100:10
*** slaweq has joined #tripleo00:11
weshayhttp://mirror.ord.rax.openstack.org:8082 "GET /cloudflare/registry-v2/docker/registry/v2/blobs/sha256/dd/dd3ac262a99ad0d7e9ea2504e86d3a242e8dd149739daa60c6af56432398ac40/data?verify=1552483745-maJDyCCr1sCRpbzYRVTCI7ULWgQ%3D HTTP/1.1" 200 2627300:11
EmilienMwhere did you see that?00:11
weshayhttp://logs.openstack.org/08/642808/3/check/tripleo-ci-centos-7-containers-multinode/d75b478/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz00:11
EmilienMthe 40100:11
weshayya.. in that log00:12
EmilienMok00:12
EmilienMlet me look00:12
EmilienMurllib3.connectionpool [  ] http://mirror.ord.rax.openstack.org:8082 "GET /v2/tripleomaster/centos-binary-tempest/manifests/current-tripleo HTTP/1.1" 200 616200:13
EmilienMthe GET /v2/... works fine00:13
EmilienMon rackspace00:14
EmilienMbut not in RDO00:14
EmilienMlet me see how they exposed the mirror in RDO00:14
*** mpjetta has quit IRC00:14
weshayEmilienM aye00:15
weshayI asked in sf-ops00:15
openstackgerritMerged openstack/tripleo-common master: Add ability to run osa playbooks from tripleo-deploy-openshift  https://review.openstack.org/62896000:15
openstackgerritMerged openstack/tripleo-common master: Pass additional args to tripleo-deploy-openshift as ansible options  https://review.openstack.org/62896100:15
openstackgerritMerged openstack/tripleo-common master: Option to run osa playbooks from path  https://review.openstack.org/62896200:15
openstackgerritMerged openstack/tripleo-common master: Switch to podman for tripleo-deploy-openshift  https://review.openstack.org/62849800:15
openstackgerritMerged openstack/tripleo-heat-templates master: Manage system timezone with ansible  https://review.openstack.org/64258900:15
openstackgerritMerged openstack/python-tripleoclient stable/queens: Use sync action get_deployment_failures  https://review.openstack.org/64291900:15
weshay<pabelanger>  weshay: you need to drop /v2/ after :808200:15
weshay18:11 <pabelanger>  oh, it is also possible rdo doesn't have :8082 setup00:15
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Reload rsyslog/cron when we change timezones  https://review.openstack.org/64306200:15
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Reload rsyslog/cron when we change timezones  https://review.openstack.org/64307200:15
weshayEmilienM ^00:15
openstackgerritMerged openstack/tripleo-quickstart-extras master: update reproducer doc w/ command options  https://review.openstack.org/64286600:15
EmilienMdamn the gate is on fire00:15
weshayMERGE PATY00:15
weshayPARTY00:15
*** slaweq has quit IRC00:15
EmilienMf*** yeah00:16
EmilienManyway00:16
weshayMERICA00:16
EmilienMit's on RDO side00:16
EmilienMLOL00:16
EmilienMlet me check what jpena|off did00:16
weshayEmilienM where?00:16
EmilienMI can't find his patch00:16
EmilienMnot sure00:16
* weshay looks00:16
weshayI don't always test my code, but when I do.. I do it on rdo00:16
EmilienMyour "code"00:17
EmilienMhttps://review.rdoproject.org/r/#/q/status:merged00:17
EmilienMI can't find anything tbh00:17
EmilienMlet me check again00:17
EmilienMhttps://github.com/rdo-infra/rdo-jobs/commit/4b51a6a12735effa1e9e19ec177b36c490754a9700:18
weshayEmilienM you might have to be Andy https://goo.gl/images/MUUCoU00:19
EmilienMexport NODEPOOL_DOCKER_REGISTRY_V2_PROXY=${NODEPOOL_DOCKER_REGISTRY_V2_PROXY:-http://$NODEPOOL_MIRROR_HOST:8082/00:20
openstackgerritMerged openstack/puppet-tripleo stable/queens: Adding a udev rule to save sriov interface name before configuring switchdev  https://review.openstack.org/64160000:20
EmilienMbut I don't get it00:20
EmilienMin udnercloud.conf it's good00:20
EmilienMdocker_registry_mirror = http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/00:20
EmilienMi hate computers00:21
weshay<pabelanger>  I'd check to see what is passing /v2/, it might be docker directly.00:21
weshay18:19 <pabelanger>  however, I cannot remember right now if that is correct or not00:21
weshay18:19 <pabelanger>  but, agree something is going on with proxy00:21
weshayEmilienM so this shouldn't be docker_registry_mirror = http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v200:22
weshay?00:22
*** xek has joined #tripleo00:23
EmilienMno00:23
EmilienMI don't think so00:24
* weshay sees if I have a node in rdo00:24
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: flatten the ovn service configurations  https://review.openstack.org/63911900:24
stevebakerweshay, EmilienM: I've not seen registry references include /v2 before, because the associated image references don't include it. Image ref http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/foo/bar will hit http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/foo/bar/manifests00:26
weshayso what is a test url I could curl?00:28
*** wolverineav has joined #tripleo00:28
EmilienMsorry I give up for the day00:29
weshayhttp://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/tripleomaster/centos-binary-heat-api-cfn/manifests/current-tripleo00:29
weshayI just get 500's00:29
weshay curl http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org00:29
weshayworks00:29
stevebakerweshay: actually mirrors don't provide the whole registry API, only blob/layer requests. let me get you a layer curl00:32
*** wolverineav has quit IRC00:35
stevebakerweshay: I think this should work. It does not http://paste.openstack.org/show/747756/00:36
weshayya.. get the same00:37
* weshay passes to paul00:37
*** holser_ has quit IRC00:56
*** rlandy|bbl is now known as rlandy01:00
*** mpjetta has joined #tripleo01:01
stevebakerweshay: ignore what I said about mirrors only handling layer blobs01:03
*** tzumainn_ has joined #tripleo01:05
*** tzumainn has quit IRC01:05
mwhahahaweshay: http://logs.openstack.org/44/637144/3/check/tripleo-ci-centos-7-scenario001-standalone/e8faa5e/job-output.txt.gz#_2019-03-13_22_09_14_300232 linting is failing on some standalone for scenario001/004. why are we running that on there? i thought it was supposed to only be the upgrade jobs01:07
mwhahahai bet it's the ceph playbooks breaking stuff01:07
mwhahahaclearly we have a gap somewhere if that got broken01:08
mwhahahaalso the logging on that is terrible01:10
*** rascasoft has joined #tripleo01:12
mwhahahaweshay, panda|rover|off: bug/181999101:14
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart master: Revert "Activate ansible lint after deployment."  https://review.openstack.org/64321201:14
*** mwhahaha changes topic to "CI Status: YELLOW (scenario001/004 broke), RDOCloud questionable | https://docs.openstack.org/tripleo-docs/latest/"01:15
* mwhahaha flips tables and gives up01:15
*** rascasoft has quit IRC01:19
stevebakermwhahaha++01:21
*** rlandy has quit IRC01:38
openstackgerritLars Kellogg-Stedman proposed openstack/puppet-tripleo master: add glue for nova serialproxy support  https://review.openstack.org/64310301:40
openstackgerritMerged openstack-infra/tripleo-ci master: fix typo in doc link rows  https://review.openstack.org/64315701:44
*** wolverineav has joined #tripleo02:10
*** wolverineav has quit IRC02:15
*** rascasoft has joined #tripleo02:32
*** rascasoft has quit IRC02:42
*** tzumainn_ has quit IRC02:49
*** psachin has joined #tripleo03:01
*** ykarel|away has joined #tripleo03:04
*** slaweq has joined #tripleo03:11
*** slaweq has quit IRC03:15
*** ykarel|away has quit IRC03:22
*** ramishra has joined #tripleo03:29
*** skramaja has joined #tripleo03:33
*** rascasoft has joined #tripleo03:34
*** ayoung has joined #tripleo03:39
*** rascasoft has quit IRC03:40
*** slaweq has joined #tripleo04:11
*** slaweq has quit IRC04:15
*** ramishra has quit IRC04:21
*** ramishra has joined #tripleo04:21
*** udesale has joined #tripleo04:22
*** ykarel|away has joined #tripleo04:24
*** ykarel|away is now known as ykarel04:24
*** ramishra_ has joined #tripleo04:25
*** ramishra has quit IRC04:28
*** ramishra has joined #tripleo04:29
*** ramishra_ has quit IRC04:30
*** janki has joined #tripleo04:35
*** rascasoft has joined #tripleo04:38
*** rascasoft has quit IRC04:48
*** saneax has joined #tripleo04:57
*** rascasoft has joined #tripleo05:17
*** rascasoft has quit IRC05:28
openstackgerritKarthik S proposed openstack/os-net-config master: Adding pci address in SRIOV config  https://review.openstack.org/62045605:43
openstackgerritKarthik S proposed openstack/os-net-config master: WIP: Removing the warnings generated due to yaml.load()  https://review.openstack.org/64323005:43
*** raukadah is now known as chandankumar05:50
openstackgerritJaganathan Palanisamy proposed openstack/os-net-config master: Adding pci address in SRIOV config  https://review.openstack.org/62045605:58
*** jbadiapa has quit IRC06:03
*** jbadiapa has joined #tripleo06:04
*** slaweq has joined #tripleo06:11
*** jtomasek has joined #tripleo06:15
*** slaweq has quit IRC06:16
*** wolverineav has joined #tripleo06:18
*** slaweq has joined #tripleo06:19
*** marios_|ruck has joined #tripleo06:20
*** rascasoft has joined #tripleo06:27
*** rascasoft has quit IRC06:31
Tenguhello there06:38
*** bandini has joined #tripleo06:40
bandinio/06:40
bandiniTengu: I added a comment to https://bugs.launchpad.net/tripleo/+bug/1819459 , see if I am making any sense at all ;)06:40
openstackLaunchpad bug 1819459 in tripleo "Swift service tries to relabel files" [High,Triaged]06:40
Tengubandini: lemme check that.06:41
Tengubandini: hey, that's exactly that! Lemme check some cp options.06:42
Tengubandini: yeah, ok, wait. guess we can drop the "-a" and replace it by a couple of options.06:42
*** wolverin_ has joined #tripleo06:42
Tengu"-a" is a shortcut for -dR --preserve=all06:43
Tenguwe can maybe add --no-preserve=context06:43
bandiniit has potential :)06:43
Tengulemme try that, I just booted my Beast for some deploy tests :)06:43
*** shardy has quit IRC06:44
*** yprokule has joined #tripleo06:44
bandiniTHE BEAST!06:44
TenguYEAAAHHH06:44
Tengumy tiny shiny builder :)06:44
*** wolverineav has quit IRC06:45
*** wolverin_ has quit IRC06:49
Tenguhm. I should have started a centos build -.-. rhel is slow due to the proxy thingy I have to use... pfff.06:51
Tenguon the bright side: my proxy thingy is working really well.06:51
*** shardy has joined #tripleo06:52
bandini:)06:53
jaosoriorHas anybody seen this?06:57
jaosoriorWorkflowServiceError: Exception updating plan: {u'result': u'Failed to run task [error=Invalid input [name=tripleo.parameters.update, class=tripleo_common.actions.parameters.UpdateParametersAction, unexpected=[u\'validate\']], wf=tripleo.swift_backup.v1.create_swift_backup_container_plan, task=set_tempurl]:\nTraceback (most recent call last):\n06:57
jaosoriorI'm getting those errors while trying to deploy the overcloud with oooq, using master-tripleo-ci06:57
*** pdeore has joined #tripleo06:57
Tenguhmmm nope. not yet.06:58
bandininope06:58
Tengulet's go for the deploy :]06:59
*** kopecmartin|off is now known as kopecmartin07:00
Tenguoh, ntw. Happy Pi Day !07:01
Tengus/ntw/btw/07:02
TenguI meant : oh, btw. Happy Pi Day !07:02
jaosorior:D07:04
bandinioh right Pi day07:07
*** rascasoft has joined #tripleo07:09
ramishrajaosorior: https://bugs.launchpad.net/tripleo/+bug/1818926/comments/607:14
openstackLaunchpad bug 1818926 in tripleo "deploy fails with Invalid input [name=tripleo.parameters.update, class=tripleo_common.actions.parameters.UpdateParametersAction, unexpected=[\'validate\']" [High,Fix released] - Assigned to Rabi Mishra (rabi)07:14
*** ksambor has joined #tripleo07:16
*** numans has joined #tripleo07:20
*** tosky has joined #tripleo07:23
*** wolverineav has joined #tripleo07:27
*** mschuppert has joined #tripleo07:29
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/rocky: Add tempest workspace volume for tempest  https://review.openstack.org/64323707:30
*** wolverineav has quit IRC07:31
chandankumararxcruz: https://review.openstack.org/64323707:31
*** aufi has joined #tripleo07:32
Tengubandini: ok, --no-preserve doesn't help.07:33
bandinibummer07:33
Tenguso replacing -a with "-dR --preserve"07:34
Tenguthe -a includes "--preserve=all"07:34
Tengu--preserve will keep the mode,ownership,timestamps07:34
Tenguwhich should be sufficient07:34
*** jfrancoa has joined #tripleo07:34
*** paramite has joined #tripleo07:40
*** aedc has quit IRC07:43
Tenguapparently, that's the right change. Will push a patch shortly.07:44
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Avoid "-a" cp option in order to avoid SELinux AVC  https://review.openstack.org/64324007:48
Tengubandini: -^^07:48
Tenguoh. shoo...07:48
Tenguforget to link the LP07:48
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Avoid "-a" cp option in order to avoid SELinux AVC  https://review.openstack.org/64324007:48
Tengubetter.07:48
bandiniack nice, will try it today or tomorrow07:49
bandiniright now am team-tagging network isolation with luca07:49
*** lmiccini has joined #tripleo07:49
Tengubandini: np - it's not a blocker per se, since it doesn't crash the deploy ;).07:50
*** gkadam_ has joined #tripleo07:50
Tengusame goes for the "sudo" trying to connect to dbus.07:50
bandinieheh permissive ftw :P07:50
* Tengu kills bandini with a rotten christmas tree07:50
bandiniahahah07:50
Tenguthe random-slap is always a surprise :D07:51
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates master: Check and enable openvswitch package  https://review.openstack.org/61903107:51
Tengudamn... yeah. so we really have some issues with dbus accesses from within containers.07:53
Tenguthat's bad.07:53
TenguAND some read into a tmpfs...07:53
openstackgerritMerged openstack/tripleo-heat-templates master: Request certificate for using host service principals  https://review.openstack.org/64081307:54
Tengujaosorior: heya! care to have a quick look at https://bugs.launchpad.net/tripleo/+bug/1820019 ?07:57
openstackLaunchpad bug 1820019 in tripleo "AVC reported for tmpfs accesses" [Medium,Triaged] - Assigned to Cédric Jeanneret (cjeanner)07:57
Tengubandini: -^^07:58
Tenguhmmm. allow init_t session_dbusd_tmp_t:dir read;07:59
Tengudoesn't seem that harmful?07:59
jaosoriorsounds quite strange as well07:59
jaosoriorTengu: sounds like that should go to the systemd folks07:59
jaosoriorand not necessarily an openstack issue08:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722408:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
Tengujaosorior: right. apparently this has nothing to do with containerized things08:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429308:00
Tengujaosorior: will open a BZ against systemd.08:00
*** holser_ has joined #tripleo08:03
*** aissat-mus has joined #tripleo08:05
Tengujaosorior: https://bugzilla.redhat.com/show_bug.cgi?id=168867108:06
openstackbugzilla.redhat.com bug 1688671 in systemd "SELinux is preventing init_t to read session_dbusd_tmp_t directories" [Medium,New] - Assigned to systemd-maint08:06
*** pcaruana has joined #tripleo08:07
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates master: Check and enable openvswitch package  https://review.openstack.org/61903108:09
*** hberaud|gone is now known as hberaud08:09
openstackgerritKamil Sambor proposed openstack/tripleo-quickstart master: Remove openvswitch tasks  https://review.openstack.org/61919608:10
aissat-musHi all, I'm trying to install overcloud on a production environment, 3 controller, 3 compute , 3 ceph storage. When deploying Overlcoud I keep getting :08:11
aissat-musResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"08:11
*** pcaruana has quit IRC08:11
Tengugosh... the healthchecks are really, really verbose :/08:12
Tengubandini: Mar 14 08:12:20 undercloud.localdomain podman[104121]: There is no ovn-northd process connected to socket sockets running in the container08:12
Tengubandini: already known? -^^08:13
Tenguwondering if it's not linked to the AVC...08:13
*** tkajinam__ has quit IRC08:13
bandiniTengu: +100 re healtchchecks08:14
Tengu^^'08:15
Tengutime to make some tea.08:15
bandiniTengu: nope but do you have all the ovn northd things running? I thought the undercloud does not use ovn yet08:15
Tengubandini: standalone08:15
Tengubandini: that's the healthcheck for the service apparentls08:15
Tengu*apparently08:15
Tenguhmm. and apparently, cinder healthcheck has some issues with rootwrap.08:16
bandiniah08:16
bandiniah the healthecks are largely broken08:16
bandinihave not tackled that08:16
openstackgerrityatin proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309508:16
Tenguyeah.... it's a pity.08:16
Tenguwill try to have a look.08:16
Tengubut first: teapot needs some filling.08:17
*** holser_ has quit IRC08:17
*** ccamacho has joined #tripleo08:20
openstackgerritMartin Schuppert proposed openstack/tripleo-docs master: [Documentation] Document procedure to manage additional nova cells v2  https://review.openstack.org/63628608:20
*** amoralej|off is now known as amoralej08:21
*** holser_ has joined #tripleo08:22
*** pcaruana has joined #tripleo08:23
Tenguhmm. I don't have any new AVC... something stopped trying to talk to dbus/tmpfs....08:24
Tengubandini: btw: tripleo_ovn_northd.service                                                                                                           loaded active running   ovn_northd container08:24
bandiniack08:25
Tengumeaning: it's the healthcheck shitting itself :D08:27
*** bogdando has joined #tripleo08:34
*** hjensas has quit IRC08:35
*** cylopez has joined #tripleo08:36
openstackgerritJaganathan Palanisamy proposed openstack/os-net-config master: Adding pci address in SRIOV config  https://review.openstack.org/62045608:43
openstackgerritFilippo Inzaghi proposed openstack/tripleo-heat-templates master: fix tox python3 overrides  https://review.openstack.org/60823908:51
*** slaweq has quit IRC08:53
*** slaweq has joined #tripleo08:55
openstackgerritMichele Baldessari proposed openstack/python-tripleoclient master: WIP Force list when returning map(replaced_list_value, template_part)  https://review.openstack.org/64309108:55
*** dtantsur|afk is now known as dtantsur08:59
*** jpena|off is now known as jpena09:00
*** aissat-mus has quit IRC09:03
*** panda|rover|off has quit IRC09:07
*** marios_|ruck has quit IRC09:09
*** marios_|ruck has joined #tripleo09:09
*** ykarel is now known as ykarel|lunch09:09
openstackgerritBogdan Dobrelya proposed openstack/tripleo-common master: [py3] Dereference maps to become real lists.  https://review.openstack.org/64325009:11
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309509:11
*** chem has joined #tripleo09:15
*** dmacpher has joined #tripleo09:16
*** morazi has joined #tripleo09:16
*** aedc has joined #tripleo09:18
*** jpich has joined #tripleo09:24
*** sshnaidm is now known as sshnaidm|rover09:25
openstackgerritSagi Shnaidman proposed openstack/tripleo-heat-templates stable/rocky: DNM: testing 039 job in rocky  https://review.openstack.org/64325909:36
sshnaidm|roverchandankumar, ^^09:36
sshnaidm|roverchandankumar, it passes on rocky, so let's see how tempest works there09:36
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: [DNM] Check scenario009  https://review.openstack.org/64326109:36
chandankumarsshnaidm|rover: cool!09:41
*** pcaruana has quit IRC09:48
*** ykarel|lunch has quit IRC09:52
*** cylopez has left #tripleo09:52
*** hjensas has joined #tripleo09:57
*** ykarel has joined #tripleo09:57
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix containerized tempest logic  https://review.openstack.org/64242010:07
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Make standalone job run containerized tempest  https://review.openstack.org/64326710:12
*** rpittau|afk is now known as rpittau10:13
*** saneax has quit IRC10:21
*** sri_ has joined #tripleo10:26
openstackgerritMichele Baldessari proposed openstack/python-tripleoclient master: Force list when returning map(replaced_list_value, template_part)  https://review.openstack.org/64309110:46
*** wolverineav has joined #tripleo10:49
*** wolverineav has quit IRC10:54
*** udesale has quit IRC10:56
*** udesale has joined #tripleo10:56
*** panda has joined #tripleo10:58
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Introduce a --plan option to replace --config-download-dir  https://review.openstack.org/64328710:59
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Add ability to run osa playbooks from tripleo-deploy-openshift  https://review.openstack.org/64328810:59
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Pass additional args to tripleo-deploy-openshift as ansible options  https://review.openstack.org/64328910:59
openstackgerritMartin André proposed openstack/tripleo-common stable/rocky: Option to run osa playbooks from path  https://review.openstack.org/64329010:59
pandamandre: with openstack tripleo container image prepare  what's the current way to specify a specify tag for the images ?11:01
*** jpich has quit IRC11:01
*** jpich has joined #tripleo11:02
*** yolanda has quit IRC11:02
*** morazi has quit IRC11:03
mandrepanda: you can add "--set tag=blah" to your prepare command11:03
*** yolanda has joined #tripleo11:03
mandrepanda: see https://github.com/openstack/tripleo-common/blob/master/container-images/container_image_prepare_defaults.yaml for the default values11:04
pandamandre: ack thanks11:04
*** suuuper has joined #tripleo11:05
mandrepanda: the "tag" here represents the tag of openstack images in the remote registry, you also have a way to control the tag of the images after importing them in your local registry11:05
pandamandre: this is done so for example I could pull two sets of containers, calling one "current" and the other "upgrades" ?11:07
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309511:09
*** shardy has quit IRC11:11
mandrepanda: nope, this is done because we want tags that uniquely identify images in the local registry (for update/upgrade purpose), and as such can't be generic tags such as "latest" or "tripleo-current"11:14
*** udesale has quit IRC11:15
*** udesale has joined #tripleo11:15
*** zbr has quit IRC11:16
pandamandre: that's basically what I tried to say before, probably failing11:18
pandamandre: does --set work only for openstack overcloud, or also for openstack tripleo ?11:19
pandammhh, does opesntack overcloud even work on master ?11:19
mandrepanda: should work for both11:19
pandamandre: ok thanks.11:20
ykarelmandre, but i can't see --set in https://github.com/openstack/python-tripleoclient/blob/969e01ad8820f516362ad730a0e7f3fce07c35a6/tripleoclient/v1/container_image.py#L606-L66811:21
ykarelis it inherited from some place?11:21
mandreykarel: yeah I think it's inherited, let me see11:22
mandreykarel: this should be https://github.com/openstack/python-tripleoclient/blob/969e01ad8820f516362ad730a0e7f3fce07c35a6/tripleoclient/v1/container_image.py#L322-L32811:26
ykarelmandre, but that's different command11:26
*** saneax has joined #tripleo11:27
ykarelmandre, ^^ overcloud_container_image_prepare11:27
*** udesale has quit IRC11:27
*** aedc has quit IRC11:27
ykarelovercloud_container_image_prepare vs tripleo_container_image_prepare11:27
*** udesale has joined #tripleo11:28
mandreykarel: seems like you're right tripleo_container_image_prepare doesn't have it11:28
*** derekh has joined #tripleo11:29
pandammmmh11:30
*** gfidente has joined #tripleo11:30
pandaok, so back to my original question11:30
pandahow can I specify a tag for the container pull to download instead of the default current-tripleo ?11:30
mandrepanda, ykarel: please talk to stevebaker or someone else from the same team11:31
pandamarios_|ruck: apparently you were right ^11:31
pandamandre: ok11:31
mandrepanda: it shouldn't be too difficult to submit a patch for the tripleo container image prepare command, not sure why it's not implemented11:32
*** pcaruana has joined #tripleo11:32
mandrein my understanding the tripleo command was supposed to be more or less equivalent to the overcloud one for the image prepare11:33
pandathere must be some other way, otherwise it meas that we've been just downloading containers in master always using current-tripleo. WHich sounds too weird.11:35
*** saneax has quit IRC11:38
*** saneax has joined #tripleo11:39
mandrepanda: perhaps the intention is to rely only on the values coming from the container image prepare file and disallow overriding vars on the command line?11:39
mandrepanda: so in that case, you need to modify the heat environment that contains the ContainerImagePrepare to have the right "tag" value11:40
*** bfournie has quit IRC11:42
*** pdeore has quit IRC11:52
hjensassshnaidm|rover: deloran builds of tripleoclient, looks broken. http://logs.openstack.org/62/642962/2/check/tripleo-ci-centos-7-containers-multinode/b758dfc/logs/delorean_logs/db/85/db8582e67186a33b680eff6cf1d95fe4536a6f85_dev/rpmbuild.log.txt.gz11:57
*** skramaja_ has joined #tripleo11:57
*** gcerami_ has joined #tripleo11:57
*** saneax has quit IRC11:57
*** saneax has joined #tripleo11:58
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: DNM: test 010 job with mistral patch  https://review.openstack.org/64330511:58
*** zbr has joined #tripleo11:58
*** ansmith has quit IRC11:58
*** skramaja has quit IRC11:58
sshnaidm|roverhjensas, ack, looking11:58
sshnaidm|roverhjensas, it fails in unittest tripleoclient.tests.v1.test_container_image.TestContainerImagePrepare.test_container_image_prepare_noargs11:59
hjensassshnaidm|rover: yes, on multiple different reviews.11:59
hjensassshnaidm|rover: here is another one. https://logs.rdoproject.org/91/643091/4/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/6366863/logs/delorean_logs/1f/90/1f903c2b321ffdb85873ca4fca5444c86afc9677_dev/rpmbuild.log.txt.gz11:59
sshnaidm|roverhjensas, I see12:00
sshnaidm|rovermarios, can you submit a bug please? ^^12:00
*** dprince has joined #tripleo12:01
marios_|rucksshnaidm|rover: we have call now12:01
marios_|rucksshnaidm|rover: i can submit after12:01
marios_|rucksure12:01
*** amoralej is now known as amoralej|off12:04
*** amoralej|off is now known as amoralej|lunch12:04
*** abishop has joined #tripleo12:05
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Octavia: set net configuration to none for mkdir in containers  https://review.openstack.org/64330712:08
*** blinkiz has quit IRC12:10
*** skramaja has joined #tripleo12:11
sshnaidm|roverhjensas, seems like broken by https://review.openstack.org/#/c/640716/12:11
*** skramaja_ has quit IRC12:11
sshnaidm|roverksambor, ^^12:11
*** blinkiz has joined #tripleo12:12
*** jpena is now known as jpena|lunch12:12
ksamborsshnaidm|rover, please assign bug to me. I will take care12:14
*** blinkiz has quit IRC12:18
*** bfournie has joined #tripleo12:18
sshnaidm|roverapetrich, can you take a look please? https://review.openstack.org/#/c/643304/12:18
*** trown|outtypewww is now known as trown12:19
jaosoriorsshnaidm|rover: responded ^^12:27
sshnaidm|roverjaosorior, I see12:28
ykareljaosorior, list() is defined in sub classes12:28
jaosoriorykarel: ResourceManager inherits from object12:28
ykarelof base.Resource12:28
jaosoriornot base.Resource12:28
ykareljaosorior, see http://eavesdrop.openstack.org/irclogs/%23tripleo/%23tripleo.2019-03-13.log.html#t2019-03-13T13:40:0712:29
jaosoriorsshnaidm|rover, ykarel commented again.12:29
*** ade_lee has quit IRC12:30
jaosoriorykarel: still. It doesn't inherit from base.Resource12:30
*** rlandy has joined #tripleo12:32
ykareljaosorior, sorry child class of base.ResourceManage12:32
ykareljaosorior, sorry child class of base.ResourceManager12:32
ykarelnot base.Resoruce12:33
*** hberaud is now known as hberaud|lunch12:33
*** morazi has joined #tripleo12:33
ykareljaosorior, https://github.com/openstack/python-mistralclient/search?utf8=%E2%9C%93&q=ResourceManager&type=12:33
ykarelmore clean:- https://github.com/openstack/python-mistralclient/search?q=%22%28base.ResourceManager%29%22&unscoped_q=%22%28base.ResourceManager%29%2212:34
jaosoriorykarel: understood. In that case, you might wanna define the list() function anyway, and throw  a NotImplemented exception on the base. For readability.12:34
ykareljaosorior, yes that what's mistral team is planning to do12:34
ykareljaosorior, see http://eavesdrop.openstack.org/irclogs/%23openstack-mistral/%23openstack-mistral.2019-03-14.log.html#t2019-03-14T08:31:2912:34
jaosoriorlol yeah; so I wasn't the only one confused.12:36
ykarelyes :)12:36
sshnaidm|roverykarel, do you know if there is a fixing patch?12:39
*** aissat-mus has joined #tripleo12:39
ykarelsshnaidm|rover, not yet12:39
sshnaidm|roverack12:39
ykarelit's only yours12:39
ykarelsshnaidm|rover, remember the fix will also need mistralclient release12:39
*** raildo has joined #tripleo12:39
sshnaidm|roveroh, right12:39
ykarelsshnaidm|rover, so i suggest to pin mistralclient in rdoinfo if it takes time12:39
sshnaidm|roverhow did this get through..12:40
*** panda has quit IRC12:40
sshnaidm|roverykarel, yeah, seems like better to do so12:40
*** gcerami_ is now known as panda12:40
openstackgerritBrent Eagles proposed openstack-infra/tripleo-ci master: Initial addition of scenario010 standalone job  https://review.openstack.org/64171312:44
openstackgerritBrent Eagles proposed openstack-infra/tripleo-ci master: Initial addition of scenario010 standalone job  https://review.openstack.org/64171312:45
chandankumarjaosorior: https://review.openstack.org/#/c/642948/ can we get +w on this!12:46
ykarelsshnaidm|rover, actually there was tripleo and devstack functional job running in breaking mistralclient patch but issue didn't detected there, also it was not detected by tripleo standalone jobs running in RDO u-c sync patch:https://review.rdoproject.org/r/#/c/19178/5/tags/stein-uc.yml@304, and final it god detected only in promotion jobs12:48
*** ansmith has joined #tripleo12:48
ykarelso good to understand why it's not detected earlier12:48
*** mcornea has joined #tripleo12:49
sshnaidm|roverykarel, https://review.openstack.org/64331912:50
*** ansmith_ has joined #tripleo12:50
ykarelsshnaidm|rover, ack but good to understand why not detected in standalone, it would be good to cover such issues in standalone itself wherever possible12:51
ykarelto save resources12:51
ykarelmay be mistral guys should look as devstack job also didn't detected it12:52
*** ansmith has quit IRC12:53
apetrichsshnaidm|rover, I agree with jaosorior. Just adding an abstract def list and I think it is good to go12:53
sshnaidm|roverykarel, as I understand it works different in standalone, doesn't have details though12:54
sshnaidm|roverapetrich, can you take care of this please?12:54
apetrichsshnaidm|rover, sure. no prob12:54
sshnaidm|roverapetrich, thanks!12:54
ykarelsshnaidm|rover, ack12:54
paramiteapetrich, d0ugal: Hey guys, so I am again stucked on testing https://review.openstack.org/#/c/640344/ . Execution now fails on get_database_credentials task with Can not evaluate YAQL expression [expression=task().result, error=Task execution not found [id=4f088802-5f7a-4ff4-ba28-47b7f45e98c6], data={}] . So I found out that the missing execution from the error message is not execution, but completely irrelevant task actually: http://paste.opensta12:55
paramiteck.org/show/747772/  ... any hint?12:55
*** cylopez has joined #tripleo12:55
paramitehttp://paste.openstack.org/show/747772/12:55
*** jchhatbar has joined #tripleo13:00
*** janki has quit IRC13:00
*** skramaja has quit IRC13:01
*** amoralej|lunch is now known as amoralej13:05
*** jchhatbar has quit IRC13:10
*** janki has joined #tripleo13:10
slagleowalsh: abishop fultonj : with the DST time change in the US, I have a standing conflict at 1400UTC now for the squad meeting, so we'll need to find a different time13:15
fultonjslagle: i have similar issue13:15
*** vkapalav has joined #tripleo13:17
abishopslagle: me too, so +1 for new time13:17
slagleok, i'll send out some proposals to the list13:18
*** cylopez has left #tripleo13:20
*** ade_lee has joined #tripleo13:22
fultonjslagle: fwiw my latest status on tripleo-ci is that the edge node is trying to push it's containers back to the controller node during container image preapre. it's like it's ignoring the LocalContainerRegistry parameter13:25
*** ade_lee_ has joined #tripleo13:25
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: Make standalone job run containerized tempest  https://review.openstack.org/64326713:26
TenguFolks, care to make that merge in order to avoid some noise in the audit.log? :) https://review.openstack.org/643240  thanks !13:27
*** ade_lee has quit IRC13:28
*** hberaud|lunch is now known as hberaud13:30
sshnaidm|roverksambor, I opened a bug, I think that it's your patch, but not sure. I'd appreciate if you can check it please, it blocks all patches with tripleoclient https://bugs.launchpad.net/tripleo/+bug/182006513:32
openstackLaunchpad bug 1820065 in tripleo "Tripleo client fails in unittests when is being built by DLRN" [Critical,Triaged]13:32
sshnaidm|rovermarios_|ruck, ^^13:32
*** tzumainn has joined #tripleo13:33
ksamborsshnaidm|rover, thank you, yeah you are right, I'm on it (will fix in next 30 min)13:33
sshnaidm|roverksambor, cool, thanks!13:33
*** mjturek has joined #tripleo13:38
*** mjturek has quit IRC13:38
hjensasquestion regarding undercloud endpoints _admin_host and _public_host. - If I want the TripleO UI to be available on the external network (i.e customers LAN) They would set _admin_host to a address on the customer net, and _public_host is still an address on the provisioning net?13:40
hjensasjaosorior: ^^ is'nt that how it's supposed to work?13:44
jaosoriorhjensas: not entirely sure to be honest. I thought that as part of the installation, the UI would set up a proxy13:45
jaosoriorbased on mod_proxy13:45
jaosoriorI'm unsure what IP that was using tbh13:45
hjensasjaosorior: ok ... I'll go look at the docs. :)13:45
openstackgerritNir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element  https://review.openstack.org/64333213:46
*** mjturek has joined #tripleo13:46
openstackgerritNir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element  https://review.openstack.org/64333213:47
*** jpena|lunch is now known as jpena13:50
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Enable debug mode for containers build  https://review.openstack.org/64288813:51
openstackgerritJames Slagle proposed openstack/tripleo-docs master: Document multiple stack deployment for DCN  https://review.openstack.org/64333913:52
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix containerized tempest logic  https://review.openstack.org/64242013:53
*** jfrancoa has quit IRC13:53
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309513:55
marios_|ruckweshay: ^13:55
weshaythanks13:58
weshayif we have the powa to move this to the top of the gate queue https://review.openstack.org/#/c/643212/13:58
*** janki has quit IRC13:58
weshaywe probably shoud13:58
weshayshould13:58
openstackgerritNir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element  https://review.openstack.org/64333214:02
openstackgerritJames Slagle proposed openstack/tripleo-docs master: Document multiple stack deployment for DCN  https://review.openstack.org/64333914:03
*** boazel_ has joined #tripleo14:04
chandankumarone stupid question, during standalone deployment we have unused network interface eth1 do we need to create it manually or the tht takes care of that?14:04
chandankumarif I use eth0 it loses the vm ssh connection14:05
*** ykarel is now known as ykarel|away14:05
chandankumarthis is what I am getting http://paste.openstack.org/show/747776/14:05
*** hjensas has quit IRC14:06
*** boazel has quit IRC14:07
*** rh-jelabarre has joined #tripleo14:13
*** ykarel|away has quit IRC14:14
*** morazi has quit IRC14:15
*** moisa has joined #tripleo14:16
mwhahahachandankumar: you need to create that14:16
mwhahahathat's a hardware config14:16
chandankumarmwhahaha: ack!14:16
mwhahahayou can reuse eth0 if you properly configure it, see the 1 nic example14:17
*** rcernin has quit IRC14:17
*** moisa has quit IRC14:17
mwhahahabut it's more complicated, is better to ensure 2 nics in the box14:17
*** moisa has joined #tripleo14:18
*** moisa is now known as moguimar14:18
*** hjensas has joined #tripleo14:20
*** aissat-mus has quit IRC14:23
openstackgerritNagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: Add an Endpoint in ceilometer  https://review.openstack.org/61197914:27
openstackgerritNagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: Update to the ceilometer publisher list  https://review.openstack.org/60084314:28
openstackgerritBrent Eagles proposed openstack/tripleo-common master: octavia: parameterize calls to container cli  https://review.openstack.org/64335114:37
*** gfidente has quit IRC14:39
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: octavia: pass container cli to external deploy tasks  https://review.openstack.org/64335814:40
*** dmacpher has quit IRC14:40
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: octavia: pass container cli to external deploy tasks  https://review.openstack.org/64335814:43
openstackgerritBrent Eagles proposed openstack-infra/tripleo-ci master: Initial addition of scenario010 standalone job  https://review.openstack.org/64171314:43
ksamborHi guys, who is able to bump up version in https://pypi.org/ for tripleo-common to have the newest version of master as a 10.6? We need this to be consistent between ci-jobs14:47
*** chem has quit IRC14:48
ksamboralso for future will be nice if we will use packages based on the same tripleo-common code between all zuul jobs14:48
ksamborEmilienM, mwhahaha ^^^^14:49
jaosoriorksambor: there was a release recently.14:49
jaosorior(today)14:49
jaosoriorksambor: https://review.openstack.org/#/c/642475/14:50
*** wolverineav has joined #tripleo14:50
ksamborjaosorior, yeah I saw but somehow pypi package is different than https://trunk.rdoproject.org/centos7-master/current-tripleo/14:51
mwhahahapackages will always be newer than pypi14:51
mwhahahathis is a long standing issue14:51
mwhahahawith no end in sight14:51
mwhahahaif you need a newer version of tripleo in the tox jobs, we have to release tripleo-common and bump requirements14:52
mwhahahathis is usually a problem with tripleoclient14:52
*** saneax has quit IRC14:52
mwhahahawhere we add functionality in tripleo-common but you can't use it in tripleoclient until a release of tripleo-common14:52
ksambormwhahaha, exactly we hit this issue that you describe14:52
mwhahahaso when you have that, you need to send for a release of tripleo-common14:53
mwhahahaanyone can propose14:53
*** holser_ has quit IRC14:53
mwhahahajust needs a +1 from PTL or a release liaison (EmilienM)14:54
ksamborkk14:54
ksamborthank you, mwhahaha14:54
*** wolverineav has quit IRC14:54
*** gfidente has joined #tripleo14:55
*** yolanda has quit IRC14:57
*** dmacpher has joined #tripleo15:04
*** apetrich has quit IRC15:10
*** Vorrtex has joined #tripleo15:11
*** hjensas has quit IRC15:12
*** ccamacho has quit IRC15:16
*** yprokule has quit IRC15:18
openstackgerritSerena Ziviani proposed openstack/diskimage-builder master: Also use selinuxenabled to check selinux status  https://review.openstack.org/64336715:24
*** agopi has quit IRC15:34
EmilienMdtantsur: hey15:35
dtantsurEmilienM: \o15:35
EmilienMdtantsur: if one node takes let's say forever to introspect but eventually it finish15:35
EmilienMwhere should I look?15:36
EmilienMI run vbmc on my own server, RAM is good, network is local so good15:36
EmilienMwhat could it be,15:36
dtantsurEmilienM: (on a meeting), check what's going on in the ramdisk15:36
*** ccamacho has joined #tripleo15:36
openstackgerritKamil Sambor proposed openstack/python-tripleoclient master: Fix tests for container image prepare  https://review.openstack.org/64337015:36
dtantsurthere are many places where it could happen15:36
EmilienMin /var/log/containers/ironic-inspector/ramdisk/15:36
EmilienM?15:36
dtantsurEmilienM: yeah15:37
EmilienMok indeed15:37
EmilienMlot of infos there15:37
EmilienMthx15:37
openstackgerritKamil Sambor proposed openstack/python-tripleoclient master: Fix tests for container image prepare  https://review.openstack.org/64337015:39
paramited0ugal, so I isolated the issue to line 52 in https://review.openstack.org/#/c/640344/2/workbooks/undercloud_backup.yaml . If I remove it the task passes. So the question is: why can I assign two variables? I verified that it is in environment: http://paste.openstack.org/show/747794/15:40
paramite*why can't I15:41
*** hamzy has quit IRC15:43
*** yolanda has joined #tripleo15:45
d0ugalparamite: do you have a error message?15:46
d0ugalparamite: I don't understand how removing that line fixes it, it looks correct15:47
d0ugalparamite: unless the yaql is wrong15:47
EmilienMdtantsur: http://ix.io/1Dvx15:47
EmilienMlot of errors15:48
dtantsurEmilienM: many of them are expected, esp. on virt15:48
dtantsurEmilienM: btw, what is "forever" for you? I know that LLDP collection can take a couple of minutes.15:49
EmilienMdtantsur: 25 min for one node15:49
EmilienMthat's, forever to me15:49
dtantsurvm/bm?15:49
EmilienMI run virtualbmc15:50
EmilienMon my bm15:50
dtantsurright, that's forever for a vm15:50
dtantsurbut the logs cover only the first minute. Can you match the timing, do the logs start quickly after the introspection start?15:50
dtantsurI'm trying to understand where the delays15:50
dtantsur* the delay is15:50
*** eggs has quit IRC15:51
*** ramishra has quit IRC15:51
*** matbu has quit IRC15:59
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309516:01
*** matbu has joined #tripleo16:06
*** agopi has joined #tripleo16:14
*** ykarel has joined #tripleo16:16
*** matbu has quit IRC16:16
openstackgerritMerged openstack/tripleo-common master: Stop outputting overcloudrc.v3 from workflow  https://review.openstack.org/64101516:17
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Add tempest workspace volume for tempest  https://review.openstack.org/64323716:17
*** matbu has joined #tripleo16:18
*** ksambor has quit IRC16:21
*** gfidente has quit IRC16:22
*** aufi has quit IRC16:23
openstackgerritLuke Short proposed openstack/tripleo-common stable/queens: Configurable host network for inventory  https://review.openstack.org/64306016:23
Tengufolks, care to merge that? https://review.openstack.org/#/c/643240/   thanks :)16:25
*** kopecmartin is now known as kopecmartin|off16:26
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Fix timezone lookup  https://review.openstack.org/64339016:27
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Fix timezone lookup  https://review.openstack.org/64339016:29
jaosoriorTengu: I can't; already +2ed it16:29
jaosorior:(16:29
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Fix timezone lookup  https://review.openstack.org/64339016:30
Tengujaosorior: ;_;16:33
*** jtomasek has quit IRC16:35
*** gfidente has joined #tripleo16:44
*** ykarel has quit IRC16:44
paramited0ugal, sorry, had mtg. Yes I have error message: http://paste.openstack.org/show/747800/16:44
paramiteah no, that;'s when I remove the line ... sec16:45
paramited0ugal, this is when the line is there: http://paste.openstack.org/show/747801/16:46
*** lmiccini has quit IRC16:47
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Bumped and hardened linting  https://review.openstack.org/63715116:48
paramited0ugal, http://paste.openstack.org/show/747803/16:49
paramited0ugal, to have complete information ^16:49
*** gkadam_ has quit IRC16:50
Tengudtantsur: heya! quick question - what's the "standard" time for the introspection of 1 vm (using the vbmc and so on) ?16:55
dtantsurTengu: depending on the enabled plugins, 2-5 minutes16:56
Tengudtantsur: hmm ok.16:56
dtantsurcan be longer if using nested virt16:56
Tengunot nested - BM has directly the VMs16:56
Tengudtantsur: same case as EmilienM - I'm trying to check if it's due to the VM xml or not16:56
dtantsurTengu: something-something-RHEL8-something? :)16:57
Tengu:)16:57
*** jaosorior has quit IRC16:57
*** jaosorior has joined #tripleo16:58
*** wolverineav has joined #tripleo16:59
Tengudtantsur: I see this in the introspect log: "The PXE filter driver DnsmasqFilter, state=initialized enters the fsm_reset_on_error context fsm_reset_on_error" - it enters and leave this "context" - not sure about its meaning.16:59
dtantsurTengu: ignore this17:00
Tenguok. hmm.17:00
*** rpioso is now known as rpioso|afk17:01
dtantsurit's a very technical thing that we should probably just remove from logging17:01
Tengudtantsur: would you have a moment to check what's going on my host?17:01
dtantsurkind of "a function was called" level of logging17:01
Tenguthe introspection is running right now17:01
Tenguand I can share a tmate :)17:01
dtantsurTengu: not right now, sorry, calling it a day17:01
Tengudtantsur: fair enough, didn't see the time.17:02
dtantsurbut the key is to get the logs from the ramdisk run (nova console in case of OVB)17:02
Tenguwill do the same then :).17:02
dtantsurand see what actually slows everything down17:02
*** wolverineav has quit IRC17:02
*** wolverin_ has joined #tripleo17:02
*** sshnaidm|rover is now known as sshnaidm|afk17:02
*** wolverin_ has quit IRC17:02
*** wolverineav has joined #tripleo17:02
Tengudtantsur: console-log then?17:02
dtantsuryeah17:03
*** paramite has quit IRC17:03
Tengudtantsur: nova list doesn't show any server. guess we're doomed.17:04
Tenguaanyway. tomorrow.17:04
dtantsurTengu: are you sure you're accessing the OVB nova?17:05
dtantsurnot the undercloud, for example?17:05
dtantsurbut yeah, evening :)17:05
*** rpittau is now known as rpittau|afk17:05
Tengudtantsur: hmmm I'm not using OVB per se... plain libvirt and the like. anyway. see you :) maybe tomorrow :17:05
dtantsurthen no nova for you17:05
dtantsur:)17:05
*** udesale has quit IRC17:06
*** trown is now known as trown|lunch17:06
*** matbu has quit IRC17:07
*** hamzy has joined #tripleo17:09
*** matbu has joined #tripleo17:09
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Fail fast if local clone is missing on include-gate-changes.sh  https://review.openstack.org/61580517:10
*** bogdando has quit IRC17:16
*** dtantsur is now known as dtantsur|afk17:18
*** psachin has quit IRC17:21
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix containerized tempest logic  https://review.openstack.org/64242017:26
*** dmacpher has quit IRC17:31
*** chandankumar is now known as chkumar24617:34
*** panda is now known as panda|off17:34
*** derekh has quit IRC17:38
*** jpich has quit IRC17:38
openstackgerritChandan Kumar proposed openstack-infra/tripleo-ci master: Make tripleo-ci-centos-7-standalone-os-tempest job to voting  https://review.openstack.org/64294817:42
*** aufi has joined #tripleo17:43
*** gfidente is now known as gfidente|afk17:46
openstackgerritMartin André proposed openstack-infra/tripleo-ci master: Collect /etc/modules-load.d/ in CI  https://review.openstack.org/64340617:47
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: [DNM] Check scenario009  https://review.openstack.org/64326117:49
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Stop loading nf_conntrack_proto_sctp module  https://review.openstack.org/64340717:49
*** hberaud is now known as hberaud|gone17:52
*** aufi has quit IRC17:53
*** panda|off has quit IRC18:01
*** trown|lunch is now known as trown18:01
*** panda has joined #tripleo18:04
openstackgerritMerged openstack/tripleo-quickstart-extras master: Improve warning message on docker group issue  https://review.openstack.org/64180818:07
*** wolverineav has quit IRC18:16
*** wolverineav has joined #tripleo18:19
*** wolverineav has quit IRC18:21
*** wolverineav has joined #tripleo18:22
*** rlandy is now known as rlandy|biab18:25
*** suuuper has quit IRC18:26
*** jpena is now known as jpena|off18:27
weshaymwhahaha rechecked https://review.openstack.org/#/c/643212/ post_failure18:27
beagles(c'mon baby merge merge!)18:27
*** wolverineav has quit IRC18:28
*** wolverineav has joined #tripleo18:29
* beagles wants to see the octavia standalone job pass18:33
*** amoralej is now known as amoralej|off18:33
weshaybeagles++18:54
weshayme too :)18:55
*** vkapalav has quit IRC18:55
*** aedc has joined #tripleo18:58
*** vinaykns has joined #tripleo19:00
*** rlandy|biab is now known as rlandy19:02
*** wolverineav has quit IRC19:07
weshaymwhahaha just double checking.. according to https://github.com/openstack/tripleo-quickstart-extras/commit/c4ec5a3b9fbd833a3384e062022d95c4e2364d28  the containers-prepare-paramater.yaml file is reused between the undercloud and overcloud19:08
weshayjust double confirming that is what we want19:08
mwhahahayes19:08
openstackgerritJames Slagle proposed openstack/tripleo-repos master: Include python*-tripleoclient from delorean.repo  https://review.openstack.org/64343919:14
slaglefyi, ^^ installs are currently broken on centos7 if you use tripleo-repos current-tripleo-dev19:15
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: [WIP] Fix the copy of tempest directories  https://review.openstack.org/64242019:19
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309519:19
*** rpioso|afk is now known as rpioso19:25
weshayf28 w/ f28 containers is green http://logs.openstack.org/51/637151/4/check/tripleo-ci-fedora-28-standalone/67e0fa6/logs/stackviz/#/testrepository.subunit19:28
*** absubram has joined #tripleo19:35
*** suuuper has joined #tripleo19:38
*** suuuper has quit IRC19:39
*** suuuper has joined #tripleo19:39
*** wolverineav has joined #tripleo19:42
*** wolverineav has quit IRC19:43
*** wolverineav has joined #tripleo19:44
*** yolanda has quit IRC19:46
*** wolverineav has quit IRC19:47
*** wolverineav has joined #tripleo19:51
*** wolverineav has quit IRC19:52
*** wolverineav has joined #tripleo19:53
*** suuuper has quit IRC19:55
*** gfidente|afk is now known as gfidente19:57
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: experiment w/ q35 for NovaHWMachineType  https://review.openstack.org/64254720:05
*** dsneddon has quit IRC20:27
fultonjMy WIP CI job sets LocalContainerRegistry: 192.168.24.3 [1] but container-image-prepare still tries to upload images to 192.168.24.1 [2]. Is this a bug or am I using it wrong?20:29
openstackgerritLuke Short proposed openstack/tripleo-common stable/queens: Configurable host network for inventory  https://review.openstack.org/64306020:30
fultonj[1] http://logs.openstack.org/88/615988/27/check/tripleo-ci-centos-7-split-controlplane-standalone/63d765d/logs/subnode-2/home/zuul/edge_parameters.yaml.txt.gz20:30
fultonj[2] http://logs.openstack.org/88/615988/27/check/tripleo-ci-centos-7-split-controlplane-standalone/63d765d/logs/subnode-2/var/log/tripleo-container-image-prepare.log.txt.gz20:30
fultonjstevebaker ^ do you have any ideas?20:30
*** sudeepbatra has joined #tripleo20:30
*** prometheanfire has joined #tripleo20:30
prometheanfireI think an FFE is needed for the recent tripleo releases (stein)20:31
prometheanfireat least if you want upper-constraints updated20:31
sudeepbatraHi, getting an error with quickstart > error: failed to connect to the hypervisor error: Failed to connect socket to '/run/user/1001/libvirt/libvirt-sock': No such file or directory20:32
sudeepbatraany ideas ?20:32
*** ansmith_ has quit IRC20:35
weshaysudeepbatra that's a libvirt problem...  https://bugzilla.redhat.com/show_bug.cgi?id=114062420:39
openstackbugzilla.redhat.com bug 1140624 in libvirt "error: Failed to connect socket to '/run/user/1000/libvirt/libvirt-sock': No such file or directory" [Unspecified,Closed: duplicate] - Assigned to libvirt-maint20:39
weshayhttps://bugzilla.redhat.com/show_bug.cgi?id=92736920:39
openstackweshay: Error: Error getting bugzilla.redhat.com bug #927369: NotPermitted20:39
openstackgerritNagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: Add an Endpoint in ceilometer  https://review.openstack.org/61197920:40
fultonjsudeepbatra: the user oooq sets up to run libvirt may not have permission to that file (usually it's stack)20:41
sudeepbatrado I need to edit the /etc/libvirt/libvirtd.conf > https://wiki.libvirt.org/page/Failed_to_connect_to_the_hypervisor20:42
sudeepbatra@fultonj, thanks ^20:42
fultonjsudeepbatra: i think oooq ssh's into the server as stack20:43
fultonjvirsh --connect qemu+ssh://username@remove.host.com/system list20:43
fultonjProbably the user you are using to access the server does not belong to the proper group, such as 'libvirtd'20:43
fultonji think you need to add stack to that group20:43
fultonjbottom of the same page you found20:43
sudeepbatraI tried using root and another admin user, still it keeps giving error20:44
fultonji think you need to add stack to that group20:44
fultonjlibvirtd20:44
*** hamzy has quit IRC20:45
sudeepbatraok will try that, I thought it did it automatically. the quickstart was working fine. I just got this error yesterday night in a new setup20:45
fultonji think i did this a while ago when i setup oooq i don't 100% remember but that page is familiar20:45
sudeepbatraok let me add it to libvirtd group, thanks20:45
sudeepbatrahmm strange, # sudo usermod -G libvirtd -a stack usermod: group 'libvirtd' does not exist20:47
fultonjsudeepbatra: i don't remember all fo the details, but i was on the same path when i solved that same issue for myself20:48
fultonje.g. group might be diff for centos20:48
sudeepbatraok20:49
*** mjturek has quit IRC20:53
*** Vorrtex has quit IRC20:56
*** raildo has quit IRC20:58
*** rh-jelabarre has quit IRC21:00
*** wolverineav has quit IRC21:00
*** wolverineav has joined #tripleo21:01
*** raildo has joined #tripleo21:06
*** raildo has quit IRC21:09
openstackgerritLuke Short proposed openstack/python-tripleoclient stable/queens: openstack overcloud failures  https://review.openstack.org/63982021:10
*** abishop has quit IRC21:12
*** vinaykns has quit IRC21:14
*** ade_lee_ has quit IRC21:16
*** trown is now known as trown|outtypewww21:16
*** itlinux has joined #tripleo21:17
*** agopi has quit IRC21:20
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize undercloud masquerade networks overrides  https://review.openstack.org/64292421:20
*** bfournie has quit IRC21:26
*** wolverineav has quit IRC21:36
*** wolverineav has joined #tripleo21:37
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Fix timezone lookup  https://review.openstack.org/64339021:43
*** ade_lee_ has joined #tripleo21:56
*** pcaruana has quit IRC22:21
*** rlandy is now known as rlandy|bbl22:22
*** wolverineav has quit IRC22:31
*** mcornea has quit IRC22:34
*** absubram has quit IRC22:36
*** aedc has quit IRC22:36
*** wolverineav has joined #tripleo22:36
*** gfidente has quit IRC22:50
*** rcernin has joined #tripleo23:00
*** tkajinam has joined #tripleo23:02
*** wolverineav has quit IRC23:04
*** wolverineav has joined #tripleo23:08
*** wolverineav has quit IRC23:11
openstackgerritMerged openstack/tripleo-quickstart master: Point to the right ovn environment file  https://review.openstack.org/64225423:12
*** wolverineav has joined #tripleo23:12
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Fix timezone lookup  https://review.openstack.org/64339023:18
*** slaweq has quit IRC23:25
*** tosky has quit IRC23:26
*** agopi has joined #tripleo23:40
*** agopi has quit IRC23:40
*** wolverineav has quit IRC23:41
*** wolverineav has joined #tripleo23:42
*** wolverineav has quit IRC23:46
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: attempt cleanup on container build id vars  https://review.openstack.org/64318723:47
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: Use the docker_image_tag instead of build id unless mixed upgrade  https://review.openstack.org/64309523:49
*** wolverineav has joined #tripleo23:49
*** wolverineav has quit IRC23:53
*** dsneddon has joined #tripleo23:55
*** wolverineav has joined #tripleo23:55
*** wolverineav has quit IRC23:56
*** wolverineav has joined #tripleo23:56
*** wolverineav has quit IRC23:58
*** dsneddon has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!