Tuesday, 2017-12-19

*** jcoufal has joined #tripleo00:00
*** d0ugal has joined #tripleo00:04
*** jcoufal has quit IRC00:05
openstackgerritMerged openstack/python-tripleoclient master: Remove Old baremetal commands provided by python-tripleoclient  https://review.openstack.org/52859300:07
openstackgerritMerged openstack/tripleo-quickstart master: Switch gate-check to THT patch  https://review.openstack.org/52877500:07
*** rcernin has quit IRC00:09
*** bfournie has quit IRC00:09
*** rcernin_ has joined #tripleo00:09
*** d0ugal has quit IRC00:40
*** moshele has joined #tripleo00:40
*** dhill_ has quit IRC00:50
*** rcernin_ has quit IRC00:50
*** d0ugal has joined #tripleo00:51
*** psachin has joined #tripleo00:53
*** rwsu has joined #tripleo00:54
*** catintheroof has joined #tripleo00:56
*** moshele has quit IRC01:00
*** moshele has joined #tripleo01:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/pike: Remove certificate before updating it  https://review.openstack.org/52885101:08
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Manila network plugin address family support  https://review.openstack.org/52222901:08
*** dmacpher has quit IRC01:16
openstackgerritOpenStack Proposal Bot proposed openstack/instack-undercloud master: Updated from global requirements  https://review.openstack.org/52886201:18
*** catintheroof has quit IRC01:21
*** zaneb has quit IRC01:27
*** bfournie has joined #tripleo01:41
openstackgerritwaleed mousa proposed openstack/tripleo-heat-templates master: Adds environment file for ODL OVS Hardware Offload  https://review.openstack.org/51871502:04
*** moshele has quit IRC02:08
*** rwsu has quit IRC02:09
*** rcernin has joined #tripleo02:11
*** rwsu has joined #tripleo02:11
rwsujoin #openstack-kolla02:11
*** moshele has joined #tripleo02:12
*** moshele has quit IRC02:15
*** psachin has quit IRC02:18
*** moshele has joined #tripleo02:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Introduce RHSM service  https://review.openstack.org/52680802:25
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test RHSM with role variable  https://review.openstack.org/52881602:26
openstackgerritdongwenjuan proposed openstack/tripleo-heat-templates master: fix the description of CongressDebug  https://review.openstack.org/52894102:31
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs032: activate config-download  https://review.openstack.org/52876102:31
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs032: activate config-download  https://review.openstack.org/52876102:31
*** psachin has joined #tripleo02:36
*** ramishra has joined #tripleo02:37
*** ramishra has quit IRC02:40
*** ramishra has joined #tripleo02:43
*** rlandy|rover has quit IRC03:10
openstackgerritMerged openstack/tripleo-heat-templates master: Fix permissions on .ssh directory.  https://review.openstack.org/52837703:14
*** ramishra has quit IRC03:15
tbarronovercloud-deploy.sh on current master altered only to refer to git tree for THT (with no changes yet):             "Error: Cannot allocate memory - fork(2)"03:17
*** yamahata has joined #tripleo03:17
tbarronit works with /usr/share/openstack-tripleo-heat-templates as $THT instead03:18
tbarronknown issue?03:18
tbarrongit tree is just a  pure checkout of master with no changes03:19
* tbarron wants to deploy with changes but is retrenching trying to get a known working baseline03:20
openstackgerritSteve Baker proposed openstack/tripleo-common master: Move more prepare logic into kolla_builder  https://review.openstack.org/52657903:24
openstackgerritSteve Baker proposed openstack/tripleo-common master: Use push_destination as the registry host in env file  https://review.openstack.org/52861603:24
openstackgerritSteve Baker proposed openstack/tripleo-common master: Prepare action: extra arguments  https://review.openstack.org/52658003:24
openstackgerritSteve Baker proposed openstack/tripleo-common master: Use skopeo for tag discover  https://review.openstack.org/52894503:24
*** yamahata has quit IRC03:27
*** ramishra has joined #tripleo03:43
*** dpawar has joined #tripleo03:52
openstackgerritMerged openstack/tripleo-heat-templates master: Add "clean" tox target  https://review.openstack.org/52780603:54
*** dmacpher has joined #tripleo03:56
*** itlinux_ has joined #tripleo04:04
itlinux_hello what' s the right process to upgrade Ocata to Pike Baremetal no containers..04:05
itlinux_I have Ocata on BM now..04:05
itlinux_I upgraded the Undercloud..04:05
*** shreshtha has joined #tripleo04:10
*** pdeore has joined #tripleo04:13
*** fragatina has quit IRC04:20
*** fragatina has joined #tripleo04:20
itlinux_anyone online today?04:21
mosheletrozet: hi04:24
itlinux_looks like to upgrade my baremetal for now the only thing I need to add is /usr/share/openstack-tripleo-heat-templates/ci/environments/multinode_major_upgrade.yaml04:26
itlinux_will not upgrade to docker yet..04:26
itlinux_just BM04:26
itlinux_is that correct.. can someone confirm that? Thanks04:27
*** psachin has quit IRC04:30
*** psachin has joined #tripleo04:35
*** janki has joined #tripleo04:43
*** ykarel has joined #tripleo04:44
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Introduce RHSM service  https://review.openstack.org/52680804:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test RHSM with role variable  https://review.openstack.org/52881604:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Introduce RHSM service  https://review.openstack.org/52680804:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test RHSM with role variable  https://review.openstack.org/52881604:46
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download  https://review.openstack.org/52533104:46
*** ratailor has joined #tripleo04:48
Tenguhello there04:57
itlinux_hi tengu04:58
*** pgadiya has joined #tripleo05:01
Tenguitlinux_: heya! regarding your upgrade question, there's a full upgrade doc I think. trying to get hold on it05:01
itlinux_thanks..05:02
itlinux_much appriciated.. I only see to upgrade to containers..05:02
itlinux_which I do not want at this point05:02
Tenguhmm fact is, you are able to NOT use containers in an easy way: just don't include the dedicated environment file(s) for containers05:04
itlinux_ok.. so what I posted above is correct..05:04
itlinux_just add the multinode_major_upgrade.yaml05:05
itlinux_with the old deploy file..05:05
TenguI think, yes. But I'm in no position to day "yep true go on".05:05
Tenguo_O tripleo.org unreachable ?!05:06
Tenguthat's.... annoying05:06
itlinux_ok.. just want to check.. the worst I will redeploy since this is preproduction now..05:06
itlinux_yes tripleo.org is down05:06
itlinux_cannot reach it..05:07
*** psachin has quit IRC05:07
Tenguknowing there's this doc: http://tripleo.org/install/post_deployment/upgrade.html  that will show you the steps in order to upgrade...05:07
Tenguwell, can't do more :/. if site is down, I can't point to the right doc :(. as it's not your productive install, you might give it a try05:08
*** pdeore_ has joined #tripleo05:08
itlinux_ok.. well it's becoming production after this upgrade LOL!05:09
itlinux_thanks..05:09
*** janki has quit IRC05:09
Tenguuhu, I followed the same steps, and ended up with a fresh Pike install05:09
itlinux_if the upgrade does not work I will delete and redeploy with the same options of ocata..05:09
*** pdeore has quit IRC05:09
Tenguhad some troubles, but that was mainly due to lack of knowledge.05:10
itlinux_nice to hear..05:10
Tenguand I wanted a stable/clean/fresh install for the prod.05:10
itlinux_what trouble did you  have? can you share..05:10
*** pdeore__ has joined #tripleo05:10
*** janki has joined #tripleo05:10
itlinux_the ocata has been running fine for the last two months.. for testing..05:10
*** pdeore__ has quit IRC05:11
itlinux_so I do not mind to redeploy fresh .. with pike if needed.. I upgraded the undercloud and it completed fine..05:11
Tenguthat was about 2 months ago - Pike was in release process, and, as said, I wasn't that good at openstack (on it for 3-4 moonths now)05:11
itlinux_yes.. agree. my boss did not want to try Pike :) now he is onboard..05:12
itlinux_so that's why..05:12
itlinux_I keep saying il!05:12
itlinux_but's it's ok.. as long as my options from Ocata work.. then I am happy!05:12
Tengu:) you'll probably see some "deprecated" environment files.05:13
*** pdeore_ has quit IRC05:13
Tenguyou'd better grep in the directory in order to ensure you only include current.05:13
itlinux_can you share which one do you think will be changed?05:13
itlinux_ok05:13
Tengugrep is your friend on that matter ;).05:13
itlinux_I will check the files before running the deploy..05:14
Tenguin fact, we copy the content of the /usr/share/tripleo-heat-templates and push it to git. that way, we have a nice `git diff' and `git grep'05:14
itlinux_here is my deploy file05:14
itlinux_http://paste.openstack.org/show/629271/05:14
itlinux_ahh I see..05:15
itlinux_looks like all the files are there :)05:15
TenguI followed a Redhat quick formation about Director, last year, and they kind of heavily insisted that directory must be backuped somewhere, and, well, git is the best place to push that (with some remote of course)05:16
itlinux_yea.. I know..05:17
itlinux_:) I remember when I worked at Red Hat.. and worked on Tusker.. which then became OOO :)05:17
*** psachin has joined #tripleo05:17
itlinux_crazy me that left RH :)05:17
itlinux_but now I like the place I am at.. after a few years with a company that had some great idea but never been able to execute!05:18
Tengu^^05:19
itlinux_thanks for sharing some ins..05:19
Tenguhope those small things will help.05:19
itlinux_I have the undercloud in a virsh..05:20
itlinux_so I created a snapshot..05:20
TenguI actually do have the same setup.05:20
itlinux_if case I can revert back to this stage and repush05:20
itlinux_nice!05:20
Tenguundercloud in virsh, snapshots, and banzai05:20
itlinux_yeap05:20
itlinux_very good config..05:20
Tenguand I'll create a lab with dedicated undercloud + some VMs for the "baremetal".05:20
itlinux_with HA05:20
Tenguthat will allow me to validate minor upgrades.05:21
Tenguyep, 3 controllers, 2 computes (2 more to be added shortly), ceph...05:21
Tengukind of a nice setup.05:21
itlinux_yes..05:21
itlinux_the plan here is to have 2 labs. since I need to test cells, maybe regions but they are not supported in OOO05:21
Tenguhmmm. octavia? I thought it wasn't that good integrated in tripleo05:22
itlinux_and the POC now has 3 controllers and 10 COMPUTE plush PURE storage05:22
itlinux_yes I pushed the code but it was not working..so we are adopting AVI LB05:22
Tengu(reading your deploy script)05:22
itlinux_which works fine with Ocata05:22
itlinux_cool!05:22
Tenguhmmm05:23
itlinux_I have AD and LDAP05:23
itlinux_pushing out..05:23
Tenguthe options setting the amount of nodes per role are deprecated05:23
Tenguyou can pass it through the environment directly.05:23
itlinux_how can you share05:23
itlinux_so I can fix it ?05:23
Tenguwe also have a freeIPA for authentication.05:23
Tengu2s05:23
itlinux_ok05:23
Tengujust add the following entries in some "environment.yaml" you include, in the parameter_defaults hash: ComputeCount:  ControllerCount:  CephStorageCount:05:24
itlinux_can you share the script..05:25
*** morazi has quit IRC05:25
itlinux_so I can see it long day..05:25
itlinux_sorry..05:25
Tengufact is, those options are already present for ocata, but instead of ControllerCount: it's named ControlCount:05:25
itlinux_what it looks like..05:25
Tenguhmmm we have some secrets in there.05:25
itlinux_ok take those out..05:25
itlinux_LOL!05:25
itlinux_I just need to see the part that changed..05:26
Tenguhttp://paste.openstack.org/show/629273/ you can get the starting point of the environment file.05:26
*** pdeore has joined #tripleo05:37
*** {aborted} is now known as chandankumar05:43
*** links has joined #tripleo05:47
moshelejanki: hi06:07
*** itlinux_ has quit IRC06:08
moshelejanki: it seem that this commit commit https://github.com/openstack/puppet-tripleo/commit/bc3feec75f8dc1bac6691ce8cd05a37c27aa5e42 broke odl deployment I am getting http://paste.openstack.org/show/629278/ from haproxy has it failed to start06:10
*** afazekas has quit IRC06:11
*** afazekas has joined #tripleo06:11
*** karthiks has joined #tripleo06:13
*** itlinux_ has joined #tripleo06:15
*** itlinux_ has quit IRC06:15
jankimoshele, I will have a look. Remind me by evening if I haven't replied yet06:17
*** jaganathan has joined #tripleo06:18
*** janki has quit IRC06:22
jaganathand0ugal, can you please look into patch https://review.openstack.org/#/c/522265/06:28
*** agopi has joined #tripleo06:28
openstackgerritMerged openstack/tripleo-common stable/pike: Consume NodeDataLookup in ceph-ansible  https://review.openstack.org/52875506:28
*** pdeore_ has joined #tripleo06:30
*** pdeore has quit IRC06:32
*** gkadam has joined #tripleo06:38
*** moshele has quit IRC06:40
*** masco has joined #tripleo06:41
*** yolanda has joined #tripleo06:47
*** agopi has quit IRC06:49
*** moshele has joined #tripleo06:49
*** dmacpher has quit IRC06:52
*** yprokule has joined #tripleo06:53
*** marios has joined #tripleo06:55
*** moshele has quit IRC06:55
*** jchhatbar has joined #tripleo07:00
*** dpawar has quit IRC07:06
openstackgerritKarthik S proposed openstack/os-net-config master: WIP: Add network objects sriov_vf and sriov_pf  https://review.openstack.org/52757107:08
*** ludo has joined #tripleo07:12
*** ludo is now known as Guest7202807:12
Guest72028Hello All07:13
*** pchavva has quit IRC07:14
*** dsariel has quit IRC07:17
*** holser__ has joined #tripleo07:18
*** skramaja has joined #tripleo07:21
*** zshi has joined #tripleo07:22
*** zshi has quit IRC07:22
*** lvdombrkr has quit IRC07:22
*** zshi has joined #tripleo07:22
*** dpawar has joined #tripleo07:23
*** abregman has joined #tripleo07:31
*** rcernin has quit IRC07:31
Guest72028Is Nova specialist here ? Maybe someone know if it is possible to configure Compute Nodes spares with OpenStack, more globally is it possible to reserve nova ressources , or prioritize rebuild order in nova-evacuate ?07:33
Guest72028Many thanks for advance07:33
*** jfrancoa has joined #tripleo07:34
*** dpawar has quit IRC07:35
*** dpawar has joined #tripleo07:35
*** shardy has joined #tripleo07:37
*** shardy has quit IRC07:37
*** shardy has joined #tripleo07:41
*** pdeore_ is now known as pdeore07:42
*** lvdombrkr has joined #tripleo07:42
*** mmethot_ has joined #tripleo07:49
*** mmethot has quit IRC07:51
openstackgerritJuan Badia Payno proposed openstack/puppet-tripleo master: Let collectd manage repo and added unit test  https://review.openstack.org/52162907:56
*** rcernin has joined #tripleo08:02
*** d0ugal has quit IRC08:03
*** cylopez has joined #tripleo08:06
*** d0ugal has joined #tripleo08:07
*** cylopez has left #tripleo08:07
*** gfidente has joined #tripleo08:17
openstackgerritYurii Prokulevych proposed openstack/tripleo-heat-templates master: Check for yum lock befor all yum* operations.  https://review.openstack.org/52898408:17
sileht*/108:17
*** moshele has joined #tripleo08:24
*** Guest72028 has left #tripleo08:25
*** amoralej|off is now known as amoralej08:25
*** ludovic has joined #tripleo08:26
*** yolanda__ has joined #tripleo08:34
jaganathanshardy, please look into patch https://review.openstack.org/#/c/525990/08:35
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Fix up the rabbitmq-ready check  https://review.openstack.org/52740308:35
*** yolanda has quit IRC08:36
*** florianf has joined #tripleo08:37
*** ccamacho has joined #tripleo08:39
*** jtomasek has joined #tripleo08:39
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: [WIP] Add a report of the Workflow execution on failure  https://review.openstack.org/52665308:42
openstackgerritYurii Prokulevych proposed openstack/tripleo-heat-templates master: Check for yum lock befor all yum* operations.  https://review.openstack.org/52898408:46
*** mdnadeem has joined #tripleo08:47
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/ocata: Remove osd_pool_default_min_size to allow Ceph cluster to do the right thing by default  https://review.openstack.org/50882108:48
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/newton: Remove osd_pool_default_min_size to allow Ceph cluster to do the right thing by default  https://review.openstack.org/50882308:50
*** jpena|off is now known as jpena08:51
*** karthiks has quit IRC08:56
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo stable/ocata: Update cephx keys with ACLs for openstack services.  https://review.openstack.org/52801408:57
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo stable/newton: Update cephx keys with ACLs for openstack services.  https://review.openstack.org/52859508:58
*** atarlov has joined #tripleo08:58
*** brault has joined #tripleo08:59
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/ocata: Set restrictive file permissions on Ceph keyrings for non-containerized deployment  https://review.openstack.org/52801508:59
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/newton: Set restrictive file permissions on Ceph keyrings for non-containerized deployment  https://review.openstack.org/52859908:59
shardyjaganathan: ack, done!09:04
jaganathanshardy, thanks09:04
*** sri_ has joined #tripleo09:06
*** karthiks has joined #tripleo09:08
openstackgerritJose Luis Franco proposed openstack/tripleo-upgrade master: Fix missing attribute in upgrade Infrared plugin  https://review.openstack.org/52743709:09
openstackgerritJose Luis Franco proposed openstack/tripleo-upgrade master: Use parameter to control the docker registry env file creation  https://review.openstack.org/52743809:09
*** nyechiel has joined #tripleo09:09
*** suuuper has joined #tripleo09:09
sshnaidmmandre, can you please take a look? https://review.rdoproject.org/r/#/c/10845/09:10
openstackgerritJose Luis Franco proposed openstack/tripleo-upgrade master: WIP ffu: Introduce ffu upgrade tasks  https://review.openstack.org/52824109:11
*** hewbrocca_afk is now known as hewbrocca09:14
*** fzdarsky has joined #tripleo09:15
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: Wait for rabbitmq_ready tag  https://review.openstack.org/52740409:19
jbadiapashardy, could you please take a look at https://review.openstack.org/#/c/500935/ ?09:19
shardyjbadiapa: ack will do - it'd be good to land the t-h-t part of that and deprecate the logging* interfaces for Queens09:23
jbadiapashardy, I'm on it09:23
shardyjbadiapa: nice, will review this now09:23
jbadiapashardy, I'm trying to get everything merged step by step09:24
jbadiapashardy, thx09:24
mandresshnaidm: looking09:27
*** jpich has joined #tripleo09:29
*** nyechiel has quit IRC09:30
*** lucas-afk is now known as lucasagomes09:32
*** matbu has joined #tripleo09:32
*** agopi has joined #tripleo09:36
*** derekh has joined #tripleo09:36
*** pliu_ has joined #tripleo09:40
*** ratailor has quit IRC09:44
openstackgerritSteven Hardy proposed openstack/tripleo-validations master: Move tripleo ansible inventory code to tripleo-common  https://review.openstack.org/52834409:50
*** nyechiel has joined #tripleo09:50
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829209:52
openstackgerritSergii Golovatiuk proposed openstack/instack-undercloud stable/pike: Restore OVSIntPort interfaces after OVS restarts  https://review.openstack.org/52901509:59
*** afazekas has quit IRC10:01
*** agopi has quit IRC10:02
openstackgerritMerged openstack/tripleo-common master: Remove jq package install  https://review.openstack.org/52578110:06
*** afazekas has joined #tripleo10:07
*** fzdarsky has quit IRC10:08
*** salmankhan has joined #tripleo10:09
*** yprokule has quit IRC10:15
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Set tftp to only listen to the provisioning network  https://review.openstack.org/52902310:17
*** yprokule has joined #tripleo10:18
*** karthiks has quit IRC10:20
*** fragatin_ has joined #tripleo10:22
*** fragatina has quit IRC10:22
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829210:23
*** psachin has quit IRC10:24
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829210:27
*** moshele has quit IRC10:29
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common master: SRIOV derive parameters workflows  https://review.openstack.org/52226510:32
*** jaganathan has quit IRC10:36
*** jtomasek has quit IRC10:38
*** jtomasek has joined #tripleo10:39
*** jpich has quit IRC10:40
*** jchhatbar is now known as janki10:41
*** dobson has quit IRC10:42
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Create flavors for undercloud  https://review.openstack.org/52681010:43
*** brault has quit IRC10:44
*** brault has joined #tripleo10:45
chandankumarmandre: please https://review.openstack.org/#/c/480353/ abandon this review10:45
*** dobson has joined #tripleo10:45
*** karthiks has joined #tripleo10:46
*** paramite has joined #tripleo10:47
*** brault has quit IRC10:50
*** brault has joined #tripleo10:51
*** fzdarsky has joined #tripleo10:51
shardymandre: Hey have you been testing with the containerized undercloud recently?10:53
shardymandre: it seems the added checks around --local-ip make both undercloud install --use-heat and undercloud deploy break after the first run, any idea if there are patches up to fix that?10:54
shardyI can't spot them and it's kind of a long debug loop to re-run vm_doit every time, I just want to keep running undercloud (install|deploy) to debug the use of the dynamic inventory10:54
shardyobviously I can hack up the tripleoclient code but wanted to see how others are doing it, as this seems pretty broken10:55
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: [WIP] Early release for fast forward upgrade testing.  https://review.openstack.org/52568010:56
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart master: [WIP] New featureset for fast forward upgrade testing.  https://review.openstack.org/52568610:56
*** mrunge_ has joined #tripleo10:56
*** mrunge has quit IRC10:57
*** fragatina has joined #tripleo10:57
*** fragatin_ has quit IRC10:57
mandrechandankumar: done11:00
chandankumarmandre: thanks :-)11:01
mandreshardy: I was about to get to it, seems like there are still multiple issues around the containerized undercloud11:01
shardymandre: ack, seems like we don't read the ~/undercloud.conf on the second run11:02
shardyperhaps https://review.openstack.org/#/c/523511/11/tripleoclient/v1/undercloud_config.py will help as it sets the UndercloudHomeDir11:02
shardymandre: another issues is we sudo internally when doing a (non sudo) openstack undercloud install --use-heat11:02
mandreshardy: is that something you're seeing in CI? my plan for today is to get the undercloud CI closer to green, I'm tracking the work in https://etherpad.openstack.org/p/tripleo-queens-undercloud-containers11:02
shardythat means the config download data ends up in /root even when running as "stack" which was initially confusing11:02
shardymandre: No I don't think we have any undercloud update/idempotency tests in CI for containers yet?11:03
shardymandre: it happens if you run an undercloud install --use-heat (e.g via vm_doit+run.sh) then try to run run.sh again11:03
shardypreviously this worked, I fixed a few issues some time ago to enable multiple runs of undercloud deploy, but it seems some things prevent that now11:04
mandreshardy: TBH, I've not yet deployed the containerized undercloud with quickstart, I've always been using our hacky doit.sh script for it11:04
shardybasically I want the CLI to be idempotent like the puppet undercloud install11:04
shardymandre: yeah I'm using the same11:04
shardymandre: try running run.sh, then run it again11:04
shardyit breaks due to IP validation, reporting a different (default) IP than that in the generated undercloud.conf]11:05
shardythat is, the first run works, but AFAICS it's not possible to run it again11:05
mandreshardy: AFAIK we do have a CI that tries to re-run the undercloud install, tripleo-ci-centos-7-undercloud-oooq is failing with idempotency issues11:05
shardymandre: ah, Ok maybe that will show it then, looks like:11:06
shardyhttp://paste.openstack.org/show/629296/11:06
shardymandre: you can see the Configured IP: is wrong, it's not reading the undercloud.conf11:07
*** mrunge_ has quit IRC11:07
*** mrunge has joined #tripleo11:07
shardyOh wait, now perhaps we're writing the wrong network_cidr11:07
mandreshardy: right, it will probably show up now since I fixed the first idempotency issue in https://review.openstack.org/#/c/526810/11:08
mandreshardy: lunch time, bbl11:08
shardymandre: ack bon appetit :)11:10
*** jaganathan has joined #tripleo11:11
openstackgerritMerged openstack/tripleo-common master: DPDK derive params inputs default value update  https://review.openstack.org/52599011:14
d0ugalshardy: Hey11:17
shardyd0ugal: hi!11:17
jaganathanshardy, d0ugal please look into https://review.openstack.org/#/c/522265/11:17
d0ugalshardy: You mentioned the idea of some more detail on a workflow failure. I've been playing around with it11:17
d0ugalshardy: http://logs.openstack.org/53/526653/10/check/tripleo-ci-centos-7-containers-multinode/f456cae/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2017-12-19_09_57_5511:17
d0ugalshardy: note, this is outputting the details for every workflow - not just errors. Since none should fail in CI (in theory)11:17
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common stable/pike: DPDK derive params inputs default value update  https://review.openstack.org/52903611:17
d0ugalshardy: but do you think something like that would be useful? maybe filtered for ERROR only?11:18
shardyd0ugal: yes definitely!11:18
shardyd0ugal: maybe filter for errors by default, but enable all output for --verbose?11:18
d0ugalshardy: good idea. I'll try that.11:19
*** pgadiya has quit IRC11:22
*** Vijayendra has quit IRC11:23
*** cylopez has joined #tripleo11:27
openstackgerritYurii Prokulevych proposed openstack/tripleo-heat-templates master: Check for yum lock befor all yum* operations.  https://review.openstack.org/52898411:28
*** Vijayendra has joined #tripleo11:28
*** jtomasek has quit IRC11:29
*** jackass has joined #tripleo11:30
*** jackass changes topic to "you've been hacked#"11:30
*** jackass changes topic to "YOU'VE BEEN HACKED!!!!!"11:30
*** cylopez has quit IRC11:31
*** jackass has quit IRC11:31
*** cylopez has joined #tripleo11:33
*** hewbrocca is now known as hewbrocca_afk11:35
*** brault has quit IRC11:39
*** d0ugal changes topic to "CI status: GREEN, see alerts and read ML. | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"11:42
*** brault has joined #tripleo11:45
dtantsurmorning, can someone give 2nd +2 to https://review.openstack.org/#/c/526439/ please? I think it passed all jobs in a crazy number of rechecks :)11:47
dtantsurthis closes blueprint ansible-deploy11:47
*** cylopez has quit IRC11:48
*** cylopez has joined #tripleo11:48
*** yolanda__ is now known as yolanda11:50
*** brault has quit IRC11:50
openstackgerritMerged openstack/puppet-tripleo master: logging: support service_config_settings configuration mechanism  https://review.openstack.org/50093511:50
*** skramaja_ has joined #tripleo11:53
*** skramaja has quit IRC11:55
*** raildo has joined #tripleo12:00
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: Convert tags to when statements for Q major upgrade workflow  https://review.openstack.org/51090212:00
openstackgerritKeith Schincke proposed openstack/tripleo-heat-templates stable/ocata: Set restrictive file permissions on Ceph keyrings for non-containerized deployment  https://review.openstack.org/52801512:01
*** pkovar has joined #tripleo12:09
*** fpan has joined #tripleo12:10
*** skramaja has joined #tripleo12:10
*** skramaja_ has quit IRC12:13
*** salmankhan has quit IRC12:15
*** akrivoka has joined #tripleo12:15
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Convert tags to when statements for Q major upgrade workflow  https://review.openstack.org/51090212:18
*** salmankhan has joined #tripleo12:18
*** bfournie has quit IRC12:22
*** bfournie has joined #tripleo12:22
*** links has quit IRC12:23
*** dpawar has quit IRC12:24
*** bfournie has quit IRC12:27
*** amoralej is now known as amoralej|off12:29
*** brault has joined #tripleo12:30
*** dpawar has joined #tripleo12:31
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/ocata: Set restrictive file permissions on Ceph keyrings for non-containerized deployment  https://review.openstack.org/52801512:33
*** anilvenkata has joined #tripleo12:34
dtantsurfolks, for a containerized undercloud, how do I put/change something to /etc/ironic (via shell)?12:35
dtantsurshould I use /var/log/config-data/ironic/etc/ironic/ or what?12:37
dtantsurslagle: if you know by chance ^^^12:38
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829212:40
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common master: SRIOV derive parameters workflows  https://review.openstack.org/52226512:42
*** lucasagomes is now known as lucas-hungry12:42
*** jpena is now known as jpena|lunch12:48
*** weshay_pto is now known as weshay12:48
*** morazi has joined #tripleo12:57
*** bfournie has joined #tripleo13:04
shardyflorianf: Hey when you get time please could you review https://review.openstack.org/#/c/528344/ and the associated tripleo-common patches?13:05
*** jcoufal has joined #tripleo13:06
*** hewbrocca_afk is now known as hewbrocca13:06
*** cylopez has left #tripleo13:09
*** rbowen has joined #tripleo13:09
*** janki has quit IRC13:11
*** janki has joined #tripleo13:11
*** catintheroof has joined #tripleo13:11
*** catintheroof has quit IRC13:12
*** catintheroof has joined #tripleo13:12
florianfshardy: sure, will have a look today13:13
*** d0ugal has quit IRC13:14
*** links has joined #tripleo13:18
*** rcernin has quit IRC13:20
*** d0ugal has joined #tripleo13:22
*** dprince has joined #tripleo13:25
*** dpawar has quit IRC13:25
*** skramaja has quit IRC13:26
*** fultonj has quit IRC13:28
*** rlandy has joined #tripleo13:29
*** rlandy is now known as rlandy|rover13:29
rookshardy ping so, in our heat.conf we have workers set to 1213:30
*** dsariel has joined #tripleo13:30
rookhowever we are seeing heat spawn # of cores it seems13:31
rookso heat is no longer honoring the workers?13:31
openstackgerritHarald JensÃ¥s proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754413:31
shardyrook: it should do but defaults to #cores - can you share the heat.conf, and the ps output showing the workers please?13:32
openstackgerritIlya Etingof proposed openstack/tripleo-common master: UCS Ironic node driver info added  https://review.openstack.org/52872613:33
rookshardy: according to the config file heat-engine should default to 413:33
rookif not set.13:34
rookomg13:34
* rook reads the second line.13:34
*** lucas-hungry is now known as lucasagomes13:34
rookfollow on question shardy should we be limiting the # of heat workers to the same config value as other services (including other heat processes)13:35
rookshardy we are seeing heat-engine consume quite a bit of memory, and it is likely due to the # of workers.13:36
*** trown|outtypewww is now known as trown|ruck13:36
shardyrook: I think the default workers in puppet-heat will be limited in a similar way to other services, but obviously in the case of the undercloud it's a tradeoff of memory usage vs deployment speed13:36
shardyrook: is this the undercloud or overcloud heat?13:37
rookundercloud.13:37
*** d0ugal has quit IRC13:37
rookshardy I don't see that big of an advantage of 63 workers vs 12 -- would we expect a 6x speed up?13:37
* rook wishes he caught this earlier.13:38
rook(I highly doubt we will see a 6x speed up) :)13:38
rookor reduction.13:38
*** pchavva has joined #tripleo13:38
shardyrook: No, but it all depends on the speed of ths storage, the next bottleneck is probably the DB13:38
*** fultonj has joined #tripleo13:39
*** d0ugal has joined #tripleo13:39
rookright - i just figure we would apply the puppet worker calculation to this process13:39
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common master: SRIOV derive parameters workflows  https://review.openstack.org/52226513:39
shardyrook: yeah we do for all processes except heat-engine by the looks of it13:39
rookshardy: between how heat-engine is configured and ansible (with ceph-ansible) we can easily consume ~ 100GB between the two13:40
shardyand that's probably because the config stuff in TripleO predates the sane defaults in puppet-*13:40
*** liverpooler has joined #tripleo13:40
shardyrook: yeah sure, it's a bug13:40
rooksure shardy13:40
shardyrook: sec13:40
rookshardy: i can patch heat-engine to honor the puppet calculation13:40
rookerr13:40
dtantsurdprince: hi! for a containerized undercloud, how do I put/change something to /etc/ironic (via shell)?13:40
rooktripleo/uc configuration *13:40
shardyrook: you can set an undercloud.conf hieradata override like13:40
shardyheat::engine::num_engine_workers: "%{::os_workers}"13:41
dprincedtantsur: hey, so you mean like without puppet then?13:41
rookroger.13:41
dtantsurdprince: yep13:41
dprincedtantsur: if so, then you could use an 'init container' via docker-cmd/paunch13:41
dprincedtantsur: this would be similar to how we initialize the logs directories in most of the docker/service yamls13:42
rookshardy: i feel like this has changed though... has it always been 4 OR # of CPUs?13:42
shardyrook: https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L144..L14713:42
rookI thought we limited the # of workers.13:42
*** jaganathan has quit IRC13:42
shardyrook: it was 4, then #CPUs due to the code I just linked, now I guess we need to switch to the puppet defaults13:42
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs master: [WIP] Document using Ironic Ansible deploy interface  https://review.openstack.org/52666313:42
shardyrook: the issue is when we made that change, there were no puppet calculated defaults, and stuff was choking on only 4 workers13:42
rookshardy: right, which will be at _max_ 12.13:42
dtantsurdprince: to be a bit more specific, what would I use instead of cp/mv bits in ^^^?13:42
shardyrook: yea, sec let me push a patch13:43
rookack shardy13:43
dprincedtantsur: let me review it then13:43
dtantsurthanks!13:46
dtantsurI also think people have a habit to edit /httpboot/inspector.ipxe. we have to make sure we have puppet replacement for that..13:47
openstackgerritSteven Hardy proposed openstack/instack-undercloud master: Set heat engine workers to "%{::os_workers}"  https://review.openstack.org/52906613:48
shardyrook: ^^13:48
*** ykarel has quit IRC13:48
rookthx shardy13:48
*** abishop has joined #tripleo13:49
*** jpena|lunch is now known as jpena13:49
*** dsariel has quit IRC13:53
openstackgerritMartin André proposed openstack/tripleo-common master: Install additional packages using kolla mechanism  https://review.openstack.org/52907013:54
openstackgerritMartin André proposed openstack/tripleo-common master: Install additional packages using kolla mechanism  https://review.openstack.org/52907013:54
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: Enable collectd to send metrics to Gnocchi  https://review.openstack.org/52612213:54
*** yprokule has quit IRC13:56
*** agurenko has joined #tripleo13:58
mwhahaha#startmeeting tripleo14:00
mwhahaha#topic agenda14:00
mwhahaha* Review past action items14:00
mwhahaha* One off agenda items14:00
mwhahaha* Squad status14:00
mwhahaha* Bugs & Blueprints14:00
openstackMeeting started Tue Dec 19 14:00:36 2017 UTC and is due to finish in 60 minutes.  The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
mwhahaha* Projects releases or stable backports14:00
mwhahaha* Specs14:00
mwhahaha* open discussion14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
mwhahahaAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:00
mwhahahaHi everyone! who is around today?14:00
*** openstack changes topic to " (Meeting topic: tripleo)"14:00
openstackThe meeting name has been set to 'tripleo'14:00
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:00
dtantsuro/14:00
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537214:00
dprincehi14:00
EmilienMo/14:00
ccamachoo/14:01
jankio/14:01
marioso/14:01
fultonjo/14:01
shardymandre: Hey fyi the issue seems to be in doit.sh, where we configure undercloud.conf with mismatched local_ip and network_cidr values14:01
shardyo/14:01
dpeacocko/14:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537214:02
*** shreshtha has quit IRC14:02
mwhahahalight crowd. everyone's on vacation already :D14:02
mwhahahalet's get on with it14:02
florianfo/14:02
mwhahaha#topic review past action items14:02
mwhahahajistr, gfidente and shardy look at ceph-ansible and config-download (postponed till next year) - just a reminder14:02
mwhahaha#action jistr, gfidente and shardy look at ceph-ansible and config-download (postponed till next year)14:02
mwhahahaEmilienM to start a policy doc around bug priority14:02
sshnaidmo/14:02
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:02
EmilienM#action EmilienM to start a policy doc around bug priority (postponed)14:03
mwhahahamwhahaha/EmilienM to schedule a Newton day in January before EOL14:03
EmilienMI suggest we rediscuss that in 201814:03
mwhahahak14:03
*** dciabrin_ has joined #tripleo14:03
mwhahaha#action mwhahaha/EmilienM to schedule a Newton day in January before EOL (postponed)14:03
*** links has quit IRC14:03
EmilienMsome projects haven't EOL-ed yet14:03
openstackgerritMike Burns proposed openstack/tripleo-puppet-elements master: yum-plugin-priorities should not be enabled on RHEL  https://review.openstack.org/52907414:04
EmilienM(nova)14:04
*** dciabrin has quit IRC14:04
EmilienMwe need to plan it14:04
*** rfolco has joined #tripleo14:04
mwhahahaok we'll figure it out next year14:04
mwhahaha#topic one off agenda items14:04
mwhahaha#link https://etherpad.openstack.org/p/tripleo-meeting-items14:04
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:04
mwhahahathe agenda is empty14:04
mwhahahaanyone have antyhing?14:05
dtantsurlet's go to the beach14:05
mwhahahaJust FYI, I'll send a note about canceling the meeting14:05
mwhahahashould we cancel the one on Jan 2nd as well?14:06
EmilienMno meeting, no Weekly Owl? ;-)14:06
dprinceI won't be there14:06
*** dhill_ has joined #tripleo14:06
mwhahahak so i guess i'll just cancel the next two and we'll resume on Jan 9th14:06
trown|ruckwait no weekly owl?14:06
trown|ruckhow will i know what all the different colors of the common barn owls various body parts are?14:07
EmilienMI can't stop the Weekly Owl14:07
dpeacockI love that part at the bottom14:07
openstackgerritSagi Shnaidman proposed openstack/puppet-tripleo master: DONT REVIEW: testing patch  https://review.openstack.org/52907714:07
EmilienMyour fellow reporter can't stop it anymore, it's forever14:07
mwhahahaHEH14:07
mwhahahaalright moving on to the Weekly Owl content14:08
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537214:08
mwhahaha#topic Squad status14:08
mwhahahaci14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:08
mwhahahaupgrade14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:08
mwhahahacontainers14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-containers-squad-status14:08
mwhahahaintegration14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:08
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:08
mwhahahaui/cli14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:08
mwhahahavalidations14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:08
mwhahahanetworking14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:08
*** jmelvin has joined #tripleo14:08
mwhahahaworkflows14:08
mwhahaha#link https://etherpad.openstack.org/p/tripleo-workflows-squad-status14:08
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829214:08
*** pdeore_ has joined #tripleo14:08
mwhahahacontainers needs an update14:09
mwhahahaui/cli also needs an update14:09
mwhahahanetworking didn't have a meeting yesterday due to low turnout from vacations so no update for them either14:09
*** pdeore has quit IRC14:10
mwhahahaworkflows have been spending time on ascii art14:10
mwhahahaany one off status things?14:10
EmilienMlol14:10
dtantsurnice!14:11
mwhahahasounds like no14:11
mwhahaha#topic bugs & blueprints14:11
mwhahaha#link https://launchpad.net/tripleo/+milestone/queens-314:11
mwhahahaFor Queens we currently have 34 (+4) blueprints and about 569 (+17) open bugs. 561 queens-3, 8 rocky-1.14:11
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:11
mwhahahai moved some blueprints back in to queens due to various FFE requests. Please make sure they stay up to date14:11
dpeacockWhat does FFE stand for?14:12
mwhahahaAs a reminder queens-3 is next so we should be stablizing the things14:12
EmilienMwhat is "+4" or (+17) ?14:12
mwhahahadpeacock: feature freeze exception14:12
dpeacockThank you14:12
mwhahahaEmilienM: change from last week14:12
EmilienMwow14:12
mwhahahai've been tracking it all cycle14:12
EmilienMsince when we take blueprints during queens-314:12
mwhahahaso irc meetings have all of those14:12
mwhahahaEmilienM: well i kicked them all out before the meeting last week14:13
mwhahahaso those are the ones i moved back14:13
EmilienMok14:13
mwhahahaso we pulled 4 back in to queens14:13
mwhahahasome of which are already implemented as of last week14:13
openstackgerritMartin André proposed openstack/tripleo-common master: Cleanup mistral-api override  https://review.openstack.org/52908114:13
mwhahahaany bug/blueprint questions?14:14
mwhahahasounds like no14:15
mwhahaha#topic projects releases or stable backports14:15
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:15
mwhahahaEmilienM: we doing any stable stuff over the holidays or is that also postponed until next year14:15
* mwhahaha would assume no14:15
matbuo/ (sorry /me lates)14:15
EmilienMno14:16
EmilienMnext year from now14:16
mwhahahak14:16
EmilienMnext stable release is first week of Jan I guess14:16
mwhahaha#topic specs14:16
mwhahaha#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:16
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:16
mwhahahaso we're past m2, so we should not be merging any more queens specs14:16
*** links has joined #tripleo14:16
mwhahahaany questions?14:17
*** pdeore_ has quit IRC14:18
EmilienMis Santa Claus exist?14:19
mwhahahayes (no)14:19
EmilienMs/is/does/14:19
mwhahaha#topic open discussion14:19
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:19
* mwhahaha sends EmilienM some coal14:19
mwhahahaanything else? or we'll call it for the week14:20
dpeacockSo we're mostly shutdown next week?14:21
mwhahahayea14:21
dpeacockAck14:21
EmilienMlet's say low activity14:21
mwhahahathere will still be some people around14:21
EmilienMsome folks will still be around14:21
EmilienMI'll be around14:21
mwhahahabut it'll be light14:21
mwhahahaso it's a good time to work on some long outstanding items :D14:22
*** dhill_ has quit IRC14:22
*** pdeore has joined #tripleo14:22
mwhahahaalright everyone back to work14:23
mwhahahaor not14:23
mwhahaha#endmeeting14:23
*** openstack changes topic to "CI status: GREEN, see alerts and read ML. | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"14:23
openstackMeeting ended Tue Dec 19 14:23:40 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:23
openstackMinutes:        http://eavesdrop01.openstack.org/meetings/tripleo/2017/tripleo.2017-12-19-14.00.html14:23
*** jtomasek has joined #tripleo14:23
openstackMinutes (text): http://eavesdrop01.openstack.org/meetings/tripleo/2017/tripleo.2017-12-19-14.00.txt14:23
openstackLog:            http://eavesdrop01.openstack.org/meetings/tripleo/2017/tripleo.2017-12-19-14.00.log.html14:23
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: add dry-run or list-tasks to quickstart.sh  https://review.openstack.org/52620814:26
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Ensure rsyncd PID file is removed during overcloud updates  https://review.openstack.org/52699814:26
EmilienMbnemec, dprince : http://tripleo.org/cistatus.html is down for me14:27
cmurphyhey EmilienM mwhahaha what's the verdict on https://review.openstack.org/#/c/522461 ? I see https://review.openstack.org/#/c/528751/ failed but I can't tell if that's the reason14:28
*** dhill_ has joined #tripleo14:29
EmilienMcmurphy: +1 we can go ahead14:32
EmilienMcmurphy: I tested it14:32
EmilienMI just +1-ed14:32
cmurphycool ty14:32
EmilienMcmurphy: thx for asking14:32
cmurphynp14:32
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Create flavors for undercloud  https://review.openstack.org/52681014:37
*** marrusl has quit IRC14:39
*** pdeore has quit IRC14:40
*** pliu_ has quit IRC14:40
owalshmwhahaha: ping re https://review.openstack.org/49872314:41
*** hjensas has quit IRC14:42
owalshmwhahaha: not sure what you mean... if I use recursive and the group ain't nova then it's not idempotent14:42
owalsh^^^ wrong channel, moving to #puppet-openstack14:43
*** sshnaidm has quit IRC14:46
*** ramishra has quit IRC14:47
*** sshnaidm has joined #tripleo14:48
mandredprince: is the undercloud install running heat-all from a container or is it BM?14:48
*** nyechiel has quit IRC14:48
dprincemandre: for CI it should be on baremetal.14:48
bnemecEmilienM: Sorry, got distracted yesterday after I rebooted that compute.  It should be up now.14:48
dprincemandre: we support both. but for the 'undercloud install' case we used the puppet/services/docker.yaml file to bootstrap/setup the docker service on baremetal14:49
mandredprince: ok, I'm looking at the tripleo-ci-centos-7-undercloud-containers failure, where that seems to be caused by a missing heat user on the undercloud14:50
mandredprince: http://logs.openstack.org/18/518118/10/check/tripleo-ci-centos-7-undercloud-containers/6b11ffe/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2017-12-18_19_35_2114:50
dprincemandre: looking14:50
EmilienMbnemec: thanks! no problem14:51
dprincemandre: are we missing the openstack-heat-monolith package?14:52
mandredprince: that what I was looking at, openstack-heat-monolith-10.0.0-0.20171206175731.8b5218c.el7.centos.noarch from delorean excluded (priority)14:53
mandreit's not installed14:53
dprincemandre: I think this is the issue. It is getting installed in one of the multinode jobs though14:53
dprincemandre: probably manually there via quickstart14:54
mandredprince: ok, let me see if I can find the magic place in quickstart where I can add the package14:54
openstackgerritMark Hamzy proposed openstack/tripleo-common master: [WIP] Support multiple architectures  https://review.openstack.org/52800014:54
*** lblanchard has joined #tripleo14:55
mandredprince: weird, seems like it's already there, https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/undercloud-deploy.sh.j2#L5614:58
*** eck`gone is now known as eck`14:58
*** pchavva has quit IRC14:59
*** masco has quit IRC15:00
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Fix up the rabbitmq-ready check  https://review.openstack.org/52740315:00
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: Wait for rabbitmq_ready tag  https://review.openstack.org/52740415:00
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Only create veritas rabbitmq users on the bootstrap node  https://review.openstack.org/52837815:00
*** zaneb has joined #tripleo15:01
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates stable/pike: Deprecate hyperconverged-ceph.yaml  https://review.openstack.org/52908915:01
*** abishop has quit IRC15:02
*** pchavva has joined #tripleo15:03
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add the option to run the container-check script  https://review.openstack.org/50102815:08
mwhahahatbarron: is manilla enabled in one of the scenarios?15:09
mwhahahanm found it in scenario00415:11
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537215:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Introduce RHSM service  https://review.openstack.org/52680815:15
*** abregman has quit IRC15:16
*** Vijayendra has quit IRC15:16
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test RHSM with role variable  https://review.openstack.org/52881615:16
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test RHSM with role variable  https://review.openstack.org/52881615:16
*** Vijayendra has joined #tripleo15:17
trown|ruckrlandy|rover: trying to comment on https://review.openstack.org/501028 but gerrit is taking forever... I like the solution, but I think you have a typo15:18
*** anilvenkata has quit IRC15:18
trown|ruckrlandy|rover: should be /etc/yum.repos.d instead of etc/yum.repos.d15:19
rlandy|rovertrown|ruck: ugh - you're right - thanks - copy paste error from my test code15:19
trown|ruckrlandy|rover: ah and now gerrit responds finally :P15:19
*** anilvenkata has joined #tripleo15:21
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add the option to run the container-check script  https://review.openstack.org/50102815:21
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537215:22
rlandy|rovertrown|ruck: thanks - fixed ... watching the CI on https://review.openstack.org/#/c/515372/15:22
trown|ruckrlandy|rover: yep, I went ahead and +2'd, because I was +2 before, and I tested the stuff you added and it works15:23
*** hjensas has joined #tripleo15:23
*** karthiks has quit IRC15:25
rlandy|rovertrown|ruck: I think we are going to have to test  this on rhel as well - which we have not accounted for :(15:26
rlandy|roverright now our gates are 11 not 12 but they should switch15:26
trown|ruckrlandy|rover: we could merge it as is though... and just add something for 12 if we need it15:27
trown|ruckrlandy|rover: im not sure we do though15:27
trown|ruckrlandy|rover: it just wont disable any repos on rhel... but that is not that big a deal I think15:28
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Parameterize ceph-ansible forks in Tripleo Heat templates  https://review.openstack.org/52812515:29
mwhahahaso scenario003 seems to be failing a lot on https://bugs.launchpad.net/tripleo/+bug/1738493 which is odd because i don't think we change anything on the undercloud between the scenarios15:30
openstackLaunchpad bug 1738493 in tripleo "undercloud installation fails waiting for create default plan execution to finish" [High,Triaged]15:30
*** cmurphy has left #tripleo15:31
mwhahahaspeaking of annoying changes15:32
mwhahahatrown|ruck, rlandy|rover: we seem to no longer be renaming the logs to .txt15:32
mwhahahahttp://logs.openstack.org/53/502553/5/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/c3804d7/logs/undercloud/var/log/mistral/15:32
*** nyechiel has joined #tripleo15:32
*** abregman has joined #tripleo15:36
*** ykarel has joined #tripleo15:37
rlandy|roverhmmm ... comparing to https://thirdparty.logs.rdoproject.org/jenkins-tqe-gate-rhos-11-ci-rhos-ovb-minimal-pacemaker-public-bond-448/undercloud/var/log/mistral/  ... .log.txt.gz15:37
*** liverpooler has quit IRC15:38
mwhahahacollect-logs failed15:39
mwhahahathats why15:39
mwhahahahttp://logs.openstack.org/53/502553/5/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/c3804d7/job-output.txt.gz#_2017-12-19_09_59_08_38863915:39
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: Document role specific parameters tagging  https://review.openstack.org/52883915:39
mwhahahahttp://logs.openstack.org/53/502553/5/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/c3804d7/logs/quickstart_collect_logs.txt.gz15:39
mwhahahadmsimard: -^ dns issues during pip install15:40
dmsimardmwhahaha: thanks15:40
*** hjensas has quit IRC15:43
*** marrusl has joined #tripleo15:43
*** nyechiel has quit IRC15:47
*** liverpooler has joined #tripleo15:48
*** jtomasek has quit IRC15:50
*** hjensas has joined #tripleo16:02
*** apetrich has joined #tripleo16:08
dtantsurfolks, I'd like to close the ansible-deploy blueprint. do you think https://review.openstack.org/#/c/526439/ can go in? it seems like it passed all jobs in at least one recheck, and it works for me locally16:09
socialddd16:10
socialsorry16:10
dtantsursocial: WE SAW WHAT YOU DID THERE16:10
*** ykarel has quit IRC16:11
*** agurenko has quit IRC16:12
*** agurenko has joined #tripleo16:14
*** hjensas has quit IRC16:16
*** brault has quit IRC16:20
*** brault has joined #tripleo16:21
*** atarlov has quit IRC16:25
*** brault_ has joined #tripleo16:25
openstackgerritMarios Andreou proposed openstack/tripleo-docs master: WIP: FFU Developer docs  https://review.openstack.org/52665616:25
*** brault has quit IRC16:26
*** cylopez has joined #tripleo16:27
openstackgerritRyan Brady proposed openstack/tripleo-common master: Select Roles Workflow  https://review.openstack.org/52828216:27
*** hjensas has joined #tripleo16:28
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: WIP Make sure haproxy config for redis uses on-marked-down shutdown-sessions  https://review.openstack.org/52910716:28
dtantsurre https://review.openstack.org/#/c/526663/2/doc/source/install/advanced_deployment/ansible_deploy_interface.rst@51 will the same things work with containerized undercloud?16:29
dtantsurdprince: ^^^16:29
*** janki has quit IRC16:31
shardydtantsur: No the undercloud install just reads the local filesystem16:31
shardywe might need to expose a variable to enable extra paths unless it's somewhere ansible already searches I guess16:32
dtantsurshardy: so I can e.g. keep stuff in /etc/ironic (as in the link above)?16:32
*** rbrady is now known as rbrady-afk16:33
shardydtantsur: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/undercloud_deploy.py#L38016:34
*** moshele has joined #tripleo16:34
shardydtantsur: probably not as that's not a standard location for ansible, we'd need to make it configurable e.g add a flag to that command16:34
dtantsurshardy: I think there is confusion. this is not the same ansible invocation as here16:34
dtantsurshardy: tl;dr I'm talking about ability of ironic-conductor to see /etc/ironic/<something> that was created by an operator on the undercloud16:35
dtantsurignore the word "ansible" for now16:35
shardydtantsur: Ok I was confused by the mention of deploy*artifcacts and the undercloud installer :)16:35
dtantsurif you're curious what it's all about, you can try reading http://specs.openstack.org/openstack/ironic-specs/specs/approved/ansible-deploy-driver.html16:36
dtantsurtl;dr everyone is using ansible, and ironic can now as well :)16:36
*** shreshtha has joined #tripleo16:36
shardydtantsur: Ok, in that case no, modifying /etc/ on the host won't expose anything to the ironic-conductor service container16:38
shardywe bind mount a few specific directories in here:16:39
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ironic-conductor.yaml#L11616:39
*** cylopez has quit IRC16:39
*** moshele has quit IRC16:39
dtantsurout of curiosity, why not bind /etc/ironic completely?16:41
*** abregman has quit IRC16:41
dtantsurshardy: do you have any idea how to better handle it?16:41
shardydtantsur: there is no /etc/ironic on the host in the container case, we don't necessarily have the deps there to run puppet and generate the config files16:42
shardyinstead it runs in a temporary config generating container, with the results written out to /var/lib/config-data16:42
dtantsurah, I see16:43
openstackgerritMartin André proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744516:43
dtantsurshardy: do we have a way for an operator to say "also inject this"?16:43
shardydtantsur: Not really, the bind mounts kind of do that, but arbitrarily mutating the containers is a bit of a non-containerish anti-pattern I think ;)16:44
shardydtantsur: I'd probably consider if a special "extras" directory could be added e.g in /var/lib/ironic16:44
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ironic-inspector.yaml#L14116:44
shardybut really anything the container consumes should really be built into the container image16:44
dtantsurshardy: except that operators have to be able to modify it...16:44
shardybind mounts are an OK hack for development tho16:45
shardydtantsur: Ok you'll have to provide a special bind mount then16:45
shardyand deal with the fact that you can't detect the definition of the container by the image hash16:45
Tengumwhahaha: ah, thanks for the recheck - is the CI issue still present?16:45
mwhahahaTengu: not sure it seems to be IO related on cloud nodes16:46
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add the option to run the container-check script  https://review.openstack.org/50102816:46
*** salmankhan has quit IRC16:46
Tengumwhahaha: duh :/. the kind of thing hard to correct :/16:46
shardydtantsur: e.g I assume if the data in the bind mount changes, you don't need to restart the conductor, it just reads dynamically from the ansibile stuff?16:46
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537216:46
dtantsurshardy: correct16:46
shardydtantsur: ack, a bind mount $somewhere will probably work then - dprince may have other ideas when he's back16:47
dtantsurshardy: currently where is /var/lib/ironic coming from?16:47
*** marios has quit IRC16:47
*** hewbrocca is now known as hewbrocca_afk16:52
shardydtantsur: in the image it's owned by openstack-ironic-common but since that's not installed on the host, it's created in host_prep_tasks in docker/services/ironic-pxe.yaml16:52
dtantsuraha, so it does not help much, I guess..16:53
dtantsurI'm also thinking how to make this optional.. because the feature itself is optional16:53
shardydtantsur: well, it's a directory that is mounted into the container, so you could add a directory to host_prep_tasks in the conductor template16:53
shardythe main risk is avoiding any conflict with the package installed into the image I guess16:54
dtantsurshardy: is it mounted from the host?16:54
*** agurenko has quit IRC16:54
shardydtantsur: yes I linked the line which does it16:54
dtantsuroh, true, I did not realize it16:54
*** agurenko has joined #tripleo16:55
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ironic-inspector.yaml#L14116:55
shardyOh wait you said conductor, sec16:55
dtantsurmmm, this starts shaping in my head :)16:55
*** agurenko has quit IRC16:55
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ironic-conductor.yaml#L12216:55
shardydtantsur: heh, yeah there are a few layers to it, but the individual interfaces aren't too tricky :)16:56
*** lucasagomes is now known as lucas-afk16:56
dtantsurright, thanks! it does look quite anti-FHS to use it for something like playbooks16:56
*** salmankhan has joined #tripleo16:58
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/newton: pacemaker: create Mysql_user once Galera is ready (puppet4)  https://review.openstack.org/52911416:59
* dtantsur starts getting a headache17:00
*** ebarrera_ has joined #tripleo17:01
*** Goneri has joined #tripleo17:03
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs master: [WIP] Document using Ironic Ansible deploy interface  https://review.openstack.org/52666317:11
*** ccamacho has quit IRC17:11
dtantsurshardy: switched to /var/lib/ironic for now ^^ I guess we can think of something better later on..17:12
dtantsurthanks for your help17:13
*** ebarrera_ has quit IRC17:13
*** anilvenkata has quit IRC17:19
*** akrivoka has quit IRC17:20
dtrainorhey there mandre, how's the ui containerization going?  can I be of any help?17:21
EmilienMweshay: happy birthday old man ;-)17:21
weshaythanks brotha17:21
dtrainorweshay++17:21
*** trown|ruck is now known as trown|lunch17:22
weshayturns out it's also rasca's bday :)17:22
weshaymy italian brotha from another motha17:22
*** dsariel has joined #tripleo17:22
*** gkadam has quit IRC17:22
rascaLOL17:26
ludovicHello everyone, is someone have already write a scheduler filter to prioritize rebuild of instances ?17:28
ludovicin the case of an host-evacuation for example17:28
ludovicI'm interesting about this17:29
mandredtrainor: it worked for me yesterday, I haven't looked at it today, I'm letting honza take over17:32
bnemecludovic: That sounds like a question for #openstack-nova.17:33
mandredtrainor: thanks, though17:33
dtrainorcool!  happy to help if you or honza are in a pinch.17:33
mandreweshay, rasca: wooo party time! happy bday17:34
ludovicbnemec: yes i asked the question17:34
*** brault_ has quit IRC17:38
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Add config to set neutron mtu in undercloud/overcloud  https://review.openstack.org/50966017:38
*** brault has joined #tripleo17:38
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: Test to see if MTU tempest tests work with Neutron MTU set  https://review.openstack.org/52829217:39
sri_EmilienM, Hi, quick question in stable/pike overcloud did logs enable for haproxy ?17:39
*** akrivoka has joined #tripleo17:39
*** suuuper has quit IRC17:41
*** brault has quit IRC17:43
openstackgerritJuan Badia Payno proposed openstack/puppet-tripleo master: Let collectd manage repo and added unit test  https://review.openstack.org/52162917:43
openstackgerritMatt Young proposed openstack/tripleo-quickstart master: also make newton use over-as-under  https://review.openstack.org/51246217:45
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add docker-registry service  https://review.openstack.org/52613217:46
openstackgerritDan Prince proposed openstack/python-tripleoclient master: Configure undercloud docker registry/mirror  https://review.openstack.org/52614717:46
*** derekh has quit IRC17:47
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs master: [WIP] Document using Ironic Ansible deploy interface  https://review.openstack.org/52666317:48
*** catintheroof has quit IRC17:50
mwhahahasri_: no that didn't merge until queens17:50
*** catintheroof has joined #tripleo17:51
*** mdnadeem has quit IRC17:52
sri_mwhahaha, i see thanks17:52
*** athomas has joined #tripleo17:52
aleemwhahaha, fyi -- https://review.openstack.org/52912617:53
aleemwhahaha, will pull that in later today and see if it works17:53
aleeback in a little bit17:54
mwhahahaalee: ok17:54
*** alee is now known as alee_lunch17:54
mwhahahasri_: https://review.openstack.org/#/c/497524/17:56
mwhahahathe little patch that could eventually17:56
*** jtomasek has joined #tripleo17:57
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Telemetry Needs Redis  https://review.openstack.org/52839218:03
*** catintheroof has quit IRC18:07
*** catintheroof has joined #tripleo18:08
*** dsariel has quit IRC18:09
shardymwhahaha, EmilienM: Hey when you get time pls could you see the discussion on https://review.openstack.org/#/c/52906618:16
*** dsariel has joined #tripleo18:17
EmilienMshardy: in middle of review time, so perfect ;-)18:17
shardyI'm not sure when the definition of ::os_workers was refined for puppet-heat, and/or how we can manage both a lower and upper bound18:17
shardybasically we don't want the minimum to reduce, but we need some way to cap so that e.g the baremetal box rook is using doesn't try to run 64 heat-engine workers and use 100G of RAM18:17
EmilienMshardy: ok let me dig that, I think we had new facts lately18:18
shardyEmilienM: ack thanks, I know things changed in this area but I'm not quite clear on the timeline ref the patch from trown|lunch that I'm now apparently reverting18:19
*** rmascena has joined #tripleo18:19
EmilienMit sounds like we might want a new os_worker18:19
EmilienMshardy: these are the facts that you can use *now*: https://github.com/openstack/puppet-openstacklib/blob/master/lib/facter/os_workers.rb18:20
EmilienMshardy: but we might need new use cases iiuc18:20
shardyEmilienM: hmm yeah we may need something with the minimum 4 that's capped at 12, either in the facts or calculated in the instack-undercloud manifest18:22
*** raildo has quit IRC18:22
EmilienMshardy: we have 2 options18:23
*** links has quit IRC18:23
EmilienMshardy: either creating a new fact that will match your formula, in puppet-openstacklib, and use this fact from i-u18:23
EmilienMshardy: or compute the parameter value in python, from instack-undercloud.18:24
EmilienMshardy: we need to find a solution that can be re-used by containerized undercloud18:24
shardycan we do :os_workers * 2 would probably be a reasonably improvement on what rook is currently seeing I guess18:24
shardyEmilienM: yeah I guess the new fact is easier in that regard18:24
EmilienMshardy: I'll push a patch18:27
*** dsariel has quit IRC18:27
shardyCan I get a second reviewer/approver on https://review.openstack.org/#/c/528342/ and https://review.openstack.org/#/c/528354 please18:27
shardythen we can land https://review.openstack.org/#/c/528344/ and avoid a rebase if anything changes in tripleo-validations18:28
shardyEmilienM: ack thanks!18:28
EmilienMwill review shortly after my call18:28
shardyEmilienM: thanks, I think you already +2'd the tripleo-common ones :)18:29
*** gfidente is now known as gfidente|afk18:31
*** atarlov has joined #tripleo18:32
*** jpena is now known as jpena|off18:33
EmilienMshardy: what # of workers do you want to cap?18:35
shardyEmilienM: I think the os::workers cap of 12 is probably reasonable, but CPU/2 may be too small for e.g CPUs=418:36
*** atarlov has quit IRC18:37
EmilienMwe could do [ [ (processors.to_i / 2), 4 ].max, 12 ].min18:37
shardyit's hard to ensure a perfect default, but I guess we don't gain much by running 64 heat-engine workers on a 64 core machine unless the DB is also running similar numbers of workers and it's on very fast storage18:37
*** atarlov has joined #tripleo18:37
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: DONT REVIEW: used for testing only  https://review.openstack.org/44727618:37
shardyEmilienM: yeah that's probably fine - IIRC we had issues in CI when the workers went below 418:38
EmilienMno I wonder if we want to change the default or create a new fact18:38
EmilienMI'm a bit scared to change worker default tbh, the impact can be really bad. THe fact is widely used18:38
EmilienMI would maybe create a new one18:38
shardyEmilienM: not sure - this is a bit specific to TripleO because we know heat-engine is stressed more than most other undercloud services18:38
shardyin generic small deployments 2 heat-engine workers is fine18:39
EmilienMshardy: we could call it os_workers_heat_engine for now18:40
*** trown|lunch is now known as trown18:40
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537218:40
*** trown is now known as trown|ruck18:40
EmilienMshardy: i'm preparing a patch18:40
EmilienMshardy: https://review.openstack.org/52913018:46
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test updates of container with base os repos  https://review.openstack.org/52913118:48
sshnaidmrlandy|rover, let's compare ^^18:48
rlandy|roversshnaidm: k - time and what actually gets updated18:50
*** atarlov has quit IRC18:51
*** brault has joined #tripleo18:52
*** brault has quit IRC18:53
vkmchey guys, I'm hitting some issues with gnocchi after destroying and redeploying my overcloud18:53
vkmcprecisely http://paste.openstack.org/show/629358/18:53
vkmcthis didn't happen the first time I ran it18:53
vkmcbut now I'm being able to reproduce it everytime18:53
vkmcany known workaround?18:53
*** brault has joined #tripleo18:54
*** dtantsur is now known as dtantsur|afk18:56
*** catintheroof has quit IRC18:56
mwhahahavkmc: would need to see the actual error out of the stack failures18:57
vkmchmm18:58
vkmclet me see18:58
mwhahahavkmc: i'm assuming you're running into an ordering thign that's currently being worked on by sileht around when to run the db sync18:58
vkmcmwhahaha, aybe18:58
vkmcmaybe*18:58
vkmcso I don't get anything useful from heat resources-show18:59
vkmcI guess I should ssh to the controller18:59
vkmcat least the nodes are being provisioned18:59
mwhahahaso openstack stack failures list should give you it18:59
mwhahahait shows up in our post ci let me see if ic an get the right command18:59
vkmcoh, that would be easier for me18:59
*** holser__ has quit IRC19:00
*** jokke_ has quit IRC19:00
*** fpan has quit IRC19:00
mwhahahavkmc: openstack stack resource list -n5 overcloud19:00
mwhahahaer no19:00
vkmcmwhahaha, http://paste.openstack.org/show/629360/19:00
mwhahahahttps://github.com/openstack/tripleo-quickstart-extras/blob/0bdd3c790fb3f335f26ebdf83e91767d0a4af18e/roles/collect-logs/templates/get_host_info.sh.j2#L3119:01
*** jokke_ has joined #tripleo19:01
mwhahahaso that's what we run in ci to get the postci.txt which will have more details19:01
rookso os_workers has changed shardy EmilienM ?19:03
*** gkadam has joined #tripleo19:03
mwhahaharook: yes19:03
vkmcok, I'll run that19:03
* rook looks it up19:03
EmilienMshardy: the stuff with rhsm/roles worked fine, thanks again19:04
rooklooks like it was a update to os_workers_large?19:04
*** alee_lunch is now known as alee19:04
mwhahaharook: if you're talking about the instack-undercloud change it's been this way for a while19:05
EmilienMrook: without cap, yes19:05
shardyEmilienM: great, np - thanks for the puppet patch! :)19:05
rookah, os_workers for the uc is different than the oc19:05
EmilienMrook: but we need a cap I guess19:05
rookwhich i always find a PITA19:05
rookcan we just standardize :D19:05
EmilienMI guess the containerized undercloud is doing it19:05
EmilienMsince it won't use the same manifest, but re-use the tht ones19:06
vkmcmwhahaha, I just generated a file with more than 10k lines19:06
vkmcis that right?19:07
rookos_workers *2 might get us in trouble shardy -- at small scale lets say each heat-engine worker gets 512MB of RAM...19:07
mwhahahavkmc: yea19:07
mwhahahavkmc: go look for Error19:07
mwhahahavkmc: that's usually how i find it19:07
shardyrook: feel free to suggest some other algorithm, it's really hard to optimized for unknown hardware and scale19:07
mwhahahaaka tripleo19:08
vkmcmwhahaha, cool, yes... I see something http://paste.openstack.org/show/629362/19:08
mwhahahavkmc: so there was a patch for 24448:            "Error: /Stage[main]/Gnocchi::Wsgi::Apache/Openstacklib::Wsgi::Apache[gnocchi_wsgi]/File[gnocchi_wsgi]: Could not evaluate: Could not retrieve information from environment production source(s) file:/usr/lib/python2.7/site-packages/gnocchi/rest/app.wsgi",19:08
rookshardy I am fine with *12, that means we would max out at 24 (right?)19:08
mwhahahavkmc: you've got an outdated puppet-gnocchi19:08
shardyhttps://review.openstack.org/#/c/529130/2/lib/facter/os_workers.rb19:08
vkmcall right, I'll update that19:09
vkmcthanks a lot mwhahaha :)19:09
shardyrook: ^^ that patch from Emilien is the same as the current os_workers except the minimum is 4 instead of 219:09
*** lblanchard has quit IRC19:09
rookshardy:ack19:09
shardyrook: I guess that will be better than the *2 approach, I was thinking of how we could do it without changing puppet-openstacklib on all the branches19:09
openstackgerritHarald JensÃ¥s proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754419:11
*** rbrady-afk is now known as rbrady19:13
*** shardy has quit IRC19:14
*** ludovic has quit IRC19:26
rookEmilienM: mwhahaha thoughts on having another patchset where the current max value of 12 becomes another fact os_workers_max --- which would allow us to change # of workers vs defining os_workers_large|small ?19:29
*** catintheroof has joined #tripleo19:29
EmilienMrook: do you want to push over it19:30
EmilienM?19:30
mwhahaharook: sounds complex and painful19:30
* mwhahaha would prefer no19:30
*** hjensas has quit IRC19:30
EmilienMwe should make it easy19:30
rookmwhahaha: roger.19:31
*** shreshtha has quit IRC19:32
*** fpan has joined #tripleo19:32
*** shreshtha has joined #tripleo19:32
*** hjensas has joined #tripleo19:32
*** openstack has joined #tripleo19:42
*** ChanServ sets mode: +o openstack19:42
*** hjensas has quit IRC19:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Rhsm  https://review.openstack.org/52680819:45
*** lblanchard has joined #tripleo19:45
*** yamahata has joined #tripleo19:47
*** tdasilva has quit IRC19:47
*** tdasilva has joined #tripleo19:47
itlinuxhello guys any suggestion on this error by chance http://paste.openstack.org/show/629370/ doing an upgrade from Ocata to Pike in BM no containers19:48
*** rlandy|rover is now known as rlandy|rover|bia19:49
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Manila network plugin address family support  https://review.openstack.org/52222919:52
*** itlinux has quit IRC19:52
*** itlinux has joined #tripleo19:52
*** salmankhan has quit IRC19:53
*** dprince has quit IRC19:55
*** fragatina has quit IRC19:56
*** hjensas has joined #tripleo20:09
*** pkovar has quit IRC20:11
*** jtomasek has quit IRC20:15
*** gkadam has quit IRC20:18
*** dprince has joined #tripleo20:23
openstackgerritIan Main proposed openstack/tripleo-common master: Manage legacy OS::TripleO::RandomString resources in constants  https://review.openstack.org/52615120:33
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Remove certificate before updating it  https://review.openstack.org/52885120:33
openstackgerritIan Main proposed openstack/python-tripleoclient master: WIP: Disable /etc/hosts checks until we determine requirements.  https://review.openstack.org/52528620:35
*** shardy has joined #tripleo20:36
openstackgerritMerged openstack/tripleo-heat-templates master: fix the description of CongressDebug  https://review.openstack.org/52894120:37
openstackgerritMerged openstack/tripleo-heat-templates master: Improve comment re nova live_migration_tunnelled setting  https://review.openstack.org/52868220:37
openstackgerritMerged openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/52893120:37
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Remove osd_pool_default_min_size to allow Ceph cluster to do the right thing by default  https://review.openstack.org/50882320:37
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Remove osd_pool_default_min_size to allow Ceph cluster to do the right thing by default  https://review.openstack.org/50882120:37
openstackgerritMerged openstack/tripleo-quickstart master: config: Provide a Master UC and Newton OC release config  https://review.openstack.org/51919520:37
openstackgerritIan Main proposed openstack/tripleo-quickstart-extras master: DNM: Update quickstart extras with undercloud install for containers  https://review.openstack.org/51744420:37
*** rlandy|rover|bia is now known as rlandy|rover20:38
*** atarlov has joined #tripleo20:42
*** athomas has quit IRC20:44
openstackgerritSteven Hardy proposed openstack/tripleo-common master: Add role_data* keys to inventory  https://review.openstack.org/52835420:48
openstackgerritSteven Hardy proposed openstack/tripleo-common master: Allow separate parameters to TripleoInventory class  https://review.openstack.org/52914620:48
openstackgerritSteve Baker proposed openstack/tripleo-common master: Always load rsyslog-base container  https://review.openstack.org/52152920:49
*** atarlov has quit IRC20:49
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Rhsm  https://review.openstack.org/52680820:51
*** shreshtha has quit IRC20:51
openstackgerritMerged openstack/tripleo-heat-templates master: undercloud: set default_resource_class to 'baremetal'  https://review.openstack.org/52607820:52
openstackgerritMerged openstack/tripleo-quickstart-extras master: dump out all the variables used  https://review.openstack.org/52613820:52
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add reproducer script for OVB and multinode jobs  https://review.openstack.org/52574320:52
openstackgerritMerged openstack/puppet-tripleo master: Allow wsrep_max_ws_rows and wsrep_max_ws_size to use defaults  https://review.openstack.org/52876520:52
openstackgerritMerged openstack/puppet-tripleo master: Add Basic Authentication support for HAProxy  https://review.openstack.org/52549220:52
openstackgerritMerged openstack/tripleo-heat-templates master: Enable queueing to use barbican workers for key generation  https://review.openstack.org/52838820:52
openstackgerritMerged openstack/tripleo-quickstart master: add dry-run or list-tasks to quickstart.sh  https://review.openstack.org/52620820:52
openstackgerritJohn Fulton proposed openstack/tripleo-common master: Parameterize ceph-ansible forks in Mistral Workflow  https://review.openstack.org/52812420:55
*** morazi has quit IRC20:58
mwhahahaquick we need to approve more stuff20:58
mwhahahathe gate is empty20:59
mwhahaha(for tripleo at least)20:59
fultonjha20:59
shardyhttps://review.openstack.org/#/c/528342/ needs a +A ;)21:00
shardyThen https://review.openstack.org/#/c/528344/ after it merges21:01
*** catintheroof has quit IRC21:02
mwhahahashardy: is there a distgit patch in between somewhere?21:02
mwhahahaor is that next21:02
shardymwhahaha: No as I've not moved the tripleo-ansible-inventory script (yet)21:03
mwhahahak21:03
*** catintheroof has joined #tripleo21:03
*** ebarrera_ has joined #tripleo21:06
*** catintheroof has quit IRC21:07
*** rbowen has quit IRC21:10
*** lblanchard has quit IRC21:15
*** lblanchard has joined #tripleo21:16
*** jfrancoa has quit IRC21:18
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Manila network plugin address family support  https://review.openstack.org/52222921:27
*** rmascena has quit IRC21:27
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Make UI accessable by default on OVB  https://review.openstack.org/50796921:28
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Allow OVB to attach undercloud to existing floating IP  https://review.openstack.org/50639721:28
tbarronmwhahaha: so you are seeing the lack of OOO expertise over in storage ^^^ - or at least in my squad 522229 - so thanks for the help21:28
mwhahahatbarron: i know, only one way to learn is through a barrage of -1's21:29
tbarronmwhahaha: oh, I have no reaction to -1s other than to learn and repeat, at least when administered by sensai in a domain21:30
*** salmankhan has joined #tripleo21:30
honzamandre: I remember you telling me about a hack I needed to do to fix hiera on the containerized undercloud.  I can't find it anywhere now! (I grepped my irc logs and email) // dtrainor21:31
honza$ sudo hiera admin_password => nil21:31
honzaand the run.sh script fails on hiera21:31
dtrainorhm.... not ringing any bells right now21:32
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Make UI accessable by default on OVB  https://review.openstack.org/50796921:32
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Allow OVB to attach undercloud to existing floating IP  https://review.openstack.org/50639721:32
honzadtrainor: thanks21:33
dtrainorsorry.  trying to think of why that would happen21:33
dtrainordo you have hieradata?21:33
*** salmankhan has quit IRC21:34
honzadtrainor: yes, in /etc/puppet/hieradata21:35
*** myoung is now known as myoung|bbl21:35
dtrainoris there a admin_password in it?  (a visual inspection / search with vim or grep)21:35
dtrainor*an21:35
honzadtrainor: in keystone::admin_password in service_configs.json21:36
mwhahahatbarron: give that a shot21:37
*** pchavva has quit IRC21:42
openstackgerritSteven Hardy proposed openstack/tripleo-common master: TripleoInventory make session optional  https://review.openstack.org/52916521:42
*** trown|ruck is now known as trown|outtypewww21:43
*** morazi has joined #tripleo21:44
GheRiverosl21:48
honzamandre: this is the error during doit.sh run http://paste.openstack.org/show/629382/21:50
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Manila network plugin address family support  https://review.openstack.org/52222921:51
dtrainori see21:53
dtrainori'm not sure honza, i'm sorry.  i'd expect that to work.  it might be something fundamental to containers?  i'm not sure.21:53
tbarronmwhahaha: thanks.  Just curious: this is simpler, but would that ugly way of doing heat conditions have worked?21:55
honzadtrainor: don't sweat it, thanks --- just thought you might know off the top of your head21:55
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Add connection timeout options to ssh  https://review.openstack.org/52916821:57
mwhahahatbarron: no it would not have21:57
mwhahahatbarron: there's a similar approach but that syntax was not right21:58
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs032: activate config-download  https://review.openstack.org/52876121:58
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: fs032: activate config-download  https://review.openstack.org/52876121:58
tbarronmwhahaha: k, I grok this syntax better anyways21:58
*** holser__ has joined #tripleo21:58
tbarronmaybe, makes sense to me at the moment21:59
mwhahaha:D21:59
mwhahahatbarron: actually it might have worked, there's like 12 ways to do the same thing22:00
mwhahahatbarron: https://github.com/openstack/tripleo-heat-templates/blob/107b610923ba5d39f90c3a6a63bf2d3642e1b35d/puppet/services/panko-base.yaml#L7122:00
mwhahahatbarron: oh you still need the conditions22:01
tbarronmwhahaha: I added a condition22:01
tbarronbut it's butt ugly22:01
mwhahahatbarron: it got dropped22:01
mwhahahano you need both22:01
honzamandre: and here is what happens when i run run.sh http://paste.openstack.org/show/629388/22:01
mwhahahatbarron: welcome to the wonderful world of heat templates :D22:03
tbarronmwhahaha: looking at the review, I do see it's not there.  I went back to prove you wrong and somehow it had disappeared.22:03
mwhahahatbarron: so you need the conditions item so you can use the if conditionals in the output22:03
mwhahahaand for the record patchset 4 would have probably worked, but wasn't ideal22:03
tbarronmwhahaha: yeah, I swear I had that.  Got lost in translation in the git revie wupload.22:03
tbarron:D22:03
mwhahahathe internets at your bits22:04
tbarronjust joking, obviously a brain-to-finger malfunction22:04
mwhahahadd my favorite typo22:04
mwhahahanext to i:wq22:05
tbarronmwhahaha: anyways, thanks for the help.  Think I'm going to do some haskell tonight.22:05
tbarronlol22:05
mwhahahanp22:05
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Manila network plugin address family support  https://review.openstack.org/52222922:10
*** jcoufal has quit IRC22:13
*** akrivoka has quit IRC22:17
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Create flavors for undercloud  https://review.openstack.org/52681022:21
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Telemetry Needs Redis  https://review.openstack.org/52839222:21
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Add docker-registry service  https://review.openstack.org/52613222:21
openstackgerritIan Main proposed openstack/python-tripleoclient master: WIP: Disable /etc/hosts checks until we determine requirements.  https://review.openstack.org/52528622:21
openstackgerritIan Main proposed openstack/python-tripleoclient master: Configure undercloud docker registry/mirror  https://review.openstack.org/52614722:21
*** florianf has quit IRC22:22
openstackgerritIan Main proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744522:26
*** lblanchard has quit IRC22:28
*** rcernin has joined #tripleo22:28
openstackgerritGiulio Fidente proposed openstack/tripleo-common master: Parse deploy_status_code vs status_code in deployment workbook  https://review.openstack.org/52917222:30
openstackgerritMark Hamzy proposed openstack/python-tripleoclient master: [WIP] Support multiple architectures  https://review.openstack.org/52806022:34
openstackgerritMark Hamzy proposed openstack/tripleo-common master: [WIP] Support multiple architectures  https://review.openstack.org/52800022:36
*** holser__ has quit IRC22:37
*** shardy has quit IRC22:37
aleemwhahaha, is there anywhere in the logs for a tripelo test that you can see the actual files on the system -- ie. glance.py in /usr/lib/python... /nova/ .. ?22:51
mwhahahaalee: no because they should be rpms22:51
aleemwhahaha, can I download the rpm somewhere to unpack it?22:52
mwhahahaalee: you should be able to go pull down the rpm if you're wanting to see something22:52
mwhahahaalee: do you have a change in particular that you're investigating?22:52
aleemwhahaha, yup =- getting ..22:53
aleemwhahaha, https://review.openstack.org/52406422:53
mwhahahaalee: so http://logs.openstack.org/36/527136/2/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/cc76cd0/logs/subnode-2/etc/yum.repos.d/delorean.repo.txt.gz is teh current repo22:54
mwhahahaso you could go pull down the package in question and see22:54
mwhahahaso we haven't had a promotion in ~13 days22:54
aleemwhahaha, that review depends on https://review.openstack.org/#/c/527136/  which depends on https://review.openstack.org/#/c/528067/, which depends on https://review.openstack.org/#/c/528017/22:55
aleethe last one is where I'd expect to see a change22:55
mwhahahaok so we should build the rpm22:55
mwhahahalet me see22:55
aleeassuming all of the dependent changes get pulled in, right?22:55
mwhahahayes they should22:55
mwhahahahttp://logs.openstack.org/64/524064/5/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/8345e93/logs/delorean_logs/22:56
*** myoung|bbl is now known as myoung22:56
mwhahahaso we built nova http://logs.openstack.org/64/524064/5/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/8345e93/logs/delorean_logs/e2/df/e2df09e9bb4cda7491f5ee8bf35e84975b03af88_dev/build.log.txt.gz22:56
mwhahahaI don't think we save any of the rpms we build in the logs22:57
aleemwhahaha, well I can pull it from the repo above , right?22:58
mwhahahaalee: oh we don't rebuild the containers22:58
mwhahahayou got container'd22:58
aleeoh ..22:58
*** lennyb has quit IRC22:58
aleeok ..22:59
mwhahahaEmilienM, weshay, rlandy|rover -^ we really need to figure out the rebuilding of the containers or injecting of the pacakges. this is a major regression from the non-containerized stuff22:59
*** jmelvin has quit IRC22:59
mwhahahaso no, you can't actually test depends-on openstack code in the overcloud anymore23:00
mwhahahathanks containers23:00
*** lennyb has joined #tripleo23:00
mwhahahaundercloud gets it still23:00
aleemwhahaha, cool  well - I can at least see that it looks right23:01
mwhahahamaster promotion is at 13 days too23:01
openstackgerritIan Main proposed openstack/instack-undercloud master: DNM, undercloud containers TESTING ONLY  https://review.openstack.org/51811823:01
mwhahahathanks rdo cloud23:01
* mwhahaha flips tables 23:01
aleemwhahaha, but of course that doesn't help the overcloud tests ..23:01
mwhahahayea23:01
aleemwhahaha, ok - so the only way for me to do this is to get the nova review pushed and the image rebuilt23:02
mwhahahayea23:02
aleemwhahaha, same for the barbican change I mentioned earlier23:02
mwhahahayup23:02
aleeyippee23:02
rlandy|rovermwhahaha: we're testing the patch on updating containers in the overcloud: https://review.openstack.org/#/c/50102823:03
aleemwhahaha, well, I'm glad I talked to you , instead of whacking head against table for any longer23:03
mwhahahaalee: so you could try depends on https://review.openstack.org/#/c/50102823:03
aleerlandy|rover, does it work? would I need to do anything else?23:04
mwhahahaor rather https://review.openstack.org/#/c/515372/1423:04
mwhahahayou could be the first one to find out if it works :D23:04
rlandy|roveralee: there are a series of patches23:04
rlandy|roverwe are getting rdocloud back now - so it will make testing easier23:05
aleerlandy|rover, is https://review.openstack.org/#/c/515372 the right depends?23:05
rlandy|roveryou will need to set update_containers to be true23:06
aleerlandy|rover, where?23:06
mwhahaharlandy|rover: that's the patch that sets it to true23:06
mwhahahaalee: yea 51537223:06
mwhahahahttps://review.openstack.org/#/c/515372/14/roles/overcloud-prep-containers/defaults/main.yml23:07
aleeyup23:07
rlandy|roverhttps://review.openstack.org/#/c/515372 is our test patch - we don't plan to merge that one23:07
rlandy|roverbut that is what you need23:07
aleerlandy|rover, sure -- well as a test, I can try it out :)23:07
*** paramite has quit IRC23:07
rlandy|roveralee; sure - ping if you need help23:08
aleeI dont mind being a guinea pig23:08
rlandy|roverapologies, this work got deplayed a couple times23:08
aleecool -- at least theres a way forward23:08
rlandy|roveryeah - so you should see a gating_repo file23:09
rlandy|roverwith your change23:09
rlandy|roverand then look for overcloud_prep_containers log23:09
rlandy|roverwhere you will see the containers being updated23:09
openstackgerritGiulio Fidente proposed openstack/tripleo-quickstart master: Switch Ceph repo from Jewel to Luminous for master  https://review.openstack.org/52917523:10
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Set nova to verify glance images in scenario 2  https://review.openstack.org/52713623:13
weshaymwhahaha, glad to hear it's needed :)23:20
*** fpan has quit IRC23:25
pabelangerOVB question, when you are booting them in testing, via heat, that is to simulate a baremetal.  I take it heat also drive the baremetal in production?23:25
*** bfournie has quit IRC23:27
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test updates of container with base os repos  https://review.openstack.org/52913123:28
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Add image signing tests with barbican  https://review.openstack.org/52406423:30
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Testing new barbican - do not merge  https://review.openstack.org/52918123:30
EmilienMmwhahaha: yes I know it, and I said it weeks ago23:30
*** moshele has joined #tripleo23:32
*** itlinux has quit IRC23:33
*** ebarrera_ has quit IRC23:34
weshaymwhahaha, which change did you need a containers build for?23:37
weshaymaybe I can use it as a test dummy23:37
mwhahahaweshay: alee's got it, we pointed him to sagi's patch that's testing it23:38
*** catinthe_ has joined #tripleo23:43
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: DNM: test container updates  https://review.openstack.org/51537223:44
*** atarlov has joined #tripleo23:46
dpeacockAnyone else getting "failed to query change details from  https://review.openstack.org/changes/?q=change:, error: failed to resolve the target change" out of quickstart, or am I just special? :-)23:48
*** jwb has quit IRC23:48
*** atarlov has quit IRC23:50
*** catintheroof has joined #tripleo23:52
*** catintheroof has quit IRC23:52
dpeacockAh, is this me neglecting to specify something or use --no-gate?23:53
*** catintheroof has joined #tripleo23:53
mwhahahadpeacock: most likely23:54
dpeacockmwhahaha: yeah I was staring at this for twenty minutes and of course it didn't hit me until *after* I'd gone public lol23:54
*** catinthe_ has quit IRC23:55
*** catintheroof has quit IRC23:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260723:59
*** bfournie has joined #tripleo23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!