| *** wolverineav has quit IRC | 00:02 | |
| *** rascasoft has quit IRC | 00:02 | |
| *** wolverineav has joined #tripleo | 00:02 | |
| *** sdake has joined #tripleo | 00:06 | |
| *** wolverineav has quit IRC | 00:06 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 00:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| *** jmcevoy has joined #tripleo | 00:16 | |
| *** jmcevoy has left #tripleo | 00:18 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 00:21 |
| *** wolverineav has joined #tripleo | 00:25 | |
| *** cfontain has quit IRC | 00:25 | |
| *** sdake has quit IRC | 00:29 | |
| jidar | so this is part of the ansible log that's failing for ceph-ansible: https://gist.github.com/326f57b898aacc48a20897d6bedb686d | 00:33 |
| jidar | https://github.com/ceph/ceph-ansible/blob/master/roles/ceph-config/tasks/main.yml#L74 i think is the line it's failing on, which is a template for the ceph.conf file | 00:36 |
| *** rascasoft has joined #tripleo | 00:36 | |
| NobodyCam | Good afternoon Tripleo folks | 00:42 |
| NobodyCam | I have a (dumb) question about the provider-network. in the docs I see "The provider network being added must be attached to both the controller and the compute nodes, since the compute node will attach a VM virtual network interface directly to an attached network interface." | 00:43 |
| NobodyCam | should that be attached to a system interface (eth2) or the ovs bridge? | 00:43 |
| *** rascasoft has quit IRC | 00:45 | |
| *** tosky has quit IRC | 00:46 | |
| *** sdake has joined #tripleo | 00:49 | |
| *** cfontain has joined #tripleo | 00:52 | |
| *** kplant has joined #tripleo | 00:53 | |
| jidar | anybody know how to have ansible execute with verbose on in the ceph-ansible stuff? | 01:08 |
| *** sdake has quit IRC | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 01:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| *** jcoufal has quit IRC | 01:22 | |
| *** cfontain has quit IRC | 01:24 | |
| *** rascasoft has joined #tripleo | 01:33 | |
| *** kplant has quit IRC | 01:43 | |
| *** rascasoft has quit IRC | 01:44 | |
| *** _fragatina has quit IRC | 01:56 | |
| *** jcoufal has joined #tripleo | 02:01 | |
| *** cfontain has joined #tripleo | 02:02 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 02:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| *** rascasoft has joined #tripleo | 02:11 | |
| *** rascasoft has quit IRC | 02:20 | |
| openstackgerrit | caoyuan proposed openstack/tripleo-heat-templates master: Update the home-page to docs link https://review.openstack.org/610368 | 02:28 |
| *** wolverineav has quit IRC | 02:29 | |
| jidar | > CephAnsiblePlaybookVerbosity: 3 | 02:29 |
| jidar | yay! | 02:29 |
| *** cfontain has quit IRC | 02:35 | |
| *** wolverineav has joined #tripleo | 02:36 | |
| *** kplant has joined #tripleo | 02:37 | |
| *** mschuppert has quit IRC | 02:43 | |
| *** rascasoft has joined #tripleo | 02:57 | |
| *** sdake has joined #tripleo | 03:00 | |
| *** psachin has joined #tripleo | 03:02 | |
| *** cfontain has joined #tripleo | 03:03 | |
| *** rascasoft has quit IRC | 03:08 | |
| jidar | https://gist.github.com/7d5b18ba4d9d3fedd9b33abfe9cdacee | 03:09 |
| jidar | ok so I've enabled debugging, re-run the deploy with all the correct things (I would think) and ultimately ceph-ansible is bombing out over _something_ missing, but I have no idea what | 03:09 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 03:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| *** wolverineav has quit IRC | 03:12 | |
| kplant | can you paste the CephAnsibleDisksConfig stanza from your yaml? | 03:14 |
| *** wolverineav has joined #tripleo | 03:14 | |
| *** mmethot has quit IRC | 03:14 | |
| *** apetrich has quit IRC | 03:15 | |
| jidar | https://gist.github.com/fe5f76230858f2a5590263da86f8c949 kplant | 03:15 |
| jidar | its passing along the right disks as I can verify in the inventory.yaml, it gets the scenero, etc | 03:16 |
| kplant | i see from your commented lines you too have been burned by inconsistent drive lettering :-) | 03:16 |
| jidar | hahahah | 03:16 |
| jidar | I wanted to use the serial numbers | 03:16 |
| kplant | yeah wwn's would be nice | 03:17 |
| jidar | but I didn't feel like doing the entire song and dance for each node | 03:17 |
| kplant | but a giant pita | 03:17 |
| kplant | i wish you could do it by description | 03:17 |
| kplant | like type: ssd, size: 240g or something | 03:17 |
| jidar | figured the by-path were close enough as they are the same | 03:17 |
| jidar | this particular ceph nodes disks are all 1tb, so that won't help me | 03:17 |
| kplant | that's a pretty simple yaml and i unfortunately have my own annoyances getting bluestore+lvm to work | 03:18 |
| kplant | i'm afraid i'm not much help | 03:18 |
| *** wolverineav has quit IRC | 03:18 | |
| jidar | can you confirm (at least as best you know) that you don't need dedicated drives | 03:18 |
| kplant | with lvm you don't | 03:18 |
| jidar | or a particular way to setup the journal | 03:18 |
| jidar | k, just double checking I didn't misread it (5 times) | 03:19 |
| *** saneax has joined #tripleo | 03:19 | |
| kplant | you could do a collocated setup with filestore, but that's not great | 03:19 |
| kplant | that doesn't require dedicated drives either | 03:19 |
| jidar | I'd be willing to switch to puppet-ceph at this point if I was sure it would work | 03:19 |
| kplant | maybe a stupid question | 03:20 |
| kplant | are you using ceph-ansible >= 3.2 | 03:21 |
| kplant | and >= luminous | 03:21 |
| jidar | I am | 03:21 |
| *** mmethot has joined #tripleo | 03:21 | |
| jidar | > ceph-ansible-3.2.0-1.el7cp.noarch | 03:21 |
| *** gkadam-bmgr has joined #tripleo | 03:34 | |
| *** skramaja has joined #tripleo | 03:35 | |
| *** cfontain has quit IRC | 03:35 | |
| *** tzumainn has quit IRC | 03:44 | |
| *** rlandy|bbl is now known as rlandy | 03:49 | |
| *** ramishra has joined #tripleo | 03:50 | |
| jidar | wacky, fails on filestore+collocated | 03:50 |
| kplant | same failure? | 03:51 |
| kplant | i assumed the disks were zapped already | 03:52 |
| kplant | assume* | 03:52 |
| *** rascasoft has joined #tripleo | 03:53 | |
| jidar | I don't think the disks have ever had anything on them | 03:54 |
| jidar | and yea, same error | 03:54 |
| kplant | i would specifically zap them | 03:54 |
| jidar | they don't show up in blkid | 03:54 |
| *** ykarel|away has joined #tripleo | 03:54 | |
| kplant | i hit my nodes with '[{"interface": "deploy", "step": "erase_devices_metadata"}]' regardless | 03:55 |
| kplant | don't want to get 30 minutes in only to have ceph not deploy because it didn't like what was on the disk | 03:55 |
| *** udesale has joined #tripleo | 03:55 | |
| kplant | you also have the root disk specified? | 03:56 |
| kplant | just trying to think of all the annoying "gottchas" | 03:57 |
| jidar | if I'd only had wasted 30 minutes lol | 03:57 |
| jidar | this is day 2 | 03:57 |
| jidar | 13ish hours day 1, 13 as of now day 2 | 03:57 |
| kplant | i feel ya | 03:58 |
| jidar | kplant: yea, I've got root disk specificed in the ironic node info | 03:59 |
| jidar | > properties | {u'cpu_arch': u'x86_64', u'root_device': {u'serial': u'600508b1001c85b78f861c15a9b9a479'}, | 03:59 |
| jidar | for all the ceph nodes | 03:59 |
| kplant | so both collocated+filestore and lvm+bluestore fail | 04:00 |
| *** rascasoft has quit IRC | 04:01 | |
| jidar | same error yea, I feel like it's missing something else | 04:01 |
| kplant | yep | 04:01 |
| jidar | also `"msg": "No first item, sequence was empty."` makes me think it's missing an item in a list | 04:02 |
| jidar | so I started poking around the template for things which call the first item in a list | 04:03 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Make container build job voting https://review.openstack.org/634060 | 04:03 |
| jidar | > radosgw_address_block | 04:03 |
| jidar | expects to have a first item in a list | 04:04 |
| jidar | but I've got none defined in my mistral inventory file | 04:04 |
| jidar | rgws: | 04:04 |
| jidar | hosts: {} | 04:04 |
| kplant | are you using the radosgw? | 04:04 |
| jidar | I am not | 04:05 |
| jidar | https://github.com/ceph/ceph-ansible/blob/v3.2.0/roles/ceph-config/templates/ceph.conf.j2#L182 | 04:07 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 04:10 |
| kplant | are you doing network isolation? | 04:14 |
| jidar | yep | 04:15 |
| jidar | kplant: current deploy command: https://gist.github.com/628094873a75e398bb3d18c56eaca96e | 04:16 |
| kplant | reason i ask is this: https://bugs.launchpad.net/openstack-ansible/+bug/1781321 | 04:17 |
| openstack | Launchpad bug 1781321 in openstack-ansible "ceph-ansible error No first item, sequence was empty" [Undecided,Invalid] | 04:17 |
| kplant | looks to be a similar symptom caused by a network reachability issue | 04:17 |
| *** _fragatina has joined #tripleo | 04:18 | |
| jidar | so he's had two issues, one was var caching | 04:19 |
| jidar | the other was the network stuff | 04:19 |
| *** _fragatina has quit IRC | 04:19 | |
| jidar | let me look into vars stuff though | 04:19 |
| *** _fragatina has joined #tripleo | 04:19 | |
| *** _fragatina has quit IRC | 04:21 | |
| *** _fragatina has joined #tripleo | 04:21 | |
| *** kplant has quit IRC | 04:27 | |
| *** _fragatina has quit IRC | 04:27 | |
| *** _fragatina has joined #tripleo | 04:28 | |
| *** udesale has joined #tripleo | 04:28 | |
| *** _fragatina has quit IRC | 04:31 | |
| *** _fragatina has joined #tripleo | 04:31 | |
| *** _fragatina has quit IRC | 04:32 | |
| *** _fragatina has joined #tripleo | 04:33 | |
| *** ykarel|away is now known as ykarel | 04:34 | |
| *** mdbooth_ has quit IRC | 04:38 | |
| *** mdbooth has joined #tripleo | 04:39 | |
| *** _fragatina has quit IRC | 04:41 | |
| *** _fragatina has joined #tripleo | 04:42 | |
| *** _fragatina has quit IRC | 04:42 | |
| *** _fragatina has joined #tripleo | 04:43 | |
| *** _fragatina has quit IRC | 04:44 | |
| *** _fragatina has joined #tripleo | 04:44 | |
| *** rascasoft has joined #tripleo | 04:47 | |
| *** rascasoft has quit IRC | 04:59 | |
| *** rh-jelabarre has quit IRC | 05:07 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Only request certs in step 1 https://review.openstack.org/633944 | 05:09 |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 05:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| *** _fragatina has quit IRC | 05:11 | |
| *** cfontain has joined #tripleo | 05:11 | |
| *** _fragatina has joined #tripleo | 05:11 | |
| *** _fragatina has quit IRC | 05:16 | |
| *** _fragatina has joined #tripleo | 05:16 | |
| *** _fragatina has quit IRC | 05:19 | |
| *** rascasoft has joined #tripleo | 05:19 | |
| *** _fragatina has joined #tripleo | 05:20 | |
| *** shyamb has joined #tripleo | 05:30 | |
| *** rascasoft has quit IRC | 05:32 | |
| *** _fragatina has quit IRC | 05:38 | |
| *** _fragatina has joined #tripleo | 05:39 | |
| *** wolverineav has joined #tripleo | 05:43 | |
| *** cfontain has quit IRC | 05:43 | |
| *** wolverineav has quit IRC | 05:45 | |
| *** wolverineav has joined #tripleo | 05:47 | |
| *** wolverineav has quit IRC | 05:54 | |
| *** ykarel has quit IRC | 05:59 | |
| *** rascasoft has joined #tripleo | 06:06 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 06:10 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| *** chandankumar is now known as chkumar|ruck | 06:14 | |
| *** rascasoft has quit IRC | 06:15 | |
| *** wolverineav has joined #tripleo | 06:15 | |
| *** ykarel has joined #tripleo | 06:16 | |
| *** wolverineav has quit IRC | 06:19 | |
| *** wolverineav has joined #tripleo | 06:19 | |
| *** quiquell|off is now known as quiquell_rover | 06:20 | |
| *** cfontain has joined #tripleo | 06:20 | |
| *** noama has joined #tripleo | 06:23 | |
| *** pdeore has joined #tripleo | 06:26 | |
| *** shyamb has quit IRC | 06:31 | |
| *** agurenko has joined #tripleo | 06:32 | |
| *** zzzeek has quit IRC | 06:33 | |
| *** zzzeek has joined #tripleo | 06:37 | |
| Tengu | hello there | 06:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Enable dbus-broker for Fedora 29 https://review.openstack.org/634105 | 06:45 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-quickstart-extras master: Add output for ip6tables and related ipv6 things https://review.openstack.org/633922 | 06:49 |
| *** cfontain has quit IRC | 06:53 | |
| *** waleedm has joined #tripleo | 06:54 | |
| *** radeks_ has joined #tripleo | 06:54 | |
| *** karthiks has joined #tripleo | 06:58 | |
| *** mdbooth_ has joined #tripleo | 06:58 | |
| skramaja | karthiks: waleedm is working on nic partitioning support for mellanox. the question is is it possible to add sriov_pf to a bridge | 06:59 |
| skramaja | i understand it is not implemented as of today, but I dont see any limitation in doing soo, whats your views.. | 06:59 |
| karthiks | skramaja, waleedm I haven't tried it | 06:59 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Enable dbus-broker for Fedora 29 https://review.openstack.org/634105 | 06:59 |
| waleedm | so I'll will try to add it the the schema file and check | 07:00 |
| skramaja | karthiks: is there any limitation with respect to reboot? i beleive the sriov-conifg service runs before the network.service, it PF should be configured to switchdev before adding to bridge on reboot. correct? | 07:01 |
| *** mdbooth has quit IRC | 07:01 | |
| *** cfontain has joined #tripleo | 07:01 | |
| waleedm | skramaja: yes it should but also need to make sure | 07:03 |
| karthiks | skramaja, waleedm the issue with reboot is fixed in https://review.openstack.org/#/c/632081/. sriov config service comes before network service | 07:03 |
| karthiks | skramaja, waleedm the dependency due to switchdev, bridge - I am not sure | 07:04 |
| karthiks | waleedm, meanwhile can you please state the usecase of attaching the PF to the bridge. is it for ovs offload ? | 07:05 |
| skramaja | the requirement is that the mode should be changed to switchdev before adding to the bridge (before network.service runs the ifcfg scripts) | 07:05 |
| *** rascasoft has joined #tripleo | 07:06 | |
| *** quiquell_rover is now known as quiquell|rover | 07:07 | |
| karthiks | skramaja, if thats the requirement then sriov_config comes up ahead of ovs-vswitchd during reboot ... the bridge attachments will happen in ovs-vswitchd, so it should be ok I think | 07:07 |
| *** quiquell|rover has quit IRC | 07:07 | |
| skramaja | ok | 07:07 |
| *** quiquell has joined #tripleo | 07:07 | |
| waleedm | skramaja: I don't think there will be any problem with that, the only missing is to add it the schema | 07:08 |
| skramaja | ok waleedm | 07:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Enable dbus-broker for Fedora 29 https://review.openstack.org/634105 | 07:08 |
| *** quiquell is now known as quiquell|rover | 07:09 | |
| *** jfrancoa has joined #tripleo | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1813900 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 07:10 |
| *** ksambor has joined #tripleo | 07:11 | |
| *** ccamacho has joined #tripleo | 07:15 | |
| openstackgerrit | Chandan Kumar proposed openstack/python-tripleoclient master: Use with keyword for opening target file https://review.openstack.org/634109 | 07:15 |
| *** mschuppert has joined #tripleo | 07:17 | |
| *** kopecmartin|off is now known as kopecmartin | 07:30 | |
| openstackgerrit | Raviv proposed openstack/tripleo-upgrade stable/queens: Migrate linting to pre-commit https://review.openstack.org/634113 | 07:36 |
| ykarel | Tengu, possibly alert can be removed now https://bugs.launchpad.net/tripleo/+bug/1813900, jaosorior patch is merged | 07:38 |
| openstack | Launchpad bug 1813900 in tripleo "tripleo-ci-centos-7-undercloud-containers timing out in host configuration for step 2" [Critical,In progress] - Assigned to Cédric Jeanneret (cjeanner) | 07:38 |
| *** jcoufal has quit IRC | 07:39 | |
| *** slaweq has joined #tripleo | 07:39 | |
| Tengu | ykarel: so we might not get the timeout? let's wait a bit. | 07:39 |
| *** holser_ has joined #tripleo | 07:39 | |
| ykarel | Tengu, atleast that job passed in check/gate for puppet-tripleo patch | 07:39 |
| ykarel | ok you can wait for more runs | 07:40 |
| ykarel | http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-undercloud-containers&branch=master | 07:40 |
| Tengu | hmm ok. well. let's drop the alert then. | 07:41 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Move the multipathd services into deployment https://review.openstack.org/633727 | 07:41 |
| Tengu | ykarel: do you have the link to jaosorior patch? | 07:42 |
| Tengu | just want to point it in the LP | 07:42 |
| ykarel | Tengu, https://review.openstack.org/#/c/633944/ | 07:42 |
| Tengu | thx | 07:42 |
| Tengu | have to feed the cat, else I'll be eaten bare. | 07:43 |
| *** slaweq has quit IRC | 07:43 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:47 | |
| *** florianf has joined #tripleo | 07:47 | |
| *** cylopez has joined #tripleo | 07:52 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: flatten the iscsid service https://review.openstack.org/633723 | 07:54 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: UX - Useful error msg if role is not in roles data https://review.openstack.org/631950 | 07:54 |
| * quique|rover|brb is away: brb | 07:55 | |
| *** quique|rover|brb is now known as quiquell|rover | 07:55 | |
| *** wolverineav has quit IRC | 07:55 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:56 | |
| *** quique|rover|brb is now known as quique|rover|bbl | 07:56 | |
| *** jcoufal has joined #tripleo | 07:56 | |
| *** wolverineav has joined #tripleo | 07:57 | |
| *** cfontain has quit IRC | 07:58 | |
| *** shyamb has joined #tripleo | 07:59 | |
| * quique|rover|bbl is back (gone 00:04:03) | 07:59 | |
| * quique|rover|bbl is away: bbl | 07:59 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 08:00 |
| *** yprokule has joined #tripleo | 08:01 | |
| *** wolverineav has quit IRC | 08:02 | |
| *** slaweq has joined #tripleo | 08:03 | |
| *** aufi has joined #tripleo | 08:04 | |
| *** agurenko has quit IRC | 08:06 | |
| *** tkajinam has quit IRC | 08:14 | |
| *** udesale has quit IRC | 08:14 | |
| *** udesale has joined #tripleo | 08:15 | |
| *** quique|rover|bbl is now known as quiquell|rover | 08:16 | |
| * quiquell|rover is back (gone 00:16:43) | 08:16 | |
| *** ramishra has quit IRC | 08:17 | |
| *** apetrich has joined #tripleo | 08:17 | |
| *** ykarel is now known as ykarel|lunch | 08:21 | |
| *** moguimar has joined #tripleo | 08:24 | |
| *** moguimar has quit IRC | 08:25 | |
| *** moguimar has joined #tripleo | 08:26 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Correct *_short_bootstrap_node_name variable. https://review.openstack.org/633754 | 08:26 |
| *** moguimar has quit IRC | 08:27 | |
| *** ramishra has joined #tripleo | 08:28 | |
| *** hberaud|gone is now known as hberaud | 08:29 | |
| *** tosky has joined #tripleo | 08:34 | |
| *** shyam89 has joined #tripleo | 08:37 | |
| lyarwood | ykarel: | 08:37 |
| lyarwood | ops sorry | 08:37 |
| lyarwood | ignore | 08:37 |
| openstackgerrit | Chandan Kumar proposed openstack-infra/tripleo-ci master: Run tempest using os_tempest role in standalone job https://review.openstack.org/627500 | 08:38 |
| *** jpich has joined #tripleo | 08:39 | |
| quiquell|rover | marios: I am rechecking https://review.openstack.org/#/c/633444 | 08:39 |
| marios | quiquell|rover: ack thx | 08:40 |
| quiquell|rover | marios: There is reproducer third party running there so you can check directly the libvirt jobs | 08:40 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: Remove usage of target_upgrade_version. https://review.openstack.org/607525 | 08:40 |
| *** shyamb has quit IRC | 08:40 | |
| marios | wdyt folks moving ci into new section https://review.openstack.org/#/c/633771 http://logs.openstack.org/71/633771/2/check/openstack-tox-docs/266ee8b/html/ thanks. no new content just rearranges sections in the index and adds standalone ci index @ /ci/index.html | 08:41 |
| marios | jaosorior: (if you have time thanks ^) | 08:41 |
| *** shyam89 has quit IRC | 08:42 | |
| *** sagarw has joined #tripleo | 08:43 | |
| quiquell|rover | marios: going to refactor libvirt review a little to check if we have to force | 08:44 |
| quiquell|rover | ykarel|lunch, chkumar|ruck: what's the promoter issue ? | 08:44 |
| *** jpena|off is now known as jpena | 08:46 | |
| *** shyamb has joined #tripleo | 08:46 | |
| *** bogdando has joined #tripleo | 08:46 | |
| chkumar|ruck | quiquell|rover: ykarel|lunch only problem with rocky and queens promoter | 08:47 |
| chkumar|ruck | master and ocata is fine | 08:47 |
| chkumar|ruck | quiquell|rover: ykarel|lunch http://38.145.34.55/rocky.log | 08:47 |
| chkumar|ruck | failed: [localhost] (item=manila-scheduler) => {"changed": false, "item": "manila-scheduler", "msg": "Error pulling trunk.registry.rdoproject.org/tripleorocky/centos-binary-manila-scheduler - code: None message: open /var/lib/docker/tmp/GetImageBlob999019506: no space left on device"} | 08:49 |
| chkumar|ruck | ykarel|lunch: is the disk full where promoter runs or somewhere else? | 08:50 |
| *** lhinds has quit IRC | 08:51 | |
| *** weshay has quit IRC | 08:51 | |
| *** dsneddon has quit IRC | 08:51 | |
| quiquell|rover | chkumar|ruck: if disk is full it has to affect the other ones since they run in the same machine | 08:54 |
| chkumar|ruck | quiquell|rover: do we have access to promoter vm? we can dig and check there? | 08:54 |
| *** dsneddon has joined #tripleo | 08:55 | |
| *** lhinds has joined #tripleo | 08:55 | |
| *** avivgt has joined #tripleo | 08:57 | |
| *** paramite has joined #tripleo | 08:58 | |
| openstackgerrit | Liron Kuchlani proposed openstack/tripleo-heat-templates master: [OSPD14] Add ContainerImagePrepare service to ControllerStorageNfs role https://review.openstack.org/634139 | 08:59 |
| openstackgerrit | Liron Kuchlani proposed openstack/tripleo-heat-templates master: [OSPD14] Add ContainerImagePrepare service to ControllerStorageNfs role https://review.openstack.org/634139 | 09:00 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by default https://review.openstack.org/633444 | 09:03 |
| *** haleyb has quit IRC | 09:03 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by default https://review.openstack.org/633444 | 09:04 |
| Tengu | hey folks, just a quick question: what do you think of this (new) output? http://logs.openstack.org/22/633922/5/check/tripleo-ci-centos-7-standalone/6400230/logs/undercloud/var/log/extra/network.txt.gz there are some "headers" and ipv6 related stuff (cf https://review.openstack.org/#/c/633922/) | 09:05 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 09:05 |
| *** mburned has quit IRC | 09:05 | |
| *** iurygregory has joined #tripleo | 09:05 | |
| quiquell|rover | marios, bogdando, ykarel|lunch: Can do you think now https://review.openstack.org/#/c/633444/ ? | 09:05 |
| *** shyamb has quit IRC | 09:05 | |
| quiquell|rover | fallback if libguestfs-test-tool fails | 09:05 |
| marios | i think nit quiquell|rover | 09:07 |
| marios | quiquell|rover: https://review.openstack.org/#/c/633444/18/roles/libvirt/setup/overcloud/tasks/libvirt_nodepool.yml@36 | 09:08 |
| marios | quiquell|rover: update in browser | 09:08 |
| *** nkinder has joined #tripleo | 09:08 | |
| bogdando | quiquell|rover: I saw something similar from Sagi yesterday, but can't find it | 09:09 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 09:09 |
| quiquell|rover | marios: fixed | 09:09 |
| *** shyamb has joined #tripleo | 09:09 | |
| quiquell|rover | bogdando: is same review | 09:09 |
| *** ykarel|lunch is now known as ykarel | 09:09 | |
| quiquell|rover | bogdando: we are working on that | 09:09 |
| bogdando | ah ok) | 09:09 |
| quiquell|rover | bogdando: are you ok with the fallback ? | 09:11 |
| marios | quiquell|rover: one more extra line | 09:11 |
| *** hakhande_ has joined #tripleo | 09:11 | |
| ykarel | chkumar|ruck, yes it should be disk in the promoter server | 09:12 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 09:12 |
| quiquell|rover | bogdando: done ^ | 09:12 |
| ykarel | chkumar|ruck, and for master it's currently running, you can see past attempts for e4a542b9a3ea6f459605ffbaa3c8af97eb81921f hash has failed | 09:13 |
| quiquell|rover | ykarel, chkumar|ruck:100% /var/lib/docker | 09:13 |
| ykarel | quiquell|rover, possibly multiple promotions running together or no cleanup can cause it | 09:14 |
| quiquell|rover | ykarel: the promotion stuff do the cleanup of docker images ? have to check | 09:14 |
| ykarel | quiquell|rover, any sort of cleanup would be there, otherwise i don't think disk is infinite | 09:17 |
| quiquell|rover | looking | 09:17 |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-quickstart-extras master: Skip test not supported by OVN https://review.openstack.org/634144 | 09:18 |
| shyamb | hi | 09:18 |
| quiquell|rover | yep it has a delete mechanism | 09:18 |
| *** shardy has joined #tripleo | 09:19 | |
| *** dtantsur|afk is now known as dtantsur | 09:21 | |
| quiquell|rover | ykarel, chkumar|ruck: I see rocky images from 34 hours ago | 09:22 |
| quiquell|rover | looks like push playbook didn't do the cleanup | 09:22 |
| quiquell|rover | and 6 days ago... | 09:22 |
| shyamb | Hi | 09:23 |
| shyamb | I want to create a directory on my container under /var/lib/nova/ | 09:23 |
| *** gfidente has joined #tripleo | 09:23 | |
| shyamb | but once I launch this container, this directory is getting mounted by host's /var/lib/nova | 09:24 |
| shyamb | I may need to create that directory on host itself | 09:25 |
| shyamb | so that it will get mounted as well | 09:25 |
| sshnaidm|afk | quiquell|rover, commented on https://review.openstack.org/#/c/633444/ | 09:25 |
| *** sshnaidm|afk is now known as sshnaidm | 09:25 | |
| *** janki has joined #tripleo | 09:29 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common stable/rocky: Publish a better failure message for Update Ansible https://review.openstack.org/634187 | 09:29 |
| quiquell|rover | chkumar|ruck, ykarel: promoter -> https://bugs.launchpad.net/tripleo/+bug/1814059 | 09:31 |
| openstack | Launchpad bug 1814059 in tripleo "Full disk at promoter " [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 09:31 |
| quiquell|rover | chkumar|ruck: etherpad https://review.rdoproject.org/etherpad/p/ruckrover-unisprint5 | 09:31 |
| ykarel | quiquell|rover, ack | 09:31 |
| *** wolverineav has joined #tripleo | 09:31 | |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: Switch default neutron ML2 mechanism driver to OVN https://review.openstack.org/593056 | 09:33 |
| quiquell|rover | sshnaidm: we have reproducer libgirt directly at the review :-) | 09:36 |
| *** avivgt has quit IRC | 09:36 | |
| chkumar|ruck | quiquell|rover: thanks! | 09:36 |
| *** wolverineav has quit IRC | 09:37 | |
| shyamb | Hi jaosorior: Tengu: You there? | 09:38 |
| Tengu | shyamb: hey, yep. | 09:38 |
| shyamb | Hi | 09:39 |
| sshnaidm | quiquell|rover, that's cool :) | 09:39 |
| shyamb | Tengu: I wanted to create a directory under /var/lib/nova/ on compute node | 09:39 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 09:39 |
| *** psachin has quit IRC | 09:40 | |
| shyamb | Looks like all puppet generated directories get created under /var/lib/config_data/... | 09:40 |
| shyamb | Tengu: Is there any way? | 09:40 |
| Tengu | shyamb: yeah, you'll need to hit in tripleo-heat-templates | 09:40 |
| quiquell|rover | sshnaidm: fixed thanks | 09:40 |
| shyamb | Tengu: Ok | 09:40 |
| Tengu | shyamb: also, take care of the selinux labels. you can check the "host_prep_tasks" in docker/services/*.yaml for instance. | 09:40 |
| Tengu | there are many, many examples of such directory creation - even that /var/lib/nova is managed there iirc. | 09:41 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: Move CI docs from contributor guide into new CI guide https://review.openstack.org/633771 | 09:41 |
| marios | ssbarnea: hope it addresses you commment thanks for review ^ | 09:41 |
| shyamb | Tengu: Thanks, I will take a look at it | 09:41 |
| Tengu | shyamb: docker/services/nova-compute.yaml: - { 'path': /var/lib/nova/instances, 'setype': svirt_sandbox_file_t } | 09:41 |
| shyamb | Tengu: ok | 09:42 |
| Tengu | shyamb: you can add your directory in that same template, there's already a loop for some directories. | 09:42 |
| shyamb | same template? | 09:42 |
| shyamb | Tengu: In that case I need to commit it upstream | 09:42 |
| Tengu | in docker/services/nova-compute.yaml - or maybe in the one you're adding with your own container ? | 09:43 |
| shyamb | yeah | 09:43 |
| shyamb | That would be good | 09:43 |
| quiquell|rover | chkumar|ruck: cleaning old docker images | 09:43 |
| Tengu | anyway, you have examples :) | 09:43 |
| quiquell|rover | chkumar|ruck: Later let's check why the don't get clean | 09:43 |
| *** pbandark has joined #tripleo | 09:43 | |
| chkumar|ruck | quiquell|rover: sure | 09:43 |
| chkumar|ruck | quiquell|rover: better add a view to check disk status | 09:44 |
| chkumar|ruck | in ruckrover cockpit | 09:44 |
| shyamb | Tengu: Thanks | 09:44 |
| *** moguimar has joined #tripleo | 09:46 | |
| *** panda|off is now known as panda | 09:46 | |
| *** derekh has joined #tripleo | 09:48 | |
| *** avivgt has joined #tripleo | 09:49 | |
| quiquell|rover | chkumar|ruck: cannot clean because there is no space left :-( | 09:54 |
| chkumar|ruck | quiquell|rover: is it possible to allocate additional storage? | 09:55 |
| Tengu | hey, anyone interested in some better CI logging regarding networking? https://review.openstack.org/#/c/633922/ please make it happen :) ( jaosorior, ykarel, quiquell|rover ;) ) | 09:56 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient stable/queens: Use a single queue for update/upgrade https://review.openstack.org/634190 | 09:58 |
| *** shyamb has quit IRC | 09:58 | |
| *** xek_ has joined #tripleo | 09:59 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient stable/queens: Use a single queue for update/upgrade https://review.openstack.org/634190 | 10:02 |
| quiquell|rover | chkumar|ruck, ykarel: deleting | 10:02 |
| *** pbandark has quit IRC | 10:05 | |
| *** hberaud is now known as hberaud|afk | 10:08 | |
| *** suuuper has joined #tripleo | 10:08 | |
| *** shyamb has joined #tripleo | 10:08 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient stable/rocky: Use a single queue for update/upgrade https://review.openstack.org/634194 | 10:13 |
| quiquell|rover | damn restarting promoter server | 10:14 |
| quiquell|rover | docker is stuck | 10:14 |
| *** hberaud|afk is now known as hberaud | 10:24 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common stable/queens: Publish a better failure message for Update Ansible https://review.openstack.org/634195 | 10:25 |
| *** shyamb has quit IRC | 10:25 | |
| *** shyamb has joined #tripleo | 10:25 | |
| sshnaidm | quiquell|rover, yesterday all servers were migrated, so worth to check if they are alive | 10:26 |
| sshnaidm | quiquell|rover, I mean in tripleo-infra tenant | 10:26 |
| *** apetrich has quit IRC | 10:27 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 10:28 |
| quiquell|rover | sshnaidm: restarting promoter server | 10:29 |
| quiquell|rover | sshnaidm: have to be it | 10:29 |
| quiquell|rover | sshnaidm: they do bad shutdown images does not get cleaned | 10:30 |
| sshnaidm | quiquell|rover, which images? | 10:30 |
| *** jistr is now known as jistr|chat | 10:31 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient stable/rocky: Get the failed status for update prepare https://review.openstack.org/634196 | 10:32 |
| quiquell|rover | sshnaidm: docker containers | 10:32 |
| sshnaidm | quiquell|rover, yeah, docker is better to shutdown before restart, it's not best in recovering | 10:33 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 10:33 |
| sshnaidm | quiquell|rover, ^^ | 10:33 |
| quiquell|rover | sshnaidm: deleting now | 10:34 |
| *** thrash|g0ne is now known as thrash | 10:34 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient stable/queens: Get the failed status for update prepare https://review.openstack.org/634198 | 10:35 |
| *** rnoriega has quit IRC | 10:35 | |
| *** rnoriega has joined #tripleo | 10:36 | |
| *** agurenko has joined #tripleo | 10:37 | |
| *** agurenko has quit IRC | 10:38 | |
| *** shyamb has quit IRC | 10:40 | |
| *** apetrich has joined #tripleo | 10:41 | |
| *** shyamb has joined #tripleo | 10:43 | |
| *** jaganathan has joined #tripleo | 10:54 | |
| *** shyamb has quit IRC | 10:55 | |
| *** florianf has quit IRC | 11:01 | |
| quiquell|rover | ykarel, chkumar|ruck: promoter back again clean | 11:02 |
| chkumar|ruck | quiquell|rover: thanks :-) | 11:03 |
| ykarel | quiquell|rover, Thanks, let's hope master will promote soon | 11:04 |
| *** florianf has joined #tripleo | 11:05 | |
| *** jistr|chat is now known as jistr | 11:05 | |
| *** pbandark has joined #tripleo | 11:05 | |
| *** rfolco has joined #tripleo | 11:05 | |
| quiquell|rover | sshnaidm: to monitor promoter disk at cockpit https://review.rdoproject.org/r/18645 | 11:07 |
| *** shyamb has joined #tripleo | 11:08 | |
| quiquell|rover | sshnaidm: this is just the influxdb part | 11:08 |
| openstackgerrit | Chandan Kumar proposed openstack/python-tripleoclient master: Use with keyword for opening target file https://review.openstack.org/634109 | 11:08 |
| sshnaidm | quiquell|rover, where is disk.telegraf.conf ? | 11:09 |
| quiquell|rover | sshnaidm: now | 11:10 |
| *** hberaud has quit IRC | 11:16 | |
| *** hberaud has joined #tripleo | 11:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: TLS everywhere: Set post-save command for httpd https://review.openstack.org/633170 | 11:18 |
| openstackgerrit | Raviv proposed openstack/tripleo-upgrade stable/queens: Migrate linting to pre-commit https://review.openstack.org/634113 | 11:18 |
| *** wolverineav has joined #tripleo | 11:20 | |
| *** mburned has joined #tripleo | 11:20 | |
| *** udesale has quit IRC | 11:21 | |
| *** wolverineav has quit IRC | 11:24 | |
| sshnaidm | quiquell|rover, in rdo cloud again: No more IP addresses available on network https://logs.rdoproject.org/81/633681/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ovb-master/a3a5ac3/job-output.txt.gz#_2019-01-31_10_43_31_848837 | 11:33 |
| sshnaidm | chkumar|ruck, ^^ | 11:33 |
| *** pbandark has quit IRC | 11:34 | |
| chkumar|ruck | sshnaidm: checking | 11:34 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: TLS everywhere: Set post-save command for RabbitMQ https://review.openstack.org/633217 | 11:35 |
| sshnaidm | quiquell|rover, do we monitor RETRY_LIMIT jobs? | 11:35 |
| *** karthiks has quit IRC | 11:36 | |
| *** boazel has joined #tripleo | 11:39 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Fix introspection data lookup https://review.openstack.org/631277 | 11:40 |
| *** shyamb has quit IRC | 11:40 | |
| *** shyamb has joined #tripleo | 11:41 | |
| *** sdake has quit IRC | 11:41 | |
| chkumar|ruck | sshnaidm: regarding above issue, it is seen else where | 11:41 |
| *** boazel_ has quit IRC | 11:42 | |
| chkumar|ruck | sshnaidm: there are still 40 ips available on that tenant | 11:42 |
| sshnaidm | chkumar|ruck, yeah, I see | 11:42 |
| *** suuuper has quit IRC | 11:43 | |
| chkumar|ruck | sshnaidm: will I try to do vm creation experiment there on that tenant? | 11:44 |
| sshnaidm | chkumar|ruck, go for it | 11:45 |
| quiquell|rover | sshnaidm: you look at the stacks at grafana issue ? | 11:45 |
| *** holser_ is now known as holser|burger | 11:46 | |
| sshnaidm | quiquell|rover, I look at grafana, but I don't see RETRY_LIMIT jobs | 11:46 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Fix introspection data lookup https://review.openstack.org/631277 | 11:46 |
| *** sdake has joined #tripleo | 11:48 | |
| quiquell|rover | sshnaidm: We don't skip RETRY_LIMIT I think | 11:51 |
| *** janki has quit IRC | 11:51 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS everywhere: Set post-save command for nova-vnc-proxy https://review.openstack.org/633246 | 11:51 |
| quiquell|rover | sshnaidm: we skip NODE_FAILURE | 11:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS everywhere: Mount the whole /etc/pki/libvirt/ directory in libvirt https://review.openstack.org/633531 | 11:52 |
| quiquell|rover | ssbarnea: This failure is one of the failures you try to fix with collect logs reviews ? http://logs.rdoproject.org/45/560445/234/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-pike-branch/d250d6f/job-output.txt.gz | 11:56 |
| *** pdeore has quit IRC | 11:57 | |
| ssbarnea | quiquell|rover: i don't think so, i see validating failing here - https://logs.rdoproject.org/45/560445/234/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-pike-branch/d250d6f/job-output.txt.gz#_2019-01-31_09_36_33_542266 --- between few other red-herrings i wanted to fix at some point. | 11:59 |
| *** karthiks has joined #tripleo | 12:00 | |
| quiquell|rover | ssbarnea: ahh thanks collect-logs was a red herring | 12:00 |
| *** dprince has joined #tripleo | 12:01 | |
| *** shyamb has quit IRC | 12:02 | |
| *** shyamb has joined #tripleo | 12:03 | |
| ssbarnea | quiquell|rover: now you know why i wanted to address those issues, i will be back to them when i finish more urgent issues. | 12:03 |
| ssbarnea | quiquell|rover: if it can confuse the ci team member, is more likely to confuse others too | 12:03 |
| quiquell|rover | ssbarnea: yep thanks mate | 12:03 |
| quiquell|rover | Didn't we have a review regarding cirros version ? | 12:03 |
| EmilienM | jaosorior: how is the gate today? | 12:07 |
| EmilienM | did your step patch helped? | 12:07 |
| chkumar|ruck | EmilienM: gates are back to normal undercloud timedout gone | 12:08 |
| jaosorior | EmilienM: it did | 12:12 |
| EmilienM | jaosorior: nice work | 12:13 |
| EmilienM | jaosorior: I mean | 12:13 |
| EmilienM | really nice ! :) | 12:13 |
| *** shyamb has quit IRC | 12:14 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Tweak modify playbook https://review.openstack.org/601426 | 12:14 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart-extras master: Change validate_image_url to 0.3.6 https://review.openstack.org/634213 | 12:17 |
| quiquell|rover | chkumar|ruck: fix for pike OVB https://review.openstack.org/634213 | 12:17 |
| *** hberaud is now known as hberaud|lunch | 12:20 | |
| quiquell|rover | sshnaidm, panda, marios: To fix cirros version stuff https://review.openstack.org/#/c/634213 | 12:20 |
| quiquell|rover | really easy one | 12:20 |
| marios | quiquell|rover: ack | 12:22 |
| quiquell|rover | We only validate overcloud at pike ? | 12:23 |
| *** hakhande_ has quit IRC | 12:23 | |
| chkumar|ruck | quiquell|rover: yes I think | 12:24 |
| chkumar|ruck | quiquell|rover: after that we moved to tesmpest | 12:24 |
| quiquell|rover | chkumar|ruck: ack then the cirros make sense ok | 12:24 |
| *** rh-jelabarre has joined #tripleo | 12:26 | |
| *** jpena is now known as jpena|lunch | 12:27 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add node disks validation https://review.openstack.org/604356 | 12:28 |
| *** udesale has joined #tripleo | 12:28 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add node disks validation https://review.openstack.org/604356 | 12:31 |
| *** shyamb has joined #tripleo | 12:34 | |
| *** ansmith has quit IRC | 12:34 | |
| *** morazi has joined #tripleo | 12:34 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Handle case change for dmidecode >= 3.1 in Ceph templates https://review.openstack.org/634220 | 12:39 |
| *** bfournie has quit IRC | 12:45 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Remove default post-save command from mysql https://review.openstack.org/634221 | 12:46 |
| *** rlandy has joined #tripleo | 12:54 | |
| *** weshay has joined #tripleo | 12:54 | |
| *** sagarw has quit IRC | 12:57 | |
| *** ekultails has joined #tripleo | 13:00 | |
| *** jjoyce has quit IRC | 13:03 | |
| *** mcornea has joined #tripleo | 13:04 | |
| *** jjoyce has joined #tripleo | 13:04 | |
| *** suuuper has joined #tripleo | 13:04 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Remove designate from the master skip list - in standalone-scen-3 https://review.openstack.org/634223 | 13:05 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: Add designate for scenario3 standalone tempest config https://review.openstack.org/633919 | 13:05 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 13:07 |
| *** wolverineav has joined #tripleo | 13:08 | |
| beagles | jaosorior, I've been stumbling over an issue where it appears that the PRIMARYIP isn't being set when configuring hosts_entry on the standalone/undercloud node | 13:08 |
| beagles | jaosorior, so I end up with a line that looks like " myhost.localdomain myhost" instead of "192.168.24.1 myhost.localdomain myhost" | 13:09 |
| jaosorior | oho | 13:09 |
| jaosorior | is that with master? | 13:09 |
| beagles | jaosorior, yes | 13:09 |
| beagles | (I think) | 13:10 |
| beagles | current-tripleo-dev is what I'm using for repos - not using git repos for any of the bits - just trying to get a baseline | 13:11 |
| jaosorior | uhm | 13:11 |
| jaosorior | well | 13:11 |
| weshay | https://review.openstack.org/#/c/630925/ reviews please on tc | 13:11 |
| beagles | current-tripleo as well come to think of it | 13:11 |
| weshay | jaosorior, ^ | 13:11 |
| jaosorior | that stuff comes fromt he service_net_map | 13:11 |
| beagles | jaosorior, yeah | 13:11 |
| jaosorior | beagles: does NetIpMap contain the ctlplane network? | 13:12 |
| *** wolverineav has quit IRC | 13:12 | |
| jaosorior | it does! | 13:13 |
| jaosorior | beagles: any idea if this would return an empty value? | 13:14 |
| jaosorior | {get_attr: [ControlVirtualIP, fixed_ips, 0, ip_address]} | 13:14 |
| jaosorior | in overcloud.j2.yaml | 13:14 |
| jaosorior | hjensas: ^^ | 13:14 |
| *** gkadam-bmgr is now known as gkadam | 13:15 | |
| beagles | jaosorior, I did notice that the StandaloneHostnameResolveNetwork maps to internal_api | 13:15 |
| *** shyamb has quit IRC | 13:16 | |
| hjensas | beagles: I think its a promotion issue. You don't have https://review.openstack.org/627980 but you are using master tripleo-heat-templates? | 13:16 |
| jaosorior | beagles: I noticed that recently too... I always thought that that was meant to be the ctlplane network though | 13:16 |
| jaosorior | that's weird | 13:16 |
| beagles | hjensas, nah, just the ones that installed from the repos | 13:17 |
| *** ssbarnea has quit IRC | 13:17 | |
| *** zbr has joined #tripleo | 13:17 | |
| hjensas | beagles: hm, do you have this https://review.openstack.org/627420 ? | 13:18 |
| hjensas | but not https://review.openstack.org/627980 on the UC? | 13:18 |
| *** zbr is now known as zbr|ssbarnea | 13:18 | |
| beagles | hjensas, checking | 13:18 |
| florianf | dtantsur: I addressed you comments in https://review.openstack.org/#/c/604356/ and https://review.openstack.org/#/c/631277/11 Would be grateful for another quick review. :) | 13:18 |
| hjensas | beagles: I hit the same issue /etc/hosts entry is br0ken right? | 13:18 |
| dtantsur | ok | 13:19 |
| beagles | hjensas, yep | 13:19 |
| *** zbr|ssbarnea has quit IRC | 13:19 | |
| openstackgerrit | Quique Llorente proposed openstack/python-tripleoclient master: Add standalone scenario jobs https://review.openstack.org/634225 | 13:19 |
| *** ykarel has quit IRC | 13:19 | |
| *** holser|burger is now known as holser_ | 13:19 | |
| *** ykarel has joined #tripleo | 13:19 | |
| beagles | hjensas, yeah okay so I don't have https://review.openstack.org/#/c/627980/ | 13:19 |
| *** zbr|ssbarnea has joined #tripleo | 13:20 | |
| beagles | hjensas, but do seem to have https://review.openstack.org/#/c/627420/ | 13:20 |
| *** gkadam has quit IRC | 13:20 | |
| beagles | hjensas, so you think that could cause it? | 13:21 |
| hjensas | beagles: this as well? https://review.openstack.org/628265 | 13:21 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 13:21 |
| *** agopi is now known as agopi|brb | 13:21 | |
| *** zbr|ssbarnea has quit IRC | 13:21 | |
| dtantsur | florianf: +2 to the first, I need to take some time with the second one. | 13:22 |
| beagles | hjensas, I have https://review.openstack.org/628265 | 13:22 |
| hjensas | beagles: I did'nt verify it yet, but I think those THT changes without the tripleoclient == BAD. | 13:22 |
| beagles | hjensas, yeah makes sense - it seems like it is behaving without network info | 13:23 |
| hjensas | beagles: I had depends on there for a reason ... | 13:23 |
| florianf | dtantsur: thanks! | 13:24 |
| beagles | hjensas, so in short, if I update my tripleo client things should improve? | 13:24 |
| hjensas | beagles: yes, I hope so. | 13:24 |
| beagles | hjensas, at least for this issue :b | 13:24 |
| * hjensas thinks it is likely it will improve. | 13:24 | |
| beagles | hjensas, I'll try - no harm in that :) | 13:24 |
| *** jistr is now known as jistr|call | 13:25 | |
| *** zbr|ssbarnea has joined #tripleo | 13:25 | |
| *** haleyb has joined #tripleo | 13:25 | |
| *** agopi|brb has quit IRC | 13:26 | |
| dtantsur | florianf: done. one thing looked suspicious, hence -1 | 13:27 |
| *** hberaud|lunch is now known as hberaud | 13:28 | |
| beagles | hjensas, jaosorior: thanks ! | 13:28 |
| ksambor | Hey all, I have question, there is any easy way to reproduce rdo third party ci jobs locally? | 13:28 |
| zbr|ssbarnea | to avoid some nick overlapping from now on I will be using zbr (or transitional zbr|ssbarnea nicknames) | 13:29 |
| florianf | dtantsur: thanks! looking... | 13:29 |
| *** jistr|call is now known as jistr | 13:31 | |
| *** tzumainn has joined #tripleo | 13:31 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** rh-jelabarre has quit IRC | 13:31 | |
| *** ykarel is now known as ykarel|away | 13:32 | |
| *** shyamb has joined #tripleo | 13:36 | |
| *** jcoufal has quit IRC | 13:37 | |
| *** ykarel|away has quit IRC | 13:38 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add python-tripleoclient files to stdln scenarios https://review.openstack.org/634231 | 13:44 |
| openstackgerrit | Quique Llorente proposed openstack/python-tripleoclient master: Add standalone scenario jobs https://review.openstack.org/634225 | 13:44 |
| *** xek_ has quit IRC | 13:45 | |
| *** xek_ has joined #tripleo | 13:45 | |
| quiquell|rover | chkumar|ruck, weshay: I am going to stop the promoter | 13:46 |
| chkumar|ruck | quiquell|rover: ok | 13:47 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add node disks validation https://review.openstack.org/604356 | 13:47 |
| *** ansmith has joined #tripleo | 13:47 | |
| florianf | dtantsur: Done. | 13:47 |
| florianf | dtantsur: ^ | 13:47 |
| *** pcaruana has quit IRC | 13:52 | |
| jaganathan | d0ugal, shardy please look into https://review.openstack.org/#/c/617158/ | 13:56 |
| weshay | quiquell|rover, ack | 13:57 |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient master: [Core] Add limit option to manage nodes. https://review.openstack.org/634234 | 13:57 |
| d0ugal | jaganathan: will do! | 13:58 |
| jaganathan | d0ugal, thanks! | 13:58 |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient master: [Core] Add limit option to manage nodes. https://review.openstack.org/634234 | 14:00 |
| *** agopi|brb has joined #tripleo | 14:01 | |
| beagles | hjensas, that did the trick btw | 14:02 |
| quiquell|rover | panda: mkfs.xfs -n fstype=1 -f /dev/vdb1 | 14:02 |
| hjensas | beagles: good. Thanks for testing. | 14:02 |
| slagle | #startmeeting TripleO Edge Squad Meeting | 14:02 |
| openstack | Meeting started Thu Jan 31 14:02:29 2019 UTC and is due to finish in 60 minutes. The chair is slagle. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:02 |
| slagle | ping slagle, csatari, jaosorior, owalsh, fultonj, gfidente, hjensas, jtomasek, bogdando, dtantsur, rbrady, d0ugal, toure, abishop | 14:02 |
| *** openstack changes topic to " (Meeting topic: TripleO Edge Squad Meeting)" | 14:02 | |
| slagle | #info remove or update your nick from the Meeting Template on the etherpad if you want (or don't want) to be ping'd for the start of the meeting | 14:02 |
| *** timeu has joined #tripleo | 14:02 | |
| *** agopi|brb is now known as agopi | 14:02 | |
| openstack | The meeting name has been set to 'tripleo_edge_squad_meeting' | 14:02 |
| d0ugal | o/ | 14:02 |
| jaosorior | o/ | 14:02 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-common master: Make the image pulling optional for tripleo-container-tag role. https://review.openstack.org/634239 | 14:02 |
| slagle | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:02 |
| abishop | o/ | 14:02 |
| slagle | Anyone can use the #link, #action, #help, #idea, #agreed, and #info commands, not just the moderatorǃ | 14:02 |
| slagle | hey! | 14:02 |
| hjensas | beagles: makes wonder how on earth things promoted to that state in the repos ... | 14:02 |
| *** pcaruana has joined #tripleo | 14:02 | |
| hjensas | o/ | 14:02 |
| owalsh | o/ (might be afk for a few mins) | 14:03 |
| slagle | #topic Agenda | 14:03 |
| *** openstack changes topic to "Agenda (Meeting topic: TripleO Edge Squad Meeting)" | 14:03 | |
| slagle | * Review past action items | 14:03 |
| slagle | * Goals/action items for the week | 14:03 |
| slagle | #topic Review past action items | 14:03 |
| *** openstack changes topic to "Review past action items (Meeting topic: TripleO Edge Squad Meeting)" | 14:03 | |
| bogdando | o/ | 14:03 |
| slagle | our AI's are from 2 weeks ago as we didn't meet last week | 14:04 |
| slagle | ACTION: slagle/fultonj try multi-ceph deployments with DCN | 14:04 |
| fultonj | i didn't get to | 14:05 |
| slagle | oh good me neither :) | 14:05 |
| fultonj | slagle: did you? | 14:05 |
| fultonj | punt to next week? | 14:05 |
| slagle | sure | 14:05 |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-common master: [Core] Be pep8 compliant. https://review.openstack.org/634241 | 14:05 |
| fultonj | i've only done it with standalone | 14:05 |
| slagle | #action slagle/fultonj try multi-ceph deployments with DCN | 14:05 |
| slagle | ACTION: review https://etherpad.openstack.org/p/tripleo-edge-glance-deployment | 14:05 |
| *** moguimar has quit IRC | 14:06 | |
| slagle | abishop: i took a look. seems bogdando and I had a similar question about glance-api at the edge using the central DB | 14:06 |
| slagle | does it require a direct DB connection from the edge to the central site? | 14:06 |
| abishop | quick answer (and I'll update etherpad) is yes, it does | 14:07 |
| abishop | all part of the common control plane | 14:07 |
| slagle | what functionality is affected if the connection is lost? | 14:08 |
| *** sdake has quit IRC | 14:08 | |
| abishop | well, I think the whole central control plane model presumes control plane is key to everything | 14:09 |
| bogdando | I think the anwer is the same as for DCN in general | 14:09 |
| bogdando | everything but running workloads | 14:09 |
| abishop | if connectino is lost, no new instances, no new volumes, no nothing | 14:09 |
| abishop | bogdando: ack! | 14:09 |
| slagle | ok, makes sense. just wanted to double check | 14:09 |
| fultonj | which is an accepted limitation of DCN from what i have heard | 14:10 |
| slagle | right | 14:10 |
| owalsh | mschuppert added libvirt-guest support too, so we could restart the VMs on reboot | 14:11 |
| owalsh | node reboot | 14:11 |
| slagle | abishop: do you plan to propose the work from your repo as patches, etc? | 14:11 |
| fultonj | owalsh: oh nice, on ephemeral volumes? | 14:11 |
| slagle | abishop: just wondering what the next steps are | 14:11 |
| owalsh | fultonj: yea, I expect there are caveats... volumes, networking etc... | 14:11 |
| fultonj | owalsh: cinder backed VMs too? | 14:11 |
| abishop | slagle: right now they're mainly a set of THT parameter values, but no new parameters | 14:12 |
| abishop | so not sure what I'd be patching | 14:12 |
| slagle | abishop: mostly just thinking environment files, roles, and docs | 14:12 |
| owalsh | fultonj: not quite sure TBH | 14:13 |
| abishop | slagle: I can ponder, and will start with fresh review of what's out there (in tht tree) and suggest a place to make some updates | 14:13 |
| abishop | I'm still working on getting a glance expert involved (jokke) but his time is extremely limited | 14:14 |
| slagle | ack | 14:14 |
| slagle | ok, and the last action item... | 14:14 |
| slagle | ACTION: continue investigating glance and nova image caching status/options | 14:14 |
| slagle | guess we kind of already talked about it | 14:15 |
| slagle | unless there are other proposals not caputred in the etherpad/ | 14:15 |
| slagle | ? | 14:15 |
| *** ykarel|away has joined #tripleo | 14:15 | |
| *** ykarel|away is now known as ykarel | 14:15 | |
| *** chkumar|ruck is now known as chkumar|out | 14:15 | |
| mschuppert | owalsh: the instances get started again via nova, so yes neutron needs to be available, also cinder if volumes are used. | 14:17 |
| *** shyamb has quit IRC | 14:17 | |
| owalsh | mschuppert: so we'd need the control plane | 14:17 |
| slagle | #topic Goals/action items for the week | 14:17 |
| *** openstack changes topic to "Goals/action items for the week (Meeting topic: TripleO Edge Squad Meeting)" | 14:17 | |
| mschuppert | owalsh: yes | 14:17 |
| owalsh | dang | 14:17 |
| fultonj | there was an item regarding the split control plane ci | 14:17 |
| fultonj | update: the job is extracting control plane data and putting it where it needs to be on the separate compute node | 14:18 |
| fultonj | progress ^ | 14:18 |
| slagle | fultonj: i had attempted to keep it moving forward when you were in brno | 14:18 |
| fultonj | slagle: yes, thank you | 14:18 |
| slagle | i haven't revisited yet though :/ | 14:18 |
| fultonj | it helped | 14:18 |
| slagle | i'll take another look. is there a current issue? | 14:18 |
| fultonj | the current status is... | 14:18 |
| fultonj | #action fultonj (et al) get split control plane ci job to not crash on podman container restart | 14:18 |
| fultonj | #link http://logs.openstack.org/88/615988/14/check/tripleo-ci-centos-7-split-controlplane-standalone/cd1a87f/logs/subnode-2/home/zuul/standalone_deploy.log.txt.gz | 14:19 |
| fultonj | #link https://review.openstack.org/#/c/615988/ | 14:19 |
| slagle | oh ok. yea i had to switch the job to podman | 14:19 |
| fultonj | yeah, so controller goes up with podman fine; compute fails on ^ | 14:19 |
| fultonj | error restarting $container0, $container1, ... on step 3 | 14:19 |
| bogdando | so, https://review.openstack.org/#/c/632089/ needs more eyes | 14:22 |
| slagle | fultonj: strange. wonder if it's a podman issue or something else | 14:22 |
| bogdando | attempts describe negative scenarios and expectations for failure modes supported | 14:22 |
| fultonj | slagle: right. i was going to try to reproduce in my env. last time i did edge in my env was w/ docker | 14:23 |
| slagle | #action review https://review.openstack.org/#/c/632089/ | 14:23 |
| slagle | bogdando: will review it | 14:23 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Do not pull image while tagging pcmk images in upgrade_tasks. https://review.openstack.org/634243 | 14:23 |
| slagle | anything else that folks want to highlight this week? | 14:24 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Use force_tcg by libguestfs is not ok https://review.openstack.org/633444 | 14:25 |
| fultonj | mschuppert: when you said 'yes' to owalsh... | 14:25 |
| fultonj | did you mean reboots of instances on edge nodes | 14:26 |
| fultonj | which were backed by cinder and ephemeral volumes? | 14:26 |
| *** numans has quit IRC | 14:26 | |
| fultonj | or is that still not known? | 14:26 |
| *** vinaykns has joined #tripleo | 14:26 | |
| *** skramaja has quit IRC | 14:26 | |
| mschuppert | fultonj: yes, compute needs neutron to be up. doesn't matter if ephemeral or cinder | 14:27 |
| fultonj | mschuppert: thanks. so is the problem then that neutron may not be up? | 14:27 |
| mschuppert | fultonj: yes | 14:28 |
| bogdando | no rebooting, no live migration will be possible indeed | 14:28 |
| fultonj | is that someone we need to track or simply state as a known limitation? | 14:28 |
| fultonj | running workloads keep running. if they stopped running, then you need the ctl plane back | 14:28 |
| fultonj | even with that nova chnage, no reboot | 14:29 |
| bogdando | that should be a known limitation IMO, we can't do that w/o control plane | 14:29 |
| fultonj | works for me | 14:29 |
| fultonj | thanks mschuppert | 14:29 |
| mschuppert | :) | 14:29 |
| slagle | thanks folks! | 14:31 |
| slagle | #endmeeting | 14:31 |
| *** openstack changes topic to "CI Status: GREENish | https://docs.openstack.org/tripleo-docs/latest/" | 14:31 | |
| openstack | Meeting ended Thu Jan 31 14:31:46 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:31 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-31-14.02.html | 14:31 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-31-14.02.txt | 14:31 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2019/tripleo_edge_squad_meeting.2019-01-31-14.02.log.html | 14:31 |
| fultonj | thanks | 14:32 |
| *** zul has joined #tripleo | 14:32 | |
| *** mjturek has joined #tripleo | 14:33 | |
| *** dtrainor has quit IRC | 14:34 | |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient master: [Core] Add limit option to manage nodes. https://review.openstack.org/634234 | 14:34 |
| *** dtrainor has joined #tripleo | 14:38 | |
| *** waleedm has quit IRC | 14:38 | |
| *** sdake has joined #tripleo | 14:38 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Remove postsave_cmd and restart libvirt on cert files change https://review.openstack.org/633746 | 14:44 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add output for ip6tables and related ipv6 things https://review.openstack.org/633922 | 14:44 |
| *** dmacpher has quit IRC | 14:45 | |
| *** moguimar has joined #tripleo | 14:51 | |
| *** jcoufal has joined #tripleo | 14:53 | |
| *** rh-jelabarre has joined #tripleo | 14:54 | |
| *** wolverineav has joined #tripleo | 14:56 | |
| *** avivgt has quit IRC | 14:57 | |
| *** wolverineav has quit IRC | 15:00 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-docs master: Add notes for DCN operational availability modes https://review.openstack.org/632089 | 15:05 |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient master: [Core] Add limit option to manage nodes. https://review.openstack.org/634234 | 15:07 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Fix network IpListMap - no longer need fallback https://review.openstack.org/633975 | 15:07 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Do not pull image while tagging pcmk images in upgrade_tasks. https://review.openstack.org/634243 | 15:11 |
| *** saneax has quit IRC | 15:12 | |
| mschuppert | hi, could I please get reviews on https://review.openstack.org/603343 | 15:12 |
| hjensas | trozet_: Was https://bugs.launchpad.net/tripleo/+bug/1813972 really the undercloud failing? | 15:13 |
| openstack | Launchpad bug 1813972 in tripleo "Undercloud install fails in write_static_inventory with Exception list index out of range " [Undecided,Incomplete] | 15:13 |
| trozet_ | hjensas: yeah, what do you mean? | 15:14 |
| quiquell|rover | jaosorior: I am loking at a tempest issue http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-rocky/6e4d1e6/logs/undercloud/home/zuul/tempest.log.txt.gz | 15:14 |
| quiquell|rover | jaosorior: Do you know if this is related to https://review.openstack.org/#/c/630925/ ? | 15:14 |
| trozet_ | hjensas: i just reran it in CI, let me get the logs | 15:14 |
| hjensas | trozet_: ok, I thought it was the overcloud deploy that failed. Probably because the initial issue you had was overcloud deployment. | 15:16 |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Initial pcs 0.10 support https://review.openstack.org/607873 | 15:20 |
| trozet_ | hjensas: just added a few log files for you | 15:20 |
| *** kplant has joined #tripleo | 15:21 | |
| *** avivgt has joined #tripleo | 15:21 | |
| *** dmacpher has joined #tripleo | 15:22 | |
| chkumar|out | EmilienM: jaosorior mwhahaha https://review.openstack.org/#/c/634109/ please have alook at this one | 15:23 |
| *** sdake has quit IRC | 15:24 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/python-tripleoclient master: Add overcloud admin ssh authorize command. https://review.openstack.org/633574 | 15:24 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 15:29 |
| *** mjturek has quit IRC | 15:33 | |
| hjensas | trozet_: hmm, can you get /etc/hosts of undercloud logs? | 15:36 |
| *** chkumar|out is now known as invincible | 15:37 | |
| *** trown|outtypewww is now known as trown | 15:38 | |
| trozet_ | hjensas: sure. Is it because you think that it is related to that other bug that had to do with hostnames? | 15:38 |
| trozet_ | hjensas: https://bugs.launchpad.net/tripleo/+bug/1813972/comments/5 | 15:39 |
| openstack | Launchpad bug 1813972 in tripleo "Undercloud install fails in write_static_inventory with Exception list index out of range " [Undecided,Incomplete] | 15:39 |
| hjensas | trozet_: There is an issue with the repos as last promoted. We have a mismatch in tripleoclient and THT. The THT package includes changes that depend on tripleoclient change. But somehow the triploclient package does not have the latest stuff. | 15:40 |
| *** quiquell|rover is now known as quique|rover|off | 15:40 | |
| hjensas | trozet_: but, it does'nt look like it's that in your case. | 15:40 |
| *** udesale has quit IRC | 15:40 | |
| trozet_ | hjensas: oh, well if you need i can get package versions | 15:40 |
| *** ccamacho has quit IRC | 15:41 | |
| *** mjturek has joined #tripleo | 15:41 | |
| *** ccamacho has joined #tripleo | 15:50 | |
| hjensas | trozet_: strange, looks like you have https://review.openstack.org/627420 , but you don't have https://review.openstack.org/628259 + https://review.openstack.org/628265 + https://review.openstack.org/627980 | 15:54 |
| *** aufi has quit IRC | 15:54 | |
| hjensas | trozet_: It's possible https://review.openstack.org/633975 causes the index out of range issue in this state. | 15:55 |
| trozet_ | hjensas: well, i can manually add all those patches in | 15:56 |
| trozet_ | let me look at them | 15:56 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 16:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 16:00 |
| trozet_ | hjensas: i just added all those patches (hopefully in the right order) https://gerrit.opnfv.org/gerrit/#/c/66789/2/config/deploy/common-patches.yaml | 16:01 |
| *** ksambor has quit IRC | 16:02 | |
| *** _fragatina has quit IRC | 16:03 | |
| *** saneax has joined #tripleo | 16:04 | |
| *** _fragatina has joined #tripleo | 16:04 | |
| *** _fragatina has quit IRC | 16:04 | |
| *** _fragatina has joined #tripleo | 16:05 | |
| *** yprokule has quit IRC | 16:06 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Run tempest using os_tempest role in standalone job https://review.openstack.org/627500 | 16:07 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add stop_grace_period for heat_engine container https://review.openstack.org/633864 | 16:07 |
| openstackgerrit | Merged openstack/tripleo-common master: Bump the Ceph container image to 3.2.1 https://review.openstack.org/630925 | 16:07 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Bump the Ceph container image to 3.2.1 https://review.openstack.org/630927 | 16:07 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Change NodeDataLookup type from string to json https://review.openstack.org/588640 | 16:08 |
| fultonj | gfidente: ^ | 16:08 |
| gfidente | fultonj I just realized there is an issue in that per_node stuff with py3 | 16:09 |
| gfidente | but it's unrelated | 16:09 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Remove default post-save command from mysql https://review.openstack.org/634221 | 16:09 |
| *** wolverineav has joined #tripleo | 16:09 | |
| gfidente | fultonj | 16:10 |
| gfidente | '{} | 16:10 |
| gfidente | in ceph-base | 16:10 |
| *** _fragatina has quit IRC | 16:10 | |
| fultonj | gfidente: ack | 16:10 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 16:11 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Change NodeDataLookup type from string to json https://review.openstack.org/588640 | 16:11 |
| fultonj | gfidente: ps4 ready for your review | 16:12 |
| gfidente | fultonj rebased mine too | 16:12 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Handle case change for dmidecode >= 3.1 in Ceph templates https://review.openstack.org/634220 | 16:12 |
| *** wolverineav has quit IRC | 16:12 | |
| *** wolverineav has joined #tripleo | 16:13 | |
| gfidente | panda some love https://review.openstack.org/#/c/633039/ ? | 16:14 |
| gfidente | panda and https://review.openstack.org/#/c/633041/ | 16:14 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Enable testing using ansible molecule https://review.openstack.org/613672 | 16:16 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-common stable/rocky: Bump the Ceph container image to 3.2.1 https://review.openstack.org/634260 | 16:18 |
| *** artom has quit IRC | 16:18 | |
| trozet_ | hjensas: i dont think im missing anything, it failed again the same way | 16:19 |
| fultonj | slagle: do you mind looking at gfidente's reply to your comment in https://review.openstack.org/#/c/588640 ? | 16:20 |
| gfidente | fultonj slagle I'll submit a fake submission to make sure yaml passes | 16:20 |
| fultonj | this submission has votes but i didn't want you to think we were sneaking it in | 16:21 |
| *** sdake has joined #tripleo | 16:21 | |
| fultonj | gfidente: thanks | 16:21 |
| slagle | fultonj: yea i had seen it, but there wasn't anything to reply to :) | 16:22 |
| fultonj | slagle: so if gfidente's test passes you're ok with it? | 16:22 |
| slagle | if we're sure it works, i'm not -1 | 16:22 |
| slagle | yep | 16:22 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-common stable/queens: Bump the Ceph container image to 3.2.1 https://review.openstack.org/634261 | 16:22 |
| fultonj | slagle: ok, thanks | 16:22 |
| gfidente | slagle not backport to queens | 16:22 |
| gfidente | slagle rocky / stein only | 16:22 |
| *** quique|rover|off is now known as quiquell|rover | 16:24 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: DO-NOT-MERGE: maybe yes, but now please test yaml NodeDataLookup anyway https://review.openstack.org/634262 | 16:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add PacemakerNetwork definition https://review.openstack.org/633575 | 16:30 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-quickstart master: Added NTP default variable to enable service synchronization on the overcloud https://review.openstack.org/634267 | 16:30 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Initial support for properties managed by pcs 0.10 https://review.openstack.org/626005 | 16:32 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Create tripleo-admin user on the undercloud https://review.openstack.org/633850 | 16:33 |
| *** avivgt has quit IRC | 16:33 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-common master: Make the image pulling optional for tripleo-container-tag role. https://review.openstack.org/634239 | 16:44 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Correct use of force_tcg with libguestfs https://review.openstack.org/634272 | 16:45 |
| *** bogdando has quit IRC | 16:50 | |
| *** avivgt has joined #tripleo | 16:51 | |
| *** vinaykns has quit IRC | 16:54 | |
| *** spsurya has quit IRC | 16:54 | |
| *** vinaykns has joined #tripleo | 16:55 | |
| *** ykarel is now known as ykarel|away | 16:57 | |
| *** ramishra has quit IRC | 16:58 | |
| *** mjturek has quit IRC | 17:00 | |
| *** mjturek has joined #tripleo | 17:00 | |
| *** nkinder has quit IRC | 17:01 | |
| *** dciabrin has quit IRC | 17:02 | |
| *** dciabrin has joined #tripleo | 17:02 | |
| *** kopecmartin is now known as kopecmartin|off | 17:04 | |
| *** iurygregory has quit IRC | 17:06 | |
| *** _fragatina has joined #tripleo | 17:08 | |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: flatten ceilometer service configuration https://review.openstack.org/633617 | 17:11 |
| *** cylopez has quit IRC | 17:13 | |
| *** ccamacho has quit IRC | 17:13 | |
| *** mjturek has quit IRC | 17:14 | |
| *** mjturek has joined #tripleo | 17:15 | |
| *** wolverineav has quit IRC | 17:22 | |
| *** wolverineav has joined #tripleo | 17:23 | |
| *** Vorrtex has joined #tripleo | 17:26 | |
| mschuppert | mwhahaha: you reviewed https://review.openstack.org/603343 before and +2 it. I had to rebase and add live_migration_with_native_tls nova setting. could you please review it again? | 17:27 |
| *** Vorrtex has quit IRC | 17:27 | |
| *** mschuppert is now known as mschuppert|afk | 17:27 | |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: step1: flatten nova service configuration https://review.openstack.org/632522 | 17:28 |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: step2: flatten nova service configuration https://review.openstack.org/633278 | 17:28 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: flatten haproxy service configuration https://review.openstack.org/626631 | 17:28 |
| *** sdake has quit IRC | 17:28 | |
| *** dsneddon has quit IRC | 17:29 | |
| *** avivgt has quit IRC | 17:29 | |
| *** moguimar has quit IRC | 17:30 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: WIP: Use Octavia's project for lb security group ops https://review.openstack.org/634286 | 17:35 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: WIP: Use Octavia's project for lb security group ops https://review.openstack.org/634286 | 17:36 |
| *** artom has joined #tripleo | 17:36 | |
| *** jpich has quit IRC | 17:38 | |
| *** wolverineav has quit IRC | 17:40 | |
| *** wolverineav has joined #tripleo | 17:42 | |
| *** suuuper has quit IRC | 17:43 | |
| *** noama has quit IRC | 17:44 | |
| *** dsneddon has joined #tripleo | 17:45 | |
| openstackgerrit | Merged openstack/tripleo-validations master: Fix introspection data lookup https://review.openstack.org/631277 | 17:46 |
| openstackgerrit | Merged openstack/ansible-role-tripleo-modify-image master: yum_update_buildah: add retry x3 https://review.openstack.org/633805 | 17:46 |
| *** hberaud is now known as hberaud|gone | 17:46 | |
| *** wolverineav has quit IRC | 17:50 | |
| *** ccamacho has joined #tripleo | 17:53 | |
| *** ykarel|away has quit IRC | 17:59 | |
| *** derekh has quit IRC | 18:00 | |
| *** ykarel|away has joined #tripleo | 18:00 | |
| openstackgerrit | Merged openstack/ansible-role-container-registry master: docker: add reset_connection https://review.openstack.org/634041 | 18:02 |
| *** saneax has quit IRC | 18:05 | |
| *** sdake has joined #tripleo | 18:06 | |
| *** wolverineav has joined #tripleo | 18:07 | |
| *** jpena is now known as jpena|off | 18:08 | |
| *** mdbooth has joined #tripleo | 18:12 | |
| *** ykarel|away has quit IRC | 18:13 | |
| *** mdbooth_ has quit IRC | 18:13 | |
| *** wolverineav has quit IRC | 18:15 | |
| *** wolverineav has joined #tripleo | 18:15 | |
| *** mdbooth has quit IRC | 18:17 | |
| *** holser_ has quit IRC | 18:17 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: run docker_puppet_tasks on any role https://review.openstack.org/625078 | 18:17 |
| openstackgerrit | Merged openstack/python-tripleoclient master: tripleo_deploy: symlink ansible-playbook https://review.openstack.org/632756 | 18:22 |
| *** bfournie has joined #tripleo | 18:26 | |
| *** trown is now known as trown|lunch | 18:29 | |
| *** mjturek has quit IRC | 18:36 | |
| *** mjturek has joined #tripleo | 18:38 | |
| *** dtantsur is now known as dtantsur|afk | 18:39 | |
| *** quiquell|rover is now known as quique|rover|off | 18:41 | |
| *** panda is now known as panda|off | 18:41 | |
| *** wolverineav has quit IRC | 18:41 | |
| openstackgerrit | Merged openstack/ansible-role-container-registry master: Fix become for tasks and handlers https://review.openstack.org/633812 | 18:42 |
| *** mjturek has quit IRC | 18:44 | |
| jidar | any of you ceph folks can help me out with this failure I'm seeing in overcloud-AllNodesDeploySteps-52tcg7nul5of.WorkflowTasks_Step2_Execution => https://gist.github.com/frizop/7d5b18ba4d9d3fedd9b33abfe9cdacee | 18:45 |
| *** wolverineav has joined #tripleo | 18:49 | |
| *** dciabrin has quit IRC | 18:51 | |
| *** dciabrin has joined #tripleo | 18:52 | |
| openstackgerrit | David Peacock proposed openstack/tripleo-heat-templates master: flatten database service MySQL Server https://review.openstack.org/633980 | 18:52 |
| openstackgerrit | Merged openstack/puppet-tripleo master: noop package installations inside containers https://review.openstack.org/632564 | 18:52 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use with keyword for opening target file https://review.openstack.org/634109 | 18:52 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Fail if too many nameservers are provided https://review.openstack.org/632560 | 18:52 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Change validate_image_url to 0.3.6 https://review.openstack.org/634213 | 18:52 |
| *** mjturek has joined #tripleo | 18:56 | |
| *** wolverineav has quit IRC | 18:58 | |
| EmilienM | jidar: can you share your environment files? maybe gfidente / fultonj can help too | 18:58 |
| jidar | https://gist.github.com/frizop/628094873a75e398bb3d18c56eaca96e https://gist.github.com/frizop/fe5f76230858f2a5590263da86f8c949 but this was was edited a bit to remove the commented items | 18:59 |
| *** aufi has joined #tripleo | 18:59 | |
| jidar | ceph-config.yaml : https://gist.github.com/de9291959e43e939937f8e737e84c64e that's more current | 19:01 |
| jidar | I also tried doing filestore and collectd but received the same error | 19:02 |
| jidar | (after diving up some dedicated devices) | 19:02 |
| fultonj | jidar: what version of X do you have for ceph-ansible 3.2.X ? | 19:04 |
| *** aufi has quit IRC | 19:04 | |
| jidar | 3.2 -1 iirc, one sec | 19:04 |
| jidar | 21:21:46* jidar | > ceph-ansible-3.2.0-1.el7cp.noarch | 19:05 |
| fultonj | 0 | 19:05 |
| fultonj | https://github.com/ceph/ceph-ansible/blob/v3.2.0/roles/ceph-config/tasks/main.yml#L173 | 19:06 |
| jidar | fultonj: yep, that's in this paste here: | 19:06 |
| jidar | overcloud-AllNodesDeploySteps-52tcg7nul5of.WorkflowTasks_Step2_Execution => https://gist.github.com/frizop/7d5b18ba4d9d3fedd9b33abfe9cdacee | 19:06 |
| jidar | so it's a template, and I assume it's tryign to call "first" on an array that's empty from the error | 19:07 |
| fultonj | exactly | 19:08 |
| fultonj | very likely that's it | 19:08 |
| fultonj | https://github.com/ceph/ceph-ansible/blob/v3.2.0/roles/ceph-config/templates/ceph.conf.j2 | 19:08 |
| jidar | is there any way I can get debug logs out of the ansible-ceph deployment? going from 3->5 didn't help via CephAnsiblePlaybookVerbosity: 5 | 19:09 |
| fultonj | |first occurss 8 times in the template | 19:09 |
| *** mjturek has quit IRC | 19:09 | |
| fultonj | CephAnsiblePlaybookVerbosity: 5 should produce a playbook run with -vvvvv | 19:10 |
| *** mjturek has joined #tripleo | 19:11 | |
| fultonj | jidar: every time |first is used however it's for the same purpose | 19:11 |
| jidar | rwgateway stuff | 19:11 |
| fultonj | more so getting the IP address | 19:11 |
| fultonj | hostvars[host]['ansible_all_' + ip_version + '_addresses'] | ipaddr(hostvars[host]['monitor_address_block'] | 19:11 |
| fultonj | jidar: get the those facts from your system and see where it fails to parse | 19:12 |
| fultonj | might be a ceph-ansible bug | 19:12 |
| jidar | > mons: hosts: 192.168.24.14: {} 192.168.24.15: {} 192.168.24.8: {} | 19:12 |
| fultonj | that's an inventory | 19:12 |
| jidar | in the facts file? | 19:12 |
| fultonj | on 192.168.24.8 | 19:13 |
| fultonj | https://docs.ansible.com/ansible/latest/modules/setup_module.html | 19:13 |
| jidar | monitor_address_block this is what I'm looking for no? | 19:13 |
| fultonj | hostvars[host]['ansible_all_' + ip_version + '_addresses'] | ipaddr(hostvars[host]['monitor_address_block']) | first | 19:14 |
| fultonj | this should return a list: hostvars[host]['ansible_all_' + ip_version + '_addresses'] | ipaddr(hostvars[host]['monitor_address_block']) | 19:14 |
| fultonj | it's not returning a list | 19:14 |
| *** morazi has quit IRC | 19:14 | |
| fultonj | thus |first fails | 19:14 |
| fultonj | so make it return a list | 19:14 |
| fultonj | or find out why it's not returning a list | 19:14 |
| jidar | "ansible_all_ipv4_addresses": ["192.168.24.8", "172.31.0.1"], | 19:14 |
| jidar | ipv6 has like 4 of them | 19:15 |
| fultonj | monitor_address_block: 172.16.1.0/24 | 19:15 |
| fultonj | is in your inventory | 19:15 |
| *** gfidente is now known as gfidente|afk | 19:16 | |
| jidar | I've never defined either of those ips | 19:16 |
| jidar | manually anyhow | 19:16 |
| jidar | 172.16 nor 172.31 | 19:17 |
| *** cylopez has joined #tripleo | 19:17 | |
| fultonj | tripleo network isolation sets up the Storage and StorageMgmt network | 19:18 |
| fultonj | if you use network isolation | 19:18 |
| fultonj | tripleo's ceph integration passes that to ceph-ansible | 19:18 |
| jidar | I do have network isolation included | 19:18 |
| fultonj | usually you specify a range and the it finds the IP per host | 19:18 |
| jidar | > default: [{'start': '172.16.1.4', 'end': '172.16.1.250'}] | 19:20 |
| fultonj | yeah | 19:20 |
| jidar | I'm just going with all defaults for that stuff | 19:20 |
| fultonj | yeah | 19:20 |
| fultonj | that's fine | 19:20 |
| fultonj | jidar: are you deploying rgw and mds etc? | 19:20 |
| fultonj | or just mon, osd, mgr to start? | 19:20 |
| jidar | I've using the ceph-all style | 19:20 |
| jidar | which let me double check what it has defiend | 19:21 |
| *** gfidente|afk has quit IRC | 19:21 | |
| jidar | > Standalone Storage Full Role (OSD + MON + RGW + MDS + MGR + RBD Mirroring) | 19:21 |
| *** wolverineav has joined #tripleo | 19:21 | |
| fultonj | actually i should have known based on your inventory :) | 19:21 |
| fultonj | and you're just doing osd mon mgr | 19:22 |
| fultonj | which is good to start | 19:22 |
| fultonj | get that working before you add in extra complexity | 19:22 |
| *** vinaykns has quit IRC | 19:22 | |
| fultonj | the roll alone doesn't do it | 19:22 |
| fultonj | "Standalone Storage Full Role" controls what gets put where | 19:22 |
| jidar | ahh, ok roles_data ceph-all makes it look that way | 19:22 |
| fultonj | if you want RGW + MDS you include their env files | 19:23 |
| fultonj | let's not add that yet though | 19:23 |
| *** vkapalav has joined #tripleo | 19:23 | |
| fultonj | roles control waht gets put where, but using a roll doesn't mean you get all the services included | 19:23 |
| fultonj | jidar: so that when you add the env files from https://github.com/openstack/tripleo-heat-templates/tree/master/environments/ceph-ansible | 19:24 |
| fultonj | to '-e ceph-rgw.yaml' then you'll get it | 19:24 |
| fultonj | same for -e ceph-mds.yaml | 19:24 |
| fultonj | but like i said let's just get MON/OSD/MGR working | 19:24 |
| jidar | groovy, thanks for the heads up | 19:24 |
| fultonj | anyway it let's you role out a few lines i the tempate | 19:24 |
| fultonj | rule* out | 19:25 |
| jidar | ah yea, mgr-mon-osd only | 19:26 |
| jidar | (and I guess client) | 19:26 |
| fultonj | yeah you see the inventory for the ansible roles (diff from tripleo roles) is empty for rgw | 19:26 |
| fultonj | yeah, client is implicit | 19:26 |
| fultonj | e.g. the nova nodes are ceph clients | 19:27 |
| jidar | right, the client is on all nodes | 19:27 |
| fultonj | https://github.com/ceph/ceph-ansible/blob/v3.2.0/roles/ceph-config/templates/ceph.conf.j2#L47 | 19:27 |
| fultonj | you'd think the condition is supposed to prevent these things | 19:27 |
| fultonj | this seems familiar | 19:27 |
| jidar | I poked around bugzilla last night and didn't see anything | 19:28 |
| jidar | heck, I read every single ceph-ansible bug for the past year | 19:28 |
| fultonj | me too :) | 19:28 |
| fultonj | lol | 19:28 |
| jidar | lol | 19:28 |
| *** wolverineav has quit IRC | 19:28 | |
| fultonj | jidar: you have latest osp errata? | 19:28 |
| *** wolverineav has joined #tripleo | 19:29 | |
| jidar | I did a repo-sync for all of osp13 I want to say about a month ago | 19:29 |
| jidar | > rhel-7-server-rpms rhel-7-server-extras-rpms rhel-7-server-rh-common-rpms rhel-ha-for-rhel-7-server-rpms rhel-7-server-openstack-13-rpms rhel-7-server-rhceph-3-osd-rpms rhel-7-server-rhceph-3-mon-rpms rhel-7-server-rhceph-3-tools-rpms | 19:30 |
| jidar | these were the repos I pulled down | 19:30 |
| *** pcaruana has quit IRC | 19:30 | |
| jidar | actually, jan-03 | 19:30 |
| jidar | 4 weeks to the day | 19:30 |
| *** fmount has quit IRC | 19:32 | |
| *** fmount has joined #tripleo | 19:33 | |
| *** sshnaidm is now known as sshnaidm|off | 19:33 | |
| *** wolverineav has quit IRC | 19:35 | |
| *** boazel_ has joined #tripleo | 19:35 | |
| *** wolverineav has joined #tripleo | 19:35 | |
| fultonj | https://cryptic-cliffs-32040.herokuapp.com/ | 19:35 |
| jidar | oo | 19:36 |
| fultonj | let's you tweak a j2 expression | 19:37 |
| *** boazel__ has joined #tripleo | 19:38 | |
| fultonj | jidar: what python version do you have? | 19:38 |
| *** boazel has quit IRC | 19:38 | |
| fultonj | on that node | 19:39 |
| fultonj | .8 | 19:39 |
| *** boazel has joined #tripleo | 19:40 | |
| jidar | python-2.7.5-76.el7.x86_64 | 19:41 |
| *** boazel_ has quit IRC | 19:42 | |
| *** trown|lunch is now known as trown | 19:43 | |
| *** boazel__ has quit IRC | 19:43 | |
| fultonj | jidar: do that host actually have '172.16.1.0' ? | 19:43 |
| fultonj | ip a | grep 172.16.1 | 19:43 |
| *** trozet_ is now known as trozet | 19:43 | |
| jidar | it does not | 19:44 |
| fultonj | https://github.com/ceph/ceph-ansible/issues/2757 | 19:44 |
| *** dsneddon has quit IRC | 19:46 | |
| fultonj | jidar: is there a partion ceph.conf? | 19:46 |
| fultonj | partial | 19:47 |
| jidar | no, but the directories exist | 19:47 |
| jidar | which is the task right before generating that conf | 19:47 |
| fultonj | as a hack you could see if behavior changes when you remove the |first | 19:48 |
| fultonj | and comment on that issue (perhaps repoen) | 19:49 |
| fultonj | if you know what vars are coming out you can use the jina2 site to eval them | 19:49 |
| jidar | hmmmmm | 19:49 |
| fultonj | or try ceph-ansible 3.2.2 | 19:50 |
| jidar | does this ceph host not having a 172.16 ip (yet) be the issue? | 19:50 |
| fultonj | though the template hasn't changed in a while | 19:50 |
| fultonj | jidar: you have NO 192.16 IP? | 19:50 |
| fultonj | jidar: you have NO 172.16 IP? | 19:50 |
| jidar | now I'm starting to wonder what's going on there | 19:50 |
| *** jcoufal has quit IRC | 19:50 | |
| fultonj | jidar: it needs to exist first | 19:51 |
| jidar | no, https://imgur.com/cDUSwXQ | 19:51 |
| fultonj | it's going to make a mon listen on that IP but can't if it's not there | 19:51 |
| jidar | I don't see any errors in os-collect-config | 19:51 |
| *** shardy has quit IRC | 19:51 | |
| fultonj | ah | 19:51 |
| *** ekultails has quit IRC | 19:51 | |
| fultonj | so your network needs to be configured | 19:52 |
| fultonj | you should be able to map your devices en01, en03 etc to IPs in the network | 19:52 |
| fultonj | network config | 19:52 |
| fultonj | en01 is prov network | 19:52 |
| fultonj | you have 0 network isolation | 19:53 |
| fultonj | you can drop it for now and get ceph working all on the same network | 19:53 |
| *** shardy has joined #tripleo | 19:53 | |
| fultonj | in production you'd wnat something more realistic | 19:53 |
| jidar | yea, I wonder if it's that the ceph-storage profile isn't being picked up properly, I built a bond in its template | 19:54 |
| fultonj | anyway you need to backbrack and get this stuff working first | 19:54 |
| fultonj | https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html' | 19:54 |
| fultonj | so simplify your network to be flat on en01 to finish up ceph | 19:54 |
| jidar | ie: https://gist.github.com/d3934744e81da7353604ed1f0778290c | 19:54 |
| fultonj | and then backtrack to network isolation | 19:54 |
| fultonj | or fix the network first | 19:54 |
| *** shardy has quit IRC | 19:55 | |
| *** holser_ has joined #tripleo | 19:55 | |
| *** weshay has quit IRC | 19:56 | |
| fultonj | https://imgur.com/cDUSwXQ is inconsistent with https://gist.github.com/d3934744e81da7353604ed1f0778290c | 19:56 |
| *** rnoriega has quit IRC | 19:57 | |
| fultonj | ok, that's nearly 1 hour on this. i gotta get back to my patch | 19:57 |
| jidar | yea | 19:57 |
| jidar | thanks fultonj | 19:57 |
| *** abishop has quit IRC | 19:57 | |
| *** rnoriega has joined #tripleo | 19:57 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: break out install-deps from quickstart.sh https://review.openstack.org/634301 | 20:00 |
| *** weshay has joined #tripleo | 20:01 | |
| *** ansmith has quit IRC | 20:02 | |
| *** dsneddon has joined #tripleo | 20:02 | |
| *** vinaykns has joined #tripleo | 20:02 | |
| *** mjturek has quit IRC | 20:03 | |
| *** vinaykns has quit IRC | 20:03 | |
| *** vkapalav has quit IRC | 20:04 | |
| *** wolverineav has quit IRC | 20:04 | |
| *** wolverineav has joined #tripleo | 20:05 | |
| *** mjturek has joined #tripleo | 20:05 | |
| *** mjturek has quit IRC | 20:09 | |
| *** vinaykns has joined #tripleo | 20:10 | |
| *** mjturek has joined #tripleo | 20:10 | |
| *** jfrancoa has quit IRC | 20:13 | |
| *** florianf has quit IRC | 20:13 | |
| *** wolverineav has quit IRC | 20:13 | |
| *** wolverineav has joined #tripleo | 20:14 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 20:17 |
| *** wolverineav has quit IRC | 20:18 | |
| *** holser_ has quit IRC | 20:21 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: Update to the ceilometer publisher list https://review.openstack.org/600843 | 20:21 |
| *** morazi has joined #tripleo | 20:24 | |
| *** mjturek has quit IRC | 20:34 | |
| *** rfolco has quit IRC | 20:40 | |
| *** wolverineav has joined #tripleo | 20:45 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: Add an Endpoint to Ceilometer Publish list. https://review.openstack.org/634309 | 20:47 |
| *** wolverineav has quit IRC | 20:49 | |
| *** zaneb has quit IRC | 20:50 | |
| *** wolverineav has joined #tripleo | 20:50 | |
| openstackgerrit | Merged openstack/tripleo-common master: Remove placement healthcheck from overrides https://review.openstack.org/634059 | 20:50 |
| *** Chaserjim has joined #tripleo | 20:50 | |
| Chaserjim | anyone know how the scale-out "adding a node" procedure might interact with a backup NFS mountpoint? | 20:51 |
| *** jcoufal has joined #tripleo | 21:02 | |
| *** paramite has quit IRC | 21:05 | |
| *** holser_ has joined #tripleo | 21:12 | |
| *** wolverineav has quit IRC | 21:16 | |
| *** wolverineav has joined #tripleo | 21:18 | |
| *** xek_ has quit IRC | 21:19 | |
| *** xek has joined #tripleo | 21:19 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: WIP - kolla_builder: allow to only generate templates https://review.openstack.org/634317 | 21:21 |
| *** wolverineav has quit IRC | 21:24 | |
| *** rh-jelabarre has quit IRC | 21:33 | |
| *** wolverineav has joined #tripleo | 21:34 | |
| *** radeks_ has quit IRC | 21:35 | |
| *** artom has quit IRC | 21:38 | |
| *** ansmith has joined #tripleo | 21:48 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Per role Numa aware vswitch configuration https://review.openstack.org/613000 | 21:53 |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Publish error for package_update workflow https://review.openstack.org/633966 | 21:53 |
| *** bfournie has quit IRC | 21:58 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: WIP - add --template-only and --work-dir https://review.openstack.org/634322 | 21:59 |
| *** trown is now known as trown|outtypewww | 22:02 | |
| *** ccamacho has quit IRC | 22:10 | |
| *** agopi has quit IRC | 22:16 | |
| *** jcoufal has quit IRC | 22:19 | |
| *** holser_ has quit IRC | 22:21 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: use bindep to install rpm deps https://review.openstack.org/634329 | 22:22 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: mysql: do not stop container when upgrade doesn't update mysql image https://review.openstack.org/634330 | 22:24 |
| *** kplant has quit IRC | 22:29 | |
| *** sdake has quit IRC | 22:30 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 22:40 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: WIP - kolla_builder: allow to only generate templates https://review.openstack.org/634317 | 22:40 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: WIP: Fix repo configuration https://review.openstack.org/633527 | 22:42 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: use bindep to install rpm deps https://review.openstack.org/634329 | 22:44 |
| *** sdake has joined #tripleo | 22:45 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: WIP - add --template-only and --work-dir https://review.openstack.org/634322 | 22:48 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: WIP - kolla_builder: allow to only generate templates https://review.openstack.org/634317 | 22:50 |
| *** aedc has quit IRC | 22:50 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: WIP: DNM: Design new role for zuul-based reproducer https://review.openstack.org/631067 | 22:53 |
| *** mcornea has quit IRC | 22:54 | |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: step1: flatten nova service configuration https://review.openstack.org/632522 | 22:55 |
| *** tkajinam has joined #tripleo | 22:56 | |
| *** holser_ has joined #tripleo | 22:58 | |
| *** agopi has joined #tripleo | 22:59 | |
| *** bfournie has joined #tripleo | 23:01 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: WIP: Fix repo configuration https://review.openstack.org/633527 | 23:04 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: use bindep to install rpm deps https://review.openstack.org/634329 | 23:06 |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: flatten ceilometer service configuration https://review.openstack.org/633617 | 23:09 |
| *** rascasoft has quit IRC | 23:10 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: break out install-deps from quickstart.sh https://review.openstack.org/634301 | 23:12 |
| *** slaweq has quit IRC | 23:14 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 23:16 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Make container build job voting https://review.openstack.org/634060 | 23:17 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Revert "Remove placement healthcheck from overrides" https://review.openstack.org/634064 | 23:17 |
| *** cfontain has joined #tripleo | 23:20 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Fix repo configuration https://review.openstack.org/633527 | 23:23 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Make container build job voting https://review.openstack.org/634060 | 23:24 |
| *** cfontain has quit IRC | 23:24 | |
| *** holser_ has quit IRC | 23:27 | |
| *** vinaykns has quit IRC | 23:28 | |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: step2: flatten nova service configuration https://review.openstack.org/633278 | 23:35 |
| *** nkinder has joined #tripleo | 23:39 | |
| *** wolverineav has quit IRC | 23:44 | |
| *** wolverineav has joined #tripleo | 23:44 | |
| *** wolverineav has quit IRC | 23:46 | |
| *** wolverineav has joined #tripleo | 23:46 | |
| *** rascasoft has joined #tripleo | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!