| openstackgerrit | Merged openstack/python-tripleoclient master: undercloud_deploy: add opts to setup virtual-ips https://review.openstack.org/526879 | 00:02 |
|---|---|---|
| rlandy | mwhahaha: I don't see any tripleo bug report yet from infra. I can see how CI could set up dns with ovb/other deployments but not with multinode (which runs on the donor cloud). I'll be back in a few hours to check the bug list | 00:05 |
| fungi | rlandy: i'm still working on it | 00:08 |
| fungi | just finished the correlation: https://etherpad.openstack.org/p/wpy5XvRYWu | 00:08 |
| fungi | good news is tripleo-ci only seems to be the snmp public community exposed to internet ones | 00:08 |
| fungi | openstack-helm is exposing ipmi and open recursive dns resolvers | 00:09 |
| fungi | kolla ansible is the memcached hits | 00:09 |
| fungi | and the ntp servers came out since they turned out to not be our instances (something in the provider's infrastructure, we think) | 00:09 |
| rlandy | looking | 00:09 |
| rlandy | yeah - I wasn't sure how tripleo ci multinode was exposing dns | 00:10 |
| fungi | i'll get the snmp thing summarized into the tripleo bug i'm opening, and i'll get bugs open against openstack-helm and kolla-ansible for the rest | 00:10 |
| rlandy | fungi: thanks | 00:14 |
| *** itlinux has joined #tripleo | 00:16 | |
| *** liverpoo1er has quit IRC | 00:17 | |
| *** liverpooler has quit IRC | 00:17 | |
| *** liverpooler has joined #tripleo | 00:21 | |
| *** liverpoo1er has joined #tripleo | 00:21 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Fix lint errors https://review.openstack.org/542948 | 00:26 |
| EmilienM | fungi: when you have something please link it here, I'm here all evening | 00:28 |
| *** agopi_out has joined #tripleo | 00:32 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: prep-containers: include containerized undercloud bits https://review.openstack.org/543024 | 00:32 |
| *** tosky has quit IRC | 00:33 | |
| rlandy | EmilienM: see the SNMP listings in https://etherpad.openstack.org/p/wpy5XvRYWu | 00:33 |
| EmilienM | dmsimard, pabelanger, mwhahaha : FYI third party voting works: https://review.openstack.org/#/c/544005/ | 00:34 |
| fungi | rlandy: EmilienM: mwhahaha: i've opened bug 1749324 | 00:34 |
| openstack | bug 1749324 in tripleo "Exploitable services exposed on community test nodes" [Critical,Triaged] https://launchpad.net/bugs/1749324 | 00:34 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 00:34 |
| fungi | i'm proceeding to open similar ones for openstack-helm and kolla-ansible | 00:34 |
| EmilienM | rlandy: ok I'm looking at it | 00:34 |
| EmilienM | rlandy: can you please review https://review.rdoproject.org/r/#/c/12155/ ? | 00:35 |
| rlandy | looking | 00:35 |
| EmilienM | sshnaidm|off: please update https://review.rdoproject.org/r/#/c/12153/ and remove -nv, so we can move forward. Thanks | 00:35 |
| dmsimard | fungi: thanks | 00:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run ovb-ha with a containerized undercloud https://review.openstack.org/542556 | 00:35 |
| EmilienM | I'm looking at snmp | 00:37 |
| fungi | thanks EmilienM! | 00:37 |
| EmilienM | fungi: any chance to tell if it's subnode-2? | 00:37 |
| EmilienM | but either way I'm investigating | 00:37 |
| fungi | EmilienM: the etherpad does have that detail | 00:37 |
| EmilienM | oh right sorry | 00:37 |
| fungi | looks like it is | 00:37 |
| fungi | e.g. <Node 0002477112 ['secondary-2']:centos-7> | 00:38 |
| EmilienM | it sounds like the overcloud | 00:38 |
| fungi | skimming i think it's always secondary-2 | 00:38 |
| * EmilienM pokes at it now | 00:38 | |
| *** links has joined #tripleo | 00:40 | |
| EmilienM | the snmp config is here: http://logs.openstack.org/48/542948/1/gate/tripleo-ci-centos-7-containers-multinode/64fbe0f/logs/subnode-2/etc/snmp/snmpd.conf.txt.gz | 00:45 |
| EmilienM | rocommunity public 127.0.0.1 | 00:45 |
| EmilienM | rocommunity6 public ::1 | 00:45 |
| EmilienM | fungi: "tripleo-ci only seems to be the snmp public community exposed to internet ones" where did you see that? | 00:47 |
| *** lblanchard has joined #tripleo | 00:48 | |
| EmilienM | I'm trying to see what is unsecure in the snmpd.conf | 00:48 |
| fungi | EmilienM: the provider managed to get those servers to respond to snmp gets with a read community of "public" | 00:49 |
| *** mdnadeem has joined #tripleo | 00:50 | |
| fungi | all instances where the servers responded to snmpget turned out to be running one of those tripleo-ci jobs mentioned in the bug report | 00:50 |
| EmilienM | rocommunity public 127.0.0.1 | 00:50 |
| EmilienM | I'm reading man now but to me the config looks good so far -- still investigating | 00:51 |
| fungi | did that change recently? the most recent response they got from one was 2018-02-11 04:29:13 utc, but i have no idea whether that's just the last time they scanned or if it's been fixed in the past few days | 00:51 |
| EmilienM | I can check our stable branch CI jobs | 00:52 |
| fungi | that was the most recent incident they reported to me | 00:52 |
| EmilienM | on ocata: http://logs.openstack.org/11/541311/1/gate/tripleo-ci-centos-7-nonha-multinode-oooq/451f201/logs/subnode-2/etc/snmp/snmpd.conf.txt.gz | 00:53 |
| EmilienM | same config | 00:53 |
| EmilienM | rocommunity public 127.0.0.1 | 00:53 |
| EmilienM | and we have: createUser ro_snmp_user MD5 "273a2b6e4eeae4a72ae7b762ee375b528397cf0c" | 00:53 |
| EmilienM | which is very secure afict | 00:53 |
| EmilienM | but obviously missing something | 00:53 |
| EmilienM | and same on pike: http://logs.openstack.org/69/544069/1/check/tripleo-ci-centos-7-containers-multinode/ae91c36/logs/subnode-2/etc/snmp/snmpd.conf.txt.gz | 00:54 |
| fungi | we likely need to catch one of those jobs in progress and try running an snmpget against it or something | 00:54 |
| EmilienM | fungi: easy, we can do it now if you want | 00:55 |
| EmilienM | a bunch of jobs are being run | 00:55 |
| fungi | but i would find it hard to believe they magically guessed which nodes were running as the second node of a tripleo-ci multinode job | 00:55 |
| EmilienM | and they all deploy SNMP | 00:55 |
| EmilienM | http://zuul.openstack.org/stream.html?uuid=6c8eeb95bfd64316a2dfe03a4bcfa5ba&logfile=console.log | 00:55 |
| EmilienM | for example if you can catch this one ^ | 00:56 |
| EmilienM | from 543508 | 00:56 |
| EmilienM | fungi: ^ | 00:56 |
| fungi | i'm in the middle of openening the rest of these bugs against the other projects indicated in the report | 00:56 |
| EmilienM | ok no prob | 00:56 |
| *** pliu_ has quit IRC | 00:59 | |
| *** sai___ has joined #tripleo | 01:02 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749324 | 01:10 |
| openstack | Launchpad bug 1749324 in tripleo "Exploitable services exposed on community test nodes" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| *** psahoo has joined #tripleo | 01:11 | |
| *** moshele has joined #tripleo | 01:15 | |
| fungi | EmilienM: okay, i'm done opening tickets and pinging people in irc, taking a look at a tripleo job build now to see what i can get from it | 01:15 |
| EmilienM | fungi: let me tell you one | 01:17 |
| EmilienM | one sec | 01:17 |
| EmilienM | https://review.openstack.org/543508 | 01:17 |
| EmilienM | http://zuul.openstack.org/stream.html?uuid=6c8eeb95bfd64316a2dfe03a4bcfa5ba&logfile=console.log | 01:17 |
| EmilienM | if you can get it | 01:17 |
| fungi | i think i have one, just trying to run it down | 01:17 |
| fungi | 158.69.95.72 | 01:18 |
| fungi | testing `snmpwalk -v1 -c public 158.69.95.72 .1.3.6.1.2.1.1.3.0` and get Timeout: No Response from 158.69.95.72 | 01:19 |
| *** itlinux has quit IRC | 01:19 | |
| fungi | i'll try to catch the one you linked | 01:19 |
| EmilienM | fungi: the password is generated during the deployment | 01:19 |
| EmilienM | you need to log on subnode-2 and see in /etc/snmp/snmpd.conf | 01:20 |
| EmilienM | so you can get credentials | 01:20 |
| EmilienM | alee: around? | 01:20 |
| fungi | well, the provider was finding instances responding to an snmp v1/2 with community "public" | 01:20 |
| EmilienM | sure I believe them | 01:21 |
| EmilienM | but so far the config looks secure and we aren't able to reproduce it | 01:21 |
| fungi | 23.253.109.191 is the one you suggested | 01:21 |
| fungi | also no response for me | 01:22 |
| *** tcw has quit IRC | 01:22 | |
| EmilienM | do we have more details on the attack? | 01:22 |
| fungi | they had examples of snmp retrieving the uname details: https://gist.githubusercontent.com/mgagne/5a90a68dbc829a702ab0de1b54f205c3/raw/b5c05feda6318d117722d08240ba47bccac7e7da/gistfile1.txt | 01:23 |
| EmilienM | jaosorior, alee : issue to build barbican-api containers, it blocks production chain | 01:23 |
| EmilienM | "SNMP" | 01:23 |
| EmilienM | ok? | 01:24 |
| EmilienM | yeah the port is open | 01:24 |
| EmilienM | but then you need credentials | 01:24 |
| fungi | well, i don't see how they're retrieving the uname oid (which is what it looks like there?) if snmp is locked down | 01:25 |
| fungi | somehow they're getting snmpd to report back the hostname, kernel version, build date and architecture | 01:26 |
| EmilienM | since it's inap we should ask mgagne | 01:26 |
| EmilienM | mgagne: if you around we need help on the security thing that you guys found | 01:26 |
| fungi | yep, he's the one who handed thid sata over to me | 01:27 |
| fungi | er, this data | 01:27 |
| EmilienM | probably a bit late for east coast | 01:28 |
| * EmilienM remembers fungi is east coast as well | 01:28 | |
| fungi | yeah, we can likely pick it up tomorrow | 01:28 |
| EmilienM | jaosorior: I think you broke it | 01:29 |
| EmilienM | with https://review.openstack.org/#/c/543456 | 01:29 |
| *** liverpooler has quit IRC | 01:35 | |
| EmilienM | jaganathan__: seems like "cannot operate on dangling symlink" on barbican healthchecks, I can't figure why now | 01:42 |
| EmilienM | jaosorior: ^ | 01:42 |
| EmilienM | but you can easily see it on https://review.rdoproject.org/jenkins/job/rdo-kolla-build-integration/2346/consoleFull | 01:42 |
| *** itlinux has joined #tripleo | 01:44 | |
| *** moshele has quit IRC | 01:47 | |
| EmilienM | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1749335 | 02:06 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 02:07 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 02:10 |
| *** itlinux has quit IRC | 02:10 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Prepare t-h-t for undercloud in a work dir https://review.openstack.org/542875 | 02:16 |
| EmilienM | jaosorior: also IPsec job is failing | 02:19 |
| EmilienM | https://logs.rdoproject.org/38/523638/32/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht/Z095530f2472142498828832f6e760599/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2018-02-13_16_38_49 | 02:19 |
| *** ebarrera has quit IRC | 02:20 | |
| openstackgerrit | Merged openstack/instack-undercloud master: docker: use ensure_resource https://review.openstack.org/543508 | 02:20 |
| EmilienM | dsneddon: FYI I approved https://review.openstack.org/#/c/523638/ - we need to be careful from now if things go south | 02:21 |
| EmilienM | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1749338 | 02:23 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 02:23 |
| *** itlinux has joined #tripleo | 02:32 | |
| *** jaganathan__ has quit IRC | 02:39 | |
| *** jaganathan__ has joined #tripleo | 02:41 | |
| *** mdnadeem has quit IRC | 02:42 | |
| *** yamahata has quit IRC | 02:42 | |
| EmilienM | mandre: can you look my comment on https://review.openstack.org/#/c/528679/3 please | 02:42 |
| *** itlinux has quit IRC | 02:44 | |
| *** ramishra has joined #tripleo | 02:45 | |
| *** myoung|bbl is now known as myoung | 02:46 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Add tripleo-ui container https://review.openstack.org/515407 | 02:49 |
| *** d0ugal has quit IRC | 02:49 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: [WIP] tripleo ui docker https://review.openstack.org/515490 | 02:50 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add tls roles for undercloud https://review.openstack.org/517079 | 02:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Undercloud: fix stackrc TLS URL detection https://review.openstack.org/535480 | 02:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Undercloud: support for external VIP for SSL https://review.openstack.org/535479 | 02:53 |
| EmilienM | stevebaker: hey, need your help if you're still around | 02:55 |
| stevebaker | EmilienM: yo | 02:55 |
| EmilienM | stevebaker: yo | 02:55 |
| EmilienM | stevebaker: I'm deploying a containerized undercloud and then a containerized overcloud | 02:56 |
| EmilienM | stevebaker: and it fails to prepare containers for the overcloud | 02:56 |
| EmilienM | http://logs.openstack.org/06/542906/9/check/tripleo-ci-centos-7-containers-multinode/b2198b7/logs/undercloud/home/zuul/prep_containers_overcloud.log.txt.gz#_2018-02-14_01_35_25 | 02:56 |
| EmilienM | again this error | 02:56 |
| stevebaker | EmilienM: is that still the docker group thing? | 02:57 |
| EmilienM | oh I think I know why | 02:57 |
| EmilienM | yeah the code isn't in puppet-tripleo yet | 02:57 |
| EmilienM | stevebaker: thx. I'm pushing a patch | 02:57 |
| EmilienM | stevebaker: but there is this user thing | 02:59 |
| EmilienM | I guess I need to move the parameter to undercloud.conf and consume it from puppet? mhh I hate that | 03:00 |
| *** d0ugal has joined #tripleo | 03:00 | |
| EmilienM | right now we compute the username in instack-undercloud | 03:00 |
| EmilienM | but with containerized undercloud we need to do something else | 03:00 |
| EmilienM | stevebaker: what is the recommended way for our production users? | 03:00 |
| EmilienM | to manually add the user to docker group? | 03:01 |
| *** lblanchard has quit IRC | 03:03 | |
| stevebaker | EmilienM: that doesn't seem ideal, but if we did require that we could at least add a preflight check to assert it | 03:04 |
| *** ohochman has quit IRC | 03:05 | |
| stevebaker | EmilienM: actually, just for this issue the prep_containers script could run as sudo | 03:05 |
| stevebaker | EmilienM: I mean the upload command inside the prep_containers can be sudo | 03:05 |
| stevebaker | EmilienM: ah, production docs also run the upload command as sudo | 03:08 |
| stevebaker | EmilienM: our upstream docs do not use sudo, for the upload command, maybe they should https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html#populate-local-docker-registry | 03:09 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 03:10 |
| stevebaker | EmilienM: with undercloud install --use-heat, what is it that installs/enables/starts docker and docker-registry? | 03:12 |
| *** gvrangan has joined #tripleo | 03:15 | |
| *** rlandy has quit IRC | 03:15 | |
| *** gvrangan has quit IRC | 03:19 | |
| *** skarthik has joined #tripleo | 03:22 | |
| *** gkadam has joined #tripleo | 03:24 | |
| *** gvrangan has joined #tripleo | 03:27 | |
| *** skarthik has quit IRC | 03:31 | |
| EmilienM | stevebaker: sorry I was afk for dinner | 03:31 |
| EmilienM | stevebaker: puppet-tripleo docker.pp | 03:32 |
| stevebaker | EmilienM: no problem | 03:32 |
| EmilienM | if prod doc says sudo, let's use sudo | 03:32 |
| EmilienM | so we don't have to do magics | 03:32 |
| EmilienM | wdyt? | 03:32 |
| EmilienM | (and of course fix the upstream doc) | 03:33 |
| EmilienM | I'm back in 1h. bbl | 03:33 |
| EmilienM | stevebaker: ^ | 03:34 |
| stevebaker | EmilienM: yeah, on the other hand its convenient for the undercloud user to be able to do other docker calls without sudo, and removing that capability could be seen as a regression. Why not have a user specified in undercloud.conf and have puppet-tripleo do it? | 03:35 |
| EmilienM | stevebaker: yeah also | 03:36 |
| EmilienM | stevebaker: is the default user "stack" on the undercloud right? | 03:36 |
| EmilienM | just looking for a default value | 03:36 |
| EmilienM | I need to disappear a little | 03:37 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add feature set to test public bond network isolation https://review.openstack.org/540152 | 03:38 |
| stevebaker | EmilienM: yeah, by convention we create and use a stack user, but I think it is appropriate for the user who runs the "undercloud install" command to be added to the docker group | 03:38 |
| *** skramaja has joined #tripleo | 03:42 | |
| *** atoth has quit IRC | 03:43 | |
| *** links has quit IRC | 03:50 | |
| *** skramaja has quit IRC | 03:50 | |
| *** skramaja has joined #tripleo | 03:50 | |
| *** skarthik has joined #tripleo | 03:52 | |
| *** d0ugal has quit IRC | 03:55 | |
| *** gvrangan has quit IRC | 03:57 | |
| *** pdeore has joined #tripleo | 03:58 | |
| *** gvrangan has joined #tripleo | 03:58 | |
| *** udesale has joined #tripleo | 03:59 | |
| *** d0ugal has joined #tripleo | 04:00 | |
| *** anilvenkata has joined #tripleo | 04:00 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Add docker service for neutron-ovs-dpdk-agent service https://review.openstack.org/543851 | 04:02 |
| *** links has joined #tripleo | 04:06 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 04:10 |
| *** gkadam_ has joined #tripleo | 04:13 | |
| *** gvrangan has quit IRC | 04:14 | |
| *** gkadam_ has quit IRC | 04:14 | |
| *** gkadam_ has joined #tripleo | 04:14 | |
| *** gkadam has quit IRC | 04:15 | |
| *** gvrangan has joined #tripleo | 04:15 | |
| *** cshastri has joined #tripleo | 04:20 | |
| *** ykarel_ has joined #tripleo | 04:22 | |
| *** gvrangan has quit IRC | 04:22 | |
| *** gvrangan has joined #tripleo | 04:34 | |
| *** gvrangan has quit IRC | 04:39 | |
| *** olivierbourdon38 has joined #tripleo | 04:49 | |
| *** gvrangan has joined #tripleo | 04:50 | |
| *** gvrangan has quit IRC | 04:53 | |
| jaosorior | EmilienM: fuck | 04:54 |
| jaosorior | when did it start failing?> | 04:54 |
| jaosorior | the IPSec job | 04:54 |
| jaosorior | would this help the barbican job https://review.openstack.org/#/c/543473/ ? | 04:55 |
| jaosorior | EmilienM: | 04:55 |
| *** masco has joined #tripleo | 04:57 | |
| jaganathan__ | d0ugal, please look into https://review.openstack.org/#/c/527112/ https://review.openstack.org/#/c/527877/ | 05:01 |
| *** psachin has joined #tripleo | 05:03 | |
| EmilienM | jaosorior: yesterday (for you) | 05:05 |
| EmilienM | jaosorior: you got it... | 05:06 |
| EmilienM | jaosorior: we need to be really careful of all jobs even non voting, when approving patches | 05:06 |
| EmilienM | jaosorior: we're working on making third party jobs voting but it'll take some days, in meantime, let's be super careful | 05:06 |
| EmilienM | I'm approving https://review.openstack.org/#/c/543473/ | 05:07 |
| *** liverpoo1er has quit IRC | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 05:10 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: DNM - manage docker group https://review.openstack.org/544167 | 05:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 05:11 |
| jaosorior | EmilienM: did something change recently in the dynamic inventory code? | 05:16 |
| EmilienM | jaosorior: I'm not sure about that | 05:16 |
| jaosorior | or in the hostnames? | 05:16 |
| EmilienM | ah yes | 05:16 |
| EmilienM | let me find it, give me a sec | 05:16 |
| jaosorior | the task that's failing is this one https://github.com/openstack/tripleo-ipsec/blob/master/tasks/ipsec-conf.yml#L19 | 05:17 |
| EmilienM | we did things like this https://review.openstack.org/#/c/540084/ | 05:17 |
| jaosorior | all it does it get the inventory hostname, and fetch the IP from the fact | 05:17 |
| *** moshele has joined #tripleo | 05:17 | |
| EmilienM | to generate HostnameMap in quickstart | 05:17 |
| jaosorior | is that used in the ovb jobs? | 05:18 |
| EmilienM | I'm not sure but we would probably have to | 05:18 |
| jaosorior | ok, goo dto know | 05:18 |
| EmilienM | jaosorior: I'm going to bed, was a long day | 05:19 |
| jaosorior | I'll check it out | 05:19 |
| EmilienM | I can check out too tomorrow, sorry I'm leaving now | 05:19 |
| EmilienM | ttyl | 05:19 |
| jaosorior | have a good one | 05:19 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: DNM - manage docker group https://review.openstack.org/544167 | 05:19 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 05:20 |
| *** moshele has quit IRC | 05:27 | |
| *** skarthik has quit IRC | 05:31 | |
| *** karthiks has joined #tripleo | 05:31 | |
| *** karthiks has joined #tripleo | 05:32 | |
| *** janki has joined #tripleo | 05:36 | |
| Tengu | hello there | 05:41 |
| *** dpawar has joined #tripleo | 05:41 | |
| *** akane_ has joined #tripleo | 05:43 | |
| *** psachin has quit IRC | 05:45 | |
| *** bfournie has quit IRC | 05:46 | |
| jaosorior | Tengu: sup dude | 05:48 |
| *** bfournie has joined #tripleo | 05:49 | |
| *** pgadiya has joined #tripleo | 05:49 | |
| Tengu | a bit frozen to be honnest :). was about 2°C today on the road, and some snow was falling. nice day for motorbike :). what about you? | 05:49 |
| * Tengu ready to sing "let it go" | 05:50 | |
| Tengu | :] | 05:50 |
| jaosorior | lol dafuq | 05:50 |
| jaosorior | uuuh... dealing with broken stuff | 05:50 |
| Tengu | as usual isn't it? | 05:50 |
| Tengu | ;) | 05:50 |
| Tengu | need some external eye? | 05:50 |
| jaosorior | not sure if there's relevant logs in CI | 05:51 |
| jaosorior | trying to reproduce it currently | 05:51 |
| Tengu | hmm ok | 05:51 |
| jaosorior | but the issue is this https://bugs.launchpad.net/tripleo/+bug/1749338 | 05:51 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 05:51 |
| jaosorior | it fails with this error message {"msg": "Error in jmespath.search in json_query filter plugin:\n'query' is undefined"} | 05:51 |
| Tengu | I see that. where's the ansible stuff? | 05:52 |
| jaosorior | this is the specific line where it fails https://github.com/openstack/tripleo-ipsec/blob/master/tasks/ipsec-conf.yml#L19 | 05:52 |
| *** psachin has joined #tripleo | 05:52 | |
| jaosorior | and this is the inventory https://logs.rdoproject.org/38/523638/32/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht/Z095530f2472142498828832f6e760599/undercloud/var/lib/mistral/a5f46d87-ed89-44af-9805-3c596b6f8d1b/tripleo-ansible-inventory.yaml.txt.gz | 05:53 |
| jaosorior | all I really do is this "{{ hostvars[inventory_hostname][ip_var_key] }}" | 05:53 |
| jaosorior | where inventory_hostname should be provided by ansible | 05:53 |
| jaosorior | so that part is fine | 05:53 |
| jaosorior | in the logs, which are these https://logs.rdoproject.org/38/523638/32/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht/Z095530f2472142498828832f6e760599/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2018-02-13_16_38_49 we can see "ip_var_key": "management_ip" a little bit above the failure | 05:54 |
| jaosorior | so the ip_var_key is defined properly | 05:54 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Render NIC config templates with jinja2 https://review.openstack.org/523638 | 05:54 |
| jaosorior | and management_ip is defined for all nodes in the inventory | 05:54 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: remove duplicate README file https://review.openstack.org/542275 | 05:54 |
| jaosorior | so wtf | 05:54 |
| Tengu | not for undercloud in fact, but I guess it's intended? | 05:55 |
| jaosorior | correct | 05:55 |
| jaosorior | but that doesn't run in the undercloud | 05:55 |
| Tengu | was going to ask :) | 05:55 |
| jaosorior | Tengu: also, you can see that the IPs that reported the failure are the overcloud nodes | 05:56 |
| Tengu | hmm, is it really hostvars[] and not ansible_facts[] ? | 05:56 |
| jaosorior | Tengu: http://docs.ansible.com/ansible/latest/playbooks_variables.html#magic-variables-and-how-to-access-information-about-other-hosts | 05:57 |
| Tengu | hmm ok. | 05:57 |
| Tengu | what about quoting? | 05:57 |
| Tengu | in the example they actually quote the indexes. | 05:57 |
| jaosorior | Tengu: by the way, this had been working just fine for a while, it just broke yesterday, and the ansible playbook I linked you hasn't changed in some time | 05:58 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Add role_specific tag to missing role-specific parameters https://review.openstack.org/544170 | 05:58 |
| Tengu | jaosorior: duh. ok. | 05:58 |
| Tengu | jaosorior: did the ansible version on the CI change? | 05:58 |
| jaosorior | Tengu: they quote the indexes cause in the examples they're constants. In this case they're unquoted because they're variables. | 05:58 |
| jaosorior | Tengu: so that might be a possible cause | 05:58 |
| Tengu | jaosorior: ip_var_key is a constant string, isn't it? | 05:59 |
| jaosorior | Tengu: another possible cause is the ansible_hostname not being populated correctly | 05:59 |
| Tengu | i.e. you want to access the "ip_var_key" for host foo in hostvars hash | 05:59 |
| jaosorior | Tengu: nope, defined here https://github.com/openstack/tripleo-ipsec/blob/master/tasks/ipsec-conf.yml#L14 | 05:59 |
| Tengu | humpf. ok | 05:59 |
| jaosorior | just above the "Set current IP fact" | 05:59 |
| Tengu | I didn't really dig into the "facts" in ansible until now. | 06:00 |
| *** pdeore_ has joined #tripleo | 06:05 | |
| jaosorior | Tengu: aaand there might be another possible cause | 06:05 |
| jaosorior | so, for some reason, the result of this https://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps.j2 is saved in the inventory | 06:06 |
| jaosorior | any change that is not rendered correctly will break in the inventory if ran via mistral | 06:06 |
| jaosorior | this doesn't seem to be the case if ansible is ran manually | 06:06 |
| jaosorior | so it gets extra confusing :D | 06:06 |
| Tengu | erf | 06:07 |
| *** pdeore has quit IRC | 06:07 | |
| jaosorior | Tengu: lol yeah | 06:08 |
| Tengu | grmbl. network outage -.-. | 06:09 |
| Tengu | and peering issues now that it's back. | 06:10 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 06:10 |
| *** dpawar has quit IRC | 06:10 | |
| *** dpawar has joined #tripleo | 06:12 | |
| *** dpawar has quit IRC | 06:12 | |
| *** pdeore_ has quit IRC | 06:13 | |
| *** pdeore has joined #tripleo | 06:15 | |
| Tengu | jaosorior: have to debug some issues at my current work - apparently a network provider has "some" issues… the range they allocated us is down. | 06:17 |
| jaosorior | bummer | 06:18 |
| jaosorior | good luck | 06:18 |
| *** psachin` has joined #tripleo | 06:19 | |
| *** psachin` has quit IRC | 06:19 | |
| *** psachin has quit IRC | 06:20 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Revert "ffu: Add fast-forward upgrade outputs to RoleConfig" https://review.openstack.org/544175 | 06:20 |
| *** marios has joined #tripleo | 06:21 | |
| *** aufi has joined #tripleo | 06:27 | |
| *** waleedm has joined #tripleo | 06:29 | |
| *** psachin has joined #tripleo | 06:29 | |
| *** moshele has joined #tripleo | 06:30 | |
| *** ykarel_ is now known as ykarel | 06:31 | |
| *** dparkes has quit IRC | 06:33 | |
| *** agurenko has joined #tripleo | 06:33 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Revert "ffu: Add fast-forward upgrade outputs to RoleConfig" https://review.openstack.org/544175 | 06:46 |
| jaosorior | Tengu: spotted the issue ^^ | 06:46 |
| jaosorior | marios: ^^ :/ | 06:46 |
| Tengu | jaosorior: gg :) | 06:49 |
| Tengu | and our network seems to be stable again here. | 06:49 |
| marios | jaosorior: ack checking (so it appears there is a problem with the playbook 'outputs' from ffu in that patch? | 06:50 |
| jaosorior | marios: yes | 06:52 |
| marios | jaosorior: ack, added a comment on the review (I'll revote and we should merge it today unless someone else offers a better suggestion by EU lunchtime wdyt?) | 06:55 |
| marios | jaosorior: its a little early still for the rest of the folks in the EU team wdyt? | 06:56 |
| *** dciabrin has joined #tripleo | 06:57 | |
| marios | jaosorior: i'm scanning https://review.openstack.org/#/c/499221/30/common/deploy-steps.j2 trying to spot the issue (maybe its in the yaql collector in services.yaml?) | 06:57 |
| marios | chem: fyi https://review.openstack.org/#/c/544175/2 lucas isn't around (he might still be away today lets see) | 06:57 |
| jaosorior | marios: no worries, I'm trying to fix it as well, so the revert hopefully is not the ultimate solution | 06:57 |
| marios | not lyarwood | 06:57 |
| marios | s/not/nor | 06:57 |
| jaosorior | marios: so... slagle had stumbled around some issues regarding the rendering (something about the quoting) which was ultimately fixed by shardy moving most of the deploy steps to deploy-step-tasks.yaml instead of the deploy-steps.j2 | 06:58 |
| *** stendulker has joined #tripleo | 06:59 | |
| jaosorior | however, I'm not entirely sure how that part worked. Nor why we're putting the output of deploy-steps.j2 as part of the ansible inventory | 06:59 |
| marios | jaosorior: but hold on, i mean, just reverting that ffu patch fixed it for you? | 07:00 |
| jaosorior | marios: pretty much | 07:01 |
| marios | jaosorior: heh 'pretty much' != yes but OK i'll take it as a yes :) | 07:01 |
| jaosorior | it's a yes in my book | 07:01 |
| jaosorior | :P | 07:01 |
| marios | :) | 07:01 |
| jaosorior | my deployment is still on-going, but it already finished the IPSec deployment. So yeah, reverting it fixes the issue. | 07:02 |
| marios | cool tx | 07:02 |
| *** jchhatbar has joined #tripleo | 07:05 | |
| *** janki has quit IRC | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 07:10 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Minor update steps for ODL https://review.openstack.org/536907 | 07:10 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Minor update steps for ODL https://review.openstack.org/536907 | 07:11 |
| *** dparkes has joined #tripleo | 07:11 | |
| *** jchhatbar is now known as janki | 07:14 | |
| *** yprokule has joined #tripleo | 07:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/544191 | 07:19 |
| *** mcarden has joined #tripleo | 07:22 | |
| *** sai___ has quit IRC | 07:22 | |
| *** mdnadeem has joined #tripleo | 07:30 | |
| *** lvdombrkr has joined #tripleo | 07:31 | |
| *** etingof has quit IRC | 07:32 | |
| *** agopi_out has quit IRC | 07:36 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Enable Barbican health checks https://review.openstack.org/543473 | 07:41 |
| *** dpawar has joined #tripleo | 07:41 | |
| *** cylopez has joined #tripleo | 07:44 | |
| *** ykarel_ has joined #tripleo | 07:47 | |
| *** ykarel has quit IRC | 07:48 | |
| *** ykarel__ has joined #tripleo | 07:52 | |
| *** ykarel has joined #tripleo | 07:54 | |
| *** ykarel_ has quit IRC | 07:55 | |
| *** pcaruana has joined #tripleo | 07:56 | |
| *** ykarel__ has quit IRC | 07:57 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 07:57 |
| *** pdeore has quit IRC | 07:57 | |
| *** dparkes has quit IRC | 08:03 | |
| skramaja | need reviews on https://review.openstack.org/#/c/543851/ and https://review.openstack.org/#/c/542623/ ? | 08:04 |
| *** ebarrera has joined #tripleo | 08:05 | |
| *** pdeore has joined #tripleo | 08:08 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 08:10 |
| *** holser has joined #tripleo | 08:12 | |
| *** gfidente has joined #tripleo | 08:14 | |
| *** gfidente has quit IRC | 08:14 | |
| *** gfidente has joined #tripleo | 08:14 | |
| *** jpena|off is now known as jpena | 08:16 | |
| *** florianf has joined #tripleo | 08:20 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/pike: DNM Testing O->P with tripleo-upgarde https://review.openstack.org/543907 | 08:20 |
| *** amoralej|off is now known as amoralej | 08:20 | |
| Tengu | grmbl. why doesn't tripleo find my ceph-storage nodes now… that's the last issue I have :(. | 08:22 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Set tripleo-upgrade vars in featureset. https://review.openstack.org/539910 | 08:22 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 08:22 |
| *** bogdando has joined #tripleo | 08:22 | |
| *** lvdombrkr89 has joined #tripleo | 08:23 | |
| *** ebarrera has quit IRC | 08:24 | |
| *** ebarrera has joined #tripleo | 08:24 | |
| *** lvdombrkr has quit IRC | 08:26 | |
| *** sshnaidm|off is now known as sshnaidm|ruck | 08:27 | |
| *** tesseract has joined #tripleo | 08:28 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Set docker and image value to the right value in mixed version. https://review.openstack.org/539929 | 08:29 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: [WIP] Use tripleo-upgrade role for p->m job. https://review.openstack.org/540072 | 08:29 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Convert Role detail dialog to redux-form https://review.openstack.org/544000 | 08:33 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Parameters cleanup https://review.openstack.org/544001 | 08:33 |
| *** rcernin has quit IRC | 08:33 | |
| *** lujinluo has joined #tripleo | 08:38 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Inherit UpdateCommand class from overcloud DeployCommand class https://review.openstack.org/540798 | 08:44 |
| bogdando | folks, how can I check the older builds history for https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-container-to-container-upgrades-master/ ? | 08:45 |
| bogdando | it shows only Feb 13 | 08:45 |
| openstackgerrit | Karthik S proposed openstack/os-net-config master: WIP: Adding VLAN support and other configs for sriov_vf https://review.openstack.org/541620 | 08:45 |
| bogdando | as we have now RDO CI switched voting, would be nice to know for how long those 2 minor/major upgrades jobs keep being red | 08:46 |
| *** dparkes has joined #tripleo | 08:46 | |
| bogdando | got it https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-container-to-container-upgrades-master/buildTimeTrend ! | 08:47 |
| bogdando | why did we set into voting the jobs being red for long? | 08:47 |
| bogdando | sshnaidm|ruck: ^^ perchance? | 08:48 |
| bogdando | mwhahaha: ^^ | 08:48 |
| bogdando | what is the procedure for making jobs voting? | 08:48 |
| sshnaidm|ruck | bogdando, they won't be voting, waiting on patch: https://review.rdoproject.org/r/#/c/12153/ | 08:49 |
| holser | @sshnaidm|ruck - please fix CI for https://review.openstack.org/#/c/544068/ | 08:49 |
| bogdando | sshnaidm|ruck: ack, thanks | 08:49 |
| *** gkadam__ has joined #tripleo | 08:50 | |
| holser | sshnaidm|ruck - Can we revert back CI to working stage, then fix with https://review.rdoproject.org/r/#/c/12153/ then merge it | 08:51 |
| *** oidgar has joined #tripleo | 08:51 | |
| holser | literally, all patches get -1 from RDO cloud CI | 08:51 |
| holser | and will require rerun which is time and resources | 08:52 |
| bogdando | holser: yes, I hope to see https://review.rdoproject.org/r/#/c/12153/ merged shortly though. Hope that works for us, if merged today | 08:52 |
| *** lujinluo has quit IRC | 08:52 | |
| sshnaidm|ruck | holser, I understand your concern, but reverting is not possible in current stage, we are waiting for patch to land | 08:52 |
| bogdando | we prolly can live one day blocked from merging :) | 08:52 |
| holser | Hope is not estimation | 08:52 |
| *** gkadam_ has quit IRC | 08:53 | |
| *** links has quit IRC | 08:53 | |
| holser | sshnaidm|ruck - Why can't we revert? | 08:53 |
| sshnaidm|ruck | holser, in your patch you mentioned there is gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master that fails and will vote in any case, so you need to ensure it passes in your patch firstly | 08:53 |
| holser | Can you please explain | 08:53 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 08:56 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Provide a way to change docker registry path during mixed upgrade. https://review.openstack.org/540473 | 08:56 |
| *** jpich has joined #tripleo | 08:59 | |
| *** links has joined #tripleo | 09:05 | |
| *** yamahata has joined #tripleo | 09:06 | |
| bogdando | holser: IIUC, sshnaidm|ruck means that that's gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master blocks your patch atm, not those minor/major jobs being permanently red for a while. Exploring the logs (with getthelogs) it seems like another known issue with heat stacks timing out | 09:06 |
| bogdando | you can get the tool to fetch logs locally here https://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/getthelogs holser | 09:07 |
| *** ledo_ has joined #tripleo | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 09:10 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| *** paramite has joined #tripleo | 09:13 | |
| *** waleedm has quit IRC | 09:13 | |
| jpich | EmilienM++ Thanks for the weekly owl, nice & helpful :) | 09:17 |
| *** jaganathan__ has quit IRC | 09:18 | |
| *** moguimar has quit IRC | 09:18 | |
| *** moguimar has joined #tripleo | 09:18 | |
| *** lyarwood has joined #tripleo | 09:19 | |
| *** ykarel is now known as ykarel|lunch | 09:19 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Use overcloud_templates for deployed server prepare scripts. https://review.openstack.org/535299 | 09:20 |
| *** janki has quit IRC | 09:22 | |
| *** yamahata has quit IRC | 09:24 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: [newton only] Fix get-occ-config when some roles are not defined. https://review.openstack.org/541245 | 09:24 |
| *** ledo_ has quit IRC | 09:25 | |
| *** pgadiya has quit IRC | 09:28 | |
| *** pgadiya has joined #tripleo | 09:29 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 09:29 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Use overcloud_templates for deployed server prepare scripts. https://review.openstack.org/535299 | 09:29 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add a hook to run a bash script at the end of the overcloud prep. https://review.openstack.org/533167 | 09:29 |
| *** yamahata has joined #tripleo | 09:31 | |
| *** pgadiya has quit IRC | 09:31 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/541548 | 09:32 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common-tempest-plugin master: Updated from global requirements https://review.openstack.org/544226 | 09:32 |
| *** assassin has quit IRC | 09:32 | |
| *** pgadiya has joined #tripleo | 09:33 | |
| *** pgadiya has quit IRC | 09:33 | |
| openstackgerrit | Merged openstack/tripleo-common master: Fix overcloud export plan https://review.openstack.org/543965 | 09:34 |
| jaosorior | lyarwood: the commit you pointed that looks more like what could have caused the issue. Thanks for pointing it out | 09:37 |
| *** waleedm has joined #tripleo | 09:38 | |
| *** pgadiya has joined #tripleo | 09:38 | |
| jaosorior | lyarwood: as of examples of how it gets rendered. I actually don't know how to get it. We had a similar issue before, since, for some reason, the tasks get persisted onto the ansible inventory. So slagle fixed a bunch of similar issues at some point. and ultimately shardy moved most problematic steps out of common/deploy-tasks.j2, which ended up fixing the issue. | 09:38 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 09:38 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Use overcloud_templates for deployed server prepare scripts. https://review.openstack.org/535299 | 09:38 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add a hook to run a bash script at the end of the overcloud prep. https://review.openstack.org/533167 | 09:38 |
| lyarwood | jaosorior: np, did you also back that change out in your test? | 09:39 |
| lyarwood | jaosorior: if that is the issue it should be a simple fix tbh | 09:39 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Provide a way to change docker registry path during mixed upgrade. https://review.openstack.org/540473 | 09:40 |
| jaosorior | lyarwood, marios: gonna test a revert with https://review.openstack.org/#/c/515429/ now | 09:40 |
| lyarwood | jaosorior: kk | 09:40 |
| *** derekh has joined #tripleo | 09:40 | |
| jaosorior | lyarwood: I didn't. merely reverting that one commit I posted was enough. | 09:40 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Revert "ffu: tripleo-packages repo management" https://review.openstack.org/544227 | 09:42 |
| lyarwood | jaosorior: right, thinking about this, that original revert corrects this as it removes the ffu tasks from tripleo-packages | 09:45 |
| lyarwood | jaosorior: but I think it's the actual tasks in tripleo-packages that are causing this | 09:45 |
| *** pblaho has joined #tripleo | 09:46 | |
| jaosorior | lyarwood: that makes sense. | 09:46 |
| jaosorior | lyarwood: I pushed the commit, so it'll be tested in CI. But I'm also testing locally. | 09:46 |
| lyarwood | jaosorior: ack, do we only have non-voting CI for this btw? | 09:47 |
| lyarwood | jaosorior: I assume that's how this was missed for so long? | 09:47 |
| *** etingof has joined #tripleo | 09:47 | |
| jaosorior | yep, it's non-voting | 09:47 |
| jaosorior | lyarwood: although, the failure had already manifested in that patch https://logs.rdoproject.org/29/515429/26/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht/Z94dad55087124e3189c5b8ea67cb1f92/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2018-02-09_20_06_12 | 09:49 |
| jaosorior | here https://review.openstack.org/#/c/515429/ in the gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht job | 09:49 |
| jaosorior | so yeah, it was unfortunate that the job wasn't voting | 09:50 |
| lyarwood | jaosorior: yeah indeed, chem has found something new FWIW, posting a change now that might avoid reverts | 09:51 |
| * lyarwood heads back to nova for a while | 09:51 | |
| jaosorior | lyarwood: something new? | 09:51 |
| jaosorior | lyarwood: good luck | 09:51 |
| lyarwood | jaosorior: well, a nit with the original ffu packages change I think | 09:52 |
| *** ukalifon has joined #tripleo | 09:52 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Add missing string block for fast_forward_upgrade_playbook. https://review.openstack.org/544229 | 09:54 |
| chem | jaosorior: lyarwood marios ^ this may be the root cause, even If I couldn't say why | 09:54 |
| *** stendulker_ has joined #tripleo | 09:55 | |
| *** salmankhan has joined #tripleo | 09:55 | |
| jaosorior | chem: alright! we'll see what CI says :D | 09:55 |
| jaosorior | hopefully that fixes it | 09:55 |
| lyarwood | haha, yeah that might be it | 09:55 |
| chem | jaosorior: lyarwood marios it's very hard to read actually | 09:55 |
| lyarwood | chem: without that, wouldn't we include fast_forward_upgrade_release_tasks.yaml in post_upgrade_steps_playbook ? | 09:56 |
| lyarwood | chem: ouch | 09:56 |
| lyarwood | chem: nice catch | 09:56 |
| chem | lyarwood: not sure what's going to happen to this "...: {get_param: [FastForwardUpgradeReleases]}" | 09:56 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Add healthcheck script for memcached https://review.openstack.org/544006 | 09:56 |
| chem | lyarwood: will it be rendered ok by, the level of parsing of this file make it hard to predict what's going to happen | 09:57 |
| *** stendulker has quit IRC | 09:57 | |
| chem | jaosorior: oki, let's wait on CI, until then I'll W-1, drop a not in the review when the test has run | 09:59 |
| marios | chem: checking | 09:59 |
| *** ykarel|lunch is now known as ykarel | 10:00 | |
| marios | chem: cool | 10:00 |
| marios | chem: thanks :) | 10:00 |
| chem | marios: too much magic in that file ... | 10:01 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/python-tripleoclient master: Prepare t-h-t for undercloud in a work dir https://review.openstack.org/542875 | 10:01 |
| marios | chem: honestly i was staring at it earlier | 10:02 |
| marios | chem: and didn't see that | 10:02 |
| *** shardy has joined #tripleo | 10:02 | |
| *** psahoo has quit IRC | 10:04 | |
| chem | marios: the only thing that bother me is the get_param inside this "new" string block, I'm not sure if heat parser will render it correctly | 10:04 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/python-tripleoclient master: Prepare t-h-t for undercloud in a work dir https://review.openstack.org/542875 | 10:04 |
| *** pcaruana has quit IRC | 10:05 | |
| chem | marios: I've droped a comment on my concern. | 10:06 |
| marios | chem: hmm good point | 10:06 |
| marios | chem: we might have to resolve that by setting jinja like we have for the _max things | 10:06 |
| chem | marios: a template_str may be the solution. Anyway let's wait for the ci result before moving forward, will see how to deal with that then I guess. | 10:07 |
| *** dsariel has joined #tripleo | 10:09 | |
| *** dsariel_ has joined #tripleo | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 10:10 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: New featureset for fast forward upgrade testing. https://review.openstack.org/525686 | 10:13 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-common master: WIP move password generation to deployment phase https://review.openstack.org/542143 | 10:13 |
| karthiks | chandankumar, Regarding https://review.rdoproject.org/r/#/c/12172/, the CI is failing. Could you please suggest if anything is missing ? | 10:14 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-docs master: WIP Add Instance HA docs https://review.openstack.org/544232 | 10:14 |
| openstackgerrit | hanish proposed openstack/tripleo-heat-templates master: Added and modified the services for ComputeLiquidio role https://review.openstack.org/544234 | 10:16 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: New featureset for fast forward upgrade testing. https://review.openstack.org/525686 | 10:18 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: libvirt hosts with reproducer https://review.openstack.org/543429 | 10:18 |
| *** pcaruana has joined #tripleo | 10:20 | |
| *** skramaja has quit IRC | 10:22 | |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Add support for libvirt VNC TLS with option of a dedicated CA https://review.openstack.org/536404 | 10:23 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Add support for libvirt VNC TLS with option of a dedicated CA https://review.openstack.org/536404 | 10:23 |
| openstackgerrit | Merged openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/544191 | 10:24 |
| chandankumar | karthiks: let's see recheck will fix the issue. | 10:24 |
| karthiks | thanks chandankumar | 10:24 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Add support for libvirt VNC TLS https://review.openstack.org/536469 | 10:24 |
| chandankumar | karthiks: File not found: /builddir/build/BUILDROOT/os-net-config-8.2.1-0.20180214102350.6e9eb68.el7.centos.x86_64/usr/bin/os-net-config-sriov | 10:25 |
| chandankumar | karthiks: we need to merge upstream patch first then i think we can merge rdo patch | 10:26 |
| *** ykarel_ has joined #tripleo | 10:26 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Inherit UpdateCommand class from overcloud DeployCommand class https://review.openstack.org/540798 | 10:26 |
| *** zoli is now known as zoli|lunch | 10:27 | |
| *** zoli|lunch is now known as zoli | 10:28 | |
| *** owalsh is now known as owalsh_afk | 10:28 | |
| karthiks | chandankumar, oh !!! looks like chicken-egg problem :) | 10:28 |
| *** ykarel has quit IRC | 10:29 | |
| bogdando | PTAL https://review.openstack.org/#/c/543024 | 10:30 |
| *** lucas-pto is now known as lucasagomes | 10:30 | |
| jistr | folks does it ever happen to you that with quickstart, the libvirt socket disappears? | 10:30 |
| jistr | mine was /run/user/1001/libvirt-sock | 10:31 |
| jistr | and it's not there and i have no idea how to bring it back :D | 10:31 |
| jistr | it should be autocreated, i found some bugzillas mentioning problems with this, but all are closed or don't really mention any workarounds | 10:31 |
| bogdando | jistr: did you try with priv mode? | 10:32 |
| *** anilvenkata has quit IRC | 10:32 | |
| *** anilvenkata has joined #tripleo | 10:32 | |
| jistr | bogdando: sorry i don't follow, what priv mode? if you mean container priv mode -- i'm talking about libvirt on BM host, it's not containerized | 10:33 |
| bogdando | jistr: https://review.openstack.org/#/c/540350/ | 10:33 |
| jistr | oh | 10:33 |
| bogdando | and https://review.openstack.org/#/c/539879/ | 10:33 |
| jistr | that could solve so many problems :D | 10:33 |
| jistr | thanks bogdando | 10:33 |
| bogdando | yay indeed | 10:33 |
| *** rfolco|off has quit IRC | 10:33 | |
| Tengu | jaosorior: oh, btw, I just found out why novajoin was failing on my lab: routing. because network ftw :) | 10:33 |
| jaosorior | lol | 10:34 |
| *** yamahata has quit IRC | 10:34 | |
| jistr | bogdando: i've had issues with qemu://session libvirt since forever, it's so flaky | 10:34 |
| bogdando | what's why I'm used to deploy on rdo cloud | 10:34 |
| bogdando | that's* | 10:34 |
| Tengu | jaosorior: as my DoubleNO³ mimic/spoofs the productive public network, I had to add "some" routes in order to let the lab access services that are hosted on the public range (but not in openstack), like freeIPA. Or our git repository :] | 10:34 |
| jaosorior | Tengu: remember that the overcloud nodes need a way to contact freeIPA as well | 10:35 |
| Tengu | jaosorior: yep, it's OK now - the lab-prenat instance has the right config, and I added specific routes in a "dedicated lab" env file. | 10:36 |
| Tengu | so now I should get a working lab. without any further hacks. | 10:37 |
| *** ykarel_ is now known as ykarel | 10:37 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Undercloud: fix stackrc TLS URL detection https://review.openstack.org/535480 | 10:37 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Undercloud: support for external VIP for SSL https://review.openstack.org/535479 | 10:38 |
| *** tosky has joined #tripleo | 10:44 | |
| karthiks | dsneddon, The patch https://review.openstack.org/#/c/527571/ and rdo patch https://review.rdoproject.org/r/#/c/12172/ has nested dependencies with respect to the entrypoint os-net-config-sriov. Can you please suggest a way forward ? | 10:45 |
| openstackgerrit | Merged openstack/tripleo-common master: Deprecate group:os-apply-config with config-download https://review.openstack.org/543976 | 10:50 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Updated tempest plugin names https://review.openstack.org/544242 | 10:50 |
| shardy | marios, chem: Hey, ramishra is investigating an issue with node delete, and I think it might be related to changes between versions where node delete implementation changed between e.g Newton and later releases | 10:51 |
| shardy | marios, chem: I was wondering if you guys are testing upgrades, particularly FFU from newton, if you could try doing a node delete, then upgrade, then do another node delete, and capture the openstack stack environment show (and a heat DB dump) at each stage | 10:51 |
| *** ykarel_ has joined #tripleo | 10:52 | |
| shardy | I can test it on master, but I was thinking you may have faster access to the upgrade-ready test environments? | 10:52 |
| chem | shardy: I don't have a working env right now, if that happen soon enought I'll try. | 10:53 |
| ramishra | shardy,chem: so the customer is on newton, so it's something after liberty->mitaka->newton upgrade | 10:54 |
| *** ykarel has quit IRC | 10:54 | |
| shardy | ramishra: Oh! | 10:54 |
| shardy | So it's expected that the ComputeRemovalPolicies is in parameters, but that may be a problem for subsequent versions | 10:55 |
| marios | shardy: o/ i am focusing on q upgrades these days so don't have an ffu env | 10:55 |
| marios | shardy: lbezdick might but i think he is away | 10:55 |
| shardy | ramishra: can you pls link the bz and we can decide if some upstream bug(s) are needed? | 10:55 |
| ramishra | https://bugzilla.redhat.com/show_bug.cgi?id=1544088 | 10:55 |
| openstack | ramishra: Error: Could not parse XML returned by bugzilla.redhat.com: HTTP Error 403: Forbidden (https://bugzilla.redhat.com/show_bug.cgi?id=1544088&ctype=xml) | 10:55 |
| marios | shardy: (might/very likey/i know he does/was running the workflow end of last week) | 10:55 |
| shardy | marios: ack thanks, we may need some collaboration between the upgrade and workflow squads to sort this out I think, my worry is that ComputeRemovalPolcies moves from parameters <= Newton to parameter_defaults in subsequent versions | 10:57 |
| shardy | but that may be an issue beyond the bug ramishra is looking into | 10:57 |
| marios | shardy: thanks for the context. fwiw the versions that matter here are exactly newton and queens (we don't use 'in-between' templates, all the upgrade stuff comes from q tht) | 10:58 |
| shardy | marios: ack, yeah it's not really specific to ffu, my worry is node delete is broken for all versions beyond Newton when upgrading | 10:59 |
| shardy | that would be surprising, hence wanting some other eyes/testing on it | 10:59 |
| *** rfolco has joined #tripleo | 11:04 | |
| *** rfolco is now known as rfolco|rover | 11:05 | |
| *** stendulker has joined #tripleo | 11:05 | |
| *** stendulker_ has quit IRC | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 11:10 |
| *** pdeore has quit IRC | 11:10 | |
| *** stendulker has quit IRC | 11:13 | |
| *** ansmith has quit IRC | 11:15 | |
| shardy | ramishra: AFAICT node delete doesn't work at all on master... | 11:18 |
| shardy | https://bugs.launchpad.net/tripleo/+bug/1749426 raised | 11:18 |
| openstack | Launchpad bug 1749426 in tripleo "Node delete is broken" [High,Triaged] | 11:18 |
| ramishra | shardy: no surprise:) | 11:18 |
| shardy | ramishra: I think we'll have to ask thrash|g0ne or rbrady to take a look and get that on the workflow squad backlog | 11:19 |
| shardy | perhaps they can also look at the node delete between versions issue, maybe in collaboration with lbezdick | 11:20 |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient master: Adds --all option to overcloud upgrade for running all playbooks https://review.openstack.org/543990 | 11:20 |
| ramishra | shardy: sure, as the immediate problem for the customer is resolved (though they can't use node delete, if some node has problem), I think we have time to investigate | 11:21 |
| *** akane_ has quit IRC | 11:26 | |
| *** pcaruana has quit IRC | 11:30 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: WIP: Take mixed upgrade into account for boostrap_subnode_minimal. https://review.openstack.org/544249 | 11:33 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: WIP During a minor update make sure docker service has live-restore option https://review.openstack.org/541827 | 11:34 |
| d0ugal | shardy: is there something I can help with re: workflow squad stuff? | 11:35 |
| *** udesale_ has joined #tripleo | 11:36 | |
| d0ugal | shardy: oh, the node delete bug. | 11:36 |
| *** ebarrera_ has joined #tripleo | 11:37 | |
| *** udesale has quit IRC | 11:38 | |
| *** udesale has joined #tripleo | 11:41 | |
| *** udesale_ has quit IRC | 11:41 | |
| *** moshele has quit IRC | 11:42 | |
| *** pcaruana has joined #tripleo | 11:42 | |
| shardy | d0ugal: Hey! Yeah the issue is there are some issues with node delete | 11:43 |
| shardy | d0ugal: it is broken on master AFAICS, also the heat environment is overwritten every node delete, and it's possible we have an issue where Newton and older used "parameters" but later versions switched to the workflow which I think uses parameter_defaults | 11:44 |
| shardy | essentially we need some workflow folks to give node delete some love and help figure out what is broken ;) | 11:45 |
| *** ebarrera_ has quit IRC | 11:45 | |
| d0ugal | shardy: the workflow squad meeting is today at 3pm - you may want to join, I added it to the agenda: https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 11:45 |
| *** sshnaidm|ruck has quit IRC | 11:46 | |
| d0ugal | shardy: I wont make it today unfortunately, normally I run it but rbrady will be handling it today | 11:46 |
| *** udesale has quit IRC | 11:47 | |
| d0ugal | shardy: sounds good - I can help tomorrow if you don't get any support before then. | 11:48 |
| d0ugal | I have never actually used it, so I'll have some learning to do :) | 11:48 |
| shardy | d0ugal: ack, OK I also raised https://bugs.launchpad.net/tripleo/+bug/1749435 for another issue I noticed when helping ramishra debug | 11:48 |
| openstack | Launchpad bug 1749435 in tripleo "Node delete always overwrites *RemovalPolicies" [Medium,Triaged] | 11:48 |
| shardy | d0ugal: heh, yeah that seems to be part of the problem, it's not widely used and there's no CI coverage | 11:49 |
| shardy | maybe we can look at adding some | 11:49 |
| d0ugal | Yeah, good idea | 11:49 |
| shardy | d0ugal: I suspect there may be a third bug related to doing a node delete on Newton or before, then upgrading to a more recent version, but we need some more testing to confirm that | 11:49 |
| shardy | obviously making node delete work at all on master is the first step :) | 11:50 |
| *** salmankhan has quit IRC | 11:50 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Keep a default value for NODEPOOL_CENTOS_MIRROR. https://review.openstack.org/544255 | 11:50 |
| moguimar | is it possible to run TripleO Quickstart virtualy, with something like docker or vagrant? | 11:51 |
| moguimar | I'm trying to run it on vagrant, ssh to root without password is already figured out | 11:51 |
| moguimar | but I can't get it to connect to the right port for ssh | 11:52 |
| shardy | moguimar: Some of the roles could work in such an environment, but one of the main reasons -quickstart exists is to setup VMs via libvirt | 11:52 |
| *** akane has joined #tripleo | 11:52 | |
| *** dsariel has quit IRC | 11:52 | |
| d0ugal | moguimar: if you are using virtualbox with vagrant I don't think it supports nested virt | 11:52 |
| d0ugal | or it didn't the last time I looked (a few years ago!) | 11:52 |
| *** masco has quit IRC | 11:53 | |
| *** dsariel_ has quit IRC | 11:53 | |
| moguimar | oh geez | 11:53 |
| moguimar | how could I try tripleO without messing arround my machine and not having access to other machines? | 11:54 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/python-tripleoclient master: Prepare t-h-t for undercloud in a work dir https://review.openstack.org/542875 | 11:55 |
| shardy | moguimar: do you have any access to existing OpenStack clouds? Some folks do test via quickstart on e.g RDOcloud | 11:55 |
| moguimar | RDOclould new accounts are on hold | 11:56 |
| moguimar | =T | 11:56 |
| shardy | personally I run it on a local development machine, nearly all changes are inside VMs so it doesn't do much to the host | 11:56 |
| *** salmankhan has joined #tripleo | 11:56 | |
| moguimar | I have also tried devstack, without success | 11:56 |
| moguimar | shardy: I think I'll have to go that same way | 11:57 |
| *** raildo has joined #tripleo | 11:57 | |
| moguimar | would you have any directions? | 11:57 |
| d0ugal | Yeah, I use RDOCloud and have a local dev box | 11:57 |
| shardy | devstack is unrelated to tripleo, but we do have an experimental tripleo feature which enables an all-in-one deployment | 11:58 |
| shardy | that could work in a VM, I've tested it in a libvirt VM before via https://github.com/dprince/undercloud_containers | 11:59 |
| jaosorior | chem: tested out your patch. It didn't fix the issue :() | 11:59 |
| jaosorior | :( | 11:59 |
| chem | jaosorior: same error ? | 12:00 |
| jaosorior | yep | 12:00 |
| moguimar | thanks guys | 12:01 |
| jaosorior | shardy: fyi https://bugs.launchpad.net/tripleo/+bug/1749338 | 12:01 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 12:01 |
| chem | jaosorior: marios in mtg currently | 12:03 |
| *** akane has quit IRC | 12:05 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add OVNController service to Networker role https://review.openstack.org/537926 | 12:06 |
| *** dsariel has joined #tripleo | 12:07 | |
| *** cshastri_ has joined #tripleo | 12:08 | |
| *** tcw has joined #tripleo | 12:08 | |
| *** dsariel_ has joined #tripleo | 12:08 | |
| *** tcw1 has joined #tripleo | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 12:10 |
| *** assassin has joined #tripleo | 12:11 | |
| shardy | jaosorior: We shouldn't need to use string blocks anywhere, we yaml dump in the config dowload code | 12:11 |
| shardy | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/utils/config.py#L196 | 12:11 |
| *** cshastri has quit IRC | 12:11 | |
| jaosorior | shardy: actually I think the yaml dump is the issue here | 12:12 |
| jaosorior | shardy: check my last comment in https://bugs.launchpad.net/tripleo/+bug/1749338 (I just made it) | 12:12 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 12:12 |
| shardy | jaosorior: is the problem that we're consuming FastForwardUpgradeReleases without converting it to json | 12:12 |
| shardy | e.g via str_replace? | 12:12 |
| *** tcw has quit IRC | 12:12 | |
| shardy | that may not be valid yaml syntax? | 12:12 |
| jaosorior | shardy: no, the problem is that that specific function relies on the order of the variables. | 12:12 |
| jaosorior | when it gets persisted, they're out of order | 12:13 |
| jaosorior | and so it fails | 12:13 |
| jaosorior | that's my current theory | 12:13 |
| jaosorior | I tried removing the query variable and setting the query in the cmd_args declaration. Hopefully that works. | 12:13 |
| * Tengu wonders if the fact novajoin can't be reached should really trash the tripleo deploy completely | 12:14 | |
| shardy | jaosorior: sorry which variables are out of order? | 12:14 |
| jaosorior | shardy: cmd_args and query | 12:14 |
| jaosorior | which were introduced here https://review.openstack.org/#/c/515429/26/puppet/services/tripleo-packages.yaml | 12:15 |
| shardy | https://review.openstack.org/#/c/544229/1/common/deploy-steps.j2 doesn't touch those, looking at the other patches | 12:15 |
| jaosorior | shardy: correct, which is why that patch doesn't fix the issue | 12:15 |
| jaosorior | shardy: sorry to bring it out without proper context | 12:16 |
| shardy | jaosorior: Ah I see so the j2 reference to query may not work due to the ordering, got it | 12:16 |
| jaosorior | shardy: so, the IPSec deployment started failing yesterday | 12:16 |
| *** gkadam has joined #tripleo | 12:17 | |
| jaosorior | and we tracked it down to a couple of patches from the ffu series | 12:17 |
| shardy | jaosorior: Ok, sounds like you're on it but let me know if I can help | 12:18 |
| jaosorior | shardy: thanks. | 12:18 |
| jaosorior | shardy: caught the variable issue about a minute after I poked you | 12:18 |
| *** gkadam__ has quit IRC | 12:19 | |
| *** sshnaidm|ruck has joined #tripleo | 12:20 | |
| *** assassin has quit IRC | 12:20 | |
| *** assassin has joined #tripleo | 12:20 | |
| *** dtantsur|afk is now known as dtantsur | 12:21 | |
| *** bfournie has quit IRC | 12:24 | |
| *** bfournie has joined #tripleo | 12:24 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-docs master: Update manila deployment doc https://review.openstack.org/544077 | 12:25 |
| *** bfournie has quit IRC | 12:29 | |
| shardy | Anyone know why the UI validation always runs, even if validate_ui_simple: false is set (which seems like it should be the default..)? | 12:30 |
| * shardy adds debug | 12:30 | |
| *** moshele has joined #tripleo | 12:30 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-docs master: Documentation - TripleO routed-spine-and-leaf https://review.openstack.org/539939 | 12:31 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-quickstart master: Refactor repo_cmd_before https://review.openstack.org/539525 | 12:31 |
| holser | sshnaidm|ruck - I have addressed your concerns | 12:31 |
| dalvarez | hi folks, has anyone seen this failure during the overcloud deployment? http://paste.openstack.org/show/672079/ | 12:32 |
| dalvarez | "Error: /Stage[main]/Apache::Service/Service[httpd]: Could not evaluate: Puppet::Util::Log requires a message", | 12:32 |
| *** nyechiel_ has quit IRC | 12:36 | |
| *** akane has joined #tripleo | 12:36 | |
| *** gkadam has quit IRC | 12:38 | |
| sshnaidm|ruck | holser, ?? | 12:39 |
| *** dsariel has quit IRC | 12:39 | |
| holser | I am talking about https://review.openstack.org/539525 | 12:39 |
| *** moshele has quit IRC | 12:40 | |
| *** dsariel_ has quit IRC | 12:40 | |
| chem | jaosorior: so you're on something or is the revert still needed ? I have to do another task from my mtg before switching back to that to help | 12:42 |
| *** moshele has joined #tripleo | 12:43 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Add Featureset 39 -- HA deploy w/ IPA https://review.openstack.org/533785 | 12:43 |
| jaosorior | chem: testing out a potential fix | 12:44 |
| chem | jaosorior: ack cool. | 12:44 |
| *** tosky has quit IRC | 12:47 | |
| *** udesale has joined #tripleo | 12:51 | |
| *** jpena is now known as jpena|lunch | 12:51 | |
| jtomasek | shardy: hi, network config guestion: What is the difference between e.g. ExternalIpSubnet and ExternalNetCidr ? ExternalINetCidr is used in network.j2 and is populated by ip_subnet in network_data.yaml, ExternalIpSubnet is used in role.role.j2.yaml and is empty by default (not populated by network_data.yaml) | 12:52 |
| *** dsariel has joined #tripleo | 12:52 | |
| *** abishop has joined #tripleo | 12:53 | |
| *** dsariel_ has joined #tripleo | 12:53 | |
| *** abishop has quit IRC | 12:54 | |
| *** abishop has joined #tripleo | 12:55 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Pass the queue_name to messaging_websocket function for update https://review.openstack.org/544298 | 12:56 |
| *** pradk has joined #tripleo | 12:56 | |
| *** thrash|g0ne is now known as thrash | 12:57 | |
| *** atoth has joined #tripleo | 12:57 | |
| jtomasek | tbarron: ^^maybe you know the answer too? | 12:59 |
| *** weshay_PTO is now known as weshay | 13:01 | |
| oidgar | hi, we have a small patch for tripleo-common for about 2 weeks: https://review.openstack.org/#/c/539245/ | 13:01 |
| oidgar | If someone can review this I'll be appreciate | 13:02 |
| oidgar | *thankful | 13:02 |
| *** ykarel__ has joined #tripleo | 13:03 | |
| *** bfournie has joined #tripleo | 13:05 | |
| *** jcoufal has joined #tripleo | 13:05 | |
| *** dpawar has quit IRC | 13:05 | |
| *** ykarel_ has quit IRC | 13:05 | |
| *** cylopez1 has joined #tripleo | 13:05 | |
| *** cdearborn has joined #tripleo | 13:05 | |
| *** cylopez has quit IRC | 13:06 | |
| *** tosky has joined #tripleo | 13:06 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Handle failed nodes when enabling ssh admin https://review.openstack.org/539245 | 13:06 |
| *** mcornea has joined #tripleo | 13:08 | |
| *** pkovar has joined #tripleo | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 13:10 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| *** noslzzp has joined #tripleo | 13:11 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 13:12 |
| Tengu | hmmm. weird. when I want to deploy the ceph-storage node, I don't find any trace of "ceph-storage" in the nova-scheduler log - although I actually DO traces of compute-* nodes and controller-* node… | 13:14 |
| jaosorior | chem: where does fast_forward_repo_args come from ? | 13:14 |
| *** nyechiel_ has joined #tripleo | 13:16 | |
| chem | jaosorior: looking | 13:16 |
| tbarron | jtomasek: good question, before https://review.openstack.org/#/c/523638 I see both, with the *NetCidr parameter_defaults in environment files and *IpSubnet in network/config/.. files | 13:17 |
| chem | jaosorior: http://codesearch.openstack.org/?q=fast_forward_repo_args&i=nope&files=&repos= | 13:17 |
| tbarron | jtomasek: I think https://review.openstack.org/#/c/523638/ just tries to maintain backwards compatability | 13:17 |
| *** apevec has joined #tripleo | 13:17 | |
| *** eck`gone is now known as eck` | 13:17 | |
| *** jruzicka has joined #tripleo | 13:17 | |
| jaosorior | funky | 13:17 |
| jaosorior | ok | 13:17 |
| *** ebarrera has quit IRC | 13:18 | |
| *** ebarrera has joined #tripleo | 13:18 | |
| chem | jaosorior: by default it's a json hash that describe how to switch repos | 13:18 |
| chem | jaosorior: is that what is causing the issue ? | 13:18 |
| jaosorior | chem: not sure yet. tried my fix and got a failure related to that not being defined | 13:19 |
| jtomasek | tbarron: so those 2 parameters are basically the same value? | 13:19 |
| *** sshnaidm has joined #tripleo | 13:19 | |
| *** agurenko has quit IRC | 13:20 | |
| tbarron | jtomasek: so far as I can tell, we better ask someone who knows more that I do, dsneddon or bnemec maybe? | 13:20 |
| *** oidgar has quit IRC | 13:21 | |
| *** sshnaidm|ruck has quit IRC | 13:22 | |
| chem | jaosorior: strange as it has a default value there https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/tripleo-packages.yaml#L40-L41 | 13:22 |
| *** dpawar has joined #tripleo | 13:22 | |
| *** agurenko has joined #tripleo | 13:23 | |
| jtomasek | tbarron: cool, I couldn't see *IpSubnet parameter set in any environment files examples. If those 2 parameters are the same thing, then 1. role.role.js.yaml should populate IpSubnet default value with ip_subnet from network_data.yaml or 2. *IpSubnet should be removed and replaced with *NetCidr in nic configs (role.role.j2.yaml) | 13:23 |
| *** cdearborn has quit IRC | 13:23 | |
| *** trown|outtypewww is now known as trown | 13:23 | |
| *** tosky has quit IRC | 13:23 | |
| tbarron | jtomasek: no argument from me, I'm just learning this stuff (and am a consumer of the template generation in https://review.openstack.org/#/c/523638/) | 13:25 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Set tripleo-upgrade vars in featureset. https://review.openstack.org/539910 | 13:27 |
| *** amoralej is now known as amoralej|lunch | 13:30 | |
| mwhahaha | rbrady, thrash: not sure if you've seen https://bugs.launchpad.net/tripleo/+bug/1749426 but node delete workflow might be broken | 13:31 |
| openstack | Launchpad bug 1749426 in tripleo "Node delete is broken" [High,Triaged] | 13:31 |
| *** rlandy has joined #tripleo | 13:31 | |
| d0ugal | mwhahaha: I spotted it, it is on the agenda for the squad meeting in 1.5 hours | 13:32 |
| mwhahaha | d0ugal: thanks | 13:32 |
| thrash | mwhahaha: ack. | 13:32 |
| d0ugal | mwhahaha: if nobody beats me to it, I plan to take a look tomorrow | 13:32 |
| rbrady | mwhahaha: saw the discussion with d0ugal and shardy, started looking | 13:32 |
| d0ugal | /query thrash | 13:32 |
| d0ugal | oops | 13:32 |
| thrash | lol | 13:32 |
| *** pradk has quit IRC | 13:36 | |
| *** suuuper has joined #tripleo | 13:38 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-quickstart master: Refactor repo_cmd_before https://review.openstack.org/539525 | 13:38 |
| *** lblanchard has joined #tripleo | 13:40 | |
| *** tosky has joined #tripleo | 13:40 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/instack-undercloud master: Update reno for stable/queens https://review.openstack.org/544314 | 13:41 |
| openstackgerrit | OpenStack Release Bot proposed openstack/os-net-config master: Update reno for stable/queens https://review.openstack.org/544331 | 13:43 |
| openstackgerrit | OpenStack Release Bot proposed openstack/paunch master: Update reno for stable/queens https://review.openstack.org/544337 | 13:44 |
| *** suuuper has quit IRC | 13:45 | |
| *** pchavva has joined #tripleo | 13:45 | |
| *** suuuper has joined #tripleo | 13:45 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Add featureset045: master to master update https://review.openstack.org/542302 | 13:50 |
| *** liverpooler has joined #tripleo | 13:50 | |
| *** arxcruz|sick is now known as arxcruz | 13:50 | |
| *** jpena|lunch is now known as jpena | 13:50 | |
| *** owalsh_afk is now known as owalsh | 13:52 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Use overcloud_templates for deployed server prepare scripts. https://review.openstack.org/535299 | 13:52 |
| *** cshastri_ has quit IRC | 13:53 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add a hook to run a bash script at the end of the overcloud prep. https://review.openstack.org/533167 | 13:55 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: remove duplicate README file (part deux) https://review.openstack.org/544344 | 13:55 |
| *** chlong has quit IRC | 14:01 | |
| jaosorior | chem: ok, I'm not sure what to do now :/ | 14:02 |
| jaosorior | I tried with and without your patch trying to define the query elsewhere http://paste.openstack.org/show/672207/ | 14:03 |
| jaosorior | and I get this fatal: [192.168.24.8]: FAILED! => {"msg": "Error in jmespath.search in json_query filter plugin:\n'tripleo_repos_cmd_args_query' is undefined"} | 14:03 |
| chem | jaosorior: looking | 14:03 |
| slagle | i saw the same error in failed CI on one of my patches | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 14:10 |
| slagle | wasn't sure where it was coming from | 14:10 |
| jaosorior | slagle: tracking it here https://bugs.launchpad.net/tripleo/+bug/1749338 | 14:11 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 14:11 |
| slagle | jaosorior: k. i saw the error here on this patch: https://review.openstack.org/#/c/539315/ | 14:12 |
| slagle | which is not related to ffu, but i guess somehow ansible is getting tripped up | 14:12 |
| slagle | jaosorior: i think it might be when you try and access hostvars | 14:12 |
| slagle | b/c i'm doing that in my patch | 14:13 |
| bnemec | tbarron: jtomasek: The *IpSubnet parameters are passed in based on what neutron assigns the node. They don't need to be set in an environment file. | 14:13 |
| jaosorior | slagle: correct | 14:13 |
| jaosorior | slagle: that's exactly what triggers the issue | 14:13 |
| slagle | k | 14:13 |
| jaosorior | slagle: why do we store the tasks in the inventory? | 14:13 |
| jaosorior | slagle: seems to me that what's in common/deploy-tasks.j2 ends up in the ansible inventory. which is quite sensitive regarding quoting and such | 14:14 |
| slagle | jaosorior: shardy did that, i don't recall the exact reason | 14:14 |
| jaosorior | which is similar to the issues I was initially having with the IPSec deployment | 14:14 |
| *** cshastri has joined #tripleo | 14:14 | |
| jtomasek | bnemec: I see, so these parameters are set during deployment? they should not be set at all? | 14:14 |
| slagle | jaosorior: some of the task files i noticed we can actually get rid of since they are now in the inventory as well | 14:15 |
| jaosorior | slagle: like which? | 14:16 |
| bnemec | jtomasek: Right, those are dynamic values, calculated at deploy time. | 14:16 |
| chem | jaosorior: oki, so the root cause is that order within a task is not preserved during the dump | 14:16 |
| *** oidgar has joined #tripleo | 14:16 | |
| slagle | jaosorior: well, things like Controller/docker_config.yaml appears entirely redundant now | 14:17 |
| jtomasek | bnemec: thanks, that's very helpful | 14:17 |
| jaosorior | slagle: what's the advantage of having them in the inventory and not just relying on include_task: relevant_file.yaml ? | 14:17 |
| slagle | jaosorior: as well as puppet_config.yaml | 14:18 |
| slagle | jaosorior: shardy had a reason :). let me dig up the patch and review the commit message. | 14:19 |
| slagle | jaosorior: also it helped with our quoting problems | 14:19 |
| slagle | iniitially anyway | 14:19 |
| *** jmelvin has joined #tripleo | 14:19 | |
| *** zoli is now known as zoli|afk | 14:19 | |
| *** zoli|afk is now known as zoli | 14:19 | |
| jaosorior | slagle: oh, I think I had initially understood your statement backwards then | 14:19 |
| *** Goneri has joined #tripleo | 14:20 | |
| tbarron | bnemec: jtomasek: thanks, makes sense. | 14:20 |
| jaosorior | slagle: so yeah, I remember shardy moving the stuff from deploy-tasks.j2 to deploy-steps-tasks.yaml helped with the quoting issue | 14:20 |
| jaosorior | if we can move more files that would be great | 14:20 |
| fungi | EmilienM: looking back over the snmp analysis notes, the ones which were consistently returning data in the "3nodes" jobs were the "secondary-2" nodes in the inventory (as opposed to "primary" or "secondary-1") so i think that makes them the third node in the set. maybe we were looking at the wrong config last night? | 14:21 |
| *** lucasagomes is now known as lucas-hungry | 14:21 | |
| fungi | i'm logged into one early in the lifecycle of one of those jobs now, waiting to see snmpd show up in the process list | 14:22 |
| slagle | jaosorior: i think that could be done | 14:22 |
| chem | jaosorior: marios for what it's worth as "vars" in ansible are hashes instead of list, I think that we shouldn't rely on the order so we should fix it inside that task | 14:22 |
| jaosorior | chem: that's what I tried, and for some reason am still having an issue. Did you see the diff I posted? | 14:23 |
| *** paramite_ has joined #tripleo | 14:23 | |
| shardy | slagle, jaosorior: I added all of RoleData to the inventory so I could write out the various json files we need for containers in a block at the start of step1 | 14:23 |
| *** nyechiel_ has quit IRC | 14:23 | |
| shardy | vs relying on a heat driven ansible step to do that outside of deploy_steps_tasks | 14:24 |
| *** akane has quit IRC | 14:24 | |
| shardy | and yeah, it makes the quoting easier | 14:24 |
| slagle | right, yea that | 14:24 |
| chem | jaosorior: yeah, trying to understand why that one has failed as well, but this is the right path and we should remove the inter-var order dependency | 14:24 |
| *** mdnadeem has quit IRC | 14:24 | |
| shardy | slagle, jaosorior: I didn't necessarily need to put the *tasks into the inventory, that was just a side effect of adding all role_data_* I think? | 14:24 |
| chem | marios: ^ | 14:25 |
| slagle | shardy: i think we need to clean up one or the other, b/c it's confusing as is | 14:25 |
| shardy | I agree the includes in the config download structure make sense, that was kinda what I was trying to iterate towards, e.g incrementally reducing our reliance on the shim that runs heat deployments via ansible | 14:25 |
| jaosorior | slagle, shardy: I'm just trying to understand why quoting issues get reflected when we try to access hostvars | 14:25 |
| *** cdearborn has joined #tripleo | 14:25 | |
| *** ykarel__ is now known as ykarel | 14:26 | |
| slagle | i was doing a bunch of stack-updates and config downloads last night trying to figure out why nothing i was updating was taking effect. it was b/c i wasn't regenerating the invneotry | 14:26 |
| slagle | which you now have to do after every stack-update | 14:26 |
| shardy | slagle: my original aim was to remove https://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps.j2#L245..L274 | 14:26 |
| shardy | since we can do the host_prep_tasks via includes so that isn't needed anymore in the --config-download case | 14:27 |
| *** nyechiel_ has joined #tripleo | 14:27 | |
| shardy | but then we decided against moving to config download by default, and I didn't manage to reach agreement to move docker-puppet.py into tripleo-common | 14:27 |
| shardy | so the refactoring kinda stalled | 14:27 |
| shardy | slagle: happy to help revive the work as I agree there's still more to do | 14:27 |
| shardy | slagle: can't you just use the dynamic inventory? | 14:28 |
| slagle | we should probably at least capture somewhere what you intended to do as remaining work | 14:28 |
| beagles | we are deliberately *not* using keepalived in a container, correct? | 14:28 |
| *** udesale has quit IRC | 14:29 | |
| slagle | shardy: yes of course. but I wasn't | 14:29 |
| *** udesale has joined #tripleo | 14:29 | |
| * beagles trys to think who that would be best directed at.. gfidente, jaosorior, ^ | 14:29 | |
| jaosorior | beagles: where what? | 14:29 |
| shardy | slagle: ack, I wonder if we can deprecate the static version as it'd be nice to have just one inventory (even though it's now pretty much the same code) | 14:30 |
| slagle | shardy: i think what's important is to document how people should interact with config-download. e.g., https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ansible_config_download.html | 14:30 |
| beagles | jaosorior, I just noticed that while we deploy haproxy in a container, we are not doing so for keeplived | 14:30 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/python-tripleoclient master: Autoeval container images for legacy services https://review.openstack.org/544357 | 14:30 |
| jaosorior | beagles: I thought we were | 14:30 |
| beagles | jaosorior, was wondering if that was deliberate or not | 14:30 |
| jaosorior | beagles: dunno :/ | 14:30 |
| shardy | slagle: agreed, I've always used -i /usr/bin/tripleo-ansible-inventory so perhaps it was a mistake to assume everyone did that | 14:30 |
| beagles | shardy, slagle, thoughts? | 14:31 |
| jaosorior | shardy: I did that as well, we started having the issues in CI, so I started uding --config-download since that's what gets tested | 14:31 |
| *** amoralej|lunch is now known as amoralej | 14:31 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: Support separate oslo.messaging services for RPC and Notification https://review.openstack.org/507963 | 14:32 |
| shardy | beagles: https://github.com/openstack/tripleo-heat-templates/commit/1e0056751fb22585cd981ccc02ad0e15d4cd46e6 has keepalived containerized? | 14:33 |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient master: Adds --all option to overcloud upgrade for running all playbooks https://review.openstack.org/543990 | 14:33 |
| *** links has quit IRC | 14:33 | |
| shardy | beagles: it could be we're missing some resource_registry mapping somewhere tho, I know this is not well tested compared to pacemaker | 14:33 |
| beagles | shardy, there is a container def for it ,but we aren't modifying the registry to load afaict | 14:34 |
| jaosorior | beagles: it isn't in environments/docker-ha.yaml? | 14:34 |
| shardy | beagles: I think that's because most overcloud testing enables pacemaker | 14:34 |
| beagles | don't think so.. unless it's been updated recently | 14:34 |
| shardy | docker-ha enables pacemaker, which is the default downstream, so keepalived is not needed in that case | 14:35 |
| shardy | it's only used for the containerized undercloud I think | 14:35 |
| jaosorior | chem: uhm... at this point I've spent all day working on this and can't come up with a fix, have you gave it a look? | 14:35 |
| *** leanderthal has joined #tripleo | 14:35 | |
| shardy | we could add an environment to enable keepalived, but then we'd have to test it in CI | 14:35 |
| chem | jaosorior: I have one last review for you before we go the revert way | 14:36 |
| chem | jaosorior: coming with 5min | 14:36 |
| jaosorior | thanks | 14:36 |
| chem | within | 14:36 |
| beagles | shardy, ah right (re: pacemaker) | 14:36 |
| *** udesale has quit IRC | 14:36 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Do not depends on the order of the hash vars in ffu repo switching. https://review.openstack.org/544229 | 14:37 |
| shardy | beagles: we could add it to environments/docker.yaml, then I guess it'd get some CI coverage | 14:37 |
| beagles | shardy, I don't particularly need it, I was just looking at options for the neutron agents cases and noticed that the keepalived container existed but wasn't being used | 14:37 |
| shardy | I do wonder if we should just enable pacemaker by default though to reduce the test matrix | 14:38 |
| beagles | shardy, and didn't want to *not* say something in case it was an oversight :) | 14:38 |
| shardy | beagles: ack, yeah it may be | 14:38 |
| shardy | thanks | 14:38 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/python-tripleoclient master: Autoeval container images for legacy services https://review.openstack.org/544357 | 14:39 |
| chem | jaosorior: marios https://review.openstack.org/#/c/544229/2/puppet/services/tripleo-packages.yaml | 14:39 |
| beagles | on a related note - the only way 'image prepare' will do its thing if a.) there is a docker service file for the container and b.) it configured for that deployment, correct? | 14:39 |
| jaosorior | chem: thanks, gonna try it out | 14:40 |
| beagles | I'm trying to sort out what it would take to get container images in place for dnsmasq etc, for neutron to do it's thing with, but they won't actually be started by the deployment - I guess a docker service file that doesn't actually start the service? | 14:41 |
| *** jaganathan has joined #tripleo | 14:42 | |
| *** suuuper has quit IRC | 14:43 | |
| *** suuuper has joined #tripleo | 14:43 | |
| *** bogdando has quit IRC | 14:43 | |
| *** jcoufal_ has joined #tripleo | 14:44 | |
| *** waleedm has quit IRC | 14:45 | |
| *** jcoufal__ has joined #tripleo | 14:46 | |
| *** jcoufal has quit IRC | 14:47 | |
| *** jcoufal_ has quit IRC | 14:49 | |
| EmilienM | hi | 14:50 |
| EmilienM | jpich: thanks for the feedback! | 14:51 |
| jpich | :) | 14:51 |
| EmilienM | fungi: sure, let me know if you have more details | 14:51 |
| *** jobewan has joined #tripleo | 14:52 | |
| *** yamahata has joined #tripleo | 14:53 | |
| EmilienM | shardy: have you ever deployed an overcloud with a containerized undercloud? I've been doing some testing since a week and blocker after blocker making little progress, so I just wonder if someone made it working before | 14:55 |
| *** jcoufal has joined #tripleo | 14:56 | |
| *** jcoufal__ has quit IRC | 14:57 | |
| EmilienM | sshnaidm: you missed fs042 (ipsec job), which doesn't work (in https://review.rdoproject.org/r/#/c/12187) | 14:57 |
| chem | jistr: you may want to sync with panda as there is a new featureset045 that's driving testing of update of the overcloud. | 14:58 |
| openstackgerrit | Sandhya Dasu proposed openstack/tripleo-heat-templates stable/pike: Adding new config parameters for Cisco UCSM ML2 driver https://review.openstack.org/544367 | 14:58 |
| *** akane has joined #tripleo | 14:58 | |
| jistr | panda, chem: why? :D the etherpad we have says it's 037 | 14:59 |
| jistr | https://review.openstack.org/#/c/542776/ | 14:59 |
| shardy | weshay, sshnaidm: Hey FYI I just raised https://bugs.launchpad.net/tripleo/+bug/1749477 - let me know if there's anything else I can check before I kill the async tasks and continue with other testing | 14:59 |
| openstack | Launchpad bug 1749477 in tripleo "Quickstart installed undercloud async tasks broken" [Medium,Triaged] | 14:59 |
| jistr | panda: https://etherpad.openstack.org/p/quickstart-featuresets | 14:59 |
| openstackgerrit | Sandhya Dasu proposed openstack/tripleo-heat-templates stable/ocata: Adding new config parameters for Cisco UCSM ML2 driver https://review.openstack.org/544368 | 14:59 |
| *** akane has quit IRC | 15:00 | |
| jistr | panda: please can you have a look at https://review.openstack.org/#/q/topic:updates-ci ? | 15:00 |
| rbrady | workflows squad meeting: apetrich,thrash,toure,jtomasek, https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 15:00 |
| *** akane_ has joined #tripleo | 15:00 | |
| jaosorior | chem: that did the trick so far | 15:01 |
| jtomasek | rbrady: too many meetings right now, sorry | 15:02 |
| jaosorior | chem: it did the trick! | 15:02 |
| jaosorior | chem: awwww yeah | 15:02 |
| *** agopi_out has joined #tripleo | 15:02 | |
| *** moshele has quit IRC | 15:02 | |
| *** rleander_ has joined #tripleo | 15:03 | |
| *** moshele has joined #tripleo | 15:03 | |
| *** leanderthal has quit IRC | 15:03 | |
| mcornea | dtantsur: hi o/ I'm seeing an issue with the pike to queens undercloud upgrade caused by ironic dbsync https://bugs.launchpad.net/tripleo/+bug/1749476 could you please advise what can I do to move forward when you have some cycles? | 15:03 |
| openstack | Launchpad bug 1749476 in tripleo "Pike-> Queens undercloud upgrade fails during ironic-dbsync The database is not compatible with this release of ironic (10.1.1.dev18). Please run "ironic-dbsync online_data_migrations" using the previous release." [High,Triaged] | 15:03 |
| chem | jaosorior: ah cool. So let's merge this as it. We will check later that it's still working for ffu. | 15:03 |
| *** rleander_ is now known as leanderthal | 15:03 | |
| chem | jaosorior: so that you (and other) don't randomly bump into this anymore. | 15:03 |
| sshnaidm | EmilienM, what doesn't work with it..? | 15:03 |
| *** sshnaidm is now known as sshnaidm|ruck | 15:04 | |
| dtantsur | mcornea: "Please run "ironic-dbsync online_data_migrations" using the previous release." :) | 15:04 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: WIP: Add docs for Q upgrade workflow https://review.openstack.org/535859 | 15:05 |
| dtantsur | mcornea: it may be confusing at first, but some services (nova and ironic including) have two stages of database migration | 15:05 |
| chem | marios: could you look at https://review.openstack.org/#/c/544229/ and push it. It solves the issue. We will have to test that that oneliner is still working after. | 15:05 |
| mcornea | dtantsur: I tried reverting the ironic packages to the previous release version ran online_data_migrations then retried the undercloud upgrade but got the same result | 15:05 |
| *** salmankhan has quit IRC | 15:05 | |
| -openstackstatus- NOTICE: Due to a race in stable/queens branch creation and some job removals, Zuul has reported syntax errors for the past hour; if you saw a syntax error reported for "Job tripleo-ci-centos-7-ovb-containers-oooq not defined" you can safely recheck now | 15:06 | |
| chem | jaosorior: could you paste me somewhere the compiled ansible for the fast_forward_upgrade_tasks ? | 15:07 |
| marios | chem: ack done +A it when ci is done? | 15:07 |
| chem | marios: yeah | 15:07 |
| chem | yes | 15:07 |
| chem | yo | 15:07 |
| marios | chem: and i actually just got it (you mean the ordering of the task vars... :/ | 15:08 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 15:10 |
| openstackgerrit | Ade Lee proposed openstack/tripleo-heat-templates master: Add dns info for hosts and vips to metadata https://review.openstack.org/544052 | 15:10 |
| dtantsur | mcornea: what exactly are you doing? if you just installed pike, you should have the database ready, I think.. | 15:10 |
| mcornea | dtantsur: the initial deployment is newton and I'm doing upgrade of newton->ocata->pike->queens; note that this issue is recently introduced, it used to work 2 days ago | 15:12 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Add workflows-based role listing commands https://review.openstack.org/526038 | 15:12 |
| *** gkadam has joined #tripleo | 15:12 | |
| *** moshele has quit IRC | 15:12 | |
| dtantsur | mcornea: we did not merge anything to pike, I think.. | 15:12 |
| dtantsur | mcornea: oh, one guess: are you sure you're using ironic *QUEENS*, not Rocky? | 15:13 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: WIP: Add docs for Q upgrade workflow https://review.openstack.org/535859 | 15:13 |
| dtantsur | mcornea: like, taken from stable/queens, not from master | 15:13 |
| mcornea | dtantsur: hmm, I think that might be it, I'm using the current repos provided by tripleo-repos | 15:14 |
| dtantsur | mcornea: then what you do is upgrade Pike -> Rocky. and indeed, a day or so ago we landed a change making it impossible. | 15:14 |
| mcornea | dtantsur: ok, that makes sense. I'll try with the queens repos. thanks for the help! | 15:14 |
| *** salmankhan has joined #tripleo | 15:14 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Add pacemaker upgrade_tasks for P..Q major upgrade https://review.openstack.org/539249 | 15:15 |
| *** chlong has joined #tripleo | 15:16 | |
| jaosorior | EmilienM: testing locally, this seems to fix the config-download issue we were discussing in my morning/your evening https://review.openstack.org/#/c/544229/ | 15:16 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Revert "Default to docker.io instead of trunk.registry.rdoproject.org" https://review.openstack.org/544378 | 15:20 |
| *** cshastri has quit IRC | 15:20 | |
| sshnaidm|ruck | ykarel, amoralej apevec ^^ | 15:21 |
| apevec | uhm why was this changed for promoation jobs? | 15:22 |
| apevec | ah it was single config for both | 15:22 |
| sshnaidm|ruck | apevec, it's settings for all rdo cloud jobs | 15:22 |
| mwhahaha | cause we don't have good seperation of configs for environments | 15:22 |
| sshnaidm|ruck | apevec, yeah, we need to find a way to separate it | 15:23 |
| mcornea | dtantsur: checking the repos I see I have openstack-ironic-10.1.1 for both queens and master: https://trunk.rdoproject.org/centos7-queens/current/ and https://trunk.rdoproject.org/centos7/current/ | 15:23 |
| sshnaidm|ruck | mwhahaha, actually there is separate config for environment | 15:23 |
| mcornea | dtantsur: is 10.1.1 the queens versions? | 15:23 |
| fungi | EmilienM: i don't think the one i've been watching has gotten far enough to start up its snmpd yet and i have to disappear in ~25 minutes so may not catch it (i do have a loop trying to hit it though just in case), but also i dug out the snmpd.conf for one of the exact nodes which showed up in the provider's scan: | 15:23 |
| fungi | http://logs.openstack.org/57/543157/1/check/tripleo-ci-centos-7-3nodes-multinode/ffc73d8/logs/subnode-3/etc/snmp/snmpd.conf.txt.gz | 15:23 |
| sshnaidm|ruck | mwhahaha, the problem is not with environments | 15:23 |
| mwhahaha | sshnaidm|ruck: right but we really need to audit/explicitly state where these types of configs need to live | 15:23 |
| mwhahaha | sshnaidm|ruck: this seems to be where there is an unintended (not tested) config change that occured | 15:24 |
| *** panda is now known as panda|mtg | 15:24 | |
| mwhahaha | which would indicate that we don't actually know what vars are doing and which jobs are actually affected | 15:24 |
| sshnaidm|ruck | mwhahaha, tested by promotion..? | 15:24 |
| panda|mtg | jistr: chem looking shortly | 15:24 |
| fungi | EmilienM: i need to go back through the config docs to be sure, but i _think_ that means that any get for community "public" is mapped to the notConfigGroup access entry which allows unauthenticated access to the systemview mib? | 15:24 |
| sshnaidm|ruck | mwhahaha, I think you make too big conclusions from this case | 15:24 |
| mwhahaha | sshnaidm|ruck: but that's a delayed test after CI, that's not a good test because now we've hurt the promotion pipeline for another 8 hours | 15:25 |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient master: Adds --all-playbooks option to overcloud upgrade to run all playbooks https://review.openstack.org/543990 | 15:25 |
| mwhahaha | or whatever the frequency is | 15:25 |
| apevec | mwhahaha, we can change crontab temporarily | 15:26 |
| *** akane_ has quit IRC | 15:26 | |
| mwhahaha | apevec: but we shouldn't have to is my point, yes we can work around everything but there's a point where we shouldn't have to be doing this | 15:26 |
| mwhahaha | we keep making self inflicted CI problems | 15:26 |
| apevec | yeah, we could definitely improve | 15:27 |
| sshnaidm|ruck | mwhahaha, yeah, everything is bad, Ci don't work and we don't know what are we doing at all, creating problems only | 15:27 |
| dtantsur | mcornea: the versioning is funny here. 10.1.1 is the last tag on both master and queens. note, however, that post-version (starting with .0) is different for both | 15:27 |
| mwhahaha | sshnaidm|ruck: ok thanks for the snark, i'm trying to raise the concern that the vars in quickstart are not well organized to prevent unintended changes. | 15:28 |
| dtantsur | tl;dr these are different packages | 15:28 |
| mcornea | dtantsur: oh boy that's confusing :) but indeed they are different, it worked with tripleo-repos -b queens current | 15:30 |
| EmilienM | jaosorior, fungi: in a call, I'll get back to you asap | 15:31 |
| *** lucas-hungry is now known as lucasagomes | 15:31 | |
| *** dpawar has quit IRC | 15:33 | |
| *** atoth has quit IRC | 15:33 | |
| EmilienM | jaosorior: so https://review.openstack.org/#/c/544229/ will fix ipsec job? | 15:34 |
| *** paramite_ has quit IRC | 15:34 | |
| shardy | bnemec: Hi, I've been investigating using awx/tower alongside the undercloud and saw your screencast - I wondered did you have any discussion about tower using keystone as an authentication source? | 15:34 |
| *** paramite has quit IRC | 15:34 | |
| jaosorior | EmilienM: I think so. my deployment is still on-going but it already passed the step where it failed before. | 15:35 |
| shardy | also interested to know if anyone is looking at enabling alternatives to postgres, e.g if we wanted to install awk alongside the existing pieces on either the undercloud or an overcloud management node | 15:35 |
| EmilienM | sshnaidm|ruck: the job isn't passing, I filed a bug: https://launchpad.net/bugs/1749338 | 15:35 |
| openstack | Launchpad bug 1749338 in tripleo "tripleo-ipsec : failure to set current IP fact" [High,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 15:35 |
| jaosorior | oh, now it passed haha | 15:35 |
| shardy | s/awk/awx | 15:36 |
| *** suuuper has quit IRC | 15:36 | |
| jaosorior | sshnaidm|ruck: this would be the fix https://review.openstack.org/#/c/544229/ | 15:36 |
| bnemec | shardy: sdoran had sent me a response about alternative auth methods in recent versions of Tower. He'd be the one to talk to about that. | 15:36 |
| *** suuuper has joined #tripleo | 15:36 | |
| bnemec | shardy: I looked at installing it in parallel with the undercloud, but it would be a mess. The Tower installer is a standalone Ansible playbook, and it wants to configure rabbitmq too. | 15:37 |
| shardy | bnemec: ack thanks, I figured you may have discussed it with sdoran but didn't want to repeat the exact same questions if there's something you can point me to or forward :) | 15:37 |
| shardy | bnemec: yeah, I looked at the same, was wondering how hard it'd be to make it use our rabbit and DB | 15:37 |
| bnemec | It might be possible to write a unified installer but I think you'd have to reimplement the Tower playbook in the undercloud installer. | 15:37 |
| shardy | looked fairly difficult from a quick pass at the code as it expects postgres | 15:37 |
| bnemec | Otherwise we've got two installers trying to manage the same services. | 15:37 |
| sshnaidm|ruck | jaosorior, EmilienM so to make it non-voting or to wait for fix..? | 15:38 |
| jaosorior | sshnaidm|ruck: you really wanna ask me? :D | 15:38 |
| EmilienM | I would make it non voting unless jaosorior is sure this patch works | 15:38 |
| sshnaidm|ruck | ok | 15:38 |
| EmilienM | jaosorior: if you're sure, then ok let's keep like this (voting now) | 15:38 |
| shardy | bnemec: yeah I was thinking we could use our existing container architecture to launch the awx task/web containers | 15:39 |
| jaosorior | cool | 15:39 |
| shardy | but wire those in to the tripleo memcached/rabbit/DB | 15:39 |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo master: Fix galera deployment when stack's name has capital letters https://review.openstack.org/544384 | 15:39 |
| shardy | probably a lot of work but if the auth plugin could use keystone it'd be a pretty nice setup | 15:39 |
| *** links has joined #tripleo | 15:40 | |
| *** ramishra has quit IRC | 15:40 | |
| fungi | EmilienM: confirmed, it's responding! | 15:40 |
| bnemec | shardy: In general I never followed up on anything after that video. I looked at it as the end of my transition work for that particular item. | 15:40 |
| fungi | snmpwalk -v1 -c public 158.69.66.124 .1.3.6.1.2.1.1.3.0 | 15:40 |
| bnemec | So basically everything I know is in the video. :-) | 15:41 |
| fungi | iso.3.6.1.2.1.1.3.0 = Timeticks: (9163) 0:01:31.63 | 15:41 |
| shardy | bnemec: ack, understood, just wanted to sync up as I may take a look as a background task - thanks! | 15:41 |
| EmilienM | fungi: ok so something in the config isn't good | 15:41 |
| fungi | EmilienM: so yes, definitely exposed and answering to community public | 15:41 |
| EmilienM | it's weird, we have: rocommunity public 127.0.0.1 | 15:41 |
| fungi | EmilienM: but then you have an acl allowing unauthenticated access to systemview for community public as well | 15:42 |
| fungi | access notConfigGroup "" any noauth exact systemview none none | 15:42 |
| EmilienM | fungi: and what an attacker can do with this access? just so I understand | 15:42 |
| sdoran | shardy: Tower DB has to be PSQL. As far as using another rabbitmq, that should be possible but would require some work. | 15:43 |
| fungi | EmilienM: it's over udp, so an attacker can spoof very small snmpgets over udp to the server for any oid which returns larger amounts of data and mount an amplified reflection attack. this is a common ddos tactic | 15:43 |
| EmilienM | PSQL o_O | 15:44 |
| shardy | sdoran: ack, I was wondering if there was interest in reworking to use an abstraction e.g sqlalchemy to enable aternate db backends | 15:44 |
| sdoran | shardy: I'm not sure what you mean about Keystone as an auth mechanism for Tower. Do you mean for logging in to the Tower interface, or having playbooks retrieve data from Keystone? | 15:44 |
| shardy | sdoran: I mean have keystone be the auth backend for tower, so you'd log in to tower with your openstack credentials, then interact with the nodes owned by your tenant | 15:44 |
| EmilienM | fungi: I'll work on a patch that change the iptables rules. | 15:44 |
| fungi | EmilienM: anyway, i think the issue is figured out? i need to disappear to run some errands but will be back later if you have any questions | 15:44 |
| sdoran | shardy: Tower Eng does not have plans to use other DB backends as far as I'm aware. But that's a totally separate product. You can ask in #ansible-awx. | 15:44 |
| fungi | EmilienM: thanks!!! | 15:45 |
| EmilienM | fungi: thanks, I have enough | 15:45 |
| sdoran | shardy: <ack> Does Keystone use SAML or LDAP? (Don't know much about Keystone) | 15:45 |
| openstackgerrit | Keith Schincke proposed openstack/tripleo-common master: Add file type check to upload-swift-artifacts https://review.openstack.org/544412 | 15:45 |
| shardy | sdoran: it can use ldap or several other backends, by default it's sql backed by mysql | 15:46 |
| *** atoth has joined #tripleo | 15:46 | |
| shardy | sdoran: it's the authentication project for openstack, and it provides auth for all openstack projects, so in an openstack env w/awx it could be a nice option to have | 15:47 |
| *** holser has quit IRC | 15:47 | |
| shardy | vs two sets of credentials and having to copy your openstack credentials into tower to retrieve inventory data | 15:47 |
| shardy | which is what bnemec showed in https://www.youtube.com/watch?v=0RB1fT_Gx70 | 15:48 |
| *** holser has joined #tripleo | 15:49 | |
| *** itlinux has joined #tripleo | 15:50 | |
| sdoran | shardy: Yeah, that would be ideal. If it can provide LDAP, Tower _should_ be able to talk to it. Tower uses Python Django LDAP library. | 15:51 |
| openstackgerrit | Merged openstack/tripleo-common master: Quote $@ in ansible-playbook-command.sh https://review.openstack.org/543587 | 15:51 |
| sdoran | Then you can configure Keystone as an LDAP source in Tower. | 15:51 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: WIP During a minor update make sure docker service has live-restore option https://review.openstack.org/541827 | 15:51 |
| shardy | sdoran: yeah it doesn't provide ldap, it's another API that can optionally be backed by ldap | 15:51 |
| sdoran | Setup team/org mappings, and be off to the races. | 15:51 |
| sdoran | SAML? | 15:52 |
| *** sai___ has joined #tripleo | 15:52 | |
| shardy | sdoran: it looks like we could add an openstack backend but the awx code isn't pluggable for auth atm, it's one big file | 15:52 |
| *** gkadam_ has joined #tripleo | 15:52 | |
| sdoran | Right, that's why I'm trying to see if it fits existing methods. | 15:52 |
| sdoran | Tower is not pluggable like Ansible. | 15:52 |
| sdoran | It's a much more "baked" product. | 15:52 |
| jaosorior | sdoran: keystone it could do SAML if federation is enabled. | 15:53 |
| shardy | jaosorior: yeah that'd be interesting to try | 15:54 |
| shardy | otherwise it'd be an alterative to the awx token authenication that is built in I think | 15:54 |
| *** moguimar has quit IRC | 15:57 | |
| *** ykarel is now known as ykarel|away | 15:57 | |
| *** lvdombrkr89 has quit IRC | 15:58 | |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade master: Use queens branch for the 3rd tripleo-repos FFU repository https://review.openstack.org/544512 | 15:59 |
| gfidente | slagle so I was looking at enable-ssh-admin.sh | 16:04 |
| gfidente | I figure it is the workflow consumer | 16:04 |
| gfidente | enable_admin_via_ssh | 16:04 |
| *** gkadam has quit IRC | 16:04 | |
| gfidente | while stuff like ceph-ansible workflow uses enable_admin_via_nova | 16:05 |
| *** ykarel|away has quit IRC | 16:05 | |
| *** suuuper has quit IRC | 16:08 | |
| *** suuuper has joined #tripleo | 16:08 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 16:08 |
| slagle | gfidente: the ceph-ansible workflow will use enable_admin_via_ssh in the case of deployed-server. at least it needs to | 16:09 |
| gfidente | slagle right | 16:09 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 16:10 |
| *** dparkes has quit IRC | 16:12 | |
| *** pcaruana has quit IRC | 16:13 | |
| *** aufi has quit IRC | 16:13 | |
| *** dparkes has joined #tripleo | 16:14 | |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Add support for Dell EMC XtremIO Cinder ISCSI Backend https://review.openstack.org/544065 | 16:17 |
| *** gvrangan has joined #tripleo | 16:17 | |
| *** cylopez1 has quit IRC | 16:19 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Replace devtest reference with quickstart https://review.openstack.org/541380 | 16:19 |
| *** atoth has quit IRC | 16:21 | |
| *** panda|mtg is now known as panda | 16:22 | |
| EmilienM | mwhahaha, sshnaidm|ruck, pabelanger, dmsimard : third party voting and working well https://review.openstack.org/#/c/544249/ | 16:23 |
| EmilienM | I'll take care of next steps (adding more projects) | 16:23 |
| dmsimard | EmilienM: cool gj | 16:26 |
| mwhahaha | yay voting | 16:26 |
| mwhahaha | only took how many years? :D | 16:26 |
| EmilienM | (the link is an example btw) | 16:27 |
| d0ugal | thrash: any luck with the node delete? | 16:27 |
| sshnaidm|ruck | mwhahaha, I knew something is wrong | 16:27 |
| sshnaidm|ruck | just waited to know what)) | 16:27 |
| thrash | d0ugal: Looks like I've got an environment coming up successfully. Will be trying the delete once it's done. | 16:28 |
| pabelanger | EmilienM: good work | 16:28 |
| thrash | d0ugal: meanwhile, nothing is sticking out to me just looking at the code. | 16:28 |
| d0ugal | thrash: nice, maybe I'll copy your method - mind failed. | 16:28 |
| d0ugal | "mind failed" - seems about right. | 16:28 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Inherit UpdateCommand class from overcloud DeployCommand class https://review.openstack.org/540798 | 16:28 |
| thrash | d0ugal: lol | 16:29 |
| *** psachin has quit IRC | 16:29 | |
| *** rlandy is now known as rlandy|brb | 16:32 | |
| *** leanderthal has quit IRC | 16:33 | |
| *** ukalifon has quit IRC | 16:33 | |
| gfidente | d0ugal the variable defined in with-items, can be consumed when defining the task inputs or is it only valid in the task action definition? | 16:34 |
| *** atoth has joined #tripleo | 16:34 | |
| *** gvrangan has quit IRC | 16:34 | |
| *** tosky has quit IRC | 16:36 | |
| d0ugal | gfidente: I don't understand the difference, they sound like the same thing? | 16:36 |
| d0ugal | gfidente: do you mean like this? https://github.com/openstack/tripleo-common/blob/cffc0c476647dd97e9c47301bf91014ad4b12335/workbooks/baremetal.yaml#L262-L265 | 16:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Restrict SNMP to internal network https://review.openstack.org/544530 | 16:37 |
| panda | jistr: chem rats! we doubled the effort, and we needed even more coordination | 16:37 |
| gfidente | d0ugal yes, perfect | 16:37 |
| gfidente | thanks :D | 16:37 |
| d0ugal | gfidente: np | 16:37 |
| *** suuuper has quit IRC | 16:37 | |
| EmilienM | shardy: if you have 2 min, I have this urgent patch to review ^ https://review.openstack.org/544530 - I'm not sure about the InternalApiIpSubnet usage, I probably did wrong. Maybe you can review. Thanks | 16:37 |
| *** suuuper has joined #tripleo | 16:38 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Inherit UpdateCommand class from overcloud DeployCommand class https://review.openstack.org/540798 | 16:38 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Inherit UpdateCommand class from overcloud DeployCommand class https://review.openstack.org/540798 | 16:38 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Add generic methods to upload files to plan https://review.openstack.org/543902 | 16:38 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Add generic method for preserving plan files, use for user-environment.yaml https://review.openstack.org/543903 | 16:38 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Use the generic preservation method for plan-environment.yaml https://review.openstack.org/543904 | 16:38 |
| openstackgerrit | Jiri Stransky proposed openstack/python-tripleoclient master: Preserve roles_data and network_data https://review.openstack.org/543905 | 16:38 |
| panda | jistr: can we meet ? | 16:39 |
| panda | chem: if you have time, I have to discuss something with you after | 16:39 |
| jistr | panda: yea, tomorrow maybe? | 16:40 |
| jistr | panda: sorry i should have been more vocal about the WIP patches | 16:40 |
| panda | jistr: yes, the fact is that we already merged the job and featurest45 | 16:41 |
| panda | jistr: and I wanted to understand if there's something we should port | 16:41 |
| *** jcoufal_ has joined #tripleo | 16:41 | |
| *** itlinux has quit IRC | 16:41 | |
| *** suuuper has quit IRC | 16:41 | |
| jistr | panda: the 037 test actually went as far as running `openstack overcloud update stack --init-update`, using the tripleo-upgrade role | 16:41 |
| panda | jistr: we can do it tomorrow, but our new sprints begins in the afternoon | 16:41 |
| panda | jistr: so we need to port stuff deom 04 to 037 | 16:42 |
| jistr | panda: so the 045 work is in what stage right now? just the featureset merged, or did we try to run something? | 16:42 |
| panda | jistr: sorry, my bad, I looked for upgrades to see if there was already something about updates | 16:43 |
| panda | jistr: also, I think I remember to have asked in one of the meetings if there was some fs for the updates. | 16:43 |
| panda | jistr: jsut merged | 16:43 |
| panda | jistr: but I created it with chm and jfrancoa | 16:43 |
| jistr | panda: i noticed chem also had a patch for 037 but that one is "old style", not using tripleo-upgrade yet | 16:44 |
| panda | in a ad-hoc meeting | 16:44 |
| panda | jistr: yes because we want to maintain the two paths now for some jobs | 16:44 |
| *** jcoufal has quit IRC | 16:44 | |
| jistr | panda: ok but it's not merged/implemented yet? do we want to implement an update job in 2 different ways? | 16:45 |
| panda | chem: you here, I will need to put some notes on the glue patch, so I'm adding a patchset and then it's good to go | 16:46 |
| dtantsur | bnemec: hi! mind checking https://github.com/cybertron/openstack-virtual-baremetal/pull/40 more or less soon please? | 16:46 |
| panda | jistr: probably not, if we know we are not going to run the updates with the old style | 16:46 |
| panda | jistr: we have to understand how to put your worjk into the job that we created | 16:47 |
| bnemec | dtantsur: Yep, will do. | 16:47 |
| panda | jistr: that's why I wanted to meet | 16:47 |
| *** fzdarsky has joined #tripleo | 16:47 | |
| panda | jistr: tomorrow morning good for you ? | 16:48 |
| chem | panda: thanks | 16:49 |
| jistr | panda: yea sure we can meet tomorrow. Is 10 AM ok? | 16:49 |
| panda | jistr: 10 UTC ? | 16:50 |
| jistr | panda: i meant CET but 10 UTC (11 CET) is totally fine too | 16:50 |
| jistr | panda: you're on CET right? | 16:51 |
| panda | jistr: I'm in GMT | 16:51 |
| jistr | ahh ok | 16:51 |
| panda | jistr: since september | 16:51 |
| jistr | panda: so let's make it 10 UTC? | 16:51 |
| panda | jistr: fine | 16:51 |
| *** rlandy|brb is now known as rlandy | 16:51 | |
| jistr | panda: ack thanks :) | 16:51 |
| *** itlinux has joined #tripleo | 16:51 | |
| *** marios has quit IRC | 16:55 | |
| panda | jistr: did you receive the invitation ? I'm playing with gcalcli and I'm not sure I'm doing it right | 16:57 |
| jistr | panda: no but i added it to my phone calendar | 16:57 |
| * jistr not using gcal much | 16:57 | |
| *** waleedm has joined #tripleo | 16:58 | |
| panda | jistr: I feel the same, trying to understand if using a command line tool will ease the pain :) | 16:58 |
| *** oidgar has quit IRC | 17:01 | |
| raildo | mandre, hey, what's up? I was wondering if you could find a solution for https://bugs.launchpad.net/tripleo/+bug/1748219 | 17:02 |
| openstack | Launchpad bug 1748219 in tripleo "keystone integration with LDAP broken in containerized deployment, manually keystone restart needed" [High,Triaged] | 17:02 |
| *** ledo_ has joined #tripleo | 17:05 | |
| *** waleedm has quit IRC | 17:07 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: [FFU] Hook to allow user to pass a custom script for repo switching. https://review.openstack.org/539503 | 17:08 |
| *** beagles is now known as beagles_brb | 17:08 | |
| *** dtantsur is now known as dtantsur|afk | 17:09 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 17:10 |
| Tengu | shardy: hello! are still around? | 17:13 |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Upgrade UC and OC using tripleo-upgrade role. https://review.openstack.org/542311 | 17:15 |
| *** dsariel_ has quit IRC | 17:16 | |
| panda | weshay: chem sshnaidm|ruck trown ^ first attempt at making a TD visible | 17:16 |
| *** dsariel has quit IRC | 17:16 | |
| *** pcaruana has joined #tripleo | 17:17 | |
| *** ramishra has joined #tripleo | 17:20 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: [FFU] Hook to allow user to pass a custom script for repo switching. https://review.openstack.org/539503 | 17:21 |
| *** links has quit IRC | 17:21 | |
| *** tosky has joined #tripleo | 17:23 | |
| *** waleedm has joined #tripleo | 17:23 | |
| *** etingof has quit IRC | 17:23 | |
| chem | panda: TD ? | 17:24 |
| *** jpich has quit IRC | 17:24 | |
| *** chem is now known as chem|eod | 17:25 | |
| panda | chem|eod: : technical debt | 17:25 |
| chem|eod | panda: ack | 17:25 |
| panda | chem|eod: if we are going to merge this, we have to keep track of the things that will need refactoring | 17:25 |
| *** ledo_ has quit IRC | 17:25 | |
| chem|eod | panda: ack, basically concerning upgrade, it's all around the "fixed" release file I think | 17:26 |
| chem|eod | panda: and a role for the glue | 17:27 |
| trown | panda you have a tech debt in your tech debt | 17:27 |
| panda | chem|eod: yes, adding that in the other change too | 17:27 |
| panda | trown: metadebt | 17:27 |
| trown | panda: jk of course... just wanted to say that :P | 17:27 |
| panda | trown: what exaclty, this is my fisrt attempt and I would have really liked to be more prepared for this | 17:27 |
| *** gvrangan has joined #tripleo | 17:28 | |
| panda | trown: well, I think I have, beacuse I'm not sure this will be the final form of the design/TD register | 17:28 |
| chem|eod | bye | 17:28 |
| panda | chem|eod: bye | 17:28 |
| trown | panda: no just a minor typo in the note, I think the note is simple and to the point ... maybe it could include list of known files with this type of debt? (in the note) | 17:28 |
| *** dparkes has quit IRC | 17:29 | |
| panda | trown: you want forward and backward reference ? | 17:29 |
| trown | panda: that note in particular would lend itself to a filelist, since it is specifically playbooks including tasks which should not be in playbooks | 17:29 |
| panda | trown: you want to add the files where we hae found the technical debts, or the list of files that should follow that hint ? | 17:30 |
| trown | panda: ya was just thinking, if someone wanted to tackle all the debts of a certain type it could be useful | 17:30 |
| trown | panda: no just specifically places we know that type of debt exists | 17:30 |
| panda | trown: my initial Idea was using UUIDs | 17:30 |
| panda | trown: so you take the uuid in the doc, and grep it in all the code | 17:30 |
| panda | trown: but at the moment I'm ok with keeping the title as key | 17:31 |
| panda | so you can grep that | 17:31 |
| *** yamahata has quit IRC | 17:31 | |
| trown | panda: that makes sense | 17:31 |
| *** gvrangan has quit IRC | 17:31 | |
| *** ramishra has quit IRC | 17:31 | |
| panda | not even wure if people will gete to do this, I don't want to add other overhead to the process | 17:31 |
| trown | ya for sure | 17:31 |
| trown | I like the idea of developing more tooling around tracking it... seems like it would be generically useful to other software projects | 17:32 |
| *** tcw1 is now known as tcw | 17:32 | |
| *** beagles_brb is now known as beagles | 17:33 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Disable SNMP service in all CI jobs https://review.openstack.org/544567 | 17:34 |
| EmilienM | fungi, mwhahaha, weshay ^ that's an immediate response to the security issue in CI | 17:34 |
| *** gvrangan has joined #tripleo | 17:35 | |
| *** ledo_ has joined #tripleo | 17:36 | |
| *** waleedm has quit IRC | 17:37 | |
| *** waleedm has joined #tripleo | 17:40 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Provide a way to change docker registry path during mixed upgrade. https://review.openstack.org/540473 | 17:41 |
| panda | trown: another attempt ^ | 17:41 |
| *** ledo_ has quit IRC | 17:41 | |
| shardy | Tengu: Hi, sorry was afk for a while, I am still around | 17:42 |
| *** thrash is now known as thrash|f00dz | 17:42 | |
| panda | going home | 17:43 |
| *** panda is now known as panda|off | 17:43 | |
| Tengu | shardy: ah, great :). | 17:44 |
| *** holser has quit IRC | 17:44 | |
| *** gvrangan has quit IRC | 17:44 | |
| *** jcoufal has joined #tripleo | 17:45 | |
| Tengu | shardy: I have a small issue with a tripleo deploy: apparently it's unable to find the two ceph-storage-[12] nodes in ironic. I searched for filter errors in the nova-scheduler.log, but apparently... it doesnt KNOW about those nodes, hence can't submit them to filter | 17:45 |
| Tengu | shardy: any thing I might check? I'm a bit lost, logs aren't really talking | 17:46 |
| shardy | Tengu: What state are the nodes in, e.g are they available and not in maintenance mode when viewing via openstack baremetal node list? | 17:47 |
| Tengu | shardy: I use "precise allocation", hence I have a capability "node:ceph-storage-0" and "node:ceph-storage-1", with the entry in my env file: CephStorageSchedulerHints: 'capabilities:node': 'ceph-storage-%index%' | 17:47 |
| *** jcoufal_ has quit IRC | 17:47 | |
| Tengu | shardy: "available"; "power off", maintenance flag set to Fale for both of them | 17:47 |
| *** jcoufal_ has joined #tripleo | 17:49 | |
| Tengu | it's just… I don't understand. more over, it's the same receipt I pushed on the production end-2017, and it was working just fine: I have my two ceph nodes on prod. But on the lab? nope :'(. I'm banging my head against the walls since last Friday with this issue, I wanted to find by myself… result: headache and no solution for now. | 17:50 |
| *** jcoufal has quit IRC | 17:51 | |
| *** moshele has joined #tripleo | 17:52 | |
| flaper87 | is there a way to tell oooq to add an extra disk to my nodes? | 17:52 |
| *** jcoufal has joined #tripleo | 17:53 | |
| *** waleedm has quit IRC | 17:54 | |
| Tengu | and even the python stack traces in nova-conductor logs are meaningless: raise exception.NoValidHost(reason="") yeah, empty reason :( | 17:54 |
| *** jcoufal_ has quit IRC | 17:55 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-repos master: Add networking-baremetal packages includepkgs https://review.openstack.org/544577 | 17:55 |
| shardy | Tengu: have you checked ironic-conductor is still running OK? | 17:57 |
| shardy | I've seen the empty reason before when it failed | 17:57 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Removes manila-generic-config from TripleO https://review.openstack.org/542136 | 17:57 |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Install and enable neutron baremetal mech plugin https://review.openstack.org/537830 | 17:58 |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Install and enable neutron baremetal mech plugin https://review.openstack.org/537830 | 17:58 |
| Tengu | shardy: apparently it's still logging stuff… and the ceph-storage are the only nodes I can't add - even when I've re-installed the whole lab, that's only those two. 4 computes? OK. 3 controllers? np. but 2 ceph ? nah | 17:58 |
| *** moshele has quit IRC | 17:58 | |
| EmilienM | flaper87: hey, be more specific. Like a disk for what exactly? not sure we have a role for that but if that's the case it would be in tripleo-quickstart-extras | 17:59 |
| Tengu | shardy: IPMI is working OK (via virtualbmc and som network magic), I can introspect all nodes without any problem | 17:59 |
| Tengu | so it's not an issue it can't get state or whatever. plus it would put the node in maintenance node if IPMI fails. | 17:59 |
| Tengu | the precise error message returned by the deploy command : ResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. , Code: 500" | 18:00 |
| flaper87 | EmilienM: think I found it: extradevices | 18:00 |
| flaper87 | I want those disks on the BM nodes | 18:00 |
| EmilienM | flaper87: good | 18:01 |
| shardy | Tengu: Ok so it must be a mismatch with the flavor and node profile I guess, what flavor are you using for CephStorage? | 18:02 |
| *** gvrangan has joined #tripleo | 18:03 | |
| Tengu | shardy: c2c_baremetal flavor, the same I use for all other nodes (compute + controller) | 18:03 |
| Tengu | shardy: I think flavor aren't really involved since I use precise allocation with the "node" capacity | 18:03 |
| *** derekh has quit IRC | 18:03 | |
| *** yprokule has quit IRC | 18:04 | |
| *** yamahata has joined #tripleo | 18:06 | |
| Tengu | shardy: http://paste.openstack.org/show/672491/ the c2c_baremetal flavor, if needed | 18:07 |
| shardy | Tengu: Ok well there can be a few resons for no valid host, but I don't see how it can be role specific if you're using the same flavor for all roles | 18:07 |
| Tengu | but even that… the undercloud VM is the copy of the production. The only thing I did was "openstack stack delete overcloud --yes --wait" | 18:07 |
| shardy | Tengu: are you trying the create immedately after the delete? | 18:08 |
| Tengu | shardy: nope | 18:08 |
| *** gfidente has quit IRC | 18:08 | |
| Tengu | oh, and I drop and re-import the nodes in ironic… maybe there's something with that? I read I need a fixed MAC for virtual machins overcloud in order to get ironic/PXE boot working | 18:08 |
| Tengu | so I added the MAC entry in my "baremetal.json". this is the only difference… and this actually seems to work, as ironic can import the nodes, introspect them and make them available. | 18:09 |
| shardy | Tengu: the re-import could be related, you should not need to do that, but if you do then there will be a delay before they are available to nova | 18:09 |
| shardy | Tengu: check nova hypervisor-stats | 18:10 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 18:10 |
| Tengu | shardy: I also wait for the nodes to be available. | 18:11 |
| Tengu | shardy: openstack hypervisor stats show lists 9 node, meaning it has the 4 compute + 3 controller + 2 cephs | 18:12 |
| Tengu | it shows 7 running VMs, they are the 4 compute + 3 controllers | 18:12 |
| *** waleedm has joined #tripleo | 18:13 | |
| Tengu | all of that seems perfectly good :(. | 18:13 |
| *** ledo_ has joined #tripleo | 18:14 | |
| *** jpena is now known as jpena|off | 18:17 | |
| *** waleedm has quit IRC | 18:18 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud_post: fix subnet name https://review.openstack.org/544587 | 18:19 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 18:20 |
| *** trown is now known as trown|lunch | 18:22 | |
| *** pkovar has quit IRC | 18:22 | |
| EmilienM | dpeacock: re: ansible role to restart containers, I found this blog post super useful, hope that helps (note it's not for containers but you get the same concept) http://blog.andy.mc.it/restarting-services-in-serial-in-ansible/ | 18:22 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Add non-production ceph defaults to low-memory-usage.yaml https://review.openstack.org/544588 | 18:23 |
| *** pradk has joined #tripleo | 18:24 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-specs master: blueprint tripleo-routed-networks-ironic-inspector https://review.openstack.org/421011 | 18:25 |
| *** psachin has joined #tripleo | 18:26 | |
| *** ledo__ has joined #tripleo | 18:26 | |
| EmilienM | sdoran: also ^ about the blog post, maybe you can talk to Andy, he's quite active in OpenStack community | 18:26 |
| *** aditya_r has joined #tripleo | 18:27 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: ffu: Allow FASTFORWARDUPGRADE as a StackUpdateType https://review.openstack.org/522547 | 18:27 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Use queens branch for the 3rd tripleo-repos FFU repository https://review.openstack.org/544512 | 18:27 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud_post: fix subnet name https://review.openstack.org/544587 | 18:28 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud_post: fix subnet name https://review.openstack.org/544587 | 18:28 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 18:28 |
| EmilienM | pabelanger, dmsimard : please review https://review.openstack.org/#/c/544580/ | 18:28 |
| EmilienM | sshnaidm|ruck: ^ FYI | 18:28 |
| *** assassin has quit IRC | 18:29 | |
| *** ledo_ has quit IRC | 18:30 | |
| EmilienM | I'll need urgent votes on https://review.openstack.org/#/c/544567/ please - it's for a security issue in our SNMP config | 18:33 |
| *** openstackgerrit has quit IRC | 18:33 | |
| *** ledo__ has quit IRC | 18:34 | |
| *** waleedm has joined #tripleo | 18:35 | |
| fultonj | EmilienM: makes sense to me +2 | 18:36 |
| *** openstackgerrit has joined #tripleo | 18:36 | |
| openstackgerrit | Merged openstack/tripleo-specs master: blueprint tripleo-routed-networks-ironic-inspector https://review.openstack.org/421011 | 18:36 |
| EmilienM | fultonj: thank you | 18:37 |
| *** shardy has quit IRC | 18:38 | |
| dpeacock | EmilienM: Excellent - thank you Sir | 18:39 |
| *** zoli is now known as zoli|gone | 18:41 | |
| *** zoli|gone is now known as zoli | 18:42 | |
| *** ledo_ has joined #tripleo | 18:43 | |
| *** ebarrera has quit IRC | 18:43 | |
| *** jcoufal_ has joined #tripleo | 18:45 | |
| *** waleedm has quit IRC | 18:46 | |
| *** aditya_r has quit IRC | 18:48 | |
| *** ledo_ has quit IRC | 18:48 | |
| *** jcoufal has quit IRC | 18:48 | |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Install and enable neutron baremetal mech plugin https://review.openstack.org/537830 | 18:49 |
| *** florianf has quit IRC | 18:49 | |
| *** jpena|off is now known as jpena | 18:50 | |
| *** waleedm has joined #tripleo | 18:53 | |
| *** tesseract has quit IRC | 18:54 | |
| *** ledo_ has joined #tripleo | 18:54 | |
| *** jcoufal has joined #tripleo | 18:55 | |
| *** lucasagomes is now known as lucas-afk | 18:56 | |
| *** jcoufal_ has quit IRC | 18:57 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates stable/ocata: Removes manila-generic-config from TripleO https://review.openstack.org/544602 | 18:58 |
| *** waleedm has quit IRC | 18:58 | |
| *** jpena is now known as jpena|off | 18:59 | |
| *** waleedm has joined #tripleo | 19:02 | |
| *** sshnaidm|ruck is now known as sshnaidm|off | 19:03 | |
| *** ledo_ has quit IRC | 19:04 | |
| *** atoth has quit IRC | 19:05 | |
| *** jcoufal_ has joined #tripleo | 19:07 | |
| *** jaganathan has quit IRC | 19:08 | |
| *** jcoufal has quit IRC | 19:09 | |
| *** maeca has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 19:10 |
| *** jcoufal has joined #tripleo | 19:13 | |
| EmilienM | jaosorior: still around? | 19:13 |
| *** waleedm has quit IRC | 19:14 | |
| *** jcoufal_ has quit IRC | 19:15 | |
| EmilienM | jaosorior: containers still failing on barbican | 19:15 |
| EmilienM | alee: ^ if you around, need your help | 19:15 |
| *** waleedm has joined #tripleo | 19:16 | |
| *** jcoufal_ has joined #tripleo | 19:17 | |
| *** atoth has joined #tripleo | 19:18 | |
| alee | EmilienM, whats up? | 19:18 |
| EmilienM | alee: https://bugs.launchpad.net/tripleo/+bug/1749335 | 19:19 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 19:19 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade master: DNM. Test tripleo-upgrade workarounds. https://review.openstack.org/544604 | 19:19 |
| *** jcoufal has quit IRC | 19:19 | |
| alee | EmilienM, ah - thats something jaosorior recently added | 19:20 |
| EmilienM | mwhahaha: someone created branches for stable/queens | 19:20 |
| EmilienM | I just found that out | 19:20 |
| mwhahaha | for what | 19:20 |
| mwhahaha | EmilienM: which projects? | 19:21 |
| EmilienM | mwhahaha: let's talk on #openstack-release | 19:21 |
| *** waleedm has quit IRC | 19:22 | |
| *** rwsu has quit IRC | 19:28 | |
| *** rwsu has joined #tripleo | 19:29 | |
| *** waleedm has joined #tripleo | 19:30 | |
| EmilienM | sshnaidm|off, mwhahaha : have you seen that before? (blocking promotion) https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/d879798/undercloud/home/jenkins/overcloud_prep_containers.log.gz | 19:33 |
| EmilienM | sounds like a random network issue? | 19:33 |
| EmilienM | 2018-02-14 13:46:27 | ImageUploaderException: Could not pull image docker.io/tripleomaster/centos-binary-rabbitmq | 19:33 |
| mwhahaha | yea there's a fix to go back to rdo | 19:33 |
| mwhahaha | from docker.io | 19:33 |
| mwhahaha | from this morning | 19:33 |
| mwhahaha | a revert in tripleo-ci | 19:34 |
| EmilienM | ah yeah | 19:34 |
| mwhahaha | https://review.openstack.org/#/c/544378/ | 19:34 |
| *** Jimbagwell has joined #tripleo | 19:34 | |
| EmilienM | in gate | 19:34 |
| *** moshele has joined #tripleo | 19:34 | |
| EmilienM | alee: yeah we need to look at that, why it fails - it's blocking container builds | 19:35 |
| pabelanger | https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master/d879798/undercloud/etc/docker/daemon.json.gz | 19:36 |
| pabelanger | is not using any proxy | 19:36 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-docs master: WIP Add Instance HA docs https://review.openstack.org/544232 | 19:37 |
| *** jcoufal has joined #tripleo | 19:37 | |
| pabelanger | EmilienM: mwhahaha ^should have access to it on mirror.regioneone.rdo-cloud.rdoproject.org | 19:37 |
| *** jcoufal_ has quit IRC | 19:39 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud_post: fix subnet name https://review.openstack.org/544587 | 19:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - run container-multinode (fs010) with a containerized undercloud https://review.openstack.org/542906 | 19:40 |
| alee | EmilienM, says something about a dangling symlink -- does /usr/share/openstack-tripleo-common/healthcheck/barbican-worker exist? | 19:41 |
| EmilienM | yeah I see the error | 19:41 |
| EmilienM | apparently not | 19:41 |
| *** salmankhan has quit IRC | 19:42 | |
| *** trown|lunch is now known as trown|outtypewww | 19:42 | |
| alee | EmilienM, seems to be in latest tripleo-common | 19:43 |
| *** yolanda has quit IRC | 19:44 | |
| *** paramite_ has joined #tripleo | 19:44 | |
| *** paramite has joined #tripleo | 19:44 | |
| *** jcoufal_ has joined #tripleo | 19:44 | |
| alee | https://github.com/openstack/tripleo-common/blob/master/healthcheck/barbican-worker | 19:44 |
| *** jcoufal has quit IRC | 19:44 | |
| EmilienM | the question is, is it deployed properly | 19:45 |
| *** yolanda has joined #tripleo | 19:45 | |
| *** waleedm has quit IRC | 19:45 | |
| stevebaker | morning | 19:46 |
| *** moshele has quit IRC | 19:49 | |
| EmilienM | hey stevebaker | 19:49 |
| EmilienM | alee: I can't help on that issue right now, as busy on other problems but please keep looking, otherwise we'll have to revert | 19:49 |
| *** paramite has quit IRC | 19:51 | |
| *** paramite_ has quit IRC | 19:51 | |
| alee | EmilienM, to be honest, I'm not sure where to look -- let me see if I can ping jaosorior - though its pretty late there | 19:52 |
| *** etingof has joined #tripleo | 19:57 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: DNM - WIP: add manila cephnfs to scenario004 https://review.openstack.org/544623 | 19:58 |
| *** jcoufal has joined #tripleo | 19:59 | |
| *** jcoufal_ has quit IRC | 19:59 | |
| openstackgerrit | Ben Nemec proposed openstack/puppet-tripleo master: Add support for Designate https://review.openstack.org/518983 | 19:59 |
| EmilienM | alee: in that case I'm proposing a revert | 20:00 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Designate Integration https://review.openstack.org/518998 | 20:00 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Containerized Designate https://review.openstack.org/535489 | 20:00 |
| alee | EmilienM, I'm chatting with jaosorior right now | 20:00 |
| *** jcoufal_ has joined #tripleo | 20:01 | |
| *** atoth has quit IRC | 20:01 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Revert "Add healthchecks for barbican containers" https://review.openstack.org/544626 | 20:01 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Revert "Add healthchecks for barbican containers" https://review.openstack.org/544626 | 20:01 |
| *** rfolco|rover is now known as rfolco|off | 20:01 | |
| jaosorior | EmilienM: this one merged already https://review.openstack.org/#/c/543473/ thought it would fix it | 20:02 |
| jaosorior | if it didn't, I honestly don't know how to make it work, and we should revert it. I'll ping paramite tomorrow and ask him to check it out. | 20:02 |
| EmilienM | looks like it didn't, the rdo-kolla-build job still failing | 20:02 |
| EmilienM | jaosorior: let me check again the latest run | 20:02 |
| EmilienM | give me one sec | 20:02 |
| EmilienM | https://review.rdoproject.org/jenkins/job/rdo-kolla-build-integration/2356/consoleFull | 20:03 |
| jaosorior | EmilienM: why didn't the tripleo-common patch fail in the gate? doesn't it rebuild the images? | 20:03 |
| EmilienM | jaosorior: because you approved a patch that failed a non voting job | 20:04 |
| EmilienM | we really need to be more careful | 20:04 |
| *** jcoufal has quit IRC | 20:04 | |
| alee | EmilienM, jaosorior gotta go pick up kids. back later if you need me for anything. | 20:04 |
| EmilienM | jaosorior: https://review.openstack.org/#/c/543456/ | 20:04 |
| jaosorior | EmilienM: shit | 20:04 |
| jaosorior | ok, lets revert it | 20:04 |
| EmilienM | rdo-kolla-build-integration was red | 20:05 |
| EmilienM | yes | 20:05 |
| EmilienM | we are making third party jobs voting from now | 20:05 |
| EmilienM | to avoid this situation | 20:05 |
| jaosorior | ok | 20:05 |
| jaosorior | good | 20:05 |
| EmilienM | you can see the error here: https://logs.rdoproject.org/30/498930/6/openstack-check/rdo-kolla-build-integration/Z096b27b8d4fa4f50b39760da60b3bfe2/kolla/logs/barbican-api.log | 20:06 |
| *** amoralej is now known as amoralej|off | 20:07 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add feature set to test public bond network isolation https://review.openstack.org/540152 | 20:07 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749576 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 20:10 |
| openstack | Launchpad bug 1749576 in tripleo "tripleo queens jobs failing on missing dlrn hash" [Critical,Triaged] | 20:10 |
| *** paramite has joined #tripleo | 20:12 | |
| *** paramite_ has joined #tripleo | 20:12 | |
| *** atoth has joined #tripleo | 20:12 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: Support separate oslo.messaging services for RPC and Notification https://review.openstack.org/507963 | 20:13 |
| *** lblanchard has quit IRC | 20:14 | |
| *** atoth has quit IRC | 20:14 | |
| *** karthiks has quit IRC | 20:15 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-common master: Add Designate containers https://review.openstack.org/544675 | 20:16 |
| *** paramite_ has quit IRC | 20:17 | |
| *** paramite has quit IRC | 20:17 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/os-net-config master: Update reno for stable/queens https://review.openstack.org/544677 | 20:19 |
| openstackgerrit | OpenStack Release Bot proposed openstack/paunch master: Update reno for stable/queens https://review.openstack.org/544681 | 20:20 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Revert "Default to docker.io instead of trunk.registry.rdoproject.org" https://review.openstack.org/544378 | 20:21 |
| openstackgerrit | Merged openstack/os-net-config master: Update reno for stable/queens https://review.openstack.org/544331 | 20:21 |
| EmilienM | ok good https://review.openstack.org/544378 landed | 20:22 |
| EmilienM | weshay: hopefully next round of periodic job we get promotion | 20:22 |
| *** ebarrera_ has joined #tripleo | 20:23 | |
| *** thrash|f00dz is now known as thrash | 20:23 | |
| openstackgerrit | Merged openstack/paunch master: Update reno for stable/queens https://review.openstack.org/544337 | 20:26 |
| dpeacock | dsneddon: Thank you :-) | 20:26 |
| *** etingof has quit IRC | 20:28 | |
| *** etingof has joined #tripleo | 20:30 | |
| *** karthiks has joined #tripleo | 20:32 | |
| *** jcoufal has joined #tripleo | 20:46 | |
| *** jcoufal_ has quit IRC | 20:49 | |
| *** Goneri has quit IRC | 20:50 | |
| *** cdearborn has quit IRC | 20:51 | |
| *** gvrangan has quit IRC | 20:54 | |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade master: Fix workarounds template before ffu upgrade steps playbook https://review.openstack.org/544690 | 20:56 |
| *** salmankhan has joined #tripleo | 20:57 | |
| *** salmankhan has quit IRC | 21:01 | |
| *** salmankhan has joined #tripleo | 21:01 | |
| *** dsariel_ has joined #tripleo | 21:08 | |
| *** dsariel has joined #tripleo | 21:08 | |
| *** liverpooler has quit IRC | 21:09 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 21:10 |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| *** cdearborn has joined #tripleo | 21:11 | |
| *** dsneddon has quit IRC | 21:11 | |
| *** psachin has quit IRC | 21:12 | |
| openstackgerrit | Matt Young proposed openstack/tripleo-upgrade master: Add log artifacts for upgrade steps https://review.openstack.org/544696 | 21:15 |
| openstackgerrit | Matt Young proposed openstack/tripleo-upgrade master: Add log artifacts for upgrade steps https://review.openstack.org/544696 | 21:17 |
| openstackgerrit | Ihar Hrachyshka proposed openstack/tripleo-quickstart master: Added a troubleshooting tip to clean up ~/.quickstart https://review.openstack.org/537008 | 21:19 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add steps to allow downstream to use featuresets https://review.openstack.org/536452 | 21:24 |
| *** jbadiapa has quit IRC | 21:24 | |
| *** jcoufal_ has joined #tripleo | 21:26 | |
| *** pcaruana has quit IRC | 21:28 | |
| *** jcoufal has quit IRC | 21:29 | |
| *** jcoufal has joined #tripleo | 21:32 | |
| *** pchavva has quit IRC | 21:33 | |
| *** dsneddon has joined #tripleo | 21:34 | |
| *** jcoufal_ has quit IRC | 21:36 | |
| *** ansiwen has quit IRC | 21:39 | |
| *** jcoufal_ has joined #tripleo | 21:39 | |
| *** waleedm has joined #tripleo | 21:40 | |
| *** jcoufal has quit IRC | 21:41 | |
| *** raildo has quit IRC | 21:42 | |
| *** jcoufal has joined #tripleo | 21:44 | |
| *** jcoufal_ has quit IRC | 21:45 | |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates master: Unify the Cinder HA and non-HA docker configurations https://review.openstack.org/542425 | 21:46 |
| *** Goneri has joined #tripleo | 21:48 | |
| *** jcoufal_ has joined #tripleo | 21:48 | |
| *** waleedm has quit IRC | 21:48 | |
| *** jcoufal has quit IRC | 21:49 | |
| *** jcoufal has joined #tripleo | 21:51 | |
| *** jcoufal_ has quit IRC | 21:54 | |
| *** jcoufal_ has joined #tripleo | 21:55 | |
| EmilienM | weshay: I think I have an overcloud deployed with a containerized undercloud \o/ | 21:55 |
| EmilienM | but it's still running (doing the overcloud steps atm) | 21:55 |
| weshay | nice | 21:56 |
| EmilienM | weshay: I saw your -2 on my tqe patches, it's fine but at least please review my code | 21:57 |
| EmilienM | weshay: so once we cut RC1 and stuff we can land it quickly | 21:57 |
| *** jcoufal has quit IRC | 21:57 | |
| *** rcernin has joined #tripleo | 22:00 | |
| *** jcoufal has joined #tripleo | 22:00 | |
| *** jcoufal_ has quit IRC | 22:01 | |
| weshay | EmilienM, aye | 22:02 |
| weshay | EmilienM, I'm just trying to be careful :) | 22:02 |
| EmilienM | weshay: you just like -2 myself right? :P | 22:03 |
| weshay | EmilienM, I was going to write up an RFE bug to explain what ur up to.. just need a few | 22:03 |
| weshay | EmilienM, hell ya I do | 22:03 |
| weshay | EmilienM, f.. live for that | 22:03 |
| mwhahaha | we all do | 22:03 |
| weshay | anything to keep that plane on the ground.. you might kill someone :) | 22:03 |
| mwhahaha | it's ok, we removed the engine long ago | 22:04 |
| mwhahaha | it's now just an empty cardboard box | 22:04 |
| weshay | phew | 22:04 |
| mwhahaha | that we marked as "AIRPLANE" | 22:04 |
| weshay | lolz | 22:04 |
| mwhahaha | EmilienM makes *brrrrrrr* noises himself | 22:04 |
| weshay | lol.. best laugh all day | 22:04 |
| mwhahaha | https://www.hellowonderful.co/ckfinder/userfiles/images/Grantsolo_500px.jpg | 22:05 |
| *** jcoufal_ has joined #tripleo | 22:05 | |
| weshay | that even looks like him | 22:05 |
| slagle | huh. i thought emilien was younger than that | 22:06 |
| *** wolverineav has joined #tripleo | 22:06 | |
| *** jcoufal has quit IRC | 22:07 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 22:10 |
| EmilienM | lol | 22:10 |
| EmilienM | I love you guys | 22:10 |
| *** wolverineav has quit IRC | 22:10 | |
| *** jcoufal has joined #tripleo | 22:10 | |
| *** jcoufal_ has quit IRC | 22:11 | |
| *** jcoufal_ has joined #tripleo | 22:14 | |
| *** jcoufal has quit IRC | 22:15 | |
| *** jcoufal has joined #tripleo | 22:16 | |
| *** waleedm has joined #tripleo | 22:16 | |
| *** salmankhan has quit IRC | 22:18 | |
| *** jcoufal_ has quit IRC | 22:20 | |
| *** abishop has quit IRC | 22:20 | |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Refactor building undercloud env files https://review.openstack.org/544715 | 22:22 |
| *** jcoufal_ has joined #tripleo | 22:23 | |
| *** jcoufal has quit IRC | 22:24 | |
| *** dsariel has quit IRC | 22:30 | |
| *** dsariel_ has quit IRC | 22:30 | |
| *** threestrands has joined #tripleo | 22:31 | |
| *** threestrands has quit IRC | 22:31 | |
| *** threestrands has joined #tripleo | 22:31 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: set the openstack release dynamically https://review.openstack.org/543031 | 22:31 |
| *** waleedm has quit IRC | 22:31 | |
| sshnaidm|off | don't we have queens branch for instack-undercloud? | 22:33 |
| *** pmannidi has joined #tripleo | 22:33 | |
| sshnaidm|off | need to backport this to unblock promotions https://review.openstack.org/#/c/543916/ | 22:34 |
| sshnaidm|off | EmilienM, weshay ^^ | 22:34 |
| *** salmankhan has joined #tripleo | 22:34 | |
| EmilienM | sshnaidm|off: we dropped branches, created by error | 22:34 |
| EmilienM | no branch now, probably next weekish | 22:35 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: set the openstack release dynamically https://review.openstack.org/543031 | 22:36 |
| weshay | EmilienM, wait.. you are talking about check | 22:37 |
| weshay | and gate | 22:37 |
| weshay | sshnaidm|off, is talking about queens promotion.... so we do it in stable/queens | 22:37 |
| EmilienM | I'm talking about promotion | 22:37 |
| weshay | so is that going to be built for queens? | 22:38 |
| weshay | I guess so??? | 22:38 |
| EmilienM | need to drop off 20 min | 22:38 |
| weshay | EmilienM, I guess if you haven't branched it will be | 22:38 |
| EmilienM | it should take from master | 22:38 |
| weshay | EmilienM, thanks | 22:38 |
| EmilienM | it won't be branched | 22:38 |
| EmilienM | before next week at least | 22:38 |
| EmilienM | so the queens job should take consistent from master? | 22:39 |
| EmilienM | brb | 22:39 |
| weshay | dmsimard, ^ | 22:39 |
| weshay | will https://review.openstack.org/#/c/543916/ be built by dlrn for queens and master? | 22:39 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: set the openstack release dynamically https://review.openstack.org/543031 | 22:44 |
| *** bfournie has quit IRC | 22:45 | |
| *** artom_ has joined #tripleo | 22:48 | |
| *** maeca has quit IRC | 22:50 | |
| *** liverpooler has joined #tripleo | 22:51 | |
| *** itlinux has quit IRC | 22:53 | |
| *** artom has quit IRC | 22:53 | |
| *** jcoufal has joined #tripleo | 22:53 | |
| *** jcoufal_ has quit IRC | 22:53 | |
| *** pradk has quit IRC | 22:59 | |
| *** cdearborn has quit IRC | 23:00 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: clean up image upload command https://review.openstack.org/544727 | 23:01 |
| *** chlong has quit IRC | 23:02 | |
| *** jmelvin has quit IRC | 23:03 | |
| *** apevec has quit IRC | 23:03 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1749335 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1749335 in tripleo "Failing to build barbican containers in rdo-kolla-build-integration" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 23:10 |
| dmsimard | weshay: I dunno ? | 23:11 |
| dmsimard | weshay: depends on the rdoinfo config | 23:11 |
| dmsimard | weshay: I think it falls back to master is stable/queens doesn't exist | 23:11 |
| *** akrivoka has quit IRC | 23:11 | |
| *** liverpooler has quit IRC | 23:11 | |
| dmsimard | weshay: actually it looks like every release builds from master ? https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml#L2434-L2453 | 23:12 |
| weshay | dmsimard, hrm... we're in a very grey area.. we have a queens pipeline that is testing w/ | 23:13 |
| dmsimard | weshay: I'm not sure how to interpret that rdoinfo config | 23:13 |
| weshay | baseurl=http://mirror.regionone.rdo-cloud.rdoproject.org:8080/rdo/centos7-queens/2c/37/2c37a470bd5672805bd2ae7fc6e1a5852f32e9d2_8a3e746b | 23:13 |
| dmsimard | weshay: best would be to check.. like.. https://trunk.rdoproject.org/centos7-queens/report.html and https://trunk.rdoproject.org/centos7-master/report.html to see where it built | 23:14 |
| weshay | ya.. doing so now | 23:14 |
| dmsimard | weshay: master has it | 23:15 |
| dmsimard | weshay: queens has it too | 23:15 |
| dmsimard | weshay: it's 1b914f86082c6d3401075056c8a03db269be1e44 | 23:15 |
| weshay | hrm.. | 23:16 |
| * weshay looking at https://trunk.rdoproject.org/centos7-queens/report.html | 23:16 | |
| weshay | 5dd4e93f5965e133bbf988573298b6b5cb13fb0d | 23:16 |
| weshay | is the latest instack-undercloud which is old | 23:16 |
| dmsimard | weshay: do a ctrl+f for 1b914f86082c6d3401075056c8a03db269be1e44 | 23:16 |
| dmsimard | and click on the link | 23:17 |
| dmsimard | http://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=1b914f86082c6d3401075056c8a03db269be1e44 | 23:17 |
| weshay | phew | 23:18 |
| *** Jimbagwell has quit IRC | 23:18 | |
| weshay | :) | 23:18 |
| *** jcoufal_ has joined #tripleo | 23:20 | |
| *** jcoufal has quit IRC | 23:21 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: prep-containers: include containerized undercloud bits https://review.openstack.org/543024 | 23:21 |
| EmilienM | mwhahaha: https://review.openstack.org/#/c/544626/ brings back rdo-kolla-build-integration | 23:22 |
| EmilienM | jaosorior: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht is still red, any progress so far? | 23:22 |
| mwhahaha | EmilienM: k why does 035 fail on that review? | 23:22 |
| EmilienM | oh https://review.openstack.org/#/c/544229/ still in gate, jaosorior : nevermind | 23:23 |
| EmilienM | mwhahaha: I was looking at it | 23:23 |
| EmilienM | 2018-02-14 21:57:47 | "/usr/bin/docker-current: Error response from daemon: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:258: applying cgroup configuration for process caused \\\\\\\"write /sys/fs/cgroup/pids/system.slice/docker-cb06bfe138359b3eeea3785422ac7862577e47594af15435f4e67675da7a590e.scope/cgroup.procs: no | 23:23 |
| EmilienM | such device\\\\\\\"\\\"\\n\".", | 23:23 |
| EmilienM | https://logs.rdoproject.org/26/544626/2/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/Zef2c99a3902b40899adfab90f9688e66/undercloud/home/jenkins/overcloud_deploy.log.gz | 23:24 |
| EmilienM | again this one | 23:24 |
| EmilienM | so not related | 23:24 |
| mwhahaha | k | 23:24 |
| *** tcw has quit IRC | 23:24 | |
| *** jcoufal has joined #tripleo | 23:27 | |
| *** jcoufal_ has quit IRC | 23:28 | |
| *** jcoufal__ has joined #tripleo | 23:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Undercloud: fix stackrc TLS URL detection https://review.openstack.org/535480 | 23:31 |
| *** jcoufal has quit IRC | 23:32 | |
| *** salmankhan has quit IRC | 23:33 | |
| *** rlandy is now known as rlandy|bbl | 23:41 | |
| *** dhill__ has quit IRC | 23:45 | |
| *** larsks has quit IRC | 23:51 | |
| *** jcoufal has joined #tripleo | 23:55 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Fix PublicVirtualFixedIPs in envs https://review.openstack.org/544744 | 23:57 |
| *** jcoufal__ has quit IRC | 23:57 | |
| *** bfournie has joined #tripleo | 23:58 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Undercloud: support for external VIP for SSL https://review.openstack.org/535479 | 23:58 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Undercloud: support for external VIP for SSL https://review.openstack.org/535479 | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
