| mwhahaha | that would seem to be the case | 00:00 |
|---|---|---|
| mwhahaha | which is why i asked about tomorrow :D | 00:00 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 00:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| weshay | somebody is moving cards on the escalation board w/o updating bugs | 00:13 |
| mwhahaha | not it | 00:14 |
| rlandy|ruck | bugs should be updated - when I moved the cards | 00:19 |
| rlandy|ruck | I am the guilty card mover - but the bugs are updated | 00:20 |
| *** ihrachys has quit IRC | 00:26 | |
| weshay | rlandy|ruck, it was apevec not you :) | 00:34 |
| *** salmankhan has joined #tripleo | 00:53 | |
| *** tzumainn has quit IRC | 00:54 | |
| *** chkumar246_ has joined #tripleo | 00:59 | |
| *** chkumar246 has quit IRC | 01:01 | |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 01:06 | |
| *** cshastri has joined #tripleo | 01:07 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 01:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| *** catintheroof has quit IRC | 01:10 | |
| *** catintheroof has joined #tripleo | 01:11 | |
| *** liverpooler has joined #tripleo | 01:13 | |
| EmilienM | rlandy|ruck|bbl: yes | 01:14 |
| *** gkadam has quit IRC | 01:15 | |
| *** catintheroof has quit IRC | 01:15 | |
| *** gkadam has joined #tripleo | 01:16 | |
| *** ecerquei has joined #tripleo | 01:24 | |
| *** ManoX has quit IRC | 01:37 | |
| *** liverpooler has quit IRC | 01:42 | |
| *** tonyb has quit IRC | 01:47 | |
| *** tonyb has joined #tripleo | 01:49 | |
| *** gkadam has quit IRC | 01:56 | |
| *** etingof has quit IRC | 01:57 | |
| *** salmankhan has quit IRC | 01:58 | |
| *** gkadam has joined #tripleo | 01:59 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 02:10 |
| *** etingof has joined #tripleo | 02:13 | |
| *** ecerquei has quit IRC | 02:18 | |
| *** atoth has quit IRC | 02:18 | |
| *** ecerquei has joined #tripleo | 02:18 | |
| *** yamahata has quit IRC | 02:30 | |
| *** owalsh_ has joined #tripleo | 02:42 | |
| *** owalsh has quit IRC | 02:45 | |
| *** karthiks has joined #tripleo | 02:49 | |
| *** bfournie has joined #tripleo | 02:53 | |
| *** bfournie has quit IRC | 02:54 | |
| *** gkadam has quit IRC | 03:07 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 03:10 |
| *** chlong has joined #tripleo | 03:15 | |
| *** stendulker has joined #tripleo | 03:16 | |
| *** dpawar has joined #tripleo | 03:22 | |
| *** dpawar has quit IRC | 03:22 | |
| *** dpawar has joined #tripleo | 03:23 | |
| *** matbu has quit IRC | 03:28 | |
| *** rhallisey has quit IRC | 03:30 | |
| *** psahoo has joined #tripleo | 03:32 | |
| *** ramishra has joined #tripleo | 03:33 | |
| *** rlandy|ruck|bbl is now known as rlandy|ruck | 03:38 | |
| *** matbu has joined #tripleo | 03:40 | |
| *** shreshtha has joined #tripleo | 03:45 | |
| *** ykarel has joined #tripleo | 03:49 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Update AnsibleGenerateInventoryAction to match new mistral interfaces https://review.openstack.org/530783 | 03:53 |
| *** pdeore_ has joined #tripleo | 03:58 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 04:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| *** udesale has joined #tripleo | 04:15 | |
| *** ecerquei has quit IRC | 04:20 | |
| *** agopi has joined #tripleo | 04:27 | |
| *** rlandy|ruck has quit IRC | 04:29 | |
| *** agopi has quit IRC | 04:32 | |
| *** agopi has joined #tripleo | 04:32 | |
| *** dpawar has quit IRC | 04:42 | |
| *** dpawar has joined #tripleo | 04:43 | |
| *** ratailor has joined #tripleo | 04:45 | |
| *** dpawar has quit IRC | 04:47 | |
| *** links has joined #tripleo | 04:48 | |
| *** psachin has joined #tripleo | 04:56 | |
| *** links has quit IRC | 04:58 | |
| *** rwsu has quit IRC | 04:58 | |
| *** chkumar246_ has quit IRC | 05:01 | |
| *** links has joined #tripleo | 05:03 | |
| ykarel | EmilienM, hi | 05:07 |
| ykarel | EmilienM, i got what was the issue i mentioned yesterday ragarding puppet-archive | 05:07 |
| EmilienM | ykarel: what's up? | 05:08 |
| ykarel | in stable/pike there is no archive added in Puppetfile | 05:08 |
| EmilienM | in p-o-i ? | 05:09 |
| ykarel | yes | 05:09 |
| *** pgadiya has joined #tripleo | 05:09 | |
| EmilienM | I'm not sure why we would need it | 05:09 |
| ykarel | because of puppet-rabbitmq 8.0.1 | 05:09 |
| EmilienM | but I might have missed something | 05:09 |
| ykarel | EmilienM, https://review.rdoproject.org/r/#/c/10941/ | 05:10 |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 05:10 |
| EmilienM | ykarel: but Puppetfile is when modules are deployed from source, not from RDO repos, right? | 05:10 |
| ykarel | EmilienM, it should be, but i noticed that Puppetfile is used even when modules are installed from rpm | 05:12 |
| *** ratailor has quit IRC | 05:12 | |
| *** ratailor has joined #tripleo | 05:12 | |
| ykarel | because different versions of modules exist at both places: /usr/share/openstack-puppet/modules and /etc/puppet/modules | 05:12 |
| ykarel | at /etc/puppet/modules (from Puppetfile) and at /usr/share/openstack-puppet/modules from RDO rpms | 05:13 |
| EmilienM | ykarel: what? | 05:13 |
| EmilienM | it's a bug | 05:13 |
| EmilienM | it shouldn't be the case | 05:13 |
| ykarel | EmilienM, hmm it looks a bug | 05:14 |
| ykarel | EmilienM, Ok i will find why Puppetfile is used, but if we use modulepath /usr/share/openstack-puppet/modules, it requires /etc/puppet/environments directory which doesn't exist in poi deployment | 05:15 |
| ykarel | This ^^ is what i noticed till now | 05:17 |
| *** dpawar has joined #tripleo | 05:20 | |
| *** pdeore_ has quit IRC | 05:20 | |
| *** pdeore has joined #tripleo | 05:20 | |
| EmilienM | ykarel: in weirdo jobs, Puppetfile shouldn't be used | 05:21 |
| EmilienM | but instead, rpms | 05:21 |
| EmilienM | the parameter is MANAGE_PUPPET_MODULES | 05:22 |
| ykarel | EmilienM, Ok, hmm was looking at that only | 05:22 |
| ykarel | will try local reproducer for it and check the issue i mentioned regarding /etc/puppet/environments | 05:24 |
| EmilienM | ok | 05:24 |
| EmilienM | ykarel: I go to bed now. Anything I can help before? | 05:30 |
| ykarel | EmilienM, you rest, i am looking at it | 05:31 |
| EmilienM | :) | 05:31 |
| * EmilienM off | 05:31 | |
| *** daidv has quit IRC | 05:34 | |
| *** daidv has joined #tripleo | 05:35 | |
| *** anilvenkata has joined #tripleo | 05:40 | |
| *** dmsimard has quit IRC | 05:42 | |
| *** dmsimard has joined #tripleo | 05:43 | |
| *** daidv has quit IRC | 05:45 | |
| *** daidv has joined #tripleo | 05:46 | |
| Tengu | hello there | 05:47 |
| jaosorior | sup | 05:48 |
| Tengu | hard to go back to work ;) | 05:48 |
| *** artom_ has quit IRC | 05:48 | |
| Tengu | what about you? :] | 05:48 |
| jaosorior | it was hard yesterday | 05:49 |
| jaosorior | not as bad today | 05:49 |
| Tengu | ^^ | 05:49 |
| jaosorior | it sure helps having a shorter week | 05:49 |
| Tengu | worst is: raining as hell, and had to go to the office, meaning: motorbike. | 05:49 |
| Tengu | should have taken the week off in fact: I was on call last week and, of course, there were emergencies :] | 05:50 |
| Tengu | so long for the holidays. | 05:50 |
| jaosorior | daaaamn | 05:50 |
| *** akane has joined #tripleo | 05:50 | |
| jaosorior | one of the perks of working from home :D | 05:50 |
| Tengu | ^^ | 05:52 |
| *** jaganathan has joined #tripleo | 05:53 | |
| Tengu | who knows… might get some good news in the following days. or weeks. :] | 05:53 |
| Tengu | aanyway. first task this year: get a working tripleo deploy after the huge bummer I hit a few days ago -.-' | 05:53 |
| Tengu | and that one will be hard. | 05:53 |
| jaosorior | aaaah | 05:55 |
| jaosorior | that still hurts when getting reminded of | 05:55 |
| Tengu | so true… | 05:56 |
| Tengu | but having a 1-compute deploy with three computes sleeping isn't satisfying. | 05:56 |
| Tengu | and I'm pretty sure my manager will have the same thoughts ;) | 05:57 |
| Tengu | but first: tea pot. | 05:58 |
| Tengu | the weird thing with the current status: compute-1 doesn't seem to be taken into account, although no tripleo resource should point to it and flag it as deployed… | 06:07 |
| Tengu | I bet my issue is caused by that state. | 06:07 |
| jaosorior | Tengu: did you get a chance to ask shardy about it? sounds like something he might know about it | 06:08 |
| jaosorior | * know about | 06:08 |
| Tengu | will try to catch him today :). | 06:08 |
| Tengu | I think he's in England or the like right? so more or less same timezone | 06:08 |
| jaosorior | right | 06:09 |
| jaosorior | he might log in in a couple of hours | 06:10 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 06:10 |
| *** masco has joined #tripleo | 06:11 | |
| *** matbu has quit IRC | 06:12 | |
| Tengu | good then. I'll try to poke-poke the stack in the meanwhile. | 06:14 |
| * Tengu pictures a kid with a stick poking a huge snake | 06:15 | |
| Tengu | "is it alive? will it bite? what if I…" :D | 06:15 |
| *** oidgar has joined #tripleo | 06:15 | |
| jaosorior | lol | 06:15 |
| *** cshastri has quit IRC | 06:18 | |
| *** nyechiel_ has joined #tripleo | 06:30 | |
| *** cshastri has joined #tripleo | 06:33 | |
| *** oidgar has quit IRC | 06:37 | |
| *** skramaja has joined #tripleo | 06:41 | |
| *** moshele has joined #tripleo | 06:44 | |
| *** aufi has joined #tripleo | 06:52 | |
| *** marios has joined #tripleo | 06:53 | |
| *** pmannidi has quit IRC | 06:58 | |
| *** matbu has joined #tripleo | 07:03 | |
| *** pmannidi has joined #tripleo | 07:08 | |
| *** pcaruana has joined #tripleo | 07:09 | |
| *** suuuper has joined #tripleo | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 07:10 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui stable/ocata: Imported Translations from Zanata https://review.openstack.org/530852 | 07:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui stable/pike: Imported Translations from Zanata https://review.openstack.org/530853 | 07:22 |
| *** pmannidi has quit IRC | 07:24 | |
| *** rcernin has quit IRC | 07:25 | |
| openstackgerrit | Arx Cruz proposed openstack-infra/tripleo-ci master: Fix stackviz data https://review.openstack.org/525603 | 07:34 |
| *** cshastri has quit IRC | 07:34 | |
| *** cshastri has joined #tripleo | 07:35 | |
| openstackgerrit | waleed mousa proposed openstack/tripleo-heat-templates master: Adding support for role parameters in "environment_generator.py" https://review.openstack.org/529422 | 07:37 |
| *** Guest87 has joined #tripleo | 07:39 | |
| *** mdnadeem has joined #tripleo | 07:46 | |
| *** florianf has joined #tripleo | 07:47 | |
| *** cylopez has joined #tripleo | 07:49 | |
| *** jtomasek has joined #tripleo | 07:50 | |
| *** jtomasek has quit IRC | 07:50 | |
| *** jtomasek has joined #tripleo | 07:51 | |
| *** cylopez has left #tripleo | 07:54 | |
| *** agurenko has joined #tripleo | 07:56 | |
| *** agurenko has quit IRC | 07:58 | |
| *** dciabrin has joined #tripleo | 07:58 | |
| openstackgerrit | Juan Badia Payno proposed openstack/puppet-tripleo master: Fluentd: Transform path by service https://review.openstack.org/530780 | 08:00 |
| *** agurenko has joined #tripleo | 08:01 | |
| *** Guest87 has quit IRC | 08:03 | |
| *** tesseract has joined #tripleo | 08:05 | |
| *** karthiks has quit IRC | 08:07 | |
| *** fragatina has joined #tripleo | 08:07 | |
| *** ffiore has joined #tripleo | 08:08 | |
| *** bogdando has joined #tripleo | 08:09 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 08:10 |
| *** medy has joined #tripleo | 08:15 | |
| *** medy has quit IRC | 08:16 | |
| *** apetrich has quit IRC | 08:16 | |
| *** medy has joined #tripleo | 08:16 | |
| *** apetrich has joined #tripleo | 08:16 | |
| *** medy has quit IRC | 08:18 | |
| *** medy has joined #tripleo | 08:18 | |
| *** medy has quit IRC | 08:19 | |
| *** medy has joined #tripleo | 08:19 | |
| *** medy has quit IRC | 08:21 | |
| *** medy has joined #tripleo | 08:21 | |
| *** medy has quit IRC | 08:22 | |
| *** medy has joined #tripleo | 08:22 | |
| *** artom_ has joined #tripleo | 08:22 | |
| *** medy has quit IRC | 08:24 | |
| *** mrunge has quit IRC | 08:24 | |
| *** medy has joined #tripleo | 08:24 | |
| *** medy has quit IRC | 08:25 | |
| *** medy has joined #tripleo | 08:25 | |
| *** medy has quit IRC | 08:27 | |
| *** medy has joined #tripleo | 08:27 | |
| *** medy has quit IRC | 08:28 | |
| *** medy has joined #tripleo | 08:28 | |
| *** medy has quit IRC | 08:29 | |
| *** moshele has quit IRC | 08:35 | |
| *** moshele has joined #tripleo | 08:37 | |
| *** fragatina has quit IRC | 08:38 | |
| *** mrunge has joined #tripleo | 08:38 | |
| *** tvignaud has quit IRC | 08:44 | |
| *** artom_ has quit IRC | 08:45 | |
| *** fragatina has joined #tripleo | 08:47 | |
| *** moshele has quit IRC | 08:50 | |
| *** fragatina has quit IRC | 08:50 | |
| *** moshele has joined #tripleo | 08:50 | |
| *** fragatina has joined #tripleo | 08:51 | |
| *** karthiks has joined #tripleo | 08:54 | |
| *** Guest87 has joined #tripleo | 09:00 | |
| apetrich | jkilpatr, Hi, I'm looking at https://bugs.launchpad.net/tripleo/+bug/1649350 could you ping me when you are here? | 09:00 |
| openstack | Launchpad bug 1649350 in tripleo "Introspection fails with 'mistral engine error' on Ocata if an 'ironic node-update' is done before introspection runs " [High,In progress] - Assigned to Adriano Petrich (apetrich) | 09:00 |
| *** Guest87 has quit IRC | 09:03 | |
| *** bogdando has quit IRC | 09:04 | |
| *** holser__ has joined #tripleo | 09:04 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient master: Handle user-provided TLS certificate/key for the undercloud https://review.openstack.org/530737 | 09:05 |
| *** jpich has joined #tripleo | 09:05 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 09:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| *** bogdando has joined #tripleo | 09:16 | |
| *** derekh has joined #tripleo | 09:17 | |
| *** fragatina has quit IRC | 09:17 | |
| *** fragatina has joined #tripleo | 09:18 | |
| openstackgerrit | Merged openstack/tripleo-ui stable/ocata: Imported Translations from Zanata https://review.openstack.org/530852 | 09:20 |
| openstackgerrit | Merged openstack/tripleo-ui stable/pike: Imported Translations from Zanata https://review.openstack.org/530853 | 09:20 |
| *** cshastri has quit IRC | 09:21 | |
| *** cshastri has joined #tripleo | 09:21 | |
| *** mcornea has joined #tripleo | 09:23 | |
| *** dbecker has joined #tripleo | 09:25 | |
| *** shardy has joined #tripleo | 09:27 | |
| *** d0ugal has quit IRC | 09:27 | |
| *** d0ugal has joined #tripleo | 09:28 | |
| *** d0ugal has quit IRC | 09:28 | |
| *** d0ugal has joined #tripleo | 09:28 | |
| *** tvignaud has joined #tripleo | 09:30 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-common master: Update AnsibleGenerateInventoryAction to match new mistral interfaces https://review.openstack.org/530783 | 09:30 |
| *** ccamacho has joined #tripleo | 09:32 | |
| Tengu | shardy: hello! can I have a bit of your time regarding a failed overcloud stack update? I'm really stuck, and can't manage to set it back in a stable state :(. | 09:32 |
| shardy | Tengu: Hi, sure, what's up? | 09:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Select Roles feature https://review.openstack.org/512266 | 09:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: [WIP] Submit select roles https://review.openstack.org/521510 | 09:34 |
| Tengu | shardy: you remember the issue I opened with the missing validation for the host mapping hash? well, I hit this issue of course (otherwise I wouldn't know of that missing validation ;)). | 09:34 |
| jaganathan | d0ugal, please look into https://review.openstack.org/#/c/527655/ | 09:35 |
| Tengu | shardy: current situation: I tried to add two more computes in a 2-compute deploy - unfortunately I made two typo in the mapping, resulting in "nova-compute-2" and -3 pointing to "compte-1". compute-1 was already deployed. | 09:35 |
| Tengu | shardy: in order to clean all that up, I emptied the "compute-1" node, moving all the instances to compute-0 - I then dropped compute-1 in order to ensure all related resources are cleaned up. But apparently, resources aren't all cleaned up :(. | 09:37 |
| *** ykarel_ has joined #tripleo | 09:37 | |
| Tengu | shardy: apparently, there is still pointers to compute-1 in, at least, the ssh "known_hosts" management, probably seeking the public key of the host in order to add it in /etc/ssh/ssh_known_hosts on all the nodes. | 09:37 |
| Tengu | shardy: currently, the sub-stack named "overcloud-ComputeHostsDeployment-7jdtrovjf4mg" is failing, no matter what I do. | 09:38 |
| *** mdnadeem has quit IRC | 09:39 | |
| Tengu | shardy: the stack is described as follow: http://paste.openstack.org/show/637239/ | 09:39 |
| *** ykarel has quit IRC | 09:39 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Update parameters updating https://review.openstack.org/525672 | 09:40 |
| shardy | Tengu: how did you delete compute-1? | 09:40 |
| *** lucas-afk is now known as lucasagomes | 09:40 | |
| Tengu | shardy: following the doc - 2s, I made a quick script for that. | 09:40 |
| Tengu | shardy: http://paste.openstack.org/show/637244/ | 09:40 |
| shardy | Tengu: there is a defined workflow that uses openstack overcloud node delete, you must not delete anything directly via nova/ironic | 09:40 |
| *** ykarel_ is now known as ykarel | 09:40 | |
| shardy | sec looking | 09:41 |
| Tengu | shardy: the $ENVIRONMENTS lists all the environment files I have included with the deploy part. | 09:41 |
| shardy | Tengu: hmm, and that completed without errors? | 09:41 |
| Tengu | shardy: yep :/. | 09:41 |
| Tengu | at least I don't remember seeing errors, else I wouldn't have gone further (that was a week before Christmas) | 09:42 |
| Tengu | note to self: replace the "stack" shell with something that logs all inputs AND outputs | 09:42 |
| shardy | Tengu: ackm yeah this looks Ok, but I've seen similar problems when folks do a nova delete on a node vs a overcloud node delete | 09:42 |
| shardy | Tengu: what does openstack stack resource list overcloud-ComputeHostsDeployment-7jdtrovjf4mg show? | 09:44 |
| shardy | is it still showing a resource for the removed node? | 09:44 |
| Tengu | shardy: hmm ok. I was sick, maybe I did funny things after all. I really can't remember if I ended with some other commands in order to clean the node. | 09:44 |
| *** ameeda has quit IRC | 09:44 | |
| Tengu | shardy: nope, it doesn't. Funnily, it shows a CREATE for the already-installed node: http://paste.openstack.org/show/637243/ | 09:44 |
| shardy | Tengu: Ok, well if you did a nova delete, that would explain it - the node is gone so heat hangs waiting for response from SoftwareDeployments that can never complete | 09:45 |
| Tengu | shardy: showing the only resource in the stack: http://paste.openstack.org/show/637251/ | 09:45 |
| shardy | Tengu: ack, so what does ps ax | grep heat on the node 0 show? | 09:45 |
| shardy | is there a heat hook running, or any error trying to signal completion back to the undercloud? | 09:46 |
| Tengu | 2s, connecting. | 09:46 |
| Tengu | shardy: nothing in `ps fax | grep heat' | 09:46 |
| shardy | Tengu: Ok I'd look at /var/log/messages or journalctl for os-collect-config, to see that it got the HostsDeployment and tried to run it | 09:47 |
| shardy | also check the network is all OK and can reach the undercloud | 09:47 |
| Tengu | shardy: network is OK. I grep os-collect-config in the logs. | 09:48 |
| Tengu | grmbl. how does -S takes date now… | 09:49 |
| Tengu | ah. | 09:49 |
| Tengu | shardy: humpf. nothing in journalctl :( | 09:50 |
| *** mdnadeem has joined #tripleo | 09:52 | |
| Tengu | shardy: ah… rsyslog is dead. That won't help. | 09:54 |
| shardy | Tengu: ah, Ok here are some notes which may help http://paste.openstack.org/show/637261/ | 09:55 |
| shardy | Tengu: that shows how to trace the software deployment and software config IDs on the node, to show the script was downloaded, and that it signalled back (or in your case perhaps didn't) | 09:56 |
| Tengu | shardy: oh, didn't kow the "software" subcommand. does it talk to mistral? or maybe heat? | 09:57 |
| shardy | Tengu: it lists the heat SoftwareDeployment resources | 09:57 |
| Tengu | hmm ok. so talking to heat directly. | 09:58 |
| Tengu | got an "IN_PROGRESS" for the software deployment show. | 09:58 |
| Tengu | as well as status_reason | Deploy data available | 09:58 |
| shardy | Tengu: yep there's a (heat) API behind the OS::Heat::StructuredDeployment/OS::Heat::SoftwareDeployment resources in the template | 09:58 |
| ccamacho | hey shardy :) happy new year | 09:59 |
| shardy | ccamacho: hey, happy new year to you to! :) | 09:59 |
| shardy | too I mean :) | 09:59 |
| * shardy needs coffee | 09:59 | |
| Tengu | shardy: ok, and it didn't seem to get the script. that might explain. | 09:59 |
| Tengu | the config_id doesn't match any file in heat cache. | 09:59 |
| shardy | Tengu: Ok, I'd perhaps try running os-collect-config manually and/or check it's not stuck running already | 10:00 |
| Tengu | shardy: 7953 ? Ss 98:36 /usr/bin/python /usr/bin/os-collect-config | 10:00 |
| Tengu | seems to be stuck. | 10:00 |
| ccamacho | quick question shardy, I created some cheatsheets for last year fostdem/devconf but they are now all deprecated, I did this yesterday https://github.com/ccamacho/tripleo-graphics/blob/master/cheatsheets/latest_style/WIP-01-tripleo-cheatsheet-deploying-tripleo.pdf but on step 04 im not sure the simplest command to deploy oooq with containers and HA, when having some cycles do you mind to check it briefly? | 10:00 |
| shardy | Tengu: ack, Ok that sounds suspicious, I'd check /var/log/messages carefully to see the last thing it ran | 10:00 |
| Tengu | shardy: may I kill it and re-run the deploy? well, it's not yet timed out, but will eventully. | 10:00 |
| shardy | Tengu: sec, it's worth figuring out what it's stuck on (pstree maybe?) | 10:01 |
| Tengu | shardy: /var/log/messages is empty, probably due to some issue with rsyslog.service… that wouldn't be hard enough. | 10:01 |
| shardy | but yes, you can kill it, then re-run manually, sec I'll do an example | 10:01 |
| *** numans is now known as numans_afk | 10:02 | |
| Tengu | shardy: weird, pstree doesn't show it up. | 10:02 |
| Tengu | shardy: and ps doesn't see any sub-task, and strace -p <pid> is just stuck. | 10:03 |
| shardy | Tengu: is it stuck trying to pull data from swift on the undercloud? | 10:03 |
| Tengu | shardy: hard to say :/. I see this kind of line in the strace: "write(2, "Command failed, will not cache n"..., 117) = -1 EPIPE (Broken pipe)" | 10:04 |
| shardy | Tengu: you can check by looking in /etc/os-collect-config.conf, then copy the metadata_url then do curl "<metadata_url>" | 10:04 |
| shardy | the quotes are needed | 10:04 |
| Tengu | shardy: curl goes through | 10:04 |
| Tengu | yep, quotes are of course needed due to params. | 10:05 |
| shardy | Yeah, easy to forget tho :) | 10:05 |
| Tengu | ^^ | 10:06 |
| shardy | Tengu: Ok so you can kill the os-collect-config process, then temporarily systemctl stop os-collect-config, then run: | 10:07 |
| shardy | os-collect-config --config-file /etc/os-collect-config.conf --debug --one-time --force | 10:07 |
| shardy | that may expose where/why it's getting stuck | 10:07 |
| Tengu | hmm ok. 2s | 10:09 |
| *** pcaruana has quit IRC | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 10:10 |
| Tengu | shardy: echo $? returns "0" - checking the output itself. | 10:10 |
| *** pcaruana has joined #tripleo | 10:10 | |
| Tengu | shardy: duh. Apparently it actually downloaded the file now… | 10:11 |
| shardy | Tengu: Ok, you may see it run, then signal back to heat, then the IN_PROGRESS will become complete? | 10:12 |
| shardy | if so then restart os-collect-config and all should be good | 10:12 |
| Tengu | shardy: I owe you a couple of beers I think. Just re-started the os-collect-config. | 10:13 |
| shardy | Tengu: nice, glad that worked :) | 10:13 |
| Tengu | shardy: not yet finished, but at least the commands you gave me didn't return any error ;). will wait for the rest. | 10:14 |
| Tengu | shardy: now, the question: is there some doc about those debug steps, or do you just happen to know them because you're deeply involved in it? :) | 10:14 |
| Tengu | shardy: darn… 2018-01-03 10:13:47Z [overcloud]: UPDATE_FAILED Timed out | 10:15 |
| shardy | Tengu: ack, it'd be interesting to know why it got stuck in the first place - typical reasons are some kind of network outage during the stack update | 10:15 |
| shardy | Tengu: ack if it's been stuck for $hours you may need to re-run the update | 10:15 |
| Tengu | shardy: I doubt it (network issue) because all is on a LAN, in the very same rack… | 10:15 |
| shardy | Tengu: there are some docs but nothing with that level of detail, I just have done that debug sequence a few times myself | 10:15 |
| Tengu | shardy: I re-run the deploy now in order to ensure all is back to a working state, and will add back the nodes if it's OK. | 10:16 |
| shardy | I did do a debugging tripleo blog post a long time ago, but a revisited version and/or some docs would be a good idea | 10:16 |
| Tengu | shardy: +42 :). I would really appreciate some insight - even a brain dump without form would be great. | 10:16 |
| shardy | https://docs.openstack.org/tripleo-docs/latest/install/troubleshooting/troubleshooting.html | 10:16 |
| shardy | Tengu: there is that section - patches welcome :) | 10:17 |
| Tengu | :) | 10:17 |
| shardy | Tengu: ack I'll try to find some time to add to the docs | 10:17 |
| openstackgerrit | Daniel Alvarez proposed openstack/tripleo-quickstart master: Revert "fs030: disable tempest" https://review.openstack.org/530890 | 10:17 |
| Tengu | shardy: will check that. first of all: ensure deploy is working again, then I'll see if I can contribute for that doc. | 10:17 |
| Tengu | shardy: a big thanks in any cases for your time. | 10:17 |
| shardy | np, good luck :) | 10:17 |
| * Tengu crosses fingers | 10:17 | |
| * shardy goes to find coffee now | 10:18 | |
| dalvarez | EmilienM, https://review.openstack.org/530890 <- reenabling tempest on fs030, let's see if it's already fixed, it should | 10:18 |
| jaosorior | shardy: if I want to add a custom heat constraint for tripleo, which repo should I add that to? | 10:20 |
| jaosorior | tripleo-common? | 10:20 |
| *** owalsh has joined #tripleo | 10:21 | |
| *** ykarel is now known as ykarel|lunch | 10:21 | |
| *** owalsh_ has quit IRC | 10:22 | |
| *** etingof has quit IRC | 10:24 | |
| *** mrch has joined #tripleo | 10:30 | |
| shardy | jaosorior: Hmm, constraints aren't really pluggable via out-of-tree plugins like resources IIRC | 10:30 |
| shardy | jaosorior: what is the validation, could it go directly into heat or into tripleo-common as a pre-deploy validation? | 10:31 |
| jaosorior | shardy: oh, thought I could just add the validation and register it in the entry points | 10:31 |
| jaosorior | shardy: was actually thinking of coding something quick for Tengu's bug https://bugs.launchpad.net/tripleo/+bug/1740396 | 10:31 |
| openstack | Launchpad bug 1740396 in tripleo "Validation: missing check for HostnameMap" [Medium,Triaged] | 10:31 |
| shardy | jaosorior: it may be possible to register the constraint via stevedore, but AFAIK that's not a documented/stable interface (all the contraints inherit from constraints.BaseCustomConstraint inside heat) | 10:32 |
| Tengu | jaosorior: ♥ ;) | 10:33 |
| shardy | jaosorior: that seems like a reasonable generic validation for heat? | 10:33 |
| jaosorior | shardy: and I was just thinking of adding an entry in tripleo-common's setup.cfg, such as heat.constraints =\n tripleo.hostnamemap = someClass | 10:33 |
| Tengu | shardy: I think the heat stack is back on track. wow. That was challenging (at least for me ;)). | 10:34 |
| shardy | e.g add some map unique values constraint or something? | 10:34 |
| jaosorior | shardy: honestly, no. I think it's quite tripleO specific | 10:34 |
| shardy | jaosorior: how is saying a map should have unique values tripleo specific? | 10:34 |
| jaosorior | can't think of many uses for that | 10:34 |
| jaosorior | if you think it's generic, then sure, I won't complaign :D | 10:34 |
| shardy | I can think of plenty - any kind of look up/translation table | 10:34 |
| jaosorior | but ok | 10:35 |
| jaosorior | won't argue with that :D | 10:35 |
| jaosorior | makes life easier | 10:35 |
| Tengu | oh. rsyslog is back on track as well now. | 10:35 |
| jaosorior | shardy: I'll push a patch in a bit | 10:35 |
| shardy | jaosorior: ack cool - I think it might be possible to do a tripleo specific thing, but lets try for a generic addition to heat first :) | 10:35 |
| jaosorior | ack | 10:36 |
| Tengu | jaosorior: thank you for that (coming) job :). | 10:36 |
| Tengu | jaosorior: that will prevent other people to make the same mistake I made -.-' | 10:37 |
| jaosorior | Tengu: thanks for reporting | 10:37 |
| *** etingof has joined #tripleo | 10:37 | |
| Tengu | shardy: can confirm: the deploy steps goes further - that seem to have solved the issue. Will wait for the end, and then add back the three nodes (2 new, and compute-1). Hopefully all will be OK after that. | 10:38 |
| Tengu | shardy: will create some beer budget for Berlin summit :9. | 10:39 |
| jaosorior | lol | 10:39 |
| jaosorior | Tengu: joining for Dublin? | 10:39 |
| shardy | jaosorior: looking at the existing constraints, I wonder if we could add just a new "unique_values" constraint, that would work for both map and list values | 10:40 |
| Tengu | jaosorior: unfortunately no :(. | 10:40 |
| shardy | that could be combined with allowed_pattern to enable quite strict value rules I think | 10:40 |
| shardy | Tengu: nice, glad we worked through the problem :) | 10:40 |
| Tengu | jaosorior: unless I can manage something, but my current job doesn't really involve "openstack community netwoking", and in no way "openstack dev contribs" more than the strict necessary stuff like the patches I already submitted. | 10:41 |
| Tengu | shardy: hopefully a complete and successfull deploy of the 1-compute "version" will also solve the issue I had when I tried to re-add the compute-1 - it was ignored. I think it will be fine now. | 10:42 |
| Tengu | jaosorior: but the Berlin Summit should be ok. In any cases. So beers. And German beers are good :). | 10:44 |
| jaosorior | :P | 10:44 |
| jaosorior | they indeed are | 10:44 |
| *** verdurin has quit IRC | 10:49 | |
| *** skramaja_ has joined #tripleo | 10:51 | |
| *** skramaja has quit IRC | 10:51 | |
| *** skramaja_ has quit IRC | 10:55 | |
| *** skramaja_ has joined #tripleo | 10:55 | |
| *** sshnaidm|afk is now known as sshnaidm | 10:56 | |
| *** jkilpatr has quit IRC | 11:00 | |
| *** moshele has quit IRC | 11:01 | |
| *** verdurin has joined #tripleo | 11:01 | |
| *** moshele has joined #tripleo | 11:01 | |
| *** agopi has quit IRC | 11:06 | |
| *** ykarel|lunch is now known as ykarel | 11:07 | |
| openstackgerrit | waleed mousa proposed openstack/tripleo-heat-templates master: Adding support for role parameters in "environment_generator.py" https://review.openstack.org/529422 | 11:09 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 11:10 |
| *** stendulker has quit IRC | 11:18 | |
| jaosorior | Tengu: https://review.openstack.org/530903 | 11:18 |
| Tengu | jaosorior: cool, that was fast :). Well, when you know where to poke in the code, that helps a lot, of course ;). | 11:20 |
| shardy | sshnaidm: Hi, do you know when the master tripleo ci promotion jobs are scheduled to run? | 11:23 |
| sshnaidm | shardy, near to every 4 hours | 11:23 |
| shardy | we need a promotion to pick up https://review.openstack.org/#/c/530781/ so I wanted to follow the progress | 11:23 |
| shardy | sshnaidm: Ok nice, I was looking at http://38.145.34.234/ and it seemed to only show recent results for pike | 11:24 |
| sshnaidm | shardy, they are running right now: https://review.rdoproject.org/zuul/ | 11:24 |
| sshnaidm | shardy, why? it shows all releases | 11:24 |
| shardy | sshnaidm: I visit http://cistatus.tripleo.org/ then click TripleO CI promotion jobs, there are three results listed, all are stable/pike | 11:25 |
| shardy | I expected to see some recent results for master too | 11:25 |
| shardy | I guess it's just listing that one job, I see the links to master now | 11:26 |
| Tengu | jaosorior: can't do more than +1, but you get it ;). | 11:27 |
| shardy | sshnaidm: thanks for the zuul link, looks good so far :) | 11:27 |
| jaosorior | Tengu: once that merges I'll use the constraint in t-h-t. | 11:27 |
| * shardy crosses fingers | 11:27 | |
| Tengu | jaosorior: \o/ | 11:27 |
| sshnaidm | shardy, https://i.imgur.com/sDFF67K.png | 11:30 |
| *** rfolco has joined #tripleo | 11:30 | |
| Tengu | sshnaidm: nice dashboard | 11:31 |
| shardy | sshnaidm: yeah, it's the job summary at the bottom which confused me | 11:31 |
| *** moshele has quit IRC | 11:32 | |
| sshnaidm | Tengu, thanks | 11:32 |
| sshnaidm | shardy, I think there is problem with 12 and 01 months, need to sort by date reverse | 11:33 |
| Tengu | I think I love that statement: "2018-01-03 11:33:21Z [AllNodesDeploySteps]: UPDATE_COMPLETE state changed" | 11:33 |
| Tengu | shardy: "Stack overcloud UPDATE_COMPLETE" THANK YOU !! | 11:33 |
| shardy | Tengu: \o/ | 11:34 |
| Tengu | so now: let's try to add the three missing computes :). | 11:35 |
| *** moshele has joined #tripleo | 11:35 | |
| Tengu | hmm, first of all: one. In order to ensure it does take the "compute-1". | 11:36 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Install additional packages using kolla mechanism https://review.openstack.org/529070 | 11:37 |
| *** moshele has quit IRC | 11:40 | |
| openstackgerrit | waleed mousa proposed openstack/tripleo-heat-templates master: Adding support for role parameters in "environment_generator.py" https://review.openstack.org/529422 | 11:47 |
| *** jkilpatr has joined #tripleo | 11:49 | |
| *** agopi has joined #tripleo | 11:52 | |
| *** gfidente has joined #tripleo | 11:52 | |
| *** gfidente has quit IRC | 11:52 | |
| *** gfidente has joined #tripleo | 11:52 | |
| Tengu | darn… | 11:52 |
| Tengu | shardy: new issue: apparently, "compute-1" in ironic is ignored for some reason, and tripleo deploy a 2-compute on compute-0 and compute-2. Any idea where I can look? | 11:53 |
| Tengu | shardy: compute-1 is set to "available" in `openstack baremetal node list' | 11:54 |
| Tengu | so I don't really see any reason for that situation :/. | 11:54 |
| *** tosky has joined #tripleo | 11:56 | |
| *** haint has quit IRC | 11:56 | |
| *** cshastri has quit IRC | 11:58 | |
| mhenkel | Hi All. Is there any description on how the neutron plugin containers are used? | 11:59 |
| Tengu | o_O seems like compute-1 is seen, but doesn't match something in the filters… weird. | 12:01 |
| Tengu | shall I rename it to something like compute-4 ?! | 12:01 |
| *** pkovar has joined #tripleo | 12:03 | |
| *** deadnull has joined #tripleo | 12:05 | |
| shardy | Tengu: are you using predicatable placement, e.g tagging compute-1 in ironic and using ComputeSchedulerHints? | 12:05 |
| Tengu | shardy: yep | 12:06 |
| shardy | Tengu: it's expected that it doesn't deploy a compute-1 after node delete, we permanently blacklist it so you avoid some random different node potentially coming up with the same IPs etc | 12:06 |
| Tengu | aaaahhh | 12:06 |
| shardy | Tengu: do openstack stack environment show overcloud | grep RemovalPolicies | 12:06 |
| Tengu | shardy: is there a way to "unblacklist" it, or … | 12:06 |
| Tengu | ok, 2s | 12:07 |
| shardy | you'll see the list, it's possible to pass an empty list on the next update to clear it | 12:07 |
| shardy | Tengu: if you want to reuse a node e.g just replace it, there's a different procedure that uses the heat resource mark unhealthy interface | 12:07 |
| Tengu | shardy: soo… if I pass an empty list, I should also set the ComputeScale to 3 | 12:07 |
| shardy | I'm not sure that is well documented yet though | 12:07 |
| Tengu | shardy: I saw that, but to be honnest, I wasn't really sure how to use that | 12:08 |
| shardy | Tengu: No you can leave the count at 2, assuming you want two nodes 0/1 and not 3 nodes 0/1/2 | 12:08 |
| shardy | Tengu: ack, I'll add it to my docs todo list ;) | 12:08 |
| Tengu | shardy: I want 4 in the end ;). | 12:08 |
| Tengu | shardy: hmmm the grep doesn't show any output, trying to find the entry. | 12:09 |
| shardy | Tengu: Ok so you set the count to 4 and remove the blacklist from ComputeRemovalPolicies, then it'll reuse the blacklisted compute-1 index | 12:09 |
| Tengu | shardy: I can't see anything like "RemovalPolicies" in the stack env… but if you tell me I can just add the entry in the env file, as an empty hash, I'll try that. | 12:10 |
| shardy | http://paste.openstack.org/show/637359/ | 12:10 |
| shardy | Tengu: there's another way to do it | 12:10 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 12:10 |
| Tengu | shardy: is the ComputeRemovalPolicies an entry directly in the parameter_defaults ? | 12:11 |
| Tengu | shardy: hmmmm. there's something else then: ComputeRemovalPolicies: '[]' | 12:11 |
| Tengu | already empty :/ | 12:11 |
| *** salmankhan has joined #tripleo | 12:12 | |
| *** raildo has joined #tripleo | 12:13 | |
| shardy | Tengu: hrm, and the node delete completed without errors? | 12:14 |
| Tengu | shardy: as said: I think so :/. | 12:15 |
| shardy | http://paste.openstack.org/show/637361/ | 12:15 |
| shardy | Tengu: can you please confirm the count and Compute resource group members like that? | 12:15 |
| Tengu | 2s | 12:16 |
| shardy | Tengu: the way this is supposed to work, is openstack overcloud node delete blacklists e.g "1" in that nested stack | 12:16 |
| shardy | then generates a new one with, say "0" and "2" in it, assuming a ComputeCount of 2 and a removal polciies containing a blacklist for "1" | 12:16 |
| Tengu | hmm ok. 2s, running the two openstack commands in order to check the stack. | 12:17 |
| Tengu | I thought it was something like that when you talked about blacklisting. | 12:17 |
| Tengu | shardy: http://paste.openstack.org/show/637364/ | 12:18 |
| Tengu | shardy: so id "1" just doesn't exist. I can of course rename the -1 to -4 - but that's uncool. | 12:19 |
| shardy | Tengu: Ok, that doesn't make sense to me, it should match the RemovalPolicies parameters | 12:19 |
| shardy | Tengu: please can you paste the output of openstack stack resource show overcloud Compute | 12:20 |
| Tengu | shardy: 2s | 12:20 |
| Tengu | aahh | 12:21 |
| Tengu | shardy: | attributes | {u'attributes': None, u'refs': None, u'refs_map': None, u'removed_rsrc_list': [u'1']} | 12:21 |
| Tengu | | | 12:21 |
| Tengu | there's an attribute in that stack | 12:21 |
| Tengu | and it indeed drops the "1" | 12:21 |
| Tengu | complete output: http://paste.openstack.org/show/637367/ | 12:21 |
| shardy | Tengu: right, so ComputeRemovalPolicies must have been set for that to happen | 12:21 |
| Tengu | o_O | 12:21 |
| shardy | I just don't understand why it's now unset | 12:22 |
| Tengu | weeeell… | 12:22 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L539 | 12:23 |
| Tengu | shardy: http://paste.openstack.org/show/637371/ | 12:23 |
| Tengu | shardy: humpf. I really don't know. the environment of the stack doesn't set that, as far as I can see. | 12:24 |
| shardy | Tengu: FYI that's what sets it, seems the heat attribute is sticky, either because of the ResourceGroup implementation or because we're using PATCH updates | 12:24 |
| * shardy looks at code | 12:24 | |
| *** cshastri has joined #tripleo | 12:25 | |
| Tengu | at least the compute-2 seems to be properly deployed. That's a good start :). | 12:28 |
| *** jkilpatr has quit IRC | 12:28 | |
| *** dpawar has quit IRC | 12:29 | |
| *** fragatina has quit IRC | 12:31 | |
| *** fragatina has joined #tripleo | 12:32 | |
| *** agurenko has quit IRC | 12:32 | |
| *** cshastri has quit IRC | 12:34 | |
| *** lucasagomes is now known as lucas-hungry | 12:34 | |
| *** ecerquei has joined #tripleo | 12:36 | |
| *** agurenko has joined #tripleo | 12:37 | |
| *** pdeore has quit IRC | 12:41 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-common master: Add UC backup Mistral workflow definition and actions https://review.openstack.org/517610 | 12:42 |
| mrunge | does anyone know if there's a generic issue with rdocloud? oooq-deployments currently fail on rdo-cloud | 12:42 |
| Tengu | shardy: do you find anything? :/ | 12:42 |
| mrunge | "Message: No valid host was found. There are not enough hosts available., Code: 500" | 12:42 |
| *** bfournie has joined #tripleo | 12:44 | |
| *** agopi has quit IRC | 12:44 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-common master: Add UC backup Mistral workflow definition and actions https://review.openstack.org/517610 | 12:45 |
| *** lvdombrkr has joined #tripleo | 12:45 | |
| *** akane has quit IRC | 12:47 | |
| *** atoth has joined #tripleo | 12:47 | |
| shardy | Tengu: sorry I'm on a call, bbiab | 12:48 |
| *** shardy is now known as shardy_mtg | 12:48 | |
| Tengu | shardy_mtg: no problem. Still checking the deploy of the current compute-2 :). so far so good. | 12:49 |
| *** moshele has joined #tripleo | 12:52 | |
| *** shardy_mtg is now known as shardy | 12:58 | |
| *** catintheroof has joined #tripleo | 12:58 | |
| shardy | Tengu: so I think what you're seeing is expected, but we may have a bug in tripleo-common as the ComputeRemovalPolicies should not be reset | 12:59 |
| shardy | Tengu: https://github.com/openstack/heat/blob/master/heat/engine/resources/openstack/heat/resource_group.py#L333 | 12:59 |
| *** jlabarre has quit IRC | 12:59 | |
| *** catintheroof has quit IRC | 12:59 | |
| shardy | Tengu: we can see the heat resource group only appends to the blacklist, there's no way at present to reset it | 12:59 |
| *** catintheroof has joined #tripleo | 12:59 | |
| *** dfg232 has joined #tripleo | 12:59 | |
| shardy | Tengu: we could add a new interface that enables that, but it doesn't look possible with the current implementation except via db hacking | 12:59 |
| shardy | unless I'm missing something | 13:00 |
| *** rlandy has joined #tripleo | 13:00 | |
| shardy | ramishra: ^^ maybe you can sanity check, it's a while since I looked at that code :) | 13:00 |
| *** rlandy is now known as rlandy|ruck|mtg | 13:00 | |
| *** skramaja_ has quit IRC | 13:00 | |
| Tengu | shardy: duh, ok. so I have to rename the -1 to -4 or something like that, right? | 13:04 |
| *** yprokule has joined #tripleo | 13:04 | |
| dfg232 | Hi all | 13:04 |
| Tengu | shardy: just having some other issues on the controllers, apparently pacemaker is crying. | 13:04 |
| shardy | Tengu: I think so, or we add that heat resouce, or figure out the db query to modify the resource data | 13:04 |
| dfg232 | i have deleted stack from directore and now my all vms are in error state | 13:04 |
| dfg232 | (undercloud) [stack@undercloud ~]$ openstack baremetal node list | 13:05 |
| dfg232 | +--------------------------------------+------+---------------+-------------+--------------------+-------------+ | 13:05 |
| dfg232 | | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | | 13:05 |
| dfg232 | +--------------------------------------+------+---------------+-------------+--------------------+-------------+ | 13:05 |
| dfg232 | | ae7f7e5e-fa20-4fd0-860d-a24aaf363b3a | vm1 | None | None | error | True | | 13:05 |
| dfg232 | | e98592f1-a373-420f-b84a-d04044b6e51a | vm2 | None | None | error | True | | 13:05 |
| dfg232 | | 3c374173-3c00-4067-8804-319f43d6c60f | vm3 | None | None | error | True | | 13:05 |
| dfg232 | | 741a0cb5-8fa8-42b5-b686-55585f1724c0 | vm4 | None | None | error | True | | 13:05 |
| dfg232 | | d69e7f84-1980-4b34-a385-eb50a39c708e | vm5 | None | None | error | True | | 13:05 |
| dfg232 | | 538b2ee8-e4d4-46af-aea3-a96a4e8c4349 | vm6 | None | None | error | True | | 13:05 |
| dfg232 | +--------------------------------------+------+---------------+-------------+--------------------+-------------+ | 13:05 |
| shardy | Tengu: for future reference the mark unhealthy workflow is what you want for this I think, I'll see if we can get a docs patch up explaining it | 13:05 |
| dfg232 | 4:12 PM | 13:05 |
| shardy | dfg232: please use paste.openstack.org, thanks! | 13:05 |
| *** pgadiya has quit IRC | 13:05 | |
| *** jcoufal has joined #tripleo | 13:05 | |
| shardy | dfg232: if you're using virtualbmc, see if the processes are still running (ps ax | grep vbmc) | 13:06 |
| shardy | for me they sometimes silently die so I need to restart the service, and reset the provisionining/maintenance state in ironic | 13:06 |
| dfg232 | ok let me check | 13:07 |
| dfg232 | <shardy> where should i need to check this on directore ? | 13:07 |
| jtomasek | jpich: can you please point me to the CLI code which generates overcloudrc? | 13:08 |
| *** abishop has joined #tripleo | 13:09 | |
| jpich | jtomasek: openstack overcloud credentials | 13:09 |
| dfg232 | do we have any cli to shift vip from controller to other ? | 13:10 |
| jtomasek | jpich: I am trying to find out if it uses mistral action or not | 13:10 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 13:10 |
| shardy | dfg232: http://paste.openstack.org/show/637395/ - you can see one process had failed, restarting the service restarted it | 13:10 |
| Tengu | shardy: ok :). thanks for your time - I'm to find out what's going on, rabbitmq seems to NOT listen on the right port on the controllers, preventing nova to start on the new compute. | 13:11 |
| jpich | jtomasek: Cool! Looks like it does: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_credentials.py#L41 -> https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/deployment.py#L94-L96 | 13:11 |
| Tengu | yololl | 13:11 |
| jtomasek | jpich: nice, thanks | 13:12 |
| jpich | jtomasek: Sorry, I had missed the "code" from your first question :) | 13:12 |
| *** jlabarre has joined #tripleo | 13:14 | |
| *** dfg232 has quit IRC | 13:14 | |
| *** pgadiya has joined #tripleo | 13:15 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Poll heat events and send it to messaging queue https://review.openstack.org/529567 | 13:16 |
| *** ratailor has quit IRC | 13:16 | |
| *** diga has joined #tripleo | 13:19 | |
| *** Goneri has quit IRC | 13:19 | |
| *** chlong has quit IRC | 13:20 | |
| *** karthiks has quit IRC | 13:22 | |
| Tengu | shardy: I'll check the DB, maybe it's simple (although I really doubt it ;)) | 13:23 |
| *** pchavva has joined #tripleo | 13:24 | |
| shardy | Tengu: ack, I can't recall if the resource data is all encrypted, if it is then it may be easier to just add the new heat property | 13:25 |
| Tengu | shardy: "new heat property" ? | 13:25 |
| *** yolanda__ has joined #tripleo | 13:25 | |
| *** psahoo has quit IRC | 13:25 | |
| shardy | Tengu: a new property in resource_group.py, like reset_removal_policies: True or similar | 13:26 |
| shardy | something to avoid always adding and instead overwrite the data | 13:26 |
| Tengu | ah. hmm. ok. for a "one-time" shot then? | 13:26 |
| shardy | Tengu: well it could just be a flag which says don't always append, but use the exact removal_policies as passed | 13:26 |
| shardy | that would probably be a better fit with the declarative nature of heat | 13:27 |
| Tengu | hmm ok. well… hm. I think I'll just rename the node - but I can open an issue with some details you might complete with your inputs… ? | 13:27 |
| shardy | Tengu: sure please do, and I'll try to hack on a patch if I get time | 13:28 |
| Tengu | db hacking isn't a good idea seeing how the "heat" database is :). | 13:28 |
| shardy | Tengu: yeah unless it's a dev environment probably best avoided | 13:28 |
| Tengu | seeing Stack overcloud UPDATE_COMPLETE is enough for me :D | 13:28 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-common master: Validate parameters before updating https://review.openstack.org/511249 | 13:28 |
| *** yolanda has quit IRC | 13:29 | |
| *** psachin has quit IRC | 13:29 | |
| Tengu | shardy: so now I get two computes in my openstack - will rename the compute-1 to compute-4 and update the env file in order to get my 4 computes. | 13:30 |
| shardy | Tengu: ack sounds good, and we can discuss the removal polcies thing later | 13:30 |
| *** bugzy has quit IRC | 13:30 | |
| *** stee_3 has quit IRC | 13:31 | |
| *** rhallisey has joined #tripleo | 13:31 | |
| Tengu | shardy: yup :). I'll open an issue today so that it's still fresh. | 13:31 |
| *** lucas-hungry is now known as lucasagomes | 13:32 | |
| Tengu | shardy: https://bugs.launchpad.net/tripleo/+bug/1741053 | 13:36 |
| openstack | Launchpad bug 1741053 in tripleo "Node blacklisting can't be reverted/reseted" [Undecided,New] | 13:36 |
| *** karthiks has joined #tripleo | 13:36 | |
| jaosorior | chandankumar: around? | 13:36 |
| weshay | mwhahaha, EmilienM the promotion server (dlrn_api) had a kernel panic, restarted it... we should promote master shortly | 13:37 |
| *** trown|outtypewww is now known as trown | 13:37 | |
| mwhahaha | k | 13:38 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: Define 'NeutronPluginExtensions' in OVN environment files with dns extension https://review.openstack.org/530921 | 13:39 |
| *** psachin has joined #tripleo | 13:40 | |
| *** jkilpatr has joined #tripleo | 13:42 | |
| *** ebarrera has joined #tripleo | 13:42 | |
| *** psachin has quit IRC | 13:43 | |
| *** karthiks has quit IRC | 13:44 | |
| *** agurenko has quit IRC | 13:45 | |
| *** karthiks has joined #tripleo | 13:45 | |
| mwhahaha | thrash: we promoted but mistral broke us so we'll need https://review.openstack.org/#/c/530783/ before you recheck things. jfyi | 13:47 |
| thrash | mwhahaha: interesting... I thought mistral had left in some backcompat stuff for that... | 13:48 |
| *** panda|pto is now known as panda | 13:48 | |
| mwhahaha | thrash: it did, but it missed stuff. see https://review.openstack.org/#/c/530781/ | 13:49 |
| *** agurenko has joined #tripleo | 13:49 | |
| thrash | mwhahaha: ack. thanks for that update. | 13:50 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Add undercloud-install tag to novajoin preparation https://review.openstack.org/530922 | 13:50 |
| jaosorior | sshnaidm, panda ^^ | 13:50 |
| shardy | yeah I need to re-test, I'm not sure even if with https://review.openstack.org/#/c/530781/ the backwards compatibility works in mistral, e.g you have to use the mistral_lib security context vs the old top-level interfaces I think | 13:50 |
| *** eck`gone is now known as eck` | 13:51 | |
| shardy | either way with my 2 patches I think we're good | 13:51 |
| ccamacho | hey trown, quick question, Im trying to find the simplest command to deploy OOOQ with HA+Containers. | 13:51 |
| ccamacho | I want to add this info to https://github.com/ccamacho/tripleo-graphics/blob/master/cheatsheets/latest_style/WIP-01-tripleo-cheatsheet-deploying-tripleo.pdf for this month devconf.cz, mostly on step04 but I dont have much space to add a custom config file with all the templates required | 13:51 |
| shardy | thrash: FYI ^^ | 13:51 |
| weshay | thrash, are you waiting on a rdo or osp build? | 13:51 |
| thrash | shardy: cool. | 13:52 |
| jaosorior | tosky: around? | 13:52 |
| thrash | weshay: rdo. | 13:52 |
| thrash | weshay: I had some code in mistral merge, which renders 'Depends-On' inoperative. | 13:52 |
| weshay | oh that bug | 13:52 |
| thrash | weshay: yah | 13:53 |
| tosky | jaosorior: I am | 13:53 |
| mwhahaha | slagle: can you check out https://bugs.launchpad.net/tripleo/+bug/1741053 when you have a second | 13:53 |
| openstack | Launchpad bug 1741053 in tripleo "Node blacklisting can't be reverted/reseted" [High,Triaged] | 13:53 |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 13:53 | |
| jaosorior | tosky: hey, what channel can I use for python-tempestconf questions? | 13:53 |
| tosky | jaosorior: #refstack | 13:53 |
| weshay | thrash, you should still be able to get what you need w/ https://trunk.rdoproject.org/centos7-master/current-tripleo/ | 13:53 |
| weshay | and not current-tripleo-rdo | 13:53 |
| slagle | mwhahaha: that is not for the "blacklist" parameter. sounds like a Heat bug to me. what do you think shardy? | 13:54 |
| *** aputtur_ has joined #tripleo | 13:54 | |
| shardy | slagle: yeah I just discussed it with Tengu, there may be two bugs, you can't ever reset the blacklist in heat, and tripleo-common resets the *RemovalPolicies parameters, so the blacklist isn't visible to operators unless you look at the heat resource attributes | 13:55 |
| shardy | slagle: I'll add a comment, probably we need to patch heat and tripleo-common | 13:56 |
| slagle | ack | 13:57 |
| *** links has quit IRC | 13:57 | |
| *** vpickard_ is now known as vpickard | 13:58 | |
| Tengu | shardy: thanks for the following :). It was a tricky one. I think I always hit tricky things ,). | 13:58 |
| *** bfournie has quit IRC | 14:00 | |
| *** tzumainn has joined #tripleo | 14:00 | |
| *** jaganathan has quit IRC | 14:01 | |
| shardy | slagle, mwhahaha: bug updated, I'll try to find time to look at the heat part later, maybe we can find someone from the workflow squad to look at the tripleo-common part? | 14:01 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 14:10 |
| *** Goneri has joined #tripleo | 14:12 | |
| weshay | EmilienM, can you join the tripleo-ci mtg? | 14:12 |
| weshay | or mwhahaha ? | 14:12 |
| *** sdake has quit IRC | 14:12 | |
| *** sdake has joined #tripleo | 14:13 | |
| *** sdake has joined #tripleo | 14:13 | |
| mwhahaha | weshay: gimme 20. | 14:13 |
| *** diga has quit IRC | 14:16 | |
| trown | ccamacho: https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset035.yml is ha and containers ... that only runs on ovb in CI, so it is pretty unlikely to work on libvirt ... may need a custom config for that for libvirt | 14:16 |
| *** Goneri has quit IRC | 14:16 | |
| arxcruz | ALL: Community meeting for TripleO CI was postponed 20 minutes in order to mwhahaha and EmilienM be able to join :) | 14:20 |
| *** jmelvin has joined #tripleo | 14:22 | |
| *** pgadiya has quit IRC | 14:25 | |
| *** akane_ has joined #tripleo | 14:26 | |
| *** akane_ has quit IRC | 14:26 | |
| *** akane has joined #tripleo | 14:27 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Validate roles data and network data https://review.openstack.org/508567 | 14:30 |
| *** akane has quit IRC | 14:31 | |
| *** ykarel is now known as ykarel|afk | 14:32 | |
| ccamacho | trown thanks man :) | 14:32 |
| arxcruz | fultonj: and we are back :) | 14:33 |
| *** Goneri has joined #tripleo | 14:33 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Add a Get Networks workflow https://review.openstack.org/509419 | 14:33 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Add UpdateNetworks workflow https://review.openstack.org/513463 | 14:33 |
| fultonj | arxcruz: ok, thanks | 14:34 |
| *** Goneri has quit IRC | 14:34 | |
| *** bfournie has joined #tripleo | 14:35 | |
| *** lblanchard has joined #tripleo | 14:37 | |
| *** bfournie1 has joined #tripleo | 14:37 | |
| *** thrash is now known as thrash|biab | 14:38 | |
| *** dfg232 has joined #tripleo | 14:38 | |
| *** bfournie has quit IRC | 14:39 | |
| jpich | Does quickstart persist across reboots nowadays? Wondering what are my options for setting up a virtualised environment that won't break after a power cut | 14:41 |
| *** masco has quit IRC | 14:41 | |
| jpich | Been keeping an eye on https://bugs.launchpad.net/tripleo/+bug/1692976 for now | 14:42 |
| openstack | Launchpad bug 1692976 in tripleo "tripleo-quickstart VMs can't survive a host reboot" [High,Triaged] | 14:42 |
| shardy | jpich: the bug I raised about it is still open AFAIK | 14:42 |
| Tengu | shardy: could get my rabbitmq back on the controllers, apparently we got some issue at some point that broke the rabbitmq cluster (and that one only). I'm currently updating the stack in order to add compute-3 and compute-4 (late compute-1 ;)). | 14:42 |
| *** sshnaidm is now known as sshnaidm|mtg | 14:42 | |
| *** ramishra has quit IRC | 14:43 | |
| jpich | shardy: Thanks, that would be the one indeed... Half hoping it got resolved in a different way and people maybe forgot to update the bug, but it doesn't sound like it | 14:44 |
| shardy | jpich: FWIW ref bug #1692987 I still don't know how to make quickstart configure VMs exactly like instack-virt-setup | 14:44 |
| openstack | bug 1692987 in tripleo "tripleo-quickstart deploying VMs as root user doesn't work" [High,Triaged] https://launchpad.net/bugs/1692987 | 14:44 |
| shardy | I suspect it's now possible but haven't spent the time to figure it out | 14:44 |
| openstackgerrit | Daniel Alvarez proposed openstack/tripleo-heat-templates master: Add support for OVN Metadata Agent https://review.openstack.org/502943 | 14:44 |
| shardy | some super-simple playbook that did that would be *so* helpful tho | 14:44 |
| *** anilvenkata has quit IRC | 14:45 | |
| shardy | jpich: well the stuff to run as root did land, but last time I tried it the results weren't what I expected | 14:45 |
| shardy | jpich: trozet may have some updates tho | 14:45 |
| jpich | shardy: Thanks for the updates! Maybe trozet will have some additional information as well | 14:48 |
| trozet | shardy, jpich: This patch was merged: https://review.openstack.org/#/c/471578/ | 14:48 |
| trozet | shardy, jpich: and this one https://review.openstack.org/#/c/471214/ | 14:49 |
| shardy | trozet: thanks, any chance you can update https://bugs.launchpad.net/tripleo/+bug/1692987 with some notes on the config you've tested successfully with those? | 14:50 |
| openstack | Launchpad bug 1692987 in tripleo "tripleo-quickstart deploying VMs as root user doesn't work" [High,Triaged] | 14:50 |
| *** chlong has joined #tripleo | 14:50 | |
| jpich | trozet: Thanks! Does it help with keeping a working environment across reboots then? | 14:50 |
| *** Goneri has joined #tripleo | 14:51 | |
| trozet | shardy: I'll defer to michchap. I tested it a long time ago, I dont currently use oooqs so not sure if it works or not | 14:51 |
| *** Goneri has quit IRC | 14:51 | |
| mhenkel | shardy: Hi Steven, I have a question regarding neutron configuration: how much backwards compatible to existing puppet modules can I expect? I.e. is something like that still supposed to work: | 14:52 |
| mhenkel | https://github.com/Juniper/contrail-tripleo-puppet/blob/stable/ocata/manifests/network/contrail/neutron_plugin.pp | 14:52 |
| mhenkel | trying to figure out how to write the neutron plugin config file | 14:52 |
| *** jobewan has joined #tripleo | 14:52 | |
| *** chlong has quit IRC | 14:53 | |
| trozet | jpich: when you say working across reboots, you mean like the bridges and vms are automatically back up after restart? | 14:53 |
| jpich | trozet: I think? I mean, can I ssh into the undercloud and keep working away, even if the host machine was rebooted | 14:53 |
| *** hjensas has quit IRC | 14:54 | |
| trozet | jpich: I think the VM will still be there, but you probably have to manually start it with virsh | 14:54 |
| trozet | jpich: and start any extra neworks manually with virsh | 14:54 |
| *** dpawar has joined #tripleo | 14:55 | |
| jpich | trozet: Ok, that sounds promising! Cheers | 14:55 |
| *** dpawar has quit IRC | 14:55 | |
| *** akane_ has joined #tripleo | 14:56 | |
| *** dpawar has joined #tripleo | 14:56 | |
| * beagles wonders if there is a downside to having those networks autostart | 14:56 | |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 14:56 | |
| *** karthiks has quit IRC | 14:56 | |
| trozet | jpich: michchap is in Australia so timing is hard, but if you catch him on IRC he probably has more info about this stuff | 14:56 |
| apetrich | rbrady,d0ugal,apetrich,thrash,toure,jpich,jtomasek Tripleo Workflows Squad Meeting in 5 mins https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 14:56 |
| michchap | trozet: looking | 14:57 |
| Tengu | shardy: once again: thank you. You saved my overcloud with your openstack voodoo :). The two last nodes are being deployed right now, and all seems to work as expected. | 14:57 |
| trozet | oh hi michchap :) | 14:57 |
| jpich | wow, hi | 14:57 |
| apetrich | rbrady,d0ugal,apetrich,thrash,toure,jpich,jtomasek Sorry, no. Just saw we are skipping 3rd of january | 14:57 |
| *** akane_ has quit IRC | 14:57 | |
| *** akane has joined #tripleo | 14:58 | |
| michchap | shardy: https://gerrit.opnfv.org/gerrit/#/c/28675/26/config/quickstart_defaults.yaml | 15:00 |
| *** agurenko has quit IRC | 15:01 | |
| michchap | that's what I was using a while ago, before we abandoned the conversion. I can't promise it still works | 15:01 |
| shardy | michchap: ack thanks for the update, sounds like we'll have to re-test it | 15:02 |
| *** liverpooler has joined #tripleo | 15:05 | |
| EmilienM | hi | 15:05 |
| EmilienM | weshay, arxcruz : too late I guess | 15:06 |
| arxcruz | EmilienM: yup | 15:06 |
| *** Goneri has joined #tripleo | 15:08 | |
| *** pradk has joined #tripleo | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 15:10 |
| EmilienM | shardy: I removed the Depends-On on purpose on https://review.openstack.org/#/c/530783/ | 15:11 |
| EmilienM | shardy: because if we land it, it'll break the CI, since we need a promotion with the mistral patch | 15:11 |
| shardy | EmilienM: ack yeah I wasn't aware of the thread about patching the RDO package before promotion, thanks | 15:12 |
| shardy | at least the CI was green for the first revision, which proves the fix I think | 15:12 |
| EmilienM | yeah cool | 15:12 |
| EmilienM | shardy: I proposed the revert: https://review.rdoproject.org/r/#/c/11155/ | 15:12 |
| *** udesale has quit IRC | 15:13 | |
| mwhahaha | EmilienM: i already reverted | 15:13 |
| mwhahaha | EmilienM: https://review.rdoproject.org/r/#/c/11151/ | 15:13 |
| EmilienM | mwhahaha: ok, thanks | 15:14 |
| EmilienM | shardy, mwhahaha : promotion jobs are running, hopefully they all pass | 15:15 |
| *** dtrainor has joined #tripleo | 15:16 | |
| *** dfg232 has quit IRC | 15:17 | |
| *** karthiks has joined #tripleo | 15:21 | |
| *** links has joined #tripleo | 15:24 | |
| *** nyechiel_ has quit IRC | 15:26 | |
| *** devananda has joined #tripleo | 15:27 | |
| *** almondjoy has joined #tripleo | 15:27 | |
| *** salmankhan has quit IRC | 15:28 | |
| *** akane has quit IRC | 15:29 | |
| *** links has quit IRC | 15:30 | |
| *** moshele has quit IRC | 15:32 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient master: Handle user-provided TLS certificate/key for the undercloud https://review.openstack.org/530737 | 15:32 |
| shardy | Tengu: FYI https://review.openstack.org/530948 aims to address the heat part of the problem we discussed earlier | 15:36 |
| *** shreshtha has quit IRC | 15:40 | |
| *** dfg232 has joined #tripleo | 15:42 | |
| *** mrch has quit IRC | 15:42 | |
| *** nyechiel_ has joined #tripleo | 15:43 | |
| *** vpickard has quit IRC | 15:45 | |
| openstackgerrit | Steven Hardy proposed openstack/instack-undercloud master: Set heat engine workers to "%{::os_workers_heat_engine}" https://review.openstack.org/529066 | 15:45 |
| *** sshnaidm|mtg is now known as sshnaidm | 15:46 | |
| shardy | Slower: Hey did you see my comments on https://review.openstack.org/#/c/523511/ ? | 15:49 |
| shardy | Slower: happy to work on a fix if you've not already started it | 15:49 |
| *** suuuper has quit IRC | 15:50 | |
| *** dougbtv_ has joined #tripleo | 15:51 | |
| jaosorior | Slower: around? | 15:51 |
| *** dougbtv has quit IRC | 15:51 | |
| Tengu | shardy: great, I'll check that in a moment :). Thanks for the fast process :) | 15:52 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-heat-templates master: Set tftp to only listen to the provisioning network https://review.openstack.org/529023 | 15:52 |
| *** ramishra has joined #tripleo | 15:55 | |
| *** dfg232 has quit IRC | 15:55 | |
| *** dougbtv_ has quit IRC | 15:56 | |
| *** dfg232 has joined #tripleo | 15:56 | |
| *** dougbtv_ has joined #tripleo | 15:59 | |
| Tengu | shardy: \o/ my overcloud now has its 4 compute nodes, compute-0, compute-2, compute-3 and compute-4. | 16:00 |
| Tengu | shardy: will you go to the Berlin Summit? or is it too early to know that? :) | 16:00 |
| shardy | Tengu: not sure but probably will be there | 16:01 |
| Tengu | :) | 16:01 |
| *** ramishra has quit IRC | 16:02 | |
| *** jkilpatr has quit IRC | 16:06 | |
| *** rlandy|ruck|mtg is now known as rlandy | 16:07 | |
| *** nyechiel_ has quit IRC | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 16:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| *** thrash|biab is now known as thrash | 16:10 | |
| *** panda is now known as panda|rover | 16:11 | |
| *** dfg232 has quit IRC | 16:13 | |
| *** artom has joined #tripleo | 16:14 | |
| *** dfg232 has joined #tripleo | 16:15 | |
| *** chlong has joined #tripleo | 16:16 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Convert tags to when statements for Q major upgrade workflow https://review.openstack.org/510902 | 16:18 |
| *** jaganathan has joined #tripleo | 16:18 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: WIP ffu: Introduce Heat fast_forward_upgrade_tasks https://review.openstack.org/530956 | 16:18 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart master: Add a new ci/environments/featureset_containerized_upgrade.yml https://review.openstack.org/527454 | 16:18 |
| *** kristaps_ has joined #tripleo | 16:18 | |
| *** openstackgerrit has quit IRC | 16:19 | |
| *** jkilpatr has joined #tripleo | 16:19 | |
| *** ykarel|afk is now known as ykarel | 16:21 | |
| *** lvdombrkr has quit IRC | 16:21 | |
| *** jkilpatr has quit IRC | 16:22 | |
| *** jkilpatr has joined #tripleo | 16:23 | |
| *** chlong has quit IRC | 16:24 | |
| *** deadnull has quit IRC | 16:25 | |
| *** Goneri has quit IRC | 16:28 | |
| *** salmankhan has joined #tripleo | 16:29 | |
| weshay | mwhahaha, EmilienM master is uploading images/containers now | 16:33 |
| mwhahaha | orly | 16:33 |
| EmilienM | yeah? | 16:33 |
| *** dpawar has quit IRC | 16:33 | |
| *** dpawar has joined #tripleo | 16:34 | |
| mwhahaha | i do think we need to change the tripleo mascot to http://i0.kym-cdn.com/entries/icons/original/000/000/015/oreally.jpg | 16:34 |
| *** pcaruana has quit IRC | 16:35 | |
| EmilienM | lol | 16:37 |
| *** dpawar has quit IRC | 16:38 | |
| *** openstackgerrit has joined #tripleo | 16:39 | |
| openstackgerrit | Jakub Libosvar proposed openstack/tripleo-heat-templates master: neutron: Call destroy-patch-ports before starting ovs-agent https://review.openstack.org/519354 | 16:39 |
| *** chlong has joined #tripleo | 16:40 | |
| *** moshele has joined #tripleo | 16:42 | |
| *** Goneri has joined #tripleo | 16:43 | |
| *** salmankhan has quit IRC | 16:44 | |
| *** jaganathan has quit IRC | 16:48 | |
| *** trown is now known as trown|lunch | 16:52 | |
| *** moshele has quit IRC | 16:54 | |
| *** aufi has quit IRC | 16:55 | |
| *** catintheroof has quit IRC | 16:55 | |
| *** dfg232 has quit IRC | 16:57 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-validations master: Add --static-yaml-inventory option to tripleo-ansible-inventory https://review.openstack.org/530476 | 16:57 |
| EmilienM | weshay: where do you see it? | 16:58 |
| *** fragatina has quit IRC | 17:00 | |
| *** fragatina has joined #tripleo | 17:01 | |
| *** ffiore has quit IRC | 17:08 | |
| *** dpawar has joined #tripleo | 17:08 | |
| *** chlong has quit IRC | 17:09 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Steven Hardy (shardy) | 17:10 |
| tdasilva | mwhahaha: hello, is there a script that I could re-use to upload data to the undercloud swift cluster from an overcloud node? | 17:13 |
| mwhahaha | tdasilva: no but there's a workflow | 17:13 |
| mwhahaha | tdasilva: https://github.com/openstack/tripleo-common/blob/master/workbooks/support.yaml#L60 | 17:14 |
| jaosorior | Do we have any scenario testing out the config-download stuff? | 17:17 |
| jaosorior | I would like to add coverage for ipsec, but it requires config-download, and net-iso | 17:17 |
| mwhahaha | i think there's one but we don't do net-iso in the scenarios | 17:18 |
| *** jpich has quit IRC | 17:18 | |
| mwhahaha | (at least i don't think we do) | 17:18 |
| trozet | jaosorior: can I ask you a question about TLS CA cert generation in puppet-tripleo? | 17:18 |
| tdasilva | mwhahaha: looking | 17:18 |
| jaosorior | mwhahaha: is it hard to add it? | 17:18 |
| jaosorior | trozet: sure | 17:18 |
| mwhahaha | jaosorior: scenarios run on multinode so yes? | 17:19 |
| mwhahaha | jaosorior: ovb is your only bet for net-iso i think | 17:19 |
| jaosorior | :( | 17:19 |
| *** dpawar has quit IRC | 17:19 | |
| shardy | jaosorior: I think the multinode-containers job uses config download, as we broke it when the ansible inventory generation got broken by mistral | 17:20 |
| trozet | jaosorior: the preface to this is I'm trying to add ODL support. So I can optionally pass puppet-ODL the CA cert file path. It looks like there are several ways that file gets written: https://github.com/openstack/puppet-tripleo/tree/master/manifests/certmonger/ca | 17:20 |
| trozet | jaosorior: from looking at those, it looks liek they get called in certmonger user. The libvirt.pp one looks to me like there is an option to use a different/unique Cert for just libvirt | 17:20 |
| shardy | jaosorior: unless that is generated for validations, not looked at the job in detail tbh | 17:20 |
| trozet | jaosorior: I'm wondering if i should make an opendaylight.pp there and give the option to provide a unique CA cert for ODL | 17:21 |
| jaosorior | trozet: I would need to know more about your usecase. | 17:21 |
| jaosorior | trozet: for TLS everywhere you could merely just pass a parameter with the CA cert path. Most services use the same one, so it's the same file usually. | 17:22 |
| trozet | jaosorior: yeah this is tls everywhere. So most services just use the one either locally or curl downloaded? | 17:22 |
| jaosorior | trozet: for libvirt I'm planning to take into use FreeIPA's lightweight subCAs, so then the CA cert path would vary. But for ODL you might not need that at all. | 17:22 |
| *** moshele has joined #tripleo | 17:23 | |
| *** mdnadeem has quit IRC | 17:23 | |
| jaosorior | trozet: it's in the filesystem. All the nodes register to FreeIPA (to get auth to the CA), so then you can just assume that the CA certificate PEM file is around | 17:23 |
| trozet | jaosorior: is it ever possible to not use any CA and everything is self signed? | 17:23 |
| jaosorior | trozet: no | 17:23 |
| trozet | jaosorior: oh | 17:23 |
| jaosorior | trozet: you don't want to let folks attempt to half ass a PKI | 17:24 |
| *** marios has quit IRC | 17:25 | |
| jaosorior | trozet: that's the reason I introduced FreeIPA into the picture. might as well have a full fledged PKI instead of getting maintenance hell from half assed scripts :D | 17:25 |
| trozet | jaosorior: so the locally generated CA in local.pp (/etc/pki/ca-trust/source/anchors/cm-local-ca.pem), and the curl one is if someone provides one (/etc/pki/CA/crl/overcloud-crl.pem') | 17:25 |
| trozet | jaosorior: right? | 17:25 |
| jaosorior | trozet: that's only for the undercloud | 17:25 |
| jaosorior | trozet: it would not work in the overcloud | 17:25 |
| trozet | jaosorior: whats only for undercloud? | 17:26 |
| jaosorior | trozet: the crl is the CA's crl (certificate revocation list) | 17:26 |
| trozet | jaosorior: oh i thought that meant curl LOL | 17:26 |
| jaosorior | nope | 17:26 |
| trozet | jaosorior: so basically I just need to know where the CA file will be on the overcloud | 17:26 |
| jaosorior | trozet: local.pp is just for the undercloud | 17:26 |
| trozet | jaosorior: because I need to tell ODL where it is and OVS | 17:26 |
| jaosorior | trozet: let me get that for you :D | 17:26 |
| trozet | jaosorior: ty | 17:27 |
| jaosorior | trozet: you can get it from t-h-t https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/apache.yaml#L45 | 17:27 |
| jaosorior | trozet: sorry for the confusing codebase | 17:27 |
| trozet | jaosorior: ah ok | 17:28 |
| trozet | jaosorior: i have several patches posted, but waiting to add you to them until I test them all together and verify it all works (working on the puppet-tripleo then tht now) | 17:28 |
| jaosorior | nice :D | 17:28 |
| jaosorior | trozet: let me know if you need any help | 17:28 |
| trozet | jaosorior: ok ty | 17:29 |
| jaosorior | trozet: this might help in the deployment http://tripleo.org/install/advanced_deployment/ssl.html#tls-everywhere-for-the-overcloud (at least understanding it), but the easiest way is to just use quickstart. it will deploy FreeIPA for you and do all the stuff that's in that doc :D | 17:29 |
| trozet | jaosorior: thanks i will use this | 17:30 |
| EmilienM | sounds like we got promotion | 17:30 |
| jaosorior | EmilienM: yay | 17:30 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Update AnsibleGenerateInventoryAction to match new mistral interfaces https://review.openstack.org/530783 | 17:30 |
| *** dpawar has joined #tripleo | 17:31 | |
| jaosorior | trozet: if you want to use quickstart, you need the config/nodes/1ctlr_1comp_1supp.yml topoligy configuration and the config/general_config/ipa.yml configuration files | 17:31 |
| shardy | \o/ | 17:31 |
| jaosorior | trozet: I gotta go in a bit, but if you need help with that tomorrow let me know. | 17:31 |
| *** devananda has quit IRC | 17:31 | |
| trozet | jaosorior: one more question just to make sure about something. The certmonger_certificate generates the certfile and keyfile right? It says "The path to the certificate that will be used for TLS in this service". But just want to confirm that file is to be written | 17:32 |
| EmilienM | shardy: removed Depends-On, let's see | 17:32 |
| shardy | EmilienM: ack, thanks, I was just about to update it | 17:32 |
| trozet | to be generated... | 17:32 |
| jaosorior | trozet: correct | 17:32 |
| trozet | jaosorior: ok cool. later man | 17:32 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common master: Read the tftp IP to test from the command line https://review.openstack.org/530979 | 17:32 |
| tdasilva | mwhahaha: i'm struggling with how I could make use of that workflow. I've been working on a script similar to https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-api.yaml#L126 | 17:33 |
| openstackgerrit | Steven Hardy proposed openstack/python-tripleoclient master: WIP convert undercloud deploy to use TripleoInventory class https://review.openstack.org/529652 | 17:34 |
| tdasilva | mwhahaha: and i'm not sure how I would tie both a script and that workflow together | 17:34 |
| mwhahaha | tdasilva: remember when i said having to do thing dynamically during the deploy is a giant pain? yea this is why | 17:34 |
| mwhahaha | tdasilva: so you can't use that workflow if you have to do it during the deployment unless you do an external set of tasks | 17:35 |
| mwhahaha | tdasilva: we do a similar thing for the swift rings, could you reuse that? | 17:35 |
| mwhahaha | tdasilva: we have some puppet bits that actually do that upload | 17:36 |
| mwhahaha | tdasilva: https://github.com/openstack/puppet-tripleo/blob/b77751a7111b641ef8fdaaf7730e1701ec716a1c/manifests/profile/base/swift/ringbuilder.pp#L158 | 17:37 |
| mwhahaha | tdasilva: you end up having to pass the get/put urls in to THT, https://github.com/openstack/tripleo-heat-templates/commit/44d0e1bddd352215a98839b9cf3b33ad61b2333c | 17:37 |
| mwhahaha | tdasilva: but note that there's currently a bug around this stuff that i just passed back to the storage dfg this morning because the urls expire which cause problems | 17:38 |
| *** tesseract has quit IRC | 17:38 | |
| mwhahaha | tdasilva: you might ask cschwede as he implemented the swift ring stuff | 17:38 |
| tdasilva | mwhahaha: interesting, that approach seems more likely what I would go for...so I'll ping cschwede about it | 17:40 |
| tdasilva | i wonder how those tempurls are being generated, why do they expire so quickly? | 17:40 |
| mwhahaha | i think there's a workflow to generate them | 17:40 |
| mwhahaha | it's a tempurl it shouldn't exist forever :D | 17:41 |
| tdasilva | true, but it could expire in say 24hrs | 17:41 |
| mwhahaha | yea the issue isn't that, it's on action on the cloud later | 17:42 |
| weshay | mwhahaha, EmilienM master promoted | 17:42 |
| EmilienM | weshay: yeah, we saw, thanks! | 17:42 |
| *** lucasagomes is now known as lucas-afk | 17:42 | |
| EmilienM | weshay: really good, let's continue the plan :D | 17:42 |
| mwhahaha | pffft plans | 17:42 |
| shardy | thanks for the update weshay :) | 17:43 |
| EmilienM | weshay: the plan is to land https://review.rdoproject.org/r/#/c/11157/ now so it won't happen again | 17:43 |
| EmilienM | and land https://review.openstack.org/#/c/530783 when it pass CI and problem is resolved | 17:43 |
| EmilienM | shardy: well, thanks for the patches and fixes | 17:43 |
| *** arxcruz is now known as arxcruz|ruck | 17:44 | |
| *** vpickard has joined #tripleo | 17:49 | |
| *** dpawar_ has joined #tripleo | 17:54 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-validations master: Adjust TripleoInventory arguments, don't pass config object https://review.openstack.org/530983 | 17:55 |
| *** dpawar_ has quit IRC | 17:55 | |
| *** dpawar has quit IRC | 17:57 | |
| *** ykarel has quit IRC | 17:57 | |
| *** derekh has quit IRC | 18:00 | |
| *** chlong has joined #tripleo | 18:00 | |
| *** ihrachys has joined #tripleo | 18:01 | |
| *** jkilpatr has quit IRC | 18:03 | |
| *** jkilpatr has joined #tripleo | 18:03 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 18:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| *** shardy has quit IRC | 18:10 | |
| *** yolanda__ is now known as yolanda | 18:14 | |
| *** bogdando has quit IRC | 18:14 | |
| *** dougbtv__ has joined #tripleo | 18:19 | |
| *** almondjoy has quit IRC | 18:20 | |
| *** yamahata has joined #tripleo | 18:21 | |
| *** gfidente is now known as gfidente|afk | 18:21 | |
| *** dougbtv_ has quit IRC | 18:22 | |
| *** dougbtv has joined #tripleo | 18:22 | |
| *** almondjoy has joined #tripleo | 18:22 | |
| *** dougbtv__ has quit IRC | 18:24 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Add CephAnsiblePlaybookVerbosity parameter to overcloud-ceph-ansible https://review.openstack.org/528741 | 18:28 |
| *** mcornea has quit IRC | 18:28 | |
| *** trown|lunch is now known as trown|outtypewww | 18:37 | |
| *** agopi has joined #tripleo | 18:39 | |
| *** jkilpatr has quit IRC | 18:40 | |
| *** suuuper has joined #tripleo | 18:40 | |
| *** pchavva has quit IRC | 18:43 | |
| *** pchavva has joined #tripleo | 18:43 | |
| *** pchavva has quit IRC | 18:44 | |
| *** pchavva has joined #tripleo | 18:44 | |
| *** pkovar has quit IRC | 18:50 | |
| *** pchavva is now known as pkchavva | 18:55 | |
| *** pkchavva has quit IRC | 18:55 | |
| *** pkchavva has joined #tripleo | 18:56 | |
| *** pkchavva has quit IRC | 18:57 | |
| *** pkchavva has joined #tripleo | 18:57 | |
| *** pkchavva has quit IRC | 18:57 | |
| *** pkchavva has joined #tripleo | 18:58 | |
| *** pkchavva is now known as pchavva | 18:58 | |
| *** pchavva has joined #tripleo | 18:59 | |
| EmilienM | weshay: I missed the ci squad call today - what actions have been taken to investigate scenario001/002 timeouts? | 19:02 |
| EmilienM | weshay: who is on it? | 19:02 |
| weshay | EmilienM, we'll be setting up a graphite/graphana server in rdo-cloud to monitor the jobs | 19:04 |
| weshay | and fixing the status pages | 19:04 |
| weshay | also moving out collect logs into a post | 19:05 |
| *** dhill__ has quit IRC | 19:05 | |
| weshay | EmilienM, is where we are starting https://trello.com/b/U1ITy0cu/tripleo-ci-squad?menu=filter&filter=label:Sprint%206 | 19:05 |
| EmilienM | weshay: i'm not talking about grafana but ok | 19:06 |
| EmilienM | i'm talking about actual actions that reduce jobs runtime | 19:06 |
| *** dhill_ has joined #tripleo | 19:06 | |
| EmilienM | like I did last week with scenario001 a bit | 19:06 |
| *** suuuper has quit IRC | 19:06 | |
| EmilienM | I'll keep looking I gues | 19:06 |
| weshay | moving the services around.. ya.. So I spoke to Alex in the mtg about that | 19:07 |
| weshay | and we're going to raise in the tripleo meeting regarding creating new scenarios so we can spread out the services more | 19:07 |
| EmilienM | yes, that | 19:07 |
| EmilienM | weshay: thanks - I'll keep investigate today probably | 19:08 |
| EmilienM | we need to re-enable voting asap imho | 19:08 |
| weshay | EmilienM, can we create a new scenario now.. and move out a few services to it? | 19:09 |
| EmilienM | weshay: yeah, not that easy because some integration tests in tempest require all these services | 19:09 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 19:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| EmilienM | weshay: scenario001 looks the priority, looking at http://cistatus.tripleo.org/ - it's the one that fails the most | 19:16 |
| weshay | EmilienM, did you guys remove mongo? | 19:19 |
| EmilienM | weshay: not used | 19:19 |
| weshay | we could remove heat | 19:22 |
| EmilienM | no | 19:22 |
| EmilienM | autoscaling needs it | 19:22 |
| weshay | hrm.. /me looks at the tests | 19:22 |
| EmilienM | yep | 19:22 |
| EmilienM | ceilometer.tests.tempest.scenario.test_telemetry_integration.TestTelemetryIntegration.test_autoscaling 621.914 | 19:23 |
| EmilienM | autoscaling test takes 10 min | 19:23 |
| EmilienM | see http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2018-01-03_14_07_26 | 19:24 |
| EmilienM | fwiw, it's a really good test | 19:24 |
| EmilienM | it tests all Telemetry services, and we have Ceph deployed so it test really use case | 19:24 |
| EmilienM | i'm wondering if we should just increase timeout | 19:24 |
| EmilienM | for this scenario | 19:24 |
| EmilienM | by 10 min | 19:24 |
| EmilienM | because I checked the services & tests, I haven't seen much we can improve | 19:25 |
| *** holser__ has quit IRC | 19:25 | |
| EmilienM | pabelanger: is there an hard limit in infra for multinode timeouts? Or can I just increase here? https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L123 | 19:26 |
| mwhahaha | EmilienM: my understanding is that's the upper limit on the zuul phases | 19:29 |
| mwhahaha | so pre, run, post all get 3 hours max | 19:29 |
| mwhahaha | which is why sometimes we get timedouts of 9 hours | 19:29 |
| EmilienM | ok so we can't add 10 min | 19:29 |
| EmilienM | it would be a test regression to remove ceilometer.tests.tempest.scenario.test_telemetry_integration.TestTelemetryIntegration | 19:29 |
| mwhahaha | honest we should move tempest to post :D | 19:29 |
| EmilienM | but would save us 10 min | 19:30 |
| * mwhahaha cheats | 19:30 | |
| EmilienM | mwhahaha: not a bad idea | 19:30 |
| EmilienM | but we also have collect-logs | 19:30 |
| EmilienM | if we moved tempest + collect-logs into post, we would have much less problems | 19:30 |
| EmilienM | sshnaidm is afik working on the collect-logs part at that time | 19:31 |
| mwhahaha | just moving collect logs to post gets like 10 mins back | 19:31 |
| EmilienM | yeah | 19:31 |
| pabelanger | EmilienM: believe that is the cap currently | 19:31 |
| EmilienM | + tempest, another 10 min | 19:31 |
| pabelanger | yah, logs could be moved to post-run | 19:32 |
| mwhahaha | i was kidding about tempest in post, i don't know if that's a really good idea but it's something that we could look into if we really need 3 hours for setup+deploy | 19:32 |
| mwhahaha | but that seems extremely excessive | 19:32 |
| fultonj | is it appropriate to recheck https://review.openstack.org/#/c/528125 ? | 19:33 |
| EmilienM | I wasn't kidding | 19:33 |
| EmilienM | fultonj: no recheck now | 19:33 |
| pabelanger | mwhahaha: ++ to reduce 3 hour deploys | 19:33 |
| EmilienM | fultonj: we need https://review.openstack.org/#/c/530783/ | 19:33 |
| fultonj | EmilienM: ack, i'll wait for that | 19:33 |
| EmilienM | mwhahaha: we could save a 3 min on scenario001 if we stop to run https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/integration/gabbi/gabbits-live/aodh-gnocchi-threshold-alarm.yaml | 19:35 |
| EmilienM | aka ceilometer.tests.tempest.scenario.test_telemetry_integration.TestTelemetryIntegration.test_aodh_gnocchi_threshold_alarm | 19:35 |
| EmilienM | pradk: you around? | 19:35 |
| EmilienM | I'll ask on telemetry channel | 19:36 |
| mwhahaha | honestly unless you're chopping 15 mins, it's not really worth it | 19:36 |
| mwhahaha | because 3 mins gets eaten up by poor network transit | 19:36 |
| pradk | EmilienM, yea whats up | 19:36 |
| EmilienM | pradk: let me ask on telemetry channel | 19:36 |
| pradk | k | 19:36 |
| EmilienM | mwhahaha: we both know we can't get 15 min easily | 19:38 |
| EmilienM | so trying little things | 19:38 |
| mwhahaha | right i think we're starting at the wrong end | 19:38 |
| mwhahaha | let's start at the beginning of the process not the end | 19:38 |
| mwhahaha | what can we cleanup/stop doing in the beginning | 19:38 |
| *** agopi has quit IRC | 19:39 | |
| *** ebarrera has quit IRC | 19:40 | |
| EmilienM | mwhahaha: the other day I noticed some quickstart asks were executed 2 times or more | 19:40 |
| EmilienM | let's see with ara | 19:40 |
| mwhahaha | EmilienM: so the 2nd time might be skipped and that's due to the way quickstart does stuff | 19:40 |
| EmilienM | I know but some times they aren't skipped for some of them | 19:41 |
| mwhahaha | EmilienM: but yea that's where i think we need to start is walking through the process and see what's taking setup so long prior to even getting to the deploy itself | 19:41 |
| mwhahaha | because that's limited to 80 mins | 19:41 |
| mwhahaha | so clearly there's other stuff we're doing before | 19:41 |
| mwhahaha | between undercloud install + deploy, that's about half of our time | 19:41 |
| *** mcornea has joined #tripleo | 19:41 | |
| mwhahaha | now i know the undercloud container stuff would allow us to bypass the install stuff, it's not yet ready to replace stuff | 19:41 |
| weshay | EmilienM, it wasn't running qs tasks 2+ times | 19:42 |
| weshay | you aren't remembering things clearly.. | 19:43 |
| weshay | must be flying to high :) | 19:43 |
| weshay | too | 19:43 |
| weshay | EmilienM, +1 to move collect logs and tempest | 19:43 |
| openstackgerrit | Moshe Levi proposed openstack/puppet-tripleo master: Add missing ssl for ovn_sbdb and ovn_nbdb https://review.openstack.org/530995 | 19:43 |
| EmilienM | come with me, I'll show you the stalls ;-) | 19:43 |
| weshay | :) | 19:43 |
| EmilienM | you'll just freak out and beg me to go landing :P | 19:44 |
| openstackgerrit | Moshe Levi proposed openstack/puppet-tripleo master: Add missing ssl ports for ovn_sbdb and ovn_nbdb https://review.openstack.org/530995 | 19:44 |
| EmilienM | mwhahaha, weshay: https://prnt.sc/hvo13j | 19:44 |
| *** pcaruana has joined #tripleo | 19:45 | |
| weshay | which? | 19:45 |
| EmilienM | weshay: scenario001 | 19:46 |
| mwhahaha | right so half the time is undercloud/overcloud deploy, so what's the other half of the time doing | 19:46 |
| mwhahaha | that's only 1:31 of 3 hours | 19:46 |
| EmilienM | http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/logs/ara_oooq/ | 19:46 |
| EmilienM | 8 min for collect-logs | 19:46 |
| mwhahaha | still only 1:39 | 19:47 |
| mwhahaha | so where is the other hour going | 19:47 |
| EmilienM | something not tracked in ARA probably | 19:48 |
| EmilienM | let me see | 19:48 |
| EmilienM | stuffs in tripleo-ci take time as well | 19:49 |
| EmilienM | see http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/job-output.txt.gz#_2018-01-03_11_55_26_563611 | 19:50 |
| EmilienM | 16 min on bootstrap | 19:50 |
| EmilienM | even before tripleo stuff | 19:50 |
| *** dhill_ has quit IRC | 19:51 | |
| *** dhill_ has joined #tripleo | 19:52 | |
| EmilienM | the repo thing takes crazy time | 19:52 |
| weshay | job starts at 11:39:21 | 19:52 |
| weshay | qs starts at 11:58:41 | 19:52 |
| weshay | undercloud installed at 12:35:03 | 19:52 |
| weshay | overcloud-prep done at 12:42:48 | 19:52 |
| weshay | overcloud deployed at 13:54:12 | 19:52 |
| EmilienM | yeah | 19:52 |
| EmilienM | 19 min before tripleo starts!!!!!! | 19:53 |
| EmilienM | wtf happens here | 19:53 |
| mwhahaha | remember what i said about starting tat the begginning? :D | 19:56 |
| EmilienM | pabelanger: we might need your help on this one | 19:57 |
| mwhahaha | so you have to check from RUN START | 19:57 |
| mwhahaha | http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/job-output.txt.gz#_2018-01-03_11_51_00_869533 | 19:57 |
| mwhahaha | that's the RUN phase | 19:57 |
| mwhahaha | there's PRE-RUN, RUN and POST-RUN they all have their own time limits | 19:58 |
| mwhahaha | so in that one, it was about 9 mins before quickstart gets going | 19:58 |
| *** pcaruana has quit IRC | 19:59 | |
| weshay | mwhahaha, EmilienM we could also run the quickstart bootstrap steps to pre | 20:00 |
| EmilienM | how's that? | 20:00 |
| mwhahaha | well that should be part of the convert to ansible zuul steps | 20:00 |
| weshay | can we not determine what is pre vs. run? | 20:01 |
| mwhahaha | you can, but right now we're still using the legacy shell stuff | 20:01 |
| *** rhallisey has quit IRC | 20:02 | |
| mwhahaha | it needs to get pulled apart and I think that comes in when ditching the toci scripts | 20:02 |
| mwhahaha | so like we're doing with the moving collect-logs to post | 20:02 |
| weshay | prep-containers is only taking 7-8 min | 20:04 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: set telemetry.alarm_granularity to 15s https://review.openstack.org/531002 | 20:05 |
| openstackgerrit | Tim Rozet proposed openstack/puppet-tripleo master: Adds TLS support for OpenDaylight https://review.openstack.org/531003 | 20:06 |
| *** yprokule has quit IRC | 20:06 | |
| weshay | tempest 13:54:16 - 14:08:26 | 20:06 |
| EmilienM | mwhahaha: on #openstack-telemetry, sileht helped to find some ways to accelerate the tempest run | 20:07 |
| mwhahaha | k | 20:07 |
| *** florianf has quit IRC | 20:07 | |
| *** stee_3 has joined #tripleo | 20:08 | |
| pabelanger | EmilienM: what is the issue? | 20:08 |
| EmilienM | pabelanger: we're trying to see why it takes 19 min before tripleo quickstart runs in a CI job | 20:08 |
| mwhahaha | weshay: so why is build-test-packages run after undercloud-setup? | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 20:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| *** bugzy has joined #tripleo | 20:11 | |
| pabelanger | EmilienM: which log shows that? | 20:11 |
| EmilienM | pabelanger: http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/job-output.txt.gz | 20:12 |
| weshay | mwhahaha, undercloud-setup only installs repos etc.. | 20:13 |
| weshay | it doesn't install the undercloud | 20:13 |
| weshay | that is undercloud-deploy | 20:13 |
| *** lblanchard has quit IRC | 20:14 | |
| mwhahaha | weshay: hmmm i see. | 20:14 |
| mwhahaha | weshay: still, shouldn't we run dlrn bits first? | 20:14 |
| weshay | not w/o the repos setup :) | 20:15 |
| EmilienM | weshay: in your knowledge, is there an interface in oooq to run some bash before running tempest? | 20:15 |
| EmilienM | weshay: that we could define in the featureset | 20:15 |
| pabelanger | EmilienM: I still am not sure of the issue, are you saying there is timeout issues? | 20:15 |
| mwhahaha | weshay: that's repo-setup, not undercloud-setup | 20:15 |
| weshay | EmilienM, we could add something sure | 20:15 |
| EmilienM | pabelanger: in scenario001 yes, and nodes preparation takes 19 minutes, we think it's too long | 20:15 |
| weshay | undercloud-setup calls repo-setup if iirc | 20:15 |
| EmilienM | weshay: but nothing in place? | 20:16 |
| EmilienM | I want to run 2 commands after overcloud-deploy and before tempest | 20:16 |
| mwhahaha | weshay: i don't think so, it's not obvious from the logs though | 20:16 |
| pabelanger | EmilienM: in pre-run? | 20:16 |
| EmilienM | pabelanger: yes | 20:17 |
| mwhahaha | weshay: repo setup is here, http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/job-output.txt.gz#_2018-01-03_11_59_15_592134 | 20:17 |
| mwhahaha | weshay: we run repo-setup on subnode-2 before undercloud, so it's a bit of a mixup in the output | 20:17 |
| * mwhahaha tries to untangle the rats nest | 20:18 | |
| pabelanger | EmilienM: and is the job failing in pre-run? each pre-run, run, post-run all have independent timeouts, so they shouldn't affect the other | 20:18 |
| weshay | sec | 20:18 |
| pabelanger | EmilienM: please show a failure | 20:19 |
| EmilienM | weshay: I have an idea | 20:19 |
| *** jkilpatr has joined #tripleo | 20:21 | |
| pabelanger | if your legacy jobs are still using the timeout setting in devstack-gate, you likely can stop doing that. Zuul will still run a post-run playbook to allow you to collect logs | 20:22 |
| pabelanger | EmilienM: weshay: ^ | 20:22 |
| pabelanger | then you don't need to deal with injob timeouts | 20:22 |
| mwhahaha | we haven't converted log collection to post yet | 20:22 |
| EmilienM | pabelanger: sshnaidm is on it | 20:22 |
| pabelanger | http://git.openstack.org/cgit/openstack-dev/devstack/tree/roles are the roles used to collect logs in devstack zuulv3 jobs, can likely do the same | 20:25 |
| *** raildo has quit IRC | 20:26 | |
| *** raildo has joined #tripleo | 20:26 | |
| *** atoth has quit IRC | 20:31 | |
| mwhahaha | EmilienM: so it seems that the controller step2 takes 50mins, http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-01-03_13_03_43 | 20:32 |
| mwhahaha | er sorry 40 mins | 20:33 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/ocata: Check for yum lock befor all yum* operations. https://review.openstack.org/530725 | 20:36 |
| fultonj | mwhahaha: FWIW during that run it looks like ceph-ansible run took about 4 minutes http://logs.openstack.org/70/529070/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e01762/logs/undercloud/var/log/mistral/ceph-install-workflow.log.txt.gz | 20:37 |
| mwhahaha | yea i'm wondering if it's pacemaker taking forever or something | 20:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-post: introduce gnocchi_quick_policy https://review.openstack.org/531005 | 20:37 |
| mwhahaha | the workflow bits only took <4 mins to complete | 20:38 |
| mwhahaha | so it's litterall step2 of the controller took at least 37 mins | 20:38 |
| fultonj | i wonder what it is | 20:38 |
| mwhahaha | yea now the fun begins | 20:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: set gnocchi_quick_policy to true https://review.openstack.org/531007 | 20:38 |
| EmilienM | mwhahaha: let's see how https://review.openstack.org/#/c/531007 works | 20:39 |
| mwhahaha | EmilienM: yea those are good improvements but i think it's step2 that eating all the time up | 20:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-post: introduce gnocchi_quick_policy https://review.openstack.org/531005 | 20:40 |
| mwhahaha | which is terrible for pacemaker on a single node to take that long to get it's act together | 20:40 |
| EmilienM | mwhahaha: I know... I just applied what sileht told us to do | 20:40 |
| EmilienM | we should stop deploying pacemaker on multinode then? | 20:40 |
| EmilienM | we have ovb already | 20:40 |
| mwhahaha | i don't know if that's the cause | 20:40 |
| mwhahaha | but it's something in step2 | 20:41 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: set gnocchi_quick_policy to true https://review.openstack.org/531007 | 20:41 |
| * mwhahaha is reviewing the logs | 20:41 | |
| mwhahaha | it would be awesome if these weren't all in journal.txt so we could actually follow wtf is happening | 20:41 |
| mwhahaha | we should enable puppet timings | 20:45 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 20:45 |
| mwhahaha | or whatever we turned on in p-o-i | 20:45 |
| mwhahaha | the resource timings | 20:45 |
| mwhahaha | weshay: how do you get the timings in quickstart for the ansible tasks? Is that something we can enable in heat somewhere so we can see how long it's taking to run these things? | 20:46 |
| EmilienM | mwhahaha: what's that | 20:46 |
| * EmilienM go look | 20:46 | |
| EmilienM | --summarize ? | 20:46 |
| mwhahaha | yea | 20:46 |
| mwhahaha | it prints out at the end how long the resources took | 20:47 |
| EmilienM | I don't see it in https://puppet.com/docs/puppet/5.3/man/apply.html | 20:47 |
| mwhahaha | it's one of the not-documented things | 20:47 |
| *** jbadiapa has quit IRC | 20:48 | |
| mwhahaha | where did we add that anyway | 20:49 |
| *** portdirect has quit IRC | 20:50 | |
| EmilienM | mwhahaha: where is the report in p-o-i? | 20:50 |
| EmilienM | I don't see it | 20:50 |
| mwhahaha | yea i'm trying to find it | 20:50 |
| *** portdirect has joined #tripleo | 20:50 | |
| mwhahaha | EmilienM: http://logs.openstack.org/77/529677/1/check/puppet-openstack-integration-4-scenario002-tempest-centos-7/4481d53/job-output.txt.gz#_2017-12-23_02_08_15_882348 | 20:51 |
| EmilienM | yeah | 20:51 |
| EmilienM | nice | 20:51 |
| EmilienM | we should have that | 20:52 |
| EmilienM | mwhahaha: you on it or? | 20:52 |
| mwhahaha | you do it | 20:52 |
| mwhahaha | :D | 20:52 |
| mwhahaha | or i can | 20:52 |
| mwhahaha | have to find where we burried the puppet apply again | 20:52 |
| EmilienM | ok I can do it and then go lunch | 20:52 |
| mwhahaha | cause we're not using the heat one | 20:52 |
| EmilienM | it's in THT no? | 20:52 |
| mwhahaha | maybe? | 20:52 |
| mwhahaha | remember when i said we need to simplify things | 20:53 |
| EmilienM | in common/deploy-steps-tasks.yaml | 20:53 |
| mwhahaha | cause we keep moving it around | 20:53 |
| weshay | sorry back | 20:56 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: puppet apply: add --summarize https://review.openstack.org/531013 | 20:56 |
| EmilienM | mwhahaha: something like this ^ | 20:56 |
| *** tzumainn has quit IRC | 20:56 | |
| mwhahaha | EmilienM: yea | 20:56 |
| mwhahaha | EmilienM: though i have a feeing it's not that and it's actually the docker tasks (and paunch stuff) | 20:57 |
| EmilienM | I think it's docker/docker-puppet.py mostly | 20:57 |
| mwhahaha | yea that logging is a bit light | 20:57 |
| EmilienM | https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps-tasks.yaml#n34 | 20:58 |
| EmilienM | I think that's this one ^ | 20:58 |
| mwhahaha | it's likely that | 20:58 |
| mwhahaha | the host puppet run took 55 seconds | 20:58 |
| mwhahaha | i just found the line in the logs | 20:59 |
| mwhahaha | of course docker-puppet logs next to nothing | 20:59 |
| mwhahaha | wait why aren't we running docker-puppet in debug | 21:00 |
| mwhahaha | DockerPuppetDebug | 21:01 |
| mwhahaha | because of course why wouldn't it honor Debug | 21:02 |
| *** ccamacho has quit IRC | 21:03 | |
| mwhahaha | where is the debug flag in quickstart | 21:04 |
| EmilienM | not sure we have | 21:04 |
| mwhahaha | ಠ_ಠ | 21:04 |
| EmilienM | DockerPuppetDebug should be set to true in tht/ci/environments? | 21:04 |
| mwhahaha | evidently | 21:04 |
| mwhahaha | otherwise it logs nothing | 21:05 |
| EmilienM | ok I'm sending a patch | 21:05 |
| weshay | mwhahaha, -v | 21:05 |
| EmilienM | wait, DockerPuppetDebug isn't used anywhere :D | 21:05 |
| EmilienM | don't tell mwhahaha | 21:05 |
| mwhahaha | yes it is | 21:05 |
| mwhahaha | i just grepped it | 21:05 |
| EmilienM | http://codesearch.openstack.org/?q=DockerPuppetDebug&i=nope&files=&repos= | 21:05 |
| EmilienM | yeah but not used anywhere | 21:05 |
| mwhahaha | yes it is | 21:05 |
| weshay | to enable the deployment with debug? | 21:05 |
| mwhahaha | weshay: yes | 21:06 |
| *** tzumainn has joined #tripleo | 21:06 | |
| weshay | k.. sec | 21:06 |
| mwhahaha | EmilienM: http://codesearch.openstack.org/?q=docker_puppet_debug&i=nope&files=&repos= | 21:06 |
| EmilienM | mwhahaha: right but no scenario sets it to True right? | 21:06 |
| EmilienM | it's false by default | 21:06 |
| mwhahaha | EmilienM: right that's the problem | 21:06 |
| mwhahaha | so 1) it should also honor the Debug param | 21:07 |
| mwhahaha | and 2) we don't appear to be configuring Debug in quickstart anywhere | 21:07 |
| EmilienM | I'll let you send patches for this one, I'm going lunch now | 21:07 |
| mwhahaha | or have the ability to do so, unless it's in a config file somewhere? | 21:07 |
| * EmilienM afk | 21:07 | |
| mwhahaha | not that quickstart should actually set the Debug but i'm trying to find where we're doing that for our CI | 21:07 |
| mwhahaha | so that lives in THT/ci/environments right? | 21:08 |
| mwhahaha | that's how we're setting Debug? | 21:08 |
| EmilienM | yes | 21:08 |
| EmilienM | we already set Debug param | 21:08 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 21:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Enable DockerPuppetDebug https://review.openstack.org/531015 | 21:11 |
| *** yamahata has quit IRC | 21:14 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Convert nic-config templates https://review.openstack.org/531016 | 21:15 |
| weshay | mwhahaha, I think this is what EmilienM was referring to https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode-containers.yaml#L99 | 21:20 |
| *** pchavva has quit IRC | 21:21 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Convert ipv6 nic-config templates https://review.openstack.org/531019 | 21:23 |
| *** dhill_ has quit IRC | 21:26 | |
| *** dhill_ has joined #tripleo | 21:27 | |
| mwhahaha | weshay: yea i found it, https://review.openstack.org/531015 | 21:30 |
| *** chlong has quit IRC | 21:30 | |
| *** holser__ has joined #tripleo | 21:33 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Adds Workflow to List Available Roles https://review.openstack.org/509832 | 21:34 |
| *** jtomasek has quit IRC | 21:34 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Adds UpdateRoles Workflow https://review.openstack.org/515045 | 21:34 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Select Roles Workflow https://review.openstack.org/528282 | 21:34 |
| *** raildo has quit IRC | 21:36 | |
| *** gfidente|afk has quit IRC | 21:38 | |
| weshay | thrash, it failed again eh? | 21:41 |
| thrash | weshay: looks like it... But haven't investigated fully yet. | 21:42 |
| thrash | weshay: I may have rechecked too soon earlier | 21:42 |
| weshay | itchy trigger finger, saw that | 21:42 |
| *** chlong has joined #tripleo | 21:45 | |
| *** Goneri has quit IRC | 21:51 | |
| *** jcoufal has quit IRC | 21:56 | |
| *** yamahata has joined #tripleo | 21:57 | |
| *** aputtur_ has quit IRC | 21:59 | |
| openstackgerrit | Merged openstack/tripleo-image-elements master: Exclude 'vnet' interfaces from being candidates for running a dhcp client. https://review.openstack.org/519283 | 22:00 |
| weshay | thrash, I'll recheck after I see https://review.openstack.org/#/c/530783/ merge | 22:01 |
| weshay | which is close | 22:01 |
| weshay | hopefully | 22:01 |
| *** rcernin has joined #tripleo | 22:05 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 22:10 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Convert ipv6 nic-config templates https://review.openstack.org/531019 | 22:11 |
| openstackgerrit | James Slagle proposed openstack/tripleo-quickstart master: DO NOT MERGE: test ovb-ha with config-download https://review.openstack.org/525331 | 22:13 |
| weshay | EmilienM, for the sprint starting tomorrow https://trello.com/b/U1ITy0cu/tripleo-ci-squad?menu=filter&filter=label:Sprint%206 | 22:13 |
| weshay | EmilienM, I can add some cleanup for toci | 22:13 |
| weshay | maybe that will speed it up | 22:13 |
| EmilienM | weshay: yeah good | 22:14 |
| *** bfournie1 has quit IRC | 22:16 | |
| mwhahaha | EmilienM who owns memcache itself? no one? | 22:21 |
| EmilienM | not sure, pidone? | 22:21 |
| EmilienM | what's the issue | 22:21 |
| EmilienM | ? | 22:22 |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates master: Adds SSL/TLS everywhere for OpenDaylight https://review.openstack.org/531026 | 22:23 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: add env for ci.centos to turn off tempest https://review.openstack.org/531027 | 22:27 |
| *** holser___ has joined #tripleo | 22:30 | |
| *** holser__ has quit IRC | 22:34 | |
| *** abishop has quit IRC | 22:38 | |
| *** dhill_ has quit IRC | 22:43 | |
| *** karthiks has quit IRC | 22:43 | |
| *** hjensas has joined #tripleo | 22:43 | |
| *** hjensas has joined #tripleo | 22:43 | |
| *** dhill_ has joined #tripleo | 22:44 | |
| *** moshele has quit IRC | 22:48 | |
| *** almondjoy has quit IRC | 22:52 | |
| *** hjensas has quit IRC | 22:56 | |
| *** hjensas has joined #tripleo | 22:56 | |
| *** hjensas has joined #tripleo | 22:56 | |
| *** karthiks has joined #tripleo | 22:56 | |
| *** rlandy is now known as rlandy|bbl | 22:58 | |
| *** chlong has quit IRC | 22:58 | |
| *** jlabarre has quit IRC | 23:00 | |
| *** pradk has quit IRC | 23:02 | |
| *** jlabarre has joined #tripleo | 23:03 | |
| *** jmelvin has quit IRC | 23:06 | |
| EmilienM | https://review.openstack.org/#/c/530783/ is going to land | 23:09 |
| weshay | ok.. we're good to go | 23:10 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1740891 | 23:10 |
| openstack | Launchpad bug 1740891 in tripleo "project_name and user_name no longer set in mistral action context" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| *** weshay changes topic to "CI status: TBD, see alerts and read ML. | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 23:12 | |
| *** pmannidi has joined #tripleo | 23:15 | |
| *** moshele has joined #tripleo | 23:15 | |
| openstackgerrit | Merged openstack/tripleo-common master: Update AnsibleGenerateInventoryAction to match new mistral interfaces https://review.openstack.org/530783 | 23:18 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Collect logs in post playbook https://review.openstack.org/527844 | 23:21 |
| *** sshnaidm is now known as sshnaidm|afk | 23:22 | |
| *** moshele has quit IRC | 23:23 | |
| *** bfournie has joined #tripleo | 23:23 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-common master: Validate roles data and network data https://review.openstack.org/508567 | 23:23 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: [WIP] Add networks directory https://review.openstack.org/520634 | 23:24 |
| *** moshele has joined #tripleo | 23:24 | |
| *** holser___ has quit IRC | 23:27 | |
| *** lblanchard has joined #tripleo | 23:33 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: set telemetry.alarm_granularity to 15s https://review.openstack.org/531002 | 23:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-post: introduce gnocchi_quick_policy https://review.openstack.org/531005 | 23:40 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs016: set gnocchi_quick_policy to true https://review.openstack.org/531007 | 23:42 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Fix docs syntax for OVB jobs https://review.openstack.org/531035 | 23:45 |
| *** tosky has quit IRC | 23:49 | |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Add composable network VIPs for puppet configuration https://review.openstack.org/531036 | 23:50 |
| openstackgerrit | Bob Fournier proposed openstack/puppet-tripleo master: Configure VIPs for all networks including composable networks https://review.openstack.org/531037 | 23:56 |
| EmilienM | mwhahaha: see --summarize: | 23:59 |
| EmilienM | http://logs.openstack.org/13/531013/1/check/tripleo-ci-centos-7-containers-multinode/f9b63b4/logs/subnode-2/var/log/journal.txt.gz#_Jan_03_22_32_20 | 23:59 |
| EmilienM | or here: http://logs.openstack.org/13/531013/1/check/tripleo-ci-centos-7-containers-multinode/f9b63b4/logs/subnode-2/var/log/journal.txt.gz#_Jan_03_23_17_32 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
