Thursday, 2019-06-27

*** sanjayu_ has quit IRC00:02
*** saneax has joined #tripleo00:06
*** openstack has joined #tripleo13:12
*** ChanServ sets mode: +o openstack13:12
gfidentebut sorry those are the featuresets not the scenarios13:12
sshnaidm|ruckcloudnull, it took for me some effort and time to find where a specific role is used13:12
gfidentethough some columns refer to scenarios13:12
sshnaidm|ruckcloudnull, we need to be careful when adding a new role, it's worth to check if it's tested in current jobs13:13
fultonji agree with sshnaidm|ruck13:14
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image delete command  https://review.opendev.org/66339113:14
fultonjif the code from a feature is moved, in theory CI should have been testing that feautre, so it would still be tested13:14
cloudnullits also worth noting that we are running molecule tests for each role we have in tripleo-ansible (with exception of the octavia things), those tests are (or should be) exercising all of the current code paths.13:14
fultonj(at least that's the case for the ceph moves)13:15
sshnaidm|ruckcloudnull, well, molecule didn't catch last failure, it's not a full deployment test though13:15
*** mmethot has quit IRC13:15
fultonjbut molecule is a nice extra test13:15
sshnaidm|ruckyeah, no doubt13:15
cloudnullsshnaidm|ruck yea we didn't have a test for that code path which resulted in breakage13:16
openstackgerritFrancesco Pantano proposed openstack/tripleo-common master: Check third-party container image metadata.  https://review.opendev.org/66783713:16
cloudnullbut i have added one - https://review.opendev.org/#/c/667609/13:16
zbrsshnaidm|ruck: once we have molecule jobs, we should ask people to improve testing when we make bugfixes. The old approach of no-commit without unit-test13:16
sshnaidm|ruckcloudnull, nice13:16
zbronce we have molecule we can apply it for roles: no fix without prevention measures added inside molecule folder.13:16
fultonji see molecule as a supplement but it would be nice if it passed before we ran a heavy deloyment job (we talked about that least week)13:16
cloudnull++ ^13:16
zbrwe are working in this, it will run before deployments.  now jobs run in parallel.13:17
sshnaidm|ruckcloudnull, fultonj about a fail-fast I think infra is against this, zbr talked with them iirc13:17
sshnaidm|ruckif I don't miss something13:17
cloudnullyea we talked to them earlier this week13:17
cloudnullRE fail-fast - https://review.opendev.org/#/c/667631/13:18
sshnaidm|ruckI don't have objections to fail-fast, seems fine13:19
fultonjoh, that's interesting. is infra against the fail fast? (i assumed it meant less resource use)13:19
sshnaidm|ruckfultonj, yeah, but in long range it can take more resources13:20
cloudnullhere was the convo http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2019-06-25.log.html#t2019-06-25T15:51:3413:20
sshnaidm|rucknot sure if it's relevant for us though13:20
cloudnullI did add fail-fast to the gate pipe-line in my follow on review (which needs to be rebased) the intention was to limit resource commitments when there's an obvious failure. that said, im not married to the idea and would be happy to remove .13:21
cloudnulls/gate/check/13:22
ekultailsSounds good to me.13:22
sshnaidm|ruckcloudnull, I think we need to proceed with  https://review.opendev.org/667912 before13:22
cloudnull+113:22
sshnaidm|ruckcloudnull, we don't have a good coverage atm13:23
cloudnull++13:23
sshnaidm|ruckcloudnull, and then let's get back to your patch13:23
cloudnullI had a couple of questions on that review but otherwise I think that's the way to go13:23
sshnaidm|ruckcloudnull, sure, let's talk after a mtg13:24
cloudnull++13:24
sshnaidm|ruckthat's it, is nobody has something to add13:24
*** jfrancoa has joined #tripleo13:24
cloudnullok moving on13:24
cloudnullACTION: cloudnull to figure out trello perms (cloudnull, 13:14:54)13:24
cloudnullAGREED: port the trello board to storyboard (cloudnull, 13:24:44)13:24
*** hberaud has joined #tripleo13:24
cloudnullI did port the trello dashboard to storyboard13:25
cloudnullhttps://storyboard.openstack.org/#!/board/17413:25
cloudnullthe old trello board has been locked https://trello.com/b/tEjJlT3k13:25
cloudnullall of the cards, and task states have been moved over.13:26
cloudnullsadly the card assignments have not, so if you were assigned to a given card before please go update the new cards13:26
fultonjLGTM thanks cloudnull13:26
cloudnullany questions about storyboard ?13:27
cloudnullhere's the link to the active project - https://storyboard.openstack.org/#!/project/openstack/tripleo-ansible13:27
cloudnullok moving on13:28
dpeacockhow do we add ourselves to cards?13:28
cloudnulldpeacock the cards contain tasks so you have to add yourself to the tasks within the card13:28
cloudnulls/card/story/13:28
dpeacockok we can take that offline but I don't see how to do it13:29
cloudnull++ happy to chat about it more13:29
cloudnullthe last action item from last week is13:29
cloudnullACTION: ping pabelanger and pester dmsimard about ARA and molecule (cloudnull, 13:56:23)13:29
cloudnullACTION: sshnaidm to look into ARA (cloudnull, 13:57:26)13:29
sshnaidm|ruckcloudnull, sorry, didn't have a chance to look into it13:30
fultonjdoes anyone have a quick example of linking a change to a story?13:30
cloudnullsshnaidm|ruck no worries.13:30
* fultonj can find it later if not13:30
sshnaidm|ruckcloudnull, let's postpone it for a next week13:30
cloudnullsomething we can tackle next week13:30
cloudnull++13:30
*** hberaud has left #tripleo13:31
cloudnullfultonj https://review.opendev.org/#/c/667609/13:31
*** rcernin has quit IRC13:31
cloudnullin the commit message i reference the story and a given task13:31
*** jistr is now known as jistr|call13:31
openstackgerritFrancesco Pantano proposed openstack/tripleo-common master: Check third-party container image metadata.  https://review.opendev.org/66783713:31
cloudnullso that's it for action items13:31
*** jfrancoa has quit IRC13:31
cloudnullthere was some talk about what jobs we should run, how we should run them, and how many jobs we have. Lets talk about this in depth so we can get to ground on what our go forward strategy is.-cc mwhahaha zbr sshnaidm|ruck13:32
cloudnull^ kinda already touched on this but wanted to continue the conversation, if need be.13:32
zbri also in tripleo-ci planning meeting, but I will do my best to multi-task13:32
zbrcreating one-job per molecule scenario does not scale well (current only tripleo-ansible does that)13:33
zbrothers do run a single molecule job for all scenarios inside a repo13:33
zbrreality is that there is no perfect approach13:33
cloudnullI'd argue that the job per role scales better than the alternative13:34
cloudnullespecially with the jobs being limited based on the files that changed.13:34
zbrcloudnull: you do not get pinged by openstack-infra about how much tripleo consumes of total number of resources13:34
fultonjcloudnull: "jobs being limited based on the files that changed" --> how?13:34
fultonjsounds like a great idea if possible13:35
zbrthat is why we ended up having our own queue in zuul, so they can slow us down,'13:35
cloudnullI used to :)13:35
sshnaidm|ruckfungi, "files" directive13:35
cloudnull^13:35
sshnaidm|ruckfultonj, ^^13:35
fultonjjob.files13:35
cloudnullI did ask about resource usage in the infra channel - http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2019-06-25.log.html#t2019-06-25T15:20:4013:35
mwhahahathe only danger is mass role updates13:35
zbrfiles directive works for *some* cases, historically I seen the filters missing critical parts which could have affecting the outcomes.13:36
fultonj#link https://zuul-ci.org/docs/zuul/user/config.html#job13:36
mwhahahapersonally i'd rather have a single test all the things job, but if it's going to take forever then it's not really worth it13:36
cloudnullmwhahaha at present with the limited roles we have a full run is about 45 minutes13:37
mwhahahameh13:37
cloudnullmaybe we could thread that to make it faster ?13:37
sshnaidm|ruckit's the shortest time in triple repos so far13:37
cloudnullright now its about 1513:37
zbrcloudnull: molecule does not run in parallel now, but a fix is WIP, it will be soon.13:37
*** aakarsh has joined #tripleo13:37
zbrprobably we will fix that issue in less than two months.13:38
cloudnullI think that time will exponentially grow as we add more roles, especially as we create roles from content in THT13:38
ekultailszbr: Is that fix in Zuul/CI or Molecule itself?13:38
cloudnull^ if we add a job runner that runs all the things13:38
zbrmolecule issue related to container names and temp files, but I have someone working on addresing it13:39
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add dashboard component as new ceph composable service.  https://review.opendev.org/66657813:39
zbrwe already know what needs to be done13:39
sshnaidm|ruckzbr, maybe worth to split to 2 molecule jobs?13:39
sshnaidm|ruckor it doesn't work that way13:39
zbrsshnaidm|ruck: yep, this is what i propose: try to use a single job when possible and lest split when needed13:39
zbrwe can easily split them by using pytest -k param13:40
zbrso we could have a single tox environment but run subsets by adding that parameter.13:40
zbrexample: tox -e molecule -- -k myrole13:40
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Queens only: Fixup nova cell0 url fix  https://review.opendev.org/66765113:41
zbrbecause {posargs} are pytest params.13:41
sshnaidm|ruckso we can test half roles in one job and other half in separate parallel job?13:41
*** rajinir has joined #tripleo13:41
zbrsshnaidm|ruck: yes, mainly limted on how we specify the pytest filters.13:41
sshnaidm|ruckack, so we always can scale it if it's too long13:42
zbrbut I would still want to be able to list all available test by calling: pytest --collect-only --- now it works quite nice.13:42
zbronce we have parallel run sorted, we will no longer need multiple jobs.13:42
cloudnullzbr I think we still will . unless that multi-job runner will allow for tests not running in containers.13:43
cloudnullnot everything tests in containers13:43
cloudnullso maybe we could glob all of the container based tests ?13:44
zbrcloudnull: some will need libvit or external clouds but I would be against requiring root access on the host itself, is not secure.13:44
cloudnull^ why?13:45
zbrcloudnull: yes, i would expose provider type in the dicovered pytest, easing the filtering by provider.13:45
cloudnullmost all tests in openstack run on the local test instance13:45
sshnaidm|ruckzbr, cloudnull about topic raised by ekultails of testing docker services - is it too crazy idea to run a multinode molecule job when using secondary node as testing node with delegated driver?13:45
zbrI do not want to touch/mess developer machine when they do "tox -e mocule"13:45
*** Goneri has joined #tripleo13:45
zbrthat is why I assume that you do not have root access to localhost (machine running tox)13:45
*** jfrancoa has joined #tripleo13:46
*** pcaruana has quit IRC13:46
zbrzuul does not give root to these for similar security reasons, that is why zuul has the pre-run phase which installs needed packages but it does not allow user to sudo.13:46
cloudnull^ yes it does13:46
*** pcaruana has joined #tripleo13:47
zbrnot on tox jobs.13:47
sshnaidm|ruckcloudnull, not on executor afaik13:47
sshnaidm|ruckbut on nodepool node13:47
cloudnulltox jobs have sudo, you just need to use the openstack-tox-with-sudo job parent13:47
cloudnullhttps://github.com/openstack/tripleo-ansible/blob/master/zuul.d/base.yaml#L1113:48
zbrso by default not, and I am kinda happy that's not widespread, I would have to see someone adding "sudo" commands inside a tox.ini file ;)13:48
cloudnullthat job is used in a lot of places throught the general openstack community, so i dont think its a security concern, especially not in CI13:49
cloudnullsshnaidm|ruck regarding docker in docker, maybe ?13:50
sshnaidm|ruckcloudnull, yeah13:50
zbrtbh, i do not see lack of sudo as a bug, i see it as a feature, for both devenv and ci. molecule is supposed to run as unittest, meaning that each time it runs it should run on the same environment, if we alter the host, we would not be able to revert it to initial state after the first run.13:50
cloudnullhowever, there are other things that dont want to run in containers, but can be made to by messing with capabilities.13:50
sshnaidm|ruckcloudnull, yeah, but systemd is harsh..13:51
cloudnullzbr its like a unit test. however ansible requires more of a functional test.13:51
zbrDoD is quite problematic, and with platform specific issues (debian vs redhat --- different fs)13:51
cloudnullunless we want to go down the path of using something like rubocop13:51
zbrthat is why it does its own provisioning of containers, VMs, from various sources. i want to keep molecule in charge of provisioning.13:52
zbrsshnaidm|ruck: did something interesting "virtainers" :D13:53
cloudnullzbr but it also can be delegated, requiring nested virt to test our roles, or an external cloud, seems like a non-starter13:53
sshnaidm|ruckyeah, but it's still little bit early to try it, still testing it in CI13:54
zbrone way or another the provisioning of the VM needed for testing has to be done by somoene: either libvirt (virtualbox provisioner), or a cloud,...13:54
cloudnullsshnaidm|ruck - https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tuned/molecule/default/molecule.yml#L28-L45 - maybe that will help, its one of the roles that is using systemd in containers13:54
ekultailsI manually spin up a Libvirt virtual machine with Vagrant for my weird Molecule tests that use root access (as to not ruin my workstation). Not to be confused with the vagrant-libvirt driver, I'm using the deletegated/ansible_connection: local way. It works for now. I wish there was an easier approach to it working exactly the same on my workstation as it does in CI by just running `tox -e molecule` (without having to manually setup inf13:55
ekultailsrastructure). I can't use vagrant-libvirt because it's not available in CI. I can't use OpenStack instances because I don't always have a cloud running at my home.13:55
zbrdelegated provisioneer is bit tricky as it assumes someone else provisioned the machine for you, but that means you cannot "reset" it.13:55
sshnaidm|ruckcloudnull, yeah, I ran systemd enabled containers, but was always mess when you are going to start adn restart docker services with systemd in role13:55
cloudnullzbr I think your seeing it from your developer workstation point of view, in test, zuul provisions the resource we use.13:55
zbrcloudnull: i try to look from both points of view, if we do not cover boths we fail.13:56
ekultailsThe use case of systemd+docker inside a container doesn't always work as you'd expect, which caused some headaches for me.13:56
sshnaidm|ruck+1 ^^13:56
zbrwe need to assure that developers can run "tox -e molecule" without being worried that this would mess their machine, its simplicity is key in its success.13:57
cloudnullI personally mimic what the gate gives me, in a process similar to ekultails, so from my point of view I'm never messing with my underlying workstation and if i need to reset things i just make another vm.13:57
cloudnulland would never run tox on my local machine13:57
cloudnullso maybe we add a "are you really sure" for molecule tests that dont run in contianers?13:58
cloudnullI really dont want to see us come up with some crazy fragil thing just to satisfy the urge to run everything in containers.13:59
zbrcloudnull: yep, or auto-skip them when the prerequisites are not met.13:59
cloudnulland I'd hate for us to require folks to use external resources to test things13:59
zbri can easily skip a test if it requires a provider that is not configured yet.13:59
cloudnullor limit where our jobs run because they require nested virt14:00
zbrhaving a libvirt host is not such a big deal of a requirement14:00
cloudnullit is in the our resource pool14:00
cloudnullalmost no clouds provide it14:00
sshnaidm|ruck*maybe 3party CI will provide it soon14:01
sshnaidm|ruckbut not now14:01
ekultailsAs a developer, I don't think I'd ever expect a tox test to run sudo on my local workstation. I'm all for having some kind of check to not run those special tests locally by default.14:01
zbrone real limitation with molecule is that you need to mention the provider in molecule.yml, meaning that you decide it when you write the test. no support for fallback/alternative providers yet.14:01
*** avivgts has quit IRC14:01
ekultailszbr: Not yet, anyways. We can fix that. ;-)14:02
cloudnullwelp our meeting is out of time. but im happy to continue the conversation given there was no meet bot :)14:02
*** ykarel|afk is now known as ykarel14:02
cloudnullif folks need to run, I understand, thanks all !14:02
fultonj#action fultonj land https://review.opendev.org/#/q/topic:ceph_refactor before next meeting14:02
zbryeah, with time we can. in fact we can override it by defining MOLECULE_PROVISIONEER if I remember well, Could be a trick to implement fallbacks,14:03
cloudnull++14:03
zbrindeed, thanks to all, this meeting is really useful.14:04
cloudnullI'm all for setting some kind of guard and overriding it in zuul config so we can run the normal tests and protect the developer workstation14:04
zbrcloudnull: sounds like a good approach. we can always adapt zuul behavior. we have far less control over what developer has on his machine.14:05
*** blueDuckRandom has quit IRC14:05
cloudnullekultails to be clear, the tox tests do no actually run sudo, the ansible playbooks however use become which executes sudo14:06
zbrbut we can follow the approach I used for docker, where I detect (in pytest-molecule) if the developer has a working docker setup (local or remote) before triggering the tests. We can do similar stuff about libvirt, cloud.yml and use whatever we can find, or fail with a clear message.14:06
cloudnullthe openstack-tox-with-sudo job just ensures the test environment is allowed to execute sudo14:06
zbrlike "molecule needs a VM provisioner in order to run ...., be sure you setup a working A, B or C"14:06
sshnaidm|ruckcloudnull, looks weird: https://github.com/openstack/openstack-zuul-jobs/blob/0e4136f476eaddd77f5242dd33015302cba9e373/playbooks/tox-with-sudo/run.yaml14:07
sshnaidm|ruckit's just a tox role14:08
cloudnullyup it just installs tox.14:08
cloudnullthe job keeps sudo available14:08
cloudnullso tox -e thing -- is not running sudo cmd;.14:09
cloudnullhowever in our case the playbook may use "become" which will fail if sudo is not allowed14:10
cloudnullzbr so do you think you could take a swing at updating our tox config so the following jobs are not run with a basic tox -e ... command ?14:12
cloudnullhttps://github.com/openstack/tripleo-ansible/blob/master/zuul.d/molecule.yaml#L110-L12314:12
cloudnullsshnaidm|ruck do you want to chat about https://review.opendev.org/#/c/667912 right quick ?14:14
sshnaidm|ruckcloudnull, sure14:14
sshnaidm|ruckcloudnull, exactly commenting there :)14:15
cloudnullmy one question was I don't think we need to execute the scenarios when the zuul config changes14:15
sshnaidm|ruckcloudnull, about zuul.d - When you change jobs set or definition it's always good to run it to be sure you didn't break anything, like other jobs and also to check what you configured in place14:15
sshnaidm|ruckcloudnull, if you add some job or changed its vars - need to run it14:16
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722114:16
sshnaidm|ruckcloudnull, about octavia you're totally right, fixing it now14:17
cloudnullcouldn't we run some kind of a DNM change to test things out without requiring those more heavy jobs?14:18
cloudnullmake a DNM review, see the jobs spin up, abandon it.14:19
cloudnullin that way we're not waiting for multiple hours when we make a zuul config change?14:19
*** vinaykns has joined #tripleo14:19
cloudnull^ feel free to tell me to be quite :)14:20
sshnaidm|ruckcloudnull, I'd like not to leave it uncovered, it's kind of standard and people will forget to run DNM patches :) Also it's not too much changing part of the repo14:21
sshnaidm|ruckcloudnull, usually after things are settled, we don't change layout much14:22
sshnaidm|ruckcloudnull, the idea is to see in CI result of your change right now, not to make additional effort of dummy patches14:23
Tengusshnaidm|ruck: heya! quick thing, I've hit for the second time this exact issue for a patch: http://logs.openstack.org/62/667562/3/check/tripleo-ci-centos-7-undercloud-containers/ff7c126/logs/undercloud/home/zuul/undercloud_install.log.gz#_2019-06-27_08_36_08  is it known?14:23
cloudnullsshnaidm|ruck ++14:23
sshnaidm|ruckTengu, Timeout on http://mirror.iad.rax.opendev.org/centos/7/os/x86_64/Packages/tmpwatch-2.11-5.el7.x86_64.rpm:14:24
Tengusshnaidm|ruck: yep14:24
Tengusecond time on that same package :/14:24
sshnaidm|ruckTengu, seems like infra has problems with iax again, they are not stable in last week14:24
Tenguok14:24
mwhahahawe should ping infra on the iad.rax issues14:25
sshnaidm|ruckTengu, changed something in mirrors..14:25
zbrcloudnull: yes I can do the tox conversion stuff, no worry, i will do it on monday.14:27
sshnaidm|ruckmwhahaha, of course14:27
cloudnullzbr thanks!14:27
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: Add standalone jobs to gate tripleo-ansible  https://review.opendev.org/66791214:27
sshnaidm|ruckcloudnull, ^^14:27
sshnaidm|ruckwill run some change on octavia to see if regex is good..14:28
*** rainleander has quit IRC14:28
cloudnull++ LGTM14:28
*** mmethot has joined #tripleo14:28
cloudnullthanks!14:28
*** jistr|call is now known as jistr14:31
cloudnulldpeacock want to sync up about storyboard?14:31
*** mmethot has quit IRC14:33
openstackgerritCédric Jeanneret (Tengu) proposed openstack/python-tripleoclient master: Ensure files exist before building/uploading plan  https://review.opendev.org/66756214:34
*** mcornea has quit IRC14:35
dpeacockcloudnull: sure - it's only a simple thing really; how does one add their name to a "card" or "story" if those are equivalent?14:36
*** cgrosjean has quit IRC14:39
*** cgrosjea_ has joined #tripleo14:39
*** mmethot has joined #tripleo14:42
ykareldtantsur, can u also get the fix https://review.opendev.org/#/c/667531/2 released14:42
ykarelin the meanwhile i fired a test tripleo patch https://review.rdoproject.org/r/#/c/21064/14:43
cloudnullfrom the board https://storyboard.openstack.org/#!/board/174, if you click on the card all you will only see the details. you have to open the story to see the tasks and the option to assign yourself to a given task14:43
ykarelsshnaidm|ruck, fyi release request ^^14:44
cloudnulldpeacock you'll see the link adjacent to "Task in the story"14:44
cloudnullits more steps than trello, which is kinda a bummer14:45
sshnaidm|ruckykarel, ack14:45
cloudnullbut overall it seems to work well14:45
*** cgrosjea_ has quit IRC14:45
*** ykarel is now known as ykarel|away14:46
*** cgrosjean has joined #tripleo14:47
openstackgerritEduard Barrera proposed openstack/python-tripleoclient master: Ensure password file exists  https://review.opendev.org/66761514:47
*** mcornea has joined #tripleo14:49
*** cgrosjea_ has joined #tripleo14:51
*** cgrosjean has quit IRC14:51
*** xek__ has quit IRC14:53
*** cgrosjea_ has quit IRC14:55
*** xek has joined #tripleo14:55
*** ratailor has joined #tripleo14:56
*** xek_ has joined #tripleo14:58
*** xek has quit IRC15:01
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add dashboard component as new ceph composable service.  https://review.opendev.org/66657815:02
*** cgrosjean has joined #tripleo15:11
*** hamzy has quit IRC15:13
*** pcaruana has quit IRC15:14
*** Vorrtex has joined #tripleo15:17
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add dashboard component as new ceph composable service.  https://review.opendev.org/66657815:18
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Update octavia to new tripleo-ansible PATH  https://review.opendev.org/66724715:24
*** udesale has joined #tripleo15:25
sshnaidm|ruckmwhahaha, so do you want me to add "tripleo" queue there or not? https://review.opendev.org/#/c/667912/2/zuul.d/layout.yaml15:28
*** kopecmartin is now known as kopecmartin|off15:28
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: DNM: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753715:29
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: DNM: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753715:30
mwhahahait's probably a good idea15:31
sshnaidm|ruckmwhahaha, or maybe not let slow down jobs..15:32
mwhahahai mean it's unlikely we'll hang something15:33
mwhahahabut it might happen15:33
mwhahahai guess we can roll with it for now but we should follow up when we're done making large changes15:33
mwhahahacloudnull: -^15:33
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Send terminal erase sequence on tripleo deploy  https://review.opendev.org/66617215:33
cloudnull++15:34
cloudnullI'd hate to slow things down too much, but if we need to, so be it15:34
*** chandankumar is now known as raukadah15:34
*** ayoung has quit IRC15:35
* cloudnull doesn't understand why the tripleo queue is slow to begin executing jobs?15:36
cloudnullis that a product of the aforementioned resource consumption ?15:37
openstackgerritMarios Andreou proposed openstack/tripleo-ci master: WIP: move buildimages to a role  https://review.opendev.org/66796615:38
mwhahahacloudnull: yea it's because they get grouped together and because we have sooooo many jobs we get limited15:40
mwhahahathis is why we need to smart about the jobs we actually include and not use a shotgun approach15:40
*** hamzy has joined #tripleo15:40
mwhahahaunfortunately things like tripleo-{quickstart,quickstart-extras,ci} run the world15:40
cloudnull++15:41
mnasercan someone run https://review.opendev.org/#/c/661994/ through the magical CI on rhel8 to test if it works?15:42
cloudnullIf we can get away without adding our jobs to the tripleo queue it'd be great. At least for now.15:42
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add dashboard component as new ceph composable service.  https://review.opendev.org/66657815:42
mnasersomeone has a comment testing it internally and itd be nice to at least get that fired up15:43
*** yprokule has quit IRC15:43
*** marios is now known as marios|out15:49
*** holser__ has joined #tripleo15:50
*** lucasagomes has quit IRC15:50
*** cylopez has joined #tripleo15:50
*** holser_ has quit IRC15:53
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Remove undercloud deploy support  https://review.opendev.org/66796715:54
*** gfidente has quit IRC15:55
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: DNM: test octavia job run  https://review.opendev.org/66796915:55
*** ccamacho has quit IRC15:55
VorrtexTengu gchamoul I was pointed in your general directions for tripleo validation errors.  Specifically I'm getting errors on any "send_message" execution.  I thought this was because ansible wasn't configured to target the local machine, but after making a change I thought would work to make that happen, it doesn't seem to be working.  Any initial troubleshooting steps I should look through?  I'm pretty new to all this and have15:55
Vorrtex very little experience with mistral and tripleo in general.15:55
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Add Octavia driver agent service  https://review.opendev.org/65811815:56
openstackgerritMerged openstack/ansible-role-redhat-subscription master: register: add retry mechanism  https://review.opendev.org/66744015:57
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: DNM: test octavia job run  https://review.opendev.org/66796915:57
*** ratailor has quit IRC15:57
openstackgerritBrent Eagles proposed openstack/tripleo-ci master: WIP: Octavia tempest tests  https://review.opendev.org/66664215:57
gchamoulVorrtex: Hi how do you run the validation?  through the openstack tripleo validator command?15:58
*** marios|out has quit IRC15:58
Vorrtexgchamoul specifically made an alias for easy repeating:  alias prevalidate='openstack workflow execution create tripleo.validations.v1.run_groups '\''{"group_names": ["pre-deployment"]}'\'''15:58
*** artom|gmtplus3 has quit IRC15:59
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722116:00
gchamoulVorrtex: which release are you using?16:00
VorrtexIts possible I've "zeroed in" on the wrong portions of these errors, but being that they're all "send_message" I thought that was pertinent.  Looking closer at those errors I can see it complaining about substantial things, like, "Role 'Controller' is in use, but has no flavor assigned" in one of the errors.  So does "send_message" just dictate essentially recording when a failure has occurred and is being logged or sent?16:00
Vorrtexgchamoul was working through this:  http://tripleo.org/install/basic_deployment/basic_deployment_cli.html#validate-the-overcloud16:01
VorrtexBut with just the default instructions, not a specific release.16:01
VorrtexSo current dev, right?16:01
VorrtexSorry, current master?16:01
gchamoulVorrtex: so if you are using master or stein release, you are now able to run the validations through the tripleoclient command16:03
Vorrtexgchamoul I'm not sure what you mean.16:04
*** jcoufal has joined #tripleo16:05
gchamoulVorrtex: I mean if you use a recent release, you should have "openstack tripleo validator" subcommand16:05
*** hberaud has joined #tripleo16:05
gchamoulVorrtex: on your undercloud16:05
*** sshnaidm|ruck is now known as sshnaidm|off16:06
Vorrtexgchamoul yeah, that's available.  Does that fix the errors I'm looking to fix or does it help me narrow the scope of the request or something?16:06
gchamoulVorrtex: nope, but there is a fix for that (https://review.opendev.org/#/c/659597/)16:07
gchamoulVorrtex: still in review, not merged yet16:08
*** pbandark has joined #tripleo16:08
*** bogdando has quit IRC16:08
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: DNM: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753716:08
gchamoulVorrtex: I am aware about the send_message action from mistral, especially when the workflow fails16:08
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: DNM: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753716:09
*** avivgta has quit IRC16:10
openstackgerritBrent Eagles proposed openstack/tripleo-ci master: Add new scenario007-standalone job definition & add to check  https://review.opendev.org/63149716:10
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: Implement tripleo container image delete command  https://review.opendev.org/66339116:11
gchamoulwhich Vorrtex16:12
gchamouloops16:12
Vorrtexgchamoul I'm the only me ;)16:12
Vorrtexgchamoul so I grabbed the error messages I'm looking at, because I'm trying to figure out how to better diagnose the problems I'm seeing, not looking for a fix to the way errors were being reported, if that makes sense.  https://gist.github.com/tvardema/ef7df96ad208090fb3954583b59a36ab16:14
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add standalone jobs to gate tripleo-ansible  https://review.opendev.org/66791216:17
*** xek__ has joined #tripleo16:20
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add standalone jobs to gate tripleo-ansible  https://review.opendev.org/66791216:20
*** jpich has quit IRC16:20
*** jpena|mtg is now known as jpena|off16:21
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Update jobs so we're targeting only what we need  https://review.opendev.org/66763116:22
*** xek_ has quit IRC16:23
gchamoulVorrtex: I am trying to do the same on my dev env, not ignoring yourself16:24
cloudnullany chance cores can give this a review https://review.opendev.org/#/c/667609 ?16:25
Vorrtexgchamoul no problem at all, just not sure where to go from here.16:25
Vorrtexgchamoul I suppose it might help if I link you how I made my undercloud as well:  https://docs.openstack.org/tripleo-docs/latest/install/installation/installation.html16:25
VorrtexAgain, with just the master deployment.16:26
VorrtexAll done on one VM.16:26
gchamoulVorrtex: so you just have an Undercloud installed, right?16:27
*** amoralej is now known as amoralej|off16:27
*** cylopez has quit IRC16:29
*** panda has quit IRC16:32
*** cgrosjean has quit IRC16:34
*** panda has joined #tripleo16:36
Vorrtexgchamoul that's correct.  It **could** be improperly configured or something along those lines, and if that's the case then I can handle these errors one at a time, just wanted to make sure the validations weren't the thing that was failing.16:39
Vorrtexwasn't the thing***16:39
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Use the UBI8 image for testing  https://review.opendev.org/66567716:40
gchamoulVorrtex: the validations don't seem to be run correctly. what's the output of the "tripleo-ansible-inventory --list | jq ." command on your undercloud?16:41
Vorrtexgchamoul updated at the bottom of this gist:  https://gist.github.com/tvardema/ef7df96ad208090fb3954583b59a36ab16:43
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Use the UBI8 image for testing  https://review.opendev.org/66567716:44
gchamoulVorrtex: sorry but it's time to commute back home for me. Please ping me tomorrow when you will be available on this channel?16:49
gchamoulVorrtex: we will keep looking at your issues16:50
*** gkadam has quit IRC16:51
Vorrtexgchamoul thanks, I'm in the US, central time, but I'll do my best to get online earlier and get your attention sooner.16:51
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Use the UBI8 image for testing  https://review.opendev.org/66567716:51
gchamoulVorrtex: ack see you tomorrow then!16:51
*** udesale has quit IRC16:54
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: remove 1831866 from the skip list  https://review.opendev.org/66798716:57
*** jcoufal_ has joined #tripleo16:59
*** jcoufal has quit IRC17:02
*** xek_ has joined #tripleo17:02
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image show command  https://review.opendev.org/66798817:03
*** ramishra has quit IRC17:03
*** xek__ has quit IRC17:05
openstackgerritwes hayutin proposed openstack/tripleo-common master: Revert "remove plotnetcfg from overcloud image yaml"  https://review.opendev.org/66799117:06
*** hamzy has quit IRC17:07
*** hamzy has joined #tripleo17:12
*** ayoung has joined #tripleo17:14
mwhahahaweshay: why https://review.opendev.org/#/c/667991/17:15
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Update jobs so we're targeting only what we need  https://review.opendev.org/66763117:20
*** hberaud has quit IRC17:24
weshaymwhahaha just going to check what happens w/ rhel8 now that the rpm is avail17:24
weshaymwhahaha I'll put a DNM on it17:24
weshayassaf and others are concerned about removing it.. notes in the cix card17:24
* weshay just wants the info atm17:25
weshaydon't really care much17:25
*** hberaud has joined #tripleo17:25
weshaythey can carry the patch internally17:25
mwhahahaconcerned even though it hasn't been available in many a cycle17:25
weshayfor sure.. I know.. just getting info17:25
mwhahahaಠ_ಠ17:25
openstackgerritwes hayutin proposed openstack/tripleo-common master: DNM, just testing Revert "remove plotnetcfg from overcloud image yaml"  https://review.opendev.org/66799117:25
*** holser__ has quit IRC17:26
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753717:28
*** rlandy is now known as rlandy|brb17:29
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart master: Remove featureset057  https://review.opendev.org/66799717:32
*** lmiccini has quit IRC17:32
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Remove reference to tempest-tripleo-ui  https://review.opendev.org/66799917:33
*** lmiccini has joined #tripleo17:39
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Use the UBI8 image for testing  https://review.opendev.org/66567717:41
*** xek_ has quit IRC17:43
*** rlandy|brb is now known as rlandy17:45
openstackgerritMerged openstack/tripleo-common master: Add all_nodes template to tripleo-hieradata.  https://review.opendev.org/66517317:46
*** ykarel|away has quit IRC17:51
*** pcaruana has joined #tripleo17:54
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Update jobs so we're targeting only what we need  https://review.opendev.org/66763117:59
openstackgerritDirk Mueller proposed openstack/diskimage-builder master: Enable nodepool testing for opensuse 15.1  https://review.opendev.org/66753718:00
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Use the UBI8 image for testing  https://review.opendev.org/66567718:02
openstackgerritMerged openstack/tripleo-common master: Add vip_data template to tripleo-hieradata  https://review.opendev.org/66644518:03
*** cgrosjean has joined #tripleo18:04
*** jcoufal_ has quit IRC18:08
openstackgerritSorin Sbarnea proposed openstack/tripleo-repos master: [WIP] add support for RHEL8  https://review.opendev.org/66660718:08
*** hberaud is now known as hberaud|gone18:11
openstackgerritJames Slagle proposed openstack/tripleo-common master: Don't calculate vips for some services  https://review.opendev.org/66800918:17
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for vip_data hieradata  https://review.opendev.org/66644718:18
*** hamzy has quit IRC18:24
*** hamzy has joined #tripleo18:25
openstackgerritwes hayutin proposed openstack/tripleo-repos master: [WIP] add support for RHEL8  https://review.opendev.org/66801518:29
*** hamzy has quit IRC18:30
*** suuuper has quit IRC18:34
*** khyr0n has quit IRC18:41
*** khyr0n has joined #tripleo18:43
openstackgerritMerged openstack/tripleo-heat-templates stable/stein: Make comparisons case insensitive  https://review.opendev.org/66762218:43
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Make comparisons case insensitive  https://review.opendev.org/66762518:46
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Make comparisons case insensitive  https://review.opendev.org/66762618:46
openstackgerritMerged openstack/tripleo-heat-templates stable/stein: Reintroduce upgrade tasks for stopping pacemaker cluster  https://review.opendev.org/66713718:46
zbrdoes tripleo has some man pages? i am wondering if I can find the magic fernet key rotation documented anywhere on disk? https://docs.openstack.org/tripleo-docs/latest/install/post_deployment/fernet_key_rotation.html18:48
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Fix ipaclient script  https://review.opendev.org/66764818:50
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Fix ipaclient script  https://review.opendev.org/66764918:50
openstackgerritMerged openstack/tripleo-common stable/stein: Overcloud-full image templates for RHEL8  https://review.opendev.org/66766718:51
openstackgerritMerged openstack/tripleo-common stable/stein: remove plotnetcfg from overcloud image yaml  https://review.opendev.org/66766818:51
openstackgerritMerged openstack/tripleo-heat-templates master: Avoid checking rc if we're in dry-run mode  https://review.opendev.org/66778318:51
openstackgerritMerged openstack/tripleo-heat-templates master: Enable kill script for Neutron agents  https://review.opendev.org/66352818:51
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates stable/stein: Avoid checking rc if we're in dry-run mode  https://review.opendev.org/66802318:54
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: add pcs debug info to sanity checks  https://review.opendev.org/66802418:59
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: WIP: turn sanity checks on for fs001  https://review.opendev.org/66802519:01
*** jfrancoa has quit IRC19:05
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add CephGrafana Service on ceph related roles and enable ci.  https://review.opendev.org/66802819:15
openstackgerritMerged openstack/tripleo-validations master: Remove useless roles meta directory  https://review.opendev.org/66738419:21
openstackgerritwes hayutin proposed openstack/tripleo-repos master: [WIP] add support for RHEL8  https://review.opendev.org/66801519:22
*** hberaud|gone has quit IRC19:29
*** fultonj has quit IRC19:44
*** fultonj has joined #tripleo19:47
*** jtomasek has quit IRC19:55
*** ayoung has quit IRC19:59
fultonjIs anyone from tripleo going to the PTG this novmember?20:08
mwhahahait'll be very lightly attended if so20:13
mwhahahabut i don't think many folks are20:13
*** Goneri has quit IRC20:15
*** ansmith_ has quit IRC20:19
zbrfultonj: I will proably go, if i manage to submit my talk on molecule in time20:20
fultonjmwhahaha: at PTL would the foundation contact you about a PTG track?20:22
mwhahahano20:22
mwhahahaoh yea they might20:22
mwhahahawho knows, it's been spotty previously20:22
fultonjmwhahaha: so we could end up w/ a PTG w/ no tripleo track unless someone from tripleo contacts the foundation?20:25
mwhahahai'm unsure the timing of the PTG planning20:25
*** khyr0n has quit IRC20:27
*** khyr0n has joined #tripleo20:28
openstackgerritLuke Short proposed openstack/paunch master: WIP: Handle defined containers that are stopped.  https://review.opendev.org/64291020:30
fultonjok, thanks20:33
fultonjzbr: i'm planning to go too, see you there20:34
EmilienMfultonj: I'm not going to PTG nor summit20:38
EmilienMfultonj: I thought we could organize a meeting/sprint end of this year20:39
EmilienMin US or EU20:39
EmilienMor Canada !!!!20:39
EmilienMcome at home folks20:39
EmilienMI have a big table20:39
fultonjEmilienM: i'll try to go to that too ;)20:40
EmilienMjust don't tell my gf20:41
openstackgerritwes hayutin proposed openstack/tripleo-repos master: add support for RHEL8  https://review.opendev.org/66801520:41
*** khyr0n has quit IRC20:42
*** khyr0n has joined #tripleo20:44
cloudnullEmilienM I'm game for a Canadian PTG, but only if we can have a team meeting in your plane :P20:46
EmilienMlol20:47
EmilienM"we're staling"20:47
EmilienMI can do spins while you guys figure out tripleo20:47
cloudnullshould help with decision making and gate times :D20:48
*** khyr0n has quit IRC20:50
*** jtomasek has joined #tripleo20:51
*** khyr0n has joined #tripleo20:52
*** Vorrtex has quit IRC20:56
*** morazi has quit IRC20:58
openstackgerritMerged openstack/tripleo-quickstart master: Add tripleo-ansible to tripleo packages in current  https://review.opendev.org/66662720:59
*** pcaruana has quit IRC21:05
*** ansmith_ has joined #tripleo21:06
*** raildo has quit IRC21:06
*** Goneri has joined #tripleo21:09
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for bootstrap_node hieradata  https://review.opendev.org/66517621:10
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for all_nodes hieradata  https://review.opendev.org/66521321:10
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for vip_data hieradata  https://review.opendev.org/66644721:10
*** mmethot has quit IRC21:11
*** tesseract has quit IRC21:19
*** rcernin has joined #tripleo21:24
*** jtomasek has quit IRC21:30
*** khyr0n has quit IRC21:31
*** khyr0n has joined #tripleo21:33
*** aakarsh has quit IRC21:33
*** mcornea has quit IRC21:36
*** cgrosjean has quit IRC21:52
*** Goneri has quit IRC21:54
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for all_nodes hieradata  https://review.opendev.org/66521321:55
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Use Ansible for vip_data hieradata  https://review.opendev.org/66644721:55
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: DNM: use virtainers for molecule testing  https://review.opendev.org/66771321:55
*** khyr0n has quit IRC22:01
*** ayoung has joined #tripleo22:01
*** khyr0n has joined #tripleo22:03
openstackgerritJames Slagle proposed openstack/tripleo-common master: Don't calculate vips for some services  https://review.opendev.org/66800922:07
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: [WIP] base job testing  https://review.opendev.org/66806322:10
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: [WIP] base job testing  https://review.opendev.org/66806322:11
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: [WIP] base job testing  https://review.opendev.org/66806322:13
*** cgrosjean has joined #tripleo22:20
*** slaweq has quit IRC22:23
openstackgerritLuke Short proposed openstack/paunch master: Handle defined containers that are stopped.  https://review.opendev.org/64291022:27
*** pkopec is now known as pkopec|afk22:27
*** khyr0n has quit IRC22:28
*** khyr0n has joined #tripleo22:28
*** ekultails has quit IRC22:28
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: [WIP] base job testing  https://review.opendev.org/66806322:32
*** Goneri has joined #tripleo22:36
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: [WIP] Convert tox to native zuul  https://review.opendev.org/66806322:43
*** vinaykns has quit IRC22:54
*** rlandy has quit IRC22:57
*** aakarsh has joined #tripleo22:59
*** tosky has quit IRC23:00
openstackgerritMerged openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722123:00
*** matbu has quit IRC23:05
*** matbu has joined #tripleo23:06
openstackgerritMerged openstack/tripleo-heat-templates master: ceilometer_agent_notification: disable-panko.yaml  https://review.opendev.org/65269923:09
*** tonyb has quit IRC23:32
*** tonyb has joined #tripleo23:32
*** cgrosjean has quit IRC23:32
*** mmethot has joined #tripleo23:33
*** Goneri has quit IRC23:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!