| *** bhagyashris is now known as bhagyashris|ruck | 04:53 | |
| marios | \o | 06:37 |
|---|---|---|
| ysandeep | happy friday all o/ | 06:41 |
| *** amoralej|off is now known as amoralej | 07:52 | |
| *** Tengu is now known as Tengu|rover | 08:12 | |
| * Tengu|rover imagines now a small rover trying to make his path on rocky paths | 08:12 | |
| Tengu|rover | Earth, here's TripleO Rover - roger | 08:12 |
| marios | :D | 08:13 |
| Tengu|rover | ah, we already get the hackmd for this period. thanks to whoever created it :) | 08:16 |
| Tengu|rover | bhagyashris|ruck: we're all good for now? just coming online. | 08:17 |
| marios | bhagyashris|ruck: Tengu|rover: o/ joining | 08:30 |
| bhagyashris|ruck | joining | 08:35 |
| marios | Tengu|rover: https://review.rdoproject.org/zuul/buildset/e307ac532f9e43c9a5bd6805a85be0aa train buildset for chasing/checking fails | 08:40 |
| marios | example test Tengu|rover https://review.rdoproject.org/r/c/testproject/+/44462 | 08:40 |
| marios | Tengu|rover: promoter logs http://promoter.rdoproject.org/promoter_logs/centos8_train.log | 08:42 |
| *** ysandeep is now known as ysandeep|afk | 08:46 | |
| *** jpena|off is now known as jpena | 08:47 | |
| marios | Tengu|rover: bhagyashris|ruck: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/438865 | 08:47 |
| Tengu|rover | marios, bhagyashris|ruck https://review.rdoproject.org/r/c/testproject/+/46686 train re-run | 08:54 |
| Tengu|rover | and now, breakfast. | 08:55 |
| marios | thx Tengu|rover | 08:57 |
| Tengu|rover | let's see how it goes :) | 09:07 |
| Tengu|rover | ah, thanks for pushing that directly in the hackmd | 09:08 |
| *** ysandeep|afk is now known as ysandeep | 10:05 | |
| *** dviroel|out is now known as dviroel | 10:41 | |
| Tengu|rover | sounds pretty calm today... isn't it? | 10:48 |
| Tengu|rover | jm1: what's the status of your galaxy investigations? did you find any blackholes? | 10:48 |
| Tengu|rover | jm1: also, for the env var: it did work here because I didn't remove the "--no-cache" option... my bad. | 10:49 |
| jm1 | Tengu|rover: abandoned my patches because they simply did not work for our use case | 10:50 |
| Tengu|rover | apparently ansible-config doesn't expose anything related to the no_cache. though, maybe, we can try adding it to the configuration file. Still, imho, they are doing it wrong© with the whole API answers. | 10:50 |
| Tengu|rover | jm1: the fact ansible-galaxy servers pushes the FQDN in the answer is so wrong... it breaks the actual use of the "-s <server>" imho. | 10:50 |
| jm1 | Tengu|rover: ansible 2.9 does not support installing from git so i dropped that workaround | 10:51 |
| Tengu|rover | erf | 10:51 |
| Tengu|rover | so the "only" solution would be for the proxy to inject the correct name in the JSONs. | 10:51 |
| jm1 | Tengu|rover: that sounds like too much work for too little benefit | 10:52 |
| Tengu|rover | apparently not really, from an infra point of view. | 10:52 |
| Tengu|rover | I'll follow the topic with fungi and his team, pretty sure they'll get something at some point. | 10:52 |
| jm1 | Tengu|rover: ok cool! fyi only one job in 1 out of 10 patches failed last night. pretty good stats this time :D | 10:53 |
| Tengu|rover | hehj | 10:53 |
| Tengu|rover | so, pretty sure the ansible galaxy cache IS helping | 10:54 |
| Tengu|rover | but still. imho, the whole thing is broken design. | 10:54 |
| Tengu|rover | they should get a v3 asap, with proper answers that will NOT expose the fqdn, and make the client re-build the URI using whatever server is configured on their side. | 10:54 |
| jm1 | Tengu|rover: the probably have their reasons.... hopefully.. i do not want to give up hope :D | 10:54 |
| Tengu|rover | that would make the whole thing stronger, better and, well, more proxy-friendly. | 10:55 |
| Tengu|rover | now, the proxy config itself is a bit flaky, but it's less "broken" than the API design. | 10:55 |
| jm1 | Tengu|rover: "you cant fix it all" said some wise guy >> sshnaidm | 11:00 |
| Tengu|rover | jm1: yeah - but we can help nudging things in the right track :) | 11:01 |
| jm1 | Tengu|rover: i am trying hard to falsify him but so far i have not been very succesful :D | 11:01 |
| Tengu|rover | jm1: same here. Though.... I tend to push really hard on things that resist :) | 11:02 |
| Tengu|rover | for the better.... or the worst. | 11:02 |
| Tengu|rover | already backfired. | 11:02 |
| Tengu|rover | :) | 11:02 |
| Tengu|rover | but heh. can't change what I am. | 11:02 |
| *** rlandy|out is now known as rlandy | 11:04 | |
| rlandy | Tengu|rover: hey - how are you holding up? | 11:05 |
| rlandy | bhagyashris|ruck: hello - did you reach out to attila yet? | 11:06 |
| bhagyashris|ruck | rlandy, yes | 11:06 |
| rlandy | what channel? | 11:07 |
| rlandy | slack? | 11:07 |
| bhagyashris|ruck | on rhos-ops and slack as well no response | 11:07 |
| bhagyashris|ruck | on slack pinged personally | 11:07 |
| rlandy | bhagyashris|ruck: reached out to him on openstack-pcci | 11:09 |
| rlandy | pls join review time | 11:09 |
| rlandy | we can chat afterwards | 11:10 |
| rlandy | otherwise we need to remove these jobs from criteria and promote | 11:10 |
| rlandy | Tengu|rover needs the update | 11:10 |
| bhagyashris|ruck | rlandy, sure | 11:10 |
| rlandy | bhagyashris|ruck: can you check components | 11:12 |
| bhagyashris|ruck | Attila haven't joined rhos-ops on slack so pinged him personally on slack | 11:12 |
| rlandy | a few were a out yesterday | 11:12 |
| bhagyashris|ruck | yes cheking those | 11:12 |
| rlandy | ty | 11:12 |
| rlandy | bhagyashris|ruck: and you can add 18 to the list of lines people should watch at this point | 11:12 |
| bhagyashris|ruck | rlandy, you mean in the rr tool | 11:13 |
| rlandy | bhagyashris|ruck: in the hack md to start | 11:13 |
| rlandy | yeah and then the rr tool pls | 11:13 |
| rlandy | ^^ but that can be its own card | 11:13 |
| Tengu|rover | rlandy: things seem to be stable enough. just launched a train promotion thing, there were 3 failed jobs. | 11:13 |
| Tengu|rover | it's running in testproject. otherwise, things seem pretty green. | 11:14 |
| rlandy | Tengu|rover: did you get the alt criteria explanation? | 11:14 |
| Tengu|rover | oh, right, I missed that topic with marios... | 11:15 |
| Tengu|rover | this morning was a bit.... weird. | 11:15 |
| Tengu|rover | rlandy: so nope | 11:15 |
| marios | o/ | 11:16 |
| Tengu|rover | marios: so far, so good for the jobs. seems the FS issue was indeed a temporary one. | 11:16 |
| marios | Tengu|rover: yes as discussed earlier, we often will run the test to confirm the issue (seen 1x/1x buildset is usually not enough for bug. we need at least 2x for verification -> file bug) | 11:17 |
| Tengu|rover | yup | 11:17 |
| marios | otherwise we go crazy (er?) | 11:17 |
| Tengu|rover | I guess those don't have the "retry" thingy? | 11:17 |
| Tengu|rover | for instance I've seen jobs retrying up to 3 times before really failing. | 11:18 |
| Tengu|rover | maybe something to consider, since it would probably avoid those false-flags? | 11:18 |
| marios | Tengu|rover: that is if they are failing in pre plays usually | 11:18 |
| Tengu|rover | oh, ok. | 11:18 |
| ysandeep | reviewbot, please add to review list: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/871223 | 11:24 |
| reviewbot | I have added your review to the Review list | 11:24 |
| * soniya29 will be travelling for 1 hr | 11:26 | |
| *** soniya29 is now known as soniya29|afk | 11:26 | |
| Tengu|rover | ok, #lunch. | 11:27 |
| ysandeep | rlandy, https://review.rdoproject.org/r/c/config/+/46549 | 11:29 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-internal-train&skip=0 | 11:30 |
| * pojadhav stepping out bit | 11:33 | |
| rlandy | Tengu|rover; alt criteria for train is not working :( | 11:40 |
| rlandy | ysandeep and bhagyashris|ruck are looking into that | 11:40 |
| rlandy | you should have been spared the rerun | 11:40 |
| rlandy | but ok | 11:40 |
| rlandy | Tengu|rover: also - asked bhagyashris|ruck to put in a patch to skip the jenkins criteria | 11:40 |
| ysandeep | rlandy, maybe dns/network config issue, holding a c8 train node to check further: https://code.engineering.redhat.com/gerrit/c/testproject/+/211643/178/.zuul.yaml | 11:41 |
| Tengu|rover | rlandy: humpf. seems infra issue (if it's the linked you posted earlier) | 12:19 |
| Tengu|rover | rlandy: is the "alt criteria" related to the jobs running on RH infra when it fails upstream? | 12:21 |
| rlandy | ysandeep: ty | 12:29 |
| rlandy | Tengu|rover: ack | 12:29 |
| rlandy | will explain at meeting | 12:29 |
| Tengu|rover | sure | 12:29 |
| Tengu|rover | marios: we should get train promotion, | 12:30 |
| Tengu|rover | last job is finishing | 12:30 |
| marios | Tengu|rover: yeah i think that last might may be failed | 12:43 |
| marios | i mean from https://review.rdoproject.org/zuul/stream/dc07d6fbb9364e849224479ce3de2798?logfile=console.log | 12:44 |
| marios | Tengu|rover: can just try with that one if it is yet another not seen before thing | 12:50 |
| marios | Tengu|rover: but got the other 2 which is great | 12:51 |
| Tengu|rover | marios: yeah, tempest just failed, unreachable instance. | 13:01 |
| Tengu|rover | marios: so I'll amend my patch and try to get that last one in | 13:01 |
| Tengu|rover | rlandy: oh, damn, we missed the "alt criteria" again.... | 13:01 |
| Tengu|rover | pfff. | 13:01 |
| Tengu|rover | marios: edited. Let's see. | 13:02 |
| ysandeep | rlandy, bhagyashris|ruck I looked at the train job failure using the node which I put on hold, I was NOT able to manually reproduce the issue in the affected node, but if I comment DNS entries I see the same issue - https://privatebin.corp.redhat.com/?038a3af8af44cb69#5cU5RNMwe37iHjobaV6YJHpZ2NkGnQS1PZur4wZfXHQY , need to check further why dns resolution is not properly working during job run. | 13:08 |
| *** amoralej is now known as amoralej|lunch | 13:24 | |
| frenzy_friday | 0/ pls add to your review lists https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46687 | 13:30 |
| reviewbot | I have added your review to the Review list | 13:30 |
| chandankumar | I will be out on monday & tuesday | 14:04 |
| * pojadhav afk | 14:04 | |
| * bhagyashris|ruck afk | 14:08 | |
| Tengu|rover | humpf.... I think I'll take the opportunity to ride a bit, there's a fine sun outside. -0.5°C, but still, it's doable with proper equipment. | 14:10 |
| Tengu|rover | marios, rlandy may I let you check on the things for the rest of the day? Things seem pretty calm on the upstream side. | 14:10 |
| marios | o/ Tengu|rover | 14:11 |
| Tengu|rover | perfect :). Hopefully things won't blow up over the weekend ;). | 14:11 |
| Tengu|rover | see you next week folks! Take care all | 14:11 |
| *** amoralej|lunch is now known as amoralej | 14:14 | |
| rlandy | Tengu|rover: sure - have a good weekend | 14:16 |
| ysandeep | Tengu|rover, happy weekend o/ | 14:20 |
| * marios biab | 14:38 | |
| dasm|off | o/ | 14:52 |
| *** dasm|off is now known as dasm | 14:52 | |
| *** dviroel is now known as dviroel|lunch | 15:49 | |
| *** ysandeep is now known as ysandeep|out | 16:15 | |
| * marios off in few | 16:31 | |
| *** marios is now known as marios|out | 16:43 | |
| *** dviroel|lunch is now known as dviroel | 16:46 | |
| *** amoralej is now known as amoralej|off | 17:29 | |
| *** jpena is now known as jpena|off | 17:42 | |
| * dasm => break | 20:32 | |
| dasm | vack | 21:07 |
| dasm | back | 21:07 |
| * dviroel is having fun, but it is time to start the weekend | 21:20 | |
| dviroel | have a great weekend team | 21:20 |
| dviroel | o/ | 21:20 |
| *** dviroel is now known as dviroel|out | 21:22 | |
| rlandy | see you all on monday | 21:27 |
| dasm | see you! | 21:28 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!