Tuesday, 2019-04-09

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/64440000:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429300:00
*** wolverineav has quit IRC00:02
*** wolverineav has joined #tripleo00:03
*** mcornea has quit IRC00:05
openstackgerritMerged openstack/tripleo-heat-templates master: Provide option to disable EMC in puppet-vswitch  https://review.openstack.org/64831000:07
*** wolverineav has quit IRC00:07
openstackgerritTakashi Kajinami proposed openstack/tripleo-heat-templates master: Add parameter to configure maxdelay in db purge/archive job  https://review.openstack.org/64156300:15
*** wolverineav has joined #tripleo00:20
*** wolverineav has quit IRC00:27
*** khyr0n has quit IRC00:48
*** wolverineav has joined #tripleo00:49
*** wolverineav has quit IRC00:53
openstackgerritSteve Baker proposed openstack/tripleo-common master: Fix support for v1 manifests  https://review.openstack.org/65105101:09
*** mschuppert has quit IRC01:10
*** boazel has joined #tripleo01:13
*** boazel_ has quit IRC01:16
*** aakarsh has joined #tripleo01:43
*** apetrich has quit IRC01:58
*** sanjayu_ has quit IRC02:47
*** ykarel|away has joined #tripleo02:51
*** baha has joined #tripleo02:51
*** boazel_ has joined #tripleo02:56
*** boazel has quit IRC02:59
*** dsneddon has quit IRC03:28
*** dsneddon has joined #tripleo03:53
openstackgerritMerged openstack/tripleo-heat-templates master: Stop writing /var/lib/container-startup-configs.json  https://review.openstack.org/65088903:58
*** dsneddon has quit IRC03:58
*** ramishra has joined #tripleo03:59
*** udesale has joined #tripleo04:04
*** ykarel|away is now known as ykarel04:07
*** Vorrtex has joined #tripleo04:13
*** fmount has quit IRC04:14
*** fmount has joined #tripleo04:15
*** Vorrtex has quit IRC04:22
*** ade_lee has joined #tripleo04:28
*** ade_lee__ has quit IRC04:29
*** dsneddon has joined #tripleo04:30
*** boazel has joined #tripleo04:39
*** boazel_ has quit IRC04:42
*** ykarel has quit IRC04:42
*** wolverineav has joined #tripleo04:49
*** wolverineav has quit IRC04:54
*** ykarel has joined #tripleo04:56
*** ratailor has joined #tripleo05:11
Tenguhello there :)05:19
*** paramite has joined #tripleo05:19
*** skramaja has joined #tripleo05:19
Tenguhmm...05:24
Tenguguess what. CI for queens seems to be broken, once again. http://logs.openstack.org/31/648331/9/check/puppet-openstack-unit-4.8-centos-7/e99484e/job-output.txt.gz#_2019-04-08_15_18_18_45267005:25
*** marios has joined #tripleo05:30
Tengusooo. https://bugs.launchpad.net/tripleo/+bug/182383805:31
openstackLaunchpad bug 1823838 in tripleo "[CI][Queens] Unable to manage Puppetfile content 'ntp': Could not resolve desired ref '4.2.x' and no default provided" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner)05:31
*** dsneddon has quit IRC05:34
*** quiquell|off is now known as quiquell|rover05:43
Tenguquiquell|rover: «o/05:43
quiquell|roverTengu: good morning sir05:46
quiquell|roverTengu: let me look at CI with queens05:46
Tenguquiquell|rover: trying a patch locally with rspec running.05:47
Tengumight be an easy one.05:47
*** yprokule has joined #tripleo05:51
Tenguo_O oook....... local rspec is just shitting itself.05:51
Tenguprobably due to ruby versions and so on. god.... I heartly HATE ruby. almost as much as I hate rabbitmq.05:51
Tenguquiquell|rover: do you have some stats about that failure?05:52
quiquell|roverSure05:52
quiquell|roverLet me draw something for you05:52
*** udesale has quit IRC05:52
TenguDraw me a sheep :)05:52
quiquell|roverTengu: http://dashboard-ci.tripleo.org/d/FEdraO0ik/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D%7Cpuppet-openstack-unit-4.8-centos-705:53
quiquell|roverYou can filter by branch too05:54
Tenguhmmm. weird.05:54
quiquell|roverTengu: Looks non relatrd to your review but 5th was passing05:54
Tenguboth failures are on my change. BUT my change didn't do anything with ntp calls....05:54
quiquell|roverthere is a third that is not05:55
Tengulemme create a fake review on that very same module.05:55
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: DNM - TEST CI  https://review.openstack.org/65109705:57
Tenguquiquell|rover: -^^  if this one fails, we have a problem.05:57
*** ykarel is now known as ykarel|afk05:58
Tenguhmm. some on my patch do actually pass. random issue at some level? please don't.....05:58
quiquell|roverTengu: it's going to fail I don't see the relation between your change and the failure05:58
quiquell|roverIt's rebased and all ?05:58
Tenguquiquell|rover: same for me. but we do have many changes passing.05:58
Tenguquiquell|rover: should be.,05:59
Tenguyep.05:59
Tenguit is against HEAD of stable/queens05:59
quiquell|roverMaybe you can remove the Depends-On that are merge (It also runs faster)05:59
Tenguquiquell|rover: yeah, well, let's first check what's going on.06:00
quiquell|roverkeep them as reference06:00
quiquell|roverYou can do that in parallel06:00
Tenguwill lose the votes again ;_;06:00
quiquell|roverOld days rake06:00
quiquell|rover:-)06:00
quiquell|roverUse to have a system on top of rake iwth java and c++06:00
*** jbadiapa has joined #tripleo06:01
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: Simplify logrotate job  https://review.openstack.org/64833106:01
*** ratailor has quit IRC06:01
*** ykarel|afk has quit IRC06:02
quiquell|roverLooks like ntp is transitive dependency ?06:03
quiquell|roverIt's not at puppet-openstack-integration/Puppetfile06:03
Tenguntp is only for build apparently06:03
Tenguor at least rspec06:03
*** jfrancoa has joined #tripleo06:03
Tenguoh. fuck. zuul's smart enough to NOT run the whole thingy when we just edit the readme.06:04
*** radeks has joined #tripleo06:04
quiquell|roverdamn, the files section06:04
quiquell|rover:-/06:04
quiquell|roverthat's so wrong since one of the test is failing06:04
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: DNM - TEST CI  https://review.openstack.org/65109706:04
Tenguthere, a .pp is edited now.06:05
quiquell|roveryou have to touch files at files: section from zuul definition06:05
Tenguoh. wait. pretty sure the modification I've done will hit some syntax check regarding spacing between header and code.06:05
*** kopecmartin|off is now known as kopecmartin06:05
*** dsneddon has joined #tripleo06:06
openstackgerritCédric Jeanneret proposed openstack/tripleo-heat-templates master: Ensure we get rsyslog state, even in "--check" mode  https://review.openstack.org/65110306:09
quiquell|roverhumm there is something I don't like a bit06:09
quiquell|rover-e /usr/zuul-env/bin/zuul-cloner06:10
quiquell|rover/home/zuul/workspace/openstack/puppet-openstack-integration/06:10
quiquell|roverThis is old zuul version of stuff06:10
*** dsneddon has quit IRC06:11
quiquell|roverTengu: so it's failing here https://github.com/openstack/puppet-openstack_spec_helper/blob/master/lib/puppet-openstack_spec_helper/rake_tasks.rb#L7006:11
Tenguquiquell|rover: hmm ok. My guess: it doesn't like the "4.2.x" and wants a plain version, current 4.2 is 4.2.006:12
Tenguso we should probably just replace the .X by a .006:12
*** ccamacho has joined #tripleo06:12
quiquell|roverWould be nice to see were did it change and try to add some CI there06:13
Tengusince that version is up since 2016, there are very few reasons to see a 4.2.106:13
*** ratailor has joined #tripleo06:13
*** ykarel|afk has joined #tripleo06:13
*** dsneddon has joined #tripleo06:14
quiquell|roverPuff there is a lot of stuff that depends on old zuul06:17
quiquell|roverIs going work06:17
*** dsneddon has quit IRC06:19
openstackgerritChandan Kumar (raukadah) proposed openstack/ansible-role-collect-logs master: Apply cookiecutter on collect-logs  https://review.openstack.org/65061306:21
Tenguquiquell|rover: hmm. where's that bash script that installs the deps?06:21
*** boazel_ has joined #tripleo06:22
quiquell|roverTengu: trying to follow the backtrace06:23
quiquell|roverTengu: issue is here https://github.com/openstack/puppet-openstack-integration/blob/417d496030211dd241297f74295cf36762b94f82/functions#L1206:23
quiquell|roverso modern zuulv3 clone at /home/zuul/src/...06:24
quiquell|roverand there is a ansible role that link it to /home/zuul/workspace06:24
*** boazel has quit IRC06:25
quiquell|roverI would like to see the contents of Puppetfile106:25
quiquell|roverWell is a csplit06:26
quiquell|roverIs transitive06:27
quiquell|roverOne of the upstream is broken there06:27
quiquell|roverWho knows wich06:28
*** radeks has quit IRC06:28
*** radeks has joined #tripleo06:29
quiquell|roverTengu: one stuff we can compre is Puppetfile between queens and Master06:29
*** pcaruana has joined #tripleo06:30
Tenguhmm06:31
Tenguif it's using r10k, there are significant differences between versions. maybe queens branch might be running with an old one?06:32
quiquell|roverPuff lot of differences06:32
quiquell|roverLet's hope the noop review fails :-/06:32
quiquell|roverWe can put in the job some post playbook to collect the stuff we need06:33
*** udesale has joined #tripleo06:33
*** mschuppert has joined #tripleo06:34
*** ade_lee has quit IRC06:34
*** ade_lee has joined #tripleo06:34
*** ykarel|afk is now known as ykarel06:35
Tengufeel free to do whatever you want on that noop change :)06:35
quiquell|roverhumm first try to manually run r10k on queens Puppetfile06:36
ykarelTengu, is it affecting only one job?06:36
ykareland one release?06:36
ykarellooks like puppet-ntp deleted 4.2.x branch06:37
ykarelor tag06:37
quiquell|roverykarel: I am running manually the r10k it should fail06:38
Tenguykarel: there WAS a 4.0.x tag?06:38
Tengu*4.2.x06:38
Tenguif so, we can just push 4.2.006:38
*** cylopez has joined #tripleo06:38
quiquell|roverthere is no 4.2.x branch or tag here https://github.com/puppetlabs/puppetlabs-ntp06:40
quiquell|roveryep06:40
*** aedc has quit IRC06:40
ykarelTengu, i think it was there earlier06:40
Tenguykarel: ook... still, it must fail a bit more then.06:40
*** quiquell|rover is now known as quique|rover|brb06:41
quique|rover|brbGo back in a few06:41
Tenguapparently there were successful runs. I've just pushed a "Noop" change. we'll know more.06:41
Tengubrb too, breakfast is needed for brain fuel.06:41
quique|rover|brbTengu: has fail06:41
quique|rover|brbhttp://logs.openstack.org/97/651097/2/check/puppet-openstack-unit-4.8-centos-7/9777052/job-output.txt.gz#_2019-04-09_06_31_32_92344106:41
quique|rover|brbthe noop06:41
quique|rover|brbSO we change transitive dependency or tag with 4.2.x puppet-ntp ?06:42
Tenguquique|rover|brb: we can push with the Puppetfile_extras modified to 4.2.006:43
*** florianf has joined #tripleo06:43
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: DNM - TEST CI  https://review.openstack.org/65109706:43
Tenguquique|rover|brb: -^^  if it works, I'll update content/commit message and we're good to merde.06:44
Tengu*merge06:44
*** dsneddon has joined #tripleo06:46
*** slaweq has joined #tripleo06:47
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/queens: Enable deep_compare of pcmk resources by default  https://review.openstack.org/65111006:50
Tenguquique|rover|brb: I've just added you to the trello card I've created about this issue.06:50
*** dsneddon has quit IRC06:51
*** radeks has quit IRC06:55
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: WIP: Add DPDK support for OVN  https://review.openstack.org/64455606:57
openstackgerritChandan Kumar (raukadah) proposed openstack/ansible-role-collect-logs master: Apply cookiecutter on collect-logs  https://review.openstack.org/65061307:01
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Add DPDK support for OVN  https://review.openstack.org/64455607:03
Tenguhmm. apparently it's working. probably a queens-only patch then. yay.07:03
*** gfidente has joined #tripleo07:05
*** bogdando has joined #tripleo07:07
*** apetrich has joined #tripleo07:10
*** gfidente has quit IRC07:11
*** rpittau|afk is now known as rpittau07:12
*** dciabrin_ has joined #tripleo07:13
*** quique|rover|brb is now known as quiquell|rover07:13
*** chem` has joined #tripleo07:13
quiquell|roverOk I am back07:14
Tenguquiquell|rover: I'm waiting for the end of the unit-4.8, and will push an update again.07:14
Tenguquiquell|rover: apparently moving to 4.2.0 is fine.07:14
TenguSUCCESS07:14
*** chem has quit IRC07:14
quiquell|roveryep07:14
quiquell|rover\o/ !!!07:15
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: [Queens-only] Correct reference for ntp module.  https://review.openstack.org/65109707:15
Tengubam.07:15
quiquell|roverSo doing some forensics07:15
quiquell|roverAn upstream project has remove a reference07:15
quiquell|roveris that it ?07:15
Tenguapparently yeah...07:15
quiquell|roverok07:15
quiquell|rovernothing we can do there07:15
Tengualthough I'm surprised by the .X anyway.07:15
Tengubut yeah.07:15
Tengucan't do anything.07:16
quiquell|roveryep... looks like bad taging and we referencing it07:16
Tenguyep07:16
quiquell|rover+2 there07:16
quiquell|roverPut the other in top of this07:16
*** slaweq has quit IRC07:17
*** dsneddon has joined #tripleo07:17
*** dciabrin has quit IRC07:17
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: Simplify logrotate job  https://review.openstack.org/64833107:17
Tenguquiquell|rover: -^^ done07:17
*** slaweq has joined #tripleo07:18
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/rocky: Be able to know when we are running inside a minor update workflow  https://review.openstack.org/65111407:18
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/rocky: Do not restart bundles during a minor update  https://review.openstack.org/65111507:18
quiquell|rover+207:18
Tengu:)07:18
Tenguhey, I could sort out a CI bug \o/07:18
Tenguwell, detect it, and squash it07:18
Tenguthanks quiquell|rover for your researches :)07:18
quiquell|roverand ykarel as alÃways07:18
Tengu:) ykarel is always here yep :)07:19
quiquell|roveris the openstack skynet version but without military means07:19
*** tosky has joined #tripleo07:20
ykarelquiquell|rover, Tengu it's not just queens only07:20
ykarelit's affecting other branches also07:20
quiquell|roverykarel: so we have to backport https://review.openstack.org/651097 ?07:20
quiquell|roverykarel: let's work out it here https://bugs.launchpad.net/tripleo/+bug/182383807:20
openstackLaunchpad bug 1823838 in tripleo "[CI][Queens] Unable to manage Puppetfile content 'ntp': Could not resolve desired ref '4.2.x' and no default provided" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner)07:21
ykarelquiquell|rover, Tengu rocky example: http://logs.openstack.org/24/649324/1/check/puppet-openstack-unit-4.8-centos-7/04305e8/job-output.txt.gz07:21
Tenguykarel: oh?? damn.07:21
ykarelso it's across other branches07:21
Tengulemme create a master patch then.07:21
Tenguand we'll backport down to queens.07:21
Tenguagain.07:21
Tenguand again.07:21
*** dsneddon has quit IRC07:21
quiquell|roverykarel: and  other projects ?07:21
quiquell|roverykarel: it's only at puppet-tripleo ?07:21
ykarelquiquell|rover, i checked only that job07:22
ykarelso wherever that 4.2.x is used should be affected07:22
ykarelTengu, ack07:22
*** JqckB has joined #tripleo07:22
ykarelbut good to know why that branch was deleted(if our theory is correct)07:22
ykareli think this was a tag07:22
ykarel4.2.x07:22
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65111607:23
ykarelEmilienM, mwhahaha any way to check ^^07:23
*** sshnaidm|off is now known as sshnaidm|pto07:23
Tenguykarel, quiquell|rover -^^07:23
Tenguykarel, quiquell|rover: I'll start the backport directly if you're OK with that.07:23
quiquell|roverok only puppet-tripleo have i t07:24
*** dsneddon has joined #tripleo07:24
quiquell|roverTengu: yep until pike I supose07:24
quiquell|roverqueens maybe07:24
ykarelTengu, okk for me as 4.2.0 was used earlier as well, and setting it explicitly won't hurt07:25
Tenguykarel: good07:25
Tengulemme start the backports with cherry-pick -x in //07:25
ykarelbut again good to know why it's deleted, generally tags are not deleted07:26
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/rocky: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65111807:27
Tengunot sure we can get a history.07:27
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65112007:28
quiquell|rovermwhahaha: is the one that added puppet-ntp like two years ago07:28
quiquell|roverhttps://github.com/openstack/puppet-tripleo/commit/37ba3a8db5e38955469e8bc9158388379d64abc807:28
quiquell|rovermaybe he knows07:28
*** dsneddon has quit IRC07:28
Tenguthere we are. master, rocky and queens are ready.07:28
quiquell|roverI think pike is needed too07:29
quiquell|roversince the change was done to newton and ocata07:29
quiquell|roverhttps://review.openstack.org/#/q/change:+I7a697aba7aa5a27ba4ab6e46018057f7f01dfab207:29
Tenguhmmm yeah.07:30
Tenguso adding pike.07:30
openstackgerrityatin proposed openstack/tripleo-heat-templates master: DO-NOT-MERGE Test new ceph-container builds  https://review.openstack.org/64461807:30
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/pike: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65112207:30
*** ykarel is now known as ykarel|lunch07:31
Tenguquiquell|rover: so newton and ocata would need that as well ?07:31
quiquell|roverwell07:31
quiquell|roverI don't htink so07:31
quiquell|roverjust pike07:31
Tenguok07:31
Tenguwell, we can always cherry-pick if needed.07:32
quiquell|roverI mean at least tripleo ci is no longer running newton or ocata07:32
quiquell|roverpike is the limit07:32
Tenguok. so we don't care.07:32
Tenguas it's only for rspec.07:32
*** mrunge has quit IRC07:32
quiquell|roverok +2 * releases07:32
*** tosky has quit IRC07:33
*** mrunge has joined #tripleo07:33
quiquell|roverdon' tknow if somone is adding patches to upstream newton/ocata puppet-tripleo though07:33
quiquell|roverThey will not pass07:33
*** tosky has joined #tripleo07:33
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/queens: Simplify logrotate job  https://review.openstack.org/64833107:34
Tenguquiquell|rover: also rebased my patch -^^07:34
Tenguusing the right base (cherry-picked patch)07:34
quiquell|rovergood07:34
*** holser_ has joined #tripleo07:40
*** nawar has joined #tripleo07:40
nawarhi07:40
nawarguys can you check this error https://bugs.launchpad.net/tripleo/+bug/182352707:41
openstackLaunchpad bug 1823527 in tripleo "neutron's dockers are unhealthy" [Undecided,New]07:41
*** amoralej|off is now known as amoralej07:43
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo stable/ocata: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65112607:46
*** jpena|off is now known as jpena07:47
*** hberaud|gone is now known as hberaud07:49
*** jpich has joined #tripleo07:52
openstackgerritMerged openstack/tripleo-quickstart-extras master: Add blank line to repo while creating dlrn config  https://review.openstack.org/62203707:52
bogdandojaosorior: hi please +2 https://review.openstack.org/#/c/648128/07:53
openstackgerritPiotr Kopec proposed openstack/tripleo-heat-templates master: Allow NovaNfs parameters to be role specific  https://review.openstack.org/65112807:58
*** lee1 has joined #tripleo07:58
*** mschuppert has quit IRC07:58
*** lyarwood has quit IRC07:58
*** mschuppert has joined #tripleo07:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
*** dtantsur|afk is now known as dtantsur08:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:00
*** lee1 is now known as lyarwood08:00
*** dsneddon has joined #tripleo08:00
*** rcernin has quit IRC08:01
*** ccamacho has quit IRC08:01
*** gfidente has joined #tripleo08:04
*** boazel has joined #tripleo08:04
*** sanjayu_ has joined #tripleo08:06
*** dsneddon has quit IRC08:06
*** boazel_ has quit IRC08:07
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates stable/queens: Be explicit when passing vars into deploy steps  https://review.openstack.org/65113308:11
*** marios has quit IRC08:11
*** wolverineav has joined #tripleo08:15
*** wolverineav has quit IRC08:20
*** lucasagomes has joined #tripleo08:27
*** zbr has quit IRC08:28
openstackgerritSlawek Kaplonski proposed openstack/tripleo-heat-templates master: Increase default ulimit values for Neutron agents containers  https://review.openstack.org/65113708:30
Tengudamn....08:31
Tenguquiquell|rover: http://logs.openstack.org/26/651126/1/check/puppet-openstack-syntax-3-legacy/074ee63/job-output.txt.gz#_2019-04-09_07_51_23_769488  guess ocata has some other issues.08:31
quiquell|roverarggg08:33
quiquell|rovergive me a sec I want to start long process08:34
Tengu:D08:34
quiquell|roverthen we look08:34
Tenguno problem08:34
openstackgerritChandan Kumar (raukadah) proposed openstack/ansible-role-collect-logs master: Apply cookiecutter on collect-logs  https://review.openstack.org/65061308:34
*** derekh has joined #tripleo08:40
*** dsneddon has joined #tripleo08:42
*** tkajinam has quit IRC08:43
*** dsneddon has quit IRC08:47
*** ykarel|lunch is now known as ykarel08:48
Tenguquiquell|rover: so yeah. ocata has some nice issues: https://review.openstack.org/#/c/651126/08:49
Tenguwe probably have some issues with non-fixed dependencies.08:50
*** florianf has quit IRC08:52
bogdandobeagles: hi, see https://bugs.launchpad.net/tripleo/+bug/1823377/comments/308:53
openstackLaunchpad bug 1823377 in tripleo "restarting the neutron dhcp agent container restarts dhcp sidecars" [High,Triaged]08:54
bogdandobeagles: it seems that all 'Unable to access' errors in the logs are basically for the missing pidfiles for qdhcp- namespaces08:58
*** pdeore has joined #tripleo08:59
*** aedc has joined #tripleo09:00
*** panda has joined #tripleo09:03
bogdandobeagles: oops, it's not, see the updated snippet09:07
*** marios_ has joined #tripleo09:08
bogdandobeagles: if you want access the env, let me know09:10
*** dsneddon has joined #tripleo09:13
Tenguguys, care to make those changes merge, at least down to Queens? https://review.openstack.org/#/q/status:open+project:openstack/puppet-tripleo+topic:bug/182383809:18
*** dsneddon has quit IRC09:19
jaosorior /exit09:23
*** jaosorior has quit IRC09:24
bogdandobeagles: https://bugs.launchpad.net/tripleo/+bug/1823377/comments/809:27
openstackLaunchpad bug 1823377 in tripleo "restarting the neutron dhcp agent container restarts dhcp sidecars" [High,Triaged]09:27
*** marios_ is now known as marios09:28
bogdandobeagles: sigh, please disregard, there is no path mismatches, just owners09:29
quiquell|rovermarios, panda: we need this to unblock puppet-tripleo check https://review.openstack.org/#/q/status:open+project:openstack/puppet-tripleo+topic:bug/182383809:29
quiquell|roverTengu: ocata feels like very broken09:30
quiquell|roverTengu: like no one is using it09:30
*** jaosorior has joined #tripleo09:31
Tenguquiquell|rover: yep... not sure we want to dig into it09:32
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add node-disks role  https://review.openstack.org/64075409:33
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add check-network-gateway role  https://review.openstack.org/64097509:33
quiquell|roverDon't think so09:33
quiquell|roverlet's merge the others09:33
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add node-health role  https://review.openstack.org/64076009:33
Tengu:)09:33
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add rabbitmq-limits role  https://review.openstack.org/64012409:33
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add mysql-open-files-limit role  https://review.openstack.org/64013109:34
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add openstack-endpoints role  https://review.openstack.org/64132809:34
mariosquiquell|rover: ack09:34
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add ironic-boot-configuration role  https://review.openstack.org/64133009:34
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add stonith-exists role  https://review.openstack.org/64141209:35
Tengugchamoul: massive rebase? :)09:35
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add containerized-undercloud-docker role  https://review.openstack.org/64163409:35
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add collect-flavors-and-verify-profiles role  https://review.openstack.org/64102709:35
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add deployment-images role  https://review.openstack.org/64131909:35
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add stack-health role  https://review.openstack.org/64132709:36
*** matbu has quit IRC09:36
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Rename undercloud-neutron-sanity-check role  https://review.openstack.org/64077409:36
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add network-environment role  https://review.openstack.org/64076109:36
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add switch-vlans role  https://review.openstack.org/64078609:36
gchamoulTengu: just getting rid of the dependencies ..09:37
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add default-node-count role  https://review.openstack.org/64077609:37
gchamoulTengu: to get them merged faster!09:37
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Add check-latest-minor-version role  https://review.openstack.org/64097209:37
gchamoulI hope09:37
gchamoulsorry for noise09:37
Tengugchamoul: :)09:37
openstackgerritNatal Ngétal proposed openstack/tripleo-common master: [Core] Be pep8 compliant.  https://review.openstack.org/63424109:39
pandaquiquell|rover: since when we're allowed to +W in puppet-tripleo ? :)09:40
quiquell|roverpanda: I forgot09:40
quiquell|roverpanda: have just +209:40
quiquell|roverpanda: bogdando and you workflow https://review.openstack.org/#/q/status:open+project:openstack/puppet-tripleo+topic:bug/182383809:41
quiquell|roverpanda: what about this ? https://review.rdoproject.org/r/#/c/20039/09:41
pandaquiquell|rover: was just looking at it09:41
quiquell|roverpanda: centos7 and fedora28 were using the same container list file09:42
quiquell|roverpanda: so we were pushing just a few fedora28 containers09:42
*** yolanda_ has quit IRC09:47
*** boazel_ has joined #tripleo09:47
*** boazel has quit IRC09:50
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Fail build-test-package if project not found  https://review.openstack.org/63658809:51
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Fail build-test-package if project not found  https://review.openstack.org/63658809:51
pandaquiquell|rover: do you know what outputs parsed_containers- file ?09:53
*** jfrancoa has quit IRC09:53
quiquell|roverpanda: yep the namer of the containres without fedora-binary- or centos-binary-09:54
quiquell|roverpanda: you can open them in the promoter09:55
*** jfrancoa has joined #tripleo09:55
quiquell|roverpanda: I am running this manually to promote fedora28 containers09:55
*** holser_ is now known as holser|lunch09:55
pandaquiquell|rover: ? what creates the parsed_containers-fedora28.txt file that contains the list containers ?09:56
quiquell|roverthe container-push playbook09:56
*** udesale has quit IRC09:57
quiquell|roverit get the container names from https://git.openstack.org/cgit/openstack/tripleo-common/plain/container-images/overcloud_containers.yaml.j209:57
pandammmhh, so why dos it matter how it's called ? Time to read the bug ... :)09:58
*** udesale has joined #tripleo09:58
quiquell|roverit checks distro name09:58
quiquell|roverat rdo registry09:59
quiquell|roverand push the images tha are ther e09:59
quiquell|roverso there are some distro aware logic there09:59
*** psachin has joined #tripleo10:00
quiquell|roverpanda: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/container-push.yml#L5910:00
quiquell|roverso it works for partial runs10:01
quiquell|roverper distro10:01
quiquell|roverso file has to be distro aware10:01
quiquell|roverlike promoter running in the middle of container push10:01
quiquell|roverit will construct a partial list10:02
*** aedc has quit IRC10:02
*** hberaud is now known as hberaud|lunch10:03
quiquell|roverhumm but we are wiping out the file :-/10:03
*** skramaja has quit IRC10:03
quiquell|roverand promotions does not run in parallel though10:03
pandaquiquell|rover: to me, that file could be called "goofy" and nothing would change ....10:04
quiquell|roverpanda: nah this is not going to fix it10:04
quiquell|roverpanda: good to have the distro in it though10:04
pandaquiquell|rover: agree10:04
quiquell|roverpanda: so looks like promoter did a promotion with not all the image pushed10:04
quiquell|roverpanda: now that I see the code this is possible10:05
quiquell|roverpanda: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/container-push.yml#L63 does not stop the playbook10:05
quiquell|roverdon't we miss an exit -1 there ?10:06
quiquell|roverexit 1 I mean10:06
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/queens: Be able to know when we are running inside a minor update workflow  https://review.openstack.org/65118110:08
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/queens: Do not restart bundles during a minor update  https://review.openstack.org/65118210:08
pandaquiquell|rover: no, we exit 1 on total failure, not on partial failure there, not sure why, but may be deliberate.10:09
*** aedc has joined #tripleo10:10
*** social has joined #tripleo10:10
quiquell|roverBut that can be a problem10:12
quiquell|roverPartial list of containers10:12
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Parametrize libvirt delay and retries  https://review.openstack.org/65118510:13
*** aedc has quit IRC10:15
quiquell|roverpanda: so here you can se a promotion with partial push https://launchpadlibrarian.net/418481718/fedora28_master.log-2019040810:18
quiquell|roverpanda: also is not retry since it's promoted10:18
*** florianf has joined #tripleo10:19
openstackgerritMerged openstack/os-net-config stable/rocky: Add ovs_options and ovs_extra parameters to VLAN object  https://review.openstack.org/65097310:20
openstackgerritMerged openstack/os-net-config stable/queens: Add ovs_options and ovs_extra parameters to VLAN object  https://review.openstack.org/65098310:20
quiquell|roversshnaidm|pto: Do you know why we don't do exit 1 here ? https://github.com/rdo-infra/ci-config/commit/f81c91283560db5bf8f1c4125e5a1f1e6742f12a10:21
quiquell|roverUps pto10:21
quiquell|roverpanda: this is the change https://github.com/rdo-infra/ci-config/commit/f81c91283560db5bf8f1c4125e5a1f1e6742f12a10:21
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run octavia-api under httpd  https://review.openstack.org/63638010:26
beaglesmarios: ^ added release note and fixed commit message. Thanks for the reviews!!!10:29
mariosbeagles: ack thanks10:31
beagles00212910:31
beaglesnotebook move fail10:33
* marios notes beagles pin number10:34
mariosbeagles: mother's maiden name?10:34
marios(just curious)10:34
*** pdeore has quit IRC10:34
beagleslol!10:34
*** ykarel is now known as ykarel|afk10:41
bogdandobeagles: hi10:43
beaglesbogdando: hi!10:43
bogdandobeagles: so bug 1823377 boils down to those pid files should be not created as root, which is prolly achievable via updating sudoers and allowing sueing those commands what wrappers invoke?..10:45
openstackbug 1823377 in tripleo "restarting the neutron dhcp agent container restarts dhcp sidecars" [High,Triaged] https://launchpad.net/bugs/182337710:45
bogdandolike that dnsmask should be su neutron?10:45
bogdandoI have an env if you want to look into10:45
openstackgerritMerged openstack/puppet-tripleo master: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65111610:45
bogdandobeagles: ^^10:46
beaglesbogdando: yeah - here's the thing that's weird. If the dhcp agent really cannot read these files, how does it know how to kill it when the agent is restarted10:46
beaglesbogdando: thought that doesn't seem to always be the case as the upstream CI failures seem to reflect it can't kill them properly either10:47
bogdandobeagles: may be it should use escalated privs10:47
beaglesbogdando: maybe10:47
bogdandothat's for neutron side tho10:47
bogdandomay be we can fix it at the wrappers side10:47
beaglesbogdando: yeah - I'm spinning up a devstack instance to verify that the behavior of the agent hasn't changed in some way with respect to the restarts10:48
bogdandobeagles: yes, CI may not fail but side cars always get restarted10:48
beaglesbogdando: ack10:48
bogdandoand in stein we struggled to fix that default behaviour valid for rocky10:49
Tenguah. ntp issue for puppet-tripleo merged on Master.10:49
bogdandobeagles: ack10:49
TenguCare to marge the other branches folks? https://review.openstack.org/#/q/status:open+project:openstack/puppet-tripleo+topic:bug/1823838  thanks :)10:49
Tengu*merge10:49
*** owalsh has quit IRC10:53
*** owalsh_ has joined #tripleo10:53
Tenguquiquell|rover: do you know if we need that same ntp fix downstream?10:55
*** udesale has quit IRC10:56
quiquell|roverTengu: no clue on it10:58
Tenguquiquell|rover: who might know? weshay ?10:58
weshayTengu ask Jon S10:59
Tenguweshay: hmmm I don't think I know which Jon you're taking about :/.11:01
quiquell|roverweshay: this is the LP for missing containers at fedora28 https://bugs.launchpad.net/tripleo/+bug/182313611:01
openstackLaunchpad bug 1823136 in tripleo "Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/fedora-binary-cinder-api:59bf4b8a4c6b00c7ebb9b18b67853f6e318b2a51_d9bb6f52" [Critical,In progress] - Assigned to Quique Llorente (quiquell)11:01
quiquell|roverweshay: in case you want to reference it at the criteria fix review11:01
*** florianf_ has joined #tripleo11:05
*** florianf has quit IRC11:05
*** florianf_ is now known as florianf11:05
*** holser|lunch is now known as holser_11:05
*** hberaud|lunch is now known as hberaud11:07
weshayTengu so is this ntp issue what caused upstream tripleo and 3rd party to start going red about 7 hours ago?11:07
weshaylooks like it11:07
*** zbr has joined #tripleo11:08
Tenguweshay: apparently yeah11:09
weshayTengu thanks for jumping on it!11:09
weshayTengu++11:09
Tenguour best guest: the "4.2.x" branch/tag was dropped upstream, creating some issues with puppet-tripleo unit tests11:09
weshayquiquell|rover ^11:09
Tenguweshay: so I've pushed patches for all branches, we need to make them merge: https://review.openstack.org/#/q/status:open+project:openstack/puppet-tripleo+topic:bug/182383811:09
Tenguweshay: and now the question is: do we need that downstream? :)11:10
TenguI'm pretty sure we do11:10
quiquell|roverweshay: yep we have being working on this at our mornings11:10
quiquell|roverweshay: since this is important can we workflow those patches ?11:10
* Tengu would love to see them +W all the way down :)11:11
weshay+111:11
Tenguthere's just the ocata one....11:11
Tenguocata branch seems to be broken for other reasons.11:11
Tenguso another question is: do we want to go down that hole as well? that might be really nasty and ugly.11:12
Tengueven gory.11:12
*** ykarel|afk is now known as ykarel11:12
weshayyikes, /me afraid of my shadow now11:12
quiquell|roverTengu, weshay: ocata error feels so deep that no one cares11:13
weshayTengu I'll just mark that bug triaged, promotion-blocker and it will get on rel-del's radar11:15
*** amoralej is now known as amoralej|lunch11:16
Tenguweshay: fine for me, thanks!11:19
Tenguquiquell|rover: yeah, same feeling....11:19
Tengusad, but.11:19
TenguTripleO isn't like a good wine - doesn't get better with age :D11:20
quiquell|roverweshay: so we workflow this stuff ?11:20
*** panda is now known as panda|lunch11:20
weshayquiquell|rover I am not ooo core11:20
weshaybut yes.. I think we should11:20
quiquell|roverok will do11:20
Tengu\o/11:20
Tengupatches, make patches land, yeah baby!11:20
quiquell|roverTengu: done, let me know if something is missing there11:21
Tenguquiquell|rover: guess we're good now. I'll abandon the ocata one.11:21
Tenguprobably the best thing to do, and dig a deeper hole for that one11:21
quiquell|roverYep, let's be reactive there11:23
Tengu+un in fact, but yeah ;).11:24
Tenguweshay: I've also added a CIX card - you can check if I did it right :)11:24
*** ansmith_ has quit IRC11:24
*** bogdando has quit IRC11:25
weshayTengu you don't have to create those by hand man11:26
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: run f28 standalone job  https://review.openstack.org/65119711:26
weshayTengu if you have promotion-blocker tag and it's triaged / critical it will be created automatically11:26
Tenguweshay: #magic11:27
Tenguwasn't sure how to trigger the card creation in fact.11:27
Tengubut good to know.11:27
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: run f28 standalone job  https://review.openstack.org/65119711:27
*** abishop has quit IRC11:28
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:29
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Make Ceph RGW accept ResellerAdmin role for administration  https://review.openstack.org/63781511:30
*** boazel has joined #tripleo11:30
openstackgerritPiotr Kopec proposed openstack/tripleo-heat-templates master: Allow NovaRbdPoolName to be role specific  https://review.openstack.org/65025311:30
openstackgerrityogananth subramanian proposed openstack/tripleo-heat-templates stable/queens: Provide option to disable EMC in puppet-vswitch  https://review.openstack.org/65120111:33
*** boazel_ has quit IRC11:33
*** jpena is now known as jpena|lunch11:37
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: image_uploader: use buildah instead of podman to manipulate images  https://review.openstack.org/65098511:37
*** morazi has joined #tripleo11:38
EmilienMweshay, mwhahaha: I would like to try to land https://review.openstack.org/#/c/650985/ - it seems like it wasn't tested by https://review.rdoproject.org/r/#/c/19157/ even if we do Upstream-Id, I probably missed some Depends-On another patch? Anyway, we tested it with weshay yesterday and it seemed to work fine11:39
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:40
openstackgerritGael Chamoulaud proposed openstack/tripleo-heat-templates master: Fix minor typos  https://review.openstack.org/65120311:41
weshayEmilienM cool man thanks! I'd like to get the standalone upgrade job back to working too.. I think we started there11:41
EmilienMweshay: yeah last time I looked wasn't it ovn stuff?11:43
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:43
weshayI have to go back through it11:43
weshaytoo many things to keep straight11:43
EmilienMweshay: https://review.openstack.org/#/c/649476/11:44
weshayaye11:44
EmilienMweshay: http://logs.openstack.org/76/649476/5/check/tripleo-ci-centos-7-standalone-upgrade/33cf41f/job-output.txt.gz#_2019-04-05_03_46_38_60459711:44
openstackgerritChristian Schwede proposed openstack/tripleo-common master: Improve Swift healthchecks  https://review.openstack.org/65120411:45
weshaylooking11:45
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:45
weshayah yay! debug11:45
EmilienMweshay: http://logs.openstack.org/76/649476/5/check/tripleo-ci-centos-7-standalone-upgrade/33cf41f/logs/undercloud/var/log/journal.txt.gz#_Apr_05_03_32_1111:45
EmilienMApr 05 03:33:54 standalone.localdomain podman[73793]: There is no ovn-northd process connected to socket sockets running in the container11:46
EmilienMApr 05 03:33:54 standalone.localdomain podman[73793]: exit status 111:46
EmilienMthat's where I left thing11:46
EmilienMI can run a recheck11:46
weshayEmilienM that is something for ovn folks to fix?11:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - do not stop / remove docker during upgrade to podman  https://review.openstack.org/64947611:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - do not stop / remove docker during upgrade to podman  https://review.openstack.org/64947611:46
openstackgerrityogananth subramanian proposed openstack/tripleo-heat-templates stable/rocky: Provide option to disable EMC in puppet-vswitch  https://review.openstack.org/65120511:46
EmilienMweshay: to investigate for sure11:46
weshayk11:46
quiquell|roverEmilienM, weshay: https://bugs.launchpad.net/tripleo/+bug/182304211:47
openstackLaunchpad bug 1823042 in tripleo "Upgrading from rocky to stein/master does not verify the neutron mechanism correctly" [High,In progress] - Assigned to Kamil Sambor (ksambor)11:47
quiquell|roverjfrancoa: ^ I think there were someone already into it ?11:47
ksamboryeah it even fixed here: https://review.openstack.org/65006111:48
ksamborquiquell|rover ^^^11:48
jfrancoaquiquell|rover: yes, I was testing it this morning, it was like a charm! thanks ksambor11:48
jfrancoaksambor++11:48
jfrancoa(ohhh no karma here :-D )11:48
openstackgerritGael Chamoulaud proposed openstack/tripleo-heat-templates master: Fix minor typos  https://review.openstack.org/65120311:50
EmilienMksambor, weshay: I'm not sure that will fix our case ^11:50
weshayEmilienM quiquell|rover the logs we're looking at are not related to upgrade other than the job name11:50
weshaythe initial deployment validation failed11:50
EmilienMtbh, idk, we'll see11:50
Tenguweshay, quiquell|rover : so apparently no need to import dowstream the fix - had a quick talk with rel-del, we're fine like that.11:50
weshayaight11:50
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:50
openstackgerritChristian Schwede proposed openstack/tripleo-common master: Improve Swift healthchecks  https://review.openstack.org/65120411:51
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Enable additional Swift healtchecks  https://review.openstack.org/65120711:51
quiquell|roverweshay: ack, just a pointer popping to my mind11:51
weshayaye11:51
weshayquiquell|rover your mind is compiled in C?11:52
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:52
quiquell|roverweshay: Maybe on scratch https://en.wikipedia.org/wiki/Scratch_(programming_language)11:53
* Tengu prefers brainfuck11:54
Tenguor spaces11:54
Tenguor ook11:54
quiquell|roverTengu: ook is the winner11:54
*** abishop has joined #tripleo11:55
Tenguquiquell|rover: ok, there's a terrible one, named Malbolge. that one has my love :)11:55
beaglesbogdando: from devstack looks to me like the dhcp agent is restarted dnsmasq each time ... ugh. Checking with neutron devs11:55
Tengucan't be written by human. only computer-generated.11:55
Tenguoh11:55
TenguOH11:55
Tengujust found out there's a "pikachu" language!!11:55
Tenguhttp://trove42.com/pikachu-syntax-rules/11:56
*** dsneddon has joined #tripleo11:56
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119811:56
quiquell|roverTengu: we need a ooo language11:56
quiquell|rover:-)11:56
Tenguquiquell|rover: ooo[k] ;)11:57
Tenguquiquell|rover: but seeing how complicated and evil all is inside ooo, we might go for the malbolge.11:57
Tengualready existing.11:57
openstackgerritChristian Schwede proposed openstack/tripleo-common master: Improve Swift healthchecks  https://review.openstack.org/65120411:58
*** aakarsh has quit IRC12:01
*** aakarsh has joined #tripleo12:01
*** dsneddon has quit IRC12:01
*** dsneddon has joined #tripleo12:02
*** dsneddon has quit IRC12:07
*** quiquell|rover is now known as quique|rover|eat12:07
*** aedc has joined #tripleo12:07
openstackgerritMerged openstack/puppet-tripleo stable/queens: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65112012:08
openstackgerritMerged openstack/puppet-tripleo stable/rocky: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65111812:08
openstackgerritMerged openstack/puppet-tripleo stable/pike: Fix ntp module version to an existing tag/branch  https://review.openstack.org/65112212:08
quique|rover|eatykarel: I think build-test-package fix to fail is working now https://review.openstack.org/#/c/636588/12:09
*** jfrancoa has quit IRC12:09
Tengu\o/12:11
ykarelquique|rover|eat, ack but still looks like it's not handling Depends-On, part12:11
ykarellet me update ur testing patch12:11
ykarelso it's more clear12:11
quique|rover|eatykarel: ack12:11
*** dsneddon has joined #tripleo12:12
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Remove validations from featureset056  https://review.openstack.org/65121312:12
marios Tengu: fyi - not sure yet but we are going to move it voting /want to anyway so maybe we need to move that ^^^12:12
mariosTengu: i'll -1 it for now12:12
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: WIP: run more jobs on requirements changes  https://review.openstack.org/65121412:12
*** gfidente has quit IRC12:13
quique|rover|eatykarel: stein centosci have still issues ?12:13
*** aakarsh has quit IRC12:13
*** gfidente has joined #tripleo12:13
Tengumarios: hmm yeah, we will need to configure validations for the CI.12:14
Tenguthat's in the pipe anyway :)12:14
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: DNM: ansible 2.7.1 test  https://review.openstack.org/65016912:14
Tenguoh, sorry. didn't see what validation you dropped.12:14
Tengumarios: does it create issues with those jobs ?12:15
ykarelquique|rover|eat, didn't checked that12:15
ykarellooking now12:15
ykarelquique|rover|eat, i see a pass on 8th12:15
mariosTengu: validation is failing right now (ovn northd). regardless of it is is a valid fail i.e. there is a problem with some service, going on your comment there, it seems you want it in a non voting job. and this job is going voting.12:15
mariosTengu: i'd like to see if it passes after we remove that validatino for now anyway12:16
*** wolverineav has joined #tripleo12:16
*** dsneddon has quit IRC12:16
gchamoulTengu, marios: this is not related to the validations from tripleo-validations, right?12:19
*** wolverineav has quit IRC12:20
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: containerized-undercloud-upgrade back to voting  https://review.openstack.org/65121812:20
mariosgchamoul: i think this is a different12:21
mariosTengu: ^ have a pointer to where you're checking th containers etc i dont have it handy12:21
gchamoulmarios: yes it is12:21
gchamoulsorry12:21
*** jfrancoa has joined #tripleo12:23
*** ratailor has quit IRC12:24
ykarelTengu, so just to clear the guess about puppet ntp, i can see 4.2.x branch in a fork12:25
ykarelhttps://github.com/amuraleedhar/puppetlabs-ntp/branches/stale12:25
ykareland those are cleaned up in main repository12:25
ykareldon't know the reason though, but clean all branches doesn't seem ideal12:26
ykarelquique|rover|eat, fyi ^^12:26
*** ansmith_ has joined #tripleo12:26
mariosgchamoul: np12:27
*** matbu has joined #tripleo12:27
beaglesmwhahaha: EmilienM, if you get the chance https://review.openstack.org/#/c/636380/12:29
*** jpena|lunch is now known as jpena12:29
EmilienMbeagles: sure, will wait for CI to vote12:30
beaglesEmilienM: cool thanks!12:30
*** toure|gone is now known as toure12:32
*** rlandy has joined #tripleo12:32
*** apetrich has quit IRC12:32
*** rlandy is now known as rlandy|ruck12:32
openstackgerritAlan Bishop proposed openstack/tripleo-quickstart-extras master: WIP/DNM: Add split-controlplane-standalone.yml playbook  https://review.openstack.org/61736812:34
*** ade_lee has quit IRC12:37
weshayCI Status: GREENISH, RDOCloud Status: YELLOW| https://docs.openstack.org/tripleo-docs/latest/12:37
*** aakarsh has joined #tripleo12:38
*** apetrich has joined #tripleo12:39
abishopowalsh_: can we proceed with https://review.openstack.org/650287 ? works great for me!12:39
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates stable/ocata: [DNM] Test tq refactorings  https://review.openstack.org/65119812:42
*** openstackgerrit has quit IRC12:44
*** openstackgerrit has joined #tripleo12:44
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates stable/queens: [DNM] Test tq refactorings  https://review.openstack.org/65122112:44
*** quique|rover|eat is now known as quiquell|rover12:44
*** apetrich has quit IRC12:45
Tengufolks, care to +W that one (once again...) https://review.openstack.org/#/c/648331/  ? It's been some weeks now I'm stuck with it due to CI issues :D12:45
quiquell|roverykarel: so we don't want to add rdoinfo, rdo-jobs, etc to skiplist ? https://review.rdoproject.org/r/#/c/18892/12:45
quiquell|roverykarel: the list is too massive ?12:45
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates stable/rocky: [DNM] Test tq refactorings  https://review.openstack.org/65122412:45
*** dtantsur is now known as dtantsur|brb12:46
*** mcornea has joined #tripleo12:47
ykarelquiquell|rover, so i added just two examples in ur review and they failed, there can/are other projects as well12:48
ykarelsome i know some i don't12:48
quiquell|roverykarel: so all depends on how we we to design this12:49
ykarelquiquell|rover, i would say we should go step back, and see what's the problem and then fix it12:49
quiquell|roverykarel: will bring it today at the community call I think12:49
*** Goneri has joined #tripleo12:50
ykarelquiquell|rover, so good to avoid breaking jobs by merging it as it is, this needs more work, can you put -W there12:50
quiquell|roverykarel: yep, thanks12:51
quiquell|roverTengu: merged https://review.openstack.org/#/q/I6a5a0541d2cccc2e210954c4d3b3c45d24d1d64912:52
Tenguquiquell|rover: yep!12:53
Tenguquiquell|rover: was about to move the card out of the CIX.12:53
beaglesweshay: what would the criteria be for making scenario010-standalone voting?12:54
*** apetrich has joined #tripleo12:54
Tenguquiquell|rover: moved to "Done", since we probably don't have any preventative action for that kind of things.12:54
*** sanjayu_ has quit IRC12:55
weshaybeagles mwhahaha and others will look at http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario010-multinode-oooq-container12:55
owalsh_abishop: ack, just want to figure out which is best for https://review.openstack.org/#/c/650287/1/overcloud.j2.yaml@296, then I'll remove the WIP later today12:55
weshayand it needs to be consistently passing for a few days to a week12:55
*** jroll has quit IRC12:55
beaglesweshay: k - it actually does beg the question of the deployments that involve external deploy tasks12:56
weshaybeagles so it would be good to have to the reviews queued up to make it voting and we'll check it..12:56
beaglesweshay: been doing that :)12:56
abishopowalsh_: ok, thx!12:56
weshaybeagles get it on the #tripleo agenda so more folks are working towards that goal12:56
quiquell|roverTengu: thanks mate12:56
beaglesweshay: because standalone doesn't run these tasks under mistral, there are certain types of issues that are not caught by the standalone jobs12:56
weshaybeagles can you rephrase "external deploy tasks"12:56
Tenguquiquell|rover: well, thank you for your support (as well as ykarel of course ;))12:57
* weshay is not 100% clear on what you mean12:57
Tenguweshay: standalone doesn't involve mistral at all, thus we might not see some issues caused by the mistral involvement12:57
*** baha has joined #tripleo12:57
beaglesweshay: they are basically ansible tasks that are run "external" to the normal deployment framework - octavia uses them as does ceph-ansible12:58
*** owalsh_ is now known as owalsh_afk12:58
*** jroll has joined #tripleo12:59
weshayTengu yes.. we need multinode jobs on mistral where mistral is in play12:59
Tengu:)12:59
weshaybeagles so we need to use zuul to run check or 3rd party  on the src of the external ansible tasks12:59
weshaybeagles same as what gfidente has done or is doing for ceph13:00
weshaygfidente you and jpena had that in progress last I checked13:00
gfidenteweshay hey yes I think jpena did all its pieces already13:00
gfidenteweshay we can build an rpm out of PR13:00
gfidentebut I don't know which job to trigger yet13:01
beaglesweshay: for octavia, they are run during the actual deploy - i.e. they are tested in scenario010 standalone and multinode so...13:01
weshayjpena /me wonders if we have doc on running zuul jobs on 3rd party repos13:01
gfidenteweshay maybe scenario1013:01
*** leanderthal has joined #tripleo13:01
jpenaweshay: of course :). Let me find the link13:01
weshayjpena rock on :)13:01
jpenaweshay: https://www.softwarefactory-project.io/gating-projects-on-github-using-zuul.html13:01
gfidentejpena weshay not sure what infos we need, only the job name?13:01
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: DNM: run f28 standalone job  https://review.openstack.org/65119713:01
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Make standalone f28 job voting again  https://review.openstack.org/65123013:01
beaglesweshay: in octavia's case it isn't really "external" per-se so much as a mechanism we can use to do things that can't be done through other means13:02
jpenaweshay: this is actually in place at review.rdoproject.org, so you can check the config repo13:02
*** ykarel is now known as ykarel|afk13:02
beaglesweshay: anyways - I'll add to agenda about the voting thing and work on patches to reintroduce multinode testing for octavia and see where we go from there13:02
gfidentejpena weshay I think the job we need to trigger is tripleo-ci-centos-7-scenario010-multinode-oooq-container13:03
*** panda|lunch is now known as panda13:03
gfidenteweshay jpena (for ceph that is)13:05
*** mjturek has joined #tripleo13:07
*** ykarel|afk has quit IRC13:09
*** sai_p_ has joined #tripleo13:10
*** jcoufal has joined #tripleo13:10
rfolcoThe CI Community Meeting (office open hours) has been moved to 13:30 UTC (30 minutes before the TripleO meeting).13:11
*** bfournie has joined #tripleo13:11
TenguEmilienM: care to +W this one? https://review.openstack.org/#/c/648331/  hopefully it won't hit any CI issue this time -.-'13:11
*** raildo has joined #tripleo13:12
*** boazel_ has joined #tripleo13:13
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: [DNM] Hack br-ex to test scenario001-standalone for Nautilus  https://review.openstack.org/65123113:13
*** boazel has quit IRC13:16
rfolcoThe CI Community Meeting (office open hours) has been moved to 13:30 UTC (30 minutes before the TripleO meeting) - starts in 10 min at https://bluejeans.com/411356779813:21
*** nawar has left #tripleo13:26
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: DO-NOT-MERGE Test new ceph-container builds  https://review.openstack.org/65123913:26
*** amoralej|lunch is now known as amoralej13:27
*** hberaud has quit IRC13:28
*** hberaud has joined #tripleo13:29
*** tzumainn has joined #tripleo13:30
EmilienMbeagles: see my comment on https://review.openstack.org/#/c/636380/13:30
beaglesEmilienM: eep thanks13:31
rfolcoThe CI Community Meeting (office open hours) start now at https://bluejeans.com/4113567798 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting13:31
weshaybeagles you are welcome to join the ci community call...  open door13:32
*** ykarel has joined #tripleo13:32
beaglesEmilienM: seems the ovn provider needs to be in the image https://review.openstack.org/#/c/644587/ - there was a window where this wasn't necessary - seems this has changed13:33
beaglesweshay: thanks will do13:33
* beagles scrambles for some headphones13:33
*** ade_lee has joined #tripleo13:34
EmilienMbeagles: +W13:34
*** mjturek has quit IRC13:35
beaglesEmilienM: thanks13:35
EmilienMbeagles: but not the one which failed13:36
*** mjturek has joined #tripleo13:36
beaglesEmilienM: of course :)13:36
EmilienMbeagles: I would "recheck" once the latest is landed and is in containers13:36
*** udesale has joined #tripleo13:36
beaglesEmilienM: yeah - we were planning on adding config to to control the providers that are being loaded13:36
*** aedc has quit IRC13:37
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image list command  https://review.openstack.org/64961913:37
*** vinaykns has joined #tripleo13:38
beaglesEmilienM: might've also been usable to work around this issue but meh better to have default behavior work from the get go13:38
*** mwhahaha changes topic to "CI Status: GREENISH, RDOCloud Status: YELLOW, promoted current-tripleo 3/18 | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs/latest/"13:45
mwhahahabeagles: http://logs.openstack.org/80/636380/27/check/tripleo-ci-centos-7-scenario010-multinode-oooq-container/fb50405/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-04-09_12_37_16 do you have a fix for the permission denined problem?13:47
*** baha has quit IRC13:53
beaglesmwhahaha: looking13:53
beaglesmwhahaha: crap thought that wasn't happening any longer13:53
*** Vorrtex has joined #tripleo13:58
jaosorior#startmeeting tripleo14:00
openstackMeeting started Tue Apr  9 14:00:49 2019 UTC and is due to finish in 60 minutes.  The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: tripleo)"14:00
openstackThe meeting name has been set to 'tripleo'14:00
jaosorior#topic agenda14:00
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:00
mwhahahao/14:00
dpeacocko/14:00
jaosorior* Review past action items14:01
ksamboro/14:01
jaosorior* One off agenda items14:01
Vorrtexo/14:01
jaosorior* Squad status14:01
jaosorior* Bugs & Blueprints14:01
jristo/14:01
jfrancoao/\o14:01
jaosorior* Projects releases or stable backports14:01
jaosorior* Specs14:01
jaosorior* open discussion14:01
marioso/14:01
rfolcoo/14:01
jaosoriorAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:01
jaosoriorHey folks! who's around?14:01
beagleso/14:01
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: WIP: Add rocky standalone job  https://review.openstack.org/65124714:01
weshay0/14:01
rlandy|rucko/14:01
zbro/14:01
jaosorior#topic review past action items14:03
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:03
jaosoriorNone.14:03
jaosorior#topic one off agenda items14:03
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:03
jaosorior#link https://etherpad.openstack.org/p/tripleo-meeting-items14:03
jaosoriorThe first topic is from beagles : working towards making scenario010-standalone (octavia) voting14:04
rh-jelabarreo/14:04
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Modify roles to remove unused services  https://review.openstack.org/64203214:04
beagleshi all, octavia has been around for a bit but we've not had voting CI jobs for it14:04
EmilienM+214:05
mwhahahacause they aren't passing </troll>14:05
fultonjo/14:05
mwhahahalet's get them green and we can switch them to voting14:05
beaglesas such things tend to get broken and unnoticed14:05
*** baha has joined #tripleo14:05
beaglesyeah. multinode probably needs to be voting as well because it is catching issues standalone is not14:06
beaglesquesitons that come to mind, do we need both - and if so, might be more of a priority to get multinode voting stable and voting first14:07
fultonjIf multinode goes back to voting it might catch things standalone doesn't, but if history is a lesson, then I think the cost would be less reliable CI.14:07
beagles(catches more stuff)14:07
fultonj(tripleo ci team should say though)14:08
fmounto/14:08
mwhahahaso the concern with adding a multinode is resources14:08
beaglesfultonj: good point14:08
weshaywe could just run the multinode against mistral projects14:08
Tengu«o/14:08
mwhahahawe've worked really hard to eliminate multinode as much as possible14:08
beaglesmwhahaha: I wonder if there would be advantages to be careful about what triggers what14:09
mwhahahamaybe keep both?14:10
mwhahahaand only run the multinode on mistral specific code14:10
weshay+114:10
mwhahahathis is one of those extra services that we can kinda cover upstream but the multinode should probably be tested downstream due to resource constraints14:10
mwhahahasince octavia is an extra and not core service i'm not sure we should use the resources upstream for it14:11
mjtureko/14:12
*** dprince has joined #tripleo14:13
jaosoriormwhahaha: +114:14
beaglesI've also been thinking about backports  - would be good to get that up and voting as well but for that it would be multinode14:14
fultonjmwhahaha: +114:14
beaglesbut I'm getting ahead of myself14:14
fultonjat least that was my experience with ceph once14:14
mwhahahalets get the jobs fixed up and have the conversation then14:14
fultonje.g. a downstream testing had to catch something multinode14:14
mwhahaha+1 to standalone version going forward being voting14:15
fultonjnot ideal but resources are limited and downstream testing was sufficient14:15
beaglesk14:15
jaosoriorthe next topic is from rfolco: stein branching: we're watching tripleo channel to make sure stein jobs will run successfully.14:17
rfolcoyep, the jobs should run as soon as we branch tripleo projects, ci bits are in place14:18
weshaydid the blockers to branching get resolved, /me needs to look14:18
weshayEmilienM mentioned a few last week14:18
mwhahahawhen do we want to branch stein? this week?14:19
EmilienMmaybe we could branch tripleoclient first14:19
EmilienMwait to see how CI works etc14:19
EmilienMand in few weeks we branch the rest14:19
EmilienMthat's how we do usually14:19
mwhahahatripleoclient or tripleo-common?14:19
marioswell they need each other14:20
EmilienMclient iirc14:20
mwhahahak lets get it started i guess14:20
jristra ra sisk boom bah14:20
ksambormwhahaha, +1 we definitely need tripleo-common otherwise we will not test all changes14:21
weshayplease don't sing14:21
jristI was cheerleading thank you very much14:21
EmilienMmwhahaha: you want me to propose the patch?14:22
mwhahahayes plz14:22
EmilienM#action EmilienM to branch tripleoclient stable/stein this week14:22
*** raildo has quit IRC14:22
*** raildo has joined #tripleo14:22
rfolcoEmilienM, I offer support for watching jobs and making sure they run well14:23
jaosorior#topic Squad status14:24
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:24
jaosoriorci14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting14:24
jaosoriorupgrade14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:24
jaosoriorcontainers14:24
jaosorior#link https://trello.com/b/S8TmOU0u/tripleo-podman14:24
jaosoriorintegration14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:24
jaosoriorcli14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:24
jaosoriorvalidations14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:24
jaosoriornetworking14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:24
jaosorioredge14:24
jaosorior#link https://etherpad.openstack.org/p/tripleo-edge-squad-status14:24
jaosoriorAny squad wanting to bring up their status, or a topic for the general public?14:24
TenguValidations: we still need a ton of reviews.14:25
Tengu#link https://trello.com/c/k96ySxA0/18-review-migrating-validations-playbooks-to-their-own-ansible-role14:25
Tenguplease-please-please :)14:25
weshayplease don't sing14:25
EmilienMmwhahaha: https://review.openstack.org/65125314:25
Tenguweshay: bah, sun's shining here.14:26
Tenguweshay: as long as I'm not "singing under the rain", "it's alright, it's ok" :D14:26
ykarelEmilienM, we would also need to do branching for tripleo in RDO as well14:27
mwhahahathought we already branched rdo?14:27
mwhahahaor did you folks just ask about it and not do it14:27
ykarelmwhahaha, not branched yet, current status: https://review.rdoproject.org/etherpad/p/stein-release-preparation14:27
ykarelL 11114:27
ykarelif you can propose branching, builds will be automatic by bot14:28
mwhahahak14:28
jaosorior#topic bugs & blueprints14:29
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:29
jaosorior#link https://launchpad.net/tripleo/+milestone/stein-rc114:29
jaosoriorFor Stein we currently have 2 (for stein-rc1) blueprints open in Launchpad.14:29
jaosoriorBugs: 221 (+9) stein-rc1, 288 (-1) train-1 in Launchpad. 104 (0) open Storyboard bugs.14:29
jaosorior#link https://storyboard.openstack.org/#!/project_group/7614:30
*** dtantsur|brb is now known as dtantsur14:30
jaosorior#topic projects releases or stable backports14:30
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:30
jaosorior#topic specs14:32
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:32
jaosorior#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:32
jaosoriorAny specs folks want to bring attention to?14:32
Tenguyeah14:32
Tenguwe still need some reviews on https://review.openstack.org/645508 - note it's for Train already14:32
Tengubut that one, the sooner we're OK with it, the sooner work can start. It will be long :)14:33
jaosoriorack14:34
jaosorior#topic open discussion14:36
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:36
jaosoriorAnything else that folks want to bring up to the meeting?14:36
mjturekI got a quick one, baha and I uploaded this rdo patch yesterday to move our ppc64le container build job to the zuul playbooks used by tripleo-ci. If anyone can take a look at the patch we'd really appreciate some feedback. https://review.rdoproject.org/r/#/c/20031/14:36
*** devep has joined #tripleo14:38
mjtureks/zuul playbooks/playbooks14:38
*** quiquell|rover is now known as quiquell|off14:38
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Make scenario000 multinode upgrades job voting  https://review.openstack.org/65125814:41
*** owalsh_afk is now known as owalsh14:41
jaosorior#endmeeting14:43
*** openstack changes topic to "CI Status: GREENISH, RDOCloud Status: YELLOW, promoted current-tripleo 3/18 | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs/latest/"14:43
openstackMeeting ended Tue Apr  9 14:43:12 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:43
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-04-09-14.00.html14:43
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-04-09-14.00.txt14:43
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-04-09-14.00.log.html14:43
*** devep has quit IRC14:45
*** jistr is now known as jistr|call14:51
*** rh-jelabarre has left #tripleo14:55
*** boazel has joined #tripleo14:56
*** rh-jelabarre has joined #tripleo14:56
*** boazel_ has quit IRC14:58
EmilienMjaosorior: can you review https://review.openstack.org/#/c/650985/ please ?15:08
TenguEmilienM: would buildah be faster for that task?15:11
EmilienMTengu: possibly15:11
TenguEmilienM: ok. well. let's see and hope :)15:12
Tengusee you tomorrow folks :)15:12
openstackgerritAlan Bishop proposed openstack/tripleo-quickstart-extras master: WIP/DNM: Add split-controlplane-standalone.yml playbook  https://review.openstack.org/61736815:16
openstackgerritMerged openstack/puppet-tripleo stable/queens: Simplify logrotate job  https://review.openstack.org/64833115:19
EmilienMjaosorior: thx15:20
jaosoriornp15:21
*** panda has quit IRC15:22
*** aedc has joined #tripleo15:22
*** bogdando has joined #tripleo15:25
Tenguwoohooo! my queens patch merged at last!15:26
*** leanderthal has quit IRC15:27
*** iurygregory_ has joined #tripleo15:32
*** panda has joined #tripleo15:35
*** iurygregory has quit IRC15:35
*** udesale has quit IRC15:37
*** iurygregory_ is now known as iurygregory15:38
openstackgerritMerged openstack/tripleo-validations master: Add network-environment role  https://review.openstack.org/64076115:42
openstackgerritGiulio Fidente proposed openstack/tripleo-quickstart-extras master: Use Ceph Nautilus for newer OpenStack releases  https://review.openstack.org/65128115:45
*** aedc has quit IRC15:45
*** cylopez has quit IRC15:48
*** apetrich has quit IRC15:50
openstackgerritPedro Alvarez Piedehierro proposed openstack/diskimage-builder master: Fix Fedora aarch64 image location  https://review.openstack.org/65128315:50
*** jistr|call is now known as jistr16:00
EmilienMksambor, jfrancoa, weshay : still broken: http://logs.openstack.org/76/649476/7/check/tripleo-ci-centos-7-standalone-upgrade/23f7e49/logs/undercloud/var/log/journal.txt.gz#_Apr_09_12_47_3316:00
*** jfrancoa has quit IRC16:01
ksamborhttps://review.openstack.org/#/c/650061/ is still not merged16:02
ksamborEmilienM, ^^^16:02
*** aedc has joined #tripleo16:05
*** mjturek has quit IRC16:05
*** morazi has quit IRC16:06
openstackgerritGiulio Fidente proposed openstack/tripleo-quickstart master: Update all Stein and master jobs to use Ceph Nautilus  https://review.openstack.org/65128616:07
openstackgerritMerged openstack/tripleo-common master: Update required packages for octavia api  https://review.openstack.org/64458716:09
*** yprokule has quit IRC16:09
ksamborAlso when I'm thinking now, we probably should provide /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs.yaml16:12
ksambor otherwise we python client during 'upgrade prepare'  will return error. But I'm not sure where we should provied this env file, probably in job definiot in standalone_custom_env_files,  EmilienM16:12
openstackgerritGiulio Fidente proposed openstack/tripleo-common master: Update default Ceph container image to use to the Nautilus version  https://review.openstack.org/65128716:13
*** marios has quit IRC16:14
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Test new ceph-container builds  https://review.openstack.org/65123916:14
*** aedc has quit IRC16:20
*** lucasagomes has quit IRC16:22
*** dsneddon has joined #tripleo16:23
beaglesbandini: re comment to  https://review.openstack.org/#/c/649599/ do you know if there is a patch out there to enable network? I vaguely recall seeing this somewhere16:23
beaglesdsneddon: ^^16:25
bandinibeagles: https://bugzilla.redhat.com/show_bug.cgi?id=1666387 is tracking it16:26
openstackbugzilla.redhat.com bug 1666387 in rhosp-director "br-ctlplane interface doesn't come up at boot time after rebooting the undercloud post install" [Urgent,On_dev] - Assigned to emacchi16:26
*** paramite has quit IRC16:26
EmilienMksambor: I'm not testing with an overcloud16:27
EmilienMksambor: and https://review.openstack.org/#/c/650061/ is for overcloud16:27
*** dsneddon has quit IRC16:28
bogdandobeagles: I have some ideas with fixing the wrappers via rootwrap16:28
*** dsneddon has joined #tripleo16:29
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Do not set the cell endpoints for regular split-controlplane child stacks  https://review.openstack.org/65028716:29
abishopslagle: I saw your recent edge demo (which was slick, btw!) and am curious how you extract the control plan info that is used to deploy the secondary stacks16:29
beaglesbogdando: 1s16:29
bogdandocreating a filter dropin for CLI, bind mounting it and the rootwrap conf into the wrappers, then invoke those commands via the wrapper, beagles16:29
bogdandook16:30
owalshabishop: ^^^ https://review.openstack.org/65028716:30
bogdandofor $CLI in the script I mean16:30
EmilienMApr 09 12:48:41 standalone.localdomain podman[87992]: There is no ovn-northd process connected to socket sockets running in the container16:30
EmilienMksambor: ^16:30
abishopowalsh: thx, I'll rebased (and un-WIP) my patch that's on top16:30
EmilienMthat's the error I hit, on standalone16:30
slagleabishop: it's doc'd in here: https://review.openstack.org/#/c/643339/16:31
abishopslagle: are you running a series of "openstack stack output show" commands by hand? am curious about any plans to automate16:31
slagleabishop: yes, by hand for now.16:31
slagleabishop: no specific plans, but it wouldn't be very hard to write a client command to do it16:31
fultonjlines 289 of https://review.openstack.org/#/c/643339/3/doc/source/install/advanced_deployment/distributed_compute_node.rst16:32
slaglei just don't want to put it behind a mistral workflow. too opaque and complex16:32
abishopslagle: this came up as a possible action item in the review of the (still WIP) split-controlplane CI job16:32
slagleit could be done fairly easily16:33
abishopslagle: see mwhahaha's comment at https://review.openstack.org/#/c/617368/43/roles/standalone/templates/standalone-control.sh.j2@3616:33
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Ensure support for legacy networking is enabled  https://review.openstack.org/65132416:33
beaglesbandini: EmilienM: would something like the above do the job or is this a no-no?16:34
beagles^16:34
beagleseventually I suspect that won't work but we can finesse it then16:35
slagleabishop: a tripleoclient only command would be an improvement imo16:35
EmilienMbeagles: have you seen https://review.openstack.org/#/q/topic:bug/1823353+(status:open+OR+status:merged) ?16:35
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image list command  https://review.openstack.org/64961916:36
abishopslagle: ack, mainly wanted to check in case another effort was already underway16:36
beaglesEmilienM: that must've been what I saw and couldn't find16:37
beaglesok so no worries then16:37
*** florianf has quit IRC16:38
*** boazel_ has joined #tripleo16:38
beaglesbogdando: what do you mean by "filter dropin"?16:40
*** boazel has quit IRC16:41
beaglesbogdando: also just found this https://review.openstack.org/#/c/643732/  looks suspicious16:43
* beagles goes to recheck mario's patch16:44
openstackgerritAlan Bishop proposed openstack/tripleo-heat-templates master: Set Standalone's br-ctlplane MTU  https://review.openstack.org/65011716:50
*** jpena is now known as jpena|away16:50
*** mjturek has joined #tripleo16:51
*** jpich has quit IRC16:54
openstackgerritMerged openstack/tripleo-validations master: Add openstack-endpoints role  https://review.openstack.org/64132816:56
openstackgerritMerged openstack/tripleo-heat-templates master: Fix minor typos  https://review.openstack.org/65120316:56
*** derekh has quit IRC17:00
*** psachin has quit IRC17:00
*** JqckB has quit IRC17:03
*** dtantsur is now known as dtantsur|afk17:03
*** ykarel is now known as ykarel|away17:12
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: playbooks/rotate-keys: add support for podman  https://review.openstack.org/65133317:13
*** dsneddon has quit IRC17:17
*** ykarel|away has quit IRC17:17
*** ramishra has quit IRC17:19
openstackgerritMerged openstack/python-tripleoclient master: Add checking mechanism driver during upgrade  https://review.openstack.org/65006117:23
*** gfidente is now known as gfidente|afk17:23
*** aedc has joined #tripleo17:27
*** holser_ has quit IRC17:30
*** bogdando has quit IRC17:34
mwhahahahjensas, bfournie: is StorageNetworkVlanID now required somewhere? It wasn't previously required17:36
*** kopecmartin is now known as kopecmartin|off17:36
*** devep has joined #tripleo17:41
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: playbooks/rotate-keys: add support for podman  https://review.openstack.org/65133317:42
*** dprince has quit IRC17:42
*** jpena|away is now known as jpena17:43
*** panda has quit IRC17:43
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: playbooks/rotate-keys: add support for podman  https://review.openstack.org/65133317:44
mwhahahahjensas, bfournie https://bugs.launchpad.net/tripleo/+bug/182399717:45
openstackLaunchpad bug 1823997 in tripleo "default network config deployment fails with Unknown Property StorageNetworkVlanID" [High,Triaged]17:45
*** devep has quit IRC17:46
*** wolverineav has joined #tripleo17:48
*** hberaud is now known as hberaud|gone17:48
*** holser_ has joined #tripleo17:50
*** dsneddon has joined #tripleo17:51
*** wolverineav has quit IRC17:52
*** wolverineav has joined #tripleo17:53
*** jpena is now known as jpena|off17:53
*** aakarsh has quit IRC18:08
*** amoralej is now known as amoralej|off18:15
redrobotohai again, tripleo friends!  I noticed the docker-registry stuff is deprecated now.  Whate are y'all using for a registry now? Quay?18:16
*** rlandy|ruck is now known as rlandy|ruck|hw18:19
*** jcoufal has quit IRC18:19
*** rpittau is now known as rpittau|afk18:21
*** boazel has joined #tripleo18:21
*** dprince has joined #tripleo18:22
*** boazel_ has quit IRC18:24
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: playbooks/rotate-keys: add support for podman  https://review.openstack.org/65133318:25
*** abishop has quit IRC18:25
EmilienMredrobot: no, we use httpd serving containers, through a vhost called image-serve18:26
EmilienMhttps://github.com/openstack/tripleo-common/blob/master/roles/tripleo-image-serve/templates/image-serve.conf.j218:26
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Do not archive ceph-ansible directory if it is empty  https://review.openstack.org/65095318:30
*** JqckB has joined #tripleo18:38
*** betherly has joined #tripleo18:38
*** dsneddon has quit IRC18:40
*** dsneddon has joined #tripleo18:41
*** aakarsh has joined #tripleo18:41
*** morazi has joined #tripleo18:43
*** betherly has quit IRC18:43
*** wolverineav has quit IRC18:46
*** wolverineav has joined #tripleo18:47
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Do not archive ceph-ansible fetch directory if it is empty  https://review.openstack.org/65095318:49
*** wolverineav has quit IRC18:52
*** betherly has joined #tripleo18:59
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: WIP - Run ansible before scale down  https://review.openstack.org/65135519:03
*** betherly has quit IRC19:04
*** morazi has quit IRC19:15
*** morazi has joined #tripleo19:15
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Remove ctlplane data from CI network-environment  https://review.openstack.org/57778619:20
*** betherly has joined #tripleo19:20
*** wolverineav has joined #tripleo19:23
*** wolverineav has quit IRC19:23
*** wolverineav has joined #tripleo19:24
*** morazi has quit IRC19:24
*** morazi has joined #tripleo19:24
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image list command  https://review.openstack.org/64961919:25
*** betherly has quit IRC19:25
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image list command  https://review.openstack.org/64961919:26
openstackgerritMerged openstack/puppet-tripleo stable/queens: Allow neutron wrapper containers log to syslog  https://review.openstack.org/64812819:28
openstackgerritMerged openstack-infra/tripleo-ci master: Fix tripleo-repos cross-platform installation  https://review.openstack.org/64562619:28
openstackgerritDavid Peacock proposed openstack/python-tripleoclient master: WIP - Implement tripleo container image list command  https://review.openstack.org/64961919:32
*** devep has joined #tripleo19:35
*** morazi has quit IRC19:37
*** morazi has joined #tripleo19:37
*** betherly has joined #tripleo19:41
*** toure has quit IRC19:44
*** thrash has quit IRC19:44
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: Add container buildah check jobs  https://review.openstack.org/65136219:45
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test buildah check jobs  https://review.openstack.org/65136319:45
*** betherly has quit IRC19:45
*** thrash has joined #tripleo19:46
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: turn off docker registry mirror if using local libvirt  https://review.openstack.org/65136419:51
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: DNM fix zuul, and buildset_artifacts_url  https://review.openstack.org/65045419:51
*** betherly has joined #tripleo20:01
*** betherly has quit IRC20:06
*** baha has quit IRC20:09
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test buildah check jobs  https://review.openstack.org/65136320:11
*** aakarsh has quit IRC20:15
*** devep has quit IRC20:17
*** mjturek has quit IRC20:18
*** betherly has joined #tripleo20:22
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Do not archive ceph-ansible fetch directory if it is empty  https://review.openstack.org/65095320:23
*** Goneri has quit IRC20:24
*** betherly has quit IRC20:26
*** rlandy|ruck|hw is now known as rlandy|ruck20:27
*** wolverineav has quit IRC20:27
*** morazi has quit IRC20:27
*** holser_ has quit IRC20:28
openstackgerritMerged openstack/tripleo-heat-templates master: Use tripleo-admin as octavia groups_vars owner  https://review.openstack.org/63657120:29
*** morazi has joined #tripleo20:29
*** pcaruana has quit IRC20:31
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run octavia-api under httpd  https://review.openstack.org/63638020:31
*** wolverineav has joined #tripleo20:32
*** aakarsh has joined #tripleo20:32
*** pcaruana has joined #tripleo20:33
*** pcaruana has quit IRC20:36
*** wolverineav has quit IRC20:37
*** aakarsh has quit IRC20:37
*** pcaruana has joined #tripleo20:39
*** aakarsh has joined #tripleo20:40
hjensasmwhahaha: I see we have another issue with network config params.20:42
hjensasmwhahaha: if you drop the vlan from network_data.yaml it'd work right?20:42
mwhahahahjensas: not sure I'll try that next. Had to rebuild my undercloud20:43
hjensasmwhahaha: you could also drop networks from role.networks in roles data...20:43
*** ansmith_ has quit IRC20:44
*** holser_ has joined #tripleo20:44
mwhahahahjensas: i'll try that too, but it should just work out of the box. this is the same way i've been deploying w/o network isolation for years20:45
*** aakarsh has quit IRC20:45
hjensasmwhahaha: so since the default networks_data.yaml and roles_data.yaml include all the networks. We end up rendering templates with all of that stuff in, but when we don't include environment/network-isolation.yaml it's all just Noop.20:47
*** pcaruana has quit IRC20:47
*** Vorrtex has quit IRC20:47
openstackgerritSlawek Kaplonski proposed openstack/tripleo-heat-templates master: Increase default ulimit values for Neutron agents containers  https://review.openstack.org/65113720:50
hjensasmwhahaha: If you update your nic-config adding the vlan parameters for the default nets in networks_data.yaml/roles_data.yml it should also work. tools/merge-new-params-nic-config-script.py can be helpful.20:50
*** wolverineav has joined #tripleo20:50
mwhahahahjensas: so that ths thing, i'm not using any nic-configs20:50
mwhahahahjensas: this is litterally no config20:50
hjensasmwhahaha: oh, so I must have missed some default config in tree.20:50
mwhahahahjensas: historically it just all fell back to the ctlplane with nothing else required20:51
mwhahahahjensas: yea that's my confusion as to why this property is erroring20:51
hjensasmwhahaha: well, is queens history? https://bugzilla.redhat.com/show_bug.cgi?id=1696208#c1220:51
openstackbugzilla.redhat.com bug 1696208 in openstack-tripleo "RHOSP13z5 overcloud stack failed in Step1 with following Error: Evaluation Error: Error while evaluating a Function Call, \"\" is not a valid IP address" [High,Assigned] - Assigned to hjensas20:51
mwhahahano?>20:52
mwhahahabut that's a different setup20:52
mwhahahathis is using just the defaults where all the networks magically fall back to ctlplane20:52
mwhahahanot specified as ctlplane20:52
* mwhahaha digs up the place where this happens20:52
mwhahahai think it's in a net_ip_map thing somewhere20:53
mwhahahabrb20:53
hjensasmwhahaha: I forgot to patch THT/net-config-noop.yaml20:54
hjensasmwhahaha: and net-config-bridge.yaml20:55
hjensasOS::TripleO::{{role.name}}::Net::SoftwareConfig and OS::TripleO::Controller::Net::SoftwareConfig uses those by default.20:55
hjensasmwhahaha: I will write up a patch. Sorry, I should remember nothing in CI tests the plain defaults some day. :(20:56
*** wolverineav has quit IRC20:57
*** holser_ has quit IRC20:59
*** wolverineav has joined #tripleo20:59
*** wolverineav has quit IRC21:00
*** wolverineav has joined #tripleo21:00
openstackgerritwes hayutin proposed openstack/tripleo-common master: WIP, opendaylight container failed to build stein  https://review.openstack.org/65137921:02
*** betherly has joined #tripleo21:03
mwhahahahjensas: ah ok.21:06
hjensasmwhahaha: patch in a few sec's.21:06
mwhahahai tried patching the noop file but didn't patch brdige so maybe that was it21:07
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add {{network.name}}NetworkVlanID to THT/net-config-*  https://review.openstack.org/65138121:07
mwhahahai'll try that patch out too21:07
*** betherly has quit IRC21:08
mwhahahaugh now my undercloud won't install21:09
mwhahahatypical21:09
hjensasmwhahaha: dejavu ...21:11
*** boazel_ has joined #tripleo21:13
*** wolverineav has quit IRC21:15
*** boazel has quit IRC21:15
*** wolverineav has joined #tripleo21:16
weshaymwhahaha when you have a sec.. /me curious what think of this https://review.openstack.org/#/c/651379/  removing opendaylight from tc/containers21:20
mwhahahaI think it's deprecated in stein, to be removed in train?21:21
*** wolverineav has quit IRC21:21
mwhahahaweshay: can we not fix the build for stein?21:21
weshayok..  I'll see what's up w/ the build21:21
*** morazi has quit IRC21:22
*** bfournie has quit IRC21:23
*** wolverineav has joined #tripleo21:23
openstackgerritSteve Baker proposed openstack/tripleo-common master: Use ContainerImageRegistryCredentials for registry auth  https://review.openstack.org/65100521:24
*** Goneri has joined #tripleo21:27
*** wolverineav has quit IRC21:27
*** wolverineav has joined #tripleo21:28
*** morazi has joined #tripleo21:28
*** wolverineav has quit IRC21:28
*** wolverin_ has joined #tripleo21:29
*** wolverin_ has quit IRC21:31
*** wolverineav has joined #tripleo21:33
*** wolverineav has quit IRC21:38
*** dprince has quit IRC21:39
openstackgerritPierre Riteau proposed openstack/diskimage-builder master: Increase size of EFI system partition (again)  https://review.openstack.org/64170221:41
*** betherly has joined #tripleo21:44
*** chem` has quit IRC21:45
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Do not archive ceph-ansible fetch directory if it is empty  https://review.openstack.org/65095321:48
*** betherly has quit IRC21:49
*** gfidente|afk has quit IRC21:54
*** aedc has quit IRC21:57
*** wolverineav has joined #tripleo21:58
*** morazi has quit IRC21:59
*** aedc has joined #tripleo22:00
*** vinaykns has quit IRC22:00
*** morazi has joined #tripleo22:00
*** slaweq has quit IRC22:01
*** morazi has quit IRC22:08
*** morazi has joined #tripleo22:09
*** JqckB has quit IRC22:13
*** mcornea has quit IRC22:26
*** rcernin has joined #tripleo22:28
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add OS::TripleO::NovaAZConfig  https://review.openstack.org/64490322:28
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add Etcd to DistributedCompute roles  https://review.openstack.org/64749422:28
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Add CinderEtcdLocalConnect parameter  https://review.openstack.org/64749522:28
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Handle any network name in ServiceNetMap  https://review.openstack.org/65140122:28
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Merge EndpointMapOverride data last  https://review.openstack.org/65140222:28
*** aedc has quit IRC22:32
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Ensure non-pacemaker versions for undercloud  https://review.openstack.org/65140322:33
*** morazi has quit IRC22:35
*** ade_lee has quit IRC22:37
*** morazi has joined #tripleo22:40
*** ansmith_ has joined #tripleo22:43
*** boazel has joined #tripleo22:47
*** thrash has quit IRC22:47
*** bfournie has joined #tripleo22:48
*** boazel_ has quit IRC22:50
*** tkajinam has joined #tripleo22:53
*** thrash has joined #tripleo22:55
*** betherly has joined #tripleo22:57
mwhahahahjensas: i can confirm that patch fixes it. yay22:58
*** betherly has quit IRC23:02
*** raildo has quit IRC23:05
*** rlandy|ruck is now known as rlandy23:06
*** rlandy has quit IRC23:09
*** ade_lee has joined #tripleo23:11
*** devep has joined #tripleo23:13
*** devep has quit IRC23:18
*** betherly has joined #tripleo23:18
*** ade_lee has quit IRC23:22
*** ade_lee has joined #tripleo23:22
*** betherly has quit IRC23:22
*** wolverineav has quit IRC23:31
*** morazi has quit IRC23:32
*** morazi has joined #tripleo23:33
*** wolverineav has joined #tripleo23:33
*** tosky has quit IRC23:36
*** betherly has joined #tripleo23:38
*** wolverineav has quit IRC23:41
*** betherly has quit IRC23:44
*** slaweq has joined #tripleo23:48
*** slaweq has quit IRC23:52
*** morazi has quit IRC23:56
*** betherly has joined #tripleo23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!