Friday, 2019-11-29

*** tosky has quit IRC00:14
*** caowei99 has joined #oooq01:33
*** caowei99_ has joined #oooq01:54
*** caowei99 has quit IRC01:57
*** ysandeep has joined #oooq02:08
*** ysandeep has quit IRC02:33
*** caowei99 has joined #oooq02:38
*** caowei99_ has quit IRC02:40
*** apetrich has quit IRC03:10
*** ysandeep has joined #oooq03:59
raukadahzbr_: donot loose hope04:20
*** bhagyashris has joined #oooq04:23
*** surpatil has joined #oooq04:28
*** epoojad1 has joined #oooq04:32
*** skramaja_ has quit IRC04:50
*** ratailor has joined #oooq04:52
*** skramaja has joined #oooq04:52
*** udesale has joined #oooq04:54
*** caowei99 has quit IRC05:00
*** caowei99 has joined #oooq05:17
*** raukadah is now known as chkumar|rover05:22
*** soniya29 has joined #oooq05:25
*** caowei99 has quit IRC05:37
*** caowei99 has joined #oooq05:38
*** caowei99 has quit IRC05:43
*** bhagyashris has quit IRC05:51
*** jtomasek has joined #oooq05:54
*** udesale has quit IRC06:07
*** bhagyashris has joined #oooq06:09
*** skramaja has quit IRC06:17
*** skramaja has joined #oooq06:17
*** caowei99 has joined #oooq06:37
*** udesale has joined #oooq06:37
*** udesale has quit IRC06:39
*** udesale has joined #oooq06:39
*** udesale has quit IRC06:42
*** marios|ruck has joined #oooq06:48
*** jfrancoa has joined #oooq06:51
*** udesale has joined #oooq06:52
*** bhagyashris_ has joined #oooq06:57
*** bhagyashris has quit IRC06:59
*** kopecmartin|sick is now known as kopecmartin07:00
*** udesale has quit IRC07:13
*** udesale has joined #oooq07:13
*** caowei99 has quit IRC07:16
*** caowei99 has joined #oooq07:28
*** bhagyashris_ has quit IRC07:31
*** caowei99 has quit IRC07:34
*** bhagyashris has joined #oooq07:34
*** caowei99 has joined #oooq07:35
*** skramaja_ has joined #oooq07:39
*** skramaja has quit IRC07:40
*** caowei99_ has joined #oooq07:41
*** caowei99 has quit IRC07:44
*** caowei99_ has quit IRC07:54
*** caowei99_ has joined #oooq07:55
*** ccamacho has quit IRC07:58
*** ccamacho has joined #oooq07:59
*** saneax has joined #oooq08:14
*** caowei99 has joined #oooq08:16
*** caowei99_ has quit IRC08:19
*** tesseract has joined #oooq08:24
*** udesale has quit IRC08:28
*** udesale has joined #oooq08:28
chkumar|rovermarios|ruck: \o/08:29
*** udesale has quit IRC08:29
chkumar|rovermarios|ruck: this review is needed https://review.opendev.org/#/c/696532/3 for ironic python agent build failure for train and master08:29
marios|ruckchkumar|rover: o/08:30
chkumar|rovermarios|ruck: I have rekicked rhel8 master job on rdo-jobs08:30
*** amoralej|off is now known as amoralej08:32
*** apetrich has joined #oooq08:34
marios|ruckchkumar|rover: ack ... looks like master scen1/2 finally passed in periodic (after timeout issue seems finally gone )08:37
arxcruzzbr_: please check https://review.rdoproject.org/r/#/c/23853/ and https://review.rdoproject.org/r/#/c/23852/08:41
zbr_arxcruz: reoplied. just do the same stuff I did on https://review.rdoproject.org/r/#/c/23826/23/zuul.d/projects.yaml08:46
*** tosky has joined #oooq08:46
zbr_so we can see what the job definition you are adding is also working.08:46
zbr_also this makes it easy to update in the future.08:46
zbr_that is beauty of having different job definition files, we can test them.08:47
*** jpena|off is now known as jpena08:49
chkumar|roverzbr_: did you get a chance to try out the new rhui rpm?09:03
chkumar|rovermarios|ruck: you are looking at fs039 master periodic failure?09:05
marios|ruckchkumar|rover: yeah but can't see the error it times out09:06
marios|ruck        * http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-master/07e19d8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz09:06
marios|ruck            * 2019-11-29 04:26:17 |         "puppet-user: error: Could not connect to cluster (is it running?)",09:06
marios|ruck            * 2019-11-29 05:18:00 | Ansible timed out at 5385 seconds.09:06
marios|ruck            * 2019-11-29 05:18:00 | Error: unable to find resource 'ovn-dbs-bundle'09:06
chkumar|roverchecking09:06
marios|ruckchkumar|rover: might try again there it was green yesterday https://review.rdoproject.org/r/#/c/23872/09:06
marios|ruckchkumar|rover: i will post testproject for that and fs1 is also missing (weird one too like http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/eddd2b5/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz * 2019-11-29 03:24:35 | 2019-11-29 03:19:13Z [oveTimed out waiting for port 22 from09:06
marios|ruck192.168.24.1909:06
chkumar|rover ?/swift_ringbuilder/etc/swift/account.ring.gz? -> ?/etc/swift/account.ring.gz? also looks interesting09:07
chkumar|rovermarios|ruck: rechecked the above patch09:08
marios|ruckchkumar|rover: no we need fs1 and fs39 going to abandon it and post new one09:09
chkumar|roverok09:09
chkumar|rovermarios|ruck: we have fs039 error on vexxhost yesterday09:10
chkumar|rover*have seen09:10
chkumar|rovermight be some real issue there09:10
marios|ruckchkumar|rover: so fs39 is not normally in criteria but weshay added in manually last friday if we continue to have problems there we should reconsider it09:11
chkumar|roverI am digging more into the logs to find out more info09:11
marios|ruckchkumar|rover: ack ... looks like maybe another race :/ it is not consistent09:12
*** migi has joined #oooq09:12
*** bogdando has joined #oooq09:15
*** derekh has joined #oooq09:28
*** bhagyashris has quit IRC09:40
zbr_chkumar|rover: yep, i installed in on my lenovo and it did not break anything.09:49
chkumar|roverzbr_: lucky you :-)09:49
chkumar|roverzbr_: can you sed some lines from podman spec file as per my fixes?09:50
chkumar|roveror may be btrfs part will also do the job09:50
chkumar|roverzbr_: please reply to the apevec email so we have got all the deps09:51
zbr_chkumar|rover: can you create a branch where we can put all libpod fixes and work on that one? now we have 3-4 patches and i need all-in-one to validate that they work.09:51
zbr_once we get all done, we can split them and ping them to merge them, one by one.09:51
chkumar|roverzbr_: I know about mine 1 and 2 where are 3 and 4?09:51
zbr_https://github.com/containers/libpod/pull/458909:52
zbr_also got another one merged yesterday09:52
chkumar|roverok09:53
zbr_do you know a cli solution to cherry pick PRs?09:53
chkumar|roverzbr_: nope, but found this https://gist.github.com/ozh/cbce675ba35e0d336cff09:53
chkumar|roverzbr_: you are making the pr or will i do?09:54
zbr_let me try to do it, and I will give you push access to my fork, so we can team-up. sound ok?09:55
chkumar|roverzbr_: ack09:56
*** sanjayu_ has joined #oooq10:00
arxcruzzbr_: done10:11
chkumar|roverzbr_: marios|ruck FYI rdo log server is hitting perf issue, fbo/mhu is looking into that10:11
marios|ruckchkumar|rover: thanks10:12
zbr_chkumar|rover: what do I need to do to disable doc building?10:13
zbr_i cherrypicked your docs change in spec ,but i need to pass the tag somehow10:14
zbr_chkumar|rover: https://github.com/ssbarnea/libpod/invitations10:17
*** soniya29 has quit IRC10:20
zbr_nevermind, found10:21
chkumar|roverzbr_: doc will be merged soon10:22
chkumar|roverzbr_: now both approved10:25
chkumar|roverby daniel walsh10:25
chkumar|roverit will be merged soon10:25
bogdandoola. PTAL https://review.opendev.org/#/c/696056/ https://review.opendev.org/#/c/696091/ https://review.opendev.org/#/c/696292/ https://review.opendev.org/#/c/696306/10:25
bogdandoweshay ^^ :)10:25
zbr_chkumar|rover: there is something weird, I am trying to use the --without option with buildrpm and it does not work with your with_doc var10:26
zbr_what is the correct way to pass it?10:26
zbr_in case it was not obvious, rpmbuild is not on my resume ;)10:28
zbr_chkumar|rover: so  you don't want to tell me the magic trick?10:40
chkumar|roverzbr_: rpmbuild --without doc10:41
chkumar|rovergot the help from jpena10:41
chkumar|roverneed one more fix10:41
chkumar|roverzbr_: proposing one more pr10:45
zbr_sure post link and I will cherry pick10:46
*** soniya29 has joined #oooq10:51
*** udesale has joined #oooq10:53
zbr_arxcruz: as you can see I am trying to trigger all 3 at https://review.rdoproject.org/r/#/c/23852/ but something is not working10:57
zbr_defining branches like this should override the inhered value from the definition and convince zuul to build them too10:58
marios|ruckpanda: ack on the meeting invite but i may be few mins late today (possibility i need to go pickup kids from school half hour before that call.. likely back in time but fyi)11:01
chkumar|roverpanda:  can we do 30 mins later?11:03
chkumar|roverzbr_: https://github.com/containers/libpod/pull/459911:03
*** caowei99 has quit IRC11:03
*** panda has quit IRC11:08
*** panda has joined #oooq11:09
*** skramaja_ has quit IRC11:21
zbr_chkumar|rover: apparently your implementation does not use with/without features correctly,11:21
zbr_is see there conditionals without underlines11:22
*** epoojad1 has quit IRC11:23
*** dsneddon has quit IRC11:30
chkumar|roverjust use export extra_arg='--without doc'11:34
chkumar|roveryou are done11:34
chkumar|roverlet's test this pr using tripleo ci11:34
chkumar|roverzbr_: ^^11:34
zbr_chkumar|rover: this is what I did but regardless what I did it was impossible for me to pass the error: Failed build dependencies:  go-md2man11:35
zbr_which means that the condition was not evaluated correctly.11:35
chkumar|roverzbr_: where is the review?11:37
chkumar|roverkopecmartin: arxcruz soniya29 surpatil Please have a look at the agenda https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view and update it11:38
surpatilok11:39
*** ratailor has quit IRC11:45
*** ratailor has joined #oooq11:47
*** ysandeep has quit IRC11:49
zbr_chkumar|rover:  to test it be sure you remove ms2man from your host, there are several other changes needed to make it work.11:57
zbr_i know we can compile it with go itself, but be sure you remove it.11:58
*** jpena is now known as jpena|lunch12:01
*** rfolco has joined #oooq12:02
*** jbadiapa has joined #oooq12:12
arxcruzzbr_: it will work once get merged, according jpena|lunch12:18
marios|ruckchkumar|rover: the log issue you mentioned for rdo cloud looks like that? "POST_FAILURE" e.g. https://review.rdoproject.org/r/#/c/23882/12:18
chkumar|rovermarios|ruck: I think yes, I have poked them internally12:33
marios|ruckchkumar|rover: yeah sorry it was silly questino it must be that cos there are no logs reported ;)12:33
marios|ruckchkumar|rover: but it basically blocks all the things if the jobs cant report12:34
marios|ruckchkumar|rover: (I mean promotions)12:34
zbr_chkumar|rover: please try test this branch and let me know what you get https://github.com/ssbarnea/libpod/tree/fix/test-build-rpm-changes12:36
*** dtantsur|afk is now known as dtantsur12:43
*** marios|ruck has quit IRC12:47
*** marios|ruck has joined #oooq12:48
*** jpena|lunch is now known as jpena12:57
*** ratailor has quit IRC12:58
arxcruzzbr_: i remove the debug: true13:01
weshayrfolco, please join13:03
rfolcoweshay, joining13:03
marios|ruckchkumar|rover: hmmm i think good news maybe the logging issue does not block promotions13:19
marios|ruckchkumar|rover: e.g. there https://review.rdoproject.org/r/#/c/23873/  POST FAILURE but it looks like it already reported to delorean before that post failure because in http://38.145.34.55/centos7_rocky.log "2019-11-29 11:53:35,357 18461 DEBUG    promoter periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky at 2019-11-29T11:43:57, logs at13:19
marios|ruckhttps://logs.rdoproject.org/73/23873/4/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky/b90090e  in successful jobs13:19
marios|rucklooks like rocky is promoting chkumar|rover13:20
chkumar|roverawesome13:20
marios|ruckmaybe master then too /me checking13:20
*** Goneri has quit IRC13:27
*** Goneri has joined #oooq13:40
chkumar|rovermigi: https://code.engineering.redhat.com/gerrit/#/q/project:openstack/tripleo-ci-internal-jobs13:44
arxcruzchkumar|rover: i know you're rover, but are you going to join us?13:50
chkumar|roverweshay: tempest meeting time13:50
chkumar|roverweshay: you are joining?13:53
chkumar|roverzbr_: done13:54
chkumar|roverzbr_: you are free to test it with work13:54
chkumar|roverexport extra_arg='--without doc'13:54
chkumar|roverthen run contrib/build_rpm.sh13:54
*** amoralej is now known as amoralej|lunch14:20
weshayrfolco, k.. let's chat about services14:21
weshayrfolco, https://meet.google.com/zvw-hnds-ncu?authuser=114:21
*** soniya29 has quit IRC14:21
*** caowei99 has joined #oooq14:26
rfolcoweshay, sorry I have a dentist appt.14:27
weshayrfolco, ok.. dang14:28
rfolcoweshay, we can chat later today if you still on14:28
weshayrfolco, hold off on that work.. until we chat re: the services etc14:28
rfolcoweshay, ack14:28
weshayrfolco, I'll try and get you later :)14:28
rfolcook thx14:28
marios|ruckpanda: chkumar|rover: ? boat party14:30
marios|ruck(or something)14:30
weshaymarios|ruck, doing ok?14:31
marios|ruckweshay: yeah, chasing master/rocky promotions today. rocky is happening hoping master will be next14:32
weshayk.. thanks for rucking14:33
marios|ruckweshay: nothing new today but we have a few ongoing bugs (all in https://etherpad.openstack.org/p/ruckroversprint1914:33
marios|ruckpanda: chkumar|rover: i dropped let me know14:33
*** rfolco has quit IRC14:34
*** surpatil has quit IRC14:34
chkumar|rovermarios|ruck: yes14:36
chkumar|roverpanda: let me know when ready?14:36
marios|ruckchkumar|rover: :/ master container build push timed out and buidimage failed :/ all the things skipped latest run14:41
*** rfolco has joined #oooq14:41
marios|ruckchkumar|rover: still hoping that will give us the promotion though (we have to wait for rocky to finish then hopefully master will promote that hash )14:41
marios|ruckchkumar|rover: 'that' https://review.rdoproject.org/r/#/c/23882/14:42
chkumar|rovermarios|ruck: post_failure got fixed?14:42
marios|ruckchkumar|rover: cos they both reported even though POST_FAILURE14:42
chkumar|roverok14:42
marios|ruckchkumar|rover: no the post failure doesn't make a difference14:42
marios|ruckchkumar|rover: it looks like the job reports before that failed14:42
chkumar|roverok14:43
marios|ruckchkumar|rover: which is why r9ocky is promoting (from rocky promoter logs i saw it took the result from my check patch promoter periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky at 2019-11-29T11:43:57, logs at https://logs.rdoproject.org/73/23873/4/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky/b90090e  in successful jobs14:43
*** rfolco has quit IRC14:46
*** amoralej|lunch is now known as amoralej14:46
*** caowei99 has quit IRC14:48
*** caowei99 has joined #oooq14:48
pandamarios|ruck: chkumar|rover you seem quite busy.14:50
pandamarios|ruck: chkumar|rover but I'm there if yuo want to join14:50
pandamarios|ruck: chkumar|rover otherwise we'll have one meeting each day for the next week.14:51
marios|ruckpanda: o/ i joined at the scheduled time (as is customary i find) but i was alone14:52
marios|ruckpanda: i can join now mostly to hear about the boat i am intrigued14:52
marios|ruckpanda: ?14:54
*** caowei99 has quit IRC14:54
chkumar|roverpanda: marios|ruck sorry I will not make it today, need some rest14:55
chkumar|roverHappy weekend, Happy thanks giving14:55
chkumar|roversee ya14:55
*** chkumar|rover is now known as raukadah14:55
marios|ruck2019-11-29 11:51:34,595 18305 INFO     promoter Skipping promotion of {'timestamp': 1574986351, 'distro_hash': 'd5fe1de9877b138a0c2f619f8669aee324210e45', 'promote_name': 'tripleo-ci-testing', 'user': 'review_rdoproject_org', 'repo_url': 'https://trunk.rdoproject.org/centos7/3c/80/3c80d51b92b8d37a3994a0f921564f7ad62480a4_d5fe1de9', 'full_hash': '3c80d51b92b8d37a3994a0f921564f7ad62480a4_d5fe1de9',14:58
marios|ruck'repo_hash': '3c80d51b92b8d37a3994a0f921564f7ad62480a4_d5fe1de9', 'commit_hash': '3c80d51b92b8d37a3994a0f921564f7ad62480a4'} from tripleo-ci-testing to current-tripleo, missing successful jobs: [u'periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-master', u'periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master']14:58
marios|ruckpanda: that one because https://review.rdoproject.org/r/#/c/23882/14:58
*** sanjayu_ has quit IRC15:08
*** saneax has quit IRC15:08
marios|ruckack raukadah15:09
marios|ruckhave a good one15:09
marios|ruckhttps://bugs.launchpad.net/tripleo/+bug/185365215:10
openstackLaunchpad bug 1853652 in tripleo "openstack overcloud node provide --all-manageable timing out and failing periodic rhel-8-ovb-3ctlr_1comp-featureset001-master" [Critical,Triaged]15:10
marios|ruckraukadah: rocky promoted finally finished \o/15:27
marios|ruckraukadah: fyi i filed a bug in the end because it does block promotions... train containers-push failed on that so it blocks the whole pipeline https://bugs.launchpad.net/tripleo/+bug/185451715:49
openstackLaunchpad bug 1854517 in tripleo "RDO jobs failing with POST_FAILURE" [Critical,Triaged]15:49
marios|ruckweshay: fyi ^15:49
*** udesale has quit IRC15:49
*** rfolco has joined #oooq15:52
raukadahmarios|ruck: I fixed that I think15:54
marios|ruckraukadah: which one the POST_FAILURE?15:54
raukadahrhel-8 container build15:55
raukadahthere was a merge conflict in the patch15:55
marios|ruckraukadah: man master also failed looks like there are issues with rdo registry? like it times out pushing to images http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/178a569/logs/build.log.txt.gz15:55
marios|ruckraukadah: no rhel8 container build failed on post-failure no merge conflict?15:55
marios|ruckraukadah: so even with your merge conflict fix it still fails15:56
*** bogdando has quit IRC15:56
marios|ruckraukadah: with post_failure and then the rest of the pipeline skips :/15:56
raukadahok15:56
raukadahjpena: can you take a look at this http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/178a569/logs/build.log.txt.gz15:56
raukadahjpena: there is some issue with rdo registry while pushing containers15:56
*** rfolco has quit IRC15:56
jpenaraukadah: the ceph cluster it's running on is currently having I/O issues (some disks failed and it is rebuilding)15:57
raukadahjpena: ok15:57
raukadahjpena: and reagrding rdo log server perf issue?15:57
raukadahwe have got hitten by post failures15:57
jpenaraukadah: same thing, it's running on the same ceph15:57
raukadahjpena: ah ok15:58
raukadahjpena: thanks!15:58
marios|ruckjpena: thanks (FYI filed that as we need to track it as blocking promotions https://bugs.launchpad.net/tripleo/+bug/185451715:58
openstackLaunchpad bug 1854517 in tripleo "RDO jobs failing with POST_FAILURE" [Critical,Triaged]15:58
raukadahweshay: marios|ruck is ok if we move rhel nodepool image from 8.0 to 8.1 next monday?16:01
marios|ruckraukadah: ack16:02
marios|ruckraukadah: wondering if we want to wait for a rhel promotion first though we are long overdue16:02
raukadahmarios|ruck: ok16:02
marios|ruckraukadah: but shouldn't affect I mean if it does we can revert anyway16:02
marios|ruckraukadah: lets talk monday in any case16:02
raukadahokok16:03
*** marios|ruck is now known as marios|out16:36
*** marios|out has quit IRC16:46
*** jtomasek has quit IRC16:48
*** jtomasek has joined #oooq16:54
*** jtomasek has quit IRC16:56
*** dtantsur is now known as dtantsur|afk17:04
*** brault has joined #oooq17:37
*** tosky has quit IRC17:50
*** jpena is now known as jpena|off17:57
*** amoralej is now known as amoralej|off17:59
*** derekh has quit IRC18:59
*** brault has quit IRC19:00
*** brault has joined #oooq19:26
*** brault has quit IRC19:26
*** brault has joined #oooq19:27
*** tosky has joined #oooq19:36
*** tesseract has quit IRC20:00
*** jfrancoa has quit IRC20:03
*** brault has quit IRC20:03
*** Goneri has quit IRC20:13
*** jtomasek has joined #oooq20:35
arxcruzweshay_: weshay Folco had an issue and he'll be afk21:00
*** jbadiapa has quit IRC21:03
*** jtomasek_ has joined #oooq21:29
*** jtomasek has quit IRC21:32

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!