| *** mary_grace has quit IRC | 00:01 | |
| rdogerrit | rdo-trunk proposed rdoinfo master: Bump rdoinfo queens-uc tags to latest upper-constraints https://review.rdoproject.org/r/10368 | 00:04 |
|---|---|---|
| rdogerrit | rdo-trunk created rdoinfo master: Promote CBS tags update for newton-release https://review.rdoproject.org/r/10377 | 00:04 |
| rdogerrit | rdo-trunk created rdoinfo master: Promote CBS tags update for ocata-release https://review.rdoproject.org/r/10378 | 00:04 |
| rdogerrit | rdo-trunk created rdoinfo master: Promote CBS tags update for pike-release https://review.rdoproject.org/r/10379 | 00:05 |
| *** dmacpher has joined #rdo | 00:09 | |
| *** itamarjp has quit IRC | 00:12 | |
| *** sasha22 has quit IRC | 00:13 | |
| *** sasha21 has joined #rdo | 00:15 | |
| *** rhallisey has quit IRC | 00:19 | |
| *** rhallisey has joined #rdo | 00:20 | |
| *** Son_Goku has quit IRC | 00:27 | |
| *** sasha21 has quit IRC | 00:31 | |
| *** rhallisey has quit IRC | 00:32 | |
| *** kellyp has joined #rdo | 00:49 | |
| *** mary_grace has joined #rdo | 00:51 | |
| *** itamarjp has joined #rdo | 00:53 | |
| *** kellyp has quit IRC | 00:54 | |
| *** julim has joined #rdo | 00:56 | |
| *** rwsu has joined #rdo | 01:00 | |
| *** brault has joined #rdo | 01:02 | |
| *** rwsu_ has joined #rdo | 01:03 | |
| *** brault has quit IRC | 01:06 | |
| *** mary_grace has quit IRC | 01:15 | |
| *** fragatina has quit IRC | 01:15 | |
| *** fragatina has joined #rdo | 01:16 | |
| *** fragatin_ has joined #rdo | 01:17 | |
| *** rlandy|afk is now known as rlandy | 01:17 | |
| *** fragatina has quit IRC | 01:20 | |
| *** fragatin_ has quit IRC | 01:21 | |
| *** gildub has quit IRC | 01:27 | |
| *** liujiong_lj has joined #rdo | 01:32 | |
| *** kellyp has joined #rdo | 01:32 | |
| rdogerrit | Emilien Macchi created openstack/ceilometer-distgit pike-rdo: Add python-tooz to requirements for common package https://review.rdoproject.org/r/10380 | 01:32 |
| EmilienM | mnaser: ^ | 01:33 |
| mnaser | EmilienM: thanks, guess we'll have to get that in and get a promotion | 01:33 |
| *** kellyp has quit IRC | 01:37 | |
| *** gildub has joined #rdo | 01:39 | |
| *** liujiong_lj is now known as liujiong | 01:43 | |
| *** suresh12 has joined #rdo | 01:49 | |
| *** vnogin has quit IRC | 01:53 | |
| *** suresh12 has quit IRC | 01:53 | |
| *** TikTok has quit IRC | 01:55 | |
| *** TikTok has joined #rdo | 01:57 | |
| *** vnogin has joined #rdo | 02:01 | |
| *** artom has joined #rdo | 02:02 | |
| *** vnogin has quit IRC | 02:03 | |
| *** rlandy has quit IRC | 02:11 | |
| *** vnogin has joined #rdo | 02:13 | |
| *** Son_Goku has joined #rdo | 02:26 | |
| *** fragatina has joined #rdo | 02:26 | |
| *** ykarel|afk has joined #rdo | 02:31 | |
| *** fragatina has quit IRC | 02:31 | |
| *** noslzzp has joined #rdo | 02:39 | |
| *** ykarel|afk has quit IRC | 02:42 | |
| *** TikTok has quit IRC | 02:57 | |
| *** TikTok has joined #rdo | 02:59 | |
| *** artom has quit IRC | 03:00 | |
| *** paragan has joined #rdo | 03:04 | |
| *** rwsu_ has quit IRC | 03:06 | |
| *** rwsu has quit IRC | 03:06 | |
| *** TikTok is now known as michealb | 03:10 | |
| *** udesale has joined #rdo | 03:16 | |
| *** anilvenkata has joined #rdo | 03:25 | |
| *** threestrands has quit IRC | 03:29 | |
| *** vnogin has quit IRC | 03:33 | |
| *** vnogin has joined #rdo | 03:34 | |
| *** vnogin has quit IRC | 03:38 | |
| *** jistr has quit IRC | 03:42 | |
| *** jistr has joined #rdo | 03:43 | |
| *** jhershbe has joined #rdo | 03:50 | |
| *** shreshtha has joined #rdo | 03:55 | |
| *** shreshtha has joined #rdo | 03:56 | |
| *** suresh12 has joined #rdo | 03:59 | |
| *** jhershbe has quit IRC | 04:02 | |
| *** imcleod has quit IRC | 04:06 | |
| *** coolsvap has joined #rdo | 04:06 | |
| *** suresh12 has quit IRC | 04:08 | |
| *** rwsu_ has joined #rdo | 04:13 | |
| *** rwsu has joined #rdo | 04:13 | |
| *** mary_grace has joined #rdo | 04:17 | |
| *** suresh12 has joined #rdo | 04:19 | |
| *** imcleod has joined #rdo | 04:20 | |
| *** gkadam_ has joined #rdo | 04:21 | |
| *** gkadam_ has quit IRC | 04:22 | |
| *** gkadam_ has joined #rdo | 04:22 | |
| *** gkadam_ is now known as gkadam | 04:23 | |
| *** michealb has quit IRC | 04:23 | |
| *** TikTok has joined #rdo | 04:33 | |
| *** paragan has quit IRC | 04:34 | |
| *** mary_grace has quit IRC | 04:36 | |
| *** imcleod has quit IRC | 04:36 | |
| *** Son_Goku has quit IRC | 04:42 | |
| *** gvrangan_ has joined #rdo | 04:43 | |
| *** suresh12_ has joined #rdo | 04:45 | |
| *** suresh12 has quit IRC | 04:45 | |
| *** ykarel has joined #rdo | 04:48 | |
| *** paragan has joined #rdo | 04:50 | |
| *** gvrangan_ has quit IRC | 04:55 | |
| *** paragan has quit IRC | 05:06 | |
| *** suresh12_ has quit IRC | 05:14 | |
| EmilienM | mnaser: I'll self approve | 05:15 |
| *** paragan has joined #rdo | 05:21 | |
| *** abregman has joined #rdo | 05:21 | |
| *** jatanmalde has joined #rdo | 05:24 | |
| *** jatanmalde has joined #rdo | 05:25 | |
| *** jatanmalde has joined #rdo | 05:26 | |
| *** jatanmalde has joined #rdo | 05:26 | |
| *** janki has joined #rdo | 05:27 | |
| *** fragatina has joined #rdo | 05:28 | |
| *** gildub has quit IRC | 05:29 | |
| *** jatanmalde has joined #rdo | 05:30 | |
| *** jatanmalde has quit IRC | 05:30 | |
| *** jatanmalde has joined #rdo | 05:31 | |
| *** TikTok has quit IRC | 05:32 | |
| *** kellyp has joined #rdo | 05:33 | |
| *** fragatina has quit IRC | 05:33 | |
| *** TikTok has joined #rdo | 05:34 | |
| *** tshefi has joined #rdo | 05:36 | |
| *** kellyp has quit IRC | 05:37 | |
| *** anshul has joined #rdo | 05:41 | |
| *** ratailor has joined #rdo | 05:41 | |
| *** kellyp has joined #rdo | 05:44 | |
| *** gildub has joined #rdo | 05:46 | |
| *** kellyp has quit IRC | 05:48 | |
| *** cshastri has joined #rdo | 05:50 | |
| *** jbadiapa has joined #rdo | 05:52 | |
| *** rwsu_ has quit IRC | 05:55 | |
| *** rwsu has quit IRC | 05:55 | |
| *** rwsu has joined #rdo | 05:56 | |
| *** sbonazzo has joined #rdo | 05:56 | |
| *** pcaruana has joined #rdo | 05:58 | |
| *** lkuchlan has joined #rdo | 06:02 | |
| *** lkuchlan has quit IRC | 06:07 | |
| *** Amita has quit IRC | 06:07 | |
| *** lkuchlan has joined #rdo | 06:07 | |
| *** emalikov has joined #rdo | 06:08 | |
| *** itamarjp has quit IRC | 06:22 | |
| *** itamarjp has joined #rdo | 06:23 | |
| *** fragatina has joined #rdo | 06:30 | |
| *** Amita has joined #rdo | 06:30 | |
| *** Amita has quit IRC | 06:31 | |
| *** Amita has joined #rdo | 06:31 | |
| *** TikTok has quit IRC | 06:33 | |
| *** fragatina has quit IRC | 06:34 | |
| *** vnogin has joined #rdo | 06:37 | |
| *** TikTok has joined #rdo | 06:39 | |
| *** ratailor is now known as ratailor|afk | 06:40 | |
| *** sbulage has joined #rdo | 06:41 | |
| *** vnogin has quit IRC | 06:41 | |
| *** ratailor|afk is now known as ratailor | 06:45 | |
| *** julim has quit IRC | 06:55 | |
| *** jhershbe has joined #rdo | 06:55 | |
| *** julim has joined #rdo | 06:56 | |
| *** jhershbe has quit IRC | 06:59 | |
| *** vnogin has joined #rdo | 07:01 | |
| *** iranzo has joined #rdo | 07:04 | |
| *** iranzo has quit IRC | 07:04 | |
| *** iranzo has joined #rdo | 07:04 | |
| *** vnogin has quit IRC | 07:05 | |
| rdogerrit | Matthias Runge created openstack/collectd-gnocchi-distgit rpm-master: initial import into RDO https://review.rdoproject.org/r/10381 | 07:06 |
| rdogerrit | Matthias Runge proposed openstack/collectd-gnocchi-distgit rpm-master: initial import into RDO https://review.rdoproject.org/r/10381 | 07:07 |
| *** iranzo has quit IRC | 07:09 | |
| *** TikTok has quit IRC | 07:13 | |
| *** jhershbe has joined #rdo | 07:14 | |
| *** liujiong has quit IRC | 07:15 | |
| *** gildub has quit IRC | 07:15 | |
| *** liujiong has joined #rdo | 07:16 | |
| *** egonzalez has joined #rdo | 07:18 | |
| rdogerrit | Matthias Runge proposed openstack/collectd-gnocchi-distgit rpm-master: initial import into RDO https://review.rdoproject.org/r/10381 | 07:19 |
| *** Caterpillar has joined #rdo | 07:19 | |
| *** jprovazn has joined #rdo | 07:19 | |
| rdogerrit | Matthias Runge proposed openstack/collectd-gnocchi-distgit rpm-master: initial import into RDO https://review.rdoproject.org/r/10381 | 07:25 |
| *** dmacpher has quit IRC | 07:26 | |
| *** agurenko has joined #rdo | 07:27 | |
| *** rcernin has quit IRC | 07:27 | |
| *** vnogin has joined #rdo | 07:31 | |
| *** smeyer has joined #rdo | 07:34 | |
| *** rdopiera has quit IRC | 07:35 | |
| *** brault has joined #rdo | 07:37 | |
| *** vnogin has quit IRC | 07:38 | |
| *** martinkopec has joined #rdo | 07:40 | |
| *** lebauce has joined #rdo | 07:44 | |
| *** tshefi has quit IRC | 07:47 | |
| *** vnogin has joined #rdo | 07:49 | |
| *** kellyp has joined #rdo | 07:51 | |
| *** aufi has joined #rdo | 07:51 | |
| *** rbartal__ has joined #rdo | 07:53 | |
| *** rbartal_ has joined #rdo | 07:54 | |
| *** rbartal has joined #rdo | 07:54 | |
| *** kellyp has quit IRC | 07:55 | |
| *** vnogin has quit IRC | 07:55 | |
| *** jackNemrod has joined #rdo | 07:56 | |
| rdogerrit | Merged openstack/congressclient-distgit rpm-master: Fix python3 build https://review.rdoproject.org/r/10363 | 08:02 |
| *** vnogin has joined #rdo | 08:04 | |
| rdogerrit | Merged openstack/collectd-gnocchi-distgit rpm-master: initial import into RDO https://review.rdoproject.org/r/10381 | 08:05 |
| *** vnogin has quit IRC | 08:08 | |
| *** racedo_ has quit IRC | 08:09 | |
| *** tesseract has joined #rdo | 08:10 | |
| *** shardy has joined #rdo | 08:16 | |
| *** apevec has joined #rdo | 08:18 | |
| *** apevec has quit IRC | 08:18 | |
| *** apevec has joined #rdo | 08:18 | |
| *** oidgar has joined #rdo | 08:21 | |
| apevec | https://www.rdoproject.org/testday/ was not updated? | 08:21 |
| apevec | that was conclusion I saw in meeting minutes from yesterday... | 08:21 |
| apevec | ykarel, ^ | 08:21 |
| ykarel | rbowen, has the action for this | 08:22 |
| ykarel | may be he missed | 08:22 |
| apevec | hmm, I don't see PR | 08:22 |
| *** MrAbaddon has quit IRC | 08:22 | |
| apevec | ok, I'll edit it on github directly | 08:22 |
| ykarel | apevec, Ok | 08:23 |
| apevec | so dates are now Nov 9/10 | 08:23 |
| *** Tengu has left #rdo | 08:23 | |
| ykarel | rbowen, told that due to summit 15 would be good | 08:23 |
| ykarel | but we can make it to 9/10 | 08:23 |
| apevec | ah Summit is next week | 08:24 |
| apevec | well, Summit is 6-8. | 08:25 |
| *** _bsurfer has joined #rdo | 08:25 | |
| apevec | but yeah, better move to 15 to avoid moving it again | 08:25 |
| *** rcernin has joined #rdo | 08:26 | |
| *** vnogin has joined #rdo | 08:26 | |
| apevec | so Nov 16,17 it is | 08:27 |
| *** oanson has quit IRC | 08:30 | |
| *** paramite has joined #rdo | 08:30 | |
| *** vnogin has quit IRC | 08:30 | |
| *** oanson has joined #rdo | 08:31 | |
| *** fragatina has joined #rdo | 08:31 | |
| *** jpena|off is now known as jpena | 08:33 | |
| *** gfidente has joined #rdo | 08:33 | |
| *** gfidente has quit IRC | 08:33 | |
| *** gfidente has joined #rdo | 08:33 | |
| jpena | good morning | 08:34 |
| *** gkadam has quit IRC | 08:34 | |
| *** gkadam has joined #rdo | 08:35 | |
| *** tshefi has joined #rdo | 08:35 | |
| rdogerrit | Javier Peña proposed rdoinfo master: Pin branchless projects for stable releases https://review.rdoproject.org/r/10328 | 08:36 |
| *** fragatina has quit IRC | 08:36 | |
| *** spectr has joined #rdo | 08:36 | |
| *** ykarel is now known as ykarel|lunch | 08:37 | |
| apevec | moin | 08:37 |
| *** iranzo has joined #rdo | 08:38 | |
| rdogerrit | Merged rdo-infra/puppet-dlrn master: Improve permissions for DLRN configuration files https://review.rdoproject.org/r/10373 | 08:40 |
| panda|ruck|off | jpena: we have a problem with dlrnapi client | 08:42 |
| jpena | panda|ruck|off: what is it? | 08:43 |
| panda|ruck|off | jpena: when specifying the password with env variable we get an error | 08:43 |
| jpena | panda|ruck|off: link? | 08:43 |
| panda|ruck|off | Traceback (most recent call last): | 08:44 |
| panda|ruck|off | File "/home/centos/promoter_venv/bin/dlrnapi", line 10, in <module> | 08:44 |
| panda|ruck|off | sys.exit(main()) | 08:44 |
| panda|ruck|off | File "/home/centos/promoter_venv/lib/python2.7/site-packages/dlrnapi_client/shell.py", line 295, in main | 08:44 |
| panda|ruck|off | raise e | 08:44 |
| panda|ruck|off | TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' | 08:44 |
| *** iranzo has quit IRC | 08:44 | |
| *** oanson has quit IRC | 08:45 | |
| *** cylopez has joined #rdo | 08:46 | |
| *** oanson has joined #rdo | 08:46 | |
| *** ushkalim has joined #rdo | 08:50 | |
| number80 | btw, I'm updating dateutil to 2.6.1 | 08:53 |
| number80 | follow-up to https://review.rdoproject.org/r/#/c/10380/ | 08:53 |
| number80 | (queens only) | 08:53 |
| *** hseipp has joined #rdo | 08:54 | |
| rdogerrit | Merged puppet/puppet-sensu-distgit pike-rdo: convert to real release tarball instead of git-tarball https://review.rdoproject.org/r/10374 | 08:56 |
| *** lucas-afk is now known as lucasagomes | 08:59 | |
| *** ykarel|lunch is now known as ykarel | 09:01 | |
| *** jpich has joined #rdo | 09:02 | |
| *** mcornea has joined #rdo | 09:03 | |
| *** dtantsur|afk is now known as dtantsur | 09:04 | |
| *** amoralej|off is now known as amoralej | 09:04 | |
| *** egonzalez has quit IRC | 09:11 | |
| amoralej | apevec, wrt croniter update, what is needing it? | 09:11 |
| amoralej | it's breaking builds wich uses common-candidate | 09:12 |
| amoralej | https://review.rdoproject.org/jenkins/job/pike-distgit-cbs-validate/817/console | 09:12 |
| amoralej | number80, i'll untag croniter update from common-candidate | 09:12 |
| amoralej | once we have new dateutil in queens we can re-tag it to queen | 09:12 |
| *** jaosorior has quit IRC | 09:15 | |
| amoralej | apevec, number80, python-croniter-0.3.19-1.el7 is untagged | 09:16 |
| number80 | amoralej: it's building | 09:16 |
| *** sshnaidm|afk is now known as sshnaidm | 09:16 | |
| *** ktibi has joined #rdo | 09:16 | |
| number80 | well, actually it's built | 09:17 |
| amoralej | number80, dateutil? | 09:17 |
| number80 | yeah | 09:17 |
| *** ktibi has left #rdo | 09:17 | |
| number80 | I updated it to 2.6.1 (version in u-c) in fedora first, then in CBS | 09:17 |
| number80 | ah, scratch completed | 09:17 |
| number80 | but final is coming right now | 09:18 |
| amoralej | ok | 09:18 |
| amoralej | number80, do you know if something requires croniter update? | 09:19 |
| *** liujiong has quit IRC | 09:19 | |
| amoralej | or it's just to sync with u-c? | 09:19 |
| *** liujiong has joined #rdo | 09:19 | |
| *** egonzalez has joined #rdo | 09:20 | |
| *** vnogin has joined #rdo | 09:22 | |
| *** number80 has quit IRC | 09:23 | |
| *** vnogin has quit IRC | 09:26 | |
| apevec | amoralej, u-c sync croniter===0.3.19 | 09:27 |
| rdogerrit | Javier Peña created openstack/tripleo-common-distgit rpm-master: Change the dependency from /bin/hostname to /usr/bin/hostname https://review.rdoproject.org/r/10382 | 09:27 |
| apevec | I had request to update it in epele so I pushed to cbs too | 09:27 |
| apevec | I'll look at it why is it breaking | 09:27 |
| amoralej | apevec, it has a requires in python2-dateutil | 09:28 |
| amoralej | and last version is python-dateutil | 09:28 |
| rdogerrit | hguemar created rdoinfo master: Update dateutil/croniter for queens https://review.rdoproject.org/r/10383 | 09:30 |
| apevec | damn again python2-* | 09:30 |
| *** paragan has quit IRC | 09:31 | |
| apevec | ok let's move them both to queens-candidate | 09:32 |
| *** number80 has joined #rdo | 09:33 | |
| number80 | damn, lost IP at the worst timing ever | 09:33 |
| amoralej | number80, i'm changing cronniter and dateutil to queens-candidate | 09:34 |
| amoralej | instead of common | 09:34 |
| number80 | amoralej: they're already tagged queens-candidate | 09:34 |
| amoralej | i saw them in common... | 09:34 |
| amoralej | maybe i missread it | 09:34 |
| number80 | dateutil is still in common-candidate | 09:34 |
| number80 | but you already removed croniter from common | 09:34 |
| apevec | I've moved it | 09:34 |
| number80 | ah | 09:35 |
| amoralej | ok, looks fine now | 09:35 |
| amoralej | going for coffee | 09:35 |
| amoralej | apevec, we lost our window to promote | 09:35 |
| apevec | alright, now we just need that master PROMOTION | 09:35 |
| amoralej | now we have new blockers | 09:35 |
| apevec | amoralej, heh, which ones? | 09:35 |
| amoralej | https://bugs.launchpad.net/tripleo/+bug/1729479 | 09:36 |
| openstack | Launchpad bug 1729479 in tripleo "puppet-collectd change on datatype" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 09:36 |
| amoralej | and there is a tempest issue | 09:36 |
| amoralej | i just started checking | 09:36 |
| * number80 gone for daily run not to miss today's calls/marathon of the afternoon | 09:36 | |
| amoralej | i'm frustrated with promotions | 09:36 |
| apevec | tell me about it | 09:37 |
| apevec | so with puppet-collectd, could we not just pin it? | 09:37 |
| apevec | I mean, we have a time machine! | 09:37 |
| amoralej | there was a better option that yesterday's promotion revert, IMO | 09:37 |
| apevec | yeah, I didn't quite understand why the revert, we could have just re-run jobs which picked different hash! | 09:38 |
| apevec | panda|ruck|off, ^ | 09:38 |
| apevec | IIUC that was the issue, | 09:38 |
| apevec | not all jobs run with the same hash? | 09:38 |
| amoralej | in fact we had a commit with everything ok to be promoted | 09:39 |
| amoralej | apevec, yes, that was the issue | 09:39 |
| amoralej | we had two pipelines overlapping | 09:39 |
| apevec | so why wasn't it promoted?! | 09:39 |
| apevec | which commit is good enough? | 09:39 |
| apevec | amoralej, and can you document it via dlrn api? | 09:39 |
| amoralej | and some of the jobs in the first promotion reported to the wrong hash | 09:39 |
| apevec | so we can see which jobs did not run with that proposed promotion? | 09:40 |
| amoralej | but were executed with the first one | 09:40 |
| amoralej | apevec, so | 09:40 |
| apevec | hmm, how did that happen? | 09:40 |
| amoralej | in the first run, the last four jobs (containers ones) started after the second pipeline had changed the hash to be tested | 09:40 |
| amoralej | so reported to the second hash | 09:41 |
| amoralej | but | 09:41 |
| amoralej | efectively they were using the containers created with the first commit | 09:41 |
| amoralej | and in fact | 09:41 |
| apevec | ah! | 09:41 |
| amoralej | the only difference between the two hashes was a single commit in tripleo-ui | 09:41 |
| apevec | wth | 09:41 |
| amoralej | so the first commit should be fine to be promoted | 09:41 |
| amoralej | it had containers etc... | 09:42 |
| apevec | panda|ruck|off, weshay ^ based on that let's discuss manual promotion, lemme know if you want call to discuss | 09:42 |
| amoralej | and the fact that containers-build job is so unreliable, it failed with second hash | 09:42 |
| apevec | are those containers still in the registry or they were GCed already? | 09:42 |
| amoralej | let me check | 09:43 |
| apevec | amoralej, unreliable is this localhost push issue? https://bugs.launchpad.net/tripleo/+bug/1729328 | 09:43 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:43 |
| amoralej | good hash was 3b718f3fecc866332ec0663fa77e758f8346ab93 btw | 09:43 |
| apevec | (which is weird, I mean _timeout_ on a localhost...) | 09:43 |
| amoralej | yeah | 09:44 |
| apevec | that just mean docker is crap imho :) | 09:44 |
| amoralej | containers are still in the registry, i'd say | 09:44 |
| amoralej | i need coffee, brb | 09:44 |
| * apevec too | 09:44 | |
| *** links has quit IRC | 09:44 | |
| *** vnogin has joined #rdo | 09:46 | |
| *** racedo has joined #rdo | 09:46 | |
| *** pgadiya has joined #rdo | 09:49 | |
| *** egallen has joined #rdo | 09:54 | |
| *** panda|ruck|off is now known as panda|ruck | 09:54 | |
| panda|ruck | amoralej: apevec we can discuss manual promotion now, but with 3 jobs missing from the pipeline, my priority tuesday evening was reverting to the last hash. | 09:56 |
| *** Amita has quit IRC | 10:01 | |
| *** Amita has joined #rdo | 10:02 | |
| *** Son_Goku has joined #rdo | 10:12 | |
| *** egallen has quit IRC | 10:14 | |
| *** BarBQ has joined #rdo | 10:16 | |
| *** anilvenkata has quit IRC | 10:16 | |
| *** liujiong has quit IRC | 10:19 | |
| *** derekh has joined #rdo | 10:20 | |
| *** mvk has quit IRC | 10:22 | |
| *** dpawlik has quit IRC | 10:24 | |
| *** Son_Goku has quit IRC | 10:25 | |
| rdogerrit | Merged openstack/ceilometer-distgit pike-rdo: Add python-tooz to requirements for common package https://review.rdoproject.org/r/10380 | 10:30 |
| *** mvk has joined #rdo | 10:30 | |
| *** dpawlik has joined #rdo | 10:32 | |
| *** BarBQ has quit IRC | 10:33 | |
| *** fragatina has joined #rdo | 10:34 | |
| *** iranzo has joined #rdo | 10:34 | |
| *** paragan has joined #rdo | 10:36 | |
| *** iranzo has quit IRC | 10:36 | |
| *** gszasz has joined #rdo | 10:38 | |
| *** fragatina has quit IRC | 10:38 | |
| amoralej | panda|ruck, apevec we can't promote hash 3b718f3fecc866332ec0663fa77e758f8346ab93, images were deleted in images.r.o | 10:39 |
| amoralej | :( | 10:39 |
| *** MrAbaddon has joined #rdo | 10:54 | |
| *** gildub has joined #rdo | 10:56 | |
| rdogerrit | Javier Peña proposed openstack/tripleo-common-distgit rpm-master: Change the dependency from /bin/hostname to hostname https://review.rdoproject.org/r/10382 | 10:56 |
| apevec | amoralej, was it by the automatic purge? | 10:58 |
| apevec | so we are stuck on https://bugs.launchpad.net/tripleo/+bug/1729328 ... | 10:59 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:59 |
| amoralej | apevec, yes, once a hash is promoted | 10:59 |
| amoralej | the rest are deleted | 10:59 |
| amoralej | apevec, other blocker i just reported https://bugs.launchpad.net/tripleo/+bug/1729569 | 11:00 |
| openstack | Launchpad bug 1729569 in tripleo "Periodic jobs using swift as glance backend failing" [Undecided,New] | 11:00 |
| *** nstrug has joined #rdo | 11:00 | |
| amoralej | apevec, we could try to use known good repo in next periodic, instead of consistent, wdyt? | 11:01 |
| rdogerrit | Aditya Ramteke proposed openstack/oslo-db-distgit rpm-master: Added with_doc conditional over -doc subpackage https://review.rdoproject.org/r/9399 | 11:02 |
| amoralej | ykarel, yes it seems the same one as in p-o-i | 11:03 |
| amoralej | ykarel, do you think it's swift or puppet-swift issue? | 11:05 |
| amoralej | could you debug it? | 11:05 |
| *** hewbrocca_afk is now known as hewbrocca | 11:12 | |
| *** fultonj has quit IRC | 11:12 | |
| *** Son_Goku has joined #rdo | 11:17 | |
| apevec | amoralej, which one is "known good" ? That's what periodic is supposed to find out :) | 11:36 |
| amoralej | zaneb, :) | 11:37 |
| amoralej | zaneb, oops, was not for you | 11:37 |
| apevec | but yeah I think we could force that by setting tripleo-testing symlink manually ? | 11:37 |
| apevec | which hash did you have in mind? | 11:37 |
| amoralej | the one that was reverted | 11:38 |
| apevec | iirc first job sets that symlink | 11:38 |
| amoralej | yes | 11:38 |
| amoralej | we'd have to trick that | 11:38 |
| apevec | (which is btw not what we wanted with dlrnapi, it's single global lock...but separate discussion) | 11:38 |
| panda|ruck | wait | 11:38 |
| amoralej | and use the desired hash instead of consistent | 11:38 |
| apevec | panda|ruck, we are waiting >50 days :) | 11:38 |
| panda|ruck | we can just run the promoter script with an hardoced hash | 11:39 |
| panda|ruck | no need to trick | 11:39 |
| panda|ruck | if we decided we want to do it | 11:39 |
| apevec | panda|ruck, but we don't have container images | 11:39 |
| panda|ruck | but we have to be sure we have the images | 11:39 |
| apevec | we don't | 11:39 |
| apevec | we need to re-run the whole thing | 11:39 |
| panda|ruck | the container build job will fail anyway | 11:39 |
| amoralej | panda|ruck, we need to re-run to build containers and images | 11:40 |
| apevec | could we hack that 1st job to run on specific hash vs "consistent" ? | 11:40 |
| amoralej | panda|ruck, container build is passing sometimes, right? | 11:40 |
| amoralej | or failing every single time? | 11:40 |
| amoralej | apevec, yes we could patching config or ci-config iirc | 11:40 |
| panda|ruck | amoralej: lately it failed consistently, yestarday one of the slaves wen down also, I have to check what happened this morning | 11:41 |
| ykarel | amoralej, yes i reproduced it | 11:41 |
| panda|ruck | to rerun the job, we'll have to change the part that sets tripleo-ci-testing, and stop the periodic pipeline | 11:41 |
| panda|ruck | then rerun rerun rerun until the container job passes | 11:41 |
| amoralej | ykarel, do you have a local reproducer to give access to the swift team? | 11:41 |
| ykarel | amoralej, yes | 11:42 |
| *** jaosorior has joined #rdo | 11:42 | |
| ykarel | share the public keys, i will give access | 11:42 |
| panda|ruck | it passed just once in the last two days | 11:43 |
| *** choirboy has quit IRC | 11:43 | |
| *** anilvenkata has joined #rdo | 11:43 | |
| amoralej | panda|ruck, apevec we have containers for 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 | 11:43 |
| amoralej | if we re-run jobs for that to get tripleo images uploaded for it | 11:44 |
| amoralej | we could promote it | 11:44 |
| amoralej | it's sane repo | 11:44 |
| panda|ruck | I'll have to open another bug for rdocloud | 11:44 |
| panda|ruck | now slaves are going offline during jobs | 11:44 |
| amoralej | panda|ruck, apevec i wrote status of promotion mess last tuesday in https://review.rdoproject.org/etherpad/p/promotion-mess-31-oct | 11:45 |
| *** choirboy has joined #rdo | 11:45 | |
| *** pgadiya has quit IRC | 11:48 | |
| panda|ruck | https://bugs.launchpad.net/tripleo/+bug/1729586 | 11:49 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Confirmed] - Assigned to Gabriele Cerami (gcerami) | 11:49 |
| *** fultonj has joined #rdo | 11:50 | |
| *** jlabarre has joined #rdo | 11:51 | |
| *** kellyp has joined #rdo | 11:51 | |
| panda|ruck | if we want to trigger single jobs we'll have to restore the manual pipeline | 11:52 |
| *** nstrug has quit IRC | 11:54 | |
| *** kellyp has quit IRC | 11:56 | |
| *** rcernin has quit IRC | 11:56 | |
| rdogerrit | isaagar created openstack/novajoin-distgit rpm-master: Added docs subpackage for novajoin Fixed easyfix#19 https://review.rdoproject.org/r/10384 | 11:58 |
| *** cylopez has quit IRC | 11:58 | |
| *** jpena is now known as jpena|lunch | 11:59 | |
| *** shardy is now known as shardy_lunch | 12:01 | |
| *** MrAbaddon has quit IRC | 12:03 | |
| amoralej | apevec, we are proposing to pin swift until an issue in puppet-swift is fixed | 12:03 |
| amoralej | ykarel will send it | 12:03 |
| *** amoralej is now known as amoralej|lunch | 12:06 | |
| *** MrAbaddon has joined #rdo | 12:07 | |
| radez | dmsimard: I wonder if a good place to start with those arm servers is to replicate the overcloud-full build closer to how RDO is doing it, I just using my script now: https://paste.fedoraproject.org/paste/V8t4mRMdLIS1zKhrCMYceg | 12:07 |
| *** BarBQ has joined #rdo | 12:08 | |
| *** jaosorior has quit IRC | 12:08 | |
| *** eck` is now known as eck`gone | 12:08 | |
| rdogerrit | Yatin Karel created rdoinfo master: Pin swift to last good hash https://review.rdoproject.org/r/10385 | 12:11 |
| *** artom has joined #rdo | 12:11 | |
| ykarel | amoralej|lunch, ^^ | 12:11 |
| *** imcleod has joined #rdo | 12:11 | |
| *** lebauce has quit IRC | 12:15 | |
| *** ratailor has quit IRC | 12:15 | |
| *** pkovar has joined #rdo | 12:15 | |
| *** jkilpatr has joined #rdo | 12:16 | |
| *** jayg|g0n3 is now known as jayg | 12:19 | |
| *** vpickard_ is now known as vpickard | 12:19 | |
| *** rhallisey has joined #rdo | 12:21 | |
| *** trown|outtypewww is now known as trown | 12:22 | |
| *** ccamacho has joined #rdo | 12:22 | |
| *** lucasagomes is now known as lucas-hungry | 12:23 | |
| *** hseipp has quit IRC | 12:25 | |
| rdogerrit | Yatin Karel proposed rdoinfo master: Pin swift to last good hash https://review.rdoproject.org/r/10385 | 12:25 |
| *** abregman is now known as abregman|afk | 12:25 | |
| *** oidgar has quit IRC | 12:25 | |
| *** kgiusti has joined #rdo | 12:27 | |
| *** jcoufal has joined #rdo | 12:28 | |
| *** hseipp has joined #rdo | 12:28 | |
| *** kellyp has joined #rdo | 12:28 | |
| *** rlandy has joined #rdo | 12:30 | |
| *** pradk has quit IRC | 12:30 | |
| *** gkadam has quit IRC | 12:30 | |
| *** kellyp has quit IRC | 12:32 | |
| *** udesale has quit IRC | 12:33 | |
| *** mrunge has quit IRC | 12:33 | |
| *** jaosorior has joined #rdo | 12:34 | |
| *** catintheroof has joined #rdo | 12:34 | |
| apevec | ykarel, amoralej|lunch re. swift pin - 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 is before that breaking change? | 12:35 |
| *** fragatina has joined #rdo | 12:36 | |
| ykarel | apevec, from where you got that? | 12:37 |
| apevec | hash from above, proposed by amoralej | 12:37 |
| apevec | <amoralej> panda|ruck, apevec we have containers for 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 | 12:38 |
| *** shreshtha has quit IRC | 12:38 | |
| apevec | review you linked was merged Nov 1 | 12:38 |
| apevec | hash is Oct 31 so should be good | 12:38 |
| apevec | wrt swift | 12:38 |
| ykarel | yes this passed in poi: https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario002/3374/rpm_packages.txt.gz | 12:39 |
| *** fragatina has quit IRC | 12:40 | |
| ykarel | in 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 also swift hash is 449d83fb0c8668f444baf3c74cf15c26a70bb02e, so we are good in: https://review.rdoproject.org/r/#/c/10385 | 12:41 |
| *** dtantsur is now known as dtantsur|bbl | 12:45 | |
| *** egonzalez has quit IRC | 12:48 | |
| *** egonzalez has joined #rdo | 12:48 | |
| *** jmelvin has joined #rdo | 12:48 | |
| *** dmacpher has joined #rdo | 12:49 | |
| *** gildub has quit IRC | 12:51 | |
| *** wolverineav has joined #rdo | 12:52 | |
| *** eck`gone is now known as eck` | 12:58 | |
| *** sasha2 has joined #rdo | 12:58 | |
| *** lucas-hungry is now known as lucasagomes | 12:59 | |
| *** jcoufal has quit IRC | 12:59 | |
| *** jcoufal_ has joined #rdo | 13:00 | |
| number80 | can someone pm me link to SF sprint review? | 13:01 |
| *** sshnaidm has quit IRC | 13:02 | |
| *** sshnaidm has joined #rdo | 13:03 | |
| *** ccamacho has quit IRC | 13:04 | |
| *** ccamacho has joined #rdo | 13:04 | |
| *** jlibosva has joined #rdo | 13:05 | |
| *** abregman|afk is now known as abregman | 13:08 | |
| *** ktibi has joined #rdo | 13:09 | |
| *** jlibosva has quit IRC | 13:10 | |
| *** aufi has quit IRC | 13:11 | |
| *** aufi has joined #rdo | 13:13 | |
| *** toure_biab is now known as toure | 13:14 | |
| *** dustins has joined #rdo | 13:16 | |
| tbarron | number80: dmsimard: a few days ago we had a conversation about assigning uids via specfiles: http://eavesdrop.openstack.org/irclogs/%23rdo/%23rdo.2017-10-24.log.html#t2017-10-24T20:31:29 | 13:17 |
| tbarron | number80: dmsimard: I brought up the issue b/c I have a manila bug raised by the ceph deployment folks asking for this: https://bugzilla.redhat.com/show_bug.cgi?id=1506023 | 13:17 |
| openstack | bugzilla.redhat.com bug 1506023 in openstack-manila "manila uid is not consistent between deployments" [Low,Assigned] - Assigned to tbarron | 13:17 |
| tbarron | number80: dmsimard: so in that BZ I explained the reasons for not using fixed uids and asked if there was a concrete reason for actually needing it | 13:17 |
| tbarron | number80: dmsimard: reply here: https://bugzilla.redhat.com/show_bug.cgi?id=1506023#c5 | 13:17 |
| tbarron | number80: dmsimard: This is an OSP12 blocker CVE which manifests in RDO as https://bugzilla.redhat.com/show_bug.cgi?id=1493311 | 13:17 |
| openstack | bugzilla.redhat.com bug 1493311 in openstack-tripleo-heat-templates "CVE-2017-12155 openstack-tripleo-heat-templates: Ceph client keyring is world-readable when deployed by director [openstack-rdo]" [Medium,New] - Assigned to johfulto | 13:17 |
| tbarron | colonwq: fultonj ^^^ chime in, correct, etc. | 13:17 |
| *** cylopez has joined #rdo | 13:17 | |
| tbarron | fultonj: would manila be the only component with this uid issue given that CVE? | 13:18 |
| fultonj | tbarron: no, we'd ask the same of the gnochi package | 13:19 |
| fultonj | number80 dmsimard do you have questions about this request? | 13:19 |
| colonwq | numans, dmsimard these are the basics of needing predictable uids for the mainlia (and gnocchi) service. | 13:19 |
| *** ktibi has left #rdo | 13:19 | |
| * number80 browsing backlog | 13:19 | |
| * number80 reads | 13:20 | |
| fultonj | colonwq is leading the backport of this CVE for ocata and back in time | 13:20 |
| fultonj | the alternative, getting the two puppet modules to coordinate key set up and user creation, would be a much larger change to backport | 13:21 |
| fultonj | number80 dmsimard i think the TLDR of this is https://review.openstack.org/#/c/509020/10/puppet/services/ceph-base.yaml line 137 | 13:24 |
| fultonj | if that helps | 13:25 |
| *** shardy_lunch is now known as shardy | 13:25 | |
| *** pradk has joined #rdo | 13:25 | |
| fultonj | mwhahaha: i'd be curious on your view on ^ | 13:26 |
| *** mrunge has joined #rdo | 13:27 | |
| number80 | tbarron, fultonj: answered bugzilla, short answer: if there's no other way, you have to find UID/GID pair that is not used on RHEL, Fedora, and any RH-products based upon them | 13:29 |
| *** thrash is now known as thrash|biab | 13:29 | |
| number80 | if we break something for another product, it means that we'll get to change that again or fix someone's else package | 13:29 |
| number80 | (and I'm not even including whatever end-users have on their machines ...) | 13:30 |
| fultonj | number80: so suppose we find such a UID, called X... would the next step be to propose a change in the RPMs for manila (and gnochi might use Y) to set the UID on install | 13:31 |
| fultonj | ? | 13:31 |
| number80 | yes | 13:31 |
| fultonj | ack | 13:31 |
| tbarron | number80: You explain the reason for resisting this very well. | 13:31 |
| dciabrin | apevec, number80 hey guys with https://review.rdoproject.org/r/#/c/9971/ we bumped version of mariadb-server to 10.1.20-2 starting pike. I can see it in docker image tripleopike/centos-binary-mariadb:current-tripleo but not in tripleomaster/centos-binary-mariadb:current-tripleo. I was wondering if that's expected or if I did something wrong in the review | 13:32 |
| *** amoralej|lunch is now known as amoralej | 13:32 | |
| amoralej | apevec, yes, hash is before breaking change in swift | 13:32 |
| fultonj | colonwq: so what's harder finding X and Y, or getting to unrelated puppet modules do execute their internal processes separately? | 13:32 |
| fultonj | two* unrelated | 13:33 |
| fultonj | "separately" is not the right word | 13:33 |
| number80 | dciabrin: review is correct, it is updated for queens too. | 13:33 |
| fultonj | thanks number80 | 13:33 |
| tbarron | number80: I think you made the point also that even if we have X and Y there's really no guarantee that some other service won't use X or Y too | 13:33 |
| EmilienM | amoralej, jpena|lunch : thx for https://review.rdoproject.org/r/#/c/10380/ :-) | 13:33 |
| *** jpena|lunch is now known as jpena | 13:34 | |
| number80 | Yes, that's why I insisted a little bit into trying to find another way (and not slamming the door :o) ) | 13:34 |
| *** jprovazn has quit IRC | 13:34 | |
| colonwq | fultonj, I have been thinking about the manila alternate solution and just need to work out some details. I'll have to do the same for gnocchi. | 13:34 |
| fultonj | colonwq: ok, let me know what you find then | 13:35 |
| amoralej | ykarel, i just left a message in https://review.rdoproject.org/r/#/c/10385/ | 13:35 |
| fultonj | thanks number80 tbarron colonwq | 13:35 |
| tbarron | number80: thanks for the help! | 13:35 |
| number80 | np | 13:35 |
| ykarel | amoralej, ok updating | 13:35 |
| colonwq | numans, tbarron fultonj thanks for helping. | 13:36 |
| *** jmlowe_ has quit IRC | 13:36 | |
| rdogerrit | Yatin Karel proposed rdoinfo master: Pin swift to last good hash https://review.rdoproject.org/r/10385 | 13:36 |
| ykarel | amoralej, ^^ | 13:37 |
| openstackgerrit | Javier Peña proposed openstack/packstack master: Fix neutron-vpnaas, re-enable it in tests https://review.openstack.org/517307 | 13:37 |
| amoralej | apevec, puppet modules in periodic jobs are installed from delorean-current so i guess we'll hit the collectd issue if we re-run | 13:37 |
| amoralej | btw, is good idea to use delorean-current in periodic jobs? | 13:38 |
| *** nstrug has joined #rdo | 13:39 | |
| amoralej | no, i was wrong, puppet modules are installed from the tested repo, not from current | 13:40 |
| *** suresh12 has joined #rdo | 13:40 | |
| *** suresh12 has quit IRC | 13:40 | |
| *** eck` is now known as eck`gone | 13:41 | |
| *** suresh12 has joined #rdo | 13:42 | |
| rdogerrit | Yatin Karel proposed rdoinfo master: Pin swift to last good hash https://review.rdoproject.org/r/10385 | 13:42 |
| *** abregman has quit IRC | 13:45 | |
| *** abregman has joined #rdo | 13:46 | |
| number80 | amoralej: any reason to remove +W on https://review.rdoproject.org/r/#/c/10385/4 ? | 13:51 |
| rdogerrit | Javier Peña proposed rdo-infra/puppet-dlrn master: Add support for a public rsync option for workers https://review.rdoproject.org/r/6112 | 13:51 |
| rdogerrit | Javier Peña proposed rdo-infra/puppet-dlrn master: Add symlink from curent-passed-ci to current-tripleo-rdo https://review.rdoproject.org/r/9158 | 13:51 |
| rdogerrit | Javier Peña proposed rdo-infra/puppet-dlrn master: Set the right SQLAlchemy string in workers' alembic.ini https://review.rdoproject.org/r/9614 | 13:51 |
| rdogerrit | Javier Peña proposed rdo-infra/puppet-dlrn master: Support the release_numbering parameter in projects.ini https://review.rdoproject.org/r/10161 | 13:51 |
| amoralej | number80, just waiting for jobs | 13:52 |
| amoralej | but ok | 13:52 |
| number80 | amoralej: ack | 13:52 |
| amoralej | if jobs fail it will not merge anyway :) | 13:52 |
| number80 | Well, usually I let CI flip the coin :) | 13:52 |
| amoralej | cschwede, FYI https://review.rdoproject.org/r/#/c/10385/ | 13:55 |
| apevec | amoralej, periodic shouldn't install anything from current | 13:55 |
| apevec | tripleo gate jobs do have a list of packages to install from current | 13:55 |
| amoralej | apevec, it's a bit tricky but it's installing packages from the right repos | 13:55 |
| apevec | (for reasons I always forget...) | 13:55 |
| *** eharney has joined #rdo | 13:56 | |
| amoralej | they first configure current repo but remove it later | 13:56 |
| apevec | that's... confusing | 13:56 |
| *** Amita has quit IRC | 13:58 | |
| *** spr has quit IRC | 13:59 | |
| amoralej | apevec, in multinode it needs some openstack packages to configure subodes before starting oooq so it configures the repos to install os-apply and family | 13:59 |
| amoralej | at that stage it has -current | 13:59 |
| amoralej | in fact we are getting three packages from -current | 14:00 |
| amoralej | https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset005-master/108b487/subnode-2/var/log/extra/yum-list-installed.txt.gz | 14:00 |
| apevec | hmm | 14:00 |
| amoralej | then it removes all rdo repos | 14:00 |
| amoralej | and let oooq configure them again | 14:00 |
| apevec | but those packages from current stay? | 14:00 |
| amoralej | yes | 14:00 |
| amoralej | it's bad | 14:00 |
| apevec | doesn't that invalidate testing ? | 14:00 |
| apevec | very | 14:00 |
| apevec | panda|ruck, ^ | 14:00 |
| *** thrash|biab is now known as thrash | 14:01 | |
| amoralej | yes, but it's not as bad as having -current enabled during the whole deployment, which is what i was afraid | 14:01 |
| *** spr has joined #rdo | 14:03 | |
| *** spr has quit IRC | 14:04 | |
| *** emalikov has quit IRC | 14:04 | |
| panda|ruck | mmmh | 14:04 |
| *** jmlowe has joined #rdo | 14:05 | |
| *** dmarlin has joined #rdo | 14:06 | |
| *** jkilpatr has quit IRC | 14:06 | |
| panda|ruck | amoralej: what are those packages ? | 14:08 |
| amoralej | panda|ruck, i think they are installed in subnodes as part of the node preparation | 14:08 |
| amoralej | panda|ruck, https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset005-master/108b487/undercloud/var/log/bootstrap-subnodes.log.txt.gz | 14:10 |
| *** jmlowe has quit IRC | 14:11 | |
| amoralej | https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset005-master/108b487/console.txt.gz#_2017-11-02_04_06_35_915 | 14:12 |
| *** janki has quit IRC | 14:12 | |
| *** ccamacho has quit IRC | 14:12 | |
| *** ccamacho has joined #rdo | 14:13 | |
| amoralej | panda|ruck, according to https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L340 | 14:15 |
| amoralej | it seems there is a variable to set if -current is enabled for promotion job | 14:15 |
| amoralej | but we may be missing to enable it | 14:15 |
| panda|ruck | amoralej: yes that's possible, but which packages ? | 14:16 |
| amoralej | panda|ruck, accornding to the logs os-apply-config, os-collect-config and os-refresh-config are being installed from -current | 14:17 |
| amoralej | panda|ruck, https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset005-master/108b487/subnode-2/var/log/extra/yum-list-installed.txt.gz | 14:17 |
| amoralej | look for delorean-current | 14:17 |
| *** vnogin has quit IRC | 14:17 | |
| *** spr has joined #rdo | 14:17 | |
| *** emalikov has joined #rdo | 14:18 | |
| *** vnogin has joined #rdo | 14:18 | |
| *** vnogin has quit IRC | 14:19 | |
| *** vnogin has joined #rdo | 14:20 | |
| amoralej | panda|ruck, some ovb jobs seems stuck in "modify-image : Close qcow2 image" step for hours | 14:23 |
| amoralej | modify-image : Close qcow2 image | 14:24 |
| amoralej | https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-master-nv/940/console | 14:24 |
| amoralej | https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-master-nv/945/console | 14:24 |
| amoralej | mmm, this one looks different | 14:24 |
| amoralej | is it perf issue? | 14:24 |
| *** jkilpatr has joined #rdo | 14:24 | |
| *** vnogin has quit IRC | 14:25 | |
| *** coolsvap has quit IRC | 14:26 | |
| *** jcoufal_ has quit IRC | 14:27 | |
| *** jmlowe has joined #rdo | 14:27 | |
| *** egonzalez has quit IRC | 14:27 | |
| *** rbowen has quit IRC | 14:27 | |
| *** jcoufal has joined #rdo | 14:28 | |
| *** vnogin has joined #rdo | 14:29 | |
| *** spectr-RH has joined #rdo | 14:30 | |
| sfbender | Merged rdopkg master: refactor: nice error messages on invalid Version/patches_base https://softwarefactory-project.io/r/10066 | 14:31 |
| *** egonzalez has joined #rdo | 14:31 | |
| *** BarBQ has quit IRC | 14:31 | |
| *** spectr has quit IRC | 14:33 | |
| *** martinkopec has quit IRC | 14:33 | |
| *** jprovazn has joined #rdo | 14:35 | |
| *** eck`gone is now known as eck` | 14:36 | |
| *** ryansb has quit IRC | 14:38 | |
| *** janki has joined #rdo | 14:38 | |
| *** ryansb has joined #rdo | 14:41 | |
| *** ryansb has joined #rdo | 14:41 | |
| *** cshastri has quit IRC | 14:41 | |
| *** thrash is now known as thrash|biab | 14:41 | |
| *** jcoufal_ has joined #rdo | 14:42 | |
| jruzicka | jschlueter, added 27 complex test scenarios (composed of 165 test steps) for 5 core rdopkg features in last 4 months since feature tests were added \o/ | 14:43 |
| *** jcoufal has quit IRC | 14:45 | |
| *** tshefi has quit IRC | 14:47 | |
| *** suresh12 has quit IRC | 14:47 | |
| dmsimard | fultonj: sorry I've been in back to back meetings since this morning, did you get your answer(s) ? | 14:53 |
| *** Son_Goku has quit IRC | 14:53 | |
| fultonj | dmsimard: yes, number80 updated the bug | 14:54 |
| number80 | fultonj: ack | 14:54 |
| fultonj | colonwq: is looking into next steps | 14:54 |
| * number80 still in calls | 14:54 | |
| fultonj | thanks dmsimard | 14:54 |
| dmsimard | ok, thanks | 14:55 |
| *** sshnaidm is now known as sshnaidm|afk | 14:55 | |
| *** thrash|biab is now known as thrash | 14:58 | |
| *** mary_grace has joined #rdo | 14:58 | |
| *** aufi has quit IRC | 14:58 | |
| *** mosulica has joined #rdo | 14:59 | |
| *** brault has quit IRC | 14:59 | |
| *** brault has joined #rdo | 15:00 | |
| *** emalikov has quit IRC | 15:01 | |
| *** hseipp has quit IRC | 15:02 | |
| number80 | jruzicka: btw, good progress on rdopkg :) | 15:02 |
| * number80 didn't get to say that during the call | 15:02 | |
| jruzicka | number80, indeed, I'm very happy. Aside from stabilizing I'm also getting at long-standing quality of life issues \o/ | 15:03 |
| jruzicka | number80, thanks ;) | 15:04 |
| *** mary_grace has quit IRC | 15:05 | |
| number80 | :) | 15:05 |
| *** jbrooks has joined #rdo | 15:05 | |
| *** lkuchlan has quit IRC | 15:07 | |
| *** janki has quit IRC | 15:07 | |
| *** mary_grace has joined #rdo | 15:08 | |
| rdogerrit | Merged rdoinfo master: Pin swift to last good hash https://review.rdoproject.org/r/10385 | 15:08 |
| *** hseipp has joined #rdo | 15:08 | |
| *** emalikov has joined #rdo | 15:09 | |
| *** aditya_r has joined #rdo | 15:09 | |
| *** abishop has joined #rdo | 15:10 | |
| rdogerrit | hguemar proposed rdoinfo master: Update dateutil/croniter for queens https://review.rdoproject.org/r/10383 | 15:11 |
| *** sshnaidm|afk is now known as sshnaidm | 15:15 | |
| panda|ruck | amoralej: did you open a bug about the three packages or I should ? | 15:18 |
| *** fragatina has joined #rdo | 15:18 | |
| amoralej | panda|ruck, i didn't | 15:18 |
| rdogerrit | Alfredo Moralejo created openstack/swift-distgit rpm-master: Dummy commit to force rebuild https://review.rdoproject.org/r/10386 | 15:18 |
| *** rbartal has quit IRC | 15:18 | |
| *** fragatina has quit IRC | 15:18 | |
| *** rbartal__ has quit IRC | 15:18 | |
| *** rbartal_ has quit IRC | 15:18 | |
| *** fragatina has joined #rdo | 15:19 | |
| *** newmember has joined #rdo | 15:19 | |
| abishop | amoralej: any update on https://review.rdoproject.org/r/10365 making its way to -release? | 15:21 |
| amoralej | abishop, it's in https://review.rdoproject.org/r/#/c/10379 | 15:22 |
| amoralej | but anyway it will not be pushed until monday | 15:22 |
| amoralej | i think new packages are only signed and pushed from mon-thu morning iirc | 15:22 |
| abishop | amoralej: that's cool, just knowing the timeline is good! thx | 15:23 |
| *** gkadam has joined #rdo | 15:23 | |
| *** suresh12 has joined #rdo | 15:24 | |
| *** nstrug has quit IRC | 15:25 | |
| apevec | abishop, why is it important to get it into stable repo? CI should be using trunk repo anyway | 15:26 |
| *** janki has joined #rdo | 15:26 | |
| abishop | apevec: I'm under informed about how all this works, but am trying to keep an eye out for help getting this downstream | 15:27 |
| abishop | apevec: am going to alert folks downstream about this new RPM landing upstream | 15:27 |
| *** dpawlik has quit IRC | 15:28 | |
| *** emalikov has quit IRC | 15:31 | |
| *** bhavik has quit IRC | 15:32 | |
| rdogerrit | Merged openstack/swift-distgit rpm-master: Dummy commit to force rebuild https://review.rdoproject.org/r/10386 | 15:32 |
| *** hseipp has quit IRC | 15:33 | |
| panda|ruck | weshay: apevec amoralej https://bugs.launchpad.net/tripleo/+bug/1729586 | 15:36 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 15:36 |
| *** Amita has joined #rdo | 15:38 | |
| *** thrash is now known as thrash|biab | 15:38 | |
| number80 | adarazs|rover: https://review.rdoproject.org/r/#/c/10356/2 needs +W | 15:39 |
| *** agurenko has quit IRC | 15:40 | |
| *** hseipp has joined #rdo | 15:41 | |
| *** spectr has joined #rdo | 15:41 | |
| *** rbowen has joined #rdo | 15:42 | |
| *** ChanServ sets mode: +o rbowen | 15:42 | |
| *** spectr-RH has quit IRC | 15:44 | |
| *** janki has quit IRC | 15:51 | |
| apevec | jpena, dmsimard - how can we debug why this died https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-containers-build/254/consoleFull ? | 15:53 |
| jpena | mmm... "Slave went offline during the build" | 15:54 |
| *** rbowen has quit IRC | 15:54 | |
| jpena | let's check what nodepool can tell us | 15:54 |
| dmsimard | I've seen that happen once yesterday iirc | 15:54 |
| dmsimard | brb picking up kids from school for lunch | 15:54 |
| *** spectr has quit IRC | 15:55 | |
| apevec | how big is that nodepool node upstream-centos-7-rdo-cloud-tripleo-30473 (upstream-centos-7) ? | 15:55 |
| jpena | apevec: those are 8 CPU, 8 GB | 15:57 |
| *** jhershbe has quit IRC | 15:58 | |
| *** shardy has quit IRC | 15:59 | |
| *** ykarel is now known as ykarel|away | 16:00 | |
| *** anshul has quit IRC | 16:00 | |
| *** pkovar has quit IRC | 16:01 | |
| rdogerrit | Alex Schultz proposed rdoinfo master: Revert "Pin puppet-firewall" https://review.rdoproject.org/r/10375 | 16:02 |
| *** aditya_r has quit IRC | 16:03 | |
| jpena | apevec: do we know the usual RAM pressure of that job? It looks like we did not have any infra issue, so I'm wondering if we went oom | 16:03 |
| *** egonzalez has quit IRC | 16:03 | |
| *** aditya_r has joined #rdo | 16:04 | |
| *** kellyp has joined #rdo | 16:06 | |
| *** newmember has quit IRC | 16:06 | |
| pabelanger | Did you setup a swap? | 16:07 |
| *** mcornea has quit IRC | 16:07 | |
| *** smeyer has quit IRC | 16:07 | |
| *** newmember has joined #rdo | 16:07 | |
| *** sbonazzo has quit IRC | 16:07 | |
| jpena | pabelanger: I don't know if that job set up a swap. panda|ruck, dmsimard do you know? ^^ | 16:08 |
| *** julim has quit IRC | 16:08 | |
| *** jaosorior has quit IRC | 16:08 | |
| *** links has joined #rdo | 16:09 | |
| panda|ruck | I don't think so | 16:10 |
| panda|ruck | the only swap option I remember was for the libvirt | 16:10 |
| rdogerrit | isaagar proposed openstack/novajoin-distgit rpm-master: Added docs subpackage for novajoin Fixed easyfix#19 https://review.rdoproject.org/r/10384 | 16:12 |
| *** morazi has quit IRC | 16:14 | |
| *** jbrooks has quit IRC | 16:15 | |
| *** mosulica has quit IRC | 16:15 | |
| *** gvrangan has joined #rdo | 16:15 | |
| *** agurenko has joined #rdo | 16:16 | |
| amoralej | panda|ruck, i checkec centos-binary-heat-api and tripleo-ci-testing is not applied to 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 | 16:16 |
| amoralej | :( | 16:16 |
| amoralej | apevec, ^ | 16:16 |
| *** dmacpher has quit IRC | 16:17 | |
| panda|ruck | that can be changed | 16:18 |
| jpena | panda|ruck: I found it on another job's logs. It looks like there's 8 GB of swap: https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-centos-7-master-containers-build/bea2241/undercloud/var/log/host_info.txt.gz | 16:18 |
| panda|ruck | but we have to stop the pipeline fist | 16:18 |
| panda|ruck | jpena: userspace file or partition ? | 16:19 |
| jpena | panda|ruck: it looks like a file (/root/swapfile) | 16:20 |
| *** aufi has joined #rdo | 16:22 | |
| dmsimard | That sounds like the swap setup by devstack-gate automatically | 16:25 |
| *** hewbrocca is now known as hewbrocca_afk | 16:25 | |
| *** newmember has quit IRC | 16:26 | |
| *** catintheroof has quit IRC | 16:26 | |
| dmsimard | jpena: we can't discard a legit jenkins issue | 16:26 |
| *** newmember has joined #rdo | 16:27 | |
| dmsimard | We're kind of pushing the limits of a single Jenkins master already | 16:27 |
| apevec | amoralej, so "tripleo-ci-testing" tag moved on i.e. we had newer successfull containers-build job? | 16:27 |
| dmsimard | Running >150 concurrent jobs at capacity | 16:27 |
| dmsimard | Upstream OpenStack had several Jenkins masters befote moving to zuul 2.5 with zuul launcher | 16:28 |
| dmsimard | Jenkins needs to keep the connection open and etc | 16:28 |
| *** catintheroof has joined #rdo | 16:30 | |
| *** dmacpher has joined #rdo | 16:30 | |
| amoralej | apevec, apevec yes, https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-containers-build/255/ | 16:31 |
| *** trown is now known as trown|lunch | 16:31 | |
| apevec | ok, so as panda|ruck said, need to stop that first | 16:31 |
| amoralej | apevec, but jobs in pipeline use containers with tagg tripleo-ci-testing, so we need to re-tag it if we want to retry old hash | 16:33 |
| apevec | right | 16:33 |
| *** cylopez has quit IRC | 16:33 | |
| *** ushkalim has quit IRC | 16:34 | |
| *** noslzzp has quit IRC | 16:35 | |
| *** links has quit IRC | 16:35 | |
| *** pkovar has joined #rdo | 16:35 | |
| *** fragatina has quit IRC | 16:36 | |
| panda|ruck | amoralej: do you have the distro hash for me ? | 16:36 |
| amoralej | panda|ruck, yes, gimme a while | 16:37 |
| *** aditya_r has quit IRC | 16:38 | |
| *** aditya_r has joined #rdo | 16:38 | |
| amoralej | panda|ruck, https://trunk.rdoproject.org/centos7-master/3b/71/3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89/commit.yaml | 16:38 |
| amoralej | commit_hash: 3b718f3fecc866332ec0663fa77e758f8346ab93 | 16:39 |
| amoralej | distro_hash: 4204ba89997cae739e41526b575027e333a2277d | 16:39 |
| panda|ruck | amoralej: thanks | 16:39 |
| chandankumar | amoralej: EmilienM please ack on zaqar tempest plugin review https://review.openstack.org/#/c/504899/ so that they can merge it | 16:39 |
| chandankumar | number80: jpena please have a look on zaqar tempest plugin review https://review.rdoproject.org/r/#/c/10318/ | 16:40 |
| *** dtantsur|bbl is now known as dtantsur | 16:41 | |
| *** rwsu has quit IRC | 16:41 | |
| rdogerrit | Gabriele Cerami created rdo-infra/ci-config master: temporarily pin master to a fixed hash https://review.rdoproject.org/r/10387 | 16:41 |
| *** MrAbaddon has quit IRC | 16:45 | |
| *** rwsu has joined #rdo | 16:47 | |
| *** MrAbaddon has joined #rdo | 16:48 | |
| *** jhershbe has joined #rdo | 16:50 | |
| *** julim has joined #rdo | 16:52 | |
| rdogerrit | Merged rdo-infra/ci-config master: temporarily pin master to a fixed hash https://review.rdoproject.org/r/10387 | 16:53 |
| apevec | panda|ruck, ^ see comments from amoralej and me, | 16:53 |
| apevec | re. tagging containers, is that done? | 16:53 |
| apevec | we need it before jobs run | 16:53 |
| apevec | amoralej, maybe we shouldn't merge it before that is done? | 16:54 |
| panda|ruck | could you wait for me to ask for review ? | 16:54 |
| apevec | when is the next run? | 16:54 |
| apevec | panda|ruck, no :) | 16:54 |
| apevec | we want to have dinner | 16:54 |
| panda|ruck | well, that's bad, that was a mistake | 16:54 |
| apevec | revert it | 16:55 |
| apevec | also use -W for WIP... | 16:55 |
| amoralej | apevec, i don't think we need to merge, tbh | 16:55 |
| apevec | or git-review --draft | 16:55 |
| apevec | amoralej, what do you mean? | 16:56 |
| amoralej | sorry | 16:56 |
| amoralej | we don't need to revert, i meawnt | 16:56 |
| amoralej | meant | 16:56 |
| amoralej | i think order is not important | 16:56 |
| rdogerrit | Gabriele Cerami created rdo-infra/ci-config master: tripleo: fix the pin https://review.rdoproject.org/r/10388 | 16:57 |
| apevec | yeah just needs to be done before next periodic run | 16:57 |
| apevec | when is it? | 16:57 |
| panda|ruck | this was the correct one ^ | 16:57 |
| number80 | chandankumar: commented, should be a quick one but hey, should be late for you right now :) | 16:57 |
| amoralej | apevec, 19:00 UTC | 16:57 |
| apevec | so we have 2h | 16:58 |
| panda|ruck | --draft is not working in openstack | 16:58 |
| rdogerrit | Merged rdoinfo master: Update dateutil/croniter for queens https://review.rdoproject.org/r/10383 | 16:58 |
| *** aditya_r has quit IRC | 16:58 | |
| amoralej | in fact we could reschedule to be started before in https://review.rdoproject.org/r/#/c/10388/ | 16:58 |
| amoralej | but yeah | 16:58 |
| *** jackNemrod has quit IRC | 16:58 | |
| *** mvk has quit IRC | 16:59 | |
| *** paragan has quit IRC | 16:59 | |
| rdogerrit | Gabriele Cerami created config master: temporarily disable containers build https://review.rdoproject.org/r/10389 | 16:59 |
| panda|ruck | also this ^ | 16:59 |
| amoralej | panda|ruck, i'm not very familiar with all those scripts but is not better to pin in get-hash.sh that in promote-hash.sh? | 17:00 |
| apevec | amoralej, number80, let's not push any new deps to queens-testing until we get promotion | 17:00 |
| amoralej | promote-hsah.sh uses hash_.info.sh which is created in get-hash.sh | 17:00 |
| apevec | just to avoid aditional variable... | 17:01 |
| panda|ruck | amoralej: if something gets hash directly from the link name, get-hash will not be enough | 17:01 |
| apevec | panda|ruck, can you get someone familiar with those scripts to review? | 17:01 |
| *** suresh12 has quit IRC | 17:01 | |
| *** jatanmalde has quit IRC | 17:01 | |
| apevec | adarazs|rover, weshay, trown|lunch ? | 17:01 |
| *** julim_ has joined #rdo | 17:02 | |
| number80 | apevec: ack, this one was fixing FTBFS though | 17:02 |
| amoralej | panda|ruck, but promote-hash.sh is used to any promotion action other that tripleo-ci-testing? | 17:02 |
| rdogerrit | chkumar246 proposed openstack/zaqar-tempest-plugin-distgit rpm-master: Initial import of zaqar tempest plugin https://review.rdoproject.org/r/10318 | 17:02 |
| apevec | number80, which one? | 17:02 |
| * weshay looks | 17:02 | |
| amoralej | my concern is that we pin undesired actions | 17:02 |
| *** julim has quit IRC | 17:02 | |
| pabelanger | dmsimard: what is max-servers for nodepool? | 17:02 |
| panda|ruck | amoralej: no, only by the first job in the pipeline | 17:02 |
| pabelanger | you're likely correct that jenkins master might not be keeping up | 17:02 |
| dmsimard | pabelanger: in aggregate we have >150 | 17:03 |
| number80 | apevec: this morning, we have reviews failing due to croniter not picking up dateutil | 17:03 |
| pabelanger | dmsimard: yah, you are pushing the limit | 17:03 |
| apevec | number80, ah that was not ftbfs, but in cbs check jobs? | 17:03 |
| pabelanger | we tried to keep it ~100 per master upstream | 17:03 |
| apevec | number80, b/c it was in common-candidate | 17:03 |
| number80 | apevec: yep | 17:03 |
| dmsimard | pabelanger: that's what I thought too.. | 17:04 |
| apevec | number80, so untagging fixed that | 17:04 |
| pabelanger | dmsimard: I mean, you could stand up another master, they are independant of each other | 17:04 |
| apevec | or we get rid of Jenkins... coming soon! | 17:04 |
| number80 | well, now next update I'm working on is facter3 but it is not something I'm pushing without a plan to test upstream | 17:04 |
| dmsimard | pabelanger: I'd rather we try to hang in there until v3 hopefully sooner rather than later | 17:04 |
| panda|ruck | amoralej: only that job uses promote-hash, the one that sets the hash for the whole pipeline | 17:06 |
| pabelanger | Yah, I'm guessing we'll tag in the new year 3.0, but just guessing | 17:06 |
| amoralej | panda|ruck, how is created hash_info.sh used in https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/jobs/tripleo-upstream.yml#L326 ? | 17:07 |
| dmsimard | pabelanger: SF already has a zuul v3 implementation, we'll definitely start using it ahead of the official release | 17:07 |
| *** pkovar has quit IRC | 17:07 | |
| dmsimard | Most of the fire-y large issues are resolved already or will be fixed by the time we start using it | 17:08 |
| panda|ruck | amoralej: with get_hash | 17:08 |
| amoralej | oh i see it now | 17:08 |
| panda|ruck | amoralej: every job calls get_hash using tripleo-ci-testing as name reference | 17:08 |
| amoralej | it's created from the commit.yaml downloaded from the symlink | 17:08 |
| amoralej | yeah | 17:08 |
| *** MrAbaddon has quit IRC | 17:10 | |
| *** sbulage has quit IRC | 17:12 | |
| *** gvrangan has quit IRC | 17:12 | |
| *** Amita has quit IRC | 17:12 | |
| *** fragatina has joined #rdo | 17:13 | |
| pabelanger | dmsimard: how many executors are you aiming to bring online? | 17:13 |
| *** gszasz has quit IRC | 17:13 | |
| *** gvrangan has joined #rdo | 17:14 | |
| *** thrash|biab is now known as thrash | 17:16 | |
| *** vnogin has quit IRC | 17:16 | |
| *** gvrangan has quit IRC | 17:18 | |
| rdogerrit | Merged rdoinfo master: Revert "Pin puppet-firewall" https://review.rdoproject.org/r/10375 | 17:20 |
| *** TikTok has joined #rdo | 17:20 | |
| *** lucasagomes is now known as lucas-afk | 17:22 | |
| dmsimard | pabelanger: don't have a strong opinion on that yet. I'd tend to look at how many executors are running upstream, their specs, how many jobs they're running and how loaded they are and make an informed decision on that | 17:22 |
| dmsimard | I'd like to have no more than 3 if we can help it | 17:23 |
| *** MrAbaddon has joined #rdo | 17:24 | |
| pabelanger | dmsimard: yah, we're doing about 75 jobs on average right now per executor. but would be interesting to get min/max/mean on that | 17:24 |
| pabelanger | should be able to get it from graphite.o.o | 17:24 |
| rdogerrit | Merged rdo-infra/ci-config master: tripleo: fix the pin https://review.rdoproject.org/r/10388 | 17:25 |
| dmsimard | pabelanger: what's their specs? cpu/ram | 17:26 |
| *** hseipp has quit IRC | 17:26 | |
| *** myoung is now known as myoung|lunch | 17:27 | |
| pabelanger | performance1-8 in rackspace | 17:27 |
| pabelanger | 40GB /dev/xvde1 and 80 GB /dev/xvde2 | 17:28 |
| pabelanger | 8vcpu / 8gb RAM | 17:28 |
| weshay | need reviews on https://review.rdoproject.org/r/#/c/10389/ | 17:29 |
| *** cylopez has joined #rdo | 17:29 | |
| pabelanger | I actually think we might be able to drop back to 8 executors and push more jobs, would be an interesting test but think we'd run out of HDD space before CPU and RAM | 17:30 |
| *** jhershbe has quit IRC | 17:31 | |
| mnaser | http://review.openstack.org/517146 - it would be nice to get python-octaviaclient packaged :D | 17:33 |
| *** jhershbe has joined #rdo | 17:34 | |
| *** MrAbaddon has quit IRC | 17:34 | |
| *** abishop has left #rdo | 17:37 | |
| panda|ruck | jpena: are the replies to api_repo_status_get paginated ? | 17:37 |
| panda|ruck | jpena: is there a limit to the number of jobs it reports ? | 17:37 |
| panda|ruck | well results | 17:37 |
| panda|ruck | not jobs | 17:37 |
| *** fultonj has quit IRC | 17:39 | |
| *** racedo has quit IRC | 17:39 | |
| *** gfidente has quit IRC | 17:41 | |
| dmsimard | pabelanger: something to think about is limitations of the network/SSH stack | 17:43 |
| dmsimard | Also I/O performance | 17:43 |
| amoralej | panda|ruck, i think it's not limited | 17:43 |
| rdogerrit | Merged openstack/novajoin-distgit rpm-master: Added docs subpackage for novajoin Fixed easyfix#19 https://review.rdoproject.org/r/10384 | 17:44 |
| dmsimard | There's probably diminishing returns at some point where increasing the cpu/ram doesn't improve the output | 17:44 |
| *** trown|lunch is now known as trown | 17:44 | |
| panda|ruck | amoralej: thanks. I think I'll try manually, because I'm getting different results when using cli or python lib | 17:45 |
| amoralej | oops | 17:45 |
| panda|ruck | amoralej: apevec are we going to merge this https://review.rdoproject.org/api/10389 ? | 17:45 |
| amoralej | panda|ruck, sorry, i didn't see it coming | 17:46 |
| panda|ruck | amoralej: yeah didn't ping people, just general | 17:46 |
| rdogerrit | rdo-trunk created openstack/glare-distgit rpm-master: openstack-glare: failed to build d114613 https://review.rdoproject.org/r/10390 | 17:48 |
| *** MrAbaddon has joined #rdo | 17:50 | |
| apevec | 10389 failed in gate ? | 17:54 |
| apevec | error cloning ?! | 17:54 |
| *** _bsurfer has quit IRC | 17:54 | |
| *** vnogin has joined #rdo | 17:55 | |
| *** vnogin has quit IRC | 17:55 | |
| panda|ruck | /o\ | 17:55 |
| apevec | fatal: The remote end hung up unexpectedly | 17:56 |
| apevec | I'm rechecking it | 17:56 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-current @ http://tinyurl.com/y8hxewm2 |#| Build failure on centos7-master/current: glare: http://trunk.rdoproject.org/centos7-master/report.html | 17:56 |
| apevec | but do we have issues in the rdo gerrit? | 17:56 |
| *** jpich has quit IRC | 17:57 | |
| rdogerrit | Merged config master: temporarily disable containers build https://review.rdoproject.org/r/10389 | 17:57 |
| apevec | panda|ruck, heh now it worked ^ | 17:57 |
| apevec | jpena, dmsimard ^ anything in gerrit logs around that time? https://review.rdoproject.org/jenkins/job/config-sanity-check/3643/console | 17:58 |
| apevec | 17:50 | 17:58 |
| apevec | or "just" network sheniangans? | 17:59 |
| *** derekh has quit IRC | 17:59 | |
| dmsimard | apevec: we've been chasing those git clone errors for a while now, we have no leads right now | 18:01 |
| apevec | mfedosin, glare switched to stestr, need to fix it in spec: https://review.rdoproject.org/r/10390 | 18:01 |
| dmsimard | I'll try and see if I can find something else.. | 18:01 |
| *** anilvenkata has quit IRC | 18:02 | |
| mfedosin | apevec: got it | 18:02 |
| *** chlong has quit IRC | 18:02 | |
| apevec | amoralej, jpena - last call for https://review.rdoproject.org/r/10376 | 18:04 |
| *** vnogin has joined #rdo | 18:04 | |
| apevec | I'd like to push it so we can run packstack smoke on multiarch w/ pike-testing | 18:04 |
| *** suresh12 has joined #rdo | 18:05 | |
| *** MrAbaddon has quit IRC | 18:05 | |
| dmsimard | apevec, jpena: this might be a hint https://stackoverflow.com/a/17981885 | 18:05 |
| dmsimard | could try GIT_CURL_VERBOSE=1 | 18:06 |
| apevec | I think I saw somethig pushed to change that git config param | 18:06 |
| dmsimard | yeah, it obviously doesn't help :) | 18:07 |
| apevec | https://review.rdoproject.org/r/10271 | 18:07 |
| apevec | that verbose env was reverted https://review.rdoproject.org/r/10255 | 18:08 |
| *** gkadam has quit IRC | 18:08 | |
| apevec | meh | 18:08 |
| *** sasha2 has quit IRC | 18:09 | |
| *** abregman is now known as abregman|afk | 18:09 | |
| *** sasha2 has joined #rdo | 18:09 | |
| *** gkadam has joined #rdo | 18:09 | |
| dmsimard | oh, I forgot yeah we had to revert | 18:09 |
| dmsimard | it broke things | 18:09 |
| amoralej | dmsimard, i think jpena already tried it | 18:10 |
| *** chlong has joined #rdo | 18:10 | |
| amoralej | yeah, that | 18:10 |
| *** pcaruana has quit IRC | 18:10 | |
| amoralej | zuul-clone doesn't like it | 18:10 |
| dmsimard | apevec: it started happening once ceph started having I/O issues :( | 18:10 |
| *** suresh12 has quit IRC | 18:11 | |
| apevec | so potentially just a pref issue? | 18:12 |
| apevec | "just" | 18:12 |
| *** jbrooks has joined #rdo | 18:12 | |
| dmsimard | something we can consider is to clean up the merger cached repositories | 18:12 |
| dmsimard | it'll just re-clone them | 18:13 |
| jpena | dmsimard: re. stackoverflow, I tested that. The env var confuses zuul-cloner (yes, it parses the output), and the postBuffer change did not fix anything | 18:13 |
| jpena | sorry for the lack of replies, I was giving a Gerrit talk at the Mad office | 18:14 |
| jpena | with some SF spam included :) | 18:14 |
| *** fultonj has joined #rdo | 18:14 | |
| dmsimard | apevec, amoralej, jpena: unless there are objections, I'd like to wipe /var/lib/zuul/git/** which is the merger cache | 18:16 |
| *** cylopez has quit IRC | 18:16 | |
| rdogerrit | Merged openstack/packstack-distgit pike-rdo: Update to 11.0.0 https://review.rdoproject.org/r/10376 | 18:16 |
| dmsimard | No clue if that'll resolve the problems but there's no harm and worth a try | 18:16 |
| jpena | dmsimard: is there a service we should stop before doing that? | 18:16 |
| dmsimard | nope | 18:16 |
| jpena | ok, then +1 for me | 18:17 |
| dmsimard | it's only used by jobs that clone repositories at the beginning | 18:17 |
| dmsimard | Like "DEBUG:zuul.Repo:Cloning from https://review.rdoproject.org/p/gerrit.review.rdoproject.org/openstack/packstack-distgit to /home/jenkins/workspace/DLRN-rpmbuild/openstack/packstack-distgit" | 18:17 |
| dmsimard | s,https://review.rdoproject.org/p,/var/lib/zuul/git, | 18:17 |
| *** MrAbaddon has joined #rdo | 18:19 | |
| *** jhershbe has quit IRC | 18:20 | |
| apevec | mfedosin, so setup.py testr subcommand is still there in glare upstream on master and there isn't "stestr" ... | 18:21 |
| *** nstrug has joined #rdo | 18:21 | |
| apevec | looks like that needs to be fixed in pbr? | 18:21 |
| dmsimard | amoralej, jpena, apevec: renamed the root git repositories to _old (faster and also atomic..) will monitor for a while | 18:22 |
| apevec | panda|ruck, containers are re-tagged ? | 18:25 |
| apevec | next run in ~0.5h ! | 18:25 |
| *** MrAbaddon has quit IRC | 18:25 | |
| *** tesseract has quit IRC | 18:25 | |
| dmsimard | jpena: repositories were re-created successfully, deleting old repos | 18:26 |
| *** jcoufal_ has quit IRC | 18:27 | |
| dmsimard | let's see if that helps... | 18:27 |
| *** jcoufal has joined #rdo | 18:28 | |
| rdogerrit | Merged config master: Add a check for Jenkins jobs to have an assigned node, take 2 https://review.rdoproject.org/r/10269 | 18:28 |
| *** suresh12 has joined #rdo | 18:31 | |
| *** hrw has quit IRC | 18:33 | |
| jpena | dmsimard: cool | 18:33 |
| * jpena logs off | 18:33 | |
| *** jpena is now known as jpena|off | 18:34 | |
| *** hrw has joined #rdo | 18:35 | |
| *** dtrainor has quit IRC | 18:41 | |
| *** myoung|lunch is now known as myoung | 18:41 | |
| dmsimard | review.rdo is regularly hitting 50% i/o wait.. | 18:42 |
| dmsimard | http://paste.openstack.org/raw/625380/ | 18:43 |
| amoralej | apevec, panda|ruck i see tripleo-ci-testing tags is still pointing to the wrong containers | 18:43 |
| apevec | panda|ruck, ^^^ | 18:44 |
| amoralej | panda|ruck, don't we need to re-tag? | 18:44 |
| *** dtantsur is now known as dtantsur|afk | 18:44 | |
| apevec | iiuc we need | 18:44 |
| apevec | otherwise next run will be bad | 18:44 |
| amoralej | yes, that's my guess too | 18:44 |
| dmsimard | We have a job that can take a tag and retag | 18:44 |
| dmsimard | What do we want to do ? | 18:45 |
| panda|ruck | dmsimard: is it done locally ? | 18:45 |
| panda|ruck | dmsimard: because I'm pulling tagging and pushing | 18:45 |
| panda|ruck | and it's taking time | 18:45 |
| dmsimard | panda|ruck: omg don't do that | 18:45 |
| dmsimard | please use the openshift client, you don't need to pull or push anything | 18:45 |
| dmsimard | https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/tag-promotion.yml | 18:46 |
| panda|ruck | this is what I had ready ... | 18:46 |
| dmsimard | that playbook ? | 18:46 |
| apevec | dciabrin, back to your question re. tripleomaster/centos-binary-mariadb:current-tripleo - master was not promoted | 18:46 |
| panda|ruck | amoralej: can you try now ? | 18:46 |
| *** aufi has quit IRC | 18:47 | |
| amoralej | panda|ruck, let me show how i'm testing it | 18:47 |
| amoralej | when i do | 18:47 |
| amoralej | skopeo inspect docker://trunk.registry.rdoproject.org/master/centos-binary-nova-api:tripleo-ci-testing | 18:47 |
| panda|ruck | dmsimard: yes, it's done, the push is fast at least, because it just tags, since the image is already there | 18:47 |
| amoralej | in labels i get | 18:47 |
| amoralej | "rdo_version": "824d796775f05d99b211d0a27729fdba490ebab3_0e5ce64f", | 18:47 |
| *** pradk has quit IRC | 18:48 | |
| panda|ruck | amoralej: now too ? | 18:48 |
| amoralej | which is not the right tag | 18:48 |
| amoralej | let me try again | 18:48 |
| amoralej | yeah | 18:48 |
| dmsimard | panda|ruck: the playbook that I linked doesn't pull or push, it tags directly on the registry -- it's what I had set up to promote containers on ci.centos https://ci.centos.org/view/rdo/view/all/job/rdo-tripleo-containers-promote/ | 18:48 |
| amoralej | should we change the schedule for openstack-periodic? | 18:48 |
| amoralej | jobs will start in 12 minutes | 18:48 |
| *** dtrainor has joined #rdo | 18:49 | |
| dmsimard | panda|ruck: it takes like 3 minutes to run | 18:49 |
| apevec | yeah, let's push it until tag is resolved | 18:49 |
| *** sshnaidm is now known as sshnaidm|off | 18:49 | |
| panda|ruck | wait just 5 minutes | 18:50 |
| amoralej | i'll send the review, just in case | 18:50 |
| amoralej | it takes some minutes to apply the config changes | 18:51 |
| panda|ruck | dmsimard: can you retag 3b718f3fecc866332ec0663fa77e758f8346ab93_4204ba89 to tripleo-ci-testing ? | 18:51 |
| dmsimard | for master ? | 18:51 |
| panda|ruck | yes | 18:51 |
| panda|ruck | my run is doing the same | 18:51 |
| dmsimard | what do you mean the same | 18:51 |
| panda|ruck | not sure how fast that is | 18:51 |
| panda|ruck | retagging | 18:51 |
| dmsimard | let's not both do the same thing at once | 18:51 |
| *** eharney has quit IRC | 18:52 | |
| *** dneary has quit IRC | 18:52 | |
| *** toure is now known as toure_biab | 18:52 | |
| apevec | yeah, openshift registry has some issues with concurrency... | 18:52 |
| dmsimard | panda|ruck: are you going to finish in time ? | 18:53 |
| rdogerrit | Alfredo Moralejo created config master: Delay the execution of openstack-periodic https://review.rdoproject.org/r/10391 | 18:53 |
| panda|ruck | amoralej: I think we are good, the containers will not be pulled right away | 18:53 |
| panda|ruck | the test jobs need to install undercloud | 18:53 |
| amoralej | that's right | 18:53 |
| amoralej | ok, | 18:53 |
| panda|ruck | I'll tell you when it's finished | 18:53 |
| amoralej | let them start, then | 18:53 |
| panda|ruck | then I'm going to learn the hell out of openshift clinet | 18:54 |
| amoralej | we can abort them if needed | 18:54 |
| amoralej | i see heat-api changed it already, good | 18:54 |
| *** agurenko has quit IRC | 18:58 | |
| *** TikTok has quit IRC | 18:59 | |
| dmsimard | panda|ruck: This is what we used to promote containers: https://github.com/rdo-infra/ci-config/blob/master/jenkins/jobs/tripleo-containers.yml#L70-L116 and https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/tag-promotion.yml | 19:00 |
| *** nstrug has quit IRC | 19:00 | |
| dmsimard | panda|ruck: What it basically does is to find all images that have a tag X and tags them Y | 19:00 |
| dmsimard | https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/tag-promotion.yml#L33-L41 | 19:00 |
| dmsimard | There is https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/files/k8s_image_tag_filter.py that exists solely for that purpose | 19:01 |
| rdogerrit | Gabriele Cerami created config master: [WIP] don't install undercloud in the containers build job https://review.rdoproject.org/r/10392 | 19:02 |
| panda|ruck | I don't know what's the proper way to find all the deps the undercloud was providing ^ | 19:02 |
| panda|ruck | dmsimard: we need the pull/push approach anyway for dockerhub, but we can certainly optimize the rdo registry part | 19:03 |
| amoralej | jobs are not in openstack-periodic queue | 19:05 |
| amoralej | mmmm | 19:05 |
| amoralej | they should | 19:05 |
| panda|ruck | amoralej: retagging completed | 19:06 |
| *** julim has joined #rdo | 19:06 | |
| panda|ruck | amoralej: which jobs ? | 19:06 |
| amoralej | periodic | 19:06 |
| amoralej | it's 19:06 UTC now | 19:07 |
| amoralej | they should be starting | 19:07 |
| amoralej | panda|ruck, cool | 19:07 |
| *** TikTok has joined #rdo | 19:07 | |
| *** julim_ has quit IRC | 19:08 | |
| *** julim has quit IRC | 19:08 | |
| panda|ruck | I've blocked master promotions until we're sure we have the correct success | 19:09 |
| panda|ruck | I don't want the promoter script to mess with tripleo-ci-testing | 19:09 |
| rdogerrit | slagle created openstack/mistral-distgit rpm-master: Package /var/lib/mistral https://review.rdoproject.org/r/10393 | 19:10 |
| amoralej | panda|ruck, ok | 19:10 |
| dmsimard | panda|ruck: it looks like there's a way to get openshift to push to dockerhub (for example), I'm not exactly sure how it works but it seems possible https://blog.openshift.com/pushing-application-images-to-an-external-registry/ | 19:10 |
| amoralej | but you didn't anything to block jobs from starting, right? | 19:11 |
| dmsimard | panda|ruck: it might require spinning up an openshift "app", though, I'm not sure. | 19:11 |
| amoralej | dmsimard, jobs for openstack-periodic should have started 13 minutes ago but they didn't. I don't see nothing strange in logs | 19:14 |
| amoralej | but i see config-update job running for +40mins | 19:15 |
| amoralej | https://review.rdoproject.org/jenkins/job/config-update/1670/console | 19:15 |
| dmsimard | amoralej: review.rdo is super slow right now, it's over >20% iowait | 19:15 |
| *** kellyp has quit IRC | 19:15 | |
| dmsimard | and I've been keeping an eye on that job as well | 19:15 |
| amoralej | yeah | 19:15 |
| amoralej | ok, maybe it's just that | 19:15 |
| amoralej | i'm going home, i'll look at it later | 19:16 |
| dmsimard | I'll kill that job and re-run it | 19:16 |
| *** amoralej is now known as amoralej|off | 19:19 | |
| dmsimard | panda|ruck: I think you're one hour off for the periodic jobs | 19:20 |
| panda|ruck | dmsimard: ? | 19:21 |
| dmsimard | hmmm, wait, nevermind | 19:21 |
| dmsimard | the zuul node is in UTC and "19" did pass | 19:21 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-current @ http://tinyurl.com/y8hxewm2 |#| Build failure on centos7-master/current: glare: http://trunk.rdoproject.org/centos7-master/report.html | 19:22 |
| dmsimard | don't have an explanation | 19:23 |
| dmsimard | it triggered properly at 14:00, 2017-11-02 14:00:00,762 INFO zuul.Scheduler: Adding openstack-infra/tripleo-ci, <NullChange for openstack-infra/tripleo-ci> to <Pipeline openstack-periodic> | 19:24 |
| dmsimard | then after that, nothing | 19:25 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-pike-consistent @ http://tinyurl.com/y8jwnbku |#| Last check execution was 186 seconds ago | 19:25 |
| *** rhallisey has quit IRC | 19:28 | |
| *** dprince has joined #rdo | 19:28 | |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-current @ http://tinyurl.com/y8hxewm2 |#| Build failure on centos7-master/current: glare: http://trunk.rdoproject.org/centos7-master/report.html | 19:28 |
| apevec | jobs MIA ? | 19:31 |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-pike-consistent @ http://tinyurl.com/y8jwnbku |#| No build failures detected: http://trunk.rdoproject.org/centos7-pike/report.html | 19:31 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-consistent @ http://tinyurl.com/y8rdsllp |#| Last check execution was 222 seconds ago | 19:32 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-pike-current-passed-ci @ http://tinyurl.com/y9kngr6a |#| Last check execution was 229 seconds ago | 19:32 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-ocata-consistent @ http://tinyurl.com/y9v2ywjh |#| Last check execution was 235 seconds ago | 19:33 |
| *** vnogin has quit IRC | 19:34 | |
| dmsimard | apevec: lots of things wrong right now... ^ those "last execution" are the monitoring node not being able to complete it's checks | 19:34 |
| *** jbrooks has quit IRC | 19:35 | |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-pike-current-passed-ci @ http://tinyurl.com/y9kngr6a |#| No build failures detected: http://trunk.rdoproject.org/centos7-pike/report.html | 19:35 |
| *** rdogerrit has quit IRC | 19:35 | |
| apevec | uhoh | 19:35 |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-ocata-consistent @ http://tinyurl.com/y9v2ywjh |#| No build failures detected: http://trunk.rdoproject.org/centos7-ocata/report.html | 19:35 |
| *** vnogin has joined #rdo | 19:35 | |
| dmsimard | apevec: I sent https://review.rdoproject.org/r/#/c/10391/ through | 19:36 |
| *** julim has joined #rdo | 19:36 | |
| dmsimard | asked rhos-ops how Ceph is doing ... | 19:36 |
| *** rdogerrit has joined #rdo | 19:36 | |
| rdogerrit | Merged config master: Delay the execution of openstack-periodic https://review.rdoproject.org/r/10391 | 19:36 |
| *** vnogin has quit IRC | 19:36 | |
| dmsimard | apevec: We have some headroom, I'll schedule an emergency maintenance to resize review.rdo, jenkins is struggling and running out of ram | 19:37 |
| *** brault has quit IRC | 19:37 | |
| dmsimard | we can always resize down once we move to zuul v3 | 19:37 |
| *** brault has joined #rdo | 19:37 | |
| dmsimard | swapping into slow ceph i/o is not something we want to be doing | 19:37 |
| *** vnogin has joined #rdo | 19:37 | |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-master-consistent @ http://tinyurl.com/y8rdsllp |#| No build failures detected: http://trunk.rdoproject.org/centos7-master/report.html | 19:37 |
| apevec | ok | 19:38 |
| apevec | hmm, will delaying to 2000 UTC be enough? | 19:39 |
| apevec | that's 20 from now | 19:39 |
| apevec | 20min | 19:39 |
| dmsimard | apevec: yeah | 19:39 |
| dmsimard | apevec: everyrhing is ready | 19:39 |
| dmsimard | apevec: we were ready for 19 | 19:39 |
| dmsimard | but for some unknown reason it didn't trigger | 19:39 |
| dmsimard | we have no containers to build so we'll be fine for the 00:00 window | 19:40 |
| dmsimard | brb | 19:40 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-current @ http://tinyurl.com/y8hxewm2 |#| Build failure on centos7-master/current: glare: http://trunk.rdoproject.org/centos7-master/report.html | 19:40 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-ocata-consistent @ http://tinyurl.com/y9v2ywjh |#| Last check execution was 235 seconds ago | 19:40 |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-ocata-consistent @ http://tinyurl.com/y9v2ywjh |#| No build failures detected: http://trunk.rdoproject.org/centos7-ocata/report.html | 19:42 |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-pike-current-passed-ci @ http://tinyurl.com/y9kngr6a |#| Last check execution was 229 seconds ago | 19:42 |
| apevec | dmsimard, ah so you are not going to resize review.rdo in <20min :) I misunderstood | 19:43 |
| rdobot | [sensu] RESOLVED: master.monitoring.rdoproject.org - check-delorean-pike-current-passed-ci @ http://tinyurl.com/y9kngr6a |#| No build failures detected: http://trunk.rdoproject.org/centos7-pike/report.html | 19:45 |
| *** kellyp has joined #rdo | 19:45 | |
| *** kellyp has quit IRC | 19:51 | |
| *** kellyp has joined #rdo | 19:51 | |
| dmsimard | apevec: no, going to do it later tonight ... | 19:52 |
| dmsimard | need to step out for appointment, afk for a bit | 19:52 |
| *** brault has quit IRC | 19:52 | |
| *** kellyp has joined #rdo | 19:52 | |
| *** gkadam has quit IRC | 19:54 | |
| *** MrAbaddon has joined #rdo | 19:55 | |
| *** vnogin has quit IRC | 20:00 | |
| *** jcoufal has quit IRC | 20:02 | |
| *** eharney has joined #rdo | 20:02 | |
| *** jcoufal has joined #rdo | 20:02 | |
| *** TikTok has quit IRC | 20:06 | |
| *** mary_grace has quit IRC | 20:06 | |
| *** jackNemrod has joined #rdo | 20:08 | |
| *** dprince has quit IRC | 20:08 | |
| *** TikTok has joined #rdo | 20:09 | |
| *** jayg is now known as jayg|g0n3 | 20:12 | |
| *** dprince has joined #rdo | 20:14 | |
| *** atoth has quit IRC | 20:16 | |
| *** jcoufal_ has joined #rdo | 20:16 | |
| *** dneary has joined #rdo | 20:17 | |
| *** dprince has quit IRC | 20:17 | |
| *** julim has quit IRC | 20:17 | |
| *** dprince has joined #rdo | 20:17 | |
| *** jcoufal has quit IRC | 20:19 | |
| *** jcoufal has joined #rdo | 20:20 | |
| *** slagle has quit IRC | 20:20 | |
| *** eck` is now known as eck`gone | 20:21 | |
| *** jcoufal_ has quit IRC | 20:22 | |
| *** kgiusti has left #rdo | 20:23 | |
| *** slagle has joined #rdo | 20:23 | |
| *** jcoufal_ has joined #rdo | 20:24 | |
| *** vnogin has joined #rdo | 20:24 | |
| *** vnogin has quit IRC | 20:24 | |
| *** jcoufal has quit IRC | 20:27 | |
| *** newmember has quit IRC | 20:34 | |
| *** newmember has joined #rdo | 20:34 | |
| *** dneary has quit IRC | 20:35 | |
| *** vnogin has joined #rdo | 20:35 | |
| *** brault has joined #rdo | 20:39 | |
| *** vnogin has quit IRC | 20:40 | |
| *** vnogin has joined #rdo | 20:43 | |
| *** mary_grace has joined #rdo | 20:44 | |
| *** brault has quit IRC | 20:44 | |
| *** rhallisey has joined #rdo | 20:44 | |
| *** jprovazn has quit IRC | 20:46 | |
| *** jmlowe has quit IRC | 20:48 | |
| *** trown is now known as trown|outtypewww | 20:49 | |
| *** brault has joined #rdo | 20:49 | |
| *** vnogin has quit IRC | 20:51 | |
| *** myoung is now known as myoung|afk | 20:51 | |
| *** myoung|afk is now known as myoung | 20:53 | |
| *** dprince has quit IRC | 20:54 | |
| *** myoung is now known as myoung|bbl | 20:56 | |
| *** brault has quit IRC | 20:57 | |
| *** mary_grace has quit IRC | 21:00 | |
| *** mary_grace has joined #rdo | 21:00 | |
| *** rhallisey has quit IRC | 21:01 | |
| *** eharney has quit IRC | 21:08 | |
| *** vnogin has joined #rdo | 21:11 | |
| *** vnogin has quit IRC | 21:16 | |
| *** vnogin has joined #rdo | 21:26 | |
| *** vnogin has quit IRC | 21:30 | |
| *** ccamacho has quit IRC | 21:33 | |
| *** jtomasek has quit IRC | 21:35 | |
| *** d0ugal has quit IRC | 21:40 | |
| *** jmlowe has joined #rdo | 21:41 | |
| *** mvk has joined #rdo | 21:44 | |
| *** kellyp has quit IRC | 21:45 | |
| *** d0ugal has joined #rdo | 21:47 | |
| *** julim has joined #rdo | 21:48 | |
| *** vnogin has joined #rdo | 21:49 | |
| *** vnogin has quit IRC | 21:50 | |
| *** yselkowitz has quit IRC | 21:50 | |
| *** yselkowitz has joined #rdo | 21:51 | |
| *** catintheroof has quit IRC | 21:53 | |
| *** gildub has joined #rdo | 22:01 | |
| *** rcernin has joined #rdo | 22:01 | |
| *** rbowen has joined #rdo | 22:02 | |
| *** ChanServ sets mode: +o rbowen | 22:02 | |
| *** jmelvin has quit IRC | 22:04 | |
| *** Son_Goku has joined #rdo | 22:06 | |
| *** threestrands has joined #rdo | 22:10 | |
| *** threestrands has quit IRC | 22:10 | |
| *** threestrands has joined #rdo | 22:10 | |
| rdogerrit | Alan Pevec proposed rdoinfo master: Pin branchless projects for stable releases https://review.rdoproject.org/r/10328 | 22:11 |
| *** rbowen has quit IRC | 22:15 | |
| rdogerrit | Merged openstack/mistral-distgit rpm-master: Package /var/lib/mistral https://review.rdoproject.org/r/10393 | 22:15 |
| *** vnogin has joined #rdo | 22:16 | |
| dmsimard | apevec: master looking good so far | 22:21 |
| * apevec knocks | 22:24 | |
| apevec | dmsimard, I saw one red but it's -pike | 22:25 |
| apevec | so periodic runs all releases at once? | 22:25 |
| dmsimard | That's the way things are set up right now. | 22:28 |
| *** dustins has quit IRC | 22:28 | |
| *** kellyp has joined #rdo | 22:29 | |
| *** jbrooks has joined #rdo | 22:31 | |
| *** vnogin has quit IRC | 22:32 | |
| *** jackNemrod has quit IRC | 22:36 | |
| rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-master-current @ http://tinyurl.com/y8hxewm2 |#| Build failure on centos7-master/current: glare: http://trunk.rdoproject.org/centos7-master/report.html | 22:37 |
| *** kellyp has quit IRC | 22:39 | |
| *** kellyp has joined #rdo | 22:40 | |
| apevec | dmsimard, so you estimate periodic jobs will finish in <1h ? | 22:43 |
| *** rlandy is now known as rlandy|afk | 22:43 | |
| dmsimard | apevec: looks like I might need to interrupt the pike ones. | 22:44 |
| apevec | ack, just let master finish | 22:44 |
| apevec | pike have now 2 failures | 22:44 |
| *** kellyp has quit IRC | 22:45 | |
| *** kellyp has joined #rdo | 22:47 | |
| *** kellyp has quit IRC | 22:47 | |
| *** kellyp has joined #rdo | 22:47 | |
| *** vpickard is now known as vpickard_ | 22:48 | |
| apevec | this job was finish under 2h https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-master/buildTimeTrend | 22:50 |
| apevec | now it's almost 2,5h | 22:51 |
| apevec | and it failed :( | 22:52 |
| apevec | panda|ruck, ^ bad news | 22:52 |
| apevec | [overcloud]: CREATE_FAILED Timed out | 22:54 |
| panda|ruck | ... | 23:02 |
| apevec | dmsimard, I think you can go ahead and abort the jobs, since master will not pass | 23:02 |
| *** paramite has quit IRC | 23:02 | |
| apevec | panda|ruck, let's keep this pin running few more times | 23:02 |
| panda|ruck | yep | 23:02 |
| apevec | there will be now short outage to resize review.rdo | 23:02 |
| apevec | it might be also that this job is too heavy for the resources we have... https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset024.yml | 23:03 |
| apevec | this is with encryption everywhere, right? | 23:03 |
| apevec | hmm, or it's not? ssl_overcloud: false | 23:04 |
| *** morazi has joined #rdo | 23:06 | |
| *** wolverineav has quit IRC | 23:08 | |
| *** wolverineav has joined #rdo | 23:08 | |
| apevec | where is the license label coming from? | 23:09 |
| apevec | "license": "GPLv2", | 23:09 |
| apevec | "maintainer": "TripleO Project (http://tripleo.org)", | 23:09 |
| apevec | "name": "nova-api", | 23:09 |
| apevec | that's not correct | 23:09 |
| *** TikTok has quit IRC | 23:12 | |
| *** wolverineav has quit IRC | 23:13 | |
| *** TikTok has joined #rdo | 23:14 | |
| *** Son_Goku has quit IRC | 23:15 | |
| *** apevec has quit IRC | 23:15 | |
| *** rcernin has quit IRC | 23:17 | |
| *** Son_Goku has joined #rdo | 23:20 | |
| *** Son_Goku has quit IRC | 23:35 | |
| *** jcoufal has joined #rdo | 23:41 | |
| *** rdogerrit has quit IRC | 23:42 | |
| *** jcoufal_ has quit IRC | 23:43 | |
| *** links has joined #rdo | 23:44 | |
| dmsimard | review.rdoproject.org is under maintenance, currently doing a selinux relabel. | 23:56 |
| *** noslzzp has joined #rdo | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!