rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ http://tinyurl.com/gud2vup |#| Build failure on centos7-master/current: swift: http://trunk.rdoproject.org/centos7-master/report.html | 00:06 |
---|---|---|
EmilienM | http://logs.openstack.org/55/354355/1/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/2d1322b/logs/nova/nova-compute.txt.gz#_2016-08-11_20_20_52_301 | 00:13 |
EmilienM | fuck | 00:13 |
EmilienM | I'll investigate later, diner now | 00:14 |
EmilienM | promotion fails for puppet CI | 00:14 |
*** Ryjedo_ has joined #rdo | 00:19 | |
*** Ryjedo has quit IRC | 00:20 | |
*** Ryjedo_ is now known as Ryjedo | 00:20 | |
*** limao has joined #rdo | 00:28 | |
*** alexismonville has quit IRC | 00:28 | |
*** limao has quit IRC | 00:29 | |
*** limao has joined #rdo | 00:29 | |
*** manous has quit IRC | 00:33 | |
*** Son_Goku has joined #rdo | 00:45 | |
*** sdake has joined #rdo | 00:46 | |
dmsimard | EmilienM: it looks like it might be https://review.openstack.org/#/c/319387/ | 01:01 |
EmilienM | dmsimard: what? | 01:02 |
EmilienM | dmsimard: and the nova thing? | 01:02 |
dmsimard | EmilienM: dunno didn't look | 01:03 |
dmsimard | EmilienM: I was investigating the failure to build and stumbled on the swift ring builder ipv6 commit | 01:03 |
dmsimard | EmilienM: and we're getting swift ring errors that seem related to ipv6 :) | 01:03 |
EmilienM | dmsimard: nova.conf looks good | 01:04 |
EmilienM | i don't understand | 01:04 |
EmilienM | oh wait | 01:05 |
EmilienM | scenario001 actually works | 01:05 |
EmilienM | so it might be a random issue for nova? | 01:06 |
rdogerrit | David Moreau Simard proposed openstack/swift-distgit: Add python-netifaces as a buildrequires for docs http://review.rdoproject.org/r/1839 | 01:11 |
EmilienM | dmsimard: I'm looking the puppet swift code | 01:11 |
EmilienM | it's in the second puppet run | 01:13 |
EmilienM | so it's not idempotent | 01:13 |
dmsimard | all of a sudden ? | 01:14 |
dmsimard | not a ton of things have merged recently in swift https://review.openstack.org/#/q/project:openstack/swift+status:merged | 01:15 |
EmilienM | https://review.openstack.org/#/c/319387/6/swift/cli/ringbuilder.py | 01:16 |
EmilienM | the output seems different now | 01:16 |
*** alexismonville has joined #rdo | 01:16 | |
EmilienM | and our ***ing parser needs an update ai guess | 01:16 |
dmsimard | ohhhhhhhhhhh | 01:17 |
dmsimard | that's right | 01:17 |
dmsimard | we parse the output | 01:17 |
dmsimard | muahahahahaha | 01:17 |
dmsimard | I touched that parser ONCE to add multi region support | 01:17 |
EmilienM | we need to compare the print | 01:17 |
EmilienM | yeah | 01:17 |
EmilienM | I never touched it | 01:17 |
EmilienM | it's against my religion | 01:17 |
EmilienM | really I didn't write this code | 01:17 |
dmsimard | It's almost as bad as that one pretty table parsing script somewhere in nova | 01:18 |
ggillies | does anyone happen to know how the tripleo overcloud images are built at http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/mitaka/cbs/ | 01:18 |
EmilienM | ok i'm trying to find what changed in output | 01:18 |
dmsimard | ggillies: I would guess those are out of our CI | 01:19 |
ggillies | it's obviously not using python-tripleoclient | 01:19 |
dmsimard | ggillies: they are built through tripleo-quickstart and uploaded there if the job pipeline was successful | 01:19 |
EmilienM | ah | 01:19 |
EmilienM | I found why | 01:19 |
EmilienM | before it was "ip port" | 01:19 |
dmsimard | ggillies: these jobs: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-mitaka/ | 01:19 |
EmilienM | now it's "ip:port" | 01:19 |
ggillies | what has tripleo-quckstart got to do with building images | 01:19 |
ggillies | jesus this stuff is a mess | 01:20 |
EmilienM | lol | 01:20 |
dmsimard | ggillies: /me shrugs | 01:20 |
EmilienM | don't call jesus! | 01:20 |
EmilienM | I'll have to invoke stars again | 01:20 |
ggillies | woops | 01:20 |
dmsimard | I'm going to go hide because I have nothing to do with quickstart (but really though I'm going to shower and sleep) | 01:20 |
ggillies | ok I think I need to put a spec or something to do overhaul of image building | 01:20 |
ggillies | it looks like we have a lot of legacy related to "old" tripleo | 01:21 |
ggillies | might be time to clean it out | 01:21 |
EmilienM | dmsimard: dude I'm preparing a patch | 01:22 |
* EmilienM invoking stars tonight | 01:22 | |
* EmilienM music | 01:22 | |
EmilienM | I swear I don't take drugs | 01:22 |
dmsimard | EmilienM: good, you can +1 https://review.rdoproject.org/r/#/c/1839/ when the dlrn job passes so it looks like I didn't ninja self merge | 01:23 |
dmsimard | in the state things are I definitely don't want unconsistent dlrn | 01:23 |
EmilienM | dude I'm doing a regex now | 01:23 |
dmsimard | need help with the regex ? | 01:23 |
EmilienM | yes | 01:23 |
EmilienM | I'm really bad | 01:23 |
dmsimard | EmilienM: http://xkcd.com/208/ | 01:23 |
EmilienM | lol | 01:24 |
*** rhallisey has joined #rdo | 01:24 | |
EmilienM | CAN WE BE SERIOUS ?? | 01:24 |
EmilienM | I'm doing a regex. | 01:24 |
EmilienM | lol the regex in puppet-swift: | 01:25 |
EmilienM | if row =~ /^\s*(\d+)\s+(\d+)\s+(\d+)\s+(\S+)\s+(\d+)\s+\S+\s+\d+\s+(\S+)\s+(\d+\.\d+)\s+(\d+)\s*((-|\s-?)?\d+\.\d+)\s*(\S*)/ | 01:25 |
dmsimard | EmilienM: https://xkcd.com/1171/ | 01:25 |
EmilienM | ahah | 01:25 |
ggillies | ok so if I am reading this correctly | 01:25 |
ggillies | all our CI jobs no longer actually use python-tripleoclient to build images | 01:25 |
*** paragan has quit IRC | 01:25 | |
dmsimard | ggillies: tripleoci probably uses python-tripleoclient | 01:25 |
ggillies | so the whole 'openstack overcloud image build' is basically now untested | 01:26 |
dmsimard | ggillies: I don't know tbh | 01:26 |
*** zeroshft has joined #rdo | 01:26 | |
dmsimard | #tripleo probably a better place to ask | 01:26 |
EmilienM | I go on http://rubular.com/ shamelesly | 01:26 |
dmsimard | EmilienM: https://xkcd.com/1313/ | 01:27 |
EmilienM | stop! | 01:27 |
dmsimard | BUT THERE IS SO MUCH FUN TO BE HAD WITH REGEX | 01:27 |
*** zeroshft has quit IRC | 01:30 | |
*** limao_ has joined #rdo | 01:36 | |
EmilienM | doing a regex at 9.30pm | 01:38 |
EmilienM | life is sad | 01:38 |
EmilienM | anyway: https://review.openstack.org/#/c/354476/ | 01:38 |
*** limao has quit IRC | 01:39 | |
*** alexismonville has quit IRC | 01:52 | |
*** Amita has joined #rdo | 02:00 | |
*** sdake has quit IRC | 02:01 | |
*** rhallisey has quit IRC | 02:01 | |
rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ http://tinyurl.com/gud2vup |#| Build failure on centos7-master/current: swift: http://trunk.rdoproject.org/centos7-master/report.html | 02:03 |
rdogerrit | David Moreau Simard proposed openstack/swift-distgit: Add python-netifaces and eclib as a buildrequires for docs http://review.rdoproject.org/r/1839 | 02:05 |
*** gfidente has quit IRC | 02:11 | |
*** sdake has joined #rdo | 02:13 | |
*** shaunm has quit IRC | 02:16 | |
*** gildub has joined #rdo | 02:19 | |
*** zeroshft has joined #rdo | 02:27 | |
*** bswartz has quit IRC | 02:33 | |
*** sdake has quit IRC | 02:34 | |
*** zeroshft has quit IRC | 02:36 | |
*** bswartz has joined #rdo | 02:37 | |
*** crossbuilder has joined #rdo | 02:37 | |
*** imcleod has quit IRC | 02:37 | |
*** crossbuilder_ has quit IRC | 02:38 | |
*** zeroshft has joined #rdo | 02:42 | |
*** zeroshft has quit IRC | 02:46 | |
rdogerrit | David Moreau Simard proposed openstack/swift-distgit: Add new build dependencies for Swift documentation http://review.rdoproject.org/r/1839 | 02:46 |
*** ayoung has joined #rdo | 02:49 | |
*** imcleod has joined #rdo | 02:50 | |
*** sdake has joined #rdo | 02:54 | |
*** paragan has joined #rdo | 02:55 | |
*** rhefner has quit IRC | 02:55 | |
*** rhefner has joined #rdo | 02:55 | |
*** mwhahaha has quit IRC | 02:55 | |
*** mwhahaha has joined #rdo | 02:59 | |
*** paragan has quit IRC | 03:01 | |
*** Ryjedo has quit IRC | 03:01 | |
*** zeroshft has joined #rdo | 03:02 | |
*** zeroshft has quit IRC | 03:03 | |
*** Ryjedo has joined #rdo | 03:03 | |
*** zeroshft has joined #rdo | 03:03 | |
*** imcleod has quit IRC | 03:05 | |
*** zeroshft has quit IRC | 03:12 | |
*** limao_ has quit IRC | 03:13 | |
rdogerrit | David Moreau Simard proposed openstack/swift-distgit: Add new build dependencies for Swift documentation http://review.rdoproject.org/r/1839 | 03:13 |
*** imcleod has joined #rdo | 03:21 | |
*** paragan has joined #rdo | 03:24 | |
*** sdake has quit IRC | 03:24 | |
pabelanger | ggillies: you might be interested in: http://logs.openstack.org/29/353929/3/experimental/gate-tripleo-buildimage-overcloud-full-centos-7-nv/59f4e05//console.html | 03:25 |
pabelanger | ggillies: I just added an experimental job upstream to build overcloud-full using python-tripleoclient | 03:25 |
pabelanger | ggillies: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/tripleo.yaml#n64 is the job-template | 03:26 |
pabelanger | ggillies: my next step is to make it non-voting, but eventually all jobs for all our images too | 03:27 |
ggillies | pabelanger: thanks | 03:30 |
ggillies | I'm about to send out an email with what I've found and some suggestions | 03:30 |
pabelanger | ggillies: Ya, I'm sure we are on the same page. | 03:30 |
rdogerrit | David Moreau Simard proposed rdoinfo: Update newton-uc branch tags to latest upper-constraints http://review.rdoproject.org/r/1834 | 03:30 |
*** links has joined #rdo | 03:33 | |
rdogerrit | Merged rdoinfo: Update newton-uc branch tags to latest upper-constraints http://review.rdoproject.org/r/1834 | 03:34 |
*** limao has joined #rdo | 03:38 | |
*** ayoung has quit IRC | 03:45 | |
*** numans has joined #rdo | 03:46 | |
*** sdake has joined #rdo | 03:54 | |
*** vimal has joined #rdo | 03:54 | |
*** paragan has quit IRC | 03:56 | |
*** paragan has joined #rdo | 03:57 | |
rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ http://tinyurl.com/gud2vup |#| Build failure on centos7-master/current: swift: http://trunk.rdoproject.org/centos7-master/report.html | 04:03 |
rdogerrit | Merged openstack/swift-distgit: Add new build dependencies for Swift documentation http://review.rdoproject.org/r/1839 | 04:03 |
*** numans has quit IRC | 04:03 | |
rdogerrit | Merged DLRN: Add timestamp to log output http://review.rdoproject.org/r/1830 | 04:04 |
*** vimal has quit IRC | 04:07 | |
*** Amita has quit IRC | 04:07 | |
*** imcleod has quit IRC | 04:16 | |
*** fragatina has quit IRC | 04:16 | |
*** limao has quit IRC | 04:19 | |
*** limao has joined #rdo | 04:20 | |
*** vimal has joined #rdo | 04:29 | |
*** nehar has joined #rdo | 04:34 | |
*** nehar has quit IRC | 04:39 | |
*** nehar has joined #rdo | 04:51 | |
*** fragatina has joined #rdo | 04:58 | |
*** rdas has joined #rdo | 05:01 | |
*** fragatina has quit IRC | 05:03 | |
*** ohochman has left #rdo | 05:04 | |
*** jaosorior has joined #rdo | 05:04 | |
*** nehar has quit IRC | 05:05 | |
*** anilvenkata has joined #rdo | 05:06 | |
*** chandankumar has joined #rdo | 05:16 | |
*** Son_Goku has quit IRC | 05:20 | |
*** ggillies has quit IRC | 05:31 | |
*** sdake has quit IRC | 05:32 | |
*** coolsvap has joined #rdo | 05:34 | |
*** rcernin has quit IRC | 05:51 | |
*** hewbrocca-afk is now known as hewbrocca | 05:53 | |
*** saneax_AFK is now known as saneax | 05:54 | |
*** saneax is now known as Guest89739 | 05:54 | |
*** Son_Goku has joined #rdo | 05:59 | |
*** kaminohana has quit IRC | 06:00 | |
rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ http://tinyurl.com/gud2vup |#| Build failure on centos7-master/current: swift: http://trunk.rdoproject.org/centos7-master/report.html | 06:03 |
*** limao has quit IRC | 06:08 | |
*** limao_ has joined #rdo | 06:08 | |
chandankumar | number80, \o/ | 06:08 |
*** manous has joined #rdo | 06:10 | |
*** jprovazn has joined #rdo | 06:11 | |
*** manous has quit IRC | 06:15 | |
*** limao_ has quit IRC | 06:15 | |
*** limao has joined #rdo | 06:15 | |
*** satya4ever has joined #rdo | 06:16 | |
*** fultonj has quit IRC | 06:17 | |
*** manous has joined #rdo | 06:18 | |
*** rcernin has joined #rdo | 06:21 | |
*** manous has quit IRC | 06:22 | |
*** aufi has joined #rdo | 06:23 | |
*** anilvenkata is now known as anilvenkata_afk | 06:24 | |
*** Amita has joined #rdo | 06:26 | |
*** manous has joined #rdo | 06:26 | |
*** lkuchlan has joined #rdo | 06:27 | |
*** furlongm has quit IRC | 06:28 | |
*** pgadiya has joined #rdo | 06:30 | |
*** pgadiya has quit IRC | 06:31 | |
*** manous has quit IRC | 06:31 | |
*** vaneldik has joined #rdo | 06:31 | |
*** vaneldik has quit IRC | 06:32 | |
*** furlongm has joined #rdo | 06:32 | |
*** manous has joined #rdo | 06:35 | |
*** lkuchlan has quit IRC | 06:39 | |
*** smeyer has joined #rdo | 06:40 | |
*** manous has quit IRC | 06:40 | |
*** nyechiel has joined #rdo | 06:41 | |
*** manous has joined #rdo | 06:43 | |
*** mrunge has quit IRC | 06:47 | |
*** manous has quit IRC | 06:48 | |
*** nyechiel has quit IRC | 06:49 | |
*** manous has joined #rdo | 06:50 | |
*** iranzo has joined #rdo | 06:50 | |
*** manous has quit IRC | 06:55 | |
*** anilvenkata_afk is now known as anilvenkata | 06:58 | |
*** manous has joined #rdo | 06:58 | |
*** smeyer has quit IRC | 06:58 | |
*** tesseract- has joined #rdo | 06:59 | |
*** manous has quit IRC | 07:02 | |
*** smeyer has joined #rdo | 07:04 | |
*** manous has joined #rdo | 07:05 | |
*** florianf has joined #rdo | 07:06 | |
*** manous has quit IRC | 07:10 | |
*** martinkopec has joined #rdo | 07:11 | |
*** toanju has joined #rdo | 07:13 | |
*** manous has joined #rdo | 07:13 | |
*** manous has quit IRC | 07:18 | |
*** manous has joined #rdo | 07:20 | |
*** tumble has joined #rdo | 07:20 | |
*** hynekm has joined #rdo | 07:22 | |
*** anilvenkata is now known as anilvenakata_lun | 07:24 | |
imcsk8 | good night! | 07:25 |
*** manous has quit IRC | 07:25 | |
*** jpena|off is now known as jpena | 07:25 | |
jpena | good morning | 07:26 |
*** manous has joined #rdo | 07:27 | |
*** manous has quit IRC | 07:32 | |
*** jlibosva has joined #rdo | 07:34 | |
rdogerrit | Javier Peña proposed DLRN: Parallel mock builds in DLRN http://review.rdoproject.org/r/1806 | 07:34 |
rdogerrit | Javier Peña proposed DLRN: rsync all files/dirs in a single command http://review.rdoproject.org/r/1772 | 07:34 |
*** manous has joined #rdo | 07:34 | |
number80 | o/ | 07:37 |
*** manous has quit IRC | 07:39 | |
*** abehl has joined #rdo | 07:40 | |
*** abehl has quit IRC | 07:41 | |
*** manous has joined #rdo | 07:42 | |
*** pgadiya has joined #rdo | 07:44 | |
*** pgadiya has quit IRC | 07:44 | |
*** amoralej|off is now known as amoralej | 07:44 | |
amoralej | good morning | 07:44 |
*** Guest89739 has quit IRC | 07:45 | |
*** ifarkas_afk is now known as ifarkas | 07:46 | |
*** saneax has joined #rdo | 07:46 | |
*** saneax is now known as Guest34085 | 07:47 | |
*** manous has quit IRC | 07:47 | |
*** gildub_ has joined #rdo | 07:48 | |
*** jpich has joined #rdo | 07:49 | |
*** ohamada has joined #rdo | 07:49 | |
*** gildub has quit IRC | 07:49 | |
*** milan has joined #rdo | 07:50 | |
*** hewbrocca is now known as hewbrocca-afk | 07:54 | |
flepied | good morning | 07:56 |
*** manous has joined #rdo | 07:59 | |
flepied | so we are blocked by swift/puppet-swift in the RDO promotion? | 08:00 |
*** smeyer has quit IRC | 08:01 | |
rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ http://tinyurl.com/gud2vup |#| Build failure on centos7-master/current: swift: http://trunk.rdoproject.org/centos7-master/report.html | 08:03 |
*** Goneri has joined #rdo | 08:04 | |
*** manous has quit IRC | 08:04 | |
amoralej | flepied, i'm checking puppet-openstack-scenario003 issue | 08:06 |
amoralej | that's not identified, right? | 08:07 |
*** athomas has joined #rdo | 08:07 | |
*** manous has joined #rdo | 08:07 | |
*** social has joined #rdo | 08:08 | |
*** nstrug has quit IRC | 08:11 | |
*** shardy has joined #rdo | 08:11 | |
*** manous has quit IRC | 08:12 | |
*** danpawlik has joined #rdo | 08:12 | |
*** ade_b has joined #rdo | 08:13 | |
*** martinkopec has quit IRC | 08:13 | |
*** lucas-dinner is now known as lucasagomes | 08:15 | |
*** pnavarro has joined #rdo | 08:15 | |
*** manous has joined #rdo | 08:16 | |
*** Amita has quit IRC | 08:16 | |
*** dbecker has joined #rdo | 08:20 | |
*** jlibosva has quit IRC | 08:22 | |
flepied | amoralej: yes it seems | 08:23 |
amoralej | i'm checking, it looks like a issue introduced by os-vif... | 08:23 |
*** nstrug has joined #rdo | 08:24 | |
*** Amita has joined #rdo | 08:27 | |
*** pnavarro has quit IRC | 08:29 | |
*** hewbrocca-afk is now known as hewbrocca | 08:29 | |
*** jlibosva has joined #rdo | 08:31 | |
rdogerrit | Javier Peña proposed DLRN: Add option in gitrepo driver to determine source branch from spec http://review.rdoproject.org/r/1818 | 08:31 |
rdogerrit | Javier Peña created DLRN: Allow package name substitution in gitrepo driver http://review.rdoproject.org/r/1842 | 08:31 |
*** Amita is now known as Amita|testday | 08:31 | |
*** anilvenakata_lun has quit IRC | 08:34 | |
rdogerrit | rdo-trunk created openstack/oslo-messaging-distgit: python-oslo-messaging: failed to build 94c958c http://review.rdoproject.org/r/1843 | 08:35 |
*** kozhukalov_ has joined #rdo | 08:35 | |
*** iranzo has quit IRC | 08:35 | |
*** gfidente has joined #rdo | 08:37 | |
*** gfidente has quit IRC | 08:37 | |
*** gfidente has joined #rdo | 08:37 | |
*** anilvenakata_lun has joined #rdo | 08:37 | |
*** numans has joined #rdo | 08:37 | |
*** anilvenakata_lun is now known as anilvenkata | 08:37 | |
*** Goneri has quit IRC | 08:38 | |
*** kozhukalov has quit IRC | 08:38 | |
*** kozhukalov_ is now known as kozhukalov | 08:38 | |
*** joelinux73 has quit IRC | 08:41 | |
*** anshul has joined #rdo | 08:43 | |
*** ade_b has quit IRC | 08:46 | |
*** ihrachys has joined #rdo | 08:47 | |
*** paragan has quit IRC | 08:47 | |
*** oshvartz has joined #rdo | 08:49 | |
*** dtantsur|afk is now known as dtantsur | 08:52 | |
*** d0ugal has quit IRC | 08:52 | |
*** Goneri has joined #rdo | 08:53 | |
*** gszasz has joined #rdo | 08:54 | |
*** athomas has quit IRC | 08:58 | |
*** iberezovskiy|afk is now known as iberezovskiy | 08:58 | |
*** numans has quit IRC | 09:01 | |
*** ade_b has joined #rdo | 09:01 | |
misc | a/win 51 | 09:02 |
*** athomas has joined #rdo | 09:03 | |
*** smeyer has joined #rdo | 09:07 | |
*** dgurtner has joined #rdo | 09:09 | |
*** dgurtner has joined #rdo | 09:09 | |
*** smeyer has quit IRC | 09:10 | |
*** panda|zz is now known as panda | 09:12 | |
*** pkovar has joined #rdo | 09:13 | |
*** numans has joined #rdo | 09:13 | |
flepied | jpena: are we lagging in dlrn master? python-swift has not been rebuilt it seems. | 09:17 |
*** limao has quit IRC | 09:18 | |
jpena | flepied: I'm checking. It looks like the rdoinfo change yesterday (updating tags for many projects) has started a build storm, with a lot of commits being built | 09:18 |
jpena | I hope it won't take long to catch up | 09:20 |
rdogerrit | Javier Peña proposed DLRN: Allow package name substitution in gitrepo driver http://review.rdoproject.org/r/1842 | 09:21 |
flepied | jpena: also I don't see jenkins result in your patchset https://review.rdoproject.org/r/#/c/1844/ | 09:24 |
jpena | flepied: maybe it's not checking commits to the newton-rdo branch? Let me check | 09:25 |
flepied | jpena: yes can be the case | 09:26 |
flepied | jpena: the jobs on the stable branches are being worked out by fbo and number80 | 09:26 |
*** hynekm has quit IRC | 09:27 | |
number80 | jpena: DLRN should not be checking newton-rdo branch | 09:28 |
number80 | it's not even in rdoinfo | 09:28 |
*** mosulica has joined #rdo | 09:28 | |
jpena | number80: but we're using it for 2 projects: oslo-messaging and mistralclient | 09:29 |
number80 | how? | 09:29 |
jpena | when using tagged releases, we needed a spec that was before rpm-master | 09:29 |
number80 | ... | 09:29 |
jpena | because some changes were not merged yet | 09:30 |
jpena | and it looks like newton-rdo was chosen | 09:30 |
number80 | we haven't synced branches yet | 09:30 |
number80 | damn, I'm taking over tripleo-ui package and get this done | 09:31 |
jpena | number80: see https://review.rdoproject.org/r/1784 and https://review.rdoproject.org/r/1786 | 09:31 |
jpena | maybe we can switch oslo-messaging back to rpm-master | 09:32 |
jpena | actually that sounds better to me | 09:32 |
number80 | that's why I was not enthusiastic with -uc branches, we're just workaround upstream flawed process with oslo libs | 09:33 |
number80 | well, not that we can do much about it | 09:34 |
rdogerrit | Javier Peña created rdoinfo: Unpin distgit branch for oslo-messaging in newton-uc http://review.rdoproject.org/r/1845 | 09:34 |
*** mrunge has joined #rdo | 09:41 | |
*** martinkopec has joined #rdo | 09:43 | |
*** manous has quit IRC | 09:45 | |
rdobot | [sensu] NEW: master.monitoring.rdoproject.org - check-delorean-newton-current @ Error |#| Build failure on centos7-master/current: swift, oslo.messaging: http://trunk.rdoproject.org/centos7-master/report.html | 10:03 |
*** paragan has joined #rdo | 10:04 | |
*** Poornima has joined #rdo | 10:06 | |
*** Ryjedo_ has joined #rdo | 10:11 | |
*** Ryjedo has quit IRC | 10:12 | |
*** Ryjedo_ is now known as Ryjedo | 10:12 | |
*** jlibosva has quit IRC | 10:13 | |
*** coolsvap is now known as _coolsvap_ | 10:13 | |
*** mvk has quit IRC | 10:14 | |
*** mvk has joined #rdo | 10:14 | |
*** tosky has joined #rdo | 10:15 | |
*** jfjoly has joined #rdo | 10:18 | |
*** dtantsur is now known as dtantsur|brb | 10:20 | |
*** leanderthal|afk is now known as leanderthal | 10:27 | |
rook | wfoster can we send dmsimard rough guesses on our EK needs -- for Browbeat in CI. | 10:27 |
rook | maybe via email | 10:28 |
*** pkovar has quit IRC | 10:34 | |
number80 | jruzicka: could you give a look at RDO-NEWTON reviews ? | 10:38 |
* number80 has his hands full w/ 3o-ui | 10:39 | |
*** hlkv6 has quit IRC | 10:43 | |
*** hlkv6 has joined #rdo | 10:44 | |
*** manous has joined #rdo | 10:49 | |
number80 | error: Unable to create immutable header region. | 10:52 |
number80 | I think I'm getting lunch | 10:53 |
*** manous has quit IRC | 10:54 | |
*** manous has joined #rdo | 10:58 | |
social | ihrachys: :( the ovs issue is still not fixed? | 10:59 |
ihrachys | social: which one? | 10:59 |
social | oh maybe I just have too old delorean build | 11:00 |
social | ihrachys: brs not created with secure mode | 11:00 |
ihrachys | social: not sure what you refer to | 11:00 |
ihrachys | social: do you have a bug in place? | 11:00 |
social | https://review.openstack.org/#/c/348889/ | 11:00 |
*** manous has quit IRC | 11:03 | |
*** jlibosva has joined #rdo | 11:04 | |
ihrachys | social: we probably did not have proper people on reviewing it | 11:04 |
ihrachys | it's funny that it's 1 line of a fix and 200+ of tests :) | 11:04 |
*** manous has joined #rdo | 11:05 | |
dmsimard | jpena: ugh, wait. That rdoinfo bump will have dlrn build every commit for the project up until the tag ? | 11:09 |
social | ihrachys: pretty please, it's kinda blocking me | 11:09 |
ihrachys | social: I will raise it in our team, and currently reviewing it | 11:10 |
dmsimard | I mean, it makes sense that dlrn would do that but I didn't realize that until now. Merging rdoinfo bumps can really mean a LOT of commits will have to be built (I guess like now) | 11:10 |
jpena | dmsimard: I'm afraid so. The only fixes I can think of are a) run it with --head-only after an rdoinfo bump, or b) remove old commits from the db | 11:10 |
jpena | or changing dlrn to behave as --head-only when switching branches | 11:10 |
dmsimard | Ok.. and what's the story with the messaging ftbfs ? | 11:11 |
jpena | after setting a newer tag, we need to switch distro-branch back to rpm-master (it was pinned to newton-rdo because master required a spec change that was not compatible with the previous tag) | 11:12 |
dmsimard | Ok not sure I understand why master != newton at this point | 11:14 |
*** weshay has joined #rdo | 11:14 | |
dmsimard | Oh, by previous tag you mean between that last tag and master | 11:14 |
jpena | yep | 11:14 |
dmsimard | jpena: so.. re: rdoinfo updates | 11:15 |
dmsimard | I dreamt of a solution | 11:15 |
dmsimard | I'll try and set up dlrn-rpmbuild job to be able to listen in order to build a specific package | 11:16 |
dmsimard | i.e, "recheck rpmbuild python-oslo-messaging" | 11:17 |
dmsimard | Need https://github.com/openstack-packages/DLRN/issues/24 to do that though | 11:17 |
*** thrash|g0ne is now known as thrash | 11:18 | |
slagle | dlrn is building commits from the end of July, is this correct? https://trunk.rdoproject.org/centos7/report.html | 11:18 |
slagle | we still dont yet have a build for puppet-swift https://review.openstack.org/#/c/354476/, that's why i'm asking | 11:18 |
dmsimard | slagle: yes, we updated clients and librairies because they were outdated | 11:18 |
slagle | just saw in scrollback :) | 11:19 |
*** jkilpatr has joined #rdo | 11:19 | |
slagle | should've read first :) | 11:19 |
slagle | is there anyway to bump the puppet-swift to the top of the queue? | 11:19 |
slagle | didn't want to get to deep into the day before trying to get a promote to pass | 11:20 |
jpena | I could stop the current run, build puppet-swift manually, then let it run again. It'll require some cleanup but it could work | 11:20 |
dmsimard | jpena: but we also need that messaging fix to merge to fix ftbfs | 11:21 |
dmsimard | Or we won't have consistent | 11:21 |
jpena | dmsimard: yep, waiting for the rdoinfo review | 11:21 |
dmsimard | +3 | 11:22 |
amoralej | slagle, we still have an issue with scenario003 also, right? | 11:22 |
amoralej | we are hitting it in rdo-ci also | 11:22 |
dmsimard | amoralej: the nova one ? | 11:22 |
jpena | ok, I'll stop the worker and do some hacking | 11:22 |
amoralej | i'm investigating | 11:22 |
amoralej | yes | 11:22 |
rdogerrit | Merged rdoinfo: Unpin distgit branch for oslo-messaging in newton-uc http://review.rdoproject.org/r/1845 | 11:22 |
dmsimard | amoralej: yeah I didn't find the root cause. When I checked last night, I saw some mysql deadlock errors in neutron-server but not sure if those are fatal. | 11:23 |
dmsimard | amoralej: http://logs.openstack.org/55/354355/3/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/c78e2a5/logs/nova/nova-compute.txt.gz#_2016-08-12_02_42_27_366 | 11:24 |
amoralej | they shouldn't | 11:24 |
amoralej | it retries | 11:24 |
dmsimard | http://logs.openstack.org/55/354355/3/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/c78e2a5/logs/neutron/server.txt.gz#_2016-08-12_02_49_49_497 | 11:24 |
amoralej | and didn't get error | 11:24 |
dmsimard | http://logs.openstack.org/55/354355/3/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/c78e2a5/console.html#_2016-08-12_02_51_47_688517 | 11:24 |
amoralej | i added some notes in the etherpad | 11:24 |
dmsimard | so, part of the reason why I was keen on merging the rdoinfo u-c bump | 11:24 |
dmsimard | was a neutronclient significant update | 11:24 |
dmsimard | we were effectively really not testing was upstream was testing anymore | 11:25 |
*** rbowen has quit IRC | 11:25 | |
dmsimard | it's nothing more than my intuition on that one, though | 11:25 |
dmsimard | nothing to back that up. | 11:25 |
*** ashw has joined #rdo | 11:26 | |
jpena | slagle: it should be openstack-swift, not puppet-swift, right? | 11:26 |
*** jfjoly has quit IRC | 11:27 | |
amoralej | jpena, i think they are talking about the puppet-swift, as a fix was added to fix one of the p-o-i scenarios | 11:27 |
amoralej | but we need the openstack-swift to get consistent link anyway | 11:27 |
jpena | amoralej: ack. puppet-swift rebuilt, openstack-swift rebuilding | 11:27 |
dmsimard | jpena: aye, I merged a ftbfs fix for openstack-swift last night | 11:28 |
amoralej | wouldn't be a good idea to run head-only for pinned packages? | 11:28 |
dmsimard | more or less | 11:29 |
dmsimard | it's really fairly the same logic as pinning to a branch | 11:29 |
*** jaosorior has quit IRC | 11:29 | |
dmsimard | the granularity of building every commit lets us know when something starts failing to build | 11:29 |
amoralej | but as soon as we are pinning packages, that's only partly true | 11:30 |
amoralej | i'd say | 11:30 |
*** jaosorior has joined #rdo | 11:30 | |
*** alexismonville has joined #rdo | 11:30 | |
dmsimard | yeah, but unpinning them would potentially mean building >100 commits into a single build | 11:30 |
dmsimard | I don't have a strong opinion on either direction btw, just discussing. | 11:31 |
jpena | yes, that's why my idea was to behave as --head-only for packages that switched branches | 11:31 |
dmsimard | I don't think we'd be discussing this if there was no performance issues | 11:31 |
dmsimard | which brings me to another topic I want to discuss, probably when apevec is back | 11:31 |
amoralej | right | 11:31 |
weshay | adarazs, is https://etherpad.openstack.org/p/delorean_master_current_issues up to date? | 11:31 |
weshay | dmsimard, morning | 11:32 |
dmsimard | let's be honest with ourselves, the CDN thing is really not working out | 11:32 |
*** maeca1 has joined #rdo | 11:32 | |
*** dpeacock has joined #rdo | 11:32 | |
dmsimard | no one's using it, it's a pain to maintain | 11:32 |
weshay | dmsimard, thought about it some more.. we can create a delorean link for the job that runs every four hours, but we wouldn't upload an image | 11:32 |
amoralej | dmsimard, are there stats about CDN usage | 11:34 |
*** dtantsur|brb is now known as dtantsur | 11:34 | |
dmsimard | weshay: upload the image in that second step of the job ? or the last step ? | 11:34 |
dmsimard | amoralej: limited | 11:34 |
slagle | jpena: thanks | 11:35 |
*** kragniz is now known as Guest66666 | 11:36 | |
dmsimard | jpena, amoralej: so I just thought about the rdoinfo bump some more | 11:36 |
dmsimard | jpena, amoralej: I think we really need to do --head-only because the master branches of the other packages are designed to work with the tag itself, not the versions before it | 11:36 |
EmilienM | hello | 11:36 |
dmsimard | if that makes sense | 11:37 |
amoralej | yeap, a tag correspond to a specific commit, if we really want to test that tag, that's a commit | 11:37 |
dmsimard | i.e, if neutronclient has an update from 4.4 to 5.0, we need to go straight to 5.0 as things maybe couldnt work in 4.5 or whatever | 11:37 |
weshay | dmsimard, second to last.. the pin name needs to exclude tripleo.. as to not confuse folks | 11:38 |
jpena | ok, so that works with my proposal | 11:38 |
amoralej | the nice point of per-commit is to troubleshooting purposes | 11:38 |
dmsimard | amoralej: yeah but in this context it doesn't make sense | 11:38 |
amoralej | in those cases it's useful move to a specific commit | 11:38 |
dmsimard | amoralej: HOWEVER, we absolutely need to test the rdoinfo bumps | 11:38 |
dmsimard | with a dlrn-rpmbuild job for the affected packages that are being updated | 11:38 |
* jpena nods | 11:39 | |
dmsimard | with https://github.com/openstack-packages/DLRN/issues/24 | 11:39 |
amoralej | yeap, make sense | 11:39 |
*** jistr has quit IRC | 11:39 | |
jpena | dmsimard: gooot it, I'll start working on it once we clean up the current newton-uc mess :) | 11:39 |
dmsimard | jpena: yeah, sure, it's already so early for me but I couldn't sleep anymore | 11:40 |
dmsimard | jpena: I'll go get some caffeine and work on the job | 11:40 |
weshay | dmsimard, what should the soft link name be? | 11:40 |
dmsimard | weshay: ok i'll take another look at the review after I properly wake up | 11:40 |
*** jruzicka has quit IRC | 11:40 | |
jpena | ok, we have openstack-swift, puppet-swift and python-oslo-messaging rebuilt. I'm running another DLRN run with --head-only, so all clients catch up to the tagged release, and we're good for another CI run | 11:41 |
* dmsimard cross fingers | 11:42 | |
dmsimard | afk a bit | 11:42 |
EmilienM | so we have a nova error on scenario003 | 11:42 |
dmsimard | EmilienM: amoralej investigated it | 11:43 |
dmsimard | EmilienM: and me | 11:43 |
dmsimard | EmilienM: not sure what's the problem tbh, but we are hoping that a upper-constraints update will resolve it. | 11:43 |
EmilienM | ok | 11:43 |
amoralej | i'm still on it | 11:43 |
amoralej | trying to reproduce it in a local environment | 11:43 |
dmsimard | amoralej: oh, you can use weirdo for that :) | 11:44 |
dmsimard | amoralej: can tell you how if you'd like | 11:44 |
EmilienM | dmsimard: did you update something in RDO ? can I try with latest rpo? | 11:44 |
amoralej | i'm trying with run-test.sh | 11:44 |
dmsimard | EmilienM: dlrn is still catching up | 11:44 |
amoralej | is documented somewhere dmsimard? | 11:44 |
EmilienM | maybe we can ask networking folks | 11:44 |
dmsimard | amoralej: yup, hang on | 11:45 |
amoralej | i was trying to have a more clear idea about the failure | 11:45 |
*** rbowen has joined #rdo | 11:45 | |
*** ChanServ sets mode: +o rbowen | 11:45 | |
amoralej | before asking network guys | 11:45 |
amoralej | in fact, i'm not sure if it's nova or neutron issue yet | 11:46 |
EmilienM | well, it's highly related | 11:47 |
amoralej | yes, it is | 11:48 |
dmsimard | amoralej: https://gist.github.com/dmsimard/1e1b6fb7123c5c50519c065299f2e47c | 11:48 |
EmilienM | regarding the urgency to promote, I would ping someone from neutron folks | 11:48 |
pabelanger | dmsimard: did you see the email from tristanC about gerrit_connections? | 11:48 |
amoralej | that easy? | 11:48 |
amoralej | cool | 11:48 |
dmsimard | amoralej: that's the whole point of weirdo ! :D | 11:48 |
EmilienM | ihrachys: good afternoon, you or someone else from your team could help | 11:48 |
EmilienM | ihrachys: we're trying to promote RDO CI and we currently have a blocker that might be related to neutron or nova | 11:49 |
dmsimard | pabelanger: I saw the email but did not read it yet | 11:49 |
dmsimard | pabelanger: I haven't woken up properly yet, just got at the keyboard to discuss some crisis stuff | 11:49 |
EmilienM | ihrachys: we have 2 tempests tests failing when using linuxbridge, more context is documented at the end of https://etherpad.openstack.org/p/delorean_master_current_issues | 11:49 |
ihrachys | EmilienM: details? | 11:49 |
ihrachys | looking | 11:49 |
pabelanger | dmsimard: ack | 11:50 |
ihrachys | EmilienM: yeah, it's definitely nova issue | 11:50 |
ihrachys | EmilienM: good news it's fixed now in their gate | 11:50 |
EmilienM | ihrachys: ok, what was the fix? | 11:50 |
ihrachys | EmilienM: so just a matter of a new nova package | 11:50 |
dmsimard | sweet | 11:50 |
ihrachys | EmilienM: sec | 11:50 |
EmilienM | I'm glad I asked | 11:50 |
ihrachys | EmilienM: https://review.openstack.org/#/c/354143/ | 11:50 |
EmilienM | you're a machine | 11:51 |
EmilienM | thanks :) | 11:51 |
EmilienM | so we need delorean to catchup and have this patch | 11:51 |
EmilienM | and we can try again | 11:51 |
ihrachys | I am. please insert a coin. | 11:51 |
EmilienM | ihrachys: dude, thanks a lot | 11:51 |
EmilienM | ok it's in delorean | 11:51 |
* hewbrocca inserts beer | 11:51 | |
EmilienM | see 3f642cff8ea837705568f1281f66fa54c7c59ea4 | 11:52 |
*** ashw has quit IRC | 11:52 | |
dmsimard | we have consistent right now | 11:52 |
EmilienM | I'm going to try again | 11:52 |
dmsimard | but it's probably a weird consistent | 11:52 |
ihrachys | EmilienM: happy to be helpful | 11:53 |
EmilienM | I ran puppet CI promotion, let's see | 11:53 |
adarazs | weshay: good morning, I just got back from lunch; so I'm waiting for the compute/tempest issue (39) to get solved before retriggering a build, otherwise we might be good to go and yes it's up to date afaik | 11:53 |
dmsimard | jpena: head is still building right now, right ? | 11:54 |
jpena | dmsimard: yep | 11:54 |
*** anilvenkata has quit IRC | 11:54 | |
jpena | I'll let you know when it's done | 11:54 |
adarazs | dmsimard: will you trigger the promotion job? | 11:54 |
dmsimard | adarazs: dlrn is in a weird state right now because we interrupted builds to priorize some packages | 11:55 |
jpena | yes, let's please give dlrn some minutes to catch up | 11:55 |
dmsimard | adarazs: jpena is rebuilding the tip of the packages right now, I'd rather wait | 11:55 |
adarazs | dmsimard: yeah, I meant after jpena finishes :) | 11:55 |
adarazs | or rather after dlrn finishes. | 11:56 |
*** jfjoly has joined #rdo | 11:56 | |
dmsimard | adarazs: yes, sure | 11:56 |
*** Poornima has quit IRC | 11:57 | |
*** Guest34085 is now known as saneax_AFK | 12:00 | |
*** lucasagomes is now known as lucas-hungry | 12:00 | |
*** ccamacho is now known as ccamacho|lunch | 12:02 | |
weshay | adarazs, cool.. fyi.. I'll post new diagrams for this https://review.gerrithub.io/#/c/287144/ | 12:02 |
*** morazi has joined #rdo | 12:04 | |
*** jayg|g0n3 is now known as jayg | 12:06 | |
*** jfjoly has quit IRC | 12:06 | |
*** pkovar has joined #rdo | 12:07 | |
*** jfjoly has joined #rdo | 12:08 | |
*** morazi has quit IRC | 12:08 | |
*** morazi has joined #rdo | 12:09 | |
*** ohamada has quit IRC | 12:13 | |
*** rhallisey has joined #rdo | 12:15 | |
*** ohamada has joined #rdo | 12:15 | |
*** jruzicka has joined #rdo | 12:17 | |
*** rbowen has quit IRC | 12:18 | |
*** amoralej is now known as amoralej|lunch | 12:23 | |
*** jistr has joined #rdo | 12:24 | |
* jruzicka gets pwned librpm again and again | 12:24 | |
jruzicka | *by librpm | 12:24 |
* jpena goes to lunch, will check dlrn execution from time to time | 12:25 | |
flepied | why the rdo-delorean-promote-master is disabled? | 12:25 |
*** jpena is now known as jpena|lunch | 12:25 | |
*** fultonj has joined #rdo | 12:27 | |
flepied | adarazs: did you disable it to wait for the right packages? ^^ | 12:30 |
*** jruzicka has quit IRC | 12:31 | |
*** jistr has quit IRC | 12:31 | |
adarazs | flepied: I didn't disable it, but weshay or dmsimard might have. | 12:31 |
adarazs | flepied: and yeah, we're waiting for dlrn to finish building stuff. | 12:31 |
flepied | adarazs: ok | 12:32 |
*** jistr has joined #rdo | 12:32 | |
*** jruzicka has joined #rdo | 12:33 | |
*** chlong|mtg has quit IRC | 12:33 | |
weshay | adarazs, I updated https://review.openstack.org/#/c/354365 | 12:35 |
*** jfjoly has quit IRC | 12:35 | |
*** fultonj has quit IRC | 12:35 | |
*** jfjoly has joined #rdo | 12:36 | |
*** fultonj has joined #rdo | 12:36 | |
adarazs | weshay: and what about my comments, apart from the semicolon stuff? :) | 12:37 |
weshay | adarazs, oh.. /me looks | 12:37 |
*** dbecker has quit IRC | 12:37 | |
*** gildub_ has quit IRC | 12:38 | |
weshay | adarazs, a different job type? | 12:38 |
weshay | adarazs, dmsimard wants the job that is running every 4 hours RDO-CI Master, to promote a delorean hash | 12:38 |
weshay | so.. it's a pipeline but not for tripleo | 12:39 |
*** jistr has quit IRC | 12:41 | |
*** jruzicka has quit IRC | 12:41 | |
adarazs | weshay: the purpose of this patch is not very clear to me to begin with, but what I didn't like is that we're starting to magically override env variables from outside, when currently we are setting them based on the job type. so if we need another job type, then let's add it rather than mess around with vars. | 12:41 |
EmilienM | dmsimard: https://review.openstack.org/#/c/354355/ | 12:43 |
EmilienM | green | 12:43 |
weshay | adarazs, the purpose of the patch is to implement the diagram | 12:43 |
weshay | adarazs, if you disagree w/ the diagram.. please be specific | 12:43 |
EmilienM | dmsimard: some puppet4 error, i'll fix them today | 12:43 |
adarazs | weshay: I don't disagree with the diagram, I'm disagreeing with how you're hacking in the extra parts :) I'll comment some more and be more specific in a sec. | 12:44 |
weshay | setting variables is not a hack | 12:44 |
EmilienM | amoralej|lunch: fyi ^ | 12:44 |
weshay | I'm removing formally hardcoded values | 12:45 |
adarazs | weshay: so far each job type is setting the $LOCATION itself. | 12:45 |
*** rlandy has joined #rdo | 12:45 | |
weshay | only the gethash job sets the variable | 12:45 |
adarazs | weshay: yeah, but now we have a 1:1 relationship with jobtype and locations, which is kind of nice. I think we should have a different job type to promote different stuff. | 12:45 |
weshay | there are other variables used throughout the promote set in that job | 12:45 |
weshay | adarazs, I don't think you understand the workflow.. can you review the image role change | 12:46 |
*** ashw has joined #rdo | 12:46 | |
adarazs | weshay: can you link me the diagram again? | 12:46 |
*** rbowen has joined #rdo | 12:50 | |
*** ChanServ sets mode: +o rbowen | 12:50 | |
*** anshul has quit IRC | 12:51 | |
*** ohochman has joined #rdo | 12:52 | |
*** dbecker has joined #rdo | 12:52 | |
*** links has quit IRC | 12:53 | |
*** aortega has joined #rdo | 12:55 | |
*** amuller has joined #rdo | 12:56 | |
*** jdob has quit IRC | 12:56 | |
*** Amita|testday has quit IRC | 12:56 | |
*** jdob has joined #rdo | 12:57 | |
*** jruzicka has joined #rdo | 12:58 | |
*** amoralej|lunch is now known as amoralej | 12:59 | |
amoralej | let's see if we can get a promotion today EmilienM | 12:59 |
EmilienM | puppet ci is being promoted atm | 13:00 |
EmilienM | I hope it will be the same for rdo | 13:00 |
amoralej | we'll wait until current dlrn run is done | 13:00 |
*** jlibosva has quit IRC | 13:00 | |
*** lucas-hungry is now known as lucasagomes | 13:00 | |
*** _elmiko is now known as elmiko | 13:00 | |
*** iranzo has joined #rdo | 13:01 | |
*** jlibosva has joined #rdo | 13:03 | |
*** jistr has joined #rdo | 13:03 | |
*** jmelvin has joined #rdo | 13:03 | |
*** pkovar has quit IRC | 13:04 | |
*** jfjoly has quit IRC | 13:05 | |
*** jruzicka has quit IRC | 13:07 | |
dmsimard | EmilienM: you're promoting a weird build | 13:07 |
EmilienM | why? | 13:07 |
EmilienM | I took latest | 13:07 |
dmsimard | we interrupted builds to priorize and build heads | 13:08 |
dmsimard | http://paste.openstack.org/show/555983/ | 13:08 |
dmsimard | but you mostly ignored me so /me shrugs | 13:08 |
dmsimard | that's why our promote job is disabled | 13:08 |
dmsimard | we will be able to kick it soon | 13:08 |
dmsimard | In fact from looking at the builds I would say now could be a good time | 13:09 |
dmsimard | jpena|lunch: what do you think ? | 13:09 |
dmsimard | looks like all libs finished building | 13:10 |
dmsimard | I'll kick it. | 13:10 |
EmilienM | dmsimard: I didn't ignore you dude, I just missed the information. | 13:10 |
dmsimard | EmilienM: no offense taken :P | 13:11 |
EmilienM | is 2a/04 good? | 13:11 |
EmilienM | https://trunk.rdoproject.org/centos7-master/2a/04/2a040c25781dd5f907842a862ca4df5c55f56f11_99d1c446/ | 13:11 |
dmsimard | EmilienM: that's the one I'm targetting right now | 13:12 |
EmilienM | k | 13:12 |
*** vimal has quit IRC | 13:12 | |
*** jistr has quit IRC | 13:13 | |
*** manous has quit IRC | 13:13 | |
*** fragatina has joined #rdo | 13:13 | |
*** ccamacho|lunch is now known as ccamacho | 13:14 | |
*** unclemarc has joined #rdo | 13:14 | |
*** sdake has joined #rdo | 13:14 | |
*** jfjoly has joined #rdo | 13:14 | |
*** jcoufal has joined #rdo | 13:15 | |
*** vimal has joined #rdo | 13:16 | |
*** tbarron|gone is now known as tbarron | 13:16 | |
*** fragatina has quit IRC | 13:18 | |
*** dprince has joined #rdo | 13:18 | |
*** paragan has quit IRC | 13:19 | |
*** dprince has quit IRC | 13:19 | |
*** dustins has joined #rdo | 13:19 | |
*** dprince has joined #rdo | 13:20 | |
*** paragan has joined #rdo | 13:20 | |
*** rbowen is now known as DrBacchus | 13:23 | |
*** ppowell has joined #rdo | 13:23 | |
*** vimal has quit IRC | 13:26 | |
*** Amita|testday has joined #rdo | 13:26 | |
*** manous has joined #rdo | 13:27 | |
*** Amita|testday has quit IRC | 13:27 | |
*** Amita has joined #rdo | 13:28 | |
*** jruzicka has joined #rdo | 13:29 | |
*** jpena|lunch is now known as jpena | 13:31 | |
jpena | build complete (3 mins ago), time for CI! | 13:31 |
*** chandankumar has quit IRC | 13:31 | |
*** manous has quit IRC | 13:32 | |
jpena | EmilienM 2a/04 is quite good, we've only had service updates after that (neutron, nova...) plus puppet-neutron | 13:32 |
jpena | all pinned versions should be there | 13:32 |
EmilienM | jpena: I'll promote to latest consistent in a few | 13:32 |
*** jfjoly has quit IRC | 13:32 | |
EmilienM | weirdo jobs for puppet should be good I think | 13:33 |
EmilienM | stars are aligned I guess :P | 13:33 |
*** manous has joined #rdo | 13:33 | |
* jpena crosses his fingers | 13:33 | |
jpena | btw, purge complete, we have > 100GB of free space in the DLRN instance | 13:34 |
*** jruzicka has quit IRC | 13:34 | |
*** manous has quit IRC | 13:38 | |
*** rodrigods has quit IRC | 13:39 | |
*** rodrigods has joined #rdo | 13:40 | |
*** trozet has quit IRC | 13:41 | |
*** manous has joined #rdo | 13:42 | |
tristanC | hey dmsimard, i'll prepare the cherry-pick for gerrit_connections setting | 13:44 |
dmsimard | tristanC: cool | 13:44 |
dmsimard | pabelanger: ^ | 13:44 |
dmsimard | pabelanger: did you ever create the launchpad account ? | 13:44 |
*** sdake has quit IRC | 13:46 | |
*** manous has quit IRC | 13:47 | |
* social stares at http://trunk.rdoproject.org/centos7/report.html waiting for new openstack-neutron build | 13:49 | |
dmsimard | social: why ? | 13:49 |
*** manous has joined #rdo | 13:49 | |
social | dmsimard: https://review.openstack.org/#/c/348889/ | 13:49 |
social | ihrachys: thanks a lot | 13:49 |
dmsimard | social: what's that fix | 13:50 |
*** trozet has joined #rdo | 13:50 | |
social | dmsimard: machines randomly loosing ips | 13:50 |
*** jruzicka has joined #rdo | 13:50 | |
dmsimard | ah, that's inconvenient :) | 13:50 |
*** Son_Goku has quit IRC | 13:50 | |
*** d0ugal has joined #rdo | 13:51 | |
pabelanger | dmsimard: I have not, did you want to do it now? | 13:52 |
dmsimard | I'm on fires left and right, if you have time to do it please do :) | 13:52 |
social | dmsimard: is there way to see queue for delorean? | 13:55 |
dmsimard | no. | 13:55 |
dmsimard | but it's fairly up to date, it shouldn't take that long. | 13:55 |
*** pnavarro has joined #rdo | 13:56 | |
*** eharney has joined #rdo | 13:56 | |
*** pkovar has joined #rdo | 13:56 | |
*** sdake has joined #rdo | 13:56 | |
*** rdogerrit has quit IRC | 13:57 | |
EmilienM | dmsimard, social: they were not loosing IP, it's just, under heavy load, a node can miss probes from ovsdb | 13:59 |
tristanC | dmsimard: pabelanger: change is now applied on review.rdoproject.org, the configuration template is added at the top of sfconfig.yaml, all you have to do is follow this doc now: http://softwarefactory-project.io/r/#/c/4546/2/docs/faqs.rst | 13:59 |
*** myoung is now known as myoung|mtg | 13:59 | |
EmilienM | so I guess the vswitch can't handle all traffic | 13:59 |
EmilienM | but VMs do not loose IP. | 14:00 |
*** imcleod has joined #rdo | 14:00 | |
*** dustins has quit IRC | 14:00 | |
pabelanger | tristanC: great! | 14:01 |
jpena | social: the last commit built (vpnaas) was merged 36 mins ago | 14:02 |
social | EmilienM: after that ovs cleans up | 14:02 |
*** pkovar has quit IRC | 14:03 | |
jruzicka | dmsimard, oh noes, the bot doesn't announce new reviews anymore? | 14:03 |
dmsimard | supposed to, maybe it's not around | 14:03 |
dmsimard | yeah it's crashed | 14:04 |
dmsimard | tristanC: sometimes the gerritbot isn't restarted properly | 14:04 |
dmsimard | tristanC: iirc is also restarts even if there are no config changes | 14:04 |
*** danpawlik has quit IRC | 14:04 | |
honza | number80: hi! let me know if i can help with tripleo-ui packaging today | 14:04 |
number80 | honza: I'm currently fixing openstack-tripleo-ui build | 14:04 |
*** pradk has quit IRC | 14:05 | |
*** rdogerrit has joined #rdo | 14:05 | |
number80 | it breaks RPM upper caps on number of files in a single package | 14:05 |
dmsimard | tristanC: http://paste.openstack.org/show/556058/ | 14:05 |
*** dmsimard sets mode: +v rdogerrit | 14:05 | |
dmsimard | jruzicka: thanks, rdogerrit bot is back now :) | 14:05 |
jpena | number80: is there such a limit? o_O | 14:05 |
*** manous has quit IRC | 14:05 | |
number80 | jpena: yes, and I was confirmed it will not be lifted in the RPM shipped by EL7 | 14:06 |
jpena | number80: just curious, how many files is that? | 14:06 |
rdogerrit | Javier Peña created DLRN: Use zuul-cloner in tests to clone rdoinfo, if available http://review.rdoproject.org/r/1848 | 14:07 |
honza | oh my | 14:07 |
*** bnemec is now known as beekneemech | 14:07 | |
*** pradk has joined #rdo | 14:07 | |
number80 | jpena: theorically an uint64 max, but it's been set lower, I'll find the definition | 14:08 |
hewbrocca | that's a lot of files | 14:09 |
number80 | http://rpm.org/api/4.4.2.2/package_8c-source.html#l00086 | 14:10 |
number80 | when RPM header > 16 MB, RPM goes boom | 14:11 |
number80 | solution is to split into subpackages | 14:11 |
*** satya4ever has quit IRC | 14:13 | |
jpena | 16 MB header is an awful lot of files :-/ | 14:14 |
number80 | well, node.js | 14:15 |
*** iranzo has quit IRC | 14:16 | |
*** jlibosva has quit IRC | 14:16 | |
number80 | we have libraries that are copied multiple times | 14:16 |
*** _coolsvap_ is now known as coolsvap | 14:16 | |
*** jlibosva has joined #rdo | 14:17 | |
*** pnavarro has quit IRC | 14:19 | |
*** manous has joined #rdo | 14:21 | |
jruzicka | number80, https://review.rdoproject.org/r/#/c/1847 | 14:23 |
number80 | jruzicka: ack | 14:24 |
number80 | that's huge changeset :) | 14:24 |
*** manous has quit IRC | 14:26 | |
*** manous has joined #rdo | 14:27 | |
dmsimard | jpena, weshay, adarazs: jobs are in progress: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/ | 14:27 |
dmsimard | well, the "actual" jobs | 14:27 |
*** saneax_AFK is now known as saneax | 14:28 | |
*** iberezovskiy is now known as iberezovskiy|off | 14:28 | |
*** saneax is now known as Guest47565 | 14:28 | |
jpena | social: neutron rpm is now ready | 14:29 |
social | jpena: already using it | 14:30 |
rdogerrit | Javier Peña proposed DLRN: Use zuul-cloner in tests to clone rdoinfo, if available http://review.rdoproject.org/r/1848 | 14:30 |
pabelanger | dmsimard: did you get the launchpad user? | 14:32 |
*** manous has quit IRC | 14:32 | |
pabelanger | if not I can do it | 14:32 |
*** pnavarro has joined #rdo | 14:32 | |
*** paramite has quit IRC | 14:33 | |
*** manous has joined #rdo | 14:34 | |
*** myoung|mtg is now known as myoung | 14:34 | |
*** Amita has quit IRC | 14:35 | |
*** jistr has joined #rdo | 14:39 | |
*** pkovar has joined #rdo | 14:40 | |
jpena | have you seen this? -> http://lists.openstack.org/pipermail/openstack-dev/2016-August/101539.html | 14:40 |
jpena | we're supposed to set lock_path to the same directory for cinder and nova :? | 14:41 |
EmilienM | o_O | 14:42 |
*** Amita has joined #rdo | 14:42 | |
jpena | of course, it all started with a patch to devstack | 14:42 |
*** Amita has quit IRC | 14:42 | |
EmilienM | you're allowed to troll, it's Friday. | 14:43 |
number80 | let's merge everything into devstack | 14:43 |
DrBacchus | Let's rewrite packstack in Go. | 14:44 |
*** pkovar has quit IRC | 14:44 | |
number80 | Like that, we can include kubernetes \o/ | 14:44 |
number80 | pack8s | 14:45 |
social | why not rust | 14:45 |
honza | ha | 14:46 |
number80 | social: If I do a keynote, I'll announce that we'll rewrite OpenStack in Rust | 14:46 |
number80 | wait, Intel already did that | 14:47 |
misc | no, I think openstack requires a new language | 14:48 |
*** rdas has quit IRC | 14:49 | |
number80 | misc: ChuckD ? which would be what to Chuck Norris what ArnoldC is to the governator https://github.com/lhartikk/ArnoldC | 14:49 |
jruzicka | number80, with huge changeset comes nice unit tests ;) | 14:50 |
dmsimard | jpena, amoralej: got a tempest failure in one of the packstack scenarios on the current pipeline https://ci.centos.org/artifacts/rdo/weirdo-master-promote-packstack-scenario003/472/packstack/logs/latest/tempest.log.txt.gz | 14:51 |
amoralej | i was just checking... | 14:51 |
dmsimard | could be a flap ? | 14:51 |
dmsimard | it's a timeout .. | 14:51 |
number80 | jruzicka: true | 14:51 |
dmsimard | I'm going to start a generic job on that hash | 14:51 |
*** pnavarro has quit IRC | 14:51 | |
number80 | at least, we have a mitaka promote | 14:52 |
dmsimard | amoralej: https://ci.centos.org/view/rdo/view/weirdo/job/weirdo-generic-packstack-scenario003/46/ running with the same hash, we'll be able to see if it was a fluke | 14:53 |
amoralej | error creating a security-group | 14:53 |
jpena | it finished, 1 minute later | 14:54 |
jpena | no, 10 seconds later | 14:54 |
jpena | [12/Aug/2016 15:41:45] "POST /v2.0/security-groups.json HTTP/1.1" 201 1208 38.932637 | 14:54 |
number80 | dmsimard, amoralej, jpena: would it be possible to have a list of changed packages between two snapshots in CI? | 14:54 |
dmsimard | number80: we recover a rpm -qa in jobs | 14:54 |
dmsimard | so we can compare if needed | 14:55 |
dmsimard | we also recover yum repolist | 14:55 |
amoralej | repodiff? | 14:55 |
number80 | in which job? | 14:55 |
amoralej | in the jobs you can see the hash repo used | 14:56 |
number80 | ok, found it, it's just not displayed in Jenkins directly | 14:56 |
*** martinkopec has quit IRC | 14:58 | |
number80 | only heat, nova and puppet-neutron changed in between that run and the successful previous one | 15:00 |
number80 | (and glance_store) | 15:00 |
jruzicka | I got mad at maintainability of regexp so as an experiment I replaced some of it with parsley grammar and I'm curious how that will suck :) | 15:00 |
amoralej | dmsimard, we are not setting debug in packstack jobs? | 15:01 |
amoralej | ConnectTimeout: Request to http://172.19.2.95:9696/v2.0/security-groups.json timed out | 15:01 |
dmsimard | amoralej: pretty sure the debug logs are in the actual logs | 15:01 |
dmsimard | amoralej: https://ci.centos.org/artifacts/rdo/weirdo-master-promote-packstack-scenario003/472/packstack/logs/latest/manifests/172.19.2.95_compute.pp.log.txt.gz is very verbose :) | 15:01 |
amoralej | i see debug=False | 15:01 |
amoralej | in nova.conf and neutron.conf | 15:01 |
amoralej | i though we add --debut to packstack | 15:02 |
amoralej | that scenario configuration, right? | 15:02 |
jpena | amoralej: so it took 39 seconds for the security-groups call to complete, and it timed out after 30 | 15:03 |
dmsimard | amoralej: I think the debug parameter is for packstack itself, not for the services it configures | 15:03 |
rdogerrit | hguemar created rdo-infra/ansible-role-weirdo-common: Sort output of rpm -qa command for easier diffing http://review.rdoproject.org/r/1849 | 15:04 |
*** ade_b has quit IRC | 15:05 | |
DrBacchus | gchamoul: Sorry about that. I didn't see that you'd already fixed the same thing I was patching. | 15:05 |
*** jlibosva has quit IRC | 15:05 | |
*** Guest47565 is now known as saneax_AFK | 15:05 | |
gchamoul | DrBacchus: well, originally it was not my patch and it was not really fixing the same thing | 15:08 |
amoralej | yes, jpena maybe just slow ci server | 15:08 |
DrBacchus | But that's included in there, and that's all mine was. I would hate to interrupt a much longer-running thing for just that one tweak. | 15:08 |
number80 | jruzicka: since we'll have to parse properly spec files soon, not a bad change | 15:08 |
*** leanderthal is now known as leanderthal|afk | 15:09 | |
jpena | amoralej: could be, it looks like that. | 15:10 |
gchamoul | DrBacchus: no worries! I will propose a new PS and add the rst format instead. originally it was fixing the doc directory name | 15:10 |
amoralej | we should add --os-debug-mode to ci jobs also | 15:10 |
*** ayoung has joined #rdo | 15:10 | |
amoralej | to enable debug in services | 15:10 |
gchamoul | DrBacchus: or feel free do it if you want :-) | 15:10 |
amoralej | i though --debug did it | 15:10 |
jruzicka | number80, pure python .spec parser would be nice | 15:11 |
amoralej | i think debug is a must for ci | 15:11 |
jruzicka | as well as pure python rpm module without dark magic | 15:11 |
*** eharney has quit IRC | 15:15 | |
*** eharney_ has joined #rdo | 15:15 | |
*** jaosorior has quit IRC | 15:15 | |
*** hewbrocca is now known as hewbrocca-afk | 15:15 | |
number80 | jruzicka: it will be soon a requirement :( | 15:16 |
*** eharney_ is now known as eharney | 15:16 | |
amoralej | number80, i did a script some time ago to get the hash repo, versions.csv and packages in delorean-deps.repo for a promotion job | 15:21 |
amoralej | if you are trying to get information from the jobs, it may be useful, it's easy to adapt | 15:22 |
amoralej | https://gist.github.com/amoralej/65117ecdb1056df029454ce35e517b7b | 15:22 |
*** florianf has quit IRC | 15:23 | |
*** pcaruana has quit IRC | 15:23 | |
*** jistr is now known as jistr|call | 15:23 | |
*** eharney_ has joined #rdo | 15:27 | |
number80 | amoralej++ | 15:27 |
number80 | that's useful | 15:27 |
social | gah shouldn't the mistral issue be fixed already? | 15:28 |
amoralej | what are you looking for?, comparing packages between different promotions? | 15:28 |
*** eharney has quit IRC | 15:29 | |
*** beagles is now known as beagles_brb | 15:29 | |
*** florianf has joined #rdo | 15:31 | |
*** ifarkas is now known as ifarkas_afk | 15:33 | |
*** ayoung has quit IRC | 15:33 | |
number80 | honza: if you don't mind, I'll submit a revision of your patchset | 15:35 |
number80 | I fixed the -deps package build | 15:36 |
*** mosulica has quit IRC | 15:36 | |
social | hmm is this new one for mistral? http://paste.fedoraproject.org/406958/16217147/ | 15:37 |
d0ugal | social: Looks like it might be :/ | 15:38 |
number80 | amoralej: comparing stuff between successful and failed runs of a CI job | 15:38 |
number80 | hence my patchset to order rpm -qa output | 15:38 |
*** sdake_ has joined #rdo | 15:39 | |
*** aufi has quit IRC | 15:39 | |
*** numans has quit IRC | 15:40 | |
*** dyasny has quit IRC | 15:40 | |
social | d0ugal: I just bumped delrean to latest repo and did openstack undercloud upgrade :( it fails on db-sync again :( | 15:40 |
* social sad | 15:40 | |
*** tumble has quit IRC | 15:41 | |
*** sdake has quit IRC | 15:41 | |
*** jistr|call is now known as jistr | 15:42 | |
honza | number80: go right ahead | 15:42 |
dmsimard | amoralej, jpena: the generic job passed on the same hash, so it was a fluke https://ci.centos.org/view/rdo/view/weirdo/job/weirdo-generic-packstack-scenario003/46/console | 15:42 |
dmsimard | I will promote manually if the quickstart jobs pass | 15:43 |
amoralej | ok | 15:43 |
jpena | ack | 15:43 |
dmsimard | weshay: fyi all packstack and puppet jobs passed on the promote hash, just waiting for quickstart now | 15:43 |
EmilienM | awesome | 15:43 |
weshay | dmsimard, I see scenario003 as red | 15:43 |
dmsimard | weshay: it's a flap (timeout when creating security group), I ran a one-off job on the same hash and it passed https://ci.centos.org/view/rdo/view/weirdo/job/weirdo-generic-packstack-scenario003/46/console | 15:44 |
adarazs | still then this image won't be promoted with this job :( | 15:44 |
*** jpich has quit IRC | 15:45 | |
dmsimard | adarazs: I'll promote manually if the quickstart jobs pass | 15:46 |
adarazs | ack | 15:46 |
dmsimard | I don't want to run the same job again (even on the same hash) because it just takes too long to complete. | 15:46 |
adarazs | dmsimard: you'll do the testing -> stable image copy as well? | 15:46 |
dmsimard | adarazs: I'll run https://ci.centos.org/job/rdo-promote-upload-master manually yes | 15:46 |
weshay | :( | 15:47 |
weshay | that's fine this time, but this is what we need to avoid. the jobs really all need to pass at the same time | 15:47 |
*** rcernin has quit IRC | 15:47 | |
weshay | dmsimard, I'm sure ooo will fail anyway :P | 15:48 |
dmsimard | weshay: well there are no known issues right now | 15:48 |
dmsimard | weshay: and everything else passed so I'm optimistic | 15:48 |
weshay | k.. /me grabs a bite | 15:48 |
weshay | adarazs, I suppose I can just check the review for updates | 15:49 |
weshay | adarazs, /me wonders re: the deps for this jjb patch though | 15:49 |
weshay | panda, you still here? https://review.openstack.org/#/c/354365/ | 15:49 |
weshay | larsks, ^ | 15:49 |
dmsimard | weshay, adarazs: did you see the ci-users thread re: Jenkins v2 update for cico ? | 15:50 |
weshay | dmsimard, not yet | 15:50 |
weshay | myoung, we're clear to merge https://review.gerrithub.io/#/c/287153/ ? | 15:50 |
*** weshay is now known as weshay_brb | 15:51 | |
adarazs | woot, minimal passed :) | 15:52 |
myoung | weshay_brb I think so...that particular portion (copy of the the newly passing image) is one of the only portions not covered by the images-ci-check gates, as they never promote from the gerrit hooks :) | 15:53 |
myoung | weshay_brb, adarazs, dmsimard: speaking of image build gates... https://review.gerrithub.io/#/c/287168/ | 15:54 |
*** dyasny has joined #rdo | 15:55 | |
*** beagles_brb is now known as beagles | 15:57 | |
imcsk8 | jpena: stil around? | 15:58 |
*** imcleod has quit IRC | 15:59 | |
*** sdake_ has quit IRC | 15:59 | |
flepied | amoralej: is it the same issue with weirdo-master-promote-packstack-scenario003 than in the previous run? | 15:59 |
*** weshay_brb is now known as weshay | 15:59 | |
amoralej | not flepied | 16:00 |
amoralej | former issue is fixed | 16:00 |
*** eharney_ is now known as eharney | 16:00 | |
amoralej | it has been just a timeout in a neutron api operation | 16:00 |
amoralej | we've re-run the job with the same hash and it passed | 16:00 |
dmsimard | flepied: I already ran the packstack 003 job on the same hash and it passed | 16:00 |
dmsimard | flepied: https://ci.centos.org/view/rdo/view/weirdo/job/weirdo-generic-packstack-scenario003/46/console | 16:00 |
weshay | myoung, is that needed for the image build only? | 16:01 |
dmsimard | flepied: I will promote manually if both quickstart jobs pass | 16:01 |
jpena | imcsk8: yep, I'm here | 16:01 |
flepied | dmsimard: ack | 16:01 |
dmsimard | afk food | 16:01 |
amoralej | first tripleo job passed, crossing fingers... | 16:02 |
dmsimard | EmilienM: ALIGN THE STARS! | 16:02 |
EmilienM | dmsimard: doing it | 16:02 |
weshay | lolz | 16:03 |
weshay | EmilienM, we want PONIES | 16:03 |
imcsk8 | jpena: last night i was checking the lp bugs and i noticed that we have a lot of with the "Fix Commited" status that have the review merged. i've written a script that retrieves those bugs and changes their status to "Fix Released" | 16:03 |
dmsimard | weshay: actual EmilienM commit message "Align stars to deploy OpenStack trunk on June 22th" | 16:04 |
adarazs | myoung: reviewed. | 16:04 |
weshay | lolz | 16:04 |
myoung | weshay, it's the jobs that get kicked when images-ci-check appears in the comment text to build images and test with quickstart. dmsimard and I were talking about this earlier in the week (or was it last?) - the validate stuff added to quickstart a few weeks ago bumps job times > max and they fail. | 16:04 |
jpena | imcsk8: oh, cool. So it's not done automatically :) | 16:04 |
dmsimard | jpena: it's supposed to | 16:04 |
dmsimard | jpena: but in my experience it's rather unreliable | 16:04 |
imcsk8 | jpena: just to get a second opinion: i'm willing to deal with false positives but i'm not shure if it's very reckless | 16:04 |
dmsimard | jpena: sometimes the gerrit < - > lp hooks work, sometimes it's not | 16:05 |
dmsimard | ok afk food for real before I start digesting myself | 16:05 |
imcsk8 | i also thught that, but it's not being done | 16:05 |
imcsk8 | s/thught/thought/ | 16:05 |
jpena | imcsk8: yeah, if the fix is committed, it's been released, even if just as a package from dlrn | 16:05 |
* adarazs is logging off now before we discover something else preventing a master promote. ;) | 16:06 | |
adarazs | see you all, have a good weekend. | 16:06 |
weshay | myoung, did this hit CI? https://review.gerrithub.io/#/c/287153/ | 16:07 |
imcsk8 | jpena: cool, ill run that then 'cause it's a little noisy with all those "Fix commited" bugs | 16:07 |
jpena | imcsk8: just make sure you do a dry run first, we don't want all bugs to be closed :P | 16:08 |
imcsk8 | oh no jajaja | 16:08 |
*** dsneddon has quit IRC | 16:08 | |
*** dsneddon has joined #rdo | 16:09 | |
imcsk8 | jpena: the script is based on the expire_old_bug_reports.py from infra-release-tools | 16:09 |
weshay | myoung, we'll need internal rdo master firing on all cylinders shortly | 16:10 |
myoung | weshay, re: 287153, the ci checks we have in place won't execute the file changed...it's only invoked AFAIK during promotion after something has passed. | 16:11 |
weshay | myoung, ok.. so I'll just merge | 16:12 |
myoung | weshay ok...the additional comment was optional | 16:12 |
weshay | myoung, If I understand.. you were saying combine the three into one variable | 16:12 |
weshay | 00:01:50.198 The requested operation failed as no inventory is available. | 16:13 |
weshay | yikes.. haven't seen that before dmsimard ^ | 16:13 |
myoung | weshay, aye - but it's just stylistic sugar. It looks correct in it's current form. | 16:13 |
*** dyasny has quit IRC | 16:14 | |
weshay | adarazs, wow.. deploy passed on pacemaker | 16:16 |
*** toanju has quit IRC | 16:17 | |
*** bkero is now known as bkero-pto | 16:18 | |
*** florianf has quit IRC | 16:18 | |
jpena | number80: I've tested the new python-suds package. It still fails, this time with a different error message -> http://209.132.178.209/repos/37/28/37283c8689238fe0bcd038267893fc7d002d1383_dev/rpmbuild.log | 16:21 |
weshay | dmsimard, ooo passed.. you were right | 16:21 |
*** dyasny has joined #rdo | 16:21 | |
*** hrw has quit IRC | 16:21 | |
jpena | number80: after some checks, I see the issue is in this line -> https://bitbucket.org/jurko/suds/src/94664ddd46a61d06862fa8fb6ba7b9e054214f57/suds/reader.py?at=default&fileviewer=file-view-default#reader.py-61 . name is None in the test, so name.encode() fails. We could fix it by patching it to str(name).encode() | 16:21 |
*** hrw has joined #rdo | 16:23 | |
amoralej | dmsimard, it seems it's going to pass, it's copying logs now | 16:23 |
number80 | jpena: ack | 16:24 |
flepied | \o/ | 16:27 |
number80 | jpena: new suds building | 16:28 |
jpena | number80++ | 16:28 |
amoralej | a new job is running | 16:30 |
*** fragatina has joined #rdo | 16:30 | |
*** fragatin_ has joined #rdo | 16:31 | |
*** athomas has quit IRC | 16:32 | |
weshay | dmsimard, ping me when you are back | 16:32 |
*** fragatina has quit IRC | 16:35 | |
*** ohamada has quit IRC | 16:35 | |
*** tesseract- has quit IRC | 16:35 | |
number80 | tagged | 16:38 |
*** dtantsur is now known as dtantsur|afk | 16:41 | |
weshay | number80, tagged? | 16:42 |
weshay | as in promoted? | 16:42 |
number80 | weshay: I meant I tagged newer python-suds build | 16:49 |
weshay | ack.. thanks | 16:49 |
* number80 has no access to CI stuff anyway | 16:50 | |
jpena | have a nice weekend everyone! BTW, Monday is a bank holiday in Spain | 16:50 |
number80 | jpena: enjoy your week-end | 16:50 |
number80 | the same here, we're celebrating the fact that the mother of god gave birth to Napoleon the first | 16:51 |
number80 | or something like that | 16:51 |
*** jpena is now known as jpena|off | 16:52 | |
number80 | Well, Napoleon may have thought of himself as some kind of godly being | 16:53 |
*** imcleod has joined #rdo | 16:54 | |
*** lucasagomes is now known as lucas-dinner | 16:55 | |
*** paragan has quit IRC | 17:05 | |
dmsimard | weshay: did someone start a promote job manually ? | 17:08 |
weshay | I did | 17:08 |
dmsimard | so I can't promote manually the last hash anymore ? | 17:08 |
weshay | we still could if that is what we want to do | 17:09 |
dmsimard | I would | 17:09 |
flepied | +1 | 17:09 |
weshay | k.. I need to build a oooq image for it | 17:09 |
dmsimard | But I don't want the current build job to overwrite the image that just passed | 17:09 |
*** paragan has joined #rdo | 17:10 | |
dmsimard | can I interrupt the current job | 17:10 |
weshay | sure | 17:10 |
dmsimard | interrupted, will let is finish semi quickly | 17:11 |
dmsimard | s/quickly/cleanly/ | 17:11 |
dmsimard | going to promote 2a/04/2a040c25781dd5f907842a862ca4df5c55f56f11_99d1c446 . | 17:11 |
*** ayoung has joined #rdo | 17:13 | |
weshay | myoung, incoming.. rdo master | 17:13 |
dmsimard | jpena|off: you gone yet ? | 17:13 |
weshay | jschlueter, I recommend not importing until internal promotes | 17:14 |
*** paragan has quit IRC | 17:14 | |
*** paragan has joined #rdo | 17:15 | |
weshay | rlandy, is bm avail for rdo master? | 17:15 |
rlandy | weshay: yes | 17:15 |
myoung | weshay: woot! are we about to promote (reading scrollback) | 17:15 |
dmsimard | jschlueter: where are imports done from ? | 17:15 |
weshay | rlandy, k.. thanks.. myoung rlandy all eyes on rdo master please :) | 17:15 |
weshay | dmsimard, from delorean, the link you are about to promote | 17:16 |
*** paragan has quit IRC | 17:16 | |
*** dustins has joined #rdo | 17:16 | |
dmsimard | weshay: doesn't answer my question - does it import from http://trunk.rdoproject.org/centos7-master/current-passed-ci (which redirects to CDN and thus has several copies of packages) | 17:16 |
dmsimard | or does it import from literal http://trunk.rdoproject.org/centos7-master/ha/hash | 17:16 |
amoralej | dmsimard, they use the hash | 17:17 |
weshay | I thought the latter | 17:17 |
dmsimard | ok, it's better to use the hash, the CDN is out of sync right now | 17:17 |
dmsimard | just making sure | 17:17 |
dmsimard | weshay: the current-passed-ci on the symlink will not be updated properly | 17:17 |
dmsimard | er | 17:17 |
amoralej | but jschlueter, could you confirm? ^^ | 17:17 |
*** russellb has joined #rdo | 17:18 | |
dmsimard | the current-passed-ci I just promoted will not be synchronized to the CDN | 17:18 |
dmsimard | so the delorean.repo there will not be updated to carry the hash | 17:18 |
*** paragan has joined #rdo | 17:18 | |
flepied | jschlueter: you can use 2a/04/2a040c25781dd5f907842a862ca4df5c55f56f11_99d1c446 to import | 17:19 |
dmsimard | flepied: fyi we sort of broke the cdn sync when we added the new worker for upper constraints | 17:20 |
dmsimard | it's... a mess | 17:20 |
flepied | dmsimard: how can we fix it? | 17:20 |
amoralej | i'm leaving, have a nice weekend !!! | 17:21 |
*** coolsvap has quit IRC | 17:21 | |
dmsimard | flepied: it's not easy. Long story short, KB doesn't delete things off of the CDN (it's hard apparently). And so, we end up with all versions of the packages on the CDN. It's not so bad with yum-plugin-priorities usually but now we have packages we actually want to use that are older (due to uppper-constraints) | 17:22 |
dmsimard | flepied: Not only that, but also the path from which to sync has changed. "centos-master-head" is at /home/centos-master (this is where the CDN pulls from) | 17:22 |
dmsimard | and "centos-newton" (the one with upper constraints) is at /homec/centos-newton (the CDN doesn't pull data from there) | 17:22 |
dmsimard | The CDN doesn't turn out to be such a great idea and more trouble than it's worth -- at least thus far | 17:23 |
dmsimard | I want to discuss it with apevec when he's back | 17:23 |
dmsimard | and truthfully, almost no one uses it anyway | 17:23 |
flepied | dmsimard: yes that's what I understand too | 17:24 |
*** ppowell has quit IRC | 17:24 | |
flepied | dmsimard: more pain than gain | 17:24 |
amoralej | flepied, how the dashboard calculates days from last promotion?, doies it uses http://trunk.rdoproject.org/centos7-master/current-passed-ci ? | 17:24 |
dmsimard | amoralej: https://github.com/rdo-infra/rdo-dashboards/blob/master/feed-dashboard.sh | 17:24 |
flepied | amoralej: http://trunk.rdoproject.org/centos7/current-passed-ci/ | 17:25 |
*** imcleod has quit IRC | 17:25 | |
amoralej | so, it will not reflect real promotions now, until we fix cdn, right? | 17:25 |
dmsimard | amoralej: right | 17:25 |
dmsimard | I /could/ hack something manually | 17:26 |
amoralej | could it access to trunk-primary somehow? | 17:26 |
dmsimard | no network access, no | 17:26 |
dmsimard | weshay: special request | 17:27 |
weshay | aye | 17:27 |
dmsimard | weshay: also upload a manifest file to https://ci.centos.org/artifacts/rdo/images/master/delorean/testing/ so we can have a look at what repository it's from | 17:27 |
*** amoralej is now known as amoralej|off | 17:27 | |
dmsimard | i.e, just a dumb text file with the repositories that were used | 17:28 |
weshay | dmsimard, agree.. myoung ^ | 17:28 |
dmsimard | weshay: I'm going to download an image just to check if it's indeed the right hash in it | 17:28 |
dmsimard | (better be safe than sorry) | 17:28 |
myoung | ^ ditto...happening now. dmsimard/weshay, where's the job that produced the images you just linked? <buildhost>/var/log/artib will have the build.log, repo files, etc used to build the image | 17:32 |
jschlueter | flepied: I will have to work harder but might be able to | 17:34 |
dmsimard | myoung: promotion pipeline | 17:34 |
dmsimard | myoung: https://ci.centos.org/job/tripleo-quickstart-promote-master-delorean-build-images/ | 17:34 |
dmsimard | myoung: for this one time I just downloaded the overcloud image to check the hash that was in it | 17:35 |
myoung | dmsimard i see...i've almost downloaded it as well. If this was from #551 then check out here... https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-delorean-build-images-551/172.19.3.117/var/log/artib/ | 17:36 |
myoung | dmsimard: this was the script used to set up the repos in the base image https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-delorean-build-images-551/172.19.3.117/var/log/artib/repo_setup.sh.gz | 17:36 |
myoung | ^ and here's the canonical builder.log from the overcloud: https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-delorean-build-images-551/172.19.3.117/var/log/artib/builder-overcloud.log.gz | 17:37 |
myoung | dmsimard: http://trunk.rdoproject.org/centos7-master/2a/04/2a040c25781dd5f907842a862ca4df5c55f56f11_99d1c446/delorean.repo | 17:37 |
dmsimard | myoung: yeah, but now you know that it was from job 441 | 17:37 |
dmsimard | 551* | 17:37 |
dmsimard | myoung: what if you don't | 17:37 |
weshay | flepied, did you get my review to update the osp10 dashboard? | 17:38 |
flepied | weshay: no | 17:38 |
flepied | weshay: do you have a link? | 17:38 |
myoung | dmsimard: for the downstream stuff, I've been archiving a log bundle and publishing it with images, hoping to land that soon for this sort of reason. sorry for wading into the thread without context, i now see we're doing a by hand promote | 17:39 |
dmsimard | myoung: yeah, unfortunately required given the circumstances | 17:40 |
myoung | dmsimard: just downloaded the image you linked and checked...that image has http://trunk.rdoproject.org/centos7/2a/04/2a040c25781dd5f907842a862ca4df5c55f56f11_99d1c446 ( https://paste.fedoraproject.org/407055/47102366/ ) | 17:41 |
dmsimard | myoung: yeah I had already checked that | 17:41 |
dmsimard | myoung: what I was saying is that it'd be trivial to just add a text file to that directory with the repository it was built from | 17:42 |
* myoung shakes his fist at verizon and VPN | 17:42 | |
myoung | took me a while :) | 17:42 |
myoung | to download | 17:42 |
dmsimard | weshay: ooo-q image is promoted | 17:42 |
dmsimard | as is dlrn, however not visible from cdn | 17:42 |
weshay | dmsimard, k.. thanks | 17:42 |
dmsimard | I'll have to send an e-mail to explain the cdn disconnect | 17:42 |
dmsimard | because people will be panicking at the dashboard | 17:43 |
dmsimard | wait | 17:44 |
dmsimard | the promote job is set to promote centos-master right now but we want to be promoting centos-newton | 17:44 |
dmsimard | ugh | 17:44 |
dmsimard | we really changed the workers to upper constraints too quickly and didn't take care of changing dependencies | 17:44 |
dmsimard | weshay: right now if the promote job passes | 17:45 |
dmsimard | weshay: the actual dlrn repository promotion will fail | 17:45 |
dmsimard | because it will try to promote centos-master and the hash won't exist there | 17:45 |
*** ihrachys has quit IRC | 17:46 | |
weshay | until a cdn sync? | 17:46 |
dmsimard | no, has nothing to do with cdn | 17:46 |
myoung | dmsimard: ack, and agreed. I put this in for that reason (https://github.com/redhat-openstack/ansible-role-tripleo-image-build/blob/master/tasks/main.yml#L62), we just need to publish it when we publish/promote the images. for downstream I've been running off this (which does that + a bit more) and hope to land it soon: https://review.gerrithub.io/#/c/277420/24/defaults/main.yml, https://review.gerrithub.io/#/c/277420/24/tasks/main.yml | 17:46 |
weshay | dmsimard, if we're all set.. let's rekick | 17:46 |
weshay | dmsimard, may I? | 17:46 |
dmsimard | weshay: feel free to re-kick but the repository promotion will fail | 17:46 |
* weshay confused | 17:46 | |
dmsimard | bluejeans ? | 17:47 |
weshay | sure | 17:47 |
weshay | whayutin | 17:47 |
*** jdennis1 has joined #rdo | 17:50 | |
*** jdennis has quit IRC | 17:51 | |
*** gfidente has quit IRC | 17:52 | |
*** pblaho has quit IRC | 17:53 | |
*** number80 has quit IRC | 17:55 | |
*** jdob has quit IRC | 17:56 | |
*** jdob has joined #rdo | 17:58 | |
*** dgurtner has quit IRC | 17:58 | |
*** fragatin_ has quit IRC | 18:02 | |
*** Son_Goku has joined #rdo | 18:02 | |
*** sdake has joined #rdo | 18:21 | |
*** jdob has quit IRC | 18:27 | |
*** jdob has joined #rdo | 18:27 | |
*** v1k0d3n has joined #rdo | 18:31 | |
*** weshay is now known as weshay_mtg | 18:33 | |
dmsimard | weshay_mtg: how long is trown out for again ? | 18:38 |
weshay_mtg | two more weeks I think | 18:38 |
dmsimard | well that's depressing | 18:39 |
*** sdake has quit IRC | 18:45 | |
*** jdob has quit IRC | 18:46 | |
*** jdob has joined #rdo | 18:47 | |
weshay_mtg | local test of master passed | 18:56 |
*** tosky has quit IRC | 18:58 | |
*** dustins has quit IRC | 18:59 | |
*** dustins has joined #rdo | 19:00 | |
*** fragatina has joined #rdo | 19:01 | |
*** jprovazn has quit IRC | 19:03 | |
*** scorcoran has joined #rdo | 19:04 | |
*** fultonj has quit IRC | 19:04 | |
openstackgerrit | Andreas Jaeger proposed openstack/packstack: Move other-requirements.txt to bindep.txt https://review.openstack.org/354972 | 19:04 |
*** fultonj has joined #rdo | 19:04 | |
*** dpeacock has quit IRC | 19:06 | |
*** smeyer has joined #rdo | 19:11 | |
*** manous has quit IRC | 19:12 | |
*** jdob has quit IRC | 19:13 | |
*** jdob has joined #rdo | 19:15 | |
*** weshay_mtg is now known as weshay | 19:17 | |
*** milan has quit IRC | 19:20 | |
*** smeyer has quit IRC | 19:20 | |
*** dustins has quit IRC | 19:25 | |
*** dustins has joined #rdo | 19:26 | |
*** fragatina has quit IRC | 19:28 | |
*** fragatina has joined #rdo | 19:29 | |
*** fragatina has quit IRC | 19:29 | |
weshay | myoung, is there an email brewin? | 19:30 |
myoung | weshay, yes...been sidetracked. I'll get to it. | 19:30 |
*** Son_Goku has quit IRC | 19:33 | |
myoung | weshay, actually if you have a moment to chat BJ I have an error that's just got me stymied, I could use a quick look. The issue is probably staring me in the face. | 19:33 |
weshay | k | 19:34 |
weshay | going in | 19:34 |
*** scorcoran is now known as scorcoran_afk | 19:35 | |
pabelanger | dmsimard: what email address did we want for 3rd party CI user? | 19:35 |
*** scorcoran_afk has quit IRC | 19:35 | |
weshay | myoung, ^ | 19:35 |
myoung | weshay inc | 19:35 |
*** pnavarro has joined #rdo | 19:36 | |
*** imcleod has joined #rdo | 19:38 | |
*** aortega has quit IRC | 19:39 | |
*** jmelvin has quit IRC | 19:45 | |
*** mvk has quit IRC | 19:54 | |
*** steveg_afk has quit IRC | 19:54 | |
*** alexismonville has quit IRC | 19:54 | |
*** alexismonville has joined #rdo | 19:54 | |
*** unclemarc has quit IRC | 20:04 | |
*** steveg_afk has joined #rdo | 20:08 | |
*** alexismonville has quit IRC | 20:10 | |
*** jdob has quit IRC | 20:11 | |
myoung | weshay, you were right. the version of tripleo-common we're pulling in doesn't yet contain the recent changes. the env block was being completely ignored, so it wasn't even using minimal-base, it was using the centos cloud image | 20:17 |
weshay | ah | 20:17 |
myoung | weshay openstack-tripleo-common-5.0.0-0.20160725153340.e7377aa.el7.centos.noarch.rpm | 20:17 |
myoung | weshay for downstream we were running off that gerrit hook gen'd rpm | 20:17 |
myoung | weshay thanks for the eyes and probing q's :) | 20:18 |
*** dprince has quit IRC | 20:25 | |
*** alexismonville has joined #rdo | 20:27 | |
*** aortega has joined #rdo | 20:29 | |
*** sdake has joined #rdo | 20:32 | |
*** shardy has quit IRC | 20:32 | |
*** ihrachys has joined #rdo | 20:33 | |
*** jayg is now known as jayg|g0n3 | 20:34 | |
*** rhallisey has quit IRC | 20:43 | |
*** jkilpatr has quit IRC | 20:43 | |
*** weshay is now known as weshay_afk | 20:47 | |
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting for a scheduled upgrade, but should return to service momentarily: http://lists.openstack.org/pipermail/openstack-dev/2016-August/101394.html | 20:51 | |
*** aortega has quit IRC | 20:55 | |
*** jkilpatr has joined #rdo | 20:56 | |
*** number80 has joined #rdo | 20:56 | |
*** TicToc has joined #rdo | 21:00 | |
-openstackstatus- NOTICE: The Mediawiki service at wiki.openstack.org will be offline from 21:00 UTC until approximately 23:00 UTC for a planned upgrade http://lists.openstack.org/pipermail/openstack-dev/2016-August/101395.html | 21:00 | |
*** ChanServ changes topic to "The Mediawiki service at wiki.openstack.org will be offline from 21:00 UTC until approximately 23:00 UTC for a planned upgrade http://lists.openstack.org/pipermail/openstack-dev/2016-August/101395.html" | 21:00 | |
*** jkilpatr has quit IRC | 21:00 | |
*** fultonj has quit IRC | 21:01 | |
*** DrBacchus has quit IRC | 21:02 | |
*** shivrao has joined #rdo | 21:04 | |
*** ohochman has left #rdo | 21:08 | |
*** fragatina has joined #rdo | 21:11 | |
*** fragatina has quit IRC | 21:19 | |
TicToc | Can anyone explain "tripleo quick start ~/.quickstart/ssh.config.ansible will try to proxy through the localhost to ssh to the localhost and will cause an error if ssh is not setup to support it" how do you proxy through localhost to ssh? | 21:23 |
*** sdake has quit IRC | 21:23 | |
*** amuller has quit IRC | 21:28 | |
*** dustins has quit IRC | 21:29 | |
*** manous has joined #rdo | 21:30 | |
*** decause has left #rdo | 21:31 | |
*** myoung is now known as myoung|biab | 21:38 | |
*** ihrachys has quit IRC | 21:39 | |
*** dneary has quit IRC | 21:40 | |
*** gszasz has quit IRC | 21:42 | |
*** imcleod has quit IRC | 21:43 | |
*** jistr has quit IRC | 21:47 | |
*** jruzicka has quit IRC | 21:48 | |
*** fragatina has joined #rdo | 21:48 | |
*** fragatina has quit IRC | 21:50 | |
*** jruzicka has joined #rdo | 21:50 | |
*** fragatina has joined #rdo | 21:50 | |
*** jistr has joined #rdo | 21:51 | |
*** manous has quit IRC | 21:53 | |
*** jruzicka has quit IRC | 21:56 | |
*** jruzicka has joined #rdo | 21:57 | |
*** imcleod has joined #rdo | 21:59 | |
*** pradk has quit IRC | 22:00 | |
*** jistr has quit IRC | 22:00 | |
*** shivrao has quit IRC | 22:04 | |
*** jruzicka has quit IRC | 22:04 | |
*** ashw has quit IRC | 22:07 | |
*** jistr has joined #rdo | 22:09 | |
*** shivrao has joined #rdo | 22:10 | |
*** jruzicka has joined #rdo | 22:10 | |
*** elmiko is now known as _elmiko | 22:12 | |
*** beekneemech is now known as bnemec-pto | 22:12 | |
*** jistr has quit IRC | 22:19 | |
*** iranzo has joined #rdo | 22:19 | |
*** iranzo has quit IRC | 22:19 | |
*** iranzo has joined #rdo | 22:19 | |
*** jruzicka has quit IRC | 22:20 | |
*** jruzicka has joined #rdo | 22:25 | |
*** jistr has joined #rdo | 22:25 | |
*** TicToc has quit IRC | 22:27 | |
*** tumble has joined #rdo | 22:29 | |
*** panda has quit IRC | 22:31 | |
*** imcleod has quit IRC | 22:32 | |
*** pnavarro has quit IRC | 22:32 | |
*** TicToc has joined #rdo | 22:34 | |
*** maeca1 has quit IRC | 22:39 | |
*** imcleod has joined #rdo | 22:48 | |
*** apetrich has quit IRC | 22:54 | |
-openstackstatus- NOTICE: ok https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong | 23:02 | |
*** ChanServ changes topic to "ok https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong" | 23:02 | |
*** iranzo has quit IRC | 23:03 | |
-openstackstatus- NOTICE: https://wiki.openstack.org/ is now running Mediawiki 1.27.0; please let us know in #openstack-infra if anything seems wrong | 23:08 | |
*** ChanServ changes topic to "RDO: An OpenStack Distribution for CentOS/Fedora/RHEL | Community meeting is held here every Wednesday at 15:00UTC | Please read our IRC community guidelines: http://rdoproject.org/IRC_etiquette | Logged at http://eavesdrop.openstack.org/irclogs/%23rdo/"" | 23:08 | |
*** imcleod has quit IRC | 23:13 | |
*** harmw has quit IRC | 23:20 | |
*** Ryjedo_ has joined #rdo | 23:25 | |
*** Goneri has quit IRC | 23:26 | |
*** saneax_AFK is now known as saneax | 23:26 | |
*** saneax is now known as Guest48269 | 23:27 | |
*** Ryjedo has quit IRC | 23:29 | |
*** Ryjedo_ is now known as Ryjedo | 23:29 | |
*** rlandy has quit IRC | 23:29 | |
*** trozet has quit IRC | 23:31 | |
*** aortega has joined #rdo | 23:31 | |
*** trozet has joined #rdo | 23:32 | |
*** shivrao has quit IRC | 23:32 | |
*** sdake has joined #rdo | 23:33 | |
*** TicToc has quit IRC | 23:34 | |
*** TicToc has joined #rdo | 23:36 | |
*** Son_Goku has joined #rdo | 23:37 | |
*** Son_Goku has quit IRC | 23:39 | |
*** Kappa has joined #rdo | 23:43 | |
*** Kappa has left #rdo | 23:43 | |
*** aortega has quit IRC | 23:47 | |
*** alexismonville has quit IRC | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!