*** rossella_s has quit IRC | 00:00 | |
*** rossella_s has joined #openstack-infra | 00:00 | |
*** amitgandhinz has joined #openstack-infra | 00:02 | |
*** mriedem has joined #openstack-infra | 00:02 | |
openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Include subnodes in updateStats function v2 https://review.openstack.org/351435 | 00:04 |
---|---|---|
*** SumitNaiksatam has quit IRC | 00:04 | |
pabelanger | jeblair: ^ should be faster | 00:04 |
*** sdake_ has joined #openstack-infra | 00:06 | |
*** thorst has quit IRC | 00:06 | |
*** zhurong has quit IRC | 00:07 | |
*** amitgandhinz has quit IRC | 00:07 | |
*** sdake has quit IRC | 00:09 | |
*** _ari_ has joined #openstack-infra | 00:12 | |
*** markvoelker has joined #openstack-infra | 00:12 | |
jeblair | the only ready nodes sitting around now (~10-20 mins) are for lesser-used image types. | 00:13 |
jeblair | i spot checked a couple and they really are idle | 00:13 |
fungi | i just cleaned up 3 more used nodes that crept past the 3 hour mark | 00:14 |
jeblair | fungi: what do you think about doing the 2-3 hour ones? | 00:14 |
*** markvoelker_ has quit IRC | 00:14 | |
fungi | there's >100 of those | 00:14 |
*** _ari_ has quit IRC | 00:14 | |
jeblair | which if we assume we should be running 450 jobs, is about our deficit (gearman says 332 are running) | 00:15 |
fungi | i guess any really long-running jobs are not that great in number and aren't a huge loss if they get rescheduled | 00:15 |
fungi | yeah, i buy that math. i'll clean those out now too | 00:15 |
jeblair | my feeling is that jobs > 1 hour are 'not officially supported' :) | 00:15 |
fungi | i can get behind that | 00:15 |
*** sputnik13 has quit IRC | 00:17 | |
ianw | hey, is it appropriate to copy stuff to static.openstack.org manually? particularly a kernel for fedora 24 which we're going to need until a fix gets in and released? my plan is to add it as a fixup in infra-package-needs until then | 00:18 |
fungi | feels a little weird, i don't think we've done that in the past | 00:18 |
*** pahuang has quit IRC | 00:18 | |
*** kzaitsev_mb has quit IRC | 00:19 | |
ianw | ok, np ... we have copr for fedora to build packages | 00:19 |
fungi | on the other hand, there's all sorts of relatively irreplaceable artifacts on tarballs.o.o | 00:19 |
ianw | i'm just having issues getting it signing the packages, etc ... just unrelated stuff | 00:19 |
fungi | i'm not particularly opposed if others aren't | 00:19 |
fungi | certainly wouldn't want that checked into a git repo anyway | 00:20 |
ianw | cool, well i need to double-check that dib doesn't make this too hard by screwing up the grub config and install the wrong kernel .. but that's the plan | 00:20 |
openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-log_processor: Make the Gemfile zuul-aware https://review.openstack.org/351437 | 00:21 |
fungi | basically, i don't have a better suggestion on where to put that at the moment, so unless someone else does... | 00:21 |
ianw | copr is made for this ... i've started again with https://copr.fedorainfracloud.org/coprs/iwienand/kernel-4.6.4-301.rhbz1361414/ ... last time it took like 7 hours and still failed | 00:22 |
*** pahuang has joined #openstack-infra | 00:22 | |
jeblair | i don't like any part of the ci system requiring manual intervention on the part of roots. i'm not going to object to anyone else doing it, but i would not do it if asked. | 00:22 |
*** cody-somerville has joined #openstack-infra | 00:22 | |
mgagne | pabelanger: so I went with a solution similar to this one for my earlier problem with voluptuous: http://paste.openstack.org/show/549324/ | 00:24 |
jeblair | is f24 gating anything? can we switch jobs to non-voting until it's fixed? | 00:24 |
*** tqtran has quit IRC | 00:25 | |
ianw | jeblair: no it's not gating any, just doesn't work at all because it oops. i've root caused it & fixed it (http://marc.info/?l=linux-netdev&m=147020530109885&w=2) but it's just going to take a while to make it through the system | 00:25 |
*** karthikp_ has quit IRC | 00:26 | |
*** pvaneck_ has quit IRC | 00:26 | |
jeblair | gotcha. | 00:28 |
*** mkoderer__ has quit IRC | 00:29 | |
jeblair | fungi: i think some more nodes crept past the 2 hour mark | 00:29 |
jeblair | we're at 429 now, so we probably only have about 30 more leaked | 00:30 |
jeblair | er, 20 more :) | 00:31 |
fungi | looks like 34 | 00:31 |
fungi | deleting | 00:31 |
*** chlong has quit IRC | 00:32 | |
*** r-mibu has quit IRC | 00:35 | |
*** mkoderer__ has joined #openstack-infra | 00:37 | |
*** mtanino has quit IRC | 00:37 | |
jeblair | i'm happy with all the squiggly lines now | 00:39 |
*** amotoki has joined #openstack-infra | 00:39 | |
*** r-mibu has joined #openstack-infra | 00:40 | |
*** rhallisey has joined #openstack-infra | 00:43 | |
*** gouthamr_ has joined #openstack-infra | 00:44 | |
*** xnox has quit IRC | 00:48 | |
*** xarses has joined #openstack-infra | 00:48 | |
*** dimtruck is now known as zz_dimtruck | 00:48 | |
rbergeron | jeblair: squiggly lines make all the things better. | 00:50 |
*** hichihara has quit IRC | 00:51 | |
*** hemanthm|afk is now known as hemanthm | 00:51 | |
*** hichihara has joined #openstack-infra | 00:51 | |
*** elo has joined #openstack-infra | 00:54 | |
*** jamielennox is now known as jamielennox|away | 00:55 | |
pabelanger | that's the first time I can remember have to deal with leaked nodes since switching from jenkins | 00:56 |
pabelanger | so that is good | 00:56 |
*** jamielennox|away is now known as jamielennox | 00:57 | |
*** markvoelker has quit IRC | 00:59 | |
*** amitgandhinz has joined #openstack-infra | 01:03 | |
*** thorst has joined #openstack-infra | 01:04 | |
*** xnox has joined #openstack-infra | 01:05 | |
*** gildub has joined #openstack-infra | 01:05 | |
*** shashank_hegde has quit IRC | 01:06 | |
*** amitgandhinz has quit IRC | 01:07 | |
*** ayoung has joined #openstack-infra | 01:08 | |
openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Add dvr/linuxbridge neutron jobs to devstack experimental queue https://review.openstack.org/351450 | 01:08 |
*** Sukhdev has quit IRC | 01:08 | |
*** xnox has quit IRC | 01:10 | |
*** thorst has quit IRC | 01:11 | |
*** jcoufal has joined #openstack-infra | 01:13 | |
*** xnox has joined #openstack-infra | 01:14 | |
*** Apoorva has quit IRC | 01:14 | |
*** thorst has joined #openstack-infra | 01:15 | |
*** zhurong has joined #openstack-infra | 01:16 | |
*** piet_ has joined #openstack-infra | 01:16 | |
*** thorst has quit IRC | 01:16 | |
*** Apoorva has joined #openstack-infra | 01:17 | |
*** amotoki has quit IRC | 01:18 | |
*** xnox has quit IRC | 01:19 | |
*** zhurong has quit IRC | 01:21 | |
*** zhurong has joined #openstack-infra | 01:22 | |
*** ddieterly has joined #openstack-infra | 01:25 | |
*** yanyanhu has joined #openstack-infra | 01:28 | |
*** piet_ has quit IRC | 01:30 | |
*** dingyichen has quit IRC | 01:32 | |
openstackgerrit | Tang Chen proposed openstack-infra/project-config: Fix Ironic functional job by properly setting PROJECTS https://review.openstack.org/351138 | 01:33 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 01:35 |
*** cody-somerville has quit IRC | 01:39 | |
*** esberglu has quit IRC | 01:43 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Cloud Foundry plugin https://review.openstack.org/350940 | 01:46 |
openstackgerrit | Merged openstack-infra/project-config: Add channel #openstack-dib for diskimage-builder https://review.openstack.org/350997 | 01:47 |
prometheanfire | woo | 01:56 |
*** mriedem has quit IRC | 01:58 | |
*** Apoorva_ has joined #openstack-infra | 01:58 | |
*** Apoorva has quit IRC | 02:01 | |
*** Apoorva_ has quit IRC | 02:02 | |
*** sdake has joined #openstack-infra | 02:03 | |
*** tphummel has quit IRC | 02:03 | |
*** amitgandhinz has joined #openstack-infra | 02:04 | |
*** sdake_ has quit IRC | 02:04 | |
*** rhallisey has quit IRC | 02:05 | |
*** yamahata has quit IRC | 02:07 | |
*** amitgandhinz has quit IRC | 02:08 | |
*** jamielennox is now known as jamielennox|away | 02:12 | |
*** amotoki has joined #openstack-infra | 02:13 | |
*** sdake_ has joined #openstack-infra | 02:14 | |
*** esberglu has joined #openstack-infra | 02:14 | |
*** sflanigan is now known as sflanigan_afk | 02:17 | |
*** thorst has joined #openstack-infra | 02:17 | |
*** sdake has quit IRC | 02:17 | |
*** cody-somerville has joined #openstack-infra | 02:20 | |
*** cody-somerville has joined #openstack-infra | 02:20 | |
*** thorst has quit IRC | 02:25 | |
*** ddieterly has quit IRC | 02:26 | |
*** julim has joined #openstack-infra | 02:28 | |
*** gouthamr_ has quit IRC | 02:29 | |
*** daemontool_ has quit IRC | 02:31 | |
*** huanxie has joined #openstack-infra | 02:32 | |
*** sflanigan_afk is now known as sflanigan | 02:32 | |
*** esberglu has quit IRC | 02:35 | |
*** zhurong_ has joined #openstack-infra | 02:37 | |
*** zhurong has quit IRC | 02:39 | |
*** itisha has quit IRC | 02:40 | |
*** thorst has joined #openstack-infra | 02:43 | |
*** thorst has quit IRC | 02:43 | |
*** zul has joined #openstack-infra | 02:44 | |
openstackgerrit | qinchunhua proposed openstack-infra/project-config: Make py35 voting https://review.openstack.org/351464 | 02:44 |
*** dchen has joined #openstack-infra | 02:44 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Replace gate-{name}-tox-{envlist} with gate-{name}-tox-{envlist}-{node} https://review.openstack.org/351219 | 02:45 |
*** pt_15 has quit IRC | 02:46 | |
openstackgerrit | qinchunhua proposed openstack-infra/project-config: Make py35 voting for networking-midonet https://review.openstack.org/351464 | 02:46 |
*** ddieterly has joined #openstack-infra | 02:47 | |
*** amotoki has quit IRC | 02:48 | |
*** yuanying has quit IRC | 02:49 | |
*** yamahata has joined #openstack-infra | 02:49 | |
*** xnox has joined #openstack-infra | 02:54 | |
*** esberglu has joined #openstack-infra | 02:58 | |
*** esberglu has quit IRC | 02:59 | |
*** piet_ has joined #openstack-infra | 02:59 | |
*** xnox has quit IRC | 02:59 | |
*** amitgandhinz has joined #openstack-infra | 03:04 | |
*** amitgandhinz has quit IRC | 03:09 | |
openstackgerrit | qinchunhua proposed openstack-infra/project-config: Make py35 voting for networking-midonet https://review.openstack.org/351464 | 03:12 |
*** amotoki has joined #openstack-infra | 03:15 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Replace gate-{name}-tox-{envlist} with gate-{name}-tox-{envlist}-{node} https://review.openstack.org/351219 | 03:16 |
*** ddieterly has quit IRC | 03:17 | |
*** yamahata has quit IRC | 03:18 | |
*** dchen has quit IRC | 03:19 | |
*** liusheng has joined #openstack-infra | 03:21 | |
*** xnox has joined #openstack-infra | 03:26 | |
*** ayoung is now known as ayoung_ZZZzzz | 03:26 | |
*** xnox has quit IRC | 03:29 | |
*** julim has quit IRC | 03:31 | |
*** amotoki has quit IRC | 03:31 | |
*** Srinu has joined #openstack-infra | 03:34 | |
*** jamielennox|away is now known as jamielennox | 03:35 | |
Srinu | hi | 03:36 |
*** xnox has joined #openstack-infra | 03:37 | |
Srinu | please help me. this issue is not sloving for me long ago........http://paste.openstack.org/show/549835/ | 03:40 |
*** sdake_ has quit IRC | 03:42 | |
Srinu | In nodepool configuration how can i use same provider for 2 slaves but different flavor on the provider | 03:43 |
*** thorst has joined #openstack-infra | 03:44 | |
*** yuanying has joined #openstack-infra | 03:48 | |
*** chlong has joined #openstack-infra | 03:49 | |
*** karthikp_ has joined #openstack-infra | 03:52 | |
*** thorst has quit IRC | 03:52 | |
*** piet_ has quit IRC | 03:52 | |
*** esberglu has joined #openstack-infra | 03:55 | |
*** sdake has joined #openstack-infra | 03:57 | |
*** esberglu has quit IRC | 04:02 | |
*** jcoufal has quit IRC | 04:02 | |
*** karthik__ has joined #openstack-infra | 04:03 | |
*** karthikp_ has quit IRC | 04:05 | |
*** amitgandhinz has joined #openstack-infra | 04:05 | |
*** rlandy has quit IRC | 04:08 | |
*** amitgandhinz has quit IRC | 04:10 | |
*** armax has quit IRC | 04:10 | |
*** sdake has quit IRC | 04:14 | |
*** amotoki has joined #openstack-infra | 04:16 | |
*** cody-somerville has quit IRC | 04:22 | |
*** javeriak has joined #openstack-infra | 04:22 | |
*** dingyichen has joined #openstack-infra | 04:24 | |
*** sdake has joined #openstack-infra | 04:24 | |
*** dingyichen has quit IRC | 04:27 | |
*** dingyichen has joined #openstack-infra | 04:32 | |
*** Sukhdev has joined #openstack-infra | 04:34 | |
*** markvoelker has joined #openstack-infra | 04:38 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Cloud Foundry plugin https://review.openstack.org/350940 | 04:40 |
*** Sukhdev has quit IRC | 04:43 | |
*** jamielennox is now known as jamielennox|away | 04:43 | |
*** Sukhdev has joined #openstack-infra | 04:43 | |
*** markvoelker has quit IRC | 04:43 | |
*** jaosorior has joined #openstack-infra | 04:45 | |
*** jaosorior has quit IRC | 04:51 | |
*** thorst has joined #openstack-infra | 04:51 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Cloud Foundry plugin https://review.openstack.org/350940 | 04:51 |
*** SumitNaiksatam has joined #openstack-infra | 04:52 | |
*** roxanaghe has joined #openstack-infra | 04:52 | |
*** SumitNaiksatam has quit IRC | 04:52 | |
*** jaosorior has joined #openstack-infra | 04:52 | |
*** psachin has joined #openstack-infra | 04:54 | |
*** javeriak has quit IRC | 04:54 | |
*** jaosorior has quit IRC | 04:54 | |
*** yottatsa has joined #openstack-infra | 04:55 | |
*** jaosorior has joined #openstack-infra | 04:55 | |
*** thorst has quit IRC | 04:58 | |
*** rbuzatu has joined #openstack-infra | 04:58 | |
*** rbuzatu has quit IRC | 05:01 | |
*** sarob has joined #openstack-infra | 05:02 | |
*** pahuang has quit IRC | 05:03 | |
*** amitgandhinz has joined #openstack-infra | 05:06 | |
*** yottatsa has quit IRC | 05:07 | |
*** yottatsa has joined #openstack-infra | 05:09 | |
*** amitgandhinz has quit IRC | 05:10 | |
*** Sukhdev has quit IRC | 05:12 | |
*** jaosorior has quit IRC | 05:14 | |
*** M-docaedo_vector has quit IRC | 05:14 | |
*** _nadya_ has joined #openstack-infra | 05:14 | |
*** jaosorior has joined #openstack-infra | 05:15 | |
*** yottatsa has quit IRC | 05:15 | |
*** narayrak has joined #openstack-infra | 05:15 | |
*** pahuang has joined #openstack-infra | 05:15 | |
*** jaosorior has quit IRC | 05:17 | |
*** jaosorior has joined #openstack-infra | 05:18 | |
*** roxanaghe has quit IRC | 05:25 | |
*** sarob has quit IRC | 05:27 | |
*** sarob has joined #openstack-infra | 05:27 | |
*** rcernin has joined #openstack-infra | 05:31 | |
*** e0ne has joined #openstack-infra | 05:31 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Replace gate-{name}-tox-{envlist} with gate-{name}-tox-{envlist}-{node} https://review.openstack.org/351219 | 05:33 |
*** sarob has quit IRC | 05:35 | |
*** kien-ha has quit IRC | 05:36 | |
*** links has joined #openstack-infra | 05:36 | |
*** _nadya_ has quit IRC | 05:37 | |
*** _nadya_ has joined #openstack-infra | 05:40 | |
*** M-docaedo_vector has joined #openstack-infra | 05:42 | |
*** nmagnezi has joined #openstack-infra | 05:44 | |
*** _nadya_ has quit IRC | 05:46 | |
openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 05:49 |
*** rbuzatu has joined #openstack-infra | 05:54 | |
*** rbuzatu has quit IRC | 05:55 | |
*** rbuzatu has joined #openstack-infra | 05:56 | |
*** thorst has joined #openstack-infra | 05:56 | |
*** pgadiya has joined #openstack-infra | 05:57 | |
*** ekarlso has quit IRC | 05:58 | |
*** chem has joined #openstack-infra | 06:00 | |
*** thorst has quit IRC | 06:03 | |
*** Ravikiran_K has joined #openstack-infra | 06:06 | |
*** amitgandhinz has joined #openstack-infra | 06:07 | |
openstackgerrit | Timothy R. Chavez proposed openstack-infra/jenkins-job-builder: Add support for the random string parameter https://review.openstack.org/351384 | 06:07 |
*** chlong has quit IRC | 06:08 | |
*** amitgandhinz has quit IRC | 06:11 | |
*** armax has joined #openstack-infra | 06:12 | |
*** coolsvap has joined #openstack-infra | 06:14 | |
*** camunoz has quit IRC | 06:14 | |
*** kaisers has quit IRC | 06:15 | |
*** kaisers has joined #openstack-infra | 06:16 | |
*** armax has quit IRC | 06:17 | |
*** ianychoi has quit IRC | 06:18 | |
*** jimbaker has quit IRC | 06:19 | |
*** jianghuaw has quit IRC | 06:20 | |
*** sandanar has joined #openstack-infra | 06:20 | |
*** jimbaker has joined #openstack-infra | 06:23 | |
*** jimbaker has quit IRC | 06:23 | |
*** jimbaker has joined #openstack-infra | 06:23 | |
*** javeriak has joined #openstack-infra | 06:29 | |
*** Hal has joined #openstack-infra | 06:32 | |
*** rbuzatu_ has joined #openstack-infra | 06:32 | |
odyssey4me | do we have any idea when the new bindep will be available on nodepool images? | 06:32 |
*** rbuzatu_ has quit IRC | 06:33 | |
*** rbuzatu_ has joined #openstack-infra | 06:35 | |
*** rbuzatu has quit IRC | 06:36 | |
*** markvoelker has joined #openstack-infra | 06:40 | |
*** chlong has joined #openstack-infra | 06:40 | |
*** jtomasek has joined #openstack-infra | 06:44 | |
*** aeng has joined #openstack-infra | 06:44 | |
*** markvoelker has quit IRC | 06:44 | |
*** cody-somerville has joined #openstack-infra | 06:49 | |
*** e0ne has quit IRC | 06:51 | |
*** chlong has quit IRC | 06:52 | |
*** ianychoi has joined #openstack-infra | 06:56 | |
*** yamamot__ has joined #openstack-infra | 07:00 | |
*** thorst has joined #openstack-infra | 07:01 | |
*** Na3iL has joined #openstack-infra | 07:03 | |
*** gildub has quit IRC | 07:07 | |
*** amitgandhinz has joined #openstack-infra | 07:07 | |
*** thorst has quit IRC | 07:08 | |
*** cody-somerville has quit IRC | 07:08 | |
*** mikelk has joined #openstack-infra | 07:09 | |
*** amitgandhinz has quit IRC | 07:12 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute014.vanilla metadata for better tracking https://review.openstack.org/351527 | 07:12 |
*** cody-somerville has joined #openstack-infra | 07:13 | |
yolanda | good morning | 07:13 |
AJaeger | good morning, yolanda | 07:14 |
AJaeger | fungi, set topic to constraints - see https://review.openstack.org/#/q/topic:constraints+status:open | 07:14 |
* rcarrillocruz waves | 07:14 | |
AJaeger | morning rcarrillocruz | 07:15 |
*** jimbaker has quit IRC | 07:16 | |
*** pahuang has quit IRC | 07:16 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute012.vanilla metadata for better tracking https://review.openstack.org/351531 | 07:18 |
*** jimbaker has joined #openstack-infra | 07:19 | |
*** jimbaker has quit IRC | 07:20 | |
*** jimbaker has joined #openstack-infra | 07:20 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute011.vanilla metadata for better tracking https://review.openstack.org/351533 | 07:21 |
*** cody-somerville has quit IRC | 07:23 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP for compute024.vanilla https://review.openstack.org/351535 | 07:25 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP for compute030.vanilla https://review.openstack.org/351539 | 07:29 |
sdake | AJaeger about? | 07:29 |
AJaeger | sdake: yes | 07:29 |
sdake | say do you know anything about the afs mirroring? | 07:29 |
openstackgerrit | Merged openstack-infra/project-config: Add release-notes-jobs to osc-lib https://review.openstack.org/351342 | 07:30 |
sdake | https://review.openstack.org/349278 | 07:30 |
openstackgerrit | Merged openstack-infra/project-config: Add dvr/linuxbridge neutron jobs to devstack experimental queue https://review.openstack.org/351450 | 07:30 |
sdake | not sure if thats correct or not ^^ | 07:30 |
AJaeger | sdake: not really. pabelanger, jeblair, mordred are the experts | 07:30 |
openstackgerrit | Merged openstack-infra/project-config: Add gate-tripleo-quickstart-base-centos-7-nv job https://review.openstack.org/350301 | 07:30 |
AJaeger | But let me check your review | 07:30 |
sdake | i need to do about 3 or 4 like em for ubuntu | 07:30 |
sdake | wanted to get cracking | 07:31 |
openstackgerrit | Merged openstack-infra/project-config: Fix Ironic functional job by properly setting PROJECTS https://review.openstack.org/351138 | 07:31 |
AJaeger | sdake: I cannot really comment on that, best way for the rest of the team to wake up and ask | 07:32 |
openstackgerrit | qinchunhua proposed openstack-infra/project-config: Make py35 voting for networking-midonet https://review.openstack.org/351464 | 07:32 |
sdake | i did ask pabelanger last week to take a look but eveyrone is super overloaded atm | 07:32 |
sdake | i'll try again in the morning | 07:32 |
sdake | thanks :) | 07:32 |
AJaeger | sdake: get some rest ;) | 07:32 |
sdake | 48 hours or bust! | 07:33 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute023.vanilla metadata for better tracking https://review.openstack.org/351541 | 07:35 |
*** jpavlik has joined #openstack-infra | 07:35 | |
jpavlik | Hello all, please can anybody take a look at following review https://review.openstack.org/#/c/343663/ | 07:35 |
jpavlik | it hangs on "needs verify" for more than week without any action | 07:36 |
*** flaper87 has joined #openstack-infra | 07:36 | |
*** flaper87 is now known as Guest73548 | 07:36 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute009.vanilla https://review.openstack.org/351542 | 07:38 |
*** jpich has joined #openstack-infra | 07:39 | |
AJaeger | jpavlik: remove the +Workflow and add it again. It might have been approved during an outage - and now we react only to state changes. A recheck will not give a state cahnge | 07:39 |
AJaeger | jpavlik: I see the problem: You first did a +W, and afterwards a +2. | 07:40 |
jpavlik | OK, so remove and add +1 on workflow? | 07:40 |
AJaeger | jpavlik: that's the problem! Your +W is only valid if there's a prior +2 already. Since we only look at state change, the later +2 does is not seen. | 07:40 |
AJaeger | jpavlik: yes, just toggle it. | 07:40 |
AJaeger | jpavlik: and it already starts in the gate as expected ;) | 07:41 |
jpavlik | OK, thanks a lot. | 07:42 |
jpavlik | so it is same problem here https://review.openstack.org/#/c/346810/ | 07:42 |
jpavlik | but this must be done by Thierry | 07:42 |
AJaeger | jpavlik: that change has "Depends-On: I662a011f0fcf2f5694da058df7e3c08a6493e3b6" | 07:42 |
*** vincentll has joined #openstack-infra | 07:42 | |
AJaeger | And the dependent change has not merged. And indeed, once that dependent change has merged, ttx needs to toggle the +W. | 07:43 |
openstackgerrit | Merged openstack-infra/project-config: Ironic: Make ironic-multitenant-network a voting job https://review.openstack.org/350703 | 07:43 |
AJaeger | yolanda: care to review https://review.openstack.org/#/c/346674/ as well, please? | 07:43 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute015.vanilla metadata for better tracking https://review.openstack.org/351543 | 07:44 |
jpavlik | OK, thanks Andreas | 07:44 |
ttx | Let me see | 07:44 |
yolanda | sure | 07:44 |
ttx | yeah, dependent change must merge first | 07:45 |
*** karthik__ has quit IRC | 07:45 | |
openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Fix senlin integration test job https://review.openstack.org/351544 | 07:46 |
AJaeger | ttx, should be merged in half an hour ;). Thanks, yolanda | 07:47 |
jpavlik | Thanks guys! | 07:47 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for baremetal01.vanilla https://review.openstack.org/351545 | 07:49 |
*** markmcd has quit IRC | 07:51 | |
*** aeng has quit IRC | 07:51 | |
AJaeger | yolanda: do you have time for these older ones as well, please? https://review.openstack.org/350616 and https://review.openstack.org/350113 https://review.openstack.org/#/c/346953/ https://review.openstack.org/#/c/317672/ | 07:51 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute033.vanilla metadata for better tracking https://review.openstack.org/351547 | 07:52 |
yolanda | AJaeger, sure | 07:52 |
*** markmcd has joined #openstack-infra | 07:53 | |
AJaeger | thanks! | 07:55 |
*** shardy has joined #openstack-infra | 07:55 | |
*** dtantsur|afk is now known as dtantsur | 07:56 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update compute032.vanilla metadata for better tracking https://review.openstack.org/351551 | 07:57 |
openstackgerrit | Merged openstack-infra/project-config: Adding OpenStack-Salt formula kubernetes https://review.openstack.org/346674 | 07:59 |
*** zzzeek has quit IRC | 08:00 | |
*** rossella_s has quit IRC | 08:00 | |
*** zzzeek has joined #openstack-infra | 08:00 | |
*** rossella_s has joined #openstack-infra | 08:00 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute025.vanilla metadata for better tracking https://review.openstack.org/351553 | 08:02 |
*** migi_ is now known as migi | 08:02 | |
openstackgerrit | Merged openstack-infra/project-config: provide a manual override mode https://review.openstack.org/350616 | 08:03 |
*** openstackgerrit has quit IRC | 08:03 | |
*** openstackgerrit has joined #openstack-infra | 08:03 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/351555 | 08:04 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute036.vanilla metadata for better tracking https://review.openstack.org/351558 | 08:08 |
*** thorst has joined #openstack-infra | 08:08 | |
openstackgerrit | Merged openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 08:09 |
openstackgerrit | Merged openstack-infra/project-config: grafana: split neutron integrated gate dashboard https://review.openstack.org/346953 | 08:10 |
openstackgerrit | Merged openstack-infra/project-config: Add python jobs to pyeclib project https://review.openstack.org/317672 | 08:10 |
*** mhickey has joined #openstack-infra | 08:10 | |
AJaeger | rcarrillocruz: could you review https://review.openstack.org/#/c/350397/ today, please? It adds python jobs to a a docs tool repo... | 08:10 |
rcarrillocruz | +2+a | 08:11 |
*** jaosorior has quit IRC | 08:13 | |
*** thorst has quit IRC | 08:13 | |
AJaeger | thanks! | 08:13 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Replace gate-{name}-tox-{envlist} with gate-{name}-tox-{envlist}-{node} https://review.openstack.org/351219 | 08:13 |
*** shardy has quit IRC | 08:14 | |
*** jaosorior has joined #openstack-infra | 08:14 | |
*** armax has joined #openstack-infra | 08:14 | |
*** shardy has joined #openstack-infra | 08:14 | |
*** yaume has joined #openstack-infra | 08:14 | |
*** nmagnezi has quit IRC | 08:15 | |
*** tesseract- has joined #openstack-infra | 08:16 | |
*** Goneri has joined #openstack-infra | 08:16 | |
*** armax has quit IRC | 08:18 | |
*** jpavlik has quit IRC | 08:21 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add controller00.vanilla metadata for better tracking https://review.openstack.org/351563 | 08:21 |
*** asettle has joined #openstack-infra | 08:21 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute035.vanilla https://review.openstack.org/351567 | 08:24 |
*** sflanigan has quit IRC | 08:26 | |
openstackgerrit | Merged openstack-infra/project-config: Add py27, py35 jobs for openstack-doc-tools https://review.openstack.org/350397 | 08:26 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack of compute031.vanilla https://review.openstack.org/351570 | 08:26 |
*** ggnel_t has joined #openstack-infra | 08:27 | |
*** lucas|dinner is now known as lucasagomes | 08:28 | |
openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/351555 | 08:28 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP for baremetal00 https://review.openstack.org/351291 | 08:29 |
*** chem has quit IRC | 08:30 | |
*** chem has joined #openstack-infra | 08:30 | |
*** d0ugal has quit IRC | 08:32 | |
openstackgerrit | Merged openstack-dev/pbr: Expose deb version to match exposing rpm version https://review.openstack.org/350861 | 08:32 |
*** chem has quit IRC | 08:32 | |
*** d0ugal has joined #openstack-infra | 08:33 | |
*** d0ugal has quit IRC | 08:33 | |
*** d0ugal has joined #openstack-infra | 08:33 | |
*** d0ugal has quit IRC | 08:33 | |
*** d0ugal has joined #openstack-infra | 08:33 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute007.vanilla https://review.openstack.org/351574 | 08:35 |
*** ams__ has joined #openstack-infra | 08:35 | |
vsaienko | infra team, I'm wondered if it is allowed to extend sudo rights for temepst user at the gates job? | 08:35 |
*** sdake has quit IRC | 08:37 | |
*** Guest73548 is now known as flaper87 | 08:39 | |
*** hichihara has quit IRC | 08:39 | |
*** flaper87 is now known as Guest78641 | 08:39 | |
*** markvoelker has joined #openstack-infra | 08:40 | |
*** Guest78641 has quit IRC | 08:44 | |
*** markvoelker has quit IRC | 08:45 | |
openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: Add dstat to gate-tempest-dsvm-monasca-log job https://review.openstack.org/351580 | 08:50 |
*** acoles_ is now known as acoles | 08:52 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for {name}-branch-tarball https://review.openstack.org/351585 | 08:56 |
*** narayrak has quit IRC | 08:59 | |
*** narayrak has joined #openstack-infra | 09:00 | |
*** itisha has joined #openstack-infra | 09:01 | |
*** pt_15 has joined #openstack-infra | 09:02 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for grenade tests https://review.openstack.org/351586 | 09:02 |
*** chem has joined #openstack-infra | 09:02 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for gate-horizon-selenium-headless https://review.openstack.org/351587 | 09:04 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update controller00.chocolate metadata for better tracking https://review.openstack.org/351589 | 09:07 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for {pipeline}-kolla-kubernetes-{action}-{distro}-{type} https://review.openstack.org/351590 | 09:07 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for kolla https://review.openstack.org/351590 | 09:08 |
*** amitgandhinz has joined #openstack-infra | 09:09 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute002.chocolate metadata for better tracking https://review.openstack.org/351591 | 09:10 |
*** thorst has joined #openstack-infra | 09:10 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for bashate job https://review.openstack.org/351597 | 09:13 |
openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Position archived items at the bottom of worklists https://review.openstack.org/351265 | 09:13 |
*** amitgandhinz has quit IRC | 09:14 | |
*** e0ne has joined #openstack-infra | 09:14 | |
yolanda | hi AJaeger, i have a problem with one project, tripleo-validations | 09:14 |
yolanda | we need to consume jsonschema | 09:14 |
yolanda | in global-requirements is set to global-requirements.txt:jsonschema>=2.0.0,<3.0.0,!=2.5.0 # MIT | 09:15 |
yolanda | but upper requirements sets to 2.5.0 | 09:15 |
*** Goneri has quit IRC | 09:15 | |
*** sambetts|afk is now known as sambetts | 09:15 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for docs publishing https://review.openstack.org/351599 | 09:16 |
AJaeger | yolanda: let me check... | 09:17 |
AJaeger | yolanda: saw your comment that you solved it yourself | 09:17 |
*** thorst has quit IRC | 09:18 | |
*** Goneri has joined #openstack-infra | 09:18 | |
yolanda | yep, my fault | 09:18 |
AJaeger | no worries | 09:18 |
AJaeger | fungi, converted jobs that use tox and might use constraints (so, no infra jobs) to use proper zuul-cloner. | 09:22 |
*** rbuzatu_ has quit IRC | 09:28 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute094.chocolate metadata for better tracking https://review.openstack.org/351606 | 09:28 |
*** javeriak has quit IRC | 09:29 | |
*** claudiub has joined #openstack-infra | 09:30 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute001.chocolate metadata for better tracking https://review.openstack.org/351609 | 09:33 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute059.chocolate metadata for better tracking https://review.openstack.org/351610 | 09:35 |
*** flaper87 has joined #openstack-infra | 09:40 | |
*** flaper87 has quit IRC | 09:40 | |
*** flaper87 has joined #openstack-infra | 09:40 | |
*** kzaitsev_mb has joined #openstack-infra | 09:43 | |
*** amoralej is now known as amoralej|brb | 09:46 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute058.chocolate metadata for better tracking https://review.openstack.org/351614 | 09:46 |
*** senk has joined #openstack-infra | 09:47 | |
senk | someone able to tell me why jenkins is not testing https://review.openstack.org/#/c/350222/ ? | 09:48 |
*** markmcd has quit IRC | 09:48 | |
*** rbuzatu has joined #openstack-infra | 09:49 | |
openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 09:50 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute053.chocolate metadata for better tracking https://review.openstack.org/351616 | 09:51 |
*** rbuzatu has quit IRC | 09:53 | |
*** _nadya_ has joined #openstack-infra | 09:55 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute060.chocolate metadata for better tracking https://review.openstack.org/351622 | 09:58 |
*** _nadya_ has quit IRC | 09:58 | |
AJaeger | senk: interesting ;( Jenkins never tested it. Let me dig a bit... | 10:00 |
senk | thanks AJaeger | 10:00 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute082.chocolate metadata for better tracking https://review.openstack.org/351623 | 10:00 |
*** jcoufal has joined #openstack-infra | 10:00 | |
*** javeriak has joined #openstack-infra | 10:03 | |
*** _nadya_ has joined #openstack-infra | 10:03 | |
*** _nadya_ has quit IRC | 10:03 | |
AJaeger | senk: We need to change infra setup, missed one place. Problem is that for a job to be tested, we need at least one job. And the jobs are setup right now that not a single job runs for your change. | 10:03 |
senk | AJaeger: ah okay.. but there are existing jobs for the openstack/kolla project in general? | 10:04 |
*** amoralej|brb is now known as amoralej | 10:06 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix kolla docs skip https://review.openstack.org/351626 | 10:06 |
AJaeger | senk: ^ | 10:06 |
AJaeger | senk: we renamed the job from gate-kolla-docs to gate-kolla-docs-ubuntu-xenial (and ...-trusty) and the skip-if was forgotten. | 10:06 |
AJaeger | yolanda, rcarrillocruz: Could you review 351626, please? | 10:06 |
rcarrillocruz | lgtm | 10:08 |
AJaeger | senk: once that change is in and live (will be live for sure 1h after merge), add a "recheck" comment to your change and then ask a core to either add another +W or toggle it - remove it and add it back again. Then your change hsould merge | 10:08 |
AJaeger | senk: but please do it all in order ;) | 10:08 |
senk | AJaeger: of course :) thanks for your help! | 10:09 |
*** zhurong_ has quit IRC | 10:10 | |
*** amitgandhinz has joined #openstack-infra | 10:10 | |
AJaeger | senk: sorry for this, we missed this when we moved jobs from trusty to xenial | 10:10 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute005.chocolate metadata for better tracking https://review.openstack.org/351629 | 10:10 |
senk | AJaeger: nevermind, was only a very small change. | 10:11 |
*** yanyanhu has quit IRC | 10:12 | |
AJaeger | consistency counts - git.openstack.org is really our master, github is just a mirror... | 10:13 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute000.chocolate metadata for better tracking https://review.openstack.org/351631 | 10:14 |
*** amitgandhinz has quit IRC | 10:14 | |
*** asselin has quit IRC | 10:15 | |
yolanda | AJaeger, approved | 10:15 |
AJaeger | thanks, yolanda | 10:15 |
*** armax has joined #openstack-infra | 10:15 | |
*** thorst has joined #openstack-infra | 10:16 | |
*** armax has quit IRC | 10:21 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute039.chocolate metadata for better tracking https://review.openstack.org/351634 | 10:22 |
*** thorst has quit IRC | 10:23 | |
senk | AJaeger: there are more occurences in kolla especially in the codebase.. try to fix that later... | 10:23 |
*** yamamot__ has quit IRC | 10:24 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute007.chocolate metadata for better tracking https://review.openstack.org/351635 | 10:24 |
openstackgerrit | Thierry Carrez proposed openstack-infra/release-tools: Initial version for ACL manager https://review.openstack.org/351637 | 10:28 |
*** Goneri has quit IRC | 10:29 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute047.chocolate metadata for better tracking https://review.openstack.org/351638 | 10:30 |
*** Goneri has joined #openstack-infra | 10:31 | |
openstackgerrit | Merged openstack-infra/project-config: Fix kolla docs skip https://review.openstack.org/351626 | 10:32 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute008.chocolate metadata for better tracking https://review.openstack.org/351642 | 10:32 |
dtantsur | folks, how to forcibly reset "This change depends on a change that failed to merge." on a change? | 10:33 |
dtantsur | is "recheck" enough? | 10:33 |
*** _nadya_ has joined #openstack-infra | 10:34 | |
*** _nadya_ has quit IRC | 10:35 | |
*** _nadya_ has joined #openstack-infra | 10:35 | |
openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: ec2api experimental job: fix rally job definition https://review.openstack.org/351643 | 10:39 |
*** tosky has joined #openstack-infra | 10:43 | |
jroll | dtantsur: it is, that happens when a lower patch in the chain is updated while using depends-on | 10:43 |
dtantsur | sigh | 10:44 |
AJaeger | dtantsur: yes, recheck should be fine. | 10:44 |
*** amotoki has quit IRC | 10:44 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute083.chocolate metadata for better tracking https://review.openstack.org/351645 | 10:46 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute018.chocolate metadata for better tracking https://review.openstack.org/351647 | 10:49 |
*** javeriak has quit IRC | 10:50 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute003.chocolate metadata for better tracking https://review.openstack.org/351649 | 10:51 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute087.chocolate https://review.openstack.org/351651 | 10:54 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute023.chocolate metadata for better tracking https://review.openstack.org/351653 | 10:57 |
*** Ravikiran_K has quit IRC | 10:58 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute021.chocolate metadata for better tracking https://review.openstack.org/351655 | 10:59 |
*** jcoufal has quit IRC | 11:01 | |
openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Use ssh-agent instead of ugly ssh-wrapper scripts https://review.openstack.org/331148 | 11:01 |
openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Add ability to use SSH instead of HTTP to get refs https://review.openstack.org/246433 | 11:01 |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute049.chocolate metadata for better tracking https://review.openstack.org/351658 | 11:04 |
*** rhallisey has joined #openstack-infra | 11:07 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute062.chocolate metadata for better tracking https://review.openstack.org/351661 | 11:08 |
*** amitgandhinz has joined #openstack-infra | 11:11 | |
*** lucasagomes is now known as lucas-hungry | 11:11 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute015.chocolate https://review.openstack.org/351663 | 11:11 |
*** amitgandhinz has quit IRC | 11:15 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Update iLO IP and rack for compute017.chocolate https://review.openstack.org/351664 | 11:16 |
*** armax has joined #openstack-infra | 11:18 | |
*** yamamoto has joined #openstack-infra | 11:19 | |
*** thorst has joined #openstack-infra | 11:20 | |
*** javeriak has joined #openstack-infra | 11:20 | |
*** rbuzatu has joined #openstack-infra | 11:21 | |
*** armax has quit IRC | 11:22 | |
*** tpsilva has joined #openstack-infra | 11:24 | |
*** ganesan has joined #openstack-infra | 11:25 | |
*** florianf has joined #openstack-infra | 11:27 | |
*** thorst has quit IRC | 11:28 | |
ganesan | I am getting following error when nodepool is launching node | 11:28 |
ganesan | ERROR nodepool.NodeLauncher: Timeout launching node id: 23480 in provider: local_02 error: Timeout waiting for ssh access | 11:28 |
*** karlos_ has joined #openstack-infra | 11:28 | |
*** sdague has joined #openstack-infra | 11:30 | |
*** jkilpatr has joined #openstack-infra | 11:31 | |
*** zhurong has joined #openstack-infra | 11:31 | |
karlos_ | Hello, can someone please help me with following question ? I am runnning one of 3rd party CI's and I am trying to configure additional jobs via jenkins job builder in yaml format. I would like to use and change some variables, like using small shell script. MY_IP_ADDRESS=`ip route get 8.8.8.8 | awk '{print $NF; exit}'` echo "$MY_IP_ADDRESS ubuntu" | sudo tee -a /etc/hosts. This is working fine when job is configured in j | 11:31 |
*** Srinu has quit IRC | 11:34 | |
*** javeriak has quit IRC | 11:35 | |
openstackgerrit | Monty Taylor proposed openstack/os-client-config: Add test for precedence rules https://review.openstack.org/351668 | 11:36 |
*** javeriak has joined #openstack-infra | 11:37 | |
*** ldnunes has joined #openstack-infra | 11:39 | |
*** gouthamr has joined #openstack-infra | 11:43 | |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Migrate functional tests to betamax https://review.openstack.org/344397 | 11:44 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Stop creating cloud objects in functional tests https://review.openstack.org/344396 | 11:44 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Move list_magnum_services to OperatorCloud https://review.openstack.org/344395 | 11:44 |
mordred | rcarrillocruz: keystoneauth made a release with the new yamljsonserializer - so we don't need to forklift it in anymore (thanks for approving the patch though :) ) | 11:45 |
mordred | I have now abandoned it | 11:45 |
*** yamamoto has quit IRC | 11:45 | |
*** asettle has quit IRC | 11:45 | |
*** rfolco_ has joined #openstack-infra | 11:46 | |
*** yamamoto has joined #openstack-infra | 11:47 | |
*** thorst has joined #openstack-infra | 11:48 | |
*** yamamoto has quit IRC | 11:51 | |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Lay the groundwork for multi-cloud in unit tests https://review.openstack.org/343916 | 11:52 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add new test with betamax for create flavors https://review.openstack.org/302192 | 11:52 |
*** ganesan has quit IRC | 11:53 | |
*** gordc has joined #openstack-infra | 11:56 | |
zigo | mordred: Hi there! Could you please review https://review.openstack.org/#/c/346095/ ? | 11:56 |
zigo | mordred: It's the last infra patch before we can start adding packages and do the actual work! :) | 11:56 |
mordred | woot! | 11:58 |
*** asettle has joined #openstack-infra | 11:58 | |
*** psilvad has joined #openstack-infra | 11:58 | |
mordred | zigo: looks good in general - although I think we want echo 'deb http://mirror.dfw.rax.openstack.org/debian-openstack jessie-newton main' >/etc/a | 11:59 |
mordred | pt/sources.list.d/jessie-newto | 11:59 |
openstackgerrit | Thierry Carrez proposed openstack-infra/release-tools: Initial version for ACL manager https://review.openstack.org/351637 | 11:59 |
mordred | to not be hardcoded to dfw ... one sec while I look for a thing you can use | 11:59 |
*** dingyichen has quit IRC | 11:59 | |
*** weshay has joined #openstack-infra | 11:59 | |
*** rossella_s has quit IRC | 12:00 | |
zigo | mordred: I believe that's not even needed, considering that it should be there by default in the image. | 12:00 |
*** rossella_s has joined #openstack-infra | 12:00 | |
mordred | zigo: no, I think it is - I do not believe we'll have that enabled by default unless I'm missing something | 12:01 |
mordred | zigo: but luckily - it's easy to grab the right one ... | 12:01 |
zigo | mordred: I trust you! :) | 12:01 |
zigo | mordred: I'm also interested to know how. | 12:01 |
AJaeger | mordred: see https://review.openstack.org/341856 - that file should be used | 12:01 |
mordred | oh! | 12:02 |
mordred | zigo: I stand corrected. thanks AJaeger | 12:02 |
AJaeger | So, just copy or link to it. | 12:02 |
AJaeger | But don't create one of your own. | 12:02 |
zigo | Just remove the thing, IMO. | 12:02 |
AJaeger | that only merged on Tuesday, hope we have images with these in now | 12:02 |
*** Goneri has quit IRC | 12:03 | |
mordred | zigo: so: cp /etc/apt/sources.list.available.d/debian-openstack-newton.list /etc/apt/sources.list.d | 12:03 |
mordred | zigo: (left it on the review) | 12:04 |
openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 12:04 |
zigo | Oh... | 12:04 |
rcarrillocruz | mordred: sweet | 12:05 |
openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 12:05 |
zigo | mordred: There you go! :) | 12:05 |
mordred | yes! | 12:05 |
mordred | zigo: looks great to me - maybe AJaeger will give you the other +2 | 12:05 |
zigo | :) | 12:06 |
* AJaeger checks | 12:06 | |
*** rodrigods has quit IRC | 12:07 | |
*** rodrigods has joined #openstack-infra | 12:07 | |
AJaeger | +2A | 12:08 |
AJaeger | wait - where's the governance change for that? | 12:09 |
*** amoralej is now known as amoralej|lunch | 12:09 | |
AJaeger | zigo: Why did you remove the "Needed-By: Ie96af7d7d9717ef3fb634f1b676a05fd4196121a" line fro mthe commit message? Could you add it back again, please? | 12:10 |
AJaeger | That was in patchset 9 | 12:10 |
*** baoli has joined #openstack-infra | 12:11 | |
*** amitgandhinz has joined #openstack-infra | 12:11 | |
*** alaski is now known as lascii | 12:12 | |
*** julim has joined #openstack-infra | 12:12 | |
*** baoli_ has joined #openstack-infra | 12:12 | |
AJaeger | so, changed to -1 and wait for that one to be added back... | 12:12 |
openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci: Set --dev option for dlrn execution instead of env var https://review.openstack.org/351682 | 12:13 |
AJaeger | zigo, mordred I'll just do it quickly myself... | 12:13 |
*** dprince has joined #openstack-infra | 12:13 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 12:13 |
AJaeger | mordred: now you can +2A ;) | 12:13 |
mordred | \o/ | 12:14 |
*** Goneri has joined #openstack-infra | 12:14 | |
*** mkoderer_ has joined #openstack-infra | 12:15 | |
*** amitgandhinz has quit IRC | 12:16 | |
*** baoli has quit IRC | 12:16 | |
*** cgross has quit IRC | 12:17 | |
*** asettle has quit IRC | 12:17 | |
*** lmiccini_ has joined #openstack-infra | 12:17 | |
*** lmiccini has quit IRC | 12:17 | |
*** pradk has joined #openstack-infra | 12:18 | |
*** mkoderer has quit IRC | 12:18 | |
*** cgross has joined #openstack-infra | 12:18 | |
*** huanxie has quit IRC | 12:19 | |
*** ggnel_t has quit IRC | 12:21 | |
zigo | gr8 !!! :) | 12:21 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add plumbing for betamax functional tests. https://review.openstack.org/344397 | 12:23 |
*** lucas-hungry is now known as lucasagomes | 12:24 | |
*** rbuzatu has quit IRC | 12:25 | |
*** amotoki has joined #openstack-infra | 12:25 | |
*** aviau has quit IRC | 12:25 | |
*** aviau has joined #openstack-infra | 12:25 | |
*** piet_ has joined #openstack-infra | 12:26 | |
*** amotoki has quit IRC | 12:29 | |
*** ramishra has quit IRC | 12:31 | |
*** savihou has joined #openstack-infra | 12:32 | |
senk | AJaeger: FYI Jenkins works now fine on kolla docs | 12:32 |
*** ramishra has joined #openstack-infra | 12:32 | |
AJaeger | senk: glad to hear! Thanks | 12:34 |
jroll | hey infra folks, we have some post jobs that publish to tarballs.o.o, and one doesn't seem to be working correctly | 12:34 |
jroll | here we haven't seen updates since june 14: http://tarballs.openstack.org/ironic-python-agent/tinyipa/ | 12:34 |
jroll | but the job seems successful: http://logs.openstack.org/27/277d97ba2b27fdd193b8daf91b375241d4f2c385/post/ironic-python-agent-buildimage-tinyipa/1ce0376/console.html | 12:35 |
jroll | and appears similar to the job that is publishing okay: http://logs.openstack.org/27/277d97ba2b27fdd193b8daf91b375241d4f2c385/post/ironic-python-agent-buildimage-coreos/2c32993/console.html | 12:35 |
AJaeger | jroll: did you check the ansible files as well? | 12:35 |
jroll | AJaeger: ah | 12:36 |
AJaeger | jroll: http://logs.openstack.org/27/277d97ba2b27fdd193b8daf91b375241d4f2c385/post/ironic-python-agent-buildimage-tinyipa/1ce0376/_zuul_ansible/ansible_log.txt | 12:36 |
AJaeger | last line is the one to look at | 12:36 |
jroll | yeah, I see it | 12:36 |
* jroll hrms, will come back if I need help | 12:36 | |
jroll | thanks AJaeger ! | 12:36 |
AJaeger | jroll: best talk with jeblair and pabelanger once you're back ;) | 12:37 |
jroll | sure :) | 12:37 |
*** rlandy has joined #openstack-infra | 12:43 | |
*** piet_ has quit IRC | 12:45 | |
*** thorst_ has joined #openstack-infra | 12:45 | |
*** amotoki has joined #openstack-infra | 12:47 | |
*** Goneri has quit IRC | 12:48 | |
*** thorst has quit IRC | 12:49 | |
*** xyang1 has joined #openstack-infra | 12:49 | |
*** jcoufal has joined #openstack-infra | 12:52 | |
*** amotoki has quit IRC | 12:52 | |
*** ccamacho is now known as ccamacho|lunch | 12:53 | |
*** jaosorior has quit IRC | 12:54 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/puppet: enable integration jobs on puppet-barbican https://review.openstack.org/351732 | 12:55 |
*** psilvad has quit IRC | 12:55 | |
*** markvoelker has joined #openstack-infra | 12:56 | |
*** matt-borland has joined #openstack-infra | 12:57 | |
*** rbuzatu has joined #openstack-infra | 12:57 | |
*** psilvad has joined #openstack-infra | 12:58 | |
pabelanger | AJaeger: mordred: zigo: 346095 is going to fail to cp source.list, you need to use sudo | 12:58 |
zigo | Oh ! | 12:58 |
mordred | blerg | 12:58 |
zigo | bad me ... | 12:58 |
mordred | pabelanger: can't you be awake all the time? | 12:58 |
zigo | ;P | 12:58 |
pabelanger | zigo: -2 it from merging so you can update it | 12:59 |
pabelanger | mordred: if only | 12:59 |
pabelanger | also, morning | 13:00 |
zigo | pabelanger: Is there a /etc/apt/sources.list.available.d/debian-openstack-newton-backports.list file too? | 13:00 |
zigo | I guess, not yet, since the repo is currently empty, right? | 13:00 |
openstackgerrit | Darragh Bailey proposed openstack-infra/git-review: Limit reviews by branch when specified https://review.openstack.org/312056 | 13:00 |
*** asselin has joined #openstack-infra | 13:00 | |
*** tldr_ has joined #openstack-infra | 13:01 | |
openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 13:01 |
zigo | mordred: pabelanger: Should be fine now. | 13:01 |
pabelanger | zigo: no, we'll likely add jessie-backports into the exist debian-openstack-newton.list, if you are fine with that | 13:01 |
zigo | Fine for me. | 13:01 |
AJaeger | rcarrillocruz, mordred : Could you +2A pabelangers change for moving tox t oxenial, please? https://review.openstack.org/351219 | 13:01 |
*** psilvad has quit IRC | 13:02 | |
AJaeger | pabelanger: I fixed the merge conflict on it and the one problem it still had ^. LGTM now - let's merge! | 13:02 |
*** Goneri has joined #openstack-infra | 13:03 | |
mordred | AJaeger: lgtm | 13:03 |
*** ddieterly has joined #openstack-infra | 13:04 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add $LSBDISTCODENAME-newton-backports for debian-openstack mirror list https://review.openstack.org/351734 | 13:04 |
pabelanger | zigo: ^ will add jessie-backports | 13:04 |
AJaeger | thanks, mordred | 13:04 |
pabelanger | AJaeger: ++ | 13:04 |
openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Add 'ceph status' and tripleo-overcloud-passwords to host_info https://review.openstack.org/348862 | 13:05 |
*** _nadya_ has quit IRC | 13:07 | |
*** yamamoto has joined #openstack-infra | 13:08 | |
*** ddieterly has quit IRC | 13:08 | |
*** piet has joined #openstack-infra | 13:09 | |
openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Add 'ceph status' and tripleo-overcloud-passwords to host_info https://review.openstack.org/348862 | 13:09 |
zigo | mordred: Can you +workflow https://review.openstack.org/346095 ? | 13:10 |
zigo | So we can get going ... :P | 13:10 |
*** javeriak has quit IRC | 13:10 | |
*** kzaitsev_mb has quit IRC | 13:11 | |
*** sdake has joined #openstack-infra | 13:11 | |
*** esberglu has joined #openstack-infra | 13:12 | |
*** amitgandhinz has joined #openstack-infra | 13:12 | |
*** psilvad has joined #openstack-infra | 13:12 | |
mordred | zigo: done | 13:12 |
zigo | :) | 13:12 |
*** asettle has joined #openstack-infra | 13:13 | |
pabelanger | zigo: have you done any commit to deb-openstack-pkg-tools recently? | 13:13 |
zigo | pabelanger: My colleague did a dumb commit today, yes, and it was published. | 13:14 |
pabelanger | cool, let me check something | 13:14 |
*** mdrabe has joined #openstack-infra | 13:14 | |
zigo | I'm not sure why they wanted to do that. | 13:14 |
openstackgerrit | Merged openstack-infra/project-config: Replace gate-{name}-tox-{envlist} with gate-{name}-tox-{envlist}-{node} https://review.openstack.org/351219 | 13:14 |
*** mkoderer has joined #openstack-infra | 13:14 | |
*** lmiccini has joined #openstack-infra | 13:15 | |
*** amitgandhinz has quit IRC | 13:17 | |
*** mriedem has joined #openstack-infra | 13:17 | |
*** mkoderer_ has quit IRC | 13:17 | |
*** lmiccini_ has quit IRC | 13:18 | |
vponomaryov | something wrong with zuul, it does not start jobs anymore | 13:18 |
*** cgross has quit IRC | 13:18 | |
*** rbuzatu has quit IRC | 13:18 | |
*** cgross has joined #openstack-infra | 13:18 | |
*** kien-ha has joined #openstack-infra | 13:19 | |
*** amoralej|lunch is now known as amoralej | 13:20 | |
sdake | pabelanger jeblair i'm told either of you have a solid enough understanding of afs shares in infra to review the patch i have in queu to add percona as a mirror | 13:21 |
*** mtanino has joined #openstack-infra | 13:21 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gpg signing job for reprepro https://review.openstack.org/343336 | 13:21 |
pabelanger | zigo: mordred: ^ should give us signed commits for the debian-openstack repo | 13:22 |
sdake | pabelanger jeblair any chance you folks could burn some cycles on that soon - I have a few more to do and want to make sure the work is correct as I am not entirely clear if it is | 13:22 |
pabelanger | sdake: yes, it is on my list. I have to land a few other mirrors before I can get to that. | 13:22 |
pabelanger | sdake: but it does look correct at first glance | 13:22 |
sdake | pabelanger roger didnt know you had ot land other mirrors | 13:22 |
sdake | pabelanger cool thats what i was lookinig for | 13:22 |
sdake | i'll get working on submitting my other patches | 13:22 |
sdake | and you can land them when convienent :) | 13:22 |
pabelanger | ya, I need to get our gem mirror online, puppet team has been waiting for a few months on that | 13:23 |
sdake | does it take few months to get mirrors in place ? | 13:23 |
sdake | (or was base dependency softwre not available) | 13:23 |
pabelanger | sdake: lets see if we can find another infra-root to do it, it should be a good review for people wanting to see how we setup our AFS bit | 13:23 |
pabelanger | sdake: just takes time to schedule the work | 13:24 |
sdake | pabelanger there is no rush now that i know the patch looks correct | 13:24 |
sdake | would be nice for it not to tkae months obviously | 13:24 |
sdake | but understand its a big request :) | 13:24 |
*** pt_15 has quit IRC | 13:26 | |
*** senk has quit IRC | 13:26 | |
sdake | pabelanger just a heads up - we have about 5-6 centos mirrors needed and 2-3 ubuntu mirrors | 13:27 |
zigo | pabelanger: So, you're not producing a InRelease file? :/ | 13:27 |
sdake | pabelanger the good news is other teams will probably want these mirrors as well | 13:28 |
zigo | pabelanger: Example InRelease file: http://ftp.fr.debian.org/debian/dists/testing/InRelease | 13:29 |
zigo | Best would be to have both. | 13:29 |
*** _ari_ has joined #openstack-infra | 13:29 | |
* zigo comments in the patch | 13:29 | |
*** itisha has quit IRC | 13:30 | |
pabelanger | zigo: sure, if you tell me how to do it, I can add it | 13:31 |
pabelanger | sdake: sure | 13:31 |
openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci: Set --use-public option for dlrn execution instead of env var https://review.openstack.org/351682 | 13:32 |
zigo | pabelanger: Done. FYI, the advantage is that InRelease updates are atomic (ie: it is a single file instead of 2 files with Release and Release.gpg) | 13:32 |
zigo | So the repo itself becomes atomically updatable. | 13:32 |
zigo | (and mirrorable) | 13:32 |
*** Ravikiran_K has joined #openstack-infra | 13:33 | |
*** ccamacho|lunch is now known as ccamacho | 13:33 | |
mordred | pabelanger: you wanna wait to update based on zigo's thing? | 13:33 |
*** rbuzatu has joined #openstack-infra | 13:33 | |
zigo | pabelanger: I commented in. | 13:34 |
Shrews | pabelanger: I *think* those random zookeeper timeout errors we see in nodepool might be we just aren't waiting long enough for the java stuff | 13:34 |
Shrews | pabelanger: the kazoo python client waits only 10sec by default | 13:34 |
Shrews | cccccceviitdrrtdgljjvjfjbrenjgcrckircdrlubgl | 13:34 |
Shrews | doh | 13:34 |
zigo | gate-project-config-compare-xml really takes forever ... :/ | 13:35 |
openstackgerrit | Merged openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 13:35 |
*** knikolla has joined #openstack-infra | 13:37 | |
*** mat128 is now known as mat128|away | 13:37 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gpg signing job for reprepro https://review.openstack.org/343336 | 13:38 |
pabelanger | updated for zigo comments and added jessie-backports too | 13:39 |
pabelanger | Shrews: Ah, that would explain it. We likely need to increase that for slower clouds | 13:39 |
zigo | How long until I can git clone https://git.openstack.org/openstack/deb-auto-backports and work on it? | 13:40 |
openstackgerrit | Sam Betts proposed openstack-infra/project-config: Clean up TinyIPA chroot directories in POST job https://review.openstack.org/351742 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: TestSubCommand inherit from UpdateSubCommand. https://review.openstack.org/319619 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Don't expose YamlParser methods unnecessarily. https://review.openstack.org/319618 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move subcommand logic into subcommand classes. https://review.openstack.org/319617 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move 'update' yaml parsing to JenkinsJobs.execute. https://review.openstack.org/319616 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Factor XmlJobGenerator out of YamlParser. https://review.openstack.org/319623 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Disentangle YamlParser and ModuleRegistry classes https://review.openstack.org/319622 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove deprecated 'extended-choice' property. https://review.openstack.org/319621 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove YamlParser references from JJB modules. https://review.openstack.org/319620 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Be explicit about objects exported by modules. https://review.openstack.org/319625 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Cleanup various deprecation warnings. https://review.openstack.org/319624 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove unused builder.Builder.update_job method https://review.openstack.org/319752 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename Builder.delete_job to Builder.delete_jobs. https://review.openstack.org/319753 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename "parallelize" decorator to "concurrent" https://review.openstack.org/333780 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Merge builder.Builder and builder.Jenkins https://review.openstack.org/319754 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Test at more appropriate boundaries for update. https://review.openstack.org/319615 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move CacheStorage to its own module. https://review.openstack.org/319755 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Make internal builder.Jenkins attrs internal-only. https://review.openstack.org/319756 | 13:41 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move 'delete' yaml parsing to JenkinsJobs.execute. https://review.openstack.org/262954 | 13:41 |
pabelanger | zigo: after is merges, should take ~30mins for it to work | 13:41 |
*** wewe0901 has quit IRC | 13:42 | |
openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Allow extra time for initial ZooKeeper connection https://review.openstack.org/351745 | 13:45 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gpg signing job for reprepro https://review.openstack.org/343336 | 13:45 |
Shrews | pabelanger: hopefully that fixes it ^^^^ | 13:45 |
pabelanger | Shrews: ack | 13:45 |
openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Allow extra time for initial ZooKeeper connection https://review.openstack.org/351745 | 13:46 |
*** kgiusti has joined #openstack-infra | 13:47 | |
Shrews | cccccceviitdlrkngfetujiirrclijdkhguekvthitne | 13:47 |
Shrews | crap | 13:47 |
mordred | Shrews: you're winning | 13:47 |
Shrews | so much | 13:47 |
*** karlos_ has quit IRC | 13:48 | |
*** krtaylor has quit IRC | 13:48 | |
*** krtaylor has joined #openstack-infra | 13:50 | |
matt-borland | Hello...I would like someone to help review https://review.openstack.org/#/c/349077/ ...Splitting the Horizon integration test job into two jobs...thanks in advance! | 13:52 |
*** tonytan4ever has joined #openstack-infra | 13:53 | |
*** tonytan4ever has quit IRC | 13:55 | |
*** tonytan4ever has joined #openstack-infra | 13:55 | |
openstackgerrit | Merged openstack/os-client-config: Revert "Fix precedence for pass-in options" https://review.openstack.org/351394 | 13:56 |
openstackgerrit | Merged openstack/os-client-config: Pass the argparse data into to validate_auth https://review.openstack.org/351405 | 13:56 |
*** mriedem1 has joined #openstack-infra | 13:56 | |
*** amotoki has joined #openstack-infra | 13:57 | |
*** amitgandhinz has joined #openstack-infra | 13:57 | |
*** mriedem has quit IRC | 13:57 | |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-python-tripleoclient-buildimage jobs https://review.openstack.org/351278 | 13:57 |
fungi | crinkle: does my answer to your question on https://review.openstack.org/349023 make sense? | 13:58 |
openstackgerrit | Merged openstack/os-client-config: Add test for precedence rules https://review.openstack.org/351668 | 13:59 |
openstackgerrit | Aleksey Zvyagintsev proposed openstack-infra/project-config: [wip] https://review.openstack.org/351757 | 13:59 |
crinkle | fungi: yes, thank you | 14:00 |
*** _nadya_ has joined #openstack-infra | 14:00 | |
fungi | thanks for reviewing that | 14:00 |
*** kzaitsev_mb has joined #openstack-infra | 14:01 | |
fungi | just trying to get these flushed through since they're blocking the release team testing tag automation | 14:01 |
*** burgerk has joined #openstack-infra | 14:01 | |
*** mriedem1 is now known as mriedem | 14:02 | |
*** openstackgerrit has quit IRC | 14:03 | |
*** openstackgerrit has joined #openstack-infra | 14:03 | |
*** edmondsw has joined #openstack-infra | 14:04 | |
*** e0ne has quit IRC | 14:05 | |
*** e0ne has joined #openstack-infra | 14:05 | |
*** amitgandhinz has quit IRC | 14:06 | |
*** amitgandhinz has joined #openstack-infra | 14:06 | |
*** mkoderer has quit IRC | 14:08 | |
openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Fix requirements for broken os-client-config https://review.openstack.org/351762 | 14:08 |
*** cgross has quit IRC | 14:08 | |
Shrews | mordred: ^^^ | 14:08 |
*** ddieterly has joined #openstack-infra | 14:09 | |
*** lmiccini has quit IRC | 14:09 | |
mordred | Shrews: thanks! | 14:10 |
mordred | Shrews: I have +Ad | 14:10 |
*** mctaylor has joined #openstack-infra | 14:10 | |
*** PollyZ has joined #openstack-infra | 14:11 | |
*** coolsvap has quit IRC | 14:11 | |
*** rbrndt has joined #openstack-infra | 14:13 | |
azvyagintsev | mmedvede fudgi Could you please take a look into "PoC" patches https://review.openstack.org/#/c/351757/ https://review.openstack.org/#/c/344706/ ? | 14:15 |
*** mkoderer has joined #openstack-infra | 14:17 | |
*** tonytan4ever has quit IRC | 14:17 | |
*** jaosorior has joined #openstack-infra | 14:17 | |
*** tonytan4ever has joined #openstack-infra | 14:17 | |
zigo | pabelanger: Still no repo after 45 minutes, should I be worried? | 14:18 |
pabelanger | mordred: AJaeger: mind helping land https://review.openstack.org/#/c/351734/ that should be the last step for zigo | 14:18 |
*** lmiccini has joined #openstack-infra | 14:19 | |
openstackgerrit | Merged openstack-infra/puppet-jenkins: Add user.signingkey to .gitconfig https://review.openstack.org/349022 | 14:19 |
pabelanger | zigo: let me look | 14:19 |
*** cgross has joined #openstack-infra | 14:19 | |
*** armax has joined #openstack-infra | 14:20 | |
pabelanger | zigo: import has been running for 15mins, a lot of projects validate these days | 14:20 |
zigo | Okay. | 14:20 |
*** zz_dimtruck is now known as dimtruck | 14:20 | |
AJaeger | matt-borland: please have some horizon cores review that one as well | 14:20 |
matt-borland | AJaeger, robcresswell is the Horizon PTL. | 14:21 |
matt-borland | I can get more as well | 14:21 |
AJaeger | matt-borland: last time I looked robcresswell didn't +1 - so now I'll review ;) | 14:21 |
matt-borland | sorry...thanks for checking in on it :) | 14:21 |
mtreinish | infra-root: cfriesen_ pointed out on the -qa channel that there is data missing from the subunit2sql db (noticed via o-h) I have a feeling one of the workers isn't working. Can someone take a look at the logs and see if anything is up? | 14:23 |
fungi | looking | 14:23 |
AJaeger | matt-borland: so, both jobs should be run on mitaka branch and newer, correct? | 14:23 |
fungi | mtreinish: i rebooted logstash.o.o yesterday after log processor debug logs filled up the rootfs | 14:23 |
matt-borland | that's right AJaeger | 14:24 |
*** pcrews has quit IRC | 14:24 | |
mtreinish | fungi: this is going back a few days | 14:24 |
fungi | mtreinish: ahh, got it | 14:24 |
mtreinish | like the nova mitaka job doesn't have any results since Aug. 1st | 14:24 |
mtreinish | s/mitaka job/mitaka py27 job/ | 14:24 |
AJaeger | matt-borland: ok, then LGTM. | 14:24 |
*** armax has quit IRC | 14:25 | |
matt-borland | awseome AJaeger...thanks for that! | 14:25 |
*** pcrews has joined #openstack-infra | 14:25 | |
fungi | mtreinish: subunit-worker-A is running on worker01 and worker02, checking logs for them now | 14:25 |
*** pgadiya has quit IRC | 14:25 | |
*** senk has joined #openstack-infra | 14:25 | |
*** mctaylor has quit IRC | 14:26 | |
*** daemontool has joined #openstack-infra | 14:26 | |
*** yamamoto has quit IRC | 14:27 | |
*** zhurong has quit IRC | 14:27 | |
fungi | mtreinish: /var/log/subunit2sql/subunit-worker-A-debug.log is empty on worker01, but on worker02 it's full of http://paste.openstack.org/ | 14:27 |
mtreinish | fungi: I think you forgot the numbers on the paste :) | 14:28 |
mtreinish | fungi: hmm, the log also shouldn't be empty | 14:28 |
mtreinish | IIRC things are normally kinda verbose | 14:29 |
fungi | oops, http://paste.openstack.org/show/550608/ | 14:29 |
*** Jeffrey4l has quit IRC | 14:29 | |
*** Jeffrey4l has joined #openstack-infra | 14:29 | |
mtreinish | fungi: ok, so that means worker 2 probably isn't doing anything. But that shouldn't result in missing data. If it can't connect to the gearman server it just won't process any jobs | 14:30 |
fungi | worker02 had "receiving packet from" log entries up until 22:15:14 utc yesterday | 14:30 |
anteaya | fungi: you saw this failed in the gate, yeah? https://review.openstack.org/#/c/349023/ | 14:31 |
mtreinish | fungi: that would correspond to our last data in the db | 14:31 |
*** karthikp_ has joined #openstack-infra | 14:31 | |
mtreinish | which is from: 2016-08-04 22:12:43 | 14:31 |
fungi | mtreinish: oh, looks like subunit-worker-A-debug.log on worker01 has been empty since late may | 14:32 |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Keivox KMAP plugin https://review.openstack.org/351771 | 14:32 |
fungi | so no idea what's up there | 14:32 |
mtreinish | fungi: http://paste.openstack.org/show/550611/ | 14:32 |
fungi | anteaya: yeah, that's some fun bitrot | 14:32 |
anteaya | :( | 14:32 |
mtreinish | fungi: yeah that's weird, it's running but not logging. I'm going to assume that worker 1 is the problem child. | 14:33 |
mtreinish | and our data halted after your logstash.o.o reboot because worker 2 can't connect anymore | 14:33 |
AJaeger | sambetts: thanks for solving that puzzle! | 14:34 |
anteaya | fungi: bit rot in puppet-openstackid? | 14:34 |
prometheanfire | so, do we have a problem with log size? http://logs.openstack.org/71/349371/5/check/gate-cross-keystone-python27-db-ubuntu-xenial/e4eeaf6/console.html#_2016-08-05_07_15_27_879800 | 14:34 |
fungi | anteaya: i think it's another broken git repo on our git backend actually | 14:34 |
anteaya | ah | 14:35 |
fungi | anteaya: that's the same error we had last time when there was a repo that spontaneously lost its master branch | 14:35 |
anteaya | I missed that there are some broken git repos on our git backend | 14:35 |
fungi | anteaya: me too | 14:35 |
anteaya | I must have missed that error the other time | 14:35 |
fungi | i need to look into this once i'm done looking at subunit2sql | 14:35 |
anteaya | I don't know how it was fixed | 14:35 |
* anteaya waits | 14:36 | |
*** karthikp_ has quit IRC | 14:36 | |
fungi | anteaya: i found which git server had the broken repo content, blew it away, reran the createcgitrepos script and then retriggered a mirror sync from gerrit for that repo | 14:36 |
fungi | last time | 14:36 |
fungi | so this time will likely be similar | 14:36 |
*** Goneri has quit IRC | 14:36 | |
anteaya | oh I can check which git server | 14:36 |
AJaeger | voting time again - thanks anteaya for supervising again! | 14:36 |
mtreinish | fungi: I really wish we had something to go on worker-1. It feels like it was accepting jobs from gearman but never doing anything with it (which is a scary bug) | 14:37 |
anteaya | just cloning that repo should tell me, yeah? | 14:37 |
mtreinish | because that's the only thing I could think would cause the missing data | 14:37 |
fungi | anteaya: finding which git server is quick, but i want to audit all the git repos on all the git servers again to make sure this is the only one | 14:37 |
anteaya | AJaeger: thanks for caring, glad you also appreciate elections :) | 14:37 |
anteaya | fungi: ah okay | 14:37 |
fungi | mtreinish: i can strace the running process tehre | 14:37 |
mtreinish | fungi: yeah, I lets see if that shows anything | 14:38 |
mtreinish | if it is doing that it means we've likely lost all the data since the logstash reboot :( | 14:38 |
prometheanfire | are logs capped at 50MB? | 14:38 |
AJaeger | fungi, we discussed yesterday that stackalytics.o.o and stackalytics.com had different numbers. stackalaytics.o.o shows me "The data was last updated on 01 Aug 2016 23:48:34 UTC", so that could explain differences in stats | 14:38 |
*** pgadiya has joined #openstack-infra | 14:39 | |
fungi | mtreinish: looks like it was last restarted on may 20 at 14:39 and stopped logging after 02:24:08 on may 27 | 14:39 |
fungi | mtreinish: it's just sitting at a "futex(0x4092d40, FUTEX_WAIT_PRIVATE, 0, NULL" | 14:40 |
fungi | deadlocked maybe? | 14:40 |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Keivox KMAP plugin https://review.openstack.org/351771 | 14:40 |
mtreinish | fungi: it could have been misbehaving all this time. It would have been very difficult to tell, since worker 2 would fill in half the data | 14:40 |
fungi | mtreinish: claims to have an established gearman socket: tcp 0 0 23.253.73.200:40801 23.253.230.235:4730 ESTABLISHED | 14:41 |
mtreinish | fungi: hmm, the only explict lock file in the worker script is the pid lockfile which I doubt is that | 14:42 |
sambetts | AJaeger: yeah, that was a weird one, is there a way to get a notification if a post job like the IPA one fails? | 14:42 |
*** e0ne has quit IRC | 14:42 | |
mtreinish | fungi: yeah, that's the bit that's probably all wrong | 14:42 |
*** ramishra_ has joined #openstack-infra | 14:42 | |
*** ayoung_ZZZzzz is now known as ayoung | 14:42 | |
mtreinish | if it does that, but never actually writes anything to the db | 14:42 |
*** jdennis has joined #openstack-infra | 14:42 | |
fungi | mtreinish: i think that's a dead socket it's listening for | 14:42 |
fungi | i find no evidence it exists on the logstash.o.o side | 14:43 |
mtreinish | ah, ok. that would make sense | 14:43 |
fungi | so probably some sort of network interruption killed the logstash end of that open connection but failed to signal the worker end | 14:43 |
mtreinish | but it does kinda give some evidence that it was probably talking to gearman before the reboot | 14:43 |
fungi | also after rebooting the logstash server i see no indication anything is listening on 4730/tcp so we probably have a service that isn't set to start at boot there | 14:44 |
fungi | what normally listens on that port on the logstash server? | 14:44 |
*** yamamoto has joined #openstack-infra | 14:45 | |
*** ramishra has quit IRC | 14:45 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Keivox KMAP plugin https://review.openstack.org/351771 | 14:45 |
AJaeger | sambetts: not really, you could only check for it. For example via health.openstack.org | 14:46 |
mtreinish | fungi: that's how we have the workers configured: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-subunit-worker.yaml | 14:46 |
mtreinish | it's the same on the logstash side: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-log-worker.yaml | 14:46 |
fungi | mtreinish: like, it's a gearman server of some sort, just trying to figure out what the initscript for it is | 14:46 |
mtreinish | fungi: it would be the gearman server | 14:46 |
jeblair | it's a geard | 14:46 |
*** slaweq has quit IRC | 14:47 | |
fungi | yep, i just don't see any obvious gearman or geard initscript so it's presumably embedded in something... jenkins-log-client? | 14:47 |
mtreinish | fungi: yeah it's the gearman client | 14:47 |
fungi | looks like that script runs /usr/local/bin/log-gearman-client.py so maybe the client is a server | 14:47 |
mtreinish | the server is baked into the script iirc | 14:47 |
fungi | confusingly named ;) | 14:48 |
mtreinish | fungi: http://git.openstack.org/cgit/openstack-infra/puppet-log_processor/tree/files/log-gearman-client.py#n150 | 14:48 |
jeblair | yeah, it embeds a gear server | 14:48 |
fungi | okay, after starting jenkins-log-client manually, it's listening on 4730/tcp again | 14:49 |
*** eharney has joined #openstack-infra | 14:49 | |
jeblair | (but is also both a zmq and gearman client) | 14:49 |
fungi | so presumably worker02 will reconnect automatically. i have a feeling the process on worker01 needs a restart to clear that hung socket | 14:49 |
mtreinish | fungi: yeah, that will solve the lack of data since yesterday. I'm just a bit worried about the lost data | 14:49 |
fungi | looks like we have /etc/rc2.d/S20jenkins-log-client symlinked to that initscript, so maybe we have a startup race that prevented it from starting correctly at boot | 14:50 |
sambetts | AJaeger: what project do the post jobs show up under? I don't see the buildimage-* jobs under http://status.openstack.org/openstack-health/#/g/project/openstack~2Fironic-python-agent | 14:50 |
*** mkoderer_ has joined #openstack-infra | 14:51 | |
*** lmiccini_ has joined #openstack-infra | 14:51 | |
AJaeger | sambetts: they need to provide subunit data as well - and not all do, mtreinish can tell | 14:52 |
pabelanger | AJaeger: have a minute to help me trace down why https://review.openstack.org/#/c/336642/ isn't launching jobs? I suspect the skip-if logic is preventing it, but I cannot seem to find that | 14:52 |
AJaeger | pabelanger: A minute? Ok ;) | 14:52 |
*** roxanaghe has joined #openstack-infra | 14:52 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Keivox KMAP plugin https://review.openstack.org/351771 | 14:52 |
mtreinish | AJaeger: right, they need to put subunit data in the log output for it to show up there | 14:52 |
pabelanger | AJaeger: sure, take your time | 14:53 |
AJaeger | pabelanger: I see "Queue lengths: 155 events, 120 results. " - let's wait until that is down to zero... | 14:53 |
AJaeger | but I'll check zuul output as well | 14:53 |
pabelanger | AJaeger: sure | 14:53 |
mtreinish | fungi: it looks like we're getting data again: http://paste.openstack.org/show/550615/ | 14:53 |
*** cgross has quit IRC | 14:54 | |
*** mkoderer has quit IRC | 14:54 | |
*** igorn has quit IRC | 14:54 | |
*** lmiccini has quit IRC | 14:54 | |
*** clenimar has quit IRC | 14:54 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for Keivox KMAP plugin https://review.openstack.org/351771 | 14:54 |
prometheanfire | can we talk about the 50MB log size limit? is there any way to change it? | 14:55 |
*** mdrabe has quit IRC | 14:55 | |
*** igorn has joined #openstack-infra | 14:55 | |
*** clenimar has joined #openstack-infra | 14:55 | |
*** cgross has joined #openstack-infra | 14:55 | |
AJaeger | pabelanger: skip-ifs look fine, at least the docs,pep8 and python27 jobs should always run | 14:55 |
sambetts | mtreinish: This is the job https://review.openstack.org/#/c/351742/1/jenkins/jobs/ironic-python-agent-jobs.yaml@16 that I'd like to get the status of, its a post job that builds an image, and then uploads it to tarballs.openstack.org, what would we need to add to this to make it appear in the health.openstack.org tracker | 14:56 |
sambetts | ? | 14:56 |
pabelanger | AJaeger: ya, I would expect that too | 14:56 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Test at more appropriate boundaries for update. https://review.openstack.org/319615 | 14:56 |
mtreinish | fungi: I'm thinking we need some sort of feedback loop here. There is no way to tell we're missing stuff short of manual inspection right now, and everything has proven to be pretty unreliable over time | 14:56 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Move 'update' yaml parsing to JenkinsJobs.execute. https://review.openstack.org/319616 | 14:56 |
mtreinish | fungi: it was easier to detect with a single worker, just look at the most recent run in the db. But with >1 worker that doesnt really work | 14:56 |
*** roxanaghe has quit IRC | 14:57 | |
AJaeger | pabelanger: but looks strange indeed, i would have expected some more jenkins entries in the history for that change | 14:57 |
*** yamamoto has quit IRC | 14:57 | |
mtreinish | sambetts: you need to add subunit output to the job. Here is an example AJaeger did for the translation jobs: https://review.openstack.org/#/c/311459/ | 14:59 |
*** karthikp_ has joined #openstack-infra | 15:00 | |
pabelanger | AJaeger: okay, 351785 works as expected | 15:00 |
mmedvede | azvyagintsev: reviewed | 15:00 |
*** vincentll has quit IRC | 15:01 | |
pabelanger | AJaeger: for some reason I think https://review.openstack.org/#/c/336524 needs a new commit, so fix the problem | 15:01 |
pabelanger | testing that now | 15:01 |
AJaeger | pabelanger: I suggest to check zuul log files on the server. | 15:01 |
AJaeger | pabelanger: let's figure out the real problem | 15:02 |
azvyagintsev | mmedvede: thanks! | 15:02 |
*** mdrabe has joined #openstack-infra | 15:02 | |
fungi | ttx: https://review.openstack.org/#/admin/groups/1525,members | 15:03 |
pabelanger | AJaeger: yes, checking that now | 15:03 |
wznoinsk | mtreinish: (just looking at your feedback loop suggestion out of curiosity), do you think something like reconciliation against jenkins build history here? | 15:03 |
ttx | fungi: great, thx! | 15:03 |
fungi | mtreinish: i've restarted jenkins-subunit-worker-A on worker01 now and there's an established gearman connection from it again on logstash.o.o | 15:04 |
fungi | so i think that must have been disruption back in mat | 15:04 |
fungi | may | 15:04 |
fungi | network disruption | 15:04 |
*** d0ugal has quit IRC | 15:05 | |
fungi | mtreinish: what do you think about statsd reporting for the number of work requests processed from each worker, and a grafana dashboard trending those? | 15:05 |
mtreinish | fungi: something had to have been popping gearman jobs off the queue and not populating the db for things to go missing. I'd really like to know what caused that | 15:05 |
*** clenimar_ has joined #openstack-infra | 15:05 | |
mtreinish | fungi: so graph the number of incoming requests and then have the actual number of things done ontop or next to it? | 15:06 |
AJaeger | fungi, I've used topic constraints for all changes to use zuul-cloner with constraints - did a few this morning: https://review.openstack.org/#/q/status:open+project:openstack-infra/project-config+branch:master+topic:constraints I would appreciate review of at least the two oldest ones. | 15:06 |
mtreinish | wznoinsk: that was something I was thinking about. I just thin we should have a way to make sure we don't drop something on the floor and never realize it. | 15:06 |
fungi | AJaeger: yep, saw that. it's queued up behind the broken things i need to look into this morning | 15:06 |
AJaeger | fungi, sure. Let's first fix before we potentially break more ;) | 15:07 |
*** dansmith is now known as superdan | 15:07 | |
*** shardy is now known as shardy_afk | 15:07 | |
openstackgerrit | Aleksey Zvyagintsev proposed openstack-infra/project-config: [jenkinss]Add {murano-app}-unittest job https://review.openstack.org/351757 | 15:07 |
*** gomarivera has joined #openstack-infra | 15:07 | |
mtreinish | wznoinsk: the issue with that though is that there isn't always subunit output from the run (even if it normally has it) | 15:08 |
pabelanger | AJaeger: working: http://paste.openstack.org/show/550620/ | 15:08 |
wznoinsk | mtreinish: know exactly that feeling, I'm paranoid on that bit | 15:08 |
*** ddieterly is now known as ddieterly[away] | 15:08 | |
*** ddieterly[away] is now known as ddieterly | 15:08 | |
*** ddieterly is now known as ddieterly[away] | 15:08 | |
pabelanger | AJaeger: not-working: http://paste.openstack.org/show/550621/ | 15:09 |
*** xarses has quit IRC | 15:09 | |
wznoinsk | mtreinish: if that to be something you can rely on has to be from the outside of the job so post_test_hook is not an option | 15:09 |
openstackgerrit | Aleksey Zvyagintsev proposed openstack-infra/project-config: [jenkinss]Add {murano-app}-unittest job https://review.openstack.org/351757 | 15:10 |
fungi | anteaya: looks like this time the git problem is a missing master branch in /var/lib/git/openstack-infra/puppet-openstackid.git on git08.o.o (and nowhere else) | 15:10 |
fungi | what's strange is that it's the same repo that broke last time, except last time it broke this way on git04 | 15:11 |
fungi | given we have >1k repos, this can't be a coincidence | 15:11 |
*** yamamoto has joined #openstack-infra | 15:11 | |
fungi | i'm at a loss as to what's different about that repo though | 15:12 |
*** olaph is now known as beefboymattface | 15:13 | |
*** jistr is now known as jistr|mtg | 15:14 | |
sambetts | mtreinish: Uploading the subunit information won't catch if the problem occurs in the publisher right? | 15:14 |
*** ajmiller has joined #openstack-infra | 15:14 | |
*** beefboymattface is now known as beefboymf | 15:14 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 15:15 |
sambetts | mtreinish: ie. the recent error we saw was caused by the scp publishers rsync command trying to read a file that it didn't have permissions for, so the job failed to upload the files we wanted | 15:15 |
*** beefboymf is now known as beefboymattface | 15:15 | |
AJaeger | pabelanger: could there be a cycle in these depends-on from https://review.openstack.org/#/c/337676/ ? | 15:15 |
mtreinish | sambetts: no they won't. There has been a plan to use a zuul mysql reporter for the top level info for a while. (which would catch those and other failures we miss) But, I don't think the patch adding that has landed | 15:16 |
openstackgerrit | Timothy R. Chavez proposed openstack-infra/jenkins-job-builder: Add support for the random string parameter https://review.openstack.org/351384 | 15:16 |
pabelanger | AJaeger: oh, maybe | 15:17 |
*** e0ne has joined #openstack-infra | 15:17 | |
Shrews | pabelanger: looks like the timeout bump might be good: https://review.openstack.org/351745 | 15:17 |
pabelanger | AJaeger: going to have them remove https://review.openstack.org/#/c/337669/ depends-on as it is circular | 15:17 |
*** d0ugal has joined #openstack-infra | 15:18 | |
*** mhickey has quit IRC | 15:18 | |
mtreinish | sambetts: nope it's still open: https://review.openstack.org/#/c/223333/ | 15:18 |
*** xnox has quit IRC | 15:18 | |
*** e0ne has quit IRC | 15:19 | |
pabelanger | Shrews: great | 15:19 |
AJaeger | pabelanger: yep, 337676 -> 337669 -> 333612 -> 337676. Just don't understand how 336642 plays into it. But fixing that cycle is important | 15:19 |
pabelanger | AJaeger: Agreed, lets start with that | 15:20 |
AJaeger | pabelanger: ah, 336312 depends-on 336551 - and your change 642 is stacked on top of 551. So, that shows the relation. | 15:20 |
*** javeriak has joined #openstack-infra | 15:20 | |
AJaeger | pabelanger: Let them break the cycle and be happy ;) | 15:20 |
*** tesseract- has quit IRC | 15:21 | |
anteaya | fungi: that is really odd that that is the repo that has broken twice | 15:22 |
*** ganesan has joined #openstack-infra | 15:23 | |
anteaya | fungi: how does a repo even lose its master branch | 15:23 |
fungi | anteaya: some sort of interruption/corruption/truncation during a mirror push from gerrit is the only way i can think of | 15:24 |
anteaya | that is really odd | 15:25 |
*** pt_15 has joined #openstack-infra | 15:25 | |
*** armax has joined #openstack-infra | 15:26 | |
*** rcernin has quit IRC | 15:26 | |
*** itisha has joined #openstack-infra | 15:26 | |
ganesan | I am setting up 3rd party ci and struck with node pool. nodepool spins vm everytime due to ssh issue " ERROR nodepool.NodeLauncher: Timeout launching node id: 23480 in provider: local_02 error: Timeout waiting for ssh access" | 15:27 |
wznoinsk | ganesan: can you spawn a VM manually on local_02 cloud and connect to it? | 15:28 |
wznoinsk | ganesan: we can discuss it here so others can learn too, if you/nodepool can reach the machine over network, it may indicate a problem with logging in to the machine (user/pass/ssh key) | 15:30 |
ganesan | sure | 15:30 |
pabelanger | AJaeger: nice work, that appears to have fixed it | 15:30 |
ganesan | manually it works | 15:31 |
*** ddieterly[away] is now known as ddieterly | 15:31 | |
ganesan | I do have 2 network, 1 - private, 2- public (floating IP) | 15:31 |
*** Teresita-Warrior has joined #openstack-infra | 15:31 | |
wznoinsk | ganesan: have you tried to log in to the VM spawned by nodepool while it is waiting to be timedout? (it's probably 1-2 mins before it does it) | 15:31 |
AJaeger | great, pabelanger | 15:32 |
AJaeger | pabelanger: could you review sambetts's change https://review.openstack.org/351742 , please? | 15:33 |
*** hurgleburgler has joined #openstack-infra | 15:33 | |
ganesan | yes, I tried, but couldn't due private network is assigned to vm , which is not accessible from nodepool machine | 15:33 |
*** links has quit IRC | 15:33 | |
wznoinsk | fungi: maybe something like iwatch or audit on the directory would be good if the problem persists? ... I guess the branches may be dropped/recreated on sync to avoid potential problem... storage may not be syncing quick enough or something's still using the files and branch can't be deleted | 15:34 |
ganesan | I do have floating IP and trying to assign to those vm | 15:34 |
*** gouthamr has quit IRC | 15:34 | |
wznoinsk | ... leaving it in a dirty state | 15:34 |
*** gouthamr has joined #openstack-infra | 15:34 | |
fungi | wznoinsk: that would be strange for sure, but doesn't explain that out of the only two incidences in recent history it happened to the same repo (out of more than a thousand) on two different git servers | 15:35 |
*** yamamoto has quit IRC | 15:35 | |
openstackgerrit | Sam Yaple proposed openstack-infra/shade: [WIP] Add update_endpoint() https://review.openstack.org/289052 | 15:36 |
wznoinsk | are there other (infra) scripts using that repo that may collide at the time of sync? | 15:36 |
wznoinsk | ganesan: I don't use private network in my cloud I give to nodepool, never tested what you have | 15:36 |
*** burgerk has quit IRC | 15:36 | |
ganesan | is it possible to specify floating IP to vm's in addition to private network IP | 15:37 |
*** _nadya_ has quit IRC | 15:37 | |
wznoinsk | wznoinsk: but it sounds like a floating ip issue (networks setting in nodepool.yaml) | 15:37 |
wznoinsk | under 'providers' section | 15:38 |
wznoinsk | s/wznoinsk/ganesan - talking to myself again | 15:38 |
ganesan | I am trying to config nodepool to set floating ip for vm's. But I could not find how to specify | 15:39 |
openstackgerrit | Robin Naundorf proposed openstack-infra/project-config: Changed comment which is no longer valid https://review.openstack.org/351804 | 15:39 |
wznoinsk | https://github.com/openstack-infra/project-config/blob/master/nodepool/nodepool.yaml#L633, use public instead | 15:40 |
wznoinsk | or whatever the name your public network has | 15:40 |
openstackgerrit | Sam Betts proposed openstack-infra/project-config: Get IPA buildimage POST job into healthcheck https://review.openstack.org/351806 | 15:42 |
*** javeriak has quit IRC | 15:42 | |
sambetts | mtreinish, AJaeger ^^ | 15:42 |
*** sdague has quit IRC | 15:42 | |
*** javeriak has joined #openstack-infra | 15:43 | |
sambetts | mtreinish, AJaeger: based on the example patch, I think this looks more or less the right way to do it for our job | 15:43 |
*** javeriak has quit IRC | 15:43 | |
*** HenryG has quit IRC | 15:43 | |
fungi | wznoinsk: the only other possibility i can think of is that something is going wrong during git repack, but i' still at a loss as to why it would be just this one repo | 15:43 |
*** HenryG has joined #openstack-infra | 15:43 | |
anteaya | fungi: has this just started happening since zaro's patch merged? | 15:45 |
openstackgerrit | Merged openstack-infra/project-config: Clean up TinyIPA chroot directories in POST job https://review.openstack.org/351742 | 15:45 |
AJaeger | sambetts: yeah, looks good. | 15:46 |
AJaeger | sambetts: I regularly check for translations jobs: http://status.openstack.org/openstack-health/#/g/build_queue/post?groupKey=build_queue&searchJob=translation | 15:46 |
openstackgerrit | Merged openstack-infra/storyboard: Position archived items at the bottom of worklists https://review.openstack.org/351265 | 15:46 |
zaro | morning | 15:46 |
*** rfolco_ has quit IRC | 15:47 | |
AJaeger | dhellmann: do you know why reno fails to generate translations in the nova post job? See http://logs.openstack.org/4c/4cbfed44f4b1dc4af21e9a09ed830617cb3caa05/post/nova-upstream-translation-update/2fd7f7b/console.html#_2016-08-05_15_02_44_417760 | 15:47 |
*** pgadiya has quit IRC | 15:48 | |
anteaya | morning zaro | 15:48 |
*** florianf has quit IRC | 15:48 | |
fungi | anteaya: yeah, that's what i'm trying to determine. the timestamps are suspicious... git fsck says my archived copies of the broken repo on git04 and git08 have bad packfiles. the broken packed-refs file on git04 was modified at 04:07 utc on july 27, and on git08 at 04:07 utc on august 5 | 15:48 |
fungi | 04:07 is when the git gc runs every night on our git server backends | 15:49 |
*** florianf has joined #openstack-infra | 15:49 | |
sambetts | AJaeger: yeah it would be super useful for us to be able to do that with the buildimage jobs, one of them has been failing for >1 month without us realising | 15:49 |
*** xarses has joined #openstack-infra | 15:49 | |
sambetts | AJaeger: and if that data appears in health.openstack.org, I have an idea to build a bot that watches for it too | 15:50 |
fungi | anteaya: zaro: oh, this could be the problem... :( | 15:50 |
AJaeger | sambetts: integrate it with harlowja's new irc bot and have it complain to your channel whenever something fails ;) | 15:50 |
fungi | anteaya: zaro: we have two cronjobs running on our git servers at 04:07. one that does git pack-ref and another than does git gc | 15:50 |
anteaya | oh my | 15:50 |
AJaeger | fungi, in parallel? | 15:51 |
sambetts | AJaeger: ooo interesting, is there a link to that somewhere? | 15:51 |
zaro | that's not good | 15:51 |
anteaya | the point was to change the cronjob: https://review.openstack.org/#/c/342388/ | 15:51 |
fungi | anteaya: zaro: i bet the puppet change to add the mirror_gitgc cronjob on them didn't ensure=>absent on the mirror_repack cronjob | 15:51 |
AJaeger | sambetts: openstack-dev mailing list discussion | 15:51 |
anteaya | fungi: ah | 15:51 |
anteaya | good hunting | 15:51 |
zaro | i thought i set puppet to remove the repack cron job? | 15:51 |
fungi | so we have a repack and a gc happening at the same time, stepping on one another | 15:52 |
anteaya | zaro: this is the patch to change the cronjob, right? https://review.openstack.org/#/c/342388/ | 15:52 |
anteaya | fungi: I'm surprised we didn't have more races | 15:52 |
fungi | anteaya: that's on gerrit, not on our git backends | 15:52 |
sambetts | AJaeger: ah sweet, I'll try to dig that up | 15:52 |
anteaya | fungi: ah | 15:52 |
AJaeger | sambetts: http://lists.openstack.org/pipermail/openstack-dev/2016-July/100409.html - repo is openstack-infra/err2d2 | 15:52 |
AJaeger | sambetts: http://git.openstack.org/cgit/openstack-infra/err2d2 | 15:52 |
sambetts | AJaeger: :D thanks! | 15:52 |
*** jlanoux has joined #openstack-infra | 15:52 | |
krotscheck | AJaeger: I changed the job names as you requested, is this what you were looking for? https://review.openstack.org/#/c/346130/ | 15:52 |
krotscheck | Oh, right. | 15:53 |
krotscheck | Sorry sick brain. | 15:53 |
* krotscheck remembers the conversation about bindep | 15:53 | |
*** florianf has quit IRC | 15:53 | |
fungi | anteaya: zaro: i see the problem :/ | 15:53 |
anteaya | :( | 15:53 |
AJaeger | krotscheck: yes, that's what I was looking for regarding names. Did you see my comment on installing a package of sphinx instead of globally pip installing? | 15:53 |
AJaeger | krotscheck: on revision 8 | 15:54 |
*** narayrak has quit IRC | 15:54 | |
krotscheck | AJaeger: Ah, I did not. Let me check that for you. | 15:55 |
zaro | fungi: i think it's set https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/git_backend.pp#n120 | 15:55 |
krotscheck | AJaeger: I suspect the debian package may be out of date. | 15:55 |
krotscheck | But I'll check | 15:55 |
fungi | zaro: you might think that, but check the coming patch | 15:55 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Ensure git cronjob absent for correct user https://review.openstack.org/351816 | 15:55 |
fungi | zaro: anteaya: ^ | 15:55 |
*** sdake has quit IRC | 15:56 | |
*** sdake has joined #openstack-infra | 15:56 | |
anteaya | tricky | 15:56 |
zaro | thanks fungi. | 15:56 |
fungi | anteaya: zaro: it was ensuring teh mirror_repack cronjob was absent from root's crontab, which it was, because it never existed | 15:56 |
anteaya | thank you fungi | 15:56 |
fungi | i'll check the gerrit manifest to make sure it's not broken similarly | 15:57 |
anteaya | easy to ensure something is absent if it never existed | 15:57 |
zaro | i think there were a couple other spots that need this as well? | 15:57 |
*** mikelk has quit IRC | 15:57 | |
AJaeger | dhellmann: normal release notes for nova are broken as well - how could they merged something that broke the releasenotes? See http://logs.openstack.org/e9/e99c3665431462fc32877b71ba876da81cb61f3a/post/nova-releasenotes/0d85028/console.html | 15:57 |
zaro | ohh yeah, looks like your on it already | 15:57 |
openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Fix /v1/worklists/:id/items endpoint https://review.openstack.org/351819 | 15:58 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Ensure git cronjob absent for correct user https://review.openstack.org/351816 | 15:58 |
fungi | there was indeed a second one in system-config | 15:58 |
anteaya | thank you | 15:59 |
zaro | fungi: there's another one in puppet-gerrit project in manifests/cron.pp | 15:59 |
*** gyee has joined #openstack-infra | 16:00 | |
fungi | zaro: i'm aware--in the middle of editing that now | 16:00 |
*** jlanoux has quit IRC | 16:01 | |
openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-gerrit: Ensure git cronjob absent for correct user https://review.openstack.org/351820 | 16:01 |
fungi | zaro: anteaya: ^ | 16:01 |
fungi | i think those two changes cover everything | 16:02 |
*** psachin has quit IRC | 16:02 | |
*** signed8bit has joined #openstack-infra | 16:03 | |
pabelanger | zigo: So, you didn't update gerrit/projects.yaml to include deb-auto-backports, that's why the repo isn't created | 16:03 |
zaro | me too | 16:03 |
*** PollyZ has quit IRC | 16:04 | |
krotscheck | AJaeger: Current version of xenial python-sphinx is 1.3.6 - will that do? | 16:04 |
anteaya | thanks fungi and zaro | 16:04 |
*** davideagnello has joined #openstack-infra | 16:04 | |
pabelanger | fungi: anteaya: do you have a moment to help review https://review.openstack.org/#/c/351251/ adds a grafana dashboard for tripleo-test-cloud-rh1 | 16:04 |
*** pcaruana has quit IRC | 16:05 | |
anteaya | pabelanger: swap you for https://review.openstack.org/#/c/351816/2 and https://review.openstack.org/#/c/351820/1 which fixes a too many cronjobs issue with the git repos | 16:05 |
pabelanger | anteaya: sure, looking | 16:06 |
*** roxanaghe has joined #openstack-infra | 16:08 | |
anteaya | pabelanger: thank you, +3 on yours | 16:08 |
*** dtantsur is now known as dtantsur|afk | 16:08 | |
pabelanger | anteaya: thanks, +2 on both patches | 16:08 |
AJaeger | krotscheck: try it ;) For python jobs, we have in global requiremnents capped to < 1.3 | 16:08 |
anteaya | pabelanger: thank you | 16:08 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-tripleo-quickstart-docs-ubuntu-xenial job https://review.openstack.org/349087 | 16:08 |
*** senk has quit IRC | 16:09 | |
pabelanger | AJaeger: ^decided not to use openstack-server-publish-jobs as ansible-role-jobs is our intended template | 16:09 |
*** jimbaker has quit IRC | 16:11 | |
*** shardy_afk is now known as shardy | 16:12 | |
*** davideagnello has quit IRC | 16:12 | |
*** roxanaghe has quit IRC | 16:12 | |
*** PollyZ has joined #openstack-infra | 16:12 | |
AJaeger | pabelanger: please keep consistency in mind. Let's see how that looks later... | 16:12 |
*** davideagnello has joined #openstack-infra | 16:13 | |
AJaeger | note that openstack-server-publish-jobs would publish, your change just builds but does not publish | 16:13 |
*** amoralej is now known as amoralej|off | 16:13 | |
pabelanger | AJaeger: right, I am trying to figure out the difference between {name}-docs-{node} and gate-{name}-docs-{node} | 16:13 |
pabelanger | AJaeger: right now, we use gate-* | 16:13 |
*** roxanaghe has joined #openstack-infra | 16:13 | |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: TestSubCommand inherit from UpdateSubCommand. https://review.openstack.org/319619 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Don't expose YamlParser methods unnecessarily. https://review.openstack.org/319618 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move subcommand logic into subcommand classes. https://review.openstack.org/319617 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Factor XmlJobGenerator out of YamlParser. https://review.openstack.org/319623 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Disentangle YamlParser and ModuleRegistry classes https://review.openstack.org/319622 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove deprecated 'extended-choice' property. https://review.openstack.org/319621 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove YamlParser references from JJB modules. https://review.openstack.org/319620 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Be explicit about objects exported by modules. https://review.openstack.org/319625 | 16:13 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Convert multiline strings to concatenated single-quote. https://review.openstack.org/351743 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Cleanup various deprecation warnings. https://review.openstack.org/319624 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Remove unused builder.Builder.update_job method https://review.openstack.org/319752 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename Builder.delete_job to Builder.delete_jobs. https://review.openstack.org/319753 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename "parallelize" decorator to "concurrent" https://review.openstack.org/333780 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Merge builder.Builder and builder.Jenkins https://review.openstack.org/319754 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move CacheStorage to its own module. https://review.openstack.org/319755 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Make internal builder.Jenkins attrs internal-only. https://review.openstack.org/319756 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move 'delete' yaml parsing to JenkinsJobs.execute. https://review.openstack.org/262954 | 16:14 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Move macro expansion into YamlParser. https://review.openstack.org/333076 | 16:14 |
pabelanger | AJaeger: okay, I see | 16:14 |
pabelanger | AJaeger: one is post job | 16:14 |
openstackgerrit | Merged openstack-infra/system-config: Set the gitpgpkey for signing nodes https://review.openstack.org/349023 | 16:14 |
pabelanger | other is gate | 16:14 |
AJaeger | pabelanger: exactly | 16:14 |
pabelanger | AJaeger: in that case, we don't need post jobs right now. But will in the future | 16:15 |
*** jimbaker has joined #openstack-infra | 16:15 | |
*** jimbaker has quit IRC | 16:15 | |
*** jimbaker has joined #openstack-infra | 16:15 | |
pabelanger | AJaeger: actually, I can add them now | 16:15 |
openstackgerrit | Merged openstack-infra/project-config: Add tripleo-test-cloud grafana dashboard https://review.openstack.org/351251 | 16:15 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename "parallelize" decorator to "concurrent" https://review.openstack.org/333780 | 16:17 |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Convert multiline strings to concatenated single-quote. https://review.openstack.org/351743 | 16:17 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-tripleo-quickstart-docs-ubuntu-xenial job https://review.openstack.org/349087 | 16:17 |
pabelanger | AJaeger: update^ | 16:17 |
*** Swami has joined #openstack-infra | 16:19 | |
*** PollyZ has quit IRC | 16:19 | |
*** mriedem is now known as mriedem_lunch | 16:20 | |
pabelanger | cloudnull: how did you make out adding ipv6 to osic? | 16:20 |
cloudnull | pabelanger: not yet. im doing it next week | 16:20 |
pabelanger | cloudnull: nice | 16:20 |
krotscheck | AJaeger: Alright, so, just to clarify your comment - should I be using the install-sphinx macro I created, wait for the bindep changes to land, or... | 16:21 |
pabelanger | was just looking at nodepool logs and ipv6 address scrolled by, reminded me about the upgrade | 16:21 |
cloudnull | pabelanger: i wanted to do it this week but its been a little hectic | 16:21 |
*** vvalderrv has joined #openstack-infra | 16:21 | |
openstackgerrit | Merged openstack-infra/shade: Fix requirements for broken os-client-config https://review.openstack.org/351762 | 16:22 |
*** asettle has quit IRC | 16:22 | |
AJaeger | krotscheck: my personal preference would be for not installing in project-config - and that means solving the bindep issue first. And that needs some discussion between a not ill krotscheck, jeblair and perhaps others on naming first... | 16:23 |
*** asettle has joined #openstack-infra | 16:23 | |
AJaeger | krotscheck: and then you could just install python-sphinx via the bindep file. | 16:23 |
AJaeger | You could also for now change the install-sphinx macro to use the package instead of pip - to make it clearer what you want and ease the migration to bindep. | 16:24 |
*** roxanaghe has quit IRC | 16:24 | |
AJaeger | krotscheck: sorry, I'm out for today. Feel free to discuss without me on how to move forward. | 16:26 |
krotscheck | AJaeger: No worries | 16:26 |
* AJaeger wishes a great weekend to everybody | 16:26 | |
krotscheck | AJaeger: Well, once we do bindep we can remove all the install-xvfb macros etc. | 16:26 |
krotscheck | So I'll assume cleanup will happen and do that. | 16:27 |
AJaeger | krotscheck: yeah, I would love that ;) | 16:27 |
jeblair | i'm available for bindep bikeshedding if others are | 16:27 |
*** asettle has quit IRC | 16:27 | |
*** burgerk has joined #openstack-infra | 16:27 | |
AJaeger | jeblair: I might stay a bit longer for a nice bikeshed but might need to leave rather quickly then... | 16:27 |
*** baoli_ has quit IRC | 16:27 | |
*** baoli has joined #openstack-infra | 16:28 | |
jeblair | fungi: around? | 16:28 |
fungi | jeblair: yep | 16:28 |
fungi | i vote fuscha | 16:28 |
*** rwsu has quit IRC | 16:28 | |
AJaeger | jeblair: this change triggered the discussion: https://review.openstack.org/334870 | 16:28 |
jeblair | so i think the important thing is whether bindep reqs files are meant to be user facing | 16:29 |
*** coolsvap has joined #openstack-infra | 16:29 | |
fungi | in my mind, their primary importance is so that developers know what distro packages they will need to install before they can test changes they make to the repo they've checked out | 16:29 |
jeblair | my understanding was that they were, and that we wanted other-requirements to be not only consumed by ci, but information provided to users as well | 16:29 |
openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: fix its_rules parameterization in template https://review.openstack.org/351323 | 16:30 |
fungi | the goal with the ci integration, was to hook into that user-facing documentation and reuse it to avoid duplicating information about dependencies | 16:30 |
mordred | yes. I agree with both of those goals | 16:30 |
AJaeger | I agree with you - I just want to point out that this will never be a perfect information. We test it in the gate, so this is the minimal package set. We might miss a package or two - but that can be easily changed | 16:30 |
AJaeger | krotscheck, still around and reading the above? | 16:31 |
jeblair | right, so my question for krotscheck is -- was the objection to 'other-requirements.txt' something about the filename (like "use a word other than requirements"), or that the filename was visible to developers when they look at the repo? | 16:31 |
*** gomarivera has quit IRC | 16:31 | |
fungi | in addition to the list serving as documentation, it's formatted in a standardized machine-parsable way, and there is an available tool (bindep) to check whether you have them installed on your system | 16:31 |
*** rwsu has joined #openstack-infra | 16:31 | |
openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Add 'ceph status' and tripleo-overcloud-passwords to host_info https://review.openstack.org/348862 | 16:32 |
*** sambetts is now known as sambetts|afk | 16:32 | |
pabelanger | rcarrillocruz: just approved a bunch of omfracloud patches of yours. Does that get us to run bifrost now? | 16:32 |
*** baoli has quit IRC | 16:33 | |
fungi | but to divine bindep's purpose, it was originally conceived because someone was locally testing patches to different openstack projects, and got tired of having to keep track in ad-hoc prose documentation of what distro packages they needed to install to do that | 16:33 |
*** Teresita-Warrior has quit IRC | 16:33 | |
fungi | because it differed from one project to the next, and some documented it better than others, or in different places than others, or often not at all | 16:34 |
*** shashank_hegde has joined #openstack-infra | 16:34 | |
mordred | I could see just calling it "bindep.txt" - the filename "other-requirements.txt" does not inspire me to think that the bindep tool might be involved somehow | 16:35 |
fungi | leveraging a standardized descriptive format in our ci was seen also as a means of encouraging projects to adopt a standard around it | 16:35 |
fungi | er, to adopt that standard | 16:35 |
*** Apoorva has joined #openstack-infra | 16:35 | |
jeblair | 'system-requirements.txt' if the word 'other' is problematic. but bindep.txt wfm. | 16:36 |
*** tphummel has joined #openstack-infra | 16:36 | |
*** lucasagomes is now known as lucas|dinner | 16:36 | |
fungi | and yeah, i'm not married to the other-requirements.txt filename. that was simply chosen by the initial author early on, and was unfortunately inspired by contemporary work in tracking python package requirements in "other" requirements.txt files | 16:36 |
*** jpich has quit IRC | 16:36 | |
openstackgerrit | Merged openstack-infra/system-config: Add controller01.vanilla metadata for better tracking https://review.openstack.org/351331 | 16:37 |
fungi | we should keep it working for backwards compatibility, but i'm fine with a different filename | 16:37 |
mordred | I could imagine 'requirements.txt' having python connotations and folks from other languages not wanting to have 'python' files in their source repos | 16:37 |
mordred | but yes to keeping it working for backwards compat | 16:37 |
openstackgerrit | Merged openstack-infra/system-config: Add compute040.vanilla metadata for better tracking https://review.openstack.org/351332 | 16:37 |
jeblair | yeah, the previous alternate suggestion was .bindeprc as implemented here: https://review.openstack.org/#/c/350184/ | 16:38 |
jeblair | and i feel that counteracts the goal of developer visibility | 16:38 |
krotscheck | jeblair: I have three objections. | 16:38 |
jeblair | (so just want to try to get us on the same page) | 16:38 |
krotscheck | jeblair: Firstly, 'other-requirements.txt' is fairly clearly a python thing, as it's very similar to requirements.txt, test-requirements.txt, etc. | 16:38 |
krotscheck | jeblair: Secondly, it depends on those other files to inform its context - If I see those three files in a repository, they provide meaning to each other's purpose. If I only see other-requirements.txt (as I would in a non-python project like java or javascript), then the prefix other-... means what? What's going on? I don't know! | 16:38 |
krotscheck | jeblair: The third objection is that it in no way informs what tool it is related to. JS developer comes along, sees that file, opens it and sees package names, they can infer what the purpose is, but would they know how to properly use it? There's nothing that says "this is a bidep thing". | 16:39 |
AJaeger | krotscheck: So, system-requirements.txt would work? | 16:39 |
openstackgerrit | Merged openstack-infra/system-config: Add compute042.vanilla metadata for better tracking https://review.openstack.org/351334 | 16:39 |
mordred | krotscheck: or bindep.txt ? | 16:39 |
jeblair | ya, both words in "other" and "requirements" are problematic then. :) | 16:39 |
*** tonytan4ever has quit IRC | 16:39 | |
krotscheck | mordred: yes, AJaeger: yes, but less so. | 16:40 |
fungi | krotscheck: the file supports comments. in the changes where i've proposed it to other projects i've included a very descriptive comment preamble explaining what it is, why it's there, and where to find further documentation on it | 16:40 |
jeblair | system-requirements.txt arguably addresses objections 1 and 2. probably not 3. maybe that could be addressed by a boilerplate comment in the file. bindep.txt addresses all 3, i think? | 16:40 |
AJaeger | krotscheck: https://review.openstack.org/#/c/334342/10/other-requirements.txt has in the first lines a comment what the file is about | 16:40 |
krotscheck | I feel that if we're going to change the name to begin with, we should go with something that makes sense cross languages, like .bindeprc or /.config/bindep.?? | 16:40 |
mordred | well, we would like to not have it be hidden | 16:40 |
AJaeger | krotscheck: but then it's not visible | 16:41 |
mordred | and I think there's good precedent for that in other languages - bower.json is not a hidden file, for instance | 16:41 |
krotscheck | mordred: Yes, but .eslintrc is, as is .travis.yml | 16:41 |
krotscheck | mordred: The precedent that I'm familiar with is that files directly related to the code are visible, files related to the meta tooling are not. | 16:42 |
jeblair | the difference is that bindep is meant to be consumed by humans, not only machines | 16:42 |
openstackgerrit | Merged openstack-infra/system-config: Add compute021.vanilla metadata for better tracking https://review.openstack.org/351348 | 16:42 |
openstackgerrit | Merged openstack-infra/system-config: Add compute018.vanilla metadata for better tracking https://review.openstack.org/351350 | 16:42 |
mordred | krotscheck: yah - in this case the file seems related to the code to me, although I can see how a different pov could exist | 16:42 |
openstackgerrit | Merged openstack-infra/system-config: Add compute027.vanilla metadata for better tracking https://review.openstack.org/351352 | 16:42 |
openstackgerrit | Merged openstack-infra/system-config: Add compute028.vanilla metadata for better tracking https://review.openstack.org/351354 | 16:42 |
fungi | krotscheck: is documentation related to the code, or the tooling? | 16:43 |
mordred | krotscheck: as in "this code is not going to work if you don't install these packages on your host os" | 16:43 |
fungi | or "you won't be able to test changes to this code if you don't install the following tools first" | 16:43 |
*** senk has joined #openstack-infra | 16:44 | |
krotscheck | Right, I don't actually care about the visibility of the file. I would like a name where there's enough established cross-project patterns so there's a reasonable chance someone can infer the file's context from its name. | 16:44 |
krotscheck | I'm using .bindeprc as an example. The .<name>rc pattern is pretty well established, but others also exist. | 16:44 |
*** ddieterly is now known as ddieterly[away] | 16:44 | |
*** d0ugal has quit IRC | 16:44 | |
*** tonytan4ever has joined #openstack-infra | 16:45 | |
fungi | and if people want to know what packages your project needs, you tell them to look in this hidden file | 16:45 |
jeblair | then i think the best compromise here is going to be either bindep.txt or system-requirements.txt. | 16:45 |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update properties.slack to use convert_mapping_to_xml https://review.openstack.org/351841 | 16:46 |
* mordred votes for bindep.txt | 16:46 | |
mordred | largely since system-requirements.txt doesn't tie to tool and also has a python implication | 16:46 |
krotscheck | mordred: Well, system-requirements I feel carries its own context. | 16:47 |
krotscheck | mordred: Anyone who's ever played a videogame knows what system-requirements imply. | 16:48 |
jeblair | 4mb of ram! | 16:48 |
AJaeger | What should the order be? Read other-requirements.txt if it exists; if not try system-requirements/bindep (or whatever we choose)? | 16:48 |
*** Swami_ has joined #openstack-infra | 16:48 | |
*** ddieterly[away] is now known as ddieterly | 16:48 | |
krotscheck | jeblair: DirectX5! | 16:48 |
krotscheck | So I'm fine with either | 16:48 |
mordred | krotscheck: ah. good point | 16:48 |
* krotscheck wonders if system-requirements.txt will include a line called '3.5" floppy' | 16:49 | |
*** Swami__ has joined #openstack-infra | 16:49 | |
*** Swami has quit IRC | 16:49 | |
pabelanger | I like bindep.txt, as it relates to mordred comments about bower.json | 16:49 |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update properties.slack to use convert xml https://review.openstack.org/351841 | 16:49 |
jeblair | 2 votes for bindep.txt! | 16:49 |
fungi | i have no preference, i just think having it as a dotfile makes it a bit nonobvious when it's meant to be obvious | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute029.vanilla metadata for better tracking https://review.openstack.org/351357 | 16:50 |
krotscheck | Is the extension correct? Is it just a text file, or a .conf style file, or a .yaml? | 16:50 |
*** senk has quit IRC | 16:50 | |
openstackgerrit | Merged openstack-infra/system-config: Add compute020.vanilla metadata for better tracking https://review.openstack.org/351359 | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute013.vanilla metadata for better tracking https://review.openstack.org/351369 | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute010.vanilla metadata for better tracking https://review.openstack.org/351371 | 16:50 |
AJaeger | krotscheck: just text file that allows comments | 16:50 |
fungi | krotscheck: it's its own text-based format | 16:50 |
mordred | krotscheck: it's not .conf or .yaml | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute019.vanilla metadata for better tracking https://review.openstack.org/351372 | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute006.vanilla metadata for better tracking https://review.openstack.org/351374 | 16:50 |
openstackgerrit | Merged openstack-infra/system-config: Add compute001.vanilla metadata for better tracking https://review.openstack.org/351378 | 16:51 |
openstackgerrit | Merged openstack-infra/system-config: Add compute000.vanilla metadata for better tracking https://review.openstack.org/351377 | 16:51 |
openstackgerrit | Merged openstack-infra/system-config: Add compute034.vanilla metadata for better tracking https://review.openstack.org/351379 | 16:51 |
jeblair | i think bindep.txt is the winner then | 16:51 |
openstackgerrit | Merged openstack-infra/system-config: Add compute017.vanilla metadata for better tracking https://review.openstack.org/351380 | 16:51 |
*** tphummel has quit IRC | 16:51 | |
jeblair | also, does anyone else suddenly want ice cream? | 16:51 |
openstackgerrit | Merged openstack-infra/system-config: Add compute039.vanilla metadata for better tracking https://review.openstack.org/351383 | 16:51 |
fungi | we could make up a vanity extension to denote that it's in "bindep format" but that seems unnecessary | 16:51 |
pabelanger | jeblair: yes! | 16:51 |
openstackgerrit | Merged openstack-infra/system-config: Update iLO IP for compute016.vanilla https://review.openstack.org/351392 | 16:51 |
fungi | jeblair: i suddenly have a craving for vanilla | 16:52 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Add .bindeprc as additional filename https://review.openstack.org/350184 | 16:52 |
* AJaeger forgot to update commit message, try again... | 16:52 | |
jeblair | :) | 16:52 |
*** Swami_ has quit IRC | 16:52 | |
krotscheck | jeblair: I like vanilla :) | 16:52 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Add bindep.txt as additional filename https://review.openstack.org/350184 | 16:53 |
pabelanger | aiming for maple walnut for a 3rd zone | 16:53 |
jeblair | krotscheck, AJaeger, fungi, mordred, pabelanger: thanks for sticking through a bikeshed convo :) | 16:53 |
* krotscheck prefers red. | 16:53 | |
AJaeger | please review whether the logic is fine in 350184 - I tested the previous version and that worked fine. | 16:53 |
AJaeger | jeblair: was a pleasure ;) | 16:54 |
jeblair | pabelanger: i thought we were going neapolitan -- maybe zone 4? | 16:54 |
*** jbernard has quit IRC | 16:54 | |
*** kzaitsev_mb has quit IRC | 16:55 | |
pabelanger | jeblair: actually, I think we should name the cloud Neapolitan and have 3 zones of f vanilla, chocolate, and strawberry | 16:55 |
* fungi wants a cosmopolitan now | 16:55 | |
fungi | it _is_ friday after all | 16:55 |
*** ajmiller_ has joined #openstack-infra | 16:55 | |
krotscheck | The next project I'm a member of we're going to use ice cream flavors as release names. | 16:57 |
mordred | krotscheck: what about types of parsley? | 16:57 |
jeblair | AJaeger: left a comment | 16:58 |
*** Swami__ has quit IRC | 16:58 | |
krotscheck | mordred: I don't know, parsley doesn't seem like a great theme for a release party. | 16:58 |
jeblair | krotscheck: "drinks made with parsley", however... | 16:58 |
*** mat128|away is now known as mat128 | 16:58 | |
krotscheck | mordred: Ah Yes. | 16:58 |
mordred | krotscheck: come on ... giant tubs of parsely? you could also have a parsley bowl next to a cilantro bowl and play a guessing game ... | 16:59 |
krotscheck | jeblair: I shall put that on my list-of-playa-cocktails-to-design. | 16:59 |
fungi | at least everyone would have fresh breath | 16:59 |
AJaeger | jeblair: right now it uses specified file - and uses other-requirements as default for that. | 16:59 |
krotscheck | mordred: The first party? Sure.... but in a CI/CD project, I feel like that might get old. | 16:59 |
AJaeger | jeblair: I can implement what you suggested. Could others comment as well on what the behaviour should be ,pl easE? | 16:59 |
mordred | krotscheck: :) | 16:59 |
krotscheck | mordred: "We released 3 times today, here's your tub of greens" | 17:00 |
mordred | krotscheck: the gimmick could be that all of the parties are always identically boring | 17:00 |
*** ajmiller has quit IRC | 17:00 | |
mordred | which is actually what CI/CD should really be - as boring as possible, right? | 17:00 |
krotscheck | mordred: Yeah, if CI/CD is interesting, that's a problem. | 17:00 |
mordred | AJaeger, jeblair: I think it should look for both files, work if either are found, and error if both are found - but that's just me | 17:01 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix incorrect labels for RH1 in grafyaml https://review.openstack.org/351846 | 17:02 |
krotscheck | ianw: If you're around, we've got a new version of the NPM-devstack job https://review.openstack.org/#/c/348056/ | 17:02 |
AJaeger | mordred, jeblair : IF a command line arg is given, use that file. Otherwise check for bindep, check for other-requirements - and fail if both don't exist? | 17:03 |
fungi | krotscheck: i expect ianw's asleep for a while still. we usually don't see him around in here until 20:00 utc or later (but also it's his saturday already) | 17:04 |
jeblair | AJaeger: yes, though mordred also suggested failing if _both_ exist | 17:04 |
krotscheck | fungi: Ah, wasn't aware that he's down under. | 17:05 |
krotscheck | Timezones are hard on IRC | 17:05 |
AJaeger | jeblair: indeed. Let's go with that... | 17:05 |
AJaeger | thanks | 17:07 |
*** d0ugal has joined #openstack-infra | 17:11 | |
*** gomarivera has joined #openstack-infra | 17:12 | |
dhellmann | fungi : how do I get the launchpad libraries and their dependencies installed on the signing node? http://logs.openstack.org/30/3076a6c65b3f1e93da77fe61881f60f0b10d3759/release-post/tag-releases/2645a3a/console.html | 17:13 |
*** daemontool_ has joined #openstack-infra | 17:13 | |
*** tphummel has joined #openstack-infra | 17:13 | |
fungi | dhellmann: oh, we can puppet the distro packages for those, but the bigger concern is how to puppet authentication for that | 17:14 |
fungi | i can look at what we do to update bugs from gerrit hooks, though i think there may be a manual bootstrapping step in there we never fully automated | 17:15 |
*** signed8bit is now known as signed8bit_Zzz | 17:15 | |
dhellmann | ok | 17:15 |
fungi | since last i looked you have to use their openid provider to authenticate your api connection | 17:15 |
dhellmann | yes, that's true | 17:16 |
*** sdague has joined #openstack-infra | 17:16 | |
*** daemontool has quit IRC | 17:16 | |
fungi | mordred: jeblair: ^ do you remember if we ever found a way around that, to avoid having to ssh into the server and use a text-based web browser to step through the openid authentication and save the token? | 17:17 |
jeblair | fungi: i recall doing it on my workstation and copying the resulting files | 17:17 |
jeblair | this was 5 years ago? :) | 17:17 |
fungi | but we're not using private hiera for that at the moment i guess | 17:18 |
jeblair | fungi: we can probably stick it in hiera and use it on both servers? | 17:18 |
fungi | i can probably put the token in hiera | 17:18 |
fungi | yeah | 17:18 |
fungi | that | 17:18 |
*** HeOS has quit IRC | 17:18 | |
fungi | dhellmann: lemme noodle on it a little, but i'm pretty sure i can just reuse some of what we have in puppet for gerrit/jeepyb | 17:19 |
dhellmann | fungi : ok, sounds good | 17:19 |
*** davideagnello has quit IRC | 17:20 | |
*** dizquierdo has joined #openstack-infra | 17:20 | |
dhellmann | hmm, the tarball signing job for that test release failed http://logs.openstack.org/c4/c4ef0ccb90a827366b71e935994fb974c8f8950e/release/release-test-tarball-signing/1182ccd/console.html | 17:20 |
*** electrofelix has quit IRC | 17:21 | |
*** gomarivera has quit IRC | 17:22 | |
openstackgerrit | Merged openstack-infra/nodepool: Allow extra time for initial ZooKeeper connection https://review.openstack.org/351745 | 17:22 |
*** signed8bit_Zzz is now known as signed8bit | 17:22 | |
fungi | dhellmann: interesting, it's erroring on a script that we also use for pypi uploads, though looks like maybe the logic in it has a bit of a hole | 17:22 |
*** Sukhdev has joined #openstack-infra | 17:22 | |
fungi | it's trying to use len(sys.argv) as a bool, but i don't think that will ever be false | 17:23 |
dhellmann | and I'm getting a "relay not permitted" error from the announce email script | 17:23 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Add bindep.txt as default filename https://review.openstack.org/350184 | 17:24 |
dhellmann | so all of the steps are being run, and we have a few pieces missing to make them work | 17:24 |
dhellmann | this is definitely progress | 17:24 |
AJaeger | krotscheck, jeblair, mordred, fungi: Updated 350184 ^ - the bindep.txt chnage | 17:24 |
dhellmann | announce log: http://logs.openstack.org/c4/c4ef0ccb90a827366b71e935994fb974c8f8950e/release/release-test-announce-release/47980fb/console.html | 17:24 |
fungi | should probably be "if len(sys.argv) != 1" instead of "if not len(sys.argv)" | 17:24 |
dhellmann | fungi : that's right, it should do "if sys.argv[1:]" | 17:24 |
*** karthikp_ has quit IRC | 17:24 | |
krotscheck | AJaeger: woot | 17:24 |
krotscheck | AJaeger: You should go have a weekend. | 17:24 |
dhellmann | fungi : or what you said, sure | 17:24 |
fungi | something like that, yeah | 17:24 |
dhellmann | oh, the relay issue is probably because I'm sending to myself and not the real mailing list | 17:25 |
dhellmann | so that's not an issue | 17:25 |
AJaeger | krotscheck: let me try again ;) | 17:25 |
dhellmann | I didn't want to spam the dev list with test announcements | 17:25 |
fungi | indeed | 17:25 |
* AJaeger waves good-bye | 17:25 | |
fungi | have a good weekend, AJaeger | 17:25 |
jeblair | Shrews, mordred, pabelanger: re 351745 -- i the zk fixture should only exit once the server is actually running (it watches the logfile); in practice, that usually takes only 1 or 2 seconds. | 17:26 |
AJaeger | thanks, fungi | 17:26 |
*** tosky has quit IRC | 17:27 | |
jeblair | dhellmann: we can make a test list if you want | 17:27 |
*** davideagnello has joined #openstack-infra | 17:27 | |
dhellmann | jeblair : nah, that's ok, the email sending part of this is already working | 17:28 |
jeblair | k | 17:28 |
dhellmann | jeblair : the only new parts are doing the bug updates and tags on this node | 17:28 |
dhellmann | the tag push worked great | 17:28 |
dhellmann | the bug update step is missing some software dependencies (the launchpad stuff) | 17:28 |
krotscheck | mordred: Side question. DOes the auth_url feature care about keystone version path suffix? i.e. will both http://example.com:35357/ and http://example.com:35357/v2.0 work? | 17:28 |
mordred | krotscheck: it depends on the keystone - but any modern keystone should properly handle the unversioned auth_url | 17:29 |
krotscheck | mordred: Ok. I'm doing design by documentation, trying to teach JS developers how to set up their clouds.yaml file. | 17:29 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Correct command line logic in pypi-extract-name.py https://review.openstack.org/351857 | 17:29 |
fungi | dhellmann: ^ | 17:29 |
mordred | krotscheck: cool. people should use the unversioned url unless their keystone is too old/broken | 17:30 |
Shrews | jeblair: hrm, then perhaps the log string the fixture is looking for is incorrect in a newer version? either way, the change in 351745 can't hurt | 17:30 |
dhellmann | fungi : you know, argparse is really really easy to set up | 17:30 |
mordred | krotscheck: although it's safe to blindly put in whatever the OS_AUTH_URL is from the openrc that horizon provides | 17:30 |
*** pradk has quit IRC | 17:30 | |
fungi | dhellmann: yes, it is. now that we're not on python 2.6 anyway | 17:30 |
dhellmann | fair | 17:31 |
mordred | krotscheck: so if they are using a cloud that is not in the set of known vendors, and they don't know how else to find the information, that is a source that is safe to consume from | 17:31 |
jeblair | Shrews: yes, i agree it's harmless. just wanted to mention that in case the situation doesn't improve. | 17:31 |
mordred | Shrews: I rechecked a bunch of shade changes now that the occ pin landed | 17:31 |
fungi | dhellmann: i think in general most of the scripts in there don't end up using argparse unless they have >1 non-exclusive option to handle, but i wouldn't reject patches converting them to use it consistently | 17:32 |
fungi | (for reasons such as this bug) | 17:32 |
pabelanger | jeblair: Shrews: it I understand, ZookeeperServerFixture will wait up to 30 seconds to start? | 17:32 |
dhellmann | fungi : ok, I'll put that on my "spare time" list | 17:32 |
*** piet has quit IRC | 17:32 | |
Shrews | jeblair: noted. might need to hold a failed node to figure out what's happening | 17:32 |
Shrews | mordred: \o/ | 17:33 |
jeblair | pabelanger: yep; it usually either starts or fails within a few seconds. | 17:33 |
fungi | dhellmann: but now i'm back to figuring out why we don't normally trip that failure, and how we tripped it here | 17:33 |
*** Swami has joined #openstack-infra | 17:33 | |
jeblair | Shrews: want me to set a trap now? | 17:33 |
*** pradk has joined #openstack-infra | 17:33 | |
dhellmann | fungi : keep in mind that this test project may be missing pieces that "real" projects have | 17:33 |
SpamapS | hm | 17:34 |
SpamapS | our jjb just deleted all of our jenkins jobs | 17:34 |
SpamapS | (running from master) | 17:34 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-tripleo-quickstart-docs-ubuntu-xenial job https://review.openstack.org/349087 | 17:34 |
SpamapS | I believe --delete-old is broken | 17:34 |
Shrews | jeblair: let's wait to see if my change has any impact first | 17:34 |
* SpamapS is investigating | 17:34 | |
openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Add config support for ZooKeeper servers https://review.openstack.org/350623 | 17:34 |
*** jed56 has quit IRC | 17:35 | |
*** amitgandhinz has quit IRC | 17:35 | |
Shrews | jeblair: pabelanger: just rebased 623 ^^^ on master to see if it passes now | 17:35 |
*** amitgandhinz has joined #openstack-infra | 17:35 | |
*** yamahata has joined #openstack-infra | 17:36 | |
*** shashank_hegde has quit IRC | 17:37 | |
fungi | SpamapS: what other command-line options were you passing along with --delete-old? | 17:38 |
openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: if we fail to send mail, the job should fail https://review.openstack.org/351860 | 17:38 |
*** tldr_ has quit IRC | 17:38 | |
fungi | SpamapS: wondering if there's an untested interaction between options | 17:38 |
pabelanger | jeblair: is 351435 what you were thinking to fix our database issue with subnodes and updateStats function? | 17:39 |
Shrews | SpamapS: sounds like it needs an '--i-am-a-dummy' option, ala mysql :) | 17:40 |
SpamapS | jenkins-jobs -l debug --conf "/etc/jenkins_jobs/jenkins_jobs.ini" update --delete-old "/etc/jenkins_jobs/config" | 17:40 |
SpamapS | that's the command we use | 17:41 |
SpamapS | and what it does is, it creates all the jobs, then deletes them all | 17:41 |
SpamapS | which is.. really.. really awesome | 17:41 |
mordred | SpamapS: you want it to create them and then not delete them? | 17:41 |
fungi | SpamapS: sounds like it's probably a very recent regression. had you been using --delete-old previously? | 17:41 |
fungi | "these are so old. some are from several minutes ago!" | 17:42 |
mordred | is it possible that a file permission issue on the jjb cache could be causing jjb to think the jobs in question are not ones that it wants? | 17:42 |
openstackgerrit | Merged openstack-infra/system-config: Update iLO IP for compute004.vanilla https://review.openstack.org/351395 | 17:42 |
openstackgerrit | Merged openstack-infra/system-config: Add compute002.vanilla metadata for better tracking https://review.openstack.org/351335 | 17:42 |
openstackgerrit | Merged openstack-infra/system-config: Add docs for firehose.o.o https://review.openstack.org/350764 | 17:43 |
SpamapS | mordred: I was thinking that would be the best way | 17:43 |
*** dizquierdo has quit IRC | 17:43 | |
SpamapS | this morning when our job ran, it deleted the job that runs jjb, which killed the job | 17:43 |
*** e0ne has joined #openstack-infra | 17:44 | |
SpamapS | so I have a pretty clear time frame when this happened.. some small things seem to have merged in that time frame | 17:44 |
mordred | SpamapS: just for giggles, can you check permissions on your cache location and make sure the user running jjb can write to it? | 17:44 |
fungi | SpamapS: there were only a couple changes merged in the past week, both just a few hours ago | 17:44 |
jeblair | pabelanger: yep | 17:45 |
SpamapS | It happened some time just before 05/Aug/2016:15:23:14 +0000 | 17:46 |
mordred | oh - so ... https://review.openstack.org/#/c/319616/9 | 17:46 |
mordred | darragh's comment seems related: "If someone was to call delete_old_managed() without any arguments, after calling update_jobs() they might have a reasonable expectation that it doesn't go a delete the jobs they just updated. It seems a bit of an inconvenience to have to supply the information on what jobs were updated." | 17:46 |
SpamapS | I have some meatspace urgent issues to attend to.. back in 10 | 17:46 |
mordred | SpamapS: also, I take it that you are running jjb from master? | 17:47 |
fungi | waynr: zxiiro: zaro: looks like https://review.openstack.org/319616 might warrant an emergency revert | 17:47 |
mordred | SpamapS: I think they're working towards a 2.0 ... it might be safer to run the latest release | 17:47 |
openstackgerrit | Monty Taylor proposed openstack-infra/jenkins-job-builder: Revert "Move 'update' yaml parsing to JenkinsJobs.execute." https://review.openstack.org/351864 | 17:48 |
* mordred submitted the revert patch for consideration ^^ | 17:48 | |
fungi | waynr: zxiiro: zaro: ^ | 17:48 |
fungi | electrofelix seems to have knocked off for the weekend | 17:48 |
*** rbrndt has quit IRC | 17:49 | |
fungi | kien-ha: ^ you reviewed the original too | 17:49 |
mordred | I put in SpamapS command into the change for reference | 17:49 |
*** eharney has quit IRC | 17:49 | |
*** ganesan has quit IRC | 17:49 | |
*** claudiub has quit IRC | 17:50 | |
*** baoli has joined #openstack-infra | 17:50 | |
fungi | would have bitten us as well if we hadn't already moved off jenkins | 17:50 |
fungi | our puppet exec looked similar to that | 17:50 |
fungi | quite possible others running a similar ci are hitting it now as well | 17:51 |
openstackgerrit | Monty Taylor proposed openstack-infra/puppet-openstackci: Pin jjb to 1.6.1 https://review.openstack.org/351865 | 17:51 |
jeblair | asselin, fungi: maybe it's worth... heh, that^ | 17:51 |
mordred | I recommend we recommend people pin | 17:52 |
jeblair | i recommend we follow mordred's recommendation | 17:52 |
fungi | that looks like a programmatic way to recommend that | 17:52 |
mordred | that would also give the 2.0 effort a bit of space to get 2.0 up and happy | 17:52 |
*** sarob has joined #openstack-infra | 17:53 | |
*** mriedem_lunch is now known as mriedem | 17:53 | |
* krotscheck reccomends that we recommend people follow jeblair's reccomendation. | 17:53 | |
kien-ha | fungi: I believe zxiiro and electrofelix will be back soon | 17:53 |
mordred | \o/ | 17:54 |
openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Fix gerrit/projects.yaml for deb-auto-backports https://review.openstack.org/351866 | 17:54 |
mordred | I just got a spam with the subject: "Tactical Lasers - 65% Off" | 17:54 |
fungi | mordred: are you sure it's spam? | 17:54 |
mordred | fungi: NO | 17:54 |
zigo | pabelanger: mordred: That should fix the not created git repo. | 17:54 |
mordred | fungi: I clearly need some disount tactical lasers | 17:54 |
Shrews | mordred: with shark-approved head straps? | 17:55 |
mordred | Shrews: definitely | 17:55 |
openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Ansible launcher: Pin JJB to 1.6.1 https://review.openstack.org/351867 | 17:56 |
jeblair | not urgent, but while we're thinking about it ^ | 17:56 |
SpamapS | mordred: indeed, I think we'll just pin to 1.6.1 | 17:57 |
mordred | SpamapS: cool | 17:57 |
greghaynes | haha, I just proposed that same patch :) | 17:57 |
*** xnox has joined #openstack-infra | 17:58 | |
*** weshay has quit IRC | 17:58 | |
*** sarob has quit IRC | 17:59 | |
*** sarob has joined #openstack-infra | 17:59 | |
greghaynes | (internally) | 18:00 |
*** ddieterly is now known as ddieterly[away] | 18:00 | |
waynr | yeah i missed that comment, too much going on | 18:01 |
*** weshay has joined #openstack-infra | 18:01 | |
waynr | (not in the JJB 2.0 api review process, but between $dayjob and 2.0 api review) | 18:01 |
SpamapS | IMO the thing that is missing is functional test coverage in the gate | 18:02 |
waynr | and people talking loudly about pimped out cars next to my desk | 18:02 |
SpamapS | We're humans, we miss things. | 18:02 |
*** shardy is now known as shardy_afk | 18:03 | |
*** sandanar has quit IRC | 18:03 | |
*** tonytan4ever has quit IRC | 18:03 | |
*** eharney has joined #openstack-infra | 18:03 | |
openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 18:04 |
*** Na3iL has quit IRC | 18:04 | |
asselin | jeblair, fungi mordred, yeah that was proposed for 3rd party ci folks here: https://review.openstack.org/#/c/348035/4/contrib/single_node_ci_data.yaml | 18:05 |
waynr | zaro: zxiiro I think we should merge this: https://review.openstack.org/#/c/351864 | 18:05 |
*** e0ne_ has joined #openstack-infra | 18:07 | |
*** e0ne has quit IRC | 18:07 | |
fungi | asselin: so i guess 351865 is unlikely to make any difference for anyone using single_node_ci_data.yaml? | 18:08 |
waynr | while i'm glad someone caught this bug before release, i don't think it's a very good idea to run from a development head of a repo in a production system | 18:09 |
timrc | ++ | 18:09 |
*** piet has joined #openstack-infra | 18:09 | |
timrc | Pin all the things (in production) is core to my belief system... but that would imply you have some way of testing "latest" (e.g. you have a staging env) | 18:10 |
jeblair | waynr: that is a change from the practice we have recommended with jjb so far. that's fine with me if you want to change it -- i just want to make it clear it is a change | 18:10 |
*** gouthamr has quit IRC | 18:10 | |
jeblair | timrc: i don't agree with that, but that's okay. :) | 18:10 |
asselin | fungi, it might still if they didn't 'manually' pin in their own hiera config. | 18:10 |
timrc | jeblair: I mean... if you care about uptime... | 18:11 |
jeblair | timrc: we will absolutely continue to run zuul and nodepool as tip of master in openstack's production env. | 18:11 |
waynr | jeblair: ah, i didn't realize that was being recommended | 18:11 |
SpamapS | waynr: we run from the development head of all of openstack.. so........... | 18:11 |
phschwartz | waynr: I am more concerned that things like this are making it into the repo. They should be caught/fixed during the review process. | 18:11 |
jeblair | timrc: i care about continuous deployment | 18:11 |
*** acoles is now known as acoles_ | 18:11 | |
*** karthik__ has joined #openstack-infra | 18:11 | |
timrc | jeblair: You can achieve that by deploying to staging first in a continuous way and then once validated, deploy to production. | 18:11 |
asselin | fungi, I didn't send any e-mail about it to the ML. I guess there's enough need to warrant that now. | 18:11 |
jeblair | timrc: yes you can. i won't be doing that. | 18:11 |
waynr | phschwartz: bugs are a risk associated with software development in general | 18:11 |
timrc | Sure there's the "push button" on staging/production boundry | 18:12 |
waynr | but yes this was a pretty stupid bug if that's the point you're trying to make | 18:12 |
timrc | jeblair: :) Oh, I know | 18:12 |
phschwartz | waynr: My point is maybe there hasn't been enough oversite lately with jjb and things are being rushed to merge. | 18:13 |
fungi | waynr: i don't personally think that it's a stupid bug, just that it's a case worth testing which wasn't previously tested | 18:13 |
jeblair | fungi: ++ | 18:13 |
notmorgan | fungi: as are most bugs | 18:13 |
*** eharney_ has joined #openstack-infra | 18:13 | |
fungi | i can totally see how that could have broken (granted, electrofelix also spotted it in review and maybe should have pressed for adding a test for that scenario) | 18:14 |
jeblair | anyway, if the jjb folks want to have room to make master unstable, i think that's fine and folks can pin to the release (or, v2 work could happen in a feature branch) | 18:14 |
timrc | phschwartz: I dunno, hopefully we can create systems that support rapid change. | 18:15 |
*** eharney has quit IRC | 18:15 | |
SpamapS | I don't thing bugs are stupid. I do think we were a little silly for not running our own tests for something we run from master. ;) | 18:15 |
waynr | i also don't think these patches are necessarily being rushed to merge so much as we were unaware of the impact that merging questionable code into master would have on others using JJB | 18:15 |
*** shashank_hegde has joined #openstack-infra | 18:16 | |
fungi | all code is questionable | 18:16 |
SpamapS | yep, it's our fault for not having tests for our use case | 18:17 |
SpamapS | so don't think anybody here is blaming jjb for this :) | 18:17 |
timrc | waynr: FWIW I really appreciate all the activity around jjb. Stuff happens, we learn from it, we fix it, we move on :) | 18:17 |
SpamapS | And, we're pinning to 1.6.1 now | 18:17 |
SpamapS | because we don't actually want to write those tests. ;) | 18:17 |
waynr | ha | 18:17 |
SpamapS | Though I do kind of wonder... | 18:18 |
SpamapS | would we have exploded when we unpinned to 2.0? | 18:18 |
SpamapS | not sure how one gets their use case tested | 18:18 |
waynr | patches are definitely welcome but I am making a note in the 2.0 API etherpad to add a test for that particular case | 18:18 |
*** gouthamr has joined #openstack-infra | 18:18 | |
waynr | (when we fix and re-submit the original patch) | 18:18 |
*** eharney_ has quit IRC | 18:19 | |
phschwartz | waynr: which reminds me, have you revisted that review I put up fixing the config issue faced by people not passing --conf and expecting to get the standard conf files used. | 18:19 |
pabelanger | zigo: do you need to import an existing repo? | 18:20 |
pabelanger | zigo: if so, you need to use upstream: | 18:20 |
zigo | pabelanger: Yes. | 18:20 |
zigo | Fixing. | 18:20 |
pabelanger | jeblair: great, thanks | 18:20 |
waynr | phschwartz: it's in my browser now, looks like it causes XML output to change: http://logs.openstack.org/84/347984/3/check/jenkins-job-builder-compare-xml/f6f72c2/ | 18:20 |
*** davideagnello has quit IRC | 18:21 | |
openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Fix gerrit/projects.yaml for deb-auto-backports https://review.openstack.org/351866 | 18:22 |
*** _nadya_ has joined #openstack-infra | 18:22 | |
jlk | Quick poll, does anybody care that the Blue Box cloud you use is named "MontyTaylor" ? | 18:23 |
*** daemontool_ has quit IRC | 18:23 | |
pabelanger | is OverloadMontyTaylor an option? | 18:24 |
timrc | pabelanger: You may get -EMONTY... | 18:25 |
jeblair | jlk: not really -- we mask that and call it 'bluebox-sjc1' in anything public facing | 18:25 |
jeblair | jlk: if you wanted to change it tho, it would not be a problem | 18:26 |
jlk | I don't want to change it | 18:26 |
phschwartz | waynr: hmm, not sure why that is failing. The compare-xml-new never returns a response. | 18:26 |
*** mctaylor has joined #openstack-infra | 18:26 | |
jlk | because changing endpoints and such is lame. | 18:26 |
pabelanger | I'm fine with status quo, makes bringing the cloud online easier IMO | 18:27 |
jeblair | jlk: then likely not worth it :) | 18:27 |
*** senk has joined #openstack-infra | 18:27 | |
fungi | jlk: i think it will lead to much amusement. let's not rename ;) | 18:28 |
fungi | "has anybody noticed that MontyTaylor is full of failing nodes this morning?" "did you see how fast that devstack job ran in MontyTaylor?" | 18:29 |
jlk | haha | 18:29 |
fungi | the possibilities are endless | 18:29 |
pabelanger | :) | 18:29 |
jlk | montytaylor has fallen over again. | 18:29 |
jlk | Completely exhausted. | 18:29 |
fungi | thankfully that doesn't happen like it used to | 18:30 |
openstackgerrit | Merged openstack-infra/infra-specs: Initial commit of the Pholio spec https://review.openstack.org/340641 | 18:30 |
jlk | Too bad we didn't name it mordred so that his IRC gets pinged every time we talk about it. | 18:30 |
*** davideagnello has joined #openstack-infra | 18:30 | |
fungi | i already get amused whenever i update a ticket in one of our rackspace accounts and see support address us as "monty" (mordred) or "jonathan" (jbryce) or "paul" (pvo) depending on the account | 18:31 |
waynr | phschwartz: i'm working on an update to your patch right now | 18:32 |
*** eharney_ has joined #openstack-infra | 18:33 | |
jlk | oh pvo. | 18:34 |
*** _ari_ has quit IRC | 18:34 | |
openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Correct configuration file usage. https://review.openstack.org/347984 | 18:35 |
*** kzaitsev_mb has joined #openstack-infra | 18:35 | |
waynr | phschwartz: does that address your problem? ^^ | 18:37 |
*** roxanaghe has joined #openstack-infra | 18:38 | |
waynr | i can try to remember to take another look tonight if not, but i really can't split my attention between work i am being paid for (and have to stay later to make progress on) and JJB anymore today | 18:39 |
phschwartz | waynr: that looks good. | 18:39 |
*** gomarivera has joined #openstack-infra | 18:40 | |
*** sputnik13 has joined #openstack-infra | 18:41 | |
*** senk has quit IRC | 18:44 | |
*** gomarivera has quit IRC | 18:44 | |
*** rbrndt has joined #openstack-infra | 18:45 | |
*** roxanaghe has quit IRC | 18:46 | |
*** e0ne_ is now known as e0ne | 18:46 | |
*** xnox has quit IRC | 18:47 | |
*** PollyZ has joined #openstack-infra | 18:49 | |
*** tongli has quit IRC | 18:51 | |
*** electrofelix has joined #openstack-infra | 18:52 | |
*** xnox has joined #openstack-infra | 18:52 | |
*** _nadya_ has quit IRC | 18:52 | |
*** amitgandhinz has quit IRC | 18:53 | |
*** esberglu has quit IRC | 18:53 | |
*** amitgandhinz has joined #openstack-infra | 18:54 | |
*** PollyZ_ has joined #openstack-infra | 18:54 | |
openstackgerrit | Kevin Daniels proposed openstack-infra/jenkins-job-builder: Add support for xcode plugin https://review.openstack.org/351883 | 18:55 |
jroll | fungi: curious if the pvo accounts get a faster response :) | 18:55 |
fungi | jroll: hard to tell, honestly | 18:56 |
fungi | we don't open as many tickets under that account | 18:56 |
jroll | ah | 18:56 |
*** PollyZ has quit IRC | 18:57 | |
*** cody-somerville has joined #openstack-infra | 18:58 | |
*** cody-somerville has joined #openstack-infra | 18:58 | |
zxiiro | sorry was out for lunch, catching up on chat now | 18:58 |
*** jimbaker has quit IRC | 18:58 | |
*** ddieterly[away] has quit IRC | 19:00 | |
*** jimbaker has joined #openstack-infra | 19:02 | |
*** jimbaker has joined #openstack-infra | 19:02 | |
*** fifieldt has quit IRC | 19:02 | |
*** xnox has quit IRC | 19:03 | |
*** roxanaghe has joined #openstack-infra | 19:03 | |
*** karthik__ has quit IRC | 19:04 | |
*** electrofelix has quit IRC | 19:04 | |
*** jaosorior has quit IRC | 19:04 | |
*** ddieterly has joined #openstack-infra | 19:05 | |
*** baoli has quit IRC | 19:07 | |
zxiiro | ouch that's a bad bug, sorry to hear we caused someone to lose their jenkins jobs | 19:08 |
openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add labels panel to nodepool dashboard https://review.openstack.org/351886 | 19:08 |
*** xnox has joined #openstack-infra | 19:09 | |
*** esberglu has joined #openstack-infra | 19:10 | |
pabelanger | fungi: I think you were talking about node type in grafana this week. ^should address that. cc jeblair | 19:12 |
pabelanger | adds another panel to display that info | 19:12 |
*** sarob has quit IRC | 19:13 | |
jeblair | pabelanger: ++ | 19:14 |
openstackgerrit | Zane Bitter proposed openstack-infra/git-review: Clarify that submitting multiple commits is OK https://review.openstack.org/351888 | 19:14 |
*** eharney_ is now known as eharney | 19:16 | |
*** roxanaghe has quit IRC | 19:18 | |
*** fifieldt has joined #openstack-infra | 19:19 | |
*** xnox has quit IRC | 19:19 | |
*** piet has quit IRC | 19:20 | |
*** karthik__ has joined #openstack-infra | 19:20 | |
*** rbuzatu has quit IRC | 19:21 | |
*** rbuzatu has joined #openstack-infra | 19:21 | |
*** thorst__ has joined #openstack-infra | 19:23 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Don't run undercloud idempotent on tempest jobs https://review.openstack.org/351066 | 19:23 |
openstackgerrit | Mathieu Gagné proposed openstack/os-client-config: Update Internap information https://review.openstack.org/351889 | 19:24 |
*** burgerk_ has joined #openstack-infra | 19:25 | |
*** thorst_ has quit IRC | 19:26 | |
*** thorst__ is now known as thorst_ | 19:26 | |
*** xnox has joined #openstack-infra | 19:26 | |
*** sdake has quit IRC | 19:26 | |
zaro | mtreinish: would you have time to take another look at the gerrit plugin to save test metadata? | 19:27 |
*** sdake has joined #openstack-infra | 19:28 | |
*** burgerk has quit IRC | 19:29 | |
*** e0ne has quit IRC | 19:29 | |
*** roxanaghe has joined #openstack-infra | 19:30 | |
*** cody-somerville has quit IRC | 19:31 | |
*** davideagnello has quit IRC | 19:33 | |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update cobertura to use convert xml https://review.openstack.org/351891 | 19:33 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul: Cloner: Update heads and tags from the cache https://review.openstack.org/350637 | 19:34 |
*** davideagnello has joined #openstack-infra | 19:34 | |
*** shashank_hegde has quit IRC | 19:36 | |
*** xnox has quit IRC | 19:37 | |
mtreinish | zaro: do you have a link? | 19:41 |
*** baoli has joined #openstack-infra | 19:42 | |
jeblair | fungi: is there a reason not to use the same format for the fetch calls we just added here? http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/merger/merger.py#n181 | 19:43 |
jeblair | (side question, i don't immediately see why we couldn't actually use the Repo() class there even.... | 19:44 |
fungi | jeblair: oh, i missed those | 19:44 |
*** xnox has joined #openstack-infra | 19:44 | |
jeblair | but that's of secondary importance) | 19:44 |
fungi | yep, now i remember when we added those | 19:45 |
jeblair | fungi: so, specifically, i'm wondering if: 1) the '+refs:refs' bit is unecessary (my guess is yes?) 2) i dunno about update_head_ok, maybe that is a difference we need?. 3) since that bit of code is getting complicated, is it possible to use the Repo class instead. | 19:46 |
zaro | pleia2: weekend is upon us. can i interst you to review these gerrit backports? https://review.openstack.org/#/q/status:open+project:openstack-infra/gerrit+branch:openstack/2.11.4+topic:gerrit-upgrade | 19:46 |
fungi | jeblair: yeah, i think we can reuse some of this. really i just want to make sure that we fetch heads and tags from the cache, so we're only subsequently fetching new heads and tags from the remote when they weren't present in the cache | 19:47 |
fungi | it dawned on me that the fact that we jump through hoops to pre-cache all teh heads and tags is pointless with zuul-cloner just doing a straight clone from the cache | 19:48 |
*** thiagolib has quit IRC | 19:48 | |
jeblair | fungi: yep, i'm down with the improvement. | 19:49 |
fungi | jeblair: we could probably reuse merger.Repo, i agree | 19:49 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add ability to upload duplicate images https://review.openstack.org/349710 | 19:50 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add a 'meta' passthrough parameter for glance images https://review.openstack.org/350114 | 19:50 |
fungi | though will take a bit more refactoring than i expect to find time for this afternoon, so i'll wip this and make a note | 19:50 |
asselin | Can I get a quick review on this (simple) change: https://review.openstack.org/#/c/344336/ | 19:53 |
asselin | for openstack-infra/ansible-role-puppet | 19:54 |
*** roxanaghe has quit IRC | 19:55 | |
openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-gerrit: Ensure git cronjob absent for correct user https://review.openstack.org/351820 | 19:56 |
fungi | pabelanger: ^ that one had angry rocketships | 19:56 |
fungi | puppet-lint now also seems to object when you have extra spaces before your rockets, even if they're aligned | 19:57 |
fungi | though it doesn't complain if there's only one rocketshipped line with extra space | 19:57 |
pabelanger | fungi: +2 | 19:57 |
anteaya | thank goodness for lint | 19:57 |
fungi | dunno how i'd ever survive without it looking after my spacebar | 19:58 |
anteaya | ha ha ha | 19:58 |
*** clenimar_ has quit IRC | 19:58 | |
fungi | end spacebar abuse now! | 19:58 |
anteaya | ha ha ha | 19:58 |
mordred | fungi: you never want the wrong amount of space aroud your rocketship | 19:58 |
jeblair | mordred: truer words have never been said | 19:58 |
anteaya | hurry you can make the weekend protest schedule | 19:58 |
fungi | mordred: unless you only have one | 19:58 |
fungi | and then it's apparently okay | 19:58 |
fungi | (more likely an oversight the whitespace fascists will address as soon as they find out) | 19:59 |
*** megm has quit IRC | 19:59 | |
*** xnox has quit IRC | 19:59 | |
*** xnox has joined #openstack-infra | 19:59 | |
anteaya | asselin: I only have +1 on ansible-role-puppet, but you have it | 19:59 |
*** rossella_s has quit IRC | 20:00 | |
asselin | anteaya, thanks | 20:00 |
anteaya | welcome, your patch makes sense to me | 20:00 |
*** rossella_s has joined #openstack-infra | 20:00 | |
fungi | mordred: jeblair: you may be interested in 351816 and 351820 which i'm surprised haven't caused more corruption already but we should fix before 04:07 utc | 20:01 |
*** megm has joined #openstack-infra | 20:01 | |
*** ddieterly is now known as ddieterly[away] | 20:02 | |
fungi | it seems to probably be the culprit for the puppet-openstackid repo breaking on git04 a week ago and now git08 today | 20:02 |
openstackgerrit | Merged openstack-infra/ansible-role-puppet: Honor show-diff options with puppet apply https://review.openstack.org/344336 | 20:03 |
fungi | somehow that repo has won the race twice and been killed for its trouble | 20:03 |
anteaya | hopefully nevermore | 20:03 |
anteaya | thanks jeblair | 20:04 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Move 'delete' yaml parsing to JenkinsJobs.execute. https://review.openstack.org/262954 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Correct default value for hold-following-changes https://review.openstack.org/350725 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: Improve support in fake builds https://review.openstack.org/350726 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Restore test_parallel_changes https://review.openstack.org/350727 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove unused Inheritable class https://review.openstack.org/350728 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: remove FakeWorker https://review.openstack.org/350730 | 20:04 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: ensure fake builds recorded in order https://review.openstack.org/351897 | 20:04 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Move subcommand logic into subcommand classes. https://review.openstack.org/319617 | 20:05 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Don't expose YamlParser methods unnecessarily. https://review.openstack.org/319618 | 20:05 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: TestSubCommand inherit from UpdateSubCommand. https://review.openstack.org/319619 | 20:05 |
jeblair | i inserted a new change in that stack that should address the random test failures | 20:05 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Remove YamlParser references from JJB modules. https://review.openstack.org/319620 | 20:05 |
openstackgerrit | Merged openstack-infra/jenkins-job-builder: Remove deprecated 'extended-choice' property. https://review.openstack.org/319621 | 20:06 |
*** amitgandhinz has quit IRC | 20:07 | |
*** Apoorva has quit IRC | 20:07 | |
*** ddieterly[away] is now known as ddieterly | 20:07 | |
*** ddieterly is now known as ddieterly[away] | 20:08 | |
*** _nadya_ has joined #openstack-infra | 20:08 | |
*** gomarivera has joined #openstack-infra | 20:09 | |
*** xnox has quit IRC | 20:10 | |
*** _nadya_ has quit IRC | 20:10 | |
*** cody-somerville has joined #openstack-infra | 20:10 | |
krotscheck | Hey, anyone around who can de-queue/kill a job for us? | 20:11 |
openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update Build Name Setter plugin to use convert xml https://review.openstack.org/351899 | 20:12 |
anteaya | krotscheck: which patch/job? | 20:12 |
krotscheck | anteaya: The release job for the 4.0.0 tag on eslint-config-openstack | 20:13 |
* krotscheck is trying to find it. | 20:13 | |
*** rbuzatu has quit IRC | 20:14 | |
krotscheck | anteaya: I don't see it in zuul. | 20:14 |
*** gomarivera has quit IRC | 20:14 | |
Shrews | jeblair: can you re-approve https://review.openstack.org/350623 for us? | 20:14 |
Shrews | jeblair: i rebased it to get the timeout change | 20:14 |
anteaya | I don't see anything in the tag pipeline in zuul | 20:15 |
*** Apoorva has joined #openstack-infra | 20:15 | |
jeblair | Shrews: done | 20:15 |
krotscheck | anteaya: I don't either. | 20:15 |
anteaya | is this job running on a specific patch? | 20:15 |
krotscheck | anteaya: I do see the remote tag in gerrit though | 20:15 |
anteaya | can you share what you are seeing? | 20:16 |
anteaya | so far I don't have a reference for what you are asking help for | 20:16 |
*** xnox has joined #openstack-infra | 20:16 | |
krotscheck | anteaya: http://paste.openstack.org/show/550675/ | 20:16 |
fungi | anteaya: krotscheck: http://logs.openstack.org/13/13a4e1f9ab9a0d5be3abece054ce3eaab96a396b/ | 20:16 |
*** thorst__ has joined #openstack-infra | 20:16 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add documentation on howto consume events from firehose.o.o https://review.openstack.org/351900 | 20:16 |
mtreinish | fungi: ^^^ I had a spare sec so I wrote up the client side docs | 20:17 |
krotscheck | fungi, anteaya: Thanks! I'm walking betherly through the release process, and we accidentally incremented once too often. | 20:17 |
fungi | mtreinish: oh, thank you! my last 48 hours have been less conducive than i expected | 20:17 |
betherly | thanks for your help sorting this krotscheck, fungi, anteaya :) sorry!!! | 20:17 |
anteaya | no no worries looks like from the timestamp the job finished on its own | 20:18 |
anteaya | kudos for taking the time to mentor someone krotscheck :) | 20:18 |
*** thorst_ has quit IRC | 20:18 | |
anteaya | good learning betherly | 20:18 |
mtreinish | fungi: no worries, I figured that was the case :) | 20:18 |
openstackgerrit | Merged openstack-infra/system-config: Ensure git cronjob absent for correct user https://review.openstack.org/351816 | 20:18 |
krotscheck | anteaya: Yeah, that job finishing is not a good thing. We skipped a version we didn't want to skip. | 20:19 |
fungi | krotscheck: betherly: that 13a4e1f9ab9a0d5be3abece054ce3eaab96a396b is the result of `git show-ref 4.0.0` in that repo, or looking at the parenthetical sha shown at http://git.openstack.org/cgit/openstack/eslint-config-openstack/tag/?h=4.0.0 | 20:19 |
krotscheck | It's not a huge problem. | 20:19 |
anteaya | krotscheck: ah well I think we missed the boat on being able to stop it | 20:19 |
krotscheck | Right, so do we care enough to unpublish the package from npm. | 20:20 |
krotscheck | Hrm - it's not published to npm yet. | 20:20 |
anteaya | I personally do not | 20:20 |
anteaya | however this is the first I have heard of this repo | 20:20 |
anteaya | so I have zero emotional investment in its tagging history | 20:20 |
jeblair | it skipped the upload because the tarball build failed | 20:21 |
anteaya | so I might be the wrong person to ask | 20:21 |
openstackgerrit | Merged openstack-infra/nodepool: Add config support for ZooKeeper servers https://review.openstack.org/350623 | 20:21 |
fungi | yeah, it seems it tried to checkout the tag from the zuul-merger, which didn't have it on hand | 20:21 |
*** davideagnello has quit IRC | 20:22 | |
*** baoli has quit IRC | 20:23 | |
*** _nadya_ has joined #openstack-infra | 20:23 | |
openstackgerrit | Merged openstack-infra/shade: Remove discover from test-requirements https://review.openstack.org/345806 | 20:23 |
krotscheck | fungi: Was that in response to my thread? | 20:23 |
fungi | krotscheck: yes | 20:23 |
openstackgerrit | Merged openstack-infra/shade: Move list_magnum_services to OperatorCloud https://review.openstack.org/344395 | 20:23 |
krotscheck | fungi: Hrm. So the actual publish failed? That's a little alarming. | 20:23 |
*** xnox has quit IRC | 20:23 | |
fungi | krotscheck: yes, the gerrit-git-prep builder failed to find the tag, and so the job bailed out at that point | 20:24 |
*** _nadya_ has quit IRC | 20:24 | |
krotscheck | fungi: Ok, so help me here. When we want to do a release, we push a tag, but that tag wasn't available, so... is there an order-of-operations problem here? | 20:25 |
fungi | krotscheck: for reasons i don't yet know, http://zm08.openstack.org/p/openstack/eslint-config-openstack was missing a copy of the tag which should have been retrieved from gerrit before running the job (i think) | 20:25 |
krotscheck | fungi: Got it. Sounds like we can't do anything right now until there's more information? | 20:26 |
krotscheck | (We being myself and betherly) | 20:26 |
fungi | krotscheck: yes, did you want that to get published to npm? i can reenqueue that ref | 20:27 |
betherly | fungi: krotscheck: so i ran 'npm version major' not the push tags part or git review etc etc | 20:27 |
krotscheck | fungi: Well, could you just delete the ref for us in gerrit so we can try again? | 20:27 |
krotscheck | (or can we do that?) | 20:27 |
openstackgerrit | Merged openstack-infra/shade: Stop creating cloud objects in functional tests https://review.openstack.org/344396 | 20:27 |
* krotscheck tries to push a tag delete. | 20:28 | |
* krotscheck expects the world to explode | 20:28 | |
*** diltram has joined #openstack-infra | 20:28 | |
diltram | hey | 20:28 |
betherly | fungi: because i didnt have gpg set up. so then once gpg was sorted i did the process again this time with the push tag part | 20:28 |
fungi | krotscheck: no, deleting and recreating that tag would only make things worse. deletion or replacement of existing tags don't propagate via pull/fetch | 20:28 |
diltram | how is configured sending updates from review to launchpad? | 20:28 |
betherly | could that have been why v3.0.0 missed the tag? | 20:28 |
krotscheck | betherly: Naah, v3.0.0 failed because npm requires that you sign tags. | 20:29 |
krotscheck | (as do we) | 20:29 |
fungi | diltram: http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_bug.py | 20:29 |
krotscheck | And you didn't have gpg set up | 20:29 |
betherly | ok | 20:29 |
krotscheck | fungi: In that case, just re-enqueue that ref. | 20:29 |
fungi | diltram: you want to know how it works, or where to configure it in our system? | 20:29 |
diltram | where to configure | 20:29 |
diltram | because for octavia this is not working | 20:29 |
diltram | I just uploaded a review and there is no info on launchpad about that | 20:30 |
fungi | diltram: there are a number of potential reasons for it not to work. rather than trying to rule them all out, let's start with an example of the change you feel should have updated a bug | 20:30 |
rm_work | I think sometimes it has to do with placement of the Closes-Bug: message | 20:30 |
rm_work | https://review.openstack.org/#/c/265065/ | 20:30 |
rm_work | ^^ that one | 20:30 |
*** davideagnello has joined #openstack-infra | 20:31 | |
diltram | exactly | 20:31 |
rm_work | I see it happen a lot with changes in Octavia though... maybe it's just because we are bad at Gerrit :P | 20:31 |
diltram | hahaha :P | 20:31 |
fungi | diltram: rm_work: https://bugs.launchpad.net/octavia shows your bug supervisor is "michael johnson" | 20:32 |
fungi | that's the problem | 20:32 |
rm_work | michael johnson is the problem? :P | 20:32 |
diltram | the problem is with our PTL? ok changing :P | 20:32 |
*** dimtruck is now known as zz_dimtruck | 20:32 | |
fungi | well, the fact that it's not a group containing the OpenStack Infra (hudson-openstack) lp account is the problem | 20:33 |
anteaya | http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad | 20:33 |
*** ddieterly[away] is now known as ddieterly | 20:33 | |
rm_work | can't find a user named "octavia-bugs" | 20:34 |
fungi | if our automation account isn't in your bug supervisor group, it lacks sufficient permissions to reassign a bug, so if the bug update would also involve reassignment (for example, initial change upload and the bug is not yet assigned to the change uploader) then it will fail to reassign the bug due to insufficient permissions and bail out before commenting | 20:34 |
rm_work | maybe I skipped a step | 20:34 |
anteaya | rm_work: which is why I found the set up launchpad docs: http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad | 20:34 |
openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Add ZooKeeper connection listener https://review.openstack.org/351910 | 20:34 |
diltram | ok, I'm gonna send that document to Michael to update that stuff, thx fungi :) | 20:34 |
rm_work | ah, the note | 20:34 |
rm_work | anteaya: yeah that is what I was reading | 20:34 |
anteaya | great | 20:35 |
anteaya | rm_work: if you don't find the solution in there, please send a patch to the doc | 20:35 |
*** Jeffrey4l has quit IRC | 20:35 | |
*** Jeffrey4l has joined #openstack-infra | 20:36 | |
fungi | krotscheck: confirmed, zm08 still seems to not have the 4.0.0 tag in its copy of the repo. checking logs to see if i can figure out why | 20:36 |
anteaya | fungi: also I'm not sure we can do anything about it but bknudson is finding spam where he is expecting keystone bug reports | 20:36 |
krotscheck | fungi: Thanks. Does the review have to land for that to get propagated? | 20:37 |
fungi | anteaya: i think lp has a "report as spam" link for bug comments | 20:37 |
anteaya | fungi: ah thank you | 20:37 |
fungi | krotscheck: what review? | 20:37 |
fungi | krotscheck: did you push a tag for a commit which isn't in the branch yet? | 20:37 |
krotscheck | fungi: I believe that may have happened, because communication breakdown. | 20:38 |
diltram | thx fungi and CU people | 20:38 |
fungi | krotscheck: oh... if so, that would completely explain this. i'm confirming now | 20:38 |
*** diltram has left #openstack-infra | 20:38 | |
krotscheck | fungi: We're working on getting the correct patch up | 20:38 |
*** shardy_afk is now known as shardy | 20:38 | |
fungi | krotscheck: betherly: for future reference, http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release | 20:39 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add ability to upload duplicate images https://review.openstack.org/349710 | 20:39 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add a 'meta' passthrough parameter for glance images https://review.openstack.org/350114 | 20:39 |
fungi | krotscheck: betherly: yep, mystery solved. i don't see the tagged commit showing up in the history of master (your only branch) | 20:40 |
*** r-daneel has joined #openstack-infra | 20:41 | |
fungi | krotscheck: betherly: in fact, i don't see the tagged commit showing up anywhere in gerrit, even proposed | 20:41 |
*** HeOS has joined #openstack-infra | 20:41 | |
*** gouthamr has quit IRC | 20:41 | |
krotscheck | fungi: betherly is doing an interactive rebase right now, give her a sec. | 20:42 |
fungi | as a result, i recommend not using 4.0.0 as a version number at all. probably need 4.0.1 when you do push a tag for an actual merged commit | 20:42 |
fungi | though it's _possible_ this is one of those rare cases where deleting and replacing the tag isn't a problem, since nobody could have fetched it | 20:43 |
betherly | fungi: so ive just done the rebase and review | 20:43 |
betherly | should i amend the commit to 4.0.1? | 20:43 |
fungi | it's probably safest to do that, yes | 20:43 |
fungi | krotscheck: ^ ? 4.0.1 okay? | 20:43 |
krotscheck | No problems here. | 20:44 |
*** kien-ha has quit IRC | 20:44 | |
openstackgerrit | Merged openstack-infra/puppet-openstackci: Pin jjb to 1.6.1 https://review.openstack.org/351865 | 20:44 |
betherly | do i do that just by literally amending the file or do i need to do something in terminal too? | 20:44 |
betherly | fungi: ^ | 20:44 |
*** gomarivera has joined #openstack-infra | 20:44 | |
betherly | krotscheck: fungi: and im guessing change the commit message to 4.0.1 too? | 20:44 |
fungi | betherly: i'm not a javascript expert, but it sounds like you need to edit package.json to say 4.0.1, propose that with git-review, wait for it to get approved and merge, then checkout the master branch tip and tag that | 20:45 |
openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Implement TripleO centos-7-nonha-multinode-updates https://review.openstack.org/351914 | 20:46 |
fungi | because it's always possible in gerrit that another change races yours to merging, and then the head of master is no longer the same commit which was the local parent of your commit, so gerrit ends up needing to create a merge commit to stitch it into the branch | 20:46 |
*** thorst__ has quit IRC | 20:46 | |
fungi | so it's best to avoid tagging your commit until you know the branch state is updated and then tag the branch tip | 20:46 |
fungi | from the remote repo | 20:47 |
fungi | that's why our instructions mention `git pull --ff-only` | 20:48 |
fungi | you never want to tag a possibly out of sync local copy of the branch | 20:48 |
*** burgerk_ has quit IRC | 20:49 | |
betherly | fungi: ok that all makes sense. thanks for helping with this | 20:49 |
anteaya | seems reporting launchpad spam is done here: https://answers.launchpad.net/launchpad by posting a question, like can you remove this spam please | 20:50 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Reinstall python-requests in bootstrap script https://review.openstack.org/350521 | 20:50 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Make $TRIPLEO_ROOT overrideable https://review.openstack.org/350539 | 20:50 |
*** thorst_ has joined #openstack-infra | 20:50 | |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Raise ping test count after multinode setup https://review.openstack.org/351916 | 20:50 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Fix resetting /etc/hosts https://review.openstack.org/351917 | 20:50 |
*** zz_dimtruck is now known as dimtruck | 20:52 | |
*** csomerville has joined #openstack-infra | 20:53 | |
*** shashank_hegde has joined #openstack-infra | 20:55 | |
*** thorst_ has quit IRC | 20:55 | |
*** vvalderrv has left #openstack-infra | 20:55 | |
*** netsinn has joined #openstack-infra | 20:56 | |
*** cody-somerville has quit IRC | 20:56 | |
*** shardy has quit IRC | 20:58 | |
*** netsin has quit IRC | 20:58 | |
*** ddieterly is now known as ddieterly[away] | 20:59 | |
*** jkilpatr has quit IRC | 20:59 | |
*** kgiusti has left #openstack-infra | 21:00 | |
*** gomarivera has quit IRC | 21:00 | |
*** netsin has joined #openstack-infra | 21:01 | |
*** mat128 is now known as mat128|weekend | 21:01 | |
*** pradk has quit IRC | 21:03 | |
*** netsinn has quit IRC | 21:04 | |
*** psilvad has quit IRC | 21:04 | |
*** gouthamr has joined #openstack-infra | 21:05 | |
*** gordc has quit IRC | 21:07 | |
*** amitgandhinz has joined #openstack-infra | 21:07 | |
*** yamamoto has joined #openstack-infra | 21:09 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: WIP: Fix the integrated gate filtering on uncategorized https://review.openstack.org/351924 | 21:10 |
*** karthik__ has quit IRC | 21:12 | |
*** karthik__ has joined #openstack-infra | 21:12 | |
*** sdake has quit IRC | 21:12 | |
*** ddieterly[away] is now known as ddieterly | 21:12 | |
*** ldnunes has quit IRC | 21:12 | |
*** dprince has quit IRC | 21:12 | |
*** amitgandhinz has quit IRC | 21:12 | |
*** mriedem has quit IRC | 21:13 | |
*** geguileo has quit IRC | 21:13 | |
*** rbuzatu has joined #openstack-infra | 21:14 | |
*** yamamoto has quit IRC | 21:16 | |
*** rbuzatu has quit IRC | 21:16 | |
*** rbuzatu has joined #openstack-infra | 21:16 | |
openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Correct command wrapping in tarball signing script https://review.openstack.org/351927 | 21:17 |
fungi | dhellmann: ^ finally circled back around to the signing error from earlier and tracked down the issue | 21:17 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Correct command wrapping in tarball signing script https://review.openstack.org/351927 | 21:18 |
*** geguileo has joined #openstack-infra | 21:19 | |
fungi | dhellmann: apparently the brokenness in the script was unnoticed because it still worked for projects whose package name matched their repo name, but in the case of the release-test repo/openstack-release-test package this was not the case | 21:19 |
*** rfolco_ has joined #openstack-infra | 21:21 | |
*** Ravikiran_K has quit IRC | 21:21 | |
*** rfolco_ has quit IRC | 21:21 | |
*** rfolco_ has joined #openstack-infra | 21:21 | |
*** rfolco_ has quit IRC | 21:21 | |
rm_work | fungi: weird, I didn't add it -- must have happened automatically when I set it as the owner | 21:22 |
*** shashank_hegde has quit IRC | 21:23 | |
*** jkilpatr has joined #openstack-infra | 21:25 | |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add some documentation on functional testing tools https://review.openstack.org/351931 | 21:28 |
jeblair | mordred, notmorgan, Shrews: ^ that's not complete by a longshot, but it's perhaps a start. :) | 21:28 |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add plumbing for betamax functional tests. https://review.openstack.org/344397 | 21:29 |
*** matt-borland has quit IRC | 21:30 | |
openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add plumbing for betamax functional tests. https://review.openstack.org/344397 | 21:31 |
mordred | jeblair: woot! | 21:31 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add some documentation on functional testing tools https://review.openstack.org/351931 | 21:31 |
jeblair | self-review :) | 21:31 |
*** rbuzatu has quit IRC | 21:35 | |
*** bnemec is now known as beekneemech | 21:36 | |
zigo | mordred: Hey, can you +2 workflow the last patch ? https://review.openstack.org/#/c/351866/ | 21:38 |
openstackgerrit | Roopak Parikh proposed openstack-infra/project-config: Adding first version of project mors https://review.openstack.org/351933 | 21:38 |
*** rbuzatu has joined #openstack-infra | 21:38 | |
zigo | That way, I can get to build the backports over the week-end... | 21:39 |
*** davideagnello has quit IRC | 21:40 | |
fungi | rm_work: yeah, i think lp auto-adds the group owner as an administrator member | 21:40 |
*** xnox has joined #openstack-infra | 21:41 | |
*** edmondsw has quit IRC | 21:42 | |
*** karthik__ has quit IRC | 21:44 | |
*** mixpix3ls has joined #openstack-infra | 21:44 | |
mixpix3ls | Hey folks, JJB question here. I'd like to reference the name of the job within the job definition. | 21:45 |
mixpix3ls | Is that possible? | 21:45 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 21:46 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 21:46 |
*** mctaylor has quit IRC | 21:48 | |
zaro | mixpix3ls: you need to have configure a project example here: http://docs.openstack.org/infra/jenkins-job-builder/definition.html#project | 21:49 |
mixpix3ls | Woah, I was just looking at that :) | 21:50 |
mixpix3ls | Thanks zaro: | 21:50 |
*** gomarivera has joined #openstack-infra | 21:52 | |
*** davideagnello has joined #openstack-infra | 21:52 | |
*** sdake has joined #openstack-infra | 21:54 | |
*** karthik__ has joined #openstack-infra | 21:56 | |
*** gomarivera has quit IRC | 21:56 | |
*** yamamoto has joined #openstack-infra | 21:59 | |
*** tdasilva has quit IRC | 21:59 | |
*** rlandy has quit IRC | 22:00 | |
*** mixpix3ls has quit IRC | 22:00 | |
*** jcoufal_ has joined #openstack-infra | 22:00 | |
*** timl0101 has quit IRC | 22:02 | |
*** weshay has quit IRC | 22:02 | |
mtreinish | anteaya: if you get a sec can you take a look at: https://review.openstack.org/351900 it's the firehose doc patch for users | 22:03 |
*** jcoufal has quit IRC | 22:04 | |
fungi | ooh, cool the draft is posted now | 22:04 |
fungi | as of a couple minutes ago | 22:04 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add Major upgrade function to the tripleo ci script https://review.openstack.org/323750 | 22:04 |
*** mdrabe has quit IRC | 22:05 | |
fungi | or not... i thought we had a gate-system-config-docs job | 22:05 |
EmilienM | hello there, tripleo folks is preparing some work to run > 2 nodes jobs. I was wondering what infra folks thinks about it, what would be the challenges, etc | 22:05 |
mtreinish | fungi: it's there: http://docs-draft.openstack.org/00/351900/1/check/gate-system-config-docs-ubuntu-xenial/34213ac//doc/build/html/firehose.html | 22:06 |
fungi | mtreinish: thanks--i'm blind | 22:06 |
mtreinish | fungi: looking at that I need to add syntax highlighting to the script | 22:06 |
mtreinish | that's the first thing that stands out :) | 22:06 |
*** signed8b_ has joined #openstack-infra | 22:06 | |
fungi | mtreinish: yeah, you can specify the highlighting method | 22:07 |
fungi | as "python" | 22:07 |
fungi | should do the trick | 22:07 |
fungi | .. code-block:: python | 22:07 |
*** tpsilva has quit IRC | 22:08 | |
openstackgerrit | Merged openstack-infra/project-config: Correct command line logic in pypi-extract-name.py https://review.openstack.org/351857 | 22:08 |
*** signed8bit has quit IRC | 22:09 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Implement TripleO centos-7-nonha-multinode-updates https://review.openstack.org/351914 | 22:09 |
jeblair | fungi: http://docs-draft.openstack.org/00/351900/1/check/gate-system-config-docs-ubuntu-xenial/34213ac//doc/build/html/ | 22:10 |
jeblair | fungi: it's called gate-system-config-docs-ubuntu-xenial now | 22:10 |
*** msuriar has left #openstack-infra | 22:10 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Fix the integrated gate filtering on uncategorized https://review.openstack.org/351924 | 22:11 |
mtreinish | fungi: ^^^ now with highlights | 22:11 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: upgrades job is renamed to updates https://review.openstack.org/351937 | 22:12 |
*** karthik__ has quit IRC | 22:13 | |
*** karthik__ has joined #openstack-infra | 22:13 | |
*** dimtruck is now known as zz_dimtruck | 22:14 | |
*** zz_dimtruck is now known as dimtruck | 22:14 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 22:14 |
zigo | fungi: Can you help me finalize the backport repo stuff ? https://review.openstack.org/#/c/351866/ | 22:14 |
*** jcoufal_ has quit IRC | 22:15 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 22:17 |
*** larainema has quit IRC | 22:19 | |
*** jcoufal has joined #openstack-infra | 22:20 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 22:20 |
*** yamahata has quit IRC | 22:21 | |
mtreinish | fungi: heh, you should of called me out on hitting git review on the wrong terminal :) | 22:21 |
openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add documentation on howto consume events from firehose.o.o https://review.openstack.org/351900 | 22:23 |
*** dimtruck is now known as zz_dimtruck | 22:23 | |
*** ddieterly has quit IRC | 22:23 | |
*** _nadya_ has joined #openstack-infra | 22:25 | |
*** karthik__ has quit IRC | 22:26 | |
*** yamamoto has quit IRC | 22:26 | |
*** tdasilva has joined #openstack-infra | 22:27 | |
*** yamahata has joined #openstack-infra | 22:27 | |
openstackgerrit | Merged openstack-infra/puppet-gerrit: Ensure git cronjob absent for correct user https://review.openstack.org/351820 | 22:28 |
*** gomarivera has joined #openstack-infra | 22:29 | |
*** _nadya_ has quit IRC | 22:29 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 22:30 |
*** csomerville has quit IRC | 22:30 | |
*** PollyZ_ has quit IRC | 22:32 | |
*** kzaitsev_mb has quit IRC | 22:33 | |
*** ljjjustin has quit IRC | 22:33 | |
*** gomarivera has quit IRC | 22:33 | |
*** rbuzatu has quit IRC | 22:33 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 22:34 |
openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add conditional check for tripleo-test-cloud-rh1 https://review.openstack.org/346985 | 22:35 |
*** jcoufal has quit IRC | 22:35 | |
openstackgerrit | Corentin Ardeois proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 22:37 |
*** ljjjustin has joined #openstack-infra | 22:37 | |
*** jcoufal has joined #openstack-infra | 22:40 | |
*** roxanaghe has joined #openstack-infra | 22:41 | |
*** kzaitsev_mb has joined #openstack-infra | 22:43 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Fix the integrated gate filtering on uncategorized https://review.openstack.org/351924 | 22:44 |
*** hurgleburgler1 has joined #openstack-infra | 22:46 | |
*** hurgleburgler has quit IRC | 22:47 | |
*** gouthamr has quit IRC | 22:49 | |
*** kzaitsev_mb has quit IRC | 22:51 | |
EmilienM | clarkb: hola, what are your thoughts if tripleo folks would propose a 4-nodes CI job? | 22:51 |
clarkb | EmilienM: what are you trying to accomplish? | 22:52 |
EmilienM | clarkb: deploying TripleO in multinode > 2 nodes | 22:52 |
EmilienM | like having an undercloud (1 node), an overcloud (1 controller, 1 compute, 1 storage) | 22:53 |
fungi | are there common failure modes where an aio controller/compute/storage would be blind? | 22:53 |
clarkb | ah yes that was my next question, will this cover things in a way all in one cant? | 22:54 |
EmilienM | we are testing network isolation | 22:54 |
EmilienM | and also pacemaker, with HA services | 22:54 |
EmilienM | having such a scenario is really close to reality | 22:54 |
EmilienM | most of the bugs we find in production are multinode things, like networking, clustering, etc | 22:55 |
clarkb | is the 2 node job working? | 22:55 |
EmilienM | yes, very good | 22:55 |
clarkb | and does it not cover those bitd? | 22:55 |
EmilienM | we're super happy with it | 22:55 |
openstackgerrit | Merged openstack-infra/project-config: zuul/puppet: enable integration jobs on puppet-barbican https://review.openstack.org/351732 | 22:55 |
*** kzaitsev_mb has joined #openstack-infra | 22:56 | |
clarkb | becausr you get network isolation and can test ha? | 22:56 |
EmilienM | right now, the 2-nodes job deploys an undercloud and an overcloud (aio) | 22:56 |
EmilienM | but installing an overcloud on one node is like a single node devstack | 22:56 |
EmilienM | you don't test live migration | 22:56 |
EmilienM | network isolation, neutron tunneling, etc | 22:56 |
*** signed8b_ is now known as signed8bit_Zzz | 22:56 | |
*** yamamoto has joined #openstack-infra | 22:57 | |
clarkb | neutron tunneling is actually the samr in aio as mulyinide if you dont dvr | 22:57 |
clarkb | just fyi | 22:57 |
*** rbrndt has quit IRC | 22:57 | |
openstackgerrit | Merged openstack-infra/project-config: Fix gerrit/projects.yaml for deb-auto-backports https://review.openstack.org/351866 | 22:57 |
fungi | playing devil's advocate, nova already tests live migration across separate job nodes. why would their testing not be sufficient to keep live migration working in tripleo deployments? | 22:57 |
*** davideagnello has quit IRC | 22:58 | |
EmilienM | fungi: they test it in devstack | 22:58 |
clarkb | my only concern is that peoplr seem to think more nodes == more coverage | 22:58 |
EmilienM | and tripleo/puppet CI have found a lot of interesting bugs that devstack didn't find. | 22:58 |
clarkb | if you actually have a scenatio that requires more nodes then thats great we can probably do it | 22:58 |
fungi | i think the testing framework they use is irrelevant. devstack is just a means to and end. they're not testing live migration in devstack. they're testing live migration | 22:58 |
EmilienM | right but the config is not the same | 22:59 |
fungi | that's sort of like saying you need a job to run nova unit tests under nose because right now they're only tested under testr | 22:59 |
EmilienM | tripleo CI deploys SSL and IPv6 by default | 22:59 |
EmilienM | Puppet CI and TripleO CI deploy SSL and IPv6 in fact | 22:59 |
mtreinish | fungi: they likely wouldn't work at all under nose... :) | 22:59 |
EmilienM | it helped a lot over the last months | 22:59 |
fungi | mtreinish: shush ;) | 22:59 |
EmilienM | you can ask mtreinish how useful was Puppet CI over the last 6 months | 22:59 |
*** hurgleburgler1 has quit IRC | 23:00 | |
EmilienM | AFIK devstack doesn't deploy SSL & Ipv6 in gate | 23:00 |
*** itisha has quit IRC | 23:00 | |
fungi | i'm sure testing the same thing in different ways does often expose different bugs, but might also be a sign that the existing tests need more work to cover some code paths | 23:00 |
EmilienM | and this is a big problem because in reality, most of people deploy at least SSL, etc. That's an example | 23:00 |
jeblair | "well, there's the problem" :) | 23:00 |
fungi | we've suggested in the past ipv6 and ssl should be the default test scenario, fwiw | 23:01 |
EmilienM | I'm sure you did | 23:01 |
EmilienM | but we (puppet/tripleo) made it | 23:01 |
*** ajmiller_ has quit IRC | 23:02 | |
EmilienM | anyway, just asking around about feedback on multinodes | 23:02 |
jeblair | i guess it works now? there shouldn't be any objection to adding it to devstack then | 23:02 |
*** asettle has joined #openstack-infra | 23:02 | |
jeblair | it used to "not work" | 23:02 |
mtreinish | clarkb, fungi: any idea why this job has 2 statuses listed: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_uuid:%20%5C%22f656a52474d9441ebc124008f8cb65bc%5C%22%20AND%20error_pr:%5B%5C%22-1000.0%5C%22%20TO%20%5C%22-10.0%5C%22%5D%20&from=864000s | 23:02 |
EmilienM | jeblair: ssl/ipv6 was an example | 23:03 |
mtreinish | jeblair: the issue with turning it on in devstack wasn't that people didn't want it or it didn't work, but that the devstack support for those isn't really there. The neutron config in particular was in a very bad place | 23:03 |
mtreinish | we're only now getting to a place where lib/neutron is usable | 23:04 |
*** ams__ has quit IRC | 23:04 | |
EmilienM | jeblair: I (and some other folks) am trying to help TripleO project to move their jobs from Red Hat private cloud to use more upstream resources | 23:04 |
*** yamahata has quit IRC | 23:04 | |
mtreinish | clarkb, fungi: oh, there are 2 jobs listed there | 23:04 |
fungi | luckily defining worker types with different numbers of nodes shouldn't require new images and the current devstack-gate implementation is generalized to support additional nodes beyond 2 (though i gather tripleo-ci doesn't use devstack-gate), but it does mean that we have n x min-ready nodes occupied waiting around for jobs that require that node count | 23:04 |
mtreinish | I missed that, I just saw the double status | 23:05 |
*** PollyZ has joined #openstack-infra | 23:05 | |
jeblair | EmilienM: are we really all on board with that? | 23:05 |
EmilienM | jeblair: and the second thing is IMHO tripleo jobs bring super useful feedback as we test things with production environment, with HA, SSL IPV6 | 23:05 |
jeblair | EmilienM: is the tripleo cloud ready to accept jobs from other projects? | 23:05 |
*** pt_15 has quit IRC | 23:06 | |
EmilienM | jeblair: I'm not sure about the future of tripleo cloud right now. | 23:06 |
*** davideagnello has joined #openstack-infra | 23:06 | |
fungi | EmilienM: is tripleo prepared to integrate their scenarios into devstack-gate or are they stuck on using a separate framework? | 23:06 |
EmilienM | pabelanger: if you around, you can certainly help here :) | 23:06 |
*** asettle has quit IRC | 23:06 | |
pabelanger | I think tripleo-test-cloud-rh1 could run other jobs today. | 23:07 |
pabelanger | we have AFS setup | 23:07 |
pabelanger | and using DIBs now | 23:07 |
jeblair | see there was this mailing list thread a while ago: http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html | 23:07 |
jeblair | and it kind of went to a bad place with some folks getting really defensive | 23:07 |
jeblair | which is unfortunate, because in my mind, we're now in a kind of awkward limbo | 23:07 |
EmilienM | jeblair: you asked me if 'we' were all onboard with that. Ask pabelanger, he's working hard on it. | 23:08 |
pabelanger | I still think tripleo-test-cloud-rh1 should be moved to third party | 23:08 |
fungi | ooh, i recognize that e-mail! | 23:08 |
pabelanger | however, if they do exist in nodepool.o.o, there is nothing stopping the cloud from excepting more jobs today. We have everything in place now | 23:08 |
pabelanger | except people agreeing to it | 23:09 |
*** amitgandhinz has joined #openstack-infra | 23:09 | |
jeblair | i like what you are doing with tripleo testing -- afaict, it's getting better and better which is great :) | 23:09 |
EmilienM | thanks | 23:10 |
EmilienM | (if it was addressed to us :)) | 23:10 |
jeblair | yep | 23:10 |
jeblair | we took on the tripleo cloud expecting to have multiple regions for stability, and, at the time, the tripleo project was the "official" deployment project | 23:10 |
jeblair | but *many* things have changed since then | 23:10 |
EmilienM | so back to the request. I'm going to dress a list of benefits to have > 2 nodes jobs | 23:11 |
jeblair | and based on the way it's currently operated, it better fits the third-party ci model (if tripleo were to stand this up today, we'd say "3rd party ci") | 23:11 |
betherly | hi all | 23:11 |
EmilienM | and we'll make sure it's really worth it | 23:11 |
fungi | betherly: welcome back | 23:12 |
betherly | ready to do the final tagging for the release of eslint-config-openstack 4.0.1 | 23:12 |
fungi | betherly: what commit are you tagging? | 23:12 |
betherly | fungi: https://review.openstack.org/#/c/351913/ has merged | 23:12 |
jeblair | EmilienM, pabelanger: i feel it would be better as a third-party ci, but if there are plans for multi-region support and a willingness to make the platform available for all projects, that seems like it might be a good fit for upstream openstack infra | 23:12 |
pabelanger | jeblair: the objections from moving tripleo-test-cloud-rh1 to 3rd party CI was it would result in the loss of voting (check) rights in the gate. | 23:13 |
pabelanger | which, in fact, they do not have today | 23:13 |
betherly | so is it just a matter of checking out the master branch, git tag 4.0.1, git push gerrit --tags | 23:13 |
fungi | pabelanger: it doesn't have those now, no | 23:13 |
*** rhallisey has quit IRC | 23:13 | |
*** csomerville has joined #openstack-infra | 23:13 | |
betherly | fungi: ^ | 23:13 |
jeblair | pabelanger: indeed, it will have the exact same privileges | 23:13 |
*** xarses has quit IRC | 23:13 | |
jeblair | it will just be an acknowledgement that it has a different set of operators than the rest of the system, which is also actually the case today :) | 23:14 |
pabelanger | jeblair: right, this is what I was explaining as well. There was not technical reason they could not vote, but the lack of multi-region cloud was the problem | 23:14 |
*** amitgandhinz has quit IRC | 23:14 | |
pabelanger | But, with that said. There is actually tripleo-test-cloud-rh2 online at the moment, which _could_ be the 2nd region | 23:14 |
pabelanger | right now it is disabled | 23:15 |
fungi | betherly: yeah, `git show-ref HEAD` should indicate it's 4549ae61e2ed19659c3d641ebd37ebdf12eb4d7f after you checkout master and pull --ff-only like in the example in our docs | 23:15 |
betherly | yup | 23:15 |
fungi | betherly: `git tag -s 4.0.1` and `git push gerrit 4.0.1` | 23:16 |
pabelanger | jeblair: but I agree, now that tripleo-jobs are starting to work across other clouds in the gate. There is no technical reasons why we couldn't enable more jobs in tripleo-test-cloud-rh1, even if its 10 nodes | 23:16 |
betherly | fungi: should i do that from the master branch? | 23:16 |
EmilienM | pabelanger: exactly | 23:16 |
jeblair | pabelanger: would it be okay for fuel to use it? | 23:17 |
betherly | fungi: apologies to double check everything. brain not entirely functional at this time so making sure i dont make any more screw ups today lol | 23:17 |
fungi | betherly: you want to make sure you use -s (which you didn't say above) and you need to push a specific tag name since --tags could push all sorts of other local tags you might have made in addition to the one you mean | 23:17 |
fungi | betherly: the example at http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release is the one to follow | 23:17 |
pabelanger | jeblair: I personally don't have an objection. | 23:17 |
pabelanger | jeblair: but that is likely a question for somebody at Red Hat | 23:17 |
betherly | fungi: ok so not in master then. create another branch which is a clone of master? | 23:18 |
jeblair | pabelanger: heh, somebody else at red hat :) | 23:18 |
pabelanger | :D | 23:18 |
fungi | betherly: the example means "master" unless you're making a stable branch tag, and then "stable/something" instead | 23:18 |
betherly | ok | 23:18 |
EmilienM | I haven't seen efforts from Fuel to run their multinode jobs in osinfra afik | 23:18 |
fungi | betherly: you don't need to make a separate branch for this | 23:19 |
pabelanger | EmilienM: I have, there was people in Austin wanting to do it | 23:19 |
betherly | fungi: should the tag message just be the version? | 23:20 |
fungi | EmilienM: fuel team members have expressed interest in running their jobs upstream, but right now a lot of their testing relies on bare-metal access and much larger multinode counts | 23:20 |
*** davideagnello has quit IRC | 23:20 | |
fungi | betherly: i usually look at earlier tags for the same repo, or if there are none then for other related repos, and try to keep the new tag message consistent with them | 23:20 |
fungi | betherly: it could for example be something like "Release eslint-config-openstack version 4.0.1" | 23:21 |
openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: WIP: Re-enable test_connection tests https://review.openstack.org/351949 | 23:21 |
*** claudiub has joined #openstack-infra | 23:22 | |
jeblair | it feels like the tripleo cloud has been grandfathered in its current state for a while. i'd just like to make sure we're being fair to everyone. so if tripleo wants to run tripleo jobs, then i think we should move it to 3rd party ci. i think that's a fine choice and we can continue to work together (please!) but with better division of reponsibilities. or, if we want to revise the idea of a multi-provider hardware platform that's ... | 23:22 |
jeblair | ... available for all openstack projects, i'm game for that. it would be great, but more work. | 23:22 |
betherly | fungi: trying to figure out how to view previous tags to keep it consistent. are they viewable as just previous commits at some point? | 23:22 |
fungi | betherly: `git tag` will list the names of existing tags, and `git show 2.0.0` will show that tag including its message | 23:23 |
fungi | betherly: or you can click on them to browse them on https://git.openstack.org/ | 23:24 |
fungi | betherly: for example http://git.openstack.org/cgit/openstack/eslint-config-openstack/tag/?h=2.0.0 shows the tag message was just "2.0.0" | 23:24 |
betherly | fungi: i was literally about to type just that lol | 23:24 |
fungi | betherly: whereas http://git.openstack.org/cgit/openstack-infra/bindep/tag/?h=2.0.0 shows a tag message of "Release bindep 2.0.0." | 23:25 |
betherly | fungi: would you say best practise would be to keep that consistent or make the next one more descriptive with you suggestion? | 23:25 |
betherly | ie "Release eslint-config-openstack version 4.0.1" | 23:25 |
fungi | i tend to just keep it consistent | 23:25 |
*** Swami has quit IRC | 23:26 | |
fungi | if you switch to release management automation later you'll end up with all sorts of fun stuff injected into the tag message, for example http://git.openstack.org/cgit/openstack/python-novaclient/tag/?h=5.0.0 | 23:26 |
openstackgerrit | Roopak Parikh proposed openstack-infra/project-config: Adding first version of project mors https://review.openstack.org/351933 | 23:26 |
pabelanger | EmilienM: jeblair: I would agree with that. Let me compose an email summarizing when we talked about today, and move it to a thread on the ML. That will get more eyes on the topic. | 23:27 |
EmilienM | pabelanger: thanks | 23:27 |
fungi | i'm going to do some non-openstack things with the rest of my friday night, but will be around this weekend some | 23:27 |
fungi | have a good night, all | 23:27 |
*** karthik__ has joined #openstack-infra | 23:28 | |
*** karthik__ has quit IRC | 23:28 | |
* bkero waves | 23:28 | |
bkero | Have a good weekend | 23:28 |
*** karthik__ has joined #openstack-infra | 23:29 | |
betherly | ughhhhh for some reason i am now getting that the repo does not appear to be a gerrit repo. it seems to have lost its link to gerrit somehow | 23:30 |
*** jcoufal has quit IRC | 23:30 | |
betherly | goodness only knows. friday evenings | 23:30 |
* betherly facepalms | 23:30 | |
*** sdake has quit IRC | 23:31 | |
clarkb | betherly: git review -s configures the repo to talk yo gerrit with git review | 23:32 |
clarkb | not sure what you mean by repo does nit appear to be a gerrit repo | 23:32 |
betherly | clarkb: thanks for sorting me out there | 23:33 |
betherly | can someone confirm for me that eslint-config-openstack v4.0.1 has now tagged and released correctly? | 23:34 |
*** roxanaghe has quit IRC | 23:34 | |
*** xyang1 has quit IRC | 23:35 | |
*** PollyZ has quit IRC | 23:37 | |
*** sdague has quit IRC | 23:38 | |
*** yamamoto has quit IRC | 23:41 | |
*** jcoufal has joined #openstack-infra | 23:41 | |
*** yamamoto has joined #openstack-infra | 23:41 | |
*** yamamoto has quit IRC | 23:41 | |
*** kzaitsev_mb has quit IRC | 23:41 | |
*** zz_dimtruck is now known as dimtruck | 23:43 | |
*** karthik__ has quit IRC | 23:45 | |
*** ljjjustin has quit IRC | 23:45 | |
fungi | i'm not really here but looks okay... http://git.openstack.org/cgit/openstack/eslint-config-openstack http://logs.openstack.org/1d/1d8ecdb54568df15d94ee826bd36c6b33ca61a90/release/ | 23:47 |
fungi | betherly: ^ | 23:47 |
* fungi is really gone now | 23:47 | |
zaro | jeblair: thought you might be interested, gerrit talking about zuul CRD https://groups.google.com/d/msg/repo-discuss/32oU6IaulW4/Ze-MTnNxCgAJ | 23:48 |
*** ljjjustin has joined #openstack-infra | 23:49 | |
betherly | thanks fungi :D | 23:50 |
*** xarses has joined #openstack-infra | 23:51 | |
*** esberglu has quit IRC | 23:52 | |
*** lascii is now known as alaski | 23:56 | |
openstackgerrit | Roopak Parikh proposed openstack-infra/project-config: Adding first version of project mors https://review.openstack.org/351956 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!