| *** atiwari has quit IRC | 00:03 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova bug 1315201 https://review.openstack.org/91707 | 00:07 |
|---|---|---|
| uvirtbot | Launchpad bug 1315201 in nova "test_create_server TimeoutException failed while waiting for server to build in setup" [Undecided,New] https://launchpad.net/bugs/1315201 | 00:07 |
| *** lakshmiS has quit IRC | 00:08 | |
| *** crc32 has joined #openstack-infra | 00:09 | |
| *** bhuvan has quit IRC | 00:15 | |
| *** mriedem has quit IRC | 00:17 | |
| *** arnaud__ has joined #openstack-infra | 00:20 | |
| *** jhesketh has quit IRC | 00:24 | |
| *** matsuhashi has joined #openstack-infra | 00:29 | |
| *** aconrad has quit IRC | 00:31 | |
| *** alexpilotti has quit IRC | 00:37 | |
| *** mestery has joined #openstack-infra | 00:37 | |
| *** amcrn has quit IRC | 00:44 | |
| *** ArxCruz has joined #openstack-infra | 00:47 | |
| *** ArxCruz has quit IRC | 00:47 | |
| *** jamielennox is now known as jamielennox|away | 00:53 | |
| *** arborism has joined #openstack-infra | 00:53 | |
| *** mrodden has quit IRC | 01:00 | |
| yfried | fungi: how does logstash parse tempest test results? from console.html log? | 01:01 |
| yfried | fungi: didn't see any other relvant filter | 01:01 |
| *** marcoemorais has quit IRC | 01:02 | |
| *** arborism has quit IRC | 01:03 | |
| *** arborism has joined #openstack-infra | 01:09 | |
| *** yamahata has quit IRC | 01:10 | |
| *** ramashri has quit IRC | 01:14 | |
| *** ArxCruz has joined #openstack-infra | 01:15 | |
| *** ArxCruz has quit IRC | 01:15 | |
| *** swesto___ has joined #openstack-infra | 01:15 | |
| *** swesto___ has quit IRC | 01:15 | |
| *** etoews has joined #openstack-infra | 01:16 | |
| *** sweston__ has joined #openstack-infra | 01:17 | |
| *** zns has quit IRC | 01:17 | |
| *** sweston__ is now known as sweston | 01:17 | |
| *** tjones has joined #openstack-infra | 01:20 | |
| tjones | hi folks - this review https://review.openstack.org/#/c/81905/ shows jenkins still running, but when i go to the zuul page i do not see it any more | 01:20 |
| fungi | yfried: we have logstash filters defined... there's a "console" one it uses for jenkins job console logs... i'll get you a link | 01:21 |
| *** bhuvan has joined #openstack-infra | 01:23 | |
| jeblair | tjones: i'll fix that in just a sec | 01:23 |
| *** arborism has quit IRC | 01:24 | |
| jeblair | tjones: done, sorry about that | 01:24 |
| *** whoops has joined #openstack-infra | 01:25 | |
| *** saschpe has quit IRC | 01:27 | |
| *** bhuvan has quit IRC | 01:27 | |
| fungi | yfried: i think http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb#n15 is what you're looking for? | 01:28 |
| *** jamielennox|away is now known as jamielennox | 01:28 | |
| fungi | yfried: though the actual result (success/failure) is reported separately in the metadata for the job i think, not parsed from the console log | 01:30 |
| *** mriedem has joined #openstack-infra | 01:31 | |
| *** saschpe has joined #openstack-infra | 01:32 | |
| *** arnaud__ has quit IRC | 01:32 | |
| lifeless | https://review.openstack.org/#/c/89691/ now has one +2; could we get a second and +A pleasE? | 01:34 |
| *** pcrews_ has quit IRC | 01:35 | |
| *** nosnos has joined #openstack-infra | 01:36 | |
| *** praneshp has quit IRC | 01:40 | |
| *** sacharya has joined #openstack-infra | 01:40 | |
| *** arborism has joined #openstack-infra | 01:41 | |
| *** arborism has quit IRC | 01:42 | |
| *** arborism has joined #openstack-infra | 01:42 | |
| *** nosnos has quit IRC | 01:44 | |
| *** nosnos has joined #openstack-infra | 01:44 | |
| *** timrc is now known as timrc-afk | 01:46 | |
| *** arborism has quit IRC | 01:46 | |
| *** weshay has quit IRC | 01:49 | |
| *** nati_uen_ has joined #openstack-infra | 01:55 | |
| *** nati_uen_ has quit IRC | 01:56 | |
| openstackgerrit | A change was merged to openstack-infra/config: Create tripleo-specs git repo https://review.openstack.org/89691 | 01:58 |
| *** nati_ueno has quit IRC | 01:59 | |
| *** signed8bit_ZZZzz has quit IRC | 01:59 | |
| fungi | watching to see if the rube goldberg machine does stuff with things | 02:00 |
| *** annegentle has joined #openstack-infra | 02:00 | |
| tjones | thanks jeblair | 02:01 |
| *** ArxCruz has joined #openstack-infra | 02:02 | |
| *** mfer has quit IRC | 02:03 | |
| tjones | jeblair: i still do not see it on zuul and jenkins still shows it running. did i miss something? | 02:03 |
| *** sdake_ has joined #openstack-infra | 02:04 | |
| *** sdake_ has quit IRC | 02:04 | |
| *** sdake_ has joined #openstack-infra | 02:04 | |
| *** yamahata has joined #openstack-infra | 02:04 | |
| tjones | jeblair: never mind. i see it now. the filter only works on the check column and not the gate column | 02:04 |
| tjones | :-D | 02:04 |
| *** openstackgerrit has quit IRC | 02:04 | |
| *** openstackgerrit has joined #openstack-infra | 02:06 | |
| jgriffith | jeblair: around? | 02:07 |
| fungi | woah, a jgriffith | 02:07 |
| fungi | rare sighting ;) | 02:07 |
| jgriffith | fungi: :) | 02:08 |
| jgriffith | I live | 02:08 |
| jgriffith | It's nice to know I was missed | 02:08 |
| fungi | of course! | 02:08 |
| *** whoops has quit IRC | 02:08 | |
| jgriffith | fungi: So I though I'd get back into it and was getting ready to push my specs repo change | 02:08 |
| jgriffith | fungi: but remembered something.... | 02:09 |
| jgriffith | http://openstack.10931.n7.nabble.com/Spec-repos-for-blueprint-development-and-review-td36105.html | 02:09 |
| jgriffith | wondering what the "real-world" experience has been like | 02:09 |
| jgriffith | and if we're still moving forward on this | 02:09 |
| fungi | seems so (i just approved the tripleo-specs creation change anyway) | 02:10 |
| jgriffith | hehe | 02:10 |
| fungi | nova, qa and neutron are using it already, at least | 02:10 |
| fungi | maybe glance, based on a comment i saw | 02:10 |
| fungi | checking | 02:10 |
| jgriffith | seems I should jump on the bandwagon then I suppose :) | 02:10 |
| *** zz_gondoi is now known as gondoi | 02:11 | |
| jgriffith | It "seems" like a good process improvement, just want to make sure nobody discovered a down-side while I was away | 02:11 |
| fungi | jgriffith: http://paste.openstack.org/show/78349/ | 02:12 |
| fungi | four projects, and someone cared enough to write a cookiecutter for it | 02:12 |
| jgriffith | fungi: well there ya have it then | 02:13 |
| fungi | i was away for much of the same time, so i'm probably not the best to ask about downsidew | 02:13 |
| jgriffith | fungi: if nothing else I don't want to confuse people but not doing it :) | 02:13 |
| fungi | downsides | 02:13 |
| jgriffith | fungi: fair enough, thanks for the input | 02:14 |
| *** mrodden has joined #openstack-infra | 02:14 | |
| jgriffith | fungi: and nice to "see you virtually" again :) | 02:14 |
| fungi | though other projects are pressing forward using it, and discussions have been had about being complimentary to storyboard rather than contradictory, so seems in vogue anyhow | 02:14 |
| openstackgerrit | John Griffith proposed a change to openstack-infra/config: Create cinder-specs git repo https://review.openstack.org/91723 | 02:15 |
| jgriffith | fungi: I am all about being in style ya know | 02:15 |
| fungi | gotta wear the latest cravat | 02:15 |
| jgriffith | haha! | 02:15 |
| jgriffith | fungi: I had to google "cravat" | 02:16 |
| jgriffith | Show's how "stylish" I am | 02:16 |
| jgriffith | as it appears they're becoming popular again | 02:16 |
| lifeless | uhm | 02:17 |
| lifeless | the summit page with developers in action thing is a bit useless | 02:17 |
| lifeless | ' | 02:17 |
| fungi | i'm mildly disturbed | 02:17 |
| lifeless | Developers In Action: | 02:17 |
| lifeless | Jenkins recently made a contribution to OpenStack Compute (nova) r 10176 ' | 02:17 |
| fungi | shoulsa said "ascot" | 02:17 |
| lifeless | Yay Jenkins. | 02:17 |
| jgriffith | fungi: You should've :) | 02:17 |
| lifeless | sorry, community - http://www.openstack.org/community/ | 02:17 |
| *** markwash has quit IRC | 02:18 | |
| jgriffith | lifeless: I think it's neat | 02:18 |
| lifeless | jgriffith: I like the idea | 02:19 |
| lifeless | jgriffith: but Jenkins... | 02:19 |
| fungi | lifeless: i'll bet jenkins wears a smart ascot | 02:19 |
| lifeless | jgriffith: vs 'Fred Bloggs' | 02:19 |
| jgriffith | lifeless: hehe... well, fair enough | 02:19 |
| lifeless | could be a gerrit fallout | 02:19 |
| lifeless | where do bugs on that belong ? | 02:20 |
| *** gondoi is now known as zz_gondoi | 02:20 | |
| fungi | lifeless: if you think we've configured gerrit incorrectly or are maintaining incorrect glue/tooling, then file a bug against openstack-ci on lp | 02:23 |
| fungi | lifeless: if you think new gerrit is broken in some way, then upstream is where the bugs end up (though we'll redirect them if there's some doubt) | 02:23 |
| lifeless | fungi: I haven't read the code (can I) for that community page. | 02:23 |
| lifeless | fungi: so I have no idea what it assumes or how its glued int. | 02:24 |
| fungi | lifeless: good point... lemme see if that ever got opened up | 02:24 |
| lifeless | fungi: I was making a good faith assumption that the community page *did* show actual devs in the past | 02:24 |
| lifeless | fungi: rather than Jenkins since forever, since that would imply it was not tested with actual OpenStack data. | 02:24 |
| fungi | oh, it's on www... reed might know what the deal is with the community page | 02:25 |
| fungi | or mebbe fifieldt | 02:25 |
| lifeless | fungi: I'm also not super blessed with spare time to go spelunking into it | 02:25 |
| lifeless | fifieldt: ^ tag | 02:25 |
| fungi | lifeless: completely sympathize ;) | 02:25 |
| *** mriedem has quit IRC | 02:28 | |
| *** etoews has quit IRC | 02:29 | |
| *** nati_ueno has joined #openstack-infra | 02:33 | |
| *** etoews has joined #openstack-infra | 02:33 | |
| *** bhuvan has joined #openstack-infra | 02:35 | |
| *** talluri has quit IRC | 02:35 | |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/config: Enable more solum services in devstack gate https://review.openstack.org/91726 | 02:36 |
| *** michchap has quit IRC | 02:43 | |
| *** lnxnut has joined #openstack-infra | 02:44 | |
| *** michchap has joined #openstack-infra | 02:45 | |
| *** PaulHend has joined #openstack-infra | 02:45 | |
| *** UtahDave has quit IRC | 02:45 | |
| *** jhesketh has joined #openstack-infra | 02:45 | |
| *** signed8bit has joined #openstack-infra | 02:45 | |
| *** etoews has quit IRC | 02:47 | |
| *** Ryan_Lane has quit IRC | 02:47 | |
| *** harlowja is now known as harlowja_away | 02:50 | |
| *** nosnos has quit IRC | 02:51 | |
| *** nati_ueno has quit IRC | 02:55 | |
| *** jerryz has joined #openstack-infra | 03:02 | |
| *** lnxnut has quit IRC | 03:09 | |
| *** etoews has joined #openstack-infra | 03:09 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Add devstack/f20 experimental test https://review.openstack.org/91728 | 03:10 |
| *** signed8bit has quit IRC | 03:10 | |
| *** jerryz has quit IRC | 03:11 | |
| *** eharney has quit IRC | 03:20 | |
| *** wenlock has joined #openstack-infra | 03:22 | |
| *** etoews has quit IRC | 03:22 | |
| *** gyee has quit IRC | 03:25 | |
| *** matsuhashi has quit IRC | 03:27 | |
| *** sweston has quit IRC | 03:34 | |
| *** eharney has joined #openstack-infra | 03:34 | |
| *** marun has joined #openstack-infra | 03:37 | |
| *** praneshp has joined #openstack-infra | 03:40 | |
| *** crc32 has quit IRC | 03:43 | |
| *** praneshp_ has joined #openstack-infra | 03:44 | |
| *** arnaud has joined #openstack-infra | 03:47 | |
| *** praneshp has quit IRC | 03:47 | |
| *** praneshp_ is now known as praneshp | 03:47 | |
| stevebaker | hai! would someone be able to restore this abandoned change please https://review.openstack.org/#/c/83831/ | 03:49 |
| mattoliverau | stevebaker: You'll need a core or a SpamapS i guess then.. fungi, clarkb, jeblair are any of you here? | 03:52 |
| *** nati_ueno has joined #openstack-infra | 03:53 | |
| fungi | stevebaker: done! | 03:53 |
| stevebaker | fungi: \o/ thanks | 03:53 |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/config: Enable more solum services and neutron in solum in devstack gate https://review.openstack.org/91726 | 03:54 |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/config: Enable more solum services and neutron in solum devstack gate https://review.openstack.org/91726 | 03:55 |
| *** nosnos has joined #openstack-infra | 04:05 | |
| *** maxbit has quit IRC | 04:09 | |
| *** SettleDown is now known as asettle | 04:17 | |
| *** Alexandra_ has quit IRC | 04:25 | |
| *** dstanek is now known as dstanekafk | 04:27 | |
| *** ramashri has joined #openstack-infra | 04:27 | |
| *** yfried has quit IRC | 04:29 | |
| *** ramashri has quit IRC | 04:32 | |
| *** tnurlygayanov has quit IRC | 04:33 | |
| *** tnurlygayanov has joined #openstack-infra | 04:35 | |
| *** chandan_kumar has joined #openstack-infra | 04:36 | |
| *** nati_ueno has quit IRC | 04:43 | |
| *** Ryan_Lane has joined #openstack-infra | 04:46 | |
| *** reed has quit IRC | 04:50 | |
| *** yfried has joined #openstack-infra | 04:53 | |
| yfried | fungi: tnx, I've seen this file. where is the conf for the result report? | 05:00 |
| fifieldt | lol@ community page | 05:01 |
| fifieldt | go Jenkins! | 05:01 |
| *** matsuhashi has joined #openstack-infra | 05:03 | |
| *** nati_ueno has joined #openstack-infra | 05:04 | |
| *** nati_ueno has quit IRC | 05:04 | |
| *** arnaud has quit IRC | 05:07 | |
| *** Ryan_Lane1 has joined #openstack-infra | 05:10 | |
| *** changbl has joined #openstack-infra | 05:21 | |
| *** Ryan_Lane has quit IRC | 05:23 | |
| *** PaulHend has quit IRC | 05:27 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add apache-configured devstack tempest test https://review.openstack.org/91738 | 05:31 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Enable apache-configured-service tempest for keystone https://review.openstack.org/91739 | 05:31 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add apache-configured devstack tempest test https://review.openstack.org/91738 | 05:31 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Enable apache-configured-service tempest for keystone https://review.openstack.org/91739 | 05:32 |
| *** amcrn has joined #openstack-infra | 05:37 | |
| *** ArxCruz has quit IRC | 05:37 | |
| *** PaulHend has joined #openstack-infra | 05:42 | |
| *** chandan_kumar has quit IRC | 05:47 | |
| *** yfried has quit IRC | 05:53 | |
| *** harlowja_at_home has joined #openstack-infra | 05:55 | |
| *** harlowja_at_home has quit IRC | 05:59 | |
| *** chandan_kumar has joined #openstack-infra | 06:01 | |
| *** morganfainberg is now known as morganfainberg_Z | 06:03 | |
| *** rwsu is now known as rwsu-away | 06:14 | |
| *** thomasbiege has joined #openstack-infra | 06:27 | |
| *** thomasbiege has quit IRC | 06:28 | |
| *** amcrn has quit IRC | 06:42 | |
| *** PaulHend has quit IRC | 06:42 | |
| *** PaulHend has joined #openstack-infra | 06:44 | |
| *** flaper87|afk is now known as flaper87 | 06:50 | |
| *** chandan_kumar has quit IRC | 06:51 | |
| *** e0ne has joined #openstack-infra | 06:54 | |
| *** ihrachyshka has joined #openstack-infra | 06:56 | |
| *** dstanekafk is now known as dstanekafk_zzz | 06:57 | |
| *** openstackgerrit has quit IRC | 06:57 | |
| *** ihrachyshka has quit IRC | 06:57 | |
| *** chandan_kumar has joined #openstack-infra | 07:05 | |
| *** _nadya_ has joined #openstack-infra | 07:10 | |
| *** waterkinfe has joined #openstack-infra | 07:10 | |
| *** wenlock has quit IRC | 07:11 | |
| *** arborism has joined #openstack-infra | 07:12 | |
| *** arborism is now known as amcrn | 07:12 | |
| *** jamielennox is now known as jamielennox|away | 07:14 | |
| *** ihrachyshka has joined #openstack-infra | 07:20 | |
| *** e0ne has quit IRC | 07:21 | |
| *** sweston has joined #openstack-infra | 07:30 | |
| *** jooools has joined #openstack-infra | 07:33 | |
| *** _nadya_ has quit IRC | 07:41 | |
| *** _nadya_ has joined #openstack-infra | 07:46 | |
| *** ihrachyshka has quit IRC | 07:47 | |
| *** dstanekafk_zzz is now known as dstanek | 07:48 | |
| *** nati_ueno has joined #openstack-infra | 07:55 | |
| *** jpich has joined #openstack-infra | 07:57 | |
| *** waterkinfe has quit IRC | 08:02 | |
| *** stillbert is now known as mikal | 08:03 | |
| *** dstanek is now known as dstanek_zzz | 08:03 | |
| *** ihrachyshka has joined #openstack-infra | 08:05 | |
| *** derekh_ has joined #openstack-infra | 08:06 | |
| *** Ryan_Lane has joined #openstack-infra | 08:08 | |
| *** _nadya_ has quit IRC | 08:12 | |
| *** enikanorov has joined #openstack-infra | 08:17 | |
| *** ihrachyshka has quit IRC | 08:17 | |
| *** Ryan_Lane2 has joined #openstack-infra | 08:18 | |
| *** ihrachyshka has joined #openstack-infra | 08:18 | |
| *** gnuoy` has joined #openstack-infra | 08:19 | |
| *** greghaynes_ has joined #openstack-infra | 08:20 | |
| *** gnuoy has quit IRC | 08:20 | |
| *** amotoki has quit IRC | 08:20 | |
| *** dguerri_ has joined #openstack-infra | 08:21 | |
| *** greghaynes has quit IRC | 08:21 | |
| *** cyeoh has quit IRC | 08:21 | |
| *** krotscheck has quit IRC | 08:21 | |
| *** Ryan_Lane has quit IRC | 08:21 | |
| *** tru_tru_ has joined #openstack-infra | 08:21 | |
| *** NobodyCa1 has joined #openstack-infra | 08:21 | |
| *** e0ne has joined #openstack-infra | 08:21 | |
| *** Adri2000_ has joined #openstack-infra | 08:22 | |
| *** Mithrand1r has joined #openstack-infra | 08:23 | |
| *** KurtMartin has joined #openstack-infra | 08:25 | |
| *** yaguang has joined #openstack-infra | 08:26 | |
| *** krotscheck has joined #openstack-infra | 08:26 | |
| *** e0ne has quit IRC | 08:26 | |
| *** cyeoh has joined #openstack-infra | 08:27 | |
| *** beekneemech has joined #openstack-infra | 08:27 | |
| *** nibalize1 has joined #openstack-infra | 08:29 | |
| *** dkehn_ has joined #openstack-infra | 08:29 | |
| *** blkperl_ has joined #openstack-infra | 08:30 | |
| *** kiall_ has joined #openstack-infra | 08:31 | |
| *** dhellmann_ has joined #openstack-infra | 08:32 | |
| *** bradm1 has joined #openstack-infra | 08:33 | |
| *** mrodden has quit IRC | 08:34 | |
| *** saschpe has quit IRC | 08:34 | |
| *** bknudson1 has quit IRC | 08:34 | |
| *** dkehn has quit IRC | 08:34 | |
| *** Shrews has quit IRC | 08:34 | |
| *** kiall has quit IRC | 08:34 | |
| *** bnemec has quit IRC | 08:34 | |
| *** hemna__ has quit IRC | 08:34 | |
| *** asselin__ has quit IRC | 08:34 | |
| *** d0ugal has quit IRC | 08:34 | |
| *** LarsN has quit IRC | 08:34 | |
| *** mattymo has quit IRC | 08:34 | |
| *** creiht has quit IRC | 08:34 | |
| *** nibalizer has quit IRC | 08:34 | |
| *** dhellmann has quit IRC | 08:34 | |
| *** blkperl has quit IRC | 08:34 | |
| *** flaper87 has quit IRC | 08:34 | |
| *** wayneeseguin has quit IRC | 08:34 | |
| *** isviridov has quit IRC | 08:34 | |
| *** dhellmann_ is now known as dhellmann | 08:34 | |
| *** Ryan_Lane1 has quit IRC | 08:34 | |
| *** ianw has quit IRC | 08:34 | |
| *** Adri2000 has quit IRC | 08:34 | |
| *** dguerri has quit IRC | 08:34 | |
| *** Mithrandir has quit IRC | 08:34 | |
| *** bradm has quit IRC | 08:34 | |
| *** enikanorov_ has quit IRC | 08:34 | |
| *** tru_tru has quit IRC | 08:34 | |
| *** kmartin has quit IRC | 08:34 | |
| *** NobodyCam has quit IRC | 08:34 | |
| *** asadoughi has quit IRC | 08:34 | |
| *** creiht has joined #openstack-infra | 08:34 | |
| *** wayneeseguin has joined #openstack-infra | 08:35 | |
| *** ianw has joined #openstack-infra | 08:35 | |
| *** Longgeek has joined #openstack-infra | 08:35 | |
| *** asadoughi has joined #openstack-infra | 08:35 | |
| *** saschpe has joined #openstack-infra | 08:35 | |
| *** amotoki has joined #openstack-infra | 08:36 | |
| *** LarsN has joined #openstack-infra | 08:40 | |
| *** chandan_kumar has quit IRC | 08:40 | |
| *** hemna__ has joined #openstack-infra | 08:42 | |
| *** Ryan_Lane has joined #openstack-infra | 08:44 | |
| *** bknudson has joined #openstack-infra | 08:46 | |
| *** mrodden has joined #openstack-infra | 08:46 | |
| *** hashar has joined #openstack-infra | 08:46 | |
| *** ihrachyshka has quit IRC | 08:47 | |
| *** mattymo has joined #openstack-infra | 08:49 | |
| *** PaulHend has quit IRC | 08:51 | |
| *** Shrews has joined #openstack-infra | 08:52 | |
| *** andreykurilin_ has joined #openstack-infra | 08:52 | |
| *** d0ugal has joined #openstack-infra | 08:53 | |
| *** asselin__ has joined #openstack-infra | 08:53 | |
| *** flaper87 has joined #openstack-infra | 08:53 | |
| *** isviridov has joined #openstack-infra | 08:53 | |
| *** chandan_kumar has joined #openstack-infra | 08:53 | |
| *** dstanek_zzz is now known as dstanek | 08:55 | |
| *** rwsu-away has quit IRC | 08:56 | |
| *** nati_ueno has quit IRC | 08:56 | |
| *** praneshp has quit IRC | 08:58 | |
| *** andreykurilin_ has quit IRC | 09:01 | |
| *** andreykurilin_ has joined #openstack-infra | 09:01 | |
| *** matsuhashi has quit IRC | 09:04 | |
| *** dstanek is now known as dstanek_zzz | 09:05 | |
| *** matsuhashi has joined #openstack-infra | 09:05 | |
| *** PaulHend has joined #openstack-infra | 09:06 | |
| *** Adri2000_ is now known as Adri2000 | 09:07 | |
| *** bhuvan has quit IRC | 09:14 | |
| *** e0ne has joined #openstack-infra | 09:22 | |
| *** alexpilotti has joined #openstack-infra | 09:25 | |
| *** darraghb has joined #openstack-infra | 09:27 | |
| *** e0ne has quit IRC | 09:27 | |
| *** yamahata has quit IRC | 09:30 | |
| *** Mithrand1r is now known as Mithrandir | 09:32 | |
| *** jcoufal has joined #openstack-infra | 09:39 | |
| *** dstanek_zzz is now known as dstanek | 09:48 | |
| *** nati_ueno has joined #openstack-infra | 09:57 | |
| *** dstanek is now known as dstanek_zzz | 09:58 | |
| *** mfer has joined #openstack-infra | 10:01 | |
| *** nati_ueno has quit IRC | 10:01 | |
| *** nati_ueno has joined #openstack-infra | 10:02 | |
| *** Ryan_Lane2 has quit IRC | 10:03 | |
| *** nati_uen_ has joined #openstack-infra | 10:04 | |
| *** yfried has joined #openstack-infra | 10:07 | |
| *** PaulHend has quit IRC | 10:07 | |
| *** nati_ueno has quit IRC | 10:07 | |
| *** PaulHend has joined #openstack-infra | 10:08 | |
| *** mfer has quit IRC | 10:08 | |
| *** nati_uen_ has quit IRC | 10:09 | |
| *** jcoufal has quit IRC | 10:13 | |
| *** asselin__ has quit IRC | 10:15 | |
| *** yolanda has joined #openstack-infra | 10:17 | |
| *** e0ne has joined #openstack-infra | 10:21 | |
| *** hashar has quit IRC | 10:22 | |
| *** e0ne has quit IRC | 10:22 | |
| *** dguerri_ is now known as dguerri | 10:23 | |
| *** e0ne has joined #openstack-infra | 10:23 | |
| *** jcoufal has joined #openstack-infra | 10:23 | |
| *** hashar has joined #openstack-infra | 10:24 | |
| *** e0ne has quit IRC | 10:25 | |
| *** markmc has joined #openstack-infra | 10:31 | |
| *** matsuhashi has quit IRC | 10:35 | |
| *** aysyd has joined #openstack-infra | 10:35 | |
| *** matsuhashi has joined #openstack-infra | 10:37 | |
| *** yaguang has quit IRC | 10:39 | |
| *** aysyd has quit IRC | 10:41 | |
| *** dstanek_zzz is now known as dstanek | 10:49 | |
| *** e0ne has joined #openstack-infra | 10:49 | |
| *** talluri has joined #openstack-infra | 10:50 | |
| *** jcoufal has quit IRC | 10:53 | |
| *** nati_ueno has joined #openstack-infra | 10:53 | |
| *** nati_ueno has quit IRC | 10:58 | |
| *** dstanek is now known as dstanek_zzz | 10:59 | |
| *** Ryan_Lane has quit IRC | 11:00 | |
| *** Ryan_Lane has joined #openstack-infra | 11:00 | |
| *** hashar has quit IRC | 11:02 | |
| *** mfer has joined #openstack-infra | 11:05 | |
| *** ildikov has quit IRC | 11:08 | |
| *** talluri has quit IRC | 11:10 | |
| *** talluri has joined #openstack-infra | 11:10 | |
| *** etoews has joined #openstack-infra | 11:12 | |
| *** Longgeek has quit IRC | 11:13 | |
| *** talluri has quit IRC | 11:15 | |
| *** matsuhashi has quit IRC | 11:22 | |
| *** Alexei_987 has quit IRC | 11:23 | |
| *** Alexei_987 has joined #openstack-infra | 11:25 | |
| *** _nadya_ has joined #openstack-infra | 11:27 | |
| *** talluri has joined #openstack-infra | 11:29 | |
| *** _nadya_ has quit IRC | 11:31 | |
| *** talluri has quit IRC | 11:32 | |
| *** nosnos has quit IRC | 11:37 | |
| *** jhesketh has quit IRC | 11:41 | |
| *** mburned_out is now known as mburned | 11:43 | |
| *** matsuhashi has joined #openstack-infra | 11:48 | |
| *** hashar has joined #openstack-infra | 11:48 | |
| *** dstanek_zzz is now known as dstanek | 11:50 | |
| *** nosnos has joined #openstack-infra | 11:53 | |
| *** imcsk8 has quit IRC | 11:53 | |
| *** nati_ueno has joined #openstack-infra | 11:54 | |
| *** imcsk8 has joined #openstack-infra | 11:55 | |
| *** nati_ueno has quit IRC | 11:59 | |
| *** dstanek is now known as dstanek_zzz | 12:00 | |
| *** etoews has quit IRC | 12:03 | |
| *** ArxCruz has joined #openstack-infra | 12:03 | |
| *** dstanek_zzz is now known as dstanek | 12:09 | |
| *** yaguang has joined #openstack-infra | 12:12 | |
| *** PaulHend has quit IRC | 12:12 | |
| *** mfer has quit IRC | 12:17 | |
| *** dkranz has joined #openstack-infra | 12:29 | |
| *** weshay has joined #openstack-infra | 12:29 | |
| sdague | fungi: curious issue - when looking up the history of a past patch - https://review.openstack.org/#/c/78548/ | 12:31 |
| *** pdmars has joined #openstack-infra | 12:31 | |
| sdague | We don't have a translation of the Approved votes? | 12:31 |
| *** talluri has joined #openstack-infra | 12:33 | |
| *** talluri has quit IRC | 12:37 | |
| *** nosnos has quit IRC | 12:39 | |
| *** miqui has joined #openstack-infra | 12:42 | |
| *** mbacchi has joined #openstack-infra | 12:43 | |
| *** nati_ueno has joined #openstack-infra | 12:44 | |
| *** rlandy has joined #openstack-infra | 12:45 | |
| *** PaulHend has joined #openstack-infra | 12:46 | |
| *** nati_ueno has quit IRC | 12:48 | |
| *** etoews has joined #openstack-infra | 12:49 | |
| *** dprince has joined #openstack-infra | 12:50 | |
| fungi | sdague: well, since gerrit doesn't know what that approved label is, it seems to not show that voter's votes at all | 12:50 |
| fungi | i think it's just confused by it | 12:50 |
| fungi | sdague: however, git notes to the rescue... | 12:51 |
| sdague | yep | 12:51 |
| sdague | so I can get what I need out of the comment stream | 12:52 |
| *** che-arne has joined #openstack-infra | 12:52 | |
| fungi | http://git.openstack.org/cgit/openstack/tempest/commit/?id=6a6f647 | 12:52 |
| fungi | as an alternative place to see it | 12:52 |
| sdague | however, I was curious if there was any thoughts in porting that history across | 12:52 |
| fungi | i don't think we had discussed it | 12:53 |
| sdague | ok, I might be in the minority. I sometimes go back and look at old reviews when looking at races to figure out who reviewed relevant changes to sort out if there are people I can ask questions on. | 12:54 |
| fungi | sdague: well, now that jeblair has a query for correcting timestamps on all the old reviews, perhaps it could be modified for changing the labels on approve votes | 12:55 |
| fungi | i'd have to look back at the table layout | 12:56 |
| sdague | right, that would be cool | 12:56 |
| sdague | I wonder if that's part of the reason stackalytics review counting seems to have gone insane | 12:57 |
| fungi | might be as simple as update patch_set_approvals set granted=granted category_id="Workflow" where category_id="Approved"; | 12:58 |
| fungi | er, i missed a comma in there, but you get the idea | 12:59 |
| *** matty_dubs|gone is now known as matty_dubs | 13:00 | |
| sdague | yeh, that's what I was thinking actually | 13:00 |
| fungi | that should, in theory, swap out the labels without touching the granted timestamp | 13:00 |
| fungi | unset approve and set approve maintain their 0 and 1 values correctly in this case i think | 13:00 |
| fungi | anyway, i'll let others shoot holes in that | 13:01 |
| *** alexpilotti has quit IRC | 13:03 | |
| *** matsuhashi has quit IRC | 13:04 | |
| *** dstanek is now known as dstanek_zzz | 13:04 | |
| *** matsuhashi has joined #openstack-infra | 13:04 | |
| *** markmcclain has joined #openstack-infra | 13:05 | |
| *** mfer has joined #openstack-infra | 13:06 | |
| *** timrc-afk is now known as timrc | 13:07 | |
| *** matsuhas_ has joined #openstack-infra | 13:08 | |
| *** matsuhashi has quit IRC | 13:08 | |
| *** maxbit has joined #openstack-infra | 13:08 | |
| *** mflobo has joined #openstack-infra | 13:11 | |
| *** mflobo has quit IRC | 13:11 | |
| *** ildikov has joined #openstack-infra | 13:14 | |
| *** matsuhas_ has quit IRC | 13:18 | |
| *** mriedem has joined #openstack-infra | 13:21 | |
| *** sweston has quit IRC | 13:22 | |
| *** dansmith is now known as superdan | 13:23 | |
| *** doug-fish has joined #openstack-infra | 13:23 | |
| *** dstanek_zzz is now known as dstanek | 13:28 | |
| *** Ajaeger has joined #openstack-infra | 13:29 | |
| *** alexpilotti has joined #openstack-infra | 13:30 | |
| *** thomasem has joined #openstack-infra | 13:31 | |
| *** jgrimm has joined #openstack-infra | 13:31 | |
| *** talluri has joined #openstack-infra | 13:34 | |
| *** andreykurilin_ has quit IRC | 13:36 | |
| *** talluri has quit IRC | 13:38 | |
| *** kiall_ is now known as Kiall | 13:38 | |
| *** che-arne has quit IRC | 13:42 | |
| Shrews | fungi: any luck on the bug hook bug? | 13:43 |
| fungi | Shrews: the rabbit hole went deeper | 13:43 |
| fungi | Shrews: do you happen to remember the last time you saw it successfully reassign a bug? | 13:43 |
| fungi | because the closer i look, the less i think it was working before the upgrade | 13:44 |
| Shrews | fungi: hrm, no. just sometime before the upgrade, so that timing may just be a coincidence | 13:44 |
| fungi | judging from logs, it may not have even been working before we updated passwords for heartbleed | 13:45 |
| *** nati_ueno has joined #openstack-infra | 13:45 | |
| *** zhiyan_ is now known as zhiyan | 13:45 | |
| fungi | seems to be consistently broken across all projects, but it's subtle because unless it needs to reassign the bug you won't see the breakage | 13:45 |
| *** PaulHend has quit IRC | 13:48 | |
| *** PaulHend has joined #openstack-infra | 13:48 | |
| *** nati_ueno has quit IRC | 13:50 | |
| *** lnxnut has joined #openstack-infra | 13:51 | |
| fungi | i'm about to re-test it and then start hacking at the hook script to see if i can tell where it's falling apart. either something about the lp account changed, or something about the lp api did | 13:52 |
| *** rcarrill` has joined #openstack-infra | 13:52 | |
| *** alexpilotti has quit IRC | 13:52 | |
| *** dstanek is now known as dstanek_zzz | 13:52 | |
| *** rcarrillocruz has quit IRC | 13:53 | |
| *** krtaylor has quit IRC | 13:56 | |
| *** yfried has quit IRC | 13:56 | |
| *** etoews has quit IRC | 13:57 | |
| *** _nadya_ has joined #openstack-infra | 13:58 | |
| *** smarcet has joined #openstack-infra | 13:58 | |
| *** zns has joined #openstack-infra | 13:59 | |
| *** packet has joined #openstack-infra | 14:00 | |
| *** prad_ has joined #openstack-infra | 14:00 | |
| *** jaypipes has joined #openstack-infra | 14:01 | |
| beekneemech | jeblair: \o/ | 14:02 |
| beekneemech | Thanks everyone for your help with the timestamp thing | 14:02 |
| dhellmann | has the bot stopped reporting changes to this channel? | 14:04 |
| fungi | openstackgerrit seems to be mia | 14:04 |
| fungi | i'll get it reconnected | 14:04 |
| dhellmann | thanks, fungi | 14:04 |
| dhellmann | jeblair: I've updated https://review.openstack.org/#/c/91149/ to fix the issue you pointed out | 14:04 |
| *** eharney has quit IRC | 14:06 | |
| dhellmann | jeblair: I had "new-project" as the topic for the first patchset, but it seems git-review overwrote it when I forgot to list it explicitly on the second patchset. I wonder if there's a way to cache that locally in the sandbox. | 14:06 |
| *** dstanek_zzz is now known as dstanek | 14:06 | |
| fungi | dhellmann: now in new gerrit, you should be able to just fix the topic in the webui without uploading a new patchset | 14:07 |
| fungi | though i haven't tried it myself yet | 14:07 |
| fungi | 2014-05-02 06:57:54 <-- openstackgerrit (~openstack@review.openstack.org) has quit (Ping timeout: 240 seconds) | 14:07 |
| fungi | ServerNotConnectedError: Not connected. | 14:08 |
| * fungi grumbles | 14:08 | |
| *** zz_gondoi is now known as gondoi | 14:08 | |
| dhellmann | fungi: cool, I had to make a new patchset anyway but that's good to know | 14:09 |
| *** openstackgerrit has joined #openstack-infra | 14:09 | |
| fungi | welcome back, openstackgerrit | 14:09 |
| *** _nadya_ has quit IRC | 14:10 | |
| *** talluri has joined #openstack-infra | 14:11 | |
| *** etoews has joined #openstack-infra | 14:11 | |
| *** thomasbiege has joined #openstack-infra | 14:12 | |
| *** markmcclain has quit IRC | 14:13 | |
| *** zhiyan is now known as zhiyan_ | 14:13 | |
| *** talluri has quit IRC | 14:15 | |
| *** etoews has quit IRC | 14:15 | |
| sdague | fungi: so here is an interesting thing - https://review.openstack.org/#/c/77962/ | 14:17 |
| sdague | it depends on an abandoned change | 14:17 |
| sdague | but zuul didn't reject it | 14:17 |
| *** comstud is now known as bearhands | 14:18 | |
| *** NobodyCa1 is now known as NobodyCam | 14:21 | |
| *** fifieldt has quit IRC | 14:21 | |
| *** etoews has joined #openstack-infra | 14:21 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Allow core reviewers to abandon and unabandon https://review.openstack.org/91797 | 14:23 |
| fungi | hrm... looking | 14:23 |
| *** yfried has joined #openstack-infra | 14:24 | |
| fungi | i expect gerrit is going to fail to merge it, but yeah i thought the can-merge gerrit api call zuul performs ought to catch that condition | 14:24 |
| *** homeless has joined #openstack-infra | 14:24 | |
| sdague | I had +2 / +A mostly because I assumed zuul was going to say it was unmergable and provide the -1 | 14:25 |
| sdague | apparently I guessed wrongly | 14:25 |
| *** pcrews_ has joined #openstack-infra | 14:25 | |
| fungi | i wonder if that api call changed in some way | 14:27 |
| *** salv-orlando has joined #openstack-infra | 14:27 | |
| *** salv-orlando has quit IRC | 14:27 | |
| *** salv-orlando has joined #openstack-infra | 14:28 | |
| *** david-lyle has joined #openstack-infra | 14:28 | |
| *** markmcclain has joined #openstack-infra | 14:28 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: remove pypy from python jobs https://review.openstack.org/91799 | 14:29 |
| sdague | we'll see if that ^^^ either inspires someone to take that on, or let us get those nodes back. | 14:29 |
| dhellmann | sdague: what are the "setuptools issues" you refer to there? | 14:30 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for Tempest volume test bug 1313970 https://review.openstack.org/91800 | 14:31 |
| uvirtbot | Launchpad bug 1313970 in tempest "tempest.api.volume.v2.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_details_with_multiple_params" [High,In progress] https://launchpad.net/bugs/1313970 | 14:31 |
| sdague | dhellmann: https://jenkins07.openstack.org/job/gate-python-ceilometerclient-pypy/48/console | 14:32 |
| sdague | see also, basically all -pypy jobs in the system | 14:32 |
| *** gondoi is now known as zz_gondoi | 14:32 | |
| dhellmann | sdague: weird | 14:33 |
| dhellmann | that's a pypy bug? | 14:33 |
| sdague | honestly, I have no idea what the bug is. My contribution was to make it non voting back in early march | 14:33 |
| Alex_Gaynor | dhellmann: Nope! It's a debian randomly patched stuff bug | 14:34 |
| fungi | dhellmann: i think the consensus is that it's a debian packaging bug | 14:34 |
| sdague | and try to get anyone to fix it | 14:34 |
| fungi | er, what Alex_Gaynor said | 14:34 |
| *** zns has quit IRC | 14:34 | |
| Alex_Gaynor | It's been worked around on pip-devel, not sure when it'll be in a release | 14:34 |
| sdague | so regardless, the tests are currently useless | 14:34 |
| dhellmann | patched pypy or patched setuptools? | 14:34 |
| *** zehicle_at_dell has quit IRC | 14:34 | |
| *** wenlock has joined #openstack-infra | 14:35 | |
| Alex_Gaynor | patched pypy | 14:35 |
| dhellmann | sorry to rehash history, folks :-/ | 14:35 |
| Alex_Gaynor | (I'm sure they patched setuptools as well, but that's unrelated :P) | 14:35 |
| sdague | so I think unless someone is taking ownership of making this all work again, we should just pull it out of the system | 14:35 |
| dhellmann | sdague: yeah, I'm not arguing against, just making sure I understand why I'm agreeing with you :-) | 14:35 |
| sdague | :) | 14:35 |
| dhellmann | who do we need to put pressure on to fix the problem? does debian care if we use pypy? can the pypy folks convince debian to fix it? | 14:37 |
| *** zns has joined #openstack-infra | 14:37 | |
| *** krtaylor has joined #openstack-infra | 14:37 | |
| *** markmcclain has quit IRC | 14:38 | |
| sdague | dhellmann: all of these are good questions. But this issue clearly doesn't seem to have an owner. | 14:38 |
| sdague | which, to me, means no one actually cares | 14:38 |
| sdague | which is fine, but if no one else cares, I don't think we should :) | 14:38 |
| Alex_Gaynor | dhellmann: The proper fix is with teh debian-pypy maintainer, Im reaching out to another friend whose a debian developer to see if he can fix it. Failing that, when the fix is in a pip release, just using that pip solves it /cc dstufft | 14:39 |
| *** dims is now known as swip | 14:39 | |
| sdague | Alex_Gaynor: that doesn't fix it for us | 14:39 |
| sdague | because we aren't using latest pip | 14:40 |
| *** dstanek is now known as dstanek_zzz | 14:40 | |
| sdague | so I still see this as an open loop with no solution | 14:40 |
| dhellmann | what prevents us from upgrading to the latest pip? | 14:40 |
| sdague | I don't know | 14:40 |
| dhellmann | ok | 14:40 |
| sdague | dhellmann: if you want to own this one, go for it :) | 14:41 |
| * dhellmann touches his nose | 14:41 | |
| *** jaypipes is now known as leakypipes | 14:41 | |
| sdague | this actually runs into a meta issue I've been thinking about a lot | 14:41 |
| Alex_Gaynor | I don't think it's fair to say it's an unclosed loop, people are actively working on botht he pip side, and the "yelling at debian folks" side; but neither of those sets of people are employed by "OpenStack companies" so it's not proceeding on our schedule. | 14:42 |
| fungi | i think we're probably okay with latest pip at this point. job logs seem to indicate we've been using it. the bigger issue is that we're still on old tox until the posargs thing gets straightened out, so we sometimes test with older pip if it doesn't first get upgraded inside the virtualenv | 14:42 |
| dhellmann | Alex_Gaynor: I see sdague's point about not consuming resources when we know tests won't pass; we can add pypy testing back when the problem is resolved | 14:42 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for Tempest volume test bug 1313970 https://review.openstack.org/91800 | 14:42 |
| uvirtbot | Launchpad bug 1313970 in tempest "tempest.api.volume.v2.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_details_with_multiple_params" [High,In progress] https://launchpad.net/bugs/1313970 | 14:42 |
| *** markmcclain has joined #openstack-infra | 14:43 | |
| *** zz_gondoi is now known as gondoi | 14:43 | |
| * dhellmann resists the urge to fork tox to make Better OpenStack TOX just for the bacronym | 14:44 | |
| sdague | Alex_Gaynor: that doesn't address the then infra issues as well, with it only being fixed in upstream pip. So there really isn't a full plan to making this running again in infra. | 14:44 |
| sdague | and given that it's been non running for almost 2 months, I expect it will be broken once it runs again | 14:45 |
| sdague | and so someone needs to be on the hook to address those issues as well | 14:45 |
| *** eharney has joined #openstack-infra | 14:45 | |
| BobBall | Bostox... unless you're planning to capitalise Openstack badly :P | 14:45 |
| sdague | the meta issue I've been thinking about is we've got a ton of tests, and once people have landed the tests in infra, they assume "someone else will fix them" | 14:46 |
| dhellmann | BobBall: I will go push the point for a joke :-) | 14:46 |
| sdague | so we've actually got things, like the pypy jobs, which no one feels sufficient ownership of to keep on top of issues | 14:46 |
| BobBall | *grin* | 14:46 |
| sdague | dhellmann / BobBall: clarkb already has that fork | 14:46 |
| dhellmann | nice | 14:47 |
| BobBall | hehe | 14:47 |
| sdague | cells devstack job is another instance of that | 14:47 |
| *** talluri has joined #openstack-infra | 14:47 | |
| sdague | for 6 weeks, turns out it wasn't running cells, but no one really cares about the job to look at it enough to know | 14:47 |
| *** amcrn has quit IRC | 14:47 | |
| dhellmann | sdague: in oslo we have a lead maintainer for each library; in nova they have sub-teams with leads; neutron does, too; maybe infra/qa need to designate owners for test jobs in a similar way? | 14:48 |
| sdague | dhellmann: yep, I was thinking the same thing | 14:48 |
| dhellmann | we're big enough that the small team that built the initial versions of these things can't keep up with everything on their own any more | 14:48 |
| sdague | for instance, there is a cross section of tempest jobs that I think the qa team would own, because they are considered core cross project integration. But it's not all of them | 14:49 |
| sdague | and if people aren't keeping up with categorizing race bugs on a job, for instance, then the job is automatically deprecated | 14:49 |
| sdague | after some sufficient warnings | 14:50 |
| sdague | that's actually a pretty good indication on whether anyone is ever looking at failed output | 14:50 |
| *** PaulHend has quit IRC | 14:50 | |
| *** imcsk8 has quit IRC | 14:50 | |
| dhellmann | that seems reasonable | 14:50 |
| *** imcsk8 has joined #openstack-infra | 14:51 | |
| jeblair | sdague: ++ | 14:51 |
| dhellmann | although what does removing the test job mean for the thing being tested? | 14:51 |
| dhellmann | for example, cells? | 14:51 |
| dhellmann | do we deprecate the feature because we're no longer testing it? | 14:51 |
| sdague | that's up to the team in question | 14:51 |
| dhellmann | or do we just declare it "experimental" or "untested" or something? | 14:51 |
| *** PaulHend has joined #openstack-infra | 14:52 | |
| *** etoews has quit IRC | 14:52 | |
| jeblair | anything we don't test is broken (within about 90 minutes actually, based on past experience)... so i we should just declare it "broken". ;) | 14:52 |
| fungi | ergh... i just noticed that nodepool isn't reporting node counts back to graphite again | 14:52 |
| jeblair | fungi: oh, it must have restarted! | 14:52 |
| sdague | jeblair: well, saying the current cells job tests cells, is a massive overstatement :) | 14:52 |
| fungi | that's what i'm expecting to find | 14:52 |
| fungi | rebooted 8 hours 40 minutes ago | 14:53 |
| jeblair | yup | 14:53 |
| fungi | right when the graph went on snooze | 14:53 |
| jeblair | checking tcpdump | 14:53 |
| jeblair | we should probably resize that vm to stop the bleeding; maybe this weekend | 14:53 |
| sdague | I also call this plan: stop assume jogo and sdague are going to fix your gate jobs :) | 14:53 |
| jeblair | restarting nodepool | 14:53 |
| fungi | i'm happy to take the time to grow the nodepool vm on, say, saturday | 14:54 |
| fungi | which i guess is tomorrow | 14:54 |
| *** etoews has joined #openstack-infra | 14:54 | |
| fungi | where did the week go? | 14:54 |
| jeblair | i see stats being vomited | 14:54 |
| dhellmann | sdague: I endorse the name & plan. | 14:54 |
| jeblair | on the plus side, we should be getting the new build stats now | 14:55 |
| BobBall | speaking of nodepool... it seems to be very heavily weighted to the first provider if we're allocating nodes slowly. Any thoughts of an easy fix for that? | 14:55 |
| jeblair | BobBall: it's weighted by demand | 14:55 |
| BobBall | xenserver CI might allocate just one node each pass - and that always goes to the first provider (I've got three providers that can build the same image) | 14:56 |
| jeblair | BobBall: oh, provider, not label, sorry. | 14:56 |
| BobBall | Ideally I'd like to see something like weighting by space left | 14:56 |
| BobBall | but I looked at the code and got confused whether there was something similar already let alone how I could add it ;) | 14:57 |
| *** Longgeek has joined #openstack-infra | 14:58 | |
| *** ociuhandu has quit IRC | 14:58 | |
| *** Longgeek has quit IRC | 15:00 | |
| jeblair | BobBall: there is some balancing of nodes across providers, but it's unlikely to do what you want in the 'add one node at a time' case | 15:00 |
| BobBall | yeah - I saw that it allocates the nodes based on the size of the pool - so if one is double the size of the next (which, actually, it is), it'll get double the allocation | 15:01 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Apply defaults on job templates and groups https://review.openstack.org/91807 | 15:01 |
| BobBall | the upshot of it is that 75%+ of the xenserver tests run in IAD rather than DFW or ORD :P | 15:01 |
| jeblair | BobBall: yep. but when you get down to just adding one, the math works out the same each time. | 15:02 |
| *** nati_ueno has joined #openstack-infra | 15:02 | |
| jeblair | (assuming you're adding/removing at about the same rate) | 15:02 |
| BobBall | Indeed. Thinking about it, would changing it to weight based on free space rather than total available nodes be an OK fix? | 15:03 |
| *** che-arne has joined #openstack-infra | 15:03 | |
| sdague | jeblair: did you see my question about porting the Approved votes to the new schema? | 15:03 |
| jeblair | BobBall: i think that's already how it works | 15:03 |
| BobBall | Oh... hmmm... maybe it's just because we tend to have <15 nodes that the others don't weigh in... OK | 15:04 |
| jeblair | sdague: updating approved to workflow? i think we can do that, and i think fungi's query will be all that's needed. | 15:04 |
| sdague | jeblair: yep | 15:05 |
| fungi | okay, good. i was expecting i was missing something obvious | 15:05 |
| sdague | mostly just to make it simpler in reading old reviews | 15:05 |
| sdague | cool | 15:05 |
| jeblair | i'm going tho think about that while i eat breakfast and see if any gotchas come to me | 15:05 |
| fungi | jeblair: how did you go about mass-updating that table last night? did you stop gerrit briefly while the update ran? | 15:06 |
| *** creiht has quit IRC | 15:06 | |
| openstackgerrit | John Griffith proposed a change to openstack-infra/config: Create cinder-specs git repo https://review.openstack.org/91723 | 15:07 |
| *** thedodd has joined #openstack-infra | 15:08 | |
| *** ociuhandu has joined #openstack-infra | 15:10 | |
| *** amcrn has joined #openstack-infra | 15:10 | |
| jeblair | fungi: https://etherpad.openstack.org/p/gerrit-2.8-reviewstats | 15:11 |
| jeblair | fungi: procedure starts at line 139: BETTER idea from mordred: | 15:11 |
| sdague | dhellmann: oslo.db is in the integrated gate but not in devstack? | 15:11 |
| jeblair | fungi: and no stopping, the actual update was in a single transaction | 15:12 |
| fungi | jeblair: oh, cool. i guess none of that risks being cached in memory in gerrit anyway | 15:12 |
| *** reed has joined #openstack-infra | 15:13 | |
| dhellmann | sdague: did we land something out of order? I think there's a patch to add it to devstack | 15:13 |
| *** zns has quit IRC | 15:13 | |
| dhellmann | sdague: nothing is using it yet anyway | 15:13 |
| sdague | from what I can see oslo.db is running 7 tempest jobs and it's not ever installed | 15:13 |
| sdague | so if nothing is using it, why are we co-gating on it? | 15:13 |
| dhellmann | sdague: the idea was to set up the test jobs before allowing anyone to adopt it | 15:14 |
| dhellmann | sdague: if you have time to review https://wiki.openstack.org/wiki/Oslo/CreatingANewLibrary I would love feedback about the order of steps and whether things should be split up | 15:14 |
| sdague | https://review.openstack.org/#/c/91405/ | 15:14 |
| sdague | yeh, it's -1ed by jenkins, which is why I never saw it | 15:14 |
| fungi | bknudson: i think i've fixed the stable branch wip problem you pointed out yesterday. can you confirm it works for you now? | 15:15 |
| sdague | dhellmann: so the current order is probably fine, I was just checking that people were proceeding | 15:15 |
| dhellmann | sdague: victors and his team are on holiday until the summit, so I don't expect any changes to oslo.db for a bit | 15:15 |
| bknudson | fungi: will do. | 15:15 |
| sdague | dhellmann: ok, so then I'd actually be inclined to remove the integrated gate jobs from oslo.db | 15:16 |
| sdague | because right now an oslo.db unit test fail will trigger a gate reset | 15:16 |
| bknudson | fungi: works now, was able to set my review to wip | 15:16 |
| fungi | bknudson: short explanation is that we had exclusive workflow control delegated to release managers and stable maint teams on milestone-proposed and stable/.* branches, so i added workflow -1..0 control for change owners to those sets | 15:17 |
| *** thomasbiege has quit IRC | 15:17 | |
| *** UtahDave has joined #openstack-infra | 15:17 | |
| bknudson | this seems similar to the "problem" of being able to WIP other people's changes. | 15:17 |
| *** rcarrill` has quit IRC | 15:18 | |
| fungi | well, in this case they have the ability to wip and approve changes on those branches, and both are covered by one vote label now | 15:18 |
| dhellmann | sdague: a unit test failure? | 15:19 |
| sdague | dhellmann: if gate-oslo.db-python27 fails in the gate | 15:19 |
| sdague | it will reset | 15:19 |
| sdague | http://status.openstack.org/zuul/ | 15:19 |
| sdague | notice oslo.db in change pos #4 | 15:20 |
| dhellmann | sdague: ah, I see what you mean | 15:20 |
| dhellmann | sdague: if the tests pass, though, what's the harm? | 15:21 |
| sdague | sure, but they don't always | 15:21 |
| sdague | granted, this wasn't it's fault - http://logs.openstack.org/84/91384/1/gate/gate-oslo.db-python27/4858f4b/console.html | 15:22 |
| sdague | but that was a gate reset | 15:22 |
| sdague | a couple hours ago | 15:22 |
| sdague | maybe that was a forced reset though | 15:22 |
| sdague | by something else further up | 15:22 |
| *** rcarrillocruz has joined #openstack-infra | 15:23 | |
| sdague | dhellmann: honestly, it's probably not a big deal | 15:23 |
| sdague | but just something to think about | 15:23 |
| sdague | because unit tests in parallel often have their own class of races | 15:23 |
| *** dkehn_ is now known as dkehn | 15:25 | |
| dhellmann | sdague: sure | 15:25 |
| dhellmann | this is the sort of thing I intended to discuss in the session on testing pre-releases of oslo libs, too | 15:25 |
| *** zns has joined #openstack-infra | 15:25 | |
| *** zns has quit IRC | 15:26 | |
| sdague | dhellmann: cool | 15:26 |
| *** zns has joined #openstack-infra | 15:26 | |
| sdague | dhellmann: unrelated - http://status.openstack.org/elastic-recheck/data/uncategorized.html#gate-stevedore-docs - 8 uncategorized stevedore-docs gate fails | 15:26 |
| *** jooools has quit IRC | 15:27 | |
| fungi | sdague: so... your earlier question about 77962 not failing even though its parent is abandoned... i just realized that we got faked out there ;) | 15:29 |
| sdague | ok | 15:29 |
| fungi | sdague: it started a recheck of the change because you commented on it when you hit approve | 15:29 |
| sdague | I should have zigged when I zagged? | 15:29 |
| fungi | sdague: it did *not* enqueue it to the gate when it was done with check | 15:30 |
| sdague | ok, did it leave negative feedback? | 15:30 |
| *** andreykurilin_ has joined #openstack-infra | 15:30 | |
| anteaya | mordred: in patchset 1 of your acl change patch, you had removed the create line from refs/tags/* yet included it in the merged version: https://review.openstack.org/#/c/90842/1/modules/openstack_project/files/gerrit/acls/openstack-dev/cookiecutter.config | 15:30 |
| fungi | nope, but i don't think it ever has for that case | 15:30 |
| *** marun has quit IRC | 15:30 | |
| sdague | really? | 15:30 |
| anteaya | mordred: can you help me understand your thinking? I had mistakenly thought you were correct and would like to understand a bit better | 15:30 |
| *** marun has joined #openstack-infra | 15:31 | |
| fungi | sdague: the circumstance is similar to hitting approve on any change whose parent change is not approved. it just waits for the parent to be approved (even though in this case the parent is abandoned) | 15:31 |
| sdague | hmmm.... ok | 15:31 |
| sdague | it would be nice to have zuul negative that situation | 15:31 |
| sdague | because it leaves it in a state where the system believes all is good, but it's unmergable forever | 15:32 |
| fungi | sdague: it's not necessarily unmergeable forever. if the parent change gets un-abandoned and approved without a new patchset, then that one will get enqueued along with it | 15:33 |
| sdague | sure | 15:33 |
| sdague | though it's kind of lost | 15:33 |
| sdague | because people will look at their review list | 15:33 |
| sdague | everything looks good | 15:33 |
| sdague | no negative marks | 15:33 |
| sdague | and yet.... changes can't make it in | 15:33 |
| fungi | eh... they might wonder after a while why it didn't merge even though it was approved. you'd think the change owner might get antsy about it after a while | 15:34 |
| fungi | and if you look at the summary for that change, you'll see that it has an unmerged parent (which is not all that uncommon), which in this case also just happens to be abandoned | 15:35 |
| *** nati_ueno has quit IRC | 15:35 | |
| sdague | my experience is things get lost | 15:35 |
| sdague | people assume there will be negative feedback if it requires action on their side | 15:35 |
| sdague | which isn't a terrible assumption | 15:35 |
| fungi | it may make more sense to have gerrit automatically abandon depending changes when a parent change is abandoned | 15:35 |
| jeblair | yeah, there's nothing intrinsic about the change or even its parent that would prevent it from merging, so a -1 verified isn't appropriate. if someone unabandones the change, it hasn't changed anything that zuul is testing. | 15:36 |
| *** markmcclain has quit IRC | 15:36 | |
| sdague | so how is it merge testing? | 15:36 |
| jeblair | sorry? | 15:36 |
| *** atiwari has joined #openstack-infra | 15:36 | |
| sdague | is it actually trying to rebase the abandoned + the change in question on master before running tests? | 15:36 |
| fungi | zuul still merges the entire dependent tree together and then cherry-picks it onto the target branch, regardless of whether some of those changes might be abandoned | 15:37 |
| sdague | so if the abandoned change is now in merge conflict, it will -1 the chain? | 15:37 |
| jeblair | in check it checks out the change as written, that includes the abandoned change. in gate it merges the changes as written, also including the abandoned change. | 15:37 |
| fungi | sdague: i would expect so, yes | 15:37 |
| sdague | jeblair: but it's not doing a master rebase in check? | 15:37 |
| jeblair | sdague: on gate, yes, but not on check | 15:38 |
| sdague | yeh, so I think that's part of the issue | 15:38 |
| jeblair | gimme a sec, that may be wrong | 15:38 |
| sdague | so I think if we are doing the rebase and merge conflict detection on check, we're good | 15:39 |
| sdague | because then the abandoned change could actually restore and off we go | 15:39 |
| sdague | but if not, then it's weird | 15:39 |
| fungi | that's what i think it does | 15:39 |
| jeblair | yeah, sorry i was wrong, it merges to branch tip in all cases | 15:39 |
| sdague | ok, then I withdraw my objection | 15:40 |
| fungi | there's nothing special about the abandoned parent which would cause merge conflict detection to suddenly start getting skipped on check | 15:40 |
| sdague | sure | 15:40 |
| dhellmann | sdague: https://review.openstack.org/#/c/85008/3 | 15:41 |
| fungi | and they'll never get enqueued into the gate while the parent is abandoned anyway, because zuul will check that it can submit (gerrit terminology) the change before it enqueues it, and abandoned changes can't be submitted to merge | 15:41 |
| jeblair | fungi: i have not thought of any reason not to do that change | 15:41 |
| jeblair | approved->workflow | 15:42 |
| dhellmann | sdague: I'll force that stevedore fix through today | 15:42 |
| sdague | dhellmann: great | 15:42 |
| fungi | jeblair: okay, cool beans. i'm happy to press forward with running the query after taking another mysqldump right before (for insurance) | 15:42 |
| jeblair | fungi: ++ | 15:42 |
| fungi | unless we want to wait for anyone else to chime in | 15:42 |
| sdague | docs not voting on stevedore? or were you just wedged | 15:42 |
| jeblair | i'm about to head out for the morning, i should be back in the afternoon | 15:42 |
| *** Ajaeger has quit IRC | 15:43 | |
| dhellmann | sdague: we're wedged but nothing was actually blocked, irrc -- I've been focusing on oslo.db and oslo.i18n for a couple of weeks | 15:43 |
| fungi | jeblair: awesome. i assume you're travelling this weekend, but i can pick a slow period tomorrow to do the nodepool expansion | 15:43 |
| sdague | actually, I just looked, docs isn't voting | 15:43 |
| dhellmann | sdague: that's odd | 15:43 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Creates a publishing job for the infra-manual repo https://review.openstack.org/89421 | 15:43 |
| dhellmann | sdague: oh, wait, we made it non-voting for a bit because we had to land some requirements changes to make it work | 15:44 |
| jeblair | fungi: i think i'm actually traveling on monday, so i may be in and out this wknd | 15:44 |
| dhellmann | sdague: and it's possible we need to add graphviz to the build nodes, too | 15:44 |
| sdague | all the foundation folks gathering up before summit? | 15:44 |
| fungi | dhellmann: i can check that for you | 15:44 |
| fungi | sdague: not intentionally | 15:45 |
| dhellmann | fungi: cool, thanks, I was going to run the job once and see whether it produced the diagrams | 15:45 |
| fungi | sdague: though apparently jeblair and i will both be in the same state next week, through sheer coincidence | 15:45 |
| jeblair | sdague: no, i'm visiting 'back home' since it's (relatively) near the summit | 15:45 |
| sdague | ah cool :) | 15:45 |
| ildikov | hi guys | 15:47 |
| *** packet has quit IRC | 15:47 | |
| ildikov | I need some help with a failing py26 job | 15:47 |
| fungi | dhellmann: ii graphviz 2.26.3-10ubuntu1.1 rich set of graph drawing tools | 15:47 |
| fungi | dhellmann: should work | 15:48 |
| fungi | ildikov: what seems to be the trouble with it? | 15:48 |
| ildikov | gcc fails while installing netifaces | 15:48 |
| *** gyee has joined #openstack-infra | 15:48 | |
| fungi | ildikov: oh, fun. do you have a link to the log? | 15:48 |
| ildikov | fungi: here is a console log for the issue: https://jenkins01.openstack.org/job/gate-ceilometer-python26/4806/console | 15:48 |
| dhellmann | fungi: perfect | 15:50 |
| *** yamahata has joined #openstack-infra | 15:51 | |
| fungi | ildikov: netifaces 0.10.1 was released today, looks like... https://pypi.python.org/pypi/netifaces | 15:52 |
| *** matty_dubs is now known as matty_dubs|lunch | 15:53 | |
| *** yfried has quit IRC | 15:53 | |
| *** chandan_kumar has quit IRC | 15:55 | |
| *** dangers_away is now known as dangers | 15:56 | |
| ildikov | fungi: hmm, so the problem is with this latest version | 15:57 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added task priority to edit and display of tasks https://review.openstack.org/91693 | 15:58 |
| fungi | ildikov: well, 0.10.1 looks like a brown-bag fix on top of the 0.10.0 release which happened yesterday, and both talk about changes to the gateways function which is where gcc is starting to vomit | 15:59 |
| fungi | so it might be either 0.10.0 or 0.10.1 where the bug is introduced | 15:59 |
| fungi | but your job definitely ran with 0.10.1 | 15:59 |
| ildikov | fungi: that does not look good | 16:00 |
| ildikov | fungi: yes, I saw that it is on 0.10.1 | 16:00 |
| fungi | i'll see if i can reproduce this manually on centos6 with 0.10.0 too | 16:01 |
| ildikov | fungi: thanks, that would be good if you could check | 16:02 |
| *** amcrn has quit IRC | 16:02 | |
| fungi | ildikov: yep, netifaces==0.10.0 also runs into the same compilation errors | 16:03 |
| anteaya | fungi let me know when you have a minute to give me a hand | 16:04 |
| ildikov | fungi: pffffffffff, that's not good | 16:04 |
| anteaya | I dug myself a git hole and can't seem to dig myself out | 16:04 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add .gitignore https://review.openstack.org/90188 | 16:05 |
| fungi | ildikov: so 0.8 (which is what we would have been using prior to yesterday's 0.10.0 release) installs fine, and 0.10.0 through 0.10.1 fail | 16:06 |
| fungi | ildikov: we probably want to introduce a global requirements pin on netifaces to add !0.10.0,!0.10.1 and then report the bug upstream | 16:06 |
| fungi | anteaya: i've got a git backhoe right here. what do you need dug? | 16:08 |
| anteaya | thanks https://review.openstack.org/#/c/89421/10 | 16:08 |
| anteaya | so patch 9 was where I had been and I saw the dependency merged so i thought I had better rebase | 16:08 |
| *** sdake_ has quit IRC | 16:08 | |
| ildikov | fungi: ok, so I need to propose a change to global-requirements first to fix this here, then I will report this bug | 16:08 |
| fungi | ildikov: right | 16:09 |
| anteaya | I did it wrong, I updated remote and then did git rebase master, which removed my changes | 16:09 |
| anteaya | so all I had from my patch was the commit message | 16:09 |
| ildikov | fungi: k, great, will do that | 16:09 |
| anteaya | so I created patchset 10 which has infra-manual.yaml included which was not my intent | 16:09 |
| ildikov | fungi: thanks much for your help | 16:09 |
| anteaya | blew away config in my local env and got another copy | 16:09 |
| fungi | ildikov: you're welcome! | 16:10 |
| anteaya | git review -d to pull down patchset 9 to start again | 16:10 |
| anteaya | now I can't submit the patchset and im in rebase wonderland | 16:10 |
| anteaya | any thoughts? | 16:10 |
| fungi | anteaya: oh, so you just want to recover your working tree to a sane state now? | 16:10 |
| anteaya | that would work | 16:11 |
| anteaya | or try again from patchest 10 but remove infra-manual.yaml which has never been part of my patch | 16:11 |
| fungi | run 'git status' there and put the results in a paste | 16:11 |
| fungi | so i can see what state you've got it in | 16:12 |
| anteaya | http://paste.openstack.org/show/78484/ | 16:12 |
| *** mrodden has quit IRC | 16:13 | |
| anteaya | I have no idea where the acl file got into this patch | 16:13 |
| zaro | morning | 16:15 |
| *** cp16net is now known as cramillionaire | 16:15 | |
| anteaya | morning zaro | 16:15 |
| fungi | i can't seem to get a response from paste.o.o. weird | 16:15 |
| *** marcoemorais has joined #openstack-infra | 16:15 | |
| fungi | there it finally goes | 16:15 |
| anteaya | wow | 16:15 |
| anteaya | did mordred do any trove magic on paste yesterday? | 16:16 |
| anteaya | so it appears that review.o.o is giving me timestamps but it is browser time :( | 16:17 |
| fungi | anteaya: okay, so i would do... 'git rebase --abort ; git remote update ; git reset --hard origin/master ; git review -x 89421,9 ; [edit things to resolve any merge conflicts ; git cherry-pick --continue ; git review' | 16:17 |
| anteaya | I will try that | 16:17 |
| anteaya | also paste has a lot of memory in cache: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=14&page=2 | 16:18 |
| zaro | fungi: what's that question you had for me earlier? | 16:18 |
| zaro | fungi: about exclusive permission? | 16:18 |
| fungi | anteaya: which is basically equivalent to doing... 'git rebase --abort ; git remote update ; git review -d 89421,9 ; git rebase -i origin/master ; [edit things to resolve any merge conflicts] ; git rebase --continue ; git review' | 16:18 |
| *** asselin has joined #openstack-infra | 16:19 | |
| fungi | zaro: oh, i confirmed it myself and fixed it this morning | 16:19 |
| *** freyes has joined #openstack-infra | 16:19 | |
| *** dstanek_zzz is now known as dstanek | 16:19 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: 100% test coverage https://review.openstack.org/90500 | 16:19 |
| zaro | fungi: cool | 16:20 |
| fungi | zaro: we have exclusive workflow -1..+1 for release managers and stable maintainers on some branches set via the all-projects acl, and that was preventing change owners from doing wip on changes targeting those branches | 16:20 |
| fungi | zaro: so i just added -1..0 workflow for change owners to the exclusive set for those branches | 16:21 |
| fungi | seems to have done the trick | 16:21 |
| *** hemna__ is now known as hemna | 16:21 | |
| fungi | zaro: also, in a few minutes, i'm going to update the production database to switch "approved" labels to "workflow" for old changes, so people will be able to see the old approve votes (adding granted=granted to the update query so that the granted timestamp doesn't auto-update this time) | 16:22 |
| *** aconrad has joined #openstack-infra | 16:22 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Autoassign user when state moves into progress https://review.openstack.org/91814 | 16:23 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/requirements: Exclude netifaces versions 0.10.0 & 0.10.1 https://review.openstack.org/91815 | 16:24 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/requirements: Exclude netifaces versions 0.10.0 & 0.10.1 https://review.openstack.org/91815 | 16:24 |
| *** _nadya_ has joined #openstack-infra | 16:25 | |
| *** mrodden has joined #openstack-infra | 16:25 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Creates a publishing job for the infra-manual repo https://review.openstack.org/89421 | 16:26 |
| zaro | fungi: why do people need to set old approve votes? | 16:26 |
| anteaya | fungi: thank you | 16:27 |
| *** manishg has joined #openstack-infra | 16:28 | |
| anteaya | mgagne: I think I have implimented your suggestion about the jjb variable as you intended, I am not sure that the rest of the file is accurate though: https://review.openstack.org/#/c/89421/ | 16:28 |
| mgagne | anteaya: looking | 16:28 |
| *** jpich has quit IRC | 16:28 | |
| anteaya | thanks | 16:28 |
| mgagne | anteaya: first thing, the template names are a bit too generic, I commented about it in a previous patch set | 16:29 |
| *** jogo is now known as flashgordon | 16:29 | |
| anteaya | yes, I remember that, not sure what they should be now | 16:29 |
| mgagne | anteaya: we should make it clear that those jobs are for infra manuals | 16:30 |
| anteaya | mgagne: you had suggested the use of the gate-{name} job which isn't named that anymore | 16:30 |
| * anteaya nods | 16:30 | |
| mgagne | anteaya: in fact, the 1st job, I'm sure there is already a job for that specific task | 16:30 |
| anteaya | really? | 16:31 |
| anteaya | where might I find it? | 16:31 |
| mgagne | anteaya: checking | 16:31 |
| mgagne | anteaya: only the "publish" job is infra specific | 16:31 |
| anteaya | thanks | 16:31 |
| anteaya | oh okay, good | 16:31 |
| *** zns has quit IRC | 16:31 | |
| *** yamahata has quit IRC | 16:31 | |
| *** yamahata has joined #openstack-infra | 16:31 | |
| mgagne | anteaya: 2nd job, I suggest '{name}-infra-docs' instead | 16:33 |
| *** zns has joined #openstack-infra | 16:33 | |
| anteaya | I'll use that | 16:33 |
| JayF | If my email gets bumped from the list for too many bounces, is there a way I can see the bounces and their cause? I would like to go thwap the email provider if they are bouncing emails or rate-limiting the mailing list | 16:34 |
| *** tjones has left #openstack-infra | 16:34 | |
| *** e0ne has quit IRC | 16:34 | |
| fungi | JayF: i can find the rejection messages in the mta log | 16:34 |
| mgagne | anteaya: 'gate-{name}-docs' is defined in python-jobs.yaml. it's the same job, 100% identical | 16:34 |
| anteaya | mgagne: and don't look at layout.yaml yet | 16:34 |
| JayF | fungi: if it's not too difficult, I'd love to see a sampling. jay at jvf dot cc | 16:35 |
| anteaya | mgagne: ah okay great | 16:35 |
| fungi | zaro: people don't need to set old approve votes. they want to *see* the old approve votes | 16:35 |
| anteaya | let me make those changes | 16:35 |
| *** aconrad has quit IRC | 16:36 | |
| fungi | zaro: right now, if you look at a review approved prior to the upgrade, you'll see that the table of votes doesn't display any of the votes made by the reviewer who approved the change (presumably because it doesn't know how to interpret the old approve label now) | 16:36 |
| *** hogepodge has joined #openstack-infra | 16:37 | |
| *** hashar has quit IRC | 16:38 | |
| *** _nadya_ has quit IRC | 16:38 | |
| fungi | zaro: so anyway, by replacing the "approve" label on those votes with "workflow" (in the database) should have the desired effect | 16:39 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Creates a publishing job for the infra-manual repo https://review.openstack.org/89421 | 16:40 |
| *** zns has quit IRC | 16:42 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Creates a publishing job for the infra-manual repo https://review.openstack.org/89421 | 16:42 |
| *** derekh_ has quit IRC | 16:43 | |
| *** yaguang has quit IRC | 16:43 | |
| *** _nadya_ has joined #openstack-infra | 16:44 | |
| anteaya | mgagne: now can I remove lines 1 to 16 in infra-publish-jobs.yaml? since you say that job is already defined in python-jobs.yaml? | 16:44 |
| *** zns has joined #openstack-infra | 16:44 | |
| *** gondoi is now known as zz_gondoi | 16:45 | |
| mgagne | anteaya: I think that's the way to go, this job template definition will trigger a duplication error anyway | 16:45 |
| *** lakshmiS has joined #openstack-infra | 16:45 | |
| anteaya | k | 16:46 |
| *** hashar has joined #openstack-infra | 16:47 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Creates a publishing job for the infra-manual repo https://review.openstack.org/89421 | 16:47 |
| zaro | fungi: sorry, misread as set. read makes much more sense. | 16:47 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript https://review.openstack.org/90507 | 16:47 |
| anteaya | mgagne: I think those are the big pieces in place now, any suggestions for further refinement? | 16:48 |
| mgagne | anteaya: none I can think of | 16:49 |
| *** miqui has quit IRC | 16:49 | |
| anteaya | great, thanks for your guidance building this | 16:50 |
| anteaya | I really appreciate it | 16:50 |
| mgagne | anteaya: np =) | 16:50 |
| *** amcrn has joined #openstack-infra | 16:51 | |
| *** sweston__ has joined #openstack-infra | 16:51 | |
| *** amcrn has quit IRC | 16:52 | |
| lakshmiS | how to fix issue related to gerrit review (Cannot connect to host review.openstack.org port 29418: Network is unreachable). Stopped working suddenly for everyone in the project? | 16:52 |
| *** amcrn has joined #openstack-infra | 16:53 | |
| *** matty_dubs|lunch is now known as matty_dubs | 16:54 | |
| dhellmann | sdague: have any idea about this failure? the logs say pillow isn't a global requirement, but it's in the global requirement list - http://logs.openstack.org/08/85008/3/check/check-grenade-dsvm/8c3f536/logs/grenade.sh.log.2014-05-02-155017 | 16:54 |
| dhellmann | sdague: pillow is only needed to generate the docs for stevedore, not for tests or at runtime | 16:54 |
| *** talluri has quit IRC | 16:55 | |
| *** zns has quit IRC | 16:55 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Hotfix: Story search project filter bug https://review.openstack.org/91827 | 16:55 |
| anteaya | lakshmiS: hi we upgraded gerrit on monday | 16:55 |
| *** talluri has joined #openstack-infra | 16:55 | |
| anteaya | lakshmiS: https://wiki.openstack.org/wiki/GerritUpgrade | 16:55 |
| anteaya | lakshmiS: are you trying to connect via ssh or via your browser? | 16:56 |
| lakshmiS | anteaya: via command line (git review) | 16:57 |
| anteaya | you will have to update your host key for the server | 16:57 |
| anteaya | lakshmiS: follow the instructions in this section: https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning | 16:57 |
| *** melwitt has joined #openstack-infra | 16:57 | |
| lakshmiS | thanks. will read it | 16:57 |
| anteaya | lakshmiS: also are you behind a firewall? | 16:57 |
| anteaya | if yes, the ip for review.o.o has changed | 16:58 |
| anteaya | you will need to update your firewall settings | 16:58 |
| *** bhuvan has joined #openstack-infra | 16:58 | |
| lakshmiS | anteaya: I go through a proxy, so it might have firewall as well. does the url link you provided give details on firewall settings as well? | 16:59 |
| *** talluri has quit IRC | 17:00 | |
| anteaya | lakshmiS: no it doesn't | 17:00 |
| anteaya | let me get the new ip for you | 17:00 |
| anteaya | here is the new gerrit ip and fingerprint for the new host key: New Gerrit: 23.253.232.87 2001:4800:7815:104:3bc3:d7f6:ff03:bf5d | 17:01 |
| fungi | anteaya: lakshmiS: the old ip addresses were 198.101.231.251 and 2001:4800:780d:509:3bc3:d7f6:ff04:39f0 but those have now been replaced by the new addresses 23.253.232.87 and 2001:4800:7815:104:3bc3:d7f6:ff03:bf5d | 17:01 |
| lakshmiS | thanks anteaya and fungi! | 17:01 |
| anteaya | let us know how you do | 17:01 |
| lakshmiS | sure | 17:01 |
| anteaya | thanks | 17:02 |
| fungi | lakshmiS: you need to be able to reach the new gerrit on port 29418/tcp specifically | 17:02 |
| lakshmiS | ok fungi. | 17:02 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Pass argv to main https://review.openstack.org/90542 | 17:03 |
| *** marcoemorais has quit IRC | 17:03 | |
| mgagne | zaro: ping | 17:04 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Doc fixes https://review.openstack.org/90458 | 17:04 |
| *** e0ne has joined #openstack-infra | 17:05 | |
| *** marcoemorais has joined #openstack-infra | 17:05 | |
| *** msabramo has joined #openstack-infra | 17:05 | |
| *** harlowja_away is now known as harlowja | 17:06 | |
| *** e0ne has quit IRC | 17:09 | |
| *** PaulHend has quit IRC | 17:12 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Create an initial README.rst file https://review.openstack.org/91833 | 17:13 |
| *** dims has joined #openstack-infra | 17:13 | |
| *** dims has quit IRC | 17:13 | |
| *** hashar has quit IRC | 17:14 | |
| *** swip has quit IRC | 17:15 | |
| *** yidclare has joined #openstack-infra | 17:15 | |
| *** dims has joined #openstack-infra | 17:16 | |
| *** aconrad has joined #openstack-infra | 17:16 | |
| *** e0ne has joined #openstack-infra | 17:17 | |
| *** sweston__ has quit IRC | 17:17 | |
| *** markmcclain has joined #openstack-infra | 17:18 | |
| *** e0ne has quit IRC | 17:19 | |
| *** dims has quit IRC | 17:20 | |
| *** dims has joined #openstack-infra | 17:20 | |
| ildikov | fungi: can you send me the conftest.out file that was generated while building netifaces? | 17:21 |
| ildikov | fungi: I reported the build issue, but I do not have the infrastructure locally to reproduce the failure, so I cannot attach this file that was requested by the author | 17:23 |
| *** khyati has joined #openstack-infra | 17:24 | |
| *** matty_dubs has left #openstack-infra | 17:25 | |
| zaro | mgagne: yo! | 17:25 |
| mgagne | zaro: hi! | 17:25 |
| mgagne | zaro: JJB question: how would you go to generate a bunch of publishers (the same one) based on project variables? | 17:26 |
| *** SumitNaiksatam has quit IRC | 17:26 | |
| mgagne | zaro: "for all X in variable Y, generate a publisher Z with X as its parameter" | 17:27 |
| fungi | sdague: the proposed update query seems to have worked as expected. details at the end of https://etherpad.openstack.org/p/gerrit-2.8-reviewstats but please do poke around and confirm it has solved what you were seeing | 17:27 |
| *** KurtMartin has quit IRC | 17:28 | |
| fungi | ildikov: i reproduced it by creating a virtualenv on an up-to-date centos 6.x vm and installed netifaces into it using pip | 17:28 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Create an initial README.rst file https://review.openstack.org/91833 | 17:29 |
| fungi | ildikov: i'll redo it and get the results pasted up for you | 17:29 |
| * zaro ponders | 17:29 | |
| ildikov | fungi: cool, thanks, my office notebook with Win7 is not the right env for reproducing issues like this :) | 17:30 |
| fungi | ildikov: i can understand that ;) | 17:31 |
| ildikov | fungi: :) | 17:32 |
| *** sweston__ has joined #openstack-infra | 17:33 | |
| ildikov | fungi: but I will write the details up, to be able to follow up with the bug report, when I'm back in the office and have access to some servers on Monday | 17:33 |
| *** sweston__ has quit IRC | 17:34 | |
| *** sweston has joined #openstack-infra | 17:34 | |
| *** Ryan_Lane1 has joined #openstack-infra | 17:35 | |
| *** dims has quit IRC | 17:36 | |
| annegentle | sdague: or jeblair: got a moment to review the latest programs.yaml? https://review.openstack.org/#/c/81859/ | 17:36 |
| krotscheck | I need some eyes on a prod bug: https://review.openstack.org/#/c/91827/ | 17:36 |
| *** dims has joined #openstack-infra | 17:36 | |
| annegentle | or dhellmann care to review https://review.openstack.org/#/c/81859/ ? | 17:36 |
| fungi | ildikov: hrm... i need to figure out how to tell pip not to clean up after it fails to build a module | 17:37 |
| *** morganfainberg_Z is now known as morganfainberg | 17:37 | |
| zaro | mgagne: i don't think there's a jjb feature to extrapolate list vars. | 17:37 |
| *** freyes has quit IRC | 17:38 | |
| mgagne | zaro: alright, I'll revisit my pipeline =) | 17:38 |
| *** msabramo1 has joined #openstack-infra | 17:39 | |
| *** msabramo has quit IRC | 17:39 | |
| ildikov | fungi: hmm, I do not know either :( | 17:39 |
| zaro | mgagne: would be a nice feature though. | 17:39 |
| fungi | pip install --no-clean (deprecated in 1.5.3) | 17:39 |
| ildikov | well, sounds logical, could have figured it out :) | 17:44 |
| *** msabramo1 has quit IRC | 17:44 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Create doc/source/index.rst with some initial content https://review.openstack.org/91839 | 17:44 |
| fungi | ildikov: http://paste.openstack.org/show/78499/ | 17:44 |
| zaro | mgagne: wait i'm wrong about that. | 17:45 |
| *** msabramo has joined #openstack-infra | 17:45 | |
| zaro | mgagne: will this work? http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests/yamlparser/fixtures/custom_distri.yaml | 17:45 |
| *** markmc has quit IRC | 17:46 | |
| mgagne | zaro: I think I'll revisit my way of doing things instead =) | 17:47 |
| ildikov | fungi: thanks | 17:49 |
| anteaya | Ryan_Lane1: hey, fungi mentioned you might know something about a bug in the wiki that is affecting indexing? | 17:49 |
| Ryan_Lane1 | ah | 17:49 |
| Ryan_Lane1 | hm | 17:49 |
| Ryan_Lane1 | we may need to regenerate the data in ES | 17:50 |
| Ryan_Lane1 | we should probably upgrade MW | 17:50 |
| Ryan_Lane1 | I'll do that tomorrow and deal with the ES indexes | 17:50 |
| anteaya | you are awesome | 17:50 |
| anteaya | thank you good sir | 17:50 |
| Ryan_Lane1 | yw | 17:50 |
| anteaya | :D | 17:50 |
| Ryan_Lane1 | how's things going in infra land? | 17:50 |
| anteaya | right now, no large fires | 17:51 |
| Ryan_Lane1 | looking forward to the summit? :) | 17:51 |
| anteaya | the latest gcc makes things blow up | 17:51 |
| Ryan_Lane1 | I'm thinking of using JJB for something soon :) | 17:51 |
| anteaya | that is the worst I am aware of atm | 17:51 |
| anteaya | yay | 17:51 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add apache-configured devstack tempest test https://review.openstack.org/91738 | 17:51 |
| anteaya | I am looking forward to the summit, very much so | 17:51 |
| Ryan_Lane1 | that works without needing zuul, right? it just takes yaml and turns it into xml? | 17:51 |
| anteaya | that is my understanding | 17:51 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Enable apache-configured-service tempest for keystone https://review.openstack.org/91739 | 17:51 |
| anteaya | zaro mgagne zuul isn't required to use JJB is it? | 17:52 |
| mgagne | anteaya: no | 17:52 |
| Ryan_Lane1 | \o/ | 17:52 |
| anteaya | \o/ | 17:52 |
| *** e0ne has joined #openstack-infra | 17:52 | |
| mgagne | anteaya: I don't have Zuul installed here nor did I ever tried to install it =) | 17:52 |
| Ryan_Lane1 | we're doing a thing where application owners manage all of their own infra, and I want them to own their jenkins pipelines too | 17:52 |
| anteaya | mgagne: great | 17:53 |
| anteaya | Ryan_Lane1: wise man | 17:53 |
| *** rcarrill` has joined #openstack-infra | 17:53 | |
| Ryan_Lane1 | so we're thinking of using JJB to keep the YAML in their repos, then when their repo changes it'll update their pipelines if their YAML changes | 17:53 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Tasks can now be filtered by assignee https://review.openstack.org/91840 | 17:53 |
| *** otherwiseguy has joined #openstack-infra | 17:53 | |
| anteaya | fungi: otherwiseguy is having an issue with jenkins email spam | 17:54 |
| *** rcarrillocruz has quit IRC | 17:54 | |
| *** Ryan_Lane1 has quit IRC | 17:54 | |
| *** Ryan_Lane1 has joined #openstack-infra | 17:54 | |
| otherwiseguy | fungi: 960 emails with the title "Change in openstack/neutron[master]: Adds GRE tunnel DF flag options" on https://review.openstack.org/75281 | 17:54 |
| anteaya | Ryan_Lane1: sounds like a scalable vision | 17:55 |
| Ryan_Lane1 | that's what we're hoping :) | 17:55 |
| otherwiseguy | fungi: make that 1152. In the last 30 minutes. | 17:56 |
| Ryan_Lane1 | we're also letting them manage their own config management via salt states | 17:56 |
| Ryan_Lane1 | <3 salt | 17:56 |
| anteaya | Ryan_Lane1: :D | 17:56 |
| flashgordon | is anyone fixing gate-requirements-pypy? | 17:56 |
| *** e0ne has quit IRC | 17:56 | |
| Ryan_Lane1 | I should make this work for Wikimedia Labs and give a talk at an OpenStack summit | 17:56 |
| *** praneshp has joined #openstack-infra | 17:57 | |
| anteaya | Ryan_Lane1: that would be great | 17:57 |
| *** kibad has joined #openstack-infra | 17:57 | |
| anteaya | Ryan_Lane1: you don't mean Atlanta, do you? | 17:57 |
| Ryan_Lane1 | nope. would need to be paris or the next | 17:57 |
| anteaya | flashgordon: I haven't seen that job being discussed | 17:57 |
| kibad | any documentation on complete openstack infra configuration? :) | 17:58 |
| anteaya | flashgordon: you might want to chat with sdague who is developing new policy around broken jobs | 17:58 |
| flashgordon | anteaya: thanks | 17:58 |
| flashgordon | sdague: ^ | 17:58 |
| *** kmartin has joined #openstack-infra | 17:58 | |
| anteaya | kibad: what we have is here: http://ci.openstack.org/ | 17:58 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Use pbr, pep8 checks & fixes, add "doc" tox target https://review.openstack.org/90455 | 17:58 |
| anteaya | Ryan_Lane1: paris would be good | 17:58 |
| *** rhsu has joined #openstack-infra | 17:59 | |
| fungi | anteaya: what's that about gcc blowing things up? or are you thinking of the latest netifaces release? | 17:59 |
| *** SumitNaiksatam has joined #openstack-infra | 18:00 | |
| anteaya | yes | 18:00 |
| anteaya | that was what I was thinking of | 18:00 |
| fungi | otherwiseguy: looking | 18:00 |
| anteaya | and https://review.openstack.org/#/c/75281/ isnt' just email spam | 18:00 |
| otherwiseguy | yeah, jenkins is freaking out | 18:00 |
| anteaya | it is caught in a loop | 18:00 |
| fungi | otherwiseguy: looks like we have a testing loop. i'll get it stopped | 18:00 |
| sdague | flashgordon: did you read scrollback? | 18:03 |
| notmyname | are you aware of any issue with netifaces and python 2.6? | 18:04 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Pass argv to main https://review.openstack.org/90542 | 18:04 |
| *** greghaynes_ is now known as greghaynes | 18:04 | |
| anteaya | notmyname: yes | 18:05 |
| notmyname | anteaya: any resolution yet? what's the cause? | 18:05 |
| fungi | notmyname: yes, ildikov proposed a global-reqs pin to block the new 0.10.0 and 0.10.1 releases and is opening a bug upstream | 18:05 |
| anteaya | netifaces came out with a new version | 18:05 |
| notmyname | ok | 18:05 |
| fungi | notmyname: failure is summarized in http://paste.openstack.org/show/78499/ | 18:06 |
| *** miqui has joined #openstack-infra | 18:06 | |
| notmyname | annoying since the trove info for netifaces still says py25 py26 py27 and py3 | 18:06 |
| *** marcoemorais has quit IRC | 18:06 | |
| ildikov | fungi: BTW 0.10.0 works on bare-precise-hpcloud | 18:06 |
| *** cody-somerville has quit IRC | 18:06 | |
| fungi | ildikov: yeah, we're not seeing this issue on ubuntu, just centos | 18:06 |
| notmyname | fungi: oh yes, I'm aware of that output ;-) | 18:06 |
| notmyname | ildikov: fungi: ah, interesting | 18:07 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1315468 https://review.openstack.org/91842 | 18:07 |
| uvirtbot | Launchpad bug 1315468 in openstack-ci "gate-requirements-pypy Failure: "No such file or directory: '/home/jenkins/workspace/gate-requirements-pypy/.tox/pypy/site-packages/setuptools-2.2.dist-info/METADATA"" [Undecided,New] https://launchpad.net/bugs/1315468 | 18:07 |
| notmyname | ildikov: can you check 0.10.1 too? | 18:07 |
| fungi | notmyname: basically latest netifaces releases yesterday and today can't be installed into a virtualenv on centos 6.x | 18:07 |
| ildikov | fungi: it seems to be failing only on CentOS, the only response I get so far fo rthe bug report that some kernel headers may be missing | 18:07 |
| *** marcoemorais has joined #openstack-infra | 18:08 | |
| flashgordon | sdague: I did not, there is soo much these days I don't read it | 18:08 |
| sdague | :P | 18:09 |
| ildikov | notmyname: I saw it in the gate logs, so I'm not sure about 0.10.1, but fungi said that 0.10.0 also fails on CentOS, so I assume 0.10.1 should work on Ubuntu, not sure | 18:09 |
| sdague | tl;dr - I think every test config should have a sponsor, and if issues with it aren't being dealt with in a timely manner, we just delete the configuration | 18:10 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Catch EPIPE exception and return https://review.openstack.org/90552 | 18:10 |
| anteaya | sdague: how should the test config sponsor indicate themselves? | 18:11 |
| anteaya | sdague: with a comment in the code? | 18:11 |
| *** etoews has quit IRC | 18:11 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1315468 https://review.openstack.org/91842 | 18:11 |
| uvirtbot | Launchpad bug 1315468 in openstack-ci "gate-requirements-pypy Failure: "No such file or directory: '/home/jenkins/workspace/gate-requirements-pypy/.tox/pypy/site-packages/setuptools-2.2.dist-info/METADATA"" [Undecided,New] https://launchpad.net/bugs/1315468 | 18:11 |
| lakshmiS | anteaya, fungi: It worked. Our corporate firewall needs a ticket to change the ip settings, but we were able to get around temporarily by using proxy. Thanks! | 18:12 |
| sdague | anteaya: I would actually make it another field in the yaml definition | 18:12 |
| flashgordon | sdague: I just skimmed backlog and yeah. It would be good if we can get some automation around that process (detect when job breaks etc) | 18:12 |
| sdague | it's an early idea though, I figured I'd socialize at summit | 18:12 |
| sdague | flashgordon: honestly, I'm not even interested in automation. I can point you at lots of jobs that are bad :) | 18:12 |
| sdague | I just want a decision process where we realize no one cares and we drop it | 18:13 |
| ildikov | fungi, notmyname: the netifaces limitation is proposed to openstack/requirements | 18:13 |
| flashgordon | point away | 18:13 |
| notmyname | ildikov: thanks | 18:13 |
| ildikov | fungi, notmyname: the review: https://review.openstack.org/#/c/91815/ | 18:14 |
| anteaya | lakshmiS: yay | 18:15 |
| anteaya | sdague: sounds reasonable | 18:15 |
| *** arnaud__ has joined #openstack-infra | 18:16 | |
| mordred | sdague: I like the idea - although I Think there are probably some 'central' test configs that are owned project-wide | 18:16 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Generalize fingerprint 1315468 to cover all pypy jobs https://review.openstack.org/91845 | 18:16 |
| flashgordon | sdague: you merged my patch a little too fast ^ | 18:16 |
| mordred | sdague: like, one config, for instance | 18:16 |
| sdague | mordred: agreed | 18:16 |
| ildikov | fungi, notmyname: I guess this quick fix is acceptable for now, while it turns out what to do on CentOS to make it work | 18:16 |
| mordred | sdague: and then when people want to add iterations or whatnot, they've got to sponsor them | 18:16 |
| fungi | otherwiseguy: okay, hopefully the spam has subsided. i also think i know why it happened and what recent zuul patches are suspect in creating that bug. hopefully i can work out a fix before it happens on another patch | 18:16 |
| anteaya | ildikov: jenkins is unhappy with the patch as written | 18:16 |
| *** talluri has joined #openstack-infra | 18:16 | |
| notmyname | ildikov: seems that the patch has other problems though. | 18:16 |
| sdague | that was actually in my scrollback comments. I think some set of test configs we consider core and are owned by the qa / infra teams to fix | 18:16 |
| sdague | but definitely not all of them | 18:17 |
| mordred | sdague: I'd nominate "devstack on ubuntu on python 2.7 with nova-network, rabbitmq and mysql" as 'the config' | 18:17 |
| sdague | sure | 18:17 |
| mordred | sdague: and other things, like pypy, python2.6, python3.3, postgres, centos, etc need an on-the-hook party | 18:17 |
| sdague | yep, that's actually more far reaching than I was thinking, but I'd be fine with that | 18:18 |
| flashgordon | sdague: so I like the idea, but the goal should be to never delete jobs but rather motivate owners. Which is where some automation would be nice ... for the owners/ | 18:18 |
| fungi | /opt/stack/new/devstack-gate/devstack-vm-gate.sh: line 223: [: : integer expression expected | 18:18 |
| fungi | huhwha? | 18:18 |
| fungi | http://logs.openstack.org/15/91815/2/check/check-tempest-dsvm-full/071c027/console.html | 18:18 |
| otherwiseguy | fungi: thanks | 18:18 |
| *** cody-somerville has joined #openstack-infra | 18:18 | |
| *** cody-somerville has quit IRC | 18:18 | |
| *** cody-somerville has joined #openstack-infra | 18:18 | |
| otherwiseguy | my inbox thanks you too. :) | 18:18 |
| flashgordon | such as a list of jobs and pass rate, and classification rates. and when something goes below a specific threshold the job goes red or something | 18:19 |
| sdague | this was also the: "don't assume jogo and sdague are going to fix all your gate jobs" plan | 18:19 |
| anteaya | that one had quite a few subscribers | 18:19 |
| flashgordon | sdague: hehe I am a big fan of that plan | 18:19 |
| mordred | fungi: perhaps DEVSTACK_GATE_USE_APACHE is not set? | 18:19 |
| mordred | fungi: yup | 18:19 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Doc: Use VersionInfo from pbr and remove openstack common modules https://review.openstack.org/90658 | 18:19 |
| mordred | fungi: nothing ever sets that to a default value | 18:19 |
| fungi | mordred: yeah, how did we not catch that in gating that change? | 18:20 |
| mordred | fungi: it might be set in the main jobs | 18:20 |
| mordred | fungi: one sec | 18:20 |
| fungi | mordred: it failed the same way for every devstack-gate job on a reqs change | 18:21 |
| *** darraghb has quit IRC | 18:21 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Generalize fingerprint 1315468 to cover all pypy jobs https://review.openstack.org/91845 | 18:21 |
| fungi | https://review.openstack.org/#/c/91815/2 | 18:21 |
| sdague | https://review.openstack.org/#/c/91646/ | 18:21 |
| mordred | fungi: nice | 18:21 |
| sdague | wa proposed | 18:21 |
| *** amcrn has quit IRC | 18:21 | |
| sdague | I just +Aed | 18:21 |
| sdague | guess we didn't get to it in time | 18:21 |
| fungi | oh, maybe that's not why those jobs failed then | 18:21 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Set a default value for DEVSTACK_GATE_USE_APACHE https://review.openstack.org/91850 | 18:21 |
| mordred | fungi: ^^ | 18:21 |
| sdague | mordred: see 5 lines back | 18:22 |
| mordred | sdague: dammit man | 18:22 |
| * mordred abandons his change | 18:22 | |
| fungi | okay, so that's apparently benign and merely misleading in the console log | 18:22 |
| fungi | ValueError: ('Expected version spec in', 'netifaces>=0.5,!0.10.0,!0.10.1', 'at', '!0.10.0,!0.10.1') | 18:22 |
| fungi | is the real problem | 18:22 |
| ildikov | anteaya, notmyname: yeap, I see now, will fix | 18:22 |
| anteaya | ildikov: k, thanks | 18:22 |
| sdague | fungi: yeh, !0.10.0 didn't seem right | 18:22 |
| sdague | so that's a correct fail | 18:22 |
| fungi | ahh, should be != | 18:23 |
| sdague | yep | 18:23 |
| ildikov | fungi: yeap, the first log I checked was not that informative, so it seemed to be another problem | 18:23 |
| fungi | ildikov: ^ | 18:23 |
| ildikov | fungi: yes, I see now the source of the issue | 18:23 |
| fungi | i missed that reviewing the change too | 18:23 |
| fungi | i probably mistyped it that way earlier when i was chatting with you about the patch | 18:24 |
| fungi | sorry about that! | 18:24 |
| mordred | anteaya: I have not done anything with paste and trove yet | 18:24 |
| * fungi shifts back to the other fire drill again | 18:24 | |
| mordred | fungi: what's the other fire drilll? | 18:24 |
| anteaya | mordred: okay | 18:24 |
| fungi | mordred: i haven't opened a bug to summarize yet, but we've inadvertently changed zuul's commenting behavior in unforseen ways while updating it to work with gerrit 2.8 | 18:25 |
| mordred | fungi: oh! great | 18:25 |
| mordred | fungi: and you're contending that this is undesirable | 18:25 |
| fungi | mordred: it seems that if a change is abandoned and someone comments on it, zuul reacts to that comment and tries to retest the change, then comments on the change but can't leave a vote (and we set it to now ignore the inability to vote and comment anyway) | 18:26 |
| *** andreykurilin_ has quit IRC | 18:26 | |
| fungi | and so then zuul sees that comment, notices that it hasn't voted on the change, and... | 18:26 |
| mordred | oh. god | 18:26 |
| mordred | that's amazing | 18:26 |
| fungi | at least one reviewer reported having a *very* full inbox a few minutes ago | 18:27 |
| fungi | i think part of the trick here is that the change has to get abandoned while zuul doesn't have a vote on it (so before it has a chance to report) | 18:28 |
| *** alexpilotti has joined #openstack-infra | 18:28 | |
| ildikov | fungi: no probs, the fun is that it wasn't me who proposed the patch, because of some unfortunate proxy and other networking issue on my notebook and I haven't seen other limitation lines in the gerrit window, when I checked, so I didn't realize that it's not valid syntax :S | 18:29 |
| fungi | also, i have inlaws in town and have to disappear for an early dinner soonish, so am trying to wrap my head around it first | 18:29 |
| ildikov | fungi: tl:dr Murphy likes me today :) | 18:29 |
| fungi | ildikov: welcome to the club ;) | 18:29 |
| ildikov | fungi: I'm happy to join :) | 18:30 |
| fungi | mordred: i wonder if there's a way to get zuul to notice that a change/patchset is "closed" and just not bother enqueuing it into any pipeline if so | 18:32 |
| *** isviridov has quit IRC | 18:32 | |
| fungi | or if perhaps it in fact used to do that and we somehow regressed that check | 18:32 |
| anteaya | fungi: can we force a zuul vote on the patch if it is abandoned and doesn't have one | 18:32 |
| anteaya | a default -1, patch abandoned with no vote | 18:33 |
| fungi | anteaya: not really. if a patchset is closed then noboy is able to apply a vote to it | 18:33 |
| fungi | s/noboy/nobody/ | 18:33 |
| mtreinish | clarkb, fungi: http://logs.openstack.org/03/90603/2/gate/gate-grenade-dsvm-partial-ncpu/fa44f67/logs/rabbitmq/rabbit@devstack-precise-hpcloud-region-b-3928729.txt.gz | 18:33 |
| anteaya | so that can't be added as part of the abandonment process | 18:33 |
| mtreinish | is that actually being generated by rabbit? | 18:33 |
| *** isviridov has joined #openstack-infra | 18:34 | |
| *** d0ugal has quit IRC | 18:34 | |
| mtreinish | or is a file being created with that by jenkins when it doesn't exist | 18:34 |
| *** sweston has quit IRC | 18:34 | |
| *** d0ugal has joined #openstack-infra | 18:34 | |
| *** d0ugal has quit IRC | 18:34 | |
| *** d0ugal has joined #openstack-infra | 18:34 | |
| fungi | anteaya: well, hard-coding a vote by a specific account into gerrit's abandon change button is probably not something we'd be likely to convince upstream to accept, for one thing | 18:34 |
| anteaya | k | 18:34 |
| *** sweston has joined #openstack-infra | 18:34 | |
| anteaya | getting zuul to recognize abandoned is probably a better direction | 18:34 |
| fungi | but also, i think the more correct behavior is for zuul to not run jobs on changes on which it will be unable to vote, regardless of whether it would eventually leave a vote on it | 18:35 |
| anteaya | ah can-I-vote? no? do nothing | 18:35 |
| *** etoews has joined #openstack-infra | 18:36 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add apache-configured devstack tempest test https://review.openstack.org/91738 | 18:36 |
| fungi | worst case, the change is abandoned after zuul starts testing, and so it leaves a comment with no vote once, but it won't react to its own comment at that point and reenqueue the change since it's now abandoned | 18:36 |
| anteaya | checks the comment author? | 18:37 |
| ildikov | fungi: it seems I will need more time to fix that patch as my great friend who helped to propose that patch is already spending his weekend time, so I need some time to fix my notebook or find someone, who's able to push code for review and correct the patch | 18:37 |
| ildikov | fungi: Murphy seems to be extra nice to the new club memebers :) | 18:37 |
| fungi | anteaya: that's a possibility which we might be able to encode in configuration (i'd have to ponder it a bit) but still leaves land mines for other zuul deployments who don't configure things precisely like we do | 18:38 |
| anteaya | hmmmm | 18:38 |
| fungi | ildikov: i'll update it for you | 18:38 |
| ildikov | fungi: cool, sorry for annoying you so much with this | 18:40 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add tests, make it work on Python 3 https://review.openstack.org/90114 | 18:40 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Enable apache-configured-service tempest for keystone https://review.openstack.org/91739 | 18:40 |
| anteaya | ildikov: annoy us with the fix | 18:40 |
| anteaya | saves us a lot of time in the future | 18:40 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack/requirements: Exclude netifaces versions 0.10.0 & 0.10.1 https://review.openstack.org/91815 | 18:40 |
| fungi | ildikov: ^ | 18:40 |
| *** aconrad has quit IRC | 18:40 | |
| ildikov | fungi: you're my guest for a beer in ATL, I assume you'll come | 18:41 |
| fungi | ildikov: yeah, my employer won't let me skip it even if i want to ;) | 18:41 |
| fungi | ildikov: but a handshake will suffice. i think i'll probably already be drowning in beer as it is | 18:42 |
| ildikov | fungi: LOL :) | 18:42 |
| ildikov | fungi: sure, that will be cool, I will change my hair color to blonde to fit to this today's mess I made here with this quick fix ;) | 18:44 |
| *** zns has joined #openstack-infra | 18:45 | |
| *** blkperl_ is now known as blkperl | 18:48 | |
| *** khyati has quit IRC | 18:50 | |
| anteaya | fungi: you are leaving in a bit, mordred has driven by | 18:51 |
| anteaya | fungi: when is jeblair expected back? | 18:51 |
| fungi | no idea | 18:52 |
| anteaya | k | 18:52 |
| anteaya | go when you have to, I will do my best to disemminate info | 18:52 |
| fungi | well, i'll summarize where i am with the zuul investigation before i vanish | 18:52 |
| anteaya | thanks | 18:53 |
| fungi | either in here or by opening a bug | 18:53 |
| anteaya | k | 18:53 |
| *** mriedem has quit IRC | 18:53 | |
| anteaya | don't get me wrong, I want you to stay, but I'm just saying I will do my best alone when you have to go | 18:53 |
| *** mrodden is now known as tylerdurden | 18:53 | |
| *** e0ne has joined #openstack-infra | 18:55 | |
| *** e0ne has quit IRC | 18:56 | |
| praneshp | hi fungi yt? | 18:57 |
| fungi | praneshp: what's up? | 18:57 |
| praneshp | I have a question about https://review.openstack.org/#/c/87427/ | 18:57 |
| *** aconrad has joined #openstack-infra | 18:57 | |
| praneshp | Why was my last patchset reverified? I just abandoned the change | 18:57 |
| fungi | praneshp: yeah, i think that's incorrect behavior. i'm currently digging in zuul's entrails to see if i can divine the fix | 18:58 |
| *** tylerdurden is now known as mrodden | 18:58 | |
| praneshp | oh ok, its a bug. Thanks for looking into it fungi. Good luck in zuul’s entrails | 18:58 |
| fungi | thanks | 18:59 |
| *** marcoemorais has quit IRC | 18:59 | |
| anteaya | it isn't being tested atm | 18:59 |
| fungi | yeah, i'm still unsure what the initial factors are which lead to the loop i saw earloer | 19:00 |
| *** marcoemorais has joined #openstack-infra | 19:01 | |
| anteaya | the trigger seems to be the comment left by sowmini | 19:02 |
| *** hogepodge_ has joined #openstack-infra | 19:02 | |
| anteaya | but there is a jenkins +1 showing up on the patch | 19:02 |
| *** freyes has joined #openstack-infra | 19:02 | |
| *** hogepodge has quit IRC | 19:03 | |
| *** hogepodge_ is now known as hogepodge | 19:03 | |
| *** kibad has quit IRC | 19:03 | |
| *** _nadya_ has quit IRC | 19:07 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Create doc/source/index.rst with some initial content https://review.openstack.org/91839 | 19:08 |
| mordred | fungi, jeblair: I have just learned that mysql config values for trove dbs can be managed via trove cli | 19:10 |
| mordred | fungi: for instance - to import a dump of the pastebin db, I ran trove configuration-create big-packet '{"max_allowed_packet":104857600}' | 19:10 |
| fungi | mordred: awesome! | 19:11 |
| mordred | and then did trove configation-attach of that config to the paste-mysql instance | 19:11 |
| *** mriedem has joined #openstack-infra | 19:12 | |
| *** talluri has quit IRC | 19:15 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: TripleO: use the common-venv element by default. https://review.openstack.org/91870 | 19:17 |
| dhellmann | annegentle: +1, nice! | 19:18 |
| *** otherwiseguy has quit IRC | 19:19 | |
| pabelanger | How does the WIP module get installed for gerrit? I don't seem to see it in -infra/config. Are you somehow including it withing the gerrit.war? | 19:21 |
| anteaya | hi pabelanger | 19:21 |
| anteaya | what WIP module? | 19:21 |
| anteaya | gerrit 2.8 uses a workflow status | 19:22 |
| pabelanger | pre 2.8 sorry, | 19:22 |
| anteaya | we indicate wip in 2.8 using workflow -1 | 19:22 |
| anteaya | ah | 19:22 |
| anteaya | pabelanger: zaro is probably the person you want to talk to | 19:22 |
| anteaya | I don't know the answer, fungi is tracking a zuul bug before heading to a family dinner | 19:23 |
| anteaya | and jeblair isn't hear right now | 19:23 |
| anteaya | mordred is here-ish | 19:23 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Set default for DEVSTACK_GATE_USE_APACHE https://review.openstack.org/91646 | 19:23 |
| zaro | pabelanger, anteaya : we are not using the WIP module (or plugin). | 19:24 |
| *** PaulHend has joined #openstack-infra | 19:24 | |
| anteaya | zaro: did we use it on gerrit 2.4.4? | 19:24 |
| zaro | anteaya: yes, but it wasn't available as a plugin. it was custom coded in 2.4 | 19:25 |
| *** julim_ has joined #openstack-infra | 19:25 | |
| anteaya | kk | 19:25 |
| anteaya | zaro: is it in our openstack-infra/gerrit repo? | 19:26 |
| zaro | anteaya: it's available as a gerrit plugin in ver 2.9. so if we migrate to newer version of gerrit we could use that instead of the label. | 19:26 |
| pabelanger | okay, looks like the wiki is out of date a bit: https://wiki.openstack.org/wiki/Gerrit_Workflow#Work_in_Progress | 19:26 |
| pabelanger | will check the gerrit repo and poke around | 19:26 |
| anteaya | http://git.openstack.org/cgit/openstack-infra/gerrit/?h=openstack%2F2.4.4 | 19:26 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Checkstyle publisher did not honor 0 values https://review.openstack.org/85562 | 19:26 |
| anteaya | is where I would suggest starting | 19:27 |
| anteaya | pabelanger: you are correct, I wll work on updating the wiki | 19:27 |
| zaro | anteaya: should link point to 2.8 instead? | 19:28 |
| pabelanger | zaro, so, you do a database modification for the WIP in 2.8? | 19:28 |
| pabelanger | or that can be controlled via project ACLs | 19:28 |
| *** julim has quit IRC | 19:28 | |
| zaro | pabelanger: no, db mods. it's a setting in project.config | 19:29 |
| pabelanger | cool | 19:29 |
| anteaya | zaro: whereever the wip custom code was where I was trying to point | 19:30 |
| anteaya | if you know where it is, do share | 19:30 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/zuul: Skip closed changes https://review.openstack.org/91875 | 19:32 |
| mordred | fungi: nice | 19:32 |
| fungi | still work in progress, but i'm zeroing in on the right code paths, i think | 19:32 |
| *** hashar has joined #openstack-infra | 19:32 | |
| fungi | also haven't written any zuul tests before | 19:32 |
| zaro | anteaya: it's here https://review.openstack.org/#/c/35440/ but i don't think pabelanger is interested in that. | 19:32 |
| mordred | fungi: did we ever fix the gerrit db backups? | 19:32 |
| fungi | mordred: not yet i don't think | 19:32 |
| pabelanger | Also, it looks like http://ci.openstack.org/gerrit.html is no longer getting refreshed | 19:33 |
| pabelanger | there is no mention about the new label-Workflow syntax | 19:33 |
| mordred | pabelanger: yeah - we've noticed that and haven't osrted it out yet | 19:33 |
| fungi | pabelanger: oh, i wonder whether the doc update for that hasn't merged yet | 19:33 |
| mordred | sorry bout that | 19:33 |
| mordred | fungi: docs for ci.o.o aren't publishing atm | 19:33 |
| fungi | oh! | 19:34 |
| fungi | another tidbit i missed while out | 19:34 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix gerrit database backups https://review.openstack.org/91876 | 19:34 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Migrate to using trove db for pastebin https://review.openstack.org/91877 | 19:34 |
| *** david-lyle has quit IRC | 19:34 | |
| mordred | fungi: ^^ there's a fix for the gerrit db backups, as well as a patch to migrate paste to trove | 19:34 |
| pabelanger | Ya, no problem. Was going in circles trying to understand workflow stuff | 19:34 |
| *** david-lyle has joined #openstack-infra | 19:34 | |
| pabelanger | zaro, okay, cool will check it out. however, I'll likely just schedule an upgrade to 2.8 some time in the near future | 19:35 |
| zaro | pabelanger: are you going to upgrade from upstream release or OS build of gerrit? | 19:36 |
| zaro | pabelanger: workflow label won't work with upstream version | 19:37 |
| pabelanger | zaro, not sure, what do you recommend | 19:37 |
| pabelanger | okay then | 19:37 |
| *** _nadya_ has joined #openstack-infra | 19:37 | |
| zaro | pabelanger: we applied a few patches to ACLs to allow the change owner to set workflow label in our version. | 19:37 |
| zaro | mordred: i see that spearce applied your gerrit upstream patch to 2.8.5, nice! | 19:39 |
| mordred | zaro: w00t! I'm a gerrit contributor! | 19:39 |
| zaro | pabelanger: ^ you'll want that. | 19:39 |
| jeblair | fungi: back | 19:39 |
| mordred | pabelanger: yah. otherwise your datas will be unhappy | 19:39 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/zuul: Skip closed changes https://review.openstack.org/91875 | 19:39 |
| mordred | yay it's jeblair ! | 19:39 |
| fungi | jeblair: i may be going down a rathole with https://review.openstack.org/91875 | 19:40 |
| anteaya | yay, jim's back | 19:40 |
| fungi | jeblair: one of the behaviors i'm hoping to solve with it, which i think is new since the gerrit upgrade, can be seen where comments are being rapid-fire looped on change https://review.openstack.org/75281 | 19:40 |
| fungi | jeblair: though i've seen it comment on another abandoned change too and not go into a loop... https://review.openstack.org/87427 | 19:41 |
| fungi | jeblair: so i'm uncertain as to the conditions required to cause the spamming loop behavior | 19:42 |
| fungi | jeblair: and i now need to run away to dine with inlaws, but will be back later | 19:42 |
| jeblair | fungi: thanks, i'll take over | 19:42 |
| fungi | jeblair: in good news, old approvals are visible in gerrit once more, and there were no unforseen side effects of that update query | 19:43 |
| *** aconrad has quit IRC | 19:43 | |
| jeblair | fungi: awesome thanks! | 19:43 |
| * fungi will bbkl | 19:43 | |
| fungi | bbl | 19:44 |
| anteaya | fungi: enjoy dinner | 19:44 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add tool to parse puppet reports https://review.openstack.org/91207 | 19:46 |
| *** dprince has quit IRC | 19:48 | |
| *** rcarrillocruz has joined #openstack-infra | 19:50 | |
| pabelanger | mordred, zaro okay, thanks for explaining | 19:50 |
| jeblair | fungi: ah, i think the conditions are: comment on an abondoned change where the newest jenkins approval is older than our required freshness check -- zuul will never be able to satisfy that because it can't vote on it. | 19:50 |
| *** rcarrill` has quit IRC | 19:52 | |
| *** arborism has joined #openstack-infra | 19:53 | |
| *** ociuhandu has quit IRC | 19:58 | |
| *** arborism has quit IRC | 20:00 | |
| *** thomasem has quit IRC | 20:00 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Fix "delete-all" https://review.openstack.org/91208 | 20:00 |
| *** freyes has quit IRC | 20:00 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Allow custom settings configs for maven projects https://review.openstack.org/84879 | 20:00 |
| *** zehicle_at_dell has joined #openstack-infra | 20:02 | |
| *** ramashri has joined #openstack-infra | 20:04 | |
| jeblair | fungi: i think we may want to solve it a slightly different way because there are some valid cases of commenting on closed changes (particularly if someone uses the 'change-merged' trigger) | 20:05 |
| *** aconrad has joined #openstack-infra | 20:05 | |
| anteaya | jenkins seems to like the netifaces pin patch now: https://review.openstack.org/#/c/91815/3 | 20:05 |
| ildikov | anteaya: I just wanted to ping, that finally this patch looks good | 20:06 |
| anteaya | yay | 20:06 |
| anteaya | notmyname: ^ | 20:06 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1315095 https://review.openstack.org/91886 | 20:10 |
| uvirtbot | Launchpad bug 1315095 in grenade "grenade nova network (n-net) fails to start" [Undecided,New] https://launchpad.net/bugs/1315095 | 20:10 |
| *** SumitNaiksatam has quit IRC | 20:13 | |
| openstackgerrit | A change was merged to openstack-infra/gear: geard: be more careful removing jobs https://review.openstack.org/91610 | 20:14 |
| openstackgerrit | A change was merged to openstack-infra/gear: Restructure logging https://review.openstack.org/91643 | 20:14 |
| jeblair | Tagged gear 0.5.5 | 20:15 |
| *** jamespage_ has joined #openstack-infra | 20:17 | |
| hashar | \O/ | 20:17 |
| *** jamespage_ has quit IRC | 20:17 | |
| hashar | now I need to update the Debian package hehe | 20:17 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: remove pypy from python jobs https://review.openstack.org/91799 | 20:17 |
| *** mbacchi has quit IRC | 20:18 | |
| *** e0ne has joined #openstack-infra | 20:20 | |
| *** sweston has quit IRC | 20:25 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1315095 https://review.openstack.org/91886 | 20:26 |
| uvirtbot | Launchpad bug 1315095 in grenade "grenade nova network (n-net) fails to start" [Undecided,New] https://launchpad.net/bugs/1315095 | 20:26 |
| *** PaulHend has quit IRC | 20:26 | |
| *** dims has quit IRC | 20:27 | |
| hashar | may I get some spare cycles to get python-jenkins triggers added in Zuul by any chance? Got a change that has been reasonably reviewed already https://review.openstack.org/#/c/89523/ | 20:28 |
| jeblair | hashar: you merged the tox.ini? | 20:29 |
| hashar | yes! | 20:29 |
| hashar | I wasn't sure which of the egg (tox) or chicken (zuul trigger) should be merged first | 20:29 |
| hashar | eventually got the tox one first | 20:29 |
| hashar | we are missing the doc target but there is a pending change for it | 20:30 |
| jeblair | hashar: doc target isn't needed -- the "official" way we build docs is with the venv target; does it have that? | 20:30 |
| hashar | jeblair checking | 20:32 |
| hashar | [testenv:venv] | 20:32 |
| hashar | commands = {posargs} | 20:32 |
| hashar | seems my copy paste from Zuul tox.ini is fully complete | 20:33 |
| *** ociuhandu has joined #openstack-infra | 20:33 | |
| *** harlowja is now known as harlowja_away | 20:33 | |
| jeblair | hashar: cool. also, we did add openstackci to the pypi project, so it's safe to add the release jobs | 20:34 |
| jeblair | hashar: do you want me to merge the change as-is, or do you want to push a new patchset with those other jobs added? | 20:34 |
| hashar | jeblair: I am looking at the release trigger in zuul conf right now | 20:35 |
| *** harlowja_away is now known as harlowja | 20:37 | |
| dhellmann | sdague: ping? | 20:37 |
| sdague | pong | 20:37 |
| dhellmann | sdague: I have an error from check-grenade-dsvm that doesn't make any sense, probably because I don't understand enough about what that job is testing: http://logs.openstack.org/08/85008/3/check/check-grenade-dsvm/8c3f536/logs/grenade.sh.log.2014-05-02-155017 | 20:38 |
| dhellmann | sdague: that's from me trying to add pillow as a doc requirement for stevedore | 20:38 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/config: Trigger jobs for stackforge/python-jenkins https://review.openstack.org/89523 | 20:39 |
| markmcclain | lists.o.o question… I've been starting get reports of my emails from the -dev list ending up the spam folder for some recipients. I'm fairly certain it is because we enabled stricter DMARC policies | 20:39 |
| hashar | jeblair: release: pipeline added https://review.openstack.org/#/c/89523/4/modules/openstack_project/files/zuul/layout.yaml,unified :-) | 20:39 |
| hashar | jeblair: the two python-jenkins jobs exist on jenkins07. I have no clue what the post-mirror-pythonXX jobs are doing but they seem to always be around. | 20:40 |
| *** _nadya_ has quit IRC | 20:40 | |
| jeblair | hashar: they update the openstack mirror on release. not strictly necessary but it's fine. | 20:41 |
| jeblair | hashar: did you want to add docs now? | 20:41 |
| hashar | the change that adds the doc hasn't been reviewed/tested yet | 20:41 |
| hashar | so I guess that will be for later | 20:41 |
| jeblair | k | 20:41 |
| hashar | will have also to figure out how to get them published somewhere | 20:41 |
| hashar | I am not sure where stackforge projects pushes their docs | 20:41 |
| sdague | dhellmann: you've made stevedore part of g-r sync? | 20:42 |
| dhellmann | sdague: stevedore is an oslo lib, so yeah | 20:43 |
| sdague | so it's not allowed to have requirements not in g-r | 20:43 |
| jeblair | hashar: generally rtfd, so we'll want to add a hook job for that. but we can still run the gate job to check syntax and make reviewable doc drafts | 20:43 |
| dhellmann | pillow *is* in the global requirements | 20:43 |
| sdague | in stable | 20:43 |
| sdague | ? | 20:43 |
| markmcclain | DMARC tends to break lists as is, but there are few workarounds… anyone interested discussing the alternative configs for dmarc? | 20:43 |
| anteaya | markmcclain: fungi added a new rule that should reduce that | 20:43 |
| dhellmann | sdague: is that what that test is? | 20:43 |
| notmyname | ildikov: anteaya: any follow up to the comment on the netifaces version patch? | 20:43 |
| sdague | dhellmann: it's the old side | 20:43 |
| sdague | so that would be stable | 20:44 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/config: Trigger jobs for stackforge/python-jenkins https://review.openstack.org/89523 | 20:44 |
| jeblair | hashar: oh, i see a syntax error -- no colon after the pypi-upload. since you'll have to do a new patchset anyway, want to go ahead and drop the post-mirror jobs? | 20:44 |
| dhellmann | sdague: ok, that error message could use some details :-) | 20:44 |
| hashar | jeblair: fixed. I forgot a : | 20:44 |
| anteaya | I don't know where in the logs he did that, let me see if he said so, I think it happened in the last 24 hours markmcclain | 20:44 |
| dhellmann | sdague: where would I go to add those details? | 20:44 |
| hashar | jeblair: yeah dropping the post-mirror sorry. | 20:44 |
| sdague | dhellmann: well the error message is from update.py | 20:44 |
| sdague | from global requirements, it doesn't know where it's running | 20:44 |
| dhellmann | sdague: yeah, I guess the test job needs to have its own error about which step failed? | 20:44 |
| ildikov | notmyname: I'm about to write it, but we can discuss it here too :) | 20:44 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/config: Trigger jobs for stackforge/python-jenkins https://review.openstack.org/89523 | 20:45 |
| markmcclain | anteaya: ok I didn't anything related in our mm_cfg | 20:45 |
| hashar | jeblair: sorry :-/ I am not being careful. | 20:45 |
| dhellmann | sdague: and I assume this is stable/icehouse? | 20:45 |
| *** marun has quit IRC | 20:45 | |
| notmyname | ildikov: in gerrit is good. I'm just trying to push it fast since it's blocking every change in swift :-) | 20:45 |
| ildikov | notmyname, anteaya: so netifaces 0.10.0 works fine on bare-precise-hpcloud | 20:45 |
| jeblair | hashar: is okay, we have gate jobs for that. :) | 20:45 |
| sdague | dhellmann: probably, further up in the log you should be able to see what's being checked out | 20:46 |
| dhellmann | sdague: why are we checking the requirements of the new version of the lib against the old stable list in that job? | 20:46 |
| ildikov | notmyname: Ceilo jobs are failing too :( | 20:46 |
| hashar | jeblair: exactly what I thought. "why bother validating the layout locally when Jenkins can figure it out for me" :D | 20:46 |
| sdague | dhellmann: because it's a lib that doesn't have stable | 20:46 |
| dhellmann | ah | 20:46 |
| dhellmann | sdague: so is the fix to make a stable branch of stevedore from the icehouse timeframe, or to add Pillow to the requirements under stable/icehouse? | 20:47 |
| sdague | this is definitely one of those gray areas | 20:47 |
| sdague | so the straight forward, is to backport the requirement add | 20:47 |
| * dhellmann nods | 20:47 | |
| sdague | because icehouse people are going to need that if they install latest stevedore | 20:47 |
| ildikov | notmyname: so it fails on CentOS on rax and I got a comment on the bugreport for netifaces that it fails on old Ubuntu too | 20:47 |
| dhellmann | sdague: only if they want to build the docs | 20:47 |
| dhellmann | sdague: pillow isn't a runtime requirement | 20:48 |
| *** khyati has joined #openstack-infra | 20:48 | |
| sdague | sure, we don't make that distinction today though | 20:48 |
| dhellmann | it isn't even used in the tests | 20:48 |
| dhellmann | yeah | 20:48 |
| notmyname | ildikov: ok, I think the only outstanding question is about keeping the version number <=0.8 rather than specifically excluding the last 2 releases | 20:48 |
| ildikov | notmyname: the problem is that I do not know by heart that what precise knows that the other two distros don't | 20:48 |
| dhellmann | sdague: ok, I'll backport the requirement change and maybe during K I'll have time to help make the doc/test requirements split real | 20:49 |
| anteaya | markmcclain: your mm_cfg, what is that? | 20:49 |
| notmyname | ildikov: we know that 0.8 works everywhere (cause that's what was working before). so why not keep the version number down until {netifaces,red hat, canonical} figure out how to play together | 20:49 |
| ildikov | notmyname: my additional problem is that my notebook has a halfly failed config, so personally I cannot upload the change now, but if anyone does, I'm not against limiting the version as <=0.8 | 20:50 |
| markmcclain | anteaya: mailman config | 20:50 |
| hashar | jeblair: gate is happy now https://review.openstack.org/#/c/89523/ :] | 20:51 |
| jeblair | hashar: done | 20:52 |
| *** ildikov_ has joined #openstack-infra | 20:52 | |
| hashar | jeblair: does puppet reload Zuul as well ? | 20:53 |
| anteaya | I can't find the reference, I know I saw something in the logs though | 20:53 |
| jeblair | hashar: yep, that should be fully automatic. you can check the 'last reloaded' timestamp on the status page. | 20:53 |
| anteaya | markmcclain: <JayF> fungi: ty for modifying the SPF record, I can tell when you did it because the emails started going to my inbox :) | 20:54 |
| hashar | jeblair: awesome! Thank you very much for all your help on getting python-jenkins integrated under stackforge! | 20:54 |
| jeblair | hashar: puppet runs every 15 minutes, aligned to the hour, so if it make it in by 2100 it should show up about 10 minutes after that | 20:54 |
| anteaya | http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-05-01.log timestamp 2014-05-01T18:09:49 | 20:54 |
| openstackgerrit | John Dickinson proposed a change to openstack/requirements: Exclude netifaces versions >0.8 https://review.openstack.org/91815 | 20:54 |
| jeblair | hashar: thank you! you've done more work than i have. now i just need to find time to actually upload my changes. :) | 20:54 |
| notmyname | ildikov: done ^ | 20:54 |
| anteaya | markmcclain: but I don't know what fungi modified in the SPF record | 20:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Trigger jobs for stackforge/python-jenkins https://review.openstack.org/89523 | 20:55 |
| anteaya | markmcclain: so would a change to the SPF record address your concerns? | 20:55 |
| hashar | jeblair: I already have uploaded one of your change made to add support for nodes configuration. I made you the author :-D | 20:55 |
| *** ildikov has quit IRC | 20:55 | |
| jeblair | markmcclain, anteaya, what's up? | 20:56 |
| dhellmann | sdague: https://review.openstack.org/#/c/91902/ and https://review.openstack.org/#/c/91903/ | 20:56 |
| markmcclain | jeblair: we activated a stronger corp dmarc policy | 20:56 |
| jeblair | markmcclain: oh, yeah. dmarc breaks list. | 20:56 |
| jeblair | markmcclain: i believe that effectively the consensus is that if you enable strict dmarc policies, then you do not want to receive email from mailing lists. | 20:57 |
| markmcclain | jeblair: well the issue is I get the email | 20:57 |
| markmcclain | it's that others are unknowingly either getting the messages dropped or routed to spam folder (gmail based users) | 20:58 |
| jeblair | markmcclain: eg, such entities value things like "ensuring that their customers can rely on the From field to 'authenticate' their email" is more important than having their employees participate in mailing lists... | 20:58 |
| jeblair | markmcclain: which is why generally such companies recommend to their employees that they use an alternate domain for mailing list correspondence | 20:59 |
| markmcclain | yeah understood was trying to figure out if it was worth exploring the alternative config options we have for mailman or just moving to using another domain | 21:00 |
| *** pdmars has quit IRC | 21:00 | |
| *** reed has quit IRC | 21:00 | |
| ildikov_ | notmyname: cool, thanks | 21:00 |
| *** etoews has quit IRC | 21:01 | |
| ildikov_ | notmyname: this patch will be a nice example of real team work by the end :) | 21:01 |
| *** ArxCruz has quit IRC | 21:01 | |
| jeblair | markmcclain: i don't find any of the alternatives palatable. particularly since dmarc intentionally breaks rfc compliance. this is going to be an issue over and over for domains like yours. i'd recommend an alternate domain | 21:01 |
| jeblair | markmcclain: (and/or pushback, depending on whether you agree that it's the right corporate policy :) | 21:01 |
| jeblair | markmcclain: it took me a while to understand this, but basically, using dmarc is like opting out of the public email system. it may actually make some sense for companies to do that... | 21:02 |
| markmcclain | jeblair: I definitely have my own issues with dmarc :) | 21:02 |
| *** etoews has joined #openstack-infra | 21:02 | |
| jeblair | markmcclain: if they have a particularly valuable "email brand"... | 21:02 |
| jeblair | markmcclain: but i think the rest of the internet shouldn't bend over backwards for them if they want to opt-out. | 21:03 |
| markmcclain | jeblair: yeah wish the solution we're more elegant than this | 21:04 |
| *** smarcet has quit IRC | 21:04 | |
| markmcclain | s/we're/was/ | 21:04 |
| jeblair | markmcclain: agreed :( | 21:04 |
| *** ArxCruz has joined #openstack-infra | 21:07 | |
| *** ArxCruz has quit IRC | 21:08 | |
| *** giustoXricordarl has joined #openstack-infra | 21:08 | |
| *** e0ne has quit IRC | 21:08 | |
| mordred | jeblair: everything about the above makes me sad | 21:10 |
| *** eharney has quit IRC | 21:10 | |
| anteaya | mordred: can you fill me in on the acl patch that originally removed the create line from refs/tags/* | 21:12 |
| JayF | markmcclain: jeblair: fungi and I confirmed that is /also/ causes bounces to 'hotmail' (domains.live.com -- which I think would also be all office365 cusomters) | 21:12 |
| jeblair | mordred: direct interpersonal communication unmediated through corporations or governments is apparently an aberration that is in the process of being corrected. | 21:12 |
| *** mfer has quit IRC | 21:12 | |
| *** david-lyle has quit IRC | 21:12 | |
| jeblair | mordred: that's as pithy as i could get and it still didn't fit in a tweet. | 21:12 |
| JayF | primarily because I got bounced off the list today due to some yahoo mails to the list bouncing on my domain | 21:12 |
| anteaya | jeblair: you aren't even on twitter | 21:13 |
| jeblair | anteaya: exactly | 21:13 |
| anteaya | ha ha ha | 21:13 |
| *** ArxCruz has joined #openstack-infra | 21:14 | |
| *** ArxCruz has quit IRC | 21:14 | |
| mordred | jeblair: :) | 21:15 |
| jeblair | JayF: yeah, dmarc is a really really bad idea. as it turns out, there are consequences to intentionally breaking rfc compliance. :( | 21:16 |
| JayF | jeblair: Me and fungi talked about it quite a bit in PM, have a sticky note on my monitor to file a bug, but apparently you're already on the case | 21:16 |
| mordred | anteaya: sure! so - I originally removed it as part of that patch because it's not needed just for tag uploading | 21:16 |
| jeblair | JayF: well, yeah, we looked into this months ago. i do not believe there is a viable solution other than 'do not use a mail provider that breaks email' | 21:17 |
| anteaya | mordred: it is back in now though, why is it back? | 21:17 |
| mordred | anteaya: but fungi pointed out that there are some repos in which the permission is used for itself | 21:17 |
| jeblair | JayF: that is actually the advice that dmarc-adopting domains give their users -- use a different domain for email | 21:17 |
| anteaya | people create tags and don't upload them? | 21:17 |
| mordred | so, getting tags working again was more important than cleaning up mostly harmless create permissions places | 21:17 |
| mordred | anteaya: no - the create permission is a permission to create refs at all | 21:17 |
| jeblair | JayF: (that's what eg, yahoo and twitter, say to their employees) | 21:17 |
| JayF | russell_h used to joke with us that it's 'unprofessional to signup to open source mailing lists with corporate email accounts'. Joke comes true and it ceases to be funny. | 21:17 |
| anteaya | because every acl I see now has a create line for refs/tags/* | 21:17 |
| mordred | but with the new signedTag permission, you don't need to be granted create to upload a tg | 21:18 |
| mordred | tag | 21:18 |
| jeblair | JayF: heh | 21:18 |
| jeblair | :( | 21:18 |
| mordred | you needed the create perm in 2.4 | 21:18 |
| anteaya | I had been reviewing saying this wasn't necessary, but now I feel I am wrong | 21:18 |
| mordred | it's not necessary | 21:18 |
| mordred | it's legacy | 21:18 |
| mordred | you are correct | 21:18 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: remove pypy from python jobs https://review.openstack.org/91799 | 21:18 |
| JayF | jeblair: can we ('we' == infra) do anything to prevent it from causing false bounce unsubscribes? Ban people from dmarc-using domains from posting to the list? | 21:18 |
| anteaya | so acl files for 2.8 do not need a create line for refs/tags/* | 21:18 |
| anteaya | okay thanks | 21:19 |
| mordred | anteaya: that's correct | 21:19 |
| *** cramillionaire is now known as cp16net | 21:19 | |
| *** weshay has quit IRC | 21:19 | |
| openstackgerrit | Antoine Musso proposed a change to stackforge/python-jenkins: Clarify job_exists() error messages https://review.openstack.org/89587 | 21:19 |
| anteaya | thank you | 21:20 |
| jeblair | JayF: i'm _assuming_ that would be self correcting after a little bit (presumably this affects the posters too), but if not, i would be okay with that ("you're trying to sign up from a domain that breaks email, please try a different one") | 21:20 |
| *** weshay has joined #openstack-infra | 21:20 | |
| *** PaulHend has joined #openstack-infra | 21:21 | |
| JayF | jeblair: My concern is mainly selfish; I don't want to be unsubscribed from the list again because someone elses' misconfiguration is causing my mail to bounce | 21:21 |
| jeblair | JayF: yeah, i don't want that either. please keep me up to date on what's going on. | 21:22 |
| anteaya | JayF: whoever russell_h is I agree with him | 21:22 |
| JayF | :) | 21:23 |
| *** Sukhdev has joined #openstack-infra | 21:24 | |
| *** weshay has quit IRC | 21:24 | |
| *** dims has joined #openstack-infra | 21:24 | |
| *** sdake_ has joined #openstack-infra | 21:24 | |
| *** sdake_ has quit IRC | 21:24 | |
| *** sdake_ has joined #openstack-infra | 21:24 | |
| jeblair | okay, i have to dig in and finish this zuul change | 21:25 |
| *** weshay has joined #openstack-infra | 21:27 | |
| *** annegentle has quit IRC | 21:29 | |
| *** ArxCruz has joined #openstack-infra | 21:29 | |
| *** ArxCruz has quit IRC | 21:29 | |
| *** jooools has joined #openstack-infra | 21:32 | |
| *** aconrad has quit IRC | 21:32 | |
| *** d0ugal has quit IRC | 21:33 | |
| *** flaper87 is now known as flaper87|afk | 21:34 | |
| *** sweston has joined #openstack-infra | 21:35 | |
| zaro | SergeyLukjanov: no approve on this? https://review.openstack.org/#/c/91192/ | 21:38 |
| *** otherwiseguy has joined #openstack-infra | 21:39 | |
| *** ArxCruz has joined #openstack-infra | 21:44 | |
| *** ArxCruz has quit IRC | 21:44 | |
| *** weshay has quit IRC | 21:45 | |
| *** miqui has quit IRC | 21:45 | |
| *** aconrad has joined #openstack-infra | 21:45 | |
| *** mriedem has quit IRC | 21:48 | |
| *** hashar has quit IRC | 21:49 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add pipeline requirements https://review.openstack.org/91911 | 21:50 |
| *** dims has quit IRC | 21:51 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add pipeline requirements https://review.openstack.org/91911 | 21:52 |
| jeblair | fungi: ^ alternate solution (partly based on yours) that still allows reporting on closed changes if you want | 21:53 |
| *** SumitNaiksatam has joined #openstack-infra | 21:53 | |
| jeblair | sdague: ^ that should also let us close the logic hole we found in require-approval | 21:53 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Stories can now be filtered by assignee_id https://review.openstack.org/91912 | 21:53 |
| *** Ryan_Lane1 has quit IRC | 21:53 | |
| *** Ryan_Lane1 has joined #openstack-infra | 21:54 | |
| jeblair | i'm going to restart zuul; there will be a few minutes of downtime while i move some log files around | 21:55 |
| *** lcheng_ has joined #openstack-infra | 21:57 | |
| jeblair | #status notice Zuul is being restarted with some dependency upgrades and configuration changes; ETA 2215 | 21:58 |
| openstackstatus | jeblair: sending notice | 21:58 |
| -openstackstatus- NOTICE: Zuul is being restarted with some dependency upgrades and configuration changes; ETA 2215 | 21:59 | |
| *** ArxCruz has joined #openstack-infra | 21:59 | |
| *** ArxCruz has quit IRC | 21:59 | |
| openstackstatus | jeblair: finished sending notice | 22:00 |
| *** UtahDave has quit IRC | 22:01 | |
| *** marun has joined #openstack-infra | 22:01 | |
| mordred | jeblair: you are entirely too fast at writing zuul changes | 22:03 |
| jeblair | mordred: ;) | 22:03 |
| jeblair | okay, zuul is restarted and i have re-enqueued changes | 22:05 |
| jeblair | mordred, fungi: i moved zuul logs to /opt/log/zuul and bind-mounted that to /var/log/zuul (and added it to fstab) | 22:06 |
| mordred | jeblair: kk | 22:06 |
| jeblair | i used a bind mount because puppet wants that to be a directory (it would have removed the symlink) | 22:06 |
| jeblair | we are now logging quite a bit of gear data, so we'll want to keep an eye on usage in /opt | 22:07 |
| *** packet has joined #openstack-infra | 22:08 | |
| *** timrc has quit IRC | 22:08 | |
| jeblair | i don't believe the new version of nodepool was installed, so i'm installing it now and will restart it. | 22:09 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Fix typo in launch stats https://review.openstack.org/91914 | 22:10 |
| *** ArxCruz has joined #openstack-infra | 22:14 | |
| *** ArxCruz has quit IRC | 22:14 | |
| mordred | jeblair: +@ | 22:16 |
| mordred | jeblair: +2 | 22:16 |
| *** prad_ has quit IRC | 22:18 | |
| jeblair | mordred: i went ahead and applied that manually and restarted nodepool; it's reporting per-provider results now | 22:18 |
| jeblair | mordred: the latest changes may break your log parsing script, btw | 22:18 |
| jeblair | mordred: but we'll get more detailed, greppable, reasons for _why_ nodes have failed | 22:18 |
| jeblair | mordred: and the counts will go to graphite | 22:19 |
| *** prad has joined #openstack-infra | 22:19 | |
| *** sdake_ has quit IRC | 22:20 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix gerrit database backups https://review.openstack.org/91876 | 22:21 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Migrate to using trove db for pastebin https://review.openstack.org/91877 | 22:21 |
| *** PaulHend has quit IRC | 22:22 | |
| mordred | jeblair: I'm thrilled with all of those things | 22:23 |
| *** prad has quit IRC | 22:23 | |
| mordred | jeblair: btw - I don't know if you caught it in the scrollback - but I used the trove command line client to make a config change in the paste-mysql instance | 22:23 |
| mordred | and I did an import of the current paste db into the trove db and that's all successful - so I think we're set to move that whenver | 22:24 |
| jeblair | mordred: oh cool, should we do that right now? | 22:24 |
| *** timrc has joined #openstack-infra | 22:25 | |
| *** melwitt has quit IRC | 22:25 | |
| mordred | jeblair: might as well. I figure if paste stops working for 2 minutes it probably will not kill anyone | 22:25 |
| *** PaulHend has joined #openstack-infra | 22:25 | |
| jeblair | mordred: i agree. we can probably move it in the downtime between hitting a url and getting a response. ;) | 22:26 |
| jeblair | mordred: aprvd | 22:26 |
| *** ianw has quit IRC | 22:26 | |
| *** ianw has joined #openstack-infra | 22:27 | |
| *** ramashri has quit IRC | 22:27 | |
| *** jooools has quit IRC | 22:29 | |
| *** ArxCruz has joined #openstack-infra | 22:29 | |
| *** ArxCruz has quit IRC | 22:29 | |
| *** doug-fish has left #openstack-infra | 22:31 | |
| *** rlandy has quit IRC | 22:32 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Migrate to using trove db for pastebin https://review.openstack.org/91877 | 22:32 |
| *** lnxnut has quit IRC | 22:33 | |
| *** otherwiseguy has quit IRC | 22:34 | |
| anteaya | mordred jeblair had approved that already | 22:34 |
| openstackgerrit | A change was merged to stackforge/python-jenkins: Doc fixes https://review.openstack.org/90458 | 22:37 |
| *** homeless has quit IRC | 22:38 | |
| openstackgerrit | A change was merged to stackforge/python-jenkins: Clarify job_exists() error messages https://review.openstack.org/89587 | 22:40 |
| *** manishg has quit IRC | 22:42 | |
| *** lcheng_ has quit IRC | 22:44 | |
| *** ArxCruz has joined #openstack-infra | 22:44 | |
| *** ArxCruz has quit IRC | 22:44 | |
| flashgordon | is there anyway to record if a job is voting or not for logStash? | 22:45 |
| *** thedodd has quit IRC | 22:45 | |
| harlowja | hey, qq, now that there next ubuntu LTS is out, any idea when we can use it (and use its newer dependencies?) | 22:49 |
| *** leakypipes has quit IRC | 22:49 | |
| *** msabramo has quit IRC | 22:56 | |
| *** manishg has joined #openstack-infra | 22:58 | |
| *** ArxCruz has joined #openstack-infra | 22:59 | |
| * fungi is back | 22:59 | |
| *** ArxCruz has quit IRC | 22:59 | |
| fungi | mordred: jeblair: paste regularly stops working for two minutes already, so i'm not sure anyone will be the wiser | 23:00 |
| fungi | they'll just be surprised when it suddenly becomes responsive with some regularity | 23:00 |
| fungi | oh, and jeblair already made that joke. damn | 23:01 |
| *** HenryG has quit IRC | 23:01 | |
| *** aconrad has quit IRC | 23:03 | |
| sdague | harlowja: probably mid to end of summer, earlier if more people help with the transition. | 23:03 |
| harlowja | cool | 23:03 |
| *** msabramo has joined #openstack-infra | 23:04 | |
| *** sweston has quit IRC | 23:05 | |
| dstufft | does openstack-infra/config go through gerrit | 23:05 |
| dstufft | I want to change https://github.com/openstack-infra/config/blob/master/install_puppet.sh#L20 | 23:05 |
| mordred | dstufft: yup. goes through the puppet | 23:05 |
| mordred | dstufft: what want you to change? | 23:05 |
| dstufft | mordred: bootstrap.pypa.io is the new cannonical location | 23:06 |
| sdague | dstufft: all things go through gerrit :) | 23:06 |
| dstufft | specifically https://bootstrap.pypa.io/get-pip.py | 23:06 |
| mordred | dstufft: we JUST changed that :) | 23:06 |
| dstufft | oh | 23:06 |
| mordred | kk. will change again | 23:06 |
| *** hemna is now known as hemnafk | 23:06 | |
| sdague | dstufft: you also want to submit a change to devstack for that? | 23:06 |
| dstufft | also if you reference ez_setup.py anywhere you'll want to use bootstrap.pypa.io too | 23:06 |
| dstufft | sdague: probably, I guess I'll have to figure out where to do that too, assumingly it's a gerrit repo too :] | 23:07 |
| sdague | https://github.com/openstack-dev/devstack | 23:07 |
| sdague | and yes, gerrit | 23:07 |
| sdague | git review is your friend :) | 23:07 |
| dstufft | okies | 23:08 |
| dstufft | i'll see about doing that once I cut a new pip release | 23:08 |
| dstufft | should fix y'all's PyPy problem | 23:08 |
| *** sandywalsh has quit IRC | 23:08 | |
| anteaya | mordred: jenkins is unhappy: https://review.openstack.org/#/c/91877/ | 23:09 |
| *** manishg has quit IRC | 23:09 | |
| fungi | well, once someone first takes care of our tox problem, so that we get newer virtualenv with new pip | 23:09 |
| fungi | which in turn works around debubtu's broken pypy | 23:09 |
| dstufft | well I can't fix all the things | 23:09 |
| dstufft | :] | 23:09 |
| fungi | dstufft: you already fix plenty | 23:10 |
| dstufft | at least people referencing ez_setup.py from bitbucket won't get rate limited anymore with bootstrap.pypa.io :| | 23:10 |
| sdague | anteaya: that's the DNS fail issue | 23:10 |
| *** sandywalsh has joined #openstack-infra | 23:10 | |
| sdague | http://logs.openstack.org/77/91877/3/check/gate-config-layout/5305b81/console.html#_2014-05-02_22_56_29_887 | 23:10 |
| fungi | that's an interesting one, because it's not happening in rackspace | 23:11 |
| fungi | but it is in hpcloud az2, which has been worse than the rest of hp where random network and disk i/o failures are concerned | 23:12 |
| jeblair | huh | 23:13 |
| *** HenryG has joined #openstack-infra | 23:14 | |
| *** ArxCruz has joined #openstack-infra | 23:14 | |
| *** ArxCruz has quit IRC | 23:14 | |
| jeblair | i've seen 2 more dns failures in az2 | 23:16 |
| *** aconrad has joined #openstack-infra | 23:18 | |
| jeblair | picked a new az2 ready node and dns seems to be working there | 23:18 |
| jeblair | jenkins06 is responding _very_ slowly; i'm going to put it in shutdown mode | 23:19 |
| jeblair | if i can log in | 23:19 |
| anteaya | does the log say what jenkins a job ran on? | 23:20 |
| fungi | anteaya: sort of. it's embedded in the slave url | 23:21 |
| jeblair | anteaya: if you mouse-over the hostname, you can see it | 23:21 |
| jeblair | "Building remotely on ..." | 23:21 |
| anteaya | got it jenkins 01 for the failed job on the paste trove patch | 23:22 |
| jeblair | anteaya: my mention of jenkins06 is not related to any job failures | 23:22 |
| jeblair | just to be clear | 23:22 |
| *** mrodden has quit IRC | 23:22 | |
| fungi | jenkins seems to just get crufty if it runs for a while without a restart. in good news, we haven't needed to restart jenkins masters for a while apparently (i'm guessing not since the heartbleed cleanup?) | 23:23 |
| jeblair | yeah | 23:23 |
| anteaya | jeblair: I see that now, but it did motivate me to ask my question | 23:24 |
| *** Sukhdev has quit IRC | 23:24 | |
| jeblair | i don't think this will happen in a timely manner; i'm going to just stop it. | 23:24 |
| openstackgerrit | A change was merged to openstack-infra/config: Migrate to using trove db for pastebin https://review.openstack.org/91877 | 23:24 |
| mordred | yay | 23:24 |
| mordred | jeblair: do you think I should noticebot the paste downtime? | 23:25 |
| anteaya | ha ha ha | 23:25 |
| jeblair | mordred: sure, maybe just a notice. couldn't hurt | 23:25 |
| mordred | #status notice paste.openstack.org is going down for a short database upgrade | 23:25 |
| openstackstatus | mordred: sending notice | 23:25 |
| *** PaulHend has quit IRC | 23:26 | |
| -openstackstatus- NOTICE: paste.openstack.org is going down for a short database upgrade | 23:26 | |
| openstackstatus | mordred: finished sending notice | 23:27 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: 100% test coverage https://review.openstack.org/90500 | 23:28 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add .gitignore https://review.openstack.org/90188 | 23:28 |
| jerryz_ | fungi: what is the procedure to create a new branch for long live development or experiment ? | 23:29 |
| *** ArxCruz has joined #openstack-infra | 23:29 | |
| *** ArxCruz has quit IRC | 23:29 | |
| fungi | jerryz_: we have a feature branch workflow we've used in a few projects, described at https://wiki.openstack.org/wiki/GerritJenkinsGit#Merge_Commits | 23:32 |
| *** dims has joined #openstack-infra | 23:32 | |
| *** SumitNaiksatam has quit IRC | 23:33 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix a very strange dyslexia https://review.openstack.org/91922 | 23:34 |
| mordred | jeblair, fungi ^^ | 23:34 |
| jeblair | haha | 23:34 |
| mordred | paste running in prod - had to chage that one thing :) | 23:34 |
| fungi | teehee | 23:35 |
| mordred | anteaya: paste should be snappier now | 23:35 |
| *** msabramo1 has joined #openstack-infra | 23:35 | |
| jeblair | mordred: yaay! | 23:36 |
| mordred | also, and more importantly, that's one more server that should be puppet clean now | 23:37 |
| anteaya | yes | 23:37 |
| anteaya | woohoo | 23:37 |
| anteaya | yay | 23:38 |
| *** msabramo has quit IRC | 23:38 | |
| fungi | this puppet is clean (apologies to zelda rubinstein) | 23:38 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix a very strange dyslexia https://review.openstack.org/91922 | 23:38 |
| anteaya | that reference must be very witty | 23:38 |
| anteaya | I have no connection | 23:39 |
| fungi | paraphrasing a film quote, sorry | 23:39 |
| fungi | from poltergeist | 23:39 |
| *** zns has quit IRC | 23:39 | |
| jeblair | jenkins06 is back up | 23:40 |
| anteaya | ah never saw the poltergeist | 23:40 |
| anteaya | I don't work with malevolents | 23:40 |
| anteaya | running away to the motel for some sleep | 23:41 |
| anteaya | probably be monday before I am online again | 23:41 |
| anteaya | have a good weekend | 23:41 |
| fungi | you too, as much as you can manage | 23:42 |
| jeblair | anteaya: ++ | 23:42 |
| fungi | jeblair: your zuul patch is great. should patch up that hole and some others once we update our configuration to match | 23:42 |
| jeblair | fungi: yay, glad you like it! | 23:43 |
| fungi | and new flexibility in the bargain | 23:43 |
| fungi | plus backwards compatibility that my broken change completely neglected | 23:43 |
| fungi | i agree i'd completely spaced on people triggering jobs post-merge. which is how most of the world actually does ci | 23:44 |
| *** dims has quit IRC | 23:44 | |
| fungi | well, actually, *most* of the world does ci by waiting on their users to complain they broke something, but anyway | 23:44 |
| jerryz_ | fungi: thanks for the info. i have a question.. what's the interval or circumstance to do a merge of master into feature branch? would rebase be better (without creating merge commits) | 23:44 |
| *** ArxCruz has joined #openstack-infra | 23:44 | |
| *** ArxCruz has quit IRC | 23:44 | |
| jeblair | at some point i'd actually like to get the merge commit into the event stream so we can sensibly do some post-commit reports | 23:45 |
| mordred | jeblair: what is it that's broken about that now? | 23:45 |
| fungi | jerryz_: generally you would do your potentially breaking work which you don't want hitting mainline on the feature branch instead, then periodically merge from mainline to your feature branch to keep it fresh so you minimize surprises, then when ready merge the feature branch back into mainline and delete the feature branch itself | 23:46 |
| jeblair | mordred: it's the reason we do ref-updated instead of change-merged -- change-merged doesn't tell you the sha of the commit that was generated if gerrit makes a merge commit | 23:46 |
| mordred | right. that was it | 23:46 |
| mordred | yeah - we should get that into the event stream | 23:46 |
| jeblair | somewhere i have a 90% patch for that | 23:47 |
| fungi | jerryz_: generally gerrit has enough of that sort of workflow already that you shouldn't often need separate long-lived feature branches, but sometimes if you've got a feature which is going to take a while to implement in multiple steps and you can't do that safely in mainline for some reason, then there's at least an alternative | 23:47 |
| *** khyati has quit IRC | 23:47 | |
| *** alexpilotti has quit IRC | 23:47 | |
| fungi | jerryz_: but that could also imply that there's something broken with your development model if you have to resort to it a lot | 23:48 |
| fungi | jeblair: is that still the case, or have they maybe fixed the event details while we were languishing on old gerrit still? | 23:49 |
| mordred | generally speaking, feature branches are the opposite of CI | 23:49 |
| *** flashgordon is now known as jogo | 23:49 | |
| fungi | jerryz_: ^ i agree with mordred | 23:49 |
| jerryz_ | fungi: yeah. i understand. since the the project is pretty new and also grows fast. something need to be refactored eg. api2.0 | 23:50 |
| mordred | they're a tool you use when you don't have CI - and they're a good way to mitigate the pain of integrating large changed to defined integration times | 23:50 |
| mordred | jerryz_: nod | 23:50 |
| jeblair | fungi: not sure | 23:50 |
| *** bhuvan has quit IRC | 23:51 | |
| mordred | k. puppet agent is enabled again on paste | 23:52 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add tests, make it work on Python 3 https://review.openstack.org/90114 | 23:53 |
| jerryz_ | fungi: who is going to be in $PROJECT-milestone group. do i need to apply for a group or just do a patch adding the privilege to $PROJECT-ptl group | 23:54 |
| fungi | jerryz_: using your -ptl group for that should probably be fine | 23:55 |
| *** dims has joined #openstack-infra | 23:55 | |
| *** dims has quit IRC | 23:55 | |
| *** dangers is now known as dangers_away | 23:56 | |
| *** dims has joined #openstack-infra | 23:56 | |
| jerryz_ | fungi: will zuul trigger jobs on the feature branch ? | 23:58 |
| fungi | jerryz_: yes, unless configured not to do so in the jobs list | 23:59 |
| *** msabramo1 has quit IRC | 23:59 | |
| *** ArxCruz has joined #openstack-infra | 23:59 | |
| *** ArxCruz has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!