| anteaya | oh I love that idea | 00:00 |
|---|---|---|
| *** salv-orlando has joined #openstack-infra | 00:00 | |
| anteaya | it is a great idea | 00:00 |
| anteaya | it is the reality I find that sometimes comes up short | 00:00 |
| pleia2 | clarkb: so, arch with CI? :) | 00:00 |
| clarkb | pleia2: bsaically | 00:01 |
| *** tsg has joined #openstack-infra | 00:01 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: dhcp-all-interfaces depends on dhcp https://review.openstack.org/296626 | 00:04 |
| ianw | greghaynes / prometheanfire : ^ i think that keeps everyone happy | 00:04 |
| *** salv-orlando has quit IRC | 00:06 | |
| *** fawadkhaliq has quit IRC | 00:07 | |
| *** fawadkhaliq has joined #openstack-infra | 00:07 | |
| prometheanfire | ianw: we do have an isc dhcp client, but I think I'd rather use dhcpcd as it's simpler | 00:07 |
| prometheanfire | I don't need that script or a custom init script for instance :P | 00:07 |
| bkero | heh dhcpcd | 00:08 |
| *** gyee has quit IRC | 00:08 | |
| ianw | right, it maps isc-dhcp-client -> dhcpcd right? | 00:08 |
| prometheanfire | bkero: wat :P | 00:08 |
| prometheanfire | ianw: nope :P | 00:08 |
| prometheanfire | net-misc/dhcp - Description: ISC Dynamic Host Configuration Protocol (DHCP) client/server | 00:08 |
| bkero | dhcpcd is a different dhcp client implementation | 00:08 |
| ianw | prometheanfire: i mean the package map does that | 00:08 |
| prometheanfire | dhcpcd - Description: A fully featured, yet light weight RFC2131 compliant DHCP client | 00:08 |
| bkero | prometheanfire: Oh nothing. Just giggling at the gentoo reference | 00:08 |
| prometheanfire | oh, sure | 00:08 |
| ianw | maybe we should call it something else | 00:08 |
| prometheanfire | bkero: iirc, it's a bsd thing :P | 00:09 |
| prometheanfire | http://roy.marples.name/projects/dhcpcd/index | 00:09 |
| *** sflanigan has joined #openstack-infra | 00:10 | |
| *** sflanigan has joined #openstack-infra | 00:10 | |
| bkero | Funny, I don't think it's the default in freebsd | 00:10 |
| prometheanfire | ianw: that script doesn't work for us, so I'm happy keeping the mapping to dhcpcd, all the workaround you guys do for ubuntu is funny :P | 00:10 |
| prometheanfire | bkero: he refrences netbsd, dunno | 00:11 |
| *** ayoung has joined #openstack-infra | 00:11 | |
| *** aeng has quit IRC | 00:11 | |
| prometheanfire | ianw: I guess the question is weather or not we want to diverge gentoo to have dhcpcd instead of what everyone else has. | 00:12 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Use generic "dhcp-client" name https://review.openstack.org/300256 | 00:14 |
| *** mylu has joined #openstack-infra | 00:14 | |
| ianw | ok, maybe that makes thing clearer. l like the idea we just specify "dhcp-client" and let each platform figure it out | 00:14 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add experimental ubuntu-xenial puppet-apply job https://review.openstack.org/300257 | 00:14 |
| *** mylu has quit IRC | 00:15 | |
| prometheanfire | ianw: worksforme | 00:16 |
| prometheanfire | ianw: did your edit go through | 00:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add ubuntu-xenial to openstack_project::single_use_slave https://review.openstack.org/300258 | 00:17 |
| ianw | prometheanfire: which edit? | 00:17 |
| prometheanfire | oh, that was another review | 00:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add experimental bindep-fallback job for ubuntu-xenial https://review.openstack.org/300259 | 00:18 |
| *** MarkAtwood has quit IRC | 00:20 | |
| *** jaypipes has quit IRC | 00:21 | |
| *** hichihara has joined #openstack-infra | 00:21 | |
| *** mtanino has quit IRC | 00:21 | |
| pabelanger | clarkb: ubuntu-xenial uploads happing now | 00:23 |
| pabelanger | have a few patches up to enable experimental jobs for puppet-apply and bindep | 00:23 |
| pabelanger | we shouldn't move anything to check / gate until ubuntu xenial is stable release | 00:23 |
| *** aeng has joined #openstack-infra | 00:24 | |
| prometheanfire | pabelanger: are gate checks moving to 16.04 already? | 00:24 |
| pabelanger | prometheanfire: we're just adding the ubuntu-xenial now, how projects use it is another story | 00:25 |
| prometheanfire | ah | 00:25 |
| clarkb | pabelanger: ya I wouldnt switch voting jobs until post release | 00:25 |
| prometheanfire | thought the default was switch and freaking out :P | 00:25 |
| clarkb | just because another openjdk can happen | 00:25 |
| pabelanger | honestly, I don't know the migration plan for the projects. I'm just making sure our infra works on xenial | 00:25 |
| pabelanger | clarkb: ++ | 00:25 |
| pabelanger | prometheanfire: no, ubuntu-trusty will be around for a while yet | 00:25 |
| clarkb | pabelanger: prometheanfire we do someehat forcefullu switch the base jobs | 00:26 |
| pabelanger | we have stable branches to support | 00:26 |
| clarkb | so unittests and devstack will all likely move sometime in may | 00:26 |
| clarkb | and yes mitaka will run on trusty | 00:26 |
| clarkb | we have so many jobs now though... I dont intend on personally proofing each one | 00:28 |
| pabelanger | indeed | 00:29 |
| pabelanger | and ubuntu-xenial on bluebox | 00:30 |
| clarkb | pabelanger an experimental set of jobs for grenade, devstack and devstack gate would probably be good | 00:30 |
| pabelanger | Ya, I've had people ask about that already | 00:30 |
| pabelanger | I want to confirm https://review.openstack.org/#/c/300257 and https://review.openstack.org/#/c/300259 work first. So jenkins can attach, then setup the devstack experimental job | 00:31 |
| openstackgerrit | Fei Long Wang proposed openstack/requirements: Add tempest to requirements https://review.openstack.org/300263 | 00:31 |
| pabelanger | then let people hack | 00:31 |
| clarkb | prometheanfire: and fwiw the switch last time was pretty smooth | 00:32 |
| pabelanger | maybe give fedora-24 a try in the morning! cc ianw | 00:33 |
| clarkb | this time might be harder due to systemd | 00:33 |
| prometheanfire | ya | 00:33 |
| prometheanfire | we do have a systemd profile though :D | 00:33 |
| *** Qiming has quit IRC | 00:34 | |
| *** tiswanso has quit IRC | 00:39 | |
| *** mylu has joined #openstack-infra | 00:39 | |
| *** tiswanso has joined #openstack-infra | 00:39 | |
| *** mylu has quit IRC | 00:41 | |
| *** tqtran has quit IRC | 00:41 | |
| *** mylu has joined #openstack-infra | 00:42 | |
| *** _amrith_ is now known as amrith | 00:42 | |
| *** mylu has quit IRC | 00:44 | |
| *** thorst has joined #openstack-infra | 00:44 | |
| *** mylu has joined #openstack-infra | 00:44 | |
| *** tsg has quit IRC | 00:49 | |
| *** moravec has quit IRC | 00:50 | |
| *** jerryz has quit IRC | 00:50 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Add fix_disk_layout script and builder https://review.openstack.org/300122 | 00:50 |
| *** mylu has quit IRC | 00:50 | |
| *** tsg has joined #openstack-infra | 00:51 | |
| *** thorst has quit IRC | 00:51 | |
| *** mylu has joined #openstack-infra | 00:52 | |
| *** markvoelker_ has quit IRC | 00:53 | |
| *** Daisy has joined #openstack-infra | 00:54 | |
| *** edmondsw has joined #openstack-infra | 00:54 | |
| *** tsg has quit IRC | 00:56 | |
| *** gildub has joined #openstack-infra | 00:56 | |
| *** Daisy_ has joined #openstack-infra | 00:56 | |
| *** Daisy__ has joined #openstack-infra | 00:58 | |
| *** Jeffrey4l has joined #openstack-infra | 01:00 | |
| *** baoli has joined #openstack-infra | 01:00 | |
| *** Swami has quit IRC | 01:00 | |
| *** aeng has quit IRC | 01:00 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for infra traceroute6 bug 1564688 https://review.openstack.org/300270 | 01:00 |
| openstack | bug 1564688 in OpenStack-Gate "/tmp/hudson5902744734221751235.sh: traceroute6: not found" [Undecided,New] https://launchpad.net/bugs/1564688 | 01:00 |
| *** Daisy has quit IRC | 01:00 | |
| *** esikachev has joined #openstack-infra | 01:02 | |
| *** Daisy_ has quit IRC | 01:02 | |
| *** tqtran has joined #openstack-infra | 01:03 | |
| *** mylu has quit IRC | 01:03 | |
| *** mylu has joined #openstack-infra | 01:04 | |
| *** esikachev has quit IRC | 01:06 | |
| *** tqtran has quit IRC | 01:07 | |
| *** mylu has quit IRC | 01:09 | |
| *** pvaneck has quit IRC | 01:10 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for infra traceroute6 bug 1564688 https://review.openstack.org/300270 | 01:11 |
| openstack | bug 1564688 in OpenStack-Gate "/tmp/hudson5902744734221751235.sh: traceroute6: not found" [Undecided,Fix released] https://launchpad.net/bugs/1564688 - Assigned to Jeremy Stanley (fungi) | 01:11 |
| *** angdraug has quit IRC | 01:14 | |
| *** mylu has joined #openstack-infra | 01:15 | |
| *** Daisy__ has quit IRC | 01:16 | |
| *** thorst has joined #openstack-infra | 01:17 | |
| *** Jeffrey4l has quit IRC | 01:17 | |
| *** asettle has quit IRC | 01:19 | |
| *** tiswanso has quit IRC | 01:20 | |
| *** tiswanso has joined #openstack-infra | 01:21 | |
| *** mylu has quit IRC | 01:22 | |
| *** thorst has quit IRC | 01:23 | |
| *** mylu has joined #openstack-infra | 01:24 | |
| *** knikolla has quit IRC | 01:25 | |
| *** woodster_ has quit IRC | 01:27 | |
| *** mtanino has joined #openstack-infra | 01:28 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add Non-ATC Recognition WG Meeting https://review.openstack.org/300278 | 01:28 |
| *** edmondsw has quit IRC | 01:30 | |
| *** Qiming has joined #openstack-infra | 01:31 | |
| *** krtaylor has joined #openstack-infra | 01:34 | |
| *** fawadkhaliq has quit IRC | 01:40 | |
| *** thorst has joined #openstack-infra | 01:42 | |
| *** moravec has joined #openstack-infra | 01:43 | |
| *** thorst has quit IRC | 01:43 | |
| *** zz_dimtruck is now known as dimtruck | 01:44 | |
| *** apoorvad has quit IRC | 01:44 | |
| *** mriedem1 has joined #openstack-infra | 01:45 | |
| *** mriedem has quit IRC | 01:47 | |
| openstackgerrit | Fei Long Wang proposed openstack/requirements: Add tempest to requirements https://review.openstack.org/300263 | 01:48 |
| *** yamahata has quit IRC | 01:49 | |
| *** mylu has quit IRC | 01:50 | |
| *** fawadkhaliq has joined #openstack-infra | 01:54 | |
| *** dimtruck is now known as zz_dimtruck | 01:55 | |
| *** asettle has joined #openstack-infra | 01:55 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:56 | |
| fungi | what, so jenkins07 still wasn't fixed even after rerunning jjb on it again? | 01:56 |
| fungi | we seem to have very spotty puppet updates going on | 01:56 |
| clarkb | it claimed to be up to date but didnt really update until I flushed cache | 01:57 |
| fungi | got it. so we still have cache weirdness sounds like | 01:57 |
| *** baoli has quit IRC | 01:58 | |
| *** baoli has joined #openstack-infra | 01:58 | |
| fungi | from now on i'll double-check a job on every single master i guess, not just spot check a few of them | 01:58 |
| *** zz_dimtruck is now known as dimtruck | 01:59 | |
| *** Sukhdev has quit IRC | 01:59 | |
| *** asettle has quit IRC | 02:01 | |
| *** tqtran has joined #openstack-infra | 02:04 | |
| *** bpokorny has quit IRC | 02:06 | |
| *** shashank_hegde has quit IRC | 02:07 | |
| *** tqtran has quit IRC | 02:08 | |
| *** tphummel_ has quit IRC | 02:10 | |
| *** achanda has joined #openstack-infra | 02:14 | |
| *** mylu has joined #openstack-infra | 02:19 | |
| *** asettle has joined #openstack-infra | 02:23 | |
| *** akshai_ has quit IRC | 02:23 | |
| *** yamamoto has joined #openstack-infra | 02:24 | |
| *** baoli has quit IRC | 02:25 | |
| *** amitgandhinz has joined #openstack-infra | 02:27 | |
| *** EricGonc_ has quit IRC | 02:32 | |
| *** fawadkhaliq has quit IRC | 02:33 | |
| *** mriedem1 has quit IRC | 02:38 | |
| *** asettle has quit IRC | 02:39 | |
| *** aeng has joined #openstack-infra | 02:41 | |
| *** fawadkhaliq has joined #openstack-infra | 02:42 | |
| *** baoli has joined #openstack-infra | 02:42 | |
| *** amitgandhinz has quit IRC | 02:43 | |
| *** fawadkhaliq has quit IRC | 02:43 | |
| *** asettle has joined #openstack-infra | 02:45 | |
| *** baoli has quit IRC | 02:45 | |
| *** fawadkhaliq has joined #openstack-infra | 02:45 | |
| *** asettle has quit IRC | 02:50 | |
| *** shashank_hegde has joined #openstack-infra | 02:50 | |
| *** yuanying has quit IRC | 02:51 | |
| *** sflanigan is now known as sflanigan_afk | 02:52 | |
| *** fawadkhaliq has quit IRC | 02:53 | |
| *** fawadkhaliq has joined #openstack-infra | 02:53 | |
| *** yamamoto has quit IRC | 02:54 | |
| *** flepied has quit IRC | 02:54 | |
| *** esikachev has joined #openstack-infra | 03:03 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Change download command to point to git.o.o https://review.openstack.org/300187 | 03:04 |
| *** sflanigan_afk is now known as sflanigan | 03:07 | |
| *** esker has joined #openstack-infra | 03:07 | |
| *** esikachev has quit IRC | 03:08 | |
| *** mylu has quit IRC | 03:08 | |
| *** fawadkhaliq has quit IRC | 03:08 | |
| *** bpokorny has joined #openstack-infra | 03:09 | |
| *** mylu has joined #openstack-infra | 03:09 | |
| *** fawadkhaliq has joined #openstack-infra | 03:10 | |
| *** mylu has quit IRC | 03:11 | |
| *** mylu has joined #openstack-infra | 03:11 | |
| *** fawadkhaliq has quit IRC | 03:12 | |
| *** krotscheck is now known as krotscheck_vaca | 03:20 | |
| *** fawadkhaliq has joined #openstack-infra | 03:32 | |
| *** amitgandhinz has joined #openstack-infra | 03:34 | |
| *** fedexo has joined #openstack-infra | 03:38 | |
| *** links has joined #openstack-infra | 03:41 | |
| *** yamamoto has joined #openstack-infra | 03:43 | |
| *** markvoelker has joined #openstack-infra | 03:47 | |
| *** yuanying has joined #openstack-infra | 03:49 | |
| *** mylu has quit IRC | 03:54 | |
| *** markvoelker_ has joined #openstack-infra | 03:56 | |
| *** markvoelker has quit IRC | 03:56 | |
| *** kushal has quit IRC | 04:01 | |
| *** amitgandhinz has quit IRC | 04:01 | |
| *** achanda has quit IRC | 04:04 | |
| *** bpokorny has quit IRC | 04:06 | |
| *** achanda has joined #openstack-infra | 04:08 | |
| *** mylu has joined #openstack-infra | 04:10 | |
| *** mylu has quit IRC | 04:10 | |
| *** mylu has joined #openstack-infra | 04:13 | |
| *** achanda has quit IRC | 04:13 | |
| *** kushal has joined #openstack-infra | 04:14 | |
| *** asselin__ has joined #openstack-infra | 04:17 | |
| *** asettle has joined #openstack-infra | 04:17 | |
| *** asselin has quit IRC | 04:18 | |
| *** harlowja_at_home has joined #openstack-infra | 04:19 | |
| *** asettle has quit IRC | 04:21 | |
| *** Sukhdev has joined #openstack-infra | 04:25 | |
| *** fawadkhaliq has quit IRC | 04:26 | |
| *** sdake has joined #openstack-infra | 04:29 | |
| *** markvoelker_ has quit IRC | 04:33 | |
| *** markvoelker has joined #openstack-infra | 04:33 | |
| *** dimtruck is now known as zz_dimtruck | 04:34 | |
| *** markvoelker has quit IRC | 04:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Move periodic-package-afs-centos7 to centos-7 https://review.openstack.org/300126 | 04:39 |
| *** mylu has quit IRC | 04:41 | |
| *** mtreinish has quit IRC | 04:44 | |
| *** kengo_sakai has joined #openstack-infra | 04:44 | |
| *** mylu has joined #openstack-infra | 04:46 | |
| *** shashank_hegde has quit IRC | 04:51 | |
| *** mylu has quit IRC | 04:57 | |
| *** rcernin has joined #openstack-infra | 04:57 | |
| *** fawadkhaliq has joined #openstack-infra | 04:57 | |
| *** achanda has joined #openstack-infra | 05:02 | |
| *** maishsk has joined #openstack-infra | 05:02 | |
| *** stevemar_ has quit IRC | 05:02 | |
| *** stevemar has joined #openstack-infra | 05:02 | |
| *** esikachev has joined #openstack-infra | 05:04 | |
| *** mylu has joined #openstack-infra | 05:05 | |
| *** torandu has quit IRC | 05:06 | |
| *** torandu has joined #openstack-infra | 05:06 | |
| *** maishsk has quit IRC | 05:06 | |
| *** esikachev has quit IRC | 05:08 | |
| *** torandu has quit IRC | 05:08 | |
| *** torandu has joined #openstack-infra | 05:09 | |
| *** achanda has quit IRC | 05:10 | |
| *** maishsk has joined #openstack-infra | 05:11 | |
| frickler | does http://nodepool.openstack.org/176634.qcow2 exist intentionally or is that some stray artefact? | 05:12 |
| khappone_ | Hi, a question for the infra people. What are the usage policies of the openstack etherpad? We're starting a finnish monthly operators video call, and we're wondering if we could use the openstack etherpad for notes? | 05:15 |
| *** [1]Thelo has joined #openstack-infra | 05:18 | |
| *** harlowja_at_home has quit IRC | 05:19 | |
| *** Thelo has quit IRC | 05:20 | |
| *** [1]Thelo is now known as Thelo | 05:20 | |
| *** tpeoples is now known as tpeoples_vacatio | 05:23 | |
| *** tpeoples_vacatio is now known as tpeoplesvacation | 05:24 | |
| *** davideag_ has joined #openstack-infra | 05:24 | |
| *** jaosorior has joined #openstack-infra | 05:25 | |
| *** davideagnello has quit IRC | 05:26 | |
| *** shashank_hegde has joined #openstack-infra | 05:28 | |
| *** markvoelker has joined #openstack-infra | 05:29 | |
| *** toabctl has left #openstack-infra | 05:29 | |
| *** sdake_ has joined #openstack-infra | 05:30 | |
| *** sdake_ has quit IRC | 05:31 | |
| *** sdake has quit IRC | 05:32 | |
| *** markvoelker has quit IRC | 05:37 | |
| *** yolanda has quit IRC | 05:39 | |
| *** davideag_ has quit IRC | 05:42 | |
| *** asettle has joined #openstack-infra | 05:43 | |
| *** mylu has quit IRC | 05:43 | |
| *** shashank_hegde has quit IRC | 05:45 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:46 | |
| *** mylu has joined #openstack-infra | 05:48 | |
| *** mylu has quit IRC | 05:48 | |
| *** shashank_hegde has joined #openstack-infra | 05:49 | |
| *** mtreinish has joined #openstack-infra | 05:52 | |
| *** kzaitsev_mb has quit IRC | 05:53 | |
| *** ifarkas has quit IRC | 05:54 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add Gentoo to the dhcp-all-interfaces element https://review.openstack.org/299694 | 05:54 |
| *** yamahata has joined #openstack-infra | 05:58 | |
| *** sputnik13net has quit IRC | 05:58 | |
| *** fedexo has quit IRC | 06:01 | |
| openstackgerrit | Anton Arefiev proposed openstack-infra/project-config: Add Ironic Inspector basic tempest job https://review.openstack.org/299821 | 06:02 |
| AJaeger | khappone_: Is that an OpenStack operator meeting? | 06:04 |
| *** tqtran has joined #openstack-infra | 06:06 | |
| *** ramishra has quit IRC | 06:06 | |
| *** YorikSar has quit IRC | 06:06 | |
| *** YorikSar has joined #openstack-infra | 06:08 | |
| *** ramishra has joined #openstack-infra | 06:08 | |
| *** tqtran has quit IRC | 06:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/300318 | 06:11 |
| *** MarkAtwood has joined #openstack-infra | 06:11 | |
| *** mikelk has joined #openstack-infra | 06:13 | |
| *** yolanda has joined #openstack-infra | 06:15 | |
| AJaeger | infra-root, tripleo-quickstart merged yesterday but setup looks broken on https://review.openstack.org/#/admin/projects/openstack/tripleo-quickstart,access . I don't see the import from the upstream repo run either. Details at https://review.openstack.org/299932 | 06:15 |
| *** armax has quit IRC | 06:18 | |
| *** MarkAtwood has quit IRC | 06:20 | |
| *** zhurong_ has joined #openstack-infra | 06:22 | |
| *** Sukhdev has quit IRC | 06:24 | |
| *** mikelk has quit IRC | 06:29 | |
| *** achanda has joined #openstack-infra | 06:30 | |
| *** camunoz has quit IRC | 06:31 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 06:31 |
| *** asettle has quit IRC | 06:35 | |
| *** sdake has joined #openstack-infra | 06:36 | |
| *** fawadkhaliq has quit IRC | 06:38 | |
| *** fawadkhaliq has joined #openstack-infra | 06:38 | |
| *** asettle has joined #openstack-infra | 06:39 | |
| khappone_ | AJaeger: yes, it's an openstack operators meeting, although in no official capacity, just a bunch of us that decided that common rant sessiosn might be useful | 06:39 |
| AJaeger | khappone_: I'm not aware of any official policy but if it's an openstack operator meeting, just go ahead. I consider that perfectly fine usage. | 06:40 |
| khappone_ | AJaeger: Thanks! Yeah, it seems like the use case would fit | 06:41 |
| *** akaszuba has joined #openstack-infra | 06:45 | |
| *** jlanoux has joined #openstack-infra | 06:47 | |
| *** rlandy has quit IRC | 06:52 | |
| *** asettle has quit IRC | 06:52 | |
| AJaeger | khappone_: it's community infrastructure and operators - especially finnish ones ;) - are part of the community | 06:52 |
| *** gnuoy_ has joined #openstack-infra | 06:54 | |
| *** gnuoy_ has quit IRC | 06:54 | |
| *** shashank_hegde has quit IRC | 06:55 | |
| openstackgerrit | Koteswara Rao Kelam proposed openstack-infra/project-config: Import from existing repository. https://review.openstack.org/300331 | 06:56 |
| *** asettle has joined #openstack-infra | 06:57 | |
| *** asettle has quit IRC | 07:02 | |
| *** salv-orlando has joined #openstack-infra | 07:05 | |
| *** aeng has quit IRC | 07:06 | |
| *** MarkAtwood has joined #openstack-infra | 07:07 | |
| *** e0ne has joined #openstack-infra | 07:07 | |
| *** MarkAtwood has quit IRC | 07:11 | |
| *** |-paul-| has joined #openstack-infra | 07:12 | |
| *** asettle has joined #openstack-infra | 07:14 | |
| *** asettle has quit IRC | 07:14 | |
| *** gildub has quit IRC | 07:16 | |
| *** dizquierdo has joined #openstack-infra | 07:16 | |
| *** matrohon has joined #openstack-infra | 07:16 | |
| *** jamielennox is now known as jamielennox|away | 07:17 | |
| *** dizquierdo has quit IRC | 07:18 | |
| *** dizquierdo has joined #openstack-infra | 07:18 | |
| *** markvoelker has joined #openstack-infra | 07:19 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:19 | |
| *** zeih has joined #openstack-infra | 07:20 | |
| *** salv-orl_ has joined #openstack-infra | 07:23 | |
| *** mrmartin has joined #openstack-infra | 07:23 | |
| *** kzaitsev_mb has quit IRC | 07:25 | |
| *** pcaruana has joined #openstack-infra | 07:25 | |
| *** markvoelker has quit IRC | 07:25 | |
| *** salv-orlando has quit IRC | 07:26 | |
| *** mrmartin has quit IRC | 07:29 | |
| therve | We're seeing quota issues when running grenade tests in Heat | 07:30 |
| *** lezbar has joined #openstack-infra | 07:30 | |
| therve | Did something change? Is it an issue with a provider? | 07:30 |
| *** nadya has joined #openstack-infra | 07:31 | |
| *** yaume has joined #openstack-infra | 07:32 | |
| *** nadya has quit IRC | 07:34 | |
| *** e0ne has quit IRC | 07:35 | |
| *** asselin__ has quit IRC | 07:35 | |
| *** e0ne has joined #openstack-infra | 07:35 | |
| *** hichihara has quit IRC | 07:35 | |
| *** dtantsur|afk is now known as dtantsur | 07:37 | |
| *** jcoufal has joined #openstack-infra | 07:39 | |
| *** e0ne has quit IRC | 07:40 | |
| *** yamahata has quit IRC | 07:42 | |
| *** shashank_hegde has joined #openstack-infra | 07:43 | |
| *** e0ne has joined #openstack-infra | 07:45 | |
| *** jaosorior has quit IRC | 07:46 | |
| *** jaosorior has joined #openstack-infra | 07:46 | |
| *** ociuhandu has quit IRC | 07:51 | |
| *** amotoki has joined #openstack-infra | 07:52 | |
| *** e0ne has quit IRC | 07:54 | |
| *** tesseract has joined #openstack-infra | 07:56 | |
| *** fawadkhaliq has quit IRC | 07:56 | |
| *** tesseract is now known as Guest90877 | 07:56 | |
| *** e0ne has joined #openstack-infra | 07:57 | |
| *** abregman has joined #openstack-infra | 07:58 | |
| *** dtantsur is now known as dtantsur|bbl | 07:59 | |
| *** arxcruz has joined #openstack-infra | 08:01 | |
| *** ifarkas has joined #openstack-infra | 08:02 | |
| *** yolanda has quit IRC | 08:04 | |
| *** shashank_hegde has quit IRC | 08:05 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:06 | |
| *** jistr has joined #openstack-infra | 08:07 | |
| *** MarkAtwood has joined #openstack-infra | 08:08 | |
| *** mikelk has joined #openstack-infra | 08:08 | |
| *** mrmartin has joined #openstack-infra | 08:08 | |
| *** e0ne has quit IRC | 08:10 | |
| *** MarkAtwood has quit IRC | 08:12 | |
| *** e0ne has joined #openstack-infra | 08:13 | |
| *** cdent has joined #openstack-infra | 08:16 | |
| *** baoli has joined #openstack-infra | 08:16 | |
| *** e0ne has quit IRC | 08:16 | |
| *** kzaitsev_mb has quit IRC | 08:18 | |
| *** sflanigan has quit IRC | 08:19 | |
| *** toabctl has joined #openstack-infra | 08:20 | |
| *** e0ne has joined #openstack-infra | 08:20 | |
| *** baoli has quit IRC | 08:20 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:21 | |
| *** sfinucan has joined #openstack-infra | 08:22 | |
| *** achanda has quit IRC | 08:22 | |
| *** cdent has quit IRC | 08:24 | |
| *** cdent has joined #openstack-infra | 08:24 | |
| *** achanda has joined #openstack-infra | 08:25 | |
| *** e0ne has quit IRC | 08:26 | |
| *** e0ne has joined #openstack-infra | 08:29 | |
| *** cdent has quit IRC | 08:30 | |
| *** e0ne_ has joined #openstack-infra | 08:31 | |
| *** e0ne has quit IRC | 08:32 | |
| *** achanda has quit IRC | 08:32 | |
| *** hashar has joined #openstack-infra | 08:32 | |
| *** cdent has joined #openstack-infra | 08:32 | |
| *** e0ne_ has quit IRC | 08:33 | |
| *** achanda has joined #openstack-infra | 08:38 | |
| *** flepied has joined #openstack-infra | 08:39 | |
| openstackgerrit | Kairat Kushaev proposed openstack/requirements: Bump oslo.context >= 2.2.0 to store roles https://review.openstack.org/300361 | 08:43 |
| *** vgridnev has joined #openstack-infra | 08:44 | |
| *** e0ne has joined #openstack-infra | 08:48 | |
| *** asilenkov has quit IRC | 08:48 | |
| *** katyafervent_awa has quit IRC | 08:48 | |
| *** amit213 has quit IRC | 08:49 | |
| *** abregman is now known as abregman|afk | 08:51 | |
| *** asilenkov has joined #openstack-infra | 08:51 | |
| *** kzaitsev_mb has quit IRC | 08:51 | |
| AJaeger | therve: do you have a log file to point to? | 08:52 |
| *** e0ne has quit IRC | 08:53 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:53 | |
| therve | AJaeger, http://logs.openstack.org/75/299575/1/gate/gate-grenade-dsvm-heat/b697039/ | 08:53 |
| AJaeger | therve: do you have a direct link to the file/line that shows it? | 08:55 |
| *** gnuoy_ has joined #openstack-infra | 08:55 | |
| therve | AJaeger, sure http://logs.openstack.org/75/299575/1/gate/gate-grenade-dsvm-heat/b697039/logs/grenade.sh.txt.gz | 08:56 |
| *** katyafervent_awa has joined #openstack-infra | 08:56 | |
| *** gnuoy_ has quit IRC | 08:57 | |
| therve | At the end of it | 08:57 |
| AJaeger | therve: I looked at the others first ;) | 08:57 |
| *** achanda has quit IRC | 08:57 | |
| odyssey4me | gerrit seems to be very slow loading over the last 10 mins or so, is this being experienced by others? | 08:57 |
| AJaeger | infra-root, could you check that log file by therve - this is bluebox ^ | 08:58 |
| *** kzaitsev_mb has quit IRC | 08:58 | |
| *** e0ne has joined #openstack-infra | 08:58 | |
| AJaeger | therve: thanks, hope somebody else can look further. I can't help directly further | 08:59 |
| therve | AJaeger, No problem, thank you | 08:59 |
| *** MarkAtwood has joined #openstack-infra | 09:00 | |
| therve | It seems fairly localized, not sure it has happened since then | 09:01 |
| *** mrmartin has quit IRC | 09:01 | |
| *** MarkAtwood has quit IRC | 09:02 | |
| *** MarkAtwood has joined #openstack-infra | 09:02 | |
| *** vgridnev has quit IRC | 09:04 | |
| *** MarkAtwood has quit IRC | 09:04 | |
| *** MarkAtwood has joined #openstack-infra | 09:04 | |
| *** MarkAtwood has quit IRC | 09:06 | |
| *** kelepirci_ has joined #openstack-infra | 09:06 | |
| openstackgerrit | Ferhat Ozkasgarli proposed openstack-infra/project-config: i18n Support for freezer-web-ui https://review.openstack.org/300370 | 09:06 |
| *** vgridnev has joined #openstack-infra | 09:06 | |
| *** shardy has joined #openstack-infra | 09:10 | |
| *** markvoelker has joined #openstack-infra | 09:11 | |
| *** MarkAtwood has joined #openstack-infra | 09:12 | |
| *** sdake has quit IRC | 09:13 | |
| *** MarkAtwood has quit IRC | 09:14 | |
| *** fhubik has joined #openstack-infra | 09:14 | |
| *** esikachev has joined #openstack-infra | 09:15 | |
| *** MarkAtwood has joined #openstack-infra | 09:16 | |
| *** markvoelker has quit IRC | 09:16 | |
| *** MarkAtwood has quit IRC | 09:18 | |
| *** vgridnev has quit IRC | 09:18 | |
| *** hrubi has quit IRC | 09:19 | |
| *** MarkAtwood has joined #openstack-infra | 09:20 | |
| *** MarkAtwood has quit IRC | 09:21 | |
| *** jaosorior has quit IRC | 09:21 | |
| *** MarkAtwood has joined #openstack-infra | 09:22 | |
| *** skolekonov|away is now known as skolekonov | 09:22 | |
| *** MarkAtwood has quit IRC | 09:23 | |
| *** MarkAtwood has joined #openstack-infra | 09:24 | |
| *** YorikSar has quit IRC | 09:25 | |
| *** MarkAtwood has quit IRC | 09:26 | |
| *** MarkAtwood has joined #openstack-infra | 09:26 | |
| *** YorikSar has joined #openstack-infra | 09:26 | |
| *** amotoki has quit IRC | 09:27 | |
| *** MarkAtwood has quit IRC | 09:27 | |
| *** MarkAtwood has joined #openstack-infra | 09:28 | |
| *** MarkAtwood has quit IRC | 09:29 | |
| *** MarkAtwood has joined #openstack-infra | 09:30 | |
| *** MarkAtwo_ has joined #openstack-infra | 09:32 | |
| *** MarkAtwood has quit IRC | 09:32 | |
| *** MarkAtwo_ has quit IRC | 09:33 | |
| *** smcginnis_ has joined #openstack-infra | 09:34 | |
| smcginnis_ | Not sure if anything can be done about it, but the Cinder meeting logs got cut off and there is no corresponding html summaries: http://eavesdrop.openstack.org/meetings/cinder/2016/cinder.2016-03-30-16.02.log.txt | 09:35 |
| *** tosky has joined #openstack-infra | 09:35 | |
| *** e0ne has quit IRC | 09:36 | |
| *** MarkAtwood has joined #openstack-infra | 09:42 | |
| *** amotoki has joined #openstack-infra | 09:42 | |
| *** jcoufal has quit IRC | 09:42 | |
| *** MarkAtwood has quit IRC | 09:43 | |
| *** MarkAtwood has joined #openstack-infra | 09:44 | |
| *** ams_ has joined #openstack-infra | 09:44 | |
| *** amotoki has quit IRC | 09:44 | |
| *** hrubi has joined #openstack-infra | 09:45 | |
| *** MarkAtwo_ has joined #openstack-infra | 09:46 | |
| *** amotoki has joined #openstack-infra | 09:46 | |
| *** e0ne has joined #openstack-infra | 09:47 | |
| *** MarkAtwo_ has quit IRC | 09:47 | |
| *** MarkAtwo_ has joined #openstack-infra | 09:48 | |
| *** MarkAtwood has quit IRC | 09:49 | |
| *** MarkAtwo_ has quit IRC | 09:49 | |
| *** MarkAtwood has joined #openstack-infra | 09:50 | |
| *** amotoki has quit IRC | 09:51 | |
| openstackgerrit | Maish Saidel-Keesing proposed openstack-infra/irc-meetings: Added nonATCrecogniction meeting https://review.openstack.org/300391 | 09:51 |
| *** MarkAtwood has quit IRC | 09:51 | |
| *** MarkAtwood has joined #openstack-infra | 09:52 | |
| *** MarkAtwood has quit IRC | 09:53 | |
| *** MarkAtwood has joined #openstack-infra | 09:53 | |
| *** hrubi_ has joined #openstack-infra | 09:54 | |
| *** Douhet has quit IRC | 09:54 | |
| *** MarkAtwood has quit IRC | 09:55 | |
| *** hrubi has quit IRC | 09:55 | |
| *** Douhet has joined #openstack-infra | 09:56 | |
| *** amotoki has joined #openstack-infra | 09:56 | |
| *** amotoki has quit IRC | 09:56 | |
| *** daemontool has joined #openstack-infra | 09:57 | |
| *** achanda has joined #openstack-infra | 09:58 | |
| *** amotoki has joined #openstack-infra | 09:59 | |
| *** hrubi_ has quit IRC | 09:59 | |
| *** MarkAtwood has joined #openstack-infra | 09:59 | |
| *** hrubi has joined #openstack-infra | 10:00 | |
| *** zeih has quit IRC | 10:01 | |
| *** vgridnev has joined #openstack-infra | 10:01 | |
| *** MarkAtwood has quit IRC | 10:01 | |
| *** MarkAtwo_ has joined #openstack-infra | 10:01 | |
| *** achanda has quit IRC | 10:02 | |
| *** MarkAtwood has joined #openstack-infra | 10:03 | |
| *** MarkAtwo_ has quit IRC | 10:04 | |
| *** MarkAtwood has quit IRC | 10:05 | |
| *** smcginnis_ has quit IRC | 10:05 | |
| *** markvoelker has joined #openstack-infra | 10:06 | |
| *** e0ne has quit IRC | 10:08 | |
| *** ihrachys has joined #openstack-infra | 10:12 | |
| *** markvoelker has quit IRC | 10:12 | |
| *** e0ne has joined #openstack-infra | 10:12 | |
| *** abregman|afk is now known as abregman|now_rea | 10:12 | |
| *** sdague has joined #openstack-infra | 10:14 | |
| *** Qiming has quit IRC | 10:15 | |
| *** zeih has joined #openstack-infra | 10:15 | |
| *** amotoki has quit IRC | 10:16 | |
| *** cdent has quit IRC | 10:17 | |
| *** zeih has quit IRC | 10:20 | |
| *** mrmartin has joined #openstack-infra | 10:20 | |
| *** daemontool_ has joined #openstack-infra | 10:24 | |
| *** jcoufal has joined #openstack-infra | 10:25 | |
| *** nadya has joined #openstack-infra | 10:26 | |
| *** daemontool has quit IRC | 10:27 | |
| *** bauzas is now known as bauwser | 10:27 | |
| *** yamamoto has quit IRC | 10:30 | |
| *** ociuhandu has joined #openstack-infra | 10:31 | |
| *** kong has quit IRC | 10:36 | |
| *** alaski is now known as lascii | 10:38 | |
| *** dtantsur|bbl is now known as dtantsur | 10:40 | |
| *** cdent has joined #openstack-infra | 10:43 | |
| *** amotoki has joined #openstack-infra | 10:45 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:46 | |
| *** e0ne has quit IRC | 10:47 | |
| *** mrmartin has quit IRC | 10:50 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 10:50 |
| openstackgerrit | Aparna proposed openstack/diskimage-builder: Support to add self-signed certificate in ironic-agent https://review.openstack.org/295203 | 10:50 |
| *** e0ne has joined #openstack-infra | 10:52 | |
| *** sbalukoff has quit IRC | 10:53 | |
| *** aspiers_ is now known as aspiers | 10:54 | |
| *** zeih has joined #openstack-infra | 10:58 | |
| *** trown|outtypewww is now known as trown | 10:59 | |
| *** markvoelker has joined #openstack-infra | 11:02 | |
| *** e0ne has quit IRC | 11:04 | |
| *** markvoelker has quit IRC | 11:06 | |
| *** zeih has quit IRC | 11:08 | |
| *** maishsk_ has joined #openstack-infra | 11:08 | |
| *** e0ne has joined #openstack-infra | 11:08 | |
| *** yamamoto has joined #openstack-infra | 11:09 | |
| *** yamamoto has quit IRC | 11:09 | |
| *** yamamoto has joined #openstack-infra | 11:09 | |
| *** maishsk has quit IRC | 11:09 | |
| *** maishsk_ is now known as maishsk | 11:09 | |
| *** e0ne has quit IRC | 11:11 | |
| *** baoli has joined #openstack-infra | 11:12 | |
| *** hashar has quit IRC | 11:13 | |
| *** baoli has quit IRC | 11:16 | |
| *** tpsilva has joined #openstack-infra | 11:17 | |
| *** hrubi_ has joined #openstack-infra | 11:20 | |
| *** hrubi has quit IRC | 11:21 | |
| *** _degorenko|afk is now known as degorenko | 11:24 | |
| *** hrubi_ has quit IRC | 11:25 | |
| *** ldnunes has joined #openstack-infra | 11:28 | |
| *** weshay has quit IRC | 11:30 | |
| *** achanda has joined #openstack-infra | 11:30 | |
| openstackgerrit | roman plevka proposed openstack/python-jenkins: Add ssl_verify option to be able to skip server SSL verification https://review.openstack.org/300425 | 11:32 |
| *** e0ne has joined #openstack-infra | 11:33 | |
| *** amotoki has quit IRC | 11:34 | |
| *** thorst has joined #openstack-infra | 11:35 | |
| *** achanda has quit IRC | 11:35 | |
| *** dizquierdo has quit IRC | 11:36 | |
| *** dalgaaf has joined #openstack-infra | 11:37 | |
| daemontool_ | all, anyone can help with http://logs.openstack.org/75/300075/1/check/gate-freezer-devstack-dsvm/88e6814/console.html | 11:37 |
| daemontool_ | thanks | 11:37 |
| *** daemontool_ is now known as daemontool | 11:37 | |
| *** abregman|now_rea is now known as abregman | 11:38 | |
| *** abregman is now known as abregman|afk | 11:41 | |
| *** baoli has joined #openstack-infra | 11:42 | |
| *** snikitin has left #openstack-infra | 11:44 | |
| *** baoli_ has joined #openstack-infra | 11:45 | |
| *** nadya has quit IRC | 11:46 | |
| *** zhurong_ has quit IRC | 11:46 | |
| *** arxcruz has quit IRC | 11:46 | |
| *** baoli has quit IRC | 11:48 | |
| *** e0ne has quit IRC | 11:50 | |
| *** Goneri has quit IRC | 11:50 | |
| *** kzaitsev_mb has quit IRC | 11:50 | |
| *** rfolco has joined #openstack-infra | 11:53 | |
| *** e0ne has joined #openstack-infra | 11:54 | |
| *** zeih has joined #openstack-infra | 11:55 | |
| *** kgiusti has joined #openstack-infra | 11:56 | |
| *** e0ne has quit IRC | 11:57 | |
| *** dprince has joined #openstack-infra | 11:57 | |
| *** arxcruz has joined #openstack-infra | 11:58 | |
| *** vhejral has joined #openstack-infra | 11:59 | |
| *** zeih has quit IRC | 11:59 | |
| *** abregman|afk is now known as abregman | 12:03 | |
| *** tiswanso has quit IRC | 12:05 | |
| *** mrmartin has joined #openstack-infra | 12:05 | |
| *** kengo_sakai has quit IRC | 12:06 | |
| *** sambetts has joined #openstack-infra | 12:06 | |
| sambetts | AJaeger: QQ, Is the PROJECTS= variable in devstack-gate.yaml order agnostic? E.g. we need the inspector plugin to load after the ironic plugin loads, so do we need to make sure its before Ironic in PROJECTS? | 12:08 |
| *** e0ne has joined #openstack-infra | 12:09 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:09 | |
| AJaeger | daemontool: that was fixed yesterday already, we had a brief period with broken jobs out. Just add a "recheck" comment. | 12:10 |
| AJaeger | sambetts: better ask on #openstack-qa the devstack experts or chck the source docde | 12:10 |
| *** binbincong has quit IRC | 12:11 | |
| sambetts | AJaeger: thanks will do :) | 12:12 |
| pabelanger | morning | 12:13 |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Fix branch passing into IPA post job https://review.openstack.org/300441 | 12:14 |
| jroll | stupid bugfix, easy review | 12:14 |
| *** mrmartin has quit IRC | 12:16 | |
| AJaeger | morning, pabelanger | 12:16 |
| odyssey4me | o/ how's xenial going? | 12:16 |
| AJaeger | jroll: which script uses BRANCH_PATH in your change? | 12:18 |
| daemontool | AJaeger, ty | 12:18 |
| jroll | AJaeger: https://github.com/openstack/ironic-python-agent/blob/master/imagebuild/coreos/full_trusty_build.sh | 12:18 |
| jroll | AJaeger: failure here: http://logs.openstack.org/23/237b76e0d532ff5c6bf45fefc89c81d8f89a3d49/post/ironic-python-agent-buildimage-coreos/8a37d7f/console.html#_2016-04-01_00_24_55_784 | 12:18 |
| AJaeger | jroll: my local copy was out of date - found it now... | 12:19 |
| AJaeger | jroll: thanks | 12:19 |
| jroll | AJaeger: np | 12:19 |
| jroll | AJaeger: defaults to master, so didn't see this until it was backported :( | 12:19 |
| AJaeger | jroll: ;) +2 | 12:19 |
| jroll | ty | 12:20 |
| pabelanger | odyssey4me: ubuntu-xenial was uploaded to clouds last night. Today, we do some initial testing to make sure our CI is working (eg: jenkins) | 12:20 |
| *** chem has joined #openstack-infra | 12:21 | |
| odyssey4me | pabelanger excellent :) I think what I'd like to do is to have CentOS7 & Xenial run as -nv roles on all our repositories, but only for the master branch - we'll be doing a big push for Xenial support in Newton to get done, and hope that we get some volunteers to complete the CentOS work on top of that in parallel | 12:24 |
| *** gordc has joined #openstack-infra | 12:24 | |
| AJaeger | odyssey4me: let's start with experimental patches first - those we can trigger on demand - until we're sure that xenial works fine everywhere... | 12:25 |
| AJaeger | odyssey4me: an experimental change I would +2 today (if pabelanger is happy with it), an -nv I would wait at least another week... | 12:25 |
| *** aysyd has joined #openstack-infra | 12:27 | |
| pabelanger | Right, so we need to land https://review.openstack.org/#/c/300257/ and https://review.openstack.org/#/c/300259/ this morning | 12:28 |
| *** SumitNaiksatam has quit IRC | 12:28 | |
| pabelanger | and https://review.openstack.org/#/c/300258/ for system-config | 12:28 |
| odyssey4me | AJaeger yup, we're not yet ready to have a go at it anyway as we're rounding out the Mitaka release anyway - but in a week or two we'll start heading in that direction | 12:28 |
| odyssey4me | I was just sharing the intent. | 12:28 |
| *** nwkarsten has joined #openstack-infra | 12:29 | |
| *** EricGonczer_ has joined #openstack-infra | 12:30 | |
| *** mrmartin has joined #openstack-infra | 12:30 | |
| *** nadya has joined #openstack-infra | 12:30 | |
| *** EricGonc_ has joined #openstack-infra | 12:31 | |
| *** erlon has joined #openstack-infra | 12:32 | |
| *** achanda has joined #openstack-infra | 12:32 | |
| *** |-paul-| has quit IRC | 12:34 | |
| *** EricGonczer_ has quit IRC | 12:35 | |
| AJaeger | pabelanger: gave already +4 ;) let's see who can help landing them later... | 12:35 |
| *** achanda has quit IRC | 12:37 | |
| *** flepied has quit IRC | 12:39 | |
| *** flepied has joined #openstack-infra | 12:39 | |
| *** niska has quit IRC | 12:40 | |
| *** pradk has joined #openstack-infra | 12:40 | |
| *** rlandy has joined #openstack-infra | 12:40 | |
| *** cbader has quit IRC | 12:41 | |
| *** chem has quit IRC | 12:41 | |
| *** devananda has quit IRC | 12:43 | |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Add scripts to generate tripleo.org website https://review.openstack.org/300452 | 12:43 |
| *** keedya has joined #openstack-infra | 12:43 | |
| *** devananda has joined #openstack-infra | 12:44 | |
| *** flepied has quit IRC | 12:44 | |
| *** annegentle has joined #openstack-infra | 12:44 | |
| *** niska has joined #openstack-infra | 12:44 | |
| *** Qiming has joined #openstack-infra | 12:46 | |
| Zara | Hi, so I'm looking at setting up an account in gerrit to read the events stream, to talk to storyboard. I gather I should be using the hudson lp account for this. my question is, how do I do things on the gerrit side? I don't know how to log into gerrit with this account, and I'm not sure I'm the person who should be doing that. | 12:48 |
| *** vaidy has quit IRC | 12:49 | |
| *** dims_ has quit IRC | 12:49 | |
| *** dims has joined #openstack-infra | 12:49 | |
| Zara | (I'm looking at the docs here, for reference, though this isn't a CI system, and should only be *reading* the stream, not posting to gerrit or anything like that. http://docs.openstack.org/infra/system-config/third_party.html#creating-a-service-account ) | 12:50 |
| *** cbader has joined #openstack-infra | 12:50 | |
| *** hashar has joined #openstack-infra | 12:52 | |
| *** edmondsw has joined #openstack-infra | 12:53 | |
| openstackgerrit | Julien Danjou proposed openstack-dev/pbr: Fix wsgiref script use with oslo.config https://review.openstack.org/296061 | 12:54 |
| *** vhejral has quit IRC | 12:57 | |
| *** e0ne has quit IRC | 12:57 | |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: Add non-voting gate-trove-python34 check https://review.openstack.org/279108 | 12:58 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Added comments to dashboard.html https://review.openstack.org/299856 | 12:58 |
| openstackgerrit | sebastian marcet proposed openstack-infra/puppet-redis: improved pesistence model https://review.openstack.org/300461 | 12:59 |
| fungi | frickler: http://nodepool.openstack.org/176634.qcow2 is an image one of our root admins made available for someone to download for debugging purposes and (probably) hasn't gotten around to deleting yet | 12:59 |
| *** akshai has joined #openstack-infra | 13:00 | |
| *** yamamoto has quit IRC | 13:00 | |
| *** mtanino has quit IRC | 13:01 | |
| *** akshai_ has joined #openstack-infra | 13:02 | |
| *** smarcet has joined #openstack-infra | 13:02 | |
| pabelanger | Ah, I noticed that the other day but forgot to ask the purpose of it | 13:03 |
| *** eharney has joined #openstack-infra | 13:03 | |
| *** esikachev has quit IRC | 13:03 | |
| pabelanger | fungi: to early for a +A on 300257 and 300259 ? puppet-apply and bindep experimental jobs for ubuntu-xenial | 13:03 |
| *** vaidy has joined #openstack-infra | 13:04 | |
| *** _ari_ has joined #openstack-infra | 13:05 | |
| *** flepied has joined #openstack-infra | 13:05 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: add missing release jobs https://review.openstack.org/300474 | 13:05 |
| *** akshai has quit IRC | 13:05 | |
| *** amotoki has joined #openstack-infra | 13:06 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Fix on attendees schedule https://review.openstack.org/300476 | 13:07 |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Fix on attendees schedule https://review.openstack.org/300476 | 13:07 |
| *** tiswanso has joined #openstack-infra | 13:08 | |
| fungi | AJaeger: therve: yeah, looks like maybe bumping up against a network quota in devstack? http://logs.openstack.org/75/299575/1/gate/gate-grenade-dsvm-heat/b697039/logs/grenade.sh.txt.gz#_2016-04-01_06_56_31_273 you might have more luck asking in #openstack-qa | 13:08 |
| AJaeger | fungi, pabelanger (from backscroll), tripleo-quickstart merged yesterday but setup looks broken on https://review.openstack.org/#/admin/projects/openstack/tripleo-quickstart,access . I don't see the import from the upstream repo run either. Details at https://review.openstack.org/299932 | 13:09 |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: Add non-voting gate-trove-python34 check https://review.openstack.org/279108 | 13:09 |
| fungi | AJaeger: yep, saw. i'm digging through the change now looking to see if i can spot a problem | 13:10 |
| AJaeger | thanks, fungi ! | 13:10 |
| pabelanger | AJaeger: I am not sure how to debug that. But happy to learn if fungi or other root steps me thru | 13:11 |
| *** aysyd has quit IRC | 13:11 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: add missing release jobs https://review.openstack.org/300474 | 13:12 |
| fungi | pabelanger: mostly it'll be something going wrong when ansible/puppet runs manage-projects on review.o.o | 13:12 |
| *** links has quit IRC | 13:12 | |
| fungi | pabelanger: most common cause is a missing or syntactically incorrect acl, or an invalid "upstream" clone url, but i'm not immediately seeing that as the case here so going to check syslog | 13:13 |
| AJaeger | fungi: invalid upstream urls should not happen anymore - part of our tests involve cloning the repo | 13:13 |
| fungi | and then possibly try rerunning manage-projects myself with the debug flag if i still don't see what's up | 13:13 |
| AJaeger | (and checking that it has a master branch ;) | 13:14 |
| fungi | AJaeger: yep, or at least there would be a very tight window where they'd have to become invalid between gating and puppeting | 13:14 |
| *** Goneri has joined #openstack-infra | 13:14 | |
| AJaeger | fungi: yeah | 13:14 |
| *** e0ne has joined #openstack-infra | 13:15 | |
| openstackgerrit | Niall Bunting proposed openstack/requirements: Don't use 1.3.0 version of cryptography https://review.openstack.org/300479 | 13:15 |
| *** aysyd has joined #openstack-infra | 13:15 | |
| *** vgridnev has quit IRC | 13:15 | |
| fungi | another possibility is that manage-projects ran too long for some reason and hit a timeout (average runtimes for it appear to be around 20-22 minutes lately and i think we have puppet set to time that exec out at 30?) | 13:17 |
| *** weshay has joined #openstack-infra | 13:17 | |
| fungi | the change merged yesterday at 18:48 utc and there was a manage_projects exec logged by puppet completing at 19:23 utc after running about 20 minutes. doesn't claim a nonzero return code | 13:19 |
| pabelanger | exec default timeout is 300 seconds: https://docs.puppetlabs.com/puppet/latest/reference/type.html#exec-attribute-timeout checking if we set it to something else | 13:19 |
| *** julim_ has quit IRC | 13:20 | |
| *** binbincong has joined #openstack-infra | 13:20 | |
| *** vgridnev has joined #openstack-infra | 13:21 | |
| *** julim has joined #openstack-infra | 13:22 | |
| jroll | fungi: easy review I'm blocked on, if you get a moment this morning: https://review.openstack.org/#/c/300441/ | 13:22 |
| fungi | aha, i keep forgetting we started echoing m-p output to a separate log | 13:22 |
| openstackgerrit | Merged openstack-infra/project-config: Add designate stable/mitaka to gerritbot https://review.openstack.org/300058 | 13:22 |
| fungi | Exception: Gerrit error executing gerrit replicate openstack/tripleo-quickstart | 13:22 |
| *** matt-borland has joined #openstack-infra | 13:23 | |
| fungi | looks like it may have been a transient error, but because it hit that, it short-circuited the rest of the setup for it | 13:23 |
| fungi | i bet if i rerun it, things will work fine | 13:23 |
| *** salv-orlando has joined #openstack-infra | 13:23 | |
| *** esikachev has joined #openstack-infra | 13:24 | |
| *** dansmith is now known as bizarroman | 13:25 | |
| fungi | AJaeger: pabelanger: after rerunning `sudo manage-projects openstack/tripleo-quickstart` the acl seems correctly applied https://review.openstack.org/#/admin/projects/openstack/tripleo-quickstart,access | 13:25 |
| *** bizarroman is now known as bizarrodan | 13:25 | |
| fungi | for future reference, when puppet runs manage-projects it gets logged to /var/log/manage_projects.log | 13:26 |
| *** salv-orl_ has quit IRC | 13:26 | |
| *** bauwser is now known as lawn_bauwer | 13:27 | |
| pabelanger | fungi: great | 13:27 |
| *** dims_ has joined #openstack-infra | 13:27 | |
| *** yamamoto has joined #openstack-infra | 13:28 | |
| AJaeger | fungi, and even github is setup at https://github.com/openstack/tripleo-quickstart/ | 13:28 |
| AJaeger | thanks, fungi! | 13:28 |
| *** mrmartin has quit IRC | 13:28 | |
| *** e0ne has quit IRC | 13:29 | |
| AJaeger | so, we can approve https://review.openstack.org/#/c/300318/1 - the daily gerrit/projects normalize bot ;) | 13:29 |
| AJaeger | That's how I noticed... | 13:29 |
| jroll | thanks for the +W fungi :) | 13:29 |
| fungi | yw | 13:29 |
| *** asselin__ has joined #openstack-infra | 13:32 | |
| *** xyang1 has joined #openstack-infra | 13:32 | |
| *** jistr has quit IRC | 13:32 | |
| fungi | clarkb: it looks like you created /var/log/nodepool/image/176634.qcow2 on nodepool back on march 18 for mnaser to inspect. is it safe for me to delete now? | 13:33 |
| mnaser | ^ i can say yes | 13:33 |
| *** openstackgerrit has quit IRC | 13:33 | |
| *** openstackgerrit has joined #openstack-infra | 13:33 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Add post job for building and uploading tinyipa https://review.openstack.org/299485 | 13:34 |
| fungi | mnaser: thanks, i was pretty sure i recalled you saying he could delete it, so he likely just got pulled away by some other emergency at that point and forgot. i've removed it now | 13:34 |
| *** achanda has joined #openstack-infra | 13:34 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix branch passing into IPA post job https://review.openstack.org/300441 | 13:34 |
| fungi | jroll: ^ will probably be live in the next 15-30 minutes | 13:35 |
| *** krtaylor has quit IRC | 13:35 | |
| jroll | fungi: yep, I'll keep an eye out | 13:35 |
| *** chem has joined #openstack-infra | 13:35 | |
| *** salv-orlando has quit IRC | 13:35 | |
| jroll | fungi: oh, the update isn't through a post job, is it | 13:36 |
| *** asselin__ has quit IRC | 13:36 | |
| fungi | jroll: no, ansible runs from cron every 15 minutes and connects to our servers and updates configuration on them | 13:37 |
| *** aysyd has quit IRC | 13:37 | |
| fungi | but sometimes it takes in excess of 15 minutes to run | 13:37 |
| jroll | right on, thanks | 13:37 |
| *** achanda has quit IRC | 13:39 | |
| *** zz_dimtruck is now known as dimtruck | 13:39 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/300318 | 13:39 |
| *** jsavak has joined #openstack-infra | 13:39 | |
| *** abregman is now known as abregman|afk | 13:41 | |
| *** electrofelix has joined #openstack-infra | 13:41 | |
| *** aysyd has joined #openstack-infra | 13:41 | |
| *** thiagop-afk has joined #openstack-infra | 13:43 | |
| *** thiagop-afk is now known as thiagop | 13:43 | |
| *** annegentle has quit IRC | 13:46 | |
| *** markvoelker has joined #openstack-infra | 13:47 | |
| *** mriedem has joined #openstack-infra | 13:47 | |
| *** Liuqing has joined #openstack-infra | 13:48 | |
| *** e0ne has joined #openstack-infra | 13:51 | |
| *** markvoelker has quit IRC | 13:51 | |
| *** mtanino has joined #openstack-infra | 13:51 | |
| *** lascii is now known as lutf-8 | 13:52 | |
| *** binbincong has quit IRC | 13:58 | |
| *** aysyd has quit IRC | 13:58 | |
| *** zeih has joined #openstack-infra | 13:58 | |
| dmsimard | the devstack-logs builder is the one we currently use to upload logs for the packstack jobs https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/macros.yaml#L880-L889 | 13:58 |
| dmsimard | I don't see anything specific in it that would explain the devstack "explanations and troubleshooting info" that appears when browsing logs | 13:58 |
| *** Hal has joined #openstack-infra | 13:58 | |
| dmsimard | Is there a way to opt-out of that ? devstack-specific info is not super relevant to Packstack jobs as you can imagine. | 13:58 |
| *** Hal is now known as Guest54251 | 13:58 | |
| *** openstack has quit IRC | 13:58 | |
| *** openstack has joined #openstack-infra | 13:59 | |
| *** daemontool has quit IRC | 13:59 | |
| *** zhurong has joined #openstack-infra | 13:59 | |
| *** daemontool_ is now known as daemontool | 14:01 | |
| *** arxcruz has quit IRC | 14:04 | |
| pabelanger | infra-root and project-config-core: if you mind taking a look at: https://review.openstack.org/#/q/topic:experimental-ubuntu-xenial I'd like to do some smoke testing of ubuntu-xenial today if at all possible | 14:04 |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: Add non-voting gate-trove-python34-db check https://review.openstack.org/279108 | 14:04 |
| *** erlon has quit IRC | 14:04 | |
| *** rbrndt has joined #openstack-infra | 14:05 | |
| *** yamahata has joined #openstack-infra | 14:05 | |
| *** erlon has joined #openstack-infra | 14:05 | |
| *** yaume_ has joined #openstack-infra | 14:05 | |
| *** sdake has joined #openstack-infra | 14:06 | |
| *** e0ne has quit IRC | 14:07 | |
| AJaeger | fungi, git timed just out on rax-dfw here: http://logs.openstack.org/74/300474/2/check/gate-project-config-layout/140491e/console.html - is that something you want to know about? | 14:08 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:08 | |
| *** annegentle has joined #openstack-infra | 14:08 | |
| *** yaume has quit IRC | 14:08 | |
| *** jsavak has quit IRC | 14:09 | |
| *** jsavak has joined #openstack-infra | 14:10 | |
| mordred | pabelanger: https://review.openstack.org/#/c/300258/1 will mean that we'll be voting on xenial for infra puppet patches | 14:10 |
| mordred | pabelanger: I think given the context I'm ok with that - but want fungi to weigh in on it | 14:10 |
| *** tonytan4ever has joined #openstack-infra | 14:10 | |
| pabelanger | mordred: it should be fine, because of https://review.openstack.org/#/c/300257/ | 14:10 |
| pabelanger | which uses the experimental pipeline | 14:11 |
| mordred | kk | 14:12 |
| mordred | I mean, I'm +2 on it - just wanted to make sure fungi expressed an opionion before it +Ad | 14:13 |
| pabelanger | ack | 14:13 |
| *** nwkarsten has quit IRC | 14:13 | |
| Zara | pabelanger: btw, to close stories in storyboard, easiest way is to mark all tasks as invalid or merged as appropriate; that should automatically move the story from the 'active' list. (I've marked tasks invalid on a couple of old stories you said should be closed) | 14:13 |
| *** nwkarsten has joined #openstack-infra | 14:14 | |
| pabelanger | Zara: Ah, cool. Do you mind doing that to one of my stories? So I can see what an email notification looks like? | 14:14 |
| *** weshay has quit IRC | 14:15 | |
| *** dimtruck is now known as zz_dimtruck | 14:15 | |
| *** burgerk has joined #openstack-infra | 14:18 | |
| *** yamamoto has quit IRC | 14:18 | |
| *** aysyd has quit IRC | 14:18 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental ubuntu-xenial puppet-apply job https://review.openstack.org/300257 | 14:18 |
| *** nwkarsten has quit IRC | 14:18 | |
| *** amitgandhinz has joined #openstack-infra | 14:18 | |
| *** tiswanso has quit IRC | 14:18 | |
| *** yamamoto has joined #openstack-infra | 14:20 | |
| *** tiswanso has joined #openstack-infra | 14:20 | |
| *** aysyd has joined #openstack-infra | 14:20 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental bindep-fallback job for ubuntu-xenial https://review.openstack.org/300259 | 14:20 |
| *** pradk has quit IRC | 14:21 | |
| *** pradk has joined #openstack-infra | 14:21 | |
| *** cdent has quit IRC | 14:22 | |
| *** zeih has quit IRC | 14:22 | |
| Kiall | Q - Anyone know of existing projects using the tempest plugin interface? | 14:23 |
| *** ajmiller has joined #openstack-infra | 14:24 | |
| *** nwkarsten has joined #openstack-infra | 14:24 | |
| fungi | AJaeger: it's something we know about, but not something we're able to troubleshoot until we decide where we want to (carefully!) wedge traceroute into our setup | 14:26 |
| fungi | so that we can get better diagnostic data back to rackspace support | 14:26 |
| Zara | pabelanger: I have done, though bear in mind it doesn't subscribe you to a story you create automatically (it probably *should*...), so you'll only get a notification if you manually subscribe. Also that will use an email template for tasks, and those are the ugliest (also on my todo list, yay). | 14:27 |
| sambetts | Hi AJaeger, about your comment on the ironic python agent tinyipa job patch, I'm not sure I understand what you mean by use the same setup? | 14:27 |
| Zara | btw, someone's trying to install storyboard and they're getting an error installing mysql (v5.6, if helpful). this is via an ansible role that we know works. wondering if anyone else has seen any mysql troubles of late? | 14:28 |
| pabelanger | anybody know what is up with jenkins05.o.o? It appears to be stopped | 14:28 |
| pabelanger | checking logs now | 14:29 |
| pabelanger | # There is insufficient memory for the Java Runtime Environment to continue. | 14:30 |
| pabelanger | guess jenkins05 crashed | 14:30 |
| *** knikolla has joined #openstack-infra | 14:31 | |
| *** dtardivel has joined #openstack-infra | 14:32 | |
| pabelanger | fungi: have a min to walk me through jenkins05? | 14:33 |
| jeblair_ | fungi: regarding vexxhost slowness -- i think getting uuids might be the first thing -- it feels like it could even be some misconfigured nova compute host or something | 14:33 |
| *** jeblair_ is now known as jeblair | 14:33 | |
| fungi | Zara: meant to answer your earlier questions in scrollback here... the hudson-openstack account in launchpad is what we use to update launchpad bugs/blueprints _from_ gerrit, but i think yesterday's discussion may have gone sideways a bit because i'm not sure how that will be relevant to storyboard | 14:33 |
| fungi | Zara: other than we need some account in gerrit to listen on the event stream, and could possibly use that launchpad account to do openid authentication for a gerrit account | 14:35 |
| fungi | assuming event stream listener is the model you're wanting to go with (rather than how we do our current launchpad integration, which is triggered by gerrit hook scripts instead) | 14:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: [WIP] Example bindep commit https://review.openstack.org/300526 | 14:37 |
| fungi | i recall the event stream solution was preferred when we were discussing models for this long ago, we're simply using gerrit hook scripts for lp updating because we needed some way to call arbitrary scripts to interface with their already defined api | 14:37 |
| Zara | fungi: That sound like what I was asking after. I've been advised to try an event stream listener, and don't know why it's preferred over hook scripts, but there seems to be a general view that it's easier and I'm inclined to trust that. (this is my first foray into either approach) :D | 14:38 |
| fungi | Zara: i don't know that it's _necessarily_ easier, but it does seem like it will be more robust | 14:39 |
| fungi | basically you can look at it as active vs passive models | 14:39 |
| fungi | if storyboard listens on to and triggers actions from the gerrit event stream (like zuul does), then it has more control over what happens and gets to encode more of the logic internally | 14:40 |
| *** zz_dimtruck is now known as dimtruck | 14:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Bring a single ubuntu-xenial node online https://review.openstack.org/300530 | 14:40 |
| Zara | fungi: aiui, we want an event stream listener to send information to a script that imports the python-storyboardclient, and that then talks to a specified storyboard instance to update it. so we want a gerrit account for that listener. | 14:41 |
| pabelanger | AJaeger: ^ we're ready to launch a single ubuntu-xenial node | 14:41 |
| fungi | if storyboard presents an action-oriented api and gerrit calls that through triggered hooks, then a lot of the logic ends up bolted on from outside and storyboard ends up with less control over the sequence of actions taken | 14:41 |
| openstackgerrit | Merged openstack-infra/system-config: Add ubuntu-xenial to openstack_project::single_use_slave https://review.openstack.org/300258 | 14:41 |
| *** markvoelker has joined #openstack-infra | 14:41 | |
| *** zeih has joined #openstack-infra | 14:42 | |
| *** abregman|afk is now known as abregman | 14:43 | |
| Zara | fungi: ah, I'm not actually sure how much control it needs. I suspect the conversations happened before I joined the project. right now we just want to update task statuses from gerrit, as a start... I don't think it has the potential to cause as much chaos as zuul if misconfigured | 14:43 |
| fungi | Zara: yep, so anyway there's no specific need for launchpad involvement in the desired case, you just need a gerrit account. you can use basically any gerrit account for testing this (you're just planning to use it to listen to the event stream, not actually leave comments/votes on changes in gerrit), though once we put something into production for, say, storyboard.openstack.org the infra team would | 14:44 |
| fungi | create a fresh service account for that or reuse one it already has for some similar purpose | 14:44 |
| Zara | ahh, okay | 14:44 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 14:44 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:45 | |
| Zara | heh, I'm still fuzzy on how this will be generalised for production, but I think that will become clearer as I work on it. | 14:45 |
| fungi | now that said, if you're going to create an account in gerrit to use for this (for testing now, or for use by another non-infra-controlled storyboard instance) you'll need to have _some_ lp account you used to authenticate to gerrit so that you can set your username, ad ssh keys and so on | 14:45 |
| Zara | yeah, I thought that was why hudson was suggested? or was that jumping ahead to the production instance? | 14:45 |
| *** jistr has joined #openstack-infra | 14:46 | |
| fungi | that could just be the personal gerrit account you already had, or you could add a second lp account solely for that purpose | 14:46 |
| fungi | for production use on storyboard.openstack.org we'd likely create or reuse an existing service account in gerrit, but we don't generally tie those to launchpad accounts | 14:46 |
| *** zhurong has quit IRC | 14:46 | |
| *** markvoelker has quit IRC | 14:47 | |
| fungi | since we can configure/update them via the gerrit api as admins | 14:47 |
| *** zeih has quit IRC | 14:47 | |
| *** Liuqing has quit IRC | 14:47 | |
| AJaeger | sambetts: what I mean is: Both jobs are on the same repo, each is invoked differently and files are placed in different locations. Why can't you put all files in an upload folder - and use the same invocation with just one diffrent parameter? | 14:47 |
| *** Liuqing has joined #openstack-infra | 14:48 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Enable DB access for Designate unittests https://review.openstack.org/300537 | 14:48 |
| AJaeger | pabelanger: you don't need this. min-ready: 0 will allow you starting one. | 14:49 |
| Zara | fungi: okay, so right now it sounds like the simplest thing to do is to just use my personal account to read the events stream, and work on getting the python client to send info from there to my test storyboard instance? | 14:49 |
| AJaeger | min-ready: 1 means: Have always a node ready to launch in the queue. | 14:49 |
| fungi | Zara: yep, you don't need anything special to be able to do that. everyone's account has access to listen to the ssh stream-events | 14:50 |
| *** cdent has joined #openstack-infra | 14:50 | |
| pabelanger | AJaeger: but, isn't the issue we haven't registered a ubuntu-xenial job to gearman yet? And we'll always return NOT_REGISTERED for jobs that need ubuntu-xenial? | 14:50 |
| Zara | fungi: okay, brilliant. I wanted to make sure I'm not breaking any rules, but that sounds fine to me. | 14:50 |
| fungi | jeblair: was that an accidental repeat from your input history? or are you still seeing slow behavior in vexxhost? i've been getting uuids to mnaser and he's been running down possible causes for the past couple days | 14:51 |
| Zara | hopefully I'll be able to come back with something more interesting in a little whle | 14:51 |
| Zara | *while | 14:51 |
| AJaeger | pabelanger: let's ask fungi and jeblair about whether "min-ready:0" allows to start jobs - AFAIK it does... | 14:51 |
| fungi | pabelanger: sure, i can help you through jenkins05 | 14:51 |
| pabelanger | fungi: neat | 14:51 |
| AJaeger | pabelanger: it's just slower since none is busy waiting for us. That's my udnerstanding... | 14:51 |
| fungi | pabelanger: i take it you were looking at the jenkins service log and it ran out of memory inside the jvm? | 14:51 |
| pabelanger | fungi: right, I am looking at /tmp/hs_err_pid8695.log on jenkins05 | 14:52 |
| pabelanger | we ran out of ram | 14:52 |
| fungi | AJaeger: pabelanger: yes, min-ready: 0 just means don't build any new nodes unless you see demand for them. it doesn't prevent running anything (nodepool doesn't decide who can run what jobs, it just makes servers) | 14:52 |
| pabelanger | and this server doesn't have a swap | 14:52 |
| sdague | fungi: ... random friday topic | 14:53 |
| *** tmorin has joined #openstack-infra | 14:53 | |
| AJaeger | pabelanger: so, run "check experimental" ;) | 14:53 |
| sdague | has anyone considered mirroring the gerrit event stream to something like mqtt | 14:53 |
| sdague | so that there could be anonymous read on it? | 14:53 |
| jeblair | fungi: no, that was a response to skimming history, thanks for the update | 14:53 |
| pabelanger | AJaeger: have see 300526 | 14:53 |
| AJaeger | pabelanger: with min-ready: -1 nothing would run... | 14:53 |
| *** mylu has joined #openstack-infra | 14:53 | |
| *** woodster_ has joined #openstack-infra | 14:54 | |
| AJaeger | pabelanger: that could also mean that no jenkins had the job so far ;( | 14:54 |
| fungi | pabelanger: so two things to do before starting jenkins: 1. make sure all nodes for it are deleted in nodepool, 2. make sure any lingering node definitions are wiped via `sudo rm -rf /var/lib/jenkins/nodes/*` | 14:54 |
| pabelanger | AJaeger: I always though you need a node to connect to jenkins to register the new job names into gearman | 14:54 |
| pabelanger | fungi: Yup I was going to do that | 14:55 |
| Zara | sdague: sounds cool to me; anteaya mentioned there were drawbacks to keeping it visible to everyone, so she might have something interesting to add about it | 14:55 |
| sdague | been playing with mosquitto for homeautomation stuff, and an mqtt with websockets built in might expand the uses for the data stream | 14:55 |
| sdague | without adding load to gerrit | 14:55 |
| fungi | sdague: we did ponder some possible ways to proxy gerrit's event stream. the design it has with dedicated worker threads handling stream events for ssh sockets has some inherent scalability issues | 14:56 |
| fungi | i don't think we got very far into designing what that would look like though | 14:56 |
| pabelanger | AJaeger: https://jenkins04.openstack.org/job/gate-project-config-bindep-fallback-ubuntu-xenial/ | 14:56 |
| *** e0ne has joined #openstack-infra | 14:57 | |
| pabelanger | AJaeger: 300526 just failed with NOT_REGISTERED | 14:57 |
| sdague | yeh, understood. I've been reasonably impressed with mosquitto at my scall, and the fact that you can direct websocket connect from js is neat - https://github.com/jpmens/simple-mqtt-websocket-example/blob/master/index.html | 14:57 |
| pabelanger | pretty sure we need to launch a ubuntu-xenial image first otherwise gearman won't register the new job | 14:57 |
| jeblair | fungi: it's a pool of threads, not dedicated, but yes, it does take resources | 14:57 |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Enable DB access for Designate unittests https://review.openstack.org/300537 | 14:57 |
| fungi | jeblair: right, the pool has a limited size was what i meant | 14:57 |
| sdague | I don't know how it would handle the load of openstack. But the conceptual model is pretty nice. | 14:57 |
| AJaeger | pabelanger: I thought not - but let's see what others say. Happy to learn something new... | 14:57 |
| sdague | like I said, random friday thought | 14:58 |
| fungi | sdague: i think _ideally_ gerrit would provide some modern stream api which didn't rely on ssh and could be handled more robustly through a web service | 14:58 |
| sdague | fungi: sure | 14:58 |
| fungi | which would remove the last real thing you can only do via ssh | 14:58 |
| sdague | I would agree | 14:58 |
| sdague | but... barring that | 14:58 |
| fungi | i wonder if a gerrit plugin could be used to do that | 14:59 |
| *** nwkarsten has quit IRC | 14:59 | |
| sdague | fungi: possibly, though you'd still be putting some more load on gerrit to do that | 14:59 |
| sdague | if the events mirror off | 15:00 |
| sdague | then the load shifts to another thing | 15:00 |
| sdague | which, with my understanding of gerrit's load, would probably be good | 15:00 |
| pabelanger | fungi: what about the nodes that are attached to jenkins, but in used state. They are safe to stay or have to be deleted? | 15:00 |
| *** nwkarsten has joined #openstack-infra | 15:00 | |
| AJaeger | pabelanger: Have to leave now, will read backscroll later and I'm eager to learn the solution. | 15:00 |
| pabelanger | jenkins05 | 15:00 |
| pabelanger | AJaeger: ack | 15:00 |
| *** kzaitsev_mb has quit IRC | 15:00 | |
| fungi | yeah, just wondering if the gerrit plugin could produce something which apache/whatever could serve to multiple consumers without having to hook in via ssh | 15:00 |
| *** thorst has quit IRC | 15:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 15:01 | |
| fungi | pabelanger: definitely delete used nodes from jenkins, those are the most dangerous | 15:01 |
| pabelanger | fungi: okay | 15:01 |
| *** mrmartin has joined #openstack-infra | 15:02 | |
| fungi | jenkins will automatically put them back online to run new jobs if it can | 15:02 |
| *** maishsk has quit IRC | 15:02 | |
| *** jsavak has quit IRC | 15:02 | |
| *** thorst has joined #openstack-infra | 15:02 | |
| *** jsavak has joined #openstack-infra | 15:03 | |
| *** electrofelix has quit IRC | 15:03 | |
| sdague | fungi: yeh, possibly. I know how you could build it with a python glue script in about a day. | 15:03 |
| sdague | but how to do it as a gerrit plugin... no real idea | 15:03 |
| fungi | sdague: right, that's what we were leaning toward doing, or just using some existing multiplexer even | 15:04 |
| fungi | just thinking more what an upstream wishlist item might be for this so we didn't end up maintaining it indefinitely | 15:04 |
| *** maishsk has joined #openstack-infra | 15:05 | |
| *** nwkarsten has quit IRC | 15:05 | |
| *** markvoelker has joined #openstack-infra | 15:05 | |
| *** esikache1 has joined #openstack-infra | 15:05 | |
| *** imcsk8 is now known as imcsk8|plane | 15:05 | |
| *** esikachev has quit IRC | 15:05 | |
| pabelanger | fungi: thanks | 15:06 |
| pabelanger | jenkins05 back online | 15:06 |
| pabelanger | I'll run JJb shortly | 15:06 |
| *** Liuqing has quit IRC | 15:07 | |
| pabelanger | So, back to gearman. Using the command clarkb showed me the other day, echo "status" | nc localhost 4730 | grep ubuntu-xenial , I don't see any xenial jobs registered with gearman | 15:07 |
| pabelanger | which, and this is where I could be wrong, jobs only register with gearman when a new jenkins slave comes online | 15:07 |
| *** links has joined #openstack-infra | 15:07 | |
| pabelanger | and because ubuntu-xenial is min-ready: 0, it will never launch | 15:07 |
| daemontool | at your convinience, please review https://review.openstack.org/#/c/300370/ +A only is needed | 15:08 |
| *** Liuqing has joined #openstack-infra | 15:09 | |
| jeblair | pabelanger: that is correct; we put servers at min-ready:1 when we're using them experimentally. | 15:09 |
| pabelanger | great, so 300530 is the patch then | 15:09 |
| pabelanger | jeblair: thanks | 15:10 |
| *** yamamoto has quit IRC | 15:11 | |
| *** vgridnev has quit IRC | 15:11 | |
| *** Liuqing has quit IRC | 15:12 | |
| openstackgerrit | Jeffrey Zhang proposed openstack-infra/project-config: add Ubuntu binary build and deploy gate for kolla https://review.openstack.org/292942 | 15:12 |
| fungi | yep, there's a chicken-and-egg issue with the current design, where nodepool won't build any nodes at min-ready: 0 unless it sees demand (jobs waiting to run on them), but if no node gets built jenkins won't register the jobs for it into gearman, and zuul will instantly fail them as NOT_REGISTERED resulting in no actual demand, so nodepool continues to not build any | 15:13 |
| jeblair | well, the current design does not involve min-ready:0 -- the design is to always try to have nodes ready | 15:13 |
| jeblair | (the 'pool' in nodepool) | 15:14 |
| *** abregman is now known as abregman|nb | 15:14 | |
| fungi | true. by "current design" i meant as far as zuul's behavior of reporting NOT_REGISTERED rather than waiting (possibly indefinitely) for the job to be registered | 15:15 |
| pabelanger | and JJB update to date on jenkins05 now | 15:16 |
| Swanson | so when you get cannot merge listed on gerrit how do you resolve that? gill pull --rebase and git review again? (Trying to avoid screwing this up like I did last time I did this... Would love to find that post-it note..) | 15:16 |
| *** [1]Thelo has joined #openstack-infra | 15:18 | |
| *** nwkarsten has joined #openstack-infra | 15:18 | |
| fungi | Swanson: i generally: git review -d <change_num>; git remote update; git rebase -i origin/master; [resolve merge conflicts and continue]; git review | 15:20 |
| *** Thelo has quit IRC | 15:20 | |
| *** [1]Thelo is now known as Thelo | 15:20 | |
| Swanson | fungi, That sounds really familiar. Thanks for the help! | 15:20 |
| fungi | we used to have a section on that in http://docs.openstack.org/infra/manual/developers.html but i think it got taken out because people interpreted it to mean that you should rebase every time you update a change, which is really caustic for your reviewers | 15:20 |
| *** maishsk has quit IRC | 15:21 | |
| *** regXboi has joined #openstack-infra | 15:21 | |
| fungi | best to avoid rebasing the change unless there's actually a merge conflict which needs resolving | 15:21 |
| fungi | or to get it to pick up a new parent change | 15:21 |
| fungi | (in the event that you're stacking multiple changes for review) | 15:21 |
| Swanson | Yep. I think I just made the same changes in two reviews and didn't stack them. | 15:22 |
| *** thorst has quit IRC | 15:22 | |
| Swanson | Thanks again! | 15:23 |
| *** yamamoto has joined #openstack-infra | 15:23 | |
| *** mrmartin has quit IRC | 15:24 | |
| *** mylu has quit IRC | 15:24 | |
| *** mylu has joined #openstack-infra | 15:28 | |
| *** jsavak has quit IRC | 15:29 | |
| mnaser | oh boy | 15:30 |
| openstackgerrit | Merged openstack-infra/project-config: Bring a single ubuntu-xenial node online https://review.openstack.org/300530 | 15:30 |
| *** thorst has joined #openstack-infra | 15:30 | |
| *** ajmiller has quit IRC | 15:30 | |
| mnaser | all instances had wide open quotas in their respective cgroups | 15:30 |
| mnaser | even though libvirt is configured to set it (including the xml) | 15:30 |
| mnaser | this explains a bunch of things. | 15:30 |
| *** e0ne has quit IRC | 15:31 | |
| *** matrohon has quit IRC | 15:31 | |
| *** ajmiller has joined #openstack-infra | 15:32 | |
| *** jsavak has joined #openstack-infra | 15:33 | |
| matt-borland | Hello, in the Horizon jobs we've been getting an increased number of errors with our npm-run-* jobs. We get 403 errors on getting specific resources...e.g. http://logs.openstack.org/46/300446/1/check/gate-horizon-npm-run-lint/da1e264/console.html#_2016-04-01_12_34_26_376 | 15:34 |
| matt-borland | It's possible that this is a StrictSSL problem, or could be a proxy issue on the node. | 15:34 |
| matt-borland | If anyone has perspective on those kinds of errors...it would be much appreciated. :) | 15:35 |
| *** bpokorny has joined #openstack-infra | 15:36 | |
| clarkb | the test machines shouldnt be behind any proxy | 15:36 |
| *** markmcd1 has joined #openstack-infra | 15:36 | |
| matt-borland | clarkb, thanks | 15:36 |
| *** achanda has joined #openstack-infra | 15:37 | |
| *** cdent has quit IRC | 15:37 | |
| clarkb | looks like bitbucket is saying go away | 15:37 |
| clarkb | its funny hiw pypi had this same ptoblem forever | 15:38 |
| *** mylu has quit IRC | 15:38 | |
| clarkb | and now npm is doomed to repeat its mistake | 15:38 |
| matt-borland | "funny" is a funny word :) | 15:38 |
| jeblair | i believe there's ongoing work to make a local npm mirror | 15:39 |
| matt-borland | awesome jeblair | 15:39 |
| *** akaszuba has quit IRC | 15:39 | |
| clarkb | that will probably just mirror the redirect though | 15:39 |
| *** ociuhandu has quit IRC | 15:39 | |
| clarkb | which us ehat all the pypi mirror software did and forced us to wrote our own | 15:40 |
| *** mylu has joined #openstack-infra | 15:40 | |
| jeblair | clarkb: oh, you're saying this is a 'package removed' situation? | 15:40 |
| matt-borland | the only thing is it is intermittent, so it doesn't always reject. | 15:40 |
| jeblair | i was assuming it was a 'transient hosting failure' situation | 15:40 |
| matt-borland | yeah, I am guessing so jeblair | 15:40 |
| clarkb | jeblair: its an exterbal package link situation and the package host (bitbucket) is 403ing | 15:40 |
| *** armax has joined #openstack-infra | 15:41 | |
| jeblair | clarkb: ahh, that. and the original pypi-mirror software. gotcha. | 15:41 |
| clarkb | so mirroring npm doesnt fix bitbucket | 15:41 |
| *** mikelk has quit IRC | 15:41 | |
| jeblair | clarkb: i assumed our npm mirror would hold content | 15:41 |
| *** achanda has quit IRC | 15:42 | |
| *** jlanoux has quit IRC | 15:42 | |
| *** fawadkhaliq has joined #openstack-infra | 15:42 | |
| *** LaT0rtue has joined #openstack-infra | 15:42 | |
| clarkb | I think it will for the things npm actually hosts | 15:42 |
| clarkb | but would expect similar behavior to pypi for external link things | 15:42 |
| matt-borland | interesting | 15:43 |
| *** jlanoux has joined #openstack-infra | 15:43 | |
| jeblair | https://review.openstack.org/235617 is the most recent version of the infra change to add the npm mirror | 15:43 |
| jeblair | it says it uses https://github.com/davglass/registry-static | 15:44 |
| clarkb | because the view the package repos have taken is you are mirroring what they have which is an external link | 15:44 |
| *** krtaylor has joined #openstack-infra | 15:44 | |
| matt-borland | yep, interesting | 15:44 |
| jeblair | clarkb, fungi, mordred: what's the current nodepool-shade status? | 15:44 |
| clarkb | it is worth confirming | 15:44 |
| *** Guest90877 has quit IRC | 15:44 | |
| fungi | jeblair: there were shade releases yesterday | 15:45 |
| clarkb | jeblair: I reviewed a bunch if stuff that I think needs merge and release. I also have not seen or heard what the tripleo work around is | 15:45 |
| fungi | no nodepool restarts yet afaik | 15:45 |
| *** jsavak has left #openstack-infra | 15:45 | |
| jeblair | clarkb: workaround is http://git.openstack.org/cgit/openstack-infra/shade/commit/?id=e43625d30025a11762e1cdd9ff6973ac3ec28db6 | 15:45 |
| fungi | oh, i guess there was _a_ shade release yesterday, then Shrews pointed out that there was still an outstanding patch which didn't get approved which will need another shade release | 15:46 |
| *** ajmiller has quit IRC | 15:46 | |
| *** yamamoto has quit IRC | 15:46 | |
| fungi | but not sure if that remaining patch was necessary for our use with nodepool | 15:46 |
| *** roxanaghe has joined #openstack-infra | 15:46 | |
| jeblair | fungi, Shrews: was it this one? https://review.openstack.org/299672 | 15:47 |
| Shrews | fungi: i think it was | 15:47 |
| Shrews | i just rechecked it | 15:47 |
| Shrews | jeblair: yes | 15:47 |
| clarkb | thats a fun workaround | 15:47 |
| *** ajmiller has joined #openstack-infra | 15:47 | |
| jeblair | it seems to be failing a lot -- are we sure there isn't something to those failures? | 15:47 |
| *** yaume_ has quit IRC | 15:47 | |
| *** tmorin has left #openstack-infra | 15:48 | |
| fungi | jeblair: those last failures were my fauly | 15:48 |
| fungi | fault | 15:48 |
| jeblair | "faulty fungi" | 15:49 |
| fungi | we merged my change to start collecting traceroutes (because rackspace says it needs them to have any hope of diagnosing network issues between ord and dfw when we start using ord again) but i didn't check that traceroute/traceroute6 are actually installed on our new images | 15:49 |
| jeblair | this is what ubuntu should do after "Z" | 15:49 |
| timrc | Sounds like a potential Ubuntu release, jeblair | 15:49 |
| timrc | jeblair: lol | 15:49 |
| timrc | :hivemind: | 15:49 |
| fungi | indeed | 15:49 |
| fungi | so... many jobs using the net-info job macro began insta-failing | 15:50 |
| *** tonytan4ever has quit IRC | 15:50 | |
| Shrews | faulty fungi, jumpy jeblair, moldy mordred | 15:50 |
| Shrews | jeblair: good idea | 15:50 |
| jeblair | silly shrews! | 15:50 |
| pabelanger | \o/ | 15:50 |
| pabelanger | https://jenkins02.openstack.org/job/gate-project-config-bindep-fallback-ubuntu-xenial/1/ | 15:50 |
| fungi | the one we really need is cloned clarkb | 15:50 |
| *** tonytan4ever has joined #openstack-infra | 15:51 | |
| pabelanger | .trolo | 15:51 |
| pabelanger | first issue | 15:51 |
| pabelanger | lack of ubuntu xenial mirror | 15:51 |
| pabelanger | But, Success that jenkins worked | 15:51 |
| jeblair | Shrews, clarkb, fungi: so i think we can restart nodepool now (that hasn't happened, right)? the change in 299672 will reduce thundering herds, but i think we can suffer through that now to see how things are otherwise. | 15:52 |
| *** weshay has joined #openstack-infra | 15:52 | |
| *** yolanda has joined #openstack-infra | 15:53 | |
| *** tonytan4ever has quit IRC | 15:53 | |
| pabelanger | okay, afk for 25mins | 15:53 |
| *** tonytan4ever has joined #openstack-infra | 15:54 | |
| openstackgerrit | Merged openstack-infra/nodepool: Set cache parameters like we do in production https://review.openstack.org/299702 | 15:55 |
| *** links has quit IRC | 15:55 | |
| jeblair | clarkb: also that change worked ^ which gives me some confidence in the port caching patch i wrote wednesday | 15:55 |
| timrc | jeblair says "restart nodepool" and pabelanger goes afk. Smart :P | 15:55 |
| *** IlyaG has joined #openstack-infra | 15:56 | |
| timrc | Oh look at the time! I need to go stir my clarkb clone vat. | 15:56 |
| fungi | yeah, it seems like we should be safe for a restart | 15:56 |
| clarkb | jeblair: cool I suppose we could restart and just watch for stampedes | 15:56 |
| *** ajmiller has quit IRC | 15:56 | |
| clarkb | timrc: those are unlicensed replicas, they have been sold without covers | 15:56 |
| jeblair | yeah, we saw that last time, but it settles out | 15:57 |
| clarkb | jeblair: this is the startup stampede with exclusive blocking locks? | 15:57 |
| *** ajmiller has joined #openstack-infra | 15:58 | |
| jeblair | clarkb: yep | 16:00 |
| jgriffith | anybody know of a reason os-client-config may not be reading my .config/openstack file? | 16:02 |
| jgriffith | errr... ok, that's a pretty dumb question to ask I guess | 16:02 |
| mtreinish | jgriffith: well you work for a storage company, so hardware corruption? | 16:02 |
| jgriffith | mtreinish: touchet! | 16:02 |
| clarkb | permissions? also I think openstack is supposed to be a dir then you put clouds.yaml in it | 16:03 |
| *** Swami has joined #openstack-infra | 16:03 | |
| jgriffith | I'm just trying this: http://paste.openstack.org/show/492747/ | 16:03 |
| jgriffith | running as root | 16:03 |
| jgriffith | oh.. what? clouds.yaml | 16:03 |
| jgriffith | son of a | 16:03 |
| *** amotoki has quit IRC | 16:04 | |
| * jgriffith is a moron | 16:04 | |
| jgriffith | sorry.. and thanks clarkb | 16:04 |
| jgriffith | I just created ~/.config/openstack NOT ~/.config/openstack/clouds.yaml | 16:04 |
| jgriffith | that fixed it.. thank you very much | 16:05 |
| *** hashar has quit IRC | 16:05 | |
| jgriffith | mtreinish: not the hardware this time :) | 16:06 |
| *** apoorvad has joined #openstack-infra | 16:06 | |
| *** trown is now known as trown|lunch | 16:06 | |
| clarkb | jgriffith: np | 16:06 |
| *** austin81 has joined #openstack-infra | 16:06 | |
| mtreinish | jgriffith: hah, fair enough :) | 16:07 |
| *** fhubik has quit IRC | 16:08 | |
| *** roxanaghe has quit IRC | 16:08 | |
| *** ihrachys has quit IRC | 16:09 | |
| zaro | morning | 16:11 |
| fungi | jgriffith: could have been subtle corruption affecting the filesystem table, which had the effect of renaming ~/.config/openstack/clouds.yaml to ~/.config/openstack (then you can be blameless!) | 16:11 |
| jgriffith | fungi: I like the way you think!! In fact.. I"m certain that's EXACTLY what happened! | 16:12 |
| *** esikache1 has quit IRC | 16:12 | |
| *** mriedem is now known as mriedem_lunch | 16:12 | |
| jgriffith | fungi: it happens just like that ya know! | 16:12 |
| *** esikachev has joined #openstack-infra | 16:12 | |
| *** zzzeek has quit IRC | 16:13 | |
| * fungi is a terrible craftsman, always blames his tools | 16:13 | |
| jeblair | clarkb, fungi: we should land https://review.openstack.org/298432 before restarting nodepool | 16:13 |
| jgriffith | fungi: that saying is bunk! | 16:13 |
| clarkb | jeblair: looking | 16:13 |
| clarkb | oh yes I meant to approve that one as part of my reviewing yesterday | 16:14 |
| *** zzzeek has joined #openstack-infra | 16:14 | |
| fungi | lgtm | 16:14 |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Move tox-bandit to check as non voting https://review.openstack.org/300564 | 16:14 |
| clarkb | I forgot to do it after I reviewed the change that consumed that | 16:14 |
| *** sfinucan has quit IRC | 16:15 | |
| *** mylu has quit IRC | 16:15 | |
| *** jcoufal has quit IRC | 16:15 | |
| clarkb | the xenial host that nodepool booted is in rax and both network interfaces look happy | 16:15 |
| clarkb | and journalctl works | 16:16 |
| clarkb | such excite, this is really going to be the killer feature for dib long term. No longer dependent on providers publishing images | 16:16 |
| *** nadya has quit IRC | 16:17 | |
| clarkb | want to test on plan9? we can do that | 16:17 |
| *** jlanoux has quit IRC | 16:17 | |
| pabelanger | back, catching up on back scroll | 16:18 |
| pabelanger | clarkb: Yup, worked first time. Exciting | 16:19 |
| *** yolanda has quit IRC | 16:19 | |
| clarkb | /dev/mapper/main-opt 504G 468G 37G 93% /opt removing devstack-centos7 will help that | 16:20 |
| pleia2 | good morning | 16:20 |
| clarkb | morning | 16:20 |
| pabelanger | o/ | 16:20 |
| openstackgerrit | Merged openstack-infra/system-config: Add server and port caching settings for nodepool https://review.openstack.org/298432 | 16:20 |
| *** jlanoux has joined #openstack-infra | 16:20 | |
| *** roxanaghe has joined #openstack-infra | 16:21 | |
| clarkb | pabelanger: the other thing we will need to do before adding xenial to gating jobs is add it to the ubuntu mirror | 16:22 |
| anteaya | morning pleia2 | 16:24 |
| anteaya | I'm mostly not here today | 16:24 |
| anteaya | was up all night | 16:24 |
| jeblair | did we roll-out and then roll-back anything for meta-april-fools-day? | 16:24 |
| clarkb | I havne't april fools | 16:24 |
| *** e0ne has joined #openstack-infra | 16:24 | |
| anteaya | I didn't think anyone in here had any time | 16:25 |
| anteaya | neutron seems to be taking the lions share of the jokes this year | 16:25 |
| clarkb | but if anyone would like to be pranked where should I ship twin 9 month olds? they excel at this | 16:25 |
| *** salv-orlando has joined #openstack-infra | 16:25 | |
| anteaya | ha ha ha | 16:25 |
| anteaya | do they ship well? | 16:25 |
| *** jlanoux has quit IRC | 16:25 | |
| clarkb | probably not, they need constant care | 16:25 |
| pleia2 | anteaya: ugh, hope the flooding problem is a bit improved at least and you can finally get some rest | 16:26 |
| anteaya | yeah, had to get a courier to take that contract | 16:26 |
| clarkb | looks like we need ~45GB overhead to build an image | 16:26 |
| *** Qiming has quit IRC | 16:26 | |
| anteaya | pleia2: well the rain has stopped, so I hope the speed of the water coming in starts to slow by this evening | 16:26 |
| clarkb | and we are sitting at ~73GB free? so should be ok for now and the removal of devstack-centos7 will clean up about 40GB | 16:26 |
| anteaya | so that's hopeful | 16:26 |
| pleia2 | anteaya: good, I hope so :) | 16:26 |
| anteaya | thank you | 16:27 |
| anteaya | :) | 16:27 |
| anteaya | me too | 16:27 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add ubuntu xenial to reprepro https://review.openstack.org/300566 | 16:27 |
| pabelanger | clarkb: I think that is all we need ^ | 16:27 |
| pabelanger | just looked over the code | 16:27 |
| *** diana_clarke has joined #openstack-infra | 16:27 | |
| *** e0ne has quit IRC | 16:29 | |
| *** esker has quit IRC | 16:29 | |
| *** tsg has joined #openstack-infra | 16:29 | |
| clarkb | pabelanger: I think that is correct, everything else is templated to splat out per release | 16:29 |
| *** roxanaghe has quit IRC | 16:29 | |
| pabelanger | clarkb: Ya, that's what I see too | 16:29 |
| *** e0ne has joined #openstack-infra | 16:29 | |
| *** yolanda has joined #openstack-infra | 16:30 | |
| *** antonym has quit IRC | 16:31 | |
| *** openstackgerrit has quit IRC | 16:31 | |
| pabelanger | clarkb: just checked the afs volume too, we are currently at 83GB of 500GB usage | 16:31 |
| *** rcernin has quit IRC | 16:32 | |
| *** kzaitsev_mb has quit IRC | 16:32 | |
| clarkb | so should have plenty of room there | 16:32 |
| *** tphummel has joined #openstack-infra | 16:33 | |
| NobodyCam | morning INFRA folks | 16:34 |
| NobodyCam | is there a good beginner's guide to setting up a Ci system I can point folks to? | 16:36 |
| *** jsavak has joined #openstack-infra | 16:36 | |
| AJaeger | fungi, thanks for explanation about the chicken-egg problem. Now I understand. Sorry, pabelanger ! | 16:36 |
| fungi | jeblair: well, there is an amusing governance resolution courtesy of russellb, if you haven't seen it yet (300280) | 16:36 |
| fungi | but no, i don't think infra found time for pranks this year | 16:37 |
| fungi | saddened, as this is my favorite annual holiday | 16:37 |
| *** gyee has joined #openstack-infra | 16:37 | |
| fungi | NobodyCam: there is! | 16:37 |
| pleia2 | NobodyCam: http://docs.openstack.org/infra/openstackci/ | 16:37 |
| anteaya | gerrit 502'd on me, it has been a while since it did that | 16:38 |
| pleia2 | NobodyCam: that's the documentation for our puppet module that does it | 16:38 |
| jeblair | fungi: ha! thanks russellb! | 16:38 |
| *** achanda has joined #openstack-infra | 16:38 | |
| fungi | NobodyCam: http://docs.openstack.org/infra/system-config/third_party.html | 16:38 |
| NobodyCam | awesome thank you :) | 16:38 |
| fungi | NobodyCam: yes, also what pleia2 linked | 16:38 |
| *** e0ne has quit IRC | 16:39 | |
| anteaya | NobodyCam: the link fungi provided is required reading for anyone setting up a third party ci that interacts with our gerrit | 16:39 |
| russellb | jeblair: what did i do | 16:39 |
| *** amitgandhinz has quit IRC | 16:39 | |
| NobodyCam | :) | 16:39 |
| anteaya | NobodyCam: the link pleia2 provided is the documentation for how to set up the code | 16:39 |
| fungi | yep, for some reason i thought the thing i pasted included links to the thing pleia2 pasted, but maybe it doesn't yet | 16:39 |
| pleia2 | ah yes, depends on whether you're looking to do it just as a standalone CI for a project somewhere, third party is for interacting with our gerrit for tests in an openstack project that reports to us | 16:39 |
| fungi | though also i think the openstackci documentation/module don't cover setting up a gerrit yet as it was mostly starting from the third-party ci use case. it's at least on the road map to add though if it's not in there yet | 16:40 |
| pabelanger | #success Launched our first ubuntu-xenial job with nodepool! http://logs.openstack.org/26/300526/1/experimental/gate-project-config-bindep-fallback-ubuntu-xenial/4dca548/console.html | 16:41 |
| openstackstatus | pabelanger: Added success to Success page | 16:41 |
| * fungi cheers | 16:41 | |
| jeblair | russellb: you made me laugh :) 300280 | 16:41 |
| russellb | oh, yay | 16:41 |
| *** smarcet has quit IRC | 16:41 | |
| clarkb | pabelanger: it failed on the mirror :) ok that means we are on the right track adding that in | 16:42 |
| pabelanger | clarkb: indeed | 16:42 |
| pleia2 | fungi: yeah, we have the out-dated http://docs.openstack.org/infra/system-config/running-your-own.html that includes gerrit stuff, but it is a bit too stale (fortunately it warns as such) | 16:42 |
| fungi | pleia2: right, i purposefully didn't suggest that one | 16:42 |
| *** shardy has quit IRC | 16:42 | |
| pleia2 | fungi: nods | 16:42 |
| jeblair | the nodepool config change has landed, i'm going to restart it now | 16:43 |
| *** fedexo has joined #openstack-infra | 16:43 | |
| *** nwkarsten has quit IRC | 16:43 | |
| pabelanger | jeblair: fungi: https://review.openstack.org/#/c/300566/ all that is needed to bring our xenial mirrors online. Do you mind confirming | 16:43 |
| pabelanger | when you have a moment | 16:43 |
| *** rcarrillocruz has quit IRC | 16:43 | |
| *** nwkarsten has joined #openstack-infra | 16:43 | |
| *** antonym has joined #openstack-infra | 16:44 | |
| *** pcaruana has quit IRC | 16:44 | |
| jeblair | fungi, pabelanger: is it easy for one of you to delete leaked bluebox fips? | 16:44 |
| pabelanger | sure, I can | 16:44 |
| jeblair | pabelanger: thanks; (cc fungi) | 16:44 |
| *** openstackgerrit has joined #openstack-infra | 16:45 | |
| *** achanda has quit IRC | 16:45 | |
| pabelanger | fungi: jeblair: deleted on bluebox, will check osic too | 16:46 |
| clarkb | osic leaks more slowly | 16:46 |
| pabelanger | osic looks good | 16:46 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config: Ironic grenade jobs do not work until after Mitaka https://review.openstack.org/300082 | 16:46 |
| clarkb | if at all | 16:46 |
| *** mmedvede_ has quit IRC | 16:46 | |
| pabelanger | ya, I see 1 | 16:46 |
| *** yamamoto has joined #openstack-infra | 16:47 | |
| *** dtantsur is now known as dtantsur|afk | 16:47 | |
| *** nwkarsten has quit IRC | 16:48 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's canonicalGitUrl option https://review.openstack.org/300572 | 16:48 |
| fungi | i haven't seen fip leaks on osic before, fwiw | 16:49 |
| fungi | and we're hoping this restart means the last of the fip leaks on bluebox, yes? | 16:49 |
| *** roxanaghe has joined #openstack-infra | 16:50 | |
| *** fawadkhaliq has quit IRC | 16:50 | |
| *** edmondsw has quit IRC | 16:50 | |
| fungi | or at least renders them more transparent and debuggable going forward | 16:50 |
| jeblair | 2016-04-01 16:50:12,218 DEBUG nodepool.ProviderManager: Manager ovh-gra1 running task KeypairDelete (queue: 16) | 16:50 |
| jeblair | does that seem like something we should be doing ^ ? | 16:50 |
| *** fawadkhaliq has joined #openstack-infra | 16:50 | |
| clarkb | the change should mean that fips get deleted whenever the server they were attached to is deleted thus preventing leaks | 16:51 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Change http download command to point to git.o.o https://review.openstack.org/300187 | 16:51 |
| *** kzaitsev_mb has joined #openstack-infra | 16:51 | |
| clarkb | jeblair: its definitely something that happens when doing snapshot builds. ovh doesn't have snapshot builds | 16:51 |
| *** fedexo has quit IRC | 16:52 | |
| pabelanger | I think something is up with jenkins03.o.o | 16:52 |
| pabelanger | checking | 16:52 |
| *** yamamoto has quit IRC | 16:52 | |
| jeblair | clarkb: yeah, and even that should be in the builder now, not the main log | 16:52 |
| jeblair | also weird -- "KeypairDelete" is the only "Keypair" task i see in the log | 16:53 |
| *** tsg has quit IRC | 16:54 | |
| openstackgerrit | Merged openstack-infra/system-config: Add ubuntu xenial to reprepro https://review.openstack.org/300566 | 16:54 |
| pabelanger | thread count on jenkins03 looks to be okay, but it is running slow. Load average up to 4 | 16:54 |
| pabelanger | Oh | 16:55 |
| pabelanger | I think is just crashed | 16:55 |
| clarkb | jeblair: the only keypair code I see is in snapshot builds | 16:55 |
| *** e0ne has joined #openstack-infra | 16:55 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting gate-trove-python34-db check https://review.openstack.org/279108 | 16:55 |
| *** jsavak has quit IRC | 16:55 | |
| *** jsavak has joined #openstack-infra | 16:56 | |
| pabelanger | eep. I think I was the one who crashed jenkins03 | 16:56 |
| pabelanger | Current thread (0x00007fced001d000): JavaThread "Handling GET /monitoring from 23.233.28.9 : RequestHandlerThread[#608]" daemon [_thread_in_native, id=20860, stack(0x00007fce96ced000,0x00007fce96dee000)] | 16:56 |
| clarkb | it crashed? | 16:57 |
| jeblair | clarkb: ah, we call delete_keypair in cleanupServer() in nodepool | 16:57 |
| pabelanger | clarkb: Ya, check /tmp/hs_err_pid14369.log on jenkins03 | 16:57 |
| *** bpokorny has quit IRC | 16:57 | |
| *** kzaitsev_mb has quit IRC | 16:57 | |
| pabelanger | that is the 2nd jenkins master today to be down | 16:57 |
| openstackgerrit | Merged openstack-infra/project-config: i18n Support for freezer-web-ui https://review.openstack.org/300370 | 16:57 |
| *** tqtran has joined #openstack-infra | 16:57 | |
| pabelanger | jenkins05 this morning | 16:57 |
| clarkb | pabelanger: oh thats the I ran out of memory because thread stack ran out of space behavior | 16:58 |
| clarkb | pabelanger: it was probably leaking threads and couldn't make new ones if that was the case | 16:58 |
| pabelanger | ok | 16:58 |
| clarkb | you can check that log file and see how many htreads there were | 16:58 |
| *** bpokorny has joined #openstack-infra | 16:58 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Change git download command to point to git.o.o https://review.openstack.org/300576 | 16:58 |
| clarkb | hrm though in this case it is complaining of sigsegv | 16:59 |
| clarkb | fun | 16:59 |
| openstackgerrit | Merged openstack-infra/project-config: Add Ironic Inspector basic tempest job https://review.openstack.org/299821 | 16:59 |
| *** bpokorny has quit IRC | 16:59 | |
| jeblair | ah, i understand the keypair thing. explanation forthcoming. | 16:59 |
| *** bpokorny has joined #openstack-infra | 17:00 | |
| clarkb | and thread count looks reasonable | 17:00 |
| clarkb | so this may be new fun and exciting jenkins brokeness | 17:00 |
| pabelanger | clarkb: okay, I'll start the process of cleaning up nodes. jeblair is it okay to interact with nodepool or should I wait | 17:00 |
| clarkb | or monitoring plugin brokeness | 17:00 |
| clarkb | pabelanger: we might consider clearing out the java melody data on all the servers if we restart them | 17:00 |
| clarkb | since at least one had problems with that data during hte upgrade | 17:00 |
| pabelanger | clarkb: which folder is that? | 17:01 |
| *** jsavak has quit IRC | 17:01 | |
| clarkb | pabelanger: /var/lib/jenkins/monitoring | 17:01 |
| *** chem has quit IRC | 17:02 | |
| clarkb | secret.key.not-so-secret is my new favorite file name | 17:02 |
| jeblair | clarkb, mordred: http://git.openstack.org/cgit/openstack-infra/nodepool/diff/?id=e1f4a12949016e57db9b716b00f0f80f90329e74#n680 | 17:02 |
| pabelanger | clarkb: okay, will wait until current nodepool testing is over before starting the clean up | 17:02 |
| *** tosky has quit IRC | 17:02 | |
| jeblair | clarkb, mordred: we used to have a conditional around the keypair delete there | 17:02 |
| *** jsavak has joined #openstack-infra | 17:02 | |
| *** rbrndt has quit IRC | 17:02 | |
| jeblair | clarkb, mordred: without that, we're doing an extra API call for every server delete | 17:02 |
| *** eharney has quit IRC | 17:02 | |
| clarkb | jeblair: nice | 17:02 |
| fungi | which is hopefully not crippling but would still be good to skip? | 17:03 |
| clarkb | jeblair: so we just need to add back in the check around server key_name? | 17:03 |
| mordred | jeblair: which api call are we making? | 17:04 |
| jeblair | clarkb: ideally, but i don't know what else may have changed wrt to that...(does nodepool still create the keys or did we shade that; i haven't gotten there yet) | 17:04 |
| jeblair | mordred: delete key pair | 17:04 |
| jeblair | KeypairDelete task | 17:04 |
| mordred | oh - the if server['key_name'] == self.provider.keypair yeah? | 17:05 |
| *** aysyd has quit IRC | 17:05 | |
| mordred | the self.hasExtension conditional doesn't need to be there - but the other one could certainly be there, yeah | 17:05 |
| jeblair | fungi: yeah, we're mostly keeping up (provider task queues are low) so it's not crippling, just extra load, which at nodepool scale is a real thing | 17:05 |
| jeblair | mordred: yep | 17:06 |
| mordred | jeblair: the keypair create logic should be the same | 17:06 |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's gitHttpUrl option https://review.openstack.org/300160 | 17:06 |
| mordred | jeblair: I'm guessing you're on it - let me know if you want me to look at a thing | 17:07 |
| jeblair | mordred: yep, i'm still making sure i have full context, but can write a patch when i am. | 17:07 |
| mordred | cool | 17:07 |
| mordred | I will review it | 17:07 |
| anteaya | according to our etherpad we have 4 workrooms, 3 fishbowl and a day of meetup: https://etherpad.openstack.org/p/infra-newton-summit-planning | 17:07 |
| mordred | also, happy to hear it's keeping up and otherwise mostly working | 17:08 |
| anteaya | I think the current schedule is missing some sessions: https://www.openstack.org/summit/austin-2016/summit-schedule/#day=2016-04-27&tags=3536 | 17:08 |
| fungi | anteaya: what sessions specifically do you think are missing? | 17:09 |
| *** trown|lunch is now known as trown | 17:09 | |
| anteaya | well I see 2 workroom and 2 fishbowl in my browser at that link | 17:09 |
| anteaya | so 2 of the workroom and 1 of the fishbowl | 17:09 |
| anteaya | not sure if meetup is expected to be on that schedule or not | 17:10 |
| fungi | double-checking. i saw them on the pdf versions but hadn't looked at the final on the www site yet | 17:10 |
| *** mrmartin has joined #openstack-infra | 17:10 | |
| anteaya | sorry my fault | 17:11 |
| anteaya | that ws just for wednesday | 17:11 |
| *** shashank_hegde has joined #openstack-infra | 17:11 | |
| anteaya | my fault my fault | 17:11 |
| fungi | yep, we have sessions on wednesday/thursday and then sprint friday | 17:11 |
| *** yamahata has quit IRC | 17:11 | |
| anteaya | in the past the project track filter gave you the whole week, not day at a time | 17:11 |
| anteaya | sorry about that | 17:11 |
| diana_clarke | Hi folks, where can I see a list of passing and failing runs for a particular job in the check pipeline (before I "recheck" something)? | 17:12 |
| jeblair | mordred: will server['key_name'] exist on a server returned from a cloud without the keypair extension (or whatever is equivalent to that now)? | 17:12 |
| diana_clarke | I've found the "OpenStack Health" tab for the "gate" pipeline (rather than "check"). My apologies if I'm missing the obvious. | 17:12 |
| stevemar | is there a page where infra has a list of projects for interns/students? | 17:12 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Add post job for building and uploading tinyipa https://review.openstack.org/299485 | 17:12 |
| anteaya | stevemar: if the wiki still has a outreachy page | 17:12 |
| anteaya | I'd start there | 17:12 |
| pleia2 | stevemar: there are a couple places where folks can start | 17:12 |
| anteaya | diana_clarke: yes health is just on gating | 17:12 |
| pleia2 | stevemar: they can have a look at http://specs.openstack.org/openstack-infra/infra-specs/ to see if there are any major projects we're working on that are of interest to them | 17:13 |
| pleia2 | stevemar: I've also started seeing about doing more tagging of our bugs in storyboard | 17:13 |
| stevemar | pleia2: trying to think of an idea for a full stack developer, i'll take a peak | 17:13 |
| mordred | jeblair: there is no such thing as a cloud without the keypair extension anymore | 17:14 |
| anteaya | diana_clarke: use this and feed in a job name: https://github.com/sdague/failopotamus | 17:14 |
| fungi | stevemar: the last outreacy intern we had, we came up with a suitably focused/scoped/bounded project and wrote a spec detailing what needed to be done | 17:14 |
| anteaya | diana_clarke: it will open your browser with a graph from graphite for the job | 17:14 |
| jeblair | mordred: so that entry will always be in the server dict? | 17:14 |
| pabelanger | diana_clarke: zuul will post the results in review.o.o patchset too for each gate. Be sure to have the 'Toggle CI' button in right way | 17:14 |
| pabelanger | diana_clarke: sorry, jenkins user | 17:14 |
| mordred | jeblair: should be. but let me emperically check that real quick | 17:15 |
| pleia2 | stevemar: ah, for all of openstack, not just infra? | 17:15 |
| *** e0ne has quit IRC | 17:15 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Only delete keypairs if needed https://review.openstack.org/300583 | 17:15 |
| fungi | stevemar: right, if you're looking more for an example to get you started planning/scoping a task, the last one we did for an infra intern was http://specs.openstack.org/openstack-infra/infra-specs/specs/code-search.html | 17:16 |
| jeblair | mordred: i went aheand and wrote it the 'safe' way, but if you think we shoud make it unsafe to fail hard on an unexpected condition, i can redo ^ | 17:16 |
| mordred | jeblair: ok. well, I think shade should ensure the key is always in the dict regardless | 17:16 |
| mordred | which ist can do in normalize _servers - but I think the safe way is fine for now | 17:16 |
| pleia2 | stevemar: if you are focused on infra if you go to https://storyboard.openstack.org/#!/search and type in low-hanging-fruit and select the tag looking one, then add openstack-ci and look for the 3 boxes, then type active to find Story Status: Active; that will list all the open infra bugs we at some point thought would be suitable for newcomers (which I'd say means students too) | 17:16 |
| diana_clarke | pabelanger: Yup, but I want to know if it's failing for a bunch of people before I enter "recheck". Normally, I just check other reviews around the same time, but I figure there must be a better way. | 17:17 |
| diana_clarke | anteaya: Thanks! I'll give that a go. | 17:17 |
| anteaya | diana_clarke: I've found it useful for looking at a single job | 17:17 |
| *** mrmartin has quit IRC | 17:18 | |
| pabelanger | diana_clarke: Ah, ya. I tend to look at the project view in gerrit and see if there are a bunch of recent -1 on reviews. If so, dive deeper into gerrit | 17:18 |
| *** yolanda has quit IRC | 17:19 | |
| *** mrmartin has joined #openstack-infra | 17:19 | |
| *** davideagnello has joined #openstack-infra | 17:19 | |
| *** amitgandhinz has joined #openstack-infra | 17:19 | |
| *** cdent has joined #openstack-infra | 17:20 | |
| mordred | jeblair: +2 | 17:22 |
| *** eharney has joined #openstack-infra | 17:22 | |
| jeblair | we leaked a bunch of nodes in rax-ord with no metadata | 17:23 |
| jeblair | this apparently happened a while ago | 17:23 |
| jeblair | mar 25 | 17:23 |
| *** mrmartin has quit IRC | 17:23 | |
| jeblair | i will nova delete them | 17:23 |
| *** mriedem_lunch is now known as mriedem | 17:24 | |
| mordred | jeblair: I have verified that we get key_name in the server dict even if we did not launch the node with a keypair | 17:24 |
| *** imcsk8|plane is now known as imcsk8 | 17:24 | |
| clarkb | I approved the change so if testing is happy with it it should be mergedin around an hour | 17:25 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's canonicalGitUrl option https://review.openstack.org/300572 | 17:25 |
| mordred | jeblair: we do not ensure that key_anme is in the dict in normalize_server though | 17:25 |
| pabelanger | jeblair: if you are deleting nodes, jenkins03 is down any existing nodes can be removed from nodepool | 17:25 |
| *** jsavak has left #openstack-infra | 17:26 | |
| *** sambetts is now known as sambetts|afk | 17:26 | |
| jeblair | pabelanger: done | 17:27 |
| pabelanger | danke | 17:27 |
| *** sputnik13 has joined #openstack-infra | 17:28 | |
| zaro | clarkb: hey, you have time to take a look at a change to enable puppet code highlighting in gerrit? https://review.openstack.org/#/c/260324/ | 17:28 |
| anteaya | I just submitted https://bugs.launchpad.net/openstack-org/+bug/1565018 | 17:29 |
| openstack | Launchpad bug 1565018 in openstack-org "Austin Summit Schedule: Project Design Summit Track view please" [Undecided,New] | 17:29 |
| jeblair | 2016-04-01 17:28:52,274 ERROR nodepool.NodeLauncher: OpenStackCloudException launching node id: 9278519 in provider: tripleo-test-cloud-rh1 error: | 17:29 |
| jeblair | OpenStackCloudException: unable to create floating IP for net None: Bad floatingip request: Cannot add floating IP to port on subnet f6603146-7ed8-4e09-8e64-f7e17dde8704 which has no gateway_ip | 17:29 |
| jeblair | Neutron server returns request_ids: ['req-f1594bd5-6a79-4869-9973-428694b4cce7'] | 17:29 |
| jeblair | mordred, clarkb: ^ | 17:29 |
| jeblair | mordred: does that mean that we wanted the second network instead of the first? | 17:30 |
| clarkb | jeblair: in this particular case yes, I do not think that is generally true | 17:30 |
| pabelanger | clarkb: so you are fine with me deleting the content of /var/lib/jenkins/monitoring? | 17:30 |
| clarkb | so the work around is too simple to trick neutron | 17:30 |
| clarkb | pabelanger: ya I moved it aside on the host that broke during the upgrade | 17:30 |
| clarkb | pabelanger: in case we needed or wanted to dig into why it might be causing problems | 17:30 |
| openstackgerrit | Merged openstack-infra/shade: Remove conditional blocking on server list https://review.openstack.org/299672 | 17:30 |
| jeblair | clarkb: yeah. i think mordred had a suspicion that nova must have been doing something like 'pick the first' for this to work at all using the nova api... but maybe there's more magic in there than suspected. | 17:31 |
| clarkb | I think when we did it before nova/neutron would be smart enough to find the port htat has a gateway and use that one | 17:31 |
| jeblair | that would be sufficient magic, it seems | 17:31 |
| clarkb | but when you explicitly list the port you have to do that yourselves | 17:31 |
| pabelanger | clarkb: okay, purging directory | 17:32 |
| fungi | anteaya: apparently non-intuitive but one of the schedule app devs suggested searching like this to get the full-week schedule for a specific team's design summit sessions... https://www.openstack.org/summit/austin-2016/summit-schedule/global-search?t=Infrastructure: | 17:32 |
| anteaya | thanks I will try that | 17:32 |
| fungi | seemed to work for me, fwiw | 17:32 |
| mordred | jeblair: ah! oh awesome | 17:32 |
| mordred | that is sufficient magic - we cna do the same thing | 17:33 |
| *** nwkarsten has joined #openstack-infra | 17:33 | |
| mordred | jeblair: in fact, I can rework my follow up patch to just do that | 17:33 |
| pabelanger | jenkins03.o.o back online, running JJB now | 17:33 |
| anteaya | fungi: yes, that is what I want | 17:33 |
| anteaya | thank you | 17:33 |
| *** skolekonov is now known as skolekonov|away | 17:33 | |
| jeblair | mordred: do you want to go down that path, or the occ path? | 17:33 |
| *** mriedem has quit IRC | 17:33 | |
| anteaya | I'll append that to my bug | 17:33 |
| mordred | jeblair: I want to go down both paths | 17:33 |
| *** nadya has joined #openstack-infra | 17:34 | |
| *** mriedem has joined #openstack-infra | 17:34 | |
| mordred | jeblair: if we can discover and avoid the need for config, great. but golly what if someone makes two networks with gateway_ips - so we will likely need a way to override the discovery | 17:35 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Consume config values for NAT destination https://review.openstack.org/300592 | 17:36 |
| jeblair | mordred: yeah. and to be fair, that is reasonable. (the general problem is not that openstack allows complexity, but that it does not support simplicity) | 17:36 |
| mordred | jeblair: there is the the occ patch | 17:36 |
| mordred | jeblair: minus tests | 17:36 |
| *** salv-orlando has quit IRC | 17:36 | |
| anteaya | fungi: I see you put Infrastructure: in the search field | 17:36 |
| mordred | jeblair: let me start working on a follow up to it that will do the discovery | 17:36 |
| jeblair | mordred: do you think we should rollback nodepool? cause either solution will require either an occ or shade release, yeah? | 17:37 |
| mordred | since the structures put into place in that patch are still helpful | 17:37 |
| mordred | jeblair: yes. I do think we should rollback nodepool | 17:37 |
| jeblair | ok. will do now. | 17:37 |
| mordred | but I'm excited that a) other than that it's working and b) I actually understand a good fix for this now :) | 17:37 |
| fungi | anteaya: yep, basically just taking advantage of the fact that all design summit sessions start with the team name followed by a colon, while conference sessions (mostly) do not. they're working on improving this for barcelona a bit apparently but this will suffice for now | 17:38 |
| *** nwkarsten has quit IRC | 17:38 | |
| jeblair | mordred: yeah, the punch-list is getting short | 17:38 |
| jeblair | nodepool has been restarted as pre-shade | 17:38 |
| *** yamahata has joined #openstack-infra | 17:39 | |
| mordred | jeblair: cool. thanks! | 17:40 |
| *** burgerk has quit IRC | 17:40 | |
| anteaya | fungi: cool, now that I know how it works, the magical colon | 17:41 |
| anteaya | fungi: do they have any objection to me posting about the colon to the -dev list? | 17:41 |
| anteaya | or does anyone else? | 17:41 |
| mordred | jeblair: I think we can test a create_server(auto_ip=True) against the tripleo cloud before we re-try the nodepool restart | 17:41 |
| fungi | anteaya: i can't imagine anyone would object to that | 17:41 |
| anteaya | I can't be the only one that wants this | 17:41 |
| anteaya | fungi: awesome | 17:42 |
| * anteaya composes | 17:42 | |
| *** salv-orlando has joined #openstack-infra | 17:45 | |
| clarkb | jeblair: looks like there may be image build cleanup necessary due to the restarts :/ | 17:46 |
| anteaya | my email draft if anyone cares to review: https://etherpad.openstack.org/p/Vp4ZL1sCXs | 17:48 |
| *** yamamoto has joined #openstack-infra | 17:48 | |
| jeblair | clarkb: hrm. i thought we fixed that? | 17:48 |
| clarkb | jeblair: I thought so to, but for some reason the building images aren't going away. If we restart the builder they do | 17:49 |
| clarkb | I even thought I wrote a test for this | 17:49 |
| clarkb | maybe that change never merged? | 17:49 |
| clarkb | https://review.openstack.org/#/c/226114/ it merge | 17:50 |
| clarkb | oh! | 17:50 |
| clarkb | ew have two image tables | 17:50 |
| clarkb | this cleans up one of them | 17:50 |
| clarkb | need to do the other | 17:50 |
| clarkb | /ew/we/ | 17:51 |
| *** nwkarsten has joined #openstack-infra | 17:51 | |
| *** fawadkhaliq has quit IRC | 17:52 | |
| *** fawadkhaliq has joined #openstack-infra | 17:52 | |
| *** markvoelker has quit IRC | 17:52 | |
| *** yamamoto has quit IRC | 17:52 | |
| jeblair | clarkb: when that patch was written, it was sufficient. :) | 17:52 |
| jeblair | but yeah | 17:53 |
| *** austin81 has left #openstack-infra | 17:53 | |
| anteaya | last call for feedback on an email to the list about how to search the design summit app: https://etherpad.openstack.org/p/Vp4ZL1sCXs | 17:55 |
| anteaya | thanks to those who have read so far | 17:55 |
| *** markvoelker has joined #openstack-infra | 17:56 | |
| *** rbrndt has joined #openstack-infra | 17:56 | |
| fungi | anteaya: lgtm | 17:56 |
| anteaya | fungi: thank you | 17:57 |
| *** nhandler is now known as StaffUnicorn | 17:57 | |
| *** jistr has quit IRC | 17:58 | |
| pabelanger | JJB finished on jenkins03 now | 17:59 |
| anteaya | typo in the subject line, d'oh | 18:00 |
| *** flepied has quit IRC | 18:00 | |
| mnaser | things should be much improved on our side | 18:00 |
| anteaya | yay | 18:01 |
| *** tsg has joined #openstack-infra | 18:01 | |
| anteaya | improvements | 18:01 |
| *** asselin__ has joined #openstack-infra | 18:02 | |
| *** harlowja has quit IRC | 18:02 | |
| *** jed56 has quit IRC | 18:03 | |
| *** nadya has quit IRC | 18:03 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's download protocols https://review.openstack.org/300596 | 18:04 |
| *** piet has joined #openstack-infra | 18:05 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Enable DB access for Designate unittests https://review.openstack.org/300537 | 18:05 |
| *** pvaneck has joined #openstack-infra | 18:06 | |
| *** ybathia has joined #openstack-infra | 18:07 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's canonicalGitUrl option https://review.openstack.org/300572 | 18:07 |
| *** dtardivel has quit IRC | 18:08 | |
| openstackgerrit | Merged openstack-infra/nodepool: Only delete keypairs if needed https://review.openstack.org/300583 | 18:08 |
| *** jsavak has joined #openstack-infra | 18:08 | |
| *** bpokorny has quit IRC | 18:08 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add trailing whitespace to reprepo distributions https://review.openstack.org/300597 | 18:09 |
| pabelanger | fungi: clarkb: fix a syntax error in reprepro configuration ^ | 18:09 |
| *** gordc has quit IRC | 18:09 | |
| pabelanger | confirmed it fixes mirror-update | 18:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Search subnets for gateway_ip to discover NAT dest https://review.openstack.org/300598 | 18:09 |
| *** bpokorny has joined #openstack-infra | 18:10 | |
| mordred | jeblair: ^^ that is not the final version of the patch | 18:10 |
| mordred | I just wanted to get it up there | 18:10 |
| *** asselin__ has quit IRC | 18:10 | |
| mordred | I actually want to poke at the occ logic for config a little before we land this | 18:10 |
| *** smarcet has joined #openstack-infra | 18:11 | |
| *** nihilifer has quit IRC | 18:12 | |
| *** nihilifer has joined #openstack-infra | 18:14 | |
| *** nihilifer has quit IRC | 18:14 | |
| *** nihilifer has joined #openstack-infra | 18:15 | |
| mordred | Shrews: ^^ fyi - but I've marked the first patch WIP because I want to rework what we did in OCC the other day before we release it | 18:15 |
| *** piet has quit IRC | 18:15 | |
| mordred | Shrews: writing this caused me to realize that I _think_ we might be conflating two different issues and want to noodle on it when i'm not streaming through my phonein the passenger seat of a car | 18:15 |
| jeblair | mordred: left some minor comments | 18:17 |
| * AJaeger signs off and wishes a great weekend... | 18:17 | |
| *** lezbar has quit IRC | 18:17 | |
| anteaya | AJaeger: thanks you too | 18:17 |
| *** daemontool has quit IRC | 18:18 | |
| morgan | mordred: so i think ZK dogpile stuff is just going t be in dogpile.core | 18:18 |
| morgan | mordred: which is nice. | 18:18 |
| fungi | have a great weekend, AJaeger! | 18:18 |
| morgan | mordred: since the mutex can be defined there. | 18:19 |
| fungi | need to pop out for groceries before the tourists turn feral. back shortly | 18:19 |
| *** asselin has joined #openstack-infra | 18:19 | |
| morgan | mordred: also there is a push to move dogpile.core and dogpile.cache into a single dogpile package because of namespace things (same issue with oslo.*) | 18:19 |
| EmilienM | I don't think this patch deserves a -2, but I might be wrong https://review.openstack.org/#/c/297469/ | 18:19 |
| morgan | cc jeblair ^ (re nodepool things and dogpile) | 18:19 |
| morgan | EmilienM: it doesn't have a -2. | 18:20 |
| EmilienM | morgan: it has WIP, indeed but to me it is not a WIP | 18:20 |
| morgan | EmilienM: it has a -1 Workflow until mitaka is released. | 18:20 |
| morgan | EmilienM: according to the comment. | 18:21 |
| mordred | morgan: oh good | 18:21 |
| EmilienM | morgan: I can read it too :-) | 18:21 |
| morgan | EmilienM: ;) i know! | 18:21 |
| EmilienM | I just think, (and again, imo), we should not block patches because a release is happening. | 18:21 |
| morgan | EmilienM: hm. thats sticky | 18:21 |
| *** baoli_ has quit IRC | 18:22 | |
| jeblair | EmilienM: we have in the past nearly frozen (we called it 'slush') infrastructure around release because of the potential disruption | 18:22 |
| morgan | EmilienM: i think the alternative is a stable/<release> or similar branch to apply fixes pre-release to | 18:22 |
| *** IlyaG has quit IRC | 18:22 | |
| morgan | and with puppet-* that iirc is much harder | 18:22 |
| morgan | jeblair: ++ | 18:22 |
| jeblair | EmilienM: because the tarball script is baked into images, if something goes wrong there, it will take quite some time to correct (probably a full day for image rebuilds) | 18:23 |
| EmilienM | jeblair: fair enough, I'll wait :-) I need to be patient ;-) | 18:23 |
| morgan | oh this is project-config :P | 18:23 |
| pabelanger | EmilienM: If fungi wants to remove the WIP, I have no objections. I just didn't want it to land and break our run-tarball.sh when mitaka is being released | 18:23 |
| morgan | jeblair: can we make snowcones with the slush? | 18:23 |
| *** eil397 has joined #openstack-infra | 18:23 | |
| pabelanger | rather error on the side of caution | 18:24 |
| EmilienM | pabelanger: np. Just wanted to understand the background and I got it. | 18:24 |
| jeblair | morgan: yes, but i'd stay away from the yellow flavors | 18:24 |
| morgan | jeblair: good idea! | 18:24 |
| pabelanger | EmilienM: Ya, once neutron is officially open, we can land it | 18:24 |
| EmilienM | cool! | 18:24 |
| pabelanger | err, mitaka is release | 18:24 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Customize Gerrit's download command and schema https://review.openstack.org/300603 | 18:24 |
| Shrews | mordred: i left you some comments. i have to admit, even i'm starting to get befuddled by all of the various network options | 18:25 |
| morgan | Shrews: sooooo many network options | 18:26 |
| morgan | i did a by-hand openstack deploy | 18:26 |
| morgan | man... | 18:26 |
| morgan | was crazy to find all the infos. | 18:26 |
| Shrews | morgan: i meant just in shade | 18:26 |
| *** asselin__ has joined #openstack-infra | 18:27 | |
| mordred | Shrews: yah. | 18:28 |
| mordred | Shrews: I need to rework the config and then do a really clear doc about it | 18:28 |
| mordred | Shrews: but I think I understand the landscape well enough to write that doc now | 18:28 |
| Shrews | mordred: i look forward to reading it with a nice glass of wine and various soft cheeses | 18:29 |
| mordred | Shrews: mmm. cheese | 18:29 |
| Shrews | mordred: or maybe just beer and slim jims | 18:29 |
| *** asselin has quit IRC | 18:29 | |
| Shrews | yeah... that second thing | 18:29 |
| *** piet has joined #openstack-infra | 18:30 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's canonicalGitUrl option https://review.openstack.org/300572 | 18:31 |
| openstackgerrit | Ronald Bradford proposed openstack/requirements: Bump oslo.context to 2.2.0 https://review.openstack.org/300607 | 18:31 |
| *** flepied has joined #openstack-infra | 18:31 | |
| *** markvoelker has quit IRC | 18:31 | |
| *** edmondsw has joined #openstack-infra | 18:32 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Enable DB access for Designate unittests https://review.openstack.org/300537 | 18:34 |
| *** kushal has quit IRC | 18:35 | |
| *** baoli has joined #openstack-infra | 18:35 | |
| openstackgerrit | Merged openstack-infra/shade: Make get_stack fetch a single full stack https://review.openstack.org/292590 | 18:36 |
| *** dalgaaf has quit IRC | 18:38 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit's download protocols https://review.openstack.org/300596 | 18:40 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Customize Gerrit's download command and schema https://review.openstack.org/300603 | 18:42 |
| *** ifarkas has quit IRC | 18:43 | |
| *** keedya has quit IRC | 18:45 | |
| pabelanger | mordred: have a moment for https://review.openstack.org/#/c/300597/ | 18:47 |
| *** gordc has joined #openstack-infra | 18:47 | |
| mordred | pabelanger: looing | 18:47 |
| mordred | lgtm | 18:48 |
| *** shashank_hegde has quit IRC | 18:49 | |
| *** kushal has joined #openstack-infra | 18:49 | |
| *** yamamoto has joined #openstack-infra | 18:49 | |
| *** elo has quit IRC | 18:49 | |
| *** Sukhdev has joined #openstack-infra | 18:53 | |
| *** david-lyle has quit IRC | 18:53 | |
| *** aysyd has joined #openstack-infra | 18:53 | |
| *** markvoelker has joined #openstack-infra | 18:53 | |
| *** markvoelker has quit IRC | 18:53 | |
| *** markvoelker has joined #openstack-infra | 18:54 | |
| *** markvoelker has quit IRC | 18:54 | |
| *** bpokorny has quit IRC | 18:54 | |
| *** yamamoto has quit IRC | 18:54 | |
| *** markvoelker has joined #openstack-infra | 18:54 | |
| openstackgerrit | Merged openstack-infra/system-config: Add trailing whitespace to reprepo distributions https://review.openstack.org/300597 | 18:55 |
| *** esikachev has quit IRC | 18:55 | |
| *** piet has quit IRC | 18:56 | |
| dstufft | clarkb: https://github.com/blog/2141-squash-your-commits ;) | 18:56 |
| clarkb | heh | 18:56 |
| clarkb | (gerrit doesn't actually require squashed commits ... but everyone seems to interpret it that way) | 18:57 |
| dstufft | yea, it just requires you to review each commit | 18:57 |
| dstufft | which leads people to squash anyways I think most of the time | 18:58 |
| clarkb | I dunno I almost never squash | 18:58 |
| *** piet has joined #openstack-infra | 18:58 | |
| clarkb | usually its more due to gate reasons (this won't work unless 3 bugs are fixed beacuse why) situations | 18:58 |
| *** degorenko is now known as _degorenko|afk | 18:58 | |
| clarkb | and those tend to be rare | 18:59 |
| *** david-lyle has joined #openstack-infra | 18:59 | |
| *** Goneri has quit IRC | 18:59 | |
| *** Sukhdev has quit IRC | 19:00 | |
| clarkb | jeblair: should I clean up the nodepool db now? | 19:01 |
| *** piet has quit IRC | 19:03 | |
| *** fawadkhaliq has quit IRC | 19:05 | |
| clarkb | pabelanger: also the xenial mirror change merged but I don't see it yet, that might need intervention? | 19:05 |
| *** matt-borland has quit IRC | 19:05 | |
| clarkb | I am not sure how to check on reprepro progress | 19:06 |
| *** annegentle has quit IRC | 19:06 | |
| *** fawadkhaliq has joined #openstack-infra | 19:06 | |
| *** fawadkhaliq has quit IRC | 19:07 | |
| *** salv-orlando has quit IRC | 19:07 | |
| *** shashank_hegde has joined #openstack-infra | 19:10 | |
| *** tsg has quit IRC | 19:10 | |
| *** annegentle has joined #openstack-infra | 19:11 | |
| *** jsavak has quit IRC | 19:11 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Enable DB access for Designate unittests https://review.openstack.org/300537 | 19:12 |
| *** dimtruck is now known as zz_dimtruck | 19:12 | |
| *** jsavak has joined #openstack-infra | 19:13 | |
| *** fawadkhaliq has joined #openstack-infra | 19:13 | |
| pabelanger | clarkb: Ya, watching now. It is a crontab | 19:15 |
| pabelanger | clarkb: I think mordred wanted it to run every 2 hours right? | 19:17 |
| pabelanger | what actually happens is it runs every 1min each 2 hours | 19:17 |
| pabelanger | so, we need to update the crontab | 19:17 |
| pabelanger | for example: http://paste.openstack.org/show/492773/ | 19:17 |
| clayton | it appears that our internal nodepool instance got updated and now it's no longer trying to associate FIPs | 19:18 |
| *** baoli has quit IRC | 19:19 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Run 'reprepro ubuntu' cron once ever 2 hours https://review.openstack.org/300620 | 19:20 |
| pabelanger | clayton: there have been some new commit to nodepool using shade. If you want to review the commit history | 19:21 |
| clayton | been looking, nothing obvious so far | 19:21 |
| pabelanger | clayton: I _think_ there is some new syntax too for public / floating IP networks | 19:21 |
| pabelanger | but mordred and jeblair would know more | 19:21 |
| *** hashar has joined #openstack-infra | 19:22 | |
| mordred | clayton: you may want to revert nodepool to afdd58c10a8b85ea1466d32c3a29c0d7a1b0aea9 | 19:23 |
| clayton | trying that. it's running the CreateFloatingIPTask, which is a plus | 19:25 |
| openstackgerrit | Merged openstack/requirements: Add tempest to requirements https://review.openstack.org/300263 | 19:26 |
| *** nadya has joined #openstack-infra | 19:27 | |
| *** nwkarsten has quit IRC | 19:27 | |
| *** nwkarsten has joined #openstack-infra | 19:27 | |
| *** baoli has joined #openstack-infra | 19:29 | |
| *** xarses has joined #openstack-infra | 19:30 | |
| xarses | How can I clone node pool images outside of CI? (https://github.com/openstack/puppet-ceph/blob/master/spec/acceptance/nodesets/nodepool-centos7.yml) has no url so beaker locally doesn't like me | 19:30 |
| pabelanger | clarkb: mirror should be updating in 30mins | 19:31 |
| xarses | I can use a vagrant image https://github.com/openstack/puppet-ceph/blob/master/spec/acceptance/nodesets/centos-70-x64.yml but it returns a different result as the CI instance | 19:31 |
| *** esikachev has joined #openstack-infra | 19:32 | |
| *** nwkarsten has quit IRC | 19:32 | |
| clarkb | pabelanger: I need to fix that cron in my change too | 19:32 |
| pabelanger | xarses: in openstack-infra/project-config there is a build script. You could do DISTRO=centos-minimal ./tools/build-image.sh | 19:32 |
| xarses | pabelanger: we don't have a URL we can clone it externally from? | 19:34 |
| *** roxanaghe has quit IRC | 19:34 | |
| xarses | pabelanger: I'll give that a shot | 19:34 |
| pabelanger | xarses: https://git.openstack.org/cgit/openstack-infra/project-config/ | 19:34 |
| xarses | pabelanger: sorry, the image that is already being used in CI, I have the project-config repo | 19:35 |
| *** ybathia has quit IRC | 19:36 | |
| *** harlowja has joined #openstack-infra | 19:37 | |
| pabelanger | xarses: right, you need to use diskimage-builder to create the images we use | 19:37 |
| pabelanger | xarses: with the tools/build-image.sh script does for you | 19:37 |
| *** esikachev has quit IRC | 19:38 | |
| xarses | pabelanger: so the answer is no, we don't have a public URL to clone the current image from, it has to be built by hand? | 19:38 |
| pabelanger | xarses: right, no public images. | 19:38 |
| xarses | ack | 19:38 |
| *** jsavak has quit IRC | 19:39 | |
| *** auggy is now known as auggy_rodin | 19:39 | |
| clarkb | pabelanger: jeblair I will clean up nodepool db situation after lunch | 19:40 |
| pabelanger | clarkb: ack | 19:40 |
| *** fawadkhaliq has quit IRC | 19:43 | |
| *** baoli has quit IRC | 19:43 | |
| *** fawadkhaliq has joined #openstack-infra | 19:44 | |
| *** fawadkhaliq has quit IRC | 19:45 | |
| *** fawadkhaliq has joined #openstack-infra | 19:46 | |
| *** elo has joined #openstack-infra | 19:48 | |
| *** salv-orlando has joined #openstack-infra | 19:48 | |
| xarses | so where am I supposed to get diskimage-builder from? | 19:49 |
| xarses | pip? | 19:49 |
| clarkb | yes | 19:50 |
| *** yamamoto has joined #openstack-infra | 19:50 | |
| fungi | xarses: if you installed nodepool (did you say you're using nodepool, right?) then diskimage-builder is a prerequisite. we have a puppet-nodepool module you can use to install nodepool and all its dependencies, or you can pip install nodepool and it should pull in diskimage-builder as a dependency for you https://git.openstack.org/cgit/openstack-infra/nodepool/tree/requirements.txt#n19 | 19:51 |
| fungi | if you're not using nodepool, you can still pip install diskimagebuilder | 19:51 |
| fungi | er, pip install diskimage-builder | 19:51 |
| xarses | fungi: as probably derived, I have no clue what I'm doing here | 19:52 |
| *** Sukhdev has joined #openstack-infra | 19:52 | |
| fungi | xarses: what is it you ultimately want to use/do? | 19:52 |
| fungi | xarses: are you setting up a ci system? | 19:52 |
| fungi | and what kind of configuration management framework are you using (if any)? | 19:53 |
| xarses | I need an image that I can hand to beaker so that I can diagnose why the CI image fails my project's CI | 19:53 |
| xarses | for a specific change | 19:53 |
| xarses | basically, I need a live instance I can troubleshoot | 19:54 |
| fungi | oh, this may be a question for the #openstack-puppet channel (what change are you diagnosing?) | 19:54 |
| *** jsavak has joined #openstack-infra | 19:54 | |
| fungi | xarses: i'm guessing this is some magic being used by the puppet-ceph tests | 19:54 |
| xarses | https://review.openstack.org/#/c/299773/ | 19:54 |
| *** baoli has joined #openstack-infra | 19:55 | |
| *** yamamoto has quit IRC | 19:55 | |
| xarses | changing the puppet service handler from init -> redhat breaks redhat, but it's only happening this way on the CI instance, if I use the companion vagrant image, it doesn't have the same failure | 19:55 |
| fungi | xarses: got it. and so you want to be able to run the same job manually on an image similar to what we tested on (our centos-minimal based diskimage-builder images) | 19:57 |
| *** annegentle has quit IRC | 19:57 | |
| fungi | for a moment i thought you were trying to run something which consumes an image, so was off on a completely different tangent | 19:57 |
| xarses | oh, no the former | 19:58 |
| fungi | xarses: in that case pabelanger was on the right track. you want diskimage-builder from somewhere but probably pip install latest release from pypi will get you closest to what we're using | 19:59 |
| fungi | and the http://git.openstack.org/cgit/openstack-infra/project-config/tree/tools/build-image.sh script to build an image | 19:59 |
| clarkb | ok nodepool is cleaned up | 20:00 |
| fungi | jroll was, i think, just doing the same earlier this week and may have some suggestions too | 20:00 |
| xarses | hrm... | 20:00 |
| xarses | ERROR: Element 'openstack-repos' not found in 'nodepool/elements:/home/andreww/.virtualenvs/fuel/bin/../share/diskimage-builder/elements' | 20:00 |
| xarses | + IMAGE_ELEMENT= | 20:00 |
| xarses | I've pip installed diskimage-builder | 20:01 |
| fungi | yeah, that's complaining about one of our dib elements from the project-config repo, in the nodepool/elements directory | 20:01 |
| fungi | rather, complaining that it's not finding it | 20:01 |
| *** annegentle has joined #openstack-infra | 20:02 | |
| xarses | and project-config's test-requirements | 20:02 |
| pabelanger | where did you run the command from? | 20:03 |
| pabelanger | should be top-level project-config | 20:03 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 20:03 |
| *** zz_dimtruck is now known as dimtruck | 20:03 | |
| fungi | so like ./tools/build-image.sh | 20:03 |
| xarses | ran it from the tools folder | 20:03 |
| fungi | sounds like the script could use a comment noting the cwd it expects | 20:03 |
| *** apoorvad has quit IRC | 20:04 | |
| lifeless | fungi: does my bindep patch feedback make sense to you? | 20:04 |
| *** fawadkhaliq has quit IRC | 20:04 | |
| pabelanger | clarkb: reprepro running now | 20:04 |
| clarkb | pabelanger: kk I should fix my change now before I forget | 20:04 |
| xarses | yay, no RPM command found... | 20:04 |
| *** fawadkhaliq has joined #openstack-infra | 20:05 | |
| pabelanger | Ya, so you need some dependencies too | 20:05 |
| pabelanger | what is your host OS? | 20:05 |
| xarses | deb | 20:05 |
| pabelanger | https://github.com/openstack/ansible-role-diskimage-builder/blob/master/vars/Debian.yaml#L19 | 20:05 |
| pabelanger | what you need at minimum | 20:06 |
| xarses | might as well just run this in docker | 20:06 |
| xarses | ya, but that doesn't seem to help the fact that we just tried to call rpm tools | 20:06 |
| xarses | (im building the centos image) | 20:07 |
| pabelanger | Ya, you need yum-utils | 20:07 |
| *** bpokorny has joined #openstack-infra | 20:07 | |
| pabelanger | which that role is missing | 20:07 |
| pabelanger | need to fix that | 20:07 |
| fungi | lifeless: i haven't had a chance to look yet but hope to shortly | 20:07 |
| fungi | will let you know | 20:07 |
| *** jsavak has quit IRC | 20:07 | |
| *** Sukhdev has quit IRC | 20:08 | |
| *** jsavak has joined #openstack-infra | 20:08 | |
| *** roxanaghe has joined #openstack-infra | 20:09 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release https://review.openstack.org/293155 | 20:10 |
| *** angdraug has joined #openstack-infra | 20:11 | |
| openstackgerrit | Clint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate grenade plugins list https://review.openstack.org/291514 | 20:12 |
| xarses | http://paste.openstack.org/show/492780/ | 20:15 |
| pabelanger | xarses: what version of debian are you using | 20:16 |
| xarses | sid | 20:16 |
| pabelanger | Ya, not surprised it doesn't work | 20:16 |
| pabelanger | I'd use centos7 or ubuntu-trusty as your build host | 20:16 |
| fungi | did yumdownloader get non-backward-compatible changes in newer debian packages, such that it can no longer support centos 7 era rpmdb? | 20:17 |
| fungi | yikes | 20:17 |
| *** sdake has quit IRC | 20:17 | |
| pabelanger | I think yumdownloader is deprecated now | 20:18 |
| pabelanger | something dnf related replaced it | 20:18 |
| openstackgerrit | Clint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate tempest plugins list https://review.openstack.org/291517 | 20:18 |
| openstackgerrit | Clint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate grenade plugins list https://review.openstack.org/291514 | 20:18 |
| xarses | something like that | 20:18 |
| fungi | ohhhh | 20:18 |
| fungi | so dib presumably needs updating | 20:18 |
| fungi | to conditionally use whatever has superseded yumdownloader when dealing with later rh releases | 20:19 |
| fungi | but still makes me wonder why the newer yum-utils on debian isn't working with centos 7 when the version on ubuntu trusty is | 20:20 |
| pabelanger | Right, we are actually using the host to build the yum cache. Which needs to be updated to dnf | 20:20 |
| pabelanger | conditionally | 20:20 |
| fungi | i would expect a debian bug reported against yum-utils for this (not compatible with --releasever=7 centos-release or something like that), definitely not seeing one | 20:21 |
| fungi | though there is a bug open about it missing a depends on python-lzma | 20:22 |
| fungi | oh, or maybe just a recommends | 20:22 |
| fungi | SpamapS reported it one year and a day ago | 20:23 |
| fungi | doesn't look like it could be responsible for the issue you're hitting though | 20:23 |
| xarses | sounds like I just need to wrap this all in a container | 20:23 |
| *** salv-orl_ has joined #openstack-infra | 20:23 | |
| clarkb | it may not work in a container | 20:24 |
| fungi | yeah, we _successfully_ (for semi-reasonable definitions of "success") run this daily on ubuntu 14.04 lts | 20:24 |
| clarkb | or at the very least require oyu to punch holes for mounting and things | 20:24 |
| * xarses sighs | 20:24 | |
| xarses | so again, we can't publish the current image? | 20:25 |
| clarkb | we could but we don't | 20:25 |
| clarkb | its been a wishlist item needs someone to work onit | 20:25 |
| fungi | xarses: we could, though it's not generally usable without modification or introduction of some mechanism to add your authentication credentials to it | 20:25 |
| *** roxanaghe has quit IRC | 20:26 | |
| fungi | right now it has access from our ci system hard-coded into the image, password auth disabled, no other generic accounts, et cetera | 20:26 |
| clarkb | apache to search the files with an index to human names and if we grow a second builder host some sort of aggregation | 20:26 |
| clarkb | s/search/serve/ | 20:26 |
| *** salv-orlando has quit IRC | 20:26 | |
| fungi | yeah, i can imagine just pointing apache at the build dir along with a linkfarm we update for human-readable names | 20:27 |
| clarkb | then there is the issue of formats, do we serve all of them? the smallest one (qcow2) etc | 20:27 |
| fungi | but definitely would need some documentation around how to log into the images, since doing so in their current state is decidedly nontrivial unless you're our ci system | 20:27 |
| *** dprince has quit IRC | 20:28 | |
| clarkb | fungi: you just mount image, add your key, boot | 20:28 |
| clarkb | but maybe I build too many images now and that is easy to me :) | 20:28 |
| fungi | yeah, modify image while mounted on a loopback is certainly one solution. i was counting that as nontrivial (for typical python developers) but maybe my characterization is overbroad | 20:28 |
| clarkb | fwiw I tend to build the images locally rather than downloda them because I can maintain a cache for dib which is easier on the network bandwidth | 20:28 |
| clarkb | but I have to trade cpu time and memory to et an imae | 20:29 |
| *** |-paul-| has joined #openstack-infra | 20:29 | |
| fungi | and also you happen to have easy access to an ubuntu trusty system on which to build them | 20:29 |
| xarses | so this is increasingly sounding like a futile exercise, after I have this image built, It sounds like I won't be able to login anyways | 20:29 |
| clarkb | fungi: it runs in a VM on my HTPC | 20:29 |
| fungi | or something similar enough that you're not encountering this particular build bug | 20:29 |
| clarkb | xarses: no, you can you just have to make an additional edit to the image to add your key | 20:30 |
| fungi | xarses: when building the image you can tell it to add your credentials with a "devuser" element | 20:30 |
| fungi | which is why so far the recommendation has been to recreate rather than download, as you need to jump through hoops to be able to log into it anyway | 20:30 |
| *** thinrichs has joined #openstack-infra | 20:30 | |
| clarkb | I think glean will support metadata injection for ssh keys but that assumes you have a config drive or metadata server | 20:31 |
| fungi | yeah, booting it with vagrant likely won't provide a mechanism for that | 20:31 |
| fungi | though maybe you can do it with a second block device mounted as an additional drive | 20:32 |
| fungi | anyway, falls into the "nontrivial" category again | 20:32 |
| fungi | i agree it's pretty silly to need to have an ubuntu trusty vm to build a centos 7 image if you're starting from a system which is neither of those things | 20:33 |
| thinrichs | Besides openstack-server-release-jobs, what jobs do I need in zuul for the release team to include Congress in a release? The release team said I should ask here. | 20:33 |
| *** sdake has joined #openstack-infra | 20:33 | |
| * xarses goes to lunch | 20:34 | |
| *** roxanaghe has joined #openstack-infra | 20:35 | |
| fungi | thinrichs: if it's not something being uploaded to pypi, that's probably it? i can't say for sure since we don't set the release team's policies for them. you also need python-jobs to be able to have a tarball job, but you seem to have that already. also maybe you need release-notes-jobs if you're using reno for your release notes? | 20:36 |
| *** tmorin has joined #openstack-infra | 20:37 | |
| *** IlyaG has joined #openstack-infra | 20:37 | |
| *** markvoelker has quit IRC | 20:37 | |
| *** tmorin has left #openstack-infra | 20:37 | |
| *** apoorvad has joined #openstack-infra | 20:38 | |
| pabelanger | xarses: export NODEPOOL_SSH_KEY=`sudo cat /var/lib/nodepool/.ssh/id_rsa.pub | awk '{print $2}'` | 20:38 |
| pabelanger | that will let you login via the jenkins user | 20:38 |
| pabelanger | well, your path to your public SSH key | 20:39 |
| *** claudiub has quit IRC | 20:40 | |
| clarkb | pabelanger: and then login as the jenkins user? | 20:41 |
| *** jsavak has quit IRC | 20:41 | |
| fungi | i think the more general recommendation from dib'ers is to us ethe devuser element | 20:41 |
| *** jsavak has joined #openstack-infra | 20:41 | |
| clarkb | ya gives you more control | 20:41 |
| pabelanger | well, if your using your build-image.sh script | 20:42 |
| pabelanger | otherwise, I would agree devuser element | 20:42 |
| clarkb | maybe we should just add the devuser element to that script and delete the old ssh injection support | 20:42 |
| clarkb | then you cna get a devuser thats useless by default or you can configure it and it will be useful | 20:42 |
| pabelanger | I'd be okay with that | 20:42 |
| fungi | also there are some elements you can safely turn off when recreating for local use, particularly the openstack-repos which causes the image to be an order of magnitude larger and lake a ton more time/bandwidth to build | 20:42 |
| pabelanger | as I always forget the export syntax | 20:42 |
| *** mylu has joined #openstack-infra | 20:45 | |
| *** ybathia has joined #openstack-infra | 20:45 | |
| *** roxanaghe has quit IRC | 20:46 | |
| clarkb | maybe we should just turn that into a doc | 20:47 |
| clarkb | describe the elements we use, point out which can be exlcuded to save space (the repos and devstack caching) | 20:47 |
| fungi | might be nice to make at least openstack-repos easy to skip | 20:47 |
| fungi | yep | 20:47 |
| clarkb | and an example disk image builder invocation to do something useful | 20:47 |
| clarkb | rather than try and can a thing int oa script | 20:47 |
| fungi | and also how to avoid it adding our accounts to your image ;) | 20:47 |
| clarkb | oh I don't mind that | 20:47 |
| *** annegentle has quit IRC | 20:48 | |
| thinrichs | fungi: thanks. I'll follow up with the release team. | 20:48 |
| pabelanger | well, it does help when you are trying to get a new image online! :) | 20:48 |
| *** thinrichs has left #openstack-infra | 20:48 | |
| fungi | also i wonder if they come up pointed at our dfw package mirrors | 20:48 |
| pabelanger | However, I would much rather use nodepool.yaml then a script | 20:48 |
| *** annegentle has joined #openstack-infra | 20:48 | |
| clarkb | I am saying don't do either | 20:48 |
| pabelanger | Ya, default ubuntu mirrors | 20:48 |
| pabelanger | can confirm | 20:49 |
| clarkb | and just have a doc that explains how to be self sufficient | 20:49 |
| fungi | similar to what i did in the devstack-gate readme | 20:49 |
| *** roxanaghe has joined #openstack-infra | 20:49 | |
| fungi | a walkthrough essentially, with enough background info on prerequisites/gotchas that most readers won't get irrevocable stuck | 20:49 |
| fungi | irrevocably | 20:50 |
| *** annegentle has quit IRC | 20:50 | |
| *** ricky1 has joined #openstack-infra | 20:50 | |
| *** annegentle has joined #openstack-infra | 20:50 | |
| *** yamamoto has joined #openstack-infra | 20:51 | |
| clarkb | pabelanger: any idea what reprepro is doing? I don't see any xenial logs or other indication it is xenialing yet | 20:52 |
| clarkb | maybe I just need to be more patient | 20:53 |
| fungi | lifeless: read your comments on 200672 and generally agree, though AJaeger has also already replied asking the followup questions i was going to ask | 20:53 |
| pabelanger | clarkb: Ya, I think it is working. | 20:53 |
| clarkb | I should probably go read more manpages | 20:53 |
| pabelanger | but we should likely enable logging for it | 20:54 |
| clarkb | pabelanger: I think logging is enabled | 20:54 |
| clarkb | pabelanger: there are logs for all the trusty things | 20:54 |
| clarkb | maybe its determining what needs updating and doesn't have anything ot log yet? | 20:54 |
| *** ricky1 has quit IRC | 20:55 | |
| *** annegentle has quit IRC | 20:55 | |
| pabelanger | clarkb: Ya, we need to add logdir to the options file IIRC | 20:55 |
| *** yamamoto has quit IRC | 20:56 | |
| *** mriedem1 has joined #openstack-infra | 20:56 | |
| pabelanger | or verbose | 20:56 |
| lifeless | fungi: cool, will look at the q's on moday | 20:57 |
| *** annegentle has joined #openstack-infra | 20:57 | |
| fungi | lifeless: there's no hurry. i'm answering some of his questions and expounding on others as we speak. and thanks for the input! | 20:57 |
| clarkb | pabelanger: gotcha | 20:58 |
| *** eharney has quit IRC | 20:58 | |
| *** eharney has joined #openstack-infra | 20:59 | |
| *** mriedem has quit IRC | 20:59 | |
| *** smarcet has quit IRC | 20:59 | |
| *** roxanaghe has quit IRC | 20:59 | |
| *** elo has quit IRC | 21:00 | |
| *** gordc has quit IRC | 21:02 | |
| *** amitgandhinz has quit IRC | 21:03 | |
| pabelanger | clarkb: okay, logs enabled but they look to be in / | 21:03 |
| pabelanger | should move them into afs volume | 21:04 |
| clarkb | pabelanger: I would put them in /var/log/reprepro with the other logs | 21:04 |
| pabelanger | that works too | 21:04 |
| clarkb | odd that they end up in / guessing thats just misconfigured | 21:04 |
| pabelanger | clarkb: ya, it is no long running. Maybe we should run it manually to see what is going on | 21:06 |
| pabelanger | maybe something misconfigured | 21:06 |
| *** roxanaghe has joined #openstack-infra | 21:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add back logdir for reprepro.pp https://review.openstack.org/300661 | 21:11 |
| pabelanger | clarkb: ^ | 21:11 |
| *** trown is now known as trown|outtypewww | 21:11 | |
| *** devananda has quit IRC | 21:12 | |
| clarkb | whoops | 21:12 |
| *** devananda has joined #openstack-infra | 21:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Increase verbosity for reprepro command https://review.openstack.org/300663 | 21:13 |
| pabelanger | and verbosity ^ | 21:13 |
| clarkb | I went ahead and fast approved the fix for my derp | 21:13 |
| pabelanger | sure | 21:13 |
| clarkb | I see now that it is used by a template | 21:13 |
| clarkb | which I must've missed when cleaning up the parameters | 21:14 |
| * pabelanger nods | 21:14 | |
| *** hahyes has joined #openstack-infra | 21:14 | |
| *** cdent has quit IRC | 21:14 | |
| hahyes | hi new to openstack-infra.. trying to get some answer for jenkins job builder... | 21:15 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: add missing release jobs https://review.openstack.org/300474 | 21:15 |
| *** _ari_ has quit IRC | 21:15 | |
| pabelanger | clarkb: Ya, so reprepro finished but didn't pull down xenial. Not sure why atm | 21:16 |
| pabelanger | but have to afk | 21:16 |
| openstackgerrit | Merged openstack-infra/system-config: Change http download command to point to git.o.o https://review.openstack.org/300187 | 21:16 |
| clarkb | hahyes: ask away | 21:16 |
| *** thorst has quit IRC | 21:17 | |
| *** regXboi has quit IRC | 21:17 | |
| *** EricGonc_ has quit IRC | 21:18 | |
| *** rfolco has quit IRC | 21:21 | |
| *** jsavak has quit IRC | 21:21 | |
| *** tmartins has joined #openstack-infra | 21:22 | |
| tmartins | hey guys, how can I fill a bug or a request for a feature, for Python Shade? | 21:22 |
| clarkb | tmartins: https://storyboard.openstack.org/#!/project/760 is the bug tracker for shade | 21:23 |
| tmartins | sounds awesome, thank you! | 21:24 |
| openstackgerrit | Julia Kreger proposed openstack-infra/project-config: Correct missing bifrost release setting https://review.openstack.org/300665 | 21:24 |
| *** ayoung is now known as ayoung-afk | 21:25 | |
| *** kakocyan has joined #openstack-infra | 21:25 | |
| *** jcooley_ has quit IRC | 21:25 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: Give project teams access to approve stable/mitaka changes https://review.openstack.org/300667 | 21:26 |
| *** ybathia has quit IRC | 21:26 | |
| kakocyan | Hi. I am new here and trying to use Jenkins job builder. I need to use folders and also installed dashboard views. How can I specify the jobs under the view and folders? | 21:26 |
| clarkb | I don't know that jjb supports that | 21:27 |
| clarkb | until relatively recently the global config for jenkins has included state information about the slaves too making it hard to edit | 21:28 |
| openstackgerrit | Merged openstack-infra/system-config: Add back logdir for reprepro.pp https://review.openstack.org/300661 | 21:29 |
| *** mriedem1 is now known as mriedem | 21:29 | |
| *** Sukhdev has joined #openstack-infra | 21:29 | |
| *** akshai_ has quit IRC | 21:30 | |
| *** ayoung-afk is now known as ayoung | 21:30 | |
| openstackgerrit | Merged openstack-infra/system-config: Increase verbosity for reprepro command https://review.openstack.org/300663 | 21:30 |
| *** ayoung is now known as ayoung_afk | 21:30 | |
| *** Sukhdev has quit IRC | 21:31 | |
| kakocyan | I run my local jenkins server and with these plugins, I could inject my job definition(just by specifying job name like 'folder1/childjob'). But when I tried on our central jenkins server, it complained. JenkinsException: Cannot create job[keymaster/Alchemy-Keymaster] because folder for the job does not exist | 21:31 |
| *** Goneri has joined #openstack-infra | 21:31 | |
| clarkb | its possible you could add that to jenkins job builder | 21:32 |
| clarkb | have it create dirs if necessary | 21:32 |
| *** ldnunes has quit IRC | 21:32 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: update tag ACLs for all official projects https://review.openstack.org/298866 | 21:33 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: Give project teams access to approve stable/mitaka changes https://review.openstack.org/300667 | 21:33 |
| kakocyan | Actually the dir was already there. On my local server, it could find and add my job, but not on the main server that I do not have much knowledge and control. | 21:34 |
| *** baoli has quit IRC | 21:36 | |
| *** kong has joined #openstack-infra | 21:36 | |
| clarkb | kakocyan: my reading of that error is that it failed because the dir keymaster did not exist | 21:36 |
| clarkb | kakocyan: it is probably possible to detect that scenario and create the dir first | 21:36 |
| *** thorst has joined #openstack-infra | 21:38 | |
| *** mriedem has quit IRC | 21:38 | |
| *** nadya has quit IRC | 21:38 | |
| *** annegentle has quit IRC | 21:39 | |
| jeblair | i want to use openstackclient to view our quota, but i don't see how... | 21:39 |
| clarkb | jeblair: one sec | 21:39 |
| jeblair | 'quota show' wants a 'project' argument... | 21:39 |
| clarkb | hrm I thought there was a server quota show | 21:40 |
| clarkb | but not seeing it | 21:40 |
| *** tiswanso has quit IRC | 21:41 | |
| jeblair | oh, 'quota show' is an admin method | 21:41 |
| clarkb | you might need to use nova client to see the nova quota and $other client for that quota | 21:42 |
| *** thorst has quit IRC | 21:42 | |
| jeblair | 'limits show --absolute' is a thing | 21:42 |
| kakocyan | Oops. we have so many jobs/folders created under that view(keymaster) already. keymaster is not a job. It is a view. But my local server seems to be able to find it. Am I missing something? | 21:42 |
| clarkb | thats an odd thing to make inconsistent | 21:43 |
| clarkb | quota vs limit | 21:43 |
| clarkb | I wonder if osc would be open to making them the same | 21:43 |
| clarkb | kakocyan: on your non local server does keymaster exist? | 21:43 |
| jeblair | clarkb: i recall 'nova absolute-limits' | 21:43 |
| *** bpokorny has quit IRC | 21:43 | |
| *** daemontool has joined #openstack-infra | 21:44 | |
| clarkb | kakocyan: the implication of that error message is that it does not | 21:44 |
| dtroyer | clarkb: agreed, there is a distinction that I don't remember offhand | 21:44 |
| dtroyer | it needs cleaning up | 21:44 |
| jeblair | dtroyer: i'm wondering if 'quota' is used on the admin side and 'limits' is used on the user side? | 21:44 |
| jeblair | dtroyer, clarkb: but yeah, that feels like a distinction without a difference that could probably be cleaned up | 21:45 |
| clarkb | the other weird one I ran into recently is with security groups | 21:45 |
| kakocyan | Yes. On my local server, I created keymaster manually. That is the same with the main server. It was created manually. | 21:45 |
| clarkb | half the commands map to nova and half map to neutron | 21:45 |
| clarkb | and its sort of not clearwhich do which | 21:45 |
| dtroyer | those are under active transition | 21:46 |
| clarkb | dtroyer: good to hear | 21:46 |
| dtroyer | should be much better Real Soon Now | 21:46 |
| clarkb | kakocyan: maybe the user doesn't have access to it? | 21:46 |
| dtroyer | they will detect Neutron in the service catalog | 21:46 |
| clarkb | kakocyan: jjb basically needs full admin rights iirc | 21:46 |
| jeblair | i have a command i want to run against every cloud+region in clouds.yaml | 21:46 |
| jeblair | (i bet you can guess which command) | 21:47 |
| jeblair | is there any neat occ/openstackclient trick to help with that? | 21:47 |
| dtroyer | jeblair: not at present | 21:47 |
| kakocyan | Oh. That probably is. I was hoping I can handle with my own id since the view was under my team's control. | 21:47 |
| dtroyer | we could add something to select the config in interactive mode and pipe stuff in | 21:47 |
| jeblair | dtroyer: or something to output the full list that i could then use in a bash for loop would be cool | 21:48 |
| dtroyer | ah, sure. we may have something already, hang on | 21:48 |
| kakocyan | I will ask the server team if they can let us handle it. Thanks! | 21:48 |
| jeblair | dtroyer: ooh | 21:48 |
| *** ayoung_afk is now known as ayoung | 21:49 | |
| dtroyer | just 'configuration show' adding a list would be easy | 21:49 |
| *** xyang1 has quit IRC | 21:49 | |
| *** hahyes has quit IRC | 21:52 | |
| *** EricGonczer_ has joined #openstack-infra | 21:52 | |
| *** Sukhdev has joined #openstack-infra | 21:52 | |
| *** Sukhdev has quit IRC | 21:55 | |
| *** sdague has quit IRC | 21:56 | |
| jeblair | for the purposes of what i'm doing, i'm running this command against 23 cloud+region combos | 21:56 |
| jeblair | first-world problems, i know | 21:56 |
| tmartins | clarkb, new story added! https://storyboard.openstack.org/#!/story/2000537 :-D | 21:57 |
| *** hashar has quit IRC | 21:59 | |
| *** kakocyan has quit IRC | 21:59 | |
| *** mdenny has joined #openstack-infra | 21:59 | |
| jeblair | clarkb, mordred: my survey says we have a limit of 40 server metadata entries (i assume this is a per-server limit) in rax, 20 in internap, and 128 elsewhere. | 22:00 |
| *** Sukhdev has joined #openstack-infra | 22:00 | |
| *** markvoelker_ has joined #openstack-infra | 22:01 | |
| tmartins | One more: https://storyboard.openstack.org/#!/story/2000538 ^_^ | 22:02 |
| *** dimtruck is now known as zz_dimtruck | 22:03 | |
| *** daemontool has quit IRC | 22:03 | |
| *** fawadkhaliq has quit IRC | 22:05 | |
| *** fawadkhaliq has joined #openstack-infra | 22:05 | |
| pvaneck | jeblair: Hey, can you initiate the workflow for this: https://review.openstack.org/#/c/281737 . Want to get the live site updated. | 22:05 |
| *** Sukhdev has quit IRC | 22:07 | |
| *** ZZelle_ has joined #openstack-infra | 22:10 | |
| *** lutf-8 is now known as alaski | 22:12 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:13 | |
| *** sdake_ has joined #openstack-infra | 22:13 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Cleanup building dib images on startup https://review.openstack.org/300678 | 22:13 |
| *** sdake has quit IRC | 22:13 | |
| clarkb | jeblair: ^ I think that will address the problem from earlire today | 22:13 |
| *** edmondsw has quit IRC | 22:15 | |
| clarkb | I should just make a permanent mysql install for nodepool testing | 22:17 |
| openstackgerrit | Tim Hinrichs proposed openstack-infra/project-config: Add openstack-server-release-jobs to Congress in zuul https://review.openstack.org/300682 | 22:18 |
| *** sdake has joined #openstack-infra | 22:19 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Cleanup building dib images on startup https://review.openstack.org/300678 | 22:19 |
| *** sdake_ has quit IRC | 22:21 | |
| jeblair | pvaneck: sure thing | 22:22 |
| jeblair | clarkb: ++ | 22:23 |
| clarkb | oddly I think the most recent failure is in the fakes not mysql | 22:24 |
| clarkb | it is somehow booting a dib based node without a valid dib image | 22:24 |
| clarkb | oh wait I know | 22:24 |
| *** LaT0rtue has quit IRC | 22:24 | |
| *** Sukhdev has joined #openstack-infra | 22:25 | |
| pvaneck | jeblair: thanks! | 22:25 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Cleanup building dib images on startup https://review.openstack.org/300678 | 22:26 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add Non-ATC Recognition WG Meeting https://review.openstack.org/300278 | 22:28 |
| *** HeOS has quit IRC | 22:28 | |
| *** achanda has joined #openstack-infra | 22:29 | |
| *** mixos has joined #openstack-infra | 22:29 | |
| *** nwkarsten has joined #openstack-infra | 22:30 | |
| *** aysyd has quit IRC | 22:31 | |
| *** HeOS has joined #openstack-infra | 22:31 | |
| *** mylu has quit IRC | 22:32 | |
| *** mylu has joined #openstack-infra | 22:33 | |
| openstackgerrit | Merged openstack-infra/puppet-refstack: Make puppet pull releases from Pypi https://review.openstack.org/281737 | 22:33 |
| *** nwkarsten has quit IRC | 22:35 | |
| *** markvoelker_ has quit IRC | 22:39 | |
| *** flepied1 has joined #openstack-infra | 22:39 | |
| openstackgerrit | Ian Cordasco proposed openstack-dev/pbr: package: fix wrong catch in email parsing https://review.openstack.org/289227 | 22:41 |
| *** pradk has quit IRC | 22:41 | |
| *** flepied has quit IRC | 22:43 | |
| *** krtaylor has quit IRC | 22:44 | |
| *** |-paul-| has quit IRC | 22:44 | |
| *** fawadkhaliq has quit IRC | 22:44 | |
| *** fawadkhaliq has joined #openstack-infra | 22:45 | |
| *** mixos has quit IRC | 22:46 | |
| *** rvasilets__ has joined #openstack-infra | 22:48 | |
| *** daemontool has joined #openstack-infra | 22:51 | |
| *** knikolla has quit IRC | 22:53 | |
| *** IlyaG has quit IRC | 22:54 | |
| *** IlyaG has joined #openstack-infra | 22:54 | |
| *** yamamoto has joined #openstack-infra | 22:55 | |
| *** IlyaG has joined #openstack-infra | 22:55 | |
| *** IlyaG has quit IRC | 22:55 | |
| *** krtaylor has joined #openstack-infra | 22:56 | |
| *** keedya has joined #openstack-infra | 22:58 | |
| *** yamamoto has quit IRC | 23:00 | |
| *** rlandy has quit IRC | 23:02 | |
| *** ams_ has quit IRC | 23:03 | |
| *** mylu has quit IRC | 23:03 | |
| *** fawadkhaliq has quit IRC | 23:04 | |
| *** auggy_rodin is now known as auggy | 23:04 | |
| *** fawadkhaliq has joined #openstack-infra | 23:04 | |
| *** rbrndt has quit IRC | 23:06 | |
| *** blogan has quit IRC | 23:09 | |
| *** roxanaghe has quit IRC | 23:10 | |
| *** roxanaghe has joined #openstack-infra | 23:10 | |
| *** mtanino has quit IRC | 23:13 | |
| *** Swami has quit IRC | 23:13 | |
| *** xarses has quit IRC | 23:16 | |
| *** armax has quit IRC | 23:17 | |
| *** daemontool has quit IRC | 23:17 | |
| *** daemontool has joined #openstack-infra | 23:18 | |
| *** armax has joined #openstack-infra | 23:19 | |
| *** trown|outtypewww is now known as trown | 23:21 | |
| *** knikolla has joined #openstack-infra | 23:22 | |
| *** Sukhdev has quit IRC | 23:23 | |
| *** erlon has quit IRC | 23:27 | |
| asselin__ | zaro, our gerrit stopped e-mail folks. any tips on where to look for debugging it? | 23:28 |
| clarkb | asselin__: do you use our puppet manifests for it? if so you are likely running exim. check the logs for that | 23:29 |
| asselin__ | ok thanks | 23:29 |
| clarkb | gerrit will hand off to exim which will then send email | 23:30 |
| *** angdraug has quit IRC | 23:34 | |
| *** harlowja has quit IRC | 23:36 | |
| *** yamahata has quit IRC | 23:36 | |
| *** LaT0rtue has joined #openstack-infra | 23:37 | |
| *** marcusvrn_ has quit IRC | 23:37 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/project-config: Fix broken ironic gate jobs https://review.openstack.org/300694 | 23:37 |
| *** daemontool has quit IRC | 23:38 | |
| jeblair | pvaneck: refstack.o.o looks dead. it looks like the npm installation failed due to npm errors | 23:38 |
| clarkb | did leftpad get deleted again? | 23:39 |
| pvaneck | jeblair: yea, trying to see if the next puppet run will correct it as a new tag ws as just pushed | 23:40 |
| pvaneck | was* | 23:40 |
| jeblair | clarkb: 403 on phantomjs | 23:41 |
| clarkb | jeblair: same thing as that horizon job | 23:41 |
| clarkb | I think bitbucket is unhappy | 23:41 |
| jeblair | pvaneck: it just ran again; i'm reading logs | 23:42 |
| *** esker has joined #openstack-infra | 23:43 | |
| *** mixos has joined #openstack-infra | 23:43 | |
| jeblair | Apr 1 23:38:42 refstack puppet-user[863]: (/Stage[main]/Refstack::App/Exec[npm install]/returns) npm ERR! cb() never called! | 23:43 |
| jeblair | Apr 1 23:38:42 refstack puppet-user[863]: (/Stage[main]/Refstack::App/Exec[npm install]/returns) npm ERR! not ok code 0 | 23:43 |
| jeblair | i have no idea what that means. i don't see any other errors | 23:43 |
| jeblair | pvaneck: it looks like anything under /assets is 404 | 23:46 |
| pvaneck | hmm, on the refstack server, does doing 'npm install' in /opt/refstack yield the same thing? | 23:46 |
| *** esker has quit IRC | 23:47 | |
| jeblair | pvaneck: npm install in that directory as refstack succeeds, and the output doesn't look anything like what was in the logs. | 23:48 |
| pvaneck | cant recreate it locally. not sure why puppet is failing to npm install | 23:54 |
| jeblair | pvaneck: the site is working now... | 23:54 |
| pvaneck | but it seems like you performing the npm install fixed the issue | 23:54 |
| jeblair | i'm not sure it was that | 23:55 |
| jeblair | there has been another puppet run, and *that* one performed the sync of the www directory | 23:55 |
| *** yamamoto has joined #openstack-infra | 23:56 | |
| pvaneck | to pick up the changes in /opt/refstack i believe from your manual npm install | 23:56 |
| jeblair | or possibly from the npm install that puppet ran | 23:57 |
| jeblair | hard to say which | 23:57 |
| *** ajmiller has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!