| clarkb | fungi: I think I have decided I really don't like this magic _dir() method in the git-review tests | 00:00 |
|---|---|---|
| clarkb | impossible to know what any path actually is | 00:00 |
| hub_cap | mordred: i have verified. no black magic. | 00:00 |
| fungi | clarkb: reasonable concern. i would +2 a replacement | 00:00 |
| hub_cap | obviously, we will freak out if you put your own backup in its place (checksum validation) | 00:00 |
| hub_cap | and try to restore from it | 00:00 |
| *** mriedem has quit IRC | 00:01 | |
| hub_cap | but you can freely take the xtra stuff out | 00:01 |
| hub_cap | and put it in your pocket | 00:01 |
| fungi | that's actually non-obvious, but also something we would be unlikely to randomly try anyway | 00:01 |
| hub_cap | and fire proof gun safe | 00:01 |
| *** hashar has quit IRC | 00:01 | |
| mordred | hub_cap: that's great. | 00:01 |
| mordred | hub_cap: I think we'll put on the todo list starting to have you do backups | 00:02 |
| fungi | (the putting our backup in place of the rs-provided backup in swift causing an insta-restore from it, i mean) | 00:02 |
| hub_cap | fungi: /me hopes | 00:02 |
| *** hemna is now known as hemnafk | 00:02 | |
| *** Ryan_Lane has quit IRC | 00:04 | |
| *** HenryG_ has joined #openstack-infra | 00:04 | |
| *** HenryG__ has joined #openstack-infra | 00:08 | |
| *** dkehn has quit IRC | 00:08 | |
| *** HenryG__ has quit IRC | 00:08 | |
| *** HenryG has quit IRC | 00:08 | |
| *** dkehn has joined #openstack-infra | 00:08 | |
| *** HenryG_ has quit IRC | 00:12 | |
| *** gyee has quit IRC | 00:14 | |
| *** gyee has joined #openstack-infra | 00:15 | |
| *** prad has joined #openstack-infra | 00:15 | |
| *** zul has quit IRC | 00:17 | |
| *** atiwari has quit IRC | 00:19 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Update old bug format in recheckwatch https://review.openstack.org/46166 | 00:23 |
| jog0 | jeblair: hopefully 3rd time is the charm :/ | 00:24 |
| *** prad has quit IRC | 00:24 | |
| *** zul has joined #openstack-infra | 00:26 | |
| *** gyee has quit IRC | 00:28 | |
| * mordred believes in jog0 | 00:28 | |
| clarkb | mordred: fungi https://review.openstack.org/#/c/45789/ | 00:41 |
| clarkb | though at this point maybe waiting for tomorrow is better? (bup doesn't run for anount ~6 hours so it happens over night anyways) | 00:41 |
| * fungi has a look | 00:41 | |
| fungi | i'm cool with now assuming it looks okay | 00:42 |
| fungi | 0041 clarkb mordred: fungi https://review.openstack.org/#/c/45789/ | 00:43 |
| mordred | clarkb: looking | 00:43 |
| fungi | er, paste fun there. excuse me ;) | 00:43 |
| mordred | clarkb: I +2'd, but did not aprv | 00:44 |
| *** FallenPegasus has joined #openstack-infra | 00:44 | |
| *** prad has joined #openstack-infra | 00:44 | |
| *** MarkAtwood has quit IRC | 00:48 | |
| *** FallenPegasus has quit IRC | 00:49 | |
| fungi | lgtm, aprv'd | 00:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Use bup to backup review.o.o to an offsite server https://review.openstack.org/45789 | 00:51 |
| *** weshay has quit IRC | 00:52 | |
| fungi | (slowly, from the wearable, as gerrit is a bit of a pain on a tiny screen) | 00:52 |
| *** prad has quit IRC | 00:52 | |
| *** dcramer_ has quit IRC | 00:56 | |
| *** marun has quit IRC | 01:03 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Delete closed bugs after 5 days in recheckwatch https://review.openstack.org/46169 | 01:08 |
| *** dcramer_ has joined #openstack-infra | 01:09 | |
| clarkb | woo more apparent gerrit differences in the new version | 01:12 |
| clarkb | looks like bouncy castle in the new version will generate its own key and kill your old one | 01:12 |
| clarkb | and it does it in a weird format | 01:12 |
| clarkb | maybe, I am still having trouble making this consistent | 01:13 |
| jog0 | jeblair: thanks for the recheck pickle it helped make debugging much easier | 01:18 |
| *** dkliban has joined #openstack-infra | 01:19 | |
| *** nosnos has joined #openstack-infra | 01:22 | |
| *** svarnau has quit IRC | 01:25 | |
| *** reed has quit IRC | 01:31 | |
| *** dprince has joined #openstack-infra | 01:36 | |
| *** fifieldt has joined #openstack-infra | 01:38 | |
| *** dzyu has joined #openstack-infra | 01:44 | |
| *** melwitt has quit IRC | 01:51 | |
| *** yaguang has joined #openstack-infra | 01:52 | |
| *** HenryG has joined #openstack-infra | 01:56 | |
| *** thomasm has joined #openstack-infra | 01:59 | |
| *** dcramer_ has quit IRC | 02:02 | |
| *** dzyu has quit IRC | 02:05 | |
| *** xchu has joined #openstack-infra | 02:09 | |
| *** dzyu_ has joined #openstack-infra | 02:10 | |
| *** dzyu_ is now known as dzyu | 02:10 | |
| *** dzyu_ has joined #openstack-infra | 02:14 | |
| *** dprince has quit IRC | 02:15 | |
| *** dzyu has quit IRC | 02:17 | |
| *** dzyu_ is now known as dzyu | 02:17 | |
| *** nati_ueno has quit IRC | 02:22 | |
| *** dims has quit IRC | 02:25 | |
| *** xchu has quit IRC | 02:27 | |
| *** jcoufal has quit IRC | 02:28 | |
| *** fifieldt has quit IRC | 02:29 | |
| *** xchu has joined #openstack-infra | 02:39 | |
| *** senk has joined #openstack-infra | 02:41 | |
| *** dzyu_ has joined #openstack-infra | 02:41 | |
| *** jhesketh__ has quit IRC | 02:41 | |
| *** dzyu has quit IRC | 02:43 | |
| *** dzyu_ is now known as dzyu | 02:43 | |
| mordred | that's the best sentence in here ever ^^ | 02:46 |
| * mordred wants a recheck pickle | 02:46 | |
| *** HenryG has quit IRC | 02:53 | |
| *** jhesketh__ has joined #openstack-infra | 02:54 | |
| *** nosnos has quit IRC | 02:57 | |
| *** anteaya__ is now known as anteaya | 02:57 | |
| *** nosnos has joined #openstack-infra | 02:58 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/git-review: Add test cleanups before things they cleanup. https://review.openstack.org/46179 | 03:01 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/git-review: Use bouncy castle when testing if available. https://review.openstack.org/46180 | 03:01 |
| clarkb | fungi: ^ so many hacks, I am not entirely happy with that but it gives you a general idea of what is needed | 03:01 |
| *** UtahDave has joined #openstack-infra | 03:03 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/git-review: Use bouncy castle when testing if available. https://review.openstack.org/46180 | 03:10 |
| *** xchu has quit IRC | 03:12 | |
| *** xchu has joined #openstack-infra | 03:30 | |
| *** anteaya has quit IRC | 03:33 | |
| *** hashar has joined #openstack-infra | 03:36 | |
| hub_cap | mordred: a recheck pickle sounds like a dreaded followup appt w/ your doctor | 03:37 |
| *** FallenPegasus has joined #openstack-infra | 03:45 | |
| *** vogxn has joined #openstack-infra | 03:46 | |
| *** reed has joined #openstack-infra | 03:53 | |
| *** fifieldt has joined #openstack-infra | 03:55 | |
| *** llu has joined #openstack-infra | 03:57 | |
| *** dcramer_ has joined #openstack-infra | 03:58 | |
| llu | It seems the openstack-requirements Jenkin's job keeps failing, https://jenkins01.openstack.org/job/gate-requirements-install/225/console | 04:04 |
| llu | pip seems complaining about duplicated requirement of hacking in both global-requirements and test-requirements | 04:05 |
| llu | which file should we remove the hacking requirement from? global or test? | 04:05 |
| *** jhesketh__ has quit IRC | 04:07 | |
| *** jhesketh has joined #openstack-infra | 04:07 | |
| *** senk has quit IRC | 04:11 | |
| *** dcramer_ has quit IRC | 04:11 | |
| clarkb | llu: neither | 04:11 |
| clarkb | llu: https://review.openstack.org/#/c/46107/ is the fix | 04:12 |
| *** SergeyLukjanov has joined #openstack-infra | 04:13 | |
| *** SergeyLukjanov has quit IRC | 04:14 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:20 | |
| *** FallenPegasus has quit IRC | 04:21 | |
| *** dcramer_ has joined #openstack-infra | 04:24 | |
| *** SergeyLukjanov has quit IRC | 04:30 | |
| *** hashar has quit IRC | 04:32 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:33 | |
| *** dguitarbite has quit IRC | 04:33 | |
| *** dguitarbite has joined #openstack-infra | 04:35 | |
| *** FallenPegasus has joined #openstack-infra | 04:37 | |
| openstackgerrit | Tom Fifield proposed a change to openstack-infra/config: adding staging server for activity board https://review.openstack.org/42998 | 04:41 |
| *** dcramer_ has quit IRC | 04:41 | |
| *** senk has joined #openstack-infra | 04:42 | |
| *** FallenPegasus has quit IRC | 04:43 | |
| *** senk has quit IRC | 04:46 | |
| *** dmakogon_ has joined #openstack-infra | 04:48 | |
| *** SergeyLukjanov has quit IRC | 04:48 | |
| *** pcrews has joined #openstack-infra | 04:50 | |
| *** Ryan_Lane has joined #openstack-infra | 04:52 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:57 | |
| *** SergeyLukjanov has quit IRC | 04:57 | |
| *** vogxn has quit IRC | 05:00 | |
| *** vogxn has joined #openstack-infra | 05:02 | |
| *** mrodden has joined #openstack-infra | 05:05 | |
| *** reed has quit IRC | 05:11 | |
| *** dguitarbite has quit IRC | 05:13 | |
| *** senk has joined #openstack-infra | 05:14 | |
| *** llu has left #openstack-infra | 05:15 | |
| *** dguitarbite has joined #openstack-infra | 05:16 | |
| *** pmoosh has quit IRC | 05:18 | |
| *** senk has quit IRC | 05:18 | |
| *** dkliban has quit IRC | 05:21 | |
| *** boris-42 has joined #openstack-infra | 05:22 | |
| *** dguitarbite has quit IRC | 05:22 | |
| *** dguitarbite has joined #openstack-infra | 05:23 | |
| *** nicedice has quit IRC | 05:29 | |
| *** dguitarbite has quit IRC | 05:30 | |
| *** harlowja has quit IRC | 05:31 | |
| *** harlowja has joined #openstack-infra | 05:31 | |
| *** harlowja_ has joined #openstack-infra | 05:36 | |
| *** harlowja has quit IRC | 05:38 | |
| *** harlowja has joined #openstack-infra | 05:39 | |
| *** harlowja_ has quit IRC | 05:42 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:42 | |
| *** pblaho has joined #openstack-infra | 05:42 | |
| *** UtahDave has quit IRC | 06:01 | |
| *** nosnos has quit IRC | 06:17 | |
| *** nosnos has joined #openstack-infra | 06:24 | |
| *** yolanda has joined #openstack-infra | 06:29 | |
| *** flaper87|afk is now known as flaper87 | 06:33 | |
| *** yolanda has quit IRC | 06:37 | |
| *** vogxn has quit IRC | 06:39 | |
| *** vogxn has joined #openstack-infra | 06:41 | |
| *** yongli_away has quit IRC | 06:43 | |
| *** SergeyLukjanov has quit IRC | 06:43 | |
| *** amotoki has joined #openstack-infra | 06:52 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Publish Over SSH Plugin option added https://review.openstack.org/46112 | 06:55 |
| *** xchu has quit IRC | 06:56 | |
| *** afazekas has joined #openstack-infra | 06:57 | |
| *** pcrews has quit IRC | 06:58 | |
| *** flaper87 is now known as flaper87|afk | 07:05 | |
| *** afazekas_ has joined #openstack-infra | 07:05 | |
| *** afazekas_ has quit IRC | 07:06 | |
| *** dachary has quit IRC | 07:06 | |
| *** dachary has joined #openstack-infra | 07:06 | |
| *** flaper87|afk is now known as flaper87 | 07:14 | |
| *** shardy_afk is now known as shardy | 07:15 | |
| *** xchu has joined #openstack-infra | 07:15 | |
| *** vogxn has quit IRC | 07:15 | |
| *** vogxn has joined #openstack-infra | 07:16 | |
| *** Ryan_Lane has quit IRC | 07:21 | |
| *** xchu has quit IRC | 07:26 | |
| *** vogxn has quit IRC | 07:28 | |
| *** dmakogon_ has quit IRC | 07:28 | |
| *** jpich has joined #openstack-infra | 07:41 | |
| *** xchu has joined #openstack-infra | 07:44 | |
| *** flaper87 is now known as flaper87|afk | 07:47 | |
| *** Bada has joined #openstack-infra | 07:48 | |
| *** afazekas has quit IRC | 07:55 | |
| *** boris-42 has quit IRC | 07:55 | |
| *** flaper87|afk is now known as flaper87 | 07:58 | |
| *** xchu has quit IRC | 08:08 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:09 | |
| *** yolanda has joined #openstack-infra | 08:10 | |
| *** fbo_away is now known as fbo | 08:17 | |
| *** locke105 has quit IRC | 08:23 | |
| *** bashok has joined #openstack-infra | 08:50 | |
| *** kspear has quit IRC | 09:04 | |
| *** zaro has quit IRC | 09:05 | |
| *** zaro has joined #openstack-infra | 09:06 | |
| *** fifieldt has quit IRC | 09:13 | |
| *** salv-orlando has joined #openstack-infra | 09:20 | |
| *** boris-42 has joined #openstack-infra | 09:22 | |
| *** dzyu has quit IRC | 09:35 | |
| *** xchu has joined #openstack-infra | 09:51 | |
| *** odyssey4me has joined #openstack-infra | 09:57 | |
| *** Bada has quit IRC | 10:03 | |
| *** xchu has quit IRC | 10:08 | |
| *** branen_ has quit IRC | 10:21 | |
| *** yaguang has quit IRC | 10:26 | |
| *** mdenny has quit IRC | 10:26 | |
| *** dims has joined #openstack-infra | 10:29 | |
| *** SergeyLukjanov has quit IRC | 10:36 | |
| *** SergeyLukjanov has joined #openstack-infra | 10:38 | |
| *** afazekas has joined #openstack-infra | 10:51 | |
| *** SergeyLukjanov has quit IRC | 10:54 | |
| *** kspear has joined #openstack-infra | 11:00 | |
| *** weshay has joined #openstack-infra | 11:02 | |
| *** dizquierdo has joined #openstack-infra | 11:03 | |
| *** dprince has joined #openstack-infra | 11:05 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:07 | |
| *** DennyZhang has joined #openstack-infra | 11:27 | |
| *** pcm_ has joined #openstack-infra | 11:32 | |
| *** jhesketh has quit IRC | 11:40 | |
| *** HenryG has joined #openstack-infra | 11:40 | |
| *** ArxCruz has joined #openstack-infra | 11:43 | |
| *** chuckieb has joined #openstack-infra | 11:44 | |
| *** rfolco has joined #openstack-infra | 11:45 | |
| *** kspear has quit IRC | 11:51 | |
| *** kspear has joined #openstack-infra | 11:52 | |
| *** zeus has joined #openstack-infra | 11:59 | |
| *** jcoufal has joined #openstack-infra | 12:03 | |
| *** jcoufal has quit IRC | 12:03 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Build Blocker Plugin support added https://review.openstack.org/46262 | 12:03 |
| *** jcoufal has joined #openstack-infra | 12:04 | |
| *** jcoufal has quit IRC | 12:04 | |
| *** mgagne has quit IRC | 12:05 | |
| *** mgagne has joined #openstack-infra | 12:05 | |
| *** mgagne has joined #openstack-infra | 12:05 | |
| *** jcoufal has joined #openstack-infra | 12:06 | |
| *** sld has quit IRC | 12:07 | |
| *** annegentle has quit IRC | 12:08 | |
| *** sld has joined #openstack-infra | 12:09 | |
| *** dprince has quit IRC | 12:09 | |
| *** alexpilotti has joined #openstack-infra | 12:18 | |
| *** flaper87 is now known as flaper87|afk | 12:23 | |
| *** changbl has quit IRC | 12:27 | |
| *** dizquierdo has quit IRC | 12:28 | |
| *** flaper87|afk is now known as flaper87 | 12:31 | |
| *** DennyZhang has quit IRC | 12:34 | |
| *** afazekas_ has joined #openstack-infra | 12:39 | |
| *** KLevenstein has joined #openstack-infra | 12:47 | |
| *** adalbas has joined #openstack-infra | 12:48 | |
| *** afazekas_ has quit IRC | 12:49 | |
| *** nhm has quit IRC | 12:50 | |
| *** anteaya has joined #openstack-infra | 12:56 | |
| *** annegentle has joined #openstack-infra | 12:57 | |
| *** annegentle is now known as annegentle_away | 13:02 | |
| *** sandywalsh has quit IRC | 13:08 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:10 | |
| *** mriedem has joined #openstack-infra | 13:15 | |
| *** zul has quit IRC | 13:18 | |
| *** zul has joined #openstack-infra | 13:18 | |
| *** nosnos has quit IRC | 13:26 | |
| *** dprince has joined #openstack-infra | 13:27 | |
| *** lnxnut has joined #openstack-infra | 13:29 | |
| sdague | hub_cap: I'll try to look today, it's been a kind of crazy week | 13:32 |
| *** ssafron has joined #openstack-infra | 13:32 | |
| ssafron | Is it possible to delete an Etherpad from https://etherpad.openstack.org? I accidentally put some sensitive information on one, and now it looks like I cannot get rid of the history. | 13:36 |
| anteaya | ssafron: when fungi is around perhaps you can send him an email with the link to the etherpad in question | 13:37 |
| anteaya | or pm him (your choice) | 13:37 |
| *** changbl has joined #openstack-infra | 13:38 | |
| ssafron | OK thanks anteaya | 13:38 |
| anteaya | I hope he can do something to help | 13:38 |
| sdague | ssafron: the way etherpads do versioning this isn't very easy | 13:38 |
| sdague | I think the only option is to nuke the whole thing, but that's not really an option if it's one community members have been using | 13:39 |
| *** dkranz has joined #openstack-infra | 13:39 | |
| ssafron | Nope, this was a new etherpad just used by two people one day. So I'd be fine if it just gets totally deleted. | 13:40 |
| fungi | ssafron: i'll have to take a look at the db schema. if it's a fairly safe and simple set of delete queries then i'm happy to redact it for you | 13:43 |
| fungi | ssafron: just /msg me the url | 13:43 |
| *** markmcclain has quit IRC | 13:45 | |
| *** lnxnut has quit IRC | 13:50 | |
| *** burt has joined #openstack-infra | 13:53 | |
| *** alexpilotti has quit IRC | 13:53 | |
| *** alexpilotti has joined #openstack-infra | 13:56 | |
| *** safron has joined #openstack-infra | 14:00 | |
| *** ssafron has quit IRC | 14:00 | |
| *** safron has quit IRC | 14:02 | |
| *** safron has joined #openstack-infra | 14:02 | |
| *** dizquierdo has joined #openstack-infra | 14:03 | |
| *** vogxn has joined #openstack-infra | 14:04 | |
| anteaya | fungi from what I am reading puppet-dashboard needs a puppet-master to talk to, the master sends requests to the dashboard | 14:05 |
| anteaya | I am not seeing how to get a puppet agent talking to the dashboard | 14:06 |
| anteaya | is this consistent with your understanding of the puppet-dashboard? | 14:06 |
| *** markmcclain has joined #openstack-infra | 14:07 | |
| fungi | anteaya: what we have in /etc/puppet/puppet.conf on our servers includes a line for it in the [master] section... | 14:07 |
| fungi | reporturl=http://puppet-dashboard.openstack.org:3000/reports/upload | 14:07 |
| *** alexpilotti has quit IRC | 14:07 | |
| anteaya | ah ha, thanks | 14:07 |
| fungi | i believe that configures each server to report directly to the dashboard server | 14:08 |
| anteaya | each server, meaning each puppet-master? | 14:08 |
| fungi | no, each puppeted machine | 14:08 |
| fungi | each server running an agent | 14:09 |
| fungi | (i snarfed that example directly from review.openstack.org but it's on all of them) | 14:09 |
| anteaya | really, okay I will see if inserting that line into puppet.conf on the agent I am running gives me any results | 14:09 |
| anteaya | okay thanks | 14:10 |
| fungi | in the [master] section we also have: | 14:11 |
| fungi | reports=store, http | 14:11 |
| fungi | and then in the [agent] section: | 14:11 |
| fungi | report=true | 14:11 |
| fungi | i'm running tcpdump on a server right now to see if it's actually trying to connect directly to puppet-dashboard.openstack.org to submit reports, just as a sanity check | 14:12 |
| fungi | might take up to 10 minutes to know for sure | 14:12 |
| anteaya | thank you | 14:14 |
| *** dkliban has joined #openstack-infra | 14:15 | |
| *** alexpilotti has joined #openstack-infra | 14:15 | |
| *** pabelanger_ has joined #openstack-infra | 14:16 | |
| *** pabelanger__ has joined #openstack-infra | 14:16 | |
| *** pabelanger__ has quit IRC | 14:17 | |
| *** pabelanger__ has joined #openstack-infra | 14:17 | |
| *** pabelanger has quit IRC | 14:17 | |
| *** pabelanger__ is now known as pabelanger | 14:17 | |
| *** pabelanger_ has quit IRC | 14:17 | |
| *** pabelanger_ has joined #openstack-infra | 14:17 | |
| *** pabelanger has quit IRC | 14:17 | |
| *** pabelanger has joined #openstack-infra | 14:18 | |
| *** dcramer_ has joined #openstack-infra | 14:19 | |
| *** mrodden has quit IRC | 14:24 | |
| *** annegentle_away is now known as annegentle | 14:24 | |
| *** adalbas has quit IRC | 14:28 | |
| *** dkliban has quit IRC | 14:30 | |
| *** afazekas has quit IRC | 14:30 | |
| fungi | mmm, you may be right. i don't see the server trying to open a socket to the dashboard to report anything. lemme make sure the agent is running and not stuck | 14:30 |
| *** senk1 has joined #openstack-infra | 14:33 | |
| sld | is there anyone around that can answer a quick jenkins job question? | 14:33 |
| Mithrandir | that depends on what the question is | 14:33 |
| Mithrandir | don't ask to ask | 14:34 |
| sld | gotcha | 14:34 |
| fungi | sld: yeah, just ask and we'll answer if we know ;) | 14:34 |
| *** mrodden has joined #openstack-infra | 14:34 | |
| sdague | fungi: the db schema for etherpad is "interesting" :) | 14:35 |
| sld | looking to clean remote ftp directory, and then put newly sphinx-generated content there...the problem is this -- after the clean remote is in place, and while the ftp upload is in process, there is no site content.. are there any other suggestions for ways to accomplish the same thing, but maybe done a different way? | 14:35 |
| fungi | sdague: yes, i ended up using the api, which worked (somewhat to my surprise) | 14:35 |
| *** atiwari has joined #openstack-infra | 14:36 | |
| sdague | fungi: nice, you got an example? I think I only managed to bork up things the last time I tried to do that | 14:36 |
| Mithrandir | sld: rsync with --delete-after? | 14:36 |
| fungi | sld: so that's more of an ftp publisher plugin question i guess. for openstack, we don't use the clean remote option... we just dump new files on top of old ones and accept (currently) that cruft will pile up | 14:36 |
| sld | fungi: exactly - that is what i am trying to solve for... cleaning that. ;) | 14:37 |
| fungi | but yes, something like what Mithrandir suggests would be cleaner. no idea of jenkins has an rsync pubisher plugin or if you'd have to hack something up for it | 14:37 |
| *** senk1 has quit IRC | 14:37 | |
| sld | i didn't see any rsync plugin on the list, but if i need to make a new one, that's fine too. | 14:37 |
| *** pentameter has joined #openstack-infra | 14:37 | |
| sld | err rsync publisher, rather. | 14:37 |
| Mithrandir | you could also use the ftp publisher and have something that rm's based on ctime. | 14:38 |
| *** senk1 has joined #openstack-infra | 14:38 | |
| fungi | in our case, we're using ftp with a somewhat braindead web hosting service which we can't install arbitrary binaries ike rsync onto, so we haven't really played around with it. we do have jobs which generate content and then rsync them to an arbitrary server via ssh however | 14:38 |
| fungi | and that works fairly well | 14:38 |
| sld | if there is any way to pass options to rsync, ...that might be good. ;-) | 14:39 |
| fungi | sdague: yes, writing something up to add to our server documentation here in a bit | 14:39 |
| sld | i'll scour source for an rsync example... thanks. :) | 14:39 |
| Mithrandir | or you could have a symlink that you make sure sorts last and which is then flipped afterwards. | 14:39 |
| Mithrandir | so you publish to foo-$jobid and have a foo/ symlink pointing to the right place | 14:39 |
| fungi | sld: yes, in our case the job is parameterized and passes things into a shell script which ends with the rsync to put the results into place, rather than relying on an explicit jenkins publisher | 14:40 |
| anteaya | fungi k | 14:40 |
| jeblair | fungi: that requires a level of access that i don't think we want to give the builders | 14:40 |
| fungi | jeblair: right, for our doc jobs i don't think it's a good idea | 14:41 |
| jeblair | fungi: i think sld is talking about our jobs :) | 14:41 |
| fungi | oh! | 14:41 |
| sld | fungi: can you think of a job that does that, that i can look at as an example? | 14:41 |
| fungi | i totally misread and thought he was talking about another jenkins | 14:41 |
| Mithrandir | jeblair: not really, you can use command= in the authorized_keys file or one of the rsynconly scripts. | 14:41 |
| *** vogxn has quit IRC | 14:42 | |
| *** SergeyLukjanov has quit IRC | 14:42 | |
| jeblair | Mithrandir: yes, that's possible once we restructure where we publish docs | 14:43 |
| Mithrandir | well, that's something I have no idea how's done today. :-) | 14:43 |
| Mithrandir | just saying you don't need to give out extra perms to the builders. | 14:43 |
| annegentle | jeblair: one idea we're thinking of is docs-draft for everything published from master | 14:43 |
| jeblair | Mithrandir: two things need to change: 1) use a site we can ssh to instead of ftp, and 2) generate special purpose keys, so the same key isn't used for publishing logs or tarballs as docs | 14:43 |
| *** senk has joined #openstack-infra | 14:43 | |
| annegentle | jeblair: sld is working through it but wanted to throw that out there | 14:44 |
| jeblair | Mithrandir: but yeah, if we did that, we could use command= | 14:44 |
| Mithrandir | (I'd also like a central-publish plugin that grabs the bits to be published, sends them to the master and the master does the publishing, so you don't need to let arbitrary slaves log in lots of places) | 14:44 |
| annegentle | doh afk for a bit | 14:44 |
| *** adalbas has joined #openstack-infra | 14:44 | |
| jeblair | Mithrandir: well, that's what happens with most publishers, so that's the behavior that ftp, scp, and ssh publishers have now | 14:44 |
| *** senk2 has joined #openstack-infra | 14:44 | |
| *** pblaho has quit IRC | 14:45 | |
| Mithrandir | jeblair: oh, ok. It used not to be that way, so if that's fixed, that's great. :-) | 14:45 |
| fungi | i suppose one workaround to not letting arbitrary build slaves rsync to the (eventual not rackspace cloudsites) webserver would be to generate and archive the content, then have a follow-on job which retrieves it on a trusted slave and does the rsync | 14:46 |
| jeblair | Mithrandir: i just double checked -- the ssh logins for the scp publishing plugin only come from our two masters | 14:46 |
| fungi | which i guess is more or less what Mithrandir just suggested as well | 14:46 |
| jeblair | Mithrandir: i'm assuming the others work similarly | 14:46 |
| *** senk has quit IRC | 14:48 | |
| *** senk1 has quit IRC | 14:48 | |
| jeblair | on further thought, there's one other 'security' aspect that we get from the current config, and that is that the destination path is configured as part of the job | 14:48 |
| Mithrandir | you get that with the rsynconly shell things too | 14:49 |
| jeblair | if the 'docs' ssh key were available inside the job (and not just to jenkins) then it means that any docs job could alter all docs sites, whereas currently, the worst it can do is alter its subset (eg, nova) | 14:49 |
| *** CaptTofu has quit IRC | 14:50 | |
| Mithrandir | then have one ssh key per job? | 14:50 |
| jeblair | the artifact storage system i wrote about in an email to openstack-infra could probably solve this problem too. | 14:51 |
| *** senk2 has quit IRC | 14:51 | |
| *** CaptTofu has joined #openstack-infra | 14:51 | |
| *** afazekas has joined #openstack-infra | 14:51 | |
| jeblair | (since it's designed to scope things by job) | 14:51 |
| sld | when was it? or do you have a link handy? | 14:52 |
| jeblair | Mithrandir: yes, i think that would work. | 14:52 |
| jeblair | sld: http://lists.openstack.org/pipermail/openstack-infra/2013-September/000238.html | 14:52 |
| *** senk has joined #openstack-infra | 14:53 | |
| *** rnirmal has joined #openstack-infra | 14:54 | |
| sld | thx...i'll finish reading it tonight when i get back to the office...headed home and to bed now. :) | 14:55 |
| *** senk1 has joined #openstack-infra | 14:56 | |
| anteaya | fungi: was the puppet agent stuck? | 14:56 |
| *** sld is now known as sld-away | 14:56 | |
| fungi | anteaya: no, i got sidetracked when i discovered we have servers where puppet is complaining that the pip provider is broken | 14:57 |
| anteaya | ah okay | 14:57 |
| *** DennyZhang has joined #openstack-infra | 14:58 | |
| fungi | current suspicion is that a recent non-distro-package upgrade of pip has moved the binary to a path where puppet doesn't know to look for it | 14:58 |
| fungi | trying to nail this down real quick | 14:58 |
| *** senk has quit IRC | 14:58 | |
| anteaya | yup | 15:00 |
| dhellmann_ | good morning everyone | 15:00 |
| *** dhellmann_ is now known as dhellmann | 15:00 | |
| anteaya | good morning dhellmann | 15:00 |
| dhellmann | things seem quiet, so I'm hoping that means people have the bandwidth to review https://review.openstack.org/#/c/46106/ today :-) | 15:01 |
| *** mrodden1 has joined #openstack-infra | 15:02 | |
| *** thedodd has joined #openstack-infra | 15:02 | |
| *** mrodden has quit IRC | 15:02 | |
| fungi | ha, apparently lifeless already tripped over this issue a couple weeks ago... bug 1218696 | 15:03 |
| uvirtbot | Launchpad bug 1218696 in openstack-ci "new puppetmaster ends up with broken pip provider" [Undecided,New] https://launchpad.net/bugs/1218696 | 15:03 |
| *** senk1 has quit IRC | 15:03 | |
| fungi | though it's not necessarily just puppet master servers affected. i see it on our gerrit servers | 15:04 |
| anteaya | hmmm | 15:06 |
| anteaya | I'll just go ahead and work on installing a puppet-master to talk to the dashboard | 15:06 |
| anteaya | and if you get a chance later, I would be interested in your tcpdump findings regarding puppet agent/dashboard communication or lack thereof | 15:07 |
| anteaya | I'm referring to my little test infra | 15:07 |
| *** dkliban has joined #openstack-infra | 15:11 | |
| fungi | jeblair: did you mean to leave puppet stopped on zuul yesterday? | 15:14 |
| jeblair | fungi: i did not; oops. | 15:15 |
| fungi | just making sure | 15:15 |
| jeblair | fungi: it should be safe to restart now if you want | 15:15 |
| fungi | done | 15:15 |
| *** pcrews has joined #openstack-infra | 15:15 | |
| *** senk1 has joined #openstack-infra | 15:16 | |
| *** dcramer_ has quit IRC | 15:17 | |
| *** odyssey4me has quit IRC | 15:17 | |
| *** mrodden has joined #openstack-infra | 15:18 | |
| *** senk has joined #openstack-infra | 15:19 | |
| *** pycabrera has joined #openstack-infra | 15:19 | |
| *** senk1 has quit IRC | 15:21 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:21 | |
| *** mrodden1 has quit IRC | 15:23 | |
| *** mrodden1 has joined #openstack-infra | 15:26 | |
| *** mrodden has quit IRC | 15:26 | |
| *** UtahDave has joined #openstack-infra | 15:27 | |
| *** prad has joined #openstack-infra | 15:28 | |
| *** dcramer_ has joined #openstack-infra | 15:29 | |
| fungi | anteaya: yeah, trying a few systems, i don't see any of the agents trying to report to the dashboard directly, only the master server seems to want to report to the dashboard | 15:32 |
| ekarlso- | would it be easy to make JJB accept multiple directories? | 15:34 |
| *** markmcclain has quit IRC | 15:35 | |
| *** bashok has quit IRC | 15:36 | |
| *** rnirmal_ has joined #openstack-infra | 15:39 | |
| anteaya | fungi: thanks, your findings are consistent with the puppetlabs documentation | 15:40 |
| *** branen has joined #openstack-infra | 15:40 | |
| anteaya | so my assumption that I could get an agent talking to the dashboard is wrong | 15:40 |
| hub_cap | sdague: hey i got a +2 from dtroyer, can you look @ trove+devstack https://review.openstack.org/#/c/38169/ | 15:40 |
| *** rnirmal has quit IRC | 15:40 | |
| *** rnirmal_ is now known as rnirmal | 15:40 | |
| hub_cap | then i can say im making progress to the infra guys when they ask about trove gating tests!! | 15:40 |
| *** pabelanger has quit IRC | 15:44 | |
| *** rnirmal has quit IRC | 15:50 | |
| *** rnirmal has joined #openstack-infra | 15:51 | |
| *** KennethWilke has joined #openstack-infra | 15:51 | |
| fungi | okay, so restarting the puppet agent daemon solves the dilemma of the moving pip executable. bug 1218696 updated with relevant info | 15:53 |
| uvirtbot | Launchpad bug 1218696 in openstack-ci "new puppetmaster ends up with broken pip provider" [Low,Triaged] https://launchpad.net/bugs/1218696 | 15:53 |
| *** mgagne has quit IRC | 15:54 | |
| pleia2 | anteaya: doing some headache triage away from the screen for a bit, hopefully will be back in not too long :) | 15:54 |
| anteaya | :( | 15:54 |
| anteaya | let me see what I can do with the notes you left me | 15:55 |
| pleia2 | ok, great | 15:55 |
| anteaya | take care of your headache | 15:55 |
| *** pabelanger_ has quit IRC | 15:55 | |
| *** pabelanger has joined #openstack-infra | 15:56 | |
| *** svarnau has joined #openstack-infra | 15:59 | |
| *** mgagne has joined #openstack-infra | 16:00 | |
| *** mgagne has joined #openstack-infra | 16:00 | |
| *** lnxnut_ has joined #openstack-infra | 16:01 | |
| *** boris-42_ has joined #openstack-infra | 16:02 | |
| *** marun has joined #openstack-infra | 16:02 | |
| *** boris-42_ has quit IRC | 16:04 | |
| *** mgagne1 has joined #openstack-infra | 16:04 | |
| *** mgagne1 has joined #openstack-infra | 16:04 | |
| *** boris-42 has quit IRC | 16:06 | |
| *** lnxnut_ has quit IRC | 16:06 | |
| *** mgagne has quit IRC | 16:06 | |
| *** senk has quit IRC | 16:08 | |
| *** thedodd has quit IRC | 16:10 | |
| *** fbo is now known as fbo_away | 16:11 | |
| *** mgagne1 has quit IRC | 16:13 | |
| *** changbl has quit IRC | 16:14 | |
| *** dcramer_ has quit IRC | 16:16 | |
| *** fbo_away is now known as fbo | 16:24 | |
| *** rnirmal has quit IRC | 16:25 | |
| *** rnirmal has joined #openstack-infra | 16:25 | |
| *** dims has quit IRC | 16:25 | |
| *** mgagne has joined #openstack-infra | 16:26 | |
| *** mgagne has joined #openstack-infra | 16:26 | |
| UtahDave | fungi: did you get a chance to test a Salt Jenkins slave? | 16:27 |
| fungi | UtahDave: i'm currently playing with determining why we're not autoupgrading our salt packages to the available later version... http://paste.openstack.org/show/46888/ | 16:28 |
| fungi | UtahDave: don't want to revert the change where we disabled it until i can confirm we're properly upgrading | 16:29 |
| fungi | need to spot check a few more systems still | 16:29 |
| *** markmcclain has joined #openstack-infra | 16:30 | |
| fungi | looks like it's upgrading correctly on our centos6 slaves but not on precise | 16:31 |
| *** moted has quit IRC | 16:31 | |
| *** moted has joined #openstack-infra | 16:32 | |
| fungi | i think possibly our auto package updates may not upgrade all packages, but instead only from certain package repositories. i'll test that out | 16:32 |
| *** danger_fo_away is now known as danger_fo | 16:34 | |
| fungi | ahh yes, we need to add an entry to Unattended-Upgrade::Allowed-Origins | 16:34 |
| *** dcramer_ has joined #openstack-infra | 16:34 | |
| clarkb | morning | 16:36 |
| *** fbo is now known as fbo_away | 16:36 | |
| *** dims has joined #openstack-infra | 16:37 | |
| jeblair | clarkb: is it correct to say that the swift hpcloud issue is that they don't recognize directory markers when generating indexes? | 16:39 |
| UtahDave | fungi: ok, good. | 16:39 |
| fungi | assuming this test works as intended, i'll upload the auto-updates fix, the one to reenable salt, one to make a small adjustment to the launch instructions for the new --salt option and the one to puppetize the special-purpose trigger slave as a stack of reviews | 16:39 |
| jeblair | clarkb: or they don't support setting directory markers at all, or ...? | 16:39 |
| clarkb | jeblair: it is a little more subtle than that | 16:40 |
| clarkb | jeblair: they seem to do the correct thing for your private view of containers, but not in the public view | 16:40 |
| clarkb | I assume to reduce load as private is presumably going to be accessed less often? | 16:40 |
| *** reed has joined #openstack-infra | 16:40 | |
| fungi | presumably similar to why rackspace only allows accessing public swift object urls via cdn | 16:41 |
| UtahDave | fungi: that sounds great. Will I need to make any changes to my review? I think mine still is set to work on all jenkins slaves. | 16:41 |
| jeblair | clarkb: ok, so is 'they don't recognize directory markers when generating index pages for the public view of containers' accurate enough to commit to a mailing list archive? :) | 16:41 |
| clarkb | jeblair: I think so, it is possible that they do support it afterall but I couldn't find documentation to that effect | 16:41 |
| fungi | UtahDave: yeah, we'll rebase yours on top of the change which adds that slavce | 16:41 |
| fungi | slave | 16:41 |
| fungi | and make some minor tweaks to have it be specific to that host | 16:42 |
| UtahDave | fungi: great | 16:42 |
| notmyname | clarkb: jeblair: "when generating index pages" as in swift's staticweb feature? | 16:42 |
| clarkb | notmyname: aye | 16:42 |
| notmyname | clarkb: ah, so perhaps we have a bug in staticweb? | 16:42 |
| clarkb | notmyname: or pebkac | 16:42 |
| clarkb | with me being between chair and keyboard | 16:43 |
| fungi | clark exists between chair and keyboard | 16:43 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:44 | |
| clarkb | jeblair: notmyname: let me give it another shot right now | 16:45 |
| *** clayg has joined #openstack-infra | 16:46 | |
| clayg | If I'm diggin' on a failed gate job logs, is there anyway to get back to the jenkins job-id? | 16:47 |
| clayg | like I have http://logs.openstack.org/91/46191/3/check/gate-tempest-devstack-vm-postgres-full/ab2d8fe/ - but I want https://jenkins.openstack.org/view/All/job/gate-tempest-devstack-vm-postgres-full/??????/ | 16:48 |
| *** dcramer_ has quit IRC | 16:49 | |
| jeblair | notmyname: FYI http://lists.openstack.org/pipermail/openstack-infra/2013-September/000238.html (we're trying very hard to store build artifacts in swift) | 16:51 |
| jeblair | clayg: why? what's missing from the logs.o.o site? | 16:51 |
| clayg | jeblair: just curious I suppose? | 16:52 |
| jeblair | clayg: (i am thinking about your question; it's difficult) | 16:53 |
| clayg | jeblair: I mean I *am* sorta at the moment sorta wondering about "Other changes tested concurrently with this change" | 16:54 |
| jeblair | clayg: I think it's not possible to do directly; you'd have to hit each of the jenkins masters and look at all the jobs from around that time and examine the parameters | 16:54 |
| *** mgagne has quit IRC | 16:54 | |
| jeblair | clayg: we could probably add some output to that to make it easier to trace back | 16:54 |
| clayg | jeblair: so but surely the... :D | 16:54 |
| jeblair | clayg: we can also add 'Other changes tested concurrently with this change' output to the console log as well | 16:55 |
| jeblair | clayg: the reason why we have not made it easy to go in the direction you're asking about... | 16:55 |
| jeblair | clayg: is that build logs are deleted from jenkins fairly quickly (i think they stay around for 24 hours) | 16:55 |
| jeblair | clayg: logs.o.o logs stay around for 6 months | 16:55 |
| clayg | nice | 16:56 |
| *** mgagne has joined #openstack-infra | 16:57 | |
| *** mgagne has joined #openstack-infra | 16:57 | |
| jeblair | clayg: you can fetch the zuul ref and get exactly what was tested for each project | 16:57 |
| jeblair | clayg: (and if you look at git log, you should see where zuul merged the other changes) | 16:57 |
| jeblair | clayg: look at http://logs.openstack.org/91/46191/3/check/gate-tempest-devstack-vm-postgres-full/ab2d8fe/logs/devstack-gate-setup-workspace-new.txt.gz | 16:57 |
| jeblair | clayg: and looke for lines like: git fetch http://zuul.openstack.org/p/openstack/python-neutronclient refs/zuul/master/Z3b4fde5321bf4207889978b69131ae87 | 16:58 |
| *** ruhe has joined #openstack-infra | 16:58 | |
| jeblair | clayg: so you can always use that to locally checkout and examine exactly what was tested | 16:58 |
| clayg | cool - didn't know about that - helpful | 16:59 |
| *** mdenny has joined #openstack-infra | 16:59 | |
| *** Ryan_Lane has joined #openstack-infra | 17:00 | |
| *** hashar has joined #openstack-infra | 17:01 | |
| *** senk has joined #openstack-infra | 17:01 | |
| *** senk has quit IRC | 17:02 | |
| *** DennyZhang has quit IRC | 17:02 | |
| clarkb | jeblair: so some pebkac apparently https://region-a.geo-1.objects.hpcloudsvc.com/v1/89381323713085/jenkinstesting2/ but it isn't perfect | 17:03 |
| *** dcramer_ has joined #openstack-infra | 17:03 | |
| clarkb | jeblair: notice that dir names are listed twice, once as an actual object and again rendered as a dir | 17:03 |
| jeblair | notmyname: ^ | 17:03 |
| jeblair | clarkb: what's 89381323713085 ? | 17:04 |
| clarkb | jeblair: magical public url for my tenant+project I think | 17:04 |
| *** thedodd has joined #openstack-infra | 17:04 | |
| clarkb | I am going to disable the rendering to see if foo/ and foo are distinct objects | 17:05 |
| *** jpich has quit IRC | 17:05 | |
| *** Bada has joined #openstack-infra | 17:05 | |
| clarkb | apparently setting web-listings to false isn't the correct way to do that | 17:06 |
| *** dcramer_ has quit IRC | 17:06 | |
| clayg | clarkb: is this staticweb? | 17:06 |
| clarkb | clayg: yes | 17:07 |
| *** gyee has joined #openstack-infra | 17:07 | |
| *** nati_ueno has joined #openstack-infra | 17:07 | |
| clarkb | http://paste.openstack.org/show/46896/ maybe foo and testing are not needed? | 17:08 |
| clayg | clarkb: the explicity directory markers aren't needed unless you need empty directories | 17:08 |
| clayg | not sure why static web isn't parsing them out and rolling them up - that may be a bug | 17:08 |
| clarkb | I think jclouds creates those because other object stores want them | 17:08 |
| clayg | hrmmm... might be static web wants a specific content type | 17:09 |
| *** pcrews has quit IRC | 17:10 | |
| clayg | X-Container-Meta-Web-Directory-Type - default is application/directory? | 17:10 |
| clarkb | clayg: no clue, how do I check? | 17:11 |
| clayg | swift stat jenkinstesting2 foo | 17:11 |
| *** nati_ueno has quit IRC | 17:12 | |
| clarkb | http://paste.openstack.org/show/46898/ | 17:12 |
| clayg | soo that looks right :P | 17:13 |
| *** ruhe has quit IRC | 17:13 | |
| clarkb | clayg: it is also possible this is a bug in whatever version of swift hp has deployed | 17:13 |
| clarkb | and may not be a problem in current swift | 17:13 |
| jeblair | ha we're talking about swift here and infra in #openstack-swift | 17:13 |
| jeblair | oh! you only need directory markers for empty directories? i thought you needed them in order to 'list objects in this pseudo-directory'? | 17:15 |
| clarkb | jeblair: apparently. I can delete foo and see what happens | 17:15 |
| clarkb | foo deleted, that seems to make it happy | 17:16 |
| clarkb | (note jclouds would need an update if we want to use it this way) | 17:16 |
| clayg | clarkb: jeblair: looking at master I dont' see any code to sqeueeze them together - no i think staticweb needs a fix | 17:17 |
| *** pmoosh has joined #openstack-infra | 17:17 | |
| clarkb | clayg: cool I found a bug maybe | 17:19 |
| *** zehicle has quit IRC | 17:20 | |
| *** rockyg has joined #openstack-infra | 17:21 | |
| clayg | clarkb: I'm trying out a fix now - it's possible gholt is the only one that really *knows* how staticweb works | 17:21 |
| clayg | the rest of us just sorta poke at it with sticks | 17:21 |
| *** Ryan_Lane has quit IRC | 17:23 | |
| *** Ryan_Lane has joined #openstack-infra | 17:23 | |
| *** Ryan_Lane has quit IRC | 17:23 | |
| *** Ryan_Lane has joined #openstack-infra | 17:23 | |
| *** zehicle has joined #openstack-infra | 17:24 | |
| *** markmcclain has quit IRC | 17:27 | |
| *** markmcclain has joined #openstack-infra | 17:28 | |
| *** vipul is now known as vipul-away | 17:30 | |
| *** vipul-away is now known as vipul | 17:31 | |
| *** pcm__ has joined #openstack-infra | 17:31 | |
| clarkb | fungi: for the pip stuff you ran into, my observation was that once pip upgraded itself, puppet needed to be run again. I thought it was per catalog run and not per process | 17:32 |
| *** svarnau has quit IRC | 17:32 | |
| fungi | clarkb: on our unit test slaves where we run it from cron, yes | 17:33 |
| fungi | clarkb: on systems where the agent is running persistently as a daemon, that needs restarting | 17:33 |
| clarkb | I see. That is a bit annoying | 17:33 |
| fungi | yes, isn't it though | 17:33 |
| *** HenryG_ has joined #openstack-infra | 17:33 | |
| clarkb | puppet probably shouldn't make config assumptions for things between runs | 17:34 |
| *** pcm_ has quit IRC | 17:34 | |
| clarkb | jeblair: so for hpcloud I think we can probably make this work, we would need to deal with the public url location (proxy? or cave and use CDN and potentially use rackspace and hp?) | 17:35 |
| *** dcramer_ has joined #openstack-infra | 17:35 | |
| *** HenryG has quit IRC | 17:36 | |
| *** svarnau has joined #openstack-infra | 17:36 | |
| clarkb | fungi: was review.o.o one of the hosts with pip puppet trouble? trying to sort out if I should attempt a bup join on that host | 17:38 |
| *** dizquierdo has quit IRC | 17:39 | |
| clayg | clarkb: so listings don't really handle directory markers all that great :\ | 17:39 |
| fungi | clarkb: it was. i only restarted puppet on it and review-dev, so there may be others out there still showing symptoms | 17:39 |
| clayg | clarkb: this gets rid of the dupes -> http://paste.openstack.org/show/46903/ | 17:40 |
| *** dcramer_ has quit IRC | 17:40 | |
| clayg | but I really should make it a proper fix and promote the application/directory objects to be treated more like real subdir listing entries | 17:40 |
| clarkb | fungi: ok, so bup join tomorrow. thanks | 17:40 |
| fungi | clayg: also, it wasn't so much a config assumption... if you look at the ruby implementing the pip package provider, it just uses the unqualified command name but presumably the path to that ends up being remembered for subsequent invocations | 17:41 |
| clayg | at one time I wanted to just teach the container servers how to do it so up stream applications don't all have to independetly solve empty directories themslves | 17:41 |
| fungi | er, clarkb not clayg | 17:41 |
| clayg | my mere presense is causing problems | 17:41 |
| * fungi curses his own lazy nick tab completion habits | 17:41 | |
| *** SergeyLukjanov has quit IRC | 17:42 | |
| *** thomasm has quit IRC | 17:42 | |
| *** Bada has quit IRC | 17:42 | |
| clarkb | clayg: while you are here and have looked at that code. Any idea what performance for weblisting would be like when a container has hundreds of thousands of objects? | 17:42 |
| *** markmcclain has quit IRC | 17:43 | |
| clarkb | (maybe millions) | 17:43 |
| *** Bada has joined #openstack-infra | 17:43 | |
| clarkb | jeblair: ^ is another thing to consider, by maintaining an external index we can cache a lot of the info rather than reprocessing | 17:43 |
| jeblair | clayg: i think on balance you're solving more problems than causing, it's nice to have you around :) | 17:43 |
| *** UtahDave has quit IRC | 17:43 | |
| jeblair | clarkb: yeah, that's why i was initially thinking we'd have a shadow db; but if we use formpost, then that's not so easy | 17:44 |
| *** SergeyLukjanov has joined #openstack-infra | 17:44 | |
| jeblair | clarkb: btw, have you caught up on the email thread? | 17:44 |
| clarkb | jeblair: mostly I think | 17:44 |
| clayg | clarkb: the container does most of the work (of doing the sudeo nested) - and millions is ok, but not super, hundreds of thousands is probably fine | 17:45 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add saltstack APT repository to automatic upgrades https://review.openstack.org/46317 | 17:45 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Revert "Disable salt master and minions globally" https://review.openstack.org/46318 | 17:45 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add a Jenkins slave for triggering salt commands https://review.openstack.org/46319 | 17:45 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Mention the --salt option in launch instructions https://review.openstack.org/46320 | 17:45 |
| clayg | *pseudo | 17:45 |
| clarkb | jeblair: you are talking about making jobs only able to upload to one place | 17:45 |
| mrodden1 | so what version of distribute/setuptools should i be on... i can't "python setup.py develop" anything anymore | 17:46 |
| clarkb | jeblair: and possibly having zuul manage that with the swift formpost undocumetned thing | 17:46 |
| mrodden1 | and pbr complains that a bunch of build targets are missing | 17:46 |
| clarkb | mrodden1: setuptools > 0.7 | 17:46 |
| mrodden1 | UserWarning: Unknown distribution option: 'install_requires' | 17:46 |
| mrodden1 | is the error | 17:46 |
| clayg | ... well acctually for many quanties of "millions" millions is probably not fine, it's bad. But for < a few million we're back into "not super" | 17:46 |
| mrodden1 | clarkb: i have 1.1 i believe | 17:46 |
| clarkb | mrodden1: I wonder if you are running into the thing clayg found with setuptools lazy loading on top of distutils | 17:47 |
| clarkb | mrodden1: can you paste a complete traceback? | 17:47 |
| mrodden1 | clarkb: sure | 17:47 |
| jeblair | clayg: any individual pseudo-dir would only have up to thousands of entries; but with thousands of those, the whole container would have millions | 17:47 |
| mrodden1 | http://paste.openstack.org/show/46904/ | 17:48 |
| jeblair | clayg: ^ just to be more explicit about the load clarkb is asking about... | 17:48 |
| jeblair | clayg: does that change anything? | 17:48 |
| clayg | yeah... I mean many millions of objects in the container will make listings and adding lots of new objects at once sorta... slower than you want | 17:50 |
| *** rfolco has quit IRC | 17:50 | |
| clarkb | ya so we will probably need container rollover of some sort | 17:51 |
| clayg | jeblair: clarkb: with the cardinality you're thinking of having hundreds of thousands of x-web-listing containers with hundreds of thousands of objects would probably be a better plan | 17:51 |
| jeblair | clarkb: deja-vu! container rollover with the cdn is hard and the cdn is needed for staticweb on rackspace! | 17:51 |
| *** dcramer_ has joined #openstack-infra | 17:51 | |
| *** thomasbiege has joined #openstack-infra | 17:51 | |
| clarkb | jeblair: right! | 17:51 |
| fungi | this definitely sounds... familiar | 17:51 |
| * clayg doesn't have thing hung of the term "container rollover" | 17:52 | |
| * clayg raises KeyError | 17:52 | |
| jeblair | clayg: i think you and clark are saying similar things -- adding more containers makes this easier | 17:52 |
| jeblair | clayg: except that on rackspace, you need to use the cdn in order to use staticweb | 17:52 |
| jeblair | clayg: and the cdn is a random hostname per-container | 17:53 |
| *** boris-42 has joined #openstack-infra | 17:53 | |
| jeblair | clayg: which means that adding a new container means adding dns entries to openstack.org each time | 17:53 |
| clarkb | right every time we try to solve this completely with swift I end up thinking swift wasn't built for this | 17:53 |
| clarkb | swift is meant to back an application | 17:53 |
| jeblair | clayg: which brings dns ops into it | 17:53 |
| clarkb | if we write an application that manages all of this stuff it is fine, but there are a few moving pieces to consider | 17:53 |
| jeblair | i tried to suggest we write a swift application, but ttx and jhesketh have not received that idea very well. | 17:54 |
| jeblair | swift-backed application | 17:54 |
| clarkb | jeblair: ya | 17:54 |
| notmyname | jeblair: clayg: actually, the CDN URLs at RAX are deterministic, IIRC. you should be able to calculate them up front | 17:54 |
| clarkb | notmyname: they looked like a uuid | 17:54 |
| clayg | notmyname: they didn't used to be, but it's still a dns entry | 17:55 |
| fungi | the dns piece would be less worrisome with an actual openstack answer to dns management, but we likely don't want to implement automation based on rackspace's proprietary dns api | 17:55 |
| notmyname | clarkb: clayg: let me check... | 17:55 |
| jeblair | fungi: definitely not my first choice. also, it's worth taking a step back and noting "constantly changing dns entries" is not a design goal for us, but actually feels like a very hacky workaround. | 17:56 |
| fungi | that too, for sure | 17:56 |
| *** dcramer_ has quit IRC | 17:57 | |
| *** markmcclain1 has joined #openstack-infra | 17:57 | |
| notmyname | clarkb: clayg: looks like an HMAC with a secret component from the config, so you prob can't precompute it https://github.com/dpgoetz/sos/blob/master/sos/origin.py | 17:57 |
| clayg | notmyname: CF doesn't use domain remap at all huh? wildcard dns is out? | 17:58 |
| clarkb | jeblair: the HP non CDN url is at least deterministic once you know the root | 17:58 |
| clarkb | jeblair: not sure about the CDN urls | 17:58 |
| jeblair | clarkb: do you know if hpcloud support formpost? | 17:59 |
| notmyname | clayg: they don't use domain remap, but they have similar functionality through the cdn. but youd still have to have the entries set right | 17:59 |
| notmyname | clarkb: HP and RAX (at least used to) use the same CDN glue code, so they should behave the same way | 18:00 |
| jeblair | http://docs.hpcloud.com/api/object-storage#formpost-jumplink-span | 18:00 |
| jeblair | looks like it | 18:00 |
| *** nati_ueno has joined #openstack-infra | 18:00 | |
| openstackgerrit | Peter Mooshammer proposed a change to openstack/requirements: adding sphinxcontrib-programoutput to global requirements https://review.openstack.org/46325 | 18:01 |
| *** safron has quit IRC | 18:01 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix paths for glossary https://review.openstack.org/46326 | 18:01 |
| *** rockyg has quit IRC | 18:03 | |
| *** thomasm has joined #openstack-infra | 18:04 | |
| *** BobBall is now known as BobBallAway | 18:06 | |
| *** salv-orlando has quit IRC | 18:06 | |
| clarkb | notmyname: any idea why CDN would be done that way? seems like with the tenant + project + container you can deterministically create unique keys | 18:07 |
| *** nicedice has joined #openstack-infra | 18:07 | |
| notmyname | clarkb: actually are really cool feature of it is that the edge (Akamai in both cases here) can validate requests. this can provide _huge_ wins for protecting the origin servers (ie swift) | 18:08 |
| notmyname | clarkb: IOW DDOS'ing RAX or HP actually turn into trying to DDOS Akamai (good luck!) | 18:09 |
| *** dcramer_ has joined #openstack-infra | 18:09 | |
| *** pcrews has joined #openstack-infra | 18:09 | |
| portante | notmyname: where is the code for that cdn stuff, do you know? | 18:10 |
| notmyname | clarkb: but I may be reading that code wrong (I didn't spend too much tim on it). let me see if I can ask someone | 18:10 |
| clarkb | notmyname: but akamai does that for you regardless of the DNS record | 18:10 |
| notmyname | portante: it's the link I pasted above | 18:10 |
| notmyname | clarkb: no, edge requests are pattern matched to an origin. and since there are multiple endpoints, the pattern match ends up being (for RAX) *.<region>.rackcdn.com | 18:12 |
| *** pblaho has joined #openstack-infra | 18:12 | |
| *** hashar has quit IRC | 18:12 | |
| *** ruhe has joined #openstack-infra | 18:12 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Document how to delete a pad from Etherpad Lite https://review.openstack.org/46329 | 18:12 |
| notmyname | clarkb: where * is any one of containers that have been created. so by adding an hmac, an invalid domain can be immediately rejected without querying the origin | 18:13 |
| notmyname | clarkb: ie the attack would be <uuid>.<region>.rackcdn.com | 18:13 |
| clarkb | I see | 18:13 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Ensure Conditional Builder produces valid class paths https://review.openstack.org/45960 | 18:14 |
| *** DennyZhang has joined #openstack-infra | 18:14 | |
| mordred | morning all | 18:18 |
| clarkb | jeblair: are etherpad(-dev), review(-dev), wiki, and the new groups servers the only servers that need bup? | 18:18 |
| clarkb | jeblair: maybe paste? | 18:18 |
| mordred | clarkb: might as well bup paste | 18:20 |
| mordred | clarkb: I think the db there is pretty small anyway | 18:20 |
| clarkb | mordred: yeah. mysqldump should be friends with drizzle right? | 18:20 |
| mordred | yup. well, I'd actually use drizzledump - but it should work about the same | 18:20 |
| mordred | clarkb: maybe we should make paste the next server to use cloud databases | 18:21 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Document how to delete a pad from Etherpad Lite https://review.openstack.org/46329 | 18:21 |
| clarkb | mordred: wfm. To go a completely different direction is paste something we should continue to run? | 18:22 |
| *** Ryan_Lane has quit IRC | 18:22 | |
| mordred | hm | 18:22 |
| mordred | well, I use it | 18:22 |
| mordred | and it doesn't take a lot of admin resources from us | 18:22 |
| *** afazekas is now known as afazekas_zz | 18:23 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Document how to delete a pad from Etherpad Lite https://review.openstack.org/46329 | 18:25 |
| * fungi is having a bad spelling day. have to remember to 'set spell' in vim | 18:26 | |
| fungi | guess i ought to just have it always turn that on for rst and md files | 18:26 |
| *** hashar has joined #openstack-infra | 18:28 | |
| *** Ryan_Lane has joined #openstack-infra | 18:28 | |
| *** vipul is now known as vipul-away | 18:30 | |
| *** UtahDave has joined #openstack-infra | 18:31 | |
| *** melwitt has joined #openstack-infra | 18:31 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Document source location as git.openstack.org https://review.openstack.org/42479 | 18:33 |
| clarkb | fungi: but then we don't get to enjoy fun tyops | 18:33 |
| fungi | clarkb: i promise i'll still make plenty of fun typos (more than my fair share, probably) | 18:34 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix publishing of High Availibility Guide https://review.openstack.org/46333 | 18:34 |
| *** matty_dubs|lunch is now known as matty_dubs | 18:34 | |
| *** yolanda has quit IRC | 18:35 | |
| *** vipul-away is now known as vipul | 18:37 | |
| *** atiwari has quit IRC | 18:39 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Minify jquery visibility https://review.openstack.org/46335 | 18:40 |
| *** atiwari has joined #openstack-infra | 18:40 | |
| *** mrmartin has joined #openstack-infra | 18:41 | |
| dkranz | clarkb: Did you see the query from sbaker in this channel about why https://review.openstack.org/#/c/45967/ is failing? | 18:42 |
| jeblair | clarkb: i'm not sure we should worry too much about the dev servers | 18:42 |
| clarkb | dkranz: I did not | 18:42 |
| *** rfolco has joined #openstack-infra | 18:42 | |
| clarkb | jeblair: ok | 18:42 |
| jeblair | clarkb: i just started with jenkins-dev so i didn't step on any toes | 18:42 |
| dkranz | clarkb: You gave a +1 but jenkins fails with what looks like a real error | 18:42 |
| dkranz | clarkb: But we don't understand its nature. | 18:42 |
| clarkb | dkranz: https://review.openstack.org/#/c/46107/ | 18:43 |
| clarkb | jeblair: fungi ^ that could use review | 18:43 |
| fungi | lookin | 18:43 |
| jeblair | me too | 18:44 |
| *** Bada has quit IRC | 18:44 | |
| dkranz | clarkb: I was pointing to https://review.openstack.org/#/c/45967/ | 18:44 |
| clarkb | dkranz: yes, 46107 fixes the problem 45967 has | 18:44 |
| lifeless | jeblair: when convenient I have a q on https://review.openstack.org/#/c/44980/ | 18:44 |
| dkranz | clarkb: Ah, I see. Great. | 18:44 |
| clarkb | dkranz: should explain the issue as well | 18:44 |
| jeblair | mordred, clarkb, fungi: dev-requirements will not get sync'd? | 18:45 |
| fungi | clarkb: on the topic of last night's bouncy castle discussion, i noticed that not only are we carting around bcprov-jdk16-144.jar but also mysql-connector-java-5.1.10.jar (seeing what i can do to get it added to puppet) | 18:45 |
| *** Bada has joined #openstack-infra | 18:45 | |
| clarkb | fungi: for bcprov I think you can do what you did for bcpg. not sure about mysql-connector | 18:46 |
| jeblair | lifeless: what's up? | 18:46 |
| fungi | clarkb: yeah, i think so but i'm checking to make sure gerrit isn't married to specific revs of those | 18:46 |
| clarkb | jeblair: good point, I think we may need global-requirements and dev-requirements in the list of reqs files | 18:46 |
| clarkb | mordred: ^ | 18:46 |
| fungi | mmm, dev-requirements is a good question... that's where we were stuffing non-pypi tarball urls from now on right? | 18:47 |
| lifeless | jeblair: I put it n the comments, but - I basically need guidance about what will make the patch ok; I don't have your environment or your dns tool to test with :) | 18:48 |
| hashar | hey there :-] I am trying to package up the 'gear' python module, but get blocked by setup.cfg missing a version: "": Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository.""" | 18:48 |
| hashar | has anyone ever packaged a pbr based module using a source tar ball ? | 18:48 |
| hashar | the lame error I got is http://paste.openstack.org/show/46913/ :D | 18:49 |
| mordred | jeblair, clarkb: what fungi said - dev-requirements is just a place to put non-pypi tarball links which are under our control already | 18:49 |
| *** yolanda has joined #openstack-infra | 18:49 | |
| mordred | hashar: yes | 18:49 |
| mordred | you need to use an actual source tarball though | 18:49 |
| jeblair | mordred: so it doesn't get synced? and that job shouldn't test it? | 18:49 |
| mordred | hashar: how did you get your tarball? | 18:49 |
| hashar | mordred: I am using git build package which extract the tar ball in some build area :/ | 18:50 |
| mordred | jeblair: it does get synced - I'm torn as to whether that job should test it | 18:50 |
| mordred | hashar: if you are using git buildpackage- you're going to need to inject a generated tarball using the import-orig command | 18:50 |
| jeblair | mordred: how could testing it hurt? | 18:50 |
| *** KennethWilke has quit IRC | 18:51 | |
| mordred | jeblair: fair enough | 18:51 |
| fungi | hashar: a bunch of pbr-based openstack components are already packaged by GheRivero and zigo for debian (under the debian-openstack alioth team) | 18:51 |
| clarkb | mrodden1: sorry got busy | 18:51 |
| fungi | hashar: and i think i remember they have a wiki up describing their packaging workflow, though it may not be current | 18:51 |
| hashar | mordred: will tried out. I simply copied pushed your master branch to my upstream branch :D | 18:51 |
| clarkb | mrodden1: I think that is related to setuptools lazy loading itself onto distutils in orders that sometimes do not work | 18:52 |
| reed | http://status.openstack.org/zuul/ do we really have around 100 comments per hour added on gerrit? | 18:52 |
| *** yolanda has quit IRC | 18:52 | |
| hashar | fungi: ahh I should have a look at that. | 18:52 |
| clarkb | mrodden1: can you try with setuptools 0.7.8 I have had good luck with that | 18:52 |
| fungi | reed: on a slow day ;) | 18:52 |
| mordred | hashar: the issue is that you _either_ need the upstream git repo of you need the PKG-INFO file that's generated at sdist time | 18:52 |
| reed | fantastic | 18:52 |
| fungi | reed: note that those include test results coming back from ci | 18:52 |
| mordred | mrodden1, clarkb: is there a setuptools issue? | 18:52 |
| reed | yeah, I imagined it's not just people :) | 18:53 |
| fungi | reed: since they end up as review comments | 18:53 |
| clarkb | mordred: http://paste.openstack.org/show/46904/ | 18:53 |
| clarkb | mordred: it has symptoms similar to the thing that clayg was working on | 18:53 |
| reed | around 20 patches per hour is an interesting number too | 18:53 |
| mrodden1 | clarkb: trying with 0.7.8 | 18:54 |
| jog0 | anyone have a minute to review my 3rd attempt to unbreak rechekwatch https://review.openstack.org/#/c/46166/ | 18:54 |
| mrodden1 | same output... :( | 18:54 |
| jog0 | (that I broke ... sorry) | 18:54 |
| clarkb | mrodden1: hmm maybe not related then | 18:55 |
| clarkb | mordred: also unrelated but interesting bug 1206730. I believe your merge milestone proposed branches back into master is the fix | 18:55 |
| uvirtbot | Launchpad bug 1206730 in pbr "pbr generates versions that are not monotonic" [Undecided,New] https://launchpad.net/bugs/1206730 | 18:55 |
| mordred | mrodden1: is this just on a plain devstack? | 18:56 |
| mordred | clarkb: it's not | 18:56 |
| *** lnxnut_ has joined #openstack-infra | 18:56 | |
| mrodden1 | yea | 18:56 |
| mrodden1 | well its kind of an old install | 18:56 |
| mordred | the versions will still not be monotonically increasing | 18:56 |
| *** thomasbiege has quit IRC | 18:56 | |
| mordred | mrodden1: shouldn't matter - it shoudl work | 18:56 |
| mrodden1 | i updated nova today and tried to do setup.py develop and it not happy | 18:56 |
| clarkb | mordred: don't we count after last tag? | 18:56 |
| mordred | clarkb: no | 18:56 |
| mordred | not for server projects | 18:56 |
| clarkb | mordred: so you get $base_version.incrementing_number.sha_thing | 18:56 |
| *** bswrchrd_ has joined #openstack-infra | 18:56 | |
| clarkb | mordred: oh | 18:56 |
| mordred | we use pre-version numbering | 18:56 |
| jeblair | lifeless: i don't have a good answer for you; however, i would like it to generate output that works for us, and not generate output that does not work for us. the current output does work for us. | 18:57 |
| clarkb | pre-version doesn't prevent you from doing that I thought | 18:57 |
| openstackgerrit | A change was merged to openstack-infra/config: Update old bug format in recheckwatch https://review.openstack.org/46166 | 18:57 |
| jog0 | thanks mordred | 18:57 |
| clarkb | oh except those versions will be greater than the base | 18:57 |
| clarkb | hmm | 18:57 |
| mordred | clarkb: the versions have no relation to previous tags | 18:57 |
| lifeless | jeblair: ok. I can make putting the domain on the rcreate calls optional | 18:57 |
| jeblair | lifeless: that should work | 18:58 |
| mordred | so you'll have 2013.1.a1.g234523 then 2013.1.b1 (produced by tag) then 2013.1.a3.g234225 | 18:58 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Puppet the Gerrit bcprov and mysql-connector deps https://review.openstack.org/46339 | 18:58 |
| mrodden1 | brb need to step away for a bit... | 18:58 |
| mordred | mrodden1: ok. I'd love to figure out why it broke - I'm fairly confident I can get you fixed again | 18:58 |
| *** bswrchrd has quit IRC | 18:59 | |
| *** bswrchrd has joined #openstack-infra | 19:03 | |
| *** changbl has joined #openstack-infra | 19:05 | |
| *** bswrchrd_ has quit IRC | 19:05 | |
| *** ruhe has quit IRC | 19:08 | |
| *** fbo_away is now known as fbo | 19:12 | |
| clarkb | fungi: re https://review.openstack.org/#/c/46179/ and the change that "fixes" ssh hash mismatch errors. I can squash them together. The one that deals with bouncy castle needs work though | 19:14 |
| clarkb | fungi: I am beginning to think that one central gerrit server should be used and we create a project per test | 19:14 |
| fungi | yeah, we could potentially just reorder those patches too | 19:15 |
| clarkb | fungi: because gerrit start up time is insane with bouncy castle | 19:15 |
| clarkb | fungi: I have seen that 5 minute timeout not be long enough on my test box | 19:15 |
| fungi | i take it bc wasn't actually getting used i the current rev of that task then? | 19:15 |
| mordred | clarkb: might be a good opportunity to look at testresources for that then | 19:15 |
| clarkb | fungi: correct, because the lib isn't installed globally on the slaves | 19:15 |
| fungi | because the tests actually completed in about the same time | 19:15 |
| clarkb | mordred: yup | 19:15 |
| fungi | aha | 19:15 |
| fungi | i overlooked the bit where it wasn't in place in that patch. no wonder | 19:16 |
| clarkb | fungi: so basically prep the golden site, spin it up and tear it down with a testresource | 19:16 |
| clarkb | and do away with per test gerrit servers | 19:16 |
| fungi | that makes more sense, yes | 19:16 |
| fungi | so only one gerrit install happens per job run that way | 19:17 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix paths for glossary https://review.openstack.org/46326 | 19:17 |
| clarkb | fungi: right | 19:17 |
| clarkb | fungi: the slowness appears to be at least partially load related | 19:17 |
| clarkb | fungi: spinning up one gerrit is typically ok. spinning up 4 at the same time is not (I assume IO related?) | 19:17 |
| fungi | does not surprise me too much | 19:18 |
| *** salv-orlando has joined #openstack-infra | 19:18 | |
| *** vipul is now known as vipul-away | 19:23 | |
| *** vipul-away is now known as vipul | 19:23 | |
| mordred | jeblair: on https://review.openstack.org/#/c/46107/ | 19:28 |
| mordred | jeblair: dev-requirements is already tested in that file | 19:28 |
| mordred | jeblair: it has a block later in the script | 19:28 |
| *** gyee has quit IRC | 19:29 | |
| mrodden1 | mordred: back, sorry had to run out to fix something else quick | 19:30 |
| mrodden1 | i'm wondering if its a python path issue but i'm not positive | 19:31 |
| *** alexpilotti has quit IRC | 19:32 | |
| mordred | mrodden1: so, newer devstack has some workarounds for a setuptools upgrade issue | 19:35 |
| mrodden1 | oh? | 19:35 |
| mordred | mrodden1: try this "pip uninstall setuptools; pip install -U setuptools; pip install -U pip; pip install -U pbr" | 19:35 |
| mordred | yea - there was some carnage caused by upstream re-merging of setuptools and distribute | 19:36 |
| annegentle | hey infra, did anyone coach some of the training guys (sarob or colin) to do a subproject within openstack-manuals repo? | 19:36 |
| *** Ryan_Lane has quit IRC | 19:36 | |
| annegentle | I'm really clueless and trying to figure out what's up with some git history | 19:37 |
| jeblair | mordred: commit message bug then? | 19:37 |
| jeblair | annegentle: i haven't caught on to what you're asking yet... is there a git commit i should look at? | 19:37 |
| mordred | annegentle: we never suggest subprojects to people - but also, I agree with jeblair in that I don't fully follow you | 19:38 |
| jeblair | mordred: annegentle may not have meant 'git submodule' which i assume is what you were thinking | 19:38 |
| annegentle | jeblair: just now spelunking as a new src folder has appeared | 19:38 |
| fungi | not to my knowledge anyway | 19:38 |
| annegentle | jeblair: c748436dd23da10861a98da63440acba23b39e01 is one | 19:38 |
| *** vipul is now known as vipul-away | 19:38 | |
| annegentle | jeblair: 090a2dfb1ca701ef31cf30579628c083002894ed is another, those are commit numbers, I can also give you Change-IDs | 19:38 |
| annegentle | jeblair: yeah I don't mean submodule | 19:39 |
| jeblair | annegentle: i brainstormed some ideas with sarob | 19:39 |
| annegentle | jeblair: ok | 19:40 |
| annegentle | jeblair: https://review.openstack.org/#/c/46182/ has these extra directories | 19:40 |
| jeblair | at least, i assume i did; i brainstormed with sean roberts, whose name looks like sarob | 19:40 |
| jeblair | ooh, self-approved organizational changes! | 19:40 |
| mordred | time for some public shame perhpas | 19:41 |
| annegentle | jeblair: yes sarob is sean roberts | 19:41 |
| jeblair | annegentle: i _definitely_ did not suggest anything that involved "go to trello" | 19:41 |
| annegentle | jeblair: well they are "incubating" under docs... and no I haven't asked them to stop using trello :) | 19:42 |
| annegentle | jeblair: this is all highly ungoverned :) | 19:42 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Publish Over SSH Plugin option added https://review.openstack.org/46112 | 19:42 |
| mordred | https://review.openstack.org/#/c/46182/1/doc/src/docbkx/openstack-training/sources/operations-guide,unified <-- what is that? what's a Subproject? | 19:42 |
| annegentle | mordred: I dunno either | 19:42 |
| *** Ajaeger has joined #openstack-infra | 19:42 | |
| annegentle | mordred: their goal is: use the existing docs to make custom training manuals | 19:43 |
| annegentle | mordred: and I LIKE that goal | 19:43 |
| annegentle | mordred: it'll make our docs better too | 19:43 |
| jeblair | annegentle: i suggested some ideas around how they could automatically pull in stuff from other repos by using some technology similar to what mordred is working on for auto-proposing requirements changes | 19:43 |
| annegentle | mordred: but, achieving "give me these bits and pieces" takes some work | 19:43 |
| jeblair | annegentle: (where a change to a file in one repo would trigger a job that copies the change to another repo) | 19:43 |
| mordred | ah. interesting | 19:44 |
| annegentle | jeblair: oo! mordred! I need that for "watching" the nova api samples directory! | 19:44 |
| fungi | subprojects do in fact seem to bear some relation to submodules | 19:44 |
| annegentle | WANT | 19:44 |
| jeblair | (copying files around to different repos is, of course, insane, but i'm not sure how you release a documentation snippet as a library, so hey) | 19:44 |
| *** adalbas has quit IRC | 19:44 | |
| jeblair | (and insane has never stopped us from trying an idea) | 19:44 |
| annegentle | jeblair: hee | 19:45 |
| annegentle | less sanity, more experimentation | 19:45 |
| *** zul has quit IRC | 19:45 | |
| jeblair | fungi: ok, so a subproject commit is actually a git submodule | 19:45 |
| jeblair | that's great that they have two names | 19:45 |
| mordred | that's stellar | 19:45 |
| fungi | jeblair: beware the hobgoblin | 19:46 |
| mordred | so, that means we're using submodules now in the doc repo? | 19:46 |
| annegentle | mordred: the use case is: nova dev makes some updates to api samples, the openstack/api-site automatically gets those new samples and makes a new patch and builds new docs with fresher samples | 19:46 |
| jeblair | mordred: well, someone is; i doubt "we" are. | 19:46 |
| *** adalbas has joined #openstack-infra | 19:46 | |
| annegentle | mordred: do not want. submodules. | 19:46 |
| mordred | sorry, I mean "we" are using submodules in the doc repo now? | 19:46 |
| annegentle | mordred: do. not. want :) | 19:46 |
| jeblair | we don't want it either. no one knows how to use submodules. even people who think they do. | 19:46 |
| annegentle | jeblair: zactly | 19:46 |
| mordred | yup. also, zuul does not know how to use submodules | 19:47 |
| *** hashar has quit IRC | 19:47 | |
| *** dcramer_ has quit IRC | 19:48 | |
| fungi | i think one of the suggestions i had was that the api samples in the software projects should have associated publish jobs to upload them separately from the docs jobs, or that the docs jobs should depend on checkouts of the software projects containing the api samples. but maybe automated embedding of that stuff isn't straightforward | 19:49 |
| fungi | but at the time it was all still very brainstormy | 19:49 |
| mordred | they're only half there | 19:49 |
| annegentle | and I don't mean to conflate the two goals/use cases -- API samples are different from custom training manuals... | 19:49 |
| jeblair | fungi: yeah, any of those things sounds like a good direction to explore | 19:49 |
| mordred | the .gitmodules file is missing | 19:49 |
| annegentle | mordred: that's good right? | 19:50 |
| annegentle | mordred: as in, reversible? | 19:50 |
| annegentle | I wish sarob or colin were here to talk to... I'll email them I think. | 19:50 |
| fungi | annegentle: yeah, clearly the training manuals case is different from the api docs case, but it seemed to share some similar needs | 19:51 |
| Ajaeger | And they committed to the wrong directory, doc/src/docbkx is obsolete. | 19:51 |
| annegentle | fungi: for sure. | 19:51 |
| jeblair | mordred: given that we've established no one understand submodules, that means there's no one that could add support for them to zuul. :) | 19:51 |
| * fungi suspects zuul will become sentient at some stage, and might add them itself then | 19:52 | |
| jeblair | fungi: it would be the only thing that understood them then. | 19:52 |
| fungi | kernel.org will start getting commits directly from zuul to correct git's current submodule implementation | 19:53 |
| mordred | jeblair: well.... I'm going to need to add support for them to zuul at some point, but I'm deferring thinking about it for right now | 19:53 |
| Ajaeger | while you all talk manuals, could I bother you with some reviews for fallout of our big directory rename? These three are needed for getting everything published: https://review.openstack.org/#/c/46333/ https://review.openstack.org/#/c/46326/ https://review.openstack.org/#/c/45938/ | 19:54 |
| clarkb | I just found out that you couldn't cleanly remove submodules until git 1.8.3 | 19:54 |
| *** SergeyLukjanov has quit IRC | 19:54 | |
| jeblair | mordred: good luck! | 19:55 |
| mordred | jeblair: do you want me to rework that commit message? I'd like to land the requirements fix | 19:55 |
| *** DennyZhang has quit IRC | 19:55 | |
| mordred | jeblair: thanks! | 19:55 |
| annegentle | Thanks Ajaeger for the patch to remove, I'll see if I can get Sean to take a look | 19:55 |
| jeblair | mordred: i kinda would like you to, it is pretty wrong... | 19:56 |
| mordred | ok | 19:56 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove old files from the requirements list https://review.openstack.org/46107 | 19:56 |
| jeblair | mordred: i'll review it all quick-like | 19:56 |
| mordred | how about that? | 19:56 |
| jeblair | loverly | 19:57 |
| *** zul has joined #openstack-infra | 19:59 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove old files from the requirements list https://review.openstack.org/46107 | 19:59 |
| *** mriedem has quit IRC | 19:59 | |
| *** mriedem1 has joined #openstack-infra | 20:00 | |
| *** bswrchrd has quit IRC | 20:01 | |
| *** mriedem1 has quit IRC | 20:01 | |
| *** pcm__ has quit IRC | 20:01 | |
| *** mriedem has joined #openstack-infra | 20:03 | |
| *** jcoufal has quit IRC | 20:03 | |
| *** HenryG_ has quit IRC | 20:03 | |
| mordred | woot | 20:03 |
| fungi | mordred: looking at 40068 with an eye for extending it to stable branches, would it make more sense to relocate that list of projects to which requirements changes should be autoproposed out of the slave script and into a file in openstack/requirements instead? (since it will change over time between releases) | 20:03 |
| mordred | hrm. that's not a bad idea | 20:04 |
| fungi | otherwise we start implementing devstack-gate-like branch conditionals to build up the list, which gets hairy | 20:04 |
| Ajaeger | jeblair, another question regarding templates for manual build jobs (see https://review.openstack.org/#/c/45574/) | 20:04 |
| mordred | I do think it means that we may wind up with information in that repo which is not really consumed by anyone but infra - but I think I can deal with that | 20:04 |
| fungi | doable, but would be simpler to just key off something in the requirements repo which can differ by branch | 20:05 |
| mordred | yup | 20:05 |
| fungi | yeah | 20:05 |
| Ajaeger | will the following start two build jobs: | 20:05 |
| Ajaeger | - project: | 20:05 |
| Ajaeger | name: openstack-manuals | 20:05 |
| Ajaeger | manual: | 20:05 |
| Ajaeger | - user-guide | 20:05 |
| Ajaeger | - image-guide | 20:05 |
| Ajaeger | jobs: | 20:05 |
| Ajaeger | - '{manual}-documentation' | 20:05 |
| annegentle | reed: around? | 20:05 |
| reed | o/ | 20:05 |
| mrodden1 | mordred: i got it | 20:06 |
| mrodden1 | i had some setuptools egg info hanging around from the .deb packages | 20:06 |
| reed | annegentle, wazzup? | 20:06 |
| mrodden1 | deleted that and the pip installed version kicked in and now everything is happy | 20:06 |
| annegentle | reed: have a q for ya | 20:07 |
| *** afazekas_zz has quit IRC | 20:07 | |
| mordred | woot! | 20:08 |
| annegentle | Ajaeger: I think so- jeblair? | 20:09 |
| annegentle | Ajaeger: I was even at the tutorial Monday and followed it :) | 20:09 |
| annegentle | Ajaeger: but will still ask jeblair | 20:09 |
| annegentle | :) | 20:09 |
| clarkb | Ajaeger: annegentle: that looks correct to me | 20:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack/requirements: Add a list of projects that we want to sync to https://review.openstack.org/46347 | 20:09 |
| mordred | fungi: ^^ | 20:09 |
| clarkb | as JJB should iterate the manual values automagically | 20:09 |
| Ajaeger | annegentle: was the tutorial recorded or is material available from it? | 20:10 |
| fungi | mordred: awesome! | 20:10 |
| Ajaeger | clarkb, annegentle thansk for the confirmation! That brings me a step further to simplifiy the jjb manual jobs | 20:10 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update projects when requirements change https://review.openstack.org/40068 | 20:10 |
| mordred | fungi: updated to consume it | 20:10 |
| annegentle | Ajaeger: it'll be out in a week or two | 20:11 |
| Ajaeger | annegentle: great, thanks | 20:11 |
| *** weshay has quit IRC | 20:12 | |
| *** jcoufal has joined #openstack-infra | 20:12 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update projects when requirements change https://review.openstack.org/40068 | 20:13 |
| fungi | mordred: also, just from a separation of procedure and data perspective, i like that we don't have yet-another-list-embedded-in-a-script syndrome there | 20:13 |
| mordred | fungi: ++ | 20:13 |
| mordred | great idea | 20:13 |
| jeblair | Ajaeger: http://paste.openstack.org/show/46925/ | 20:13 |
| jeblair | Ajaeger: there's the commands from the lab; if you run those, you'll end up with a local copy of jenkins-job builder, and you can use it to test out the syntax | 20:14 |
| jeblair | Ajaeger: (if you use it as described, you'll end up with a directory with one xml file for each job, so you can see if it's creating the jobs you expect) | 20:14 |
| *** mgagne1 has joined #openstack-infra | 20:15 | |
| *** mgagne1 has joined #openstack-infra | 20:15 | |
| *** mgagne has quit IRC | 20:16 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/pypi-mirror: Remove spurious uses of output return values https://review.openstack.org/46348 | 20:16 |
| *** fbo is now known as fbo_away | 20:16 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Start using templates in manual build jobs https://review.openstack.org/45574 | 20:19 |
| Ajaeger | jeblair: Thanks, will test later. | 20:19 |
| *** weshay has joined #openstack-infra | 20:21 | |
| *** niska has quit IRC | 20:24 | |
| mordred | jeblair, clarkb, fungi: if you aren't subscribed to openstack-infra/pypi-mirror, now might be a good time :) | 20:24 |
| fungi | oof--good reminder | 20:25 |
| fungi | i should re-audit my subscriptions against cgit's openstack-infra list | 20:25 |
| jeblair | Ajaeger: /win 23 | 20:25 |
| jeblair | ga | 20:26 |
| *** niska has joined #openstack-infra | 20:26 | |
| fungi | full of win | 20:26 |
| *** dprince has quit IRC | 20:28 | |
| *** dcramer_ has joined #openstack-infra | 20:29 | |
| clarkb | I think claws wins the current MUA competition | 20:30 |
| clarkb | also I may have spent lunch derping over MUAs | 20:30 |
| *** lnxnut_ has quit IRC | 20:30 | |
| fungi | i have not tried claws. you liked it better than gnus huh? | 20:31 |
| clarkb | I am not super comfortable with emacs | 20:32 |
| fungi | that would definitely make gnus less attractive | 20:32 |
| clarkb | I can't get it to verify ttx's signature though | 20:33 |
| clarkb | but I think that may be because the uids don't match? | 20:33 |
| fungi | that shouldn't matter | 20:33 |
| clarkb | hmm, needs more hammering then | 20:33 |
| fungi | as long as the public part of the signing key's parent is in your keychain, it should at least tell you the signature is valid, even if not valid for the sender's address | 20:34 |
| fungi | was the message signed inline or multi-part pgp+mime? | 20:35 |
| clarkb | oh I see the error now. His key is not available to verify the signature | 20:35 |
| clarkb | probably keychain issues | 20:35 |
| clarkb | fungi: inline I believe (one of the recent CVE announcements) | 20:35 |
| fungi | yeah, his client seems to sign inline unless there are attachments | 20:36 |
| fungi | whereas mutt always uses pgp+mime by default for me | 20:36 |
| fungi | i also don't have mutt configured to validate inline signatures, but it's only a |gpgv away | 20:36 |
| Ajaeger | jeblair: thanks for the step-for-step instructions, now I can test my changes! | 20:39 |
| jeblair | yay! i should put those in the jjb docs | 20:39 |
| *** KennethWilke has joined #openstack-infra | 20:40 | |
| mordred | HAHAHAHAHAHAHAHAHAAHAHAHAH | 20:42 |
| *** pblaho has quit IRC | 20:42 | |
| mordred | I just got the best unsolicited recruiter email ever | 20:42 |
| mordred | Need Java Architect – Google Web Technologies (GWT) expert | 20:42 |
| mordred | yeah. THAT's me | 20:42 |
| *** vipul-away is now known as vipul | 20:42 | |
| *** Ryan_Lane has joined #openstack-infra | 20:43 | |
| *** Ryan_Lane has joined #openstack-infra | 20:43 | |
| ekarlso- | jeblair: what's a way to have zuul notify a vendor jenkins upon change ? | 20:45 |
| Shrews | mordred: OMG. PLZ TO SEND TO ME | 20:45 |
| clarkb | Shrews: mordred hahaha | 20:45 |
| jeblair | ekarlso-: there isn't; see this instead: http://ci.openstack.org/third_party.html | 20:46 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Start using templates in manual build jobs https://review.openstack.org/45574 | 20:48 |
| ekarlso- | jeblair: would be cool if there was a feed or so from zuul | 20:49 |
| jeblair | ekarlso-: there is a feed from gerrit | 20:50 |
| ekarlso- | ok :) | 20:50 |
| Ajaeger | jeblair: thanks to the testing, 45574 is valid now ;) I would appreciate some review comments on it before I continue. | 20:50 |
| jeblair | Ajaeger: yay! | 20:52 |
| *** mrmartin has quit IRC | 20:56 | |
| *** nhm has joined #openstack-infra | 20:59 | |
| *** mriedem1 has joined #openstack-infra | 21:00 | |
| Ajaeger | jeblair: thanks, will do another iteration tomorrow. Bye for now! | 21:00 |
| *** hashar has joined #openstack-infra | 21:00 | |
| *** Ajaeger has quit IRC | 21:01 | |
| *** mriedem has quit IRC | 21:02 | |
| *** ArxCruz has quit IRC | 21:02 | |
| *** zeus has quit IRC | 21:03 | |
| *** rfolco has quit IRC | 21:04 | |
| *** Bada has quit IRC | 21:07 | |
| *** gyee has joined #openstack-infra | 21:08 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/publications: Make make-index ignore template branch. https://review.openstack.org/46356 | 21:09 |
| *** dmakogon_ has joined #openstack-infra | 21:09 | |
| clarkb | fungi: jeblair mordred ^ that is a start to creating a template branch for publications and writing a howto create new publication | 21:10 |
| jeblair | neat | 21:10 |
| clarkb | I haven't created the template branch or anything else yet :) | 21:11 |
| clarkb | but working on it | 21:11 |
| fungi | righteous | 21:11 |
| *** mgagne1 has quit IRC | 21:11 | |
| *** dkliban has quit IRC | 21:12 | |
| jog0 | so I am still failing at getting unbreaking rechecks :( | 21:14 |
| hashar | fungi: mordred: I found out how to get the python module version injected in Debian packaging tool :-) Got to include the openstack packaging tool then do something like: OSLO_PACKAGE_VERSION=$(VERSION) :-] | 21:18 |
| fungi | hashar: yes, that's one of several backdoor solutions mordred added for packagers who want to override versioning | 21:18 |
| hashar | fungi: that went well for me, I am building from a the tarballs.openstack.org source tar :-] | 21:19 |
| fungi | excellent! | 21:19 |
| hashar | now I get to backport all your dependencies huhu (d2to1 .. pbr …) | 21:19 |
| fungi | hashar: i thought those were already packaged for debian | 21:20 |
| *** svarnau has quit IRC | 21:20 | |
| hashar | yup, got to get them backported to Ubuntu Precise which is what is being used by Wikimedia | 21:20 |
| *** dcramer_ has quit IRC | 21:20 | |
| fungi | they may still be in the new queue or experimental/sid/testing... | 21:20 |
| jog0 | clarkb: have a minute to put me back on the right path for not breaking things? | 21:20 |
| fungi | hashar: ahh, right. shouldn't be too tough then | 21:21 |
| hashar | fungi: they landed in saucy already iirc. I will be fine (hopefully) | 21:21 |
| lifeless | did review.openstack.org just die ? | 21:21 |
| hashar | !status gerrit | 21:21 |
| openstack | hashar: Error: "status" is not a valid command. | 21:21 |
| jeblair | lifeless: no | 21:21 |
| SpamapS | I think pypi.openstack.org is having issues .. from my local box here in LA .. Could not fetch URL http://pypi.openstack.org/openstack/anyjson/: timed out | 21:21 |
| fungi | seems up to me | 21:21 |
| lifeless | 19: core3.dfw1.rackspace.net 306.860ms asymm 18 | 21:21 |
| lifeless | 20: review.openstack.org 511.968ms !H | 21:21 |
| hashar | lifeless: gerrit web interface works for me | 21:21 |
| lifeless | Resume: pmtu 1500 | 21:21 |
| *** pycabrera has left #openstack-infra | 21:21 | |
| *** dkranz has quit IRC | 21:21 | |
| fungi | ooh, that's fun | 21:21 |
| *** lcestari has quit IRC | 21:22 | |
| lifeless | fungi: FSVO | 21:22 |
| fungi | fftwf | 21:22 |
| hashar | lifeless: ssh works for me as well | 21:22 |
| fungi | i'm getting to it via core3 as well and getting an actual reply | 21:22 |
| jeblair | ditto | 21:23 |
| fungi | also via ipv6 through whatever's at 2001:4800:800:c3:150a:4:0:1 as the penultimate hop | 21:23 |
| lifeless | fungi: sadly my ISP is not enlightened | 21:23 |
| lifeless | so I can't check ipv6 | 21:23 |
| *** weshay has quit IRC | 21:24 | |
| lifeless | I'm also getting | 21:24 |
| lifeless | 20: core3.dfw1.rackspace.net 307.901ms asymm 18 | 21:24 |
| jeblair | i am seeing some ipv6 packet loss on rax's network | 21:24 |
| lifeless | 21: static.openstack.org 716.756ms !H | 21:24 |
| fungi | lifeless: neither is mine. i tunnel to hurricane electric | 21:24 |
| lifeless | so this seems larger than that one host. | 21:24 |
| lifeless | Resume: pmtu 1500 | 21:24 |
| fungi | https://status.rackspace.com/ | 21:24 |
| fungi | they're at least not admitting to any issues, or haven't gotten around to doing so at any rate | 21:25 |
| lifeless | hmm | 21:26 |
| lifeless | fungi: can you see if you see traffic from me ? | 21:26 |
| *** senk has joined #openstack-infra | 21:26 | |
| fungi | lifeless: your source ip address is...? | 21:26 |
| lifeless | it just came good | 21:27 |
| fungi | oh well | 21:27 |
| fungi | i doubt it was an issue along an asymmetric return path unless rackspace is making internet backbone choices several hops into their core | 21:28 |
| *** senk has quit IRC | 21:28 | |
| fungi | could have been flow hash related though, and only exhibited for certain remote addresses | 21:28 |
| fungi | or some provider closer to you was blackholing some of rackspace's ip space which included some of our servers but not those core routers (they are on significantly different parts of the ipv4 global address space) | 21:30 |
| fungi | this is the point where, if i were still a network engineer at an isp, i would be digging through bgplay.routeviews.org looking for whacky peering transitions out on the 'net and posting annoyed messages to the nanog ml | 21:32 |
| fungi | so glad i'm not ;) | 21:32 |
| lifeless | that seems possible, since google.com just when gnaaargh | 21:32 |
| jog0 | fungi: ping | 21:36 |
| *** mriedem1 has quit IRC | 21:36 | |
| fungi | jog0: howdy | 21:36 |
| fungi | jog0: still troubleshooting rechecks? | 21:36 |
| jog0 | fungi: sigh ... yeah | 21:36 |
| fungi | jog0: any leads? | 21:37 |
| jog0 | I was wondering if you could confirm that it tried to restart today | 21:37 |
| fungi | taking a look now | 21:37 |
| jog0 | I tested the latest patch locally and it appeared to work | 21:37 |
| jog0 | with the pickle jeblair gave me | 21:37 |
| jog0 | fungi: thanks for your patience | 21:37 |
| *** KennethWilke has quit IRC | 21:38 | |
| fungi | Sep 12 19:00:44 zuul puppet-agent[12549]: (/Stage[main]/Recheckwatch/File[/usr/local/bin/recheckwatch]/content) content changed '{md5}f649e84163517a28e0821eee382c6cdf' to '{md5}ceccb0659de7cf6e3c153eab79d8df4c' | 21:39 |
| fungi | Sep 12 19:00:48 zuul puppet-agent[12549]: (/Stage[main]/Recheckwatch/Service[recheckwatch]) Triggered 'refresh' from 1 events | 21:39 |
| fungi | though it may not have been running, so the service refresh may have been a no-op (can't remember if puppet would complain in that situation) | 21:39 |
| jog0 | I assume the process isn't running now though? | 21:39 |
| fungi | nope | 21:39 |
| clarkb | fungi: jeblair: is 'template' a good name for the publications template branch? if so I am going to create that branch and push a change to it | 21:40 |
| fungi | just started it back up, jog0 | 21:40 |
| jog0 | thanks | 21:40 |
| jog0 | can you check in a minute or two if its still running | 21:40 |
| jeblair | clarkb: wfm | 21:40 |
| fungi | jog0: seems to have stayed running for the moment, but yeah i'll take another look in a bit | 21:40 |
| jog0 | fungi: thanks if it fails it will take a few minutes | 21:41 |
| fungi | clarkb: yes, i'm 50/50 on template vs skeleton, but the former probably translates better outside of english | 21:41 |
| fungi | archetype would also work | 21:41 |
| fungi | but is a little on the unnecessarily fancy side | 21:41 |
| clarkb | template it is then | 21:41 |
| fungi | sounds good | 21:41 |
| *** markmcclain1 has quit IRC | 21:42 | |
| *** hashar has quit IRC | 21:45 | |
| clarkb | fungi: jeblair mordred pleia2 zaro https://review.openstack.org/46366 | 21:46 |
| mrodden1 | so.. now when i re-stack.sh on my precise machine... stack.sh brings in old versions of ubuntu packages that are incompatible with nova requirements | 21:47 |
| clarkb | I based template on overview so that I could hack 'n slash the existing content. This also gives you good examples in git history | 21:47 |
| mrodden1 | and that overrides the pip installed versions in /usr/local/ | 21:47 |
| mrodden1 | :( | 21:47 |
| clarkb | mrodden1: /usr/local/ should win with default python path | 21:47 |
| mrodden1 | hmmm | 21:47 |
| jog0 | fungi: it appears to be working, can you make sure the process is still up as well | 21:49 |
| fungi | jog0: just did. seems to have survived the first 10 minutes at least and is still chugging | 21:49 |
| mrodden1 | clarkb: that doesn't appear to be the case on my system | 21:49 |
| *** danger_fo is now known as danger_fo_away | 21:49 | |
| mrodden1 | probably need to figure out how that got messed up i guess | 21:50 |
| jog0 | fungi: thanks! | 21:50 |
| clarkb | mrodden1: is this ubuntu or debian? | 21:50 |
| mrodden1 | ubuntu | 21:50 |
| mrodden1 | precise | 21:50 |
| zaro | clarkb: nice work. but i probably will never branch from template. | 21:50 |
| fungi | BrokenLinux[tm] | 21:50 |
| clarkb | zaro: why not? | 21:50 |
| *** hashar has joined #openstack-infra | 21:51 | |
| zaro | clarkb: prefer to work from more to less. than other way around. | 21:51 |
| clarkb | ok, just making sure that I wasn't missing something significant | 21:51 |
| fungi | in which case you can s/template/overview/ in those instructions and it'll still be mostly relevant | 21:51 |
| clarkb | fungi: zaro: also I branched template from overview so you can just look in history (this was intentional) | 21:52 |
| fungi | yep, sounded like a good move | 21:52 |
| zaro | yes, nice to have options. | 21:52 |
| fungi | i had already done most of the cleanup getting cruft files blown away in the first iterations of the overview branch creation, so should be pretty similar | 21:53 |
| *** thomasm has quit IRC | 21:53 | |
| mrodden1 | wth... i had /usr/lib/python2.7/dist-packages in my easy-install.pth in /usr/local/lib/python2.7 | 21:55 |
| fungi | that sounds marginally recursive | 21:55 |
| mrodden1 | yeah... | 21:55 |
| mrodden1 | now that that is fixed... lets try this again (stack.sh) | 21:55 |
| *** boris-42 has quit IRC | 21:57 | |
| *** changbl has quit IRC | 22:00 | |
| lifeless | fungi: ok its broke again | 22:04 |
| lifeless | whee now I don't even get dns | 22:04 |
| lifeless | fungi: ah, it was broken due to dns this time. verra odd | 22:05 |
| jog0 | fungi: recheck appears to still be working thanks again | 22:06 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Diff between installed packages and requirements https://review.openstack.org/45329 | 22:11 |
| jog0 | fungi:now to get my second patch for recheckick in without breaking things again | 22:12 |
| *** lifeless_ has joined #openstack-infra | 22:13 | |
| *** lifeless has quit IRC | 22:15 | |
| *** lifeless_ is now known as lifeless | 22:18 | |
| *** jcoufal has quit IRC | 22:24 | |
| fungi | lifeless: i'm surprised to hear that the internet is not a bastion of stability and reliability ;) | 22:25 |
| fungi | jog0: great! | 22:26 |
| clarkb | fungi: re https://review.openstack.org/#/c/41927/6 would bash string manipulation be safer than the backticks and sed? | 22:27 |
| clarkb | oh I see it isn't the sed that is a problem | 22:27 |
| *** flaper87 is now known as flaper87|afk | 22:28 | |
| clarkb | its that the resulting of sedding might be mytag ; rm -rf | 22:28 |
| fungi | clarkb: yeah, or whatever | 22:28 |
| fungi | but i really couldn't get it to not sanely escape all the things i fed into a tag | 22:28 |
| fungi | also, what can go into a tag is a little limited... man git-check-ref-format | 22:29 |
| *** atiwari has quit IRC | 22:29 | |
| pleia2 | clarkb: thanks for writing these instructions | 22:29 |
| fungi | in particular, spaces aren't allowed which mostly limits you to running a single name (but could still be a file in your repo) | 22:29 |
| *** markmcclain has joined #openstack-infra | 22:30 | |
| clarkb | pleia2: np, let me know if they need more depth or if I got something wrong | 22:30 |
| pleia2 | clarkb: once this is merged I'll give the instructions a try with mine and let you know ;) | 22:30 |
| pleia2 | seems good though | 22:31 |
| clarkb | zaro: was https://review.openstack.org/#/c/45329/5 rebased onto mordred fix? | 22:31 |
| clarkb | zaro: looks like it was | 22:32 |
| lifeless | fungi: droll, very droll :) | 22:33 |
| lifeless | so | 22:33 |
| lifeless | who do I talk to about making sure requirements files do /not/ list URLs ? | 22:34 |
| *** hashar has quit IRC | 22:34 | |
| *** dmakogon_ has quit IRC | 22:34 | |
| clarkb | lifeless: mordred and sdage | 22:34 |
| lifeless | mordred: sdague: ^ oh HAI | 22:34 |
| *** matty_dubs is now known as matty_dubs|gone | 22:35 | |
| zaro | clarkb: yes, but i kept the orig implmentation of install-requirements.sh because it can handle multiple requirements files. | 22:36 |
| clarkb | zaro: ok | 22:36 |
| *** jhesketh has joined #openstack-infra | 22:37 | |
| *** thedodd has quit IRC | 22:38 | |
| *** mrodden1 has quit IRC | 22:39 | |
| *** pentameter has quit IRC | 22:39 | |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Update doc impact to only make bugs on merge https://review.openstack.org/45907 | 22:42 |
| *** jhesketh__ has joined #openstack-infra | 22:42 | |
| *** zoresvit has joined #openstack-infra | 22:43 | |
| zoresvit | Hi. Can anyone tell if Repo Hooks API is available in GitHub Enterprise? The Enterprise version is not the latest... | 22:44 |
| *** KLevenstein has quit IRC | 22:45 | |
| fungi | zoresvit: i think you may be looking for github's support people. pretty sure none of them hang out in here, though i could be wrong | 22:46 |
| clarkb | zoresvit: I wouldn't expect anyone here to know that. I can point you to the script that clsoes our pull requests http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/close_pull_requests.py | 22:46 |
| zoresvit | Ok, got it! Thanks a lot for this as well. | 22:47 |
| clarkb | re https://review.openstack.org/#/c/45862/3 I think we need to limit the test matrix at some point... | 22:50 |
| *** pcrews has quit IRC | 22:53 | |
| *** zoresvit has quit IRC | 22:55 | |
| *** zoresvit has joined #openstack-infra | 22:56 | |
| *** mrodden has joined #openstack-infra | 23:00 | |
| fungi | i think we raised that concern two summits ago when people wanted to start testing with postgres? | 23:00 |
| *** locke105 has joined #openstack-infra | 23:03 | |
| *** UtahDave has quit IRC | 23:05 | |
| *** hashar has joined #openstack-infra | 23:09 | |
| anteaya | so we only have one more day of UtahDave's dedicated time. I want to make sure we don't get distracted with other stuff and lose him | 23:11 |
| *** adalbas has quit IRC | 23:11 | |
| anteaya | fungi: what is the current salt status? | 23:12 |
| anteaya | I've been trying to keep track but I might have missed something | 23:12 |
| anteaya | you are currently ensuring that we can get the latest salt on all our slaves, is that correct? | 23:12 |
| fungi | anteaya: i have a stack of salt-relevant reviews up, and i'm in the process of editing his to add it on top of them | 23:13 |
| anteaya | marvelous | 23:13 |
| fungi | anteaya: https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:salt,n,z | 23:14 |
| anteaya | just what I was looking for, thanks | 23:14 |
| *** rnirmal has quit IRC | 23:18 | |
| *** mrodden has quit IRC | 23:20 | |
| *** mrodden has joined #openstack-infra | 23:20 | |
| *** mrodden has quit IRC | 23:21 | |
| *** mgagne has joined #openstack-infra | 23:24 | |
| *** mgagne has joined #openstack-infra | 23:24 | |
| *** mgagne1 has joined #openstack-infra | 23:33 | |
| *** mgagne1 has joined #openstack-infra | 23:33 | |
| clarkb | anteaya: I will look at those changes as well | 23:35 |
| *** mgagne has quit IRC | 23:36 | |
| clarkb | fungi: we did raise the concern. I don't think we have hit an upper limit either, I am just reminded that this is a thing whenever changes like jd__'s come in | 23:36 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Give Jenkins the ability to trigger puppet runs https://review.openstack.org/45898 | 23:38 |
| fungi | anteaya: clarkb: added ^ (the rebase/adjustment of UtahDave's patch) just now | 23:38 |
| fungi | need to follow that one with a jenkins job definition and script to run it | 23:38 |
| clarkb | fungi: re https://review.openstack.org/#/c/46317/ wouldn't it be better to configure auto updates for salt in the salt module? | 23:38 |
| *** mrodden has joined #openstack-infra | 23:38 | |
| clarkb | eg instaed of in slave.pp? | 23:38 |
| clarkb | or is that an issue because we can configure unattended upgrades in one place? no conf.d/*? | 23:39 |
| fungi | clarkb: oh, perhaps... can modules set things in each other in a transverse fashion like that? | 23:39 |
| fungi | clarkb: and yeah, it needs to be included into the block in that one file | 23:39 |
| clarkb | fungi: it isn;t easy when it must go in the one file | 23:40 |
| fungi | can't be in a separate file i don't think | 23:40 |
| clarkb | you have to use puppet concat and things get weird | 23:40 |
| clarkb | fungi: isn't 50unattendedupgrades in a conf.d? | 23:40 |
| clarkb | can we go another level deep and have that file include the others? or is that particular file magical? | 23:41 |
| *** hashar has quit IRC | 23:41 | |
| fungi | oh, good point, we could probably break that file up into a 50* and a 60* and split them where we want to insert lines | 23:41 |
| fungi | and then have the salt module add a 55* which includes just the one line we want | 23:42 |
| * fungi looks | 23:42 | |
| clarkb | fungi: if that isn't too difficult I think it would be a good way to go | 23:42 |
| clarkb | fungi: the salt module can then include unattended_upgrades and require that class before creating the files | 23:43 |
| clarkb | and that should keep everything in the correct order | 23:43 |
| fungi | well, yes except we still need unattended upgrades on systems where we don't use salt, right? so we'd need to apply it differently to the other systems to avoid include conflicts in puppet | 23:44 |
| clarkb | fungi: there won't be conflicts if you use include | 23:44 |
| clarkb | you can include a thing as many times as you want and the first one wins. | 23:44 |
| fungi | ahh, okay. it's just class instantiation which goes sideways if you've already included the same? | 23:45 |
| clarkb | yes | 23:46 |
| clarkb | include is special | 23:46 |
| fungi | so as long as we only include and don't need to pass in variables we should be safe | 23:47 |
| clarkb | yup | 23:47 |
| fungi | i'll finally comprehend puppet about the time we switch to something else | 23:48 |
| *** dims has quit IRC | 23:52 | |
| *** Ryan_Lane has quit IRC | 23:56 | |
| *** hashar has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!