Monday, 2015-03-02

*** baoli has quit IRC00:00
*** baoli has joined #openstack-infra00:00
*** sarob has joined #openstack-infra00:05
*** purp has quit IRC00:05
*** purp has joined #openstack-infra00:06
*** ParsectiX has quit IRC00:06
*** ParsectiX has joined #openstack-infra00:07
*** xnox has quit IRC00:11
*** spzala has joined #openstack-infra00:17
sarobany cores online?00:25
sarobi need to get this ML going https://review.openstack.org/#/c/159995/00:25
*** stevemar has joined #openstack-infra00:26
pleia2sarob: there's one +200:27
pleia2maybe someone else will wander by too :)00:27
sarobpleia2: thx!00:28
sarobmordred or fungi: you around?00:30
*** dimsum__ has joined #openstack-infra00:34
*** dims_ has joined #openstack-infra00:35
*** dims_ is now known as dims00:36
*** dimsum__ has quit IRC00:39
*** achanda has joined #openstack-infra00:46
*** dannywilson has joined #openstack-infra00:46
*** dannywilson has quit IRC00:51
*** achanda has quit IRC00:51
zxiiroAnyone available to review this patch, I need it moving along if possible? https://review.openstack.org/#/c/155865/00:54
*** sputnik13 has quit IRC00:57
*** camunoz is now known as camunoz_mtg00:58
*** dimtruck is now known as zz_dimtruck00:58
*** vilobhmm1 has joined #openstack-infra01:01
*** xnox has joined #openstack-infra01:02
*** vilobhmm has quit IRC01:05
*** sputnik13 has joined #openstack-infra01:08
*** camunoz_mtg is now known as camunoz01:10
*** amitgandhinz has joined #openstack-infra01:14
*** sputnik13 has quit IRC01:18
*** mriedem has quit IRC01:18
*** vilobhmm1 has quit IRC01:22
*** sputnik13 has joined #openstack-infra01:23
*** sputnik13 has quit IRC01:25
*** sputnik13 has joined #openstack-infra01:32
*** sputnik13 has quit IRC01:32
*** wuhg has joined #openstack-infra01:34
*** jerryz_ has joined #openstack-infra01:37
*** cody-somerville has joined #openstack-infra01:37
*** nelsnelson has quit IRC01:38
*** jyuso1 has quit IRC01:40
*** ghostpl_ has joined #openstack-infra01:42
*** tiswanso has joined #openstack-infra01:46
*** ghostpl_ has quit IRC01:47
*** tiswanso has quit IRC01:48
*** tiswanso has joined #openstack-infra01:50
*** yamamoto_ has joined #openstack-infra01:59
*** amitgandhinz has quit IRC02:08
*** jyuso1 has joined #openstack-infra02:10
*** ashleighfarnham has joined #openstack-infra02:14
*** Ryan_Lane has joined #openstack-infra02:18
*** ashleighfarnham has quit IRC02:19
*** sarob has quit IRC02:20
*** purp has quit IRC02:21
*** purp has joined #openstack-infra02:21
*** rmcall has joined #openstack-infra02:22
*** koolhead17 has joined #openstack-infra02:27
*** trey_ has joined #openstack-infra02:27
*** trey has quit IRC02:27
*** trey_ is now known as trey02:27
*** mwagner_lap has joined #openstack-infra02:35
*** patri962 has joined #openstack-infra02:38
*** patri674 has joined #openstack-infra02:38
*** patri824 has joined #openstack-infra02:38
*** patri659 has joined #openstack-infra02:38
*** patrickeast___ has quit IRC02:38
*** patrickeast has quit IRC02:38
*** patrickeast__ has quit IRC02:38
*** patrickeast_ has quit IRC02:39
*** patri28 has quit IRC02:39
*** nelsnelson has joined #openstack-infra02:39
*** patrickeast has joined #openstack-infra02:41
*** rmcall has quit IRC02:45
*** asettle is now known as asettle-FEET02:59
*** Ryan_Lane has quit IRC02:59
*** jyuso has joined #openstack-infra03:05
*** jyuso1 has quit IRC03:07
jerryz_clarkb: ping03:08
*** amitgandhinz has joined #openstack-infra03:13
clarkbhi03:14
*** spzala has quit IRC03:23
*** amitgandhinz has quit IRC03:24
*** ghostpl_ has joined #openstack-infra03:31
*** dims has quit IRC03:35
*** ghostpl_ has quit IRC03:36
*** amotoki has joined #openstack-infra03:44
*** achanda has joined #openstack-infra03:49
*** achanda has quit IRC03:54
*** jpeeler has joined #openstack-infra03:55
*** camunoz is now known as camunoz_mtg03:59
*** hdd has joined #openstack-infra04:02
*** tsg has quit IRC04:02
*** david-lyle_afk has quit IRC04:02
*** tiswanso has quit IRC04:02
*** tsg has joined #openstack-infra04:12
*** hdd has quit IRC04:15
openstackgerritMerged openstack-infra/system-config: patch to create foundation-testing-standards mailing list  https://review.openstack.org/15999504:19
*** Sukhdev has joined #openstack-infra04:26
*** patrickeast_ has joined #openstack-infra04:27
*** patrickeast has quit IRC04:27
*** patrickeast_ is now known as patrickeast04:27
*** patri659 has quit IRC04:28
*** patri962 has quit IRC04:29
*** patri824 has quit IRC04:29
*** patri674 has quit IRC04:29
*** koolhead_ has joined #openstack-infra04:30
*** patrickeast_ has joined #openstack-infra04:30
*** patrickeast__ has joined #openstack-infra04:30
*** patri484 has joined #openstack-infra04:30
*** patri533 has joined #openstack-infra04:30
*** koolhead17 has quit IRC04:32
*** BharatK has joined #openstack-infra04:34
*** dims has joined #openstack-infra04:35
*** asettle-FEET is now known as asettle04:40
*** dims has quit IRC04:40
*** carl_baldwin has joined #openstack-infra04:41
*** coolsvap_ is now known as coolsvap04:43
*** fifieldt has joined #openstack-infra04:44
*** ParsectiX has quit IRC04:44
*** ParsectiX has joined #openstack-infra04:45
*** hdd has joined #openstack-infra04:45
*** bhunter71 has quit IRC04:46
*** koolhead17 has joined #openstack-infra04:48
*** achanda has joined #openstack-infra04:49
*** koolhead_ has quit IRC04:50
*** hdd has quit IRC04:52
openstackgerritMerged openstack-dev/pbr: "packages" might list multiple directories  https://review.openstack.org/15991804:52
*** camunoz_mtg is now known as camunoz04:58
*** Ryan_Lane has joined #openstack-infra05:00
*** baoli has quit IRC05:05
*** ddieterl_ has quit IRC05:06
*** sarob has joined #openstack-infra05:09
*** nelsnelson has quit IRC05:14
*** sarob has quit IRC05:14
*** carl_baldwin has quit IRC05:18
*** yfried|afk has joined #openstack-infra05:19
*** ghostpl_ has joined #openstack-infra05:20
*** ildikov has quit IRC05:21
*** patri338 has joined #openstack-infra05:23
*** patri781 has joined #openstack-infra05:23
*** patri996 has joined #openstack-infra05:23
*** patri952 has joined #openstack-infra05:23
*** patrickeast has quit IRC05:23
*** patrickeast_ has quit IRC05:24
*** ghostpl_ has quit IRC05:24
*** patri484 has quit IRC05:24
*** patri533 has quit IRC05:24
*** patrickeast__ has quit IRC05:25
*** patrickeast has joined #openstack-infra05:27
*** BharatK has quit IRC05:39
*** koolhead17 has quit IRC05:44
*** yfried|afk has quit IRC05:46
*** yfried|afk has joined #openstack-infra05:46
*** armax has joined #openstack-infra05:51
*** tsg has quit IRC05:52
*** yfried|afk has quit IRC05:55
*** otter768 has quit IRC05:56
*** trinaths has joined #openstack-infra05:57
*** sputnik13 has joined #openstack-infra05:58
*** hdd has joined #openstack-infra05:59
*** BharatK has joined #openstack-infra06:01
*** sputnik13 has quit IRC06:01
*** sputnik13 has joined #openstack-infra06:02
*** achanda has quit IRC06:03
*** sarob has joined #openstack-infra06:04
*** sputnik1_ has joined #openstack-infra06:05
*** sputnik13 has quit IRC06:07
*** hdd has quit IRC06:07
*** deepakcs has joined #openstack-infra06:12
*** jyuso1 has joined #openstack-infra06:15
*** jyuso has quit IRC06:15
*** ghostpl_ has joined #openstack-infra06:21
GheRiveromorning all06:24
*** ghostpl_ has quit IRC06:25
*** Sukhdev has quit IRC06:26
*** achanda has joined #openstack-infra06:27
*** tnovacik has joined #openstack-infra06:36
*** ddieterly has joined #openstack-infra06:36
*** Ryan_Lane has quit IRC06:41
*** ddieterly has quit IRC06:41
*** yfried|afk has joined #openstack-infra06:52
*** armax has quit IRC06:52
*** tnovacik has quit IRC06:53
*** hogepodge has quit IRC06:55
*** yfried|afk has quit IRC06:56
*** yfried has joined #openstack-infra06:56
*** yfried is now known as yfried|afk06:56
tchaypomorning ghe07:05
tchaypoalmost feels like being back in europe07:05
*** jerryz_ has quit IRC07:10
GheRiverotchaypo: are you at home finally? :)07:12
*** ParsectiX has quit IRC07:12
*** ParsectiX has joined #openstack-infra07:13
*** achanda has quit IRC07:13
*** ParsectiX has quit IRC07:13
*** jerryz_ has joined #openstack-infra07:13
*** ParsectiX has joined #openstack-infra07:14
*** achanda has joined #openstack-infra07:15
tchaypoyes :)07:16
tchaypoWell, not quite - don’t actually *have* a home yet. At partner’s parent’s place while we battle the rental market here07:16
*** scheuran has joined #openstack-infra07:17
*** mrmartin has joined #openstack-infra07:20
*** katyafervent is now known as katyafervent_awa07:21
*** katyafervent_awa is now known as katyafervent07:23
*** sarob has quit IRC07:23
*** achanda has quit IRC07:23
*** HeOS has joined #openstack-infra07:23
*** armax has joined #openstack-infra07:26
*** mrunge has joined #openstack-infra07:27
*** achanda has joined #openstack-infra07:29
*** jcoufal has joined #openstack-infra07:29
*** yfried|afk is now known as yfried07:34
*** ddieterly has joined #openstack-infra07:37
*** ashleighfarnham has joined #openstack-infra07:41
*** ddieterly has quit IRC07:42
*** koolhead17 has joined #openstack-infra07:47
*** ashleighfarnham has quit IRC07:47
*** hogepodge has joined #openstack-infra07:49
*** woodster_ has quit IRC07:50
openstackgerritJerry Zhao proposed openstack-infra/project-config: add python26 jobs back for compass-core  https://review.openstack.org/16025107:50
*** Sukhdev has joined #openstack-infra07:52
*** achanda has quit IRC07:52
*** jgallard has joined #openstack-infra07:55
*** ildikov has joined #openstack-infra07:56
*** otter768 has joined #openstack-infra07:57
*** achanda has joined #openstack-infra07:57
*** HeOS has quit IRC07:58
*** Ryan_Lane has joined #openstack-infra07:58
*** ihrachyshka has joined #openstack-infra07:58
*** HeOS has joined #openstack-infra08:00
*** otter768 has quit IRC08:02
*** chlong has quit IRC08:02
*** Ryan_Lane has quit IRC08:03
*** markus_z has joined #openstack-infra08:06
*** mpaolino has joined #openstack-infra08:06
*** mpaolino has quit IRC08:07
*** mpaolino has joined #openstack-infra08:08
*** mpaolino has quit IRC08:09
*** unicell has joined #openstack-infra08:09
*** mpaolino has joined #openstack-infra08:09
*** unicell1 has quit IRC08:09
*** ghostpl_ has joined #openstack-infra08:09
*** arxcruz has joined #openstack-infra08:10
*** mpavone has joined #openstack-infra08:11
*** mpaolino has quit IRC08:11
*** mpaolino has joined #openstack-infra08:12
*** ghostpl_ has quit IRC08:14
*** mpaolino has quit IRC08:14
*** mpaolino has joined #openstack-infra08:15
*** teran has quit IRC08:16
*** achanda has quit IRC08:16
*** afazekas has joined #openstack-infra08:18
*** Guest78669 is now known as d0ugal08:20
*** d0ugal has joined #openstack-infra08:21
*** Sukhdev has quit IRC08:23
*** sergsh has joined #openstack-infra08:24
*** dtantsur|afk is now known as dtantsur08:26
*** Ala has joined #openstack-infra08:26
*** sergsh has quit IRC08:26
*** sergsh has joined #openstack-infra08:27
*** matrohon has joined #openstack-infra08:27
*** _nadya_ has joined #openstack-infra08:28
*** dizquierdo has joined #openstack-infra08:28
*** stevemar has quit IRC08:36
*** macjack has joined #openstack-infra08:37
*** mpaolino has quit IRC08:38
*** mpaolino has joined #openstack-infra08:38
*** ddieterly has joined #openstack-infra08:38
*** _shaps__ has joined #openstack-infra08:40
*** mpaolino has quit IRC08:40
*** _shaps__ has quit IRC08:40
*** mpaolino has joined #openstack-infra08:41
openstackgerritGauvain Pocentek proposed openstack-infra/project-config: Add cloudkitty projetcs to gerritbot configuration  https://review.openstack.org/16025608:42
*** ddieterly has quit IRC08:43
*** mpaolino has quit IRC08:43
*** mpaolino has joined #openstack-infra08:44
*** salv-orlando has joined #openstack-infra08:44
openstackgerritMerged openstack-infra/project-config: Use Neutron in Horizon integration tests job  https://review.openstack.org/15939708:45
*** viktors|afk has quit IRC08:50
macjackGood day everyone, I need help to go next step. https://review.openstack.org/#/c/158923/ Thanks for help.08:55
*** yfried is now known as yfried|afk08:55
*** yfried|afk is now known as yfried08:57
*** MaxV has joined #openstack-infra09:00
*** tnovacik has joined #openstack-infra09:00
*** Hal has joined #openstack-infra09:01
*** Hal is now known as Guest3567809:01
*** sputnik1_ has quit IRC09:05
*** tkelsey has joined #openstack-infra09:05
*** Guest35678 has quit IRC09:07
*** ihrachyshka has quit IRC09:07
*** mpaolino has quit IRC09:08
*** mpaolino has joined #openstack-infra09:09
*** jistr has joined #openstack-infra09:12
*** mpaolino has quit IRC09:13
*** salv-orlando has quit IRC09:13
*** salv-orlando has joined #openstack-infra09:14
*** mpaolino has joined #openstack-infra09:14
*** ghostpl_ has joined #openstack-infra09:17
*** doude has quit IRC09:19
*** doude_ has joined #openstack-infra09:19
*** skolekonov has joined #openstack-infra09:29
*** koolhead17 has quit IRC09:29
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Removed PostgreSQL from tests  https://review.openstack.org/16027209:31
*** Ala has quit IRC09:33
*** jcoufal_ has joined #openstack-infra09:33
*** jcoufal has quit IRC09:36
*** Ala has joined #openstack-infra09:36
*** mpaolino has quit IRC09:36
*** mpaolino has joined #openstack-infra09:37
*** sergsh has quit IRC09:37
*** _nadya_ has quit IRC09:38
*** ddieterly has joined #openstack-infra09:39
*** mpaolino has quit IRC09:39
*** mpaolino has joined #openstack-infra09:40
*** mpaolino has quit IRC09:41
*** mpaolino has joined #openstack-infra09:42
*** mpaolino has quit IRC09:43
*** mpaolino has joined #openstack-infra09:43
*** ddieterly has quit IRC09:44
*** _nadya_ has joined #openstack-infra09:47
*** e0ne has joined #openstack-infra09:49
*** sergsh has joined #openstack-infra09:49
*** mpaolino has quit IRC09:51
*** mpaolino has joined #openstack-infra09:51
*** ZZelle has joined #openstack-infra09:53
openstackgerritMerged openstack-infra/storyboard: Exclude pyMySQL v0.6.4  https://review.openstack.org/15956409:53
openstackgerritMerged openstack-infra/storyboard: Reduce the number of timezones under test.  https://review.openstack.org/15957609:55
*** mpaolino has quit IRC09:55
*** mpaolino has joined #openstack-infra09:55
*** MaxV has quit IRC09:56
openstackgerritMerged openstack-infra/storyboard: Remove is_active from task model  https://review.openstack.org/15913509:56
*** MaxV has joined #openstack-infra09:56
*** otter768 has joined #openstack-infra09:58
openstackgerritMerged openstack-infra/storyboard: Replaced python-crontab with apscheduler  https://review.openstack.org/15895509:59
*** ssam2 has joined #openstack-infra09:59
openstackgerritMerged openstack-infra/storyboard: User abort call instead of returning response  https://review.openstack.org/15251510:01
*** otter768 has quit IRC10:02
*** jgallard has quit IRC10:04
*** mpaolino has quit IRC10:06
*** mpaolino has joined #openstack-infra10:07
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added MySQL-python to requirements  https://review.openstack.org/16028010:08
*** AJaeger has joined #openstack-infra10:09
*** jedimike has joined #openstack-infra10:09
AJaeger /msg chanserv access #cloudkitty list10:09
AJaegerargh ;(10:09
mrmartinAJaeger hi10:09
mrmartincan you give me a review on this: https://review.openstack.org/#/c/159426/10:09
AJaegerhi mrmartin10:09
mrmartinThe groups portal cron jobs are broken for a while, and this patch resolves that issue10:10
AJaegermrmartin, I'm not a core there and don't know enough to review it.10:10
mrmartinif you give me a +1 it is enough10:10
mrmartinit is easier to manage a +2 at the end10:10
AJaegerLet me check whether it looks sane...10:10
*** salv-orlando has quit IRC10:11
mrmartinit is a very simple patch10:11
mrmartinjust a puppet variable changed10:11
AJaegerlooks sane, +110:11
mrmartinthanks10:11
mrmartindon't we have an infra core in EU timezone?10:12
openstackgerritThierry Carrez proposed openstack-infra/storyboard: Added milestones  https://review.openstack.org/15250010:12
AJaegermrmartin, SergeyLukjanov is in the European timezone (not EU)10:12
mrmartinyeah, but he used to be in the States too :)10:13
*** mpaolino has quit IRC10:13
AJaeger;)10:13
*** mpaolino has joined #openstack-infra10:13
mrmartinwe need to establish an EU infra team10:13
mrmartinneed to find somebody who don't want too sleep in the next 1yr10:14
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Split auth in api and change user_tokens  https://review.openstack.org/15359110:15
*** mpaolino has quit IRC10:16
openstackgerritGauvain Pocentek proposed openstack-infra/project-config: Add cloudkitty projetcs to gerritbot configuration  https://review.openstack.org/16025610:16
*** mpaolino has joined #openstack-infra10:16
deepakcsAJaeger, mrmartin can someone help respond to http://lists.openstack.org/pipermail/openstack-dev/2015-February/057948.html ( i tried on -qa, but nobody responded)10:20
mrmartindeepakcs, sorry I don't have experience with that, if you cannot find anyone in actual timezone, need to wait until the US fellas wake up10:22
deepakcsmrmartin, np, yeah will try to find someone in US tz10:23
*** dizquierdo has quit IRC10:23
openstackgerritMerged openstack-infra/storyboard: Added milestones  https://review.openstack.org/15250010:27
*** marcusvrn has joined #openstack-infra10:28
*** dtantsur is now known as dtantsur|bbl10:29
*** marcusvrn1 has quit IRC10:32
GheRiveroI'm in European TZ but not core... yet10:34
*** mpaolino has quit IRC10:35
*** mpaolino has joined #openstack-infra10:35
*** mpaolino has quit IRC10:37
*** mpaolino has joined #openstack-infra10:37
*** jyuso1 has quit IRC10:37
*** jyuso1 has joined #openstack-infra10:38
*** ddieterly has joined #openstack-infra10:40
openstackgerritMerged openstack-infra/storyboard-webclient: Singularized Timeline Event Preferences  https://review.openstack.org/15594510:41
*** yfried is now known as yfried|afk10:41
gpocentekAJaeger: any clue on the IRC test failure for https://review.openstack.org/#/c/160256/ ? it doesn't fail locally10:43
AJaegerdeepakcs, sorry, can't help with that one.10:43
AJaegergpocentek, let me check...10:44
deepakcsAJaeger, np, got someone on -qa to help10:44
*** pelix has joined #openstack-infra10:44
*** ddieterly has quit IRC10:45
*** rlandy has joined #openstack-infra10:46
*** ihrachyshka has joined #openstack-infra10:46
*** _nadya_ has quit IRC10:46
openstackgerritMerged openstack-infra/storyboard-webclient: Tweak for Superuser UI in projects.  https://review.openstack.org/15994710:48
*** AJaeger has quit IRC10:50
openstackgerritRadomir Dopieralski proposed openstack/requirements: Add ironic-discoverd  https://review.openstack.org/15821010:53
*** EmilienM is now known as EmilienM|afk10:54
openstackgerritAleksey Ripinen proposed openstack-infra/infra-specs: Foreign keys support in StoryBoard  https://review.openstack.org/15940610:55
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Removed PostgreSQL from tests  https://review.openstack.org/16027210:56
openstackgerritRadomir Dopieralski proposed openstack/requirements: Upgrade oslo.config to version 1.7  https://review.openstack.org/16029710:56
ZZelleHi everyone10:57
ZZelleDoes somone understand why jenkins say that the change https://review.openstack.org/143412 depends on a change that failed to merge?10:58
*** Hal has joined #openstack-infra11:01
*** Hal is now known as Guest3599411:01
*** armax has quit IRC11:01
*** yfried|afk is now known as yfried11:02
*** AJaeger has joined #openstack-infra11:02
*** AJaeger has quit IRC11:02
*** AJaeger has joined #openstack-infra11:02
*** dizquierdo has joined #openstack-infra11:02
AJaegergpocentek, my extra recheck and the successfull test just crossed each other - so, 160256 looks fine again.11:04
gpocentekAJaeger: thanks!11:05
*** Ala has quit IRC11:05
*** mpaolino has quit IRC11:09
*** mpaolino has joined #openstack-infra11:09
*** armax has joined #openstack-infra11:09
*** dizquierdo has quit IRC11:10
*** mpaolino has quit IRC11:11
*** yfried is now known as yfried|afk11:12
*** mpaolino has joined #openstack-infra11:12
openstackgerritJulien Danjou proposed openstack-dev/hacking: Add a check for staticmethod  https://review.openstack.org/15195211:14
openstackgerritJulien Danjou proposed openstack-dev/hacking: tests: allow to run the same test for different code  https://review.openstack.org/15210811:14
*** ryanpetrello has quit IRC11:14
*** mpaolino has quit IRC11:17
*** achanda has joined #openstack-infra11:17
*** mpaolino has joined #openstack-infra11:17
*** ryanpetrello has joined #openstack-infra11:19
*** achanda has quit IRC11:21
*** dizquierdo has joined #openstack-infra11:22
*** jerryz_ has quit IRC11:22
*** yamamoto_ has quit IRC11:22
*** yfried|afk is now known as yfried11:24
*** doude_ has quit IRC11:26
*** SergK has joined #openstack-infra11:26
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added MySQL-python to requirements  https://review.openstack.org/16028011:26
*** claudiub has joined #openstack-infra11:28
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Split auth in api and change user_tokens  https://review.openstack.org/15359111:29
*** pc_m has joined #openstack-infra11:31
*** tnovacik has quit IRC11:32
*** ldnunes has joined #openstack-infra11:32
*** mpaolino has quit IRC11:34
*** ildikov has quit IRC11:34
*** mpaolino has joined #openstack-infra11:34
*** mpaolino has quit IRC11:36
*** mpaolino has joined #openstack-infra11:36
*** jerryz_ has joined #openstack-infra11:38
*** mpaolino has quit IRC11:39
bookwarhi, everyone, there is a commit to create new repos https://review.openstack.org/#/c/147169/ which has been merged, but we have empty core-reviewers group: https://review.openstack.org/#/admin/groups/579,members11:40
*** mpaolino has joined #openstack-infra11:40
bookwarcan someone add Nikolay Markov to core reviewers?11:40
*** ddieterly has joined #openstack-infra11:41
AJaegerbookwar, one of the infra cores needs to do this - let's ask clarkb or pleia2 once they're awake (US time) for this. Could you give me the email address of NIkolay, please?11:43
bookwarAJaeger: he is the author of the original commit, Nikolay Markov <nmarkov AT mirantis.com>11:44
AJaegerbookwar, thanks. I hope pleia2 and clarkb scroll back once they're back ^ - if not, please ping again later today11:45
*** ddieterly has quit IRC11:45
bookwarAJaeger: thanks, will do11:45
*** ildikov has joined #openstack-infra11:47
*** Shrews has quit IRC11:49
*** acfleury has joined #openstack-infra11:49
*** Shrews has joined #openstack-infra11:50
*** mpaolino has quit IRC11:51
*** mpaolino has joined #openstack-infra11:51
*** denis_makogon has joined #openstack-infra11:52
*** mpaolino has quit IRC11:53
*** yamamoto has joined #openstack-infra11:53
*** mpaolino has joined #openstack-infra11:53
*** mpaolino has quit IRC11:55
*** mpaolino has joined #openstack-infra11:55
*** tnovacik has joined #openstack-infra11:57
*** mpaolino has quit IRC11:58
*** mpaolino has joined #openstack-infra11:58
*** otter768 has joined #openstack-infra11:59
*** mpaolino has quit IRC12:01
*** mpaolino has joined #openstack-infra12:01
*** e0ne is now known as e0ne_12:03
*** otter768 has quit IRC12:04
*** yamamoto has quit IRC12:04
*** spredzy is now known as spredzy|afk12:04
*** mase_x200 has joined #openstack-infra12:05
*** coolsvap is now known as coolsvap_12:06
*** koolhead17 has joined #openstack-infra12:06
*** dims has joined #openstack-infra12:09
*** mpaolino has quit IRC12:09
*** mpaolino has joined #openstack-infra12:09
*** mpaolino has quit IRC12:11
*** mpaolino has joined #openstack-infra12:11
*** e0ne_ has quit IRC12:13
*** coolsvap_ is now known as coolsvap12:15
*** BobBall_AWOL is now known as BobBall12:16
*** derekh has joined #openstack-infra12:18
*** trinaths has quit IRC12:22
*** BharatK has quit IRC12:24
*** e0ne has joined #openstack-infra12:27
mordredbookwar AJaeger: done12:28
*** yamamoto has joined #openstack-infra12:29
*** aysyd has joined #openstack-infra12:29
bookwarmordred: thanks12:30
*** Guest35994 has quit IRC12:40
*** Hal has joined #openstack-infra12:40
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added fulltext index for access tokens  https://review.openstack.org/16032612:40
*** Hal is now known as Guest5552812:40
*** ddieterly has joined #openstack-infra12:41
*** Guest55528 has quit IRC12:42
*** dkranz has joined #openstack-infra12:43
SergKmordred, could you please help me with the same issue ^, adding me (skulanov@mirantis.com) to core group https://review.openstack.org/#/admin/groups/628,members   (my patchset https://review.openstack.org/#/c/156245/)12:44
*** ddieterly has quit IRC12:46
*** Ala has joined #openstack-infra12:49
*** yamamoto has quit IRC12:50
*** _nadya_ has joined #openstack-infra12:53
*** tnovacik has quit IRC12:53
*** baoli has joined #openstack-infra12:56
*** chmod__ has joined #openstack-infra12:57
*** deepakcs has quit IRC12:57
chmod__Hi guys, someone have worked on the implementation of the openstack APIC plugin ?12:57
*** woodster_ has joined #openstack-infra12:59
*** hdd has joined #openstack-infra12:59
*** baoli has quit IRC13:02
*** baoli has joined #openstack-infra13:03
*** hdd has quit IRC13:03
*** dkranz has quit IRC13:04
*** claudiub has quit IRC13:05
*** jgallard has joined #openstack-infra13:06
openstackgerritMonty Taylor proposed openstack-infra/system-config: Add two more useful ansible playbooks  https://review.openstack.org/10505713:07
*** markvoelker has joined #openstack-infra13:08
*** claudiub has joined #openstack-infra13:08
*** ashleighfarnham has joined #openstack-infra13:09
*** weshay has joined #openstack-infra13:09
*** esker has quit IRC13:10
openstackgerritAleksey Ripinen proposed openstack-infra/python-storyboardclient: Added milestones controller  https://review.openstack.org/16033413:11
*** ashleighfarnham has quit IRC13:14
*** dkranz has joined #openstack-infra13:14
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Enforce nodepool user when running cmd commands  https://review.openstack.org/16033613:14
*** chmod__ has quit IRC13:15
*** mpaolino has quit IRC13:16
*** hdd has joined #openstack-infra13:16
*** mpaolino has joined #openstack-infra13:17
*** rfolco has joined #openstack-infra13:17
*** yfried has quit IRC13:18
*** e0ne is now known as e0ne_13:20
*** gondoi has quit IRC13:20
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Record nodeinfo as ansible facts and inventory  https://review.openstack.org/10759713:20
openstackgerritKevin Benton proposed openstack-infra/project-config: Add Big Switch Networking library to StackForge  https://review.openstack.org/16033913:21
*** nithyag__ has joined #openstack-infra13:22
mordredSergK: done13:22
*** nithyag_ has quit IRC13:24
*** yamamoto has joined #openstack-infra13:24
*** e0ne_ is now known as e0ne13:24
SergKmordred, thank a lot13:25
*** bswartz has quit IRC13:25
*** _nadya_ has quit IRC13:26
*** tmcpeak has joined #openstack-infra13:27
*** gordc has joined #openstack-infra13:30
*** hdd has quit IRC13:31
*** hdd has joined #openstack-infra13:31
*** gondoi has joined #openstack-infra13:32
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Record nodeinfo as ansible facts and inventory  https://review.openstack.org/10759713:34
*** hdd has quit IRC13:36
*** dhritishikhar has joined #openstack-infra13:36
openstackgerritMonty Taylor proposed openstack-infra/puppet-mailman: Add non-alias-modifying mailman provider  https://review.openstack.org/16034213:38
*** Ala has quit IRC13:38
openstackgerritMonty Taylor proposed openstack-infra/puppet-mailman: Add non-alias-modifying mailman provider  https://review.openstack.org/16034313:39
*** Ala has joined #openstack-infra13:40
openstackgerritKevin Benton proposed openstack-infra/project-config: Add Big Switch Networking library to StackForge  https://review.openstack.org/16033913:40
openstackgerritMonty Taylor proposed openstack-infra/system-config: Use non-alias-modifying mailman provider  https://review.openstack.org/8936713:41
*** Hal has joined #openstack-infra13:41
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Enforce nodepool user when running cmd commands  https://review.openstack.org/16033613:41
*** Hal is now known as Guest2664213:42
*** ddieterly has joined #openstack-infra13:42
*** salv-orlando has joined #openstack-infra13:43
*** _nadya_ has joined #openstack-infra13:45
openstackgerritKevin Benton proposed openstack-infra/project-config: Add Big Switch Networking library to StackForge  https://review.openstack.org/16033913:45
*** bknudson has quit IRC13:46
*** mpaolino has quit IRC13:46
*** marcusvrn1 has joined #openstack-infra13:46
*** ddieterly has quit IRC13:47
*** mpaolino has joined #openstack-infra13:47
*** yolanda has quit IRC13:48
*** marcusvrn has quit IRC13:48
*** jcoufal_ has quit IRC13:48
*** mpaolino has quit IRC13:48
*** yolanda has joined #openstack-infra13:48
*** mpaolino has joined #openstack-infra13:49
*** yolanda has quit IRC13:49
*** julim has joined #openstack-infra13:49
*** jcoufal has joined #openstack-infra13:49
*** marcusvrn has joined #openstack-infra13:50
*** claudiub has quit IRC13:50
*** yolanda has joined #openstack-infra13:50
*** thelexx is now known as lxsli13:53
*** marcusvrn1 has quit IRC13:53
*** mase_x200 has quit IRC13:53
*** amotoki has quit IRC13:55
*** gondoi has quit IRC13:55
*** gondoi has joined #openstack-infra13:59
*** dtantsur|bbl is now known as dtantsur13:59
*** yamamoto has quit IRC13:59
*** otter768 has joined #openstack-infra14:00
*** bswartz has joined #openstack-infra14:00
*** bcourt has joined #openstack-infra14:01
*** doug-fish has joined #openstack-infra14:01
*** ParsectiX has quit IRC14:02
*** ParsectiX has joined #openstack-infra14:02
openstackgerritMonty Taylor proposed openstack-infra/git-review: Use git push-url instead of a second remote  https://review.openstack.org/10986914:02
*** bhunter71 has joined #openstack-infra14:04
*** otter768 has quit IRC14:04
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Record nodeinfo as ansible facts and inventory  https://review.openstack.org/10759714:06
*** cdent has joined #openstack-infra14:06
*** afazekas has quit IRC14:06
*** jgallard has quit IRC14:08
*** dustins has joined #openstack-infra14:09
*** bknudson has joined #openstack-infra14:10
*** yfried has joined #openstack-infra14:10
*** mriedem has joined #openstack-infra14:10
*** mriedem has quit IRC14:11
*** mriedem has joined #openstack-infra14:11
mordredwee. that was fun14:12
* mordred went through his outstanding patches and abandoned things that were total crap and responded to reviews on things that still made some sense14:13
*** yamamoto has joined #openstack-infra14:17
*** mpaolino has quit IRC14:19
*** mpaolino has joined #openstack-infra14:19
*** mjturek1 has joined #openstack-infra14:20
AJaegermordred, thanks for the spring cleanup ;)14:20
*** nelsnelson has joined #openstack-infra14:20
mordredAJaeger: gotta start somewhere :)14:22
*** ParsectiX has quit IRC14:22
*** afazekas has joined #openstack-infra14:23
*** dims has quit IRC14:25
*** dims has joined #openstack-infra14:26
*** dhritishikhar has quit IRC14:26
*** radez_g0n3 is now known as radez14:27
*** mpaolino has quit IRC14:28
*** mpaolino has joined #openstack-infra14:28
*** tmcpeak1 has joined #openstack-infra14:28
*** tmcpeak has quit IRC14:29
*** tiswanso has joined #openstack-infra14:29
*** mpaolino has quit IRC14:30
*** mpaolino has joined #openstack-infra14:31
mriedemis everything a story in storyboard? including bugs?14:31
mordredmriedem: yes14:32
*** mattfarina has joined #openstack-infra14:32
*** mpaolino has quit IRC14:35
*** mpaolino has joined #openstack-infra14:35
*** tmcpeak1 has quit IRC14:36
*** armax has quit IRC14:37
*** mpaolino has quit IRC14:38
*** mpaolino has joined #openstack-infra14:38
*** ddieterly has joined #openstack-infra14:38
*** mpaolino has quit IRC14:40
*** mpaolino has joined #openstack-infra14:41
*** afazekas has quit IRC14:41
*** mpaolino has quit IRC14:43
*** mpaolino has joined #openstack-infra14:43
*** denis_makogon has quit IRC14:43
openstackgerritKun Huang proposed openstack-infra/lodgeit: update 404 link of lodgeit.py  https://review.openstack.org/16036314:44
*** mpaolino has quit IRC14:46
*** mpaolino has joined #openstack-infra14:47
*** mpaolino has quit IRC14:48
openstackgerritAndrea Frittoli  proposed openstack-infra/devstack-gate: Get default branch from feature.yaml  https://review.openstack.org/16036614:48
*** eharney has joined #openstack-infra14:49
*** mpaolino has joined #openstack-infra14:49
*** amitgandhinz has joined #openstack-infra14:49
openstackgerritMonty Taylor proposed openstack-infra/project-config: Add config-drive element  https://review.openstack.org/15413214:51
*** mpaolino has quit IRC14:52
*** mpaolino has joined #openstack-infra14:52
mordredNg: ^^ there is the static version with keypair support added14:52
Ngmordred: ta14:52
*** mpaolino has quit IRC14:54
*** tonytan4ever has joined #openstack-infra14:54
*** mpaolino has joined #openstack-infra14:55
*** mpaolino has quit IRC14:55
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade  https://review.openstack.org/15846314:57
*** yamamoto has quit IRC14:59
*** pblaho_ has joined #openstack-infra15:00
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade  https://review.openstack.org/15846315:00
*** nelsnelson has quit IRC15:00
*** nelsnelson has joined #openstack-infra15:01
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/project-config: Add functional job for Shade and enable it on check and gate pipelines  https://review.openstack.org/16019015:02
*** tsg_ has joined #openstack-infra15:03
*** pblaho has quit IRC15:03
*** tgohad has joined #openstack-infra15:05
*** rm_work has quit IRC15:05
*** spredzy|afk is now known as spredzy15:06
*** yamamoto has joined #openstack-infra15:07
*** rm_work|away has joined #openstack-infra15:07
*** rm_work|away is now known as rm_work15:07
*** rm_work has quit IRC15:07
*** rm_work has joined #openstack-infra15:07
*** tsg_ has quit IRC15:08
*** mbacchi has joined #openstack-infra15:10
*** yfried has quit IRC15:11
*** sigmavirus24_awa is now known as sigmavirus2415:11
*** rm_work has quit IRC15:12
*** dangers_away is now known as dangers15:14
*** mtanino has joined #openstack-infra15:14
*** rm_work|away has joined #openstack-infra15:15
*** rm_work|away is now known as rm_work15:15
*** rm_work has joined #openstack-infra15:15
*** claudiub has joined #openstack-infra15:16
*** mfink_ has joined #openstack-infra15:17
*** afazekas has joined #openstack-infra15:19
*** dmsimard_away is now known as dmsimard15:20
*** asselin_ has joined #openstack-infra15:22
mordredclarkb: fyi - I updated my original "just read crap out of config-drive" patch to support keypairs. it was literally 5 lines of code. it's entirely possible that it's actually more work to deal with installing, configuring and debugging cloud-init than it is to just have the simple script - so I wanted to have it for comparison15:25
mordredclarkb: because, of course, "just use cloud-init" isn't actualy "just" in any way - that would be way too sane15:26
*** pblaho__ has joined #openstack-infra15:26
openstackgerritMatt Riedemann proposed openstack-infra/release-tools: Add .venv to .gitignore  https://review.openstack.org/16037715:27
openstackgerritMatt Riedemann proposed openstack-infra/release-tools: Make git clone work for python-*client projects  https://review.openstack.org/16037815:27
*** e0ne is now known as e0ne_15:28
*** e0ne_ is now known as e0ne15:28
*** ildikov has quit IRC15:29
*** pblaho_ has quit IRC15:30
*** annegentle has joined #openstack-infra15:31
*** yamamoto has quit IRC15:32
*** stevemar has joined #openstack-infra15:32
hogepodgefungi do you have a link on how to integrate with openstackid?15:32
hogepodgeor clarkb, or anyone else15:32
*** beekneemech is now known as bnemec15:32
hogepodgeanteaya: ? ^^15:32
*** achanda has joined #openstack-infra15:33
anteayahi sorry I know zip about openstackid15:33
anteayaand fungi is on holiday15:33
hogepodge:-/15:33
anteayamordred: ^^15:33
*** radez is now known as radez_g0n315:33
hogepodgethanks anteaya15:34
jeblairhogepodge: mrmartin can probably help you15:34
*** yamahata has joined #openstack-infra15:35
*** zz_jgrimm is now known as jgrimm15:35
jeblairhogepodge: but you should only need to specify the url as an openid provider15:36
jeblairhogepodge: here's how groups.o.o does it: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/groups.pp#n7315:37
*** wenlock has joined #openstack-infra15:37
*** jlvillal|afk has quit IRC15:37
*** zz_dimtruck is now known as dimtruck15:38
timrcjeblair, clarkb, fungi, mordred and whomever else... just a random question about nodepool and is probably a product of my naivety... why do we go through the trouble of complete tear-down / build-up of nodes in nodepool? In other words, why don't we use "rebuild" to avoid costly delete operations to reset VMs?  All else be equal (in terms of required resources and what not) then simply re-imaging an existing VM seems to be a much quicker solutio15:38
timrcn than deleting everything and then recreating it with a different image.15:38
mordredtimrc: we have a todo list item to use rebuild15:38
* timrc beams. Yay.15:38
mordredtimrc: but it'll take some work in the scheduler, since the delete and create threads are separeate right now15:38
mkj_wkZZelle: any comments on the latest version of my patch?15:38
mordredso it's not a quick-fix - but we do want it15:39
ZZellemkj_wk, did'nt get the time to look at it15:39
*** jlvillal|afk has joined #openstack-infra15:39
mkj_wkZZelle: ok, just wanted to make sure it wasn't lost in the shuffle of piles of email for unrelated reasons :)15:39
*** jlvillal|afk is now known as jlvillal15:39
timrcmordred, That's enough for me.  There is some interest in the Gozer team in making nodepool more performant, so perhaps something we might be interested in looking at.15:40
mordred++15:42
*** skolekonov has quit IRC15:43
*** achanda has quit IRC15:44
ZZellemkj_wk, done15:45
jeblairtimrc: it's going to be a big enough change it should have a spec beforehand; it might want to wait until the big nodepool work i have proposed on the list.  also, we should test that using rebuild is actually faster and benchmark it.15:45
pashkinHi all! Can somebody explain, why Jenkins jobs configs stored in separate repository and not in propjects repositories themselves?15:45
timrcjeblair, I think I have quite a bit of work to do learning about nodepool before I tackle anything, so yeah...15:45
mordredjeblair: ++ to testing15:46
mordredjeblair, timrc: I _have_ verified that you can do a rebuild with a different base image15:46
jeblairmordred: oh!  that could make things much simpler!15:46
mordredyah15:46
jeblairtimrc: ^15:46
timrcmordred, As have I.  I'm super lazy.15:47
mordredjeblair: yeah - I think with that, it's really about exposing needs to the reaper15:47
timrcI hate creating VM's so if my resources don't need to change I always go with a rebuild.15:47
timrcjeblair, It should be faster inherently since there's no need to reprovision anything or create / assign floatings ips.15:48
*** armax has joined #openstack-infra15:48
mordredtimrc: yah - but 'should' and 'are' are often at odds in openstack :)15:48
*** garyh has joined #openstack-infra15:48
timrcmordred, Indeed.  I'm completely supportive of testing and benchmarking to prove that the statement is true.15:49
timrcmordred, I think that's a general conundrum in life in general :)15:49
mordredtimrc: yah. oy15:49
jeblairtimrc: maybe a short program that used shade would be in order.  it could do the steps nodepool currently does, then it could do things again with rebuild.  we can run that on both clouds.15:50
mrmartinhogepodge hi, what you like to integrate with openstackid?15:50
hogepodgemrmartin: We'd like to use it as the identity provider for refstack15:51
mordredjeblair, timrc: ++15:51
mrmartinallright, you have to options, a, integrate an openid auth or, an oauth215:51
clarkbpashkin the way jenkins is configured it helps to have code review of those jobs and to locate them in a known place so jjb can find them. We need some control of where and what projects can do considering the resources that are handed out. in particular we have nodes for publishing to maven nexus and pypi15:52
mordredtimrc: step one- add rebuild support to shade15:52
mordredtimrc: should be easy - should be a single function15:52
hogepodgemrmartin: is there one that infra prefers or recommends?15:53
mrmartinhogepodge, first I suggest to do the integration localy, and if you're ready, we can do some tests on live15:53
timrcmordred, Looking at shade now.15:53
mrmartinhogepodge: it depends on you, I'm using oauth2 for groups portal because I can query the profile pictures15:53
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add microsecond columns to the test_runs table  https://review.openstack.org/16005715:53
mrmartinbut this is the only reason15:53
mordredtimrc: yolanda has been hacking in it already - so she's a good person to poke and whatnot15:54
*** dustins_ has joined #openstack-infra15:54
hogepodgemrmartin: ok, thanks. pictures are nice. :-D15:55
*** salv-orlando has quit IRC15:55
*** tiswanso has quit IRC15:55
*** salv-orlando has joined #openstack-infra15:55
mrmartinhogepodge, I'm have vagrant scripts somewhere I'm using for testing, because it is easier to read logs from there, if something goes wrong during development15:56
mrmartinI can share it if you like15:56
pashkin<clarkb> You talking about control, but actually, if somebody put some resource-heavy or malicious code in a project - you will not be able to review it15:56
hogepodgemrmartin: that would be great. thanks.15:56
*** ildikov has joined #openstack-infra15:56
timrcmordred, That's super convenient that we're on the same team at HP and I have access to her phone number.  Thanks!15:56
*** harlowja_at_home has joined #openstack-infra15:56
*** kmartin has joined #openstack-infra15:57
zxiiropelix: any chance you'll be able to get to https://review.openstack.org/#/c/155865/ today?15:57
clarkbpashkin thats mostly fine for the single use throwaway nodes.15:57
*** dustins has quit IRC15:57
clarkbthe larger concern are the publishing nodes which do not run arbitrary code15:57
mrmartinhogepodge this one: https://github.com/fremontlabs/vagrant-openstackid15:58
pashkinclarkb Isnt it more sane to keep jobs in a project repos?16:00
*** salv-orlando has quit IRC16:00
clarkbpashkin given the current set of tools no16:00
clarkbfor the reason above16:00
*** otter768 has joined #openstack-infra16:00
clarkbpashkin is there something in particular you are trying to achieve?16:01
openstackgerritJames E. Blair proposed stackforge/gertty: Fix syncing messages attached to draft revisions  https://review.openstack.org/16038816:01
pashkinclarkb Understanding mostly16:01
jeblairstoryboard meeting in #openstack-meeting-316:01
clarkbpashkin we need control of what jobs have access to privileged info. that requires central control of the jobs given our current tooling16:02
pashkinclarkb So you saying, that aside with throwaway nodes, there are also long-living nodes - and if they will be broken - it will be bad? But they can be broken anyway by some new evil patchset16:02
clarkbpashkin jeblair started a thread on evolving zuul that describes how we can put some control in the jobs but it requires new tooling16:03
clarkbpashkin not broken but exploited. if I have control of jobs in my project I could run ajob on that node and publish to pypi16:04
*** mbacchi has quit IRC16:04
*** coolsvap is now known as coolsvap_16:05
*** mbacchi has joined #openstack-infra16:05
*** otter768 has quit IRC16:05
pashkinclarkb You mean - untrusted user can publish to PyPi which is undesired?16:05
clarkbcorrect16:06
pashkinclarkb  "jeblair started a thread #openstack-meeting-3 ?16:06
*** sputnik13 has joined #openstack-infra16:06
openstackgerritMatt Riedemann proposed openstack-infra/release-tools: Add bashate testing  https://review.openstack.org/16038916:06
*** kgiusti has joined #openstack-infra16:06
clarkbno on the openstack-dev mailing list16:06
*** _nadya_ has quit IRC16:07
*** sputnik13 has quit IRC16:07
pleia2doing some final rsyncs on the new backup server, but it should be ready soon16:10
*** annegentle has quit IRC16:10
jeblairclarkb: pashkin openstack-infra list16:10
clarkbjeblair thanks16:11
clarkbI will get a link16:11
openstackgerritMichael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting  https://review.openstack.org/15887716:11
clarkbpashkin: http://lists.openstack.org/pipermail/openstack-infra/2015-February/002471.html16:12
*** dannywilson has joined #openstack-infra16:15
*** deepakcs has joined #openstack-infra16:17
andreafjeblair: my first step towards dropping GIT_BRANCH and have a test-matrix based solution: https://review.openstack.org/#/c/160366/16:18
jeblairandreaf: thanks16:18
andreafjeblair: next I would include an additional section in the YAML to express in there the update paths to be tested, and then extract that via python script into GRENADE_BRANCH env variables16:19
*** carl_baldwin has joined #openstack-infra16:19
*** tiswanso has joined #openstack-infra16:20
*** armax has quit IRC16:20
pashkinclarkb Thanks, I see it now16:21
openstackgerritAndrea Frittoli  proposed openstack-infra/devstack-gate: Get default branch from feature.yaml  https://review.openstack.org/16036616:22
*** HeOS has quit IRC16:22
*** carl_baldwin_ has joined #openstack-infra16:24
*** patrickeast has quit IRC16:25
*** patri952 has quit IRC16:25
*** patri781 has quit IRC16:25
*** patri338 has quit IRC16:25
*** patri996 has quit IRC16:25
deepakcsclarkb, Reg. gluster CI, we are running stable on ubuntu, see http://logs.openstack.org/11/159711/1/experimental/gate-tempest-dsvm-full-glusterfs/16:26
*** carl_baldwin has quit IRC16:26
*** carl_baldwin_ is now known as carl_baldwin16:26
deepakcsclarkb, So i would request that we add the ubuntu job on openstack/cinder in experimental pipeline16:26
*** julim has quit IRC16:26
clarkbdeepakcs have you proposed that change?16:26
deepakcsclarkb, no, since you had removed it, wanted to check with you before doing it16:27
*** reed has joined #openstack-infra16:27
clarkbbest to just push those things to gerrit and we can discuss in review. I am fine with it as long as the job isnt restarting continuously and sounds like it is not16:27
deepakcsclarkb, we have 2 test case failures which are expected, we will fix that soon, in the meantime we wanted to get enuf runtime against cinder before the deadline16:27
deepakcsclarkb, yes, its running w/o the oom, restart, issue16:28
*** matrohon has quit IRC16:28
clarkbgreat, I would propose the layout change then16:28
deepakcsclarkb, thanks, will wait for your patch then16:28
clarkbI am catching up on mail looks like there is desire to get a centos7 node on rax to debug further?16:29
deepakcsclarkb, yes, there was some interest from centos-devel folks, so if you can help 1 VM each from rax and hpcloud, they would like to get some data16:29
*** ashleighfarnham has joined #openstack-infra16:30
deepakcsclarkb, BTW, fyi - we added the ubuntu experimental job on our stackforge plugin repo too, once we fix the 2 TC failures, will add it to check pipeline of our repo16:30
*** carl_baldwin has quit IRC16:32
*** boris-42 has quit IRC16:32
*** marcusvrn has quit IRC16:32
*** marcusvrn has joined #openstack-infra16:33
*** mrodden has quit IRC16:33
*** ashleighfarnham has quit IRC16:34
*** mrodden has joined #openstack-infra16:35
*** ashleighfarnham has joined #openstack-infra16:37
*** annegentle has joined #openstack-infra16:37
*** marcusvrn has quit IRC16:37
*** carl_baldwin has joined #openstack-infra16:38
*** marcusvrn has joined #openstack-infra16:38
*** derekh has quit IRC16:39
*** wuhg has quit IRC16:39
*** ddieterl_ has joined #openstack-infra16:39
*** ddieterly has quit IRC16:40
*** melwitt has joined #openstack-infra16:41
*** gyee has joined #openstack-infra16:42
*** mbacchi has quit IRC16:43
*** deepakcs has quit IRC16:44
openstackgerritKun Huang proposed openstack-infra/lodgeit: update 404 link of lodgeit.py  https://review.openstack.org/16036316:45
*** saper has quit IRC16:46
*** _nadya_ has joined #openstack-infra16:46
pleia2clarkb, jeblair: new backups server is ready at 23.253.166.59 (cinder volume mounted, bup users created with homedirs on the cinder volume, backups synced from old server), should poke around and make sure everything looks good, then see about switching over to it16:49
*** dizquierdo has quit IRC16:49
clarkbpleia2: maybe try do a restore off of new server given the current rsynced data?16:50
jeblairpleia2: cool.  we will need to have each server accept the new ssh host key when we switch16:50
clarkbdocs on that should be on the same ci.openstack.org page16:50
clarkbbut sounds good +1 to switching over asap16:51
*** yfried has joined #openstack-infra16:53
clarkbpleia2: quick glance at fstab, mount, and passwd look good16:53
clarkbalso ssh keys seem to be in place. Anything else you think should be looked at?16:53
clarkbI have a centos7 node held in rax and hpcloud for the glusterfs OOM debugging16:54
pleia2should be it, jeblair is right about this server having a new ssh key so the connecting servers will need to accept it16:54
*** markus_z has quit IRC16:54
clarkbdeepakcs is no longer here... I guess I can ask for ssh keys on the ml thread16:55
jeblairpleia2: i think try a restore, and if it seems halfway reasonable, let's go ahead and do the switch, and see if it looks like things backed up16:55
*** unicell has quit IRC16:55
pleia2ok16:55
clarkb+116:55
*** dustins_ has quit IRC16:55
*** e0ne is now known as e0ne_16:55
*** e0ne_ is now known as e0ne16:56
*** trey has quit IRC16:56
*** trey has joined #openstack-infra16:57
*** radez_g0n3 is now known as radez16:57
openstackgerritMatt Riedemann proposed openstack-infra/project-config: Add bashate job for release-tools  https://review.openstack.org/16041916:58
*** asselin__ has joined #openstack-infra16:58
clarkbgood news everyone, I will be presenting to the PSU CS capstone groups either april 20 or april 27. Then they get to rank and choose projects so hopefully we get selected16:59
mrmartinre17:00
*** jaypipes has joined #openstack-infra17:00
*** harlowja_at_home has quit IRC17:00
mrmartinif somebody have a little time, can I get an approval on this: https://review.openstack.org/#/c/159426/ ?17:01
jeblairclarkb: yay!  what do you need help with?17:01
anteayaclarkb: yay!17:01
mrmartinit is blocking the groups portal scheduled jobs.17:01
clarkbjeblair: right now not much. I need to get a first draft of the pitch together then will ask for review/edits on that17:01
Ngmordred: so, if these services are enabled on a centos7 system, they'll mount a configdrive and kick off some script or other, but I'm not really sure what the benefit is over what's already in your changeset17:01
Ngmordred: http://paste.openstack.org/show/185245/17:01
anteayaclarkb: what are the capstone groups?17:01
anteayasorry if I should know context, I don't17:02
Ngmordred: (but it would likely be worth having the mount options either way, because securitah :)17:02
*** asselin_ has quit IRC17:03
mordredNg: well, whats in my changeset doesn't work17:03
mordredNg: so if these actually run the script17:03
zaromorning17:03
mordredNg: then they are 100% better17:03
*** _shaps_ has quit IRC17:03
clarkbanteaya: PSU has two tracks for undergraduate studies, there is one track where you write a thesis and mosty that is for students that intend on going to graduate school. Other students do a capstone which is typically a 6 month projects in a small group of 4-5 people. In this case the computer science department has external entities pitch projects to the teams which they then select from and work17:03
clarkbon for the next 6 months17:03
clarkbanteaya: my capstone at PSU is how I know mordred :)17:04
mordredNg: (also, feel free to modify my changeset and submit new patchsets - no ego/ownership issues round these parts)17:04
mordredclarkb, anteaya: yup17:04
*** dizquierdo has joined #openstack-infra17:04
mordredclarkb: happy to help in any way I can17:04
clarkbso my idea is to propose vinz webui as a capstone project and see if we can get people working on that17:04
Ngmordred: :)17:04
anteayaclarkb mordred awesome!17:04
*** bdpayne has joined #openstack-infra17:04
anteayaoh yes17:05
*** Sukhdev has joined #openstack-infra17:05
anteayawhat a good idea17:05
*** mugsie has quit IRC17:05
*** yfried has quit IRC17:05
*** hdd has joined #openstack-infra17:06
*** mugsie has joined #openstack-infra17:06
*** mugsie is now known as Guest4600017:06
*** annegentle has quit IRC17:07
*** prad has joined #openstack-infra17:08
*** sergsh has quit IRC17:09
*** scheuran has quit IRC17:09
*** rwsu has joined #openstack-infra17:09
*** Ala has quit IRC17:10
*** julim has joined #openstack-infra17:10
clarkbhttp://logs.openstack.org/59/160059/1/check/gate-infra-puppet-apply-trusty/7b980a5/console.html#_2015-02-28_00_59_48_460 is interesting17:10
clarkbI am going to recheck and see if that is consistent on hpcloud images17:11
*** rkukura has quit IRC17:11
*** markvan__ has joined #openstack-infra17:13
*** markvan_ has quit IRC17:13
*** cnesa has joined #openstack-infra17:14
clarkbok next step try and replicate rcarrillocruz's devstack sigttin with devstack17:14
*** e0ne has quit IRC17:14
*** yamahata has quit IRC17:16
*** salv-orlando has joined #openstack-infra17:16
pleia2neat, bup is not installed on the new backups server17:17
*** pblaho__ is now known as pblaho17:17
jeblairwhopsie17:17
jerryz_clarkb: do you have to talk about nodepool ipv6 support?17:18
clarkbjerryz_ sure17:18
jerryz_clarkb: i see right now, nodepool db is storing both public ipv4 or v6 and private v417:19
*** Guest46000 has quit IRC17:19
*** Guest46000 has joined #openstack-infra17:19
*** Guest46000 is now known as mugsie_17:19
*** ddieterl_ has quit IRC17:20
clarkbI think just v4 public and private until your change gets in, but yes17:20
*** kirshil has quit IRC17:20
anteayacan I get third party ci account number 5170 disabled please? http://paste.openstack.org/show/185255/17:20
jerryz_clarkb: what did you mean by "recording an ipv4 address and an ipv6 address"17:20
anteayapleia2: you up to learning how?17:20
jerryz_clarkb: do you mean both ipv4 and ipv6 public ips?17:21
*** tonytan4ever has quit IRC17:21
clarkbjerryz_ yes17:21
*** jistr has quit IRC17:21
*** ddieterly has joined #openstack-infra17:22
jerryz_clarkb: in one column or two?17:22
clarkbtwo, we will need to update the db schema for this17:22
clarkbmordred has a change to add alembic support so we can likely base it on that17:23
anteayaclarkb: can I bug you for the disable since it appears pleia2 is busy?17:23
jerryz_clarkb: cool17:23
pleia2anteaya: sorry, being pulled in many directions this morning17:24
anteayapleia2: I understand17:24
anteayapleia2: don't mean to add17:24
clarkbanteaya yup give me a minute. Were you not able to track them down?17:24
anteayaclarkb: thanks17:24
*** jcoufal has quit IRC17:24
anteayano we asked for claudeb in neutron, no response17:24
anteayawe will re-enable if he does respond, currently there is a patch in neutron to remove hyper-v driver17:25
anteayaso mestery and I aren't sure what the status of the account is17:25
anteayahttps://review.openstack.org/#/c/153394/17:25
*** tgohad is now known as tsg_17:25
*** vigneshvar has joined #openstack-infra17:25
anteayanot sure but claudeb might be the hyper-v ci operator? I don't know17:26
anteayain any case it is failing to build17:26
anteayaor failing tests17:26
clarkbwell they do ci on nova too17:26
anteayahttps://review.openstack.org/#/q/reviewer:hyper-v_ci%2540microsoft.com+status:open,n,z17:26
*** patrickeast has joined #openstack-infra17:26
anteayaI went through this list17:26
*** carl_baldwin has quit IRC17:26
*** adam_g_out is now known as adam_g17:27
anteayathe top 5 patches all report failure17:27
*** sputnik13 has joined #openstack-infra17:27
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email base plugin.  https://review.openstack.org/15194817:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email SMTP Client  https://review.openstack.org/14979217:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Send Cron Worker  https://review.openstack.org/15141617:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Working Directory Utility Method  https://review.openstack.org/15194717:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Outbox  https://review.openstack.org/15141317:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Redirect Browsers from API to Client  https://review.openstack.org/15555117:28
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add microsecond columns to the test_runs table  https://review.openstack.org/16005717:29
*** ayoung is now known as ayoung-lunx17:29
anteayaas I said in neutron I would like to take an action and move on so that I don't forget17:31
anteayaas what happened with plumgrid17:31
anteayaI left it with someone else with the request that a post be made to the mailing list and the post never happened17:32
anteayajust the removal from non-voting17:32
*** asselin__ has quit IRC17:32
anteayaclarkb: belay that claudeb showed up in neutron17:34
pleia2clarkb: could use some help figuring out what I missed when setting this up that caused it to lack the bup stuff (but puppet installed all our system users fine?)17:34
clarkbpleia2: sure let me see17:34
SpamapSmordred: I agree that I hope nova's test fakes continue to function. What I'm worried about is that we start depending on how they behave in ways that change, like expecting certain names go in and from that id's come out.17:34
clarkbanteaya: ok please ping if you still need that deactivation17:35
anteayaclarkb: thanks will do17:35
anteayahopefully claudeb can shed some light17:35
clarkbpleia2: check out openstack_project::backup_server17:35
clarkbpleia2: it is quite... empty :)17:35
pleia2clarkb: heh, yeah17:36
openstackgerritJames E. Blair proposed stackforge/gertty: Fix crashing on files with no changes  https://review.openstack.org/16043217:36
clarkbpleia2: so I don't think anything you did was wrong, puppet did exactl as it was told17:36
pleia2which was nothing \o/17:36
pleia2well, set up the template system17:36
clarkbya which is iptables and admins17:36
pleia2so how'd we install these other backups servers?17:36
clarkbpleia2: jeblair set them up probably installed it by hand17:37
clarkbpleia2: we can likely just add the bup package you need to that manifest and be good right? iirc bup shouldn't need anthing other than ssh and users17:37
jeblairNikitaKonovalov, yolanda: what's the current state of session timeout in storyboard?  i'm working on closing some bugs, and my session times out quickly...17:37
*** vigneshvar has quit IRC17:37
pleia2clarkb: makes sense to me17:37
pleia2I'll write a patch now17:37
*** afazekas is now known as _afezekas|pub17:37
SpamapSmordred: I actually think what might be best is if they simply ACK that it should be used in such a way or recoil in horror.17:38
jeblairclarkb, pleia2: oh, sorry :(17:38
yolandajeblair, it keeps me logged until i close browser17:38
SpamapSmordred: if the latter, then we can suggest they be made a real public API and supported properly. :)17:38
yolandamaybe some cookies issue?17:38
mordredjogo, sdague: ^^ could we get a thought on the SpamapS question above?17:38
jeblairyolanda: can you elaborate?17:39
mordredjogo, sdague: the question si saroudn re-use of novaclient client fakes in teh shade tests17:39
*** alexpilotti has joined #openstack-infra17:39
yolandajeblair, i'm not sure on the internals of it, but i don't see a timeout for me inside storyboard webclient17:39
sdaguecan someone crisply state the question?'17:39
yolandai just need to logout, or close browser and clean cookies to logout17:39
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: User typeahead permits unassign.  https://review.openstack.org/15994817:40
anteayasdague: what is the answer to life the universe and everything?17:40
mordredsdague: is it a disaster to make the shade unittests use the Client fakes in python-novaclient17:40
*** tsg_ has quit IRC17:41
jogomordred: probably, we don't really treat it as a stable API at the moment17:41
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: User typeahead permits unassign.  https://review.openstack.org/15994817:41
SpamapSjogo: https://review.openstack.org/#/c/160155/17:41
SpamapSsdague: ^^17:41
openstackgerritMerged openstack-infra/storyboard: Removed PostgreSQL from tests  https://review.openstack.org/16027217:42
greghayneshaha, allow me to just import your tests....17:42
SpamapSgreghaynes: your tests are not _tests so I will import them as I see fit. ;-)17:42
openstackgerritElizabeth K. Joseph proposed openstack-infra/system-config: Add bup package to backup_server.pp  https://review.openstack.org/16043317:43
sdagueSpamapS / mordred: so it's probably a medium dangerous thing to do, but the reality is people don't tend to change that stuff willy nilly17:44
*** patrickeast_ has joined #openstack-infra17:44
*** patrickeast__ has joined #openstack-infra17:44
*** patri136 has joined #openstack-infra17:44
*** patri505 has joined #openstack-infra17:44
openstackgerritDmitry Tantsur proposed openstack/requirements: Add ironic-discoverd to projects.txt  https://review.openstack.org/15627017:44
SpamapSsdague: I kind of want it to be a 1st class thing. It's super useful for users.17:44
sdagueso you might just need to tweak things on releases, but I wouldn't imagine much17:44
*** yamahata has joined #openstack-infra17:44
SpamapSsdague: I'm happy to do the work to document it and explain what is supported and what isn't.17:45
mordredSpamapS, sdague: honestly, if the _interface_ of the fake changes, it ultimately probably means that the interface of the Client object itself changed17:45
*** ChuckC has quit IRC17:45
mordredand that breakage is probably something we want very much to know about :)17:46
SpamapSmordred: agree17:46
*** vigneshvar has joined #openstack-infra17:46
SpamapSIt's perfectly good as a sanity check the way I've done it there.17:46
sdagueyeh, true, the calling interface should be reasonably stable, I'd just imagine the payload might get changed over time to support testing other things the current fake payloads don't cover17:47
SpamapSbut right now, for instance, we can't test anything except 'BUILD' status in shade, because the fake has no way to signal "I want an active server".17:47
SpamapSWhich means I can't add tests for wait=True, because that does create and then immediately waits for it to turn into ACTIVE17:48
*** Ryan_Lane has joined #openstack-infra17:49
jeblairSpamapS, mordred: aware of https://github.com/rackerlabs/mimic ?17:49
clarkbrcarrillocruz: I have a node in hpcloud that wasn't booted off of our images to try and reproduce the sigttin thing17:50
clarkbrcarrillocruz: will let you know how that goes17:50
*** mpavone has quit IRC17:50
*** ihrachyshka has quit IRC17:50
mordredjeblair: yes17:51
*** tonytan4ever has joined #openstack-infra17:51
SpamapSfunny they call it a mock17:52
SpamapSbut it is clearly a fake. ;)17:52
SpamapSwhich makes me :|17:52
SpamapSjeblair: looks realy cool actually17:53
sdagueso... if you want something complicated enough to have state progression, I don't think you'll get it out of the novaclient tree easily17:53
anteayawhat is the difference between a mock and a fake?17:53
SpamapSI sort of think nova should have a way to just start nova-api --fake with exactly that behavior.17:53
sdagueSpamapS: well... I have a fixture in the nova tree for it17:53
*** MaxV has quit IRC17:53
*** carl_baldwin has joined #openstack-infra17:53
SpamapSsdague: can haz hint how to find? kthx17:54
*** yfried has joined #openstack-infra17:54
sdaguehttps://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L287-L34517:54
*** dhritishikhar has joined #openstack-infra17:54
zxiiropelix: thanks :)17:54
zxiironow to find someone else to +2 it...17:54
zxiiroany JJB committers have time to review and merge https://review.openstack.org/#/c/155865/   ?   (I'd be very greatful!)17:55
*** _nadya_ has quit IRC17:55
sdaguethough, you need to do a few more things to get a full nova, but we also have ways to start the services in the test process with an in memory sqlite and in memory oslo.message queue17:56
*** tonytan4ever has quit IRC17:56
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Bump oslo.log for critical nova regression  https://review.openstack.org/16043517:57
sdaguethat infrastructure is all in the nova test tree, it's just not super easy to export right now17:57
SpamapSsdague: I think it should stay in the nova tree17:58
pc_mCan I get a second core reviewer on the new VPNaaS functional job being created? https://review.openstack.org/#/c/158389/17:58
sdagueSpamapS: sure, by export I mean easy to consume out of tree17:58
SpamapSsdague: but if it could actually be wrapped up in a bundle as 'nova-api --fake' so test authors can just point their functional test suite at a real nova that doesn't require any infrastructure, that would be huge for end users.17:58
SpamapSI know that's a tall order.17:59
*** Guest26642 has quit IRC17:59
sdagueSpamapS: yeh, I've actually been thinking about that actually17:59
SpamapSBut it's something to consider. :)17:59
SpamapSsdague: double actually, so you know it's true. :)17:59
sdague:)17:59
sdaguesee that's the problem when I start writing one sentence, and lose context on a though until the end, having forgotten I used that word already18:00
SpamapSI think your subconscious served you perfectly there. :)18:00
*** BobBall is now known as BobBall_AWOL18:00
*** unicell has joined #openstack-infra18:00
SpamapSsdague: I'd guess nodepool would love to have tests that hit a fake nova too.18:00
sdagueanyway, in trying to come up with constructive functional test patterns in the nova tree, it has occurred to me how close it gets us to a test stack that could actually be bundled into a command to bring it up.18:01
sdagueeither with a libvirt, or with a fakevirt18:01
* sigmavirus24 is hoping someone can take a look at https://review.openstack.org/#/c/160006/ today18:01
*** otter768 has joined #openstack-infra18:01
SpamapSsdague: You have my full support in making something like that a reality and it would have direct benefit to work I'm interested in. :)18:01
* SpamapS returns to email-slog for a bit18:02
*** mbacchi has joined #openstack-infra18:02
sdagueyeh, it won't be right away, but I'll try to keep that on the list18:02
*** ivar-lazzaro has joined #openstack-infra18:03
SpamapSI'm reminded that back when I was doing work with RealtyTrac and MLS, they both have fake services that always behave a certain way, which was realy nice for writing automated tests _and_ for development.18:04
*** jerryz_ has quit IRC18:05
*** boris-42 has joined #openstack-infra18:05
SpamapSand even once uncovered an issue with our version of libssl IIRC, where they dropped support for an old cipher and we accidentally didn't have any new ciphers. :-P18:05
*** otter768 has quit IRC18:06
*** annegentle has joined #openstack-infra18:07
*** tiswanso has quit IRC18:08
*** ssam2 has quit IRC18:09
*** mbacchi has quit IRC18:09
*** dustins has joined #openstack-infra18:09
*** vilobhmm has joined #openstack-infra18:10
*** _afezekas|pub has quit IRC18:10
asselinjeblair, fyi, I'm seeing this zuul issue again. Not sure why it's not hitting -infra. Fills up the log files: http://paste.openstack.org/show/185263/18:13
mordredSpamapS: actually, nodepool is being ported to shade - so "shade having functional tests" will be the mechanism whereby nodepool can test its nova interactions18:13
mordredSpamapS: although don't forget that much of the difficulty actually comes around interacting with the other services - glance and neutron and keystone - and figuring out when to use nova and when to use glance/neutron18:14
mordredSpamapS: you'll also love the things I've been learning about keystone v318:14
jeblairmordred, SpamapS: i expect we'll have devstack-based testing for shade soon (and then later, nodepool).18:15
*** jedimike has quit IRC18:15
mordredyah18:15
mordredI'd like to see more effort focused on that than too much working standing up fake novas18:15
stevemarmordred, care to share what you've learned? :)18:15
mordredstevemar: domains and projects and roles and users are all infinitely combinable18:16
mordredstevemar: and in v3, you _must_ pass domain information, as the project list is not automatically filtered by user's current logged in domain18:16
mordredbut in v2, there are no domains, so you can't make domain an interface level required parameter18:16
stevemarmordred, bah, the second point should be fixed.18:16
mordredwhich means that to get sane defaults, I intend to pull the user's current domain from the token18:17
mordredand if the domain is not provided for any domain parameters, provide the one from the current token18:17
mordredthis is mainly to facilitate passing project and user by name and not by id18:18
stevemarmordred, this should be interesting once federated users get into the mix18:19
mordredstevemar: I'm sure it's going to be magical18:19
stevemarthe have a 'federated' domain,18:19
*** dizquierdo_ has joined #openstack-infra18:19
anteayaasselin: any idea if it is related to a change or a timestamp?18:19
*** dizquierdo has quit IRC18:20
asselinanteaya, I think it's related to a depends-on change18:20
asselinanteaya, still investigating....18:20
*** tiswanso has joined #openstack-infra18:20
anteayaasselin: okay keep us posted18:21
*** harlowja_away is now known as harlowja_18:21
*** mugsie_ is now known as mugsie18:22
mordredjogo: the patch to fix the console-scripts merged in pbr18:23
mordredjogo: in the feature/0.10 branch18:23
jogomordred: woot!18:23
jogothanks, when is the release expected?18:24
*** hdd has quit IRC18:24
mordredpleia2, jeblair, clarkb: so - we've got another patch in 0.10 of pbr that we need to release to make jogo happy - any thoughts?18:25
mordreddhellmann: ^^18:25
clarkbmordred: wfm. Though next thing that breaks without working semver I propose we revert all of semver and put that on a feature branch18:25
SpamapSjeblair: I agree, and I think that's quite important as a true integration test. I also don't really want to spin that up for every patch to shade or nodepool. :-P18:25
clarkbmordred: also any chance you will update 159159 soon?18:25
mordredclarkb: oh - piddle - I actually walked my outstanding patches-needing-work this morning but didn't see that because of the gerrit "show +2 over -1" logic18:26
SpamapSjeblair: but maybe I'm just resisting for no good reason. I like self contained unit tests for tighter development iteration.18:26
jeblairSpamapS: locally, sure.  I'm not suggesting it as a substitute for the convenienc of full local testing.18:26
*** AJaeger_ has joined #openstack-infra18:27
jeblairSpamapS: however, we do have a system that will do that for you automatically, so we can have that testing now at very little cost.  :)18:27
SpamapSAnd faked functional tests also blow my mind when they work right because they test actual things against some partially relevant things. :)18:27
SpamapSfast18:27
clarkbmordred: are you saing I should -2 more liberally :P18:27
jeblairSpamapS: i would really like both.18:27
SpamapSjeblair: Yeah, it's low cost, high latency.18:27
mordredclarkb: :)18:27
*** yfried has quit IRC18:28
*** dtantsur is now known as dtantsur|afk18:28
SpamapSEven if we have nodepools under capacity and images ready to go and [.. magic ..] .. It's still a few minutes.18:28
jeblairSpamapS: yep.  so the end state for me is local functional testing to help iterate quickly, and full devstack testing to make sure we don't land changes that break18:28
jeblairSpamapS: we just happen to be in a position to be able to do the second now, and the first will take more work18:28
SpamapSjeblair: we share end state. I think that means we combine into a Voltron.18:29
*** ZZelle_ has joined #openstack-infra18:29
mordredjeblair, SpamapS: also, at some point, it would be neat if we knew how to configure a devstack to behave like HP and how to configure a devstack to behave like RAX18:29
* SpamapS hopes jeblair gets that and isn't googling Voltron right now ;)18:29
anteayaclarkb: to follow up hyper-v ci disabled their system and updated their system status wikipage18:29
SpamapSmordred: the result of that is exciting. The work estimate seems daunting.18:30
mordredjeblair, dhellmann, clarkb, jeblair: ok. I'm going to cut 0.10.8 - just as an FYI in case something in our testing missed something and the world decides to break18:30
mordredI _think_ we're pretty well covered at this point though18:30
*** achanda has joined #openstack-infra18:31
pc_mclarkb: Can you review https://review.openstack.org/158389? I need a second core reviewer.18:31
* anteaya goes for a walk18:31
mordredjogo: ^^ released - mind keeping an eye out for bonghits?18:32
SpamapSmordred: 0.10.8 of what?18:33
SpamapSIs there a place all the tags -> tars gets logged?18:33
SpamapSseems like a worthy thing for gerritbot to handle.18:34
jogomordred: thanks, will do!18:34
clarkbSpamapS: pypi18:34
mordredSpamapS: pbr18:34
openstackgerritMerged openstack-infra/project-config: Fix glob pattern for os-ansible-logs publisher  https://review.openstack.org/16000618:37
*** dhritishikhar has quit IRC18:39
jeblairasselin: that's strange; i think i see one instance of it in our last month of zuul logs18:42
clarkbrcarrillocruz: I have reproduced on an hpcloud node18:43
jeblair  File "/usr/local/lib/python2.7/dist-packages/zuul/trigger/zuultrigger.py", line 74, in _createProjectChangeMergedEvent18:43
jeblair    event.project_name = change.project.name18:43
jeblairasselin: ^ actually, that's different; so we haven't seen your exact error :(18:44
asselinjeblair, it's still logging in my zuul debug.log file.18:45
SpamapSclarkb: but thats _the world_ of pypi.18:45
SpamapSclarkb: I'm kind of thinking if I care about reviews of project x, I care about releases of it too, is all.18:46
*** achanda has quit IRC18:46
*** achanda has joined #openstack-infra18:46
*** mpaolino_ has joined #openstack-infra18:47
asselinjeblair, this started occuring when the cross-project depends on change merged. Perhaps something didn't completely update on my system?18:48
patrickeastasselin: i see that error in my log as well as on my zuul status page it displays an error18:49
*** marun has joined #openstack-infra18:49
asselinpatrickeast, thanks for confirming18:50
*** pelix has quit IRC18:50
*** unicell has quit IRC18:52
*** adalba has joined #openstack-infra18:52
*** mpaolino_ has quit IRC18:52
*** acfleury has quit IRC18:52
pleia2if we can get this in, I can rerun puppet on the new backups server https://review.openstack.org/#/c/160433/18:52
clarkbrcarrillocruz: it is the apt-get install that is hanging on the sigttin18:53
*** Rockyg has joined #openstack-infra18:54
clarkbpleia2: +218:54
*** crc32 has joined #openstack-infra18:55
asselinpleia2, hi, our gerrit ip  firewall rule was applied. Let me konw when you're ready to test18:55
*** zul has quit IRC18:56
*** zul has joined #openstack-infra18:56
clarkbrcarrillocruz: it appears that root is running sudo18:57
greghaynesclarkb: You mentioned rest api patches for zuul at one point, any idea if those were posted somewhere?18:58
greghaynesfor the zuul webhook deal I need something very similar18:58
mordredgreghaynes: nobody has posted patches to my knowledge18:58
greghaynesok18:59
greghaynescode reading time, then18:59
jeblairgreghaynes: i think there is a rest api patch for nodepool18:59
openstackgerritMerged openstack-infra/system-config: OpenStack Taiwan User Group mailing list  https://review.openstack.org/15892318:59
greghaynesah, nodepool18:59
greghaynesoh, but theres already zuul.webapp :)19:00
pleia2asselin: yeah, sorry, working to get to it today19:00
asselinpleia2, ok thanks. let me know if you need any help19:00
*** ghostpl_ has quit IRC19:01
*** devlaps has joined #openstack-infra19:01
clarkbzuul has a webserver just not a real restapi19:01
jeblairgreghaynes: check in with jhesketh, i know he has/had an interest in expanding that19:01
greghaynesjhesketh: ^ hello there19:02
jeblairgreghaynes: but yeah, if you just need a hook endpoint, could probably stick it in webapp or something19:02
jeblairgreghaynes: (and will likely be some hours before jhesketh is awake)19:03
*** crc32 has quit IRC19:03
clarkbrcarrillocruz: so its definitely something hanging up http://paste.openstack.org/show/185362/ which does work when I run it in the foreground with the stack user19:04
clarkbjeblair: any ideas on why ^ would get SIGTTIN despire have the noninteractive and assume yes flags set?19:04
*** Sukhdev has quit IRC19:04
*** zz_johnthetubagu is now known as johnthetubaguy19:07
*** arxcruz has quit IRC19:08
*** annegentle has quit IRC19:10
*** ChuckC has joined #openstack-infra19:11
*** crc32 has joined #openstack-infra19:12
*** unicell has joined #openstack-infra19:13
*** hdd has joined #openstack-infra19:13
clarkbrcarrillocruz: so I think jenkins must be doing something that affects apt-get/dpkg differently than running it in an interactive shell19:13
mordredclarkb: wow19:13
clarkband its possible that devstack-gate tickles that19:13
clarkbsince we don't seem to see this problem when people just run devstack19:13
clarkbmordred: if you have an insight into that it would be appreciated19:14
mordredclarkb: only thing I can imagine is that it has something to do with how the java process is forking19:14
mordredclarkb: I seem to remember in the fog of my brain that it's one of the reasons we stick a bash -c in front of some things19:14
clarkbmordred: possible, like maybe it is handing off a different stdin or something19:14
mordredyah19:14
clarkboh also that doesn't seem to be a problem with our images19:15
clarkbso maybe jenkins isn't at fault but something we do during image build? maybe the dist upgrade? its all veryweird19:16
mordredclarkb: I am looking forward to uncomplexity this some more19:16
clarkbso after manually istalling that package set then restarting d-g it does the same thing against a subsequent apt-get install command19:17
clarkbso I don't think a specific package is at fault19:17
openstackgerritMerged openstack-infra/jenkins-job-builder: Remove unnecessary warning message.  https://review.openstack.org/15854319:17
*** tonytan4ever has joined #openstack-infra19:20
reedjeblair, in the auto-abandon discussion, I think fifieldt gave a very good suggestion. I'd pick that one up19:20
*** cdent has quit IRC19:20
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Story events may now be filtered by event_type.  https://review.openstack.org/16046219:21
*** annegentle has joined #openstack-infra19:27
clarkbAHA! I think it may be timeout that is at fault19:27
* clarkb tests some stuff19:28
reedlooks like we didn't make it to GSoC 201519:29
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade  https://review.openstack.org/15846319:29
clarkbrcarrillocruz: ^ btw I think our use of timeout is at fault, unsure why it doesn't break when jenkins uses it but going to test timeout with the --foreground optin19:29
*** devlaps has quit IRC19:29
pleia2reed: aw, sorry to hear19:29
reedwe're in good company, with mozilla, linux foundation, tor19:30
rcarrillocruzclarkb: awesome, thx for confirming re:sigttin19:30
*** yfried has joined #openstack-infra19:31
rcarrillocruzSpamapS: thx, the cleanup should indeed go in setUp to avoid duplication19:31
*** dimtruck is now known as zz_dimtruck19:32
*** e0ne has joined #openstack-infra19:32
*** e0ne is now known as e0ne_19:32
*** e0ne_ is now known as e0ne19:32
*** tkelsey has quit IRC19:33
clarkbrcarrillocruz: also unsure why it works when we use our own images19:34
rcarrillocruzour own images = dib images?19:34
clarkbor snapshot images, basically anything built by jenkins19:35
clarkber nodepool19:35
rcarrillocruzah ok19:35
nibalizerreed: so no gsoc for openstack this year?19:36
clarkbif this works I will propose a change to use the --foreground flag to double check that jenkins is happy with it too19:36
reednibalizer, doesn't seem likely19:36
jogomordred jeblair: question about the zuul evolution plan19:37
jogomordred jeblair: would that help make things like https://review.openstack.org/#/c/160194/ better19:38
jeblairreed, fifieldt: yeah, i think it is a good idea -- however, i'd want to try to keep it to a minimum (ie, only leave that message once per user or something...); i would not think it would be helpful for, say, mordred's changes to have 50 copies of that message on each :)19:38
jogoaka more obvious that its an issue, make it not an issue etc.19:38
jeblairreed, fifieldt: but i agree the general idea is good, and we can hash details like that out later19:39
reedjeblair, yeah, it's just a matter of optimizing the query or to write the text more about each individual review19:40
*** ghostpl_ has joined #openstack-infra19:40
*** ghostpl_ has quit IRC19:40
*** ghostpl_ has joined #openstack-infra19:40
jeblairjogo: i think job inheritance and/or project-specific job configuration might help with things like that19:41
clarkbthe project specific job configuration is what I was thinking would help19:41
clarkbso rather than apply job data to neutron and nova instead you would sa from neutron and nova's perspective that these jobs should do X and Y19:42
openstackgerritMerged openstack-infra/system-config: Updates to running-your-own CI docs: Initial setup section  https://review.openstack.org/15586419:42
jogoclarkb: nice19:42
jogojeblair clarkb: thanks, wanted to bring this up as an example where things are confusing in the current model. sounds like the proposal will help!19:43
clarkbpleia2: I think you can self approve the bug package change19:43
pleia2clarkb: k19:44
clarkbjogo: fwiw I am not sure it should be confusing, the current behavior is simple and deterministic19:44
*** koolhead17 has quit IRC19:44
clarkbjogo: and its always had this behavior since we first added the functionality19:44
clarkbjogo: maybe we need better documentation?19:44
pleia2re: https://review.openstack.org/#/c/148949/ does our mailman stuff actually apply changes like to the admin after a list has been created? (I know people adjust descriptions post creation)19:45
jogoclarkb: the current behavior is non obvious and its easy to miss the subtle interactions19:46
jogoas seen by the fact that no one caught this in my patch19:46
clarkbjogo: how is it non obvious?19:46
jogothe first time19:46
clarkbjogo: I think the larger issue is we are using regexes here and people often don't parse regexes well19:46
jogoclarkb: yes, that and regex and a massive config file means it is easy to literally miss a conflict19:47
clarkbpleia2: I don't think it will apply the update, but is useful should we ever need to redeploy the list server?19:47
pleia2clarkb: fair enough, I'll +A and make a comment accordingly19:48
pleia2ah, looking at the live list it looks like they did go in to mailman and update their address already19:48
AJaeger_jogo: is https://review.openstack.org/#/c/160194/ urgent to get in? How broken is the situation?19:49
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Read user preferences from backend  https://review.openstack.org/13779819:49
jogoAJaeger_: not very AFAIK, it only impacts neutron19:49
jogoand means extra jobs run19:49
jogonot fewer19:49
clarkbjogo: is the last match wins behavior not what you would expect? I guess that is my real question19:49
jeblairjogo: let's make sure marun is involved in reviews on skip-if changes19:50
clarkblast match wins is what I prefer to see as its unambiguous vs merging things and which side has priority and so on19:50
jogoclarkb: no, its not. because of the 'project' part of skip-if19:50
jogojeblair: yes please, good point19:50
clarkbjogo: not sure I follow19:50
clarkbjogo: you want something other than last match wins?19:50
jogojeblair AJaeger_: plus this impacts code marun wrote19:50
clarkbwhat would you suggest as a better alternative to last match wins?19:51
marunjogo: hmmm, so the nova regex was too broad19:51
clarkbjogo: I don't think we want to merge things becuse then you get ambiguity in a less easy to understand way19:51
marunjogo: I think it would be preferable to exclude the neutron jobs from nova rather than merging them19:51
*** zz_dimtruck is now known as dimtruck19:51
clarkbmarun: why is that?19:51
jogomarun: I don't think we can even do that19:52
marunjogo: Really?19:52
clarkbdoing so makes the regex more complicated and I think the regexes are where we acutally have the problem19:52
clarkbjogo: you can19:52
marunjogo: (!?neutron)19:52
jogomarun: job names are the same19:52
jogonova runs those tests too19:52
marunjogo: they are not the same19:52
marunjogo: hmmm19:52
clarkbeither way the current set for neutron and nova has to stay19:52
jogocheck-tempest-dsvm-neutron-full  gate-tempest-dsvm-neutron-large-ops19:52
clarkbyou can have a second set for non neutron jobs though19:52
jogoclarkb: I assumed the 'project: ^openstack/neutron$' line prevented conflicts like this19:53
marunjogo: ah, I see19:53
clarkbjogo: its not a conflict19:53
marunjogo: the skip-if application is done without regard to the target projects19:53
clarkbjogo: its specifically the lack of any conflicts via last match wins19:54
jogomarun: well I know that know19:54
marunjogo: so that the matcher for one project can't co-exist with another for the same job19:54
clarkbif however you switch to merging things that would require conflict resolution19:54
marunclarkb: might it be preferable to allow merging skip-if clauses across metajobs?19:55
clarkbmarun: imo no, because we have never merged any of that job data19:55
marunclarkb: it should be possible to detect conflicts as errors and prevent them from merging19:55
*** matrohon has joined #openstack-infra19:55
openstackgerritMerged openstack-infra/project-config: Updated the Chef repo description  https://review.openstack.org/15966419:55
AJaeger_marun: could you document this, please? I'd like some more reference for the skip-if19:55
openstackgerritMerged openstack-infra/system-config: Add bup package to backup_server.pp  https://review.openstack.org/16043319:55
clarkbmarun: it would introduce new behavior that isn't consistent with the other job flags19:55
jogomarun: so your review on https://review.openstack.org/#/c/160194 would be great (even if its a -1)19:55
marunclarkb: metajobs are hacky today19:56
clarkbmarun: they really aren't19:56
clarkblast match wins19:56
clarkbits simple19:56
marunclarkb: and that's not hackey?19:56
clarkbno19:56
*** carl_baldwin has quit IRC19:56
marunclarkb: it seems pretty easy to screw up19:56
*** crc32 has quit IRC19:56
marunclarkb: maybe not to you, but to fresh eyes?19:56
clarkbits deterministic, avoids merge conflicts problems19:56
clarkbthe problem is more with regexes than with last match wins imo19:56
clarkbbecause its harder for people to read regexes19:57
jogoclarkb: I agree with marun, its hard to reason that out on  really big config file without reading the full config file19:57
marunclarkb: really?19:57
clarkbmarun: yes, what is confusing about last match wins?19:57
marunclarkb: what is confusing about regexes?19:58
jogohttps://xkcd.com/1171/19:58
*** vilobhmm2 has joined #openstack-infra19:58
marunthat seems to me to be the least complex piece in play19:58
clarkbmarun: regexes model large state machines that are hard for people to keep track of19:58
* jogo goes to lunch19:58
marunclarkb: ...as opposed to the config itself?19:58
marunclarkb: I mean, honestly?19:58
openstackgerritMerged openstack-infra/system-config: Update mailman contact for openstack-fr mailing list  https://review.openstack.org/14894919:59
clarkbmarun: the config itself isn't a massive state machine with many transitions. Most of it is neatly self contained under its branch of the tree19:59
*** e0ne is now known as e0ne_19:59
*** xyang1 has joined #openstack-infra19:59
marunclarkb: if you say so19:59
*** saper has joined #openstack-infra19:59
clarkbmarun: last match wins is part of that, if I read through the list I know exactly what behavior I will get at the end19:59
marunclarkb: yeah, I got that *you* understand the configuration.19:59
*** bswartz has quit IRC19:59
marunclarkb: I'm not sure why you're not willing to admin that it's complicated for folks that don't live and breath it.'19:59
marunadmin -> admit19:59
clarkbI do admit that portions of it are complicated but not the last match wins behavior20:00
marunclarkb: and that's today...  this is going to keep evolving.20:00
clarkbbecause thats simple20:00
marunclarkb: simple to implement, I get.20:00
mordredclarkb: I think I'd say that, if we have people coming and saying that as new users it's complicated for them, it's worthwhile to hear that - I mean, I'm sure that keystone domains make sense to the keystone team20:00
clarkbIMO the complicatedness is part of using regexes as the keys for elements in the tree20:00
marunclarkb: simple to reason about? Not sure I can agree.20:00
*** vilobhmm has quit IRC20:00
*** e0ne has joined #openstack-infra20:00
clarkbmordred: I agree which is why I was trying to see what jogo found confusing to boil it down to a thing we can fix20:00
stevemarmordred, you'd think so - right? about that...20:00
mordredstevemar: :)20:01
AJaeger_clarkb: the problem for me is that it's not always obvious that a regex matches multiple things *by accident*20:01
clarkbAJaeger_: yes exactly20:01
marunclarkb: I'm suggesting that we add checks for conflicting skip-if matchers.20:01
clarkbmarun: you were suggesting we merge skip ifs20:01
marunclarkb: they could run on merges20:01
clarkbmarun: which is a break in behavior20:01
*** saper has quit IRC20:01
*** saper has joined #openstack-infra20:01
greghaynesclearly we should make on giant CFG out of all the regexes in our config20:01
marunclarkb: so we have 2 choices...20:02
marunclarkb: I'm hearing that manually merging is the preferred path?20:02
*** otter768 has joined #openstack-infra20:02
clarkbmarun: imo yes because it avoids surpises and maintains consistent behavior with the rest of the config20:02
marunclarkb: That's fine. And does that path imply that we don't want errors or warnings when multiple skip-if clauses are found to match?20:03
clarkbno20:03
marunclarkb: ok.20:03
marunclarkb: So I'm happy with jogo20:04
*** patrickeast__ has quit IRC20:04
*** patri505 has quit IRC20:04
*** patri136 has quit IRC20:04
*** patrickeast_ has quit IRC20:04
*** patrickeast has quit IRC20:04
marun's patch as written, and I'll add a story to track better validation to protect against unexpected skip-if overwrite20:04
marunclarkb: sound good?20:05
*** sandywalsh_ has joined #openstack-infra20:05
pleia2heading out for lunch, hopefully will finish up that backups server when I get back20:05
harlowja_could i get https://review.openstack.org/#/c/157580/ looked over, approved20:06
harlowja_just a final addition needed for debtcollector stuffs20:06
clarkbmarun: sure, I would say that we should general warn/error on any overwrites though if that is the approach. To be consistent20:06
marunclarkb: understood.20:06
*** otter768 has quit IRC20:07
marunclarkb: I don't think there is any such capability today, but it would make sense to add it.20:07
clarkb(which I think is my biggest concern here, we shouldn't treat skip-if as special if it is part of the job list config)20:07
SpamapSjeblair: I may have to hack up a vim-mode for gertty. I've cherry picked so many changes hitting x... :-P20:07
*** patrickeast has joined #openstack-infra20:07
*** fifieldt has quit IRC20:07
clarkbSpamapS: its unfortunate that urwid doesn't do it out of the box20:07
*** achanda has quit IRC20:07
*** sandywalsh has quit IRC20:07
*** patrickeast_ has joined #openstack-infra20:08
*** patrickeast__ has joined #openstack-infra20:08
*** patri875 has joined #openstack-infra20:08
*** patri928 has joined #openstack-infra20:08
*** achanda has joined #openstack-infra20:08
*** HeOS has joined #openstack-infra20:08
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: apt-get needs access to tty so give it access  https://review.openstack.org/16048020:08
clarkbrcarrillocruz: ^ I think that will fix your d-g runs20:09
clarkbrcarrillocruz: let us see if jenkins is happy with it20:09
*** EmilienM|afk is now known as EmilienM20:10
mordrednew rant - directed at nova20:11
mordred"No valid host was found. Exceeded max scheduling attempts 3"20:11
*** Rockyg has quit IRC20:11
mordredcompletely useless error message20:11
SpamapSclarkb: I don't know enough about urwid to know if what I'm suggesting would be crazy hard. :-P20:11
clarkbSpamapS: I don20:11
clarkber20:11
mordred_what_ exactly, did I do that caused fail?20:11
clarkbSpamapS: I don't think its crazy hard its just annoying that movement and multi modal behavior isn't something alrady done for you20:11
dstufftmordred: you expected a computer to do somehting for you when you asked20:11
*** cnesa has quit IRC20:11
clarkbmordred: by chance are you using qcow2 images on hpcloud?20:11
dstufftthat's what you did20:11
mordredclarkb: yes20:12
mordredclarkb: am I not supposed to?20:12
*** bswartz has joined #openstack-infra20:12
clarkbmordred: you can but hpcloud doesn't support latest qcow2 version that qemu-img gives you20:12
mordredsigh stab stab20:12
mordredclarkb: thank you20:12
clarkbmordred: you have to restrict the verison of qcow2 to 0.10 iirc, see the nodepool yaml file20:12
greghayneshahahaha, I forgot about that one20:12
mordredgreghaynes: is there a dib param I pass? /me looks at nodepool file20:13
greghaynesno20:13
clarkbthere is20:13
greghaynesno to dib I dont think20:13
clarkbQEMU_IMG_OPTIONS or similar20:13
clarkbnodepool sets it, just grab what nodepool does20:13
greghaynesyea, thats a qemu-img env var20:13
SpamapSclarkb: Honestly, I think it's crazy that I _want_ modal behavior. IMO we're all broken for loving vim. ;)20:13
*** _afezekas|pub has joined #openstack-infra20:14
greghaynesoh wait, we tunnel that through20:14
greghaynesclarkb: konws whats up20:14
greghaynesRelated, it might be sane for dib to set qemu to make qcow2's that are backwards compat by default20:15
AJaeger_jeblair, clarkb : marun has +1'ed 160194, will either of you review and approve?20:15
mordredclarkb: thank you20:15
*** Sukhdev has joined #openstack-infra20:15
*** dizquierdo_ has quit IRC20:16
mordredclarkb: but - you know - not to completely deflect - but it would be great if the nova error wasn't "scheduler error" but instead was "gah qcow2 incompatible version"20:16
clarkbmordred: definitely20:16
mordredI should, in fat, NEVER hear anything about the scheduler20:16
greghaynesReminds me of the old Heat error we would get of "Error in cloud"20:16
clarkbprinter on fire20:16
SpamapSENOCLOUD20:17
*** tsg has joined #openstack-infra20:17
mordredPC LOAD LETTER20:17
SpamapSsomebody should setup a translation for a new language: en_MEME20:18
mordredSpamapS: I can't see anything that would go wrong with that20:18
SpamapSever right?20:18
*** ddieterly has quit IRC20:18
mordredSpamapS: also - any meme reference should be acceptable as a translated string - regardless of relevance20:18
SpamapS"Authentication Failed" --> "Somebody set us up the bomb"20:19
*** ddieterly has joined #openstack-infra20:19
mordredyup20:19
*** bswartz has quit IRC20:19
* ianw on the phone to the bank again, thanks to hpcloud getting my card cut off with it's 8c transaction20:21
*** ddieterly has quit IRC20:23
*** ddieterl_ has joined #openstack-infra20:23
SpamapSianw: hahaha20:24
SpamapSianw: it's cost effective20:24
mordredianw: wow, really?20:24
mordredpurp: ^^ fyi20:24
*** matrohon has quit IRC20:24
SpamapS< $0.10 transactions _are_ the hallmark of CC theft.20:24
*** fifieldt has joined #openstack-infra20:25
ianwyep, it's all automated on the banks end, they can't seem to mark hpcloud as "safe"20:25
mtreinishfungi, clarkb: so the next subunit2sql release will likely have a db migration it. I set things up to auto run the upgrade tool on new releases but I was wondering if maybe we should make a backup before I push it20:26
*** baoli has quit IRC20:26
mtreinishis there a good way to do that? I looked at the mysql_backup module but that was just a cron job and I'm not sure that's what we want here20:26
mtreinishSpamapS: they've just seen office space one too many times :)20:27
openstackgerritMerged openstack-infra/project-config: Revert "Remove lib-forward-testing from openstackclient"  https://review.openstack.org/15959020:29
*** _afezekas|pub has quit IRC20:29
ianwok hp has their 8c and Meg Whitman's salary is covered for another nanosecond20:30
mordredianw: thank goodness20:30
jrollianw: hey, did I get back to you about fedora/rackspace things?20:30
mrmartinmtreinish: why not? just exec the same job that cron is using before your upgrade tool20:30
jrollianw: they confirmed the bug and said they would fix it in the next fedora image they build (which may be done by now, dunno)20:31
mtreinishmrmartin: I meant the cron job isn't what we wanted. But, yeah that's a good idea just run the raw mysqldump command20:31
mtreinishmrmartin: the only issue is the storage location, the upgrade command runs on logstash.o.o and I don't think we want a many gb sql dump on that node20:32
*** vigneshvar has quit IRC20:33
*** juice has quit IRC20:34
mrmartinmtreinish: maybe you could make the backup using a pipe, and it not generates a local gb sql dump20:34
mrmartinjust an idea20:35
mtreinishmrmartin: yeah I can pipe it into somthing, but to where I guess is the real question?20:35
mtreinishI ran a mysqldump locally at home over the weekend and it was ~20gb20:36
clarkbmtreinish: you could have it be in /opt somewhere on the server which is where we mount the ephemeral drive20:36
mrmartinthis bup we are using for backup does deduplication in some form20:36
clarkbmtreinish: then just keep a single backup file20:36
clarkbexcept on logstash.o.o we haven't mounted tha drive20:37
mrmartinanyway, if we have enough storage don't spend the time to find out how to keep disk usage low20:37
clarkbhrm there is no xvde maybe the falvor is wrong20:37
mtreinishclarkb: ok cool, that's easy enough. I just wasn't sure where to put things.20:37
*** david-lyle has joined #openstack-infra20:37
openstackgerritClark Boylan proposed openstack-infra/zuul: Fix race in CRD duplicates test  https://review.openstack.org/16049120:39
clarkbjeblair: ^ another zuul test race fix20:39
mtreinishmrmartin, clarkb: ok I'll push a patch up to store to run a mysqldump in /opt and we can go from there20:40
clarkbmtreinish: note that mysqldump will lock of the db when it runs too iirc20:41
clarkbmtreinish: which should be fine, all of the jobs trying to write to it will just queue up while that happens20:41
mtreinishclarkb: sure I think that's fine, although I can also do it with the single transaction flag (or whatever it is) and that should not lock everything20:42
*** tonytan4ever has quit IRC20:43
*** mfink_ has quit IRC20:45
*** AJaeger_ has quit IRC20:45
*** mrunge has quit IRC20:46
*** baoli has joined #openstack-infra20:46
*** baoli has quit IRC20:47
*** baoli has joined #openstack-infra20:48
rcarrillocruzclarkb: wootz, thx20:49
clarkbI have restarted jenkins02 it died from running out of memory, copied the error file to my homedir from /tmp20:50
clarkb04 is also being slow but is up so looking into that now20:50
*** rlandy has quit IRC20:50
*** eharney has quit IRC20:51
clarkblooks pretty consistent like spanw all the threads for compute remoting then it dies20:51
macjackhttps://review.openstack.org/#/c/158923/ Thanks for help! Appreciated it!20:51
clarkbintesting that http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=826&rra_id=all shows it didn't run out of system meory20:52
clarkbI think it is running out of memory for all the threads20:52
*** achanda has quit IRC20:52
*** kmartin has quit IRC20:53
ianwjroll: ok, thanks, let me check20:54
*** yamamoto has joined #openstack-infra20:56
ianwjeblair / fungi / someone : speaking of f21 images, if we can get f21 dib going that would be good (https://review.openstack.org/#/c/140911/)20:56
HenryGjogo: marun has spoken: https://review.openstack.org/16019420:57
clarkbzaro: have you seen this jenkins thread explosion anywhere else?20:59
clarkbI am trying to get on 04 which is chugging to see if its halfway there and maybe we can inspect it live20:59
clarkbpreparing 04 for shutdown now so that nodes are no longer distributed to it21:00
*** annegentle has quit IRC21:05
jogomarun clarkb: thanks21:07
*** penguinRaider has joined #openstack-infra21:07
*** med_ has quit IRC21:08
*** andreaf_ has joined #openstack-infra21:08
*** andreaf_ has quit IRC21:08
*** kmartin has joined #openstack-infra21:09
clarkbmelody confirms jenkins04 is currently running with 10k threads21:09
clarkbgoing to grab lunch now will see if I can get any more info out of melody while I wait for jobs to finish on that host so I can restart it21:10
clarkbzaro: ^ I think this confirms we are heavily leaking the remote compute threads21:10
mikalHello, can anyone here teach me how to use LP?21:14
*** rm_work is now known as rm_work|away21:14
mikalSpecifically, what controls who creates milestones (and therefore can do releases) for python-novaclient?21:15
mikalI'd like to open that up to more people than just me21:15
jeblairmikal: i think drivers does that?21:17
mikaljeblair: that's not working for us21:17
mikalMembers of drivers say that the milestone page has nothing actionable for them21:17
jeblairhrm21:18
jeblairmikal: maybe it's owner then... i can make drivers the owner21:18
jeblairmikal: shall i?21:18
dansmithmikal: what does the button look like?21:18
openstackgerritMatthew Treinish proposed openstack-infra/puppet-subunit2sql: Add backup before running db migrations  https://review.openstack.org/16051421:18
jeblairmikal, dansmith: this page, right? https://launchpad.net/python-novaclient/+addseries21:19
mtreinishclarkb, mrmartin: ^^^21:19
dansmithno21:19
mikaljeblair: so I'm not 100% sure I understand the side effects of that, but sure21:19
mikalLet's try it21:19
dansmithjeblair: that's to add 3.x.x right?21:19
jeblairmikal, dansmith: "(+) Register a series" under "Series and milestones", right?21:19
mikalWe;re trying to create milestones on https://launchpad.net/python-novaclient/2.x.x21:19
mriedemdansmith: yeah we don't want a new series yet21:19
dansmithright21:19
mkj_wkZZelle_: I have interactive with two unit test cases implemented -- shall I submit it separately or would you rather see a stack of patches with a dependency?21:20
jeblairmikal: erm, i don't think you have any permissions on python-novaclient that the rest of nova-drivers doesn't have21:20
mikalAlso, I also want specific non-drivers people to be able to do this, but let's start with drivers21:20
jeblairmikal: so let's start with: what page can you see that the drivers can't?21:20
mikaljeblair: https://launchpad.net/python-novaclient/2.x.x is where I create milestones21:21
dansmithmikal: can you describe the thing you can click on?21:21
mikaljeblair: I am told that other members of drivers don't see a "create milestone" link there21:21
dansmithI see no "add milestone" link or anything21:21
ZZelle_mkj_wk, if there is no merge conflicts between them, prefer independent changes21:21
mikalI have a "create milestone" under the list of milestones in the middle of the page21:21
dansmithmikal: jeblair: I have no "create milestone" on my page21:21
*** radez is now known as radez_g0n321:21
ZZelle_mkj_wk, it reduces coupling and the merge of one depends not on the other one merge21:22
jeblairdansmith: "Create milestone"?  it's under "Milestones and releases"21:22
mkj_wkit's just pulled out with no change, and unit test added, but there's a real dependency -- test suite for track patch will fail without the interactive patch21:22
dansmithjeblair: nope, don't have that21:22
*** dmitryme has quit IRC21:23
jeblairdansmith: https://launchpad.net/~danms  that's your user, right?21:23
dansmithjeblair: mikal: I just tried logging out and in to see if it was an issue, but it didn't help21:23
dansmithjeblair: yep21:23
mkj_wkbut it's really simple so if you look at it and accept it, then the review won't be stacked21:23
ZZelle_mkj_wk, i am not a core :)21:23
*** alazarev has quit IRC21:23
*** rm_work|away is now known as rm_work21:23
mkj_wkZZelle_: oh, just saw that you had plenty of commits21:23
*** chlong has joined #openstack-infra21:23
*** ruhe has quit IRC21:23
dansmithmikal: jeblair: http://imgur.com/Yjs9GI521:24
*** NikitaKonovalov has quit IRC21:24
*** DinaBelova has quit IRC21:24
mkj_wkZZelle_: who is core for git-review? :)21:24
*** SergeyLukjanov has quit IRC21:24
mikaldansmith: is that a picture of a dead clown?21:24
ZZelle_mkj_wk, jeblair, clarkb, fungi, jhesketh at least21:24
*** baoli has quit IRC21:24
mikaldansmith: yeah, the next line below "5 of 5 releases" for me is a "create milestone" link21:24
dansmithmikal: wat?21:24
mikaldansmith: sorry, imgur scares me21:25
jeblairdansmith, mikal: wow, i can't find the permission difference between the two of you in this context.  you are both members of nova-drivers and not members of openstack-admins...21:26
mikaljeblair: and I an owner of the project or something?21:26
mikalHow do I check that?21:26
*** david-lyle has quit IRC21:27
jeblairmikal: afaik, only the driver and maintainer matter (and they are nova-drivers and openstack-admins respectively, according to https://launchpad.net/python-novaclient )21:27
*** ayoung-lunx has quit IRC21:27
*** david-lyle has joined #openstack-infra21:28
*** annegentle has joined #openstack-infra21:29
jeblairmikal, dansmith: i have made nova-drivers the maintainer of python-novaclient; try it out now21:29
mikalOk, sounds like its time for a confused email to ttx on this21:29
dansmithjeblair: yep, that works now21:29
*** carl_baldwin has joined #openstack-infra21:29
mikalHuh21:29
jeblairmikal: still have the button?21:29
dansmithI have create milestone and release21:29
mikalI still have the thing21:29
dansmithso, can I create the .22 milestone to test?21:30
dansmithmriedem: ?21:30
mriedem2.22.021:30
jheskethMorning21:30
dansmithmikal: cool?21:30
mikalOk, I have to go to a 1:1 with my boss, so reply latency will increase for a bit21:30
mikaldansmith: i am cool with you making .2221:30
jeblairmikal: okay.  i think the solution is fine (several other projects have this set).  i'm weirded out by the permissions difference we saw, but i'm also okay with "<shrug> launchpad...storyboard..." as an answer to that for now.21:30
dansmithmriedem: and what for the release date.. today?21:31
mriedemyeah21:31
*** dizquierdo has joined #openstack-infra21:31
dansmithmriedem: done, look right?21:32
mriedemdansmith: yes, beautiful21:32
dansmithI even put the time of your mail for release time/date21:32
dansmithcool21:32
dansmithjeblair: thanks!21:32
mriedemi like that21:32
clarkbhudson.plugins.sshslaves.SSHLauncher.afterDisconnect(SSHLauncher.java:1063) is where all the jenkins trheads are blocking21:34
clarkbI don't think jenkins is properly cleaning up after nodes that have disconnected and been deleted21:34
*** david-lyle has quit IRC21:35
*** eharney has joined #openstack-infra21:36
zarojamespage: ping21:36
jamespagezaro, hello21:36
*** bswartz has joined #openstack-infra21:37
zarojamespage: would it be possible for you to update LP python-jenkins page and remove bzr references?21:37
zarojamespage: people think the project is dead when they look at the history.21:37
pleia2clarkb: I think we need some more docs around how we run puppet on the backups servers, I think you must have run it on the old one to get my user added, but it doesn't have a cert on the puppetmaster (there is a revoked one that I created for the new one, but I revoked it so it doesn't auto-update)21:38
jamespagezaro, I thought I had21:38
* jamespage looks21:38
pleia2our docs just say "run it manually"21:38
clarkbhttps://github.com/jenkinsci/ssh-slaves-plugin/blob/ssh-slaves-1.5/src/main/java/hudson/plugins/sshslaves/SSHLauncher.java#L1063 <- thats the line that is blocking21:38
clarkbpleia2: I did not run it on the old one to make your user, I adduser'd you21:39
pleia2clarkb: ah :)21:39
zaroclarkb: its only offlined, not disconnected correct?21:39
clarkbzaro: I do not know yet, there are ~10k threads in that state21:39
*** dizquierdo has quit IRC21:39
clarkbzaro: and they don't say what node they are associated to21:39
*** johnthetubaguy is now known as zz_johnthetubagu21:39
jamespagezaro, hmm - not sure I can get rid of all that history21:40
clarkbzaro: though I am pretty sure not all 10k belong to offlined nodes21:40
clarkbpleia2: I am not sure how we want to handle that, I think ansible may need a blacklist of nodes21:40
clarkbmordred: ^21:40
*** med_ has joined #openstack-infra21:40
jamespagezaro, I could setup an auto-import of https://github.com/stackforge/python-jenkins into lp:python-jenkins but that might be more confusing21:40
*** med_ has quit IRC21:41
*** med_ has joined #openstack-infra21:41
zarojamespage: yeah, that probably wouldn't be any better.21:41
zarojamespage: also just heads up, reference to http://git.openstack.org/cgit/stackforge/python-jenkins would be better21:41
*** kgiusti has quit IRC21:41
pleia2clarkb: ok, we need to build a second node anyway (on hp), so let's try the new puppet change when I make that change and figure out how to handle "only run puppet manually" separately. Going to use my apt-get install for the moment here21:42
*** ghostpl_ has quit IRC21:43
clarkbpleia2: sounds good. I think likely what we want is to have a cert on the puppetmaster that ansible ignores21:43
clarkbthen use puppet agent --test to run puppet when we want to21:43
* pleia2 nods21:43
*** ChuckC has quit IRC21:44
jeblairclarkb, pleia2: yeah, and also probably no root key for ansible on it...21:44
jeblairthat could be harder to put into today's puppet21:44
jeblair(things have changed radically since the last time this was run; "run puppet manually" was sufficient documentation at the time :)21:45
pleia2:)21:45
jeblairthis idea might be untenable, but i'd like to limp along until after mordred's attempt to remove the puppetmaster is complete before we make any decisions; i think things are still too much in flux to invest too much in this right now.21:46
*** e0ne is now known as e0ne_21:46
*** achanda has joined #openstack-infra21:46
pleia2wfm21:46
*** e0ne_ is now known as e0ne21:48
openstackgerritChris Jones proposed openstack-infra/project-config: Add config-drive element  https://review.openstack.org/15413221:48
clarkbzaro: https://github.com/jenkinsci/ssh-slaves-plugin/commit/b5f26ae3c685496ba942a7c18fc9659167293e43#diff-80fc627a638d10d5056a52e20a13d8cbR1208 I think that may fix it21:48
clarkbzaro: there is a bit more state checking before getting the node21:49
clarkbzaro: what do you think?21:49
zarolooking21:49
Ngmordred: ^^ only tested with local VMs so far, but it seems to work. How have you been building the images to test in RAX?21:50
*** chlong has quit IRC21:51
clarkbzaro: we should be able to update the bundled plugins right? we don't have to use the bundled versions. I will likely update that plugin to 1.921:51
*** julim has quit IRC21:51
*** sarob has joined #openstack-infra21:52
*** achanda has quit IRC21:52
openstackgerritMichael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting  https://review.openstack.org/15887721:53
openstackgerritMichael Johnson proposed openstack-infra/git-review: Add ability to run non-interactive commands  https://review.openstack.org/16052421:53
mkj_wkclarkb, jhesketh, jeblair: followed ZZelle's review and broke up my patch. I think it addresses all his concerns now. Will any of you have a chance to look those patches over?21:54
* mkj_wk noticed the other day that fungi is on vacation ☺21:54
bookwari need to test my gerrit configuration, especially the trigger on merge event, can i create branches and merge things in sandbox repository? or is it for testing commits only?21:55
zaroclarkb: yes, i believe so.21:55
*** bswartz has quit IRC21:55
anteayabookwar: we have a sandbox for testing ci systems21:56
zaroclarkb: that looks like it may help.21:56
mordredNg: there are scripts in that repo21:56
*** sarob has quit IRC21:56
anteayahttp://git.openstack.org/cgit/openstack-dev/ci-sandbox/21:56
*** _afezekas|pub has joined #openstack-infra21:56
anteayabookwar: now that is for testing ci systems21:57
mordredNg: in the tools dir - that show what needs to happen for rackspace image conversion and extra parameters and stuff21:57
anteayabut if you area just testing stuff as a gerrit user, you can use the regular sandbox21:57
anteayabookwar: try to do what you need to do and let us know if you hit an obstacle21:57
sigmavirus24Anyone here familiar with Ant glob patterns (used in the publishers part of macros)?21:57
*** sarob has joined #openstack-infra21:57
anteayaI don't know if you can create branches or not in it21:57
zaroclarkb: i'm just wondering if offline is a ssh  disconnect21:58
bookwaranteaya: ok, i'll check, thanks21:58
anteayabookwar: np21:58
clarkbzaro: I don't think so, I think thats what happens when either the node goes away under jenkins or you properly delete a node21:58
*** penguinRaider has quit IRC21:59
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Added class and view variable for state transitions.  https://review.openstack.org/16052621:59
*** markvan_ has joined #openstack-infra22:00
Ngmordred: ta22:00
*** yamamoto has quit IRC22:00
zaroclarkb: so that was my question initially.  are we setting the node to offline or disconnect after using it?22:00
openstackgerritClark Boylan proposed openstack-infra/zuul: Set __gearman_mananger on builds at launch time  https://review.openstack.org/16005022:00
clarkbzaro: disconnect22:01
clarkbzaro: we set it to offline before we use it22:01
*** markvan__ has quit IRC22:01
clarkbthen when we are done with it nodepool deletes the node which should disconnect it22:01
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Added class and view variable for state transitions.  https://review.openstack.org/16052622:01
*** cebruns has quit IRC22:01
zaroclarkb: cool, then that is a likely fix.22:02
*** mrmartin has quit IRC22:02
clarkbok I will have to install it on jenkins-dev and give it a quick test22:03
*** otter768 has joined #openstack-infra22:03
*** dannywilson has quit IRC22:04
*** dustins has quit IRC22:04
*** david-lyle has joined #openstack-infra22:05
*** wenlock has quit IRC22:06
pleia2clarkb: fyi jenkins-dev is what I'm using for the bup test, in case it's a little networky-slow (only ~3G backups, hopefully done soon)22:07
clarkbpleia2: no problem and thanks22:07
*** otter768 has quit IRC22:08
*** _afezekas|pub has quit IRC22:10
*** baoli has joined #openstack-infra22:10
anteayajogo and mtreinish can you ack this patch please? makes the nova api v2.1 test voting in nova and tempest: https://review.openstack.org/#/c/159024/222:11
jogoanteaya: neat, looking forward to that landing22:15
*** Sukhdev has quit IRC22:15
*** tiswanso has quit IRC22:15
*** penguinRaider has joined #openstack-infra22:16
*** andreykurilin_ has joined #openstack-infra22:18
anteayajogo: great thanks, waiting to hear from mtreinish22:18
*** radez_g0n3 is now known as radez22:20
anteayajogo: if you are still here can you also acknowledge this patch: https://review.openstack.org/#/c/156698/22:20
anteayait adds an experimental job to nova which is for infra testing more than anything22:21
anteayawe are trying to compress our node images to one kind, currently we have two22:21
anteayajust nice to see on the review that someone in nova is aware what this test is about22:21
*** e0ne has quit IRC22:23
anteayamarun: would you be willing to make 159672 dependent on https://review.openstack.org/#/c/159900/ ?22:24
anteayajhesketh: were you waiting on something to approve https://review.openstack.org/#/c/156252/ ?22:25
anteayalike the patch it depends on?22:26
*** pc_m has quit IRC22:27
*** e0ne has joined #openstack-infra22:27
clarkbok jenkins-dev is running ssh-slaves plugin version 1.9 which includes that commit I linked to above22:27
clarkbI don't think jenkins-dev sees the volume we need to know if it fixes the problem but I will do my best to vet this newer version then I can update one of the normal jenkins masters22:28
jheskethanteaya: no reason, I think I just wasn't going to be around if it merged and failed for some reason22:29
anteayaah okay, well the parent looks okay to me, you able to merge both and watch them?22:30
*** tsg_ has joined #openstack-infra22:30
openstackgerritMerged openstack-infra/project-config: Limit manual jobs  https://review.openstack.org/15840622:31
clarkbjhesketh: did you see the concurrent uploads change went in late last week? I haven't seen any problems iwth it so far and d-g log uploads were taking ~2minutes22:31
jheskethclarkb: oh cool :-) that's good22:32
*** tsg has quit IRC22:32
anteayajhesketh: you don't want to do that?22:33
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Test command should not attempt connection to Jenkins  https://review.openstack.org/15882622:33
*** andreykurilin_ has quit IRC22:33
*** stevemar has quit IRC22:33
*** andreykurilin_ has joined #openstack-infra22:34
*** e0ne has quit IRC22:34
mtreinishanteaya: looking now22:34
jheskethanteaya: do what sorry?22:34
anteayajhesketh: approve the two uuid logging patches, one to nodepool and one to project-config22:36
anteayamtreinish: thanks22:36
*** alazarev has joined #openstack-infra22:36
*** dmitryme has joined #openstack-infra22:36
*** NikitaKonovalov has joined #openstack-infra22:37
*** ruhe has joined #openstack-infra22:37
mtreinishanteaya: so I'm fine with it, but I have questions because I'm not really sure the strategy around gating v2.1 has been thought out22:37
openstackgerritMerged openstack-infra/system-config: Update docs for puppet 3  https://review.openstack.org/15627222:37
*** DinaBelova has joined #openstack-infra22:37
*** SergeyLukjanov has joined #openstack-infra22:37
anteayamtreinish: good, do ask your questions22:38
mtreinishanteaya: I did :)22:38
jheskethanteaya: I can do that :-)22:38
anteayamtreinish: ah cool22:38
anteayamtreinish: thank you22:38
clarkbok ssh-slaves 1.9 appears to be working on jenkins-dev22:38
anteayajhesketh: awesome, trying to dig out some of the moldier patches22:38
*** e0ne has joined #openstack-infra22:38
clarkbI have 04 in shutdown mode, any opposition to me updating this plugin there and putting it back into service?22:38
*** mtanino has quit IRC22:39
*** marun has quit IRC22:39
*** che-arne|2 has quit IRC22:39
*** patrickeast has quit IRC22:39
anteayathis plugin? sorry I have no context22:40
*** esker has joined #openstack-infra22:40
*** ldnunes has quit IRC22:40
clarkbthe ssh-slaves plugin, was talking to zaro about it earlier22:40
anteayaah22:40
*** alexpilotti has quit IRC22:41
clarkbthe current version we run seems to get stuck on a thing that was updated in the latest version22:41
clarkbmy reading of the comments are that this is a likely fixfor our issue22:41
sigmavirus24Before I go reverse-engineering things, does anyone know, off the top of their head, what directory the publisher macros run from?22:41
jheskethanteaya: so I pressed +A and then unpressed it because we'll need to make sure the uuid part for nodepool merges and is applied (nodepool will probably need restarting) first22:41
anteayaI'm for updating on one jenkins and seeing what happens22:41
anteayado you have a notes file somewhere on teh server to let others know you did so, if they miss it in backscroll?22:41
pleia2clarkb: root@jenkins-dev:~/backup-restore-02Mar2015 <-- untarred, looks like a linux filesystem, yay22:41
clarkbsigmavirus24: from the workspace which is usually /home/jenkins/workspace/$nameofjob22:42
clarkbpleia2: woot22:42
sigmavirus24clarkb: awesome thanks!22:42
nibalizerpleia2: woot22:42
*** achanda has joined #openstack-infra22:42
anteayaor will history tell them what you did?22:42
clarkbanteaya: this channel is logged :)22:43
jogosdague: when we build devstack nodepool images, why not run devstack.sh;./unstack.sh and use that?22:43
anteayaso backscroll, okay22:43
clarkbjogo: probably so that stack.sh is self testing22:43
openstackgerritMerged openstack-infra/nodepool: Write the node UUID to /etc/nodepool/uuid  https://review.openstack.org/15933922:43
anteayajust wondering what you do on servers to communicate when a server is ina different state than what puppet tells it to do22:43
clarkbjogo: which is why we cache as many things we can then let it exercise the logic22:44
*** patrickeast__ has quit IRC22:44
*** patri875 has quit IRC22:44
*** patri928 has quit IRC22:44
*** patrickeast_ has quit IRC22:44
*** mattfarina has quit IRC22:44
jogoclarkb: well we can have a devstack only test that doesn't use an image that already ran stack unstack22:45
clarkbjogo: we could, though we are explicitly trying to reduce the number of images not increase them22:45
*** weshay has quit IRC22:46
clarkbalso wouldn't that do weird things for requiing that devstack be upgradeable?22:46
clarkbwhich it isn't currently today (not saying its impossible but I think it would require use to be able to update from one env to another sanely)22:46
jogoclarkb: hmm well from unstack22:47
*** annegentle has quit IRC22:47
*** mtanino has joined #openstack-infra22:47
jogoclarkb: I'm interested to hearing what sdague thinks22:47
*** radez is now known as radez_g0n322:47
openstackgerritAnita Kuno proposed openstack-infra/project-config: Add non-voting tempest dsvm job for Midonet  https://review.openstack.org/15986722:48
*** e0ne has quit IRC22:49
clarkbI am going to go ahead and test ssh-slaves 1.9 on jenkins04 now. I didn't hear any opposition and this issue is causing us to lose jenkins fairly frequently22:49
openstackgerritMerged openstack-infra/project-config: Limit target projects for the neutron func job  https://review.openstack.org/15595822:50
openstackgerritMerged openstack-infra/project-config: Refactor Gerrit plugin build jobs  https://review.openstack.org/15760322:50
clarkbanteaya: I will write the corresponding puppet change if it all works well22:50
*** ddieterl_ has quit IRC22:51
ianwclarkb: tty thing (160480) ... isn't that papering over that debconf is probably asking you something?22:51
*** ddieterly has joined #openstack-infra22:51
clarkbianw: yes, probably. But it does seem to work even without being able to get that input22:52
mtreinishjogo: I think there is a test in the devstack repo which does the stack unstack bit22:52
anteayaclarkb: great, I hope it does22:52
*** wuhg has joined #openstack-infra22:52
clarkbianw: also devstack is pretty good at the whole non interactive, assume yes, conf old settings. so not sure why it is happening22:52
*** tiswanso has joined #openstack-infra22:53
mtreinishjogo: the thing is I don't think an unstack isn't really going to reliably put the system in a good state for restacking and running tests22:53
*** e0ne has joined #openstack-infra22:53
jogomtreinish: but if it did, that would be a better way of building ready to go images right?22:54
jogomore things installed etc22:54
clarkbianw: I am happy to wip that change if you want me/someone to dig into it more first22:54
ianwclarkb: where's the logs / failing job?  sorry probably in scrollback somehwere ...22:54
*** tiswanso has quit IRC22:54
*** ddieterly has quit IRC22:55
clarkbianw: jenkins is actually fine with it, this came up because rcarrillocruz tried using the devstack-gate readme to locally run d-g + tempest and ran into that22:55
*** ddieterly has joined #openstack-infra22:55
clarkbianw: I was able to reproduce the behavior today on hpcloud with their current trusty image22:55
clarkb(I did not use the nodepool built image)22:55
notmynamegerrit usernames pull from launchpad? (I'm looking for the gerrit user names for a few people)22:55
*** tiswanso_ has joined #openstack-infra22:56
clarkbnotmyname: they may, but can be overridden in some cases. email is likely a good key to use instaed unless you really need the username22:56
mtreinishjogo: it might, but what exactly are you trying to optimize? It's not like a devstack build takes very long with all the caching we already do22:56
jogomordred: is this the new pbr version pbr==0.10.822:56
notmynameclarkb: ok, thanks22:56
jogoif so, root wrap is fast with it22:56
jogomtreinish: I counted 20 minutes today22:56
SpamapSjeblair: Hi, do you have an infra deck that is presentty friendly?22:56
SpamapSor does anybody for that matter?22:57
mtreinishjogo: that's over 2x what it normally takes22:57
jogomtreinish: http://logs.openstack.org/94/160494/1/check/check-tempest-dsvm-full/85e0294//logs/devstacklog.txt.gz22:57
jogoyup22:57
mtreinishjogo: so my guess it's related to building the wheelhouse, we should be caching the wheels during nodepool22:58
*** unicell has quit IRC22:58
jogoyeah me too22:58
pleia2any considerations when approving gerritbot changes? (like, how eavesdrop restarts so can't be done when a meeting is in progress/coming up)22:58
*** unicell1 has joined #openstack-infra22:58
mtreinishwe're not yet but that's a good thing to work on (hint, hint)22:58
clarkbmtreinish: ya we could build the wheelhouse at image time22:58
*** resker has joined #openstack-infra22:58
clarkbpleia2: nope gerritbot will restart itself but doesn't have much state so should be ifne22:58
pleia2great, thanks22:58
jogoyeah numpy takes a few minutes to build22:59
jogohttp://logs.openstack.org/94/160494/1/check/check-tempest-dsvm-full/85e0294//logs/devstacklog.txt.gz#_2015-03-02_21_59_00_99622:59
*** annegentle has joined #openstack-infra22:59
mordredjogo: yay! I did good job?22:59
jogomtreinish: if we stack.sh & unstack.sh we wouldn't have to build it every time22:59
clarkbok restarting jenkins04 now with new plugin version22:59
*** esker has quit IRC22:59
* jogo hands mordred a cookie for doing a good job23:00
mtreinishjogo: oh, that's your fault because nova is dumb and requires numpy as a transient dep23:00
*** tiswanso_ has quit IRC23:00
mtreinishjogo: or we could just add building the wheelhouse to the nodepool scripts23:00
mtreinishwhich is really what the fix is23:00
jogomtreinish: that works for me too23:00
*** andreykurilin_ has quit IRC23:00
anteayanotmyname: you are here, do you know what the status if for this patch? https://review.openstack.org/#/c/159943/23:01
anteayanotmyname: is it still required or have you any idea?23:01
jogomtreinish: /opt/stack/new/.wheelhouse is the dir we would have to populate which works23:02
notmynameanteaya: I have no idea23:02
ianwSpamapS : i'm giving a talk locally on infra and have a deck ... it might have something useful23:02
mtreinishjogo: it actually wouldn't be that hard. You probably just need to add a loop over the branche which checks for tools/build_wheel.sh and calls it into the cache_devstack stuff in project-config23:02
notmynameanteaya: I've seen some stuff be talked about wrt swift in icehouse (maybe something with keystone not being installed?) but I don't know any details of that23:02
jogomtreinish: 'pip wheel -r' isn't enough23:03
jogobecause that fails if any one of those fails23:03
*** amitgandhinz has quit IRC23:03
SpamapSianw: I kind of want to develop a deck in presentty, and then track in in a git repo for the project to use :)23:03
anteayanotmyname: okay thanks, once we get into the third gen of reverting I need some details23:04
*** nelsnelson has quit IRC23:04
mordredianw, SpamapS: infr has a repo of presentations we all hack on and share already that you may be intersted in ...23:04
mordredianw, SpamapS: infra has a repo of presentations we all hack on and share already that you may be intersted in ...23:04
jogomtreinish: I still think stack.sh ./unstack.sh is a more robust long term answer23:04
notmynameanteaya: looks like you know as much (and probably more) than me about it23:04
*** dkliban is now known as dkliban_afk23:04
anteayanotmyname: okey dokey, thanks23:04
SpamapSmordred: perfect. :)23:04
clarkbthe one gotcha on this is pinned plugins23:05
clarkbarg jenkins why23:05
notmynameanteaya: as always, let me know if/when I can help23:05
mtreinishjogo: tools/build_wheels.sh doesn't use -r it loops over file23:05
clarkboh well should be fine to have the ssh-slaves plugin pinned as longs as it works23:05
anteayanotmyname: will do23:05
* ianw learns that presentty is *not* just a typo of "presenter" :)23:05
jogomtreinish: oh perfect23:06
*** bknudson has quit IRC23:08
*** dannywilson has joined #openstack-infra23:08
clarkbhrm we don't currently manage the jenkins bundled plugins with puppet23:09
openstackgerritStefano Maffulli 'reed' proposed openstack-infra/jeepyb: Put the link to developer infra manual  https://review.openstack.org/16054723:09
clarkbI suppose if this works out I should figure out how to do that23:09
*** dannywilson has quit IRC23:09
*** dannywilson has joined #openstack-infra23:09
openstackgerritMerged openstack-infra/project-config: Add cloudkitty projetcs to gerritbot configuration  https://review.openstack.org/16025623:09
*** cebruns has joined #openstack-infra23:11
SpamapSmordred: hm the overview one is so nice as-is.. I can't inflict presentty on that. ;)23:13
*** sarob has quit IRC23:14
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Build wheel house during nodepool devstack caching  https://review.openstack.org/16054823:15
jogomtreinish: damn, I was writing that too23:16
mtreinishjogo: ^^^ untested, but I think you can roll with it23:16
mtreinishjogo: heh, I win then :)23:16
mtreinishdtroyer: ^^^ you might be interested in that too23:16
jogomtreinish: I'll move over to doing the testing instead23:17
mtreinishjogo: what testing?23:17
jogomtreinish: if tools/build_wheels thing works without a localrc etc23:17
reedpleia2, thanks for the comment, abandoned the patch23:18
mtreinishjogo: it should: http://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/build_wheels.sh23:18
*** dimtruck is now known as zz_dimtruck23:18
pleia2reed: sure thing, fungi's patch is kind of old but I recalled in the back of my mind that he had them hanging out there for most (all?) projects23:18
mtreinishjogo: there isn't much in there but it only depends on in repo files23:19
openstackgerritClark Boylan proposed openstack-infra/puppet-jenkins: Allow for plugin pinning  https://review.openstack.org/16054923:19
jogohttp://paste.ubuntu.com/10507887/23:20
*** annegentle has quit IRC23:20
*** salv-orlando has quit IRC23:20
jogohmm commented that line out and its doing apt-get installs first23:21
jogoneat23:21
clarkbzaro: while writing 160549 I realize that I don't think the wget for installing plugins is correct23:21
clarkbzaro: any chance you can look at the url it uses and tell me if it looks wrong to you as well?23:21
*** salv-orlando has joined #openstack-infra23:21
mtreinishjogo: hmm yeah those requirements are probably for other things in those files, but they shouldn't be needed for the actual script23:22
mtreinishjogo: I would just push a devstack patch to add sane defaults for both23:22
jogowell python2.7-dev23:22
mtreinishdtroyer: ^^^ is that sane advice? :)23:22
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Support lazy resolving of include yaml tags  https://review.openstack.org/6358023:22
*** annegentle has joined #openstack-infra23:22
jogohttp://paste.ubuntu.com/10507907/23:22
mtreinishjogo: well I can run it as sudo in the patch23:23
jogomtreinish: trying that now23:23
dtroyermtreinish: I think the primary thing build_wheels.sh needs from stackrc/local.conf is the WHEELHOUSE setting, if you can work with the pip default ($HOME/.wheelhouse IIRC) you probably could remove the source stackrc line23:24
*** marun has joined #openstack-infra23:24
*** rm_work is now known as rm_work|away23:24
jogodtroyer: that would be great using default pip23:24
jogodtroyer: also thoughts on just using stack.sh ; unstack.sh  ?23:25
mtreinishdtroyer: oh, I assumed the script set WHEELHOUSE to the right dir by default, I guess I should add that to the patch23:25
dtroyerit used to, got pushback so used stackrc to set it…23:25
*** jgrimm is now known as zz_jgrimm23:26
*** yamamoto has joined #openstack-infra23:26
pleia2jeblair: exim is your thing, so I figured I'd ask first, mind if I logrotate some of the paniclogs to quiet down system emails? (last entries were from mid 2014 so far on systems I'm looking at, and logrotate keeps them around if we need them for some reason)23:26
mtreinishdtroyer: well if we call it standalone it sources stackrc so it would be set right?23:26
jeblairpleia2: please do!23:26
clarkbjogo: oh one more thing on stack + unstack during image builds, that may not work with dib as you need to have real services and things running23:26
jogoclarkb: ohhh dib isn't a full VM its a chroot or something right?23:27
dtroyermtreinish: yes23:27
mtreinishdtroyer: I'm just trying to figure out what else is needed for https://review.openstack.org/16054823:27
clarkbjogo: correct23:27
jogoclarkb: ohh never mind23:27
jogomtreinish: based on how long this is taking, I think just sudo23:28
mtreinishheh, that seems to be the cureall for the nodepool scripts :)23:28
dtroyermtreinish: that looks sane23:29
pleia2there, two fewer emails per day23:29
mtreinishclarkb: what do you think? should I add sudo preemptively to the wheelhouse patch, or try it without first?23:30
*** marun has quit IRC23:30
mtreinishdtroyer: ok cool, thanks23:30
clarkbmtreinish you shouldnt need sudo to make a wheelhouse. I must be missing something23:31
*** dannywilson has quit IRC23:31
*** prad has quit IRC23:31
*** gordc has quit IRC23:31
mtreinishclarkb: it might be needed for the outdir of where devstack expects the wheelhouse.23:32
jogoclarkb: OSError: [Errno 13] Permission denied: '/opt/stack'23:32
*** MaxV has joined #openstack-infra23:32
clarkbhrm are wheelhouses portable? I wonder if we should do like we do for the cached images23:32
*** MaxV has quit IRC23:32
jogoclarkb: not fully portable23:32
*** annegentle has quit IRC23:33
mtreinishjogo: well that could just be because your system didn't have the proper permissions on /opt, the nodepool script might create things correctly first23:33
jogobut may work for our two vendors23:33
*** dannywilson has joined #openstack-infra23:33
*** MaxV has joined #openstack-infra23:33
jogomtreinish: possibly. this is a blank image though23:33
clarkbsudo is probably fine23:33
jogoblank vm23:33
mtreinishjogo: in this case it would be, the gcc args are normally pretty portable in the pip installs23:33
*** chlong has joined #openstack-infra23:33
clarkbalso note that anything in /opt is copied anyways so not an optimization to avoid the copying23:33
jeblairtchaypo: are you still looking into 157492?  or should i look into doing my suggestion?23:33
*** dannywil_ has joined #openstack-infra23:33
clarkbmtreinish: jogo not that kind of portable, portable as in can we move them23:34
jogoI ran 'sudo tools/build_wheels.sh 2>&1 |  pastebinit'23:34
*** yamamoto has quit IRC23:34
jogoso can't even see whats going on :/23:34
clarkbmtreinish: jogo virtualenvs blow up if you move them for example23:34
mtreinishclarkb: oh, yeah that it is23:34
mtreinishI think23:34
jogoclarkb: ohh yeah wheels are self contained23:34
clarkbmtreinish: jogo since each image is distrp release specific the wheels will be built for it just fine23:34
jogoclarkb: I think23:34
mtreinishclarkb: I mean I can create the wheels and move them wherever too I jsut figured it was easier to call the devstack script23:35
tchaypojeblair: one sec23:35
clarkbbut sudoing should be fine, probably simplifies what is going on23:35
clarkbmtreinish: sure I just wanted to point out that /opt is copied anyways so don't expect this to be faster23:35
clarkbbut if there is already a script that does it that is probably worht it23:35
*** rfolco has quit IRC23:35
tchaypojeblair: I think your suggestion is the right way to proceed, I just haven't had time to dig into it yet.23:37
jogohttp://paste.ubuntu.com/10508007/23:37
*** dannywilson has quit IRC23:37
tchaypoWas travelling too much last week. Should be able to look at it this week23:37
*** melwitt has quit IRC23:37
mtreinishclarkb: sure, the optimization we're looking for here is to stop compiling wheels during the jobs. Compared to the 10+ min hit we take for that everything will be better23:37
clarkbmtreinish: jogo maybe we should back up a second, in theory all of those wheelhouse items are being built regardless23:38
jogoclarkb: would /opt/stack/.wheelhouse23:38
greghaynesOut of curiousity - how is the wheelhouse that gets built there served to the jobs?23:38
clarkbso its not a 10 minute hit23:38
*** openstackgerrit has quit IRC23:38
*** rm_work|away is now known as rm_work23:38
jogo get copied to the right location23:38
clarkbgreghaynes: PIP env vars set in stackrc23:38
*** openstackgerrit has joined #openstack-infra23:39
*** baoli has quit IRC23:39
clarkbmtreinish: jogo I am worried that either A) we are not using the wheelhouse properly if it has increased job runtime or B) something else is actually slowing devstack down23:39
jogoof /opt/stack/new/.wheelhouse23:39
clarkbjogo: no23:39
clarkbjogo: it would get copied to /opt/stack/.wheelhouse on the newly mounted filesystem under /opt23:39
jogoclarkb: yes wheelhouse *shouldn't slow things down23:39
jogoso something else may be wrong23:40
mtreinishclarkb: it's probably how devstack is checking and using the wheels23:40
jogobut this is a good win even if this isn't the regression itself23:40
clarkbjogo: or its possible that we don't use the wheelhouse so we don't save any time23:40
mtreinishclarkb: because it's calling tools/build_wheels.sh on every run23:40
clarkbjogo: so we just shifted the time sink23:40
jogoclarkb: I think we are using wheelhouse based on the logs23:40
clarkbjogo: kk23:40
jogoclarkb: actually not sure23:42
greghaynesI am surprised you dont hit the same issues we didn in tripleo when we tried using a wheelhouse in this way23:42
clarkbgreghaynes: what issue was that?23:42
greghaynesThere were several. The first one was an issue with fallback behavior when a wheel was not found in a directory index (which was a result of another bug where case insensitivity didnt work for local dir indexes)23:43
greghaynestchaypo: ^23:43
greghaynesI think you were dealing with these23:43
*** ddieterly has quit IRC23:44
greghaynesThen there was issues with transitive deps for a package that uses easy_insall23:44
clarkbI think pip fixed taht first issue23:44
dstufftyea23:44
clarkbfor the second thats not really a problem, we are trying to reduce compile times not avoid easy installs23:45
jeblairtchaypo: okay, i'm trying to get any outstanding bugfixes in and stabilize gertty for a release this week23:45
greghaynesyes, but the problem is that it would search the wheelhouse and then bail and not fall back when it couldnt install anything23:45
tchaypoI worked around  the file-not-found upstream23:45
tchaypoBy faking a 40423:45
*** mriedem is now known as mriedem_away23:46
tchaypoWhich fixed at least one case insensitivity issue23:46
clarkbgreghaynes: you mean easy_install wouldn't use pypi as it normally would?23:46
jogowe do a lot of compiling http://logs.openstack.org/94/160494/1/check/check-tempest-dsvm-full/85e0294//logs/devstacklog.txt.gz#_2015-03-02_21_59_00_99623:46
clarkbgreghaynes: aiui easy_intall shouldn't even try wheels it would just go straight to pypi23:46
tchaypojeblair: I'll see if I can prioritise that for today then23:47
jogomtreinish: so build_wheels.sh is pretty touchy23:47
*** ayoung-lunx has joined #openstack-infra23:47
jogomtreinish: it takes a separate list from venv-requirements23:47
jogodtroyer: ^23:47
openstackgerritMerged openstack-infra/project-config: Add non-voting tempest dsvm job for Midonet  https://review.openstack.org/15986723:47
jogoIMHO we should try building a wheel for all requirments23:47
clarkbjogo: if we are going to do that we should put wheels on the mirrors23:48
greghaynespypi-mirror? ;)23:48
*** annegentle has joined #openstack-infra23:48
dtroyerthe intent was to just build wheels for the stuff with C dependencies23:48
jogoclarkb: well yes and no. yes if that is possible that would be great ( dstufft ^ )23:48
jogoclarkb: and no, we should still cache pip packages locally if possible anyway23:48
greghaynesclarkb: acually, we used index-url, looks like stackrc uses find-links?23:49
dstufftit's not currently possibly with bandersnatch, though there's a devpi thing23:49
greghaynesno idea what the difference is there23:49
dstufft--index-url overwrites the default index of PyPI23:49
dstufft--extra-index-url adds an additional index23:49
dstufft--find-links is similar to --extra-index-url, except it doesn't expect a directory structure23:49
dstufft(this is confusing i know)23:49
clarkbdstufft: jogo its 100% possible with pypi-mirror23:49
greghaynesah, so I bet this is the reason then :)23:50
clarkbjogo: I disagree on putting it in the images if we can avoid it, our images are already something like 4GB uncompressed23:50
jogodtroyer: but you got the list wrong search for gcc on http://logs.openstack.org/94/160494/1/check/check-tempest-dsvm-full/85e0294//logs/devstacklog.txt.gz#_2015-03-02_21_59_00_99623:50
jogoclarkb: how much bigger would the image be ?23:51
dtroyerwhat is wrong?  I don't try to get them all, just the known problem children23:51
*** erlon has quit IRC23:51
jogoalso we could use devpi-server and do stuff in the background or something23:51
jogodtroyer: problem is devstack is taking 20 minutes to launch now23:51
jogodtroyer: and not 100% clear what the issue is23:51
*** salv-orlando has quit IRC23:51
mordredjogo: dont add devpi-server23:51
clarkbjogo: the entire mirror is 130GB, I would have to go rebuild a wheel only mirror with pypi-mirror to see what our subset of the world is23:52
clarkbprobably at least a gig23:52
jogodtroyer: 21:56:53 -> 22:14:4323:52
* jogo gets coffee23:52
clarkbthough we can avoid building wheels for all the platforms if we do it per image so maybe not that bad23:52
dtroyerjogo: you specifically said I had the list wrong…what about the list is wrong?23:52
*** Sukhdev has joined #openstack-infra23:52
*** dmsimard is now known as dmsimard_away23:52
clarkbso far jenkins04 does not seem to be leaking any threads, will likely leave it this way for a bit so that we can see it over time23:54
Sukhdevclarkb: May I ask a question?23:54
clarkbSukhdev: sre23:55
clarkbSukhdev: er sUre23:55
*** ddieterly has joined #openstack-infra23:55
*** ddieterly has quit IRC23:56
*** tsg_ has quit IRC23:56
SukhdevI set up the networking_arista project for our ML2 driver decomposition - I got all the infra pieces in place except for the lauchpad23:56
Sukhdevwondering what step am I missing to get it taken care of?23:56
*** tiswanso has joined #openstack-infra23:56
clarkbSukhdev: launchpad is all you, none of the project management in launchpad is automated23:57
anteayaSukhdev: http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad23:57
clarkbanteaya: ty23:58
Sukhdevclarkb: Thanks23:58
*** EmilienM is now known as EmilienM|afk23:58
Sukhdevanteaya: Thannks23:58
SukhdevThanks23:58
pleia2clarkb: so, is switching to this new backups server just a DNS change?23:59
*** aysyd has quit IRC23:59
clarkbpleia2: that and accepting the new ssh host key23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!