Thursday, 2015-08-06

*** olaph has quit IRC00:00
*** larainema has quit IRC00:00
*** olaph has joined #openstack-infra00:00
*** larainema has joined #openstack-infra00:01
dougwigminwang2: looking00:02
minwang2thanks!00:03
dougwigminwang2: i need to wait for all of the jobs to complete, so i can see the logs.  standby.00:04
greghaynesSpamapS: aye, was a missing firewall rule after reboot00:05
dougwigminwang2: actually, i don't.  looking at your review, you're enabling octavia for ALL lbaas jobs, not just the octavia job. you have to key off the testenv that's passed in.00:05
minwang2can you explain why00:07
minwang2so what i want to do is to check neutron lbaas v2 tempest against octavia00:07
*** esmute_ has quit IRC00:09
minwang2we are not testing octavia here, we are still testing neutron lbaas00:10
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969800:10
*** rguillebert has quit IRC00:11
*** esmute_ has joined #openstack-infra00:12
openstackgerritMerged openstack-infra/puppet-asterisk: Remove invalid comment  https://review.openstack.org/20423500:12
*** sputnik13 has quit IRC00:13
dougwigminwang2: let's switch to #openstack-lbaas00:13
minwang2ok00:13
*** gildub has joined #openstack-infra00:14
*** otter768 has joined #openstack-infra00:14
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969800:15
*** r-daneel has quit IRC00:16
*** MarkAtwo_ has joined #openstack-infra00:17
*** annegentle has quit IRC00:18
*** ashleighfarnham has quit IRC00:19
*** otter768 has quit IRC00:19
*** MarkAtwood has quit IRC00:19
*** boris-42 has quit IRC00:20
adam_gis there a central docs repo somewhere for users wishing to stand up a CI pipeline similar to ours?00:21
asselin_adam_g, http://docs.openstack.org/infra/system-config/third_party.html00:22
asselin_http://docs.openstack.org/infra/system-config/running-your-own.html00:22
*** crc32 has quit IRC00:22
adam_gasselin, second is the one im looking for,  thanks!00:23
openstackgerritMatthew Treinish proposed openstack-infra/elastic-recheck: WIP: Add support to filter results by failure test_ids  https://review.openstack.org/20971200:25
mtreinishclarkb: ^^^ it was to interesting to wait until tomorrow to try00:25
mtreinishalthough I haven't actually tried to run it or anything00:25
*** tsg has quit IRC00:33
*** baoli has joined #openstack-infra00:34
SpamapSgreghaynes: oh, what firewall rule?00:34
SpamapSgreghaynes: my bad for not saving that00:34
openstackgerritMerged openstack/os-testr: Set min pbr version in setup_requires  https://review.openstack.org/20969100:35
*** bapalm has joined #openstack-infra00:37
*** thorst has joined #openstack-infra00:37
greghaynesSpamapS: dhcp, and then no rule for http to grab ipxe firmware00:37
greghaynesSpamapS: I just -i brbm -j ACCEPT00:37
SpamapSgreghaynes: err.. careful00:37
*** thorst has quit IRC00:37
greghayneshehe00:38
SpamapSgreghaynes: IIRC, Ironic listens w/ noauth00:38
SpamapSgreghaynes: so all 10.x boxes can probably wreak havoc on ironic with that rule.00:38
greghaynesyes00:38
greghaynesI only have the one host enrolled though00:39
greghaynesI did mention that you would like that rule though ;)00:39
greghaynesSpamapS: Oh, I think we should try and give crinkle access to these hosts?00:39
*** bapalm has quit IRC00:39
greghaynesSpamapS: I showed her what all is where00:40
*** gyee has quit IRC00:41
*** btully has quit IRC00:41
openstackgerritMorgan Fainberg proposed openstack/requirements: Cap keystoneclient at <2.0.0  https://review.openstack.org/20971500:42
*** camunoz is now known as camunoz_away00:42
*** tsekiyam_ has joined #openstack-infra00:45
*** heyongli has joined #openstack-infra00:46
*** mtanino has quit IRC00:46
*** tsekiyama has quit IRC00:48
*** tiswanso has joined #openstack-infra00:48
*** baoli has quit IRC00:49
*** tsekiyam_ has quit IRC00:50
*** ivar-laz_ has joined #openstack-infra00:51
openstackgerritMerged openstack-infra/project-config: networking-odl: Enable python3 job  https://review.openstack.org/20921200:52
*** ivar-lazzaro has quit IRC00:55
*** ivar-laz_ has quit IRC00:56
openstackgerritMerged openstack-infra/zuul: Fix bug with shared jobs in reconfiguration  https://review.openstack.org/20968600:56
*** bcourt has quit IRC00:56
*** minwang2 has quit IRC01:00
*** tiswanso has quit IRC01:01
*** abitha has quit IRC01:02
openstackgerritVipul Sabhaya proposed openstack/requirements: Add cue and python-cueclient to projects.txt  https://review.openstack.org/15801301:03
*** baoli has joined #openstack-infra01:06
*** puranamr has joined #openstack-infra01:07
*** ybathia has quit IRC01:09
*** tiswanso has joined #openstack-infra01:10
*** tiswanso has quit IRC01:13
*** tiswanso has joined #openstack-infra01:14
*** annegentle has joined #openstack-infra01:14
*** Sukhdev has quit IRC01:15
*** gildub has quit IRC01:19
ianwgreghaynes: what's the situation with the long host names?  is that fixed yet?01:20
ianw(well it's not fixed, but is it supposed to be :)01:21
*** dims_ has joined #openstack-infra01:21
greghaynesianw: We chabged the nodepool hostname template IIRC01:21
ianwgreghaynes: http://logs.openstack.org/33/209633/2/check/gate-tempest-dsvm-centos7/b1c33cc//logs/devstacklog.txt.gz01:22
ianwit causes issues for the centos job because apache tries to make a certficate, uses the hostname and it's too long, apache fails to install, devstakc is unhappy01:22
*** dims has quit IRC01:22
clarkbyes it should be fixed we dropped the slave.openstack.org01:23
clarkbwhich gained us that many extra characters-101:23
greghaynesianw: I wonder if its getting fqdn elsewhere01:23
ianwecho "devstack-centos7-1438784108.template.openstack.org.novalocal" | wc -c01:24
ianw6101:24
ianwmaybe that's too big, let me find the bug01:24
greghaynesyes, where is it getting that domain from01:24
greghaynesbecause it shouldnt be that anymore01:24
clarkbthats image build01:24
greghaynesah01:25
clarkbwhich is different so maybe we need to fix that too?01:25
greghaynesso I thought we fixed that in glean...01:25
*** dkalleg has quit IRC01:26
*** alexsyip has quit IRC01:26
*** sdake has joined #openstack-infra01:27
greghaynesalthough it shouldnt matter if the hostname isnt that01:27
greghaynessuch confusion01:27
greghayneshttp://git.openstack.org/cgit/openstack-infra/glean/commit/?id=e6e5322b884bc587364f6f025219122723d7545f01:29
greghaynesyea, so that doubly shouldnt be happening on images :p01:29
*** sdake_ has joined #openstack-infra01:30
*** MarkAtwo_ has quit IRC01:32
StevenKclarkb: Sorry, post-conference recovery. You/we probably need to work out a time that works with myself, camunoz_away and Daisy, and then approve that change while we're all watching.01:33
*** sdake has quit IRC01:34
StevenKclarkb: I'm tempted to bring up a test zanata server with current HEAD, then apply that change, re-run puppet and see what happens.01:34
clarkbya going to try hard to hit todays meeting01:34
clarkb1am local time but will try01:34
StevenKJust ask the twins really nicely to wake you? :-P01:35
clarkbexactly01:35
*** laughterwym has joined #openstack-infra01:36
StevenKclarkb: Maybe post to the mailing list?01:36
jrollyou have to ask for babies to wake you?01:36
*** sharikapongubala has quit IRC01:37
*** warpc__ has quit IRC01:38
greghaynesits all about the timing ;)01:40
*** yamamoto has joined #openstack-infra01:46
*** yamamoto has quit IRC01:47
*** yamamoto has joined #openstack-infra01:47
*** yamamoto has quit IRC01:49
*** jogo has quit IRC01:49
*** esmute_ has quit IRC01:50
*** jogo has joined #openstack-infra01:50
*** unicell has quit IRC01:52
*** unicell has joined #openstack-infra01:53
*** skylerberg has quit IRC01:53
*** atuvenie has joined #openstack-infra01:55
*** gildub has joined #openstack-infra01:55
*** spzala has quit IRC01:57
*** unicell has quit IRC01:58
*** atuvenie_ has quit IRC01:59
*** achanda has joined #openstack-infra02:00
openstackgerritMerged openstack/diskimage-builder: Add element to disable cloud-init resizefs  https://review.openstack.org/20426302:01
*** dims has joined #openstack-infra02:03
*** dims_ has quit IRC02:05
*** dims_ has joined #openstack-infra02:06
*** dims has quit IRC02:09
*** btully has joined #openstack-infra02:09
*** sdake_ has quit IRC02:09
*** maxandersen has quit IRC02:10
*** markvoelker has quit IRC02:11
*** baoli has quit IRC02:12
*** markvoelker has joined #openstack-infra02:12
*** darrenc is now known as darrenc_afk02:12
*** enikanorov has joined #openstack-infra02:12
*** baoli has joined #openstack-infra02:12
*** bharath has quit IRC02:13
*** btully has quit IRC02:13
*** chen12 has joined #openstack-infra02:15
*** enikanorov_ has quit IRC02:15
*** otter768 has joined #openstack-infra02:15
chen12hello, can anyone review  https://review.openstack.org/#/c/207050/  &  https://review.openstack.org/#/c/209728/02:15
*** angdraug has quit IRC02:17
*** esmute_ has joined #openstack-infra02:17
*** unicell has joined #openstack-infra02:18
*** puranamr has quit IRC02:19
*** BharatK has quit IRC02:19
*** otter768 has quit IRC02:19
*** dannywilson has quit IRC02:20
*** sabeen1 has quit IRC02:21
*** unicell1 has joined #openstack-infra02:22
*** dims_ has quit IRC02:22
*** unicell has quit IRC02:22
*** tjones has joined #openstack-infra02:26
*** tsg has joined #openstack-infra02:31
*** claudiub has quit IRC02:34
*** tsg has quit IRC02:36
*** yamamoto has joined #openstack-infra02:39
*** puranamr has joined #openstack-infra02:40
openstackgerritYanyan Hu proposed openstack-infra/project-config: Add functional test jobs for Senlin project  https://review.openstack.org/20973602:44
*** aukhan has joined #openstack-infra02:44
*** BobBall_AWOL is now known as BobBall02:50
*** Somay has joined #openstack-infra02:50
*** stevemar has joined #openstack-infra02:50
openstackgerritMerged openstack-infra/puppet-bugdaystats: Remove puppet-lint-numericvariable gem  https://review.openstack.org/20435002:50
openstackgerritMatthew Treinish proposed openstack-infra/elastic-recheck: WIP: Add support to filter results by failure test_ids  https://review.openstack.org/20971202:55
*** rfolco has quit IRC02:56
openstackgerritMerged openstack-infra/puppet-bup: Remove puppet-lint-numericvariable gem  https://review.openstack.org/20435102:59
*** krtaylor has quit IRC02:59
openstackgerritSpencer Krum proposed openstack-infra/project-config: Don't run any tests on puppet-dashboard  https://review.openstack.org/20974003:01
openstackgerritSpencer Krum proposed openstack-infra/project-config: Don't run any tests on puppet-dashboard  https://review.openstack.org/20974003:02
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:03
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875103:03
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:03
*** markvoelker has quit IRC03:04
*** stevemar has quit IRC03:09
*** stevemar has joined #openstack-infra03:09
*** dingyichen has quit IRC03:10
*** dingyichen has joined #openstack-infra03:11
*** krtaylor has joined #openstack-infra03:12
*** puranamr has quit IRC03:12
*** tjones has quit IRC03:13
*** stevemar has quit IRC03:13
*** bharath has joined #openstack-infra03:13
*** fedexo has joined #openstack-infra03:15
openstackgerritMerged openstack-infra/puppet-cgit: Add better defaults for clone-prefix  https://review.openstack.org/20424503:18
*** bharath has quit IRC03:18
*** minwang2 has joined #openstack-infra03:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:23
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:23
*** harlowja_at_home has joined #openstack-infra03:24
*** dmorita has quit IRC03:25
*** dmorita has joined #openstack-infra03:25
*** baoli has quit IRC03:26
*** doug-fish has quit IRC03:26
*** doug-fish has joined #openstack-infra03:26
*** fedexo has quit IRC03:26
*** boris-42 has joined #openstack-infra03:27
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:27
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875103:27
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961703:27
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:27
*** hemna has joined #openstack-infra03:30
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:30
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875103:30
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961703:30
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:30
openstackgerritMerged openstack-infra/puppet-cgit: Ensure cgit dir is created after parent directory  https://review.openstack.org/20386703:32
*** tiswanso has quit IRC03:33
openstackgerritMerged openstack-infra/puppet-elasticsearch: Add Gemfile and puppet 4 checks  https://review.openstack.org/20388603:33
*** puranamr has joined #openstack-infra03:34
*** hemna has quit IRC03:34
*** bpokorny has quit IRC03:34
openstackgerritMerged openstack-infra/puppet-etherpad_lite: Fix warning about httpd module  https://review.openstack.org/20276103:34
*** marzif__ has joined #openstack-infra03:35
*** darrenc_afk is now known as darrenc03:36
*** dmorita has quit IRC03:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875103:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961703:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:42
*** achanda has quit IRC03:44
openstackgerritMerged openstack-infra/puppet-drupal: Add Gemfile and puppet 4 checks  https://review.openstack.org/20388103:45
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:46
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:48
nibalizerHey so I just fat-finger +a'd 20664803:49
nibalizerdoes leaving -2 on it mean that it won't land or is more action needed?03:49
nibalizerclarkb: jhesketh ?03:50
*** fedexo has joined #openstack-infra03:51
jheskethnibalizer: gerrit should reject the submit from zuul because its pre-reqs aren't met (needs verified and needs workflow)03:54
jheskethhowever, we'll find out in a minute ;-)03:54
*** pal has joined #openstack-infra03:55
clarkbcorrect -2 prevents tge merge as does breakinf any other req like removing the +a03:55
*** ayoung has quit IRC03:56
jheskethnibalizer: looks fine. zuul's left a +2, but hasn't merged it03:56
nibalizerokay03:57
nibalizerthanks03:57
*** coolsvap|away is now known as coolsvap03:58
*** fedexo has quit IRC03:59
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969803:59
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875103:59
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961703:59
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963203:59
greghaynesSpamapS: so, I got the .3 reloaded. Having some networking issues on it though. Do I need to be making an interface for tagged vlan 25 to talk via 15.x?03:59
*** Somay has quit IRC04:00
*** harlowja_at_home has quit IRC04:03
*** markvoelker has joined #openstack-infra04:04
*** fedexo has joined #openstack-infra04:06
openstackgerritMerged openstack-infra/puppet-gerritbot: Use instance variables in templates  https://review.openstack.org/20526304:06
*** markvoelker has quit IRC04:09
openstackgerritColleen Murphy proposed openstack-infra/puppet-cgit: Add Gemfile and puppet 4 checks  https://review.openstack.org/20387004:10
openstackgerritColleen Murphy proposed openstack-infra/puppet-cgit: Use instance variables in templates  https://review.openstack.org/20386904:10
*** chen12 has quit IRC04:11
*** chen12 has joined #openstack-infra04:11
*** bharath has joined #openstack-infra04:15
*** otter768 has joined #openstack-infra04:16
*** yushiro has joined #openstack-infra04:17
*** bharath has quit IRC04:19
*** otter768 has quit IRC04:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969804:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875104:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961704:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963204:21
*** puranamr has quit IRC04:22
*** dmorita has joined #openstack-infra04:24
*** stevemar has joined #openstack-infra04:24
*** tsg has joined #openstack-infra04:25
*** minwang2 has quit IRC04:25
openstackgerritIan Wienand proposed openstack-infra/project-config: Add swift to dsvm doc/unit-test skip  https://review.openstack.org/20975104:31
*** esmute_ has quit IRC04:32
mordredgreghaynes: it is my understanding that you need to tell the interface to talk on vlan 2504:35
greghayneshrm04:35
greghaynesthis is especially fun given that the host doesnt have connectivity or the vlan package04:36
*** garyk has quit IRC04:36
*** BharatK has joined #openstack-infra04:36
*** MarkAtwood has joined #openstack-infra04:37
*** pal has quit IRC04:38
greghaynesmordred: aha! you are correct04:40
*** fedexo has quit IRC04:40
*** annegentle has quit IRC04:41
*** deepakcs has joined #openstack-infra04:42
*** fedexo has joined #openstack-infra04:43
openstackgerritYushiro FURUKAWA proposed openstack-infra/infra-manual: Insert further details of the procedure  https://review.openstack.org/20975304:43
*** tsg has quit IRC04:43
*** tsg has joined #openstack-infra04:44
*** tjones has joined #openstack-infra04:44
*** yfried has quit IRC04:46
clarkbthe lack of untagged vlan bites again04:47
*** fedexo has quit IRC04:47
*** monester has joined #openstack-infra04:48
greghaynesyeaa04:48
greghaynessoo scping deb's onto the box04:49
StevenKBeen there, done that04:49
*** achanda has joined #openstack-infra04:50
StevenK"Oh great, the modules for the network card don't load. Let me build them on my laptop and copy them via USB key."04:50
*** abregman has joined #openstack-infra04:52
*** nadya has joined #openstack-infra04:54
*** btully has joined #openstack-infra04:54
*** fedexo has joined #openstack-infra04:58
*** MarkAtwood has quit IRC04:59
*** harlowja_at_home has joined #openstack-infra05:00
SpamapSgreghaynes: yes, 15.x is only on vlan2505:00
*** MarkAtwood has joined #openstack-infra05:00
*** monester has quit IRC05:04
greghaynesSpamapS: We should definitely have the vlan package in the image ;)05:05
*** minwang2 has joined #openstack-infra05:05
StevenKYou can't see vlan using ip?05:06
StevenKs/see/set/05:06
greghayneshrm05:06
greghaynesyou might05:06
StevenKip link add link eth0 name eth0.1234 type vlan id 123405:07
*** tjones has quit IRC05:08
*** bharath_ has joined #openstack-infra05:15
*** pal has joined #openstack-infra05:16
*** bharath_ has quit IRC05:20
*** sergmelikyan has joined #openstack-infra05:21
*** jkraj has joined #openstack-infra05:21
*** camunoz_away is now known as camunoz05:23
harlowja_at_homeianw, if u interested,  http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_19_45_369 (templates of spec files getting made), http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_19_30_550 (dependencies getting built automatically into rpms...)05:25
*** achanda has quit IRC05:25
harlowja_at_home*or anyone else that cares, that log is neat imho :-P05:25
harlowja_at_homestarts from http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_16_00_58105:25
harlowja_at_homegit checkouts start @ http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_16_02_624 ...05:26
harlowja_at_homefinding dependencies in yum (to avoid building them) @ http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_17_18_583 ...05:27
harlowja_at_homeanyways05:27
harlowja_at_homeoh ya, same code builds venvs to @ http://logs.openstack.org/60/209260/2/check/gate-anvil-venv-bare-trusty/fe47818/ :-P05:28
harlowja_at_homestarting @ http://logs.openstack.org/60/209260/2/check/gate-anvil-venv-bare-trusty/fe47818/console.html#_2015-08-06_01_28_27_233 :-P05:28
*** yfried has joined #openstack-infra05:28
ianwharlowja_at_home: you do love your ascii art :)05:31
harlowja_at_homehaha05:31
*** achanda has joined #openstack-infra05:31
*** yfried_ has joined #openstack-infra05:31
harlowja_at_homeianw, the cows make it the best :-P05:31
harlowja_at_homethe table creation like @ http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_16_01_565 is actually a pretty simple recursive algorithm05:32
*** yfried has quit IRC05:32
harlowja_at_homehttps://github.com/stackforge/anvil/blob/master/anvil/pprint.py05:32
harlowja_at_homethat 'somewhat simple' code, ha05:32
harlowja_at_homebut yes, happy cows like @ http://logs.openstack.org/60/209260/2/check/gate-anvil-venv-bare-trusty/fe47818/console.html#_2015-08-06_01_50_16_206 are good cows05:33
ianwhappy to have the cows and jobs working :)05:34
harlowja_at_homeya, nearly working, the building from source rpm -> real rpm is busted, but i gotta investigate that locally05:35
harlowja_at_homehttp://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_21_38_95705:35
*** k4n0 has joined #openstack-infra05:36
*** harlowja_still_a has joined #openstack-infra05:36
harlowja_still_aarg, damn wirless05:36
harlowja_still_ahttp://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_21_37_40605:36
harlowja_still_acffi something or other05:36
harlowja_still_aprobably missing a devel package05:36
harlowja_still_ajust gotta remember which one, ha05:37
ianwhmm, do you need to copy the other log files out?05:39
harlowja_still_aprobably eventually, ha05:39
harlowja_still_aits alot of log files though, ha05:39
harlowja_still_aeach rpmbuild --> its own log file....05:39
*** harlowja_at_home has quit IRC05:39
harlowja_still_a* http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_20_22_732 (all go here)05:40
harlowja_still_abut maybe, how's that done?05:40
harlowja_still_agetting at '/home/jenkins/openstack/deps/output' would have all the goodness of what broke05:40
ianwharlowja_still_a: see the scp publishers, to copy one directory should be straight-forward05:41
harlowja_still_ak05:41
harlowja_still_alet's see here05:42
*** yamahata has quit IRC05:43
*** nadya has quit IRC05:50
*** markvoelker has joined #openstack-infra05:51
*** markvoelker has quit IRC05:56
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Add scp log extraction to anvil-jobs  https://review.openstack.org/20976305:56
harlowja_still_aianw, ok dokie05:56
harlowja_still_aalthough should that be 'static.openstack.org' ?05:57
harlowja_still_anot sure05:57
harlowja_still_afor 'site: 'tarballs.openstack.org''05:57
StevenKclarkb: Your paste from your shell just makes me think of "MORBO IS PLEASED"05:57
*** jtomasek has quit IRC05:57
ianwharlowja_still_a: yeah, tarballs is not right, that where things like image outputs from builds go05:59
harlowja_still_ahmmm05:59
harlowja_still_awherever u guys want it to go, i make it happen, lol06:00
openstackgerritYanyan Hu proposed openstack-infra/project-config: Add functional test jobs for Senlin project  https://review.openstack.org/20973606:01
*** dims has joined #openstack-infra06:03
*** fedexo has quit IRC06:05
harlowja_still_aianw, switching to static.openstack.org then i guess?06:08
*** pal has quit IRC06:08
*** nadya has joined #openstack-infra06:09
*** Yanyan has joined #openstack-infra06:09
*** vlaza has joined #openstack-infra06:09
*** dims has quit IRC06:09
harlowja_still_aor maybe logs.openstack.org06:09
harlowja_still_alots of options , haha06:09
*** isaacb has joined #openstack-infra06:12
ianwahh, ping jhesketh, he's the main log guy :)  i'll watch and learn06:13
jhesketho/06:15
jheskethharlowja_still_a, ianw: what's up?06:15
harlowja_still_ajhesketh, soooo06:15
harlowja_still_aanvil job produces rpmbuild logs placing them in a directory06:16
harlowja_still_aie http://logs.openstack.org/60/209260/2/check/gate-anvil-rpms-dsvm-devstack-centos7/c42db75/console.html#_2015-08-06_01_21_37_40706:16
harlowja_still_abutttt on failure (or success) it'd be really useful to have those logs for viewing proposes06:16
harlowja_still_aso i made https://review.openstack.org/#/c/209763/06:17
*** otter768 has joined #openstack-infra06:17
harlowja_still_abut not sure on the 'site' part06:17
harlowja_still_ajhesketh, so thats where u come in, ha06:17
harlowja_still_aie what site / special sauce needed in that review to get 'Output for build being placed in /home/jenkins/openstack/deps/output/rpmbuild-python-cliff-1.10.1-0.el7.centos.src.rpm.log' into a useful place06:18
jheskethharlowja_still_a: ah okay06:18
*** ParsectiX has joined #openstack-infra06:18
jheskethharlowja_still_a: you probably want logs.o.o06:18
harlowja_still_ak06:18
jheskethtake a look at another job uploading to logs (doesn't have to be a post job either)06:19
harlowja_still_atarget: 'tarballs/anvil/deps/output/' is ok? or something else?06:19
*** pal has joined #openstack-infra06:19
jheskethharlowja_still_a: however, a better option would be to use the zuul swift uploader06:19
jheskethbut that depends on what is in the output dir at the moment06:19
harlowja_still_ai'm ok with that to06:19
harlowja_still_ait will be 'openstack/deps/output/*' which will have a bunch of those files in it06:20
*** boris-42 has quit IRC06:20
*** otter768 has quit IRC06:21
*** e0ne has joined #openstack-infra06:21
*** dmorita has quit IRC06:22
jheskethharlowja_still_a: if they are all text then swift is the way to go06:22
jheskethharlowja_still_a: otherwise I lied and it should be static.openstack.org, something like http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n87506:23
harlowja_still_aya, all text06:23
jheskethharlowja_still_a: in that case you could use the zuul-swift-upload-logs-with-console macro06:23
harlowja_still_ahmm06:23
*** mrmartin has joined #openstack-infra06:24
*** mrunge has joined #openstack-infra06:25
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Add better log + console extraction to anvil-jobs  https://review.openstack.org/20976306:25
harlowja_still_ajhesketh, like ^ maybe?06:25
*** ashleighfarnham has joined #openstack-infra06:25
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/20882006:28
*** yfried_ is now known as yfried|afk06:28
jheskethharlowja_still_a: yep, that looks good to me :-)06:29
harlowja_still_acool06:29
harlowja_still_athx!06:29
*** yfried|afk is now known as yfried_06:29
harlowja_still_ajhesketh, that will upload on failure or success (the scp jobs have a 'copy-after-failure: true' option that that one doesn't have)06:29
harlowja_still_a(if u know)06:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove icehouse building for training-guides  https://review.openstack.org/20977606:30
jheskethharlowja_still_a: yes it should06:31
harlowja_still_acools06:31
*** nadya has quit IRC06:32
openstackgerritYushiro FURUKAWA proposed openstack-infra/infra-manual: Substitute from <projectname> to <repositoryname>  https://review.openstack.org/20977706:32
*** kushal has joined #openstack-infra06:33
*** Trozz_ has joined #openstack-infra06:34
*** ihrachyshka has joined #openstack-infra06:35
*** yfried__ has joined #openstack-infra06:35
*** yfried_ has quit IRC06:36
*** harlowja_still_a has quit IRC06:37
*** afazekas_ has joined #openstack-infra06:37
mordredgreghaynes: I have an open todo list item to add grokking vlan information from config-drive to glean06:37
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove icehouse building for training-guides  https://review.openstack.org/20977606:39
*** marzif__ has quit IRC06:39
mordredgreghaynes: since I'm at the beach and mostly useless ... if you wanted to do it - then we could pass in vlan25 at boot and whatnot06:39
*** che-arne has quit IRC06:42
*** stevemar has quit IRC06:42
*** jtomasek has joined #openstack-infra06:42
mordredgreghaynes: https://review.openstack.org/#/c/167385/1/specs/liberty/approved/metadata-service-network-info.rst contains the spec for what the metadata in config-drive wants to look like06:43
*** minwang2 has quit IRC06:44
mordredgreghaynes: unfortunately, the example decided to combine vlan and bonding, so reading it hurts the brain a little bit06:45
*** tsg has quit IRC06:45
mordredgreghaynes: but best I can tell, a vlan on a non-bonded interface would just ahve the normal phy link defined and then the vlan link with vlan_link == the phy link defined06:46
StevenKmordred: Like the example I posted with ip link add ? :-)06:51
*** scheuran has joined #openstack-infra06:52
openstackgerritMonty Taylor proposed openstack-infra/glean: Add support for reading vlan information  https://review.openstack.org/20978406:54
mordredStevenK: well, no - we want something to write teh vlan information properly into /etc/network/interfaces or the RH location when it's writing out the config files06:55
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove rally-jobs job group  https://review.openstack.org/20978606:55
mordredStevenK: I mean, we DO want it to do the equivilent to your ip link add command :)06:55
*** sergmelikyan has quit IRC06:55
mordredTheJulia, greghaynes, StevenK, SpamapS: ^^ there is a VERY much non-working patch that adds a little bit of test fixture that should show what the input data from bifrost shoudl look like06:58
openstackgerritMonty Taylor proposed openstack-infra/glean: Add support for reading vlan information  https://review.openstack.org/20978406:59
mordred(amended to link to the spec - because it took me a while to find it this time)06:59
mordredand with that - I will go do laundry06:59
openstackgerritMerged openstack-infra/project-config: Add better log + console extraction to anvil-jobs  https://review.openstack.org/20976307:00
*** e0ne has quit IRC07:00
*** yushiro has quit IRC07:01
*** ifarkas has joined #openstack-infra07:02
*** blogan has quit IRC07:03
*** blogan has joined #openstack-infra07:04
*** ashleighfarnham has quit IRC07:05
*** ildikov has quit IRC07:07
*** nadya has joined #openstack-infra07:07
*** camunoz is now known as camunoz_gone07:07
*** yushiro has joined #openstack-infra07:09
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove rally-jobs job group  https://review.openstack.org/20978607:09
*** zeih has joined #openstack-infra07:10
*** e0ne has joined #openstack-infra07:11
*** juggler_ has joined #openstack-infra07:14
openstackgerritSteve Kowalik proposed openstack-infra/project-config: Add support for pulling translations from Zanata  https://review.openstack.org/20392807:15
*** juggler has quit IRC07:17
*** yfried has joined #openstack-infra07:18
*** claudiub has joined #openstack-infra07:20
*** yfried__ has quit IRC07:21
*** markus_z has joined #openstack-infra07:24
*** ub has joined #openstack-infra07:26
*** ildikov has joined #openstack-infra07:26
*** e0ne has quit IRC07:26
*** zeih has quit IRC07:27
openstackgerritWitold Bedyk proposed openstack-infra/project-config: add gerrit groups for monasca-log  https://review.openstack.org/20843007:27
*** jschwarz has joined #openstack-infra07:27
openstackgerritMerged openstack-infra/puppet-cgit: Use instance variables in templates  https://review.openstack.org/20386907:28
*** btully has quit IRC07:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove rally-jobs job group  https://review.openstack.org/20978607:31
*** witek has joined #openstack-infra07:35
*** b10n1k has quit IRC07:37
openstackgerritlyanchih proposed openstack/requirements: Bump oslo_versionedobjects to 6.0  https://review.openstack.org/20979707:37
*** garyk has joined #openstack-infra07:40
*** jcoufal has joined #openstack-infra07:45
*** bradm has quit IRC07:48
*** bradm has joined #openstack-infra07:48
*** bookwar has quit IRC07:49
*** bookwar has joined #openstack-infra07:49
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove rally-jobs job group  https://review.openstack.org/20978607:49
*** jlanoux has joined #openstack-infra07:50
*** zeih has joined #openstack-infra07:52
*** markvoelker has joined #openstack-infra07:52
*** HeOS has quit IRC07:53
*** sdake has joined #openstack-infra07:54
*** dtantsur is now known as dtantsur|brb07:56
*** markvoelker has quit IRC07:56
*** fhubik has joined #openstack-infra07:57
*** laughterwym has quit IRC07:58
*** enikanorov has quit IRC07:59
*** zeih has quit IRC07:59
*** kiran-r has joined #openstack-infra08:10
*** arxcruz has joined #openstack-infra08:10
*** alexandrov has joined #openstack-infra08:11
*** gildub has quit IRC08:12
openstackgerritSteve Kowalik proposed openstack-infra/project-config: Add support for pulling translations from Zanata  https://review.openstack.org/20392808:13
*** cdent has joined #openstack-infra08:13
*** achanda has quit IRC08:14
*** jistr has joined #openstack-infra08:14
*** achanda has joined #openstack-infra08:18
*** otter768 has joined #openstack-infra08:18
*** larainema has quit IRC08:18
*** larainema has joined #openstack-infra08:18
*** heyongli has quit IRC08:19
*** heyongli has joined #openstack-infra08:19
jamielennoxcan i get build artifacts from jenkins jobs that have already failed without waiting for all the other build jobs to finish so that zuul picks it all up?08:21
*** e0ne has joined #openstack-infra08:22
*** che-arne has joined #openstack-infra08:22
*** dmellado has joined #openstack-infra08:22
*** otter768 has quit IRC08:23
*** Ala has joined #openstack-infra08:23
*** sshnaidm has joined #openstack-infra08:25
eantyshevjamielennox: I reckon zuul writes preliminary results in debug log, you can search it08:26
jamielennoxthat just gives me console though right08:26
jamielennoxi really need like devstacklog.txt08:26
jamielennoxand i don't want ot wait 20 minutes for the rest of the job to finish08:27
jamielennoxactually i think i need g-api.txt08:28
eantyshevjust debugged something in zuul recently, and seen reports like that: http://paste.openstack.org/show/411087/08:29
*** larainema has quit IRC08:29
*** achanda has quit IRC08:29
*** larainema has joined #openstack-infra08:30
eantyshevprobably, you can find logs link for completed builds there08:30
*** fhubik is now known as fhubik_afk08:30
openstackgerritMerged openstack-infra/project-config: Adding swift3 to list of projects for particular ec2 job  https://review.openstack.org/20853508:31
eantyshevif you are about seing devstacklog.txt of a running job, my way is to ssh into the slave and read it08:32
*** mfmcdonagh has joined #openstack-infra08:36
*** mcdonagm has joined #openstack-infra08:36
*** aviau has quit IRC08:37
*** pal has quit IRC08:38
*** gaurangt has joined #openstack-infra08:38
*** HeOS has joined #openstack-infra08:38
*** pal has joined #openstack-infra08:39
*** fhubik_afk is now known as fhubik08:40
*** achanda has joined #openstack-infra08:40
*** e0ne has quit IRC08:41
*** ihrachyshka has quit IRC08:43
*** aviau has joined #openstack-infra08:43
*** dmellado has quit IRC08:43
*** dmellado has joined #openstack-infra08:44
*** dingyichen has quit IRC08:46
*** flepied has quit IRC08:47
*** NithyaG has joined #openstack-infra08:48
jamielennoxyou can ssh into the jenkins slave?08:49
*** fhubik is now known as fhubik_afk08:50
*** dmellado has quit IRC08:51
*** achanda has quit IRC08:51
*** dmellado has joined #openstack-infra08:51
*** dmellado has quit IRC08:52
*** dmellado has joined #openstack-infra08:52
*** marzif__ has joined #openstack-infra08:54
*** dmellado has quit IRC08:54
*** dmellado has joined #openstack-infra08:55
*** fhubik_afk is now known as fhubik08:55
*** fhubik is now known as fhubik_afk08:55
*** flepied has joined #openstack-infra08:55
*** dmellado has quit IRC08:57
*** yfried is now known as yfried|afk08:58
*** dmellado has joined #openstack-infra08:58
*** dmellado has quit IRC08:58
*** dmellado has joined #openstack-infra08:58
*** e0ne has joined #openstack-infra08:58
*** sobersabre has joined #openstack-infra09:01
*** jkraj has quit IRC09:04
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: Added new SF repository for Fuel Ironic plugin integration  https://review.openstack.org/20911209:05
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: Added new SF repository for Fuel NSXV plugin integration  https://review.openstack.org/20953509:05
*** ashleighfarnham has joined #openstack-infra09:06
*** Somay has joined #openstack-infra09:08
*** fhubik_afk is now known as fhubik09:08
*** ashleighfarnham has quit IRC09:10
*** zeih has joined #openstack-infra09:15
*** btully has joined #openstack-infra09:17
*** yfried|afk is now known as yfried09:18
*** monester has joined #openstack-infra09:19
*** VivCheri has joined #openstack-infra09:20
*** Somay has quit IRC09:21
mescanefhello, please review https://review.openstack.org/#/c/209535/ and https://review.openstack.org/#/c/209112/09:21
mescanef:-)09:21
*** btully has quit IRC09:22
*** MarkAtwood has quit IRC09:22
jokke_can someone boot that uvirtbot off from #openstack-meeting?09:24
*** yushiro has quit IRC09:25
*** yamamoto has quit IRC09:26
eantyshevjamielennox, I am talking about my third-party CI09:28
*** yfried is now known as yfried|afk09:28
*** Somay has joined #openstack-infra09:33
*** jkraj has joined #openstack-infra09:40
*** kushal has quit IRC09:41
*** e0ne has quit IRC09:43
*** kushal has joined #openstack-infra09:43
*** yamamoto has joined #openstack-infra09:44
*** zeih has quit IRC09:46
*** deepakcs has quit IRC09:47
*** mquin has quit IRC09:47
*** aviau has quit IRC09:48
*** dims has joined #openstack-infra09:49
*** zeih has joined #openstack-infra09:49
*** aviau has joined #openstack-infra09:50
*** markvoelker has joined #openstack-infra09:53
*** yfried|afk is now known as yfried09:53
*** zeih has quit IRC09:54
*** markvoelker has quit IRC09:57
lifelessjamespage: if you could follow up on https://github.com/testing-cabal/traceback2/pull/6 that would be good09:58
*** nagyz_ has joined #openstack-infra09:58
jamespagelifeless, I'll take another look - my fix was sceptical in the first place tbh09:59
*** e0ne has joined #openstack-infra09:59
jlanouxHello. Could I have reviews on  https://review.openstack.org/#/c/208481/ please?10:01
*** nagyz has quit IRC10:01
*** nadya has quit IRC10:06
ekarlsooii gang10:10
ekarlsodoes the gate use local.conf or localrc ?10:10
*** yfried is now known as yfried|afk10:11
*** dtantsur|brb is now known as dtantsur10:15
*** btully has joined #openstack-infra10:18
*** otter768 has joined #openstack-infra10:18
openstackgerritKirill Zaitsev proposed openstack-infra/project-config: make gate-murano-dashboard-npm-run-lint voting  https://review.openstack.org/20967610:20
*** oomichi has quit IRC10:22
*** btully has quit IRC10:22
*** otter768 has quit IRC10:23
*** Yanyan has quit IRC10:28
*** fhubik is now known as fhubik_brb10:30
*** yfried|afk is now known as yfried10:31
*** pc_m has joined #openstack-infra10:31
*** mwagner_lap has quit IRC10:39
*** ihrachyshka has joined #openstack-infra10:39
*** yfried is now known as yfried|afk10:41
*** pal has quit IRC10:42
*** mquin has joined #openstack-infra10:43
*** gildub has joined #openstack-infra10:47
*** doronshai has joined #openstack-infra10:48
*** doronshai has quit IRC10:50
*** pal has joined #openstack-infra10:50
*** dims_ has joined #openstack-infra10:51
*** e0ne has quit IRC10:52
*** dims has quit IRC10:53
*** markvoelker has joined #openstack-infra10:54
*** e0ne has joined #openstack-infra10:54
*** yfried|afk is now known as yfried10:55
*** laughterwym has joined #openstack-infra10:57
*** markvoelker has quit IRC10:58
*** jkraj has quit IRC11:01
*** fhubik_brb is now known as fhubik11:04
clarkbboth11:06
*** sdake has quit IRC11:07
*** pal has quit IRC11:07
eantyshevclarkb: I have another zuul review, which many people may be interested in: https://review.openstack.org/20709411:09
*** MarkAtwood has joined #openstack-infra11:10
*** pal has joined #openstack-infra11:15
*** MarkAtwood has quit IRC11:15
*** yamamoto has quit IRC11:16
*** jkraj has joined #openstack-infra11:19
*** vinayyadhav has joined #openstack-infra11:20
*** vinayyadhav has quit IRC11:20
*** vinayyadhav has joined #openstack-infra11:22
*** spzala has joined #openstack-infra11:24
*** thorst has joined #openstack-infra11:24
openstackgerritMerged openstack-infra/project-config: Add Mistral gate that run unit tests and use postgresql  https://review.openstack.org/20796111:24
*** BharatK has quit IRC11:24
*** gordc has joined #openstack-infra11:28
openstackgerritPaul Michali proposed openstack-infra/project-config: Enable VPN DevStack plugin for neutron client  https://review.openstack.org/20988711:29
*** pc_m has quit IRC11:29
*** hdd has joined #openstack-infra11:33
*** Somay has quit IRC11:33
*** gaurangt has left #openstack-infra11:34
*** nadya has joined #openstack-infra11:35
tobiash20968611:38
*** fhubik is now known as fhubik_brb11:38
*** dprince has joined #openstack-infra11:40
*** e0ne has quit IRC11:41
*** ub has quit IRC11:41
*** fhubik_brb is now known as fhubik11:42
*** e0ne has joined #openstack-infra11:44
*** Somay has joined #openstack-infra11:47
openstackgerritJoseph Lanoux proposed openstack-infra/project-config: Add non-voting job with ssh validation  https://review.openstack.org/20848111:51
*** pal has quit IRC11:52
*** yamamoto has joined #openstack-infra11:55
*** Somay has quit IRC11:56
*** nelsnelson has quit IRC11:57
openstackgerritMerged openstack-infra/zuul: Status: Use vertical-align for progress bar instead of margin  https://review.openstack.org/17005811:58
*** Somay has joined #openstack-infra11:58
*** Somay has left #openstack-infra11:58
*** vinay_yadhav has joined #openstack-infra11:59
*** hdd has quit IRC12:00
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Reduce the size of the ironic-agent ramdisk  https://review.openstack.org/20990212:02
*** yfried_ has joined #openstack-infra12:02
*** hdd has joined #openstack-infra12:03
*** vinayyadhav has quit IRC12:03
*** yfried has quit IRC12:03
*** vinay_yadhav has quit IRC12:04
*** yfried__ has joined #openstack-infra12:05
*** yfried_ has quit IRC12:08
*** cburroughs has joined #openstack-infra12:12
*** markvoelker has joined #openstack-infra12:16
openstackgerritTobias Henkel proposed openstack-infra/zuul: Support pagination in simplequery for Gerrit >=2.9  https://review.openstack.org/20990612:17
jamielennoxis Depends-On handled for devstack patches?12:18
jamielennoxi say this because it doesn't appear so12:18
*** pc_m has joined #openstack-infra12:18
openstackgerritTobias Henkel proposed openstack-infra/zuul: Support pagination in simplequery for Gerrit >=2.9  https://review.openstack.org/20990612:19
*** otter768 has joined #openstack-infra12:19
*** unicell has joined #openstack-infra12:20
*** unicell1 has quit IRC12:21
*** mwagner_lap has joined #openstack-infra12:22
jamielennoxactually - is Depends-On: supported for client library patches12:22
jamielennoxbecause it appears to work for patches attached to the server12:22
*** dims has joined #openstack-infra12:24
*** bexelbie has joined #openstack-infra12:24
*** otter768 has quit IRC12:24
*** dims_ has quit IRC12:27
fungijamielennox: depends-on will set up a logical dependency between changes, and in integration test jobs that know how to consume zuul refs for repos being tested it will also being the depended on change in as part of the test12:29
fungidevstack-gate for example does consume zuul refs, but it depends on whether you're running a devstack-based job which is configured to consume that library from source or from release12:30
openstackgerritMerged openstack-infra/zuul: Status: Increase width of change-progress-row-left  https://review.openstack.org/17005912:30
jamielennoxfungi: so i've been trying to debug failure https://review.openstack.org/#/c/209351/12:30
fungijamielennox: the majority of devstack-based jobs are configured to use released versions of libraries so that we avoid introducing fixes and then relying on those without first releasing them12:31
*** chen12 has quit IRC12:31
jamielennoxi've got multiple depends-on trying to figure out what's going on12:31
jamielennoxthe Depends-On that is linked to glance works, the depends-on for swiftclient and glance_store do not12:31
jamielennoxfungi: depends-on would seem to imply i want to LIBS_FROM_GIT though right?12:32
jamielennoxotherwise there's no way for me to test this stuff12:32
fungiyeah, if you read the job setup logs, you'll likely see pip installing swiftclient and glance_store from pypi12:32
jamielennoxyep12:32
tobiashhello, could I have reviews on https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul+branch:master+topic:upload-pack-via-ssh,n,z please?12:32
jamielennoxso nothing i can do but wait for the library to be released12:32
jamielennox?12:32
fungijamielennox: there are some job templates, mostly only applied to the library projects themselves, which test the libraries from source12:32
fungibut yeah, ask the library release team to expedite a release if there are pending bug fixes12:33
jamielennoxfungi: will do. thanks12:34
*** baoli has joined #openstack-infra12:35
*** aysyd has joined #openstack-infra12:36
*** hdd has quit IRC12:36
*** olaph has quit IRC12:40
*** bapalm has joined #openstack-infra12:44
*** pblaho has quit IRC12:46
*** pblaho has joined #openstack-infra12:46
*** laughterwym has quit IRC12:46
*** zul has joined #openstack-infra12:46
*** laughterwym has joined #openstack-infra12:46
*** baoli_ has joined #openstack-infra12:47
*** abregman has quit IRC12:47
*** baoli has quit IRC12:49
*** hdd has joined #openstack-infra12:50
*** afazekas_ has quit IRC12:50
*** krtaylor has quit IRC12:50
*** yamamoto has quit IRC12:50
*** ogelbukh has joined #openstack-infra12:50
*** yfried has joined #openstack-infra12:50
*** mriedem_away has quit IRC12:53
*** marzif__ has quit IRC12:53
*** marzif__ has joined #openstack-infra12:54
*** pc_m has quit IRC12:54
*** yfried__ has quit IRC12:54
*** bexelbie_ has joined #openstack-infra12:55
*** che-arne has quit IRC12:55
*** bexelbie has quit IRC12:57
*** bexelbie_ is now known as bexelbie12:57
*** olaph has joined #openstack-infra12:57
*** pal has joined #openstack-infra12:57
*** che-arne has joined #openstack-infra12:58
*** bcourt has joined #openstack-infra12:59
*** MarkAtwood has joined #openstack-infra13:00
openstackgerritJan Hruban proposed openstack-infra/infra-specs: Zuul github-integration spec.  https://review.openstack.org/16363713:00
*** yamamoto has joined #openstack-infra13:02
*** rlandy has joined #openstack-infra13:02
*** krtaylor has joined #openstack-infra13:03
*** tobiash has quit IRC13:03
*** tobiash has joined #openstack-infra13:03
*** yamamoto has quit IRC13:04
*** tobiash has quit IRC13:05
*** MarkAtwood has quit IRC13:05
*** tsg has joined #openstack-infra13:05
*** pc_m has joined #openstack-infra13:06
*** ashleighfarnham has joined #openstack-infra13:07
*** hdd has quit IRC13:08
*** yamamoto has joined #openstack-infra13:09
*** yamamoto has quit IRC13:10
*** gildub has quit IRC13:10
*** jtomasek has quit IRC13:10
*** yamamoto has joined #openstack-infra13:10
*** dolphm has left #openstack-infra13:10
*** tlian has joined #openstack-infra13:11
*** kiran-r has quit IRC13:11
*** ashleighfarnham has quit IRC13:12
*** sergmelikyan has joined #openstack-infra13:15
*** che-arne has quit IRC13:15
*** hdd has joined #openstack-infra13:15
*** hdd has quit IRC13:16
*** k4n0 has quit IRC13:18
*** rfolco has joined #openstack-infra13:18
openstackgerritVictor Ryzhenkin proposed openstack-infra/project-config: Add opportunity to install muranoclient from change  https://review.openstack.org/20415913:18
*** jpeeler has quit IRC13:20
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Add module to handle welcome rsa key  https://review.openstack.org/20992913:27
openstackgerritKhai Do proposed stackforge/python-jenkins: Rework the examples  https://review.openstack.org/20759013:27
*** tiswanso has joined #openstack-infra13:27
*** tobiash has joined #openstack-infra13:27
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/puppet-zuul: Add known_hosts class  https://review.openstack.org/20993013:27
*** amitgandhinz has joined #openstack-infra13:29
*** btully has joined #openstack-infra13:29
*** ldnunes has joined #openstack-infra13:29
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/puppet-openstackci: Reuse puppet-zuul known_hosts class  https://review.openstack.org/20993213:30
*** marcusvrn has joined #openstack-infra13:30
*** mriedem has joined #openstack-infra13:31
greghaynesmordred: hah, usually being useless on a beach doesnt involve submitting patches for new features ;)13:31
*** dkranz has joined #openstack-infra13:31
greghaynesah, just the test data updaating13:32
*** mestery has quit IRC13:32
*** jpeeler has joined #openstack-infra13:33
*** jpeeler has quit IRC13:33
*** jpeeler has joined #openstack-infra13:33
*** che-arne has joined #openstack-infra13:34
*** pc_m has quit IRC13:36
*** yfried is now known as yfried|afk13:37
*** mestery has joined #openstack-infra13:38
*** alivigni has joined #openstack-infra13:38
Shrewsgreghaynes: you don't give him enough credit. mordred can be useless from _anywhere_13:38
*** alexandrov has quit IRC13:39
*** Ala has quit IRC13:39
*** ayoung has joined #openstack-infra13:39
pabelangerjeblair: cool. looking into patch now.  Awesome that we can add a unit test to expose it13:40
*** yfried|afk is now known as yfried13:40
*** serg_melikyan has joined #openstack-infra13:41
*** sergmelikyan has quit IRC13:42
*** mriedem has quit IRC13:43
*** bknudson has joined #openstack-infra13:44
*** olaph has quit IRC13:44
*** mriedem has joined #openstack-infra13:46
greghayneshaah13:46
*** jswarren has joined #openstack-infra13:46
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Allow to specify git directory to run the mirror repack  https://review.openstack.org/20993813:46
*** olaph has joined #openstack-infra13:46
openstackgerritPaul Belanger proposed openstack-infra/puppet-etherpad_lite: Add Gemfile and puppet 4 checks  https://review.openstack.org/20424213:46
*** Ala has joined #openstack-infra13:47
yolandahi pabelanger, wanted to check with you...13:47
yolandawe found some conflict between grafana dependency, camptocamp-archive13:47
yolandaand puppetcommunity-archive13:47
yolandadid you find the same?13:47
*** MarkAtwood has joined #openstack-infra13:49
pabelangeryolanda: not sure. What are you seeing?13:49
yolandapabelanger, to bfraser/grafana module has this dep13:50
yolanda"dependencies": [13:50
yolanda    {13:50
yolanda      "name": "camptocamp/archive",13:50
yolanda      "version_requirement": ">= 0.3.2 <2.0.0"13:50
yolanda    },13:50
yolandaand we also install in our modules the nanliu archive module13:50
yolandathey have the same name13:50
yolandaso it gave a conflict in our installs, if you want to bring the grafana dependency13:50
jswarrenhi gmann, I think https://review.openstack.org/#/c/208045/ is ready to go.13:51
pabelangeryolanda: Hmm, right now we arn't actually using archive inside puppet-grafana, so not sure of the dependency issue you are running into. How are you installing the module?13:51
yolandapabelanger, we were installing from modules, not from source, so that was were we discovered the problem. I guess the grafana dep is not being installed, and just ignored , when installed from source?13:52
yolandapabelanger, and this will mean that if at some point, we need to use grafana features that use archive, we can have a problem13:52
*** hexlibris has joined #openstack-infra13:52
yolandapabelanger, basically if you install the nanliu archive module, then install grafana, it cannot be installed because a module with same name already exysts13:53
*** jaypipes has quit IRC13:53
pabelangeryolanda: right, the only feature that uses it is the install_method.  I'm actually change it to use apt-get in the next patchset. So, right now we can ignore the dependency upstream13:53
pabelangeryolanda: https://review.openstack.org/#/c/209218/ is the change to install_method=repo13:54
yolandaah yes, saw that. Well, we need to be careful about it, and don't use archive features on grafana13:55
*** sigmavirus24_awa is now known as sigmavirus2413:55
*** sdake has joined #openstack-infra13:55
pabelangerright13:56
*** austin81 has joined #openstack-infra13:57
*** isaacb has quit IRC13:57
yolandapabelanger, do you think it's worth adding a note on modules.env grafana module, about it?13:57
yolandait exploded for us downstream, so it can be an issue at some point13:57
pabelangeryolanda: no preference about the comment. If you see value, I have no issues.13:58
yolandayes, i think a reminder won't hurt and will make us remember that there is a hidden conflict there13:59
*** pal has quit IRC13:59
pabelangeryolanda: however you raise a valid issue, our dependency management in system-config and openstack-infra/puppet-* kinda stinks13:59
*** hexlibris has quit IRC13:59
*** pal has joined #openstack-infra13:59
*** olaph has quit IRC14:00
yolandapabelanger, tell us about that, for us downstream is even worse :)14:00
yolandagoing to add that comment to modules.env, just as reminder14:01
*** r-daneel has joined #openstack-infra14:01
*** pc_m has joined #openstack-infra14:01
*** olaph has joined #openstack-infra14:01
*** hexlibris has joined #openstack-infra14:01
*** dkranz has quit IRC14:01
*** pc_m has quit IRC14:02
*** ParsectiX has quit IRC14:02
*** MarkAtwood has quit IRC14:03
*** fhubik has quit IRC14:03
*** pc_m has joined #openstack-infra14:03
*** zz_dimtruck is now known as dimtruck14:05
*** MarkAtwood has joined #openstack-infra14:05
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add a reminder about grafana and puppet-archive conflict  https://review.openstack.org/20994614:05
yolandapabelanger, so that's it14:05
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova segfault unit test bug 1482230  https://review.openstack.org/20994714:05
openstackbug 1482230 in OpenStack Compute (nova) "LibvirtConnTestCase.test_clean_shutdown_first_time causes subunit segfault" [Undecided,New] https://launchpad.net/bugs/148223014:05
*** tiswanso has quit IRC14:05
*** radez_g0n3 is now known as radez14:06
*** tiswanso has joined #openstack-infra14:07
*** pal has quit IRC14:08
*** MarkAtwood has quit IRC14:09
*** tsg has quit IRC14:10
*** tvanderwiel has joined #openstack-infra14:11
*** pal has joined #openstack-infra14:12
*** sdake has quit IRC14:14
openstackgerritVictor Sergeyev proposed openstack/requirements: Fix url regexp to proceed oslo-specific url's properly  https://review.openstack.org/20943214:14
*** vlaza has quit IRC14:16
*** dkranz has joined #openstack-infra14:16
*** wenlock has joined #openstack-infra14:16
pabelangerfungi: if you have a moment this morning, do you mind a review for: https://review.openstack.org/#/c/209135/ Plan to monitor the commit for puppet-jobs too14:17
*** kushal has quit IRC14:19
*** otter768 has joined #openstack-infra14:20
*** radez is now known as radez_g0n314:20
*** radez_g0n3 is now known as radez14:20
ekarlsoWhat's the appropriate way to alter tempest.conf if we need to in the gate ?14:20
*** tvanderwiel has left #openstack-infra14:21
*** pal has quit IRC14:23
*** otter768 has quit IRC14:25
*** dims has quit IRC14:25
*** dims has joined #openstack-infra14:26
*** afazekas_ has joined #openstack-infra14:28
*** bswartz has joined #openstack-infra14:29
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Move local replication directory and repack to module  https://review.openstack.org/20995414:29
openstackgerritMerged openstack-infra/elastic-recheck: Add query for nova segfault unit test bug 1482230  https://review.openstack.org/20994714:29
openstackbug 1482230 in OpenStack Compute (nova) "LibvirtConnTestCase.test_clean_shutdown_first_time causes subunit segfault" [High,Confirmed] https://launchpad.net/bugs/148223014:29
*** EricGonczer_ has joined #openstack-infra14:29
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Move local replication directory and repack to module  https://review.openstack.org/20995414:30
*** baoli_ has quit IRC14:30
*** yfried_ has joined #openstack-infra14:30
greghaynesianw: SpamapS That sfdisk patch makes me really want to break partitioning out into a separate element14:31
*** stevemar has joined #openstack-infra14:31
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Reduce the size of the ironic-agent ramdisk  https://review.openstack.org/20990214:31
greghaynesianw: SpamapS At least then we can say "automated partitioning probably sucks for you, make your own element if you care"14:31
*** Zhongjun has joined #openstack-infra14:32
greghaynesSpamapS: Also, I think the deploy image on the bastion has both cloud-init and glean on it14:32
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add a reminder about grafana and puppet-archive conflict  https://review.openstack.org/20994614:32
*** isaacb has joined #openstack-infra14:32
*** yfried has quit IRC14:33
openstackgerritMerged openstack-infra/project-config: Don't use sudo to install bundler gem for puppet jobs  https://review.openstack.org/20913514:34
*** dmorita has joined #openstack-infra14:34
*** pal has joined #openstack-infra14:34
*** pradk has joined #openstack-infra14:34
pc_mQuestion for infra... There is a commit (https://review.openstack.org/#/c/194635/) that I added a Depends-On: clause for a commit (https://review.openstack.org/#/c/209887/) in project-config, but it doesn't seem to be using the dependency. Am I missing something? I just did a recheck of 194635 wondering if maybe the cause was because 209887 had not completed test run yet?14:35
*** yfried__ has joined #openstack-infra14:35
*** isaacb has quit IRC14:35
*** minwang2 has joined #openstack-infra14:36
fungipc_m: it doesn't work that way. depends-on only gets used in integration tests and the project-config repo isn't used by the integration test jobs you're configuring there (it's used to drive automation which configures the test infrastructure itself)14:37
*** yfried_ has quit IRC14:38
fungipc_m: the net effect of your depends-on header is that 194635 won't be allowed to merge until 209887 successfully merges14:38
*** jasondot_ is now known as jasondotstar14:38
fungibut it won't allow you to see the results of 209887 before it's actually merged and applied to the infrastructure14:39
gordchi, if any devstack folks are available, can we get help on https://review.openstack.org/#/c/207949/ to unblock a feature.14:39
fungipc_m: it's one of the things we want to improve with zuul v3 (moving job definitions into the projects being tested, which in theory would allow you to self-test changes to your job configs)14:40
fungigordc: you might consider asking in #openstack-qa14:40
pc_mfungi: Thanks for the explanation!14:40
gordcfungi: cool cool. figured everyone was here :) will move there14:41
pc_mfungi: 209887 adds a needed devstack plugin enable for the job, and I was hoping to test that it was correct (looked right).14:41
fungigordc: most of them are probably also in here, but devstack patches are more on-topic in the qa channel since it's a qa-controlled repo, so in theory they're more likely to see your request14:42
openstackgerritAustin Clark proposed openstack-infra/subunit2sql: Update API docs to include alternative db layer initialization  https://review.openstack.org/20996014:42
fungipc_m: you should be able to manually test it if you have an appropriate machine to clone and run devstack-gate on14:42
*** marzif__ has quit IRC14:43
*** yamamoto has quit IRC14:43
*** afazekas_ has quit IRC14:43
fungipc_m: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n95 (hopefully that's still current anyway)14:44
*** marzif__ has joined #openstack-infra14:44
pc_mfungi: Awesome! Didn't know how to do that.14:44
*** patrickeast has joined #openstack-infra14:44
*** tsekiyama has joined #openstack-infra14:44
*** kiran-r has joined #openstack-infra14:45
*** annegentle has joined #openstack-infra14:45
fungipc_m: we do try to make sure that any jobs we run in our test infrastructure can also be similarly run by individual developers, and do our best to maintain documentation in support of that14:46
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Move local replication directory and repack to module  https://review.openstack.org/20995414:46
*** bexelbie has quit IRC14:46
*** kurtmartin has joined #openstack-infra14:47
*** olaph has quit IRC14:47
*** xyang has joined #openstack-infra14:49
*** yamamoto has joined #openstack-infra14:49
*** tqtran-afk has joined #openstack-infra14:49
mtreinishfungi: so the patch from yesterday that fixed the if check in run-docs.sh doesn't look like it's taken effect. Does it need to wait for a nodepool build or something?14:52
fungimtreinish: yeah, if it merged after image updates started, then it won't be in effect yet14:52
fungiit's a change to a script which is baked into our worker images each day14:52
openstackgerritJan Hruban proposed openstack-infra/zuul: (WIP) Add 'push' and 'tag' github webhook events.  https://review.openstack.org/19120714:52
openstackgerritJan Hruban proposed openstack-infra/zuul: (WIP) Support GitHub PR webhooks  https://review.openstack.org/16311714:52
openstackgerritJan Hruban proposed openstack-infra/zuul: (WIP) Fix job hierarchy bug.  https://review.openstack.org/19245714:52
openstackgerritJan Hruban proposed openstack-infra/zuul: (WIP) Add basic Github Zuul Reporter.  https://review.openstack.org/19131214:52
openstackgerritJan Hruban proposed openstack-infra/zuul: (WIP) Allow list values in template parameters.  https://review.openstack.org/19120814:52
*** e0ne has quit IRC14:52
mtreinishfungi: ok, thats what I figured it. I guess I'll play the waiting game some more14:54
*** kurtmartin has quit IRC14:54
fungimtreinish: do you happen to know what time it merged?14:54
mtreinish12:01 eastern: https://review.openstack.org/#/c/209305/14:55
*** dangers_away is now known as dangers14:55
fungii make that 16:01 utc, so a couple hours after image updates had started14:57
*** e0ne has joined #openstack-infra14:57
fungiftr, our image updates start at 14:14 utc daily14:58
*** mriedem has quit IRC14:58
fungiand take upwards of 8 hours to complete14:58
fungithough most will be done in the next hour or so14:58
*** r-daneel has quit IRC14:58
*** mriedem has joined #openstack-infra14:58
*** jkraj has quit IRC14:58
*** achanda has joined #openstack-infra14:59
*** tqtran-afk is now known as tqtran14:59
*** kiran-r has quit IRC14:59
*** bapalm_ has joined #openstack-infra15:00
*** MarkAtwood has joined #openstack-infra15:00
*** r-daneel has joined #openstack-infra15:01
*** bapalm has quit IRC15:01
openstackgerritMatthew Treinish proposed openstack-infra/elastic-recheck: Add support to filter results by failure test_ids  https://review.openstack.org/20971215:01
*** witek has quit IRC15:03
*** olaph has joined #openstack-infra15:04
*** jaypipes has joined #openstack-infra15:05
*** tiswanso has quit IRC15:05
*** tiswanso has joined #openstack-infra15:06
*** baoli has joined #openstack-infra15:07
*** kurtmartin has joined #openstack-infra15:08
openstackgerritMerged openstack-infra/project-config: Move multinode dvr job to neutron check list.  https://review.openstack.org/20916515:08
*** mtanino has joined #openstack-infra15:09
*** BrandonL08 has joined #openstack-infra15:09
*** achanda has quit IRC15:09
*** bpokorny has joined #openstack-infra15:10
*** zul has quit IRC15:10
*** krtaylor has quit IRC15:13
*** Ala has quit IRC15:14
*** gyee has joined #openstack-infra15:14
*** patrickeast has quit IRC15:14
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add method to add flavor extra specs  https://review.openstack.org/20998315:17
*** patrickeast has joined #openstack-infra15:18
*** angdraug has joined #openstack-infra15:18
openstackgerritMatthew Treinish proposed openstack-infra/elastic-recheck: Add suppress-notification to the slow devstack query  https://review.openstack.org/20998415:19
openstackgerritMerged openstack-infra/elastic-recheck: Track nova net dhcp failures  https://review.openstack.org/20969915:19
*** dannywilson has joined #openstack-infra15:20
*** jgriffith has quit IRC15:21
*** WyAt has quit IRC15:21
mtreinishfungi: can you do me a favor when you get a chance and check on logstash.o.o if subunit2sql-db-manage is running? I just pushed a release that contained a migration15:22
*** btran has quit IRC15:22
mtreinishI think that we set it up to background the migrations to get around the timeouts but it makes it tricky to see if they're actually running15:23
fungimtreinish: root     27236  0.3  1.7 185032 35072 ?        S    15:20   0:00 /usr/bin/python /usr/local/bin/subunit2sql-db-manage --config-file /etc/subunit2sql.conf upgrade head15:23
mtreinishok, then it's doing its thing15:23
mtreinishI guess we'll see how long it takes to add a bunch of indexes15:23
*** tonytan4ever has joined #openstack-infra15:23
mtreinishwell assuming we don't run out of space again15:23
*** sputnik13 has joined #openstack-infra15:24
*** jgriffith has joined #openstack-infra15:25
mtreinishfungi: any bets on how long: http://git.openstack.org/cgit/openstack-infra/subunit2sql/tree/subunit2sql/migrations/versions/1ff737bef438_add_indexes_on_common_search_fields.py#n30 will take? :)15:25
*** yfried has joined #openstack-infra15:25
*** jgriffith is now known as Guest5780215:25
fungii never spin the wheel without a cocktail in hand15:27
Swansonnodepool questions here?15:27
fungiSwanson: this is the best place for them, sure15:27
*** scheuran has quit IRC15:29
SwansonSimple one.  I keep getting an error resolving archive.ubuntu.com while creating a trusty image.  Any idea what config I could have screwed up?  It doesn't seem to be a problem on the host I'm doing this on.15:29
mtreinishfungi: heh, sounds like a good reason to get a cocktail then15:29
*** devvesa has joined #openstack-infra15:29
*** abitha has joined #openstack-infra15:29
*** yfried__ has quit IRC15:30
*** ihrachyshka has quit IRC15:30
fungiSwanson: snapshot-based or using diskimage-builder?15:30
*** abitha has quit IRC15:30
*** sdake has joined #openstack-infra15:31
mescanefhello, please review https://review.openstack.org/#/c/209535/ and https://review.openstack.org/#/c/209112/15:31
mescanefSergeyLukjanov: ^^ :-)15:31
*** MarkAtwood_ has joined #openstack-infra15:31
*** ashleighfarnham has joined #openstack-infra15:31
Swansonfungi: diskimage-builder.15:32
Swansonfungi: I think.  (<- should tell you the level I'm at.)15:32
*** abitha has joined #openstack-infra15:32
*** Guest55 has joined #openstack-infra15:32
fungiSwanson: okay, so sounds like dns resolution may be misconfigured inside the chroot diskimage-builder is creating15:32
*** olaph has quit IRC15:32
*** sshnaidm has quit IRC15:33
fungiSwanson: you can tell whether it's a snapshot or dib based on whether there's a base-image parameter or a diskimage parameter15:35
fungi(base-image means you're building a snapshot from a template vm, diskimage means you're using diskimage-builder locally in a chroot on the nodepool server itself)15:35
*** sputnik13 has quit IRC15:36
Swansonfungi: diskimage.  Looking at a yaml file for nodepool now.15:36
*** sabeen1 has joined #openstack-infra15:36
*** dkranz has quit IRC15:37
fungiyep, so chances are there's an issue with resolv.conf in the chroot dib has created15:37
mesteryGood morning infra folks! Any chance I can get someone to populate these gerrit groups with me so I can pass them on to the infoblox folks? https://review.openstack.org/#/admin/groups/1010,members and https://review.openstack.org/#/admin/groups/1009,members15:38
*** yfried has quit IRC15:38
*** spzala has quit IRC15:38
fungiSwanson: i _think_ we configure the images to expect to resolve through a resolver daemon listening on the loopback interface. in a chroot i believe that means it's going to try to hit the loopback interface of the nodepool server15:39
fungiSwanson: do you have a locally-running dns resolver daemon on your nodepool server?15:39
Swansonfungi:  I will in a moment if I'm not.15:40
*** kiran-r has joined #openstack-infra15:40
fungiSwanson: see if `host archive.ubuntu.com 127.0.0.1` on your nodepool server works15:40
Swansonfungi:  This probably explains the transient error I saw fly by where it couldn't resolve the dns of the nodepool server.15:40
*** mpaolino has joined #openstack-infra15:41
openstackgerritMerged openstack-infra/elastic-recheck: Add suppress-notification to the slow devstack query  https://review.openstack.org/20998415:41
Swansonfungi:  That looks to be it.15:41
fungiSwanson: yeah, due to the unreliable nature of the dns resolvers in our service providers' networks, we run an unbound daemon on all our workers and also on our long-lived servers (including our nodepool server)15:41
*** krtaylor has joined #openstack-infra15:42
fungibut that does point out an interesting interaction/dependency for our dib setup that i hadn't considered before15:42
*** jaypipes has quit IRC15:43
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Reduce the size of the ironic-agent ramdisk  https://review.openstack.org/20990215:43
fungimestery: done15:43
mesteryfungi: thank you very much!15:43
fungiyw15:43
*** tongli has joined #openstack-infra15:44
*** tiswanso has quit IRC15:44
openstackgerritClaudiu Belu proposed openstack-infra/project-config: Adds oslo.windows repository  https://review.openstack.org/20920415:45
*** sputnik13 has joined #openstack-infra15:46
*** Guest55 has quit IRC15:48
*** BrandonL18 has joined #openstack-infra15:48
jlanouxHello. May I have reviews on https://review.openstack.org/#/c/208481/ please?15:49
*** tiswanso has joined #openstack-infra15:49
markus_zI'm preparing a query for yesterday's Nova bug triage day. I use the data from http://status.openstack.org/bugday/ What's the timezone of that?15:49
*** dguerri` is now known as dguerri15:50
*** btully has quit IRC15:50
*** sputnik13 has quit IRC15:50
*** dkranz has joined #openstack-infra15:50
*** BrandonL08 has quit IRC15:51
*** jgriffith has joined #openstack-infra15:52
*** ivar-lazzaro has joined #openstack-infra15:52
*** jaypipes has joined #openstack-infra15:56
fungimarkus_z: looks like utc15:57
*** ivar-laz_ has joined #openstack-infra15:57
markus_zfungi: UTC+0?15:58
fungimarkus_z: i think so. for me at least the graph seems to be ending at almost 16:00 on the right-hand side, and it's almost 16:00 utc now15:59
*** ivar-lazzaro has quit IRC15:59
markus_zfungi: oh, stupid me, I should have seen that. Thank you!15:59
*** yamahata has joined #openstack-infra16:00
*** pal has quit IRC16:00
pabelangerfungi: we seem to be launching less test nodes right now16:00
*** ihrachyshka has joined #openstack-infra16:00
*** olaph has joined #openstack-infra16:00
pabelangerfungi: or graphite.o.o is acting up16:00
fungipabelanger: yep, i'm checking to make sure we don't have any jenkins masters going crazy, quota problems, et cetera16:01
fungirax-iad is still basically unusable. i'm trying to get frenetical support to read the request in the open ticket correctly and clean up the 72 nodes we have stuck in a pending deletion state there16:02
*** teran has joined #openstack-infra16:02
*** rm_work is now known as rm_work|away16:02
*** jlanoux has quit IRC16:02
*** marzif__ has quit IRC16:03
*** jschwarz_ has joined #openstack-infra16:03
*** skolekonov has joined #openstack-infra16:03
fungioh yeah, jenkins07 has tipped over into brokenness16:03
*** jschwarz has quit IRC16:04
fungi[Thu Aug  6 08:05:49 2015] Out of memory: Kill process 12933 (java) score 936 or sacrifice child16:04
*** olaph has quit IRC16:04
fungilooks like it's been dead, dead, deadski for 8 hours now16:05
*** ParsectiX has joined #openstack-infra16:05
*** olaph has joined #openstack-infra16:05
openstackgerritMatthew Treinish proposed openstack/os-testr: Handle incomplete subunit streams  https://review.openstack.org/21000516:06
fungii've cleaned up and restarted jenkins07 now. also jenkins01 looks like it's starting to head into unhappyville so i'll put it into prepare for shutdown16:06
*** hogepodge has quit IRC16:06
*** hogepodge has joined #openstack-infra16:07
fungi256 disassociated floating ips in hpcloud. starting to remove them now16:07
*** pblaho has quit IRC16:08
*** geoffarnold has joined #openstack-infra16:09
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: Added new SF repository for Fuel Trove plugin integration  https://review.openstack.org/21000716:09
*** laughterwym has quit IRC16:11
*** warpc__ has joined #openstack-infra16:11
*** moravec has left #openstack-infra16:11
*** jistr has quit IRC16:12
*** kiran-r has quit IRC16:12
*** ParsectiX has quit IRC16:12
*** dtantsur is now known as dtantsur|afk16:12
*** jgriffith has left #openstack-infra16:13
*** minwang2 has quit IRC16:13
*** sputnik13 has joined #openstack-infra16:14
*** mpaolino has quit IRC16:14
*** sdake has quit IRC16:14
pabelangerfungi: looking into statsd to see if we can graph the failure some how for floating IPs16:16
*** esmute_ has joined #openstack-infra16:16
*** kurtmartin is now known as kmartin16:16
*** unicell has quit IRC16:18
*** minwang2 has joined #openstack-infra16:19
*** unicell has joined #openstack-infra16:19
*** moravec has joined #openstack-infra16:19
*** unicell has quit IRC16:20
*** otter768 has joined #openstack-infra16:21
openstackgerritDavide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO env for Cue job  https://review.openstack.org/20128516:22
*** minwang2 has quit IRC16:23
greghaynesTheJulia: Odd thing I noticed with shade - it is making a dib image by default of elements debian vm simple-init16:24
greghaynesTheJulia: That is going to have both cloud-init and simple-init, which is not so good16:24
greghaynesTheJulia: er, not shade, with bifrost16:24
*** otter768 has quit IRC16:26
greghaynesTheJulia: Maybe the default should be debian-minimal?16:26
*** ybathia has joined #openstack-infra16:27
*** mfmcdonagh has quit IRC16:28
*** mcdonagm has quit IRC16:28
*** wyatt88 has joined #openstack-infra16:29
openstackgerritPaul Michali proposed openstack-infra/project-config: Enable VPN DevStack plugin for neutron client  https://review.openstack.org/20988716:30
*** serg_melikyan has quit IRC16:30
*** sergmelikyan has joined #openstack-infra16:30
openstackgerritAndrei V. Ostapenko proposed openstack-infra/project-config: Move from stackforge/magnetodb to openstack/magnetodb  https://review.openstack.org/20352416:30
openstackgerritAndrei V. Ostapenko proposed openstack-infra/project-config: Changes jobs description for MagnetoDB  https://review.openstack.org/20447816:30
openstackgerritDawid Malinowski proposed openstack-infra/jenkins-job-builder: Add trigger-with-no-params for join-trigger  https://review.openstack.org/20627116:32
*** enikanorov2 has joined #openstack-infra16:32
*** patrickeast has quit IRC16:32
*** cdent has quit IRC16:33
openstackgerritPaul Michali proposed openstack-infra/project-config: Enable VPN DevStack plugin for neutron client  https://review.openstack.org/20988716:35
sdaguefungi: is there a way to check what's up with the unit test images wrt a libvirt update16:36
sdaguebecause nova unit tests seem to all be segfaulting now16:36
*** nadya has quit IRC16:37
*** ildikov has quit IRC16:38
annegentleHey TC members flaper87 sdague dhellmann ttx jeblair russellb lifeless dhellmann mordred jaypipes I'm still drafting the next TC blog post at https://etherpad.openstack.org/p/next-tc-blog-post, but I have to drive four hours tonight and be offline tomorrow, so I will need someone else to do the actual posting to make sure it goes out tomorrow.16:38
annegentleCan anyone already post to the blog who can take on the finalizing and copy/paste/post step?16:38
*** annegentle has quit IRC16:38
jaypipesannegentle: I cannot post to the blog.16:38
fungisdague: what do you mean by "what's up with the unit test images?"16:39
fungisdague: as in what version of libvirt they have installed?16:39
*** dguerri is now known as dguerri`16:40
*** yamamoto has quit IRC16:42
sdaguefungi: yeh16:42
sdagueand if it changed recently16:42
fungisdague: looks like it's been nearly a month since the ubuntu packages for libvirt changed in trusty16:42
sdaguewe're getting a segfault on all our unit test runs now16:42
*** puranamr has joined #openstack-infra16:42
sdaguein libvirt16:42
clarkbcheck python libvirt16:43
*** serg_melikyan has joined #openstack-infra16:43
clarkbor is it libvirt pythob?16:43
*** unicell has joined #openstack-infra16:43
*** sergmelikyan has quit IRC16:43
*** tonytan4ever has quit IRC16:43
sdagueactually, this is weirder, hold on16:44
fungii don't see the python package for it listed in the freeze16:44
clarkbyup uploaded on the 4th16:44
fungiwhat's the package called?16:45
openstackgerritli,chen proposed openstack-infra/project-config: A Devstack-plugin that installs a HDFS backend  https://review.openstack.org/20705016:45
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Listing flavors should pull all flavors  https://review.openstack.org/21002016:46
clarkblibvirt-python16:46
fungiodd it doesn't show up in the freeze at all16:46
fungior else my browser's having trouble searching in the job log16:46
clarkbits possible it isnt installed16:47
*** arxcruz has quit IRC16:47
*** btully has joined #openstack-infra16:48
clarkbI know nova unittests wanted to fakevirt (doesnt explain libvirt segfaults though)16:48
*** patrickeast has joined #openstack-infra16:48
Swansonfungi: Thanks for the help.  This looks happier.  Wish it looked faster but things are getting resolved now.16:48
*** vahidh has quit IRC16:48
fungiSwanson: diskimage-builder caches a lot of stuff, so subsequent runs should go much faster16:48
*** ParsectiX has joined #openstack-infra16:48
Swansonfungi:  Cool.  My original concern was that the "intelligent" firewalls around here were shutting me down.  So this is all good.16:50
openstackgerritMatthew Treinish proposed openstack/os-testr: Handle incomplete subunit streams  https://review.openstack.org/21000516:50
fungisdague: i take it you're unable to reproduce locally16:52
*** btully has quit IRC16:52
sdaguehonestly, I haven't gotten quite that far yet, mriedem was poking at it16:52
pabelangernibalizer: I think we are finally ready for https://review.openstack.org/#/c/205696/ to land16:53
*** patrickeast has quit IRC16:53
fungiseems like it shouldn't be related at all to the libvirt-python release earlier this week (not recent enough, also not used afaict)16:53
sdagueyeh16:53
* krotscheck is around if anyone wants to help land https://review.openstack.org/#/c/199707/16:55
*** patrickeast has joined #openstack-infra16:55
jeblairsdague: in 199091 is this correct: you are asking grenade to run devstack on the subnode after running the initial devstack setup on the main node, and before it runs the upgrade.16:56
sdaguejeblair: yes16:56
*** aukhan has quit IRC16:56
sdaguethat ensures that we bring up the workers after rabbit is up on the controller16:57
mriedemfungi: i reproduced locally16:57
mriedemtesting a change now16:57
fungimriedem: so it's specifically nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_clean_shutdown_no_wait triggering a segfault in the interpreter?16:57
nibalizerpabelanger: what do you think about putting a #date comment on the remove_module line so that those don't pile up and we can remove it in a month or something16:57
mriedemfungi: appears that way16:57
mriedemfungi: thing is we shouldn't actually be using python-libvirt anywhere16:57
mriedemso that could be something else too, still looking16:57
mriedemi found one dumb thing to remove and test anyway16:58
jeblairsdague: i was asking myself why do something like that, involving a hook in grenade, rather than in d-g.  but now i understand that generally you want grenade to handle the entire sequencing of devstack + grenade, and also it gives you fine grained control over things like what you just suggested (rabbit)16:58
mriedemthis https://github.com/openstack/nova/blob/master/nova/tests/unit/virt/libvirt/test_driver.py#L53916:58
jeblairsdague: which is fine -- i don't have a desire for d-g to be more complicated, i just wanted to make sure i understood.  :)16:58
sdagueright, because honestly, once the subnodes get brought up, we never want to touch them again. So doing just that one action point sets up exactly the scenario we want to test16:58
jeblairsdague: cool; want me to aprv?16:59
sdaguejeblair: that would be awesome16:59
openstackgerritPaul Belanger proposed openstack-infra/project-config: Remove color output from puppet apply  https://review.openstack.org/21003016:59
*** minwang2 has joined #openstack-infra16:59
sdagueit's got an experimental job in d-g, it appears to be working as designed. I read the logs pretty carefully after the success run16:59
jeblairsdague: cool, i'll work down the stack with +2s then +3.16:59
sdaguesounds good16:59
sdaguethe other 2 are basically refactorings, mostly to make this patch easier to review17:00
*** minwang2 has quit IRC17:00
*** amitgandhinz has quit IRC17:00
SpamapSgreghaynes: so.. vlan in the images.. actually.. ovs is the one we have to use.17:01
SpamapSgreghaynes: unless you figure out how to use linuxbridge. :)17:01
jeblairkrotscheck: i'd be happy to; though i'm the only +2 on that so far.  perhaps clarkb or fungi could give another +2 while i'm reviewing sdague's patch, and then i can do the legwork when complete.17:01
krotscheckjeblair: That'd be spectacular.17:01
*** bapalm_ has quit IRC17:01
* krotscheck casually points clarkb and fungi at https://review.openstack.org/#/c/199707/17:02
jeblairkrotscheck: also, istr having some questions about the change with the actual application of that, but i haven't checked back to see if you have responded17:02
*** mwagner_lap has quit IRC17:02
greghaynesSpamapS: oy17:03
krotscheckjeblair: Long story short on that response - the `npm run [thing]` command is much like tox -e venv [thing]17:03
krotscheckIt's entirely customizable.17:03
*** skolekonov has quit IRC17:03
krotscheckTo the point where rm -rf * is actually an option.17:03
jeblairkrotscheck: that could be a problem then.  'rm -rf *' is annoying but less of a worry.. 'cat /path/to/$CREDENTIALS' is much more of one17:04
greghaynesSpamapS: im about to try redeploying all the nodes from the .3 host17:04
greghaynesall the nodes but .2 and .3, that is17:04
jeblairkrotscheck: for equivalent things in python and java, we have developed ways of uploading things that are not at all customizable17:05
TheJuliagreghaynes: i was thinking debian-minimal, need to see if it working at the moment since it was broken when i was last messing with it17:05
mriedemfungi: https://review.openstack.org/#/c/210036/17:05
*** gyee has quit IRC17:05
krotscheckjeblair: Hrm. I see that. Can you give me an example of the python thing?17:05
SpamapSgreghaynes: actually... linuxbridge is exactly what we want.17:05
* SpamapS is reading17:05
greghaynesSpamapS: it is17:06
jeblairkrotscheck: in python, we run the build step, which is customizable, on a standard throwaway node, and have jenkins scp it to tarballs.o.o.  then we run the upload step as a second job on the proposal node that downloads the thing that was just uploaded to tarballs.o.o, and then, uploads it to pypi.17:06
greghaynesSpamapS: The only question is how easily we can configure neutron to use it with provider networking17:06
*** rockyg has joined #openstack-infra17:06
jeblairkrotscheck: so that upload step is fixed with code we entirely control -- so the only code that runs on the proposal slave is essentially 'download tarball from A upload tarball to B'17:06
krotscheckjeblair: Let me dig into that, I _think_ that's possible.17:07
SpamapSgreghaynes: well if we're using linuxbridge, IIRC, we'll be fine, because it's just going to bridge taps onto our vlan25.17:07
krotscheckAnd it makes sense.17:07
krotscheckAnd yes, cat does in fact expose things.17:07
*** minwang2 has joined #openstack-infra17:07
SpamapSgreghaynes: no vlan knowledge necessary. Just put the vms on the 15.x network. Done.17:07
*** jschwarz_ has quit IRC17:08
jeblairkrotscheck: at one point, we even wrote our own little script to do the XMLRPC needed for a pypi upload, but now we do it with twine, which is an upstream tool that is purpose-built for uploading things to pypi17:08
jeblairkrotscheck: cool, yeah, if you can separate build/upload, we can do something similar17:08
*** garyk has quit IRC17:08
greghaynesSpamapS: yea, that should be doable with ovs too17:08
greghaynesSpamapS: but I am definitely all for linuxbridge17:08
*** devvesa has quit IRC17:09
jeblairkrotscheck: having said all that, we don't need to block https://review.openstack.org/#/c/199707/ on that; we can continue laying groundwork there17:09
krotscheckjeblair: Yeah, but the followup patch with the publish job is already +2A &dependent.17:09
krotscheckLemme go make sure that doesn't land.17:09
jeblairkrotscheck: oh, good, yes thanks.17:09
SpamapSgreghaynes: our use case is really anti-ovs.17:10
krotscheckjeblair: Done.17:10
SpamapSgreghaynes: we never want tenant isolation.17:10
SpamapSgreghaynes: and we never want soft routers17:10
SpamapSuntil we do17:10
SpamapSand then DOH17:10
*** ParsectiX has quit IRC17:10
greghaynesSpamapS: yep, I think that is why we were being told to just do provider networking17:11
mrmartinfungi: the new ask.o.o site seems to be up now, but the theme is a bit broken. The staging site is using the feature/development branch, meanwhile the production one is tied to master branch. What is the proper process to merge the dev branch into master?17:11
mrmartinDo we have a guide for that somewhere?17:11
fungimrmartin: http://docs.openstack.org/infra/manual/drivers.html#merge-commits17:12
*** sobersabre has quit IRC17:12
*** ParsectiX has joined #openstack-infra17:12
*** spzala has joined #openstack-infra17:12
mrmartinfungi: thank you, I'm trying to merge the branches17:12
* greghaynes enrolls all the nodes and hopes the world doesnt explode17:13
*** chlong has quit IRC17:14
SpamapSgreghaynes: enroll or deploy?17:15
*** ildikov has joined #openstack-infra17:15
SpamapSenroll is like meh :)17:15
greghayneswell, just did deploy17:15
greghaynes46 nodes17:15
SpamapSoh fantastic17:15
greghayneswell see what happens...17:15
greghaynesI gave bastion01 a static ip on the 10.10.16 network btw, one above bastion0017:16
*** mrunge has quit IRC17:16
pabelangerfungi: could you point me towards a project that is using bindep right now?17:16
SpamapSbtw trying to find docs on linuxbridge is really hard17:16
SpamapSeverything is ovs17:16
greghaynesheh17:16
fungipabelanger: my wip patch for nova is https://review.openstack.org/164444 and the experimental bindep-nova-python27 job is using it and passing17:17
clarkbspamaps wat17:18
*** serg_melikyan has quit IRC17:18
fungipabelanger: the file needs trimming (i was playing with some additions/removals hence the comment lines) but it's a working example17:18
jeblairsdague: naturally it's the simplest change, 206175, that's going to take me the most time to review :)17:18
clarkbovs has no docs beyond man pages for commands that dont actualky tell what they are for or why they exist17:18
*** Somay has joined #openstack-infra17:18
*** Somay has quit IRC17:19
greghaynesclarkb: the commands are for ctling ;)17:19
SpamapSclarkb: I'm not talking about the actual tech underlying things.17:19
SpamapSclarkb: the way to tell OpenStack how to use these technologies and configure them.17:19
SpamapSovs is the leader, so every single manual just assumes it.17:20
minwang2can someone help to take a look at this patch https://review.openstack.org/#/c/209675/217:20
pabelangerfungi: cool, thanks. Will get me started on testing17:20
SpamapSthere's no good "this is why you would want linuxbridge and this is how to make it just attach all the vms to a physical network"17:20
*** dmorita has quit IRC17:20
krotscheckjeblair: Ya know, I wonder whether that vulnerability occurred to the travis people....17:20
greghaynesyea, thats my fear - linuxbr might only really be a known quantity with nova-net17:21
fungipabelanger: some things are not nailed down yet, for example it may make sense to tag some of those for a "test" profile rather than to the default profile depending on whether they're needed by things in requirements.txt or just by things in test-requirements.txt17:21
clarkbSpamapS we have devstack tempest jobs for it now17:21
SpamapSgreghaynes: Rackspace migrated all of their managed private clouds to linuxbridge+neutron17:21
*** ParsectiX has quit IRC17:21
*** austin81 has left #openstack-infra17:21
clarkbso canwork back from that17:21
greghaynesSpamapS: hah, so *someone* knows how17:22
SpamapSclarkb: yes, there's plenty of examples. I just would love some understanding.17:22
greghaynesoh, ok, win17:22
pabelangerfungi: is [platform:ubuntu] valid? Or do I need to specifiy the release?17:23
SpamapSgreghaynes: when you brought up vlan25, did you lose the 10.10 network?17:23
greghaynesSpamapS: no17:23
*** yamahata has quit IRC17:24
greghaynesSpamapS: I have eth2 with 10 net on it, eth2.25 with 15.x17:24
*** yamahata has joined #openstack-infra17:24
SpamapSgreghaynes: ah, I cant hit the IP for it from the old bastion17:25
pabelangerfungi: looks to be17:25
greghaynesSpamapS: 10.10.16.131?17:25
SpamapSgreghaynes: oh I was doing it wrong17:26
SpamapSgreghaynes: can you ssh-import-id my keys on the bastions? clint-fewbar17:27
greghaynesyerp17:27
SpamapSI want to spy ;)17:27
*** ifarkas has quit IRC17:27
greghaynesSpamapS: done, user ubuntu17:27
jeblairsdague: +3s all around17:28
greghaynesSpamapS: looks like deploys failed17:28
sdaguejeblair: thanks17:29
openstackgerritMerged openstack/os-testr: Handle incomplete subunit streams  https://review.openstack.org/21000517:29
greghaynesSpamapS: heh, I think I didnt add the ilo net17:29
*** achanda has joined #openstack-infra17:30
fungipabelanger: yeah, though using platform:ubuntu would be mostly a means of differentiating it from package names for other platform:dpkg based distros17:30
SpamapSgreghaynes: you have to route through .12917:31
greghaynesyerp17:31
SpamapSgreghaynes: we should probably add that static route to our bifrost manifest17:31
*** olaph has quit IRC17:32
fungipabelanger: since the package names on debian, ubuntu and other debian derivatives are typically identical it usually doesn't make sense unless you're referring to specific releases of a given distro17:32
clarkbor put an interface on that network (nice for ilo to avoid router fails)17:32
SpamapSclarkb: that's not an option we get.17:32
clarkb:(17:32
mrmartinfungi: this is the output of master merge, it was rejected at the end. What am I doing wrong? http://paste.openstack.org/show/411218/17:33
SpamapSSome kind of policy against provisioning networks being directly attached to workloads.17:33
*** sdake has joined #openstack-infra17:33
clarkbeven on the bastion?17:34
clarkbwow17:34
SpamapSyep. Bastion just gets let through the router.17:34
fungimrmartin: checked the acl to make sure it's set to allow your release group to push merge commits?17:34
*** baoli has quit IRC17:34
fungimrmartin: per the document i linked17:34
SpamapSclarkb: in theory thats how it works. In practice they seem to be letting the whole /25 through.17:34
clarkbha17:34
SpamapSclarkb: not a fact I'm going to alert them of. :)17:34
mrmartinchecking now17:34
*** salv-orlando has quit IRC17:35
fungimrmartin: oh! "change 179472 closed" implies you're trying to push an altered version of an already merged change17:35
SpamapSto be fair.. ilos' are really fragile17:35
SpamapSI'd take steps to control the number of hosts that can access them too.17:35
*** baoli has joined #openstack-infra17:35
fungimrmartin: i am trying now to determine how that's coming about17:36
mrmartinI did a total fresh clone of git repo17:36
*** funzo has quit IRC17:36
*** skylerberg has joined #openstack-infra17:36
clarkbSpamapS ya I would allow just the bastions but do so from l217:37
mrmartinfungi: anyway the feature/development seems to be 4 commits ahead of master branch17:37
fungimrmartin: oh! i see. the merge ended up being a fast-forward since the branches didn't both diverge17:37
greghaynesSpamapS: hrm, 6 nodes ended up in a funky state when I tried to delete17:37
fungimrmartin: so your commit --amend is altering the new branch tip which isn't a new merge commit at all17:38
fungii wonder what a good workaround is for this17:38
greghaynesSpamapS: I wonder if those are nodes with stuck ilos17:38
*** ivar-laz_ has quit IRC17:39
SpamapSclarkb: any particular reason you'd want to exert control at l2? I mean, l3's are a lot simpler to think about.17:39
fungimrmartin: possibly just resetting the master branch tip to the same as the current development branch is in order17:39
*** puranamr has quit IRC17:39
fungijeblair: clarkb: ^ opinions?17:39
clarkbSpamapS routers are less reliable than switches17:39
clarkband I hate getting someone in a closet/dc over the phone that I have seen in perso  to find the hosts and fix them17:40
clarkb*have never seeb17:40
SpamapSclarkb: I can't refute that. But I think they're still quite a bit more reliable as the ilos themselves, so that difference is somewhat mitigated.17:40
greghaynesSpamapS: ok, redeploying...17:41
clarkbhuh I never had issues with ilo (but I did have to use IE and c# app thing)17:41
mrmartinfungi: what is strange for me, that we don't have any other changes in master branch since the feature/development was created.17:41
jeblairfungi, mrmartin: i'm afraid i still don't quite understand what mrmartin is trying to do17:41
SpamapSclarkb: anyway, interesting thoughts. Maybe some day we'll get a rack with a bare TOR switch and we can really play. :)17:41
*** akshai has joined #openstack-infra17:41
mrmartinjeblair: I want to merge the feature/development branch into master at openstack-infra/askbot-theme repo17:42
fungijeblair: merge a feature branch into master, when there have been no new changes to master17:42
*** alexpilotti has joined #openstack-infra17:42
jeblairah, and just found http://paste.openstack.org/show/411218/17:42
* jeblair thinks some more17:42
SpamapSclarkb: I've never let anything in prod rely on ilos being available. They've failed me so many times at the worst possible moment..17:42
greghaynesclarkb: heh, so thers a handful of boxes in our rack where the ilo's are suck in a state that requires somene to physically go up to them and poke with a stick17:42
fungiresetting master to the feature branch commit should be safe since it's ff-able17:42
clarkbneat17:42
*** yamamoto has joined #openstack-infra17:42
greghaynesbecause these ilo's just do that when you talk too quickly to them17:42
jeblairfungi, mrmartin: you probably want to force git to create a merge commit17:42
SpamapSyep17:42
jeblairfungi, mrmartin: there's an option for that17:43
clarkboh we did have to fast ethernet all our ilos17:43
clarkbno gigabit17:43
SpamapSclarkb: ^ what greghaynes said. It's o_O given the role the thing is supposed to be playing.17:43
fungijeblair: oh! thanks, had no idea that was possible17:43
jeblairfungi, mrmartin: --no-ff17:43
*** ivar-lazzaro has joined #openstack-infra17:43
*** marcusvrn has quit IRC17:43
*** puranamr has joined #openstack-infra17:43
greghaynesSpamapS: \O/ most are in wait callback17:43
jeblairfungi, mrmartin: so maybe: git merge --no-ff origin/feature/development17:43
*** markvoelker has quit IRC17:44
mrmartinand can I push that to gerrit at the end?17:44
SpamapSgreghaynes: woot17:44
*** pvaneck has joined #openstack-infra17:44
SpamapSgreghaynes: when we get those poked with sticks, lets also get them firmware-upgraded.17:44
SpamapSgreghaynes: I seem to recall that solved many problems.17:45
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Ensure 'libffi-devel' is installed so that python-cffi can build  https://review.openstack.org/21005817:45
SpamapSand is a constant running theme in HP hardware. :)17:45
harlowjaianw fungi https://review.openstack.org/#/c/210058/ if u get a chance, seems thats needed to avoid http://logs.openstack.org/03/209203/1/check/gate-anvil-rpms-dsvm-devstack-centos7/18588d9/output/py2rpm-cffi-1.1.2.tar.gz.log17:45
*** jcoufal has quit IRC17:45
SpamapSIt would be pretty amazing if some company came out with a line of OpenCompute hardware that was also mostly open-source firmware.17:45
*** mwagner_lap has joined #openstack-infra17:45
fungimrmartin: looks like http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/acls/openstack-infra/askbot-theme.config needs to be updated with the addition documented at http://docs.openstack.org/infra/manual/drivers.html#merge-commits17:45
mrmartinfungi: so a git push gerrit at the end?17:45
SpamapSSystem76: I'm looking at you.17:46
*** garyk has joined #openstack-infra17:46
fungimrmartin: yeah, repeat the same steps as before, but just add the --no-ff to your git merge command17:46
mrmartinfungi, ok, I'm doing the project-config patch first17:47
fungimrmartin: and it's `git review -R` per the documentation, not push17:47
*** yamamoto has quit IRC17:47
*** amitgandhinz has joined #openstack-infra17:48
greghaynesSpamapS: now I kind of want to do this in hp2 just so I can boot 150ish nodes with ironic ;)17:48
greghaynesoh hey, some are going active17:49
jrollSpamapS: like openpower? :)17:49
SpamapSgreghaynes: lets dooo it17:49
jrollbut it's not x86 etc etc17:49
SpamapSjroll: I'm fine with a different cpu17:49
SpamapSjroll: did not know there was an openpower17:49
clarkbjroll how fast does qemu run in iy:P17:49
jrollheh17:50
*** cody-somerville has quit IRC17:50
jrollSpamapS: yeah, the goal is open firmware etc17:50
SpamapSI'm so meh on power though17:50
jrollyeah...17:50
jrollI think it's going to be a chicken/egg thing17:50
SpamapSbut really there's no reason you couldn't do the same thing with pc firmware, and I wonder why thats not happening?17:51
mtreinishjroll: I was thinking it's more like a dead chicken :)17:51
*** yfried has joined #openstack-infra17:51
clarkbSpamapS intel?17:51
jrollSpamapS: yeah, dunno, maybe nobody has the capital or the cares17:51
notmynameis there a documented "here's how to report and patch a security bug in openstack" document somewhere?17:51
SpamapSjroll: bingo. :-P17:51
sdaguedstufft: pypi down?17:52
clarkbnotmyname yes https://security.openstack.org17:52
mtreinishSpamapS: I thought there were open firmware projects out there17:52
notmynameclarkb: thanks17:52
SpamapSBut I could see a future where people are running containers inside vms on baremetal and going "why don't we just run this on baremetal?" and somebody goes "we don't have enough control between the firmware and container" and then somebody else says "what if we did?" and it happens.17:52
dstufftsdague: appears so17:52
funginotmyname: there's a specific section on that page i would link but my internet connection is acting up at the moment17:52
SpamapSmtreinish: there are. some. I want a box, a GOOD box, with all of it open.17:53
fungior maybe rackspace's networks are acting up17:53
mtreinishSpamapS: I think coreboot was one of the bigger ones17:53
*** cody-somerville has joined #openstack-infra17:53
*** markus_z has quit IRC17:53
jrollSpamapS: or a world where someone has a baremetal cloud that they don't want to get pwned and they make it happen :P17:53
sdaguedstufft: that wasn't even a gate thing, I was trying to do a 'pip search pbr'17:53
*** enikanorov2 has quit IRC17:53
SpamapSjroll: really thats the same thing. :)17:53
jrollya17:53
SpamapSjroll: so, get 'er done17:53
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Update query for nova seg fault bug 1482230  https://review.openstack.org/21006617:53
sdagueI think that might mean it's time for a break17:53
openstackbug 1482230 in OpenStack Compute (nova) "LibvirtConnTestCase.test_clean_shutdown_first_time causes subunit segfault" [High,In progress] https://launchpad.net/bugs/1482230 - Assigned to Matt Riedemann (mriedem)17:53
notmynameclarkb: fungi: do you know where those docs are managed? there's 2 important things I don't see there that should be (that gerrit is NOT the place for security patches and how to format a patch for review in launchpad)17:54
notmynameI'd like to patch it17:54
jrollSpamapS: send dev resources pls :P17:54
jrollSpamapS: unless you want to end up with firmware written in python17:54
openstackgerritMarton Kiss proposed openstack-infra/project-config: Enable pushMerge commits for askbot-theme repo  https://review.openstack.org/21006717:54
dstufftsdague: ``pip install`` probably works fine17:54
funginotmyname: in the openstack/ossa repo at the moment, though it'll likely get split out to a separate openstack/security-site repo later17:54
dstufftsearch doesn't have as much caching17:54
dstufft:/17:54
dstuffthttps://status.python.org/incidents/nqw6dm4cyjky17:54
mrmartinfungi: ^^^17:54
*** SumitNaiksatam has joined #openstack-infra17:54
fungimrmartin: thanks. as soon as i stop failing to reach review.openstack.org i'll have a look17:55
*** jgriffith has joined #openstack-infra17:56
mrmartingreat, thhnx17:56
fungiunfortunately it's breaking gertty _and_ my browser17:56
*** zul has joined #openstack-infra17:56
clarkbfungi try ipv417:56
*** bharath has joined #openstack-infra17:56
fungii'm going to have to i'm afraid17:57
clarkbthe ipv6 pipes have been trouble for me recebtly17:57
clarkblots of -4s in commands17:57
fungiwhatever this is, it just started in the past hour17:57
greghaynesSpamapS: Just ran into "iLO Advanced License required for this functionality." when I tried to TEXTCONS on an ilo :(17:57
*** patrickeast has quit IRC17:57
jeblairi'm on v4 and it wfm.  i'll try from a 6.17:57
greghaynesSpamapS: It appears we cant affort our own licenses17:57
fungiit looks like i'm just experiencing massive ipv6 packet loss at the moment17:58
clarkboh this has been days of off and on packet loss overv6 for me17:58
mrmartingreghaynes: try to get a trial license17:58
jeblair6wfm here17:58
fungijeblair: yeah, it may be my isp17:59
jeblairgreghaynes: neither can anyone else, so this is realistic.17:59
fungihrm, 26% packet loss. but i guess when you couple that with the lower mtu because my isp has a terrible idea of what ipv6 "support" looks like, i suppose that can be crippling18:01
*** kgiusti has joined #openstack-infra18:02
*** kmartin has quit IRC18:03
dstufftsdague: should be back now18:03
*** HeOS has quit IRC18:04
*** bapalm has joined #openstack-infra18:04
*** sputnik13 has quit IRC18:05
*** markvoelker has joined #openstack-infra18:05
pabelangerany infra-puppet-core mind approving: https://review.openstack.org/#/c/208446/ ? trivial change to fix fedora22 dibs18:05
*** sputnik13 has joined #openstack-infra18:05
SpamapSgreghaynes: vsp works fine18:05
*** sputnik13 has quit IRC18:05
*** sputnik13 has joined #openstack-infra18:06
*** che-arne has quit IRC18:08
*** liam__ has joined #openstack-infra18:09
*** esmute_ has quit IRC18:09
*** sputnik13 has quit IRC18:10
sdaguedstufft: thanks!18:10
*** Zhongjun has quit IRC18:10
openstackgerritPaul Belanger proposed openstack-infra/puppet-pip: Add Gemfile and puppet 4 checks  https://review.openstack.org/21007018:11
*** esmute_ has joined #openstack-infra18:12
*** esmute_ has quit IRC18:13
jeblairarmax: do you think you can look into this: https://review.openstack.org/20917118:14
openstackgerritMerged openstack-infra/elastic-recheck: Update query for nova seg fault bug 1482230  https://review.openstack.org/21006618:15
openstackbug 1482230 in OpenStack Compute (nova) "LibvirtConnTestCase.test_clean_shutdown_first_time causes subunit segfault" [High,In progress] https://launchpad.net/bugs/1482230 - Assigned to Matt Riedemann (mriedem)18:15
armaxjeblair: looking18:15
*** spzala has quit IRC18:17
*** esmute_ has joined #openstack-infra18:17
*** angerson has joined #openstack-infra18:19
*** esmute_ has quit IRC18:19
*** liam__ has quit IRC18:20
*** Sukhdev has joined #openstack-infra18:20
*** liam__ has joined #openstack-infra18:20
*** otter768 has joined #openstack-infra18:22
angersonHey all, I'm getting a "cannot import name constants" error when running tox on Zuul (from gear import constants). Here's a full stack trace: http://paste.openstack.org/show/411229/18:23
angersonAny clues?18:23
akraminskyHello, I'm trying to find some upstream jobs that run the test_live_migration tempest test, but I can't seem to find one that doesn't skip it. Anyone know one?18:23
clarkbangerson try tox -r to rebuild the venv.constants are relatively new iirc18:24
clarkbakraminsky gate-tempest-dsvm-multinode-full and gate-tempest-dsvm-neutron-multinode-full18:24
akraminskyclarkb: those get skipped too, SKIPPED: Block Live migration not configured for iSCSI18:25
*** esmute_ has joined #openstack-infra18:26
clarkbwe test live migration it uses the built in libvirt block copying method18:26
clarkbwe do not test it with shared storage18:26
*** rockyg has quit IRC18:26
*** sergmelikyan has joined #openstack-infra18:27
*** otter768 has quit IRC18:27
openstackgerritMarton Kiss proposed openstack-infra/askbot-theme: Standalone maintenance page for ask.o.o migration  https://review.openstack.org/21007318:27
clarkbif someone were to have devstack set up shared storage we could likely test that too18:27
*** tqtran has quit IRC18:27
*** liam__ has quit IRC18:28
*** esmute_ has quit IRC18:28
openstackgerritMarton Kiss proposed openstack-infra/askbot-theme: Standalone maintenance page for ask.o.o migration  https://review.openstack.org/21007318:28
akraminskyclarkb: so which case does run the libvirt block copying method?18:28
jeblairclarkb: https://review.openstack.org/199707 is ready for +3 if you have a sec18:28
*** davideag_ has joined #openstack-infra18:29
clarkbthis one http://logs.openstack.org/18/188118/17/check/gate-tempest-dsvm-multinode-full/165e2de/console.html#_2015-08-06_16_59_16_21318:29
angersonclarkb: Same problem after tox -r. This appeared the first time I ran the tests.18:29
clarkbjeblair looking18:29
notmynamefungi: clarkb: https://review.openstack.org/21007518:31
funginotmyname: thanks!18:32
clarkbkrotscheck jeblair is trusty npm/nodejs new enouvh?18:32
clarkbits 0.6.somwthing iirc18:32
*** tiswanso has quit IRC18:32
krotscheckclarkb: That's the precise version. Trusty's new enough18:32
notmynamefungi: turns out someone reported a sec bug in swift (invalid, fortunately) last night and then proposed the patch in the normal way. I had written something down before, but I'd love it to be in a more discoverable place :-)18:33
krotscheckclarkb: fungi was nice enough to update the publishing slave 2 weeks ago.18:33
*** baoli has quit IRC18:33
*** amitgandhinz has quit IRC18:33
*** davideagnello has quit IRC18:33
* krotscheck is having a hard time trying to figure out how to publish to NPM without leaving our credentials open to exposure.18:33
jeblairwe should probably install npm on it though18:33
krotscheckBecause there's a prepublish and a postpublish command that can just run anything.18:33
*** tiswanso has joined #openstack-infra18:34
clarkbjeblair this change instalks nom18:34
clarkbkrotscheck curl!18:34
clarkbits what we did before twine18:34
*** ayoung has quit IRC18:34
krotscheckclarkb: The protocol's not documented :/18:35
clarkbneither is pypis18:35
clarkbhad to read cheeseshop source to figure it out18:35
clarkbwas good fun18:35
*** amitgandhinz has joined #openstack-infra18:35
*** Hal has joined #openstack-infra18:36
*** btully has joined #openstack-infra18:36
clarkbas an alternative write a npmtwine18:36
*** Hal has quit IRC18:36
clarkb199707 approved18:36
clarkbangerson not sure then18:36
*** e0ne has quit IRC18:37
angersonclarkb: Darn. Thanks for assisting.18:37
*** mtreinish has quit IRC18:37
clarkbangerson double check the gear version installed in the virtualenv18:37
*** salv-orlando has joined #openstack-infra18:37
*** esmute_ has joined #openstack-infra18:37
*** baoli has joined #openstack-infra18:37
*** mtreinish has joined #openstack-infra18:39
*** spzala has joined #openstack-infra18:40
akraminskyclarkb: thanks for the help18:40
*** btully has quit IRC18:40
*** esmute_ has quit IRC18:40
*** gyee has joined #openstack-infra18:40
angersonclarkb: 0.5.8.18:41
clarkbhuh that should be new enough18:42
clarkbjeblair ^ ideas18:42
*** monester has quit IRC18:44
clarkbakraminsky does that cover what you need? if not we need to setup shared storage on those test nodes which may be a bit of work if we can use cinder volume bacjed i stances18:44
*** funzo has joined #openstack-infra18:44
*** salv-orlando has quit IRC18:44
clarkbwe do not use config drive there so that isnt an issue18:44
clarkb(becauae no live migration is supported with it apparebtly)18:45
*** puranamr has quit IRC18:45
greghaynesSpamapS: oh derp didnt make dhcp range big enough18:46
greghaynesthat is a nice problem to have though :)18:46
jeblairangerson: running on osx?18:47
angersonjeblair: Yep, Yosemite.18:47
jeblairangerson: the geard server used by zuul uses edge polling which isn't supported on osx18:48
openstackgerritMerged openstack-infra/system-config: Added NPM Publishing credentials to pypi slave.  https://review.openstack.org/19970718:48
*** funzo has quit IRC18:49
*** esmute_ has joined #openstack-infra18:49
jeblairangerson, clarkb: that's the first traceback, which is the cause of the subsequent ones18:49
angersonjeblair: Any workarounds?18:49
angersonOr should I do testing on Linux instead?18:50
pabelangerjeblair: question about merge-mode in zuul. I _thought_ at one time infra was running in cherry-pick mode.  Was curious why we didn't stick with it, or even use it today?18:50
jeblairangerson: yeah, i think run tests on linux is the way to go18:50
angersonjeblair: Alright, thanks for the help!18:50
jeblairangerson: np, sorry for the inconvenience18:51
jeblairpabelanger: it always rewrites commits which we found distasteful18:51
*** b10n1k has joined #openstack-infra18:51
jeblairpabelanger: and practially, that meant that workflows involving git branch -D don't work18:51
jeblair(because your local commit never shows up in the repo)18:51
clarkbit made rebase weird too18:52
clarkbbecause it didnt see the common history18:52
pabelangerright18:52
*** olaph has joined #openstack-infra18:53
akraminskyclarkb: I think that's what I was looking for18:53
*** baoli has quit IRC18:53
clarkbakraminsky perfect. the job is newish and not perfect but getting more robust all the time. just having a second node makes tge testing much more interrsting18:54
*** baoli_ has joined #openstack-infra18:54
*** harlowja has quit IRC18:56
*** nadya has joined #openstack-infra18:56
*** mquin_ has joined #openstack-infra18:58
*** mquin_ has quit IRC18:58
*** pc_m is now known as pc_m_AWAY19:01
*** mquin_ has joined #openstack-infra19:01
*** pc_m_AWAY has quit IRC19:04
*** achanda has quit IRC19:05
*** achanda has joined #openstack-infra19:05
*** mquin is now known as Guest3294419:06
*** mquin_ is now known as mquin19:06
*** Guest32944 has quit IRC19:06
*** puranamr has joined #openstack-infra19:06
*** markstur has joined #openstack-infra19:07
*** dahoo has joined #openstack-infra19:08
*** dahoo is now known as jerryz19:08
*** nadya has quit IRC19:08
marksturHello Infra. In Manila our "manila-coverage" job no longer captures the cover/ directory and it looks like the same thing happened to barbican19:08
marksturI was wondering if it is known problem or where I should file a bug19:08
*** annegentle has joined #openstack-infra19:09
*** tonytan4ever has joined #openstack-infra19:09
*** liam__ has joined #openstack-infra19:10
*** kmartin has joined #openstack-infra19:11
fungiseems my internet issues are not limited to ipv6. everything dies one hop into my isp's network, sometimes for up to 15 minutes at a stretch19:11
fungii'm taking this as a sign i should do some unpacking for a bit19:12
*** teran has quit IRC19:13
*** patrickeast has joined #openstack-infra19:14
*** warpc__ has quit IRC19:18
annegentledhellmann: ttx: or flaper87: any of you have an account for the OpenStack blog? I know Thierry does, maybe he can post tomorrow.19:21
clarkbttx is on vacation19:22
clarkbfungi and I can ping foundation people if it helps19:22
clarkbits the www site blog right?19:22
*** liam__ has quit IRC19:22
clarkbmarkstur please link an example change19:23
*** e0ne has joined #openstack-infra19:24
marksturclarkb, https://review.openstack.org/#/c/209852/19:24
marksturmanila-coverage used to capture cover/ directory19:25
marksturWe made a change to report regression in coverage, but that seems to be unrelated. That's why I looked at barbican19:25
openstackgerritMarton Kiss proposed openstack-infra/project-config: Enable pushMerge commits for askbot-theme repo  https://review.openstack.org/21006719:25
*** jgriffith is now known as Guest6942319:26
marksturclarkb, barbican example https://review.openstack.org/#/c/208688/19:26
*** wyatt88 is now known as jgriffith19:26
clarkbI think I know the problem will need a little time to confirm19:27
marksturclarkb, Awesome19:28
*** olaph has quit IRC19:29
annegentlepleia2: hey! Your session is right before mine at Grace Hopper! zomg wowsa!19:30
*** puranamr has quit IRC19:31
*** baoli_ has quit IRC19:32
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for stable/juno large-ops osc dep bug 1482350  https://review.openstack.org/21009519:32
openstackbug 1482350 in OpenStack-Gate "gate-tempest-dsvm-large-ops fails in stable/juno due to uncapped dep problems with osc" [Undecided,Confirmed] https://launchpad.net/bugs/148235019:32
*** armax has quit IRC19:32
*** baoli has joined #openstack-infra19:34
openstackgerritSebastien Badia proposed openstack-infra/project-config: scripts/copy_puppet_logs: Fix puppet beaker upgrade jobs  https://review.openstack.org/21009719:36
*** claudiub has quit IRC19:36
annegentleclarkb: I have an account but don't have much computer access tomorrow... I think I'll just have to plan on coffee shop stop to be sure19:36
clarkbok19:36
annegentleI mean, it's Texas, there oughta be internet :)19:37
*** hemna_ has joined #openstack-infra19:37
clarkbare you sure?19:37
*** hemna__ has joined #openstack-infra19:37
clarkbI hear its lots of empty space19:37
clarkb:)19:37
annegentle:)19:37
annegentlethere's lots of crappy radio where I'm driving19:37
*** claudiub has joined #openstack-infra19:38
*** hemna__ has quit IRC19:38
openstackgerritPaul Belanger proposed openstack-infra/project-config: Remove color output from puppet apply  https://review.openstack.org/21003019:38
jeblairannegentle: you should write a country music song about driving around empty space with nothing but crappy radio19:39
annegentlejeblair: lol19:39
*** boris-42 has joined #openstack-infra19:39
*** fifieldt_ has joined #openstack-infra19:40
*** minwang2 has quit IRC19:40
mrmartinjeblair: can you add me to the askbot-theme-release group in gerrit?19:41
jeblairmrmartin: ack19:42
mrmartinthnx19:42
jeblairmrmartin: done19:42
*** fifieldt has quit IRC19:42
mrmartinawesome, thnx19:42
openstackgerritMarton Kiss proposed openstack-infra/project-config: Enable pushMerge commits for askbot-theme repo  https://review.openstack.org/21006719:43
*** angdraug has quit IRC19:46
*** hemna_ is now known as hemna19:46
*** olaph has joined #openstack-infra19:46
*** salv-orlando has joined #openstack-infra19:47
openstackgerritMerged openstack-infra/devstack-gate: simplify setup_localrc branch logic  https://review.openstack.org/20617519:48
openstackgerritMerged openstack-infra/devstack-gate: extract setup_multinode_connectivity  https://review.openstack.org/20618919:49
*** ayoung has joined #openstack-infra19:49
mriedemanyone know if pbr will allow a 1.0.1.1 version?19:50
mriedemper semver i thought that wouldn't fly19:50
clarkbit wont19:50
openstackgerritMerged openstack-infra/devstack-gate: grenade partial multinode support  https://review.openstack.org/19909119:50
*** puranamr has joined #openstack-infra19:50
*** salv-orlando has quit IRC19:51
openstackgerritBrant Knudson proposed openstack/requirements: python-openstackclient up to 1.0.4  https://review.openstack.org/21010119:56
*** sergmelikyan has quit IRC19:57
*** sergmelikyan has joined #openstack-infra19:57
*** cburroughs has quit IRC19:57
openstackgerritMatt Riedemann proposed openstack/requirements: python-openstackclient up to 1.0.4  https://review.openstack.org/21010119:57
ryanpetrellomonty and other shade-related folks19:58
ryanpetrelloany idea when a new shade release may be coming to PyPI?19:58
ryanpetrelloI'd love to take advantage of https://github.com/openstack-infra/shade/commit/a9cf85b0cb8497757cce010e9d7accc2bf25ce06 :)19:58
ryanpetrellomordred ^19:58
*** dprince has quit IRC19:58
BrandonL18Are there any core reviewers that can look at my code and merge it if possible?19:59
BrandonL18Two other people have already reviewed my code and +1'd it.19:59
BrandonL18https://review.openstack.org/#/c/206178/19:59
BrandonL18It's for Jenkins Job Builder20:00
*** jamespage_ has joined #openstack-infra20:00
*** monester has joined #openstack-infra20:02
*** ivar-lazzaro has quit IRC20:04
*** HeOS has joined #openstack-infra20:06
*** ivar-lazzaro has joined #openstack-infra20:09
*** ybathia_ has joined #openstack-infra20:09
*** daemontool_ has joined #openstack-infra20:10
*** akraminsky_ has joined #openstack-infra20:11
*** VivCheri has quit IRC20:11
*** wayneeseguin has quit IRC20:11
*** ybathia has quit IRC20:11
*** pradk has quit IRC20:11
*** ybathia_ is now known as ybathia20:11
*** achanda_ has joined #openstack-infra20:12
*** kgiusti has quit IRC20:12
*** achanda has quit IRC20:12
*** akraminsky has quit IRC20:12
*** marzif has quit IRC20:12
*** minwang2 has joined #openstack-infra20:12
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add methods to set and unset flavor extra specs  https://review.openstack.org/20998320:12
*** pradk has joined #openstack-infra20:12
clarkbmarkstur: see http://logs.openstack.org/88/208688/3/check/barbican-coverage/bad0000/index.html its there because that index.html is served from swift20:13
*** wayneeseguin has joined #openstack-infra20:13
clarkbmarkstur: but the one without is served from apache itself looking at the on disk situation20:13
clarkbmarkstur: I think the fix is to stop copying file sto disk and having apache serve them then you will get the swift hosted index.html. I will work on that change20:13
*** ivar-lazzaro has quit IRC20:14
*** mquin has quit IRC20:14
*** mquin_ has joined #openstack-infra20:15
marksturclarkb, Thanks!20:17
mtreinishjeblair: where is the gertty bug tracker? (hopes it's not storyboard)20:17
*** marzif has joined #openstack-infra20:17
mtreinishturns out gertty explodes on gpg signed commits20:17
clarkbmtreinish: ha20:18
fungimtreinish: also, storyboard20:18
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add methods to set and unset flavor extra specs  https://review.openstack.org/20998320:19
fungimtreinish: see README.rst20:19
mtreinishfungi: who reads READMEs? :)20:19
* fungi thinks you may have just answered your own question20:19
*** Sukhdev has quit IRC20:20
mtreinishoh, the traceback is coming from GitPython20:20
fungimtreinish: i am willing to believe that gitpython doesn't grok signed commits20:22
jeblairmtreinish: gitpython folks are really good/fast about merging bugfix patches20:22
fungisupposedly support was added for them in gitpython 0.3.220:23
fungiso might be more of a corner case20:23
*** dguerri` is now known as dguerri20:23
mtreinishjeblair: yeah, although I'm looking if it's been fixed upstream because gertty pins the version20:23
*** otter768 has joined #openstack-infra20:23
mtreinishfungi: oh, you beat me to it :)20:23
*** dims_ has joined #openstack-infra20:23
mordredryanpetrello: very soon20:23
openstackgerritMerged openstack-infra/elastic-recheck: Add query for stable/juno large-ops osc dep bug 1482350  https://review.openstack.org/21009520:23
openstackbug 1482350 in OpenStack-Gate "gate-tempest-dsvm-large-ops fails in stable/juno due to uncapped dep problems with osc" [Undecided,Confirmed] https://launchpad.net/bugs/148235020:23
fungi>=0.3.2.RC1 should be getting at least 0.3.220:23
mordredryanpetrello: there's one outstanding patch I want in20:23
fungimtreinish: what version does pip freeze say you have?20:24
*** dims_ has quit IRC20:24
*** btully has joined #openstack-infra20:24
*** e0ne has quit IRC20:24
mtreinishfungi: oops you're right I was looking at the wrong terminal, the one with pip freeze output20:24
mtreinishnot the requirements output20:24
ryanpetrellomordred: okay, sounds good20:24
ryanpetrellothanks!20:24
mtreinishfungi: pip freeze shows GitPython==0.3.2rc120:25
*** mrmartin has quit IRC20:25
jeblairmtreinish: i'm on 1.0.1.20:25
mordredjeblair, clarkb, fungi, SpamapS: https://review.openstack.org/#/c/209514/ and the one that comes before it?20:25
mtreinishyeah it might have just been a stale venv20:25
*** dims_ has joined #openstack-infra20:25
mtreinishit's been a while since I last tried gertty20:26
jeblairmtreinish: have a signed commit handy i can look at?20:26
*** dims has quit IRC20:26
mtreinishit works after upgrade GitPython20:26
fungiyeah, i'm running with 1.0.1 as well, looks like20:26
mtreinishjeblair: 20971220:27
fungiso apparently when they say implemented in 0.3.2 they really mean 0.3.2 and not 0.3.2rc120:27
clarkbmordred: which one?20:27
clarkboh that one, yay lag20:27
*** otter768 has quit IRC20:27
fungimtreinish: yeah, i can browse that change without a problem20:28
jeblairmtreinish: so... how does one typically discover a commit is signed?20:28
*** btully has quit IRC20:28
mtreinishjeblair: you've got to check it out locally and do a git show or git log with --show-signature20:28
*** rm_work|away is now known as rm_work20:28
jeblairmtreinish: i haven't signed your key20:29
fungior git show --show-signature20:29
mtreinishjeblair: oh, well we can fix that at some point (I probably haven't signed yours either)20:30
clarkbmordred: reviewing those now20:30
greghaynesTheJulia: so, I am having a ton of fun with bifrost BTW :)20:30
*** jgriffith has quit IRC20:31
clarkbmordred: are you super afk or have time for a question?20:31
*** Guest69423 has quit IRC20:33
mordredclarkb: I'm totes here20:33
clarkbmordred: comments on 208105 if you can look real quick20:33
clarkbmordred: curious what yo uthink particular of the first non test comment20:34
*** tongli has quit IRC20:34
*** jamespage_ has quit IRC20:34
*** patrickeast has quit IRC20:35
mordredkk. looking20:35
mordredSpamapS: you've seen the docs from sc68cal yeah?20:36
TheJuliagreghaynes: \o/.... or is that sarcastic \o/ ?20:36
TheJulia:)20:37
openstackgerritMerged openstack-infra/elastic-recheck: Add a section to the readme about removing bug signatures  https://review.openstack.org/20470720:37
SpamapSmordred: no, I'd appreciate a pointer though.20:37
mordredclarkb: yah - I thought about adding a delete fails test case...20:37
greghaynesTheJulia: no sarchasm20:37
mordredclarkb: yes. we should log that. follow up patch?20:38
*** tonytan4ever has quit IRC20:39
*** alivigni has quit IRC20:39
*** patrickeast has joined #openstack-infra20:40
openstackgerritClark Boylan proposed openstack-infra/project-config: Fix coverage report links on logs.o.o  https://review.openstack.org/21011420:40
clarkbmarkstur: ^20:40
clarkbmordred: sure would you include that in the release?20:41
annegentlecan anyone recall how to delete a file from developer.openstack.org? I need to delete http://developer.openstack.org/api-ref-compute-v3.html and I don't have FTP to it.20:41
TheJuliagreghaynes: glad to hear it :)20:41
openstackgerritMonty Taylor proposed openstack-infra/shade: Add log message for when IP addresses fail  https://review.openstack.org/21011520:42
mordredclarkb: yup ^^ there ya go20:42
*** minwang2 has left #openstack-infra20:42
mordredannegentle: do you have a quick link to 4b?20:42
mordredannegentle: SpamapS is working on learning about provider-network/linuxbridge20:42
mordredSpamapS: there is also a devstack flag for it now too20:42
mordredsc68cal: if you're around, too, SpamapS may want to become you new best friend20:43
SpamapSclarkb: haha, we just had a +2+A collision ;)20:43
annegentlemordred: SpamapS: http://docs.openstack.org/networking-guide/deploy_scenario4b.html20:43
clarkbSpamapS: bam20:43
greghaynesThat rolls over into a -2 right?20:43
annegentlestill sad it has a 4b :)20:44
clarkbgreghaynes: it becomes a +620:44
greghaynesoh, straight to the top of the gate then20:44
greghaynes:p20:44
mordredSpamapS: annegentle beat sc68cal to being your new best friend20:44
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969820:44
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875120:44
SpamapSand a +F420:44
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961720:44
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963220:44
SpamapSannegentle: thhhhannnkkkyouuuuuu20:44
greghaynesoh h3y20:45
greghayneser, hey20:45
openstackgerritTerry Howe proposed openstack/os-testr: Convert file names to regular expressions  https://review.openstack.org/20342420:45
greghaynesthat doc looks like something we want20:45
*** funzo has joined #openstack-infra20:45
SpamapSgreghaynes: http://docs.openstack.org/networking-guide/deploy_scenario4b.html is what we're doing I think.20:47
SpamapSmordred: ^20:47
SpamapSjeblair: ^20:47
greghaynesyea20:47
*** claudiub has quit IRC20:47
greghaynesTheJulia: Bifrost question - I just made a new deploy image, if I want to push it out to some boxen what is the workflow for that20:47
greghayneswill deploy-dynamic "just do it"?20:48
*** ihrachyshka has quit IRC20:48
SpamapSgreghaynes: no20:48
clarkbgreghaynes: the Nike deployment?20:49
greghaynesYou know it20:49
mordredSpamapS: yes20:49
*** funzo has quit IRC20:49
jeblairSpamapS: http://docs.openstack.org/infra/system-config/infra-cloud.html#networking agrees with you!20:49
jeblair(it has a link to that page)20:50
*** bapalm has quit IRC20:50
fungigood ol' scenario 4b!20:50
SpamapShaha I didn't realize that had been added. Doh20:50
*** dguerri is now known as dguerri`20:51
TheJuliagreghaynes: it will pretty much just do it.  Keep in mind you can customize your workflow to fit your needs.  Checkout example-deploy-all-available-nodes.yaml20:51
greghaynesoooo20:51
*** BrandonL18 has quit IRC20:52
TheJuliagreghaynes: also, keep in mind, if it is already deployed, nothing is going to happen20:52
greghaynesyea, I need do work up something to redeploy20:52
jeblairso, at uc berkeley, before centralized mail, there was a machine called uclink, so you would be user@uclink.berkeley.edu.  it got full, so new people ended up on anotheruser@uclink2.berkeley.edu.  eventually they ended up with unfortunateuser@uclink4b.berkeley.edu before the wonders of centralized MXs were discovered.20:52
*** salv-orlando has joined #openstack-infra20:52
jeblairfungi: there's just something about "4b"...20:52
*** monester has quit IRC20:52
SpamapSgreghaynes: set them to deleted... then re-run deploy. :)20:53
TheJuliagreghaynes: yeah... never quite sa down to do that yet.  I might have a vict^H^H^Holunteer to help on supporting rebuild :)20:53
TheJulias/sa/sat/20:53
greghaynesfun fun20:53
clarkbgreghaynes: does the image build hostname still need debugging? I can probably dig into that today if so20:54
clarkb*nodepool image build hostnames20:54
greghaynesclarkb: I didnt actually look at the log to see what was going on20:54
greghaynesETOOMANYTHINGS20:54
*** dannywilson has quit IRC20:54
greghaynesgot a link?20:54
clarkbindeed you make cloud go20:54
SpamapSgreghaynes: regarding resizing /.. I think we should just make a playbook that plonks in an sfdisk command to resize / to 1100GB and runs resize2fs. Or we can put that in puppet, but I'm less inclined to have it in puppet since this is a "once ever in a box's lifetime" thing not "any time it's not right"20:54
*** dannywilson has joined #openstack-infra20:55
clarkbno ianw just mentioned that centos explodes on the hostname being too long when generating certs20:55
clarkbI can proably dig it up in the logs20:55
*** mwagner_lap has quit IRC20:55
greghaynesSpamapS: oh, I was just doing resize2fs /dev/sda 1024G20:55
greghaynesnot even sfdisk20:55
greghaynessince the config-drive is at the end of the disk20:55
SpamapSgreghaynes: true, it's not that horrible if we just leave the partition too big.20:55
greghaynesyep20:55
SpamapSgreghaynes: I'm about to give up on making diskimage-builder put a /boot on20:55
greghaynesbut yes, just ansible do it20:55
clarkbexcept sfdisk is crazy20:56
greghaynesSpamapS: heh20:56
greghaynesSpamapS: You want that so you can lvm?20:56
fungijeblair: still better than having a uucp address, or worse, one routed via a fidonet gateway20:56
clarkbsectors containers -'s who knows!20:56
SpamapSgreghaynes: no I want that so I can make the rest of the disk /20:56
greghaynesoh!20:56
greghaynesyea, that is another way to solve that20:56
jeblairclarkb: how many dockers per sector?20:56
clarkbSpamapS: just dont do what ubuntu does and make /boot too tiny for more than 2 kernels20:56
fungijeblair: i think they're measured in parsecs20:56
clarkbits so aggravating that the efi parittion is massive and has no data but /boot is tiny and has all the data20:57
openstackgerritSpencer Krum proposed openstack-infra/zuul: Log which repo zull references are created on  https://review.openstack.org/21011920:57
*** dkalleg has joined #openstack-infra20:57
clarkb(by default)20:57
SpamapSThough I do love lvm as it lets me do clever things like a small ro /usr and noexec /var/lib/nova ... but who wants to micromanage these things, right?20:57
SpamapSclarkb: inorite?20:57
fungiclarkb: SpamapS: what bootloader are you stuck on that can't boot from lvm?20:57
greghaynesfungi: we cant boot from 2TB without GPT is the issue20:58
fungiohhhh20:58
*** claudiub has joined #openstack-infra20:58
greghaynesjust trying to find path of least resistence20:58
SpamapSfungi: yeah, it's just a "numbers too big" issue. :-P20:58
fungiright-o20:58
fungithat _is_ an issue20:58
SpamapSand actually making it GPT only is one thing I do want to try20:58
SpamapSas that would be future-proofed and far simpler.20:58
fungiyah20:59
greghaynesmaking lvm with dib is on my list of things to hack on when I get some time20:59
greghaynesbecause I like to lvm my things20:59
mordredLVM == ossum20:59
SpamapSIn fact, I think ironic just needs tweaking to grok that GPT != uefi20:59
SpamapSBecause grub stage 1 is smart enough to go "oh, a GPT, I will boot that!"20:59
*** salv-orlando has quit IRC21:00
clarkbcan ironic not uefi?21:00
*** EricGonczer_ has quit IRC21:00
SpamapSclarkb: it can!21:00
SpamapSclarkb: but if it is not being told to uefi, it will freak out if there's no mbr21:00
clarkbalso we can use the freebsd bootloader with GPT instead of grub. Not sure why you want to though21:00
SpamapSbecause it will try to add a config-drive with mbr only tools21:00
clarkbSpamapS: gotcha21:01
SpamapSit's entirely just a rigidity that isn't needed.21:01
openstackgerritdevdatta-kulkarni proposed openstack-infra/project-config: Rename stackforge/solum -> openstack/solum  https://review.openstack.org/20128721:01
SpamapSI have a near-term todo for that.21:01
clarkbbut iirc GPT + no uefi is how freebsd has done it for years now21:01
*** radez is now known as radez_g0n321:01
SpamapSclarkb: yeah, grub has supported that for along time too.21:02
SpamapSa long21:02
SpamapSclarkb: but.. distros and their funny "dual boot with stupid OS's" things.. ;)21:02
*** rfolco has quit IRC21:02
SpamapSwe have no such desire. :)21:02
*** radez_g0n3 is now known as radez21:03
SpamapSsome day I'd love for a distro to _JUST_ be for servers and always favor simple things that are just for servers.21:03
annegentlejeblair: bwah ha ha finally read the 4b berkley email story ha ha ha21:03
* SpamapS waits for the BSD people to chime in21:03
annegentleso how do I delete a file from developer.openstack?21:03
clarkbSpamapS: oh is that the motivation? (dual booting)21:04
fungiSpamapS: i left my chimes in my other pands21:04
fungipants21:04
fungipandas?21:04
clarkbits funny I had to boot windows not too long ago to fill out canadian visa paperwork for my mom. I almost forgot that my desktop could do that (don't worry it was airgapped from the internet for that operation)21:05
fungiannegentle: via ftp21:05
annegentlefungi: is it a separate server from docs.openstack?21:05
annegentlefungi: maybe I don't have creds?21:05
*** tiswanso has quit IRC21:05
fungiannegentle: yeah, as far as i know cloudsites doesn't provide an easy way to have multiple fqdns for different docroots in a single site21:06
*** patrickeast has quit IRC21:06
fungiannegentle: creds will be different from docs.o.o but no recollection whether i sent you some when i originally created that one21:06
* fungi checks21:06
*** mriedem is now known as mriedem_away21:07
*** rm_work is now known as rm_work|away21:07
annegentlefungi: ok, I think I only have docs dot21:07
fungioh, right, i think you have to manage these from the "old" rackspace dashboard?21:07
fungiick21:07
annegentlefungi: the docs dot ones, yes21:07
clarkbfungi: I am glad you know all of this21:08
fungii think last time i tried it my browser was too new to allow me to connect21:08
fungigiving it another shot now21:08
*** dannywilson has quit IRC21:08
*** aysyd has quit IRC21:08
openstackgerritJeremy Stanley proposed openstack-infra/infra-manual: Recommend --no-ff on merges from feature to master  https://review.openstack.org/21012421:08
*** dannywilson has joined #openstack-infra21:08
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Update ATC and approvers scripts for deliverables  https://review.openstack.org/21012521:08
*** e0ne has joined #openstack-infra21:09
*** yamamoto has joined #openstack-infra21:10
*** patrickeast has joined #openstack-infra21:11
*** ayoung has quit IRC21:11
clarkbgreghaynes: from a snapshot log we definitely run ./prepare_node.sh bare-centos6-1438784043.template.openstack.org21:12
clarkbya thats a different name than the slave host names21:13
fungiaha! it was https://manage.rackspacecloud.com/21:13
fungithat took some searching21:13
*** ldnunes has quit IRC21:13
clarkbgreghaynes: ianw will work on a fix similar to how we fixed the slave hostname issue21:13
*** BrandonL08 has joined #openstack-infra21:15
*** dkranz has quit IRC21:15
*** e0ne has quit IRC21:16
*** annegentle has quit IRC21:18
*** sergmelikyan has quit IRC21:19
*** sergmelikyan has joined #openstack-infra21:19
*** yamamoto has quit IRC21:19
openstackgerritMerged openstack-infra/shade: Throw an exception on a server without an IP  https://review.openstack.org/20810521:19
*** beisner is now known as beisner-afk21:19
openstackgerritClark Boylan proposed openstack-infra/nodepool: Shorten template hostname  https://review.openstack.org/21012721:20
clarkbgreghaynes: ianw ^21:20
openstackgerritMerged openstack-infra/shade: Be consistent with accessing server dict  https://review.openstack.org/20951421:20
*** radez is now known as radez_g0n321:20
*** SumitNaiksatam has quit IRC21:22
*** beisner-afk is now known as beisner21:22
*** markvoelker has quit IRC21:22
*** armax has joined #openstack-infra21:22
*** reed_ has joined #openstack-infra21:23
mordredShrews: hrm. I'm torn on your flavors patch21:23
mordredShrews: because for non-admin users, "is_public" should default to true, but for admin users is_public should default to false21:24
mordredShrews: perhaps an override of the method with different default behaviors in operatorcloud?21:24
*** patrickeast has quit IRC21:24
mordred(I don't want to fall in to the trap of making a tool that is friendly to operators at the expense of doing what an end user expects)21:25
*** bswartz has quit IRC21:25
*** btully has joined #openstack-infra21:25
*** dguerri` is now known as dguerri21:26
mordredShrews: also - bonghits on the second patch21:26
*** patrickeast has joined #openstack-infra21:27
*** dguerri is now known as dguerri`21:28
Shrewsmordred: ahoy21:29
*** EricGonczer_ has joined #openstack-infra21:29
*** btully has quit IRC21:29
Shrewsmordred: perhaps i don't understand flavors. testing on devstack, i created a private flavor as admin, yet the demo user was not able to see this, even when requesting all flavors21:31
mordredShrews: OH!21:33
Shrewsmordred: so the behavior is as you outlined.21:33
mordredShrews: well, then. in that case, you understand them better than I do21:33
mordredShrews: and I agree with your patch21:33
fungido we have a good way to identify the creation date and time for the diskimage from which a nodepool node was booted?21:34
*** cburroughs has joined #openstack-infra21:34
fungii'm looking for a file which would be created by dib and not subsequently updated during/after boot21:34
*** harlowja has joined #openstack-infra21:34
*** EricGonczer_ has quit IRC21:35
clarkbfungi: the node boot process should write the image uuid to /etc/nodepool21:35
fungiyeah, i'm saying what if i don't trust the uuid to image creation mappingh21:35
fungitrying to figure out if nodepool is really uploading updated diskimages, or just believes it is21:36
clarkboh if yo don't trust that then do a nova show on the instance21:36
clarkbthat includes the image nova thinks it was booted from21:36
fungiwhich at least rules out everything except nodepool successfully uploading the wrong image21:37
fungithough /etc/apt/sources.list has a current creation time, and i don't think we tweak that with a ready script, right?21:38
*** dannywilson has quit IRC21:39
*** harlowja has quit IRC21:39
clarkbwe may for fixing the security location from local cloud to ubutun proper21:39
*** harlowja has joined #openstack-infra21:39
jeblairmordred, clarkb, fungi: i need another brain on the nodepool instance-reuse problem.  i've been staring at it for 2 days and have yet to figure out a way to reproduce, or anything that could be wrong.21:39
jeblairhere are the log entries: http://paste.openstack.org/show/411583/21:39
*** angdraug has joined #openstack-infra21:39
jeblairi annotated them with 'A' and 'B' to indicate which threads are emitting them21:39
*** dannywilson has joined #openstack-infra21:40
*** bswartz has joined #openstack-infra21:40
jeblairhttp://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/nodepool.py#n243821:40
fungijeblair: is there a lock around the database updates, or any (negative) query caching, that could cause nodepool's alien cleanup not to see the addition?21:40
*** BrandonL08 has quit IRC21:41
jeblairthe only way that sequence can happen is if thread B, the cleanup thread, can't find the database record for the node -- that line 2438 ^21:41
jeblairfungi: not to my knowledge -- we're using mysql with sqlalchemy.  each thread should have a thread-local session attached to its transaction.  and while sqlalchemy will return an existing object from the session if you request it by id, i do not believe it performs any kind of negative caching related to that.  so if the object didn't already exist in the cleanup thread session, it should hit the db and see it.21:43
fungiit looks like the launcher knows the name and uuid as early as 14:58:23,581 and then the cleanup spots it at 15:03:16,571... nearly 5 minutes have elapsed, so that's pretty stale21:43
jeblairand we know it's in the database because it's actually the _main_ thread (not represented here) that creates the record originally, and then a newly-created launch thread (thread A here) does a db lookup in its own session to retrieve the record21:44
*** b10n1k has quit IRC21:44
jeblairso "Launching node id: 4117507" can be interpreted as "thread A just successfully retrieved the node record by ID from the database"21:45
*** zul has quit IRC21:45
fungithough at that stage the uuid column is still empty for that row, yeah?21:45
jeblaircorrect21:45
*** mquin_ is now known as mquin21:45
fungipresumably it's written several seconds later once it's known?21:46
jeblairfungi: oops, i lied, it's the "creating server" line that indicates a successful lookup; 'launching node' happens right before the lookup.21:47
fungiif the periodic cleanup queries the nova list and sees the uuid before it's updated in nodepool's db, would that mark it as an alien?21:47
jeblairfungi: and then the uuid column is written to (and a commit issued) by the 'waiting for server' line.21:47
*** claudiub has quit IRC21:48
fungiwould have to be a _very_ tight race between nova list and nova boot i guess21:48
*** tiswanso has joined #openstack-infra21:48
jeblairfungi: we actually query by the nodepool id in the metadata, so the uuid column doesn't matter at that point21:48
jeblairfungi: the nova uuid will be used to delete the instance if there is not a match, but we don't use it until we decide to delete it; we operate entirely on metadata21:48
fungioh, there goes that theory21:50
lifelessmriedem_away: pbr's versioning is all three-component semver at this point; we can change that but lets do so with some discussion (a spec) since its so wide-impact21:51
*** tiswanso has quit IRC21:52
ianwclarkb: "greghaynes: ianw will work on a fix similar to how we fixed the slave hostname issue" <- was that saying you want me to do something?21:53
*** armax has quit IRC21:53
greghaynesianw: heh, I think clarkb already did it21:53
ianwoh, ok, just catching up here :)21:54
clarkbsorry your nick was missing the : too21:54
*** armax has joined #openstack-infra21:54
clarkbI was saying I will fix it and making sure you saw that I would21:54
*** reed_ has quit IRC21:54
clarkbjeblair: let me look21:54
clarkbbut my brain is probably not up for debugging thread races right now21:55
*** zul has joined #openstack-infra21:55
*** thorst has quit IRC21:55
clarkbjeblair: leak checking queries nova directly then compares against the db. so could this just be the launch node thred hasn't committed to the db by the time the leak checker grabs from the db?21:56
clarkbjeblair: I wouldn't be surprised if leak checking grabbed a db listing firs,t then a nova listing, then compared which would be a race21:56
*** BrandonL08 has joined #openstack-infra21:57
*** SumitNaiksatam has joined #openstack-infra21:57
clarkblooks like it gets a session then lists the hosts from nova. That would be in its own transaction right? so wouldn't see updates from the launch node thread?21:58
openstackgerritMerged openstack/os-testr: Convert file names to regular expressions  https://review.openstack.org/20342421:58
*** salv-orlando has joined #openstack-infra21:59
*** pradk has quit IRC22:00
lifelessNakato: how goes the tox constraints stuff? Do you need any support?22:01
*** ivar-lazzaro has joined #openstack-infra22:03
*** HeOS has quit IRC22:04
jeblairclarkb: i would expect it to see those updates....oh... so maybe this is a mysql question...22:04
*** HeOS has joined #openstack-infra22:04
jeblairclarkb, fungi: where sometimes operations inside the cleanup thread cause it not to see data from the other thread... which probably relates to when sqlalchemy does it's implicit begin transaction, which may be what i was missing in my attempted reproduction...22:05
*** rlandy has quit IRC22:06
akraminsky_clarkb, trying to run those tempest tests and am running into this error "libvirtError: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: ssse3, vme" have you ran into something like this? It looks like its an issue with the bios.22:06
*** marzif has quit IRC22:07
jeblairclarkb, fungi: i know that in one case, i started with a query for the node id before it was created, but still it showed up in the same thread after it was created, but maybe that's not enough22:07
clarkbakraminsky_: its not a bios issue, it has to do with having different cpu types across nodes. We ran into it and deal wiht it by having devstack write out a custom qemu cpu type22:07
clarkbakraminsky_: I think its call devstack64 or something like that22:07
*** gordc has quit IRC22:08
*** salv-orlando has quit IRC22:08
asselin_clarkb, hi, I'm working a bit with akraminsky_ on this. I see you wrote this script: tools/cpu_map_update.py22:11
asselin_. Is that what you're referring to?22:11
clarkbasselin_: yes22:12
clarkbasselin_: it writes out a bare minimum amd64 cpu type that works across hpcloud and rax22:12
*** cburroughs has quit IRC22:13
clarkbasselin_: I believe it will work on any amd64 cpu as long as you haven't disabled a weird set of features22:13
asselin_clarkb, ok we'll take a look. Still very odd that we would run into this since all our hardware/images/nodes should be the same22:13
clarkbas its pretty minimal22:13
*** teran has joined #openstack-infra22:14
*** armax has quit IRC22:15
*** amitgandhinz has quit IRC22:16
*** patrickeast has quit IRC22:17
*** sergmelikyan has quit IRC22:17
*** tiswanso has joined #openstack-infra22:18
jeblairclarkb: i just verified that our database is configured to use the default isolation level of repeatable-read.  :)22:19
*** yamamoto has joined #openstack-infra22:19
ianwhttps://review.openstack.org/#/c/210030/2 (Remove color output from puppet apply) <- you'd think puppet could figure this out with isatty() ...22:23
*** abitha has quit IRC22:23
*** bknudson has quit IRC22:23
*** otter768 has joined #openstack-infra22:24
clarkbjeblair: so if that session is one transaction it won't see any updates because it has a snapshot created by the first read?22:24
*** yamamoto has quit IRC22:25
*** abitha has joined #openstack-infra22:26
*** patrickeast has joined #openstack-infra22:27
jeblairclarkb, fungi: and now i know why i can't repro locally -- wrong engine.  thank you!  i knew this was one of those things i just needed to talk about to see the obvious thing.  :)22:27
clarkbjeblair: you were myisam?22:27
*** abitha has quit IRC22:27
*** abitha has joined #openstack-infra22:28
*** otter768 has quit IRC22:28
openstackgerritmin wang proposed openstack-infra/project-config: New jobs for neutron-lbaasv2 tempest tests  https://review.openstack.org/21014122:30
openstackgerritmin wang proposed openstack-infra/project-config: New jobs for neutron-lbaasv2 tempest tests  https://review.openstack.org/21014122:31
jeblairclarkb: yep22:32
*** stevemar has quit IRC22:33
jeblairclarkb: and reproduced! :)22:34
*** tiswanso has quit IRC22:34
clarkbjeblair: woot22:34
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Disable venv anvil VERBOSE setting  https://review.openstack.org/21014222:35
*** zul has quit IRC22:35
openstackgerritMerged openstack-infra/project-config: Remove color output from puppet apply  https://review.openstack.org/21003022:35
*** sergmelikyan has joined #openstack-infra22:36
clarkbjeblair: after an initial look at code I don't have great ideas for fix. Maybe set transaction read level on that specific session to something a bit more ready?22:36
*** crc32 has joined #openstack-infra22:37
*** asselin__ has joined #openstack-infra22:37
*** puranamr has quit IRC22:38
openstackgerritIan Wienand proposed openstack-infra/project-config: scripts/copy_puppet_logs: Create LOG_DIR  https://review.openstack.org/21009722:39
*** asselin has quit IRC22:39
openstackgerritMerged openstack-infra/shade: Add log message for when IP addresses fail  https://review.openstack.org/21011522:39
dkallegHey clarkb: Me again :)  I've tried running my test on a devstack that I stacked with the local.sh that Jenkins used and the test still passed on my env.  I'm having trouble printing console logs.  When doing the rest call to servers/<instance id>/action , i get an exception saying my instance could not be found.22:40
clarkbdkalleg: is the test automatically cleaning it up after failure? you may need to catch the fail, do the console log, then do that clenaup22:40
dkallegclarkb: yes it does auto cleanup. The exception is while trying to get the console log.22:42
*** marzif has joined #openstack-infra22:42
dkallegclarkb: I've been debugging for nearly two days with trial and error and waiting 2-5 hours for results from Jenkins.  How much effort goes behind keeping a Jenking VM around for me to ssh into?22:43
clarkbdkalleg: its less the effort involved and more the fact that this indicates your tests have a bug in logging22:43
clarkbdkalleg: we want to make your tests debuggable and not just hop on the VM every time it fails22:43
*** asselin__ has quit IRC22:43
dkallegclarkb: I certainly understand trying to keep it to a min.  So what else do you recommend I check before we deem it necessary?22:44
clarkbdkalleg: definitely get the console log so we can see if VM boots are failing22:44
dkallegI provision a cluster, get the cluster id, then loop over each node in the cluster, restfully calling to get the console log, then the env returns saying it doesn't know about my vms22:44
clarkbyou might also look at the tempest scenario tests as they pull ocnsole logs on failures22:45
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Fix erroneously deleting non-leaked instances  https://review.openstack.org/21014922:45
jeblairclarkb: ^22:45
clarkbjeblair: I hvae to go feed a baby but will look as soon as  Ican22:45
*** sharikapongubala has joined #openstack-infra22:45
clarkbdkalleg: those logs are very important in any case where you need the VM to do more than just boot22:45
clarkbdkalleg: that is why tempest grabs them for debugging22:46
*** stevemar has joined #openstack-infra22:47
krotscheckjeblair: I'm in conversation with npm right now, hopefully something will come of it. Best I can tell, we'll have to write our own publishing client.22:47
dkallegclarkb: I think we're having a disconnect.  I agree the console logs would be a huge help.  I'm using a framework that existed for 6+ months to request for those logs.  I'm inclined to believe the request for console logs is sound.22:47
krotscheckjeblair: PErsonally I'd love to have a --no-lifecycle option to npm publish, but we won't be able to use that until the distros catch up.22:47
clarkbdkalleg: ok then lets get that working first22:48
dkallegclarkb: so what are you recommending22:48
clarkbdkalleg: because I have a hunch it will point out the problem22:48
*** sharikapongubala has quit IRC22:48
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Remove terminated image/instance delete threads  https://review.openstack.org/21015022:48
dkallegclarkb: I'm convined that console logs would point out the problem22:48
jeblairclarkb: also noticed that ^22:49
openstackgerritTerry Howe proposed openstack/os-testr: Make use of argparse groups and add some tests  https://review.openstack.org/21015122:49
clarkbdkalleg: ok then lets get that working, should be able to look at tempest to see how they do it22:49
clarkbdkalleg: look a tthe scenario tests that boot a VM and ssh into it, on failures they dump the console log22:49
*** sharikapongubala has joined #openstack-infra22:50
*** dims_ has quit IRC22:50
jeblairkrotscheck: cool, thanks.  good thing we like writing code!  :)22:50
*** markvoelker has joined #openstack-infra22:50
krotscheckjeblair: We're going to have a chicken/egg problem though. Can't publish to npm without npm tool. Can't have npm tool without being able to publish. DOH22:51
krotscheck:D22:51
clarkbkrotscheck: unless you use curl22:51
*** sharikapongubala has quit IRC22:51
clarkbkrotscheck: and I am dead serious about that, worked great for pypi22:51
krotscheckclarkb: I do not trust the stability of the npm registry enough to rely on curl.22:51
krotscheckOr, well, the stability of their api.22:51
clarkbkrotscheck: well thats an issue with any tool22:52
jeblairthat wouldn't be solved by writing another tool, right?  unless you aim to get them to maintain it in sync with their api.22:52
krotscheckclarkb: True. But why worry when you can jsut make someone else do the work? https://github.com/npm/npm-registry-client22:52
*** alexpilotti has quit IRC22:52
krotscheckjeblair: There's already a library that does it, we'd just have to write a wrapper22:52
jeblairkrotscheck: ah nice22:52
krotscheckYep. Especially since that's what npm uses.22:53
dkallegclarkb: I'm making modifications to an integraion test that boots 3 vm's and would get console logs in certain error cases.  I made a change to have it follow the same process but with or without failures.22:53
jeblairkrotscheck: (though, considering npm is in distros, etc, i'd hope they maintain backwards compat with their api for quite some time)22:53
krotscheckjeblair: That's.... an excellent point.22:54
krotscheckI hope so too.22:54
*** baoli has quit IRC22:54
*** sharikapongubala has joined #openstack-infra22:54
dkallegclarkb: If it helps, I've made changes to how this test tries to create a cluster:  https://github.com/openstack/cue/blob/master/tests/integration/api/v1/clusters/test_clusters.py22:55
*** funzo has joined #openstack-infra22:55
*** EricGonczer_ has joined #openstack-infra22:56
asselin_nibalizer, you around to discuss https://review.openstack.org/#/c/199674/7/manifests/init.pp ?22:57
*** alexpilotti has joined #openstack-infra22:58
*** funzo_ has joined #openstack-infra22:58
jheskethMorning23:00
*** funzo has quit IRC23:00
clarkbjhesketh morning. there are comments on your change to set os-loganalyze filters, can you read them?23:00
*** marzif has quit IRC23:01
krotscheckjeblair: Any opinions on what a custom npm publish tool might be called? "openstack-npm-publish" seems meh.23:01
jheskethclarkb: cool, will do23:01
clarkbat least one needs to ve addressed for sure23:01
nibalizerasselin_: not really sorry23:02
*** funzo_ has quit IRC23:03
asselin_nibalizer, ok, will leave comments then23:03
asselin_nibalizer, going on vacation. back next week23:03
*** erlon has quit IRC23:03
mmmporkhey folks, i'm assuming that someone else probably needs to +1 my addition of myself to stackalytics?23:03
mmmporkhttps://review.openstack.org/#/c/210111/23:03
nibalizerasselin_: okay have fun23:04
*** EricGonc_ has joined #openstack-infra23:04
asselin_nibalizer, thanks23:04
Clintmmmpork: you need +2 and +A from https://review.openstack.org/#/admin/groups/183,members23:04
nibalizerhuh23:04
nibalizermmmpork: I wonder if someone needs to do a big patch to stackalytics to rename HP Development to HPE ?23:04
greghayneshah23:05
mmmporktaking into account all of the "HP" variations?23:05
Clintnibalizer: i was wondering the same thing23:05
Clintplus the email addresses23:05
jeblairkrotscheck: s/openstack-// ?23:05
*** radez_g0n3 is now known as radez23:06
*** sharikapongubala has quit IRC23:06
*** EricGonczer_ has quit IRC23:07
krotscheckjeblair: Already taken.23:07
krotscheckjeblair: npm-publish-tarball23:07
krotscheckI'll run with that.23:07
clarkbnpmtwine23:07
*** abitha has quit IRC23:07
*** patrickeast has quit IRC23:08
*** sharikapongubala has joined #openstack-infra23:08
*** sc68cal has quit IRC23:09
*** dims_ has joined #openstack-infra23:10
openstackgerritSukhdev Kapur proposed openstack-infra/irc-meetings: Adding details to ironic_neutron meeting  https://review.openstack.org/21015423:10
krotscheckclarkb: We can go with a theme. Instead of twine, call it rope?23:11
clarkbsure dstufft may have more info on the twine name23:11
*** wenlock has quit IRC23:12
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud  https://review.openstack.org/20969823:12
openstackgerritColleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty  https://review.openstack.org/20875123:12
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env  https://review.openstack.org/20961723:12
openstackgerritColleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes  https://review.openstack.org/20963223:12
*** btully has joined #openstack-infra23:13
*** bswartz has quit IRC23:13
openstackgerritMerged openstack-infra/nodepool: Shorten template hostname  https://review.openstack.org/21012723:14
*** sc68cal has joined #openstack-infra23:14
*** vmbrasseur has joined #openstack-infra23:15
mtreinishclarkb: when you get a chance I pushed up the subunit2sql elastic-recheck crossover episode: https://review.openstack.org/20971223:16
*** r-daneel has quit IRC23:17
*** atuvenie has quit IRC23:17
*** atuvenie has joined #openstack-infra23:17
*** btully has quit IRC23:17
*** jgriffith has joined #openstack-infra23:19
*** mriedem has joined #openstack-infra23:21
*** kmartin has quit IRC23:22
*** mriedem_away has quit IRC23:23
*** kurtmartin has joined #openstack-infra23:23
*** kurtmartin has quit IRC23:24
*** stevemar has quit IRC23:25
*** Daviey has quit IRC23:25
*** Daviey has joined #openstack-infra23:26
*** salv-orlando has joined #openstack-infra23:27
openstackgerritmin wang proposed openstack-infra/project-config: Set octavia gate for  neutron-lbaasv2 tempest test  https://review.openstack.org/21014123:30
*** jgriffith has quit IRC23:31
*** morganfainberg is now known as morgan_50323:31
*** yushiro has joined #openstack-infra23:31
yushirosdague, ping23:32
*** dangers is now known as dangers_away23:33
*** bknudson has joined #openstack-infra23:35
*** crc32 has quit IRC23:35
gmannjswarren: sure will look today.23:37
jswarrengmann: thanks23:37
gmannjswarren: np!23:37
*** sdake has quit IRC23:38
*** bharath has quit IRC23:38
*** camunoz_gone is now known as camunoz23:39
*** btully has joined #openstack-infra23:41
clarkbok I am back23:43
clarkbjeblair: will review those changes now and try to get to mtreinish's change23:43
*** sharikapongubala has quit IRC23:47
*** sharikapongubala has joined #openstack-infra23:49
*** BrandonL08 has quit IRC23:49
*** Sukhdev has joined #openstack-infra23:53
*** dannywil_ has joined #openstack-infra23:54
*** sharikapongubala has quit IRC23:54
clarkbmtreinish: just noting I am happy you chose to use pymysql as it makes updated the existing e-r install easier :)23:55
*** sharikapongubala has joined #openstack-infra23:56
*** puranamr has joined #openstack-infra23:56
*** dimtruck is now known as zz_dimtruck23:57
fungiclarkb: jeblair: not really around, approaching bedtime, but those two nodepool fixes lgtm. good find!23:57
clarkbbedtime? I know fungi better than that23:57
*** dannywilson has quit IRC23:57
*** ashleighfarnham has quit IRC23:57
mtreinishclarkb: heh, yep23:58
*** dannywil_ has quit IRC23:58
*** alexpilotti has quit IRC23:58
*** tsg has joined #openstack-infra23:59
*** dingyichen has joined #openstack-infra23:59
*** funzo has joined #openstack-infra23:59
*** achanda has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!