Monday, 2016-03-21

*** ajmiller has quit IRC00:00
*** ddieterly is now known as ddieterly[away]00:05
*** heyongli has quit IRC00:07
*** heyongli has joined #openstack-infra00:07
*** superdan is now known as dansmith00:08
*** baoli has joined #openstack-infra00:09
*** baoli has quit IRC00:10
*** zul has joined #openstack-infra00:11
*** baoli has joined #openstack-infra00:11
*** mylu has joined #openstack-infra00:12
*** amrith is now known as _amrith_00:12
*** Qiming has quit IRC00:14
*** baoli_ has joined #openstack-infra00:15
*** heyongli has quit IRC00:17
*** heyongli has joined #openstack-infra00:17
*** baoli has quit IRC00:18
*** _ari_ has joined #openstack-infra00:18
*** baoli_ has quit IRC00:19
*** Jeffrey4l__ has quit IRC00:19
*** baoli has joined #openstack-infra00:19
*** sflanigan has joined #openstack-infra00:19
*** sflanigan has joined #openstack-infra00:19
*** zul has quit IRC00:20
*** zul has joined #openstack-infra00:21
*** amotoki has joined #openstack-infra00:22
*** _amrith_ is now known as amrith00:23
*** sdake has quit IRC00:24
*** amotoki has quit IRC00:27
*** heyongli has quit IRC00:27
*** heyongli has joined #openstack-infra00:28
*** sdake has joined #openstack-infra00:29
*** mylu has quit IRC00:34
*** heyongli has quit IRC00:38
*** heyongli has joined #openstack-infra00:38
*** roxanaghe has quit IRC00:39
*** kzaitsev_mb has joined #openstack-infra00:41
*** baoli has quit IRC00:45
*** mylu has joined #openstack-infra00:46
openstackgerritEmilien Macchi proposed openstack-infra/project-config: enable voting for puppet-openstack-cookiecutter jobs  https://review.openstack.org/29502700:46
*** heyongli has quit IRC00:48
*** heyongli has joined #openstack-infra00:48
*** x00350071 has quit IRC00:54
*** yamamoto has joined #openstack-infra00:56
*** heyongli has quit IRC00:58
*** heyongli has joined #openstack-infra00:58
*** jiggalator has joined #openstack-infra01:01
*** zhurong has joined #openstack-infra01:02
*** ddieterly[away] has quit IRC01:05
*** baoli has joined #openstack-infra01:08
*** heyongli has quit IRC01:08
*** heyongli has joined #openstack-infra01:09
*** Sukhdev has joined #openstack-infra01:11
*** Qiming has joined #openstack-infra01:11
*** zz_dimtruck is now known as dimtruck01:13
*** kzaitsev_mb has quit IRC01:16
*** ddieterly has joined #openstack-infra01:17
*** heyongli has quit IRC01:19
*** heyongli has joined #openstack-infra01:19
*** dimtruck is now known as zz_dimtruck01:22
*** amotoki has joined #openstack-infra01:23
*** jiggalator has quit IRC01:25
*** akshai has quit IRC01:25
*** mylu has quit IRC01:25
*** mylu has joined #openstack-infra01:27
*** amotoki has quit IRC01:28
*** heyongli has quit IRC01:29
*** heyongli has joined #openstack-infra01:29
*** Liuqing has joined #openstack-infra01:30
*** yamamoto has quit IRC01:31
*** sdake has quit IRC01:36
*** heyongli has quit IRC01:39
*** heyongli has joined #openstack-infra01:39
*** _ari_ has quit IRC01:42
*** _ari_ has joined #openstack-infra01:42
*** thorst has quit IRC01:47
*** thorst has joined #openstack-infra01:48
*** heyongli has quit IRC01:49
*** heyongli has joined #openstack-infra01:50
openstackgerritMerged openstack-dev/pbr: Correct the indentation in the classifiers example  https://review.openstack.org/29450001:53
*** ajmiller has joined #openstack-infra01:54
*** thorst has quit IRC01:56
*** Liuqing has quit IRC01:57
*** ajmiller has quit IRC01:58
*** asettle has quit IRC01:59
*** heyongli has quit IRC02:00
*** asettle has joined #openstack-infra02:00
*** heyongli has joined #openstack-infra02:00
*** Daisy has joined #openstack-infra02:01
*** ddieterly has quit IRC02:02
*** bpokorny has joined #openstack-infra02:03
*** heyongli has quit IRC02:10
*** heyongli has joined #openstack-infra02:10
*** rhallisey has quit IRC02:11
*** kzaitsev_mb has joined #openstack-infra02:12
*** Daisy has quit IRC02:13
*** Daisy has joined #openstack-infra02:13
*** baoli has quit IRC02:15
*** baoli has joined #openstack-infra02:16
*** Daisy has quit IRC02:17
*** Daisy has joined #openstack-infra02:17
*** kzaitsev_mb has quit IRC02:18
*** zz_dimtruck is now known as dimtruck02:19
*** heyongli has quit IRC02:20
*** heyongli has joined #openstack-infra02:20
*** Daisy has quit IRC02:23
*** Daisy has joined #openstack-infra02:23
*** amotoki has joined #openstack-infra02:24
*** joehuang111 has joined #openstack-infra02:24
*** joehuang111 has quit IRC02:25
*** fedexo has joined #openstack-infra02:26
*** baoli has quit IRC02:28
*** amotoki has quit IRC02:28
*** heyongli has quit IRC02:30
*** heyongli has joined #openstack-infra02:31
*** yamamoto has joined #openstack-infra02:32
*** IlyaG has joined #openstack-infra02:38
*** baoli has joined #openstack-infra02:40
*** yamamoto has quit IRC02:41
*** heyongli has quit IRC02:41
*** heyongli has joined #openstack-infra02:41
*** IlyaG has quit IRC02:43
*** heyongli has quit IRC02:51
*** heyongli has joined #openstack-infra02:51
*** dimtruck is now known as zz_dimtruck02:51
*** thorst has joined #openstack-infra02:54
*** amrith is now known as _amrith_02:59
*** tphummel has quit IRC02:59
*** heyongli has quit IRC03:01
*** thorst has quit IRC03:01
*** heyongli has joined #openstack-infra03:01
*** binbincong has quit IRC03:02
*** baoli has quit IRC03:04
*** okay22many has joined #openstack-infra03:10
*** heyongli has quit IRC03:11
*** heyongli has joined #openstack-infra03:12
*** kzaitsev_mb has joined #openstack-infra03:13
*** binbincong has joined #openstack-infra03:15
*** kzaitsev_mb has quit IRC03:18
*** tdasilva has quit IRC03:18
*** zz_dimtruck is now known as dimtruck03:19
openstackgerritElizabeth K. Joseph proposed openstack-infra/system-config: Fix link to install_puppet.sh in Puppet docs  https://review.openstack.org/29507203:20
*** heyongli has quit IRC03:22
*** heyongli has joined #openstack-infra03:22
*** aviau has joined #openstack-infra03:23
*** amotoki has joined #openstack-infra03:25
*** tphummel has joined #openstack-infra03:25
*** mylu has quit IRC03:26
*** amotoki has quit IRC03:29
*** Sukhdev has quit IRC03:30
*** Sukhdev has joined #openstack-infra03:31
*** heyongli has quit IRC03:32
*** heyongli has joined #openstack-infra03:32
*** Kevin_Zhangsen has joined #openstack-infra03:34
*** Daisy_ has joined #openstack-infra03:35
*** yamamoto has joined #openstack-infra03:36
*** Kevin_Zhangsen has quit IRC03:36
*** Kevin_Zhangsen has joined #openstack-infra03:37
*** Daisy has quit IRC03:37
*** IlyaG has joined #openstack-infra03:41
*** jswarren has quit IRC03:41
*** gildub has joined #openstack-infra03:41
*** ianychoi has quit IRC03:41
*** jswarren has joined #openstack-infra03:41
*** ianychoi has joined #openstack-infra03:41
*** heyongli has quit IRC03:42
*** heyongli has joined #openstack-infra03:42
*** IlyaG has quit IRC03:44
*** mylu has joined #openstack-infra03:45
openstackgerritMerged openstack-infra/system-config: Fix link to install_puppet.sh in Puppet docs  https://review.openstack.org/29507203:46
*** sridhar_ram has joined #openstack-infra03:49
*** Daisy_ has quit IRC03:49
*** binbincong has quit IRC03:50
*** Daisy has joined #openstack-infra03:50
*** Kevin_Zhangsen has quit IRC03:50
*** heyongli has quit IRC03:52
*** heyongli has joined #openstack-infra03:53
*** binbincong has joined #openstack-infra03:54
*** Daisy has quit IRC03:57
*** thorst has joined #openstack-infra03:59
*** tdasilva has joined #openstack-infra04:02
*** links has joined #openstack-infra04:03
*** heyongli has quit IRC04:03
*** heyongli has joined #openstack-infra04:03
*** thorst has quit IRC04:07
*** heyongli has quit IRC04:13
*** yamamoto has quit IRC04:13
*** heyongli has joined #openstack-infra04:13
*** IlyaG has joined #openstack-infra04:19
*** asettle has quit IRC04:21
*** asettle has joined #openstack-infra04:22
*** dimtruck is now known as zz_dimtruck04:23
*** heyongli has quit IRC04:23
*** heyongli has joined #openstack-infra04:23
*** treaki_ has joined #openstack-infra04:24
*** roxanaghe has joined #openstack-infra04:25
*** amotoki has joined #openstack-infra04:25
*** treaki__ has quit IRC04:28
*** amotoki has quit IRC04:30
*** heyongli has quit IRC04:33
*** heyongli has joined #openstack-infra04:34
*** yamamoto has joined #openstack-infra04:34
*** IlyaG has quit IRC04:35
*** yamamoto has quit IRC04:37
*** roxanaghe has quit IRC04:38
*** yamamoto has joined #openstack-infra04:40
*** bpokorny has quit IRC04:41
*** yamamoto has quit IRC04:42
*** heyongli has quit IRC04:44
*** heyongli has joined #openstack-infra04:44
*** roxanaghe has joined #openstack-infra04:51
*** mylu has quit IRC04:53
*** Daisy has joined #openstack-infra04:54
*** Jeffrey4l__ has joined #openstack-infra04:54
*** heyongli has quit IRC04:54
*** heyongli has joined #openstack-infra04:54
*** sridhar_ram has quit IRC04:56
*** sputnik13 has joined #openstack-infra04:56
*** harlowja_at_home has quit IRC04:57
*** Daisy has quit IRC05:03
*** Daisy has joined #openstack-infra05:03
*** heyongli has quit IRC05:04
*** heyongli has joined #openstack-infra05:04
*** thorst has joined #openstack-infra05:04
*** Daisy has quit IRC05:08
*** _fortis has joined #openstack-infra05:08
*** sputnik13 has quit IRC05:08
*** thorst has quit IRC05:11
*** jcooley_ has joined #openstack-infra05:11
*** pcm_ has quit IRC05:13
*** christiana has quit IRC05:13
*** jgriffith has quit IRC05:13
*** jgriffith has joined #openstack-infra05:14
*** jgriffith is now known as Guest491605:14
*** pc_m has joined #openstack-infra05:14
*** heyongli has quit IRC05:14
*** kzaitsev_mb has joined #openstack-infra05:15
*** heyongli has joined #openstack-infra05:15
*** kzaitsev_mb has quit IRC05:20
*** maishsk has quit IRC05:22
*** heyongli has quit IRC05:25
*** heyongli has joined #openstack-infra05:25
*** amotoki has joined #openstack-infra05:26
*** yamamoto has joined #openstack-infra05:29
*** sdake has joined #openstack-infra05:29
*** maishsk has joined #openstack-infra05:29
*** maishsk has quit IRC05:30
*** amotoki has quit IRC05:31
*** heyongli has quit IRC05:35
*** heyongli has joined #openstack-infra05:35
*** tphummel has quit IRC05:36
*** sdake has quit IRC05:38
*** roxanaghe has quit IRC05:39
AJaeger_amrith_: I added a lengthy review comment. Does that help? If not, let's discuss later...05:40
*** roxanaghe has joined #openstack-infra05:41
*** sdake has joined #openstack-infra05:41
*** roxanaghe has quit IRC05:45
*** heyongli has quit IRC05:45
*** heyongli has joined #openstack-infra05:45
*** Liuqing has joined #openstack-infra05:51
*** yamamoto has quit IRC05:52
*** Jeffrey4l__ has quit IRC05:53
*** sdake has quit IRC05:55
*** heyongli has quit IRC05:55
*** heyongli has joined #openstack-infra05:56
*** nadya has joined #openstack-infra05:56
*** nadya has quit IRC05:56
*** yamamoto has joined #openstack-infra05:59
*** pcaruana has quit IRC06:02
kashyapAny admins with operator privileges -- there's some spam on #openstack-meeting, please kick the nick out.06:05
*** heyongli has quit IRC06:06
*** heyongli has joined #openstack-infra06:06
*** jaosorior has joined #openstack-infra06:06
*** jaosorior has quit IRC06:07
*** jaosorior has joined #openstack-infra06:08
kashyapAJaeger: ^  (I know you're in CET)06:08
pleia2kashyap: thanks, sorted06:08
*** thorst has joined #openstack-infra06:08
kashyappleia2: Ah, thanks!  Australia tz comes to rescue, I suppose :-)06:09
pleia2I'm usually in California ;) Singapore at the moment post FOSSASIA06:09
kashyapAh, right.  Heard about it06:09
* kashyap is in CET normally, now in IND for a few weeks.06:09
tchaypohow's the weather pleia2?06:10
tchayposick of the humidity yet?06:10
kashyapHeh06:10
pleia2tchaypo: lovely place, too humid+hot, I am ready to go home :) (in the airport now)06:10
openstackgerritMerged openstack-infra/project-config: Added repo for fuel-plugin-detach-haproxy project  https://review.openstack.org/29468306:10
*** abregman has joined #openstack-infra06:15
*** thorst has quit IRC06:16
*** kzaitsev_mb has joined #openstack-infra06:16
*** heyongli has quit IRC06:16
*** heyongli has joined #openstack-infra06:16
*** openstackgerrit has quit IRC06:17
*** openstackgerrit_ is now known as openstackgerrit06:17
*** openstackgerrit_ has joined #openstack-infra06:18
*** openstackgerrit_ is now known as openstackgerrit06:19
*** openstackgerrit_ has joined #openstack-infra06:19
*** Sukhdev_ has joined #openstack-infra06:20
*** rcernin has joined #openstack-infra06:20
*** kzaitsev_mb has quit IRC06:21
*** christiana has joined #openstack-infra06:22
*** yamamoto has quit IRC06:22
*** fedexo has quit IRC06:23
*** Sukhdev has quit IRC06:23
*** heyongli has quit IRC06:26
*** heyongli has joined #openstack-infra06:26
*** amotoki has joined #openstack-infra06:27
*** ianychoi has quit IRC06:27
*** achanda has quit IRC06:31
*** amotoki has quit IRC06:31
*** achanda has joined #openstack-infra06:34
*** achanda has quit IRC06:36
*** Jeffrey4l__ has joined #openstack-infra06:36
*** heyongli has quit IRC06:36
*** heyongli has joined #openstack-infra06:37
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/28884906:37
abregmanhallo06:38
*** hashar has joined #openstack-infra06:39
*** hashar is now known as hasharAway06:40
*** Daisy has joined #openstack-infra06:41
*** yamamoto has joined #openstack-infra06:41
*** kevin_zhang has joined #openstack-infra06:42
*** roxanaghe has joined #openstack-infra06:42
*** kevin_zhang has quit IRC06:42
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Add networking-generic-switch-dsvm job  https://review.openstack.org/29412806:43
*** Jeffrey4l__ has quit IRC06:45
*** asettle has quit IRC06:45
*** Daisy has quit IRC06:45
*** heyongli has quit IRC06:47
*** roxanaghe has quit IRC06:47
*** heyongli has joined #openstack-infra06:47
*** ihrachys has joined #openstack-infra06:48
*** ihrachys has quit IRC06:49
openstackgerritNisha Agarwal proposed openstack/diskimage-builder: Add psmisc to the packages for ironic-agent  https://review.openstack.org/28936406:55
*** heyongli has quit IRC06:57
*** asettle has joined #openstack-infra06:57
*** heyongli has joined #openstack-infra06:57
*** oanson has joined #openstack-infra06:58
kashyapWas there any prior discussion/plans to make Jenkins auto-rebase patches when there are merge conflicts?06:59
oansonHello.06:59
oansonI'm running into the issue described here: http://openvswitch.org/pipermail/dev/2016-March/067987.html. I was wondering if anyone could help?06:59
oansonOVS fails to compile under kernel 3.13.0-83, Worked on kernel 3.13.0-79.06:59
*** asettle has quit IRC07:02
openstackgerritSteve Baker proposed openstack-infra/shade: Implement update_stack  https://review.openstack.org/29259307:03
openstackgerritSteve Baker proposed openstack-infra/shade: Wrap stack operations in a heat_exceptions  https://review.openstack.org/29259207:03
openstackgerritSteve Baker proposed openstack-infra/shade: delete_stack add wait argument  https://review.openstack.org/29262607:03
openstackgerritSteve Baker proposed openstack-infra/shade: Use event_utils.poll_for_events for stack polling  https://review.openstack.org/29259107:03
*** heyongli has quit IRC07:07
*** heyongli has joined #openstack-infra07:07
*** thorst has joined #openstack-infra07:08
openstackgerritIan Wienand proposed openstack/diskimage-builder: Add releasenotes  https://review.openstack.org/29219307:08
*** esikachev has joined #openstack-infra07:12
esikachevhi all! need merge git fetch https://review.openstack.org/openstack-infra/project-config refs/changes/64/293464/2 && git checkout FETCH_HEAD07:13
esikachevpardon) https://review.openstack.org/#/c/293464/07:14
*** thorst has quit IRC07:16
*** kzaitsev_mb has joined #openstack-infra07:17
*** heyongli has quit IRC07:17
*** heyongli has joined #openstack-infra07:18
*** esikachev has quit IRC07:19
*** esikachev has joined #openstack-infra07:20
*** kzaitsev_mb has quit IRC07:21
*** zeih has joined #openstack-infra07:22
*** asettle has joined #openstack-infra07:22
openstackgerritIan Wienand proposed openstack/diskimage-builder: Add releasenotes  https://review.openstack.org/29219307:24
*** dchen has quit IRC07:25
*** |-paul-| has joined #openstack-infra07:25
esikachevthanks!07:26
*** maishsk has joined #openstack-infra07:26
*** amotoki has joined #openstack-infra07:28
*** heyongli has quit IRC07:28
*** heyongli has joined #openstack-infra07:28
*** gildub has quit IRC07:28
*** scheuran has joined #openstack-infra07:29
openstackgerritIan Wienand proposed openstack-infra/project-config: Add release notes jobs for diskimage-builder  https://review.openstack.org/29510007:29
*** amotoki has quit IRC07:32
openstackgerritMerged openstack-infra/project-config: Enable voting for gate jobs in sahara-tests  https://review.openstack.org/29346407:32
AJaegerpleia2: safe travel home!07:32
*** hasharAway has quit IRC07:35
*** achanda has joined #openstack-infra07:36
*** flepied has joined #openstack-infra07:38
*** heyongli has quit IRC07:38
*** heyongli has joined #openstack-infra07:38
*** Daisy has joined #openstack-infra07:40
*** achanda has quit IRC07:42
*** Swanson has quit IRC07:42
yolandagood morning07:42
*** ihrachys has joined #openstack-infra07:42
*** ihrachys has quit IRC07:42
*** Swanson has joined #openstack-infra07:42
*** Daisy has quit IRC07:44
*** andymaier has joined #openstack-infra07:44
AJaegermorning, yolanda!07:45
AJaegeryolanda: for all my "Use db setup for python jobs" - please do not approve yet, I'd like PTLs have the chance to review.07:46
yolandaok thanks07:46
*** HeOS has joined #openstack-infra07:47
oansonHello.07:47
*** heyongli has quit IRC07:48
oansonI'm running into the issue described here: http://openvswitch.org/pipermail/dev/2016-March/067987.html. I was wondering if anyone could help?07:48
*** heyongli has joined #openstack-infra07:48
*** daemontool has joined #openstack-infra07:50
*** tesseract has joined #openstack-infra07:50
*** vsaienko has quit IRC07:51
*** tesseract is now known as Guest944107:51
*** jtomasek has joined #openstack-infra07:52
*** zeih has quit IRC07:56
*** maishsk has quit IRC07:57
*** zeih has joined #openstack-infra07:57
*** dtardivel has joined #openstack-infra07:57
*** lezbar__ has quit IRC07:57
*** lezbar has joined #openstack-infra07:58
*** maishsk has joined #openstack-infra07:58
*** heyongli has quit IRC07:58
abregmanhmm so how exactly am I getting this merged: https://review.openstack.org/#/c/29478107:59
*** heyongli has joined #openstack-infra07:59
openstackgerritMerged openstack/diskimage-builder: Add testing for the Gentoo element  https://review.openstack.org/29089407:59
openstackgerritkevin proposed openstack-infra/project-config: New Project - jacket  https://review.openstack.org/26350008:00
*** Sukhdev_ has quit IRC08:00
*** akaszuba has joined #openstack-infra08:00
*** yamamoto has quit IRC08:01
*** mikelk has joined #openstack-infra08:01
*** yamamoto has joined #openstack-infra08:02
*** zhurong has quit IRC08:04
*** zhurong has joined #openstack-infra08:05
AJaegerabregman: every review needs two +2s - and the second reviewer giving a +2 will normally approve...08:06
abregmanAJaeger: oh k. thanks :)08:06
AJaegerabregman: we have many projects and few reviewers, so it might take some time until a second person approves.08:06
abregman:O08:06
abregman:(08:06
AJaegerBest is always to help with reviewing other changes as well, a +1 with comments or a -1 with suggestions to improve help core reviewers a lot...08:07
*** zeih has quit IRC08:08
abregmanack. will do :)08:08
*** heyongli has quit IRC08:09
*** heyongli has joined #openstack-infra08:09
*** ifarkas has joined #openstack-infra08:11
openstackgerritMerged openstack-infra/project-config: Trigger python27 job for packetary-specs  https://review.openstack.org/29458008:12
*** matrohon has joined #openstack-infra08:14
ianwabregman: seems like you could write a style-checker and integrate that into the job pretty easily for that.  see project-config:tools/* where we have a number of similar ideas checking different bits of yaml08:14
*** thorst has joined #openstack-infra08:14
openstackgerritMerged openstack-infra/infra-ansible: Fix run_puppet_on_*.yml files  https://review.openstack.org/29501308:15
abregmanianw: thanks. I'll look into it.08:16
abregmanyolanda: thanks for the reviewing :)08:17
yolandanp08:17
*** yamamoto has quit IRC08:17
*** heyongli has quit IRC08:19
*** heyongli has joined #openstack-infra08:19
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-cgit: Order of the classes parameters is refactored  https://review.openstack.org/29511708:20
*** thorst has quit IRC08:21
*** Daisy has joined #openstack-infra08:21
*** ramishra has quit IRC08:23
*** Daisy has quit IRC08:23
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-diskimage_builder: Order of the class parameters is refactored  https://review.openstack.org/29512008:25
*** nadya has joined #openstack-infra08:25
*** amotoki has joined #openstack-infra08:28
*** heyongli has quit IRC08:29
*** heyongli has joined #openstack-infra08:30
*** roxanaghe has joined #openstack-infra08:30
*** abregman_ has joined #openstack-infra08:32
*** amotoki has quit IRC08:33
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-drupal: Order of the classes parameters is refactored  https://review.openstack.org/29513008:34
*** roxanaghe has quit IRC08:34
*** abregman has quit IRC08:34
*** abregman_ is now known as abregman08:35
*** kzaitsev_mb has joined #openstack-infra08:35
*** amotoki has joined #openstack-infra08:35
*** rcarrillocruz has quit IRC08:36
*** hashar has joined #openstack-infra08:37
*** zeih has joined #openstack-infra08:38
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-elastic_recheck: Order of the classes parameters is refactored  https://review.openstack.org/29513208:38
*** dingyichen has quit IRC08:38
*** heyongli has quit IRC08:39
*** heyongli has joined #openstack-infra08:40
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-elasticsearch: Order of the class parameters is refactored  https://review.openstack.org/29513408:40
*** arxcruz has joined #openstack-infra08:40
*** permalac has joined #openstack-infra08:40
*** kzaitsev_mb has quit IRC08:41
*** esikachev has quit IRC08:41
*** dizquierdo has joined #openstack-infra08:43
*** vincentll has joined #openstack-infra08:44
openstackgerritJan Hruban proposed openstack-infra/gearman-plugin: Fix permission problem when finding builds  https://review.openstack.org/23403608:45
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-etherpad_lite: Order of the classes parameters is refactored  https://review.openstack.org/29513808:46
*** maishsk has quit IRC08:48
*** maishsk has joined #openstack-infra08:49
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-exim: Order of the class parameters is refactored  https://review.openstack.org/29513908:49
*** heyongli has quit IRC08:50
*** heyongli has joined #openstack-infra08:50
openstackgerritkevin proposed openstack-infra/project-config: New Project - jacket  https://review.openstack.org/26350008:50
*** Hal1 has joined #openstack-infra08:50
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-gerritbot: Order of the class parameters is refactored  https://review.openstack.org/29514008:51
matrohonfungi, hi08:52
*** vgridnev has joined #openstack-infra08:52
*** maishsk has quit IRC08:53
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-github: Order of the class parameters is refactored  https://review.openstack.org/29514208:53
*** yaume has joined #openstack-infra08:54
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-grafyaml: Order of the class parameters is refactored  https://review.openstack.org/29514308:55
*** daemontool has quit IRC08:55
*** Jeffrey4l__ has joined #openstack-infra08:55
*** jlanoux has joined #openstack-infra08:55
*** jlanoux has quit IRC08:56
*** jlanoux has joined #openstack-infra08:57
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-graphite: Order of the class parameters is refactored  https://review.openstack.org/29514408:57
*** ramishra has joined #openstack-infra08:58
*** okay22many has quit IRC08:59
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-hound: Order of the class parameters is refactored  https://review.openstack.org/29514608:59
*** heyongli has quit IRC09:00
*** achanda has joined #openstack-infra09:00
*** heyongli has joined #openstack-infra09:00
*** achanda has quit IRC09:01
*** shardy has joined #openstack-infra09:02
*** rossella_s has quit IRC09:03
*** rossella_s has joined #openstack-infra09:04
*** daemontool has joined #openstack-infra09:04
*** esikachev has joined #openstack-infra09:05
*** permalac has quit IRC09:05
*** permalac has joined #openstack-infra09:05
*** shardy_ has joined #openstack-infra09:06
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-etherpad_lite: Order of the classes parameters is refactored  https://review.openstack.org/29513809:08
*** heyongli has quit IRC09:10
*** heyongli has joined #openstack-infra09:10
*** maishsk has joined #openstack-infra09:12
*** mrmartin has joined #openstack-infra09:13
*** kingia_ has quit IRC09:13
*** Daisy has joined #openstack-infra09:14
*** derekh has joined #openstack-infra09:15
*** abregman is now known as abregma09:16
*** yamamoto has joined #openstack-infra09:18
*** Daisy has quit IRC09:18
*** thorst has joined #openstack-infra09:19
*** heyongli has quit IRC09:20
*** _degorenko|afk is now known as degorenko09:20
*** heyongli has joined #openstack-infra09:21
*** Jeffrey4l__ has quit IRC09:21
*** dkaigarodsev has quit IRC09:21
*** Guest42967 is now known as BobBall09:21
*** pasquier-s has quit IRC09:22
*** gsilvis has quit IRC09:23
*** SergK has quit IRC09:23
*** gsilvis has joined #openstack-infra09:23
*** SergK has joined #openstack-infra09:24
*** dkaigarodsev has joined #openstack-infra09:24
*** pasquier-s has joined #openstack-infra09:25
*** thorst has quit IRC09:25
*** yamamoto has quit IRC09:25
*** permalac has quit IRC09:26
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-httpd: Order of the classes parameters is refactored  https://review.openstack.org/29515709:27
*** heyongli has quit IRC09:31
*** heyongli has joined #openstack-infra09:31
*** placko_ has joined #openstack-infra09:32
*** samuelBartel has joined #openstack-infra09:33
*** amotoki has quit IRC09:35
openstackgerritSergey Belous proposed openstack-infra/project-config: Improve experimental tempest-dsvm-nova-os-vif job  https://review.openstack.org/29516109:40
*** zeih has quit IRC09:40
*** sbelous_ has joined #openstack-infra09:40
*** heyongli has quit IRC09:41
*** asettle has quit IRC09:41
*** heyongli has joined #openstack-infra09:41
*** Qiming has quit IRC09:42
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-httpd: Order of the classes parameters is refactored  https://review.openstack.org/29515709:45
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-infracloud: Order of the classes parameters is refactored  https://review.openstack.org/29516209:46
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-iptables: Intendataion of the class parameters is refactored  https://review.openstack.org/29516409:48
*** zhurong has quit IRC09:49
*** Liuqing has quit IRC09:51
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-jeepyb: Order of the classes parameters is refactored  https://review.openstack.org/29516609:51
*** heyongli has quit IRC09:51
openstackgerritOleksandr Berezovskyi proposed openstack-infra/project-config: Add bareon-image-elements project  https://review.openstack.org/29516709:51
*** heyongli has joined #openstack-infra09:51
*** e0ne has joined #openstack-infra09:52
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add elements related to image builds on system-config  https://review.openstack.org/28378809:52
*** asettle has joined #openstack-infra09:53
*** sdague has joined #openstack-infra09:54
*** esikachev has quit IRC09:54
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add elements related to image builds on system-config  https://review.openstack.org/28378809:54
openstackgerrityolanda.robla proposed openstack-infra/project-config: Start using infra element instead of puppet  https://review.openstack.org/29516809:56
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-jenkins: Order of the classes parameters is refactored  https://review.openstack.org/29517109:59
*** |-paul-| has quit IRC10:01
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-kerberos: Order of the classes parameters is refactored  https://review.openstack.org/29517210:01
*** esikachev has joined #openstack-infra10:01
*** heyongli has quit IRC10:01
*** heyongli has joined #openstack-infra10:02
openstackgerritOleksandr Berezovskyi proposed openstack-infra/project-config: Add bareon-image-elements project  https://review.openstack.org/29516710:02
*** zeih has joined #openstack-infra10:03
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-kibana: Order of the classes parameters is refactored  https://review.openstack.org/29517410:03
*** dizquierdo has quit IRC10:03
*** _skolekonov is now known as skolekonov10:05
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add elements related to image builds on system-config  https://review.openstack.org/28378810:06
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-lodgeit: Order of the classes parameters is refactored  https://review.openstack.org/29517710:06
openstackgerrityolanda.robla proposed openstack-infra/project-config: Start using infra element instead of puppet  https://review.openstack.org/29516810:06
*** zeih has quit IRC10:07
*** vsaienko has joined #openstack-infra10:09
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-logrotate: Order of the class parameters is refactored  https://review.openstack.org/29517910:09
*** tpsilva has joined #openstack-infra10:09
*** abregman has joined #openstack-infra10:09
*** abregma has quit IRC10:10
ogelbukh_ttx: thanks for the governance patch bump10:10
*** heyongli has quit IRC10:12
ttxogelbukh_: you're welcome!10:12
*** heyongli has joined #openstack-infra10:12
openstackgerritArie Bregman proposed openstack-infra/infra-ansible: Add keypairs to infra_config sample and option to use key file  https://review.openstack.org/29479110:13
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-logstash: Order of the classes parameters is refactored  https://review.openstack.org/29518010:13
*** electrofelix has joined #openstack-infra10:13
*** sputnik13 has joined #openstack-infra10:14
*** jaosorior has quit IRC10:15
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-mailman: Placement of the class parameter is refactored  https://review.openstack.org/29518110:15
*** jaosorior has joined #openstack-infra10:15
*** tpsilva has quit IRC10:17
*** |-paul-| has joined #openstack-infra10:17
*** tpsilva has joined #openstack-infra10:17
*** roxanaghe has joined #openstack-infra10:18
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-mediawiki: Order of the classes parameters is refactored  https://review.openstack.org/29518210:19
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client  https://review.openstack.org/25534910:21
openstackgerritOleksandr Berezovskyi proposed openstack-infra/project-config: Add bareon-image-elements project  https://review.openstack.org/29516710:21
*** kzaitsev_mb has joined #openstack-infra10:22
*** notnownikki has joined #openstack-infra10:22
*** heyongli has quit IRC10:22
*** roxanaghe has quit IRC10:22
*** heyongli has joined #openstack-infra10:22
*** thorst has joined #openstack-infra10:23
*** jordanP has joined #openstack-infra10:23
*** sc68cal has joined #openstack-infra10:26
vsaienkoHello infra team!10:27
vsaienkocould you please review https://review.openstack.org/#/c/252814/10:28
*** zeih has joined #openstack-infra10:30
*** thorst has quit IRC10:30
*** cdent has joined #openstack-infra10:31
*** heyongli has quit IRC10:32
*** heyongli has joined #openstack-infra10:32
*** Nikolay_St has joined #openstack-infra10:34
*** Qiming has joined #openstack-infra10:35
AJaegervsaienko: we have 70+ repos and not everybody is core everywhere. Please tell us a bit about your change so that the right people will review it.10:35
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client  https://review.openstack.org/25534910:36
*** sc68cal has quit IRC10:38
*** ZZelle has joined #openstack-infra10:38
*** gnuoy_ has joined #openstack-infra10:42
*** heyongli has quit IRC10:42
*** heyongli has joined #openstack-infra10:43
Nikolay_StAJaeger: hi. can you point me and kzaitsev_mb to some documents/email threads related to this patch: https://review.openstack.org/#/c/295034/10:43
openstackgerritOleksandr Berezovskyi proposed openstack-infra/project-config: Add bareon-image-elements project  https://review.openstack.org/29516710:44
*** okay22many has joined #openstack-infra10:44
*** sc68cal has joined #openstack-infra10:45
*** gnuoy_ has quit IRC10:47
*** zeih has quit IRC10:48
*** dims has joined #openstack-infra10:49
*** electrofelix has quit IRC10:50
*** yamahata has quit IRC10:51
AJaegerNikolay_St: this is part of the effort mentioned in http://lists.openstack.org/pipermail/openstack-dev/2016-March/088832.html10:51
openstackgerritAparna proposed openstack/diskimage-builder: Support to add self-signed certificate in ironic-agent  https://review.openstack.org/29520310:51
AJaegerNikolay_St, kzaitsev_mb: If your repo uses a MySQL or PostgreSQL database during *unit* tests, this is the proper change. Nothing needs to be changed on your side, you will only notice that the job names are different, we continue to call "tox -e py27" etc.10:52
AJaegerNikolay_St, kzaitsev_mb: Does that help?10:52
*** heyongli has quit IRC10:53
*** heyongli has joined #openstack-infra10:53
openstackgerritRob Cresswell proposed openstack-infra/irc-meetings: Fix Horizon meeting times  https://review.openstack.org/29520510:55
*** ociuhandu has quit IRC10:56
robcresswellSomewhat silly issue: The Horizon meeting slots appear to have been reversed. If anyone could confirm the desync between our wiki and the ical entries, that would be great https://review.openstack.org/#/c/295205/10:57
*** jlanoux_ has joined #openstack-infra10:57
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client  https://review.openstack.org/25534910:57
AJaegerrobcresswell: when did that change happen? You know about the tow odd weeks over the New Year?10:58
*** zeih has joined #openstack-infra10:58
Nikolay_StAJaeger: the effort was clear from the comments on the commit, we've just interested in background :)11:00
*** sputnik13 has quit IRC11:00
AJaegerNikolay_St: Background ;)11:00
AJaegerWe do not want to setup databases in unit tests for those projects that don't need it.11:00
*** jlanoux has quit IRC11:00
AJaegerOnce all this is done, it will allow projects to have their own "package lists" and specify what they need.11:00
robcresswellAJaeger: I'm unsure. I just pulled down the calendar today, and its opposite timeslots to the intended ones.11:01
*** yamamoto has joined #openstack-infra11:01
*** achanda has joined #openstack-infra11:01
*** achanda has quit IRC11:01
AJaegerRight now the "package list" needs to include SQL server packages just to satisfy the gates...11:01
AJaegerLunch time now, will followup later11:01
robcresswellAJaeger: As in the commit notes, it seems easier to update the cal than to tell everyone the meeting has changed time.11:01
robcresswellEnjoy lunch :)11:02
openstackgerritSergey Belous proposed openstack-infra/project-config: Improve experimental tempest-dsvm-nova-os-vif job  https://review.openstack.org/29516111:02
*** heyongli has quit IRC11:03
*** heyongli has joined #openstack-infra11:03
*** yamamoto has quit IRC11:04
openstackgerritMarton Kiss proposed openstack-infra/system-config: Add puppet-drush module to modules.env  https://review.openstack.org/29520811:05
*** maishsk has quit IRC11:05
vsaienkoAJaeger:  this change https://review.openstack.org/#/c/252814/ adds new job for Ironic/Neutron integration testing. It will be experimental job for now to verify that related patches to ironic/devstack are working and can pass CI. In future this job will be switched to voting mode.11:05
*** maishsk has joined #openstack-infra11:07
*** yamamoto has joined #openstack-infra11:10
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs  https://review.openstack.org/28944511:11
*** heyongli has quit IRC11:13
*** heyongli has joined #openstack-infra11:13
*** rhallisey has joined #openstack-infra11:14
*** Liuqing has joined #openstack-infra11:15
sbelous_AJaeger: hi, Andreas. Can you please help me with strange error from CI on this patch? https://review.openstack.org/#/c/295161/2 gate-project-config-layout failed with error "Job gate-tempest-dsvm-nova-os-vif not defined" — how can it be? :(11:19
*** electrofelix has joined #openstack-infra11:21
*** Hal1 has quit IRC11:21
*** ihrachys has joined #openstack-infra11:22
*** shardy has quit IRC11:23
*** shardy_ has quit IRC11:23
*** heyongli has quit IRC11:23
*** ricky1 has joined #openstack-infra11:23
*** heyongli has joined #openstack-infra11:24
ricky1AJaeger: heya, do you know by any chance of any project having multi-region jobs?11:24
*** binbincong has quit IRC11:25
ricky1like two devstack nodes, each one with running as independent regions11:26
ricky1?11:26
*** edmondsw has joined #openstack-infra11:27
*** shardy has joined #openstack-infra11:27
*** jaypipes has joined #openstack-infra11:28
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Generate fedora-atomic images using dib  https://review.openstack.org/28716711:31
*** dizquierdo has joined #openstack-infra11:31
*** tojuvone has quit IRC11:32
*** heyongli has quit IRC11:34
*** zeih has quit IRC11:34
AJaegerrobcresswell: your call how to update it, just wanted to point out one possible cause of this.11:34
*** heyongli has joined #openstack-infra11:34
AJaegerricky1: not aware of any of that. Best ask on #openstack-qa whether we do anything like that. We do have some multinode testing but I don't know whether it includes multi-region.11:35
ricky1k, thx11:35
AJaegervsaienko: I suggest you have neutron and ironic infra liasons review it as well. dtantsur|pto normally reviews all ironic changes - and dougwig is one of the neutron liasons11:36
AJaegersbelous_: let me check...11:36
robcresswellAJaeger: I didnt know about the new year issue, but think I'll go ahead with this course of action, and then just let people know on the mailer that the calendar entries have been fixed. My impression is most use their own entries, or the wiki, to plan their attendance.11:36
*** ldnunes has joined #openstack-infra11:37
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-jenkins: Order of the classes parameters is refactored  https://review.openstack.org/29517111:37
AJaegersbelous_: your change renames '{pipeline}-tempest-dsvm-nova-os-vif{job-suffix}' from 'gate-tempest-dsvm-nova-os-vif' to 'gate-tempest-dsvm-nova-os-vif-nv'11:37
openstackgerritMarton Kiss proposed openstack-infra/puppet-drupal: Refactor drush deployment  https://review.openstack.org/29523111:37
*** Hal1 has joined #openstack-infra11:37
vsaienko AJaeger: thank you, going to contact with them.11:38
AJaegerrobcresswell: and then you have the same problem next year again ;( See http://lists.openstack.org/pipermail/openstack-dev/2016-January/083216.html for background11:38
*** yamamoto has quit IRC11:39
* jroll hears ironic and reviews11:39
robcresswellAJaeger: Ah, I see... How do you recommend fixing it?11:40
AJaegerrobcresswell: your change might be the right one for now... I just surprised that this hits you three months into the year...11:40
jrollvsaienko: I'm on it11:40
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Generate fedora-atomic images using dib  https://review.openstack.org/28716711:40
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-infracloud: Order of the classes parameters is refactored  https://review.openstack.org/29516211:41
robcresswellAJaeger: I don't personally use the calendar entries, and had only just noticed it today.11:41
*** ociuhandu has joined #openstack-infra11:43
*** heyongli has quit IRC11:44
*** jlanoux_ has quit IRC11:44
*** heyongli has joined #openstack-infra11:44
*** cdent has quit IRC11:44
*** nadya_ has joined #openstack-infra11:47
*** gordc has joined #openstack-infra11:47
*** sbelous__ has joined #openstack-infra11:48
openstackgerritAntoine Musso proposed openstack-infra/zuul: Provide 'longname' to project templates  https://review.openstack.org/29523711:48
*** thorst has joined #openstack-infra11:48
*** sbelous_ has quit IRC11:48
*** nadya has quit IRC11:49
*** thorst_ has joined #openstack-infra11:49
sbelous__AJaeger: oops, got it, thank you, Andreas. but I can't understand where exactly I must rename job (add -nv suffix)  Should I add something like this again? https://review.openstack.org/#/c/286085/5/jenkins/jobs/devstack-gate.yaml11:50
jrollvsaienko: just a naming thing inline I'd like changed, otherwise lgtm11:52
*** thorst has quit IRC11:53
sbelous__AJaeger: sorry, I understand my mistake :) and I know what i should change11:54
*** heyongli has quit IRC11:54
vsaienkojroll: thanks for review, I'm going to update the patch.11:54
*** heyongli has joined #openstack-infra11:54
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Add mirror server  https://review.openstack.org/23841411:55
jrollvsaienko: thanks, ping me for a +1 when you do :)11:55
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Used the mirror server for the centos image and git repos  https://review.openstack.org/28525711:55
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Use instack-qcow2 from cache if available and appropriate  https://review.openstack.org/28525911:55
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Upload the instack qcow image after periodic jobs  https://review.openstack.org/28525811:55
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job  https://review.openstack.org/22978911:55
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: WIP: Add a OVB te-broker server  https://review.openstack.org/29524311:55
*** yamamoto has joined #openstack-infra11:56
openstackgerritSergey Belous proposed openstack-infra/project-config: Improve experimental tempest-dsvm-nova-os-vif job  https://review.openstack.org/29516111:56
*** Kiall has quit IRC11:56
AJaegerjroll: could you review https://review.openstack.org/294939 as well, please? That's a change for naming of jobs so that your unit tests have the database setup. AFAIU you need database setup during unit tests, correct?11:57
*** hashar has quit IRC11:57
jrollAJaeger: I don't think we do, other than a few migration tests11:57
jrollAJaeger: those are skipped when db isn't present, though I guess I'd like them running, so that's fine with me11:59
*** Kiall has joined #openstack-infra11:59
jroll(I could be totally wrong too)11:59
jroll+1'd11:59
*** esker has quit IRC12:00
AJaegerjroll: I could be wrong as well ;) That's why I asked an expert in ironic to double check ;). And I don't want to skip your tests, we should run them in the gate.12:00
jrollAJaeger: yeah, let me verify quickly12:01
*** oanson has quit IRC12:01
jrollI just spun up a box for devstack so I'll verify before I run stack.sh12:01
*** Liuqing has quit IRC12:01
*** yamamoto has quit IRC12:02
AJaegerjroll: I noticed you use OpportunisticTestCase - and that's an indicator for the behavior you descriped (database testing but skipped when not present).12:03
AJaegerjroll: thanks!12:03
*** zeih has joined #openstack-infra12:03
*** Liuqing has joined #openstack-infra12:03
*** claudiub has joined #openstack-infra12:04
*** heyongli has quit IRC12:04
*** heyongli has joined #openstack-infra12:05
*** esikachev has quit IRC12:05
*** roxanaghe has joined #openstack-infra12:06
*** cdent has joined #openstack-infra12:06
openstackgerritMerged openstack-infra/storyboard-webclient: Make task notes visible for logged-out users  https://review.openstack.org/29295712:07
*** beisner has quit IRC12:07
*** zeih has quit IRC12:07
*** zeih has joined #openstack-infra12:07
*** beisner has joined #openstack-infra12:08
openstackgerritGraham Hayes proposed openstack-infra/project-config: Create 'Review-Priority' for designate repos  https://review.openstack.org/29525312:10
*** roxanaghe has quit IRC12:10
*** aysyd has joined #openstack-infra12:11
*** Daisy has joined #openstack-infra12:11
*** dprince has joined #openstack-infra12:12
anteayaabregman: http://paste.openstack.org/show/491248/12:13
anteayaabregman: did you get the paste I left for you on my friday night (which might be saturday in the logs)?12:14
anteayaI have to go now to get ready for a breakfast date, I will be back by 15:00 utc12:14
anteayaabregman: you are doing good work, keep it up12:14
*** heyongli has quit IRC12:15
anteayaabregman: I will keep reply to your items and questions as I can in small pieces12:15
*** heyongli has joined #openstack-infra12:15
anteayas/reply/replying12:15
Shrewsmordred: at some point, i'll find the energy to go through the truck load of shade patches you dumped on me this weekend12:16
*** weshay has joined #openstack-infra12:17
*** abregman has quit IRC12:18
AJaegeranteaya, yolanda, fungi : We can merge https://review.openstack.org/294939 for renaming the python tests thanks to jroll's confirmation.12:19
openstackgerritAparna proposed openstack/diskimage-builder: Support to add self-signed certificate in ironic-agent  https://review.openstack.org/29520312:19
AJaegerfungi, for the python27 renames: I did not create -like I did for nova - a two step process to first introduce the job and then use it. Is that something you prefer me doing for everything? Or should we schedule the merge for "idle" times?12:20
*** rfolco has joined #openstack-infra12:21
*** baoli has joined #openstack-infra12:22
openstackgerritIgor Belikov proposed openstack-infra/project-config: Add experimental puppet-apply job for debian-jessie  https://review.openstack.org/29376712:22
*** nadya_ has quit IRC12:22
openstackgerritIgor Belikov proposed openstack-infra/project-config: Add experimental debian-jessie job for bindep-fallback  https://review.openstack.org/29376912:22
*** Liuqing has quit IRC12:23
*** baoli has quit IRC12:24
*** Liuqing has joined #openstack-infra12:24
*** zeih has quit IRC12:25
*** heyongli has quit IRC12:25
openstackgerritSrushti Gadadare proposed openstack-dev/hacking: Add check for mutable objects as method arguments  https://review.openstack.org/29338612:25
*** heyongli has joined #openstack-infra12:25
*** sfinucan has joined #openstack-infra12:25
*** jpeeler has joined #openstack-infra12:26
*** Liuqing has quit IRC12:27
*** rguillebert has joined #openstack-infra12:27
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Adds ironic neutron integration testing  https://review.openstack.org/25281412:28
*** _amrith_ is now known as amrith12:28
*** |-paul-| has quit IRC12:30
*** annegentle has joined #openstack-infra12:31
*** abregman has joined #openstack-infra12:31
openstackgerritMurad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo  https://review.openstack.org/29526012:32
*** baoli has joined #openstack-infra12:33
*** woodster_ has joined #openstack-infra12:34
*** heyongli has quit IRC12:35
*** heyongli has joined #openstack-infra12:35
*** baoli_ has joined #openstack-infra12:37
*** oanson has joined #openstack-infra12:38
*** nadya has joined #openstack-infra12:39
*** baoli has quit IRC12:40
*** rlandy has joined #openstack-infra12:40
*** annegentle has quit IRC12:40
*** kzaitsev_mb has quit IRC12:41
*** annegentle has joined #openstack-infra12:41
*** annegentle has quit IRC12:42
*** annegentle has joined #openstack-infra12:42
mordredShrews: sorry man12:43
mordredShrews: they're good though, I promise12:43
mordredShrews: also, the 2 patches between your patch and the last patch in the series are only broken out for reviewing - I intend to squash the last three so that we don't have to fix mocks that we're removing in the next patch12:43
*** cbader has joined #openstack-infra12:45
Shrewsmordred: ack. that was a good catch, though, about bypassing the task manager. that never occurred to me12:45
*** heyongli has quit IRC12:45
*** heyongli has joined #openstack-infra12:46
Shrewsmordred: mostly b/c i was in such disbelief about the api weirdness12:46
mordredShrews: the api weirdensss is baffling12:46
Shrews(for the second time)12:46
mordredShrews: it might even be as weird as the glance task thing12:46
jbernard_sdague: looks like ceph's spike is has returned to normal so far, but im keeping any eye on it12:46
*** doug-fis_ has joined #openstack-infra12:47
vsaienkojroll: could you please have a look at new patchset https://review.openstack.org/#/c/252814/12:48
sdaguejbernard_: cool12:48
sdaguejbernard_: what's left in the list of things to do before we make that voting?12:49
jrollvsaienko: sure, thanks12:49
*** doug-fish has quit IRC12:50
jrollvsaienko: +112:50
jbernard_sdague: i think im just waiting for the package mirror12:50
jbernard_which then requires a plugin update to make use of it12:51
jbernard_and then remove the '-nv' from the job name12:51
sdaguejbernard_: ok, cool, do we have an eta on the mirror?12:51
mordredwhich mirror?12:51
jbernard_sdague: i dont, but i know it's in clarkb 's queue12:51
mordredah. the ceph mirror12:51
*** snikitin has quit IRC12:55
*** heyongli has quit IRC12:56
*** Daisy has quit IRC12:56
*** heyongli has joined #openstack-infra12:56
*** Daisy has joined #openstack-infra12:56
*** bhunter_ has joined #openstack-infra12:57
*** _ari_ has joined #openstack-infra12:58
*** placko_ has quit IRC12:59
*** Daisy has quit IRC13:01
*** rossella_s has quit IRC13:03
*** abregman has quit IRC13:03
*** rossella_s has joined #openstack-infra13:03
*** amitgandhinz has joined #openstack-infra13:04
pleia2anteaya: anyone have an eye on disabling IBM-GPFS-CINDER-CI? (see -infra list if you haven't already)13:05
*** heyongli has quit IRC13:06
*** heyongli has joined #openstack-infra13:06
*** hashar has joined #openstack-infra13:08
BobBallTheoretically, can we have two nodepools and two jenkins connecting to the same zuul, just providing different nodes to run different jobs in different locations? :)13:10
*** nt has quit IRC13:12
*** nadya has quit IRC13:12
*** mriedem has joined #openstack-infra13:13
*** kgiusti has joined #openstack-infra13:13
*** asettle has quit IRC13:14
*** eharney has joined #openstack-infra13:14
*** heyongli has quit IRC13:16
*** heyongli has joined #openstack-infra13:16
hasharBobBall: Nodepool looks at the Zuul gearman queues to accommodate the pool of instances. So if you have say 5 instances but 20 jobs queued, Nodepool will spawn additional instances.  I guess it is a non issue with different labels13:17
*** annegentle has quit IRC13:18
*** yamamoto has joined #openstack-infra13:18
*** annegentle has joined #openstack-infra13:18
*** lucasagomes has quit IRC13:19
*** abregman has joined #openstack-infra13:19
BobBallhashar: Yeah; but I'd like to effectively host two different CI systems (short term they must run different branches of nodepool) but with the same gerrit user (which means the same zuul instance).  I'm trying to understand if this is a fools errand or should be possible.13:19
hasharBobBall: I dont think Nodepool has any interactions with Gerrit13:20
BobBallcorrect; zuul does.13:20
*** nt has joined #openstack-infra13:21
hasharso you could have two Nodepools each managing a different jenkins label like   label-pool1 and label-pool213:21
hashareach Nodepool will query the Gearman servers for jobs queued for each label  and only accommodate their pool size based on the label13:22
*** kzaitsev_mb has joined #openstack-infra13:22
hasharso if  label-pool1  get overcrowded, the Nodepool  instance tied to the other label would not notice13:22
*** lucasagomes has joined #openstack-infra13:24
BobBallOK so you believe it should be possible.13:24
*** heyongli has quit IRC13:26
*** jlanoux has joined #openstack-infra13:27
*** heyongli has joined #openstack-infra13:27
*** dizquierdo_ has joined #openstack-infra13:27
*** links has quit IRC13:28
*** jlanoux has quit IRC13:28
*** mvkil has joined #openstack-infra13:28
*** lucasagomes is now known as lucas-hungry13:29
*** jlanoux has joined #openstack-infra13:29
*** akshai has joined #openstack-infra13:29
mvkilhi guys. is there a rule of thumb when is it useful to use jenkins job builder over fully manual work?13:29
hasharBobBall: yeah13:29
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-meetbot: Intendataion of the parameters is refactored  https://review.openstack.org/29528013:30
*** dizquierdo has quit IRC13:30
*** ddieterly has joined #openstack-infra13:31
*** dizquierdo_ has quit IRC13:32
*** esikachev has joined #openstack-infra13:32
*** jcoufal has joined #openstack-infra13:32
*** permalac has joined #openstack-infra13:34
*** dims_ has joined #openstack-infra13:34
*** ddieterly has quit IRC13:35
hasharBobBall: if you get each pairs of (nodepool / jenkins) to use /watch different labels13:35
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-mysql_backup: Order of the classes parameters is refactored  https://review.openstack.org/29528313:36
BobBallyup13:36
*** dims has quit IRC13:36
*** heyongli has quit IRC13:37
openstackgerritMerged openstack-infra/shade: Log inner_exception in test runs  https://review.openstack.org/29492513:37
*** heyongli has joined #openstack-infra13:37
mriedemclarkb: i noticed that the cluster health status in the e-r status page isn't showing up (the label does, the value doesn't),13:38
mriedemclarkb: i checked the source on the page and it looks like the elastic-recheck.js isn't updated13:39
mriedemhttp://status.openstack.org/elastic-recheck/elastic-recheck.js13:39
*** ricky1 has quit IRC13:39
openstackgerritMerged openstack-infra/shade: Add constructor param to turn on inner logging  https://review.openstack.org/29492613:39
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250913:42
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-nodepool: Order of the classes parameters is refactored  https://review.openstack.org/29528513:42
mugsieAJaeger / anyone else: re https://review.openstack.org/#/c/295253/ - how would I denormalise the config? define a "designate-common.config" and then inherit it across the 3 repos?13:43
Shrewsmordred: so did you actually make changes to my flavor patch, or just rebase it off of your logging stuff? hard to tell since it was rebased13:43
*** krtaylor has quit IRC13:43
*** dizquierdo_ has joined #openstack-infra13:44
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-openafs: Order of the class parameters is refactored  https://review.openstack.org/29528813:45
AJaegermugsie: first: Entries are sorted alphabetically, so move label-Review-Priority before label-Workflow13:46
*** cdent has quit IRC13:46
*** jlanoux has quit IRC13:47
*** heyongli has quit IRC13:47
*** heyongli has joined #openstack-infra13:47
openstackgerritGraham Hayes proposed openstack-infra/project-config: Create 'Review-Priority' for designate repos  https://review.openstack.org/29525313:48
mugsieAJaeger: ah, OK. I thought it was asking me to abstract it :)13:48
mugsiethanks13:48
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-openstack_health: Order of the classes parameters is refactored  https://review.openstack.org/29529013:48
*** cdent has joined #openstack-infra13:48
*** vgridnev has quit IRC13:49
*** vgridnev has joined #openstack-infra13:49
mordredShrews: just rebased on the logging13:50
mordredShrews: I think. I might have fixed a test fail as part of the rebase13:50
AJaegermugsie: your patch 295253 will need some more discussion with the infra team, best discuss this with fungi. I see several approaches currently for slightly similar problems. Before that discussion hasn't happened, you won't see a +2 from me.13:50
*** sigmavirus24_awa is now known as sigmavirus2413:51
*** jlanoux has joined #openstack-infra13:51
mugsieAJaeger: sure, no problem. mail to -infra the best bet? or add it to the weekly agenda?13:51
openstackgerritEmilien Macchi proposed openstack-infra/project-config: enable voting for puppet-openstack-cookiecutter jobs  https://review.openstack.org/29502713:51
AJaegermugsie: talk with fungi about best approach once he's had his coffee and has time ;)13:52
mugsieack :)13:52
*** jcoufal has quit IRC13:52
Shrewsmordred: so, looks like you are only changing the get of extra specs. there are still set and unset operations that need to go through requests13:53
Shrewsmordred: but feel free to squash now, if you want13:54
*** roxanaghe has joined #openstack-infra13:54
pabelangerMorning13:54
*** nadya has joined #openstack-infra13:54
pabelangerJust catching up on what happen since Friday afternoon13:55
*** BigWillie has joined #openstack-infra13:55
pabelangerAJaeger: how are things looking today to migrate puppet jobs off devstack-trusty today?13:56
sigmavirus24lifeless: ping, could you take a look at https://review.openstack.org/#/c/280270/ when you wake up?13:57
*** heyongli has quit IRC13:57
openstackgerritMonty Taylor proposed openstack-infra/shade: Create clouds in Functional Test base class  https://review.openstack.org/29529613:57
*** heyongli has joined #openstack-infra13:57
AJaegerpabelanger: we had two problems, hadn't we? I somehow lost track over one of them. The UUID one was fixed by clarkb but what was the second one?13:58
mordredShrews: there are get and set operations?13:58
mordredShrews: /me cries13:58
* AJaeger will be back later, sorry13:58
*** sdake has joined #openstack-infra13:58
pabelangerAJaeger: we needed a new DIB element to install cron too13:58
*** roxanaghe has quit IRC13:58
mordredShrews: there's one more test-case related patch for you ... want me to move it to be at the front of the stack with the other test ones while I squash the others? or leave it as is?13:58
AJaegerpabelanger: part of https://review.openstack.org/29423313:59
AJaegerso, we fixed them together13:59
AJaegerbrb13:59
Shrewsmordred: yes. look for flavor.set_keys() and flavor.unset_keys() in operatorcloud.py13:59
mordredShrews: sob13:59
mordredShrews: k. will get those as a followup14:00
pabelangerAJaeger: okay, and I checked on Friday the element was called.  So, images have been in production for the last few days.14:00
Shrewsmordred: put the test patch where ever makes sense14:00
pabelangerI have to run out this morning but maybe this afternoon we can start the switch again14:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Don't execute the output from wait_for  https://review.openstack.org/29529714:00
pabelangerAssuming JJB is currently happy14:00
openstackgerritMerged openstack-infra/project-config: add more clients when checking for openstackclient command overlap  https://review.openstack.org/29432514:02
*** sridhar_ram has joined #openstack-infra14:02
*** bryan_att has joined #openstack-infra14:03
*** Nikolay_St has quit IRC14:03
openstackgerritMonty Taylor proposed openstack-infra/shade: Create clouds in Functional Test base class  https://review.openstack.org/29529614:04
openstackgerritMonty Taylor proposed openstack-infra/shade: Run extra specs through TaskManager and use requests  https://review.openstack.org/29492814:04
*** yaume_ has joined #openstack-infra14:04
*** pcaruana has joined #openstack-infra14:05
mordredShrews: decided to just leave it whereever14:05
Shrewsoy. so much with the reviewing14:06
*** heyongli has quit IRC14:07
*** yaume has quit IRC14:08
*** heyongli has joined #openstack-infra14:08
mordredShrews: sorry - most of the last patch is just global search and replace and then reformatting lines that were too long14:08
mordredShrews: shade/tests/functional/base.py is the only substantive piece of code in that patch14:08
*** mtanino has joined #openstack-infra14:08
*** esikachev has quit IRC14:09
*** jaosorior has quit IRC14:10
*** jaosorior has joined #openstack-infra14:11
*** boris-42 has joined #openstack-infra14:12
*** lucas-hungry is now known as lucasagomes14:12
*** knikolla has joined #openstack-infra14:12
openstackgerritMerged openstack-infra/project-config: Add db setup for periodic-*with-oslo-master  https://review.openstack.org/29494314:13
*** dizquierdo_ has quit IRC14:14
openstackgerritMerged openstack-infra/shade: Refactor unit tests to construct cloud in base  https://review.openstack.org/29492714:15
pabelangerfungi: thoughts on 287232? I think we're in good shape to start cleaning up wiki.o.o from the spam attack14:17
*** infra-red has joined #openstack-infra14:17
*** heyongli has quit IRC14:18
*** heyongli has joined #openstack-infra14:18
*** rbrndt has joined #openstack-infra14:18
*** ajmiller has joined #openstack-infra14:18
openstackgerritMerged openstack-infra/project-config: Add the releasenotes gate job to Packstack  https://review.openstack.org/29474114:19
mordredShrews: wow. _mod_flavor_specs just makes me sad14:19
*** ddieterly has joined #openstack-infra14:19
mordredShrews: that's been there for a while though - I don't think we need to fix it before doing a 1.5.114:19
openstackgerritMerged openstack-infra/project-config: Add jobs for stable/mitaka branch  https://review.openstack.org/29467814:20
openstackgerritMerged openstack-infra/project-config: Correct branch limitations for ironic-inspector  https://review.openstack.org/29410614:20
*** kencjohnston_ has joined #openstack-infra14:21
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-openstackci: Order of the classes parameters is refactored  https://review.openstack.org/29530914:22
*** yamamoto has quit IRC14:23
pabelangerso, looking for some help to land 293826 for glean (fixes issue with centos7) and maybe see what is needed to also cut a release14:24
fungiAJaeger: for 294939 and similar changes it'll be a judgement call. i was worried about it for "highly active" repos like nova, but picking a quiet time when there were few changes in progress testing was the alternative i suggested to splitting14:25
*** dizquierdo_ has joined #openstack-infra14:27
*** heyongli has quit IRC14:28
mordredpabelanger: prometheanfire wanted a glean release too14:28
*** heyongli has joined #openstack-infra14:28
openstackgerritMerged openstack-infra/glean: Better service description  https://review.openstack.org/29434214:28
mordredpabelanger: 293826 landed14:29
pabelangermordred: excellent, and thanks14:29
mordredalso ... ianw thank you for such an informative commit message14:29
vsaienkoplease review https://review.openstack.org/#/c/294128/ it is an experimental systest job for networking-generic-switch project. This job configures devstack with neutron and keystone and launch exercise.sh (which is on review https://review.openstack.org/#/c/294058/) from networking-generic-switch. Once it is stabilized we move this job to voting mode.14:29
Shrewsmordred: well, that hasn't worked since 1.214:30
*** krtaylor has joined #openstack-infra14:30
*** mrmartin has quit IRC14:30
mordredShrews: setting them?14:30
Shrewssince BEFORE 1.214:30
mordredor your bugfix?14:30
Shrewsmordred: set/unset used to work until you moved the obj_to_dict stuff to task_manager14:31
fungimugsie: AJaeger: i feel like adding a vote label in gerrit for review priority is sort of hacky since the only person who can unset it is the person who set it (for example if one person in designate-core sets Review_priority +2 and then goes on vacation, nobody else in designate-core can downgrade that to +1 or 0)14:31
Shrewsmordred: that's when we stopped getting the raw object back14:31
mordredShrews: I feel like we're talking about two different things still14:31
Shrewsmaybe?14:31
fungimugsie: this is something jeblair might also have input on since he put together the rollcall-vote solution being used by governance and some specs repos14:32
*** tosky has joined #openstack-infra14:32
mordredShrews: I'm purely talking about not waiting for a new patch to put set/unset into TaskManager/requests14:32
mugsiefungi: yeah - that is true. It is a little better than "starred-by" as at least if I go on holiday other members can add to the list14:32
mordredShrews: I htink the "fix flavor get" should certainly go in14:32
jeblairhrm what?14:32
mugsiefungi: yeah, thats where I got the idea from14:32
mugsiejeblair: https://review.openstack.org/29525314:32
fungijeblair: change 29525314:32
*** ianychoi has joined #openstack-infra14:33
*** zz_dimtruck is now known as dimtruck14:33
fungimugsie: querying on that could also be a little strange if there are conflicting review-priority votes14:33
Shrewsmordred: are we in a rush for a release?14:33
mordredShrews: I thought there was something someone needed for an ansible bug - lemme check14:33
mugsieyeah, it would have to default to the highest I suppose14:34
fungimugsie: i think other teams are collaborating on a solution using custom dashboards auto-published in gerrit14:34
mordredSamYaple: were you looking for a shade release for a $reason currently?14:34
fungiwhich can also do things similar to what reviewday does (take into account prioritiies set on corresponding bug reports, check for mapping to approved specs, et cetera)14:34
mugsiefungi: yeah - I saw that, but I would like to keep as much as possible in gerrit14:35
*** yamamoto has joined #openstack-infra14:35
mordredShrews: ah - yeah, the endpoint changes are in master and those are needed ot land the fixes to the endpoint modules14:35
mugsieI could build a dashboard that generates it from that, + a list of change ids I suppose.14:35
*** sc68cal_ has joined #openstack-infra14:36
mugsieI just liked the (subjective I admit) cleanlyness of this way.14:36
mordredShrews: but I'll try to get the set/unset thing written soon14:37
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-openstackid: Order of the classes parameters is refactored  https://review.openstack.org/29532014:37
*** sc68cal has quit IRC14:38
*** heyongli has quit IRC14:38
*** heyongli has joined #openstack-infra14:38
amrithAJaeger, ping ... had a question re: https://review.openstack.org/#/c/295046/14:39
Shrewsmordred: we don't currently have an endpoint module. there is an open PR by me for one, but it needs to be modified for the recent changes14:39
mordredkk.14:39
prometheanfiremordred: thanks, that'd be nice :P14:40
SamYaplemordred: i was but its too late14:40
prometheanfirepabelanger: woo for peer preasure14:40
*** annegentle has quit IRC14:40
*** Jeffrey4l__ has joined #openstack-infra14:40
SamYaplemordred: shade relase was so i could get service module for ansible in ansible 2.114:40
*** annegentle has joined #openstack-infra14:40
*** mrmartin has joined #openstack-infra14:41
pabelangerprometheanfire: :)14:41
mordredSamYaple: ah. k14:43
openstackgerritSam Betts proposed openstack-infra/glean: Enable support for configuring bonded interfaces  https://review.openstack.org/29364814:44
SamYaplemordred: just tag before ansible 2.2 :P14:44
*** annegentle has quit IRC14:45
*** esikachev has joined #openstack-infra14:45
pleia2going to find an airplane to take me home, will be around for the meeting on Tuesday o/14:45
fungipleia2: safe flight!14:46
openstackgerritMurad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo  https://review.openstack.org/29526014:46
pleia2thanks :)14:46
mordredsdague, AJaeger: any chance either of you know off the top of your head where I can find docs on OS-FLV-EXT-DATA:ephemeral and os-flavor-access:is_public ?14:47
mordredthe only thing I find on the API site is reference to their existence, not any sense of what they might mean or when I might or might not find them as part of the payload or what their default values might be14:48
*** tonytan4ever has joined #openstack-infra14:48
*** yamamoto has quit IRC14:48
*** heyongli has quit IRC14:48
sdagueyeh, well they should be documented on the API site... apparently that whole section is just noise14:48
sdagueI'll dig up code links14:49
*** heyongli has joined #openstack-infra14:49
*** achanda has joined #openstack-infra14:50
openstackgerritMurad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo  https://review.openstack.org/29526014:50
sdaguemordred: what are you looking for per say? types?14:50
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-packagekit: Order of the classes parameters is refactored  https://review.openstack.org/29533114:50
mordredsdague: well, mainly "what do they communicate to me" and "can I always expect them to be in the payload"14:51
*** sc68cal_ has quit IRC14:52
mordredsdague: becuase OS-FLV-EXT-DATA:ephemeral on a flavor could mean any number of things to my brain, and google wasn't helpful14:52
sdagueOS-FLV-EXT-DATA:ephemeral is the # of gb of the ephemeral disk14:52
mordredwoot! lovely. that's very clear :)14:53
sdagueyeh14:53
sdagueI'm actually trying to unfuxor the api doc space right now14:53
sdaguebut, it will be a bit14:53
mordredsdague: btw: http://developer.openstack.org/api-ref-compute-v2.1.html the value shown for swap is "" but for disk it's 1 and for ephemeral it's 0 - is "" an actual valid swap value or would it be more likley to be 014:54
*** links has joined #openstack-infra14:54
mordredsdague: specifically here: http://developer.openstack.org/api-ref-compute-v2.1.html#showFlavor14:54
sdaguemordred: probably, honestly, the current api-ref is so so bad right now, I'm not sure it can even really be saved in the current format14:54
openstackgerritPaul Belanger proposed openstack-infra/glean: Bring up glean before network.service  https://review.openstack.org/29382614:55
pabelangermordred: had to rebase ^14:55
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-pgsql_backup: Order of the class parameters is refactored  https://review.openstack.org/29533414:55
Zarawhat's that? is it a bird? is it a plane? no, it's much better-- it's the storyboard email notifications patch, awaiting reviews! https://review.openstack.org/#/c/293367/14:57
*** jcooley_ has quit IRC14:58
*** smarcet has joined #openstack-infra14:58
*** heyongli has quit IRC14:59
smarcetmorning14:59
*** tonytan4ever has quit IRC14:59
smarcetfungi: could u look at https://openstackid.org?14:59
*** heyongli has joined #openstack-infra14:59
smarcetseems to be down14:59
fungi#status log Rackspace has opened support tickets warning of disruptive maintenance March 22 05:00-07:00 UTC, March 24 03:00 to 07:00 UTC, and March 25 02:00 to 06:00 UTC which could impact network connectivity including disconnecting from Trove databases and Cinder block devices14:59
openstackstatusfungi: finished logging14:59
anteayapleia2: thanks I will take a peek14:59
*** tonytan4ever has joined #openstack-infra15:00
ddieterlyhow long should a 'recheck' on a patchset take to be recognized by zuul?15:00
*** sridhar_ram has quit IRC15:00
*** jcooley_ has joined #openstack-infra15:00
fungismarcet: seems under very heavy load15:01
fungitook a couple minutes to ssh in15:01
pabelangerindeed15:01
pabelangerhttp://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2064&rra_id=all15:01
fungismarcet: looks like it's using all available ram15:01
fungismarcet: lots of apache2 processes eating up all the memory15:01
smarcetfungi: yes i was running some stress test with jmeter15:01
smarcetseems that ws my fault15:02
smarcetsorry :(15:02
fungismarcet: well, it demonstrates that we have insufficient memory available for the current tuning of the server, so good find15:02
smarceti just run 500 users15:02
*** links has quit IRC15:02
fungismarcet: i'm going to have a look at the memory graph for th server to see whether it looks like you've exposed a memory leak15:03
*** austin81 has joined #openstack-infra15:03
fungismarcet: spiked up pretty instantly, and i don't see much growth prior to that, so probably just tuned to support too many concurrent sessions for the amount of ram we have http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2067&rra_id=all15:04
*** tonytan4ever has quit IRC15:04
smarcetfungi: could we increase ram ?15:06
fungismarcet: we would need to build a replacement server, but sure we can do that15:06
openstackgerritMurad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo  https://review.openstack.org/29526015:06
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-phabricator: Order of the classes parameters is refactored  https://review.openstack.org/29533715:06
fungismarcet: it will have new ip addresses, but presumably that won't pose a problem15:06
fungiwe'll just update dns to switch over to the replacement server15:07
smarcetfungi: got it15:07
fungideploying a new server is pretty painless as long as the puppet modules for this haven't grown any bootstrapping problems since the last deployment15:08
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-pip: Order of the class parameters is refactored  https://review.openstack.org/29534015:08
*** yamamoto has joined #openstack-infra15:08
*** heyongli has quit IRC15:09
*** rfolco has quit IRC15:09
*** yamamoto has quit IRC15:09
*** heyongli has joined #openstack-infra15:09
*** rfolco has joined #openstack-infra15:09
fungismarcet: i'll start trying to deploy a replacement openstackid.org and make sure it comes up properly... how much ram would you want to move up to?15:10
*** yamamoto has joined #openstack-infra15:10
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-project_config: Order of the class parameters is refactored  https://review.openstack.org/29534215:10
fungithen we can plan the cut-over time once we test the replacement server out and it's working correctly15:10
*** yamamoto has quit IRC15:10
*** zz_ja has quit IRC15:11
pabelangerfungi: when I last tested it before chistmas, I was able to launch the server on trusty15:11
smarcetfungi: rite now its has 2GB ?15:11
openstackgerritIan Cordasco proposed openstack/requirements: Blacklist cryptography 1.3.0  https://review.openstack.org/29534415:11
sigmavirus24mriedem: ^15:11
*** hashar has quit IRC15:11
fungismarcet: correct, 2gb currently15:11
vsaienkosdague: could you please review https://review.openstack.org/#/c/252814/ and https://review.openstack.org/#/c/294128/15:11
*** acruz has joined #openstack-infra15:11
smarceti said that 8Gb should be enough dont u think?15:11
SamYaplewhen does openstack/requirements create a mitaka branch?15:11
smarcetwe expect some heavy load due the summit ap15:11
smarcetpp15:12
fungipabelanger: yep, i remember, so this should go pretty smoothly (though we'll want to stick with precise for this replacement to avoid introducing more unknowns right before the summit)15:12
smarcetapp15:12
*** yamahata has joined #openstack-infra15:12
fungismarcet: 8gb sounds fine. i'll get going on that now15:12
smarcetfungi: thx a lot ;)15:12
openstackgerritGraham Hayes proposed openstack-infra/project-config: Create 'Review-Priority' for designate repos  https://review.openstack.org/29525315:12
*** rfolco has quit IRC15:12
fungismarcet: you're welcome. i'm in a meeting for a few more minutes, but can focus attention on the new deployment as soon as this wraps up15:13
*** sbalukoff has quit IRC15:13
*** rfolco has joined #openstack-infra15:13
smarcetfungi: roger that15:13
*** zz_ja has joined #openstack-infra15:13
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-puppet: Order of the class parameters is refactored  https://review.openstack.org/29534515:14
*** arxcruz has quit IRC15:14
openstackgerritMerged openstack-infra/glean: Bring up glean before network.service  https://review.openstack.org/29382615:15
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-redis: Order of the class parameters is refactored  https://review.openstack.org/29534615:16
pabelangerokay, have to run an errand. will be back shortly15:16
*** fesp has joined #openstack-infra15:16
openstackgerritDouglas Mendizábal proposed openstack/requirements: Update python-barbicanclient for latest release  https://review.openstack.org/29534715:18
*** achanda has quit IRC15:19
*** heyongli has quit IRC15:19
*** heyongli has joined #openstack-infra15:19
*** abregman is now known as abregman|brb15:23
mgagnefungi: I see all images are in active state at internap. Should I consider the issue solved?15:23
*** dizquierdo_ has quit IRC15:23
*** Guest4916 is now known as jgriffith15:23
mgagnefungi: also, can images stuck in "saving" state be deleted?15:24
fungimgagne: probably, yes. the increase on the token timeout seems to have done the trick for now, though obviously we're still looking at ways to make this less of an issue from the client end as well15:24
*** dizquierdo has joined #openstack-infra15:24
fungimgagne: any images still in a "saving" state are fine to delete15:24
anteayaI have a request on the mailing list to disable IBM GPFS CI which gerrit thinks is linked to this email: rsandeep@in.ibm.com however when I do a curl GET request to find the gerrit id from the email gerrit tells me not found, because there are two gerrit accounts with the same name and same email address *sigh*15:24
anteayaany infra-root have a moment to help me sort that out?15:25
anteayathe email in question: http://lists.openstack.org/pipermail/openstack-infra/2016-March/004031.html15:25
*** tonytan4ever has joined #openstack-infra15:25
anteayawhich seems to be from the operators who appear to have inherited the system15:25
vincentllhow many +1 are needed to have a new project accecpted into OpenStack CI?15:26
*** harlowja_at_home has joined #openstack-infra15:26
*** MarkAtwood has joined #openstack-infra15:26
anteayavincentll: none or as many as you like15:27
anteayavincentll: you need two +2's and a +1 workflow15:27
*** tonytan4ever has quit IRC15:27
*** IlyaG has joined #openstack-infra15:28
vincentllanteaya: ok, so two approvals from the infra team and one from jenkins right?15:28
*** abregman|brb is now known as abregman15:29
anteayatwo +2's and a +1 workflow from core reviewers on that repo, and yes jenkins has to verify check and gate15:29
*** heyongli has quit IRC15:29
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-reviewday: Order of the class parameters is refactored  https://review.openstack.org/29535715:29
vincentllanteaya: ok, thanks15:30
openstackgerritAndrew Laski proposed openstack/requirements: Bump oslo.messaging to >=4.5.0  https://review.openstack.org/29535815:30
*** heyongli has joined #openstack-infra15:30
*** ifarkas has quit IRC15:30
*** tonytan4ever has joined #openstack-infra15:30
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-simpleproxy: Intendation of the class parameter is refactored  https://review.openstack.org/29535915:32
fungismarcet: openstackid.org seems to have recovered on its own once you stopped your load test, so that's a great sign for resiliency of the system at least15:32
*** krtaylor has quit IRC15:32
openstackgerritMerged openstack-infra/openstackid-resources: Fix on instrospection error handling  https://review.openstack.org/29535515:33
*** Nikolay_St has joined #openstack-infra15:33
anteayavincentll: welcome15:33
*** tonytan_brb has joined #openstack-infra15:33
*** Qiming has quit IRC15:34
*** ddieterly has quit IRC15:34
*** vgridnev has quit IRC15:35
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-ssl_cert_check: Order of the class parameters is refactored  https://review.openstack.org/29536115:36
*** annegentle has joined #openstack-infra15:36
*** tonytan4ever has quit IRC15:36
phschwartzAre there currently any multi-node test jobs in project-config?15:36
*** mrmartin has quit IRC15:36
anteayaphschwartz: there should be15:37
anteayanova and neutron I believe use the setup currenly15:37
anteayacurrently15:37
phschwartzanteaya: grepping through everything I just don't see anyting using the mutli node labels from nodepool. Let me dig further15:37
*** yamamoto has joined #openstack-infra15:37
*** Sukhdev has joined #openstack-infra15:38
*** annegentle has quit IRC15:38
*** annegentle has joined #openstack-infra15:39
anteayaphschwartz: there's one: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n753615:39
*** heyongli has quit IRC15:40
anteayacontrol-F says "multinode" appears 47 times in zuul/layout.yaml15:40
*** heyongli has joined #openstack-infra15:40
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-stackalytics: Order of the class parameters is refactored  https://review.openstack.org/29536215:40
phschwartzanteaya: in the layout yes, but I can't seem to find the jjb def's associated with them15:41
*** yamahata has quit IRC15:41
*** yamamoto has quit IRC15:41
phschwartzanteaya: I will dig through it all. that gives me a starting point though. ty15:41
*** yamamoto has joined #openstack-infra15:42
*** roxanaghe has joined #openstack-infra15:42
sbelous__ianw: ping15:42
*** gongysh_ has joined #openstack-infra15:44
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-statusbot: Order of the class parameters is refactored  https://review.openstack.org/29536615:44
fungiphschwartz: for example, the {pipeline}-tempest-dsvm-multinode-full{job-suffix} definition at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n200115:44
*** krtaylor has joined #openstack-infra15:46
*** roxanaghe has quit IRC15:47
anteayafungi: thank you15:48
phschwartzfungi: ty15:48
*** moravec has quit IRC15:48
anteayaphschwartz: is export DEVSTACK_GATE_TOPOLOGY="multinode" what you mean by the multi node labels from nodepool?15:48
*** bpokorny has joined #openstack-infra15:49
*** mylu has joined #openstack-infra15:49
*** moravec has joined #openstack-infra15:49
*** tonytan_brb has quit IRC15:49
*** fesp has quit IRC15:50
*** heyongli has quit IRC15:50
*** tonytan4ever has joined #openstack-infra15:50
*** heyongli has joined #openstack-infra15:50
*** darvon has quit IRC15:50
*** sbelous has joined #openstack-infra15:50
*** sbelous__ has left #openstack-infra15:51
*** frauggy is now known as auggy15:52
*** annegentle has quit IRC15:53
*** sputnik13 has joined #openstack-infra15:53
*** annegentle has joined #openstack-infra15:53
*** annegentle has quit IRC15:54
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-storyboard: Order of the classes parameters is refactored  https://review.openstack.org/29537215:55
*** gongysh_ has quit IRC15:55
*** yamamoto has quit IRC15:56
*** darvon has joined #openstack-infra15:56
*** acruz has quit IRC15:57
*** maishsk has quit IRC15:57
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-subunit2sql: Order of the classes parameters is refactored  https://review.openstack.org/29537415:57
*** sridhar_ram has joined #openstack-infra15:57
*** sputnik13 has quit IRC15:57
*** rcernin has quit IRC15:59
*** daemontool has quit IRC16:00
*** IlyaG has quit IRC16:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Add mirror server  https://review.openstack.org/23841416:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Used the mirror server for the centos image and git repos  https://review.openstack.org/28525716:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Use instack-qcow2 from cache if available and appropriate  https://review.openstack.org/28525916:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Upload the instack qcow image after periodic jobs  https://review.openstack.org/28525816:00
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job  https://review.openstack.org/22978916:00
*** heyongli has quit IRC16:00
*** IlyaG has joined #openstack-infra16:00
*** yamamoto has joined #openstack-infra16:00
*** heyongli has joined #openstack-infra16:00
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-unattended_upgrades: Intendation of the class parameter is refactored  https://review.openstack.org/29537816:03
*** infra-red has quit IRC16:04
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-user: Order of the class parameters is refactored  https://review.openstack.org/29538016:05
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-yum: Order of the class parameters is refactored  https://review.openstack.org/29538116:08
*** yamamoto has quit IRC16:08
*** tphummel has joined #openstack-infra16:08
*** mylu has quit IRC16:09
*** yamahata has joined #openstack-infra16:10
*** daemontool has joined #openstack-infra16:10
*** heyongli has quit IRC16:10
*** heyongli has joined #openstack-infra16:11
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250916:11
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: WIP: Add a OVB te-broker server  https://review.openstack.org/29524316:11
anteayajroll: can I trouble you for a moment? on Friday antonym offered a patch that increased our rax nodes in ord from 55 to 195 but then realized we didn't have enough quota in that region to satisfy the node increase, so last I heard was going to request more ord quota for us to cover the 195 node amount16:12
anteayajroll: do you know if that happened?16:12
antonymi bumped the quota, if you are still having issues, let me know and i can tweak16:13
antonymanteaya: ^16:13
antonymbumped it last week at least16:13
jrolleasiest question ever16:14
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250916:14
anteayaantonym: we seem to still be having issues according to our graphs, but I don't know if it is on our side or not16:14
anteayaantonym: I wanted to know the quota status first before I started bugging folks, sorry to bother and thanks for the response16:15
anteayajroll: sorry to bother you16:15
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-zanata: Order of the classes parameters is refactored  https://review.openstack.org/29538216:15
anteayaantonym: and thanks for the quota :)16:15
jrollanteaya: no worries :)16:15
anteayaantonym: error node launch attempts is the graph I'm looking at: http://grafana.openstack.org/dashboard/db/nodepool-rackspace16:15
antonymyeah, no prob, you can ping me if you have more issues with that16:16
anteayaantonym: which is a catch all for just about anything16:16
anteayaantonym: thank you :)16:16
*** armax has joined #openstack-infra16:16
jeblairanteaya: do the values in that graph seem abnormally high?16:17
anteayato me they do16:17
jeblairanteaya: compared to when?16:17
anteayajeblair: I don't have a time line for when error node launch attempts number was less that it is now16:18
*** roxanaghe has joined #openstack-infra16:18
anteayaso unfortunately I don't have a good answer for compared to when16:18
fungianteaya: antonym: the most recent error i see launching in ord was a timeout waiting for it to become reachable: "2016-03-21 16:13:39,289 ERROR nodepool.NodeLauncher: Timeout launching node id: 8923511 in provider: rax-ord error: Timeout waiting for ssh access"16:18
*** dizquierdo has quit IRC16:19
jrollinfra-root, btw, if you didn't already know about the network maintenance that began today: https://status.rackspace.com/ first one in the list16:19
*** asettle has joined #openstack-infra16:19
jrollit will cause a ~10 second network blip on performance flavors16:19
jrollidk how well jenkins will handle that :)16:20
fungijroll: yep, i logged at https://wiki.openstack.org/wiki/Infrastructure_Status16:20
jrollawesome16:20
anteayajroll: yes fungi had mentioned that earlier16:20
* jroll now knows about that page16:20
fungisince it seems to wreak havoc with our trove and cinder usage quite often16:20
jrollnod16:20
anteayaI was just seeing that we seem to have an unhappy rax interaction and was trying to narrow down reasons16:20
*** pfallenop has quit IRC16:20
fungiwould be nice if they used utc times so i didn't have to convert them, but was nice to at least have a warning16:21
*** heyongli has quit IRC16:21
*** heyongli has joined #openstack-infra16:21
anteayawarnings are helpful16:21
*** jiggalator has joined #openstack-infra16:21
fungigranted, the only reason i knew to look for warnings about this was that i received some notices of new tickets opened for my personal servers in iad and syd16:22
fungiright now ticket e-mails for that tenant seem to go to monty@inaugust.com16:23
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250916:23
*** asettle has quit IRC16:24
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-zuul: Order of the classes parameters is refactored  https://review.openstack.org/29538516:24
*** yamamoto has joined #openstack-infra16:24
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: WIP: Add a OVB te-broker server  https://review.openstack.org/29524316:25
*** amitgandhinz has quit IRC16:25
*** pcaruana has quit IRC16:25
openstackgerritMonty Taylor proposed openstack-infra/shade: Deal with is_public and ephemeral in normalize_flavors  https://review.openstack.org/29538716:26
openstackgerritMonty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset  https://review.openstack.org/29538816:26
*** jiggalator has quit IRC16:26
*** mylu has joined #openstack-infra16:26
zaromorning16:26
*** jiggalator has joined #openstack-infra16:26
anteayamorning zaro16:27
fungiclarkb: did you encounter any issues with the current state of launch-node.py or are there outstanding patches i need to apply to it?16:27
*** jiggalator has quit IRC16:27
*** netsin has joined #openstack-infra16:27
fungithe readme in that directory hasn't been touched, so i don't know if that's an oversight or if it means usage of teh script hasn't changed at all with the introduction of shade/ansible16:28
*** yamamoto has quit IRC16:28
fungihrm, i guess just shade, no ansible16:28
fungianyway, going to take it for a spin as soon as i get some lunch in me16:29
*** Guest9441 has quit IRC16:30
*** heyongli has quit IRC16:31
*** tqtran-afk has joined #openstack-infra16:31
*** heyongli has joined #openstack-infra16:31
*** kencjohnston_ has quit IRC16:32
*** jpr has joined #openstack-infra16:33
anteayazaro: did you see on friday we upgraded all the jenkinsii?16:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset  https://review.openstack.org/29538816:34
*** lakshmiS has joined #openstack-infra16:34
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250916:35
lakshmiSappreciate any feedback on https://review.openstack.org/#/c/294224/16:35
*** abregman has quit IRC16:35
*** eil397 has joined #openstack-infra16:35
*** amitgandhinz has joined #openstack-infra16:35
*** rkukura has joined #openstack-infra16:37
*** sdake_ has joined #openstack-infra16:38
*** rcarrillocruz has joined #openstack-infra16:39
*** infra-red has joined #openstack-infra16:40
zaroanteaya: i just noticed that.  what prompted the upgrade?16:40
*** sdake has quit IRC16:41
*** heyongli has quit IRC16:41
*** amitgandhinz has quit IRC16:41
*** heyongli has joined #openstack-infra16:41
*** jpr has quit IRC16:42
fungizaro: https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2016-02-24 which seems not to have gotten mentioned in the usual places (like the oss-security ml), or else i overlooked it when it was announced16:42
*** amitgandhinz has joined #openstack-infra16:42
*** Swami has joined #openstack-infra16:47
*** ihrachys has quit IRC16:47
openstackgerritMerged openstack-infra/project-config: Improve experimental tempest-dsvm-nova-os-vif job  https://review.openstack.org/29516116:47
rossella_sfungi hi! regarding the project gerrit dashboard, do you think that I can use https://review.openstack.org/#/c/94260 ?16:47
*** infra-red has quit IRC16:48
*** infra-red has joined #openstack-infra16:48
rossella_sfungi it's patch by sdague that was abandoned long ago...I think I need something like that to enable project dashboard in gerrit. Unfortunately I am not so familiar with the infra code, I might be wrong16:48
*** HeOS has quit IRC16:49
*** achanda has joined #openstack-infra16:49
*** vincentll has quit IRC16:51
*** mriedem is now known as mriedem_lunch16:51
*** daemontool has quit IRC16:51
*** apoorvad has joined #openstack-infra16:51
*** heyongli has quit IRC16:51
*** annegentle has joined #openstack-infra16:51
zarofungi: i guess it was good thing we already had a change in review for the upgrade.  any issues so far?16:52
rossella_sjeblair, please read above I should have pinged you too16:52
*** heyongli has joined #openstack-infra16:52
openstackgerritZara proposed openstack-infra/storyboard-webclient: Improve 'Tasks Assigned to Me'  https://review.openstack.org/29250916:52
*** mikelk has quit IRC16:53
pabelangerfungi: do infra-root have access to the DNS for trystack.org? Or is that a foundation only thing?16:53
jeblairrossella_s, fungi: yes, i think it would be great for something like 94260 to land.  alternatively, we could explore allowing a group of people permission to push directly to a project's refs/meta/dashboards ref.16:55
rossella_sjeblair, if sdague agrees I can take over and help updating the patch so that it can be merged16:56
rossella_sjeblair, it's probably a cleaner solution than pushing directly to refs/mets/dashboards, what do you think?16:56
*** jpr has joined #openstack-infra16:57
*** achanda has quit IRC16:57
*** mrmartin has joined #openstack-infra16:58
*** keedya has joined #openstack-infra17:00
openstackgerritMonty Taylor proposed openstack-infra/shade: Create clouds in Functional Test base class  https://review.openstack.org/29529617:01
openstackgerritMonty Taylor proposed openstack-infra/shade: Deal with is_public and ephemeral in normalize_flavors  https://review.openstack.org/29538717:01
openstackgerritMonty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset  https://review.openstack.org/29538817:01
*** heyongli has quit IRC17:02
*** heyongli has joined #openstack-infra17:02
*** amrith is now known as _amrith_17:02
*** rossella_s has quit IRC17:03
*** daemontool has joined #openstack-infra17:03
*** cdent has quit IRC17:03
*** rossella_s has joined #openstack-infra17:04
*** matrohon has quit IRC17:04
*** esikachev has quit IRC17:05
*** scheuran has quit IRC17:05
*** annegentle has quit IRC17:06
*** tqtran-afk is now known as tqtran17:06
*** piet has joined #openstack-infra17:08
*** sputnik13 has joined #openstack-infra17:11
austin81mtreinish: Will you be around for -infra mtg tomorrow? Discussing Stackviz spec17:11
fungirossella_s: i agree with jeblair 94260 should be a good fit for what we were discussing earlier17:11
*** heyongli has quit IRC17:12
*** ihrachys has joined #openstack-infra17:12
*** heyongli has joined #openstack-infra17:12
fungijeblair: interestingly, i'm able to crash gertty repeatedly with 94260 so time to turn on debugging17:12
clarkbfungi: my launch node conflicts with recent other changes to launch node and I havne't hda the time to unravel that17:13
fungijeblair: urwid.listbox.ListBoxError: Widget <Pile selectable box/flow widget> at position 2 within listbox calculated 7 rows but rendered 8!17:13
clarkbfungi: if you grab my launch node script changes they should just work (tm) and work the same way as before17:13
clarkbmriedem_lunch: maybe you need a hard refresh to pull in the new js?17:13
*** mylu has quit IRC17:14
jeblairfungi: ah thanks. i can reproduce that (at 80x24; it works at larger sizes)17:14
clarkbmriedem_lunch: it seems to be working for me17:14
*** mylu has joined #openstack-infra17:14
anteayaI need a hand from an infra root to untangle a ci system with two accounts using the same email, IBM GPFS CI for which there is a request to disable, details here: http://paste.openstack.org/show/491334/17:15
*** samuelBartel has quit IRC17:15
fungijeblair: yep, 80x24 is what i run (80x25 terminal with a status line)17:15
fungijeblair: still need a debug log or full traceback for that?17:15
jeblairfungi: nope, i got it, thanks17:16
fungithanks!17:16
*** mrmartin has quit IRC17:16
openstackgerritThomas Maddox proposed openstack-infra/project-config: Move to Fedora 23 for Libvirt/LXC Nova Test  https://review.openstack.org/29358517:16
anteayaaustin81: it is possible he his either in singapore still or in transit, pleia2 was at the same event and posted she was flying out around 06:00 utc today17:17
fungizaro: the only issue i'm aware of (i was on the road most of the day) was that the gearman plugin upgrade got stuck somehow on one of the jenkins masters and had to be manually deleted and reinstalled17:17
*** bnemec has quit IRC17:17
clarkbanteaya: looking17:17
anteayaclarkb: thank you17:17
fungipabelanger: i believe it's under dns management for the "openstack" tenant account in rackspace, just like our other domains17:17
austin81anteaya: ah okay. thanks for the tip.17:17
*** nadya has quit IRC17:17
anteayaaustin81: sure17:17
pabelangerfungi: okay, thanks17:17
*** tdasilva has quit IRC17:18
clarkbanteaya: it looks like we should disable both accounts then when they want to be reenabled they can provide the exact username they want to use17:18
anteayaclarkb: I can live with that17:18
jeblairfungi: the cause is the related change whose subject is wider than the screen17:18
clarkbanteaya: do you have both gerrits ids handy or should I dig them up?17:18
jeblairfungi: "  Update mechanism of creating groups in database and managing access rights with jeepyb17:18
fungijeblair: aha! interesing17:18
anteayaclarkb: I don't, gerrit doesn't give them to me with curl when two accounts use the same email17:19
clarkbanteaya: gotcha I can dig them out of the db17:19
anteayaclarkb: is there any way I can get gerrit ids in this case?17:19
anteayathanks17:19
openstackgerritRossella Sblendido proposed openstack-infra/jeepyb: add dashboard support to jeepyb  https://review.openstack.org/29541817:20
*** doug-fis_ has quit IRC17:20
rossella_sfungi, jeblair ^^ pushed! thanks a lot for your help!17:21
*** doug-fish has joined #openstack-infra17:21
openstackgerritSebastian Schuberth proposed openstack-infra/jenkins-job-builder: builder: Fix hashing job configurations with Python 3  https://review.openstack.org/23769917:21
*** e0ne has quit IRC17:21
clarkbanteaya: 21058 and 12491 I am disabling them both17:22
*** heyongli has quit IRC17:22
*** rkukura has quit IRC17:22
*** heyongli has joined #openstack-infra17:22
anteayaclarkb: thank you17:23
*** tdasilva has joined #openstack-infra17:23
*** yaume_ has quit IRC17:23
*** pfallenop has joined #openstack-infra17:23
zarofungi: good to hear.17:25
*** doug-fish has quit IRC17:25
zaroclarkb: votes on every stream event was merged: https://gerrit-review.googlesource.com/#/c/7105117:26
anteayazaro: did you read the logs from the incident channel?17:26
clarkbanteaya: thats done, will work on an email response. Still trying to catch up on the monday flood of email17:26
clarkbzaro: woot17:26
anteayaclarkb: I can respond via email, unless you would prefer to do it yourself17:26
anteayaand thank you17:26
zaroanteaya: yes, just glossed over it.  anything else i should be aware of?17:27
clarkbanteaya: ya I can do it and provide specific account info (usernames in particular)17:27
clarkbanteaya: unless you are able to get that now that you have the ids?17:27
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs  https://review.openstack.org/28944517:27
*** cdent has joined #openstack-infra17:27
anteayazaro: nope, just wanted to ensure you saw it17:28
anteayazaro: yay 71051!17:28
*** chlong|wfh has quit IRC17:28
*** rkukura has joined #openstack-infra17:28
*** davideagnello has joined #openstack-infra17:28
anteayaclarkb: were the usernames different for the two accounts?17:28
clarkbanteaya: I havne't checked but I believe gerrit requires them to be17:29
*** yamamoto has joined #openstack-infra17:29
clarkbanteaya: and its a required parameter to zuul so is a good way for them to identify which account they wish to use17:29
*** akaszuba has quit IRC17:29
anteaya21058 doesn't give me a username17:29
anteaya12491 has username ibm-gpfs-ci17:30
clarkbits possible they never said a username in which case the other account is likely the one in use17:30
anteayapossibly17:30
anteayaclarkb: yeah, why don't you go ahead and reply17:30
anteayaand I can follow up if need be17:31
anteayathanks for disabling17:31
clarkbmnaser: if you are able to add more info to https://bugs.launchpad.net/bugs/1558807 that would be great. This is a bug for the volume and config drive not ending up on the devices we expect17:31
openstackLaunchpad bug 1558807 in OpenStack Compute (nova) "Volume attached at different path than reported by nova/cinder when using config drive" [Undecided,New]17:31
*** agentle has joined #openstack-infra17:31
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Add API functionality for automatic worklist filters  https://review.openstack.org/29542217:31
*** nadya has joined #openstack-infra17:32
*** heyongli has quit IRC17:32
kambizfolks pabelanger mentioned I should ask here.  Can someone help me with a DNS request to add "8.43.86.2 www.x86.trystack.org" to the trystack.org DNS ?  Please make sure NOT to change the current x86.trystack.org A record though.  And I can send an email to the infra list if needed.17:32
*** sridhar_ram has quit IRC17:33
*** heyongli has joined #openstack-infra17:33
pabelangerkambiz: I don't mind doing the work, I just wanted to make sure other admins were okay with the request17:33
fungiyeah, seems fine to me17:34
*** degorenko is now known as _degorenko|afk17:34
fungipabelanger: log into the "openstack" tenant https://mycloud.rackspace.com/ and go to networking->dns and i think there will be a trystack.org domain listed there17:35
fungipabelanger: should be able to add/update records as needed17:35
openstackgerritJames E. Blair proposed openstack/gertty: Fix rendering error in change screen  https://review.openstack.org/29542317:35
fungijeblair: will try that now--thanks again!17:35
*** jlanoux has quit IRC17:35
*** yamamoto has quit IRC17:35
jeblairfungi: ^ with any luck, that might have been one of those lingering display bugs17:36
*** yamamoto has joined #openstack-infra17:36
*** maishsk has joined #openstack-infra17:36
*** yamamoto has quit IRC17:37
*** pvaneck has joined #openstack-infra17:37
openstackgerritJames E. Blair proposed openstack-infra/infra-specs: Add Nodepool Zookeeper spec  https://review.openstack.org/27877717:38
fungijeblair: confirmed--i can load 94260 at 80x24 now, crash-free!17:38
jeblairaim high!17:39
*** sdake_ has quit IRC17:40
*** dizquierdo has joined #openstack-infra17:40
EmilienMfungi, clarkb: re- Fridays's issue with Puppet logs & symlink. We still have some CI jobs failing because the centos7 does not contain the fix, on OVH. Example: https://review.openstack.org/#/c/294803/17:41
*** chlong|wfh has joined #openstack-infra17:42
pabelangerkambiz: okay, DNS updated17:42
*** heyongli has quit IRC17:43
*** heyongli has joined #openstack-infra17:43
kambizmoo chass grassy us pabelanger17:43
*** pfallenop has quit IRC17:43
*** pfallenop has joined #openstack-infra17:44
*** dimtruck is now known as zz_dimtruck17:44
*** tonytan4ever has quit IRC17:46
fungiEmilienM: it looks like ovh-bhs1 is the only one missing updated images (last successful image build there was 3 weeks ago)17:46
fungiwell, last successful image upload i mean17:46
EmilienMfungi: ok it makes sense now17:46
fungii'll go ahead and delete those so they don't contaminate results17:46
EmilienMfungi: thanks a lot17:46
fungieverywhere else (including ovh-gra1) has updated fine17:46
EmilienMfungi: where do you read that? on http://nodepool.openstack.org/ ?17:46
fungiEmilienM: eventually, but for now i have to pull stats manually via the nodepool client tool17:47
fungiit's not public data17:47
fungibut it definitely should be17:47
jeblairfungi, EmilienM: https://review.openstack.org/29370617:48
fungii think jeblair has the beginnings of an implementation for a nodepool status api17:48
AJaeger_amrith_: I'm back, what do you want to discuss?17:48
fungiyeah, that one! ;)17:48
EmilienM\o/17:48
EmilienMjeblair: nice work17:48
EmilienMfungi: please let me know when ovh is updated and we'll recheck17:49
fungiEmilienM: the ancient devstack-centos7 images have been deleted from ovh-bhs1 now so any that reappear there later will be new enough to contain the fix17:50
fungii've confirmed there are no remaining ready nodes of that type there now17:50
EmilienMexcellent17:50
*** bnemec has joined #openstack-infra17:50
*** dizquierdo has quit IRC17:50
*** doug-fish has joined #openstack-infra17:50
fungii think clarkb had said something about slow uploads timing out in ovh-bhs1, not sure what the current ideas are on addressing that17:51
*** austin81 has left #openstack-infra17:51
*** doug-fish has quit IRC17:52
*** doug-fish has joined #openstack-infra17:52
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Add API functionality for automatic worklist filters  https://review.openstack.org/29542217:52
*** tonytan4ever has joined #openstack-infra17:53
*** heyongli has quit IRC17:53
*** heyongli has joined #openstack-infra17:53
*** jaosorior has quit IRC17:53
*** derekh has quit IRC17:54
*** jordanP has quit IRC17:55
*** erlon has joined #openstack-infra17:56
*** rkukura has quit IRC17:57
clarkbEmilienM: fungi how is the symlink issue a problem with our image s?17:58
*** achanda has joined #openstack-infra17:58
*** yamamoto has joined #openstack-infra17:59
*** lucasagomes is now known as lucas-dinner17:59
EmilienMclarkb: the jenkins script update17:59
clarkbI thought that was an issue of new jenkins treating permissions issues differently than old jenkins17:59
EmilienMclarkb: https://review.openstack.org/#/c/294845/17:59
*** bpokorny has quit IRC18:00
clarkbthat seems like the wrong fix? you should follow the symlink and use root powers to copy?18:02
*** salv-orlando has joined #openstack-infra18:02
*** heyongli has quit IRC18:03
jeblairclarkb: EmilienM said he didn't need those files.18:03
fungiclarkb: well, the issue is in the script regardless, and slave scripts are baked into our images18:03
*** heyongli has joined #openstack-infra18:03
clarkbjeblair: but only because the symlink is deeper into the dir and will be copied anyways but that doesn't hold true in the general case18:04
fungiclarkb: unless you want to consider modifications to jenkins to make it go back to its old behavior to be the proper "fix"18:04
jeblairclarkb: yeah, it's not a generalized script though18:04
anteayaany channel op, openstack meeting -alt and -4 seem to be in need of a topic refresh18:04
clarkbfungi: this works18:04
fungizaro: i forgot, we did notice one behavior change with jenkins. it used to ignore read permission issues when collecting artifacts from slave workspaces, but now it errors and aborts the job during artifact collection18:05
fungizaro: this surfaced as a sudden breakage of a job which was copying a symlink to a file which was unreadable by the jenkins user on the slave. it used to just create a zero-byte file in place of the symlink but now it throws an exception about insufficient permission to the file18:06
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: nodepool: Use db setup for python jobs  https://review.openstack.org/29543318:07
*** Hal1 has quit IRC18:08
jeblairdo we need to tag gearman-plugin?18:09
fungiAJaeger: the analysis at https://etherpad.openstack.org/p/bindep is awesome. that reduces the problem space considerably18:09
thomasemclarkb: mriedem_lunch: https://review.openstack.org/#/c/293585 added '-f23' and tests passed. Would like to get this merged so I can start running Fedora 23 in Nova CI upstream, instead of just my local simulations.18:09
jeblairfungi, clarkb, zaro: maybe i should tag master with 0.2.0 ?18:09
openstackgerritMerged openstack-infra/tripleo-ci: Test overcloud SSL  https://review.openstack.org/28198818:09
*** sputnik13 has quit IRC18:10
thomasemfedora 23 for libvirt LXC in nova CI upstream*18:10
fungijeblair: it's needed if using a secured jenkins lts release, so tagging seems friendly to our downstream consumers18:10
zarofungi: i think this might fix that issue, https://review.openstack.org/#/c/234036/18:10
*** nadya has quit IRC18:11
*** catherineD has joined #openstack-infra18:11
fungizaro: how? the permissions issue is on the slave18:11
*** infra-re_ has joined #openstack-infra18:11
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: sqlalchemy-migrate: Use db setup for python jobs  https://review.openstack.org/29543518:12
clarkbthomasem: +218:12
*** bryan_att has quit IRC18:12
*** sbalukoff has joined #openstack-infra18:12
AJaegerfungi: consider it as as best guess ;) I got some -1 from repos where I was wrong...18:13
*** yamamoto has quit IRC18:13
*** sdake has joined #openstack-infra18:13
*** heyongli has quit IRC18:13
AJaegerfungi, but it should cover 90 per cent ;)18:14
*** heyongli has joined #openstack-infra18:14
fungiAJaeger: well, best guess is all we can do. before we remove database setup from the normal templates we'll want to send an announcement with the list of ones we found and give projects a little bit of time to point out things we missed18:14
*** infra-red has quit IRC18:14
fungii think what you have here is very comprehensive18:15
AJaegerfungi: agreed. Once I'm through, I'll work on the mail. I first wanted everything done18:15
fungiright, i agree that's the best sequence18:15
pabelangerclarkb: have you seen any issues with the new cron element on ubuntu-trusty?18:15
AJaegerfungi, btw. could you abandon https://review.openstack.org/164444 and https://review.openstack.org/200672, please?18:15
zarofungi: ohh wait, that doesn't seem like it has anything to do with the gearman plugin. is the job just doing a cp or scp?18:16
clarkbpabelanger: no18:16
zarojeblair: tagging sounds good to me.18:16
clarkbpabelanger: its pretty simple just a list of packages to install18:16
pabelangerclarkb: great18:16
pabelangerclarkb: are we good to try the puppet migration again to ubuntu-trusty?18:17
jeblairfungi, zaro: tagged 0.2.018:17
fungiAJaeger: why should i abandon those? i was going to un-wip them when it came time to finalize an initial example of adding a other-requirements.txt to a project18:17
fungiand probably squash them of course18:18
clarkbpabelanger: maybe? maybe you want to run the tests on one of the images first?18:18
fungiAJaeger: if the topic is causing a problem on them i can switch it to something else for noe18:19
fungier, for now18:19
pabelangerclarkb: sure, we can see about adding an experiment job to confirm18:19
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: taskflow: Use db setup for python jobs  https://review.openstack.org/29543718:20
*** kzaitsev_mb has quit IRC18:21
AJaegerfungi, Ah, then leave them. I thought they were obsolete18:21
*** gfreeman has joined #openstack-infra18:22
*** bpokorny has joined #openstack-infra18:22
fungiAJaeger: yeah i just want us to have the database situation squared away before un-wip'ing the squashed version of those, but otherwise they're my proposal for developer-friendly bindep introduction in a project (giving them a convenient tox target to find out what packages they're missing, updated to their developer documentation showing the new simpler instructions, et cetera)18:23
*** Sukhdev has quit IRC18:23
clarkbanteaya: mail sent18:23
*** heyongli has quit IRC18:24
anteayaclarkb: thank you, sorry to impose but I do appreciate it18:24
*** dprince has quit IRC18:24
*** heyongli has joined #openstack-infra18:24
*** achanda has quit IRC18:24
gfreemanhi guys.  my CI for my FCP driver started failing suddenly with "Returning exception Invalid input received: Connector doesn't have required information: wwpns to caller"  We didn't change anything on our end, but previously our job suddenly started failing due to an upstream bug with the inherited validate_connector18:25
gfreemani just wanted to check if other people's FCP drivers are having this issue..?18:26
fungiwow. i see lots of acronyms/terms there which i don't understand18:26
fungifcp == fibre channel protocol?18:27
*** sfinucan has quit IRC18:27
AJaegerfungi, thanks for explanation. Now I reviewed the changes and not only read the commit message...18:27
gfreemansorry:  CI = continuous integration, our jenkins stuff.  fcp = fibrechannel protocol, yes18:27
*** catherineD has left #openstack-infra18:28
fungiwwpns == world wide port name service?18:28
gfreemani think so..?18:28
gfreemanbut it's a property that is expected in the "connector" object that cinder's volume/manager.py is expecting be present18:28
fungianyway, you probably want to ask the cinder devs whether something has changed that could impact that, but maybe there are other cinder driver third-party ci maintainers are listening in here too18:29
*** sputnik13 has joined #openstack-infra18:29
gfreemanok ty18:29
fungicinder devs mostly hang out in #openstack-cinder per https://wiki.openstack.org/wiki/IRC18:29
dmsimardYou guys probably know but I noticed there's a logo overlap on the jenkins pages, i.e, https://jenkins04.openstack.org/ (top left)18:29
smcginnisgfreeman: I have not seen issues.18:30
dmsimardI see both the jenkins and the openstack logo overlapping18:30
smcginnisgfreeman: But you can ask in #openstack-cinder to see if anyone else has.18:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: storyboard: Use db setup for python jobs  https://review.openstack.org/29544518:30
fungidmsimard: indeed, it's an interesting composite logo we've created there, likely to annoy both the openstack foundation and cloudbees if their trademark people found out18:30
*** sputnik13 has quit IRC18:30
gfreemansmcginnis:  ok ty, yeah, i should have asked this in the cinder channel, but thanks to you and fungi :)18:30
dmsimardfungi: lol :)18:30
fungii'd like to say that zuul v3 is the "solution" to that logo problem, but we may need to find out a fix sooner if intellectual property lawyers come knocking18:31
*** sputnik13 has joined #openstack-infra18:31
dmsimardfungi: I think that could be ruled as a CSS bug more than a liberate attempt at wrongly using a logo :)18:33
*** sambetts is now known as sambetts|afk18:33
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: nodepool: Use db setup for python jobs  https://review.openstack.org/29543318:33
fungidmsimard: yep, but lawyers are not generally known to be reasonable about such situations18:33
krotscheckAJaeger: Not certain why I was added to the taskflow python-db patch...18:34
*** heyongli has quit IRC18:34
krotscheckAJaeger: I did just add Zara to the storyboard one though.18:34
*** esker has joined #openstack-infra18:34
*** heyongli has joined #openstack-infra18:34
AJaegerkrotscheck: thanks for adding Zara. And indeed, I wanted to add somebody else to taskflow. Thanks for telling me...18:35
*** achanda has joined #openstack-infra18:35
pabelangerAJaeger: have a moment to help land 293767?18:36
*** _amrith_ is now known as amrith18:36
pabelangerthat's for debian-jessie puppet-apply testing18:36
AJaegerpabelanger: do you want to land clarkb's puppet change first? it conflicts with "Revert "Revert "Move puppet jobs to ubuntu-trusty"""18:39
pabelangerAJaeger: on the fence, we are going to do an experimental job first, to make sure puppet-jobs are happy18:39
fungiEmilienM: did you have a link to a recent "unstable" job result? that probably has the answers zaro's looking for18:40
EmilienMfungi: https://review.openstack.org/#/c/294803/18:40
AJaegerpabelanger: let's get it in, we can rebase the big puppet one...18:40
openstackgerritMerged openstack/diskimage-builder: Add psmisc to the packages for ironic-agent  https://review.openstack.org/28936418:42
AJaegerfungi, I've sent all changes for database setup. Will set changes to WIP with exception of those that are infra related and start drafting an email next18:42
fungiAJaeger: sounds great--thanks!18:42
AJaegerpabelanger: could you remvoe your +2 from https://review.openstack.org/#/c/294941/2 , please? neutron does not need it...18:44
*** heyongli has quit IRC18:44
*** heyongli has joined #openstack-infra18:44
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve the worklist UI  https://review.openstack.org/29545018:45
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Allow an update of the SearchCriteriaController to be triggered  https://review.openstack.org/29545118:45
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve the tag-complete directive  https://review.openstack.org/29545218:45
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Add a resource abstraction for worklist filters  https://review.openstack.org/29545318:45
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Allow filters to be set on automatic worklists  https://review.openstack.org/29545418:45
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs  https://review.openstack.org/28944518:46
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: taskflow: Use db setup for python jobs  https://review.openstack.org/29543718:48
openstackgerritMerged openstack-infra/project-config: Add experimental puppet-apply job for debian-jessie  https://review.openstack.org/29376718:48
thomasemclarkb: Thank you!!18:48
*** electrofelix has quit IRC18:49
AJaegerpabelanger, clarkb : I'll rebase  294203 now18:50
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Revert "Revert "Move puppet jobs to ubuntu-trusty""  https://review.openstack.org/29420318:52
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add experimental puppet-openstackci-beaker job for centos-7  https://review.openstack.org/29545518:52
*** IlyaG has quit IRC18:52
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Expose nodepool-builder workers settings  https://review.openstack.org/29433918:52
pabelangerjeblair: fungi: Add the ability to configure the amount of upload-workers for nodepool.  I just tested it locally it worked with my cloud18:53
pabelangerI just used the default value of 418:53
fungipabelanger: cool! excellent datapoint18:54
*** dprince has joined #openstack-infra18:54
fungii wonder if we still need some sort of anti-affinity to prevent trying to upload 4 images at once to the same provider, or if that will generally work with our current providers18:54
*** heyongli has quit IRC18:54
*** heyongli has joined #openstack-infra18:55
*** e0ne has joined #openstack-infra18:55
smarcethey fungi: i was speaking with :jpmaxman  about ram upgrade on openstackid.org18:55
clarkbfungi: we currently don't do that18:55
smarcetfungi: jpmaxman: what about to raise it to 16Gb instead of 8gb18:55
fungismarcet: was there further input on it? i actually haven't started launching the replacement yet (was investigating clarkb's changes to our launch script) so i can make it 16gb no problem18:56
*** zz_dimtruck is now known as dimtruck18:56
*** hashar has joined #openstack-infra18:56
*** tosky has quit IRC18:56
*** tonytan_brb has joined #openstack-infra18:56
clarkbfungi: but thats a good point we could be hampering our current uploads by spamming all the images to each provider at once18:57
*** tonytan_brb has quit IRC18:57
fungismarcet: we can always scale it back down to a smaller server later if we decide it's seriously overpowered based on your subsequent load tests18:57
*** akshai has quit IRC18:57
*** nadya has joined #openstack-infra18:57
smarcetfungi: well i kill the server with 125 concurrent connections18:57
smarcetfungi: yes it would be better 16gb and the run the tests18:58
*** akshai has joined #openstack-infra18:58
smarcetalso we could tweak the sysctl-confs ?18:58
smarcetfungi: like net.core.somaxconn var  to raise it up?18:59
*** tonytan4ever has quit IRC18:59
fungismarcet: sure, we should be able to adjust that through puppet18:59
*** maishsk_ has joined #openstack-infra19:00
*** maishsk has quit IRC19:01
*** maishsk_ is now known as maishsk19:01
smarcetfungi: cool :)19:01
*** rockyg has joined #openstack-infra19:02
*** heyongli has quit IRC19:05
*** heyongli has joined #openstack-infra19:05
smarcetfungi: there is some puppet module that is already used on puppet scripts to set up sysctl-confs ?19:07
*** sputnik13 has quit IRC19:09
*** sputnik13 has joined #openstack-infra19:11
*** mriedem_lunch is now known as mriedem19:12
pabelangerAJaeger: fungi: success for debian-jessie: http://logs.openstack.org/08/295208/1/experimental/gate-infra-puppet-apply-debian-jessie/2de1182/console.html19:13
pabelangerso, I think we can move that out of experimental19:13
pabelangerand also add debian-jessie to the other clouds, since rax works as expected19:13
*** ociuhandu has quit IRC19:13
AJaegerpabelanger: first merge https://review.openstack.org/294203, please19:13
clarkbspot checks of jenkins thread counts looks good. So new jenkins isn't any worse than old jenkins when it comes to that19:13
*** yamamoto has joined #openstack-infra19:14
AJaegerfungi, what do you think of https://etherpad.openstack.org/p/uMABGroiz0 ?19:14
anteayaclarkb: wonderful19:14
pabelangerAJaeger: we'll need 295455 first19:14
pabelangerif we can merge that19:14
pabelangerI can do some puppet job tests19:14
*** tonytan4ever has joined #openstack-infra19:14
pabelangerthen a revert on it19:14
fungismarcet: in openstack-infra/system-config for the openstack_project::slave_common class we just create a file in /etc/sysctl.d19:15
pabelangerthen we should be safe to merge 29420319:15
fungismarcet: granted, that probably only takes effect on reboot if we do it that way19:15
*** heyongli has quit IRC19:15
*** heyongli has joined #openstack-infra19:15
fungismarcet: though looks like we work around that by subscribing an exec19:15
*** jtomasek has quit IRC19:16
jeblairclarkb, fungi, pabelanger: i wouldn't worry about anti-affinity for the moment; i think whatever we do will be better than the status quo.19:16
fungijeblair: right now we push them up completely serially right?19:17
fungiupload them, i mean19:18
AJaegerpabelanger: that is for centos-7 - why do we need that?19:18
pabelangerAJaeger: Opps19:18
pabelangerAJaeger: sorry, that should be ubuntu-trusty19:18
pabelangerlet me fix19:18
jeblairsmarcet, fungi, jpmaxman: i think we'd want to tune appache connections first before we start on tcp connection backlogs19:18
*** roxanaghe has quit IRC19:18
jeblairfungi: we have 4 threads at the moment; we went to 16 during the sprint and that ~ saturated our outbound pipe/cpu19:19
jeblairfungi: (so 16 is what i would like to try for nodepool.o.o)19:19
*** tonytan4ever has quit IRC19:19
fungiwfm19:19
jpmaxmanjeblair agreed the web server should be tuned for the server config19:20
jeblairjpmaxman: cool; i'm only partly here and haven't followed everything; just wanted to make sure that wasn't missed :)19:21
*** yamamoto has quit IRC19:21
*** baoli_ has quit IRC19:22
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add experimental puppet-openstackci-beaker job for ubuntu-trusty  https://review.openstack.org/29545519:22
pabelangerAJaeger: ^19:22
openstackgerritMonty Taylor proposed openstack-infra/shade: Create clouds in Functional Test base class  https://review.openstack.org/29529619:23
openstackgerritMonty Taylor proposed openstack-infra/shade: Deal with is_public and ephemeral in normalize_flavors  https://review.openstack.org/29538719:23
openstackgerritMonty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset  https://review.openstack.org/29538819:23
fungijeblair: brief summary is that load tests indicate past about 125 concurrent users openstackid.org becomes unusable due to apache threads gobbling all available ram and swap. i'm building a 16gb replacement for the current 2gb server instance first and then we can do further load testing and figure out a sweet spot for apache tuning and if necessary also kernel tcp/ip stack tuning19:24
smarcetfungi: ok19:24
jeblairfungi: gotcha... are we targeting ~200 simulusers?19:25
*** heyongli has quit IRC19:25
*** heyongli has joined #openstack-infra19:25
AJaegerI need some zuul debugging help. Could anybody tell me why the barbican-api-guide post job is not run, please? See for example http://logs.openstack.org/06/0600a260891c84eb9550fad2cdb7bec9a9c4ed8c/post/ - the job is not listed there at all19:26
fungijeblair: i think the initial test was 500, and that knocked the server offline due to memory starvation and swap causing i/o contention19:26
jeblairfungi: yeah, just wondering what our goal is19:26
jeblairAJaeger: will do19:26
AJaegerpabelanger: LGTM19:26
fungimy guess is 500-1000 but i'll let smarcet and jpmaxman answer that19:26
AJaegerthanks, jeblair19:26
fungithey're estimating significant additional load on it due to the summit mobile app19:27
AJaegerjeblair: I already checked and couldn't find anything obviously wrong19:27
*** esker has quit IRC19:27
jpmaxmanjeblair fungi I think 500 - 1000 should be ok.  longer term we need a salability plan for this server19:27
jpmaxmanin talking to smarcet the first step toward that would be to externailize the redis server19:28
*** dtardivel has quit IRC19:28
*** gordc has quit IRC19:28
*** ociuhandu has joined #openstack-infra19:28
jpmaxmanI'm talking to jimmy later today to discuss when we want to do that19:28
jpmaxmanI'm guessing post summit19:28
*** achanda has quit IRC19:28
AJaegerpabelanger: change looks fine19:28
*** qb_ has joined #openstack-infra19:29
AJaegerjeblair: looking at the zuul gate log file I see "<Job barbican-api-guide> master "19:29
fungijpmaxman: is there much opportunity to move the high-impact bits into openstackid-resources on a separate server, or is this heavily identification/authentication code which would need to remain in the core openstackid codebase?19:29
openstackgerritEmilien Macchi proposed openstack-infra/project-config: gerritbot/puppetopenstack: add stable/mitaka support  https://review.openstack.org/29546919:30
*** angdraug has joined #openstack-infra19:31
AJaegerfungi, do you have any feedback on my draft email at https://etherpad.openstack.org/p/uMABGroiz0 ?19:31
*** mylu has quit IRC19:31
jeblairAJaeger: https://jenkins02.openstack.org/job/barbican-api-guide/1/console19:32
openstackgerritJames E. Blair proposed openstack-infra/project-config: Publish console logs for api-guide jobs  https://review.openstack.org/29547019:33
AJaegerjeblair: thanks. But why does it not show up on logs.openstack.org?19:33
fungiAJaeger: python34 as well?19:33
jeblairAJaeger: ^19:33
*** mylu has joined #openstack-infra19:33
AJaegerfungi, yes, indeed19:34
AJaegerjeblair: thanks!19:34
jeblairAJaeger: np19:34
*** smarcet has quit IRC19:34
*** Sukhdev has joined #openstack-infra19:34
AJaegerjeblair: now I'll be debugging why it didn't upload anything ;/19:35
*** agentle has quit IRC19:35
*** smarcet has joined #openstack-infra19:35
*** heyongli has quit IRC19:35
*** heyongli has joined #openstack-infra19:36
fungiAJaeger: the publishers list for that job-template is only the ftp upload for the site, no console-logs publisher19:36
*** infra-red has joined #openstack-infra19:36
jeblairAJaeger: i'm reading that draft email19:36
AJaegerjeblair: feel free to comment! Thanks!19:37
jpmaxmanfungi I need to discuss with smarcet but generally the idea would be to run an independent redis server and introduce a load balanced horizontal scaling option.  I'm not sure how much you could slice and dice the code or if that'd even make sense.19:37
AJaegerfungi, jeblair already fixed that with 29547019:37
fungijpmaxman: sure, i'm completely oblivious to what sort of queries you were load testing or whether you were load testing straight auth/id calls19:38
anteayaAJaeger: removing soon access/soon removing access ?19:38
jeblairAJaeger: how does this relate to the PTI19:38
lifelesssigmavirus24: sure19:38
fungiAJaeger: hrm, openstackgerrit didn't mention that change19:39
sigmavirus24lifeless: thanks!19:39
jeblairfungi: 19:33 < openstackgerrit> James E. Blair proposed openstack-infra/project-config: Publish console logs for api-guide jobs  https://review.openstack.org/29547019:39
*** infra-re_ has quit IRC19:39
AJaegerfungi, 6 minutes ago19:39
AJaegerjeblair: the PTI is not changed, we still use tox -e py27 etc.19:40
AJaegerit's just the name of the jobs19:40
fungiAJaeger: oh! i misread and thought you said you proposed it. now i see19:40
jeblair[fwiw, i acked AJaeger's request for root assistance so that we would all know that someone was working on helping him]19:40
pabelangerclarkb: do you mind helping 295455?19:41
*** terriyu has joined #openstack-infra19:42
jeblairAJaeger: re pti: i agree, i don't think it's affected19:42
fungiyep, i have too many irons in the fire. i should finish the things i'm in the middle of rather than failing at paying attention to everything in irc19:43
clarkbpabelanger: why do we need a new job? just run the existing jobs in experimental19:43
clarkbpabelanger: oh right the node changes19:44
pabelangerclarkb: ya, that19:44
clarkband we don't template the node?19:44
jeblairAJaeger: draft msg lgtm19:44
*** infra-re_ has joined #openstack-infra19:44
clarkbya its this broken setup I tried to address, nevermind19:44
AJaegerthanks, jeblair ! fungi, jeblair: I'll send it out then in a few minutes19:44
fungiAJaeger: yes, with the couple of tweaks i made to include python34 jobs, it looks great. thanks for working on this!19:45
AJaegerthanks for these tweaks, fungi19:46
*** heyongli has quit IRC19:46
*** heyongli has joined #openstack-infra19:46
AJaegerif any project-config core has time to review the db changes at https://review.openstack.org/#/q/project:openstack-infra/project-config+topic:bindep+status:open , it would be nice. I can then +A those myself during some more idle hours once I get proper +1s - like I have already on ironic. Most are marked as WIP, that's just to give time for review19:47
*** terriyu has quit IRC19:48
*** infra-red has quit IRC19:48
*** agentle has joined #openstack-infra19:48
*** tonytan4ever has joined #openstack-infra19:48
*** sridhar_ram has joined #openstack-infra19:48
fungiyeah, main reason to not approve immediately is for higher-change-volume repos we're going to get lots of questions about NOT_REGISTERED job results between when zuul updates its config and the changes get applied through jjb19:48
*** baoli has joined #openstack-infra19:49
*** baoli has quit IRC19:49
anteayaoh looks like -meeting needs a topic refresh as well as -alt and -419:49
*** baoli has joined #openstack-infra19:50
*** maishsk has quit IRC19:50
*** maishsk_ has joined #openstack-infra19:50
fungianteaya: took care of except -alt since it doesn't seem to be idle19:51
AJaegerfungi, I plan to do it early my morning - so around 7:00 UTC. That's quite idle on zuul...19:51
fungiAJaeger: yes, that will be ideal19:52
anteayafungi: thanks19:52
*** achanda has joined #openstack-infra19:52
*** tonytan4ever has quit IRC19:53
*** harlowja_at_home has quit IRC19:53
*** harlowja_at_home has joined #openstack-infra19:54
openstackgerritMerged openstack-infra/project-config: Publish console logs for api-guide jobs  https://review.openstack.org/29547019:54
*** heyongli has quit IRC19:56
*** heyongli has joined #openstack-infra19:56
*** IlyaG has joined #openstack-infra19:57
jeblairAJaeger: if you feel like doing more work, you could go ahead and create all the jobs in jjb :)19:57
openstackgerritMerged openstack-infra/project-config: Add experimental puppet-openstackci-beaker job for ubuntu-trusty  https://review.openstack.org/29545519:57
*** andymaier has quit IRC19:58
AJaegerjeblair: another time ;)19:58
jeblair:)19:58
*** piet has quit IRC19:59
fungianteaya: i took care of -alt now that the meeting ended. thanks for the heads up!20:00
*** akshai has quit IRC20:00
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Refactored Locations Images Model  https://review.openstack.org/29548320:01
anteayafungi: thank you20:01
openstackgerritMerged openstack-infra/openstackid-resources: Refactored Locations Images Model  https://review.openstack.org/29548320:02
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Revert "Revert "Move puppet jobs to ubuntu-trusty""  https://review.openstack.org/29420320:03
AJaegerreviews for ^ are welcome to move us off devstack-trusty. I just rebased...20:04
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE: Run tempest serially  https://review.openstack.org/29548720:05
*** heyongli has quit IRC20:06
*** kzaitsev_mb has joined #openstack-infra20:06
*** heyongli has joined #openstack-infra20:06
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Move lbaas v2 driver choice into gate hook  https://review.openstack.org/29548820:08
*** agentle has quit IRC20:10
*** gordc has joined #openstack-infra20:10
*** baoli has quit IRC20:11
*** baoli has joined #openstack-infra20:11
clarkbAJaeger: you are really on top of things, any idea if any of the periodic-mitaka stuff has been proposed or the tempest/d-g tests against mitaka are proposed?20:12
*** baoli has quit IRC20:12
clarkbAJaeger: I am trying to make sure that all gets pushed up today and want ot make sure I don't duplicate effort20:12
*** baoli has joined #openstack-infra20:13
*** rkukura has joined #openstack-infra20:13
AJaegerclarkb: I don't remember anything like that. I might have missed it if it come in today, since I wanted to get the db stuff done.20:15
pabelangermordred: Are you going to be tagging glean or should I make a request on the ML to confirm everybody has their patches in?20:15
mordredpabelanger: I can tag20:15
clarkbAJaeger: no problem and thanks20:15
*** heyongli has quit IRC20:16
*** heyongli has joined #openstack-infra20:17
clarkbAJaeger: what about translations jobs for mitaka aainst openstack-docs? is that later?20:17
*** e0ne has quit IRC20:17
mordredianw: (if you're up) you have 3 outstanding patches for glean - are they important for a release? or is your latest patch good20:17
*** tongli has joined #openstack-infra20:18
ianwmordred: i'm up :)  one of them is20:18
*** ihrachys has quit IRC20:18
ianwhttps://review.openstack.org/#/c/293826/ ... which has merged20:19
mordredianw: ok. awesome20:19
ianwoh yeah, the other 3, the ones you're actually talking about, are not important, just testing cleanups.  one has a -1 i need to correct20:20
*** akshai has joined #openstack-infra20:20
AJaegerclarkb: that's for later - I expect openstack-manuals will be branched a month or so later20:20
*** tongli has quit IRC20:20
*** pcaruana has joined #openstack-infra20:21
ianwmordred: we've tested that; i can revert https://review.openstack.org/#/c/294363/ when there's a release20:21
clarkbAJaeger: ok, works for me20:21
AJaegerclarkb: so, yes, I took care of translations so far. Sorry, didn't count that as periodic-mitaka ;((20:21
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Move lbaas v2 driver choice into gate hook  https://review.openstack.org/29548820:21
clarkbAJaeger: ya I am mostly focusing on bitrot jobs and then the tempest periodic and master + branch jobs20:21
openstackgerritMonty Taylor proposed openstack-infra/shade: Create clouds in Functional Test base class  https://review.openstack.org/29529620:23
openstackgerritMonty Taylor proposed openstack-infra/shade: Deal with is_public and ephemeral in normalize_flavors  https://review.openstack.org/29538720:23
openstackgerritMonty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset  https://review.openstack.org/29538820:23
*** |-paul-| has joined #openstack-infra20:23
*** baoli has quit IRC20:24
AJaegerclarkb: please create bitrot-db jobs like I did for liberty.20:24
clarkbAJaeger: I did20:25
AJaegerLooking at my changes at https://review.openstack.org/#/q/project:openstack-infra/project-config+topic:bindep+status:open, I suggest you use for nova the db-bitrot variant  - and the normal one everywhere else20:25
AJaegerOnce my changes are in, we can rework the mitaka bitrot ones..20:25
AJaegerOr I update - I fear this creates lots of merge conflicts for me ;(20:26
*** heyongli has quit IRC20:27
*** MarkAtwood has quit IRC20:27
*** heyongli has joined #openstack-infra20:27
*** esker has joined #openstack-infra20:28
openstackgerritClark Boylan proposed openstack-infra/project-config: Add mitaka stable branch jobs  https://review.openstack.org/29549920:30
*** kzaitsev_mb has quit IRC20:30
openstackgerritClark Boylan proposed openstack-infra/project-config: Move stable compat jobs up to mitaka  https://review.openstack.org/29550020:30
clarkbAJaeger: I think we get your changes in first20:31
*** roxanaghe has joined #openstack-infra20:31
clarkbAJaeger: and rebase 295499 on top and do the db switch as necessary20:31
sdagueAJaeger: fyi - https://review.openstack.org/#/c/295489/ - the docs theme breaks sphinx extensions that add css20:32
openstackgerritgreghaynes proposed openstack/diskimage-builder: ironic-python-agent should use console output  https://review.openstack.org/29550120:32
openstackgerritPaul Belanger proposed openstack-infra/puppet-mediawiki: Switch to ubuntu-trusty DIBS  https://review.openstack.org/29550220:32
*** dprince has quit IRC20:32
*** agentle has joined #openstack-infra20:34
*** kzaitsev_mb has joined #openstack-infra20:35
*** nadya has quit IRC20:36
*** sc68cal has joined #openstack-infra20:36
*** heyongli has quit IRC20:37
*** heyongli has joined #openstack-infra20:37
AJaegersdague: thanks20:38
AJaegerclarkb: works for me ;)20:39
*** terriyu has joined #openstack-infra20:39
*** mattt has joined #openstack-infra20:39
*** regXboi has joined #openstack-infra20:40
fungiclarkb: i'm trying to update your 285477 change, what's the sanest way to rebase that on top of the change that moved shade-launch-node.py to launch-node.py?20:40
mriedemclarkb: any idea what version of libvirt you were testing against here? https://bugs.launchpad.net/nova/+bug/155880720:40
openstackLaunchpad bug 1558807 in OpenStack Compute (nova) "Volume attached at different path than reported by nova/cinder when using config drive" [Undecided,New]20:40
AJaegerclarkb: let's see how timing works out - I still needs some reviews... https://review.openstack.org/#/c/295034/ and https://review.openstack.org/#/c/294939/ and https://review.openstack.org/294938 could merge tomorrow if there're enough reviews20:40
AJaegerand https://review.openstack.org/#/c/294953/ is ready as well20:41
clarkbfungi: :) there is a reason I haven't done it yet. I don't think there is a sane way, its going to involve editing in a vimdiff likely20:41
mattthi all, trying to understand an issue i'm having testing a gate check locally (gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty) -- this functional test spawns ansible and for the most part everything is well20:41
clarkbfungi: the problem is more than the filename change, a bunch of ansible logic replaced python logic in launch node20:41
AJaegersdague: is https://review.openstack.org/#/c/294953/ solving the same problem?20:41
fungiclarkb: right now i'm trying an --amend of it rebased onto master and cleaning up the merge conflicts20:41
fungiclarkb: oh, yeah i see that20:42
openstackgerritPaul Belanger proposed openstack-infra/puppet-openstackci: Switch to ubuntu-trusty DIBS  https://review.openstack.org/29550520:42
clarkbfungi: I should also remove the code that allows specifying the volume device name beacuse apparently nova removed that last release but the docs haven't caught up20:42
matttthe problem i'm having is that the execution does not match CI, for example if the ansible playbook installs a python package onto the host, it installs into tox's venv, however i do not see that behaviour when i run locally20:42
fungiclarkb: i don't actually need cinder support this time around. is there anything else this change fixes not mentioned in the commit message? if so i may just skip it for now20:42
clarkbfungi: no you should only need that if yo uneed cinder volume support20:43
fungiokay, cool. i'll not complicate matters for my current need20:43
clarkbmattt: it will do that if it runs under tox and ansible uses the python installed into a venv20:43
clarkbmattt: I would check tox versions and invocations match locally and not locally20:44
matttclarkb: i've tried old and new versions of tox locally, and behaviour is consistent, so i'm guessing it's invocation, can you point me to where i can see how tox is invoked in CI ?20:44
clarkbmattt: can you link to the job log where you see this behavior?20:45
clarkbmattt: it should be logged for you20:45
matttclarkb: http://logs.openstack.org/65/295265/5/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/e6ce8f8/20:45
clarkbmattt: it starts at http://logs.openstack.org/65/295265/5/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/e6ce8f8/console.html#_2016-03-21_16_54_23_48720:45
*** baoli has joined #openstack-infra20:46
*** heyongli has quit IRC20:47
*** verdurin_ has joined #openstack-infra20:47
*** heyongli has joined #openstack-infra20:48
clarkbAJaeger: ok will work on reviews now20:48
AJaegerthanks, clarkb20:48
matttclarkb: ah right, thank you !20:48
*** verdurin has quit IRC20:49
*** verdurin_ is now known as verdurin20:49
*** moravec has quit IRC20:49
*** BigWillie has quit IRC20:51
fungiclarkb: does the updated launch-node.py retain the ability to run as non-root, or will i need to sudo now?20:52
*** moravec has joined #openstack-infra20:52
clarkbfungi: I think non root broke before my changes, it has to do with unlinking the cache for ansible20:53
fungiahh, too bad. okay thanks20:53
clarkbI ran into it and just ignored it >_>20:55
*** hichihara has joined #openstack-infra20:55
clarkbAJaeger: a  couple of these changes have enough +2's what is the WIP waiting on?20:55
*** eharney has quit IRC20:56
clarkbfungi: I was already juggling the cinder stuff and didn't want to overcomplicate things20:56
fungiclarkb: hah, also we lost +x perms on that script20:56
clarkbbut that would be another good improvmeent to make the admin group unlink the ansible cache20:56
*** ihrachys has joined #openstack-infra20:56
pabelangerso, https://review.openstack.org/#/c/295502/ is the fix to get puppet beakerci jobs working on ubuntu-trusty20:56
pabelangersimple rename20:56
pabelangerbut need to see how to land it properly20:57
AJaegerclarkb: I wanted to give repo experts time to review - and since these changes will give a NOT_REGISTERED node, plan to merge them at an idle time20:57
clarkbpabelanger: I think it will have to merge after the switch as otherwise it will fail20:57
*** heyongli has quit IRC20:57
clarkbAJaeger: gotcha sounds good20:57
fungismarcet: jpmaxman: how about 15gb? rackspace has no 16gb (next flavor up from 15 is 30)20:58
*** heyongli has joined #openstack-infra20:58
pabelangerclarkb: Yup, merge JJB patch, run JJB, then deal with puppet failures20:58
pabelangerI have time tonight if people are up for it20:59
pabelangerotherwise, another day20:59
clarkbpabelanger: make sure that the experiemntal jobs you wrote work first?20:59
matttclarkb: ansible has an ansible_python_interpreter variable which i can add to the inventory, i just need to flip that to the tox venv and i should be good to go, thanks for your help!20:59
pabelangerclarkb: Ya, it did. Just waiting for it to post to review.o.o20:59
AJaegerclarkb, pabelanger, could you review https://review.openstack.org/294953 as well, please?20:59
clarkbianw: https://review.openstack.org/#/c/294871/1 would be a good one to get into dib to fix eth1 problems in rax21:00
*** rkukura has quit IRC21:01
ianwclarkb: yes, related to https://review.openstack.org/#/c/294384/21:01
*** amotoki has joined #openstack-infra21:02
*** moravec1 has joined #openstack-infra21:02
*** rossella_s has quit IRC21:03
*** sfinucan has joined #openstack-infra21:03
*** rossella_s has joined #openstack-infra21:03
smarcetfungi: 15Gb is fine21:04
*** infra-red has joined #openstack-infra21:04
dmsimardDo we have to poke -infra to delete a mistaken git tag ?21:04
*** infra-red has quit IRC21:04
pabelangerdmsimard: we usually can't delete it21:04
pabelangerso you'd have to increment and create again21:05
*** moravec has quit IRC21:05
AJaegerdmsimard: git is distributed, so everybody accessing the repo already has the tag21:05
dmsimardpabelanger: it's not a huge deal, though gerrit just mentions that it requires the force push permission21:05
dmsimardAJaeger: *nods*21:05
*** thorst_ has quit IRC21:06
*** infra-re_ has quit IRC21:07
*** rguillebert has quit IRC21:07
fungiclarkb: mordred: so... i'm unconvinced that the new launch-node.py does what we want. it looks like when you puppet a replacement server it doesn't actually use the requested hostname because the inventory decides that duplicate hostnames need to be handled as uuids instead, then proceeds to use the uuid as the hostname in the deployed instance and fails to match anything in the global site manifest21:07
*** heyongli has quit IRC21:08
clarkbfungi: fun21:08
clarkbfungi: in that case just checkout my change or revert :)21:08
*** heyongli has joined #openstack-infra21:08
pabelangerdmsimard: problem is usually downstream in our git mirrors and cached git repos on DIBs21:08
pabelangerdmsimard: double check with fungi21:08
fungiclarkb: yes, i'll give your version a shot since it's pre-ansible21:08
clarkba revert + my change then get working ansible in may be the best process21:08
pabelangerAJaeger: sorry, missed your comment21:09
*** moravec1 has quit IRC21:10
fungidmsimard: tag deletion is not recommended. it doesn't propagate on remote update, so people will still have local copies of the tag. also if you attempt to reuse that tag later, it won't actually replace the original tag in any local clones, so creates issues for things like release jobs running on long-lived workers21:10
*** thorst has joined #openstack-infra21:10
*** terriyu has quit IRC21:11
dmsimardfungi: fair enough - the tag has just been created and it's for packstack. I'll just tag on top of it. We changed our mind on the adopted semver format at the last minute.. Our bad.21:11
*** thorst_ has joined #openstack-infra21:11
AJaegerpabelanger: no problem21:12
*** kgiusti has left #openstack-infra21:13
*** kgiusti has quit IRC21:13
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config: Add hexdump to bindep-fallback.txt  https://review.openstack.org/29552121:14
AJaegerdims_: could you review https://review.openstack.org/294948 and https://review.openstack.org/295437 , please?21:15
*** thorst has quit IRC21:15
dims_AJaeger : ack21:15
AJaegerproject-config cores: Could I  get a second +2 on https://review.openstack.org/#/c/294953/ , please? pabelanger reviewed already21:15
*** thorst_ has quit IRC21:15
AJaegerAnd also please on https://review.openstack.org/#/c/295034/ - clarkb gave one +2.21:15
*** moravec has joined #openstack-infra21:15
dims_AJaeger : +121:16
*** ldnunes has quit IRC21:16
pabelangerAJaeger: so, I think we are ready for 29550521:17
pabelangeronly left a +1 since my original patch21:17
AJaegerthanks, dims_21:17
AJaegerpabelanger: I've rebased 295505 to often to review ;(21:17
*** heyongli has quit IRC21:18
AJaegerso, who wants to help get finally rid of devstack-trusty on puppet? 295505 is the review...21:18
*** heyongli has joined #openstack-infra21:18
fungiclarkb: fun, "from novaclient.v1_1 import client as Client11" is raising "ImportError: No module named v1_1"21:19
clarkbpabelanger: AJaeger where are the job results from the experimental job that ran there?21:19
fungii guess we need older novaclient?21:19
clarkbfungi: ya you have to use older novaclient21:19
*** smarcet has quit IRC21:19
fungii'll make a venv in that case21:19
clarkbfungi: my change was built on top of the shade change that addressed that21:20
clarkbfungi: I think, you would have to confirm that there are no v1_1 references21:20
pabelangerclarkb: https://jenkins02.openstack.org/job/experimental-openstackci-beaker-ubuntu-trusty/1/21:20
pabelangerstill waiting for debian-jessie job21:20
*** regXboi has quit IRC21:20
clarkbpabelanger: how worried for the puppet-openstack jobs shoulkd we be?21:21
clarkbmaybe get EmilienM to weigh in? otherwise I am redy to try again21:21
*** keedya has quit IRC21:21
pabelangerclarkb: I can confirm with EmilienM but I believe they have a gem now, so it makes it easy to update the nodesets21:21
pabelangerI was going to look into that once in 10mins21:22
EmilienMI'm on my phone right now, trying to look what's up21:23
pabelangerclarkb: I believe we just need to patch puppet-openstack_extras like we did to puppet-openstackci21:23
EmilienMOur gems does not handle nodesets afik21:24
clarkbpabelanger: well that gets tricky because on some of the jobs I kept ostype because of that weirdness21:24
clarkbpabelanger: so maybe its fine for them and we can transition them when they are ready and kill the ostype portion of those templates?21:24
lifelesscan I get someone to lookup a config setting for gerrit? cache.web_sessions.maxAge21:24
clarkblifeless: 24 hours iirc21:24
* AJaeger waves good bye21:24
lifelessI suspect its not defined - but want to be sure21:24
lifelessclarkb: ^21:24
prometheanfireianw: the dib non-voting stuff can merge now :D https://review.openstack.org/#/c/293193/21:24
clarkblifeless: https://git.openstack.org/cgit/openstack-infra/puppet-gerrit/tree/templates/gerrit.config.erb#n10821:25
EmilienMpabelanger: puppet-openstack_spec_helper does not manage nodesets21:25
lifelessclarkb: thanks21:26
lifelessclarkb: so, is there any reason we can't raise that to something more friendly - like 1m ?21:26
lifelessclarkb: I'd really like to not have to log in again at the start of every week21:26
clarkblifeless: no, iirc its 384 bytes per entry so at several thousand entries we should still be well within memory limits21:27
clarkblifeless: this came up a few months back not sure why it hasn't been bumped21:27
pabelangerEmilienM: ya, I think puppet-openstack_extras ?21:28
EmilienMNeither21:28
*** asettle has joined #openstack-infra21:28
pabelangerEmilienM: other wise, we'll need to patch each module21:28
pabelangerwhich I can script21:28
*** aysyd has quit IRC21:28
*** heyongli has quit IRC21:28
EmilienMWe use msync21:28
pabelangerclarkb: okay, make sense21:28
*** heyongli has joined #openstack-infra21:28
pabelangerEmilienM: right, forgot about that21:28
EmilienMI'm afk, will catch up later. Let me know21:29
*** rfolco has quit IRC21:29
*** doug-fish has quit IRC21:29
*** weshay has quit IRC21:30
pabelangerclarkb: So, I think we're in good shape to know what to expect. At least of the issue we seen first time around21:30
*** jpr has quit IRC21:30
lifelessclarkb: I finally found a round tuit for it :)21:30
*** rkukura has joined #openstack-infra21:31
openstackgerritlifeless proposed openstack-infra/puppet-gerrit: Set gerrit login cache to 2m  https://review.openstack.org/29553021:31
*** doug-fish has joined #openstack-infra21:32
*** asettle has quit IRC21:32
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config: [WIP] Support py34 functional tests for Oslo.Messaging  https://review.openstack.org/29553121:34
*** doug-fis_ has joined #openstack-infra21:34
*** amrith is now known as _amrith_21:35
*** achanda has quit IRC21:35
*** doug-fish has quit IRC21:36
*** lakshmiS has quit IRC21:36
*** nadya has joined #openstack-infra21:36
*** zeih has joined #openstack-infra21:37
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Fix on entity events  https://review.openstack.org/29553421:38
clarkbdebian jessie and fedora 23 builds both failed today21:38
openstackgerritMerged openstack-infra/openstackid-resources: Fix on entity events  https://review.openstack.org/29553421:38
*** doug-fis_ has quit IRC21:38
*** heyongli has quit IRC21:38
*** heyongli has joined #openstack-infra21:39
*** sdake_ has joined #openstack-infra21:39
*** gfreeman has quit IRC21:39
*** fawadkhaliq has joined #openstack-infra21:41
clarkbovh uploads appear to be suffering from the thing vexxhost was suffering from (same error on our end at least)21:41
*** nadya has quit IRC21:41
clarkband rax uploads are timing out21:41
*** sdake has quit IRC21:41
clarkbvexxhost is still suffering from that too21:41
*** terriyu has joined #openstack-infra21:42
clarkbmnaser: did you have any luck reproducing with that imgae?21:42
*** sc68cal has quit IRC21:42
fungiclarkb: your change is what i was testing that was raising ImportError21:43
fungiso presumably not21:43
clarkbfungi: using the shade-launch-node script?21:43
fungiclarkb: yeah, there's still a reference in utils.pu21:43
fungier, .py21:43
clarkbgotcha21:43
fungialso dns.py21:44
clarkboh thats something else to convert ot shade then21:44
*** rockyg has quit IRC21:44
*** rkukura has quit IRC21:44
fungiwhat vintage novaclient will this require?21:44
clarkbmy novaclient venv has 2.31.021:45
*** tpsilva has quit IRC21:47
fungiperfect--thanks!21:47
*** achanda has joined #openstack-infra21:47
jheskethMorning21:48
*** IlyaG has quit IRC21:48
*** jpr has joined #openstack-infra21:48
anteayamorning jhesketh21:49
*** heyongli has quit IRC21:49
*** heyongli has joined #openstack-infra21:49
clarkbif the ovh uploads issue persists we should see what mnaser has found and loop ovh in21:49
clarkbpossible its the same problem in both places depending on how they deploy/we upload21:50
*** |-paul-| has quit IRC21:50
*** terriyu has quit IRC21:51
*** doug-fish has joined #openstack-infra21:51
*** asettle has joined #openstack-infra21:52
*** tphummel has quit IRC21:52
*** kzaitsev_mb has quit IRC21:53
*** doug-fish has quit IRC21:55
*** hashar has quit IRC21:55
*** doug-fish has joined #openstack-infra21:55
*** cdent has quit IRC21:56
*** kzaitsev_mb has joined #openstack-infra21:56
*** sfinucan has quit IRC21:57
*** esker has quit IRC21:57
*** jpr has quit IRC21:58
*** sridhar_ram has quit IRC21:58
*** smarcet has joined #openstack-infra21:58
*** esker has joined #openstack-infra21:58
smarcetfungi: before to spin up a new opesntackid.org node21:58
smarcetwe need to fix the puppet script21:58
smarcetbc on install its try to do the seeding on the db21:58
fungismarcet: jpmaxman: Notice: /Stage[main]/Openstackid/Exec[deploy-site]/returns:   SQLSTATE[23000]: Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails (`openstackid_production`.`oauth2_client`, CONSTRAINT `oauth2_client_resource_server_id_foreign` FOREIGN KEY (`resource_server_id`) REFERENCES `oauth2_resource_server` (`id`)) (SQL: delete from21:58
fungi`oauth2_resource_server`)21:59
fungithat?21:59
*** agentle has quit IRC21:59
*** heyongli has quit IRC21:59
smarcetdo u remember this one https://review.openstack.org/#/c/221716/21:59
fungiif so, i just hit it and was starting to look into it21:59
smarcetyes21:59
*** heyongli has joined #openstack-infra21:59
smarcetits trying to re populate db21:59
smarcetthat is why i created https://review.openstack.org/#/c/221716/22:00
smarcetto disallow the db seeding22:00
*** amotoki has quit IRC22:00
*** agentle has joined #openstack-infra22:01
*** sdake_ is now known as sdake22:02
fungismarcet: are you working on finishing that change?22:02
fungii just don't want to duplicate any work, step on any toes22:02
smarcetwell its seems that something from db got broken22:03
fungiyou mentioned something about redis failing and then set it to work in progress22:03
smarcetnope i am not working22:03
*** _ari_ has quit IRC22:03
*** knikolla has quit IRC22:04
smarcetfungi: nope that is to have a boolean to not allow the db seeding22:04
*** sridhar_ram has joined #openstack-infra22:04
fungismarcet: right, i read through the change, just wondering why it's work in progress rather than ready to review22:04
smarcetfungi: produiton db got broken22:05
smarcetis any recent backup?22:05
clarkbshould be one from 0000UTC ish yesterday?22:05
clarkber I guess thats today UTC time22:05
fungismarcet: oh, you mean my attempt to deploy a replacement server modified the production database? i'll get the backup handy now22:06
*** pcaruana has quit IRC22:06
fungii agree we should make sure this deploy tooling is idempotent22:06
smarcetfungi: yes deleted the resource server settings i am afraid22:06
smarcetfungi: yes i will conclude https://review.openstack.org/#/c/221716/ tomorrow first thing morning22:07
clarkbwoot recheck of my serial tempest change caught osic on the neutron full job (previous runs used osic on nova net full) this will give us comparable data hopefully22:08
fungismarcet: yes, most recent database dump appears to be from 22 hours ago22:08
anteayaclarkb: yay22:08
fungismarcet: also i've deleted the replacement server instance so that we won't accidentally re-break the production database if puppet reruns on it22:08
smarcetfungi: ok could be restore that22:09
fungismarcet: do you want the entire dump restored or do i need to try to skip specific tables?22:09
smarcet?22:09
smarcetentire22:09
fungidoing that now22:09
smarceti dont know what got broken22:09
smarcetsorry22:09
*** heyongli has quit IRC22:09
*** rkukura has joined #openstack-infra22:09
*** heyongli has joined #openstack-infra22:09
*** tphummel has joined #openstack-infra22:11
smarcetfungi: tnx22:11
fungismarcet: it's applying now22:11
smarcetok roger that22:12
smarcetthx again :)22:12
fungismarcet: all done. do i need to restart any services?22:12
*** sdake_ has joined #openstack-infra22:12
craigeo/22:12
*** sigmavirus24 is now known as sigmavirus24_awa22:13
smarcetfungi: nope22:13
smarcetchecking on it22:13
*** sridhar_ram has quit IRC22:13
smarcetfungi: i am not seeing the api endpoints table beeing populated22:14
smarcetcheckgin on sql22:14
*** maishsk_ has quit IRC22:15
fungismarcet: the oauth2_api_endpoint table?22:15
smarcetfungi: SELECT * FROM openstackid_production.oauth2_api;22:15
smarcetis empty22:15
smarcetSELECT * FROM openstackid_production.oauth2_api_endpoint; also empty22:15
*** sdake has quit IRC22:15
smarcetalso this one SELECT * FROM openstackid_production.oauth2_api_endpoint;22:16
fungiERROR 1142 (42000): SELECT command denied to user 'os_prod_site'@'10.209.4.108' for table 'oauth2_api'22:16
clarkbhttp://paste.openstack.org/show/491369/ is the fedora-23 image fail error22:16
pabelangerclarkb: ianw thinks the rpm cache is corrupt22:16
smarcetfungi: i am using that user22:17
*** sridhar_ram has joined #openstack-infra22:17
ianwclarkb / pabelanger : yeah, i thought i fixed that with https://review.openstack.org/#/c/281596/22:17
clarkbhttp://paste.openstack.org/show/491370/ is the debian jessie fail22:17
fungismarcet: me too, which is how i got that error22:17
clarkbianw: ok, may need a new dib release then?22:17
*** claudiub has quit IRC22:18
clarkbnot sure if dib has released in the last 8 days22:18
*** achanda has quit IRC22:18
pabelangerdidn't notice debian-jessie looks like pgp server is down22:18
pabelangerwe should store the gpg keys in puppet22:18
smarcetmmmm22:18
ianwclarkb: no, that should be in a release 1.12.0, that's the problem :)  i am now think that it might be because we reuse the cache across runs22:18
smarcetis another user22:18
smarcethold on22:18
smarcetos_id_production22:19
smarcethow i send u password22:19
smarcet?22:19
fungismarcet: well, os_prod_site is the account configured to do the backups, so if it lacks permission to a table then we didn't back it up22:19
clarkbpabelanger: ya that might be a good alternative22:19
ianwclarkb: i'm pretty sure if i tell the dnf developers "well what we do is bind mount an old cache directory from a previous run" their enthusiasm for finding the bug will be ... minimal :)22:19
clarkbpabelanger: stick it in public hiera22:19
smarcetups22:19
*** heyongli has quit IRC22:19
smarceti think that i have a back up22:19
smarcethold on22:20
clarkbianw: eh the cache lives persistently on machines though22:20
*** heyongli has joined #openstack-infra22:20
clarkbianw: so I don't see why this is a problem22:20
*** bhunter_ has quit IRC22:20
fungismarcet: i guess this means the deployment is using a different account than os_prod_site to initialize the database?22:20
pabelangerclarkb: that or just use store as file.  Not sure how often they change.22:20
clarkbisn't the whole point of a cache to keep valid data from old runs?22:20
pabelangeractually need to see what we are using them for22:20
pabelangerOh22:20
pabelangerpuppetlabs key22:20
smarcetfungi: that is rite22:20
smarcetos_id_production22:20
clarkbpabelanger: yup22:20
smarcetfungi: ok i got a backup from march 1422:21
smarceti will restore empty tables by hand22:21
clarkbpabelanger: we also have one for the upstream ubuntu repo key22:21
pabelangerYa, I'd say we just cache it locally. if possible22:21
clarkbpabelanger: as part of reprepro setup22:21
pabelangerright22:21
fungismarcet: i see the account we're using as the openstackid_id_mysql_user key in hiera and it is a different account, i agree22:21
ianwclarkb: well yes, but on an actual system ... rather than say where we rebuild the system on-top and then use a cache from an old run.  versions might have changed, etc.  anyway, i need to dig further, obviously.22:21
clarkbpabelanger: though that one doesn't go in the slave image builds22:21
*** zeih has quit IRC22:22
fungismarcet: also we should adjust the backup to use an account which has access to those tables22:22
pabelangerclarkb: I can get a patch up for puppetlabs atleast22:22
pabelangergive me a min22:22
clarkbianw: the unfortunate part of that is fedora is the slave image that needs the cache the most since its mirrors are the least reliable22:22
ianwclarkb: yeah, pabelanger how is that rpm mirror going :)22:23
*** rkukura has quit IRC22:23
*** dimtruck is now known as zz_dimtruck22:23
clarkbianw: I mean in the general case though. If I was running fedora at home I would be unhappy if dnf wasn't committed to working caching22:23
*** ihrachys has quit IRC22:23
clarkb"everyone make a mirror" isn't exactly hte best of responses22:24
*** ybathia has joined #openstack-infra22:24
fungismarcet: oh! i just noticed we're doing two database backups22:24
*** bpokorny has quit IRC22:25
*** shardy has quit IRC22:25
*** amitgandhinz has quit IRC22:26
pabelangerianw: initial version: https://review.openstack.org/#/c/289520/22:26
pabelangerjust need to test it with the current mirror.pp code and add kstart5 support22:26
fungismarcet: i may have restored the backup to the wrong database. on openstackid_production i do have read access to that table22:26
*** agentle has quit IRC22:27
fungismarcet: i can try restoring today's backup again to the right database now22:27
*** mylu has quit IRC22:27
fungismarcet: i just want to make sure you aren't in the middle of modifying it first22:27
ianwclarkb: well i don't think i can really blame dnf for not considering the case of "you built an image from a chroot environment and then bind mounted in a cache you built yesterday on a different image and something goes wrong".  anyway, i haven't proven that yet.  it may well be something i've messed up with our swizzling of rpmdb's and such22:27
smarcetfungi22:28
smarcetsorry i was restoring manually22:28
fungismarcet: my first restore was trying to update the old copies of those tables on the other database from before they split22:28
clarkbianw: its a fairly common thing to share a package cace within a local env instead of runnin ga mirror22:28
clarkbianw: fwiw22:28
fungismarcet: which explains why it didn't have access to some tables22:28
smarcetfungi: could it try now?22:28
fungismarcet: it's restoring now22:29
smarcetok thx :)22:29
fungiand updating a lot more tables i think22:29
smarcetfungi: should be ok22:29
fungismarcet: should be done now. can you check again?22:29
fungisorry about that. i should have noticed/remembered we're backing up more than one remote database on that server22:29
smarcetyeah on it22:30
*** esker has quit IRC22:30
*** heyongli has quit IRC22:30
smarcetfungi: its ok  :)22:30
*** heyongli has joined #openstack-infra22:30
smarcetseems to be working22:30
smarcetthx a lot :)22:30
fungiokay, great!22:30
fungiapologies for the emergency. i should go do some non-computer-related evening chores rather than break anything else this evening22:31
fungii'll be around and keeping an eye on irc in case you spot anything else amiss22:31
*** bpokorny has joined #openstack-infra22:31
anteayafungi: happy evening breaking noncomputer things22:32
smarcetfungi: tnx22:32
fungithanks!22:32
anteayastay away from the stemware22:32
smarcetfunfgi: i will refactor https://review.openstack.org/#/c/221716/ tomorrow22:32
anteayasteins only22:32
smarcetthx again22:32
smarcetcu and have a good bite22:32
openstackgerritIan Wienand proposed openstack-infra/project-config: Install devstack "general" prereqs for unit tests  https://review.openstack.org/29555122:32
fungismarcet: sounds good, remind me tomorrow and i'll take a look at what you come up with so we can get it merged and then give this another try (i'll also make database backups rught before i start in case of any other mishaps)22:32
*** tonytan4ever has joined #openstack-infra22:35
*** smarcet has quit IRC22:37
*** bpokorny has quit IRC22:37
*** knikolla has joined #openstack-infra22:39
clarkbthings I have learned about neutron jobs. Devstack does appear to take longer to set up neutron than nova net. First theory (that I haven't checked up on) is that its the extra pip install time for neutron which nova net doesn't have when installing the rest of nova and the db migrations22:40
*** heyongli has quit IRC22:40
*** heyongli has joined #openstack-infra22:40
openstackgerritSteven Dake proposed openstack-infra/project-config: Add mitaka to gerritbot for Kolla  https://review.openstack.org/29555622:41
sdake_I hate to ask, but can I get a semi-emergency merge on that above patch22:41
mordredfungi: heya!22:41
mordredfungi: so, I'm both confused and concerned about your problems wiht launch-node22:41
*** _ari_ has joined #openstack-infra22:42
sdake_we just branched and without irc people dunno how to use gerrit review queue i think ;-)22:42
mordredfungi: there is a _deeper_ design thing I think we should talk about in austin... but for hte short term22:42
*** esker has joined #openstack-infra22:42
jeblairsdake_: :) lgtm22:42
mordredfungi: if the puppet/ansible interaction didn't work with servers with uuids, it would _never_ work with servers who are duplicate with uuids22:42
mordredfungi: (also, yolanda and I used that script to spin up a server last week - so I really do want to diagnose the problems you saw, because I would not expect them to exist)22:43
clarkbmordred: yes I think we found that it never worked with uuids due to the fix I made that PR against ansible for22:43
sdake_jeblair thanks ;)22:44
sdake_simple one liner22:44
sdake_thanks clarkb22:44
mordredclarkb: AH. right. weird. then I wonder how we got the server booted last week22:44
fungimordred: right, basically tip of system-config master i did a normal launch-node.py run per the readme in that directory (after doing a chmod +x anyway because when we replaced teh file we lost the executable permissions on it)22:44
mordredfungi: k. so I can do with a duplicate name for something and I should be able to trivially reproduce yah?22:45
fungimordred: i booted a replacement for the openstackid.org server with a larger flavor, end result was that i got a server with no openstackid.org specific puppeting, and it though the hostname of the machine (when i ssh'd into it) was the nova uuid22:46
fungimordred: so, yes, i expect this to happen with any duplicate we try to launch22:46
fungi(though _please_ don't try to launch another openstackid.org since we subsequently learned its deployment code is not idempotent and it blew away the production trove content)22:47
mordredfungi: yes. I saw that22:48
jeblairfungi: is that a current problem (do we need to restore from backup?)22:48
mordredfungi: I will boot a duplicate review-dev.openstack.org22:48
jeblair(i did not see that)22:48
fungijeblair: i restored it from backup already22:48
fungiafter discussing with smarcet22:49
jeblairfungi: ok.  anything i can help with?22:49
fungijeblair: nope, it's all buttoned up aside from not launching another replacement openstackid.org until 221716 gets finished and merged22:49
jeblaircool, sorry i missed that22:49
fungino problem, i happened to be at the wheel when smarcet popped in22:49
fungiit was pretty much _right_ after i finished trying to launch the replacement22:50
fungiwhich took a bit of time due to having to unroll the weirdness wit the current ansible launch node state22:50
*** heyongli has quit IRC22:50
*** heyongli has joined #openstack-infra22:50
jeblair716 doesn't seem like an ideal solution in that it's manual, but i do agree it will prevent future such problems.22:51
*** tonytan4ever has quit IRC22:51
fungii feel the same way22:51
fungiwould be better if the depoyment code detected whether there was already content in the db22:52
openstackgerritDeklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job  https://review.openstack.org/29143022:53
openstackgerritDeklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job  https://review.openstack.org/29143022:54
*** denisra has quit IRC22:56
*** denisra has joined #openstack-infra22:56
openstackgerritMerged openstack-infra/project-config: Add mitaka to gerritbot for Kolla  https://review.openstack.org/29555622:59
*** edmondsw has quit IRC22:59
craigeReaching out to infra puppet heqads for guidance on wher eI might be going wrong with MySQL on this change: https://review.openstack.org/#/c/289648/22:59
openstackgerritPaul Belanger proposed openstack-infra/system-config: Store puppetlabs gpg key local  https://review.openstack.org/29556023:00
pabelangerclarkb: if my puppet foo is still good ^23:00
*** heyongli has quit IRC23:00
clarkbpabelanger: I would drop the server config to force it to not talk out23:01
*** heyongli has joined #openstack-infra23:01
pabelangerclarkb: right23:01
pabelangerguess it is no longer needed23:01
openstackgerritPaul Belanger proposed openstack-infra/system-config: Store puppetlabs gpg key local  https://review.openstack.org/29556023:01
*** IlyaG has joined #openstack-infra23:01
openstackgerritDeklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job  https://review.openstack.org/29143023:02
*** openstackgerrit has quit IRC23:03
*** openstackgerrit_ is now known as openstackgerrit23:03
*** nadya has joined #openstack-infra23:03
*** openstackgerrit has quit IRC23:03
*** openstackgerrit_ has joined #openstack-infra23:03
*** boris-42 has quit IRC23:04
clarkbcraige: http://logs.openstack.org/48/289648/5/check/gate-infra-puppet-apply-ubuntu-trusty/f218073/console.html#_2016-03-21_22_29_17_125 says mysql_password is an invalid parameter to openstack_project::phabricator and yet its there23:04
*** openstackgerrit_ is now known as openstackgerrit23:04
clarkbcraige: I am not sure what is going on there yet23:04
*** _amrith_ is now known as amrith23:05
craigeOk, so that sounds like I've not written my manifests correctly.23:05
craigeTHanks clarkb :-)23:05
pabelangercraige: clarkb: mysql_user_password23:06
*** rfolco has joined #openstack-infra23:06
craigeAll the other errors left me wondering where the true problem lay.23:06
pabelangerhttps://github.com/openstack-infra/puppet-phabricator/blob/master/manifests/init.pp#L1123:06
craigeTHanks pabelanger. I'll try that instead.23:06
fungimost often it's just a matter of forgetting to plumb a variable all the way through all layers of classes you're passing from/to23:06
clarkbpabelanger: this is for openstack_project::phabricator though not phabricator23:07
clarkbpabelanger: but maybe its a scopring issue?23:07
fungibut yeah, that just looks like a consistency error23:07
*** erlon has quit IRC23:07
pabelangerclarkb: Ya, I think it is scoping23:07
pabelangermight be worth renaming our system-config phabricator to something else23:07
*** nadya has quit IRC23:08
*** openstackgerrit_ has joined #openstack-infra23:08
pabelangerclarkb: Oh no I see the issue23:09
fungii'm generally a fan of naming the server for the service it provides rather than the software it runs23:09
openstackgerritDeklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job  https://review.openstack.org/29143023:09
pabelangerlike 38 is wrong23:09
pabelangerline*23:09
*** dims_ has quit IRC23:09
*** dims has joined #openstack-infra23:09
pabelangerleft comment23:09
craige38 in which file pabelanger?23:09
anteayafungi: have we decided what this service is called?23:09
craigeThank pabelanger23:09
fungie.g. we have review.openstack.org not gerrit.openstack.org23:09
* craige reads.23:09
jeblairanteaya: "bugs"23:10
anteayabugs.openstack.org23:10
craigeI concur fungi, so which name would you suggest?23:10
anteayait does have a nice ring to it23:10
fungianteaya: very good question, and no i'm not sure what a great one-word descriptor is for what we plan to use it for23:10
anteayaokay23:10
openstackgerritRyan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again  https://review.openstack.org/28891523:10
fungibut "bugs" wfm23:10
anteayalooks like bugs has been offered for consideration23:10
fungii'm sure it will be full of them (in more ways than one)23:10
anteayanice and short23:10
anteayato the point23:10
anteayaha ha ha23:10
craigeOkay, so we call it "bugs" but UI team want to use it for Pholio, so perhaps mock-bugs ?23:10
craige;-)23:10
*** heyongli has quit IRC23:11
*** heyongli has joined #openstack-infra23:11
jeblairit's probably worth a meeting/ml bikeshed23:11
*** mylu has joined #openstack-infra23:11
fungiwell, it'll be used for task tracking in general, and incident tracking probably to some extent, and filing wishlist items... no one word will capture all that but bugs seems close enough to a major use case anyway23:11
* craige nominates "cruft:23:11
openstackgerritDeklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job  https://review.openstack.org/29143023:11
jeblairwe may want to avoid actually using that hostname until we are about ready for it to be in production for openstack itself23:12
fungiabandon-hope-all-ye-who-file-here.openstack.org23:12
clarkbon osic neutron-full takes 13 minutes 10 seconds to setup, regular full is 11 minutes 40 seconds. Thats everything until devstack runs23:12
clarkber sorry until tempest runs23:12
anteayaI've offered my tomorrows bikeshed topic for the meeting already23:12
anteayaso my quota is full at the moment23:12
*** sdague has quit IRC23:12
jeblairbecause having 'bugs.openstack.org' but not having openstack track its bugs there for any significant length of time would be weird.23:12
clarkband the serialized tempest tests are taking quite a bit longer with neutron (still not done yet) so I think that is likely where the greatest benefit can be achived with optimizing23:13
anteayafungi: very aquatic23:13
jeblairclarkb: what are you working on?23:13
fungijeblair: agreed, might be best to plan for a rename with legacy forwarding redirect right before production23:13
clarkbjeblair: well there was a thread to the ml that said "most of the neutron test time is infrastructure stuff" so trying to get timing data for neutron to show that no it isn't23:13
*** ddieterly has joined #openstack-infra23:14
anteayaclarkb: is that rossella_s's thread?23:14
clarkbjeblair: most of the time in our job is spent in devstack and tempest and tempest is a good 50% when run concurrently23:14
clarkbanteaya: yes23:14
fungi"it's your tests, optimize there"23:14
anteayacool23:14
clarkbfomr there I pushed a change to run tempest serially in order to get good per tempest test data for neutron to compare with23:14
fungithough it does appear devstack could benefit from their optimization efforts if they're keen on expending effort to speed stuff up23:15
clarkbonce this job finishes up I can respond again with the tempest serial infos23:16
openstackgerritCraige McWhirter proposed openstack-infra/system-config: Initial commit of the Phabricator puppetry.  https://review.openstack.org/28964823:16
clarkbfungi: yup, tough I think much of the cost there is overhead for additional service setup. Its more stuff to pip install and more db migrations23:17
clarkbnot sure how you avoid that overhead (but there are probably things that can be done like consolidating migrations)23:17
*** ddieterly is now known as ddieterly[away]23:17
*** ddieterly[away] has quit IRC23:17
clarkbI wonder if pip install $biglistofthingshere would be appreciably faster than pip install foo && pip install bar etc23:19
clarkbdstufft: ^ do you have any idea off the top of your hread without me trying to make devstack do that?23:19
*** rbrndt has quit IRC23:19
*** achanda has joined #openstack-infra23:20
dstufftclarkb: um, probably depends on how much overlap there is in dependencies23:20
clarkbdstufft: there is a fair bit because openstack reuses a lot of the same libs in all the places23:20
fungiclarkb: was more thinking they (and others) could benefit from optimizing devstack setup time in general, rather than neutron's case specifically23:21
*** heyongli has quit IRC23:21
clarkbfungi: ya the pip intsall consolidation might be one way to do that (cut out all of that extra python startup time and recalculating of deps over and over)23:21
dstufftclarkb: I'm not actually sure tbh, I don't know that I ever thought about it before23:21
*** heyongli has joined #openstack-infra23:21
openstackgerritPaul Belanger proposed openstack-infra/system-config: Store puppetlabs gpg key local  https://review.openstack.org/29556023:21
*** dims has quit IRC23:21
*** agentle has joined #openstack-infra23:22
*** dims has joined #openstack-infra23:22
fungiclarkb: sure, i do a similar collection phase in the package caching in our dib elements, to avoid calling bindep and the package manager multiple times23:23
fungibasically build up a deduplicated list of things to install, then pass those to the installer in one invocation23:23
*** Nepoc has quit IRC23:23
*** Nepoc has joined #openstack-infra23:24
*** sdake has joined #openstack-infra23:25
anteayathis email contains an assertion: http://lists.openstack.org/pipermail/openstack-dev/2016-March/090028.html "Switching to multinode jobs for the23:26
anteayagate means that all projects in the integrated gate will have to use the23:26
anteayamiltinode configuration.23:26
anteayaI question this assertion23:26
clarkbanteaya: the way the gate works is by running a common set of tests across all the projects23:26
clarkbanteaya: so if you update that common set of tests all projects are affected23:26
*** sdake_ has quit IRC23:26
anteayathe integrated gate jobs23:26
clarkbwhere all projects means everything in that gate pipeline23:26
*** amitgandhinz has joined #openstack-infra23:26
anteayaright23:27
*** achanda has quit IRC23:27
openstackgerritStephanie Miller proposed openstack/diskimage-builder: Use correct package for add-apt-repository  https://review.openstack.org/29556723:27
anteayabut projects have other jobs in their gate23:27
anteayain addition to the integrated gate jobs23:27
*** dims has quit IRC23:27
clarkbyes the project specific jobs can do whatever they want, its the shared common jobs that this affects23:28
*** dims has joined #openstack-infra23:28
anteayaright23:28
anteayaI think sc68cal's point was about the project specific jobs23:28
*** tphummel has quit IRC23:28
anteayaas nova is doing23:28
clarkbI didn't read it that way23:29
clarkb(but I am also in the just use multinode camp)23:29
*** dingyichen has joined #openstack-infra23:29
clarkbI did a spot check on multinode test sand those do take much longer than single node for reasons I don't understand23:29
anteayaoh perhaps I am not reading it the way it was intended23:29
clarkbwhen we started they were about the same23:30
anteaya:(23:30
clarkbmultinode tempests faster due to extra resources but setup is longer and it balanced out23:30
clarkbI am guessing that it no longer tempests faster for some reason23:30
anteayahmmmmm23:30
*** heyongli has quit IRC23:31
*** amrith is now known as _amrith_23:31
*** heyongli has joined #openstack-infra23:32
*** dims has quit IRC23:33
*** gordc has quit IRC23:33
*** Qiming has joined #openstack-infra23:33
*** smarcet has joined #openstack-infra23:34
*** tphummel has joined #openstack-infra23:35
*** fawadkhaliq has quit IRC23:35
*** agentle has quit IRC23:35
*** _ari_ has quit IRC23:36
*** Adri2000 has quit IRC23:37
openstackgerritIan Wienand proposed openstack/diskimage-builder: Turn down tracing for source-repo cache  https://review.openstack.org/29556923:37
*** amitgandhinz has quit IRC23:39
*** bpokorny has joined #openstack-infra23:41
*** heyongli has quit IRC23:41
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Added DB seeding support  https://review.openstack.org/29557323:42
*** heyongli has joined #openstack-infra23:42
pabelangergoing to call it for tonight23:42
pabelanger295560 should fix our timeout issue with gpg servers for dib builds23:42
pabelangerwill look into debian-jessie failures in the morning23:42
openstackgerritIan Wienand proposed openstack-infra/glean: Handle invalid symlinks in written files  https://review.openstack.org/25983123:43
*** jpr has joined #openstack-infra23:43
openstackgerritRyan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again  https://review.openstack.org/28891523:44
*** fawadkhaliq has joined #openstack-infra23:45
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: update mbcrypt support  https://review.openstack.org/29557623:46
*** rkukura has joined #openstack-infra23:46
*** eil397 has quit IRC23:46
*** arif-ali has quit IRC23:47
*** Adri2000 has joined #openstack-infra23:48
*** achanda has joined #openstack-infra23:49
*** sridhar_ram has quit IRC23:49
*** fawadkhaliq has quit IRC23:50
*** fawadkhaliq has joined #openstack-infra23:50
*** jpr1 has joined #openstack-infra23:52
*** heyongli has quit IRC23:52
*** heyongli has joined #openstack-infra23:52
*** sridhar_ram has joined #openstack-infra23:53
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Added apache2 service notification on ssl cert change  https://review.openstack.org/29557723:54
*** jpr has quit IRC23:54
openstackgerritCraige McWhirter proposed openstack-infra/system-config: Initial commit of the Phabricator puppetry.  https://review.openstack.org/28964823:57
*** smarcet has quit IRC23:59
*** ybathia has quit IRC23:59
*** ajmiller has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!