Friday, 2016-09-16

*** Apoorva has joined #openstack-infra00:01
*** yamamoto_ has quit IRC00:03
clarkbsdake_: I think that usually means there are duplicate accounts and berendt will need to get them deduped00:05
*** itsuugo has quit IRC00:06
*** amotoki has quit IRC00:06
sdake_clarkb how does he do that?00:06
sdake_ask in channel, amil openstack-infra?00:06
clarkbhe will need to figure out which of the three is the one to keep and then we will have to modify the others for him00:06
clarkbits usually easiest here so we can have a conversation about what is ehat00:07
*** itsuugo has joined #openstack-infra00:07
sdake_clarkb ok i'll sync up with him and let him know to come here - he is in europe so wont be around during us hours00:07
*** amotoki has joined #openstack-infra00:07
*** HeOS has quit IRC00:09
*** SumitNaiksatam has quit IRC00:09
fungicatching up, i don't want to be left in the lurch after all ;)00:10
*** gildub has joined #openstack-infra00:14
*** armax has quit IRC00:14
*** tosky has quit IRC00:16
*** kjorgensen has quit IRC00:17
*** ddieterly has joined #openstack-infra00:18
*** kjorgensen has joined #openstack-infra00:19
*** armax has joined #openstack-infra00:20
fungijeblair: clarkb: mordred: 371176 (region name) is failing lint checking, so 371193 (plumbing) and 371196 (environments) depending on it is also failing. i'm looking now to see why that is00:20
fungioh, of course. it's all-important cosmetic whitespace00:21
fungii'll "fix" it00:21
*** kjorgensen has quit IRC00:21
*** kjorgensen has joined #openstack-infra00:22
*** tqtran has quit IRC00:22
openstackgerritMerged openstack-infra/system-config: launch: stream syslog on remote host  https://review.openstack.org/37118200:23
fungiaha, one actual error: undefined name '00:23
fungiself'00:23
*** asettle has joined #openstack-infra00:23
fungi(extra carriage return mine)00:23
*** ddieterly has quit IRC00:24
fungier, wait... 371193 is a doc-only patch00:25
fungiit's only changing the readme, but our linter is suddenly picking up on launch-node.py issues?00:25
fungiand there's no depends/parent patch00:26
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120900:26
*** kjorgensen has quit IRC00:26
EmilienMfungi: could you please approve this revert ? https://review.openstack.org/#/c/371133/00:27
*** asettle has quit IRC00:28
fungihuh, so somehow 371158 (the popen change) passed the linters job and merged, but the changes after it are failing lint checks on lines it added?00:29
*** kaisers1 has joined #openstack-infra00:30
*** kaisers2 has joined #openstack-infra00:30
*** piet has quit IRC00:30
*** ijw has joined #openstack-infra00:31
EmilienMfungi: it happens if a library was updated, let me look00:31
EmilienMfungi: you mean puppet lint?00:31
fungiEmilienM: nope, our combined linters job that runs bash8 and flake8 and some other stuff on the system-config repo00:32
*** kaisers has quit IRC00:32
EmilienMah ok00:32
*** kaisers3 has quit IRC00:32
fungi"INFO:zuul.Cloner:Falling back to branch master"00:33
fungii think it tested the master branch tip and not our change00:33
fungii shouldn't start looking in rabbit holes like this one at this time of night00:33
*** ijw has quit IRC00:34
*** markvoelker has joined #openstack-infra00:34
*** ijw has joined #openstack-infra00:34
fungier, no, that was for the requirements repo checkout00:34
fungiINFO:zuul.Cloner:Prepared openstack-infra/system-co00:34
funginfig repo with commit 035610aa93cee2c342ad6a88fa7999736503855900:35
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: get_host_info: get repos list  https://review.openstack.org/36930500:35
openstackgerritMerged openstack-infra/project-config: Revert "tripleo: disable voting on gate-tripleo-ci-centos-7-nonha-multinode"  https://review.openstack.org/37113300:37
funginevermind, that checks out as 371158 on zm0500:37
fungiso it seems to have tested that change but flake8 didn't complain about those lines in launch-node.py until it tested the next change00:38
fungiokay, now i get it. 371158 got a new patchset to fix these errors, but childred/dependencies were not rebased or rechecked. fixing now00:41
*** itsuugo has quit IRC00:45
*** itsuugo has joined #openstack-infra00:46
openstackgerritShu Muto proposed openstack-infra/project-config: Change JS test jobs to 'voting' for Magnum-UI  https://review.openstack.org/37059400:47
*** tphummel has quit IRC00:49
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Plumb puppet_environment through to the adhoc playbook  https://review.openstack.org/37119300:49
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Use OS_REGION_NAME so that OCC ignore it properly  https://review.openstack.org/37117600:49
fungiwouldn't merge without a rebase on master00:49
*** piet has joined #openstack-infra00:52
*** tuanla_ has joined #openstack-infra00:53
mordredahhhh00:54
mordredalso - /me waves to everyone00:54
mordredno airplane for mordred tonight00:54
mordredthere will now be airplaning tomorrow00:54
*** jcoufal has joined #openstack-infra00:55
*** gyee has quit IRC00:56
*** itsuugo has quit IRC00:56
fungimordred: flight cancelled?00:56
fungithat's unfortunate00:56
*** aeng has quit IRC00:56
*** asettle has joined #openstack-infra00:57
*** itsuugo has joined #openstack-infra00:58
*** asettle has quit IRC01:01
openstackgerritIan Wienand proposed openstack-infra/project-config: Add pushMerge for diskimage-builder  https://review.openstack.org/37123001:03
*** kzaitsev_mb has joined #openstack-infra01:03
*** itsuugo has quit IRC01:06
openstackgerritMerged openstack-infra/system-config: Use OS_REGION_NAME so that OCC ignore it properly  https://review.openstack.org/37117601:07
*** itsuugo has joined #openstack-infra01:08
*** roxanaghe has joined #openstack-infra01:09
*** SumitNaiksatam has joined #openstack-infra01:10
*** SumitNaiksatam has left #openstack-infra01:11
*** gongysh has joined #openstack-infra01:11
openstackgerritMerged openstack-infra/system-config: Plumb puppet_environment through to the adhoc playbook  https://review.openstack.org/37119301:13
*** aeng has joined #openstack-infra01:13
*** itsuugo has quit IRC01:15
*** itsuugo has joined #openstack-infra01:17
openstackgerritMerged openstack-infra/system-config: launch: add puppet environment option  https://review.openstack.org/37119601:21
*** kzaitsev_mb has quit IRC01:21
*** itsuugo has quit IRC01:22
*** piet has quit IRC01:22
*** itsuugo has joined #openstack-infra01:22
*** xarses has joined #openstack-infra01:24
*** xarses has quit IRC01:24
*** xarses has joined #openstack-infra01:24
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update checkstyle plugin  https://review.openstack.org/37059201:26
*** baoli has joined #openstack-infra01:27
*** itsuugo has quit IRC01:30
*** itsuugo has joined #openstack-infra01:31
*** mtanin___ has joined #openstack-infra01:31
*** mtanino has quit IRC01:32
*** xarses has quit IRC01:33
openstackgerritMerged openstack-infra/irc-meetings: Change WOS-mentoring meetings to weekly  https://review.openstack.org/37089901:34
*** xarses has joined #openstack-infra01:34
*** xarses has quit IRC01:35
*** xarses has joined #openstack-infra01:35
*** Apoorva_ has joined #openstack-infra01:35
*** xarses has quit IRC01:38
*** Apoorva has quit IRC01:39
*** xarses has joined #openstack-infra01:39
*** xarses has quit IRC01:40
*** itsuugo has quit IRC01:40
*** Apoorva_ has quit IRC01:40
*** xarses has joined #openstack-infra01:41
*** salv-orl_ has joined #openstack-infra01:41
*** itsuugo has joined #openstack-infra01:41
*** salv-orlando has quit IRC01:43
*** tsg has joined #openstack-infra01:45
*** aeng has quit IRC01:46
*** tkelsey has joined #openstack-infra01:47
*** cody-somerville has joined #openstack-infra01:49
*** cody-somerville has joined #openstack-infra01:49
*** fernnest has quit IRC01:50
*** jkilpatr has quit IRC01:50
*** xarses has quit IRC01:50
*** xarses has joined #openstack-infra01:51
*** tkelsey has quit IRC01:51
*** itsuugo has quit IRC01:51
*** csomerville has quit IRC01:52
*** itsuugo has joined #openstack-infra01:53
*** annegentle has quit IRC01:53
*** dkehn has quit IRC01:55
*** baoli has quit IRC01:56
openstackgerritMerged openstack-infra/project-config: Fix missing '$' sign in ironic-inspector-enabled builder  https://review.openstack.org/37070501:56
*** aeng has joined #openstack-infra01:58
openstackgerritMerged openstack-infra/project-config: Fix the experimental VPNaaS rally job  https://review.openstack.org/37083801:59
*** armax has quit IRC01:59
*** zhurong has joined #openstack-infra02:01
*** itsuugo has quit IRC02:03
*** itsuugo has joined #openstack-infra02:03
*** caowei has joined #openstack-infra02:04
*** mtanin___ has quit IRC02:08
*** dkehn has joined #openstack-infra02:08
*** tsg has quit IRC02:10
*** itsuugo has quit IRC02:11
*** ijw has quit IRC02:11
openstackgerritMerged openstack-infra/project-config: Revert "Pin bundler to 1.12.5 for some jobs"  https://review.openstack.org/37106702:11
openstackgerritMerged openstack-infra/project-config: do not prompt before submitting multiple reviews  https://review.openstack.org/37076902:11
*** itsuugo has joined #openstack-infra02:12
*** Jeffrey4l has quit IRC02:16
*** itsuugo has quit IRC02:17
*** xarses_ has joined #openstack-infra02:17
*** itsuugo has joined #openstack-infra02:18
*** xarses has quit IRC02:20
*** fernnest has joined #openstack-infra02:22
*** itsuugo has quit IRC02:23
*** itsuugo has joined #openstack-infra02:23
*** fernnest has quit IRC02:25
*** fguillot has quit IRC02:29
*** mdrabe has joined #openstack-infra02:36
*** gouthamr has quit IRC02:38
*** zhurong has quit IRC02:45
*** yuanying has quit IRC02:47
*** zhurong has joined #openstack-infra02:49
*** itsuugo has quit IRC02:51
*** itsuugo has joined #openstack-infra02:53
*** AnarchyAo has joined #openstack-infra02:54
*** asettle has joined #openstack-infra02:59
*** itsuugo has quit IRC03:01
*** sdake_ has quit IRC03:01
*** itsuugo has joined #openstack-infra03:02
*** vinaypotluri has quit IRC03:02
*** asettle has quit IRC03:03
*** armax has joined #openstack-infra03:05
*** phschwartz has quit IRC03:05
*** hongbin has joined #openstack-infra03:06
*** itsuugo has quit IRC03:09
*** itsuugo has joined #openstack-infra03:11
*** zhurong has quit IRC03:13
*** itsuugo has quit IRC03:15
*** sarob has joined #openstack-infra03:16
*** itsuugo has joined #openstack-infra03:16
*** sarob has quit IRC03:16
*** sarob has joined #openstack-infra03:17
*** EricGonczer_ has quit IRC03:18
*** vinaypotluri has joined #openstack-infra03:18
*** tuanla_ has quit IRC03:19
*** itsuugo has quit IRC03:23
*** itsuugo has joined #openstack-infra03:25
*** itsuugo has quit IRC03:30
*** tsg has joined #openstack-infra03:30
*** itsuugo has joined #openstack-infra03:31
*** aeng_ has joined #openstack-infra03:32
*** phschwartz has joined #openstack-infra03:32
*** roxanaghe has quit IRC03:33
*** tuanla_ has joined #openstack-infra03:33
*** hongbin has quit IRC03:35
*** gmann has quit IRC03:35
*** sarob has quit IRC03:36
*** gmann has joined #openstack-infra03:37
*** Shrews has quit IRC03:37
*** mordred has quit IRC03:38
*** roxanaghe has joined #openstack-infra03:39
*** phschwartz has quit IRC03:40
*** Shrews has joined #openstack-infra03:42
*** mordred has joined #openstack-infra03:43
*** vikrant has joined #openstack-infra03:44
*** mdrabe has quit IRC03:46
*** itsuugo has quit IRC03:48
*** itsuugo has joined #openstack-infra03:50
*** tsg_ has joined #openstack-infra03:53
*** tsg has quit IRC03:53
*** itsuugo has quit IRC03:54
*** mriedem has quit IRC03:55
*** itsuugo has joined #openstack-infra03:56
*** yamamot__ has joined #openstack-infra03:58
*** dimtruck is now known as zz_dimtruck04:00
*** itsuugo has quit IRC04:00
*** kjorgensen has joined #openstack-infra04:00
*** itsuugo has joined #openstack-infra04:01
*** senk has joined #openstack-infra04:02
*** uberjay has quit IRC04:02
*** kjorgensen has quit IRC04:05
*** phschwartz has joined #openstack-infra04:06
*** itsuugo has quit IRC04:08
*** itsuugo has joined #openstack-infra04:11
*** sree has joined #openstack-infra04:12
*** yuanying has joined #openstack-infra04:13
openstackgerritCaleb Boylan proposed openstack-infra/nodepool: [WIP] Fix subnode deletion  https://review.openstack.org/37045504:13
*** vikrant has quit IRC04:19
*** itsuugo has quit IRC04:20
*** vikrant has joined #openstack-infra04:21
*** itsuugo has joined #openstack-infra04:21
*** Sukhdev has joined #openstack-infra04:21
*** vicky has joined #openstack-infra04:21
*** uberjay has joined #openstack-infra04:21
*** vicky has quit IRC04:23
*** vicky has joined #openstack-infra04:23
*** markvoelker has quit IRC04:28
*** itsuugo has quit IRC04:28
*** itsuugo has joined #openstack-infra04:29
*** caowei has quit IRC04:30
*** psachin has joined #openstack-infra04:36
*** vicky has quit IRC04:36
*** vicky has joined #openstack-infra04:37
*** vicky has quit IRC04:37
*** rcernin has quit IRC04:37
*** itsuugo has quit IRC04:37
*** hamzy has joined #openstack-infra04:38
*** itsuugo has joined #openstack-infra04:39
*** itsuugo has quit IRC04:43
*** itsuugo has joined #openstack-infra04:45
*** harlowja has quit IRC04:45
*** tkelsey has joined #openstack-infra04:49
*** itsuugo has quit IRC04:50
*** itsuugo has joined #openstack-infra04:51
*** gongysh has quit IRC04:53
*** tkelsey has quit IRC04:53
*** itsuugo has quit IRC04:56
*** itsuugo has joined #openstack-infra04:57
*** tsg_ has quit IRC04:57
*** jaosorior has joined #openstack-infra04:57
*** asettle has joined #openstack-infra05:01
*** gongysh has joined #openstack-infra05:02
*** yamamot__ has quit IRC05:03
*** itsuugo has quit IRC05:04
*** itsuugo has joined #openstack-infra05:05
*** asettle has quit IRC05:05
*** itsuugo has quit IRC05:12
*** tqtran has joined #openstack-infra05:13
*** itsuugo has joined #openstack-infra05:14
*** senk has quit IRC05:15
*** itsuugo has quit IRC05:18
openstackgerritMerged openstack-infra/project-config: skip constraints update if there is no tox.ini file  https://review.openstack.org/37121105:19
openstackgerritMerged openstack-infra/project-config: Change JS test jobs to 'voting' for Magnum-UI  https://review.openstack.org/37059405:19
*** woodster_ has quit IRC05:20
*** itsuugo has joined #openstack-infra05:20
*** markvoelker has joined #openstack-infra05:28
*** sree has quit IRC05:31
*** sree has joined #openstack-infra05:33
*** markvoelker has quit IRC05:33
openstackgerritRyo Tagami proposed openstack-infra/jenkins-job-builder: Add Workflow Project Documentation  https://review.openstack.org/37127205:34
*** itsuugo has quit IRC05:36
*** itsuugo has joined #openstack-infra05:36
*** rlandy|bbl is now known as rlandy05:39
*** pgadiya has joined #openstack-infra05:40
*** adriant has quit IRC05:40
*** claudiub has joined #openstack-infra05:40
*** itsuugo has quit IRC05:41
*** docaedo4 has quit IRC05:41
*** claudiub|2 has joined #openstack-infra05:42
*** rcernin has joined #openstack-infra05:43
*** itsuugo has joined #openstack-infra05:43
*** claudiub has quit IRC05:45
*** yamamot__ has joined #openstack-infra05:46
*** senk has joined #openstack-infra05:46
*** itsuugo has quit IRC05:48
*** itsuugo has joined #openstack-infra05:49
*** docaedo4 has joined #openstack-infra05:56
*** Genek has joined #openstack-infra06:01
*** gongysh has quit IRC06:02
*** roxanaghe has quit IRC06:02
*** hashar has joined #openstack-infra06:04
*** hashar has quit IRC06:04
*** hashar_ has joined #openstack-infra06:04
*** rajinir has quit IRC06:05
*** hashar_ is now known as pasladutout06:06
*** itsuugo has quit IRC06:07
*** aeng_ has quit IRC06:07
*** itsuugo has joined #openstack-infra06:08
*** andreas_s has joined #openstack-infra06:10
*** Douhet has quit IRC06:14
*** rcernin has quit IRC06:14
*** itsuugo has quit IRC06:15
*** itsuugo has joined #openstack-infra06:16
*** rcernin has joined #openstack-infra06:19
*** flepied has quit IRC06:19
*** YorikSar has quit IRC06:20
*** YorikSar has joined #openstack-infra06:22
openstackgerritRyo Tagami proposed openstack-infra/jenkins-job-builder: Add Workflow Project Documentation  https://review.openstack.org/37127206:23
*** sree has quit IRC06:23
*** sree has joined #openstack-infra06:23
*** pcaruana has joined #openstack-infra06:23
*** nherciu has joined #openstack-infra06:25
*** sree has quit IRC06:28
*** sree has joined #openstack-infra06:28
*** Sukhdev has quit IRC06:28
*** itsuugo has quit IRC06:33
*** thorst_ has joined #openstack-infra06:34
*** Ravikiran_K has joined #openstack-infra06:34
*** itsuugo has joined #openstack-infra06:34
*** ryanpetrello has quit IRC06:39
*** dhellmann has quit IRC06:39
*** vern has quit IRC06:40
*** dhellmann has joined #openstack-infra06:42
*** vinaypotluri has quit IRC06:42
*** Douhet has joined #openstack-infra06:42
*** matbu|brb is now known as matbu06:43
*** vern has joined #openstack-infra06:44
*** itsuugo has quit IRC06:44
*** gongysh has joined #openstack-infra06:45
*** itsuugo has joined #openstack-infra06:45
yolandagood morning06:47
*** salv-orl_ has quit IRC06:48
*** salv-orlando has joined #openstack-infra06:48
*** tkelsey has joined #openstack-infra06:51
*** jtomasek_ has joined #openstack-infra06:52
*** priteau has joined #openstack-infra06:53
*** priteau has quit IRC06:54
*** itsuugo has quit IRC06:54
*** tkelsey has quit IRC06:55
*** itsuugo has joined #openstack-infra06:56
*** florianf has joined #openstack-infra06:57
*** denisra_ has joined #openstack-infra06:58
AJaegergood morning, yolanda !06:59
*** david-lyle_ has joined #openstack-infra06:59
openstackgerritLukas Bednar proposed openstack-infra/jenkins-job-builder: properties: cover lockable-resources plugin  https://review.openstack.org/37129207:00
openstackgerritLukas Bednar proposed openstack-infra/jenkins-job-builder: properties: cover lockable-resources plugin  https://review.openstack.org/37129207:00
*** david-lyle has quit IRC07:00
*** denisra has quit IRC07:01
AJaegeryolanda: could you put https://review.openstack.org/370641 https://review.openstack.org/369660 https://review.openstack.org/371060 and https://review.openstack.org/370318 on your review queue for today, please?07:01
yolandasure, i'll do07:01
*** itsuugo has quit IRC07:01
AJaegerthanks07:02
*** itsuugo has joined #openstack-infra07:02
*** amotoki has quit IRC07:02
*** roxanaghe has joined #openstack-infra07:02
*** flepied has joined #openstack-infra07:03
*** pahuang has quit IRC07:04
*** flepied has quit IRC07:04
*** dtantsur|afk is now known as dtantsur07:05
*** flepied has joined #openstack-infra07:06
*** roxanaghe has quit IRC07:07
*** dtardivel has joined #openstack-infra07:09
odyssey4meyolanda a review of https://review.openstack.org/370646 would be appreciated when you have a moment07:10
*** matrohon has joined #openstack-infra07:10
yolandasure, going to start with the review queue07:11
*** Jokke_ has quit IRC07:13
*** itsuugo has quit IRC07:15
*** aviau has quit IRC07:16
fricklerhmm, I haven't seen this error in gate-setup yet, is this a known thing? http://logs.openstack.org/75/369675/7/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/9c29d66/logs/devstack-gate-setup-workspace-new.txt.gz07:17
*** itsuugo has joined #openstack-infra07:17
*** amoralej|off is now known as amoralej07:19
*** cschwede has quit IRC07:19
*** psanchez has quit IRC07:20
*** stevebaker has quit IRC07:20
*** slagle has quit IRC07:21
*** psanchez has joined #openstack-infra07:21
*** esikachev has joined #openstack-infra07:21
*** slagle has joined #openstack-infra07:22
*** camunoz has quit IRC07:22
*** Na3iL has joined #openstack-infra07:22
*** aviau has joined #openstack-infra07:22
*** itsuugo has quit IRC07:22
*** stevebaker has joined #openstack-infra07:23
openstackgerritMerged openstack-infra/project-config: Include a marker file at the root of all docs builds  https://review.openstack.org/36966007:23
*** itsuugo has joined #openstack-infra07:23
*** Hal1 has joined #openstack-infra07:25
*** cschwede has joined #openstack-infra07:25
*** david-lyle has joined #openstack-infra07:28
*** ryanpetrello has joined #openstack-infra07:28
*** AnarchyAo has quit IRC07:28
*** david-lyle_ has quit IRC07:29
*** markvoelker has joined #openstack-infra07:29
*** drifterza has joined #openstack-infra07:30
openstackgerritMerged openstack-infra/project-config: Run keystone jobs on xenial  https://review.openstack.org/37106007:30
openstackgerritMerged openstack-infra/project-config: Add root-marker for api-ref, api-guide,install-guide jobs  https://review.openstack.org/37031807:30
AJaegeryolanda: if you have time, here're some changes waiting for +2: https://review.openstack.org/369667 https://review.openstack.org/369519 https://review.openstack.org/368804 https://review.openstack.org/370112 https://review.openstack.org/370996 (more in the queue but those should go in sooner)07:31
openstackgerritMerged openstack-infra/project-config: Add experimental scenario jobs to OpenStack-Ansible  https://review.openstack.org/37064607:31
*** Hal1 has quit IRC07:31
*** Hal2 has joined #openstack-infra07:31
*** amotoki has joined #openstack-infra07:31
*** thorst_ has quit IRC07:31
*** jlanoux has joined #openstack-infra07:31
*** tqtran has quit IRC07:33
*** markvoelker has quit IRC07:34
*** ggnel_t has joined #openstack-infra07:34
*** zhurong has joined #openstack-infra07:34
*** Jeffrey4l has joined #openstack-infra07:37
*** jtomasek_ has quit IRC07:39
openstackgerritMerged openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078107:39
openstackgerritMerged openstack-infra/project-config: Publish docs for tripleo-validations  https://review.openstack.org/36948707:39
openstackgerritMerged openstack-infra/project-config: Add pushMerge for diskimage-builder  https://review.openstack.org/37123007:39
*** kong has quit IRC07:41
*** florianf has quit IRC07:44
*** florianf_ has joined #openstack-infra07:44
openstackgerritMerged openstack-infra/project-config: Add publish-to-pypi for the nova-lxd project to zuul  https://review.openstack.org/37028607:45
yolandaAJaeger, reviewed07:47
*** gildub has quit IRC07:47
AJaegerthanks, yolanda !07:48
yolandaAJaeger, you'll be at mid-cycle right? for one day?07:50
*** nherciu has quit IRC07:50
AJaegeryolanda: Yeah! Just one day ;(07:51
AJaegerIt's driving in on the morning and presenting on another event the next day...07:51
AJaegerAnd Monday didn't work out ;(07:51
AJaegeryolanda: could you review https://review.openstack.org/#/c/366729/ as well, please?07:52
*** amotoki has quit IRC07:52
yolandadone07:53
yolandai'm departing on sunday, it was difficult to arrive same day because airport seems to be a bit far away from the venue07:53
*** florianf_ has quit IRC07:53
AJaegeryeah, Frankfurt is the next airport and beeing there at 10am is tricky.07:55
AJaegerFor me it's 2 1/2 hours car drive on Tuesday ;)07:56
openstackgerritMerged openstack-infra/project-config: remove requirements check for networking-fortinet  https://review.openstack.org/37047007:56
openstackgerritMerged openstack-infra/project-config: add newton gate jobs for trove  https://review.openstack.org/36806707:56
openstackgerritMerged openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050607:56
openstackgerritMerged openstack-infra/project-config: Replace ironic-inspector old jobs with new tempest ones on Newton  https://review.openstack.org/37011207:56
openstackgerritMerged openstack-infra/project-config: Add mistral to scenario003 for tripleo ci  https://review.openstack.org/36880407:56
* AJaeger is looking forward to meeting you in Walldorf, yolanda !07:57
yolandaAJaeger, indeed07:57
*** yamahata has joined #openstack-infra07:58
*** shardy has joined #openstack-infra07:58
openstackgerritMerged openstack-infra/project-config: Add Experiemental PDNS job on xenial  https://review.openstack.org/37099607:59
*** zzzeek has quit IRC08:00
openstackgerritMerged openstack-infra/project-config: Create more Puppet OpenStack Gerrit groups  https://review.openstack.org/37120508:00
*** zzzeek has joined #openstack-infra08:00
openstackgerritMerged openstack-infra/project-config: Added new repos for Murano chef cookbook  https://review.openstack.org/36672908:00
*** jpich has joined #openstack-infra08:01
*** jkraj has joined #openstack-infra08:01
*** tkelsey has joined #openstack-infra08:01
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-infra08:04
*** pasladutout is now known as hashar08:05
*** jascott1 has quit IRC08:06
*** itsuugo has quit IRC08:08
*** ccamacho has joined #openstack-infra08:09
*** amotoki has joined #openstack-infra08:10
*** itsuugo has joined #openstack-infra08:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add missing root-marker for afs-publishing  https://review.openstack.org/37134108:10
*** jordanP has joined #openstack-infra08:11
*** bauzas is now known as bauwser08:11
*** itsuugo has quit IRC08:15
*** jed56 has joined #openstack-infra08:15
*** dtantsur is now known as dtantsur|bbl08:16
*** itsuugo has joined #openstack-infra08:16
*** asettle has joined #openstack-infra08:20
*** lucas-dinner is now known as lucasagomes08:21
*** yamahata has quit IRC08:21
*** itsuugo has quit IRC08:21
*** itsuugo has joined #openstack-infra08:22
AJaegerzigo, deb-python-ironic-lib is not properly setup - it's missing from gerrit/projects.yaml ;(08:25
*** claudiub|2 has quit IRC08:25
*** derekh has joined #openstack-infra08:26
*** salv-orl_ has joined #openstack-infra08:27
*** e0ne has joined #openstack-infra08:28
*** itsuugo has quit IRC08:29
*** amotoki has quit IRC08:30
*** salv-orlando has quit IRC08:31
*** itsuugo has joined #openstack-infra08:31
*** mhickey has joined #openstack-infra08:33
*** YorikSar has quit IRC08:37
*** YorikSar has joined #openstack-infra08:39
*** shardy has quit IRC08:40
*** electrofelix has joined #openstack-infra08:43
rcarrillocruzpleia2: i've gone thru the 'create an infracloud' story and updated/closed items i had assigned08:46
rcarrillocruzmorning to all btw08:46
*** andymaier has joined #openstack-infra08:48
*** itsuugo has quit IRC08:49
*** itsuugo has joined #openstack-infra08:50
*** roxanaghe has joined #openstack-infra08:51
*** HeOS has joined #openstack-infra08:52
*** itsuugo has quit IRC08:55
*** roxanaghe has quit IRC08:55
*** amotoki has joined #openstack-infra08:56
*** itsuugo has joined #openstack-infra08:56
*** _degorenko|afk is now known as degorenko08:57
*** amotoki_ has joined #openstack-infra08:57
*** amotoki has quit IRC09:01
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Install required packages for ansible  https://review.openstack.org/37137509:06
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute021.chocolate public and provisioning IP addresses  https://review.openstack.org/37137609:07
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute022.chocolate public and provisioning IP addresses  https://review.openstack.org/37137709:08
*** itsuugo has quit IRC09:08
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute023.chocolate public and provisioning IP addresses  https://review.openstack.org/37137809:09
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute024.chocolate public and provisioning IP addresses  https://review.openstack.org/37137909:10
*** itsuugo has joined #openstack-infra09:10
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute025.chocolate public and provisioning IP addresses  https://review.openstack.org/37138009:12
*** itsuugo has quit IRC09:15
*** itsuugo has joined #openstack-infra09:16
*** abregman has joined #openstack-infra09:17
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute026.chocolate public and provisioning IP addresses  https://review.openstack.org/37138409:17
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute027.chocolate public and provisioning IP addresses  https://review.openstack.org/37138709:19
*** bethwhite_ has joined #openstack-infra09:19
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute028.chocolate public and provisioning IP addresses  https://review.openstack.org/37138809:21
*** amotoki_ has quit IRC09:22
*** amotoki has joined #openstack-infra09:23
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute029.chocolate public and provisioning addresses  https://review.openstack.org/37139009:24
*** sshnaidm has quit IRC09:25
*** shardy has joined #openstack-infra09:26
*** itsuugo has quit IRC09:27
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute030.chocolate public and provisioning addresses  https://review.openstack.org/37139209:27
*** sshnaidm has joined #openstack-infra09:27
*** dtantsur|bbl is now known as dtantsur09:28
*** hughsaunders is now known as hushsaunders09:28
*** itsuugo has joined #openstack-infra09:29
*** hushsaunders is now known as hush09:29
*** hush is now known as hughsaunders09:30
*** sambetts|afk is now known as sambetts09:33
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute025.chocolate public and provisioning addresses  https://review.openstack.org/37139609:34
*** itsuugo has quit IRC09:34
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Setup translations for Newton  https://review.openstack.org/37139709:35
openstackgerritMerged openstack-infra/system-config: Update compute021.chocolate public and provisioning IP addresses  https://review.openstack.org/37137609:36
*** itsuugo has joined #openstack-infra09:36
openstackgerritMerged openstack-infra/system-config: Update compute022.chocolate public and provisioning IP addresses  https://review.openstack.org/37137709:39
openstackgerritMerged openstack-infra/system-config: Update compute023.chocolate public and provisioning IP addresses  https://review.openstack.org/37137809:39
openstackgerritMerged openstack-infra/system-config: Update compute024.chocolate public and provisioning IP addresses  https://review.openstack.org/37137909:39
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Install required packages for infracloud  https://review.openstack.org/37137509:40
*** itsuugo has quit IRC09:41
*** itsuugo has joined #openstack-infra09:42
*** ihrachys has joined #openstack-infra09:42
*** amitgandhinz has quit IRC09:43
openstackgerritMerged openstack-infra/system-config: Update compute026.chocolate public and provisioning IP addresses  https://review.openstack.org/37138409:43
openstackgerritMerged openstack-infra/system-config: Update compute027.chocolate public and provisioning IP addresses  https://review.openstack.org/37138709:43
*** tkelsey has quit IRC09:43
openstackgerritMerged openstack-infra/system-config: Update compute028.chocolate public and provisioning IP addresses  https://review.openstack.org/37138809:44
openstackgerritMerged openstack-infra/system-config: Update compute029.chocolate public and provisioning addresses  https://review.openstack.org/37139009:46
*** itsuugo has quit IRC09:47
*** itsuugo has joined #openstack-infra09:48
*** amotoki has quit IRC09:49
*** daemontool has joined #openstack-infra09:49
*** amotoki has joined #openstack-infra09:52
*** claudiub|2 has joined #openstack-infra09:53
*** tosky has joined #openstack-infra09:56
*** sree has quit IRC09:57
*** itsuugo has quit IRC09:57
*** itsuugo has joined #openstack-infra09:58
*** daemontool has quit IRC09:59
AJaegerrcarrillocruz, yolanda: Could you review https://review.openstack.org/371397 , please?10:01
*** zhurong has quit IRC10:02
openstackgerritMerged openstack-infra/system-config: Update compute030.chocolate public and provisioning addresses  https://review.openstack.org/37139210:03
openstackgerritMerged openstack-infra/system-config: Update compute025.chocolate public and provisioning addresses  https://review.openstack.org/37139610:03
rcarrillocruzAJaeger: done10:05
yolandalooking10:06
rcarrillocruzpabelanger: i'm back at work10:06
rcarrillocruzlet me know if you did that qcow/raw testing on the compute nodes10:06
*** itsuugo has quit IRC10:07
*** itsuugo has joined #openstack-infra10:09
*** sandanar has joined #openstack-infra10:09
*** thorst_ has joined #openstack-infra10:12
*** acoles_ is now known as acoles10:13
*** aglarendil_ has joined #openstack-infra10:14
*** itsuugo has quit IRC10:14
aglarendil_hello, folks, is it possible that add me to https://review.openstack.org/#/admin/groups/1575,members10:14
openstackgerritMerged openstack-infra/puppet-infracloud: Install required packages for infracloud  https://review.openstack.org/37137510:14
aglarendil_as I am a patch author of https://review.openstack.org/#/c/36734510:14
*** itsuugo has joined #openstack-infra10:15
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config: Add new project os-log-merger  https://review.openstack.org/37141110:15
*** tsg has joined #openstack-infra10:16
ajoyolanda,  ^ I'm not sure I did it completely right, probably I'll have stuff to fix :)10:17
yolandahi ajo , let me check10:18
ajoyolanda: thanks a lot, very appreciated10:18
ajoyolanda, not sure about the defined jobs, for example, I still have no automated tests in place, I want to start working on that first thing once imported to openstack10:20
openstackgerritMerged openstack-infra/project-config: Setup translations for Newton  https://review.openstack.org/37139710:20
*** tsg_ has joined #openstack-infra10:20
*** tsg has quit IRC10:20
*** itsuugo has quit IRC10:20
ajobut not sure if some of the jobs will try to trigger those via tox, for example, and hence fail10:20
yolandaajo, then maybe you can add just merge-check and noop-jobs, and go adding gradually10:21
*** itsuugo has joined #openstack-infra10:21
*** abregman has quit IRC10:21
ajoyolanda, yes, that sounds reasonable. I'm ok if the project imports but we can't merge anything until we create all the tests & tox envs.. that'd force us to provide that as the first patch :D10:22
ajobut setting noop-jobs is also fine10:22
yolandaso if you set python-jobs, and nothing is on place, i think it'll fail. AJaeger , am i right?10:23
yolandaor it just skips?10:23
ajoit would make sense not to let anything import if initially defined jobs are failing ':)10:24
HeOSHello, infra team. I'd like to ask you to review the following request: https://review.openstack.org/#/c/284843/. If you have no objections, please merge it. :)10:25
*** esikachev has quit IRC10:26
*** daemontool has joined #openstack-infra10:26
*** amotoki has quit IRC10:26
ajoyolanda: I'm going to simplify it to noop-jobs, let's walk before we run ':D10:27
yolandawise words10:27
ajoyolanda, thanks for the advice :)10:27
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config: Add new project os-log-merger  https://review.openstack.org/37141110:32
*** itsuugo has quit IRC10:33
AJaegeryolanda: indeed. Ajo would need to set up all the infrastructure with the first patch so  that they pass10:33
ajoAJaeger, I've rolled back to simpler job setup for now ^10:34
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Cleanup not used element  https://review.openstack.org/37142410:34
*** Jokke_ has joined #openstack-infra10:34
AJaegerajo: ok10:34
ajoAJaeger, then I'll prepare the tests, and submit a new patch to project-config :D :)10:35
*** itsuugo has joined #openstack-infra10:36
*** kjorgensen has joined #openstack-infra10:36
*** amitgandhinz has joined #openstack-infra10:36
*** kong has joined #openstack-infra10:38
AJaegerajo: sure10:38
AJaegerajo: will this repo part of an official team - or is it independent?10:39
*** roxanaghe has joined #openstack-infra10:39
ajoAJaeger, do you mean if I want to make it an official project?10:40
*** kjorgensen has quit IRC10:40
AJaegerajo: no. Whether there's an official project that will adopt it.10:40
ajoAJaeger, it's a command line tool to merge logs (and help debugging), it's not a library, or a service other projects can consume10:41
*** itsuugo has quit IRC10:41
ajoAJaeger, https://github.com/mangelajo/os-log-merger#basic-usage10:41
AJaegerajo, so, let's treat it as "Stackforge project" - ok.10:41
ajoAJaeger, correct, makes sense10:41
*** thorst_ has quit IRC10:42
*** itsuugo has joined #openstack-infra10:42
*** sdake has joined #openstack-infra10:42
*** roxanaghe has quit IRC10:44
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Basic folder support  https://review.openstack.org/13430710:44
*** thorst_ has joined #openstack-infra10:47
*** berendt has joined #openstack-infra10:49
*** tuanla_ has quit IRC10:50
*** itsuugo has quit IRC10:51
*** itsuugo has joined #openstack-infra10:52
*** thorst_ has quit IRC10:52
sdakesy folks10:54
sdakeyesterdya i had mentioend there are 3 bereendt accounts ingerrit - and gerrit wont let me selectany of them to add to the core team for kolla10:54
sdakeclarkb had said to have brendt show up in chnnel to select the one he wants to keep so the others couldbe merged10:54
berendti am still confused why i have 3 accounts10:55
sdakeberendt is in channel now10:55
sdakeberendt no idea10:55
sdakeanyone able to help?10:55
berendtthe account id in the account i use every day is 16710:55
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Correct iLO IP for compute029.chocolate and metadata for better tracking  https://review.openstack.org/37143810:55
*** itsuugo has quit IRC10:57
*** itsuugo has joined #openstack-infra10:58
AJaegersdake, berendt: This needs an infra-root to update. Best to wait for the US based to wake up10:59
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Cleanup not used element  https://review.openstack.org/37142410:59
sdakeAJaeger i'm us and awake :)10:59
AJaegerUS based infra-roots ;)10:59
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Correct iLO IP for compute027.chocolate and add metadata for better tracking  https://review.openstack.org/37144010:59
*** thorst_ has joined #openstack-infra10:59
*** dizquierdo has joined #openstack-infra11:00
*** zhurong has joined #openstack-infra11:01
*** ccamacho is now known as ccamacho|lunch11:05
*** kzaitsev_mb has joined #openstack-infra11:08
*** itsuugo has quit IRC11:09
*** lucasagomes is now known as lucas-hungry11:09
*** itsuugo has joined #openstack-infra11:10
openstackgerritMerged openstack-infra/system-config: Correct iLO IP for compute029.chocolate and metadata for better tracking  https://review.openstack.org/37143811:10
*** artmr has joined #openstack-infra11:15
*** itsuugo has quit IRC11:21
*** rtheis has joined #openstack-infra11:21
*** thorst_ has quit IRC11:22
*** itsuugo has joined #openstack-infra11:23
*** Hal1 has joined #openstack-infra11:24
*** thorst_ has joined #openstack-infra11:24
*** Hal2 has quit IRC11:26
*** thorst_ has quit IRC11:28
*** jkilpatr has joined #openstack-infra11:29
*** itsuugo has quit IRC11:30
*** markvoelker has joined #openstack-infra11:31
*** itsuugo has joined #openstack-infra11:31
openstackgerritMerged openstack-infra/system-config: Correct iLO IP for compute027.chocolate and add metadata for better tracking  https://review.openstack.org/37144011:33
*** thorst_ has joined #openstack-infra11:33
*** markvoelker has quit IRC11:35
*** ldnunes has joined #openstack-infra11:36
*** baoli_ has joined #openstack-infra11:37
*** pgadiya has quit IRC11:38
Zaraoh, something about I was wondering about and forgot to ask-- the storyboard api (project 456) is no longer listed in the storyboard project group here: https://storyboard.openstack.org/#!/project_group/57 I'm fairly sure it used to list all our projects; wondering if someone could return it to its rightful home?11:38
*** thorst_ has quit IRC11:38
*** itsuugo has quit IRC11:38
Zara(only admin users can edit projects and project groups)11:39
*** itsuugo has joined #openstack-infra11:39
*** andreas_s has quit IRC11:43
*** itsuugo has quit IRC11:44
openstackgerritDarren Chan proposed openstack-infra/irc-meetings: Change Ops Guide meeting time  https://review.openstack.org/37146411:45
*** itsuugo has joined #openstack-infra11:45
EmilienMAJaeger, yolanda: hello, are you gerrit admin?11:46
*** abregman has joined #openstack-infra11:46
*** sandanar_ has joined #openstack-infra11:46
*** EricGonczer_ has joined #openstack-infra11:47
EmilienMso we merged https://review.openstack.org/#/c/371205/ and we need to add puppet-manager-core in puppet-gnocchi, puppet-ceilometer, puppet-panko, puppet-aodh and puppet-ironic again11:52
rcarrillocruzi can do that11:52
EmilienMrcarrillocruz: thanks a lot11:52
rcarrillocruzgimme a sec11:52
EmilienMsure thing11:52
EmilienMrcarrillocruz: there is an example https://review.openstack.org/#/admin/groups/1165,members11:52
EmilienMwith puppet-keystone11:53
EmilienMwe just need puppet-manager-core in "included groups"11:53
*** trown|outtypewww is now known as trown11:53
rcarrillocruzi think i got them all11:55
rcarrillocruzcan you check EmilienM ?11:55
persiaInfra-Cloud folk: I just bumped into https://storyboard.openstack.org/#!/board/8 , which looks mostly out of date (lots of the things listed as "TODO" are actually done).  Depending on whether people want to use it, it may benefit from being cleaned or deleted.11:55
*** jed56 has quit IRC11:55
rcarrillocruzpersia: we are going to have a track on infra-cloud next week at the mid-cycle, i expect we'll go thru that list11:56
rcarrillocruzi checked my stuff today and updated, and cleaned some that i know have been implemented already11:56
EmilienMrcarrillocruz: everything is perfect11:56
EmilienMrcarrillocruz: thanks!11:56
rcarrillocruznp11:56
persiarcarrillocruz: Cool.  From a brief look, it mostly appeared forgotten, and I feared it might suggest to other folk that you hadn't done all the work you've been doing this past while :)11:57
rcarrillocruzheh11:57
*** dprince has joined #openstack-infra12:00
openstackgerritThomas Goirand proposed openstack-infra/project-config: Add missing openstack/deb-python-ironic-lib  https://review.openstack.org/37147912:01
*** asettle has quit IRC12:01
*** asettle has joined #openstack-infra12:01
*** ccamacho|lunch is now known as ccamacho12:04
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Allow to configure workflow job in Groovy file  https://review.openstack.org/32374412:05
*** asettle has quit IRC12:06
*** itsuugo has quit IRC12:07
*** esikachev has joined #openstack-infra12:07
amrithyolanda, thanks for approving the two trove gate jobs12:07
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add Emilien Macchi ssh key to TripleO admins  https://review.openstack.org/37148412:07
*** gongysh has quit IRC12:08
*** amoralej is now known as amoralej|lunch12:08
*** sdague has joined #openstack-infra12:09
*** itsuugo has joined #openstack-infra12:09
*** psilvad has joined #openstack-infra12:11
*** liusheng has quit IRC12:11
*** liusheng has joined #openstack-infra12:11
*** lucas-hungry is now known as lucasagomes12:11
*** asettle has joined #openstack-infra12:12
*** thorst_ has joined #openstack-infra12:14
*** thorst_ has quit IRC12:14
*** raildo has joined #openstack-infra12:15
*** thorst_ has joined #openstack-infra12:20
*** itsuugo has quit IRC12:20
openstackgerritMerged openstack-infra/puppet-infracloud: Cleanup not used element  https://review.openstack.org/37142412:21
*** itsuugo has joined #openstack-infra12:21
*** pradk has joined #openstack-infra12:23
*** sandanar__ has joined #openstack-infra12:23
*** sandanar has quit IRC12:23
*** sandanar_ has quit IRC12:23
*** fguillot has joined #openstack-infra12:24
AJaegerrcarrillocruz, yolanda : Can you help merge a broken setup, please? https://review.openstack.org/#/c/37147912:24
*** rhallisey has joined #openstack-infra12:25
AJaegerAlso, https://review.openstack.org/371134 would be nice - but needs a manual git push --force of the repo12:25
*** markvoelker has joined #openstack-infra12:26
*** itsuugo has quit IRC12:26
*** itsuugo has joined #openstack-infra12:27
*** thorst_ has quit IRC12:28
*** yamamot__ has quit IRC12:28
*** thorst_ has joined #openstack-infra12:29
*** rfolco has joined #openstack-infra12:30
*** yaume has joined #openstack-infra12:31
*** edmondsw has joined #openstack-infra12:32
*** sree has joined #openstack-infra12:32
*** rtheis has quit IRC12:33
*** itsuugo has quit IRC12:34
*** itsuugo has joined #openstack-infra12:35
*** dizquierdo has quit IRC12:37
*** mriedem has joined #openstack-infra12:38
openstackgerritMerged openstack-infra/project-config: Add missing openstack/deb-python-ironic-lib  https://review.openstack.org/37147912:38
*** mtanino has joined #openstack-infra12:38
*** itsuugo has quit IRC12:40
*** fguillot has quit IRC12:40
*** itsuugo has joined #openstack-infra12:41
openstackgerritAndy Smith proposed openstack-infra/project-config: Update the gate-tempest-neutron-dsvm-src-oslo.messaging test  https://review.openstack.org/37150212:41
*** flepied has quit IRC12:42
*** gordc has joined #openstack-infra12:42
*** rlandy has joined #openstack-infra12:44
*** yaume has quit IRC12:44
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config: Add new project os-log-merger  https://review.openstack.org/37141112:45
*** vikrant has quit IRC12:48
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add ipv6 nic-configs  https://review.openstack.org/36447912:49
*** xyang1 has joined #openstack-infra12:49
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367412:49
*** Goneri has joined #openstack-infra12:50
*** matt-borland has joined #openstack-infra12:52
*** mtanino has quit IRC12:52
*** thorst_ has quit IRC12:55
*** thorst_ has joined #openstack-infra12:56
*** jcoufal_ has joined #openstack-infra12:58
*** amitgandhinz has quit IRC12:59
*** makowals has joined #openstack-infra12:59
openstackgerritIgor Duarte Cardoso proposed openstack-infra/irc-meetings: Update Neutron Classification Framework Meeting  https://review.openstack.org/37065313:00
*** sdake_ has joined #openstack-infra13:00
*** drawsmcgraw has joined #openstack-infra13:00
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config: Add new project os-log-merger  https://review.openstack.org/37141113:01
*** makowals has quit IRC13:01
openstackgerritZara proposed openstack-infra/infra-manual: Detail what the 'Story' and 'Task' options do  https://review.openstack.org/37151513:01
*** thorst_ has quit IRC13:02
Zarafungi ^13:02
Zarahopefully what you were imagining13:02
*** drifterza has quit IRC13:02
*** makowals_ has joined #openstack-infra13:02
*** makowals_ has quit IRC13:03
*** sdake has quit IRC13:03
*** jaosorior has quit IRC13:03
*** thorst_ has joined #openstack-infra13:04
*** makowals has joined #openstack-infra13:04
*** jaosorior has joined #openstack-infra13:04
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute031.chocolate public and provisioning addresses  https://review.openstack.org/37151913:05
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute032.chocolate public and provisioning addresses  https://review.openstack.org/37152013:06
openstackgerritMonty Taylor proposed openstack-infra/puppet-statusbot: Add Twitter support  https://review.openstack.org/36823813:07
*** psilvad has quit IRC13:07
*** mdrabe has joined #openstack-infra13:08
*** yamamoto has joined #openstack-infra13:09
*** esberglu has joined #openstack-infra13:10
*** kgiusti has joined #openstack-infra13:10
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute034.chocolate public and provisioning addresses  https://review.openstack.org/37152213:12
rcarrillocruzzomg, twitter support, we're so hip13:13
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add compute035.chocolate public and provisioning addresses  https://review.openstack.org/37152413:14
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute036.chocolate public and provisioning addresses  https://review.openstack.org/37152813:15
*** amitgandhinz has joined #openstack-infra13:15
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute039.chocolate public and provisioning addresses  https://review.openstack.org/37153213:18
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute040.chocolate public and provisioning addresses  https://review.openstack.org/37153413:19
*** annegentle has joined #openstack-infra13:19
mordredrcarrillocruz: I hear twitter is dead :)13:20
rcarrillocruz:D13:20
mordredrcarrillocruz: if you wanted to +A that ...13:21
rcarrillocruzi sure can13:21
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Update compute032.chocolate public and provisioning addresses  https://review.openstack.org/37152013:21
mordredrcarrillocruz: also, https://review.openstack.org/#/c/368239/ is the follow up - it's still re-testing though13:22
* mtreinish is hungry13:22
mtreinishrcarrillocruz: I'm expecting you to bring lots of chocolate next week to the sprint now :)13:23
rcarrillocruzmordred: lgtm13:23
mordredrcarrillocruz: woot! thanks13:23
*** yamamoto has quit IRC13:24
rcarrillocruzmtreinish: heh13:24
* rcarrillocruz will sure buy a giant toblerone at the airport lounge13:24
mtreinishhah13:25
*** akshai has joined #openstack-infra13:25
mtreinishrcarrillocruz: of course I always expect germany to be like this: https://www.youtube.com/watch?v=ZOziWm_MJ9k13:26
*** amoralej|lunch is now known as amoralej13:27
*** akshai_ has joined #openstack-infra13:27
*** tonytan4ever has joined #openstack-infra13:27
rcarrillocruzoh god, that was a great episode13:27
*** ddieterly has joined #openstack-infra13:27
*** berendt has quit IRC13:28
rcarrillocruzone of the simpsons videogame even had a stage based on that episode13:28
rcarrillocruzhttps://www.youtube.com/watch?v=oTaTVGqpDas13:28
*** alaski is now known as lascii13:28
Zaraoh, I played that13:29
Zarathen I lost the disc13:29
mordreddisc13:29
mordredyou know - because things used to be on discs13:29
Zara:D13:30
ZaraI had it on PS2, I think, and one of the racing games on the computer13:30
Zara(also a disc)13:30
*** akshai has quit IRC13:30
openstackgerritMerged openstack-infra/system-config: Update compute031.chocolate public and provisioning addresses  https://review.openstack.org/37151913:30
openstackgerritMerged openstack-infra/system-config: Update compute034.chocolate public and provisioning addresses  https://review.openstack.org/37152213:31
*** thorst_ has quit IRC13:32
*** rtheis has joined #openstack-infra13:32
*** kencjohnston_ has joined #openstack-infra13:32
*** fguillot has joined #openstack-infra13:32
* rcarrillocruz remembers ages ago used to order batches of 100 floppy disks to pirate games on MS-DOS13:33
*** flepied has joined #openstack-infra13:35
*** Ravikiran_K has quit IRC13:36
openstackgerritMerged openstack-infra/system-config: Add compute035.chocolate public and provisioning addresses  https://review.openstack.org/37152413:36
*** thorst_ has joined #openstack-infra13:37
*** cardeois has joined #openstack-infra13:37
openstackgerritMerged openstack-infra/system-config: Update compute036.chocolate public and provisioning addresses  https://review.openstack.org/37152813:38
*** khivin has quit IRC13:41
openstackgerritMerged openstack-infra/system-config: Update compute039.chocolate public and provisioning addresses  https://review.openstack.org/37153213:42
openstackgerritMerged openstack-infra/nodepool: Stop using custom ZK heartbeat.  https://review.openstack.org/37035013:42
*** thorst_ has quit IRC13:42
openstackgerritMerged openstack-infra/nodepool: Allow deleting non-existing build requests  https://review.openstack.org/37039613:42
*** sree has quit IRC13:43
*** tonytan_brb has joined #openstack-infra13:44
openstackgerritPeter Szabo proposed openstack-infra/jenkins-job-builder: Add missing parameters for gitlab-notifier  https://review.openstack.org/37154713:44
*** rodrigods has quit IRC13:45
*** rodrigods has joined #openstack-infra13:45
*** piet has joined #openstack-infra13:45
*** thorst_ has joined #openstack-infra13:45
*** daemontool has quit IRC13:46
*** thorst_ has quit IRC13:46
*** tonytan4ever has quit IRC13:46
openstackgerritMerged openstack-infra/nodepool: Use sequence znodes for image build numbers  https://review.openstack.org/37043513:47
openstackgerritMerged openstack-infra/system-config: Update compute040.chocolate public and provisioning addresses  https://review.openstack.org/37153413:48
*** csomerville has joined #openstack-infra13:48
*** cody-somerville has quit IRC13:48
*** eharney has joined #openstack-infra13:48
openstackgerritMerged openstack-infra/system-config: Update compute032.chocolate public and provisioning addresses  https://review.openstack.org/37152013:48
kgiustiHi - I've made some sort of rookie mistake when I created this new test: http://logs.openstack.org/27/370927/3/check/gate-oslo.messaging-dsvm-functional-py34-amqp1-centos7-nv/7ee15d6/console.html#_2016-09-16_12_08_14_41215813:50
kgiustiany advice?13:50
*** sdake_ has quit IRC13:52
kgiustiI must have missed some setup directive to have python 3.4 installled....13:52
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add chocolate computes deployed and reachable from 21 to 30  https://review.openstack.org/37155113:53
*** thorst_ has joined #openstack-infra13:54
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Fix on Filters  https://review.openstack.org/37155713:56
*** makowals has quit IRC13:56
*** thorst_ has quit IRC13:57
*** fguillot has quit IRC13:57
*** roxanaghe has joined #openstack-infra13:58
*** thorst_ has joined #openstack-infra13:58
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Fix on Filters  https://review.openstack.org/37155713:59
*** kencjohnston_ has quit IRC13:59
openstackgerritMerged openstack-infra/openstackid-resources: Fix on Filters  https://review.openstack.org/37155713:59
*** bin_ has joined #openstack-infra14:02
*** coolsvap has quit IRC14:02
AJaegerkgiusti: devstack jobs have to specify their dependencies14:02
*** xarses_ has quit IRC14:03
openstackgerritMerged openstack-infra/puppet-statusbot: Add Twitter support  https://review.openstack.org/36823814:03
*** jcoufal__ has joined #openstack-infra14:03
*** roxanaghe has quit IRC14:03
*** ociuhandu has joined #openstack-infra14:04
*** esikachev has quit IRC14:04
*** thorst_ has quit IRC14:04
kgiustiAJaeger: thanks for replying - I'm a little unclear how that is done, can you point me to an example or the config file(s) I need to update?14:04
*** jcoufal_ has quit IRC14:06
fungihrm... sdake and berendt didn't stick around14:06
*** asettle has quit IRC14:06
AJaegerkgiusti: normally this is done like http://git.openstack.org/cgit/openstack-dev/devstack/tree/files - not sure what's the best way is in your case14:06
AJaegermorning, fungi!14:06
*** tongli has joined #openstack-infra14:06
fungilooks like berendt's duplicate account ids are 167, 21091 and 2110614:06
*** asettle has joined #openstack-infra14:07
*** asettle has quit IRC14:07
*** psilvad has joined #openstack-infra14:07
*** asettle has joined #openstack-infra14:07
AJaegerfungi, I hope berendt comes back online. I14:07
kgiustiAJaeger: kk - I'll dig around, thanks for the pointer.14:07
*** yamamoto has joined #openstack-infra14:07
*** rbrndt has joined #openstack-infra14:07
*** esikachev has joined #openstack-infra14:07
AJaegerHe knows he needs to be around for this ;)14:08
fungi167 and 21091 both have usernames but 21091 hasn't submitted contact info for the icla14:08
AJaegerfungi, do you have some time for another debian repo push, please? https://review.openstack.org/37113414:08
dulekHi, when will grenade tests on master switch from deploying stable/mitaka->master to stable/newton->master? Do I need to change anything in project-config for my project?14:08
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Start parameterizing the network config in bifrost  https://review.openstack.org/37156514:08
* AJaeger likes to have an unbroken project-config again...14:08
fungiso i expect i can deactivate those newer 21091 and 21106 accounts, but will reassign the openids for them to 167 just to be safe14:09
*** ociuhandu has quit IRC14:09
openstackgerritJulien Danjou proposed openstack-infra/project-config: Fix Python 3 gating for tooz  https://review.openstack.org/36878714:09
*** thorst_ has joined #openstack-infra14:09
fungiAJaeger: maybe in a while, but it's a fairly time-consuming process to make sure i've got all the branches and tags pulled/pushed. not entirely straightforward so takes a bit of focused concentration14:10
*** jaosorior has quit IRC14:10
AJaegerfungi, argh ;( what fun :(14:10
fungiAJaeger: as long as the 371134 patch doesn't merge first, it's fine14:10
*** ociuhandu has joined #openstack-infra14:11
*** ramishra has quit IRC14:11
AJaegerwith those comments in 371134, I hope not ;)14:11
fungiAJaeger: yeah, that's why we automated importing. complex and easy to get wrong when doing it by hand14:11
AJaegerand it does not scale ;(14:11
*** thorst_ has quit IRC14:12
*** asettle has quit IRC14:12
*** asettle has joined #openstack-infra14:13
*** ramishra has joined #openstack-infra14:13
fungiokay, berendt's duplicate accounts are all cleaned up now, in case he or sdake return14:13
*** psilvad has quit IRC14:14
*** rtheis has quit IRC14:14
*** thorst_ has joined #openstack-infra14:15
*** senk has quit IRC14:15
AJaegerfungi, I send berendt an email with the above IRC chat so that he knows. Thanks!14:15
*** ociuhandu has quit IRC14:15
*** thorst_ has quit IRC14:16
fungiAJaeger: oh, thanks!14:16
fungii'm looking into the deb-novnc reimport now14:16
fungigonna require more coffee than duplicate account cleanup though14:17
*** asettle has quit IRC14:18
*** asettle has joined #openstack-infra14:18
*** ddieterly is now known as ddieterly[away]14:19
*** ddieterly[away] is now known as ddieterly14:19
*** hongbin has joined #openstack-infra14:19
*** tonytan_brb is now known as tonytan4ever14:20
*** thorst_ has joined #openstack-infra14:22
*** asettle has quit IRC14:23
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Ensure that gerrit projects have zuul pipelines  https://review.openstack.org/36396914:23
*** salv-orl_ has quit IRC14:23
*** thorst_ has quit IRC14:23
*** asettle has joined #openstack-infra14:23
*** salv-orlando has joined #openstack-infra14:24
AJaegerfungi, do we need to import it manually at all? http://git.openstack.org/cgit/openstack/deb-novnc gives "no repositories found"14:25
*** eeiden has joined #openstack-infra14:25
AJaegerOr is gerrit setup and we don't do another import?14:26
*** jed56 has joined #openstack-infra14:26
fungiAJaeger: git.o.o says that because we generate the cgit index from the projects.yaml14:26
AJaegerah14:26
fungithe problem is the repo already exists in gerrit14:26
AJaegerunderstood14:26
fungiand needs its content overwritten forcibly14:27
*** zhurong has quit IRC14:28
*** annegentle has quit IRC14:28
*** rtheis has joined #openstack-infra14:29
rcarrillocruzeven if it's not 100% success at times, it's so self-satisfying to enroll machines from infracloud, run the deploy playbook and get them back as provisioned and reachable from the intarweb14:31
rcarrillocruztruly awesome work of so many people involved14:31
rcarrillocruzfrom ironic, to infra folks14:31
fungiespecially you, yolanda and crinkle though!14:31
fungiit's an awesome success14:31
AJaegerthanks, rcarrillocruz, yolanda and crinkle. It's great to see the updates flying in!14:32
rcarrillocruzlooking forward to get all of that up and running next week :D14:32
fungiand hopefully a pattern we can repeat with donations from other companies in the future now that you've worked out a lot of the kinks14:32
yolandai'm really happy that this effort finally arrived to success14:32
openstackgerritMerged openstack-infra/puppet-infracloud: Start parameterizing the network config in bifrost  https://review.openstack.org/37156514:33
fungii mean, being able to use it will be cool, but deploying a workable production-grade openstack environment from scratch in a colo with little hands-on help is amazing14:33
fungiand now we have a model for this which is reusable14:34
fungiand only using free/openly developed tools and automation14:34
fungiwith all our fundamental configuration published14:34
rcarrillocruz++14:35
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375014:36
*** thorst_ has joined #openstack-infra14:37
*** xarses_ has joined #openstack-infra14:37
*** bnemec has quit IRC14:37
pleia2rcarrillocruz: thanks re: infra-cloud story14:38
*** itsuugo has quit IRC14:38
rcarrillocruzyeah, i know some stuff is already fixed, but i rather leave that to assigned people yolanda and crinkle14:38
*** mdrabe has quit IRC14:39
yolandafungi well, we are in the process to make it more reusable. Look that for example we are reconsuming in opnfv now14:39
rcarrillocruzdon't want to anticipate, i may be wrong on stuff :/14:39
fungiyolanda: right! i keep forgetting that it's already been repeated outside our team now14:39
rcarrillocruzthe fact we are refactoring, not only makes it reusable to other people like opnfv, is that eases the path to have integration testing at some point in the gate14:39
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: POC DO NOT MERGE: Add virt-setup option to tripleo.sh  https://review.openstack.org/37158714:39
yolandaat the moment i only got success on vms, but right now i'm in the process of testing on baremetal14:39
rcarrillocruzlike, 'this change, let's do a deploy of a cloud of one baremetal, one controller, one compute, will it break it?'14:40
fungiawesome14:40
yolanda++14:40
*** itsuugo has joined #openstack-infra14:41
*** burgerk has joined #openstack-infra14:42
*** kzaitsev_mb has quit IRC14:43
*** samuelBartel has joined #openstack-infra14:43
*** mdrabe has joined #openstack-infra14:44
yolandapleia2, rcarrillocruz , i missed that "infra-cloud story", can you give me details?14:44
fungiAJaeger: i'll write up something for our gerrit operations page to outline the process so it's not so painful to keep reverse engineering from the code in jeepyb, but the process basically looks like this: http://paste.openstack.org/show/580046/14:44
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Make internal builder.Jenkins attrs internal-only.  https://review.openstack.org/31975614:44
rcarrillocruzyolanda: https://storyboard.openstack.org/#!/board/814:45
*** inc0 has joined #openstack-infra14:45
pleia2yolanda: the story itself is at https://storyboard.openstack.org/#!/story/200017514:47
AJaegerI see, fungi. Quite a few steps...14:47
openstackgerritFrédéric Guillot proposed openstack-infra/project-config: (WIP) Add new project: Almanach ansible roles  https://review.openstack.org/37159014:48
*** esikachev has quit IRC14:48
*** senk has joined #openstack-infra14:48
*** fguillot has joined #openstack-infra14:48
*** jcoufal__ has quit IRC14:48
yolandapleia2, ok, i updated some of the status, and i unassigned myself from the internal ones.. nothing i can do now14:48
pleia2yolanda: thanks!14:48
AJaegerfungi, so are those steps done and we can merge 371134 now?14:49
*** bnemec has joined #openstack-infra14:49
fungiAJaeger: i just approved it, so yes i hope14:49
AJaegerfungi: just seeing the gerrit mail coming in;)14:49
AJaegerthanks14:49
AJaegerfungi, could you +2a https://review.openstack.org/#/c/363969 as well?14:50
*** zz_dimtruck is now known as dimtruck14:50
*** askb has quit IRC14:50
AJaegerThat would have caught a few of the problems with imports - like fixed with https://review.openstack.org/37147914:50
*** rfolco has quit IRC14:50
*** kong has quit IRC14:51
pleia2not around much today, lots to do before I fly out at the break of dawn tomorrow14:52
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: opensuse: Add support for openSUSE Leap  https://review.openstack.org/37160214:53
*** psilvad has joined #openstack-infra14:55
*** tsg_ has quit IRC14:55
openstackgerritMerged openstack-infra/project-config: Revert "Temporarily remove deb-novnc from projects.yaml"  https://review.openstack.org/37113414:56
*** yamamoto has quit IRC14:58
*** yamamoto has joined #openstack-infra14:58
*** yamamoto has quit IRC14:59
jeblairmordred: what's the part of the ansible-puppet system that puts system-config on the remote host?14:59
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Use JJBConfig() class  https://review.openstack.org/31975614:59
AJaegerodyssey4me: please review https://review.openstack.org/#/c/37159014:59
*** yamamoto has joined #openstack-infra15:00
AJaegerjeblair: I created a few more changes for afs-docs, I hope I covered all documents now ;)15:00
jeblairAJaeger: neat!15:01
AJaegerjeblair: this is a change for project-config: https://review.openstack.org/371341 - reviews are welcome.15:01
*** kzaitsev_mb has joined #openstack-infra15:01
*** amotoki has joined #openstack-infra15:01
AJaegerjeblair: and this one neats a new release of openstack-doc-tools: https://review.openstack.org/#/c/371316/15:01
*** amotoki has quit IRC15:01
*** dizquierdo has joined #openstack-infra15:02
*** thcipriani|afk is now known as thcipriani15:02
*** jcoufal_ has joined #openstack-infra15:03
AJaegerjeblair: will we cover specs as well? Then jenkins/jobs/specs-jobs.yaml needs an update for marker files.15:04
jeblairAJaeger: specs are on static.o.o via scp right now, so not so urgent15:04
jeblairAJaeger: we could move them to this later15:04
AJaegerjeblair: ok. Just say them on the etherpad and noticed we didn't convert them yet...15:05
*** Apsu has joined #openstack-infra15:06
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add network config for puppet-infracloud on hiera  https://review.openstack.org/37160715:06
fungiAJaeger: yeah, there are a bunch of sites published to static.o.o that could take advantage of this (we could even still serve them with apache on static.o.o but straight from an afs mount)15:07
fungiconsider those possible future improvements15:08
*** Swanson has quit IRC15:08
AJaegerfungi: yeah - those are much easier than docs.o.o but once we done docs.o.o, they should be trivial ;)15:09
jeblairand then files will be deleted from them as well :)15:09
AJaegeryeah!15:09
pabelangerrcarrillocruz: sure, I only change compute001 the other day, still need to apply the change to 002-005.  We could likely do that this morning15:10
*** rfolco has joined #openstack-infra15:10
AJaegerjeblair: what is your plan on when to start publishing to AFS?15:11
jeblairAJaeger: asap15:11
AJaeger;)15:11
jeblairAJaeger: we need the launchers restarted with the afs publisher changes and also running afs and having the keytab installed on them15:12
jeblairianw was working on the keytab thing, but it's his weekend now, so maybe someone can take that over15:12
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: opensuse: Add support for openSUSE Leap  https://review.openstack.org/37160215:13
*** jkraj has quit IRC15:13
*** dtantsur is now known as dtantsur|pto15:13
*** mdrabe has quit IRC15:14
*** mdrabe has joined #openstack-infra15:14
*** sdake has joined #openstack-infra15:14
crinklepleia2: rcarrillocruz i updated the tasks i was assigned to15:14
*** annegentle has joined #openstack-infra15:15
*** rcernin has quit IRC15:15
AJaegerteam, could I get a review on a project-config change to add root markers for the AFS Docs publishing, please? https://review.openstack.org/37134115:16
*** thorst_ has quit IRC15:16
*** hashar is now known as hasharAway15:16
AJaegerAnd one more review on this change to remove a broken coverage job - from horizon: https://review.openstack.org/37064115:17
jeblairpabelanger: do you have time to take over the zuul launcher keytab puppet plumbing work?15:17
jeblair(i'm still trying to get files01 launched)15:17
*** acoles is now known as acoles_15:18
*** sdake_ has joined #openstack-infra15:18
*** jkraj has joined #openstack-infra15:19
pabelangerjeblair: sure15:19
*** sdake has quit IRC15:20
*** Hal1 has quit IRC15:20
openstackgerritMerged openstack-infra/project-config: Ensure that gerrit projects have zuul pipelines  https://review.openstack.org/36396915:20
*** annegentle has quit IRC15:21
*** thorst_ has joined #openstack-infra15:22
*** bnemec has quit IRC15:22
*** senk__ has joined #openstack-infra15:23
*** senk has quit IRC15:23
*** thorst_ has quit IRC15:24
*** gouthamr has joined #openstack-infra15:25
jeblairfungi, mordred: huh, this is weird.... i think that puppet-ansible overwrites puppet.conf (with content "A") and then runs puppet which overwrites puppet.conf (with content "B")15:25
*** bnemec has joined #openstack-infra15:25
jeblairfungi, mordred: and i think this happens on both the puppetmaster and the target hosts15:25
jeblairand every puppet run15:25
*** ramishra has quit IRC15:25
jeblairso, essentially, we're running puppet with the configuration that ansible-puppet puts in place15:25
jeblairbut we can't see it15:25
jeblairbecause puppet itself instantly overwrites it with something different15:26
openstackgerritMerged openstack-infra/jenkins-job-builder: Use JJBConfig() class  https://review.openstack.org/31975615:26
*** vinaypotluri has joined #openstack-infra15:26
*** mmedvede has quit IRC15:26
jeblairfungi, mordred: at this point, i suspect that we should not be having puppet manage puppet.conf15:26
*** ramishra has joined #openstack-infra15:27
*** thorst_ has joined #openstack-infra15:27
*** sandanar__ has quit IRC15:28
clarkbI want to say the reason for having puppet manage it is so that you can stillbootstrap without ansible15:29
*** samuelBartel has quit IRC15:29
clarkbeg puppet apply directly15:29
*** EricGonc_ has joined #openstack-infra15:29
mordredjeblair, clarkb: it sounds like we're definitely in a terrible middle ground15:30
mordredwe should at the very least not have both things manage it15:30
openstackgerritFrédéric Guillot proposed openstack-infra/project-config: (WIP) Add new project: Almanach ansible roles  https://review.openstack.org/37159015:30
*** tsg_ has joined #openstack-infra15:30
jeblairclarkb: i'm not sure that's possible at this point -- i mean, you can run puppet apply *after* ansible has copied everything in place15:30
*** jkraj has quit IRC15:30
clarkbjeblair: I do it all the time...15:30
jeblairbut with on-host hiera, i think that the ansible step is essentially required15:30
jeblairclarkb: right, i do to, *after* ansible has already run15:31
clarkbI do it all the time with zero ansible15:31
*** EricGonczer_ has quit IRC15:31
jeblairok15:31
jeblairclarkb: are you suggesting i leave this as-is?15:31
*** migi is now known as migi_afk15:32
clarkbno, just giving a use case where puppet configuring itself may still be something we want15:32
*** tsg__ has joined #openstack-infra15:32
openstackgerritZara proposed openstack-infra/python-storyboardclient: Add some sample pythonclient scripts  https://review.openstack.org/37162015:32
mordredjeblair: in ansible-role-puppet tasks/main.yml ... task "copy puppet modules" is what copies things over15:32
jeblairclarkb: okay.  i can not concieve of a way that happens in production, but i am willing to take your word for it.15:32
jeblairmordred: yep, thanks :)15:32
mordredjeblair: manage_config is the flag that tells puppet-role-ansible to manage the puppet.conf file or not15:33
mordredfwiw15:33
mordredI'm guessing you have found this already15:33
jeblairclarkb: i guess i either need a detailed description from you of your use-case, or some help from you in solving this15:33
jeblairclarkb: is it used in dib when making nodepool images?15:34
clarkbjeblair: basically you can git clone system config, run install puppet, thr puppet apply various things for testing or deployments outside of the ansible cron. dib builds is one case that does this yes15:34
jeblairclarkb: okay.  i think install_puppet is another place where some of these files are created though15:35
*** tsg_ has quit IRC15:35
jeblairclarkb: so it may still be the case that management in puppet is redundant15:35
jeblairclarkb: i mean, install_puppet.sh needs to set up puppet.conf (and friends) well enough for that apply step to work, right?15:37
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add network config for puppet-infracloud on hiera  https://review.openstack.org/37160715:37
openstackgerritPaul Belanger proposed openstack-infra/puppet-zuul: Add keytab support for sites  https://review.openstack.org/37162815:37
pabelangerwe need to patch puppet-zuul for afs-docs^15:37
pabelangerwe are missing the keytab file on launchers15:38
*** matbu is now known as matbu|brb15:38
clarkbjeblair: I think it maypass enough in on command line to avoid needing it the first run but I would have to double check that15:38
odyssey4meAJaeger done15:38
openstackgerritZara proposed openstack-infra/python-storyboardclient: Add some sample pythonclient scripts  https://review.openstack.org/37162015:38
mordredas we think about which direction to fix or not fi this - also worth remembering that we'd like to stop running puppet in dib builds15:39
*** itsuugo has quit IRC15:39
fungiyeah, i can see a way out where ansible and install_puppet.sh can both populate puppet.conf but not have puppet itself manage that file15:40
jeblairyeah, i think we need to fix this.  i don't think it's good that we run with secret config files, and i used to very much rely on the ability to run puppet on a host in an experimental environment, which i still have not been able to replicate with this system (i'm still pulling out threads)15:40
fungii think that covers both the ansible and ansibleless-bootstrap cases?15:40
jeblairi am convinced that the puppet.conf manangement in puppet itself is not used for any of the 3 use cases we know about right now15:40
*** itsuugo has joined #openstack-infra15:41
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add newton translation jobs  https://review.openstack.org/37165815:42
AJaegerthanks, odyssey4me15:42
mordredI'd vote for having ansible always manage the file and just put in a playbook in system-config that is good for running puppet locally not remotely15:42
mordredthat way we always know that we've updated the puppet.conf before running puppet in all scenarios15:43
asettleAJaeger: question - what does manuals use to get the DRAFT watermark on the back of the unstable docs?15:43
jeblairmordred: ansible-puppet updates puppet.conf15:43
jeblairalready15:43
mordredyah. I'm saying I vote that we make that the thing that does it, and kill the other thing15:43
mordredI'm agreeing with you, I believe - just doing it with too manyu words15:43
jeblair:)15:43
fungii think this also touches on asselin__'s https://review.openstack.org/363074 change15:43
AJaegerasettle: see https://review.openstack.org/#/c/365872/ - it uses brian's sphinxmark extension15:44
*** psachin has quit IRC15:44
fungior at a minimum, 363074 came about because of the mess we have around what controls the puppet configs15:44
asettleThank you AJaeger :D15:44
*** xarses_ is now known as xarses15:45
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Regression test for update command with --delete-old  https://review.openstack.org/37166115:45
*** roxanaghe has joined #openstack-infra15:46
openstackgerritZara proposed openstack-infra/python-storyboardclient: Add some sample pythonclient scripts  https://review.openstack.org/37162015:46
pabelangerjeblair: mordred: fungi: if you don't mind reviewing: 371018 once landed I believe we can start restarting launchers15:48
jeblairclarkb, fungi, mordred: i would like to stop the puppet run all cron on puppetmaster, because the current config doesn't allow me to test this without changing the production env15:48
openstackgerritZara proposed openstack-infra/python-storyboardclient: Add some sample pythonclient scripts  https://review.openstack.org/37162015:48
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Move macro expansion into YamlParser.  https://review.openstack.org/33307615:49
*** jcoufal__ has joined #openstack-infra15:49
jeblairpabelanger: i think you mean https://review.openstack.org/37162815:49
clarkbjeblair: just make sure its not halfway through the git*/review.o.oplaybook as the ordering there is due to deps. shpuld be fine once iy is running the else playbook15:49
pabelangerapologies, yes 37162815:49
*** electrofelix has quit IRC15:50
*** mriedem is now known as mriedem_afk15:50
*** jcoufal_ has quit IRC15:50
*** roxanaghe has quit IRC15:50
mordredjeblair: ok15:50
*** tphummel has joined #openstack-infra15:50
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add network config for puppet-infracloud on hiera  https://review.openstack.org/37160715:51
jeblairthis is strange15:52
jeblair2016-09-16 14:02:14,633 p=15307 u=root |  ok: [lists.openstack.org]15:52
jeblairthat's the last entry in puppet_run_all.log15:53
jeblairand ansible is not running15:53
pabelangersounds like OOM again15:53
pabelangeror ansible-playbook process still hanging around15:54
mordredI'd really love to track down the oom for real15:54
*** mhickey has quit IRC15:54
jeblairoh yep, oom.  thanks.15:56
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add newton translation jobs  https://review.openstack.org/37165815:56
*** jlanoux has quit IRC15:58
*** fernnest has joined #openstack-infra15:58
*** e0ne has quit IRC15:58
*** fernnest_ has joined #openstack-infra15:58
jeblairmordred: is there a reason why the remote adhoc playbook runs puppet on puppetmaster?15:59
*** sarob has joined #openstack-infra16:00
mordredjeblair: looking16:00
*** matt-borland has quit IRC16:00
mordredjeblair: you're running it with limit right?16:01
jeblairmordred: well, that's what's in the playbook16:01
mordredjeblair: no, I mean - it's designed so that you pass -l or --limit on the command line to ansible-playbook16:02
jeblairor is it launch-node that's doing that...16:02
*** derekh has quit IRC16:02
mordredjeblair: so that you can specify which host(s) you want it to run on16:02
*** matrohon has quit IRC16:02
mordredlaunch-node should be passing -l16:02
jeblairoh16:03
jeblairit's kick.sh that does that16:03
*** tsg_ has joined #openstack-infra16:03
jeblairso launch-node does not, but kick.sh does16:03
mordredoh - includes the puppet on puppetmaster - got it16:03
*** mtanino has joined #openstack-infra16:03
*** thorst_ has quit IRC16:03
jeblairi wonder why we thought that was necessary?16:03
*** kjorgensen has joined #openstack-infra16:04
jeblairi guess because we wanted it to behave like a production pulse16:04
mordredyah16:04
*** tonytan4ever has quit IRC16:04
*** pilgrimstack has quit IRC16:04
jeblairokay, i think i can disregard that for now16:04
jeblairso the *actual* puppet configuration we are running with is:16:06
jeblairbasemodulepath=$confdir/modules:/opt/system-config/production/modules16:06
jeblairenvironmentpath=$confdir/environments16:06
*** tsg__ has quit IRC16:06
*** roxanaghe has joined #openstack-infra16:06
jeblairi believe the problem i'm encountering is that ansible-puppet writes the environment configuration with that basemodulepath16:08
jeblairso all of the environments end up with the same module path16:08
jeblairhere, let me etherpad16:08
jeblairhttps://etherpad.openstack.org/p/QB2wi5DGNm16:08
*** Benj_ has joined #openstack-infra16:08
mordredjeblair: yes. I follow16:08
fungioh, right, before we relied on puppetmaster configuration to point a particular environment at a particular module load path16:09
*** sarob has quit IRC16:09
fungibut there's no puppetmaster process interpreting that any longer16:09
*** sarob has joined #openstack-infra16:09
*** bethwhite_ has quit IRC16:10
fungii suppose one way would be to copy all the environments and all the module trees and all the module configs to every server and let local puppet on the target sort out which to use based on the environment you request16:10
jeblairmordred: i would like the 'corvus' environment.conf to have 'system-config/corvus/modules' in its modulepath instead of 'production'16:11
*** thorst_ has joined #openstack-infra16:11
*** gyee has joined #openstack-infra16:11
*** EricGonc_ has quit IRC16:11
*** thorst_ has quit IRC16:11
jeblairmordred: (and i'm hoping that the environment.conf there will override the puppet.conf value)16:11
jeblairso i think i can pass the base modulepath in as an ansible cmd line var, yeah?16:12
jeblairthat will change the value in corvus/environment.conf, but it will also change it for puppet.conf16:12
*** Benj_ has quit IRC16:13
*** mordred has quit IRC16:13
*** Shrews has quit IRC16:13
jeblairdoes it really make sense for ansible-puppet to manage environment.conf files?16:14
*** phschwartz has quit IRC16:14
*** Benj_ has joined #openstack-infra16:14
clarkbor maybe modify the default module path to include the env?16:14
jeblairbecause it seems like if ansible-puppet is writing its own puppet.conf files every time, it almost eliminates the need for an environment.conf16:14
clarkblooks like its set in ansible-role-puppet/defaults/main.yml16:15
jeblair(ie, ansible-puppet *creates* its own environment each time it runs)16:15
*** hieulq_ has joined #openstack-infra16:15
*** Shrews has joined #openstack-infra16:15
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167916:15
clarkbjeblair: that should work too, just have to be careful to clean out the old env which asselin__ is also working on16:15
fungiright, so it's possible we could just have ansible decide what to copy based on the environment you specify, and let puppet itself stop caring about multiple environments16:15
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367416:16
*** phschwartz has joined #openstack-infra16:16
*** thorst_ has joined #openstack-infra16:16
* jeblair looks at mordred's change16:16
*** Apoorva has joined #openstack-infra16:16
*** mordred has joined #openstack-infra16:17
fungii sort of feel like there's not a lot of middle ground. either we push all the environments and the environment selection logic to the target, or we keep the selection logic at the control end and just substitute what's requested for the remote end's default environment16:17
mordredI'm back!16:17
*** jascott1 has joined #openstack-infra16:18
jeblairmordred: i think your change will work -- the only think i still wonder about is that will also change the puppet.conf each time it runs (to match environment.conf), and i wonder why we bother to have environment.conf in that case16:18
pabelangerrcarrillocruz: I'm seeing: PortLimitExceeded: Maximum number of ports exceeded from infracloud16:18
pabelangerchecking what our quota is now16:19
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167916:21
pabelangerrcarrillocruz: looks like our quota for ports is only 5016:21
mordredjeblair: oh - bother. yesI think we want a different thing in environment.conf than in puppet.conf ... on e sec16:21
fungii have to step out for a few to run some errands... bbiab16:22
*** asettle has quit IRC16:22
*** aorourke has quit IRC16:22
mordredjeblair: I wrote all of the irc statement before writing and pushing the patch, fwiw16:23
*** asettle has joined #openstack-infra16:23
*** aorourke has joined #openstack-infra16:23
pabelangerrcarrillocruz: I've bumped to 100 for now16:23
jeblairmordred: oh, okay, so we're all caught up now? :)16:23
rcarrillocruz++ , thx16:23
mordredmaybe? who even knows :)16:23
clarkbpabelanger: iirc you need at least yhe same number of ports as instamces and if you have a router one or two more16:24
pabelangergoing to see if we can manage that in cloud-launcher16:24
*** dprince has quit IRC16:24
pabelangerclarkb: Ya, we should sit down next week and map out what our quotas will look like for infracloud-vanilla, most are just using defaults right now16:24
mgagnepabelanger: do you know why infra-cloud has a 240s boot-timeout instead of 120s?16:24
pabelangerhttp://paste.openstack.org/show/580142/16:25
pabelangermgagne: not sure, rcarrillocruz?16:25
mordredclarkb: yes, that's correct16:25
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Revert "Create overcloud images for liberty using EXT4"  https://review.openstack.org/36895316:25
mordredeach server will need a port for its fixed ip16:25
mgagnepabelanger: we are investing some time to solve the thundering herd problem when new images are uploaded and there is more questions than solutions atm16:26
rcarrillocruzDunno, I do not think we ever tweaked that16:26
pabelangermgagne: great, thanks for the update.16:26
*** esberglu has quit IRC16:27
*** asettle has quit IRC16:27
mordredmgagne: this is nova slamming glance to pull images into cache?16:28
mgagnemordred: yea, into local cache on compute node16:28
jeblairmordred: oh you were in a netsplit on an airplane?16:28
jeblairthat's like the worst16:28
mordredjeblair: yah16:29
jeblairmordred: did the oxygen masks drop? :)16:29
mordredjeblair: only for some of the people on the plane16:29
mordreddepends on which side of the sp[lit you were on16:29
jeblairnice16:29
*** stajkowski1 has joined #openstack-infra16:30
stajkowski1hey all, possible to create a group for our use?  need to create osic-neutron group and make me admin if possible of that group16:30
jeblairmordred: i'm going to try your change now16:31
*** hieulq_ has quit IRC16:31
*** sambetts is now known as sambetts|afk16:31
*** Swami has joined #openstack-infra16:31
mgagnemordred: let me know if you have any suggestion or reading to do about the problem. I'm not even sure how the community or glance team suggests addressing the problem.16:32
*** hieulq_ has joined #openstack-infra16:32
pabelangermordred: rcarrillocruz: so, I think we have some leaked ports in infracloud-vanilla. We have 18 servers online, but 23 ports assigned16:32
pabelangeris there an easy way to clean that up?16:32
mordredmgagne: I was thinking cloudnull might have thoughts ...16:32
mgagnemordred: precaching "golden image" of the provider is one thing, caching customer images is not something we can systematically do for obvious reasons16:32
mordredyah16:33
*** jpich has quit IRC16:33
*** wznoinsk has quit IRC16:33
*** piet has quit IRC16:34
*** ddieterly is now known as ddieterly[away]16:34
cloudnullmordred: /me reading back16:35
*** piet has joined #openstack-infra16:35
clarkbpabelanger: something like for port in ports if not port.instance then delete port16:36
clarkbpabelanger: just be careful you dont delete the dhcp port which shpuld be annotated16:36
clarkbwe had a script for this somehwere since hpcloud leaked them too16:36
clarkbbut its been so long since we used hpcloud16:36
pabelangerokay, so something we might need to handle better in shade16:37
pabelangerI'll hold off on running it16:37
cloudnullmordred mgagne: i have no good thoughts.16:38
mgagnethanks for the brain time =)16:38
mordredugh. I really don't want to have to handle that in shade. :(16:38
mordredI mean, we obviously can if we need to - but GROSS16:39
mgagnewe were thinking about bypassing Glance which actually uses Swift to store the data. like bypass the middleman and download from Swift directly and invest resources to make Swift super performant16:39
*** wznoinsk has joined #openstack-infra16:40
*** artmr has quit IRC16:42
pabelangerinfracloud also has the cache problem too, we see scheduling timeouts for launching nodes because compute nodes needs to fetch the image.16:42
jeblairmordred: okay, with your change in place, i have a correct corvus environment.conf, but puppet is still changing puppet.conf back.16:44
*** esikachev has joined #openstack-infra16:44
jeblairi am having a very hard time understanding this.16:44
pabelangerclarkb: mordred: do mind reviewing? https://review.openstack.org/#/c/371628/ would like to start graceful restart on zuul-launchers for afs-docs16:44
*** jaosorior has joined #openstack-infra16:44
mgagnepabelanger: ok, if we come up with the time required to precache the image, could the timeout be adjusted? I'm looking for a short-term solution while we work on the long-term one.16:44
jeblairmordred: i *think* i have all of the relevant copies of system-config updated not to manage puppet.conf16:46
jeblairso i have no idea what's going on16:46
pabelangermgagne: As long as everybody is okay with it, I don't see a reason not too.16:46
jeblairpabelanger: okay with what?16:46
mgagnepabelanger: well, so far, there is no capacity available for ~1.5-2h while nodepool hammers the API and tries to spawn (and delete nodes)16:47
*** lucasagomes is now known as lucas-dinner16:47
pabelangerjeblair: mgagne was looking at increasing the boot-timeout in nodepool for internap-mtl01, to work around (short-term) some caching issues they are having.16:47
jeblairincreasing the timeout to 2h?16:47
mgagnejeblair: does nodepool delete the node if it doesn't spawn in time?16:48
jeblairmgagne: yes.16:48
pabelangerI thought boot-timeout was seconds?16:48
*** abregman is now known as abregman|afk16:48
jeblairpabelanger: mgagne just said 'no capacity for 2h'16:48
jeblair(i'm paraphrasing :)16:49
mgagnejeblair: I'm trying to see if the problem is amplified (or not) by having nodepool spawning and deleting at a crazy rate16:49
*** andymaier has quit IRC16:49
pabelangerJust seen that16:49
mgagneso according to graph, there is a bunch of nodes in building state for 1h before they are deleted. I'm not sure why there is no deleting during that period if timeout is 2m16:51
jeblairclarkb, fungi, mordred: are any of you around?16:51
mordredjeblair: what host are you on? I can try to shell in and look to verify16:51
jeblairmordred: i'm on puppetmaster and the not-yet-in-dns files0116:52
pabelangermgagne: That is our launch-timeout, which defaults to 1hr.16:52
mgagneboot-timeout is time to ssh ?16:53
*** ihrachys has quit IRC16:53
mgagneafter launch phase is completed?16:53
dimsjeblair : fungi : mordred : if i need to stop processing releases... please let me know (not fully reading backlog here)16:53
*** tqtran has joined #openstack-infra16:53
jeblairdims: why is something wrong?16:54
pabelangermgagne: yes16:54
*** cardeois has quit IRC16:54
*** hieulq_ has quit IRC16:54
dimsjeblair : i don't see any issues yet. just cautious16:54
mgagneok so we already have 1h and it's not enough. now I'm sad =(16:54
*** bexelbie has quit IRC16:55
jeblairdims: ok.  i'm not aware of any user-visible issues.  :)16:55
*** cardeois has joined #openstack-infra16:55
dimsjeblair : awesome :)16:55
jeblairmordred: oh, erm, i don't see an '--environment=corvus' argument on the run puppet task16:56
*** bexelbie has joined #openstack-infra16:57
*** bexelbie has quit IRC16:57
mordredjeblair: hrm16:58
*** ddieterly[away] is now known as ddieterly16:58
*** amitgandhinz has quit IRC16:59
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171016:59
mordredjeblair: I agree with you16:59
jeblairmordred: ++16:59
mordredwait16:59
mordredjeblair: one sec16:59
jeblairmordred: yeah, 'puppet_environment' in the 2nd half of that, right?16:59
mordredthe varialbe is puppet_environment - not environemnt16:59
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171016:59
mordredyah16:59
*** itsuugo has quit IRC17:00
*** amitgandhinz has joined #openstack-infra17:00
*** itsuugo has joined #openstack-infra17:00
jeblairmordred: okay, i think all of your changes, plus the ones i haven't pushed up yet, make this work17:01
*** bexelbie has joined #openstack-infra17:02
*** annegentle has joined #openstack-infra17:03
*** rajinir has joined #openstack-infra17:04
pabelangeris the only way for nova to download the new image from glance by creating a new server?17:04
*** bexelbie has quit IRC17:04
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171417:04
*** tonytan4ever has joined #openstack-infra17:04
jeblairmordred: ^17:04
clarkb pabelanger approves17:05
jeblairwhat's the easiest way for me to test that a dib build still works with that ^17:05
clarkber approved17:05
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config: Add new project os-log-merger  https://review.openstack.org/37141117:05
jeblairclarkb approves pabelanger17:05
pabelangeryay17:05
clarkbjeblair: you can use project-config/tools/build-image.sh and set the vars for system config location and ref17:06
*** tsg_ has quit IRC17:06
* mordred also approves pabelanger17:06
clarkbjeblair: be warned you need a fiar bit of disk and patience17:06
*** degorenko is now known as _degorenko|afk17:06
clarkbmordred: pabelanger can has review on https://review.openstack.org/#/c/369601/ so that when we get glean release its quieter with neutron running?17:07
*** jcoufal_ has joined #openstack-infra17:08
jeblairclarkb: do we have a job that builds a test image?17:08
clarkbjeblair: we do but not one that runs our puppet. nodepools integration tests make a really simple dibimage17:09
*** bexelbie has joined #openstack-infra17:10
jeblairseems like something that runs our elements in project-config would be useful17:10
*** tonytan4ever has quit IRC17:10
jeblair(i thought we had talked about doing that)17:10
pabelangerclarkb: +2, I did not test17:10
*** ddieterly is now known as ddieterly[away]17:10
*** ddieterly[away] is now known as ddieterly17:10
AJaegerproject-config cores, could you review translation sync for newton, please? https://review.openstack.org/37165817:10
jeblairclarkb: okay, what i'm actually going to do then is run launch-node without the ansible bits, and see if install_puppet.sh followed by puppet apply is sufficient.17:11
jeblairand use that as a proxy for the dib case17:11
openstackgerritMerged openstack-infra/puppet-zuul: Add keytab support for sites  https://review.openstack.org/37162817:11
*** jcoufal__ has quit IRC17:11
clarkbjeblair: ok17:11
*** thorst_ has quit IRC17:12
AJaegerstajkowski1: what do you mean with group? A gerrit one?17:12
*** tosky has quit IRC17:13
*** tphummel has quit IRC17:16
*** bin_ has quit IRC17:17
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Ignore test data in web/share/elastic-recheck  https://review.openstack.org/37172317:17
*** tsg_ has joined #openstack-infra17:17
mordredjeblair: ++17:18
*** dahoo has joined #openstack-infra17:18
*** sdake_ has quit IRC17:18
*** tphummel has joined #openstack-infra17:19
*** jordanP has quit IRC17:20
*** rhallisey has quit IRC17:21
*** rhallisey has joined #openstack-infra17:23
*** amoralej is now known as amoralej|off17:24
*** oanson has joined #openstack-infra17:25
*** sarob has quit IRC17:25
*** rvasilets___ has joined #openstack-infra17:25
*** e0ne has joined #openstack-infra17:26
jeblairmordred, clarkb: 'install_puppet && install_modules && puppet apply' works17:26
jeblairmy change to remove all of that from puppet failed the apply test because it uses the puppet copy of hiera.yaml instead of the ansible one (which is the one we have actually been using in prod)17:27
jeblairi'm thinking of changing the apply test to use ansible to run the puppet role locally17:28
nibalizerjeblair: i like that17:28
*** tonytan4ever has joined #openstack-infra17:29
fungijeblair: i'm back now and available to continue17:29
fungicatching up17:29
jeblairfungi, nibalizer, mordred, clarkb: "status:open topic:pansible"17:29
jeblairi think that about summarizes the current state :)17:30
nibalizerhaha17:31
*** kzaitsev_mb has quit IRC17:31
mordredjeblair: ++17:31
*** esberglu has joined #openstack-infra17:33
*** ddieterly is now known as ddieterly[away]17:33
*** ddieterly[away] is now known as ddieterly17:33
*** ddieterly is now known as ddieterly[away]17:34
fungihah, pansible. nice17:36
mgagnepabelanger: https://blueprints.launchpad.net/nova/+spec/nova-image-cache-management-2 not much traction. never got implemented afaik17:37
mgagnepabelanger: cern has a custom script for that: https://github.com/dberzano/cern-alice-setup/blob/master/hltcloud/openstack-prefill-cache17:37
*** annegentle has quit IRC17:39
*** rbrndt has quit IRC17:40
fungimordred: on 333459 did you have a pending example per asselin__'s request?17:41
*** antipsychiatry has joined #openstack-infra17:41
pabelangermgagne: interesting17:41
mgagnepabelanger: again, it's for golden images, not for customer's images17:41
*** antipsychiatry has quit IRC17:42
mgagneI'm sure there is plenty of examples on how as a golden image provider you can speed up pre-caching17:42
fungimordred: i'm also tempted to make 371679 and its child not depend on 333459 so the documentation can be refined separately17:42
pabelangerya17:42
asselin__mordred, fungi I can try to update 333459 to include some docs now that I have stuff working.17:42
fungiasselin__: oh, would you mind? that would be awesome17:42
*** spzala has joined #openstack-infra17:43
mordredthat would be great!17:43
mordredfungi: I can reparent real quiick17:43
asselin__of I can remove my -1 and do that in follow-up17:43
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171017:43
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167917:43
*** jamesdenton has joined #openstack-infra17:44
asselin__s/of/or/17:44
mordredthere ya go - they are now reparented17:44
mordredasselin__: follow up patch or edit that one - either is fine with me, whichever you prefer - thanks!17:44
mgagnecern made a presentation about it in Paris: https://www.openstack.org/summit/openstack-paris-summit-2014/session-videos/presentation/cold-start-booting-of-1000-vms-under-10-minutes17:44
openstackgerritMonty Taylor proposed openstack-infra/zuul: Add async action plugin to override upstream async  https://review.openstack.org/37173317:46
openstackgerritMonty Taylor proposed openstack-infra/zuul: Split playbook into vars, pre-playbook and playbook  https://review.openstack.org/37173417:46
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Temporary use master for requirements in translation job  https://review.openstack.org/37173517:47
AJaegerdims, here's my workaround ^17:47
AJaegerinfra-root, we do not have a stable/newton branch and translation jobs fail - what do you think of my temporary fix ? ^17:47
dimsAJaeger : sweet17:47
*** trown is now known as trown|lunch17:48
openstackgerritMichal Jastrzebski (inc0) proposed openstack-infra/project-config: Re-enable cloud8  https://review.openstack.org/37173617:48
mordredAJaeger: lgtm17:48
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Temporary use master for requirements in translation job  https://review.openstack.org/37173517:49
* AJaeger just updated commit message in case anybody reviewed...17:49
AJaegerthanks17:49
mordredAJaeger, dims: it might be worth chatting in barcelona about the places where things fail in the period before we've cut the stable/newton branches everywhere17:49
mordredI think there are a few we've run in to this time17:49
dimsright mordred17:49
*** Na3iL has quit IRC17:49
AJaegerdims, could you take a note of this, please?17:49
mordrediirc, grenade is another one that clarkb was doing something with a day or two ago17:50
fungimordred: i guess you missed jeblair's comments on 371679 when you were updating. they're an easy edit... mind if i do that now?17:50
mordredfungi: oh - please do17:50
fungion it17:50
stajkowski1AJaeger: yes, gerrit group17:50
mordredfungi: thanks! plane wifi is both awesome, and terrible17:50
AJaegerstajkowski1: for which repo?17:51
AJaegermordred: could you give another +2 on 371735 , please?17:51
AJaegerstajkowski1: gerrit groups are associated with repos - and need to be setup in project-config repo, directory gerrit/acls/ for the repo17:52
AJaegerthanks, mordred17:52
clarkbya grenade cant be made to do the newton to master upgrade until we have branches everywhere due to our fallback on master17:53
clarkbaiui it would do master to master where no newton branch is present17:53
dimsAJaeger : added a TODO section at the bottom of https://etherpad.openstack.org/p/newton-relmgt-plan17:53
openstackgerritJeremy Stanley proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171017:53
openstackgerritJeremy Stanley proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167917:53
AJaegerthanks, dims17:53
mordredclarkb: I +A'd the glean change17:54
fungijeblair: your comments on 371679 have been addressed now17:54
clarkbmordred: cool thanks17:54
AJaegerteam, inc0 proposes to reanable cloud8 at https://review.openstack.org/#/c/371736/ . Are we ready to take that?17:55
openstackgerritMerged openstack-infra/glean: Ignore non permanenet addresses in udev too  https://review.openstack.org/36960117:57
clarkbAJaeger: we should probably run a reproduce.sh on an instance there and check the time to run is reasonable17:57
*** yamamoto has quit IRC17:57
clarkbthen reenable it17:57
clarkbI can do that here in a few minutes17:58
*** adrian_otto has joined #openstack-infra17:58
AJaegerclarkb: thanks. Btw. I gave a -1 on your horizon xenial change. Do you want to update it yourself or shall I do it quickly for you?17:58
inc0AJaeger, clarkb we did some tests, seems ok now17:59
inc0also reason for long times was pretty obvious when I looked at configs17:59
*** adrian_otto has quit IRC17:59
njohnstonHi!  I have a quick question about how gate jobs work.  I am looking at a job - gate-neutron-fwaas-dsvm-tempest - and I assume somewhere there is a git repo that has the configs specific to that job, like the local.conf that is used to set up the devstack so it has everything configured correctly.  Where would I find those configs?17:59
inc0but by all means, validate17:59
AJaegernjohnston: check the log files, they should have everything you need. The reproduce.sh script contains everything to reproduce it.18:00
njohnstonAJaeger: Thanks very much!18:00
AJaegernjohnston: job setup is done in general in project-config repo - directory jenkins/jobs - and uses devstack-gate for dsvm jobs.18:01
*** akshai_ has quit IRC18:01
*** annegentle has joined #openstack-infra18:02
jeblairmordred: playbooks/remote_puppet_adhoc.yaml:    puppet_environment: '{{ puppet_environment | default(omit) }}'18:03
jeblairmordred: is it possible that can produce a recursion error?18:03
*** adrian_otto has joined #openstack-infra18:04
jeblairmordred: ansible seems happier when those are different var names18:05
*** jcoufal_ has quit IRC18:05
*** jcoufal_ has joined #openstack-infra18:06
*** sdague has quit IRC18:06
mordredjeblair: it _shouldn't_ ... lemme look a sec though18:06
mordredjeblair: that said - it would be easier to provide on the command line if we made it just environment18:07
mordredjeblair: so puppet_environment: '{{ environment | default(omit) }}' ... which would then just be -e environment=corvus18:07
jeblairyep18:07
mordredchange coming18:07
*** piet has quit IRC18:08
fungioh, weird, self-referential definitions in ansible don't short-circuit resolve?18:08
* fungi will keep this in mind18:09
mordredthey should not be self-referential18:09
mordredlike, this should not be a problem - but I'm about tpo land, so debugging fully == hard18:09
fungiahh, i had read the original as only set puppet_environment to default(omit) if bool(puppet_environment) is false18:09
*** rbrndt has joined #openstack-infra18:09
openstackgerritMonty Taylor proposed openstack-infra/system-config: Use environment as the parameter for ad_hoc playbook  https://review.openstack.org/37174518:10
clarkbAJaeger: thank you if you don't mind updating it that would be great asI am juggling kids currently18:10
jeblairfatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/opt/system-config/__omit_place_holder__54105d0ebbfe2e6565252f2ff98abbd6a6cfb825\" \"/opt/system-config\"", "failed": true, "msg": "rsync: link_stat \"/opt/system-config/__omit_place_holder__54105d0ebbfe2e6565252f2ff98abbd6a6cfb825\" ...18:10
jeblair... failed: No such file or directory (2)\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.0]\n", "rc": 23}18:10
jeblairthat's where i'm at now if i try to run ansible for the apply test18:10
mordredjeblair: neat18:10
mordredwhat is __omit_place_holder__54105d0ebbfe2e6565252f2ff98abbd6a6cfb82518:10
jeblairno idea18:10
clarkbI wonder if its a special object it carries through for omit so that it knows if something further down the chain should override?18:11
* mordred landing - back in a few18:11
*** EricGonczer_ has joined #openstack-infra18:11
clarkbinc0: we mostly want to make sure that our specific use case isn't hampered since its rc time and all that18:12
*** harlowja has joined #openstack-infra18:13
inc0clarkb, that's fair18:13
rcarrillocruzpabelanger: there's been a merge recently for setting quotas on the ansible openstack modules18:14
rcarrillocruzi'll put that into the launcher18:14
rcarrillocruzso we can have the quotas defined on the layouts yaml18:14
AJaegerclarkb: will do...18:14
*** nt has quit IRC18:15
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174918:16
clarkbinc0: is our image created at 2016-09-15T19:22:57Z new enough that it is expected to work?18:17
clarkbinc0: 6a7677a1-75c6-4035-92a9-a9d911b6f7a7 is the uuid18:17
*** adrian_otto has quit IRC18:17
inc0no18:17
*** dtardivel has quit IRC18:17
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Run horizon jobs on xenial  https://review.openstack.org/37120218:17
inc0we purged images18:17
AJaegerclarkb: ^18:17
inc0as we redeployed ceph alltogether18:17
inc0also, we could do raws to make sure ceph actually is useful;)18:17
*** adrian_otto has joined #openstack-infra18:18
clarkbinc0: that is a raw from 22 hours ago18:19
clarkbinc0: and it shows up on image show18:19
inc0ahh18:19
inc0then it's ok18:19
clarkbok18:19
inc0we didn't do anything in last 2 or 3 days18:19
inc0yeah, dates are hard:( sorry18:19
*** jcoufal_ has quit IRC18:20
clarkbcool I have booted off of that and will run a reproduce script to see what performance is like for our jobs18:20
inc0btw clarkb if you have a way to manually start a gatejob I could use it;)18:20
inc0let me know18:20
*** jcoufal_ has joined #openstack-infra18:20
*** kvcobb has quit IRC18:21
clarkbinc0: devstack-gate writes out a reproduce.sh script for every job it runs you just wget and bash it18:21
inc0cool, good to know18:21
inc0where can I find it?18:21
*** kvcobb has joined #openstack-infra18:21
AJaegerinc0: just look at a random change and pick a job that is interesting. Let me give you an example...18:22
fungiinc0: along with the job logs from any dsvm job which has reported back on a change18:22
*** adrian_otto has quit IRC18:23
clarkbI am running http://logs.openstack.org/96/370696/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/547b9af/logs/ on a host I booted in cloud818:23
AJaegerinc0: looking at https://review.openstack.org/#/c/360269/ , take one of the dsvm jobs like gate-tempest-dsvm-heat-apache, click on it and get the log files at http://logs.openstack.org/69/360269/3/check/gate-tempest-dsvm-heat-apache/a4326ca/logs/18:23
AJaegerinc0, and that contains a reproduce.sh - download it18:23
inc0btw guys sorry for the fuzz, since we created dedicated cloud for you it's hard to keep quality high - we build our alarming and stuff now18:23
*** annegentle has quit IRC18:23
*** adrian_otto has joined #openstack-infra18:23
*** spzala has quit IRC18:24
inc0got it, thanks!18:24
clarkbcan I get a second review on https://review.openstack.org/#/c/371202/ ? that will get horizon all xenialed. They were actually running quite a few tests there already due to nodejs18:27
stajkowski1AJaeger: ah yes, i remember, ty18:27
*** esikachev has quit IRC18:27
*** chlong_ has quit IRC18:27
openstackgerritNate Johnston proposed openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894518:30
*** e0ne has quit IRC18:30
*** vhosakot has joined #openstack-infra18:30
fungijeblair: so your efforts on puppet apply job adjustments are in service of getting 371714 passing i guess?18:30
jeblairfungi: yes18:30
*** ddieterly[away] is now known as ddieterly18:30
*** esberglu has quit IRC18:31
fungii need to step away again in a bit but can also help try to work out details of that when i get back in an hour-ish18:31
jeblairfungi: i'm about to push up a chaneg18:31
fungioh, excellent18:31
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Use sequence znodes for image upload numbers  https://review.openstack.org/37104518:32
fungiclarkb: AJaeger: since horizon already has their rc1 cut, i guess 371202 can't be _too_ disruptive now?18:32
clarkbfungi: no and things were tested and working fine and they already mostly tested on xenial anyways18:32
fungiperfect. anyway i already approved it, just double-checking18:33
AJaegerfungi, note it changes the jobs running on stable/newton as well...18:33
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove puppet_environment from the adhoc playbook  https://review.openstack.org/37175918:33
jeblair(still working on changes)18:33
*** spzala has joined #openstack-infra18:33
AJaegerfungi, clarkb, could you help me with landing translation update for newton, please? https://review.openstack.org/#/c/371735/ and https://review.openstack.org/#/c/371658/ ? Other project-config cores are welcome to review as well ;)18:34
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171418:35
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove puppet_environment from the adhoc playbook  https://review.openstack.org/37175918:35
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176218:35
jeblairfungi, clarkb, mordred: ^ that seems to work in local testing, but it will be interesting to see what happens in the actual apply tests18:36
*** yamahata has joined #openstack-infra18:37
*** spzala has quit IRC18:37
jeblairoh, that may need a depends-on: I6be88234eab32844f395df7658952b1cef944bd818:38
jeblairit will be interesting to see if it fails, and if it does, if adding that fixes it :)18:38
clarkbAJaeger: trying to understand https://review.openstack.org/#/c/371735/2 can't we just not running the newton translation jobs there?18:38
fungijeblair: very cool, i'll check in on it and see what the job results are looking like when i get back18:39
jeblairthey will look terrible :(18:39
clarkboh wait I get it now18:39
clarkbitsbecause of constraints18:39
jeblairbecause they will be in unreadable ansible stdout json18:39
fungigrar18:39
jeblairi don't know how to deal with that :(18:39
*** nt has joined #openstack-infra18:39
jeblairfungi: oh!18:40
jeblairpuppet has a --logdest file18:40
jeblairwe can use that18:40
AJaegerclarkb: yeah, constraints is the problem.18:40
AJaegerclarkb: I think we have zuul-cloner now on the proposal node (without cache) and thus could change the setup as well - but it's too late for me today to double check everything there...18:41
clarkbAJaeger: there are more upper-constriants macros do we want ot update all of them now?18:42
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176418:43
*** bnemec is now known as beekneemech18:45
fungijeblair: oh, so we can break out the puppet stream separately that way? i guess that's also better than the syslog streaming?18:45
*** oanson has quit IRC18:45
jeblairfungi: i still think syslog is the better solution for launch-node18:45
jeblairfungi: (that way it remains on the host)18:45
fungifair enough, we get more detail than just puppet that way18:46
jeblairfungi: but i think a temp file is good for the apply test.18:46
openstackgerritMerged openstack-infra/project-config: Run horizon jobs on xenial  https://review.openstack.org/37120218:46
AJaegerclarkb: zuul-cloner falls back to master if the branch does not exist18:46
fungijeblair: yep, that too18:46
AJaegerclarkb: so, no need to udpate those18:46
jeblairfungi: i missed something very amusing in the apply test.18:46
*** spzala has joined #openstack-infra18:46
jeblairi forgot 'noop'.  :)18:46
fungihah18:46
*** vhosakot has quit IRC18:46
openstackgerritNate Johnston proposed openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894518:46
fungiokay, stepping away, back soon18:46
clarkboh wow these use ggp18:46
clarkbjeblair: fungi it might be nice to have a tee to syslog and console18:47
jeblairclarkb: for what?18:47
clarkbjeblair: for eg launch node. Then it stays on host and you don't have to ru na separate thread to pull the data out via tail18:48
*** dimtruck is now known as zz_dimtruck18:48
jeblairclarkb: that's not possible18:48
*** abregman|afk is now known as abregman18:48
clarkbwould probably be too noisy for our regularly cron scheduled playbooks though18:48
*** esberglu has joined #openstack-infra18:48
jeblairclarkb: the interface to the host we are launching is through ansible18:48
clarkbansible captures stdout though right?18:49
jeblairclarkb: yes18:49
jeblairclarkb: and launch-node was outputting the stdout18:49
jeblairclarkb: at the end of the run18:49
clarkbso if we could write to syslog and stdout wouldn't that owrk?18:49
jeblairbecause that's what ansible does18:49
clarkboh right it wouldn't be able todo it live18:49
clarkbbecause ansible buffers the whole thing18:49
jeblairyes, that is the problem18:49
AJaegerclarkb: if you think we should remove those builders and use zuul-git-branch-prep and zuul-git-prep, we can evaluate that as well...18:49
openstackgerritNate Johnston proposed openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894518:49
clarkbAJaeger: I think we should but we don't need to do thatright now18:50
jeblairdoes anyone know what a 'check_mode' is for an ansible module and how to use it?18:50
AJaegerclarkb: I'll take a note and will patch soon.18:50
jeblairbecause that's apparently how we tell the puppet role to noop18:50
clarkbjeblair: I do not18:51
jeblairit's looking like we may need a new playbook for this18:51
*** EricGonczer_ has quit IRC18:51
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove horizon coverage jobs  https://review.openstack.org/37064118:51
*** EricGonczer_ has joined #openstack-infra18:51
*** pilgrimstack has joined #openstack-infra18:52
openstackgerritNate Johnston proposed openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894518:52
*** vhosakot has joined #openstack-infra18:52
*** adrian_otto has quit IRC18:52
*** hasharAway is now known as hashar18:53
*** adrian_otto has joined #openstack-infra18:55
*** adrian_otto has quit IRC18:55
*** ociuhandu has joined #openstack-infra18:55
openstackgerritJames E. Blair proposed openstack-infra/system-config: apply test: log ansible-puppet to a file  https://review.openstack.org/37176618:55
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174918:56
*** adrian_otto has joined #openstack-infra18:57
inc0clarkb, did gate finished?18:58
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Client change to support ZK image build requests  https://review.openstack.org/37176718:58
*** adrian_otto has quit IRC18:58
*** yamamoto has joined #openstack-infra18:58
*** thiagolib has quit IRC18:58
clarkbinc0: no its still running18:58
openstackgerritJames E. Blair proposed openstack-infra/system-config: apply test: log ansible-puppet to a file  https://review.openstack.org/37176618:58
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171418:58
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176218:58
inc0duh, well, it's still within norm18:58
clarkbinc0: will have to look at the times reported by tempes tand devstack logs18:59
clarkbbut it doesn't appear to be really slow at least18:59
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174918:59
inc0ok, so you just boot your vm and run reproduce.sh inside?19:00
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Client change to support ZK image build requests  https://review.openstack.org/37176719:00
clarkbyup as jenkins user19:00
clarkbthis doesn't use the mirrors though which is fine19:00
clarkbif you want ot use the mirrors you hvae to run the setup mirrors script from nodepool's scripts first19:00
inc0well, I don't want to test this part;)19:01
inc0ok, I'll try to run it myself and observe the process19:01
inc0try to debug why it's slow19:01
*** trown|lunch is now known as trown19:02
clarkbinc0: like I said it doesn't seem too slow. But will have to wait for it to finish to compare numbers (since all logs will be available at that point)19:02
*** pilgrimstack has quit IRC19:02
openstackgerritJames E. Blair proposed openstack-infra/system-config: apply test: log ansible-puppet to a file  https://review.openstack.org/37176619:02
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171419:02
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176219:02
inc0ahh I misread it, sry19:03
openstackgerritMerged openstack-infra/project-config: Temporary use master for requirements in translation job  https://review.openstack.org/37173519:03
inc0Friday is creeping on me19:03
*** rkukura has quit IRC19:04
*** EricGonc_ has joined #openstack-infra19:04
*** zz_dimtruck is now known as dimtruck19:05
*** jaosorior has quit IRC19:05
openstackgerritNate Johnston proposed openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894519:05
*** yamamoto has quit IRC19:05
*** cardeois has quit IRC19:05
jeblairclarkb, fungi, mordred, pabelanger: i have re-enabled puppet globally19:06
pabelangerneat19:06
*** EricGonczer_ has quit IRC19:06
*** rtheis_ has joined #openstack-infra19:07
openstackgerritJames E. Blair proposed openstack-infra/system-config: files.o.o: remove rewriteengine  https://review.openstack.org/37176819:07
AJaegerclarkb: do we need this change : https://review.openstack.org/365650 ?19:08
jeblairpabelanger: ^ that's safe to aprv for afs-docs19:08
pabelangerjeblair: +319:08
*** rtheis has quit IRC19:09
*** vhosakot has quit IRC19:10
clarkbAJaeger: no I think we droped the .4 and .5 entirely so it works on any pyhton 319:10
*** vhosakot has joined #openstack-infra19:10
jeblairhrm.  i don't think check_mode is what we want19:12
jeblairthere's too much other stuff it doesn't do19:12
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Short-circuit builder processing on shutdown  https://review.openstack.org/37177019:12
AJaegerclarkb: yeah, the merged requirements change uses /usr/bin/python3 - shall I update?19:12
clarkbAJaeger: please19:12
pabelangerjeblair: AJaeger: I have to run out for some errands, I can graceful restart zuul-launchers when I'm back in an hour or so.19:13
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Rework translation job git setup  https://review.openstack.org/37177219:14
AJaegerclarkb: this is the zuul-cloner usage ^ - needs carefull review about assumptions ;)19:14
clarkbinc0: Ran: 1520 tests in 1746.0000 sec. is cloud8 Ran: 1519 tests in 3829.0000 sec is where we ran the original so that looks great19:14
*** ddieterly is now known as ddieterly[away]19:14
inc0yeah quite a difference19:15
*** gouthamr has quit IRC19:15
clarkbI think thats related to cpu counts but still good19:16
*** cardeois has joined #openstack-infra19:16
*** r-mibu has quit IRC19:16
clarkbthe individual test speeds are pretty similar19:16
*** r-mibu has joined #openstack-infra19:17
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: requirement-constraints: Update for python3.5  https://review.openstack.org/36565019:17
AJaegerclarkb, updated ^19:17
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177419:17
clarkbinc0: and the devstack runtime looks good as well with just longer apt-get and pip due to not using mirror I think19:18
inc0clarkb, ceph was badly misconfigured before19:18
*** cardeois has quit IRC19:18
clarkband wall time was 50minutes so ya I think we can go ahead and reenable19:18
clarkbinc0: where is the change to do that?19:18
inc0clarkb, we already fixed it locally19:18
*** cardeois has joined #openstack-infra19:18
clarkbright I mean for nodepool sounded like there was one?19:18
inc0ahh, hold on19:19
AJaegerclarkb: https://review.openstack.org/37173619:19
AJaegerinc0: ^19:19
inc0thanks AJaeger19:19
*** ddieterly[away] is now known as ddieterly19:20
*** adrian_otto has joined #openstack-infra19:20
*** rkukura has joined #openstack-infra19:20
*** esberglu has quit IRC19:21
*** cardeois has quit IRC19:21
*** cardeois has joined #openstack-infra19:21
openstackgerritMerged openstack-infra/system-config: files.o.o: remove rewriteengine  https://review.openstack.org/37176819:21
clarkbcool that lgtm. I didn't approve because wasn't sure if we needed more consensus to go ahead and reenable it19:21
clarkbfungi: jeblair pabelanger ^ thoughts?19:21
*** ricardoas has quit IRC19:22
*** esberglu has joined #openstack-infra19:23
*** kzaitsev_mb has joined #openstack-infra19:23
*** sarob has joined #openstack-infra19:24
*** sarob has quit IRC19:24
*** sarob has joined #openstack-infra19:24
*** senk__ has quit IRC19:25
*** cloudnull is now known as cloudsaviour19:28
*** roxanaghe has quit IRC19:28
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Rework translation job git setup  https://review.openstack.org/37177219:29
*** david-lyle has quit IRC19:30
*** gouthamr has joined #openstack-infra19:30
*** david-lyle has joined #openstack-infra19:30
*** roxanaghe has joined #openstack-infra19:30
*** cloudsaviour is now known as cloudnull19:31
mtreinishjeblair: I have the traceback from yesterday again, this time with debug mode turned on19:32
mtreinishwhat do you need from it I can pastebin it19:32
*** adrian_otto has quit IRC19:32
jeblairmtreinish: i will have to pick this up after lunch, sorry19:32
mtreinishjeblair: sure, np19:32
*** kzaitsev_mb has quit IRC19:33
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176419:34
*** adrian_otto has joined #openstack-infra19:35
jeblairthere is something *very* strange going on19:35
inc0clarkb, AJaeger https://review.openstack.org/#/c/371736/ +w please;)19:35
jeblairgit-review is not allowing me to submit that change19:35
*** rossella_ has quit IRC19:36
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176419:36
jeblairi just had to *manually* push that ^19:36
openstackgerritClark Boylan proposed openstack-infra/ansible-role-puppet: Simplify puppet cmd generation  https://review.openstack.org/37177819:36
clarkbjeblair: ^ noticed we were duplicating code in one of your changes so pushed a follow up to collapse that19:37
clarkbinc0: yes I was hoping moreinfra roots would chime in, I asked just a few minutes ago19:37
*** dprince has joined #openstack-infra19:37
jeblairevery time i ran git-review it claimed a conflict19:37
inc0ahh sorry, I'm used to 2*+2 = +W19:37
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177419:37
clarkbjeblair: weird, the rebase step maybe in git review causing oddities?19:38
*** esberglu has quit IRC19:38
jeblairclarkb: i lack context to weigh in19:38
jeblairon the re-enable tihng19:38
clarkbjeblair: ok19:38
clarkbjeblair: maybe retry with -R next time to skip the rebase step19:38
jeblairclarkb: yes, it's the rebase step19:38
jeblairclarkb: my thoughts are that if we've tested it, why not put it in prod? :)19:38
*** roxanaghe has quit IRC19:38
clarkbjeblair: the onlyreason is sdague seems to want us to be ultra conservative. I am not sure we need to be and the testing checked out well19:39
*** dprince has quit IRC19:39
*** akshai has joined #openstack-infra19:39
clarkbbut I wasn't really around for when it got disabled so maybe don't have full info19:39
*** adrian_otto has quit IRC19:39
jeblairif we need someone for sdague to be mad at, i will happily be that person.19:40
jeblairperforming a dry run test seems appropriately conservative here.19:40
*** akshai_ has joined #openstack-infra19:40
*** salv-orl_ has joined #openstack-infra19:41
*** esberglu has joined #openstack-infra19:41
inc0clarkb, jeblair I'll be frank, we just totally misconfigured ceph19:41
inc0we found that and fixed that19:41
clarkbinc0: yup understood19:41
* clarkb will approve19:41
jeblairi just did19:41
*** tphummel has quit IRC19:41
*** rtheis_ is now known as rtheis19:41
inc0cool19:41
inc0I'll keep close tabs on gates for next few days19:42
inc0see if anything is out of ordinary19:42
clarkbjeblair: and does https://review.openstack.org/#/c/371778/1 makes sense to you pushed atop your series?19:42
clarkbI need to rebase if so19:42
jeblairclarkb: i think the two stanzas have different spaces19:42
jeblairi don't know why19:42
jeblairbut it looks tricky19:42
*** david-lyle has quit IRC19:43
clarkboh they do, huh19:43
*** salv-orlando has quit IRC19:43
clarkbI should be able to normalize that too19:43
clarkband then the apply job will tell us if it all works?19:43
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177419:44
jeblairclarkb: hopefully? :)19:44
*** akshai has quit IRC19:44
*** esberglu has quit IRC19:45
*** Apoorva has quit IRC19:47
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176219:48
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171419:49
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176219:49
*** fguillot has quit IRC19:49
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171419:50
jeblairthat was *extremely painful*19:50
*** itsuugo has quit IRC19:50
jeblairthanks git19:50
*** itsuugo has joined #openstack-infra19:50
jeblairand it's still not over19:51
jeblairclarkb: https://review.openstack.org/37176419:51
jeblairmaybe that is hitting whatever my local rebase was hitting?19:51
jeblairi have no idea what's going on19:51
jeblairit's a very simple change19:51
clarkbya I think it must be19:51
*** EricGonc_ has quit IRC19:51
*** thorst_ has joined #openstack-infra19:52
*** EricGonczer_ has joined #openstack-infra19:52
* clarkb attempts to reproduce a conflict locally19:52
jeblairis....19:52
jeblairis it possible mordred's change isn't based on master?19:53
*** ddieterly is now known as ddieterly[away]19:53
*** EricGonc_ has joined #openstack-infra19:53
openstackgerritMerged openstack-infra/project-config: Re-enable cloud8  https://review.openstack.org/37173619:54
clarkbjeblair: the conflict I get locally is showdiff and logdest between master and allow any logdest19:54
clarkbwhich is weird because gerrit says its mster the whole way up19:54
*** roxanaghe has joined #openstack-infra19:55
clarkbmsater is ede55a8691b396fe54a5220ef9f2008c864a3aff and parent of mordred change is b5fc21ca376bc903f8eb2373830c0d7c6e256f2b so maybe thats it19:55
jeblairso, i guess it's just that there isn't a merge conflict until my change19:55
jeblairwhat a mess19:56
jeblairbut i know how to fix it now19:56
*** thorst_ has quit IRC19:56
*** EricGon__ has joined #openstack-infra19:56
*** EricGonczer_ has quit IRC19:56
*** esberglu has joined #openstack-infra19:57
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176419:57
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171019:57
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177419:57
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167919:57
*** vhosakot has quit IRC19:57
*** openstackstatus has quit IRC19:58
*** EricGonc_ has quit IRC19:58
jeblair(btw, puppet seems to be running places)19:58
*** sdake has joined #openstack-infra19:59
*** EricGonczer_ has joined #openstack-infra19:59
*** thorst_ has joined #openstack-infra19:59
jeblairinc0: cloud8 is just starting to be used19:59
*** openstackstatus has joined #openstack-infra20:00
*** ChanServ sets mode: +v openstackstatus20:00
openstackgerritClark Boylan proposed openstack-infra/ansible-role-puppet: Simplify puppet cmd generation  https://review.openstack.org/37177820:00
clarkbjeblair: ^ that should deal with the different whitespacing20:00
*** EricGon__ has quit IRC20:01
inc0jeblair, looking:)20:01
jeblairi'm going to get some lunch now20:01
clarkbinc0: out of curiousity what was wrong with ceph? is that something that should make it back into documentation?20:01
inc0journal was way too small20:02
inc0leftover from PoCs in config20:02
*** esberglu has quit IRC20:03
*** esberglu has joined #openstack-infra20:04
inc0clarkb, so at the end of the day we will publish our configs and such as a reference implementation20:04
*** alex_xu has quit IRC20:04
inc0so you guys will be free to use any part of our config in your docs and such20:05
clarkbya most just thinking of like a troubleshooting faq but maybe that belongs more in cephs docs20:05
clarkb"if cpeh is slow check your journal size"20:05
*** esikachev has joined #openstack-infra20:05
inc0there is more to ceph than that ;)20:05
inc0but yeah20:05
*** dimtruck is now known as zz_dimtruck20:06
*** sdague has joined #openstack-infra20:07
*** alex_xu has joined #openstack-infra20:07
*** esberglu has quit IRC20:08
*** kzaitsev_mb has joined #openstack-infra20:11
*** moravec has quit IRC20:14
*** moravec has joined #openstack-infra20:14
*** ddieterly[away] is now known as ddieterly20:15
clarkbhttps://review.openstack.org/#/c/371679 is making my friday brain melt. What is the interaction between the environment module path and the puppte conf base module path and why are modules going in the hieradir?20:16
*** rhedlind has quit IRC20:17
clarkbok looks like basemodulepath is the "base" used by all puppet envs20:18
clarkbI think this means we can remove basemodulepath from the env too and just have it in basemodulepath. Will write comments20:18
*** jcoufal has quit IRC20:18
inc0time to ready in grafana means from nova boot to successful login right?20:18
*** jcoufal_ has quit IRC20:19
*** sarob has quit IRC20:19
*** kzaitsev_mb has quit IRC20:19
*** tphummel has joined #openstack-infra20:19
clarkbinc0: roughly yes20:20
* AJaeger waves good bye and wishes everybody a great weekend - and safe travels to Germany!20:21
clarkbAJaeger: you too see you tuesday20:21
AJaeger(for those that do ;)20:21
AJaegerclarkb: until Tuesday...20:21
*** scottda has quit IRC20:21
*** scottda has joined #openstack-infra20:23
fungiback and seeing where things have gotten to now20:23
*** larainema has quit IRC20:24
rcarrillocruzAJaeger: thx, see ya there20:25
clarkbinc0: the devstack jobs runtime graph on grafana should be a good thing to watch20:25
*** jheroux has joined #openstack-infra20:27
inc0yeah, but raw images did help a lot, ~20s time to ready20:27
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174920:27
clarkbinc0: it would just be cool if we could upload them compressed20:28
clarkbI should dig into glance more to see if that is possible20:28
*** zz_dimtruck is now known as dimtruck20:29
inc0yeah20:31
inc0maybe good idea to write some code there, let me know what you find20:31
fungilooks like the thing with git-review's rebase dry-run claiming there's a conflict was figured out as a stale base/parent revision?20:33
clarkbdevstack supports compresed images but it inflates them before uploading to glance20:33
openstackgerritKen Giusti proposed openstack-infra/system-config: Enable Python3 on Centos 7 and Fedora 22+  https://review.openstack.org/37179720:34
clarkbfungi: ya basically there was a stack of changes with stale parent and it didn't conflict with current master until ~4 changes in20:34
fungihave a great weekend AJaeger! sorry i won't see you next week but hopefully we can catch up soon in barcelona20:34
*** Apoorva has joined #openstack-infra20:34
*** Jeffrey4l has quit IRC20:35
*** Jeffrey4l has joined #openstack-infra20:35
*** kgiusti has left #openstack-infra20:35
clarkbkgiusti: re ^ you actually probably don't want that as it will likely not affect the places where you want python320:35
*** sdake has quit IRC20:37
*** raildo has quit IRC20:38
*** sdague has quit IRC20:39
*** itsuugo has quit IRC20:40
jeblairclarkb, fungi: looking at http://logs.openstack.org/62/371762/5/check/gate-infra-puppet-apply-ubuntu-trusty/fd30da5/console.html i see ec2 metadata errors... any thoughts on what would cause that to be a problem?20:40
*** itsuugo has joined #openstack-infra20:41
jeblair(i'll fix the logdest problem in the mean time)20:41
clarkbjeblair: facter queries ec2 metadata and sometimes gets cranky20:41
clarkbI thpught they had fixed it soaybe this is a new regression20:41
clarkbjeblair: first change in ansible stack has a -1 from me maybe you can check on that too20:41
jeblairclarkb: is it possible that something about how we were configuring/running puppet before this ansible apply change inhibited that?20:42
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171420:42
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176220:42
*** jkilpatr has quit IRC20:43
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174920:44
clarkbjeblair: I dont think so as facter runs pretty independently amd we never did the monkey patch hack to remove ec2etadata facts irrc20:44
jeblairhrm.  ok.20:44
jeblairi guess we'll see how this one goes20:45
*** gyee has quit IRC20:48
fungihttps://tickets.puppetlabs.com/browse/FACT-718 mentions seeking the ability to disable ec2 metadata server queries20:48
mordredjeblair: back online - did I miss anyhting good?20:49
openstackgerritJames E. Blair proposed openstack-infra/system-config: Make apply test output more usable  https://review.openstack.org/37180120:49
jeblairmordred: so much :)20:49
jeblairmordred: check "topic:pansible"20:49
mordredjeblair: awesome!20:49
jeblairmordred: i think the whole story is there :)20:49
*** spzala has quit IRC20:50
*** ccamacho has quit IRC20:50
mordredjeblair: am I going to like this story?20:50
jeblairmordred: it's getting better20:50
*** ccamacho has joined #openstack-infra20:51
*** ccamacho has left #openstack-infra20:51
*** edmondsw has quit IRC20:51
*** esikachev has quit IRC20:51
fungibug 997641 also seems relevant20:52
openstackbug 997641 in facter (Ubuntu) "Could not run Puppet configuration client: Could not retrieve local facts: bad URI" [High,Invalid] https://launchpad.net/bugs/99764120:52
fungimordred: might even have a happy ending!20:52
*** trown is now known as trown|outtypewww20:53
fungior at least, not everyone will die until the sequel20:53
jeblairwell, not yet20:53
mordredfungi: that can't possibly be true20:53
*** lascii is now known as alaski20:53
jeblairnow i'm looking at http://logs.openstack.org/14/371714/7/check/gate-infra-puppet-apply-ubuntu-xenial/454476f/console.html and suspecting that i'm not getting the depends-on ansible role i'm expecting20:53
mordredcrinkle, nibalizer: didn't we get somewhere at some point in time with getting facter to not try to talk to ec2 metadata?20:54
clarkbmordred: yes, I think upstream facter fixed the exception handling when we went to puppet320:55
fungiso far i'm not finding evidence there's way to prevent facter from collecting (even network-dependent) facts without hacking facter itself20:55
nibalizeri dont recall20:55
crinklemordred: that doesn't ring a bell with me20:55
clarkbmordred: the issue I think was they had an exception handler but it was raising an exception that wasn't in the same class tree so it didn't get caught. They addressed that but this looks new20:55
*** sdake has joined #openstack-infra20:55
mordredhttps://tickets.puppetlabs.com/browse/FACT-71820:56
jeblairoh, the apply test is in parallel.  that seems to be causing ansible problems.20:56
jeblairi hate to slow it down, but i will serialize it for now, and we can revisit it later.20:56
mordredas of a couple of days ago, there is a patch in review for facter that would allow us to disable ec2 collection20:56
clarkbthats interesting I just ran facter on osic1 and it thinks the ost is physical20:56
mordredjeblair: ++20:56
mordredoh - I mean - it's landed20:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171420:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176220:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Make apply test output more usable  https://review.openstack.org/37180120:57
fungimordred: oh! i missed that was still active20:58
jeblair2016-09-16 20:43:21.422651 | INFO:zuul.CloneMapper:  openstack-infra/ansible-role-puppet -> /etc/ansible/roles/puppet20:59
jeblair2016-09-16 20:43:21.608419 | INFO:zuul.Cloner:Prepared openstack-infra/ansible-role-puppet repo with commit 194c6f53e1df8484546473da0a4fb03d5fed83e720:59
jeblairto me, that says that z-c put the dependent ansible role changes in place20:59
jeblairso i can't figure out why we would get:21:00
jeblair2016-09-16 20:44:10.027665 | fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "value of logdest must be one of: stdout,syslog, got: /tmp/puppet-out"}21:00
nibalizermordred: so its fixed if we want to use cfacter from master21:00
clarkbmordred: please see comment on https://review.openstack.org/#/c/371679/521:00
clarkbjeblair: does it need to be galaxy installed?21:00
nibalizeri like the suggested workaround though, putting in an empty custom fact with the same name as the default fact masks the default fact21:01
nibalizernot sure if that would make it stop collecting it alltogether though21:01
mordrednibalizer: interesting21:01
nibalizermordred: do you have a link handy to the facter/puppet output?21:01
mordredjeblair: are we running the ansible-galaxy command?21:01
jeblairmordred: not that i'm aware of21:01
mordredgrump then21:01
*** itsuugo has quit IRC21:02
*** rhallisey has quit IRC21:02
*** itsuugo has joined #openstack-infra21:04
*** ijw has joined #openstack-infra21:04
jeblairi held a node21:05
jeblair/etc/ansible/roles/puppet looks correct21:05
fungiso presumably it's using another copy elsewhere on the fs?21:06
jeblairwait i see it21:06
mordrednibalizer: for instance: http://logs.openstack.org/39/368239/3/check/gate-infra-puppet-apply-ubuntu-trusty/873682c/console.html.gz#_2016-09-09_22_07_17_14924721:06
mordrednibalizer: it's not a problem problem - but it does  cause people to have to look twice when they're looking for the real problem21:07
*** kzaitsev_mb has joined #openstack-infra21:07
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176421:07
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177421:07
*** ijw has quit IRC21:07
jeblairanother thing lost in the rebase earlier; that should do it21:07
mordred\o/21:07
*** ijw has joined #openstack-infra21:07
*** cardeois has quit IRC21:08
openstackgerritSergey Reshetnyak proposed openstack-infra/project-config: Enable publish docs on rtfd for fuel-ccp* projects  https://review.openstack.org/37180421:08
*** askb has joined #openstack-infra21:08
*** Goneri has quit IRC21:08
fungilooks like 371759 is safe to merge at least?21:09
*** psilvad has quit IRC21:09
fungijeblair: clarkb has notes on 37167921:09
jeblairfungi: mordred should review 75921:09
fungik21:09
jeblairi'm undoing a change he did :)21:09
jeblairfungi, mordred: and it's an alternative to https://review.openstack.org/37174521:10
mordredjeblair: I'm sure I'm excited about it21:10
mordredjeblair: oh, yes. definitely in favor of that21:11
jeblairmordred: anything to take your mind off the airport/airplane you are in :)21:11
mordredTHANK YOU21:11
mordredabandoned 74521:12
*** gouthamr has quit IRC21:13
jeblairgood news! http://logs.openstack.org/01/371801/2/check/gate-infra-puppet-apply-ubuntu-xenial/ead7eea/console.html21:13
jeblairthe bad news is...21:13
jeblairi don't know if it actually did anything :)21:13
*** mdrabe has quit IRC21:14
*** burgerk has quit IRC21:14
mordredjeblair: :)21:14
*** mdrabe has joined #openstack-infra21:14
mordredjeblair: we might should maybe add some -v to the invocation?21:15
*** abregman has quit IRC21:15
*** sshnaidm is now known as sshnaidm|away21:16
*** kzaitsev_mb has quit IRC21:16
fungimore -v never hurt anybody21:16
*** Douhet has quit IRC21:16
*** jkilpatr has joined #openstack-infra21:17
mordred4 of them might hurt loggstash21:17
jeblairi don't *think* we need that just yet21:18
*** ldnunes has quit IRC21:18
*** kzaitsev_mb has joined #openstack-infra21:18
openstackgerritMerged openstack-infra/system-config: Remove puppet_environment from the adhoc playbook  https://review.openstack.org/37175921:18
*** akshai_ has quit IRC21:18
*** sdake has quit IRC21:20
*** jimbaker has quit IRC21:20
*** sdake has joined #openstack-infra21:20
*** isviridov_away has quit IRC21:20
jeblairyeah, i think it ran, i just think my trick of setting logdest to the apply file worked21:21
*** akshai has joined #openstack-infra21:21
*** adrian_otto has joined #openstack-infra21:21
jeblairoh i think i know why21:21
jeblairrelative path21:21
*** isviridov_away has joined #openstack-infra21:21
*** salv-orl_ has quit IRC21:22
*** salv-orlando has joined #openstack-infra21:22
jeblair-e puppet_logdest=`pwd`/$fileout21:23
jeblairhow does that look?21:23
*** rlandy has quit IRC21:23
cloudnullclarkb: is that because we have nested virt?21:24
cloudnullre: facter on osic121:24
*** jimbaker has joined #openstack-infra21:24
*** jimbaker has quit IRC21:24
*** jimbaker has joined #openstack-infra21:24
clarkbcloudnull: oh maybe21:24
jeblairoh is it only on osic1?21:24
jeblair(i may have missed something)21:24
*** adrian_otto has quit IRC21:24
clarkbjeblair: I don't know, but I hopped onto an osic node and ran facter and it thought it was a physical host21:24
*** jheroux has quit IRC21:24
clarkbjeblair: I haven't queried logstash tosee what clouds have ec2 metdata fails21:25
*** sdake has quit IRC21:25
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171421:26
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176221:26
*** jamesdenton has quit IRC21:26
*** kzaitsev_mb has quit IRC21:26
*** inc0 has quit IRC21:26
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180721:27
*** gouthamr has joined #openstack-infra21:27
*** akshai has quit IRC21:29
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180721:29
pabelangerback, checking on the status of zuul-launcher21:29
jeblairclarkb, mordred, fungi: how cool is this! http://logs.openstack.org/14/371714/9/check/gate-infra-puppet-apply-debian-jessie/fbfd770/puppetapplytest53.final.out21:29
fungiwoah21:30
jeblairhttp://logs.openstack.org/14/371714/9/check/gate-infra-puppet-apply-fedora-24/52a6507/console.html21:30
fungiactual puppet output21:30
jeblairf24 is failing because it needs openssl libs in order to install ansible21:30
*** tsg__ has joined #openstack-infra21:30
mordredjeblair: woot!21:31
jeblairpabelanger: can you shed any light on the f24 thing ^ ?21:31
fungii need to disappear again to get the bits of the yard i couldn't get to yesterday, but am 100% on board and happy to review the dregs in ~ an hour when i get done21:31
*** ddieterly is now known as ddieterly[away]21:33
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180721:33
mordredjeblair: why do we run puppet apply tests on fedora 24 again?21:33
*** tsg_ has quit IRC21:33
jeblairno idea21:33
pabelangerwe have f24 nodes21:33
jeblairoh right, the nodepool bit21:33
mordredoh. and we puppet their initial thing21:33
mordredyah. got it21:33
mordredso, once we no longer puppet nodepool nodes, we will not need that21:33
*** ddieterly[away] is now known as ddieterly21:34
pabelangerYa, I know ianw added the puppet-apply test, but I'm not sure the state21:34
pabelangerI don't see bindep running, so need to figure out how to fix that21:34
jeblairit's non-voting21:35
jeblairso i don't know if it worked before21:35
jeblairbut i know that this change added one more thing to prevent it from working21:35
jeblairso if there's something easy to do to fix that, i'd like to21:35
jeblairbut if it's just not ready for that yet, ok.21:35
mordredpabelanger: we don't really have bindep for system-config it doesn't seem21:35
clarkb  mordred jeblair on the ansible role side it all looks good except forthat first change21:36
*** tsg__ has quit IRC21:36
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Simplify puppet cmd generation  https://review.openstack.org/37177821:36
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add debug argument  https://review.openstack.org/37181121:36
clarkbmordred: ^ can you read my comment there and decide if thats a thing?21:36
fungiincentive to stop using puppet to configure job nodes21:36
mordredfungi: ++21:36
pabelangermordred: Ya, not sure the history of it.21:36
jeblairfungi, mordred: agreed, though i'd rather not pull another string out of this sweater today.  :)21:36
pabelangerjeblair: I won't be able to dive into it tonight, but will have bandwidth during my travels tomorrow21:36
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180721:37
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171421:38
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176221:38
jeblairokay, that adds the '--debug' option back to puppet apply21:38
*** akshai has joined #openstack-infra21:38
mordredclarkb: I think we have to have _something_ for a module path in the environment file21:41
mordredclarkb: otherwise it would be impossible for an environment to carry its own version of a module21:41
clarkbmordred: yup thats the second part of that line21:41
mordredclarkb: that said - I agree with your reasoning on the second part21:41
*** gordc has quit IRC21:41
clarkbmordred: but I don't think you want the default in there ahead of the env specific one because it will resolve first21:42
mordredclarkb: you're thinking modulepath is additive on top of basemodulepath?21:42
clarkbmordred: yup thats what the puppet docs seem to say21:42
clarkbbasically basemodulepath always applies after more specific paths21:42
mordredawesome21:42
mordredclarkb: how about we do that as a followon fix to the end of the stack21:42
clarkbif you want ot be conservative you could flip the order of the two in the env file21:42
mordredclarkb: since jeblair has tested this stack to that point21:42
clarkbmordred: as long as we get that merged today too (so its not broken then we forget about it)21:43
mordred++21:43
jeblair2016-09-16 21:40:54.564868 | fatal: [localhost]: FAILED! => {"changed": true, "failed": true, "msg": "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!"}21:43
mordredmaking th patch now21:43
jeblairthat's the f23 apply job21:43
*** itsuugo has quit IRC21:43
mordredjeblair: how fasicinating21:43
jeblairyeah, i guess it's a 'new' dependency because we're running ansible there21:44
mordredI'm ... "excited" about that?21:44
jeblairwe aren't using bindep, right?21:44
mordred(I'm going to raise that as a bug upstream, that's stupid)21:44
*** itsuugo has joined #openstack-infra21:44
mordredwe are not21:44
mordredwell, we have an empty bindep tfile21:44
jeblairshould we start using it, or should i throw a 'yum install' or whatever in there?21:44
mordredjeblair: I could go either way- which ever is the least amount of additional effort for you21:45
fungidoes that job include the install-distro-packages macro?21:45
pabelangerwe should use bindep, but that is a change to JJB21:45
pabelangerfungi: not today21:45
fungioh21:46
fungieasy enough to add21:46
jeblairpabelanger: can you take care of that?21:46
fungiit's just a line in project-config i guess21:46
pabelangersure21:46
jeblairi think we have more to fix, and can work on that while it lands21:46
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Remove basemodulepath from environment config  https://review.openstack.org/37181221:46
mordredclarkb: there's the followon removal21:46
jeblairhttp://logs.openstack.org/14/371714/10/check/gate-infra-puppet-apply-ubuntu-precise/a7b9326/puppetapplytest06.final.out.FAILED21:46
jeblairi'm regretting the '--debug' thing i just added.  :(21:47
clarkbmordred: thanks, reviews updated21:47
fungithough if this is a job that runs for all our puppet modules... if a package is needed by it we presumably need to add it to a bindep.txt in every one of our module repos?21:47
*** esikachev has joined #openstack-infra21:48
*** kzaitsev_mb has joined #openstack-infra21:48
jeblairfungi: all of our puppet modules require ansible? :)21:48
jeblairmaybe 'yum install' in the apply script is the way to go after all....21:48
mordred++21:48
fungiprobably not, but is there an ansible-using job we're running against changes to all our puppet modules?21:48
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add bindep macro to gate-infra-puppet-apply-{node}  https://review.openstack.org/37181421:49
jeblairfungi: yes there is, but it's an integration test job21:49
jeblairso we may only (rightly) need it in the one place21:49
jeblairlet's see what pabelanger came up with :)21:49
jeblairfungi, pabelanger: i think maybe it will work if pabelanger puts that line after the infra-puppet-apply-prep line21:50
pabelangeractually, my patch will fail21:50
pabelanger1 sec21:50
fungiokay, just keeping in mind that the install-distro-packages builder relies on the bindep.txt in the repo for teh change being tested... we could use bindep within that integration test job in other non-repo-specific ways though, sure21:50
clarkbalso if we bindep selinux lib will venv'd ansible see it?21:51
fungipresumably selinux is not a python lib, so i expect so21:51
clarkbwell its pyhton-selinux it wants21:51
jeblairi think it was complaining about python-selinux21:51
fungiahh21:51
jeblairso yeah21:51
jeblairi think we need to bindep openssl dev, and then pip install selinux21:52
fungiwe may want python-selinux installed into the virtualenv, yes21:52
openstackgerritPaul Belanger proposed openstack-infra/project-config: Run bindep for gate-infra-puppet-apply-{node}  https://review.openstack.org/37181421:52
zigofungi: I have a concern right now about +dfsg branches in infra.21:52
* fungi really goes out to finish yardwork now21:52
fungior not21:52
mordredI have asked about python selinux bindings in #ansible-devel21:52
fungizigo: what concern?21:52
zigomordred: fungi: I'd like for ryu to make a "master-dfsg" branch that would be upstream minus its debian folder.21:52
pabelangerjeblair: fungi: ^ updated21:53
zigoI would merge tags from upstream there, but it shouldn't attempt to build a package.21:53
pabelangerwe cannot use install-packages macro, since we don't clone into .21:53
zigo(because it would fail)21:53
jeblairpabelanger: why 2 shell builders?21:53
fungizigo: your merge commits could just handle merge conflicts between your debian dir from e.g. the debian/newton branch and te master branch, right?21:53
jeblairpabelanger: (but otherwise, i think that will work, yes)21:53
zigofungi: mordred: Would it be possible to have just a noop job in some branches, instead of attempting to build?21:53
*** kzaitsev_mb has quit IRC21:53
pabelangerjeblair: mostly other jobs follow that format21:53
jeblairpabelanger: okay, time to stop that then :)21:54
*** hashar has quit IRC21:54
pabelangerYa, once we move to playbooks, should be much cleaner21:54
zigofungi: Yes, I could do the clean-up, but how will I push a tag for ryu called 4.6+dfsg1 then?21:54
pabelangerlet me push change quick21:54
*** EricGonczer_ has quit IRC21:54
zigoIt's needed to generate the tarball.21:55
fungizigo: right now we can skip jobs on specific branches, but the existing "noop" job can't be used by any project that runs other jobs in check or gate pipelines21:55
*** esikachev has quit IRC21:55
openstackgerritPaul Belanger proposed openstack-infra/project-config: Run bindep for gate-infra-puppet-apply-{node}  https://review.openstack.org/37181421:55
zigofungi: I'd like to do that then.21:55
fungizigo: so you'd need at least one job (not noop) on any branch to which you push changes21:55
*** EricGonczer_ has joined #openstack-infra21:56
zigofungi: I'm not sure how this translates into a project-config patch... :/21:56
fungizigo: i'm probably not understanding the problem wit pushing a tag in this case. i assume you'd tag the merge commit after it's merged, or the debian/.* branch state after the merge commit lands21:56
pabelangerI've started graceful restarting zuul-launchers21:57
zigofungi: Do you have time so that I can explain the workflow?21:57
*** rfolco has quit IRC21:57
pabelangerStill need to push up keytab files, working on that now21:57
jeblairokay, i don't like --debug.  i'm going to turn it off.21:58
*** tsg__ has joined #openstack-infra21:58
*** rtheis has quit IRC21:58
fungizigo: i don't really have time right now, no21:59
zigofungi: Should I send a mail?21:59
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171421:59
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176221:59
fungizigo: the openstack-infra@lists.openstack.org ml might be a good place to hash it out, yes21:59
*** Swami has quit IRC22:00
clarkbjeblair: how long does the trusty job take serially?22:01
mordredjeblair: the heira lookups are super distracting, yeah22:01
jeblairclarkb: not sure yet22:02
jeblairclarkb: precise takes 5 min.  :)22:03
jeblairxenial 322:03
clarkbas for why ansible breaks when run in parallel I think maybe because its trying to copy all the files to the same locations and stomping over each other. Hacking around thatwill be interesting22:04
*** ddieterly has quit IRC22:04
*** xyang1 has quit IRC22:04
jeblairclarkb: yes, i believe that's the case22:04
*** akshai has quit IRC22:04
jeblairclarkb: there's the issue of the files under ansible control (ie, hiera.yaml, puppet.conf, etc).  but *also* i think there may be races with ansible itself (ie, ansible .tmp dirs)22:05
jeblairthe second is actually probably easy to fix with per-'node' ansible directories22:05
jeblairthe first may not end up being an issue (if all the files end up being the same anyway)?22:06
clarkbpossibly, though depending on how files are copied we may end up with truncated or unlinked files during small windows fo time22:06
jeblairhrm22:06
jeblairhttp://logs.openstack.org/14/371714/11/check/gate-infra-puppet-apply-ubuntu-precise/9b4f6f2/puppetapplytest06.final.out.FAILED22:06
openstackgerritPaul Belanger proposed openstack-infra/system-config: Add zuul-launcher keytab to zl0(1-8) nodes  https://review.openstack.org/37181822:06
jeblairwhy did that end up with debug output?22:06
pabelangeradds keytab to zuul-launchers^ I skipped zlstatic01.o.o, but could added it if people think I should22:07
jeblairmordred, clarkb: am i misunderstanding something about module arguments in 371811?22:08
clarkbit looked right to me22:09
clarkbbasically default to false and then omit things by default22:09
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180722:09
jeblaircause that run ended up with debug lines but it did not set puppet_debug22:09
pabelangerjeblair: try adding type=boolean22:10
clarkbheh if that fixes it we are gonna have to add that to the whole stack likely22:11
jeblairi'll go ahead and add it to both the noop and debug arg changes22:11
pabelangerI thought we needed to be more strict with type casting now22:11
pabelangertype='bool' it looks like22:11
jeblairoh yeah, like show_diff22:11
*** mdrabe has quit IRC22:11
openstackgerritPhillip Toohill proposed openstack-infra/project-config: Adding Octavia senario job  https://review.openstack.org/37180722:12
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Simplify puppet cmd generation  https://review.openstack.org/37177822:13
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add debug argument  https://review.openstack.org/37181122:13
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Remove basemodulepath from environment config  https://review.openstack.org/37181222:13
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177422:13
jeblairand rechecked22:13
*** bauwser is now known as bauzas22:14
*** gyee has joined #openstack-infra22:16
openstackgerritMerged openstack-infra/project-config: Run bindep for gate-infra-puppet-apply-{node}  https://review.openstack.org/37181422:16
*** gouthamr has quit IRC22:17
*** itsuugo has quit IRC22:18
*** itsuugo has joined #openstack-infra22:19
*** mriedem_afk is now known as mriedem22:22
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182322:22
jeblairclarkb: ^ maybe? :)22:22
jeblairpabelanger: http://logs.openstack.org/14/371714/11/check/gate-infra-puppet-apply-ubuntu-precise/4483925/puppetapplytest06.final.out.FAILED22:22
jeblairyep, the bool thing was the issue22:23
jeblairnow, i have no idea why that failed :)22:23
jeblairailed with return code: 4", "rc": 4,22:23
jeblair4: The run succeeded, and some resources failed.22:23
*** yamahata has quit IRC22:25
clarkbjeblair: I would expect that to work as long as the hiera and puppet module copies don't truncate things or otherwise handling the sync poorly. But it uses rsync under the hood so it should be smart about it iirc22:26
clarkb(atomic moves but also checking file hashes)22:26
clarkbhrm looks like it failed a lot according to zuul though22:27
jeblairwell, i'm still digging into the serial failures22:27
clarkbok22:27
* clarkb looks at the rc 4 one22:27
*** xarses has quit IRC22:28
clarkblooks like it may be mad about missing init scripts but I thought under noop those were ok (they are missing because the package installs didn't actually happen)22:28
jeblairclarkb: here is a run from the current puppet apply: http://logs.openstack.org/07/371607/3/check/gate-infra-puppet-apply-ubuntu-precise/220e751/puppetapplytest06.final.out22:29
jeblairit seems to be doing much more now.22:30
*** EricGonczer_ has quit IRC22:30
jeblairoh22:30
jeblairthe old way only output stderr to that file22:31
*** ijw has quit IRC22:31
jeblairthis has stdout and stderr22:31
*** makowals has joined #openstack-infra22:31
jeblairoh22:32
jeblairand the old way did *not* have --detailed-exitcodes22:32
clarkboh huh. But also that one you linked to me was successful22:33
jeblairclarkb: yes22:33
jeblairwas trying to compare current failed to previous success22:33
*** dizquierdo has quit IRC22:33
jeblairi may be using tenses poorly here22:33
*** kzaitsev_mb has joined #openstack-infra22:34
*** markvoelker has quit IRC22:34
clarkbgotcha. I think that it may not use detailed exit codes in the psat because then it handles noop fails properly22:34
jeblairyeah, i think i agree with that22:34
jeblairso i think we need to have our puppet role have an option there22:34
jeblairor...22:35
jeblairmaybe if we noop, we don't set detailed_exitcodes22:35
clarkb++ I like that idea22:35
jeblairk, will do that22:35
*** ddieterly has joined #openstack-infra22:37
*** ddieterly has quit IRC22:37
*** yamahata has joined #openstack-infra22:37
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Don't set detailed-exitcodes when noop  https://review.openstack.org/37182722:37
clarkbbecause then future us won't try to toggle that knob and break us then wonder why22:37
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176222:38
jeblairhehe22:38
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171422:38
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182322:38
*** itsuugo has quit IRC22:39
*** itsuugo has joined #openstack-infra22:41
jeblairthat's not looking good so far.22:42
*** rbrndt has quit IRC22:42
jeblair2016-09-16 22:41:54 +0000 Puppet (err): Could not find class snmpd for debian-jessie-osic-cloud1-4352169.openstack.org on node debian-jessie-osic-cloud1-4352169.openstack.org22:43
*** kzaitsev_mb has quit IRC22:44
*** tsg__ has quit IRC22:44
jeblairclarkb: erm, i wonder if there is an error with 37181222:46
jeblairi think this is the first run that included that22:46
jeblairi will restack and retry22:46
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Don't set detailed-exitcodes when noop  https://review.openstack.org/37182722:46
openstackgerritJames E. Blair proposed openstack-infra/ansible-role-puppet: Remove basemodulepath from environment config  https://review.openstack.org/37181222:46
clarkbits possible I read the docs wrong and we just wanted to flip the order of the path entries22:47
clarkbI definitely think we want to change the order. So that your env modules are before the base modules22:47
*** ddieterly has joined #openstack-infra22:47
*** kzaitsev_mb has joined #openstack-infra22:48
*** david-lyle has joined #openstack-infra22:48
*** Jeffrey4l has quit IRC22:48
jeblairclarkb: yeah, i think we do want something like that.  i wasn't testing custom modules this morning (aside from the openstack_project module in system-config), so i wasn't exercising this code path22:48
clarkb(in other news I am building a xenial image with the current set of glean changes just to double hceck that is still all happy)22:49
jeblairalso, i wonder if we should add an apply-test job to ansible-role-puppet after the dust settles22:49
*** Jeffrey4l has joined #openstack-infra22:49
mordredjeblair: back online (yay planes!)22:50
jeblairmordred: yay!22:51
*** esikachev has joined #openstack-infra22:51
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182322:52
*** ddieterly is now known as ddieterly[away]22:53
*** ddieterly[away] is now known as ddieterly22:53
jeblairokay, i have the change that fixes the selinux thing ready, but i'm going to give the ubuntu-trusty run a chance to finally finish first :)22:54
*** spzala has joined #openstack-infra22:54
mordredjeblair, clarkb: 371778 and 371881 both look good - we should submit those changes upstream - do you want to or do you want me to?22:54
clarkbsomething something aren't we upstream?22:55
*** esikachev has quit IRC22:55
mordrednope22:55
mordredthe puppet module exists in upstream ansible22:55
*** itsuugo has quit IRC22:55
jeblairthat's probably why i couldn't find any tags22:56
mordredwe've been keeping a copy in our role for these reasons22:56
jeblairmordred: the 'version added' may be wrong :)22:56
mordredyah22:56
mordred:)22:56
*** itsuugo has joined #openstack-infra22:56
jeblairmordred: i would appreciate it if you did that, thanks.22:56
clarkbya if there is someplace else we need to be in sync with we should make sure that happens22:57
mordredjeblair: I will do it22:57
mordredwe should make that noop parameter actually obey the ansible check mode22:58
jeblairmordred: read the change for why not :)22:58
mordredoh! neat!22:58
clarkb:)22:58
* mordred is reading changes, but there are many22:58
jeblairmordred: 37177422:58
mordredah - yes22:59
jeblairmordred: i think 371764, 371774, 371778, 371811, 371827 are all universally applicable22:59
mordredI agree with that ... but I think _also_ if someone did ansible --check that sohuld potentially imply --noop, no?22:59
*** xarses has joined #openstack-infra22:59
jeblairi think it still does?22:59
clarkbya it still does22:59
mordredcool22:59
*** ddieterly has quit IRC22:59
clarkbits an check or noop case22:59
*** kzaitsev_mb has quit IRC22:59
mordredsweet! then we're set22:59
jeblairclarkb: the trusty run is "not fast"23:00
*** kzaitsev_mb has joined #openstack-infra23:01
jeblair2016-09-16 22:55:38.890839 |   File "/tmp/apply-ansible-env/lib/python2.7/os.py", line 157, in makedirs23:02
jeblair2016-09-16 22:55:38.890903 |     mkdir(name, mode)23:02
jeblair2016-09-16 22:55:38.890975 | OSError: [Errno 17] File exists: '/root/.ansible/tmp'23:02
jeblairmordred: didn't we fix that?23:02
jeblairin ansible23:02
jeblairor is that still unreleased?23:03
jeblairat any rate, i think my export ANSIBLE_CONFIG=/tmp/tmp.VvouyB04Iy/ansible.cfg is not making it through sudo23:03
clarkbjeblair: ya I recall reading the PR for it but guessing unreleased23:03
mordred2.2 is in feature freeze, should be out soon23:04
clarkbfwiw glean master looks happy on xenial for me locally23:05
jeblairclarkb, mordred: can you take a look at 371823 and tell me why you think http://logs.openstack.org/23/371823/3/check/gate-infra-puppet-apply-ubuntu-precise/48a72b7/console.html#_2016-09-16_22_55_38_890338 is happening?23:05
jeblairi would think that the config file i write there would prevent it23:05
jeblairoh yeah, it is the env var23:06
jeblairsudo -E ?23:06
clarkbya sudo -E or set it on the other side of sudo and bash it?23:06
*** itsuugo has quit IRC23:06
clarkbsudo -H bash -c "foo=bar cmd"23:07
clarkbbut -E is probably fine here since security isn't an issue23:07
jeblairor i could cd into the tmpdir23:07
*** itsuugo has joined #openstack-infra23:08
jeblairclarkb: 15 minutes for trusty23:08
jeblairclarkb: and it passes :)23:08
clarkbjeblair: cool thats not too long23:08
clarkbjeblair: so that means we hvae to edit the puppet env file?23:09
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171423:09
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176223:09
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182323:09
jeblairthat's with "-E" because it's the least typing23:09
jeblairclarkb: ?23:09
clarkbjeblair: you mentioned earlier you were restacking thinking that https://review.openstack.org/#/c/371812/2 broke things?23:10
jeblairclarkb: yeah, it seems to have worked after i pulled that out of the dependency chain for the system-config changes23:10
jeblairclarkb: so i think it is problematic23:10
jeblairi haven't thought further about it though23:11
clarkbjeblair: let me push a new ps that flips the order of the path entries rather than removing one of them23:11
jeblairclarkb: cool.  push a DNM system-config change that depends-on it and it should get tested.23:11
jeblairclarkb: (and parent the system-config change on 371714)23:12
clarkbok23:12
openstackgerritClark Boylan proposed openstack-infra/ansible-role-puppet: Move basemodulepath to end in environment config  https://review.openstack.org/37181223:14
jeblairer, the pip install python-selinux thing isn't right23:14
jeblairoh libselinux-python23:15
*** HeOS has quit IRC23:15
mordredclarkb: if you get a sec ... https://github.com/ansible/ansible-modules-extras/pull/2945 - I had to rebase bcause there are changes upstream we don't have. I'm 95% I got it right23:15
jeblairmordred: do you happen to know what thing i should pip install to make ansible not say 'libselinux-python' needs to be installed?23:16
mordrednope. but let me ask23:16
openstackgerritClark Boylan proposed openstack-infra/system-config: DNM: test ansible-role-puppet env file  https://review.openstack.org/37183223:16
clarkbjeblair: ^ like that?23:16
jeblairclarkb: yep!  it's going to instafail because of the selinux thing, but i'll recheck it as soon as we figure that out23:17
mordredjeblair: it's a system dep23:18
mordredno pypi23:18
reedlooks like https://openstackid.org/ is taking a nap... is it?23:18
clarkbreed: its up for me23:18
reedso weird, it comes and goes "server not found"23:18
reedmaybe it's my dns though23:19
clarkbmordred: reviewd the PR23:20
*** spzala has quit IRC23:20
clarkbjeblair: but only instafail on fedora right?23:20
jeblairmordred: what's a bindep line for libselinux-python look like?23:20
jeblairclarkb: everywhere23:20
mordredjeblair: so it seems dnf install libselinux-python on our fedora nodes is the answer ... and we may want to add that to the fedora-minimal element23:20
reedah23:20
clarkboh thats sadness23:20
jeblairmordred: we can probably do a bindep thing for system-config now23:20
reedclarkb, www.openstackid.org doesn't exist23:20
reedi was typing that one :)23:21
mordredjeblair: I think it would be "libselinux-python [platform:rpm]"23:21
*** kzaitsev_mb has quit IRC23:21
clarkbmordred: jeblair but bindep for that won't go into the venv right?23:21
mordredit isn't needed in the venv23:21
clarkbor is ansible breaking out of the venv with its forking/sshing?23:21
mordredit's needed on the target host23:21
*** Jeffrey4l has quit IRC23:21
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171423:22
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176223:22
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182323:22
*** fguillot has joined #openstack-infra23:22
mordredthe venv has no bearing on the hosts ansible is remotely controlling23:22
mordred(which happen to be the same host ansible is installed on in this case)23:22
clarkbshould we be venving that and setting the remote python path to use the venv?23:22
mordrednope23:22
openstackgerritJames E. Blair proposed openstack-infra/system-config: DNM: test ansible-role-puppet env file  https://review.openstack.org/37183223:22
*** Jeffrey4l has joined #openstack-infra23:22
jeblairmordred: oooooh remote host.  gotcha.23:22
jeblairi will probably forget this one more time before it's all over.  sorry about that.23:23
mordredI do think we need to make sure libselinux-python is installed on all of our fedora hosts23:23
clarkband ubuntu apparently23:23
mordredbut that can be a next step23:23
pabelangerlibselinux-python [platform:rpm] in bindep.txt should be the right fix23:23
clarkbits called pyton-selinux on ubuntu23:23
jeblairmordred: i think i agree with that now.23:23
jeblairclarkb: i don't think we need it on ubuntu23:23
pabelangerthat's what I uses for testing roles23:23
mordredclarkb: but ubuntu doesn't run selinux23:23
clarkbjeblair: oh I thought you said all the jobs were going to fail?23:23
jeblairclarkb: (it was only instafailing because i was trying to install it on ubuntu)23:23
clarkbah23:23
jeblairso the latest changes have it in bindep for rpm systems23:24
jeblairso we'll see how that goes23:24
mordred\o/23:24
* mgagne is still waiting for the thundering herd23:24
pabelangermgagne: http://nodepool.openstack.org/image-list23:25
pabelangerdone for today23:25
mgagnestill in building state no?23:25
pabelangeroh, I was looking at mtl0123:26
mgagnewell, I'm looking at mtl01 too for xenial image23:26
openstackgerritMonty Taylor proposed openstack/diskimage-builder: Add libselinux-python to yum-minimal  https://review.openstack.org/37183423:26
mordredclarkb, greghaynes: ^^23:26
mgagneoh23:26
mgagnehaha, made the same mistake23:26
mgagnebut nothing "wrong" on our side according to grafana ¯\_(ツ)_/¯23:26
pabelangerneed to submit a patch to nodepool tomorrow to stop uploads for ovh-bhs123:27
mgagnecould it be we are Friday and there was no load on Zuul at that time?23:27
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Sync puppet module changes from upstream ansible  https://review.openstack.org/37183523:27
*** rvasilets___ has quit IRC23:27
mordredclarkb: there is the corresponding "sync from upstream"23:28
clarkbya zuul is pretty quiet today23:28
mordrednibalizer: you're a core on the puppet module upstream - could you look at https://github.com/ansible/ansible-modules-extras/pull/2945 too?23:28
pabelangerI see a few launch errors in mtl01, but nothing like the last few days23:29
*** askb has quit IRC23:29
mgagneyea, could be lack of load made it so Glance had time to download and cache instead of stacking like yesterday23:29
jeblairpabelanger: http://logs.openstack.org/14/371714/14/check/gate-infra-puppet-apply-centos-7/e6b622c/console.html#_2016-09-16_23_22_57_06760523:30
jeblairpabelanger: i don't understand the output of that23:30
jeblairit doesn't seem like it installed anything23:31
pabelangerjeblair: that would mean bindep has all the dependencies installed23:31
pabelangeror dependencies it matched23:31
jeblairi guess we need to hold a node23:31
mordredyah23:31
mordredyou type faster than me23:31
mgagnepabelanger: ok, I'm out for tonight. We will continue our optimization work next week. Hopefully weekend will be quiet23:32
openstackgerritJames E. Blair proposed openstack-infra/system-config: DNM: test ansible-role-puppet env file.  https://review.openstack.org/37183223:32
clarkbin http://grafana.openstack.org/dashboard/db/nodepool-osic the test nodes (cloud 1) and test nodes (cloud 8) graphs seem to render the orange deleting line over the blue in use line. Does anyone else see that too?23:32
mgagneclarkb: yes23:33
mgagneclarkb: confusing colors23:33
clarkbI am reading the yaml and the config for it looks correct and if you hover over the graph the actual values are correct23:33
jeblair2001:4800:1ae1:18:f816:3eff:fe0c:bcf4 and 104.239.192.21223:33
mordredfwiw,  https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/basic.py#L782-L793 is where the selinux check is happening23:34
*** Benj_ has quit IRC23:34
clarkbpabelanger: any idea why grafana is rendering that way?23:34
mordred[mordred@fedora-23-rax-ord-4352630 ~]$ rpm -ql libselinux-python-2.4-4.fc23.x86_6423:34
mordredshows files23:34
clarkboh I think it may be stacking23:35
jeblairclarkb: they are stacked23:35
clarkbjeblair: ya23:35
mordredjeblair: ok. I see selinux installed23:35
mgagneyes, changing order makes it even more confusing in some case :-/23:35
openstackgerritClark Boylan proposed openstack-infra/project-config: Add devstack runtimes for OSIC cloud8  https://review.openstack.org/37183823:36
jeblairlibselinux-python-2.4-4.fc23.x86_6423:36
clarkbadding cloud8 to the job runtime graphs in ^23:36
pabelangerI think because you have inventory=localhost, you are looking in the venv, not system.23:36
mordredjeblair: maybe since it is doing localhost it's not doing ssh23:36
mordredso yeah - what pabelanger said23:36
pabelangerif you did inventory=127.0.0.2 It would force SSH23:37
mordredso I think your original quesiton "do I need to add --site-packages" is likely correct23:37
jeblair:)23:37
mordredAND - we don't need it in bindep.txt23:37
mordredsince it's there23:37
jeblairnor the dib change, yeah?23:37
mordredyah. abandoning23:37
jeblairvirtualenv --site-packages /tmp/apply-ansible-env23:38
jeblairis that right?23:38
*** kzaitsev_mb has joined #openstack-infra23:38
mordredjeblair: I'mma try something on that node ...23:38
clarkb--system-site-packages according ot my local venv23:38
pabelangerOh ya, I forgot: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/zuul-worker/package-installs.yaml23:38
pabelangerwe have it in our zuul-worker element23:38
jeblairmordred: oh, maybe you do want the dib change after all then ^23:39
openstackgerritJames E. Blair proposed openstack-infra/system-config: Stop managing puppet config in puppet  https://review.openstack.org/37171423:40
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ansible for puppet apply test  https://review.openstack.org/37176223:40
openstackgerritJames E. Blair proposed openstack-infra/system-config: Run ansible apply test in parallel  https://review.openstack.org/37182323:40
pabelangerjeblair: however, if the install-package macro is still used, you likey want an empty bindep.txt file other wise, it will use bindep-fallback.txt23:40
openstackgerritJames E. Blair proposed openstack-infra/system-config: DNM: test ansible-role-puppet env file  https://review.openstack.org/37183223:41
jeblairpabelanger: yeah, that was already there23:41
pabelangercool23:41
jeblaira centos7 success: http://logs.openstack.org/23/371823/6/check/gate-infra-puppet-apply-centos-7/456c33e/23:43
jeblairrunning in parallel, no less!23:43
clarkbjeblair: I need to rebase my dnm change?23:44
jeblairclarkb: i did it23:44
*** kzaitsev_mb has quit IRC23:44
clarkboh cool I see itnot it came in a little after the first 323:44
jeblairclarkb: yeah, took me a sec to back up since it forks off before the parallel testing change23:45
clarkbit worked on xenial looks like so I think thats probably what we want23:45
clarkbthe order flip rather than just entry removal23:45
jeblairya23:45
mordredjeblair: fwiw, I recreated the failued by hand on that node and also then ran with site-packages and it succeeded23:45
clarkboh it just failed centos though23:45
mordredI think ypiu're alreayd ahead of me23:45
mordredbut I just wanted to double check while we had it23:45
jeblairmordred: cool, thanks23:45
mordredthe venv is /home/jenkins/workspace/gate-infra-puppet-apply-fedora-23/openstack-infra/system-config/bong23:46
jeblairclarkb: hrm, i may have botched that rebase.. checking23:46
mordredI also restored the dib change ... so - greghaynes ... ping stands23:46
jeblairclarkb: yep.  ignore failure23:46
clarkbjeblair: kk23:46
openstackgerritJames E. Blair proposed openstack-infra/system-config: DNM: test ansible-role-puppet env file  https://review.openstack.org/37183223:47
jeblairclarkb: there we go23:47
*** denisra_ has quit IRC23:47
jeblairclarkb: the parralel run finished.  all success. trusty took 6m.23:47
jeblairhttps://review.openstack.org/37182323:48
*** hongbin has quit IRC23:48
*** fguillot is now known as fguillot_afk23:48
*** fguillot_afk is now known as fguillot23:48
mordredwoot!23:50
funginow that i've bent the yard to my will for a few more days, time to see what i missed23:50
mordredfungi: all the things23:50
*** claudiub|2 has quit IRC23:51
clarkbmordred: did you see that my local image using glean master was happy. Probably don't want a new release before we all get on planes but maybe we can do that early monday?23:51
*** esikachev has joined #openstack-infra23:52
clarkbI should probably pack soon23:53
jeblairmtreinish: sorry about that -- today has been... busy.  if you can pastebin info about the http request that happens before the failure, that would be great.23:53
*** zhurong has joined #openstack-infra23:53
jeblairit's possible that's huge and the best way for us to debug this is to fly to germany and look at your compter.  :)23:54
mordredclarkb: yay!23:54
*** zhurong has quit IRC23:54
mordredjeblair: I like compters23:54
jeblairclarkb: yeah, i'm expecting test results in a couple of minutes, and then i need to afk23:54
mordredalso - clarkb, jeblair, fungi : https://www.youtube.com/watch?v=PJiRijiLwbQ23:55
jeblairmordred: i hope you brought yours23:55
clarkbmordred: rumor has it that is how perl was born23:55
*** kjorgensen has quit IRC23:55
jeblairclarkb: i heard it was modem line noise23:55
mordredclarkb: don't let larry hear you say that ;)23:55
mordredhe lives nearer to you than I do you know23:56
jeblairbut, to be fair, i heard that a long time ago23:56
mordredjeblair: what, modem line noise?23:56
*** dahoo has quit IRC23:56
jeblairyes... though... maybe i'll play some insoc and listen to it again.23:56
*** esikachev has quit IRC23:56
mordredjeblair: we should figure out a way to generate a soundcloud feed that is some data from the gate encoded into modem tones23:57
jlkmordred: STAHP23:57
jlkYou're about 3 drinks and 2 bad decisions away from Skynet23:57
mordredjlk: you know you want to listen to the gate23:57
mordredwe could all sit around and be like "oh, a nova failure"23:57
mordredand then grin knowingly23:58
clarkbjust pipe the json file into /dev/whateveritscalled23:58
*** kjorgensen has joined #openstack-infra23:58
jlkI don't even see code anymore. All I see is nova failure, recheck, grenade failure, ...23:58
mordredhahaha23:58
clarkbthe trick would be to make it sound all choral when everything is happy and heavy metal when its not23:59
jeblairmordred, pabelanger: why have we not hooked up status.json to asterisk yet so you can dial in and have asterisk read you the gate status of your change?23:59
jlkand then we can have bikeshed arguments about what color the sound should be for the passing and failing.23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!