Wednesday, 2015-09-02

*** TravT_ has joined #openstack-sprint02:50
*** TravT has quit IRC02:51
*** TravT_ has quit IRC03:27
*** TravT has joined #openstack-sprint03:28
*** mrmartin has joined #openstack-sprint04:39
*** mrmartin has quit IRC04:54
*** mrmartin has joined #openstack-sprint05:03
*** mrmartin has quit IRC05:04
*** mrmartin has joined #openstack-sprint06:31
*** mrmartin has quit IRC06:45
*** mrmartin has joined #openstack-sprint06:53
*** mrmartin has quit IRC08:50
*** mrmartin has joined #openstack-sprint09:09
*** jhesketh has quit IRC09:39
*** jhesketh has joined #openstack-sprint09:41
*** pabelanger has joined #openstack-sprint10:59
pabelangermorning10:59
*** clayton has joined #openstack-sprint11:10
*** rfolco has joined #openstack-sprint11:54
*** mordred has quit IRC12:27
*** timrc has quit IRC12:28
*** timrc has joined #openstack-sprint12:33
*** ChanServ changes topic to "Puppet Liberty Mid-Cycle Sprint https://etherpad.openstack.org/p/puppet-liberty-mid-cycle"12:51
*** mwhahaha has joined #openstack-sprint13:14
*** TravT has left #openstack-sprint13:39
EmilienMgood morning13:42
anteayaEmilienM: pabelanger morning13:42
*** vinsh has joined #openstack-sprint13:42
*** tristanC has joined #openstack-sprint13:43
*** fbo has joined #openstack-sprint13:45
*** richm has joined #openstack-sprint13:47
anteayasay hello when you arrive so we are sure to include you13:54
anteayaEmilienM pabelanger and I are face to face so we want to be sure to include folks on things13:54
vinshHello.13:56
EmilienMwe're talking about CI13:57
EmilienMhttps://etherpad.openstack.org/p/puppet-openstack-CI13:57
*** mrmartin has quit IRC14:00
mwhahahamorning folks14:07
*** mrmartin has joined #openstack-sprint14:09
pabelangerhttps://wiki.openstack.org/wiki/Infrastructure/Conferencing14:12
pabelangerwe are in conference room 600014:12
anteayavinsh mwhahaha and others you are welcome to join us on asterisk14:12
anteayafollow the wiki instructions above14:12
*** nibalizer has joined #openstack-sprint14:13
nibalizero/14:13
anteayanibalizer: hey14:13
anteayawe are on asterisk if you want to join us14:13
anteayahttps://wiki.openstack.org/wiki/Infrastructure/Conferencing14:13
anteayaconference room 600014:13
anteayaand currently are working on this etherpad: https://etherpad.openstack.org/p/puppet-openstack-CI14:14
nibalizermaybe14:14
nibalizerit is a bit early in the morning for me to phone14:14
anteayaunderstood14:14
anteayawe are talking about multinode14:14
nibalizerI'm also not sure what my puppet/infra/downstream work split is today14:15
anteayaEmilienM pabelanger and I14:15
anteayanibalizer: understood14:15
EmilienMnibalizer, we're on https://etherpad.openstack.org/p/puppet-openstack-CI14:18
* vinsh dialed in.14:20
anteayavinsh: welcome14:21
anteayahow is the audio?14:21
vinshIt sounds good!14:21
anteayagreat14:21
pabelanger60951114:22
pabelangeroops14:22
nibalizerhh14:22
nibalizerheh14:22
*** mfisch has joined #openstack-sprint14:23
EmilienMmfisch: welcome14:23
mfischhey14:24
mfischI'm only half here dealing with a production issue14:25
anteayaack14:27
EmilienMnibalizer: are you online?14:32
EmilienMon the call I mean14:32
nibalizerI am not on the call14:39
anteayathe zuul3 spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html14:40
anteayaright now I can find no code in any state regarding the zuul3 implementation14:41
nibalizeralso i'd be happy to join the call to sync up as the sprint begins, but 3 days of phone call sounds pretty brutal tbh14:42
anteayasure that is fine14:43
nibalizerI certainly can't spend all 3 days working on puppet, so if most of the chatter were here I could pop in when needed and read scrollback, if that makes sense14:43
anteayawe are taking notes in the etherpad14:43
anteayanibalizer: glad to have you for what you can help with14:43
anteayaand if you have any thoughts about current puppet ci14:44
anteayathat is the current topic14:44
nibalizerya sure14:44
nibalizerso for multinode, I'm guessing here, but a few weeks back someone (paul?) wanted a sepearte node to run tempest from14:44
crinkleI'm a little concerned that this sprint is happening over conference call, if it was happening in this channel as planned then the whole discussion could be logged and folks not in a convenient time zone could participate14:45
nibalizerand my thoughts on that are confused because devstack doesn't use a separate node for tempest, so why should we?14:45
pabelangernibalizer: Right, that was a simple method of flexing multinode nodepool14:46
EmilienMcrinkle: we can stop using the call - though we took notes on https://etherpad.openstack.org/p/puppet-openstack-CI14:46
*** _ody has joined #openstack-sprint14:47
*** cdelatte has joined #openstack-sprint14:47
EmilienMsee "Discussion around muti-node" notes14:47
mfischwhats the next topic?14:49
EmilienMcrinkle, nibalizer: during the infra sprint, did you use conferencing tool?14:49
nibalizerno we just chat in here14:49
pabelangerare we ready to move on?14:49
EmilienMok14:49
EmilienMso for "Running OpenStack AIO", we discussed with Paul and everything is written in the etherpad14:50
EmilienMany thoughts ?14:51
pabelangerwell, I think virtual people have questions about multinode / single node?14:52
EmilienMsingle node is for now making progress using scenario001.pp but we might need to split the manifest if running multiple scenarios14:53
anteayavinsh: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/projects.yaml#n325714:54
vinshTY14:54
anteayavinsh: let me get you more14:54
anteayawelcome14:54
EmilienMcrinkle: for single node, I would like last reviews on https://review.openstack.org/#/q/status:open+project:openstack/puppet-openstack-integration+owner:%22Emilien+Macchi+%253Cemilien%2540redhat.com%253E%22,n,z14:54
EmilienMso we would enable the jobs for puppet-glance,nova,neutron14:55
nibalizerpabelanger: ya I don't understand the multinode need?14:55
nibalizerinfra doesn't have infinite resources so I think we need compelling reasons to use multinode14:55
nibalizerplus it is much more complicated14:55
pabelangernibalizer: for me, it is the real world example of testing14:55
pabelangernobody runs a cloud on a single node14:55
EmilienManteaya: https://review.openstack.org/#/c/219556/ was failing to land, I rebased it, can you +A again?14:56
anteayavinsh: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n188914:56
mfischsingle node is still a good starting point test though14:56
nibalizerso the goal is to have a separate compute node?14:56
mfischof course anything is better than what we had before you guys started on it14:56
crinkleEmilienM: the integration tests are failing for 3 out of 4 of those14:56
EmilienMcrinkle: because of packaging issue that we faced14:57
nibalizerit says zuulv3 right now, which doesn't exist in code form14:57
pabelangerso, my question is, what is the downside of having multinode?14:57
EmilienMcrinkle: if can point to you latest working logs14:57
* EmilienM look for URL14:57
anteayavinsh: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n4814:57
mfischfrom a puppet pov whats the upside of a separate node?14:57
claytonI assume the issue is that it chews up x instances instead of 1 instance and there is a hard cap on the total instances nodepool can use at any given time14:57
crinkleEmilienM: okay but it seems silly to merge a patch about tests when the tests its adding are failing14:57
mwhahahatesting cross system dependancies14:57
anteayavinsh: those parts should at least get you a little bit of context, and be a jumping off place for more questions14:57
mwhahahaie keystone endpoint creation on a machine that is not the keystone server14:58
EmilienMcrinkle: I agree with you, though we are not going to change the patches because of packaging issues, they are not directly related14:58
vinshanteaya: Very good. I'll study.14:58
nibalizerpabelanger: I think 2 nodes with one controller and one compute is a good idea14:58
anteayavinsh: great thanks14:58
EmilienMcrinkle: http://logs.openstack.org/52/217352/34/check/gate-puppet-openstack-integration-dsvm-centos7/968c4af/14:58
nibalizerI think a separte node for tempest is not14:58
EmilienMcrinkle: from https://review.openstack.org/#/c/217352/14:58
mfischyes why would tempest need a node?14:58
mfischits just making api calls14:58
pabelangerRight, tempest is not an issue right now.  Having tempest on a multinode was the simplest way to show multinode working14:58
pabelangernew plan would be to have controller and compute node14:59
pabelangerthen, run tempest some place14:59
*** krotscheck is now known as kro_afk14:59
mfischok14:59
EmilienMcrinkle: I can run the jobs again so we can make sure it pass centos7 at least15:00
crinkleEmilienM: okay15:00
crinkleEmilienM: we could just wait till the packaging issue is fixed15:00
pabelangerAFK, getting drink15:00
EmilienMcool15:00
EmilienMcrinkle: lol I hope you don't want to wait for trusty15:00
mwhahahaanother point of multinode is that no one runs single nodes in production15:01
crinklewhat's the rush?15:01
EmilienMcrinkle: see my recent patches in infra, I disabling voting for all jobs that are failing15:01
mwhahahaso we should try and reproduce at least a little separation15:01
EmilienMcrinkle: there is no rush I guess, we just want to make progress and iterate our work15:01
pabelangermwhahaha: exactly15:03
crinkleright but progress in writing tests that don't give us any useful information because of packaging issues15:03
*** kro_afk is now known as krotscheck15:04
EmilienMcrinkle: I'm trying to keep centos7 green15:05
nibalizerpabelanger: I think mulitnode is working, and not puppet's job to show that it is or not15:05
EmilienMcrinkle: I consider ubuntu unstable now15:05
nibalizerbut yea a 2 node multinode test with separte controller I am in favor of15:06
pabelangernibalizer: working where?15:06
EmilienMnibalizer: controller, compute, network, etc15:06
pabelangerHow did packaging for distros get decided to gate on?15:07
pabelangereg: rdo, ubuntu15:07
vinshI think it was just pulling latest. right?15:07
nibalizerpabelanger: nodepool multinode testing works? clark has had multinode devstack things going for a while15:07
crinklewe test with packages from distros because those are the packages we support15:08
pabelangercould you consume debian packages, from debian15:08
pabelangerfor the gate15:08
crinklewe would still need to test ubuntu15:08
EmilienMright, ubuntu:UCA and centos7:RDO15:08
EmilienMcrinkle: right, but during the period where packages are really broken for trusty, I'm not sure we should bump to liberty or keep jobs voting15:09
EmilienMliberty or trunk15:09
EmilienMbecause it's blocking our CI15:09
crinkleCI is there to be blocked15:09
crinkleif we're going to ignore the tests then why are we writing tests15:09
EmilienMand keeping a backlog of bunch of current patches, is I think a mess to manage15:09
pabelangerIt would be worth the effort to get RDO to move packages upstream15:09
EmilienMcrinkle: we are not going to ignore15:10
EmilienMcrinkle: what is happening now, is, we are ignoring trusty vote for 4 modules15:10
EmilienMwe still test them, and still use centos7 to vote15:10
pabelangerRight having downstream packages block our CI, a big issue. Especially if downstream never signed off to support them for us.  How we notify them of issues and SLA for fixes?15:11
EmilienMcrinkle: this is an issue we will have in the next cycle again15:12
EmilienMcrinkle: people want to push features in puppet modules to test trunk - how can we test it if we don't download trunk packages?15:13
pabelangernibalizer: Yes, starting with a 2 node setup should be fine.  I actually have some JJB code up for an experimental job15:13
pabelangerhttps://review.openstack.org/#/c/215696/15:13
*** richm has quit IRC15:14
crinklepabelanger: but these are the packages we support, we write the modules based on decisions these packagers have made, so we need to run CI with them because that is how we're developing15:14
EmilienMcrinkle: "run the CI with them" > can you explain? Like keeping synced with their status, etc?15:15
crinkleEmilienM: i mean what we're doing now15:15
EmilienMI'm not sure Ubuntu folks is willing to help15:15
crinklewe install their packages in our ci15:15
EmilienMhonestly15:15
pabelangercrinkle: Right, I don't dispute that. How do you guarantee ubuntu will ever get a fix?15:15
EmilienMthey don't even reply to my IRC pings15:15
pabelangerOr are you expecting somebody on this team to do the package fix?15:15
crinklepabelanger: so we should stop supporting ubuntu in our modules?15:15
pabelangerno15:16
EmilienMno15:16
pabelangerI want to understand who at ubuntu is listening to us report problems15:16
EmilienMI just feel we spend too much time on packaging issues15:16
mfisch+115:16
nibalizerEmilienM: whats the alternative15:16
EmilienMI have to feeling we are making the job of packaging folks15:17
mfischyou cannot guarantee a fix nor can you guarantee that they wont break again in M15:17
EmilienMbecause we 1/ install trunk packaging 2/ have a gate - they don't have 2/15:17
EmilienMnibalizer: well, I see two 3 options, maybe there is more, tell me:15:17
EmilienM1/ use stable/* until OpenStack is released. So we are not pushing any trunk feature during the cycle15:18
EmilienMthis is not the best option, people will fork our modules and push their features15:18
claytonis there any existing tooling for generating reports for gate tests passing?15:18
mfischhave we tried a discussion with the ubuntu packagine guys?15:19
EmilienMmfisch: LOL15:19
pabelangerRight15:19
claytonwe could make ubuntu non-voting and consider putting back if it passes regularly15:19
EmilienMthat's my proposal in 3/ but I'm not here yet15:20
crinkleit seems slightly backwards to me to expect stability from something that hasn't been released yet15:20
EmilienMwell, if they had a CI that would be at least usable15:20
EmilienMwe can't even do apt-get install heat today15:20
mfischI agree not to expect stability but ignoring bugs that are filed is frustratin15:20
anteayacrinkle: that is a good point15:21
EmilienMeven if OpenStack is bugged, we should at least being able to do apt-get install heat15:21
mwhahahaimo, we should have the previous openstack package used for voting jobs and the current (dev) packages there but non-voting15:22
pabelangerYa, I don't have a good solutions. But this should all work under kilo right?15:22
pabelangerotherwise, people have broken production systems15:22
EmilienMso another option is 2/ use trunk and use zuul to vote or not if package is working or not15:23
EmilienMand keep at least one distro voting15:23
EmilienMso we make sure we continue to use beaker jobs part of our voting system15:23
pabelangerYa, I think we need to atleast reach out to RDO and ubuntu about us wanting to consume there packages for CI.15:25
pabelangerMaybe some point of contact to keep abreast of changes15:25
EmilienMI invited Ubuntu packagers to join here15:26
EmilienMso we can discuss15:26
EmilienMJames Page is about to join probably15:26
*** jamespage has joined #openstack-sprint15:27
EmilienMjamespage: welcome here, we are in the first day of Puppet sprint15:27
mwhahahaxarses did have this mail back in july which seems relevant to this conversation again http://lists.openstack.org/pipermail/openstack-dev/2015-July/069123.html15:27
EmilienMjamespage: we are talking about our current issues with Liberty packaging15:27
EmilienMand the alternatives for the next cycle15:27
EmilienMjamespage: we have two options, use stable packaging until openstack is released (this is not the best option, people will fork our modules and push their features that we could not test upstream  beacuse of packaging)15:28
EmilienMand the second one is use trunk packages (liberty staging in your case) and disable voting if packaging is really unstable15:29
EmilienMbut keep at least one distro voting15:29
EmilienMso we can actually test our patches15:29
*** spredzy has joined #openstack-sprint15:29
jamespageEmilienM, its sounds like you need some level of stability around the packaging; *staging* is not the right area todo that with15:29
pabelangerso, we are thinking about lunch in 30mins. Does that break work for people?15:29
EmilienMjamespage: you were the one who suggested me staging if I wanted to run liberty15:30
*** richm has joined #openstack-sprint15:30
jamespageEmilienM, that's cause you wanted it before we had done any testing on liberty-115:30
jamespageit was the only place we had it right then15:30
EmilienMjamespage: do you run a CI that deploy your packages and test OpenStack ?15:30
jamespageEmilienM, we do and we know that right now things are broken; and we are working to fix that15:31
jamespageEmilienM, in staging15:31
mfischjamespage: welcome15:32
jamespageonce we're happy with the packages in staging, they get promoted to proposed and then updates15:32
jamespagemfisch, hey!15:32
EmilienMjamespage: when is that happen?15:33
mfischjamespage: I think one of the frustrations is how we can point out packaging issues to you, I'm also happy to submit patches time permitting, but bugs soemtimes end up lost in the shuffle15:33
EmilienMhttps://etherpad.openstack.org/p/puppet-liberty-blocker15:33
EmilienMthey have been for 1 month and a half ^15:33
mfischI think LP bugs are easily lost15:34
*** med_ has joined #openstack-sprint15:35
EmilienMcrinkle, nibalizer: going back to alternatives, what do you suggest?15:35
jamespageEmilienM, when went offline a few weeks back you only had one outstanding issue15:35
jamespageEmilienM, I see new issues there due to some problems we've had pushing liberty-2 in to the cloud archive15:35
EmilienMthat's not true, Heat, Ironic and Glance are broken for a while15:36
nibalizerclayton: I made this which is a  hack http://spencerkrum.com/openstack/ci/15:36
EmilienMpabelanger: ^15:37
nibalizerclayton: anteaya runs things against elasticsearch15:37
mfischGuys, if we start from the idea that both of us, puppet & Ubuntu cloud can gain from working together we'll have a better conversation15:37
mfischWe're testing packages all the time and can help find bugs15:37
med_+115:38
EmilienMmfisch: yeah but until now, it's 1/ taking a lot of time who actually dig in beaker failures and report bugs to Launchpad 2/ block our CI15:38
mfischYes we have some issues15:38
EmilienMI can't handle 1/ in the next cycle anymore15:38
EmilienMand I don't want 2/ neither15:38
mfischI think we have 2 main concerns: which repo can we use and how do we better communicate issues and status back & forth15:38
jamespageEmilienM, we need to switch your CI away from staging and to either proposed or updates15:39
mfischjamespage: sorry if I missed this but is there a liberty proposed/updates?15:39
jamespageEmilienM, right now you're getting stung by some transitory packaging issues due to the way we have to managed the backport to the cloud archive15:39
jamespagemfisch, yes15:39
EmilienMjamespage: we won't test current openstack15:39
EmilienMie liberty15:39
mfischwe need a balance between "really new" and "working"15:40
jamespageagreed15:40
EmilienMyeah15:40
EmilienM+1 for working15:40
EmilienMwe don't mind having last commit from openstack15:40
EmilienMbut a package dated from one month could be acceptable15:40
mfischEmilienM: what about liberty/updates/proposed liek jamespage said?15:40
jamespageEmilienM, to give you an idea, libery-1 was ~90 packaging touchpoints and took about 2 weeks to process15:40
jamespagethat's just getting the packaging bits done and through process15:40
EmilienMjamespage: so you're suggesting to use proposed/updates only - 1/ since when could I use this repo to install liberty? 2/ is it tested enough and does it work today?15:41
pabelangernibalizer: interesting15:41
pabelangernibalizer: oops15:41
pabelangerEmilienM: interesting15:41
jamespageEmilienM, proposed currently contains the b1 packaging that I think was 'best'15:41
jamespageEmilienM, staging is volatile while we work through the last of b2 - and then we'll run straight into b315:42
jamespageunfortunately people like to take holiday15:42
EmilienMok so the proposal is to use liberty/updates/proposed as soon as it's available15:42
EmilienMjamespage: I like holidays too ;-)15:42
mfischliberty-b1 is probably okay15:43
EmilienMand you say liberty/updates/proposed should work15:43
jamespageEmilienM, I think that will give you a better balance between current/stable15:43
EmilienMok15:43
crinkle\o/15:43
EmilienMso I propose, in the next cycle, we bump our CI as soon liberty/updates/proposed is here15:43
mfischhate to bring this up but what will we do after L ships15:43
jamespageEmilienM, to the best of my knowledge yes - we're just starting on the next round of testing and updates for the juju charms we use to deploy openstack for our CI15:43
jamespagewe will find problems15:43
mfischwill we have m1 by the time we branch liberty to stable?15:44
crinklemfisch: we can just wait a bit15:44
mfischmakes sense to me15:44
EmilienMyeah15:44
EmilienMliberty/updates/proposed is out, wait a bit (run some tests if we want and see how it behaves)15:44
EmilienMexactly what we did but for staging...15:45
mfischjamespage: so about communication, I'm not sure LP is working very well for packaging bugs. besides emailing you whats the best way to communicate with ubuntu cloud team?15:45
jamespagemfisch, actually that's something we're working on right now15:45
EmilienMgood to know15:45
anteayajamespage: do you have any expectation of having more people helping with packageing?15:46
jamespageLP bugs can be a fire hose, so we've built out a specific openstack view to make that easier for triage etc...15:46
mfischjamespage: perfect15:46
anteayajamespage: as in more that just you and zul?15:46
mfischthat keeps them from getting lost15:46
jamespageanteaya, my packaging team is three - coreycb, zul and me15:46
anteayajamespage: yay15:46
anteayaI have yet to meet you15:46
jamespageanteaya, we have open headcount to general increase engineering resources in the team15:46
anteayawill you and coreyb be at summit?15:46
anteayaI'm expecting to see zul there15:46
jamespageanteaya, and just to be clear we don't *just* do packaging15:46
anteayaah sorry15:47
EmilienMjamespage: can you point me an example of apt config file to run liberty/updates/proposed please?15:47
jamespageI'll be their - ditto corey15:47
anteayamy mistake15:47
anteayathanks for the clarification15:47
mfischdeb http://ubuntu-cloud.archive.canonical.com/ubuntu trusty-updates/liberty main?15:47
jamespageEmilienM, yes one second15:47
anteayaand yay for more folks joining your team!15:47
anteaya\o/15:47
EmilienMsounds like we have a plan15:47
jamespagemfisch, that sounds right15:47
jamespagemfisch, also need the ubuntu-cloud-keyring package installed15:47
mfischyep15:47
jamespageEmilienM, stick with trusty-proposed/liberty for noew15:48
anteayajamespage: I look forward to meeting you and coreyb15:48
EmilienMjamespage: ok, I'm about to patch our CI to run trusty-proposed/liberty today and see how it behaves. If CI works, I'll ask for our team to approve patches15:49
EmilienMcrinkle, mfisch: does it work for your?15:49
EmilienMyou*15:49
jamespageEmilienM, ack15:49
crinkleEmilienM: yep this seems like the best solution15:49
mfischEmilienM: I didnt try I was basing it on my kilo entry15:49
EmilienMcrinkle: cool, we have a consensus15:49
EmilienMI honestly hope trusty-proposed/liberty is working :)15:50
mfischjamespage: whats the official status of openstack-client? still universe?15:51
jamespageEmilienM, fingers crossed - like I said we're focussing on updating charms for b2 once we have the packaging complete and to a certain level15:51
jamespagemfisch, nope main15:51
mfischjamespage: awesome, its a cornerstone of some of the puppet needs for api calls15:52
mwhahahahey for bug triaging, can I have access to update bug's severity? or should I just update the etherpad and someone else can do i t15:54
mfischmwhahaha: we're going to discuss them as a group15:55
EmilienMmwhahaha: I would sync with mfisch15:55
mfischsomeone here will have access I think I do15:55
mwhahahaK i'll make notes in the etherpad15:55
mfischmwhahaha: some of those bear discussions15:56
mwhahahayup15:56
mfischespecially the first 215:57
mfischthe keystone ones can be looked at now15:57
med_is audio still audiating?15:57
EmilienMcrinkle: I think I'm about to submit a patch to openstack_extras to support this use case, wdyt?15:58
jamespageEmilienM, the init-system-helpers problems should be resolved for keystone - I'm just looking at other potential impacts15:59
jamespagesahara and manila where impacted as well15:59
HunnerMorning!15:59
EmilienMjamespage: ok - I'm waiting for your apt config to download liberty so I'm sure I'm doing well16:00
Hunner_ody: In the office yet?16:00
nibalizerHunner: o/16:00
nibalizerHunner: _ody said he'd be in the office around nine16:00
nibalizeri'll be heading down there in a few minutes16:00
jamespageEmilienM, "deb http://ubuntu-cloud.archive.canonical.com/ubuntu trusty-proposed/liberty main"16:00
jamespageand install ubuntu-cloud-keyring package16:01
Hunnernibalizer: YAAAAY16:01
EmilienMjamespage: and http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/trusty-updates/liberty/ ?16:03
jamespageEmilienM, you only need updates or proposed, not both16:04
EmilienMok16:04
EmilienMjamespage: thanks, I'll use proposed then16:04
jamespageEmilienM, yes as updates has nothing liberty in it yet16:04
EmilienMpabelanger, anteaya and I are afk for lunch - happy hacking :-)16:05
_odyHunner: Yeah.  Just came looking for you.16:07
_odyI am in the kitchen now.  Need some espressso.16:07
mfischare we on break?16:14
_odyHunner: I am in "Big Bird"...I booked the room rest of the week so we do not have to move around if we don't want to.16:18
Hunner_ody: okay, over in a minute16:18
_odynibalizer: That is the room right behind you if you get to our front desk.16:19
mfischHunner: I hope you have your ginormous glass of milk there16:23
Hunnermfisch: I wish :(16:24
crinkleEmilienM: can't we just pass in $source_hash?16:25
jamespageEmilienM, the glanceclient problem should not be impacting the liberty proposed  UCA pocket16:41
jamespageas it does not have the newer urllib3 yet16:41
Hunner_ody: https://docs.google.com/spreadsheets/d/1Q8Db0jrv3dZQ8FbKhwpIjJRkI29Q571-FGs6VD_mVSQ/edit#gid=67753167916:59
EmilienMjamespage: ok perfect17:01
EmilienMcrinkle: you're right :) I'll test17:01
Hunner_ody: https://tldrlegal.com/17:17
EmilienMcrinkle: so I still need to patch manifests/repo/debian/ubuntu.pp17:28
EmilienMcrinkle: to benefit the "installing ubuntu-cloud-keyring" exec17:28
pabelangerjust added info about puppet unit test refactor, if people want to review17:34
anteayaI'm back17:34
EmilienMpabelanger: lgtm17:40
EmilienMit will save nodes consumption17:41
EmilienMand time17:41
EmilienMcrinkle: https://review.openstack.org/21980017:44
EmilienMand https://review.openstack.org/21980417:49
EmilienMrichm: I know you there is a lot of pending patches for keystone v3 work - please go ahead here if you need reviews17:52
nibalizerpabelanger: EmilienM how can I help with the CI progresS?17:52
EmilienMnibalizer: pabelanger is figuring out to consolidate unit test jobs in 2 jobs17:52
EmilienMmaybe without a bash script would be cool17:53
nibalizersweet17:53
EmilienMusing JJB17:53
EmilienMone job (voting) to test the versions that we *support*17:53
EmilienMone job (non-voting) to test the latest version that we *don't support yet* but will17:53
nibalizerokay17:53
EmilienMnibalizer: see https://etherpad.openstack.org/p/puppet-openstack-CI17:53
nibalizerthese two got created: http://git.openstack.org/cgit/?q=beaker I might build those out17:53
EmilienMpabelanger and nibalizer can work on this together17:53
nibalizerok17:54
nibalizerthese are the rspec unit tests?17:54
EmilienMnibalizer: yes17:54
nibalizerok17:56
pabelangerwhat is beaker-nodepool?17:56
EmilienMnibalizer: good question ^17:57
nibalizerso those are two repos I created17:58
nibalizerwith no content, thinking that we would create beaker hypervisors for our use17:58
nibalizerone, beaker-localhost, would standardize and extend the pattern we are using where beaker sshes into itself17:58
nibalizerthe other, beaker-nodepool would use either the extra ips in /etc/nodepool for multinode or the (soon to be) nodepool rest api17:58
claytoninteresting, you have more details on the nodepool rest api?17:59
EmilienMjamespage, crinkle: it works18:03
EmilienMhttps://jenkins02.openstack.org/job/gate-puppet-keystone-puppet-beaker-rspec-dsvm-trusty/149/console18:03
mfischmwhahaha: I have some time to do triage now if you want to dive in we can do it in the main channel18:03
EmilienMfor keystone18:03
mwhahahamfisch: i've got a meeting for 30 right now, you free in 30?18:03
EmilienMmfisch: can you review https://review.openstack.org/#/c/219800 please ?18:04
mfischmwhahaha: probably free but only until 2pm my time18:04
mwhahahalooks like my meeting is canceled18:04
mwhahahaso let's do this18:05
mfischEmilienM: looking18:05
mfischEmilienM: its possible to be something besides updates and proposed too but not currently that way18:06
mfischsince you dont enforce that, +218:06
mfischmwhahaha: I'd like to talk about: https://bugs.launchpad.net/puppet-keystone/+bug/147709318:07
openstackLaunchpad bug 1477093 in puppet-keystone "Creating Keystone users with a password in the puppet module (Kilo) throws error at second puppetrun" [High,New]18:07
mfischbecause its hit me too18:07
mfischits not necessarily the 2nd puppet run18:07
mfischbut if you source an admin openrc and run puppet it blowsup18:07
mwhahahaSo does it happen with a normal puppet run (not manual)18:08
mfischunsure18:08
mfischlet me see if I can repro it in my env18:08
mfischI was using a test env18:08
mfischit blows up trying to get a token18:09
mfischalthough the password is 100% right just like he says18:09
mwhahahaand the password is right in the openrc?18:09
mwhahahaos it's a priority thing18:09
mfischos?18:09
mwhahahawhere openrc > parameters18:09
mwhahahas/os/or18:09
mfischexcept the password didnt change18:09
mfischopenrc password matches puppet18:09
mfischI had them both set to something like "admin_pass"18:10
mfischat first I was seeing the same "2nd run" until I dropped the shell and came back and it went away18:10
mfischI asked Van in email about this and this is what he said18:10
mfisch"Found a “sort of” solution.18:10
mfischIf you set "replace_password => false” it will not complain."18:10
mfischso that implies to me maybe that puppet is trying to change the password even if its the same?18:11
mfischcan you repro?18:11
mwhahahathe provider can't know if the password is the same18:12
mwhahahaso i bet it's that18:12
mfischright18:12
mfischI cannot repro using our code but we're on master18:12
mfischI do see it on stable/kilo18:13
mwhahahai bet it's the auth url18:14
mfischoh18:15
mwhahahahttps://ask.openstack.org/en/question/68565/error-openstack-the-resource-could-not-be-found-http-404-openstack-user-list/18:15
mfischin openrc auth url is port 500018:15
mfischdoes puppet use 3535718:15
mfischlet me repro and see18:15
mfischI can make it happen18:15
mwhahahait may also be a v3 vs v2 issue18:15
mfischlet me unset OS_AUTH_18:15
mfischyes18:15
mfischrepro18:16
mfischError: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user[neutron]: Could not evaluate: Execution of '/usr/bin/openstack token issue --format value' returned 1: ERROR: openstack The resource could not be found. (HTTP 404) (Request-ID: req-2d82845c-a89e-4d6a-b5ab-fa506b71bf6d)18:16
mfischI will clear OS_AUTH_18:17
richmEmilienM: re: pending reviews for v3/federation18:17
richmhttps://review.openstack.org/#/c/208054 - latest patch hacks the wsgi class to add a custom fragment to the conf - this seems wrong, but I'm not sure how else he can do it18:17
richmhttps://review.openstack.org/#/c/218044/ - not quite ready18:18
richmhttps://review.openstack.org/#/c/218059/ - not quite ready18:18
EmilienMcrinkle, richm, mfisch: so the bump to trusty-proposed/liberty worked for puppet-keystone: https://review.openstack.org/#/c/219804/18:18
mfischmwhahaha: confirmed, no repro with OS_AUTH_URL unset18:18
mwhahahaso did openrc have a v2 url?18:19
mfischyes I only use V218:19
mfischfor everything18:19
mwhahahaand our provider assumes v318:19
mfischbut V2 works18:19
mfischkeystone token-get works18:19
mfischI was thinking it was a port issue18:19
mfisch5000 vs 3535718:19
mfischlet me try that18:19
richmhttps://review.openstack.org/#/c/218281/ - please review the spec/bp18:19
EmilienMrichm: ok18:20
EmilienMrichm: I dropped a comment this moring already18:20
richmhttps://review.openstack.org/218688 - not quite ready18:20
richmEmilienM: yes, I saw that - thanks18:20
richmhttps://review.openstack.org/219084 - please review18:20
mfischEmilienM: I thought we needed liberty/updates18:20
richmhttps://review.openstack.org/213957 - please review18:20
mfischmwhahaha: guess what, it works18:20
mfischwait nm18:21
mfischspoke too soon18:21
richmhttps://review.openstack.org/202409 - not quite ready18:21
EmilienMhum18:21
richmhttps://review.openstack.org/#/c/219127/ - not quite ready18:21
EmilienM<jamespage> [12:04:39] EmilienM, yes as updates has nothing liberty in it yet18:21
EmilienM<jamespage> [12:00:52] EmilienM, "deb http://ubuntu-cloud.archive.canonical.com/ubuntu trusty-proposed/liberty main"18:21
EmilienMso I think I'm right using proposed18:21
mfischk18:22
EmilienMjamespage: ^18:22
EmilienMcan you confirm?18:22
mfischupdates is more stable right?18:22
richmhttps://review.openstack.org/#/c/176150/ - please review18:22
mfischmwhahaha: no repro with this: export OS_AUTH_URL=http://127.0.0.1:5000/v3/18:22
EmilienMmfisch: right18:23
EmilienMmfisch: updates will be the one to use when jamespage will finish to upload all liberty18:23
EmilienMat the end18:23
EmilienMiiuc18:23
mwhahahamfisch: but it repro's if OS_AUTH_URL is v2 right?18:23
mfischyes18:23
mfischno matter what port18:23
EmilienManteaya: https://docs.google.com/presentation/d/1-AS5rYOdq0OVulPHGP3-w5ns3Lz2i3QNBj8pvPQZLkI/edit#slide=id.p18:24
mwhahahamfisch: https://github.com/openstack/puppet-keystone/blob/master/lib/puppet/provider/keystone_user/openstack.rb#L10618:24
mwhahahai think it's that, we need to make sure the auth url is v318:24
mfischdoes OSC not work with v2?18:24
mwhahahabut we're using v3 creds against v218:25
mwhahahai don't think that works18:25
mfischv3 creds with a domain you mean18:25
mwhahahayea18:25
mfischokay well I consider this triaged18:26
mfischwe could at least toss a useful error18:26
mfischcan you add some comment too18:26
mwhahahasure18:26
mfischrichm: since you're here, have you seen the 3 v3 related puppet-keystone bugs? https://etherpad.openstack.org/p/puppet-openstack-bugtriage18:26
mfischmwhahaha: when you are ready I'd like to cover: https://bugs.launchpad.net/puppet-keystone/+bug/134429318:28
openstackLaunchpad bug 1344293 in puppet-neutron "[library] neutron.rb uses cached call to keystone provider and can end up raising nil exception or otherwise fail to find resources created during this run of puppet" [Undecided,Confirmed]18:28
mwhahahak let me look at that one18:30
richmmfisch: ok18:30
richmhttps://bugs.launchpad.net/puppet-keystone/+bug/148853818:30
openstackLaunchpad bug 1488538 in puppet-keystone "keystone_user_role provider should be `domain` aware" [Undecided,New] - Assigned to Vasyl Saienko (vsaienko)18:30
richmThis already works in Kilo - you specify keystone_user_role {'user::domain@project::domain'}18:31
mfischso thats a wont fix18:31
mfischcan you update?18:32
mwhahahaso i don't think 1344293 is valid anymore18:34
mwhahahaas it appears the affected code is no longer there18:34
mwhahahaat least for keystone18:35
mfischlet me look18:35
mfischsorry boss came by18:35
mwhahahabosses always getting in the way :D18:36
richmhttps://bugs.launchpad.net/puppet-keystone/+bug/148801818:36
openstackLaunchpad bug 1488018 in puppet-keystone "Does not work if v3 API variables are set in the environment" [Undecided,New]18:36
richmI think Gilles solution is "Don't set OS_URL, OS_AUTH_URL, or OS_IDENTITY_API_VERSION before running puppet"18:36
richmbut I've asked him for clarification18:37
mfischmwhahaha: can you ask bogdando if its still legit?18:37
mfischrichm: thanks18:37
richmso that also would be a Won't Fix18:37
mfischa comment with "heres the fix is that okay18:37
mwhahahamfisch: i can but i can tell you we don't have that code in 6.1 or 7.0 anymore18:37
mfischrichm: + marking incomplete18:37
mfischthey will expire off18:37
mfischmwhahaha: what about upstream in puppet-keystone18:38
mwhahahait's not in upstream either18:38
mfischrichm: do you have permissions to change prio?18:38
mwhahahathat's what i was refering too18:38
mwhahahathere's no tenant_hash anymore18:38
mfischmwhahaha: okay I will fix18:38
richmmfisch: For 1488018 ?18:38
mfischdone18:38
mfischrichm: for either of the 2 we've talked about, if you dont have permissions to mark as Incomplete or Wont Fix let me know18:38
richmmfisch: I marked 1488538 as Fix Released since the fix was already in Kilo18:39
richmI would like to wait for feedback from Gilles before doing anything with 148801818:39
richmhttps://bugs.launchpad.net/puppet-keystone/+bug/147063518:40
openstackLaunchpad bug 1470635 in puppet-keystone "endpoints added with v3 are not visible with v2" [High,Confirmed]18:40
richmThis is really a Keystone bug - they would need to change Keystone to allow v2 clients to list endpoints created with v318:40
richmwe've already worked around it in puppet-keystone18:40
richmmfisch: so I'm not sure how to dispose of this bug18:41
mfischokay let me catch up18:41
mfischhttps://bugs.launchpad.net/puppet-keystone/+bug/1488538 is fixed in kilo and master, yes?18:41
openstackLaunchpad bug 1488538 in puppet-keystone "keystone_user_role provider should be `domain` aware" [Undecided,Fix released] - Assigned to Richard Megginson (rmeggins)18:41
richmmfisch: yes18:42
mfischk18:42
mfischnow let me look at the other bug18:42
richmhttps://github.com/openstack/puppet-keystone/blob/stable/kilo/lib/puppet/provider/keystone_user_role/openstack.rb#L10818:42
mfischokay richm whats the work-around?18:42
mfischgood timing18:43
richmmfisch: the work around for the endpoint bug is that we always use the v2 api for endpoints in keystone_endpoint18:43
mfischmwhahaha: can you look at 1416701 and 1416638 while we do this? they are related18:43
mwhahahasure18:43
mfischrichm: can you comment to that affect and I will mark as Wont Fix18:44
mfischsigh, prod issue, back later18:45
richmmfisch: done18:46
pabelangershould have something here in 20mins for the puppet-unit refactor18:47
pabelangerjust testing now18:47
mfischEmilienM: lets talk about https://bugs.launchpad.net/puppet-keystone/+bug/142368518:47
openstackLaunchpad bug 1423685 in puppet-keystone "default service should be run under apache, eventlet will deprecated in K, Removed in M" [Wishlist,Triaged]18:47
mfischEmilienM: I thought we'd already agreed to change the default?18:47
EmilienMyeah !!!18:47
EmilienMso18:48
mwhahahamfisch: those two bugs are dupes, we should mark one the dupe of the other18:48
*** iurygregory has joined #openstack-sprint18:48
mfischmwhahaha: go for it unless you lack perms18:48
mfischand note on etherpad plz18:48
EmilienMmfisch: the plan is: deprecate it in liberty and drop it in M18:48
mfischEmilienM: do we need a BP?18:48
EmilienMno18:48
EmilienMwe already did it for liberty18:48
EmilienMmfisch: https://github.com/openstack/puppet-keystone/commit/6c827e49bddbc85875f77c3700b0ecaea58156ea18:49
EmilienMnibalizer: sorry for github url18:49
EmilienMmfisch: what I would love is to drop acceptance code that test eventlet asap18:49
EmilienMprobably early M18:49
nibalizerEmilienM: :)18:50
mfischEmilienM: so we can mark that fixed18:50
EmilienMmfisch: go ahead18:50
mwhahahamfisch: kicked it back to the reporter because i'm not sure if he's getting an openstack error or a puppet error about duplicated resource definition18:51
mfischmwhahaha: thanks18:51
mfischthat finishes keystone and now I really need to look at this issue lets take a 15 min break18:51
mwhahahak, go put out some fires18:51
EmilienMcrinkle: I though puppet-unit-latest was not voting, but it does (which is fine but I thought it was not the plan)18:53
crinkleEmilienM: i have no information18:53
crinkleEmilienM: i agree that is surprising18:54
nibalizerpabelanger: so where are we on splitting up those rspec tests?18:54
EmilienMcrinkle: well, it works now so... let's keep it voting until it fails and block us...18:54
mfischmwhahaha: I'm somewhat back19:07
mfischI wanted to discuss https://bugs.launchpad.net/puppet-glance/+bug/1393490 with mgagne but he's nothere19:07
openstackLaunchpad bug 1393490 in puppet-glance "Cannot manage Glance package version" [Low,Triaged]19:07
mwhahahawe may want to make that more general for all modules, not sure all of them support the ability to manage specific versions19:08
mwhahahasome do, some don't19:08
mwhahahai checked glance and it does allow for the python client version to be specified19:08
mwhahahabut not the api or registry19:08
EmilienMmfisch: I pinged him to join19:09
*** mgagne has joined #openstack-sprint19:09
mgagnehi19:09
mwhahahaas for https://bugs.launchpad.net/puppet-nova/+bug/1350966, i wonder how much effort we want to put into that since we were going to deprecate the rabbitmq config out of nova19:09
openstackLaunchpad bug 1350966 in puppet-nova "Incorrect dependency on puppetlabs-rabbitmq" [Undecided,Confirmed] - Assigned to Richard Raseley (richard-raseley)19:09
mwhahahamgagne: mfisch wanted to talk about https://bugs.launchpad.net/puppet-glance/+bug/139349019:11
openstackLaunchpad bug 1393490 in puppet-glance "Cannot manage Glance package version" [Low,Triaged]19:11
mgagneAFAIK, it got fixed since: https://github.com/openstack/puppet-glance/blob/master/manifests/api.pp#L250 https://github.com/openstack/puppet-glance/blob/master/manifests/registry.pp#L17919:12
mgagnemfisch: ^19:12
mfischokay can we mark it fixed?19:12
*** mrmartin has quit IRC19:13
mgagneyes19:13
mfischthanks19:15
mfischokay19:15
mfischmwhahaha: this nova one https://bugs.launchpad.net/puppet-nova/+bug/135096619:15
openstackLaunchpad bug 1350966 in puppet-nova "Incorrect dependency on puppetlabs-rabbitmq" [Undecided,Confirmed] - Assigned to Richard Raseley (richard-raseley)19:15
mfischis Richard R online?19:15
mwhahahawell based on the irc meeting we are going to deprecate the rabbitmq out of puppet-nova19:16
mwhahahaso we can fix it only to remove it next cycle19:16
mwhahahathe rabbitmq server config that is19:16
mfischokay can you comment to that and ask Richard to close if he agrees?19:16
mwhahahasure19:16
mfischokay I want to skip around some19:19
mfischthis one19:19
mfischhttps://bugs.launchpad.net/puppet-neutron/+bug/142058619:19
openstackLaunchpad bug 1420586 in puppet-neutron "neutron net-show and subnet-show don't scale" [High,Confirmed]19:19
mfischthis is one I filed19:19
mfischthis one is so bad that we gave up managing networks and subnets with neutron19:19
mfischinstead I exec a python script if needed19:20
mfischits horrible but this was adding 45 minutes per deploy to us19:20
mgagneBTW, I'll switch development focus on LP to liberty instead of kilo19:21
mfisch+119:21
mfischmwhahaha: maybe there's not much to say here19:22
mwhahahasec still writing something19:22
mfischk19:22
mwhahahabut if it doesn't scale the question would be how many people at scale are using the functions that require net-show/subnet-show19:22
mfischand it runs on every control node we have19:23
mfischYou can disable it by removing all the nets and subnets which is what we did19:23
mwhahahahmm19:23
mwhahahalet me look19:23
mfischdorman and kris arent here I'd ask them19:23
mfischFuel may have the same issue19:23
mfischif you do a rolling delpoy like us its more painful19:23
mwhahahawe don't :D19:24
mfischbut the catalog runs were taking 15 minutes per control node, over half was this19:24
mwhahahasince we don't run puppet constantly it doesn't really affect fuel19:24
claytonmwhahaha: it's pretty horrible, it does net-list and then net-show one every network and subnet19:24
claytonand no token caching, so it takes a few seconds per network and subnetwork19:25
mfischwhen we deploy we do 1 control node, then the next, then the next19:25
mgagnedone19:25
claytononce you have a few hundred of those, it's untenable.19:25
claytonmy conclusion was that we were the only people using this with more than a few networks19:25
mfischthen lets leave this as-is19:26
mfischtriage wise19:26
mfischI will note the work around19:26
mwhahahais there a way to make it optional?19:26
mwhahahalike don't do that unless someone is explictly managing networks19:27
claytonmwhahaha: I believe neutron actually returns all the information it needs from the query in the net-list, it's just not displayed by default19:27
mfischcan the openstack client handle -f?19:27
mfischfor show certain fields?19:27
claytonI suspect a clever person could fix this with neutron net-list -c -c -c or openstack client19:28
claytonI didn't know about telling neutron to display extra fields when we replaced this with a python script and I didn't have the time to wedge openstack client into the neutron module19:28
claytonin fact, you need to do this - neutron net-list -c id -c name -c admin_state_up -c provider:network_type -c provider:physical_network -c provider:segmentation_id -c router:external -c shared -c tenant_id19:30
mfischdoes the provider use os-client?19:31
mfischor neutron19:31
mfischneutron right?19:31
claytonneutron19:31
claytonthat's all the attributes that the neutron_network provider cares about19:32
pabelangernibalizer: have something local, but ran into an issue with zuul19:33
pabelangertrying to figure out solution19:33
mfischclayton: sounds like we could fix this since we're the only ones who are hit by it19:36
claytonsure, not sure we have much incentive to at this point though19:36
nibalizerpabelanger: okay19:36
nibalizerpabelanger: let me know how I can help19:36
claytonI'm willing to in theory, but in practice it's not very high up the list of things to do19:36
mfischagree19:36
mfischlets move on19:36
mfischworking up19:37
mfischhttps://bugs.launchpad.net/puppet-neutron/+bug/134427119:37
openstackLaunchpad bug 1344271 in puppet-neutron "neutron::server::notifications settings are actually required" [Undecided,Incomplete]19:37
mwhahahai'm sure that one because we would pull in neutron::server separately from that class19:39
mwhahahabut that's my guess from the description, i'll have to look at the code19:39
mwhahahamay not be valid anymore19:40
mfischok19:40
mfischit was marked incomplete in June without update19:41
mfischit should be closed19:41
mfischmwhahaha: any objections?19:42
mwhahahaGo ahead and close it and i'll look into it later and see if it should be reopened19:42
* mfisch is following ubuntu bug control policy here ;)19:43
mfischmoving up again19:43
mfischhttps://bugs.launchpad.net/puppet-neutron/+bug/147420419:43
openstackLaunchpad bug 1474204 in puppet-neutron "vlan id are invalid" [Undecided,New]19:43
mfischis this really a puppet bug?19:43
mwhahahaoh going back to that previous bug19:44
mfischsure19:44
mwhahahaiit's saying that the values have been set to true but unless you include it, neutron won't have the nova creds19:44
mwhahahai think it's still valid19:45
mfischok19:45
mwhahahabecause the default is now true but we don't provide the nova creds anywhere19:45
mfischcan you reopen and add that comment?19:45
mwhahahasure19:45
mfischthe latter bug I moved to neutron19:45
mfischthey will likely close it19:45
mwhahahak19:47
mfischready for https://bugs.launchpad.net/puppet-neutron/+bug/148027719:47
openstackLaunchpad bug 1480277 in puppet-neutron "linuxbridge agent module in Kilo branch is broken" [Undecided,New]19:47
mfisch?19:47
EmilienMrichm: reviewing https://review.openstack.org/#/c/213957/13/manifests/endpoint.pp,cm - why not using undef?19:48
mwhahahawonder if that's a packaging issue19:48
mwhahahasimilar to the neutron_lbaas.conf missing19:48
EmilienMmwhahaha: no, crinkle did a patch for that19:48
mfischhe's saying the above folder DNE19:48
mfischthe containing folder19:48
mfischthis code is weird19:48
EmilienMhttps://review.openstack.org/#/c/216523/19:48
EmilienMoh sorry19:49
EmilienMit's not this bug19:49
mwhahahathat patch might be the fix19:50
mfischyou'd also need to change params.pp19:51
mfisch$linuxbridge_config_file    = '/etc/neutron/plugins/linuxbridge/linuxbridge_conf.ini'19:51
mfischsame for ubuntu and rhel19:52
mwhahahathat's not used19:52
mwhahahaprobably left over from a previous version19:52
mfischwell it is in 1 place19:52
mwhahahawhere19:52
mwhahahamy grep for linuxbridge_config_file returned only params19:53
mfischin manifests/plugins/linuxbridge.pp19:53
mfischlet me make sure I',m on the right tree19:53
EmilienMrichm: err, undef is actually an option, never-mind my comment19:53
mfischmwhahaha: err I dont see it all anymore on master19:53
mfischdo you?19:54
mfischI did a clean checkout19:54
mfisch^%$%$19:54
mfischmy remote was at stackforge19:55
mwhahaha:D19:55
mwhahaharight so the params.pp needs cleaning up and crinkle's patch should fix it19:55
mfischwhich?19:55
mwhahahahttps://review.openstack.org/#/c/216523/19:55
mfischI dont do much with puppet-neutron19:56
mfischmy contrib of the day19:57
mfischhttps://review.openstack.org/21986019:57
mwhahaha:D19:57
mfischmarking colleen's as a fix19:57
mfischmwhahaha: I have to go to the DMV now19:58
mfischI will tether in from there if there is a line19:58
mwhahahagood luck with that19:58
mfischvinsh: are yo uon?19:58
mfischyou on19:58
vinshhey19:58
EmilienMmfisch, mwhahaha: fyi, puppet-neutron CI is broken now, I'm unblocking it19:59
mfischvinsh: can you and mwhahaha do the swift bug triage?19:59
mfischhttps://etherpad.openstack.org/p/puppet-openstack-bugtriage19:59
mfischjust dig a bit, see if fixed/fixable update severity etc19:59
mfischno need to actually fix unles you want19:59
mfischif not I will catch up after I get my license plates19:59
vinshYeah, I could walk through those.  mwhahaha you have time?20:00
mwhahahasure, but let me take a 5 min break.20:00
mwhahahabrb20:00
vinshkk20:00
EmilienMclarkb: in case you missed mfisch's comment on https://review.openstack.org/#/c/216926/20:04
EmilienMdamn20:04
EmilienMsorry clarkb20:04
EmilienMclayton: https://review.openstack.org/#/c/216926/20:04
claytonah, I didn't see that.  replied.20:06
EmilienMif anyone can review two puppet-tempest patches: https://review.openstack.org/#/c/218467/ and https://review.openstack.org/#/c/218398/20:06
claytonI'm fine with adopting the same approach for clients also, but it's less clear what the solution for that would be.  For services its' easier, since they access the filesystem in pretty predictable ways20:07
claytonI've been focused on the service side part of this.20:07
mwhahahavinsh: i'm back20:11
vinshRight on, first on the list 146670720:11
vinshSounds like a doc update would help20:14
mwhahahayea20:14
vinshWell, I can take that one on20:15
vinshi'll make note to have mfisch assign it to me when he gets back20:15
vinshnext20:16
mwhahahahttps://bugs.launchpad.net/puppet-swift/+bug/129143320:16
openstackLaunchpad bug 1291433 in puppet-swift "swift storage services can start before databases are rsync'd" [Undecided,Incomplete] - Assigned to David Moreau Simard (dmsimard)20:16
vinshI feel like that one is no longer an issue... ive never come close to hitting it :)20:18
mwhahahaso i wonder if there's a way to do the check optionally20:18
mwhahahait is quite old20:18
mwhahahamight be worth at least documenting a work around20:19
vinshwhats up with all these launch pad patches and not a gerrit review?20:19
mwhahahagood question20:20
mfischwhich one to you vinsh ?20:22
vinshfirst one. I made a note for you20:22
vinshon etherpad20:23
mfischlemme reconnect to ep20:23
pabelangerhttps://review.openstack.org/#/c/219866/20:23
pabelangernibalizer: ^20:24
vinshmwhahaha: They could probably just add "  File['/etc/swift/container.ring.gz'] ~> Class['swift::storage::container']" to their manifest to solve that20:25
nibalizerwhere does {puppet_unit_version} come from20:25
mwhahahavinsh: yea which is why i said it should just be documented20:25
EmilienMjamespage: see https://jenkins04.openstack.org/job/gate-puppet-openstack-integration-dsvm-trusty/43/console20:26
vinshmwhahaha: Ok, so read me update?20:26
EmilienM"WARNING: The following packages cannot be authenticated!"20:26
mwhahahavinsh: it seems to be something that you can't really solve in the module as it's more of a operator usage thing20:26
mwhahahavinsh: yea that would be good20:26
EmilienMjamespage: using 'proposed'20:26
vinshI'll take it on also, seems easy enough to document.20:26
nibalizerpabelanger:  where does {puppet_unit_version} come from20:27
pabelangerI'd like to get this one too: https://review.openstack.org/#/c/215696/20:27
pabelangernibalizer: the job-group below20:27
nibalizerpabelanger: is there a review containing run_multinode.sh?20:28
vinshmwhahaha: https://bugs.launchpad.net/puppet-swift/+bug/128963120:28
openstackLaunchpad bug 1289631 in puppet-swift "Do security hardening for /etc/swift" [High,Triaged]20:28
nibalizerI don't understand why the unit tests have an {ostype}20:28
nibalizerthat seems irrelevant to this change but why?20:29
mfischguys about to take my turn at DMV let me know updates later thanks vinsh and mwhahaha20:29
pabelangerhttp://paste.openstack.org/show/442385/ JJB output20:29
mwhahahak20:29
pabelangerya, ostype could be removed20:29
pabelangerfollowing patch could do that20:29
vinshmwhahaha: wow, never would have considered this one.20:31
mwhahahabecause it doesn't work20:31
mwhahahaand it's a bad idea? :D20:31
mwhahaharecurse doesn't work like that20:31
mwhahahaat at least i don't think it does20:31
vinshWell, not sure where to go with this one. I don't have time in the next 6 weeks to touch it.20:33
pabelangernibalizer: nothing yet, this just setups on the nodepool config20:33
pabelangerhave something locally I want to push up20:33
pabelangerI've been testing something with ansible-playbook --connection=local20:33
mwhahahai'd be concerned about the load on the system with recurse = true if it does actually manage all of the items in that folder20:33
pabelangerhowever, once we get the experimental job merged, we'll have a place to sandbox some of the nodepool functionality20:34
mwhahahathat one seems triaged, it just needs someone to actuallyt est it20:34
pabelangerwhich, is something I'd like right now20:34
vinshmwhahaha: the load on /etc/swift?20:34
vinshnot a lot in there.20:34
pabelangerotherwise, I have an internal nodepool I can play with20:34
pabelangerbut, would like people to monitor results20:34
mwhahahavinsh: on the constant checking of perms, the other issue is that really only addresses people who are running puppet constantly20:35
vinshchecking only on each puppet run20:35
claytonI think the advise you see against not using recurse has more to do with people doing it with directories that have a bazillion files20:35
vinshdoesn't seem like a big deal really..20:35
claytonI've done that before, it's horrible.  <100 files has never been an issue20:35
mwhahaharight it should be ok on small directories20:36
mwhahahabut how are we to know what a user puts in /etc/swift20:36
vinsh /etc/swift is >20 files20:36
vinshI mean <20:36
mwhahahait should be < 20 files20:36
EmilienMpabelanger: http://logs.openstack.org/18/219818/1/check/gate-puppet-openstack-integration-dsvm-trusty/8d00d68/console.html#_2015-09-02_20_25_04_53120:36
mwhahahadoesn't mean it is for all cases20:36
mwhahahaanyway back to the triage part, this bug seems fine as is. just needs someone to try it out i guess and make sure it doesn't do anything crazy20:37
vinshyep, add it to the backlog.20:37
*** rfolco has quit IRC20:38
EmilienMpabelanger: see https://review.openstack.org/#/q/status:open++branch:master+topic:beaker/trusty-proposed,n,z20:39
EmilienMall the same patches20:39
mwhahahaso moving on, https://bugs.launchpad.net/puppet-swift/+bug/148461420:40
openstackLaunchpad bug 1484614 in puppet-swift "make "log_requests" flag configurable, reduce account/object/container server logs" [Undecided,New]20:40
vinshAgain.. why didn't he put the review in gerrit? :)20:40
mwhahahawas just going to say that20:41
vinshI would assign the bug to him and let him propose one as he says he can20:41
mwhahahayes20:41
vinshI use rysyslog to log.. and I like lots of log data.20:41
vinshTo each his own :)20:41
vinshi'll comment in the bug20:42
mwhahahak20:42
mwhahahanext up https://bugs.launchpad.net/puppet-swift/+bug/148461220:42
openstackLaunchpad bug 1484612 in puppet-swift "enable configuration of [incoming|outgoing]_chmod in ::swift::storage" [Undecided,New]20:42
mwhahahasame guy20:43
mwhahahasame deal20:43
vinshI think this one is already proposed20:43
*** guimaluf has joined #openstack-sprint20:43
vinshhttps://review.openstack.org/#/c/217707/20:44
vinshkinda20:44
mwhahahayup20:44
vinshsuppose we assign it to danp20:46
vinshsince he is working it20:46
mwhahahasounds ok20:46
vinshmaybe he saw the report. maybe not. :)20:46
vinshoh the next one is assigned to me :-o20:47
EmilienMmfisch, crinkle: so here are the patches to use proposed repo - you have to know, horizon is broken, ironic too and glanceclient too. that's it - https://review.openstack.org/#/q/status:open++branch:master+topic:beaker/trusty-proposed,n,z20:47
mwhahahaoh it's a dupe20:47
vinshoh man. i must need coffee20:47
vinshlol yep.20:47
mwhahahaof the first one20:47
mwhahahawe'll just make that one go away20:48
vinshRight on.20:48
vinshswift is done/traiged. mfisch.20:48
vinshactions listed by each bug for you.20:48
EmilienMmfisch, crinkle: I updated https://etherpad.openstack.org/p/puppet-liberty-blocker so you can see the diff between the 2 repos20:50
crinkleEmilienM: ceilometer is broken too?20:58
EmilienMcrinkle: on centos only20:58
EmilienMcrinkle: https://bugs.launchpad.net/puppet-ceilometer/+bug/149098620:58
openstackLaunchpad bug 1490986 in puppet-ceilometer "centos: puppet run is not idempotent (collector & notif agents)" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)20:58
EmilienMhttps://bugzilla.redhat.com/show_bug.cgi?id=125907520:59
openstackbugzilla.redhat.com bug 1259075 in Package Review "Review Request: python-jsonpath-rw-ext - Extensions for JSONPath RW" [Medium,Assigned] - Assigned to rmarko20:59
EmilienMthis is a new dep missing20:59
EmilienMRDO guys are doing it right now at this time20:59
EmilienMpabelanger: https://review.openstack.org/#/c/21981821:02
pabelangerEmilienM: sent again please21:02
EmilienMcrinkle: can you also approve https://review.openstack.org/#/c/219393/ ?21:03
EmilienMpabelanger: https://review.openstack.org/#/c/21981821:03
EmilienMpabelanger: https://review.openstack.org/21980421:03
mfischhere now21:05
mfischno new plates just a stupid piece of paper21:05
mfischmwhahaha: vinsh thanks21:05
mfischlet me read the ep21:05
vinsh:)21:06
EmilienMcrinkle: I'm highly concerned by https://review.openstack.org/219818 failures21:06
EmilienMbecause trusty fails21:07
mfischmwhahaha: can we pick up the last few tomorrow?21:07
mwhahahayea sure21:07
EmilienMand it's not failing on puppet-* modules patches because it still adds liberty repo21:07
mwhahahai'm out of here in a few mins21:07
EmilienMthat means, if we merge my patches that use 'proposed' repo, it will break CI21:08
EmilienMsince https://review.openstack.org/219818 is failung21:08
EmilienMfailing*21:08
nibalizerEmilienM: pabelanger https://review.openstack.org/#/c/219879/ I think will add nonvoting puppet4 beaker tests21:09
EmilienMcrinkle: I'm gonna but depends-on on the openstack-integ patch21:09
EmilienMto make sure we don't break our CI21:09
mfischnice work guys, we only have 6 bugs for tomorrow21:10
crinkleEmilienM: :( my +2s disappeared21:12
EmilienMcrinkle: because I added Depends-On for safety21:13
EmilienMcrinkle: do you see why I added it?21:13
pabelangercrinkle: nibalizer: EmilienM: lets get https://review.openstack.org/#/c/217346/ merged so we can get a proper depends on for puppet-keystone on puppet-openstack-integration21:13
crinkleEmilienM: yes but :(21:13
pabelangerthen you don't have to manually add Depends-On jobs21:13
EmilienMcrinkle: you must be angry at me :-P21:13
EmilienMpabelanger: looking21:14
nibalizerpabelanger: I don't understand why we need 21734621:14
EmilienMcrinkle: I think 'proposed' is not installing keystone21:15
EmilienMif I'm right, it really sucks for us21:15
pabelangernibalizer: so EmilienM can add gate-puppet-keystone-beaker... gate on puppet-openstack-integration21:16
pabelangerwhich didn't work before because the clone was not properly done21:16
EmilienMcrinkle: but looking at http://logs.openstack.org/18/219818/1/check/gate-puppet-openstack-integration-dsvm-trusty/8d00d68/console.html#_2015-09-02_20_25_04_531 from https://review.openstack.org/#/c/219818/ - it looks like it's still broken21:16
nibalizerpabelanger: I don't understand, I'll need a bit more explanation (preferably in the commit message)21:17
nibalizeroh wait maybe I do understand21:17
nibalizerso gate-keystone will get the keystone sourcecode instead of the project sourcecode21:18
pabelangercurrently gate-*-beaker uses the jenkins workspace and git-prep21:18
nibalizeryep21:19
EmilienMthis is a canari job21:19
nibalizerokay I understand now21:19
pabelangerok21:19
nibalizerI'm not sure others would though, I would like you to explain it in the commit message please21:19
EmilienMhttp://goo.gl/tgtiqq21:19
EmilienMnibalizer, crinkle: we are off now21:21
EmilienMcrinkle: I'll look status of https://review.openstack.org/#/q/status:open++branch:master+topic:beaker/trusty-proposed,n,z - and see if we should go to 'proposed' repo or not21:21
EmilienMsee you later folks21:23
nibalizerokay21:23
nibalizerpabelanger: so f22/23 question21:23
nibalizerthe puppet, puppet4 package21:24
nibalizerdoes it use /etc/puppetlabs?21:24
* mwhahaha wanders off21:33
*** krotscheck is now known as kro_paternity23:34

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!