15:00:40 <anteaya> #startmeeting third-party
15:00:40 <openstack> Meeting started Mon Apr 20 15:00:40 2015 UTC and is due to finish in 60 minutes.  The chair is anteaya. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:41 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:43 <openstack> The meeting name has been set to 'third_party'
15:01:00 <anteaya> hello
15:01:02 <tristanC> Hello!
15:01:13 <anteaya> how are you tristanC?
15:02:00 <asselin_> hi
15:02:07 <anteaya> hey asselin_
15:02:14 <fbo> Hello !
15:02:20 <tristanC> well I'm interested by that openstackci third party convergence thing
15:02:23 <anteaya> fbo: hello!
15:02:24 * BobBall is here as well.
15:02:30 <anteaya> BobBall: hi there
15:02:33 <anteaya> tristanC: wonderful
15:02:33 <BobBall> For some definition of 'here'.
15:02:41 <anteaya> BobBall: I'll take what I can get
15:02:41 <asselin_> hi fbo BobBall.. got your messages
15:02:47 <asselin_> hi anteaya
15:02:50 <anteaya> asselin_: did you want to kick things off?
15:02:58 <BobBall> Good good.
15:03:05 <anteaya> asselin_: folks seem to want to hear from you, go ahead please
15:03:48 <asselin_> sure...sent an e-mail to the mailing list to get us operators on a common ci solution
15:04:01 <anteaya> who has the link?
15:04:04 * asselin_ looks up the link
15:04:07 * anteaya goes looking
15:04:36 <asselin_> ok while anteaya finds the link....
15:04:41 <anteaya> #link http://lists.openstack.org/pipermail/openstack-dev/2015-April/061929.html
15:04:57 <asselin_> we have an infra spec approved and some stories defined (based on the spec)
15:05:20 <anteaya> #link http://specs.openstack.org/openstack-infra/infra-specs/specs/openstackci.html
15:05:48 <asselin_> #link https://storyboard.openstack.org/#!/story/2000101
15:06:10 <asselin_> as mentioned in the post, I started with the "log server".
15:06:46 <BobBall> Log server is an interesting one...
15:06:58 <asselin_> I think the other modules should be able to follow similarly
15:07:04 <BobBall> Is the proposal that we have a log server that the combined CIs can use, that's quite similar to Infra's log server?
15:07:31 <asselin_> BobBall, no, the idea is to use the same puppet script to create multiple identical log servers
15:07:37 <anteaya> #link https://review.openstack.org/#/q/topic:downstream-puppet+status:open+owner:%22Ramy+Asselin+%253Cramy.asselin%2540hp.com%253E%22,n,z
15:07:42 <asselin_> each owned by their 3rd party operator
15:07:47 <BobBall> Ah ok
15:08:05 <anteaya> just common deployment structure
15:08:13 <BobBall> So by "common CI solution" do you just mean the deployment rather than a shared jenkins etc?
15:08:17 <anteaya> making debugging and support much much easier for everyone
15:08:37 <asselin_> BobBall, yes...shared puppet scripts.
15:08:43 * BobBall mis-understood
15:08:44 <BobBall> Shame.
15:08:54 <asselin_> BobBall, np...those are good questions
15:09:33 <BobBall> I was thinking we were aiming for a shared CI infrastructure rather than independent but deployed from the same set of scripts
15:10:06 <asselin_> So, in the case of the log server, lot's of operators setup their own, but then log files require downloads & no "log analyze" filtering capability.
15:10:36 <asselin_> BobBall, I think that would be great, but complicated from a security point of view.....
15:10:58 <BobBall> Indeed.
15:11:10 * anteaya doesn't even want to think about what would be involved in co-ordinating that
15:11:31 <asselin_> So the best is just to setup an 'identical' system....and make it easy to do so
15:11:53 <anteaya> I agree with that statement
15:12:15 <anteaya> or at least, easier
15:13:12 <anteaya> do we have more here?
15:13:13 <asselin_> Yes, so I created stories for each main component
15:13:25 <anteaya> you have many assigned to you
15:13:33 <anteaya> and there are some unassigned
15:13:57 <anteaya> asselin_: would you prefer someone just getting involved in the process help you with a current story you are working on or select their own?
15:13:57 <asselin_> I think if we divide and conquer, we can all be using the same scripts sooner rather than later
15:14:12 * anteaya likes asselin_'s vision
15:14:21 <asselin_> anteaya, I would prefer each to take their own, and we help out via reviews etc.
15:14:25 <anteaya> great
15:14:37 <fbo> asselin_, I would be happy to tackle the zuul one
15:14:49 <asselin_> anteaya, personally I think it's more satisfying to get 'credit' for something.
15:14:58 <anteaya> asselin_: it is
15:15:01 <anteaya> fbo: wonderful!
15:15:04 <fbo> but as said I will need some guidances
15:15:10 <devvesa> asselin_: I answered your mail this morning. I can help you too
15:15:14 <anteaya> fbo: you will get a lot
15:15:18 <asselin_> fbo, great. I will be happy to help.
15:15:19 <anteaya> devvesa: welcome
15:15:22 <fbo> cool !
15:15:35 <devvesa> thanks anteaya
15:15:51 <asselin_> devvesa, I didn't see your mail yet, but your help would be appreciated as well!
15:16:00 <anteaya> asselin_: would you like to do some work at summit on this? not asking folks to wait until summit to work of course
15:16:04 <devvesa> I am so busy this week, but next I will be more free
15:16:29 <tristanC> do we agree that the configuration for a third party ci should be done through a common (shared) system-config ?
15:16:37 <anteaya> asselin_: I'm thinking if we can get you a table to get some work done, and others can join you, what do you think?
15:16:41 <asselin_> anteaya, I would like to make progress first before the summit...
15:16:49 <anteaya> asselin_: of course
15:17:01 <anteaya> not asking anyone to wait until summit
15:17:04 <asselin_> anteaya, and then use the summit to deal with difficult issues that arise, if any
15:17:24 <BobBall> I don't have much time pre-summit TBH but would definitely be interested in a table / BOF at the summit
15:17:25 <anteaya> asselin_: so would a work space be useful perhaps? if yes, I should work on getting you a space now
15:17:47 <anteaya> since summit spaces are quickly filling up
15:18:03 <asselin_> anteaya, ok I see. then yes, let's get some space
15:18:23 <anteaya> #action anteaya to work on getting asselin_ a working space at summit
15:18:30 <anteaya> no promises but I will try
15:18:42 <asselin_> BobBall, that's ok. If you have time to review or test some patches, that would be appreciated as well.
15:19:01 <BobBall> That I can do
15:20:20 <asselin_> devvesa, did you mention in the mail anything specific you'd like to help with?
15:20:55 <devvesa> asselin_: no, because I haven't read your links yet
15:21:10 <devvesa> but I am so frustrated with our infrastructure that I'd help with anything :)
15:21:51 <anteaya> devvesa: which is your infrastructure?
15:22:25 <asselin_> ok, so devvesa fbo self assign the 'one' story you'd like to start with. After the meeting we can discuss it in more details.
15:22:27 <devvesa> some Barcelona's Datacenter machines with an old version of OpenStack
15:22:38 <anteaya> devvesa: :(
15:22:43 <devvesa> asselin_: ok!
15:22:50 <fbo> asselin_, do you think we can try to make openstackci module as generic as possible ? I mean by generic reusable in another context than openstack (for instance I want the stack zuul nodepool jenkins connected to my own gerrit, not the openstack one)
15:22:55 <fbo> asselin_, ok
15:23:10 <asselin_> fbo, absolutely
15:24:04 <asselin_> fbo, fyi, we have 3 ci's currently all pointing to different gerrit servers.
15:24:07 <fbo> asselin_, nice ! I'm sure it can be really helpful to lot of folks to have a it generic
15:24:43 <fbo> ok good to know
15:24:56 <asselin_> so it no more questions, we can move on, and discuss more on this topic offline
15:25:37 <anteaya> asselin_: early indications are a table is only available on Tuesday afternoon
15:25:46 <anteaya> so I requested one be booked for this work
15:25:56 <anteaya> and thank you so much asselin_
15:25:57 <asselin_> anteaya, thank you
15:26:05 <anteaya> you have worked so hard to get here
15:26:08 <anteaya> this is awesome
15:26:23 <asselin_> thanks
15:26:37 <anteaya> does anyone have any questions about their current infrastructure which they would like to discuss?
15:27:11 <BobBall> Well I'm struggling to understand why DEVSTACK_GATE_TEMPEST_REGEX doesn't work in mine :)
15:27:31 <anteaya> BobBall: what happens when you try it?
15:27:37 <anteaya> BobBall: have you a stacktrace?
15:27:44 <BobBall> Nah - nothing that helpful
15:27:46 <BobBall> Just does nothing
15:27:54 <BobBall> http://d7013eaae7e632dff837-028d11a4a642ead4d20755bd13d99a1b.r55.cf5.rackcdn.com/47/174247/4/passonly/dsvm-tempest-xen/f14a26e/console.html
15:27:57 <asselin_> BobBall, make sure you're exporting it correctly
15:27:58 <anteaya> yeah, nothing isn't terribly helpful
15:28:00 <BobBall> 14:45:39 sudo: unable to resolve host ds-xen-1429452840
15:28:00 <BobBall> 15:04:07 Removing sudo privileges for devstack user
15:28:00 <BobBall> 15:04:07 Cleaning up host
15:28:00 <BobBall> 15:04:07 ... this takes 3 - 4 minutes (logs at logs/devstack-gate-cleanup-host.txt.gz)
15:28:03 <BobBall> 15:04:26 Process leaked file descriptors. See http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build for more information
15:28:17 <BobBall> Exporting it in the same way I did for the other vars
15:28:20 <BobBall> export DEVSTACK_GATE_TEMPEST_REGEX="(?!.*test_volume_boot_pattern).*"
15:28:41 <anteaya> I think volume boot pattern is having a problem
15:28:46 <anteaya> am I correct?
15:28:54 <asselin_> BobBall, I don't think that's a valid regex
15:28:57 <BobBall> mine certainly is :)
15:29:01 * asselin_ looks up correct regex
15:29:03 <BobBall> Why not asselin_?
15:29:07 <BobBall> It's a negative lookahead...
15:29:26 <asselin_> BobBall, yes, but you need to specify what you want first, then negative look ahead
15:29:29 * anteaya doesn't know what a negative lookahead is
15:29:34 <BobBall> this worked: "grep -o -P "(?!.*test_cinder_qos_list)tempest[[:alnum:]._]*" regex_test"
15:29:43 <BobBall> when I was just doing a test
15:30:00 <akerr> anteaya: run everything that doesn't match the following
15:30:06 <anteaya> akerr: thank you
15:30:15 <BobBall> do I have to use a negative lookbehind rather than a lookahead then?
15:31:06 <asselin_> BobBall, ^(?=.*volume)(?!.*test_volume_boot_pattern).*
15:31:54 <akerr> BobBall: we do something similar for our internal tests: export DEVSTACK_GATE_TEMPEST_REGEX='tempest.api.volume.netapp.(?!test_netapp_ssc)'
15:31:55 <asselin_> BobBall, at least that what works for us: http://15.126.198.151/41/174941/2/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/d1cd391/console.html
15:32:00 <BobBall> So that would include all volume tests; does that actually exclude boot_pattern or not?  I'd assume not since the positive lookahead works fine?
15:32:36 <asselin_> BobBall, it runs all volume tests except those matching "test_volume_boot_pattern"
15:33:52 <BobBall> And does that regexp only run volume tests?
15:34:19 <BobBall> I'm just confused..
15:34:26 <BobBall> The test I've done here seems to do what I expected
15:34:38 <BobBall> Perhaps it's the perl-specific part that doesn't work
15:34:46 <BobBall> Does your regexp work with standard grep?
15:34:58 <asselin_> BobBall, didn't try....
15:35:17 <BobBall> OK - your regexp seems to be doing more than just volume, good
15:35:40 <BobBall> unless you have another one to add the basic scenarios :)
15:36:04 <asselin_> BobBall, just one regex
15:36:15 <BobBall> Seems to include tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario
15:37:03 <asselin_> yes....but now that you mention it...not sure why...
15:37:45 <asselin_> there must be "volume" somewhere
15:37:45 <BobBall> So you think I need (?=.*)(?!.*pattern).*
15:37:50 <BobBall> (shortened
15:38:52 <BobBall> Basically export DEVSTACK_GATE_TEMPEST_REGEX='^(?=.*)(?!.*test_volume_boot_pattern).*'
15:38:54 <asselin_> BobBall, but that will run everything
15:39:04 <BobBall> Surely everything except boot_pattern?
15:39:11 <asselin_> yes :)
15:39:11 * BobBall hates regexp
15:39:19 <BobBall> Good - that's what I need
15:39:43 * BobBall is willing to bet that it doesn't work though...
15:39:45 <BobBall> not much :D
15:39:48 <anteaya> any additional input on regexp?
15:40:24 <anteaya> okay thanks asselin_ and akerr
15:40:33 <anteaya> BobBall: I hope it works for you
15:40:40 <anteaya> let us know
15:40:47 <anteaya> let's move on
15:40:59 <anteaya> does anyone have anything else they would like to discuss today?
15:41:52 <tristanC> well I have a quick question, Are efforts to make puppet manifest works on fedora/centos aligned with the openstackci spec ?
15:42:18 <tristanC> or making system-config more generic the better priority effort ?
15:42:31 <asselin_> tristanC, first priority is to make it more generic
15:42:50 <asselin_> tristanC, after it's merged, then follow up patches and add new support
15:43:16 <asselin_> tristanC, I'd like to try to keep the refactoring separate from new functionality.
15:43:18 <anteaya> to be clear, system-config will still be consider openstack infra specific
15:43:24 <anteaya> considered
15:43:30 <asselin_> I think that's generally good practice
15:43:51 <anteaya> the work is to create a structure to consume the puppet modules that are already outside of system-config
15:44:03 <anteaya> so a generic config repo that isnt' system-config
15:44:13 <tristanC> asselin_: alright, sounds like a plan
15:44:28 <anteaya> then after this is done, infra is planning on consuming the same new generic config file/repo
15:44:40 <anteaya> and reducing infra usage of system-config
15:44:45 <asselin_> anteaya, yes
15:44:49 <anteaya> that is my current understanding
15:44:50 <tristanC> anteaya: oh, so what would be that new structure ?
15:45:05 <anteaya> tristanC: some new config that would be generic
15:45:20 <anteaya> tristanC: I don't know as it has a name yet, or is that the openstackci repo asselin_?
15:45:31 <asselin_> tristanC, see this patch for example: https://review.openstack.org/#/c/167426/
15:45:52 <asselin_> tristanC, you can trace the cross-project 'depends on'
15:46:30 <asselin_> tristanC, this is an example of infra specific system-config consuming the reusable log server puppet script
15:47:27 <fbo> asselin_, so we need to reuse the same process when moving manifests to openstackci ?
15:47:28 <asselin_> tristanC, so if you updated the log server script in openstackci to better support fedora/centos, it would be done in the puppet-openstackci repo
15:47:59 <asselin_> fbo, yes
15:48:26 <tristanC> anteaya: asselin_: thanks for the explanation, it makes more sense
15:48:37 <anteaya> tristanC: great, thanks for asking the questions
15:48:41 <BobBall> asselin_: Sorry to go back to this... Looking at your log you pasted me, I doubt very much the REGEX is the issue; your log includes a line that says "Running tempest with a custom regex filter" but mine does not run that, it just cleans up the host immediately after "Removing sudo privileges".  Any thoughts on why I might consistently stop there without any logging (that I've found so far...) What other D-G vars are you setting?
15:49:28 <asselin_> fbo, well I don't want to dictate process....if you find a better way, then great! otherwise that one is a good starting point.
15:50:27 <anteaya> asselin_: sometimes when folks are just getting started it doesn't hurt to dictate process, as it decreases the number of decisions they have to make to get involved
15:50:40 <anteaya> asselin_: just be open to options if a block occurs
15:50:46 <anteaya> fbo: is that fair?
15:51:19 <asselin_> BobBall, I remember when the export was done right it wouldn't run. Otherwise it should pick up.  https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L635
15:51:19 <fbo> yep
15:51:26 <anteaya> fbo: great
15:51:59 <asselin_> anteaya, fbo agreed. :)
15:52:06 <anteaya> asselin_: thanks :)
15:52:13 <fbo> so thank you for the explanations it is clearer to me now :)
15:52:21 <anteaya> fbo: wonderful
15:52:52 <anteaya> 8 minutes remaining anyone with a new topic?
15:53:27 <anteaya> BobBall: are you in a happy place yet?
15:53:34 <BobBall> asselin_: For the record, I was being stupid... I was replacing DEVSTACK_GATE_TEMPEST with _REGEX, thinking the _TEMPEST was actually _TEMPEST_ALL... ... ...
15:53:45 * BobBall needs to go find something nice and soft he doesn't mind hitting his head against
15:54:01 <anteaya> BobBall: don't do that, at least you saw it
15:54:31 <anteaya> does anyone have anything else today?
15:54:33 <BobBall> Indeed.  I was expecting that one to bug me for days.
15:54:44 <anteaya> BobBall: and here it is Monday and you saw it
15:54:56 <anteaya> the rest of the week stretches out before you
15:55:08 * anteaya considers wrapping up this meeting
15:55:12 <anteaya> any objections?
15:55:57 <anteaya> so thanks everyone for your participation today, it will be great to get some additional coders/reviewers on asselin_'s work
15:56:06 <anteaya> enjoy the rest of the day
15:56:10 <anteaya> see you next week
15:56:13 <anteaya> #endmeeting