15:01:33 <jruzicka> #startmeeting RDO meeting - 2017-09-20
15:01:33 <openstack> Meeting started Wed Sep 20 15:01:33 2017 UTC and is due to finish in 60 minutes.  The chair is jruzicka. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:36 <openstack> The meeting name has been set to 'rdo_meeting___2017_09_20'
15:01:49 <rbowen> o/
15:01:57 <trown> o/
15:02:05 <jruzicka> #topic Hello #rdo o/
15:02:05 <PagliaccisCloud> o/
15:02:09 <flepied> o/
15:02:11 <amoralej> o/
15:02:41 <jatanmalde> o/
15:02:57 <jruzicka> #chair rbowen trown PagliaccisCloud flepied amoralej jatanmalde
15:02:58 <openstack> Current chairs: PagliaccisCloud amoralej flepied jatanmalde jruzicka rbowen trown
15:03:28 <snecklifter> o/
15:04:05 * Duck o/
15:04:22 <myoung> o/
15:04:37 <jschlueter> o/
15:06:17 <jruzicka> #chair Duck myoung jschlueter
15:06:18 <openstack> Current chairs: Duck PagliaccisCloud amoralej flepied jatanmalde jruzicka jschlueter myoung rbowen trown
15:06:35 <jruzicka> #topic Infrastructure topics
15:06:39 <jruzicka> Duck, Quack
15:07:32 <Duck> I did not have time to put them yet :-)
15:07:36 <rbowen> Duck asked that he be first on the agenda for Infra things, so that he can duck out for the remainder of the meeting.
15:07:41 <rbowen> I didn't actually ask if there were any issues.
15:07:43 <snecklifter> jruzicka: i demand a chair
15:07:50 <rbowen> Just that we need a standing Infra item, first thing on the agenda.
15:07:54 <rbowen> That's why that was there.
15:08:08 <rbowen> So, next week ... :-)
15:08:11 <jruzicka> #chair snecklifter
15:08:12 <openstack> Current chairs: Duck PagliaccisCloud amoralej flepied jatanmalde jruzicka jschlueter myoung rbowen snecklifter trown
15:08:19 <snecklifter> ta
15:08:24 <Duck> if there's a proficient Python guy here, helping fix the nasty Mailman3 bug could help
15:08:56 <rdogerrit> Jeremy Liu proposed openstack/karborclient-distgit rpm-master: Initial import of spec file  https://review.rdoproject.org/r/8675
15:09:14 <Duck> also I suggested using the bugzilla tracker Rich kindly opened to follow big tasks projects
15:09:31 <Duck> but more later yes
15:09:40 <apevec> rbowen, which bz is that?
15:09:51 <jschlueter> gfidente, fultonj: you want to get line item on here for ceph-ansible and ceph  and tripleo integration ci jobs?
15:10:09 <rbowen> apevec: The only ticket in that bucket is https://bugzilla.redhat.com/show_bug.cgi?id=1487324
15:10:10 <openstack> bugzilla.redhat.com bug 1487324 in Infrastructure "Move mailing lists from @redhat.com to @lists.rdoproject.org" [Unspecified,Assigned] - Assigned to duck
15:10:15 <rbowen> which may very well be the wrong ticket tracker.
15:10:19 <rbowen> This is an open question.
15:10:23 <Duck> for eg -> https://bugzilla.redhat.com/show_bug.cgi?id=1487324
15:10:42 <Duck> the ticket for MM3 is in there
15:10:59 <jruzicka> Duck, is there a bounty on it? :)
15:11:03 <Duck> the rest is Ansible rules and how all the so many extra yum repo broke the world
15:11:19 <Duck> jruzicka: NO, just the FUN and GLORY :-)
15:11:37 <gfidente> jschlueter yes but looks a bit early on
15:11:56 <Duck> so any help is appreciated
15:12:00 <rdogerrit> Jeremy Liu proposed openstack/karbor-distgit rpm-master: Initial import of spec file  https://review.rdoproject.org/r/8671
15:12:12 <Duck> or just discussing the possible solutions
15:12:26 <Duck> I'm not a BOFH so please give your opinion
15:12:55 <Duck> (solutions for coordinating, and solutions for the MM3 problems too)
15:13:11 <jruzicka> well, "bug loosing mails" sounds serious.
15:13:16 <Duck> dmsimard: :-)))
15:13:54 <Duck> fortunately oVirt lost mostly automated mails noone cares, but a handful were real ones
15:14:33 <jschlueter> #chair gfidente
15:14:33 <openstack> Current chairs: Duck PagliaccisCloud amoralej flepied gfidente jatanmalde jruzicka jschlueter myoung rbowen snecklifter trown
15:14:58 <Duck> so it's all for today on my side. do not hesitate pinging me :-)
15:15:07 <Duck> any other infra topics ?
15:15:25 <jschlueter> jruzicka: agenda item for meeting ... /me doesn't have link to etherpad close at hand ... "ceph/ceph-ansible and tripleo integration"
15:15:31 <fultonj> ktdreyer: ^
15:15:44 <jschlueter> #chair fultonj
15:15:45 <openstack> Current chairs: Duck PagliaccisCloud amoralej flepied fultonj gfidente jatanmalde jruzicka jschlueter myoung rbowen snecklifter trown
15:15:47 <jruzicka> RDO-Meeting pad: https://etherpad.openstack.org/p/RDO-Meeting
15:15:47 <gfidente> let me introduce a little the issue that we have to see if there are ideas
15:16:10 <gfidente> basically we consume ceph-ansible builds in tripleo to deploy ceph via tripleo
15:16:15 <jruzicka> jschlueter, added
15:16:24 <jschlueter> jruzicka: thanks!
15:16:25 <jruzicka> let us proceed, then
15:16:32 <gfidente> but we don't have jobs gating ceph-ansible commits with a full tripleo run to make sure they don't break
15:16:41 <jruzicka> #topic rdopkg Release bumping support
15:16:45 <gfidente> and we probably won't have
15:17:15 <jruzicka> This this is about bumping Release: in .spec files
15:17:20 <jruzicka> Currently: last numeric part of the Release string is bumped, i.e. 0.1.rc1 -> 0.2.rc1
15:17:29 <rdogerrit> Mike Burns created rdoinfo master: bump newton default for python-networking-cisco  https://review.rdoproject.org/r/9698
15:17:36 <jruzicka> DLRN support requested: https://github.com/softwarefactory-project/rdopkg/issues/77
15:17:45 <Duck> jruzicka: you skipped gfidente
15:18:03 <trown> or did gfidente skip jruzicka ... i am a bit lost
15:18:12 <jpena> o/
15:18:13 <jruzicka> 0.20160817045049.d668fcd%{?dist} -> 0.20160817045049.d668fcd.1%{?dist} (bump last part or add .1 if not numeric)
15:18:19 * jruzicka will finish the dump :)
15:18:24 <rbowen> I think we're just out of order on the agenda.
15:18:35 <jruzicka> how?
15:18:47 <jruzicka> Infrastructure topics were closed, or no?
15:18:57 <jruzicka> shall I undo?
15:19:06 <rbowen> The ceph item was at the end. of the list. Perhaps I'm just confused.
15:19:19 <Duck> gfidente: I really don't know anything about this, but it sounds like adding a CI job (and be sure to have the resources, because I guess it's not a tiny build)
15:19:26 <trown> ya lets finish this topic, then continue with ceph-ansible later
15:19:36 <Duck> ok
15:19:45 <jruzicka> yup it's down in the agenda
15:19:53 <gfidente> sorry guys I assumed we were then when jschlueter mentioned the new agenda topic
15:20:04 <jruzicka> gfidente, np np ;)
15:20:10 <Duck> daijoubu
15:20:23 <trown> so release string bumping
15:20:54 <jruzicka> So in short, DLRN uses Release format which requires a special handling so I need to detect it in rdopkg and treat separately
15:21:12 <jruzicka> since I'm doing that, it's good time to research Release: conventions in the wild and support as much of them as possible
15:21:21 <jpena> jruzicka: support for predictable short hashes was merged yesterday: https://softwarefactory-project.io/r/9721
15:21:30 <jruzicka> jpena++
15:21:46 <jpena> so now we can really use the regexp to detect DLRN specs
15:21:52 <jruzicka> so it's pretty easy to detect DLRN spec and not trigger special behavior when it shouldn't
15:21:56 <jruzicka> yes
15:22:10 <apevec> dlrn is currently 0.timestamp.hash
15:22:15 <jschlueter> jruzicka: I proposed a number of them that I'm familar with in patch for rdopkg
15:22:25 <apevec> but that is not fully following Fedora pre-release convention
15:22:27 <amoralej> jruzicka, we also use pre and post release NVRs
15:22:35 <amoralej> as defined in fedora
15:22:49 <jruzicka> oh jschlueter is one steap ahead :)
15:23:03 <apevec> fedora-compliant it would be 0.N.timestamp.whatever
15:23:03 <amoralej> and git snapshots based
15:23:09 <jruzicka> https://softwarefactory-project.io/r/#/c/9636/
15:23:27 <jruzicka> apevec, were N gets bumped?
15:23:47 <apevec> that's what rdopkg or packager would bump
15:23:48 <jruzicka> jschlueter, nice, thanks :)
15:24:06 <jschlueter> https://softwarefactory-project.io/r/#/c/9724 has some of the good behavior from current release
15:24:28 <apevec> so dlrn would need to be changed to e.g. 0.1.timestamp.hash
15:24:43 <apevec> but need to carefuly check upgrade implications
15:24:48 <jruzicka> jschlueter++
15:25:03 <apevec> since we want to ensure you can always upgrade trunk -> next stable build
15:25:05 <jruzicka> jschlueter, amazing, that's gotta make it much harder to break everything as I change this :)
15:25:29 <jschlueter> jruzicka: yep exactly!
15:25:49 <jruzicka> apevec, indeed, I'd rather do this properly as opposed to series of fixes that each introduces new unexpected implications :)
15:26:17 <jschlueter> behave scenarios make it easy to define what things should like like and how it should behave
15:27:06 <jruzicka> ight? it's working :) I suggest looking at this nice feature file by jschlueter which clearly definec what to expect from rdopkg:
15:27:20 <jruzicka> https://softwarefactory-project.io/r/#/c/9724/1/features/fix.feature
15:27:34 <apevec> jruzicka, what I'm proposing would require changes in dlrn, so need to be extra careful not to break e.g. CI use cases
15:27:44 <jschlueter> jruzicka: all of those are current behavior and acceptable by me for how rdopkg works
15:28:16 <jruzicka> apevec, changing dlrn to standard format would be best
15:28:47 <jpena> apevec: wouldn't 0.timestamp.hash.1 work better? It might not be fedora-compliant, but 0.1.timestamp will break upgrades (0.2017xxx is always higher than 0.1)
15:29:37 <apevec> jpena, yes, it would require one-time breaking
15:29:53 <apevec> so doing it now at the beginning of cycle would be best
15:30:27 <jruzicka> yeah break now and I don't have to support obsolete (current) behavior in rdopkg :)
15:30:33 <amoralej> now it's beginning for pike but not for the rest and i guess we'll apply it for all releases, right?
15:30:46 <amoralej> s/pike/queens/
15:31:29 <amoralej> so we'll breack migrations from trunk to newer trunk in the same release
15:31:32 <jruzicka> having compatible releases with Fedora will save huge amount of time and pain I'm sure
15:32:30 <amoralej> yes, sounds as the best option but i'm just thinking in minimize impact
15:33:03 <jpena> crazy idea... maybe 0.3.timestamp.hash ?
15:33:31 <apevec> how that helps?
15:33:46 <amoralej> no, 3 < 2017...
15:33:58 <Duck> quack?
15:34:13 <jpena> mmm
15:34:14 <jpena> true
15:34:35 <jruzicka> I was thinking about extracting all or some .spec files from Fedora pkgs and analyzing foudn Releases... maybe that's overkill, what do you think?
15:34:56 <Duck> epoch?
15:35:05 <amoralej> making nvr format configurable per-worker
15:35:24 <amoralej> we could change it in queens and keep as-is in previous releases?
15:35:27 <jruzicka> oh epoch would work but it's confusing.
15:35:38 <apevec> epoch could solve this but has its cost
15:35:43 <jschlueter> jruzicka: I can send you NVR's for OSP
15:35:53 <apevec> then you have to add %{epoch} _everywhere_
15:35:59 <jschlueter> and cherry-pick interesting cases into rdopkg
15:36:00 <jruzicka> jschlueter, cool please do!
15:36:02 <Duck> sure, but you could be sure to have the target version and migration are sure to work
15:36:14 <jschlueter> but dlrn builds don't have to be upgradable across the board
15:36:22 <jruzicka> jschlueter, that and Fedora ones could be enought to start with...
15:36:31 <amoralej> we'd need to update Requires to epoch also
15:36:47 <jruzicka> #action jruzicka to research Release formats in the wilds
15:36:48 <amoralej> in all specs
15:36:50 <apevec> amoralej, let's explore per-worker idea
15:38:02 <jruzicka> I'm glad I raised awarenes about this. Usually some extra special cases and detections in rdopkg mean some inconsistency/irregularity and solving this one will make things easier.
15:38:58 <jruzicka> #action everyone to think about best Trunk Release format
15:39:28 <jruzicka> let's discuss this again in the future and move on with the meeting
15:39:31 <jschlueter> #action jschlueter to collect interesting OSP NVR's to jruzicka and pick out interesting cases
15:39:35 * jschlueter nods
15:40:24 <jruzicka> I'll consider blogging about it ;)
15:40:37 <jruzicka> #topic PTG interviews starting to appear here: http://youtube.com/RDOCommunity
15:41:01 <rbowen> That's pretty much the whole topic.
15:41:01 <PagliaccisCloud> -dances-
15:41:08 <rbowen> Great interviews there. Please help promote
15:41:10 <rbowen> EOL
15:41:18 <PagliaccisCloud> btw i'm sorry i missed you all at ptg. for some reason it got in my mind that you would all be with OoO
15:41:20 <rbowen> I had 23 interviews. I have published 8 so far.
15:41:21 <jruzicka> PagliaccisCloud -dancing- is the whole topic? OK.
15:41:32 * jruzicka winks
15:42:38 <jruzicka> #topic Test day schedule for Queens
15:42:49 <jruzicka> rbowen, yours as well I think?
15:43:08 <rbowen> Ah. Yes. Just a heads up that I'll be doing the Queens test day schedule soon.
15:43:21 <rbowen> And there's always date conflicts in the second release of a year.
15:43:41 <rbowen> So I'll hopefully propose a schedule soon, so that people can tell me what dates everything conflicts with.
15:43:54 <apevec> default is upstream milestone + 1 week
15:43:55 <gfidente> rbowen++
15:43:56 <rbowen> But also we need to figure out how to better engage people on test day, given that so much of hte testing is automated.
15:44:18 <rbowen> We need to find ways to engage operators, rather than just ourselves and the usual suspects.
15:44:29 <chandankumar> rbowen: one thing would be good to include installation docs for projects currently all installation guides for each component moved to their project so may include a section
15:44:35 <rbowen> So advance promotion is part of that.
15:44:40 <jschlueter> rbowen: do we want to consider closer to trailing release (so that tripleo is stable for the milestone and testday)?
15:44:43 <chandankumar> testing each component from those docs
15:44:46 <rbowen> We also talked about having a Trystack-like thingy where people can do testing.
15:45:00 <rbowen> jschlueter: That's certainly a good idea.
15:45:24 <rbowen> Is there a chance that we could have a public deployment of each milestone where people can run specific manual testing scenarios?
15:45:42 <jruzicka> too automated to attract operators? :D
15:45:48 <jschlueter> unless we can push on tripleo and other trailing release components to be as much feature complete by Milestone and only have blockers and specific integration issues to deal with in trailing release window
15:46:15 <rbowen> chandankumar: Yeh, that's a good idea also.
15:47:14 <rbowen> Unfortunately, I've been traveling and have dropped a number of tasks along the way. So I'll try to put together some of these suggestions on the mailing list in the coming day or two.
15:47:27 <snecklifter> having a public deployment that gets reborn every ay 2 hours would be great, just for the test day
15:47:40 <rbowen> Yes, so that it's harder to abuse.
15:48:32 <rbowen> But that will involve actually having someone tasked to make that happen, as I am sure it's not trivial.
15:48:35 <jschlueter> public deployment good idea but need good coordination for rebuilding ...
15:48:49 <rbowen> Yep
15:49:59 <jruzicka> we're running late so let's quickly jump to last topic
15:50:09 <jruzicka> #topic ceph/ceph-ansible and tripleo integration
15:50:18 <PagliaccisCloud> :D
15:50:25 <jruzicka> gfidante, fultonj NOW IS THE TIME
15:50:28 <jschlueter> gfidente, fultonj: ^^
15:50:31 <jruzicka> :)
15:50:46 <fultonj> ok
15:50:53 <fultonj> We gate tripleo against ceph-ansible builds from the centos storage sig
15:50:58 <fultonj> We don't gate ceph-ansible builds against tripleo however, but want to
15:51:06 <fultonj> The closet thing we have to a gate is a WIP submission in tripleo ci
15:51:16 <fultonj> an example is linked from  https://www.redhat.com/archives/rdo-list/2017-September/msg00045.html
15:51:37 <fultonj> so we need to do something and welcome feedback
15:51:47 <jschlueter> #info hallway-track discussion at PTG brought up how we can improve ceph/ceph-ansible and tripleo integration and testing
15:51:50 <amoralej> fultonj, in RDO we have implemented automatic tagging in gate jobs in review.r.o
15:52:01 <amoralej> to promote from -candidate to -testing
15:52:07 <amoralej> and from -testing to -release
15:52:07 <gfidente> amoralej++
15:52:12 <gfidente> we were looking into same thing
15:52:20 <gfidente> and the gating job should be a tripleo run
15:52:27 <amoralej> based on a metadata repo
15:52:28 <gfidente> but we don't have ceph-ansible in review.r.o
15:52:29 <amoralej> rdoinfo
15:52:50 <jschlueter> would getting a dlrn instance for ceph/ceph-ansible be enough help get CI jobs running?
15:53:06 <amoralej> in fact you could apply that not only for ceph-ansible but for anything in storage repo
15:53:10 <jschlueter> non-voting to start with ... but need somewhere to run the jobs and instance
15:53:31 <amoralej> automation of cbs builds is different worflow as dlrn builds
15:53:33 <gfidente> I guess that should be centos infra given that's where it is hosted?
15:53:47 <jschlueter> *instance of dlrn for doing builds and kicking master chasing rpms/repos and ci jobs
15:54:12 <amoralej> jschlueter, yeah, that's a different approach, more similar to the promotion pipeline
15:54:16 <amoralej> for rdo trunk repos
15:54:29 <jschlueter> there was interest in getting "dlrn" master chasing of ceph and ceph-ansible going in the near future
15:55:13 <jschlueter> and have non-voting CI jobs of tripleo using master chasing ceph/ceph-ansible to provide quicker feedback of when and what broke
15:55:16 <fultonj> ktdreyer: ^
15:55:43 <jschlueter> but not sure what resources/people/efforts that would take to get setup for storage sig to make use of
15:55:58 <jruzicka> 5 minutes remain.
15:56:18 <amoralej> jschlueter, that can be done, but that'd require an additional repo to be configured in the periodic jobs
15:56:26 <jschlueter> ktdreyer: got semi-automated cbs builds of ceph-ansible setup at PTG but looking at what can be dome for more
15:57:12 <jschlueter> amoralej: so discussion opened up and need to get trade offs, where we can get resources, effort to maintain and what is needed to get it working and keep it working
15:57:45 <jschlueter> that's all more of an FYI and request for addition feedback/input to move it forward
15:58:08 <amoralej> yes, there are different aproaches that can be followed, we can discuss it
15:58:18 <fultonj> thanks amoralej
15:58:33 <apevec> let's start with cbs builds
15:58:36 <jschlueter> #info Looking for additional feedback and input on how Storage SIG can setup more workflow around ceph/ceph-ansible and tripleo integration
15:58:55 * jschlueter nods to apevec
15:59:08 <apevec> chasing trunk takes not just machine resources, but also org and people
15:59:33 * jschlueter nods
15:59:43 <jruzicka> chasing clouds, chasing trunks... hehe :)
16:00:38 <jruzicka> And with that we ran out of time.
16:01:00 <jruzicka> #topic Chair for next meetig
16:01:14 <amoralej> i can do it
16:01:31 <jruzicka> let the one who wants to chair put his hands up in the air!
16:01:40 <jruzicka> #action amoralej to chair next meeting
16:01:59 <jruzicka> #endmeeting