13:31:34 <chandankumar> #startmeeting RDO Office Hour - 2017-10-17
13:31:35 <openstack> Meeting started Tue Oct 17 13:31:34 2017 UTC and is due to finish in 60 minutes.  The chair is chandankumar. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:31:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:31:39 <openstack> The meeting name has been set to 'rdo_office_hour___2017_10_17'
13:31:51 <chandankumar> #topic Roll Call
13:32:03 <jpena> o/
13:32:08 <adarazs> o/
13:32:09 <chandankumar> hguemar: jpena amoralej adarazs ykarel|away dmsimard office hour
13:32:10 <aditya_r> o/
13:32:21 <chandankumar> #chair aditya_r jpena adarazs
13:32:22 <openstack> Current chairs: adarazs aditya_r chandankumar jpena
13:32:22 * jpena will have to run in 30 mins due to a conflicting meeting
13:32:30 <hguemar> o/
13:32:47 <chandankumar> #chair hguemar
13:32:48 <openstack> Current chairs: adarazs aditya_r chandankumar hguemar jpena
13:33:34 <chandankumar> adarazs: i am handing over the status to you
13:33:43 <chandankumar> #topic Config Project session
13:33:53 <chandankumar> adarazs: feel free to go ahead
13:34:59 <chandankumar> s/status/stage
13:35:48 <adarazs> well.
13:35:50 <adarazs> sorry.
13:36:09 <adarazs> so I think we should go over different folders of the "config" repository.
13:36:15 <chandankumar> adarazs: yup
13:36:32 <adarazs> I'm only familiar with the "jobs" directory and the "zuul" one.
13:36:50 <jpena> I'll start with the "gerrit" and "gerritbot" folders
13:36:52 <adarazs> can somebody describe dashboards, gerrit, gerritbot, nodepool, policies?
13:37:04 <myoung> o/
13:37:05 <adarazs> and repoxplorer, resources.
13:37:07 <chandankumar> dmsimard: ^^
13:37:10 <chandankumar> #chair myoung
13:37:11 <openstack> Current chairs: adarazs aditya_r chandankumar hguemar jpena myoung
13:37:13 <amoralej> o/
13:37:19 <chandankumar> #chair amoralej
13:37:19 <openstack> Current chairs: adarazs aditya_r amoralej chandankumar hguemar jpena myoung
13:37:26 <dmsimard> chandankumar: python-tempestconf is objectively out of scope for puppet-openstack-integration, but that doesn't mean that in the python-tempestconf gate you can't use puppet-openstack-integration to deploy openstack, run tempestconf and then check that you get the expected configuration or something.
13:37:44 <tosky> adarazs: I would summarize them as: 99% you don't need to touch them
13:38:09 <dmsimard> I'll let jpena start and I can follow up I guess
13:38:21 <jpena> The "gerrit" folder contains some configuration for Gerrit, namely hyperlink creation for commit messages (commentlinks.yaml) and Gerrit-GitHub replication configuration (replication.config)
13:38:22 <adarazs> tosky: I probably won't but I just got +2 rights on the repo, so it's better to know about them :)
13:38:50 <jpena> In general, we may only touch replication.config if we want to add a project to Gerrit and it does not match one of the existing wildcards
13:38:50 <dmsimard> BTW, I don't think I've seen it mentioned but the config repo is mirrored here: https://github.com/rdo-infra/review.rdoproject.org-config
13:39:41 <adarazs> dmsimard: that's a useful info.
13:39:46 <adarazs> dmsimard: I couldn't find it before.
13:40:01 <tosky> adarazs: gerritbot is the configuration of the bot (so where to send which notification for which project)
13:40:30 <tosky> about gerrit, is the gerrit configuration; see for example this review that adds a new project: https://review.openstack.org/#/c/508502/
13:40:46 <dmsimard> I think it's also helpful to draw parallels between the equivalence between upstream configuration and software factory configuration
13:41:31 <dmsimard> I'm starting a pad here: https://review.rdoproject.org/etherpad/p/rdo-config-office-hours
13:41:39 <jpena> we have automation for the new project creation, based on a post-job for rdoinfo changes
13:41:52 <adarazs> *jobs* directory: correct me if I'm wrong, but those are Jenkins Job Builder style definitions of jobs that are added to our Jenkins that Zuul can trigger on different conditions. the tripleo upstream related jobs are in jobs/tripleo-upstream.yml (surprisingly)
13:41:56 <adarazs> what I don't know if these are actually JJB config files or just something similar. are every jenkins plugins supported?
13:42:57 <jpena> they are JJB config files. I'm not sure if all plugins are supported, dmsimard?
13:43:14 <dmsimard> adarazs: jenkins plugins are available so long as they are installed just like any jenkins installation, however, I would not rely on the installation of plugins since the days of jenkins are counted
13:43:32 <dmsimard> adarazs: for example, 'timeout' was installed because it is not a feature available by default iirc
13:44:06 <adarazs> asking because we're apparently using postbuild script and panda|rover is adding another one just now.
13:44:14 <adarazs> and I was wondering how we're going to solve that later.
13:45:26 <dmsimard> adarazs: postbuildscript was not supported upstream, even before zuul v3
13:45:48 <dmsimard> adarazs: in zuul v3, this would be a 'post-run' playbook which is built-in
13:46:16 <dmsimard> adarazs: there is also a variable, 'zuul_success' that you can use in your playbooks to determine if the status of the run jobs were successful or not
13:46:56 <adarazs> dmsimard: and when are we going to switch to zuul v3 for rdoproject.org?
13:48:03 <adarazs> or at least what are the preliminary plans? :)
13:48:12 <dmsimard> adarazs: We're expecting to start using Zuul v3 with the next release of Software Factory, 2.7, which is due within the next 3 weeks or so. We'll then need to plan the upgrade of Software Factory itself and then the plan is to run Zuul v2 and Zuul v3 side-by-side a bit like upstream did during the partial rollback.
13:48:50 <adarazs> all right.
13:49:02 <rdogerrit> Aditya Ramteke proposed openstack/novaclient-distgit rpm-master: Moved package description to global variable.  https://review.rdoproject.org/r/9957
13:49:04 <adarazs> thank you for the timeline.
13:49:07 <dmsimard> However, we want to run Zuul v2 and Zuul v3 simultaneously for as little time as possible as doing that is very expensive to maintain
13:49:46 <jschlueter> chandankumar: o/
13:49:48 <dmsimard> Given that it's about a month out, hopefully upstream will have ironed out kinks with their jobs so the effort involved to translate review.rdo jobs to Zuul v3 should not be tremendous
13:49:52 <chandankumar> #chair jschlueter
13:49:53 <openstack> Current chairs: adarazs aditya_r amoralej chandankumar hguemar jpena jschlueter myoung
13:50:13 <jpena> dmsimard: that's optimistic, but hey :D
13:50:28 <chandankumar> I have a question: how we add an experimental job on review.rdoproject.org?
13:50:39 <dmsimard> adarazs, weshay|ruck: however, we should definitely plan for the tripleo CI team availability when that time nears because we really don't want to maintain both Zuuls simultaneously for any extended period of time
13:50:47 <panda|rover> sorry I missed part of the discussion, can we use postbuildscript in config in the end ?
13:51:04 <dmsimard> chandankumar: there's an experimental pipeline, you put a job in that pipeline for a project and that allows you to trigger it with 'check experimental'
13:51:10 <adarazs> panda|rover: if it works currently I think we could, and zuul v3 will give us something instead that is built-in.
13:51:16 <panda|rover> if not, is it ok if we don't mark build as failures ?
13:51:30 <panda|rover> because this is what is going to happen without post scripts
13:52:03 <dmsimard> panda|rover: you can use postbuildscript *currently* but you need to keep in mind that you need to translate those into playbooks that will run in zuul v3 'post-run', which is the equivalent of what 'postbuildscript' is.
13:52:38 <dmsimard> panda|rover: you could probably save time if you started writing your post playbooks right away and then use postbuildscript to run those playbooks so that when zuul v3 comes around, your playbooks are mostly written
13:53:14 <panda|rover> oh dear, this is like when I was doing an exam and the teacher was pulling the sheets under my pen, but i wanted to keep writing.
13:53:49 <adarazs> hehe
13:53:51 <dmsimard> panda|rover: but then the teacher handed you a sheet with the answers written on it because zuul v3 is so awesome
13:54:50 <dmsimard> panda|rover, adarazs: we need to discuss zuul v3 more broadly but right now is probably not the right time, I encourage you to sign up here: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123668.html
13:54:52 <rdogerrit> Aditya Ramteke proposed openstack/kuryr-distgit rpm-master: Move package description to global variable.  https://review.rdoproject.org/r/9907
13:55:41 <chandankumar> dmsimard: can i get an example review for the experimental job?
13:55:48 <dmsimard> chandankumar: sure, one sec
13:56:07 <panda|rover> dmsimard: ok, as usual there's no much time to do things the perfect way the first time, I'll add the post build, and add technical debt cards all aover to add the playbooks for the post-run
13:56:47 <dmsimard> chandankumar: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/projects.yaml#L3535-L3573
13:57:30 <dmsimard> panda|rover: up to you, if I was writing new content, I'd write it in ansible instead of bash, you're literally creating debt out of thin air if you're writing bash :D
13:57:45 <chandankumar> dmsimard: thanks :-)
13:57:57 <dmsimard> ok, let's continue with the overview of the config repository: https://github.com/rdo-infra/review.rdoproject.org-config
13:58:13 <dmsimard> jpena explained gerrit and gerritbot ? I can talk about nodepool
13:58:20 <jpena> yep
13:58:22 <chandankumar> dmsimard: yup
13:58:50 <dmsimard> our nodepool directory https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/nodepool is the equivalent to upstream's https://github.com/openstack-infra/project-config/tree/master/nodepool
13:59:19 <dmsimard> There's several things going on there -- first is the actual nodepool configuration https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/nodepool/nodepool.yaml
14:00:20 <dmsimard> This defines our disk images that are built through nodepool with diskimage-builder, so each image has a list of DIB elements picked from here: https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/elements and here: https://github.com/openstack-infra/project-config/tree/master/nodepool/elements
14:00:21 <rdogerrit> Merged rdoinfo master: Promote CBS tags update for newton-testing  https://review.rdoproject.org/r/10172
14:00:56 <dmsimard> We make the upstream elements available in order to be able to build images for TripleO that are similar to what upstream is doing, ideally with as little delta as possible
14:01:44 <dmsimard> We've stopped synchronizing project-config a while back because upstream is moving full steam ahead to Zuul v3 and there are some changes in there that are likely to break our implementation.
14:02:24 <dmsimard> Otherwise, in that nodepool configuration, this is also where you'll see our 'cloud providers' defined, what images we're uploading to each and the quota allocated to each provider.
14:03:25 <dmsimard> The authentication information for nodepool is stored on the nodepool servers themselves rather than in-tree for obvious reasons :)
14:04:03 <dmsimard> As far as software factory is concerned, this information is available in /etc/software-factory/sfconfig.yaml
14:05:14 <chandankumar> dmsimard: https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/policies is used to update the SF configuration?
14:05:27 <dmsimard> Other than that, in the nodepool directory ( https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/nodepool ) we have elements which are our DIB elements for various purposes like installing basic packages and such. You also have scripts, which are made available in.... a path I forget, on the virtual machines
14:05:45 <dmsimard> chandankumar: yup, I'll get to the policies directory in a minute
14:05:52 <chandankumar> dmsimard: sure
14:07:05 <dmsimard> fbo, mhu: correct me if I'm wrong here? the policy.yaml file ( https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/policies/policy.yaml ) controls the ACL to the software factory (managesf) API
14:07:23 <rdogerrit> Aditya Ramteke proposed openstack/tacker-distgit rpm-master: Moved package description to global variable and replace with it.  https://review.rdoproject.org/r/9552
14:07:44 <dmsimard> I don't remember having to touch that file ever, so it's there and it's used but we don't need to play with it too much.
14:08:16 <mhu> dmsimard, yes. The managesf API is consumed by the CLI sfmanager
14:09:09 <mhu> it can be useful to allow some privileged users to hold test nodes for example
14:09:14 <dmsimard> The repoexplorer directory is not in use right now because we haven't yet deployed it on review.rdoproject.org. It's a project that allows to provide statistics around the repositories hosted by the instance, it looks like this: https://softwarefactory-project.io/repoxplorer/
14:09:33 <dmsimard> mhu: okay, thanks
14:09:52 <chandankumar> and the same policy.yaml file is tied with https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/resources folder update and create acls
14:10:30 <dmsimard> chandankumar: it's not really tied, the resources directory is mostly around gerrit itself
14:10:50 <adarazs> that repoexplorer looks cool.
14:10:53 <dmsimard> the resources directory defines the gerrit ACL for every project -- who can do what, who is in which group, and so on
14:10:55 * adarazs likes stats.
14:11:07 <chandankumar> ah, thanks :-)
14:11:20 <dmsimard> for example, this should be fairly self explanatory https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/resources/config.yaml
14:11:39 <dmsimard> The idea is that the gerrit configuration is controlled by the config repository
14:12:56 <chandankumar> dmsimard: currently if we open review.rdorproject.org main page 4 Associated git repositories related to barbican appears
14:13:19 <chandankumar> how to get rid of it and it should display the list of projects i am in involved
14:13:23 <dmsimard> If we need to add a new project to review.rdoproject.org, we'll need to create a resource for it through that mechanism. A review to add a project looks like this: https://review.rdoproject.org/r/#/c/10070/ or like this: https://review.rdoproject.org/r/#/c/9451/
14:13:42 <dmsimard> chandankumar: what do you define as the 'main page' ?
14:14:05 <chandankumar> dmsimard: https://review.rdoproject.org/sf/welcome.html
14:15:10 <dmsimard> chandankumar: I don't know to be honest, I don't really use that page :/
14:15:16 <rdogerrit> Gabriele Cerami proposed config master: [WIP] tripleo upstream periodic: add dlrn reporting as post build  https://review.rdoproject.org/r/10180
14:15:19 <dmsimard> maybe mhu or fbo know how to customize the home page
14:15:58 <rdogerrit> Gabriele Cerami created rdo-infra/ci-config master: Adding dlnapi_reporting script  https://review.rdoproject.org/r/10181
14:16:10 <mhu> dmsimard, IIRC you can set a template in sfconfig.yaml for the landing page
14:16:29 <dmsimard> mhu: yeah but why am I "associated" with some specific repos ?
14:17:18 <mhu> dmsimard, good question, I don't know how the repos to display are chosen, to be honest
14:19:52 <dmsimard> I think we've gone through most of the aspects of the config dir
14:20:01 <dmsimard> unless there's any other questions, we can wrap this up
14:20:22 <chandankumar> dmsimard: do we have something in gates, if i want to trigger only failed jobs not all?
14:20:27 <dmsimard> once again, I encourage you to sign up to the zuul v3 "ask me anything" session next week: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123668.html
14:20:39 <dmsimard> chandankumar: nope, every job will be retriggered
14:22:42 <chandankumar> adarazs: anything you want to add? which is missed
14:23:21 <dmsimard> I guess we might have wanted to have sshnaidm|off around :(
14:24:05 <chandankumar> lots of info i have got, i will create a blog out of it
14:24:10 <adarazs> we didn't talk about the zuul pipelines, but I think they are fairly straightforward.
14:24:40 <myoung> who "owns" the dashboards repo?  I'm doing some work in a side project and will have some things to merge to there...
14:24:56 <chandankumar> dmsimard: one more thing can you explain https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/jobs part
14:25:02 <adarazs> you can create pipeline like so: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml#L58-L83
14:25:06 <myoung> also - are there any details about networking, host, etc that it's running on?
14:25:09 <chandankumar> dmsimard: there are bunch of files, what is for what?
14:25:45 <dmsimard> chandankumar: the jobs directory is the equivalent of https://github.com/openstack-infra/project-config/tree/master/jenkins/jobs
14:25:55 <dmsimard> chandankumar: it is jenkins configuration driven by jenkins job builder
14:26:04 <adarazs> and when you created a pipeline that can be triggered based on time/gerrit event, then you can associate jobs with projects on that pipeline, example: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml#L142-L152
14:26:06 <dmsimard> https://docs.openstack.org/infra/jenkins-job-builder/
14:27:32 <dmsimard> myoung: the dashboards repo, you mean this https://dashboards.rdoproject.org/rdo-dev ?
14:28:11 <dmsimard> the code for it is here: https://github.com/rdo-infra/rdo-dashboards and the deployment is done with https://github.com/rdo-infra/ansible-role-rdo_dashboards
14:28:25 <dmsimard> both are under code review on review.rdoproject.org
14:28:37 <myoung> dmsimard: thx
14:29:54 <chandankumar> dmsimard: do we add third party ci in review.rdoproject.org?
14:30:25 <dmsimard> chandankumar: we have jobs in review.rdoproject.org that are running against the upstream gerrit
14:30:39 <dmsimard> chandankumar: these are the pipelines prefixed by openstack-*
14:31:07 <dmsimard> You'll find those pipelines here: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml
14:32:01 <chandankumar> dmsimard: thanks :-)
14:32:06 <dmsimard> Setting up a third party CI job is as simple as defining a job in the jobs/ repo and then adding it to the layout for a particular project in the right pipeline
14:35:27 <chandankumar> dmsimard: adarazs jpena thanks for the session.
14:35:30 <chandankumar> dmsimard++
14:35:33 <chandankumar> adarazs++
14:35:36 <chandankumar> jpena++
14:35:38 <rdogerrit> trown proposed config master: tripleo upstream periodic: add dlrn reporting as post build  https://review.rdoproject.org/r/10180
14:36:27 <adarazs> chandankumar: thanks for keeping track of this session!
14:37:04 <chandankumar> dmsimard: jpena if nothing more to discuss, can we close the office hour?
14:37:10 <jpena> ok for me
14:37:14 <jpena> (still in mtg)
14:37:19 <chandankumar> sure
14:37:22 <chandankumar> #endmeeting