13:31:34 #startmeeting RDO Office Hour - 2017-10-17 13:31:35 Meeting started Tue Oct 17 13:31:34 2017 UTC and is due to finish in 60 minutes. The chair is chandankumar. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:31:36 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:31:39 The meeting name has been set to 'rdo_office_hour___2017_10_17' 13:31:51 #topic Roll Call 13:32:03 o/ 13:32:08 o/ 13:32:09 hguemar: jpena amoralej adarazs ykarel|away dmsimard office hour 13:32:10 o/ 13:32:21 #chair aditya_r jpena adarazs 13:32:22 Current chairs: adarazs aditya_r chandankumar jpena 13:32:22 * jpena will have to run in 30 mins due to a conflicting meeting 13:32:30 o/ 13:32:47 #chair hguemar 13:32:48 Current chairs: adarazs aditya_r chandankumar hguemar jpena 13:33:34 adarazs: i am handing over the status to you 13:33:43 #topic Config Project session 13:33:53 adarazs: feel free to go ahead 13:34:59 s/status/stage 13:35:48 well. 13:35:50 sorry. 13:36:09 so I think we should go over different folders of the "config" repository. 13:36:15 adarazs: yup 13:36:32 I'm only familiar with the "jobs" directory and the "zuul" one. 13:36:50 I'll start with the "gerrit" and "gerritbot" folders 13:36:52 can somebody describe dashboards, gerrit, gerritbot, nodepool, policies? 13:37:04 o/ 13:37:05 and repoxplorer, resources. 13:37:07 dmsimard: ^^ 13:37:10 #chair myoung 13:37:11 Current chairs: adarazs aditya_r chandankumar hguemar jpena myoung 13:37:13 o/ 13:37:19 #chair amoralej 13:37:19 Current chairs: adarazs aditya_r amoralej chandankumar hguemar jpena myoung 13:37:26 chandankumar: python-tempestconf is objectively out of scope for puppet-openstack-integration, but that doesn't mean that in the python-tempestconf gate you can't use puppet-openstack-integration to deploy openstack, run tempestconf and then check that you get the expected configuration or something. 13:37:44 adarazs: I would summarize them as: 99% you don't need to touch them 13:38:09 I'll let jpena start and I can follow up I guess 13:38:21 The "gerrit" folder contains some configuration for Gerrit, namely hyperlink creation for commit messages (commentlinks.yaml) and Gerrit-GitHub replication configuration (replication.config) 13:38:22 tosky: I probably won't but I just got +2 rights on the repo, so it's better to know about them :) 13:38:50 In general, we may only touch replication.config if we want to add a project to Gerrit and it does not match one of the existing wildcards 13:38:50 BTW, I don't think I've seen it mentioned but the config repo is mirrored here: https://github.com/rdo-infra/review.rdoproject.org-config 13:39:41 dmsimard: that's a useful info. 13:39:46 dmsimard: I couldn't find it before. 13:40:01 adarazs: gerritbot is the configuration of the bot (so where to send which notification for which project) 13:40:30 about gerrit, is the gerrit configuration; see for example this review that adds a new project: https://review.openstack.org/#/c/508502/ 13:40:46 I think it's also helpful to draw parallels between the equivalence between upstream configuration and software factory configuration 13:41:31 I'm starting a pad here: https://review.rdoproject.org/etherpad/p/rdo-config-office-hours 13:41:39 we have automation for the new project creation, based on a post-job for rdoinfo changes 13:41:52 *jobs* directory: correct me if I'm wrong, but those are Jenkins Job Builder style definitions of jobs that are added to our Jenkins that Zuul can trigger on different conditions. the tripleo upstream related jobs are in jobs/tripleo-upstream.yml (surprisingly) 13:41:56 what I don't know if these are actually JJB config files or just something similar. are every jenkins plugins supported? 13:42:57 they are JJB config files. I'm not sure if all plugins are supported, dmsimard? 13:43:14 adarazs: jenkins plugins are available so long as they are installed just like any jenkins installation, however, I would not rely on the installation of plugins since the days of jenkins are counted 13:43:32 adarazs: for example, 'timeout' was installed because it is not a feature available by default iirc 13:44:06 asking because we're apparently using postbuild script and panda|rover is adding another one just now. 13:44:14 and I was wondering how we're going to solve that later. 13:45:26 adarazs: postbuildscript was not supported upstream, even before zuul v3 13:45:48 adarazs: in zuul v3, this would be a 'post-run' playbook which is built-in 13:46:16 adarazs: there is also a variable, 'zuul_success' that you can use in your playbooks to determine if the status of the run jobs were successful or not 13:46:56 dmsimard: and when are we going to switch to zuul v3 for rdoproject.org? 13:48:03 or at least what are the preliminary plans? :) 13:48:12 adarazs: We're expecting to start using Zuul v3 with the next release of Software Factory, 2.7, which is due within the next 3 weeks or so. We'll then need to plan the upgrade of Software Factory itself and then the plan is to run Zuul v2 and Zuul v3 side-by-side a bit like upstream did during the partial rollback. 13:48:50 all right. 13:49:02 Aditya Ramteke proposed openstack/novaclient-distgit rpm-master: Moved package description to global variable. https://review.rdoproject.org/r/9957 13:49:04 thank you for the timeline. 13:49:07 However, we want to run Zuul v2 and Zuul v3 simultaneously for as little time as possible as doing that is very expensive to maintain 13:49:46 chandankumar: o/ 13:49:48 Given that it's about a month out, hopefully upstream will have ironed out kinks with their jobs so the effort involved to translate review.rdo jobs to Zuul v3 should not be tremendous 13:49:52 #chair jschlueter 13:49:53 Current chairs: adarazs aditya_r amoralej chandankumar hguemar jpena jschlueter myoung 13:50:13 dmsimard: that's optimistic, but hey :D 13:50:28 I have a question: how we add an experimental job on review.rdoproject.org? 13:50:39 adarazs, weshay|ruck: however, we should definitely plan for the tripleo CI team availability when that time nears because we really don't want to maintain both Zuuls simultaneously for any extended period of time 13:50:47 sorry I missed part of the discussion, can we use postbuildscript in config in the end ? 13:51:04 chandankumar: there's an experimental pipeline, you put a job in that pipeline for a project and that allows you to trigger it with 'check experimental' 13:51:10 panda|rover: if it works currently I think we could, and zuul v3 will give us something instead that is built-in. 13:51:16 if not, is it ok if we don't mark build as failures ? 13:51:30 because this is what is going to happen without post scripts 13:52:03 panda|rover: you can use postbuildscript *currently* but you need to keep in mind that you need to translate those into playbooks that will run in zuul v3 'post-run', which is the equivalent of what 'postbuildscript' is. 13:52:38 panda|rover: you could probably save time if you started writing your post playbooks right away and then use postbuildscript to run those playbooks so that when zuul v3 comes around, your playbooks are mostly written 13:53:14 oh dear, this is like when I was doing an exam and the teacher was pulling the sheets under my pen, but i wanted to keep writing. 13:53:49 hehe 13:53:51 panda|rover: but then the teacher handed you a sheet with the answers written on it because zuul v3 is so awesome 13:54:50 panda|rover, adarazs: we need to discuss zuul v3 more broadly but right now is probably not the right time, I encourage you to sign up here: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123668.html 13:54:52 Aditya Ramteke proposed openstack/kuryr-distgit rpm-master: Move package description to global variable. https://review.rdoproject.org/r/9907 13:55:41 dmsimard: can i get an example review for the experimental job? 13:55:48 chandankumar: sure, one sec 13:56:07 dmsimard: ok, as usual there's no much time to do things the perfect way the first time, I'll add the post build, and add technical debt cards all aover to add the playbooks for the post-run 13:56:47 chandankumar: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/projects.yaml#L3535-L3573 13:57:30 panda|rover: up to you, if I was writing new content, I'd write it in ansible instead of bash, you're literally creating debt out of thin air if you're writing bash :D 13:57:45 dmsimard: thanks :-) 13:57:57 ok, let's continue with the overview of the config repository: https://github.com/rdo-infra/review.rdoproject.org-config 13:58:13 jpena explained gerrit and gerritbot ? I can talk about nodepool 13:58:20 yep 13:58:22 dmsimard: yup 13:58:50 our nodepool directory https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/nodepool is the equivalent to upstream's https://github.com/openstack-infra/project-config/tree/master/nodepool 13:59:19 There's several things going on there -- first is the actual nodepool configuration https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/nodepool/nodepool.yaml 14:00:20 This defines our disk images that are built through nodepool with diskimage-builder, so each image has a list of DIB elements picked from here: https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/elements and here: https://github.com/openstack-infra/project-config/tree/master/nodepool/elements 14:00:21 Merged rdoinfo master: Promote CBS tags update for newton-testing https://review.rdoproject.org/r/10172 14:00:56 We make the upstream elements available in order to be able to build images for TripleO that are similar to what upstream is doing, ideally with as little delta as possible 14:01:44 We've stopped synchronizing project-config a while back because upstream is moving full steam ahead to Zuul v3 and there are some changes in there that are likely to break our implementation. 14:02:24 Otherwise, in that nodepool configuration, this is also where you'll see our 'cloud providers' defined, what images we're uploading to each and the quota allocated to each provider. 14:03:25 The authentication information for nodepool is stored on the nodepool servers themselves rather than in-tree for obvious reasons :) 14:04:03 As far as software factory is concerned, this information is available in /etc/software-factory/sfconfig.yaml 14:05:14 dmsimard: https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/policies is used to update the SF configuration? 14:05:27 Other than that, in the nodepool directory ( https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/nodepool ) we have elements which are our DIB elements for various purposes like installing basic packages and such. You also have scripts, which are made available in.... a path I forget, on the virtual machines 14:05:45 chandankumar: yup, I'll get to the policies directory in a minute 14:05:52 dmsimard: sure 14:07:05 fbo, mhu: correct me if I'm wrong here? the policy.yaml file ( https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/policies/policy.yaml ) controls the ACL to the software factory (managesf) API 14:07:23 Aditya Ramteke proposed openstack/tacker-distgit rpm-master: Moved package description to global variable and replace with it. https://review.rdoproject.org/r/9552 14:07:44 I don't remember having to touch that file ever, so it's there and it's used but we don't need to play with it too much. 14:08:16 dmsimard, yes. The managesf API is consumed by the CLI sfmanager 14:09:09 it can be useful to allow some privileged users to hold test nodes for example 14:09:14 The repoexplorer directory is not in use right now because we haven't yet deployed it on review.rdoproject.org. It's a project that allows to provide statistics around the repositories hosted by the instance, it looks like this: https://softwarefactory-project.io/repoxplorer/ 14:09:33 mhu: okay, thanks 14:09:52 and the same policy.yaml file is tied with https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/resources folder update and create acls 14:10:30 chandankumar: it's not really tied, the resources directory is mostly around gerrit itself 14:10:50 that repoexplorer looks cool. 14:10:53 the resources directory defines the gerrit ACL for every project -- who can do what, who is in which group, and so on 14:10:55 * adarazs likes stats. 14:11:07 ah, thanks :-) 14:11:20 for example, this should be fairly self explanatory https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/resources/config.yaml 14:11:39 The idea is that the gerrit configuration is controlled by the config repository 14:12:56 dmsimard: currently if we open review.rdorproject.org main page 4 Associated git repositories related to barbican appears 14:13:19 how to get rid of it and it should display the list of projects i am in involved 14:13:23 If we need to add a new project to review.rdoproject.org, we'll need to create a resource for it through that mechanism. A review to add a project looks like this: https://review.rdoproject.org/r/#/c/10070/ or like this: https://review.rdoproject.org/r/#/c/9451/ 14:13:42 chandankumar: what do you define as the 'main page' ? 14:14:05 dmsimard: https://review.rdoproject.org/sf/welcome.html 14:15:10 chandankumar: I don't know to be honest, I don't really use that page :/ 14:15:16 Gabriele Cerami proposed config master: [WIP] tripleo upstream periodic: add dlrn reporting as post build https://review.rdoproject.org/r/10180 14:15:19 maybe mhu or fbo know how to customize the home page 14:15:58 Gabriele Cerami created rdo-infra/ci-config master: Adding dlnapi_reporting script https://review.rdoproject.org/r/10181 14:16:10 dmsimard, IIRC you can set a template in sfconfig.yaml for the landing page 14:16:29 mhu: yeah but why am I "associated" with some specific repos ? 14:17:18 dmsimard, good question, I don't know how the repos to display are chosen, to be honest 14:19:52 I think we've gone through most of the aspects of the config dir 14:20:01 unless there's any other questions, we can wrap this up 14:20:22 dmsimard: do we have something in gates, if i want to trigger only failed jobs not all? 14:20:27 once again, I encourage you to sign up to the zuul v3 "ask me anything" session next week: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123668.html 14:20:39 chandankumar: nope, every job will be retriggered 14:22:42 adarazs: anything you want to add? which is missed 14:23:21 I guess we might have wanted to have sshnaidm|off around :( 14:24:05 lots of info i have got, i will create a blog out of it 14:24:10 we didn't talk about the zuul pipelines, but I think they are fairly straightforward. 14:24:40 who "owns" the dashboards repo? I'm doing some work in a side project and will have some things to merge to there... 14:24:56 dmsimard: one more thing can you explain https://github.com/rdo-infra/review.rdoproject.org-config/tree/master/jobs part 14:25:02 you can create pipeline like so: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml#L58-L83 14:25:06 also - are there any details about networking, host, etc that it's running on? 14:25:09 dmsimard: there are bunch of files, what is for what? 14:25:45 chandankumar: the jobs directory is the equivalent of https://github.com/openstack-infra/project-config/tree/master/jenkins/jobs 14:25:55 chandankumar: it is jenkins configuration driven by jenkins job builder 14:26:04 and when you created a pipeline that can be triggered based on time/gerrit event, then you can associate jobs with projects on that pipeline, example: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml#L142-L152 14:26:06 https://docs.openstack.org/infra/jenkins-job-builder/ 14:27:32 myoung: the dashboards repo, you mean this https://dashboards.rdoproject.org/rdo-dev ? 14:28:11 the code for it is here: https://github.com/rdo-infra/rdo-dashboards and the deployment is done with https://github.com/rdo-infra/ansible-role-rdo_dashboards 14:28:25 both are under code review on review.rdoproject.org 14:28:37 dmsimard: thx 14:29:54 dmsimard: do we add third party ci in review.rdoproject.org? 14:30:25 chandankumar: we have jobs in review.rdoproject.org that are running against the upstream gerrit 14:30:39 chandankumar: these are the pipelines prefixed by openstack-* 14:31:07 You'll find those pipelines here: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml 14:32:01 dmsimard: thanks :-) 14:32:06 Setting up a third party CI job is as simple as defining a job in the jobs/ repo and then adding it to the layout for a particular project in the right pipeline 14:35:27 dmsimard: adarazs jpena thanks for the session. 14:35:30 dmsimard++ 14:35:33 adarazs++ 14:35:36 jpena++ 14:35:38 trown proposed config master: tripleo upstream periodic: add dlrn reporting as post build https://review.rdoproject.org/r/10180 14:36:27 chandankumar: thanks for keeping track of this session! 14:37:04 dmsimard: jpena if nothing more to discuss, can we close the office hour? 14:37:10 ok for me 14:37:14 (still in mtg) 14:37:19 sure 14:37:22 #endmeeting