16:00:04 <tiffanie> #startmeeting openstack_ansible_meeting
16:00:05 <openstack> Meeting started Tue Sep 10 16:00:04 2019 UTC and is due to finish in 60 minutes.  The chair is tiffanie. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:08 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:00:20 <tiffanie> #topic office hours
16:00:34 <jrosser> o/
16:00:40 <guilhermesp> o/
16:00:43 <cjloader> o/
16:01:40 <BjoernT> Hey
16:01:49 <mnaser> 👋
16:02:23 <weshay> 0/
16:02:30 <weshay> arxcruz ^
16:02:37 <chandankumar> \o/
16:03:38 <evrardjp> o/
16:06:44 <mnaser> thanks for your work on the py3 stuff jrosser
16:06:51 <mnaser> here's to hoping centos 7.7 shapes up soon
16:06:53 <jrosser> no worries :)
16:07:05 <jrosser> the ansible-runtime venv is now py3, thats merged
16:07:14 <mnaser> what about for centos?
16:07:26 <jrosser> bootstrap-ansible.sh detects what to do
16:07:29 <arxcruz> weshay: hey boss
16:07:50 <mnaser> ok
16:08:01 <mnaser> so hopefully we can kill it soon. :<
16:08:09 <mnaser> i do think tha tplacement extraction is becoming more of a priority
16:08:17 <chandankumar> jrosser: Does python3-libselinux available in centos 7.7?
16:08:18 <jrosser> there is a topic here for py3 https://review.opendev.org/#/q/topic:osa-py3+(status:open+OR+status:merged)
16:08:35 <evrardjp> thanks jrosser
16:08:35 <jrosser> chandankumar: afaik there are no centos 7.7 test nodes
16:09:02 <jrosser> the only thing so far which broke and i've not fixed is swift
16:09:02 <noonedeadpunk> o/
16:09:27 <mnaser> centos 7.7 is not out yet apparently..
16:09:46 <mnaser> its been a month or so
16:09:59 <jrosser> i will continue to make role patches to cover everything else for opt in/out of py3
16:10:00 <noonedeadpunk> so I'm working on the placement extract and upgrade jobs right now
16:10:10 <noonedeadpunk> And I think I have good enough results
16:10:22 <mnaser> thats awesome, upgrade jobs should forever solve us and "force" us to fix things because we cant land stuff :P
16:11:02 <noonedeadpunk> https://review.opendev.org/#/c/664867/
16:11:08 <chandankumar> jrosser: in RDO swift rpm for py3 is available now tested here https://review.rdoproject.org/r/#/c/21331/
16:12:56 <evrardjp> mnaser: +1 that's awesome
16:13:03 <noonedeadpunk> Yep, I guess we may try to move centos to python3 as well
16:13:10 <evrardjp> how long would those jobs take though?
16:13:13 <mnaser> i didn't do anything, that's all noonedeadpunk's work :)
16:13:17 <mnaser> i mean 2 hours tops maybe?
16:13:27 <mnaser> if our jobs are stable, it doesn't really matter how long they take to run in ci
16:13:34 <evrardjp> that's fair too
16:14:03 <chandankumar> mnaser: for centos 7.7, we can use cr repo http://mirror.centos.org/centos-7/7/cr/x86_64/Packages/ in experimental job
16:14:15 <noonedeadpunk> so right now upgrade jobs are not really deploying aio - they prepare env and then try to run upgrade.
16:14:24 <mnaser> oh there you go i see python3 in there chandankumar
16:14:25 <evrardjp> chandankumar: how stable is that?
16:14:36 <mnaser> CR is pretty stable.  it's like RC pretty much
16:14:37 <noonedeadpunk> PRobably this should be changed to real deployment without running tempest after first deployment
16:14:42 <jawad_axd> Hi all, I am putting this here again.I am having issue with heat after stein/centos7  install. Heat-engine service is not getting started properly and showing in logs as http://paste.openstack.org/show/774843/ , http://paste.openstack.org/show/774844/ ,  http://paste.openstack.org/show/774847/ .  Any help with this ?
16:14:58 <mnaser> CR is a clean rebuild of rhel packages without testing afaik
16:15:05 <mnaser> so this is what rhel ships
16:15:05 <evrardjp> so I am confused
16:15:35 <mnaser> rhel ships 7.7 packages => centos consumes sources and builds packages and puts inside CR repo => <qa/testing/whatever> => centos 7.7 is out
16:15:46 <mnaser> its the middle ground between rhel and centos release
16:15:53 <evrardjp> I thought py3 was shipped in RHEL8 and therefore centos8. Then it is to be brough for compatibility in rhel7.7 . I thought CR was for 7.6 to have a preview of 7.7/8 py3
16:16:00 <openstackgerrit> Merged openstack/openstack-ansible-os_trove master: Updated from OpenStack Ansible Tests  https://review.opendev.org/680061
16:16:16 <evrardjp> mnaser: I see
16:16:51 <evrardjp> so centos7.7 is out ?
16:17:03 <mugsie> nope
16:17:32 <chandankumar> evrardjp: mnaser but it will a test ground to prepare the stuff early
16:17:55 <chandankumar> mnaser: ah, that part I was not knowing that
16:18:28 <chandankumar> *will provide
16:19:31 <evrardjp> ok so 7.6's CR will give us the python3-devel (3.6) that we need. Cool, now I return to my previous understanding. That linked confused me :)
16:20:15 <evrardjp> chandankumar: so you're testing this for OSA?
16:21:00 <chandankumar> evrardjp: nope, I am was relying on this job https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021-master/84b1a81/job-output.txt.gz in rdo side
16:21:22 <chandankumar> where we enable cr repo and install it from that
16:21:36 <mnaser> CR repo will give us what centos 7.7 will be when they "tag" it
16:21:49 <evrardjp> yeah that was my understanding. Cool
16:21:55 <evrardjp> great.
16:22:00 <jrosser> mnaser: so we can make a depends-on patch for this https://review.opendev.org/#/c/680470/ that enables the repo?
16:22:32 <mnaser> yah and that should enable us to test py3 under centos
16:22:34 <jrosser> we'd need to do that early as part of bootstrap-ansible i think
16:22:45 <mnaser> yeah in the same python detection code i guess
16:22:58 <evrardjp> jrosser: it might be in two places
16:23:05 <evrardjp> it might require to be in two places
16:23:16 <evrardjp> sorry for the bad sed there :)
16:23:47 <evrardjp> today I also wanted to discuss releases and cjloader wanted to discuss ironic
16:24:09 <cjloader> hey all -- i'd like to get my ironic-inspector/os-ironic combined patch in before we freeze the roles. BjoernT noonedeadpunk jrosser mnaser evrardjp et al.
16:24:19 <mnaser> we're not freezing anything for quite sometime :)
16:24:37 <noonedeadpunk> Actually I've placed freeze https://review.opendev.org/#/c/681002/
16:24:43 <noonedeadpunk> (i Hope it's freeze)
16:24:51 <evrardjp> so let's clarify what freeze is first
16:25:10 <noonedeadpunk> ++
16:25:17 <evrardjp> technically openstack-ansible is an official project whose release model is cycle-trailing
16:25:37 <evrardjp> by being cycle-trailing we are allowed some kind of latitude in the project in terms of what we have to do.
16:25:44 <evrardjp> what it used to be is:
16:26:19 <evrardjp> we have 3 milestones, at last 2 weeks later than official milestones, and the first rc will branch. We then had two weeks to release final after others
16:27:07 <evrardjp> After discussions, we relaxed the two weeks after milestones, because some milestones basically overlapped between trailing and normal, leading to instabilities.
16:27:20 <evrardjp> We then relaxed the trailing the release for more than 2 weeks.
16:27:29 <evrardjp> So nowadays we're quite relaxed :D
16:28:16 <evrardjp> what it means, is that we _should_ (not must) have milestones, and must have at least one rc (for branching), and then at least a final release in the x (I think 2) months after final
16:28:54 <evrardjp> technically there was a rule that you could only propose a release if you had proposed milestones, but trailing wording allows us more flexibility
16:29:35 <evrardjp> Having milestones has two positive aspects: We know where we are in the cycle, and it allows testers to try a "frozen" version of what will be next release
16:29:52 <evrardjp> therefore it's a good to have a "frozen" milestone containing the features we want to show.
16:29:59 <evrardjp> That's what I would call feature freeze.
16:30:25 <evrardjp> Assuming we align on openstack, we should do that around m3, which is close to nowadays.
16:30:39 <noonedeadpunk> argh - got disconnected, just see what's used to be and conlusion...
16:30:44 * noonedeadpunk went reading archive
16:30:59 <BjoernT> so why do milestones before branching, if the master is still open ?
16:31:39 <evrardjp> a milestone is required for branching, because we need a tagged point to decide where to branch. That opens up master and train branch in this case
16:31:44 <evrardjp> sorry
16:31:47 <evrardjp> wrong wording
16:31:52 <evrardjp> a tag is required for branching
16:32:02 <evrardjp> in our case
16:32:28 <BjoernT> Well for thats equivalent of just a SHA ie point in time checkout but its fine, if the freeze is just a mile stone. But there is no additional testing done on that mile stone right ?
16:32:58 <evrardjp> no additional testing, just that all the shas are frozen, instead of being moving
16:33:07 <evrardjp> rc1 is generally the branching time
16:33:18 <evrardjp> m1/2/3 are just for "testing out"
16:33:30 <evrardjp> by m3 we should have an idea of what's included in the release
16:33:38 <jrosser> evrardjp: can you be precise about "all the SHA" ?
16:33:45 <jrosser> the roles? the openstack git repos?
16:33:48 <openstackgerrit> Merged openstack/openstack-ansible-ops master: Use version test instead of version_compare  https://review.opendev.org/681068
16:33:55 <evrardjp> the git repos are already kind of frozen
16:34:04 <evrardjp> git repos of openstack services and friends*
16:34:07 <evrardjp> the roles aren't
16:34:18 <cjloader> so basically no new features can go in the milestone? potential release?
16:34:33 <evrardjp> on a milestone we freeze the roles, we ensure it's stable and working, and then we unfreeze the roles
16:34:37 <BjoernT> no its just a snapshot
16:34:39 <noonedeadpunk> evrardjp: so I did kinde right thing with https://review.opendev.org/#/c/681002/ ?:)
16:34:58 <evrardjp> noonedeadpunk: that's exactly required for a milestone indeed
16:35:02 <BjoernT> is a user visible tag which is equivalent of checking out. OSA with a SHA
16:35:15 <evrardjp> correct
16:35:30 <evrardjp> having a milestone tagged has extra niceness for the generated documentation etc.
16:35:43 <BjoernT> yeah lipstick on a pig, lol
16:35:44 <evrardjp> it's good for us to have milestones
16:35:57 <evrardjp> oh yeah it's not like it's gonna change the world either
16:36:16 <evrardjp> but master is a moving target, milestone gives a way to test a "frozen" way
16:36:18 <noonedeadpunk> yep, agree that it's goodf to have that
16:36:43 <evrardjp> so if we are freezing the roles, changes need to be in before that, to be part of the milestone
16:36:48 <noonedeadpunk> sicne otherwise you can't check and test "master" in the middle of the development
16:36:54 <cjloader> so i'd like to get my patch in the milestone then
16:37:04 <BjoernT> no you can wait for the next onw
16:37:07 <BjoernT> one
16:37:12 <BjoernT> we just keep fixing things
16:37:15 <evrardjp> or you can wait for the next one too :)
16:37:28 <cjloader> how long until next one??
16:37:31 <evrardjp> It's both fine. I think I want to encourage you to do it :)
16:37:48 <evrardjp> how do you still have to do your patch?
16:38:06 <evrardjp> If you say that your patch can land this week, then it's probably fine for us to wait for this before proposing a milestone
16:38:09 <BjoernT> we estimate 1-2 weeks left
16:38:13 <evrardjp> mmm
16:38:21 <jrosser> as far as i see the patch looks sorta reasonable but it can't go anywhere until tests pass
16:38:28 <BjoernT> yepp
16:38:30 <evrardjp> agreed
16:38:38 <jrosser> if it were me i would be spending all the time in an AIO getting it to "work"
16:38:49 <evrardjp> so let's say that we won't block the milestone for that, and we'll just deal with that when it comes
16:38:50 <cjloader> i'd say let me get it passing gates
16:39:09 <evrardjp> mnaser: is there something you want in for the milestone by any chance?
16:39:10 <evrardjp> placement?
16:40:09 <noonedeadpunk> probably python3 as well
16:40:13 <mnaser> in an ideal world :)
16:41:01 <openstackgerrit> Merged openstack/openstack-ansible-lxc_container_create master: Use version test instead of version_compare  https://review.opendev.org/681216
16:41:14 <evrardjp> ok let's wait for py3/placement extract to be in, and propose the milestone then
16:41:21 <guilhermesp> do we want also the mgmt-bind ?
16:41:40 <jrosser> ^ that would be super nice
16:41:52 <jrosser> galera needs someone who understands to fix it
16:41:56 <guilhermesp> https://review.opendev.org/#/q/topic:bind-to-mgmt+(status:open+OR+status:merged) right now we still have to fix galera
16:42:04 <jrosser> well, or to know what actually needs doing
16:42:06 <guilhermesp> yeah I'm still trying to find a best solution
16:42:06 <mnaser> i dont think anyone actually understands galera
16:42:16 <guilhermesp> I have a local deployment to test things out
16:42:18 <mnaser> its just magic that works and you pray to the gods it doesnt break
16:42:21 <guilhermesp> not fully understanding the issue
16:42:40 <guilhermesp> it is something involving listening to locahost
16:42:53 <guilhermesp> mnaser: exactly
16:43:00 <jrosser> ok so the other thing that i have is ansible 2.9 support
16:43:02 <jrosser> https://review.opendev.org/#/q/topic:osa-ansible-2.9+(status:open+OR+status:merged)
16:43:02 <openstackgerrit> Merged openstack/openstack-ansible-lxc_hosts master: Use version test instead of version_compare  https://review.opendev.org/681214
16:43:23 <noonedeadpunk> I can take a look if it still be relevant right after resolving placement and upgrades
16:43:48 <jrosser> and i don't know if there is appetite for moving to 2.9 for T is. the release aligns favourably
16:43:51 <guilhermesp> noonedeadpunk: let me know if you need help with placement stuff
16:44:21 <noonedeadpunk> I'm ok right now - just takes a lot of time for testing since I re-create environment to get clean results
16:44:50 <noonedeadpunk> not every time, but still
16:45:58 <evrardjp> ok so we have a list
16:46:10 <evrardjp> let's not wait too much too :)
16:47:42 <jrosser> we have really good progress on these things right now - it's good work going on from everyone
16:51:56 <jrosser> mnaser: oh the other thing - os-vif for stein, what are we going to do about that?
16:52:24 <mnaser> jrosser: unfortunately unless we start creating a list of our own overrides which we then override with the lower constraints .. not much. :(
16:52:41 <mnaser> it seems like openstack/requirements will not bump up those versions because "those are for testing"
16:52:51 <mnaser> im not sure if we can use the in-repo local-constraints
16:53:04 <mnaser> err lower
16:53:57 <mnaser> im at a loss because i dont want us to start becoming a distro that maintains a second version of all those dependencies
16:54:00 <mnaser> that's a lot of work
16:54:00 <jrosser> it's been backported https://github.com/openstack/os-vif/commits/stable/stein
16:54:07 <mnaser> yeah but
16:54:24 <mnaser> https://github.com/openstack/requirements/blob/stable/stein/upper-constraints.txt
16:54:32 <mnaser> i dont know if that will be bumped
16:54:33 <mnaser> or can be bumped
16:54:57 <mnaser> maybe prometheanfire can explain but afaik it cannot be changed
16:55:20 <mnaser> or maybe it can be and we just need a tag of os-vif ?
16:58:22 <jrosser> also did folks see jawad_axd issue earlier? another case of python-systemd apparrently missing for logging
16:59:19 <jrosser> http://paste.openstack.org/show/774844/ i tried to advise a bit on this earlier but i don't have a centos deploy to poke around and compare
16:59:46 <noonedeadpunk> I'd say that python-systemd should be everywhere in master after migration to journald...
16:59:54 <noonedeadpunk> But it wasn't backported
17:00:41 <jrosser> thats why i'm a bit surprised about whats happening there - a journal logging error on stein
17:01:59 <noonedeadpunk> uwsgi on some roles has been switched already btw before landing patches...
17:03:53 * jrosser travels, bbl
17:04:40 <noonedeadpunk> but for heat no journald defined https://opendev.org/openstack/openstack-ansible-os_heat/src/branch/stable/stein/templates/heat.conf.j2#L9
17:05:49 <noonedeadpunk> probably it's just me who advised jawad_axd to replace logfile with journal
17:06:20 <noonedeadpunk> yesterday
17:10:13 <tiffanie> #endmeeting