Tuesday, 2018-01-30

openstackgerritMerged openstack/releases master: Release kolla-ansible 6.0.0.0b3  https://review.openstack.org/53818800:00
openstackgerritMerged openstack/releases master: Release Kolla 6.0.0.0b3  https://review.openstack.org/53818700:00
openstackgerritMerged openstack/releases master: Release Mistral Queens 3  https://review.openstack.org/53844200:00
smcginnisdhellmann: Have you spoken to anyone from searchlight by chance?00:00
dhellmannsmcginnis : no, I haven't00:00
dhellmannI guess we should start a special mailing list thread for each case that has a configuration issue00:03
dhellmannsmcginnis : shall we do a few more, or call it a night and pick up early tomorrow?00:07
smcginnisdhellmann: I'm feeling more confident. I think there's only three more left that are ready. Let's go for it.00:07
smcginnisdhellmann: I'll be in and out a bit, but I'll be checking in on status and can at least capture failures in the etherpad if we run into anything.00:07
dhellmannok, I approved the last 300:08
pabelanger\o/00:09
smcginnisSo we have cinder, searchlight, and blazar releases yet.00:09
smcginnisI just rechecked cinder.00:10
smcginnisSearchlight and blazar look to have config issues.00:10
dhellmannyeah00:10
smcginnisI wonder if we should just update them to have release-type: python-pypi.00:11
dhellmannWe could. My concern is whether the current job will actually build the right release artifacts.00:11
dhellmannThat was the point of adding this new validation, right?00:11
smcginnisYeah00:11
dhellmannby the same token we could just update project-config, but I have the same concern of making it do the wrong thing00:11
dhellmannso we really need people on those teams to be involved in sorting out the problems00:12
smcginnisRight, probably best to have them make that call.00:12
dhellmannhow about if I start a couple of email threads asking them to look at the settings and fix whichever is incorrect00:12
dhellmannand to ask here if they need help00:12
smcginnisOK, great. You can probably explain it better than I can. :)00:13
openstackgerritMerged openstack/releases master: Release python-cloudkittyclient 1.2.0 for Queens  https://review.openstack.org/53882900:22
openstackgerritMerged openstack/releases master: Release OpenStack-Ansible queens/17.0.0.0b3  https://review.openstack.org/53902800:25
smcginnisGreat to finally see that list get smaller.00:26
dhellmannsmcginnis : emails sent00:28
smcginnisLooks good.00:28
dhellmannsmcginnis : the tripleo release needs this code change in openstack/releases: https://review.openstack.org/#/c/537764/100:29
smcginnisI was just wondering why that one wasn't showing up in the queue.00:30
dhellmannme, too00:30
dhellmannthanks00:30
smcginnisI expect the recheck of cinder to be OK. I can approve that one later tonight if I see it go through.00:31
smcginnisAnd fingers crossed, blazar and searchlight will resolve their issues by tomorrow.00:31
smcginnisThen I would like to get all of your patches through.00:31
smcginnisWe still have glance.00:32
smcginnisAnd I see python-mistralclient with -W.00:32
*** dave-mccowan has quit IRC00:32
openstackgerritDoug Hellmann proposed openstack/releases master: allow list-deliverables to support multiple types  https://review.openstack.org/53762100:33
openstackgerritDoug Hellmann proposed openstack/releases master: constrain the valid values for the --type option to list-deliverables  https://review.openstack.org/53762200:33
openstackgerritDoug Hellmann proposed openstack/releases master: add client-library deliverable type  https://review.openstack.org/53762300:33
openstackgerritDoug Hellmann proposed openstack/releases master: change propose-library-branches to use yamlutils  https://review.openstack.org/53762400:33
openstackgerritDoug Hellmann proposed openstack/releases master: add --dry-run to propose-library-branches  https://review.openstack.org/53762500:33
openstackgerritDoug Hellmann proposed openstack/releases master: update list_library_unreleased_changes.sh to include client libraries  https://review.openstack.org/53763100:33
smcginnisNot sure if this is worth bothering, but maybe for historical accuracy: https://review.openstack.org/53791800:33
dhellmann+2 on that00:34
dhellmannrosmaita : how are things looking for glance?00:34
smcginnisThey have two patches that have been sitting in gate queue for several hours.00:35
smcginnishttps://review.openstack.org/#/c/538694/2 and https://review.openstack.org/#/c/536630/00:35
smcginnisThe one is almost through.00:35
smcginnisI know he did not want to approve any other patches in glance until that testing was restored.00:35
dhellmannmaybe we can get rosmaita to update the commit message for the release request to use depends-on instead of -W00:36
smcginnis++00:36
dhellmanncinder passed00:40
dhellmannjungleboyj : are you ready for a cinder tag?00:40
smcginnisThis should pass once it can get around zuul issues: https://review.openstack.org/53826500:41
dhellmannthat cinder tag won't include a bunch of commits so I wonder if it's using the right sha00:41
jungleboyjI was going to say, I thought I had taged.00:41
dhellmannhttp://logs.openstack.org/65/538265/1/check/releases-tox-list-changes/a348167/tox/list-changes-results.log00:41
dhellmannjungleboyj : please look at ^^ and see if that means you landed bonus patches or if the SHA in the request should have used the SHA of a merge commit00:42
smcginnisjungleboyj: This list: http://logs.openstack.org/65/538265/1/check/releases-tox-list-changes/a348167/job-output.txt.gz#_2018-01-30_00_24_59_04113000:43
dhellmannthere's a list of the commits that won't be included if we tag the current request00:43
dhellmannyeah00:43
smcginnisjungleboyj: I think those are just the things that have landed since the milestone was proposed.00:43
openstackgerritMerged openstack/releases master: project_config: add nodejs8 support  https://review.openstack.org/53776400:43
jungleboyjsmcginnis:  dhellmann  I think that is what we want though. I tagged when I was supposed to and the rest is what has come in since.00:44
dhellmannjungleboyj : ok, just checking00:44
dhellmannapproved00:44
smcginnis release-openstack-python http://logs.openstack.org/53/533a5ee424ebccccf6937f03d3b1d9d5b52e8ecb/pre-release/release-openstack-python/44f2fd4/ : FAILURE in 7m 58s00:44
jungleboyjsmcginnis:  Do you agree?00:44
smcginnisjungleboyj: Yeah, nothing there looks so critical that it has to wait I think.00:45
jungleboyjsmcginnis:  Ok, that was my feeling as well.00:45
EmilienMrelease-managers: thanks for your reviews :)00:46
jungleboyj++00:47
dhellmannsmcginnis : that failure is weird. was it for mistral-extra?00:47
smcginnisIt's been an interesting mileston. ;)00:47
smcginnisdhellmann: Yes.00:47
smcginnisSelf-loathing?00:47
smcginnis*milestone00:47
dhellmannit looks like maybe they didn't update their constraints handling00:48
dhellmannwhen the release failure email comes in we can forward it to the list for them to deal with00:48
openstackgerritMerged openstack/releases master: TripleO queens milestone 3 release  https://review.openstack.org/53775200:48
dhellmannthey tagged their b2 as b3 so they haven't had any changes since the second milestone00:48
smcginnisdhellmann: What? They already did a b3 tag?00:49
openstackgerritMerged openstack/releases master: add a script to build zuul commands to reenqueue jobs  https://review.openstack.org/53833900:49
dhellmannthe request that failed was for b3. it used the same sha as b2, which was HEAD of master00:49
dhellmannthe last change merged into the repo is from october00:49
smcginnisdhellmann: Oh, OK. I misunderstood. I thought you were saying last milestone they typod a b3 and were doing another.00:50
dhellmannhttps://review.openstack.org/#/q/project:openstack/mistral-extra00:50
dhellmannoh, sorry, no, I wasn't clear00:50
smcginnisOK, got it.00:50
dhellmannso this is an example of why we do these milestone tags00:50
dhellmannwe can let them sort out the failure00:50
smcginnisSo if nothing changed, why is it failing now?00:50
smcginnisBut yes, let's let them sort that out.00:51
dhellmannbecause mistral-extra appears in the constraints list and they aren't using any mechanism to ignore that when installing things in tox00:51
dhellmannI'm not sure why their tox environment is being used to build the package00:51
dhellmannmaybe mordred or ajaegar can explain00:51
smcginnisOh right. That makes sense now.00:51
dhellmannajeager00:51
smcginnisForwarding to openstack-dev now.00:52
dhellmannthey haven't noticed because they aren't landing patches at all in the repo00:52
dhellmannprobably because it's stable00:52
dhellmannI wish we had a way to test the packaging job as part of the validation step within the releases repo00:52
dhellmannmaybe when the magical "run the zuul job locally" client exists we can use it to run the job and skip a step00:53
smcginnisThat would be nice.00:54
dhellmannI approved the searchlight release since it passed the tests00:54
dhellmannand I think that's the last one we can do today00:54
smcginnisYep, I think that's a wrap. Now we wait for the problem ones to be resolved.00:55
dhellmanng'nite all00:57
smcginnisHave a good night dhellmann. See you tomorrow.00:58
openstackgerritMerged openstack/releases master: Create Cinder Queens-3 Tag  https://review.openstack.org/53826501:02
openstackgerritMerged openstack/releases master: Release Searchlight 4.0.0.0b3  https://review.openstack.org/53832001:08
*** melwitt has quit IRC01:10
*** jroll has quit IRC01:10
*** jroll has joined #openstack-release01:11
prometheanfiredo these releases need an FFE for reqs?01:30
*** derekjhyang has joined #openstack-release01:36
*** hongbin has joined #openstack-release01:39
*** melwitt has joined #openstack-release01:56
*** melwitt is now known as Guest9430601:56
*** Guest94306 is now known as jgwentworth01:58
*** jgwentworth is now known as melwitt02:01
*** gcb has joined #openstack-release02:04
*** yamamoto has joined #openstack-release02:05
openstackgerritMasahito Muroi proposed openstack/releases master: Release blazar-nova for queens  https://review.openstack.org/53813902:10
*** harlowja has quit IRC02:10
*** ekcs has quit IRC02:14
openstackgerritBrian Rosmaita proposed openstack/releases master: Release Glance Q-3  https://review.openstack.org/53828802:21
*** ykarel|away has joined #openstack-release02:23
rosmaitadhellmann smcginnis ^^ still really a DNM, but i added the depends-on you requested and lifted the -W02:30
*** ykarel|away has quit IRC02:38
*** hongbin has quit IRC02:46
*** yamamoto_ has joined #openstack-release02:47
*** hongbin has joined #openstack-release02:47
*** yamamoto has quit IRC02:50
*** d0ugal has quit IRC02:51
*** d0ugal has joined #openstack-release02:53
*** edmondsw has joined #openstack-release02:54
*** edmondsw has quit IRC02:58
*** dave-mccowan has joined #openstack-release02:58
*** zhurong has joined #openstack-release03:03
*** yamahata has quit IRC03:05
smcginnisrosmaita: Thanks!03:07
*** armax has quit IRC03:08
rosmaitawe are #23 and #24 in the queue03:17
rosmaitasomehow we seem to have moved downwards03:18
rosmaitaor maybe it's my imagination03:18
smcginnisrosmaita: Just can't get a break, huh.03:18
prometheanfirelol03:18
prometheanfirewait til zuul restarts again :P03:19
smcginnisHah, that will be around the point they get to #2 and #3 most likely.03:19
rosmaitait's my imagination ... i read back, they were #29 and #30 a few hours ago03:19
rosmaitawhere "few" == 603:20
smcginnisEmilienM: Not a rush, but if you're around, we had some of those tripleo jobs fail and it appears to be misconfiguration.03:20
EmilienMoops03:26
EmilienMI'm around03:26
smcginnisOh good, was just going to post to the ML to make sure it wasn't lost, but probably easier to just talk to you.03:27
EmilienMyeah i'm around03:27
smcginnisEmilienM: We had four failures.03:27
smcginnisEmilienM: tripleo-ipsec, os-refresh-config, os-collect-config, and os-apply-config.03:28
prometheanfiresmcginnis: talking about in reqs or releases?03:28
prometheanfiresmcginnis: also, those have been blocked unless an FFE is approved in reqs, fyi03:28
smcginnisprometheanfire: Releases.03:28
smcginnisprometheanfire: Ah. Sorry, I did see which patch number you put in your message, but we had several outstanding releases in queue.03:29
smcginnisprometheanfire: I think we're going to have several FFEs now.03:29
prometheanfireI imagine so03:29
EmilienMsmcginnis: any logs?03:29
smcginnisEmilienM: The os-*-config repos all have the release-notes-jobs template applied to them in project-config/zuul.d/projects.yaml.03:30
smcginnisEmilienM: But none of these have release notes, so when it tried to process that it got a failure.03:30
EmilienMdamn03:30
EmilienMsmcginnis: I could kick off a basic structure for reno03:31
smcginnisEmilienM: The tripleo-ipsec release job failed here: http://logs.openstack.org/a0/a0a3a51f951e7a1ae304f40d55f09361ebbb1c1d/release/release-openstack-python/a5f4a4b/ara/03:31
EmilienMand then update release files03:31
EmilienMand propose a new tag?03:31
smcginnisEmilienM: That one due to one of the zuul jobs expecting a venv tox target, which that repo's tox.ini does not have.03:31
EmilienMor just remove the reno jobs?03:31
smcginnisEmilienM: I think just remove reno jobs on those. It didn't block anything else, so we can probably just ignore those for now.03:31
smcginnisEmilienM: But would be good to remove the reno jobs before RC so there isn't any confusion.03:32
smcginnisEmilienM: The tripleo-ipsec one did tag the release, but there is no tarball on tarballs.o.o and the announcement and upper-constraint proposal was skipped.03:33
EmilienMI don't see reno jobs for ipsec weird03:33
smcginnisEmilienM: Yeah, doesn't look like that repo has them.03:34
EmilienMI was looking in project-config03:34
smcginnisEmilienM: But that repo doesn't have release notes either, so that one is good.03:34
smcginnisEmilienM: It's just the os-*-config repos that have it that should not.03:35
EmilienMok03:35
EmilienMso what's the problem with ipsec?03:35
smcginnisEmilienM: The job assumes there is a "venv" tox target, but that tox.ini does not define one.03:35
smcginnisEmilienM: So in that repo, you will need to add one similar to what you have here: https://github.com/openstack/tripleo-upgrade/blob/master/tox.ini#L7103:36
*** yamamoto has joined #openstack-release03:36
EmilienMsmcginnis: step 1 https://review.openstack.org/53908103:36
smcginnisEmilienM: Perfect03:37
EmilienMsmcginnis: step 2 https://review.openstack.org/53908203:38
smcginnisEmilienM: Probably don't need it, but for reference, here is the tox job that expects venv: https://github.com/openstack-infra/zuul-jobs/blob/master/roles/tox/tasks/siblings.yaml#L203:38
*** yamamoto_ has quit IRC03:39
smcginnisEmilienM: OK, I need to run. We'll discuss tomorrow what we can do to get the tarball created and update constraints proposed.03:40
EmilienMsmcginnis: ok03:40
EmilienMack03:40
EmilienMsmcginnis: thanks a lot03:40
EmilienMplease ping me anytime03:40
smcginnisEmilienM: No problem, glad we were able to resolve it so quickly.03:40
smcginnisHave a good night.03:41
prometheanfirelol, 95 jobs in gate03:41
smcginnisHey, better than when we were well over 100 earlier. ;)03:41
smcginnisBut it is always funny when the gate queue gets longer than the check queue.03:41
smcginnisAlways a sure sign of getting to the end of the cycle.03:42
EmilienMsmcginnis: good night too03:42
smcginnisOK, really leaving now. :)03:42
*** hongbin has quit IRC03:44
prometheanfireya03:46
*** yamahata has joined #openstack-release03:47
*** dave-mccowan has quit IRC03:52
*** coolsvap has joined #openstack-release04:00
*** sree has joined #openstack-release04:04
*** zhurong has quit IRC04:05
*** harlowja has joined #openstack-release04:21
*** armax has joined #openstack-release04:24
*** harlowja has quit IRC04:26
*** harlowja has joined #openstack-release04:26
*** harlowja has quit IRC04:27
*** armax has quit IRC04:28
*** ykarel has joined #openstack-release04:31
*** kumarmn has joined #openstack-release04:33
*** armax has joined #openstack-release04:35
*** kumarmn has quit IRC04:37
*** rosmaita has quit IRC04:39
*** edmondsw has joined #openstack-release04:42
*** edmondsw has quit IRC04:46
*** armax has quit IRC05:05
*** armax has joined #openstack-release05:06
*** ricolin has joined #openstack-release05:32
*** zhurong has joined #openstack-release05:42
*** ekcs has joined #openstack-release05:43
*** jtomasek has quit IRC06:24
*** jtomasek has joined #openstack-release06:24
*** edmondsw has joined #openstack-release06:30
*** edmondsw has quit IRC06:34
*** ricolin has quit IRC06:42
*** yamamoto_ has joined #openstack-release06:43
*** yamamoto has quit IRC06:46
*** ricolin has joined #openstack-release06:53
*** ekcs has quit IRC06:58
*** zhurong has quit IRC06:58
*** zhurong has joined #openstack-release07:02
*** jtomasek has quit IRC07:05
*** pcaruana has joined #openstack-release07:10
*** jtomasek has joined #openstack-release07:27
*** iyamahat has joined #openstack-release07:30
*** d0ugal has quit IRC07:34
*** armax has quit IRC07:41
*** alexchadin has joined #openstack-release07:49
*** jtomasek has quit IRC07:54
*** d0ugal has joined #openstack-release08:02
*** edmondsw has joined #openstack-release08:18
*** edmondsw has quit IRC08:23
*** iyamahat has quit IRC08:43
*** lennyb has quit IRC08:50
*** jpich has joined #openstack-release08:58
*** amoralej|off is now known as amoralej09:04
*** yamahata has quit IRC09:08
*** shardy has joined #openstack-release09:20
*** ricolin has quit IRC09:30
*** lennyb has joined #openstack-release09:34
openstackgerritChandan Kumar proposed openstack/releases master: Release python-tempestconf-1.1.4  https://review.openstack.org/53884009:41
*** jtomasek has joined #openstack-release10:00
*** zhurong has quit IRC10:02
*** edmondsw has joined #openstack-release10:06
*** yamamoto has joined #openstack-release10:08
*** edmondsw has quit IRC10:10
*** yamamoto_ has quit IRC10:11
*** yamamoto has quit IRC10:12
*** edmondsw has joined #openstack-release10:12
*** edmondsw has quit IRC10:12
*** yamamoto has joined #openstack-release10:14
*** yamamoto has quit IRC10:14
*** sree has quit IRC10:29
*** alexchadin has quit IRC10:30
*** alexchadin has joined #openstack-release10:31
*** dave-mccowan has joined #openstack-release10:32
ttxRechecked both blazar entries since the release-job updates are now merged10:40
*** lucas-afk is now known as lucasagomes10:41
*** tellesnobrega has quit IRC10:43
*** pbandark has joined #openstack-release10:46
*** alexchadin has quit IRC10:47
*** alexchadin has joined #openstack-release10:49
*** kumarmn has joined #openstack-release10:49
*** kumarmn has quit IRC10:53
*** yamamoto has joined #openstack-release10:59
*** alexchadin has quit IRC11:02
*** alexchadin has joined #openstack-release11:03
*** alexchadin has quit IRC11:03
*** alexchadin has joined #openstack-release11:03
*** alexchadin has quit IRC11:04
*** yamamoto has quit IRC11:04
*** alexchadin has joined #openstack-release11:04
*** yamamoto has joined #openstack-release11:04
*** alexchadin has quit IRC11:04
*** alexchadin has joined #openstack-release11:05
*** alexchadin has quit IRC11:05
*** alexchadin has joined #openstack-release11:06
*** alexchadin has quit IRC11:06
*** d0ugal has quit IRC11:34
*** pkovar has joined #openstack-release11:35
*** yamamoto has quit IRC11:36
*** yamamoto has joined #openstack-release11:46
*** alexchadin has joined #openstack-release11:49
*** yamamoto has quit IRC11:51
*** d0ugal has joined #openstack-release11:54
*** d0ugal has quit IRC11:54
*** d0ugal has joined #openstack-release11:54
*** tellesnobrega has joined #openstack-release12:04
*** edmondsw has joined #openstack-release12:08
*** yamamoto has joined #openstack-release12:10
*** pkovar has quit IRC12:10
*** zhurong has joined #openstack-release12:13
*** yamamoto has quit IRC12:14
*** alexchadin has quit IRC12:17
*** alexchadin has joined #openstack-release12:18
*** sree has joined #openstack-release12:19
*** coolsvap has quit IRC12:19
*** alexchadin has quit IRC12:22
*** alexchadin has joined #openstack-release12:22
*** wolverineav has joined #openstack-release12:30
*** yamamoto has joined #openstack-release12:35
*** yamamoto_ has joined #openstack-release12:40
*** yamamoto_ has quit IRC12:43
*** yamamoto has quit IRC12:43
*** edmondsw_ has joined #openstack-release12:49
*** pkovar has joined #openstack-release12:51
*** edmondsw has quit IRC12:51
*** gcb has quit IRC12:56
*** edmondsw_ has quit IRC12:58
*** edmondsw has joined #openstack-release13:00
*** zhurong_ has joined #openstack-release13:03
*** sree has quit IRC13:08
*** zhurong has quit IRC13:17
smcginnisttx: What were the release-job updates?13:17
*** zhurong_ has quit IRC13:19
*** zhurong has joined #openstack-release13:20
*** zhurong has quit IRC13:20
*** tellesnobrega has quit IRC13:21
*** alexchadin has quit IRC13:21
*** ricolin has joined #openstack-release13:23
*** amoralej is now known as amoralej|lunch13:28
*** ykarel is now known as ykarel|away13:31
*** rosmaita has joined #openstack-release13:32
ttxsmcginnis: they had to adjust the associated release job, see ML13:33
*** sree has joined #openstack-release13:35
*** ykarel|away has quit IRC13:36
*** ricolin has quit IRC13:36
*** dave-mccowan has quit IRC13:37
smcginnisOh right, the job changes.13:41
*** sree has quit IRC13:41
smcginnisGuess we need to wait a bit while the new image issues are being sorted.13:42
*** ricolin has joined #openstack-release13:42
-openstackstatus- NOTICE: Our ubuntu-xenial images (used for e.g. unit tests and devstack) are currently failing to install any packages, restrain from *recheck* or *approve* until the issue has been investigated and fixed.13:43
*** yamamoto has joined #openstack-release13:44
rosmaitaarrrrgh13:44
ttxoops13:45
openstackgerritRico Lin proposed openstack/releases master: Release heat queens-3 10.0.0.0b3  https://review.openstack.org/53921213:48
*** yamamoto has quit IRC13:49
*** alexchadin has joined #openstack-release13:52
*** jpich has quit IRC13:54
*** kumarmn has joined #openstack-release13:54
dhellmanno/13:54
openstackgerritRico Lin proposed openstack/releases master: release python-heatclient 1.13.1  https://review.openstack.org/53921713:55
*** alexchadin has quit IRC13:55
*** alexchadin has joined #openstack-release13:56
*** sree has joined #openstack-release13:58
*** tellesnobrega has joined #openstack-release13:58
openstackgerritRico Lin proposed openstack/releases master: Release heat-agents 1.5.2  https://review.openstack.org/53921913:58
*** alexchadin has quit IRC14:00
*** sree_ has joined #openstack-release14:02
*** sree_ is now known as Guest4157414:03
*** sree has quit IRC14:05
smcginnisrosmaita: See, I told you. :]14:09
*** yamamoto has joined #openstack-release14:11
*** yamamoto has quit IRC14:14
*** yamamoto has joined #openstack-release14:15
*** yamamoto has quit IRC14:15
*** ykarel|away has joined #openstack-release14:17
rosmaitasmcginnis i hate it when you're right14:20
rosmaita:)14:20
smcginnisMe too. :)14:22
*** derekjhyang has quit IRC14:26
*** alexchadin has joined #openstack-release14:30
*** lucasagomes is now known as lucas-hungry14:36
*** amoralej|lunch is now known as amoralej14:39
*** yamamoto has joined #openstack-release14:40
*** yamamoto has quit IRC14:43
*** d0ugal has quit IRC14:44
*** d0ugal has joined #openstack-release14:47
*** yamamoto has joined #openstack-release14:48
*** dave-mccowan has joined #openstack-release14:55
*** d0ugal has quit IRC14:58
openstackgerritSteve McLellan proposed openstack/releases master: Release searchlight-ui 4.0.0.0b3  https://review.openstack.org/53832115:02
*** tellesnobrega has quit IRC15:03
*** yamahata has joined #openstack-release15:05
*** alexchadin has quit IRC15:05
*** yamamoto has quit IRC15:12
dhellmannsmcginnis , ttx, dims: what do we want to do with the neutron-vpnaas release? https://review.openstack.org/53829515:15
*** hongbin has joined #openstack-release15:16
*** zhongjun has quit IRC15:18
smcginnisdhellmann: Akihiro's response sounds reasonable.15:24
dhellmannthat works for me15:25
smcginnisChallenge would be if there was something more than a bugfix. We would be stuck unless we make the next release 13.0 or skip 12.1 and go to 12.2, right?15:25
*** d0ugal has joined #openstack-release15:25
dhellmannif they're going to move back to cycle-with-milestones next cycle they'll be going to 13 anyway15:26
dhellmannotherwise the validation logic will require the next release from master to be 12.115:26
smcginnisOh right.15:26
smcginnisThen that should be good.15:26
smcginnisWe also have an issue to resolve yet with tripleo-ipsec.15:26
amotokiI think we have two rooms. Queens initial release would be 12.0.0 and we can release bug fixes with 12.0.[1-...]15:27
smcginnisThe release job failed with that due to a job issue. It tagged, but did not publish anything.15:27
amotoki12.1.x is not used from the point of view of semantic versioning15:27
dhellmannamotoki : yes, that's likely to be correct15:27
amotokii will update the neutron-vpnaas release review as my hat of neutron-vpnaas core.15:28
*** yamamoto has joined #openstack-release15:30
dhellmannamotoki : thanks!15:30
*** gcb has joined #openstack-release15:31
openstackgerritAkihiro Motoki proposed openstack/releases master: Neutron-vpnaas milestone 3  https://review.openstack.org/53829515:31
*** armax has joined #openstack-release15:33
*** lucas-hungry is now known as lucasagomes15:34
ttx+115:37
ttxthe commit message could be updated15:38
*** rosmaita has quit IRC15:59
*** rosmaita has joined #openstack-release15:59
ttxdhellmann, smcginnis the gate looks slightly shorter now... should we process heat q3 ?16:01
*** rosmaita has quit IRC16:01
ttxhttps://review.openstack.org/#/c/539212/16:01
ttxhappy to +2a if all clear16:01
ttxsame for blazar-nova and blazar-dashboard16:02
*** alexchadin has joined #openstack-release16:03
dhellmannttx: go for it. I have a meeting starting but I'll be back in ~1 hr16:03
ttxack16:03
ttxI'm following the board call but that only takes one of my CPUs16:04
ttxdhellmann: maybe remove your -1 from https://review.openstack.org/#/c/53817516:04
prometheanfirettx: gate is still broken iirc16:04
prometheanfire07:45 -openstackstatus:#openstack-release- NOTICE: Our ubuntu-xenial images (used for e.g. unit tests and devstack) are currently failing to install any packages, restrain from *recheck* or *approve* until the issue has been investigated and fixed.16:04
dhellmannttx: done16:04
ttxfungi: should we still refrain from approving releases ?16:06
*** tellesnobrega has joined #openstack-release16:07
*** rosmaita has joined #openstack-release16:08
fungittx: things are a little more stable now that we have the image problems addressed, but we've been pointed to a network rate limiting issue for our mirror in rax dfw which is causing some job timeouts and errors so may still want to hold off while we rebalance there and get a ticket files with the provider16:08
ttxok, holding16:08
*** kumarmn has quit IRC16:09
*** sree has joined #openstack-release16:10
openstackgerritAkihiro Motoki proposed openstack/releases master: Release neutron-vpnaas for queens  https://review.openstack.org/53829516:10
openstackgerritRico Lin proposed openstack/releases master: Release heat-dashboard 1.0.1  https://review.openstack.org/53924316:12
*** kumarmn_ has joined #openstack-release16:12
*** Guest41574 has quit IRC16:13
smcginnisOK, good. I thought there were still issues being resolved, so I was holding off.16:15
smcginnisHopefully just a little longer.16:15
*** kumarmn_ has quit IRC16:17
prometheanfireya, I'm holding off for now as well16:17
prometheanfirehoping we get our gate time increase in16:17
*** yamamoto has quit IRC16:17
*** ykarel|away has quit IRC16:20
*** yamamoto has joined #openstack-release16:20
*** kumarmn has joined #openstack-release16:21
*** kumarmn has quit IRC16:23
*** kumarmn has joined #openstack-release16:24
*** yamamoto has quit IRC16:25
*** kumarmn has quit IRC16:27
*** kumarmn has joined #openstack-release16:28
*** kumarmn has quit IRC16:28
*** inc0 has quit IRC16:33
*** inc0 has joined #openstack-release16:33
*** kumarmn has joined #openstack-release16:34
*** alexchadin has quit IRC16:35
*** jtomasek has quit IRC16:36
*** kumarmn has quit IRC16:39
*** kumarmn has joined #openstack-release16:40
*** kumarmn has quit IRC16:45
*** kumarmn has joined #openstack-release16:45
*** pkovar has quit IRC16:53
*** kumarmn has quit IRC16:54
*** kumarmn has joined #openstack-release16:55
*** jtomasek has joined #openstack-release16:57
*** jtomasek has quit IRC16:58
*** kumarmn has quit IRC16:59
*** gcb has quit IRC17:00
*** pkovar has joined #openstack-release17:01
*** kumarmn has joined #openstack-release17:06
*** sree has quit IRC17:08
*** kumarmn has quit IRC17:10
*** sree has joined #openstack-release17:12
*** kumarmn has joined #openstack-release17:12
*** kumarmn has quit IRC17:16
*** kumarmn has joined #openstack-release17:16
*** sree has quit IRC17:17
*** kumarmn has quit IRC17:18
*** kumarmn has joined #openstack-release17:19
*** pbandark has quit IRC17:21
*** pbandark1 has joined #openstack-release17:21
*** yamamoto has joined #openstack-release17:21
*** pbandark1 is now known as pbandark17:23
*** kumarmn has quit IRC17:24
*** kumarmn has joined #openstack-release17:24
*** kumarmn_ has joined #openstack-release17:27
*** yamamoto has quit IRC17:27
*** kumarmn has quit IRC17:28
*** kumarmn has joined #openstack-release17:28
*** kumarmn_ has quit IRC17:31
sjmc7dhellmann: thanks for the email about the searchlight-ui config, not sure why that hadn’t come up before17:43
*** electrofelix has quit IRC17:49
*** electrofelix has joined #openstack-release17:50
rosmaitasmcginnis: #55 and #56 in the integrated queue17:50
smcginnisrosmaita: I'll keep my fingers crossed. :)17:51
*** armstrong has joined #openstack-release17:51
dhellmannsjmc7 : we added more validation after the zuul v3 migration so maybe that's it17:55
*** alexchadin has joined #openstack-release17:58
*** iyamahat has joined #openstack-release17:58
EmilienMsmcginnis: FYI, the 1) and 2) things that we talked about merged18:16
*** ricolin has quit IRC18:16
openstackgerritKendall Nelson proposed openstack/releases master: Fix Email Deadline & Nominations  https://review.openstack.org/53928218:20
*** yamamoto has joined #openstack-release18:23
*** alexchadin has quit IRC18:27
smcginnisEmilienM: Great, thanks!18:28
*** yamamoto has quit IRC18:29
fungithe max-servers reduction via https://review.openstack.org/539255 merged around 15 minutes ago, so just waiting for the dark blue line at http://grafana.openstack.org/dashboard/db/nodepool-rackspace?panelId=14&fullscreen to drop from 140 to 100 and then hopefully we'll see some reduction on http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=64259&rra_id=all to be comfortably18:30
fungibelow 100Mbps18:30
smcginnisdhellmann: Now I forget, Since that tripleo-ipsec job tagged the release, we can't just re-enqueue it, correct?18:30
fungiat that point i expect jobs to be reasonably stable again18:30
smcginnisfungi: That gets rid of the slow nodes?18:30
fungiwasn't slow nodes, it was packet loss communicating with the mirror server because the provider seems to have sudenly started rate-limiting its bandwidth to 100Mbps18:31
smcginnis90's mode enabled.18:31
fungimade transfers slow since packet loss means lots of retransmits18:31
*** alexchadin has joined #openstack-release18:38
*** alexchadin has quit IRC18:41
*** amoralej is now known as amoralej|off19:06
*** ekcs has joined #openstack-release19:06
*** alexchadin has joined #openstack-release19:09
*** alexchadin has quit IRC19:11
*** pkovar has quit IRC19:14
*** harlowja has joined #openstack-release19:17
*** yamamoto has joined #openstack-release19:25
*** lucasagomes is now known as lucas-afk19:26
*** ekcs has quit IRC19:27
*** yamamoto has quit IRC19:31
*** shardy has quit IRC19:42
*** alexchadin has joined #openstack-release19:43
fungiokay, bandwidth utilization seems to have steadily dropped off for mirror01.dfw.rax.o.o, so should be safe to assume we're back to "normal" levels of stability i think19:45
smcginnisfungi: So all clear to approve a few more releases?19:49
fungiyeah, it's still not clear whether all the load on that mirror is thanks to our jobs or whether there's a third-party ci system contributing to the bandwidth utilization, but things are looking okay for now19:50
*** pcaruana has quit IRC19:52
smcginnisGreat, thanks fungi.19:52
*** corvus has joined #openstack-release19:53
mordredsmcginnis, dhellmann, ttx: when does devstack get branched currently? there are some new interactions with stable branches of projects using jobs defined in the devstack repo ...19:54
mordredcorvus: ^^19:54
smcginnismordred: Hmm, I thought at RC.19:54
mordredsmcginnis: do we do it later in the process to deal with a specific thing?19:57
mordredI'm asking because right now a patch to stable/queens of openstacksdk currently does not run any of the devstack integration tests because there is no stable/queens branch of devstack ... and trying to start working through what the right way to deal with that is19:57
smcginnismordred: Not entirely sure. See the RC1 section here: https://github.com/openstack/releases/blob/master/PROCESS.rst19:57
mordred"After the minimum set of projects used by devstack have been branched, the devstack branch can be created. Devstack doesn't push a tag at RC1 it is just branched off of HEAD"19:58
mordredcorvus: ^^19:58
corvusis that now?20:01
ttxno, RC1 is usually a couple week after FF20:01
ttxand FF is... a bit delayed20:02
corvuswhen do the devstack-required projects branch?  (ie, nova,keystone,etc)20:02
smcginnisAt RC.20:02
smcginnisNext Thursday.20:03
ttx-ish20:03
corvusokay, so devstack+nova+keystone will all branch at around the same time -ish?20:03
mordreddo we know what happens if there is a stable/queens of devstack but no stable/queens of nova?20:03
*** alexchadin has quit IRC20:03
ttxcorvus: yes20:03
ttxmordred: ISTR that creates issues, hence the rule in PROCESS20:04
ttxmight be cargo-culted from an environment that non longer applies though20:04
mordredyah, oh, I'm sure it breaks something - I'm curious what the issues were20:04
mordredit also could be that people working on the services up to feature freeze need to also land changes in devstack to support those features20:05
smcginnisOh, that could very well be.20:06
dhellmannsmcginnis : since the tag exists, the tag-releases job will say "nothing to do here" and skip the step. That's why we have to manually tell zuul about the tag event again.20:08
dhellmannmordred : I think it's the landing of things up to FF that motivates us to wait to branch devstack20:09
smcginnisdhellmann: How do we "manually tell zuul about the tag event"?20:09
dhellmannthat's the thing our infra folks are doing when they run those commands20:10
dhellmann"those commands" -- very specific, I know20:10
smcginnis:)20:10
dhellmannsomeone with special access to zuul has to do that step20:11
dhellmannthe "zuul enqueue-ref" commands20:11
smcginnisdhellmann: So it's a different command than just re-enqueing a failed job.20:12
dhellmannmy mental model of that command is that it tells zuul "pretend you just saw this $ref come in"20:12
smcginnisHah, seems like a good mental model for that.20:12
dhellmannno, it's the same thing, that's what that command is doing20:12
smcginnisSo if we ask fungi real nice, he could maybe redo http://logs.openstack.org/a0/a0a3a51f951e7a1ae304f40d55f09361ebbb1c1d/release/release-openstack-python/a5f4a4b/20:13
smcginnis:)20:13
openstackgerritMerged openstack/releases master: Release heat-dashboard 1.0.1  https://review.openstack.org/53924320:14
dhellmannsmcginnis : yes20:14
*** iyamahat has quit IRC20:14
fungisure20:15
* dhellmann hands fungi a cup of sake20:15
smcginnisOoh, sake. Getting fancy now.20:15
fungii have a bottle of okunomatsu saké tokubetsu junmai in the fridge waiting for me20:16
smcginnisI have a few good ones and hoping to pick up some more at the Tokyo ops midcycle. :)20:17
smcginnisIf any of you end up in Minneapolis for some reason, would love to take you to a place that brews their own sake here. It's suprisingly good.20:18
fungismcginnis: i'd love to! i'll remember that next time i'm there20:19
smcginnisfungi: I owe you some beverage too!20:19
fungiclarkb also mentioned that momokawa in forest grove oregon does tours and tastings20:19
fungiconveniently close to portland20:20
smcginnisShoot, I was just out there visiting my sister. I'll have to make a note of that for next time.20:20
fungiif i'm there for oscon, i may try to plan for an excursion20:20
smcginnisDon't think I can make oscon, but if I do, I may want to join you.20:20
fungimomokawa is hands down the best domestic sake i've tasted here in the usa20:21
smcginnisI don't think I've tried it. I'll have to look for that.20:21
fungitheir "silver" is good. i also like the g joy genshu (under their "sakeone" brand)20:22
* smcginnis takes notes20:22
fungieasily import quality at domestic prices20:23
*** d0ugal has quit IRC20:26
*** yamamoto has joined #openstack-release20:27
fungilooks like the tripleo-ipsec release-openstack-python job is running20:29
smcginnisHmm, same error.20:30
smcginnisI thought their change merged that was supposed to fix that.20:31
smcginnisAbout to give a presentation, so I will have to check on that a little later.20:31
*** yamamoto has quit IRC20:33
fungiwas the change covered by that tag, or after the tag?20:36
openstackgerritMerged openstack/releases master: Release blazar-dashboard 1.0.0 for Queens  https://review.openstack.org/53817520:38
*** armstrong has quit IRC20:39
ttxhmmm sake20:39
*** armstrong has joined #openstack-release20:39
*** d0ugal has joined #openstack-release20:42
*** armstrong has quit IRC20:44
*** armstrong has joined #openstack-release20:47
fungismcginnis: looking back at my notes, it was the momokawa "diamond" i was thinking of, actually, not the silver20:51
smcginnisThanks!20:53
*** wolverineav has quit IRC20:53
smcginnisEmilienM: Looks like we actually need an 8.0.1 I guess.20:53
smcginnisThis merged: https://github.com/openstack/tripleo-ipsec/commit/00df8e1adb939d357805eadefaf54f4f326ee21f20:54
smcginnisFixing this: http://logs.openstack.org/a0/a0a3a51f951e7a1ae304f40d55f09361ebbb1c1d/release/release-openstack-python/b1b8f84/job-output.txt.gz#_2018-01-30_20_30_23_07581320:54
EmilienMsmcginnis: ok I'm out for lunch but I'll propose asap20:54
smcginnisBut the commit hash will need to be updated to get the fix included.20:54
smcginnisBut since we already tagged 8.0.0, we'll have to just cut our losses on that and do 8.0.1.20:54
smcginnisEmilienM: Not a huge rush, enjoy your lunch.20:55
prometheanfirehttps://gist.github.com/prometheanfire/f4e710363ebbe9a743753cfaec877847   those are the open reviews to get in before reqs is considered fully frozen (baring new FFEs)20:57
*** iyamahat has joined #openstack-release21:08
*** iyamahat has quit IRC21:09
*** iyamahat has joined #openstack-release21:09
smcginnisprometheanfire: Is that based on what's out here: http://tiny.cc/4atwny21:11
prometheanfiresmcginnis: no21:13
prometheanfiresmcginnis: anything in the queue that needs to get in needs a FFE as of yesterday21:14
* prometheanfire did send email :P21:14
smcginnisprometheanfire: Yeah, saw that. Just thought this gist was an update to that.21:14
prometheanfirenah, that's a list of reqs reviews, atm it's the canonical list21:15
smcginnisAh, gotcha.21:15
*** armstrong has quit IRC21:16
*** armstrong has joined #openstack-release21:17
*** armstron_ has joined #openstack-release21:21
*** armstrong has quit IRC21:21
*** yamamoto has joined #openstack-release21:29
*** armstron_ has quit IRC21:32
*** armstrong has joined #openstack-release21:33
*** ianychoi has quit IRC21:34
*** ianychoi has joined #openstack-release21:35
*** yamamoto has quit IRC21:35
*** armstron_ has joined #openstack-release21:36
*** armstrong has quit IRC21:36
dhellmannsmcginnis : how are things looking? is there anything I can help with?21:43
openstackgerritMerged openstack/releases master: Release blazar-nova for queens  https://review.openstack.org/53813921:43
openstackgerritMerged openstack/releases master: Release python-mistralclient 3.1.5 for Queens  https://review.openstack.org/53843821:43
smcginnisdhellmann: I've been feeding a few things through.21:43
smcginnisdhellmann: Just need a recheck on searchlight-ui once the project-config change goes through.21:44
smcginnisThen we've got the glance release that's still blocked pending two patches for the last 2+ days.21:44
dhellmannwe should probably go ahead and cut that one as-is and they can use FFEs to land those21:45
smcginnisWe'd have to update the depends-on.21:45
smcginnisHmm, yeah. The two patches are in gate, but they've just been queued for 4 hours now waiting.21:46
smcginnisrosmaita: Thoughts on above? ^^21:48
openstackgerritEmilien Macchi proposed openstack/releases master: Release tripleo-ipsec 8.0.1  https://review.openstack.org/53934021:55
EmilienMsmcginnis: done ^21:56
smcginnisThanks!21:56
rosmaitadhellmann really don't want to cut a release without those patches, they aren't features, they restore the functional tests, which haven't been run since the ubuntu py2.7 distro update22:01
rosmaitasmcginnis ^^22:01
smcginnisrosmaita: Is the fear that they will show a failure that has already merged?22:02
smcginnisrosmaita: Otherwise, that affects work on new patches going in, but wouldn't affect someone trying to test with the milestone release.22:03
smcginnisTest as in try deploying and using, obviously.22:03
rosmaitawell, my feeling is that those patches have to land before we can allow (i guess before i will allow) anything else to land, so they have to go through anyway.  Also, the functional gate broke before the image import property injection feature landed, so that's in Q-3 not having been functionally tested in-gate, just tested locally22:06
rosmaitaso there is a possibility there's something broken in there22:06
rosmaitaalso, the requirements gate removed the glance py27 test, so who knows what else has sneaked in22:07
openstackgerritMerged openstack/releases master: Release neutron-vpnaas for queens  https://review.openstack.org/53829522:07
rosmaitathe glance py27 tests were only restored yesterday22:08
rosmaitafor requirements, i mean22:08
*** edmondsw has quit IRC22:11
dhellmannok, I get that. but tagging the milestone is meant to test that packaging works and mark the feature freeze. we don't need those tests to do those things.22:12
*** edmondsw has joined #openstack-release22:12
dhellmannso we could tag where we are now, you could block all other patches to glance before landing those, and they can be part of the rc122:12
dhellmannwe have other things we need to do as part of the release process that are blocked now, so we're looking for ways to unblock them22:13
openstackgerritMerged openstack/releases master: release python-heatclient 1.13.1  https://review.openstack.org/53921722:14
dhellmannit's time to call it a day here22:14
rosmaitayeah, i understand22:15
*** edmondsw has quit IRC22:17
rosmaitadhellmann smcginnis how does this sound? let's tag Q-3 at http://git.openstack.org/cgit/openstack/glance/commit/?id=631add1ba849eac67ecb36ea4c90b57a300a96fa , which was 13 days ago, but before the ft broke22:18
rosmaitadoes that make sense?22:19
rosmaitai'll just do FFEs for everything after that22:20
openstackgerritMerged openstack/releases master: Release heat queens-3 10.0.0.0b3  https://review.openstack.org/53921222:22
openstackgerritMerged openstack/releases master: Release heat-agents 1.5.2  https://review.openstack.org/53921922:22
openstackgerritBrian Rosmaita proposed openstack/releases master: Release Glance Q-3  https://review.openstack.org/53828822:26
openstackgerritBrian Rosmaita proposed openstack/releases master: Release Glance Q-3  https://review.openstack.org/53828822:26
*** yamamoto has joined #openstack-release22:27
smcginnisrosmaita: Stepped away. Yes, that sounds like a good plan.22:30
rosmaitaok, i updated the release patch, so it should be ready to go22:30
smcginnisrosmaita: Thank you22:32
rosmaitanp, happy releasing!22:33
smcginnisThe list is finally getting small, so that is making me happy. ;)22:41
*** armstron_ has quit IRC22:43
openstackgerritMerged openstack/releases master: Release Glance Q-3  https://review.openstack.org/53828822:55
openstackgerritMerged openstack/releases master: Release tripleo-ipsec 8.0.1  https://review.openstack.org/53934022:56
*** armax has quit IRC22:56
*** mriedem is now known as mriedem_afk23:02
*** dave-mccowan has quit IRC23:03
openstackgerritMerged openstack/releases master: Release searchlight-ui 4.0.0.0b3  https://review.openstack.org/53832123:05
*** ianychoi has quit IRC23:05
*** kumarmn has quit IRC23:09
*** kumarmn has joined #openstack-release23:10
*** yamamoto_ has joined #openstack-release23:12
*** kumarmn has quit IRC23:14
*** yamamoto has quit IRC23:15
*** kumarmn has joined #openstack-release23:27
*** kumarmn has quit IRC23:32
*** kumarmn has joined #openstack-release23:38
*** yamahata has quit IRC23:44
*** iyamahat has quit IRC23:47

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!