Thursday, 2016-02-04

openstackgerritDoug Hellmann proposed openstack-infra/release-tools: remove the announce flag from the release wrapper scripts  https://review.openstack.org/27598600:03
*** bdemers has quit IRC00:19
*** bdemers_ has joined #openstack-release00:19
openstackgerritDoug Hellmann proposed openstack/reno: add earliest_version option to scanner  https://review.openstack.org/27599100:23
*** nikhil has quit IRC01:21
*** nikhil has joined #openstack-release01:35
*** nikhil_k has joined #openstack-release02:24
*** nikhil has quit IRC02:27
*** dims has joined #openstack-release03:38
*** amotoki has joined #openstack-release03:52
*** dims has quit IRC03:53
*** bdemers_ has quit IRC04:04
*** bdemers has joined #openstack-release04:05
*** SlickNik has quit IRC04:11
*** SlickNik has joined #openstack-release04:12
*** sigmavirus24_awa has quit IRC04:13
*** jroll has quit IRC04:13
*** bdemers_ has joined #openstack-release04:13
*** bdemers has quit IRC04:13
*** sdague has quit IRC04:13
*** sigmavirus24_awa has joined #openstack-release04:14
*** sdague has joined #openstack-release04:14
*** jroll has joined #openstack-release04:14
*** SergeyLukjanov has quit IRC04:19
*** mriedem_afk has quit IRC04:19
*** tristanC has quit IRC04:19
*** smcginnis has quit IRC04:19
*** Daviey has quit IRC04:19
*** tristanC has joined #openstack-release04:19
*** Daviey has joined #openstack-release04:19
*** SergeyLukjanov has joined #openstack-release04:20
*** smcginnis has joined #openstack-release04:20
openstackgerritBen Swartzlander proposed openstack/releases: Release manila-ui 1.3.0  https://review.openstack.org/27531004:41
*** bnemec has joined #openstack-release08:02
*** openstackgerrit has quit IRC09:17
*** openstackgerrit has joined #openstack-release09:17
*** mugsie has quit IRC10:39
*** mugsie has joined #openstack-release10:40
*** bnemec has quit IRC10:40
*** bnemec has joined #openstack-release10:42
*** bnemec has quit IRC11:01
*** dims has joined #openstack-release11:12
dimsttx : can we let this in please? i can clean out a whole lot of config files11:27
dimshttps://review.openstack.org/#/c/274939/11:27
dimsttx ^^11:27
ttxon it11:28
ttxdims: shouldn't that one also change the upper-contraints to match ?11:36
ttxoh, already at 0.17.311:36
dimsttx : the bot merged a chance to u-c already11:36
dimsyep11:36
*** bnemec has joined #openstack-release11:36
ttxalright then11:36
dimsthanks ttx11:37
*** dtantsur|afk is now known as dtantsur12:58
dhellmannttx, dims: you saw that the new job to send announcement emails merged yesterday, right? https://review.openstack.org/#/c/272767/13:19
*** gordc has joined #openstack-release13:22
dimsdhellmann : y, one question i had was if the pypi upload fails, does it still send the announce?13:23
ttxdhellmann: yes, review is next on my todo13:23
dhellmanndims : great question, I'm not sure. probably not, since the announce job depends on the upload job13:26
dimsdhellmann : cross my fingers :)13:33
ttxdhellmann: lgtm (good thing since it merged already :)13:37
ttxdhellmann: so the main issue with this approach is that it ties the announce to a specific tarball (vs. a deliverable)13:38
*** david-lyle has quit IRC13:38
ttxso at release time we end up sending several mails for a single 'release'13:39
ttxI think we can survive that for now13:39
ttxbut it kind of reduces the value of the "deliverable" concept, which was supposed to help with decomposition across multiple git repos13:40
ttxbut it's a tough one to crack, especially with the infrastructure happily ignoring that subtlety for now13:42
ttxthe integration was in fact manually done by the human at the announce step13:43
ttxone way to work around it with the current structure would be to have the first -announce-release jobs in a deliveravle be NOOPs and the last one for a deliverable picking up the tab. Hard to avoid races though13:45
openstackgerritgordon chung proposed openstack/releases: ceilometerclient 1.5.2 - liberty  https://review.openstack.org/27536713:47
ttxthat would replicate how humans used to do it13:48
*** sigmavirus24_awa is now known as sigmavirus2413:51
*** sigmavirus24 is now known as sigmavirus24_awa13:52
dhellmannttx: crazy idea: have the announce job on a repo tag the releases repo, and have a job on the releases repo respond to the tag14:04
openstackgerritDean Troyer proposed openstack/releases: Release OpenStackClient 2.1.0  https://review.openstack.org/27625114:04
dhellmannttx: but yeah, I think we can live with it for the time being14:04
*** dims_ has joined #openstack-release14:10
*** dims has quit IRC14:12
ttxdhellmann: heh, that would be a pretty nasty hack :) Although that could let us feed information back to releases repo... think a SHA1 for the generated tarballs14:24
ttxbut then you're back on privileges nodes for the tag signing14:25
*** wshao has joined #openstack-release14:29
dhellmannttx: what if we submit a patch with the sha, instead of tagging?14:33
dhellmannit makes the release a 2 step process (approve the release, wait for and approve the sha patch to trigger the announcement)14:33
ttxThat could be acceptable. We need to sit on top of the reviews on that repo anyway14:34
dhellmannttx: if a deliverable has several repos, we would have to wait for all repos to have an associated sha before announcing, but that's an easy check14:34
dhellmannI'll make a note of that as an improvement for next cycle. I'm more worried about release notes translations, I think, this cycle.14:35
ttxdhellmann: the job would have to propose changes on top of one another14:35
dhellmannttx: that would be idea, yeah, but adding individual lines shouldn't introduce merge conflicts, would it?14:35
ttxI wonder if they would be separated enough. I guess they would14:36
dhellmannwe could design it to ensure that. maybe that info goes into separate files, for example14:37
ttxas a workaround for this cycle we could make the announce job noop on multi-repo deliverables for this cycle, up to the human to collect them in a single one14:37
dhellmannwe still need the tool to support doing that, and that's going to be some work14:37
ttxor maybe have the tag contain the deliverable:yes info14:38
dhellmannhmm, that might work14:38
ttxI'll think about it, worst case scenario we'll just multiannounce for this cycle14:38
dhellmannok14:38
ttxI just would like to avoid training people to ignore the deliverables concept, even if just for one cycle14:39
dhellmannif we need a flag, we could allow announce-to to be empty, and not announce in that case14:39
ttxoh. We could indeed just remove announce-to from multi-repo deliverables14:39
ttxthen it's just a question of collating manually14:40
ttxanyway, don't worry about it, I'll look into it14:40
ttxfamous last words14:40
dhellmannOTOH, I wonder how many multi-repo deliverables we really want. Doesn't it confuse things like where to go look for the release notes in the first place?14:40
dhellmannI need to think about it more, too14:40
ttxhmm14:40
ttxIdeally those would have a single release note14:41
ttxbut that may be hard to retrofit14:41
dhellmannbut that means if I have a patch in repo B, I have to put a reno file in repo A14:41
ttxright, that's what I mean by hard to retrofit14:41
dhellmannyeah14:41
ttxmaybe just adding a "this component is part of the neutron 7.0.2 deliverable" on top of the announce is a good tradeoff14:42
openstackgerritDoug Hellmann proposed openstack/releases: skeleton version of newton schedule  https://review.openstack.org/27591114:56
dhellmannttx: that's certainly easy enough to do14:56
dhellmannttx, dims_ : I'm going to release keystoneclient and try out the new announce job15:03
*** doug-fish has joined #openstack-release15:03
dhellmannttx, dims_: do you have an issue with me self-approving the tool changes under that patch?15:04
dims_dhellmann : please go for it15:04
* dims_ prototyping a jenkins job for running nova, glance etc with oslo master15:05
dhellmanndims_ : ack, thanks15:05
*** sigmavirus24_awa is now known as sigmavirus2415:06
*** nikhil_k is now known as nikhil15:08
*** wshao has quit IRC15:11
openstackgerritMerged openstack/releases: show more change detail, including ancestry  https://review.openstack.org/27582915:17
openstackgerritMerged openstack/releases: do not merge stderr & stdout when checking ancestors  https://review.openstack.org/27585315:20
openstackgerritMerged openstack/releases: release keystoneclient 2.1.2  https://review.openstack.org/27568515:23
dhellmannugh, failed15:28
dims_dhellmann : where exactly?15:46
dhellmanndims_ : bug in the job definition, we have the fix merging now15:47
* dhellmann my kingdom for a backslash15:47
dims_ack thanks dhellmann :)15:47
dims_dhellmann : there's just no way to try job definitions :(15:48
dhellmanndims_ : yeah :-/15:48
*** david-lyle has joined #openstack-release16:24
dhellmannstevemar : keystoneclient 2.1.2 is released, but the announce job failed. we have a fix in the queue and will re-run that asap, but I thought I'd let you know16:26
dhellmannstevemar, bknudson_: you should go ahead and make the constraints update16:26
stevemardhellmann: ah okay - i saw it was merged, haven't gotten to the ML folder in my email yet :)16:27
smcginnisHey all, any plans to add gpg signatures on https://tarballs.openstack.org/16:38
smcginnisI'm getting pinged on the lack of that.16:39
dhellmannsmcginnis : yes, let me find that spec16:40
smcginnisdhellmann: Cool, thanks!16:40
dhellmannsmcginnis : http://specs.openstack.org/openstack-infra/infra-specs/specs/artifact-signing.html16:40
dhellmannit's an infra spec, but we're interested/involved, too16:40
smcginnisdhellmann: Perfect, thanks. I can at least point them to this.16:41
dhellmannsmcginnis : cool16:41
smcginnisSorry, one more question.16:42
smcginnisAm I supposed to hit the Create Release link here: https://launchpad.net/cinder/+milestone/2015.1.316:42
smcginnisOr does it not matter anymore?16:42
dhellmannsmcginnis : you don't need to do that any more. We're going to be referring folks to the docs published out of the releases repo (docs.openstack.org/releases now and releases.openstack.org any day now)16:43
smcginnisdhellmann: OK, thanks. Just want to make sure I'm not missing anything.16:43
dhellmannsmcginnis : yep, thanks for double-checking16:44
dims_dhellmann : stevemar : we may have broken something... http://logs.openstack.org/57/273957/1/gate/gate-tempest-dsvm-neutron-full/f2eaa44/logs/devstacklog.txt.gz16:53
dims_stevemar : bknudson_ : 3 more logs with the same problem16:54
stevemardims_: oh nice16:54
bknudson_something with the requirements?16:54
dims_http://paste.openstack.org/show/485987/16:54
dims_links to 4 jobs that failed a short while ago16:55
stevemari feel like just releasing from master at this point16:55
dims_stevemar : need to make sure those were because of the release pushed out or something else first16:56
openstackgerritGreg Hill proposed openstack/releases: Add taskflow 1.27.0  https://review.openstack.org/27635216:56
stevemardims_: if it quacks like a duck and looks like a duck16:57
stevemarlet me check pip freeze16:57
stevemarwhat the... python-keystoneclient==1.3.416:57
bknudson_old school16:57
*** jgriffith is now known as jgriffith_away16:58
stevemarlooks like those are all kilo related16:58
dims_stevemar : whew ok16:59
stevemar*wipes sweat from brow*17:00
dims_we still have to tell mriedem :)17:00
dims_we got off easy :)17:00
stevemardims_: maybe testtools needs a pin on kilo?17:01
stevemarlooks like they are pulling in fixtures 1.3.0?17:01
stevemardims_: yeah testtools released today https://pypi.python.org/pypi/testtools17:02
*** bnemec has quit IRC17:02
dims_ah cool17:02
stevemardims_: and kilo says fixtures>=0.3.14,<1.3.017:03
dims_stevemar : let's take it to the stable channel17:06
*** dtantsur is now known as dtantsur|afk17:18
bswartzdhellmann: why did you ask fro the manilaclient verison to be bumped to 1.7.0 instead of 1.6.1? Isn't a bugfix release just a +0.0.1 version bump?17:52
dhellmannbswartz : the minimum required versions of related libraries have changed18:07
bswartzah, I see18:09
*** amotoki has quit IRC18:43
*** dansmith has quit IRC19:05
*** dansmith has joined #openstack-release19:05
*** david-lyle has quit IRC19:11
*** georgewang has joined #openstack-release19:50
georgewanghi, networking-sfc submit README.rst after mitaka release, it causes the content in http://docs.openstack.org/developer/networking-sfc/ out of sync with https://pypi.python.org/pypi/networking-sfc/1.0.019:52
georgewangcan release team help update the pip document also? Thanks19:52
*** nikhil_k has joined #openstack-release19:52
*** nikhil has quit IRC19:55
*** stevemar has quit IRC19:56
*** jgriffith_away is now known as jgriffith20:19
sdaguedims_: / dhellmann it looks like the oslo.messaging release has gone poorly, all the python 3.4 constraints jobs are failing20:20
*** r1chardj0n3s has left #openstack-release20:20
dhellmanngeorgewang : your pypi readme will be updated on your next release, or someone with access to pypi can edit it directly (I don't have that permission myself)20:22
dhellmannsdague : what repo?20:23
sdague?20:23
dhellmannsdague : 3.4 constraints jobs?20:23
sdaguehttps://jenkins06.openstack.org/job/gate-neutron-python34-constraints/1689/20:23
sdague No matching distribution found for oslo.messaging===4.1.0 (from -c /home/jenkins/workspace/gate-neutron-python34-constraints/upper-constraints.txt (line 213))20:23
sdaguefungi said the wheel looked wrong on pypi20:24
dhellmannlooks like something didn't get uploaded20:24
fungiyeah, i'm juggling 10 conversations in different places but was on my way over here to say something20:24
sdaguethis seems like an excellent new thing to get tested on updating upper-constraints20:24
dhellmannsdague : the constraints job should already be trying to install things, and it runs the dsvm jobs, doesn't it?20:25
dhellmannfungi : is this something we can recover from by rerunning the existing release, or should I tag another one?20:25
dhellmannsdague : ah, requirements updates don't try with python 320:27
dhellmannI wonder if we're at a point yet where all of our requirements can be installed under python 320:27
sdaguethat I don't know20:29
sdaguehowever, it probably wouldn't be too bad to make sure they exist20:29
fungithe constraints job is supposed to try to do python 2 and python 320:31
fungidhellmann: i should be able to complete the upload20:31
dhellmannfungi : ok, thanks, I know you have a lot going on this week20:31
fungii'm trying to find a second to dig up the logs, but it's almost certainly another case of pypi returning an error after a successful upload of the wheel20:31
*** jgriffith is now known as jgriffith_away20:31
sdaguefungi: cool, that would be good. right now there are enough neutron jobs in the gate, it's just going to chew on the entire node allocation until this gets sorted20:32
dhellmannfungi : http://logs.openstack.org/68/68799083b19f07f321ec1666071ca58bdd42d1f4/release/oslo.messaging-pypi-both-upload/bc12f21/console.html20:32
dhellmannbad gateway20:32
fungiinfra's still pretty short-staffed this week (though not as bad as last week)20:32
dhellmannand yes, the py2-non-any wheel is there20:33
fungiyeah, dstufft says this is something weird with their fastly proxying for http post method20:33
fungiit's been going on for like a year though20:34
dhellmannI could try to patch twine to retry, or confirm the error by looking for the file20:34
dhellmannor I could patch our job to do that20:34
fungithere are some subtle issues around possible approaches to making it more robust, which we were discussing in the infra channel earlier this week20:35
dhellmannok20:35
*** stevemar has joined #openstack-release20:36
fungipypi upload of the sdist is complete now20:36
dhellmannfungi : thanks!20:37
fungiyw, sorry about the delay20:37
dhellmannfungi : no worries!20:37
*** bdemers has joined #openstack-release20:38
* dims_ back from picking up kid from school20:38
dims_fungi : dhellmann : this should have triggered doubts in my head - https://review.openstack.org/#/c/276058/20:40
dims_fungi : thanks!20:40
dhellmanndims_ : ah!20:40
dhellmannsdague, fungi, dims_ : I don't see a job with an obvious name indicating that it is trying to install everything while checking the requirements. Did we stop doing that? There used to be a pbr script...20:41
sdagueI thought that was - gate-requirements-integration-dsvm20:41
dims_gate-requirements-integration-dsvm-resolver?20:41
*** SlickN1k has joined #openstack-release20:42
fungithe -resolver version is lifeless's experimental sat solver branch of pip i think20:42
sdagueyeh that's different20:42
sdaguehowever, that job generates the constraints, but does not attempt to use them I don't think20:42
sdaguealso, only py27 I'm pretty sure20:43
fungii thought it also had a py34 stage20:43
dims_gotcha sdague20:43
fungias part of the same job20:43
sdaguefungi: maybe, though if it isn't using the constraints file, it would not have hit this20:43
fungifair point20:46
fungii meant the constraints generator job is supposed to do a py34 pass20:46
openstackgerritBen Swartzlander proposed openstack/releases: Release python-manilaclient 1.7.0  https://review.openstack.org/27083420:47
*** bdemers_ has quit IRC20:47
*** SlickNik has quit IRC20:47
*** bswartz has quit IRC20:47
*** ttx has quit IRC20:47
*** v12aml has quit IRC20:47
*** SlickN1k is now known as SlickNik20:47
dims_fungi dhellmann : looks like we need additional steps in the pypi uploader job to see if it can pull down everything it just uploaded?20:47
dhellmanndims_ : we were just talking about that. I'd want to add it to twine, if we could. Do you have someone who can help dstufft with that?20:48
dims_dhellmann : i can check20:49
*** ttx has joined #openstack-release20:49
*** v12aml has joined #openstack-release20:49
*** bswartz has joined #openstack-release20:50
fungiyeah, twine confirming what it uploaded is indexed, rather than just relying on the http response, might help20:51
*** stevemar has quit IRC20:52
fungiproblem is this seems to be an infrastructure/implementation issue with pypi.python.org causing a false negative result. it's sort of an odd wishlist item to ask to have the tool double-check that errors are really failures, though that's the situation we find ourselves in20:52
*** gordc has quit IRC20:52
dhellmannfungi : yeah, dstufft thinks the fix in twine might be pretty simple, so if we can find someone with time to make the patch I'll set up introductions and see if we can get it dealt with, hacky though that might be20:53
fungiawesome20:53
dhellmanndims: ^^20:53
dims_fungi : would doing a pip install with the version that we just uploaded work?20:53
fungithat someone != me btw ;)20:54
dhellmannfungi : ditto20:54
fungidims_: i don't know what sort of latency we should be expecting there, but also that doesn't necessarily cover it in situations like this one20:54
dhellmanndims_ : dstufft said we probably just need to "pass a Retry instance" to the post code20:54
dims_fungi : ack20:54
dhellmannlooking at the requirements-integration job, I see it installing the projects listed in the job definition, but not using the constraints file20:55
fungiwell, also the retry needs to idempotently noop in cases where the file was successfully uploaded and pypi refuses the retry because it won't let the file be overwritten20:55
dims_dhellmann : can you create an issue in twine? we can add notes there and point some folks to it?20:55
dims_s/can you/can you please/20:55
dims_i may be getting an intern shortly in ukraine, so this may be a quick one to test their skills20:56
fungithere's supposedly already some implementation in twine to skip files which are already on pypi when uploading, so presumably that could be used to determine whether there is a need to retry on some failures20:56
dims_fungi : ack i'll poke at the code in a little bit too20:57
dhellmanndims_ : https://github.com/pypa/twine/issues/16720:57
fungiworth noting, also, is that we're consuming twine from ubuntu's packages in 14.04 lts. we switched away from installing it from pypi because things like requests getting updated broke our release process not long ago too20:58
fungiand we'd rather not run non-distro packages on privileged job workers20:58
fungiso the cycle to consume this feature would likely be longish20:58
dims_dhellmann : ah thanks!20:59
fungilike if it somehow gets into 16.04 lts then maybe we can have it soon, but otherwise we may need to consider other options20:59
dims_fungi : ouch ok21:00
dhellmannfungi : hmm, that makes fixing it upstream a bit less useful, at least at first. We might need to modify the job, too.21:12
dims_dhellmann : right21:17
*** gordc has joined #openstack-release21:49
openstackgerritDoug Hellmann proposed openstack/releases: add a flag for whether to report the PyPI link for releases  https://review.openstack.org/27646722:05
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: remove notable changes option  https://review.openstack.org/27646822:05
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: process pypi flag for release announcements  https://review.openstack.org/27646922:05
openstackgerritgordon chung proposed openstack/releases: ceilometerclient 1.5.2 - liberty  https://review.openstack.org/27536722:47
*** david-lyle has joined #openstack-release22:48
*** david-lyle has quit IRC22:52
*** gordc has quit IRC23:28
*** dims_ has quit IRC23:31
*** dims has joined #openstack-release23:32
*** sigmavirus24 is now known as sigmavirus24_awa23:35
georgewangTo update the networking-sfc pypi documentation, I see dhellmann suggested to wait for next release or ask someone to edit pypi directly23:41
georgewangMy question is when is the next release or do you know who can access to modify the pypi?23:42
georgewangthanks23:42
lifelessfungi: sdague: constraints generation does 2.7 and 3.423:53
dhellmanngeorgewang : you'd have to ask the contributor team about either of those things23:53
lifelessand merges them23:53
fungilifeless: it seems to have squeaked by in a corner case where a py2-only wheel gets uploaded and there's no sdist23:53
dhellmannlifeless : we discovered that since the python3 compatible sdist wasn't uploaded, the constraints had different versions for py2 and py3. dims fixed that by hand, and that change passed the test, so we have a gap in coverage somewhere23:54
fungilikely not a situation to optimize for, but might be interesting to figure out why23:54
lifelesssorry, I don't have full context23:54
fungioh, that 'splains it23:54
fungiso the constraints change that merged was a "modified" one23:54
dhellmannlifeless : the oslo.messaging release broke part way through because of pypi errors, so only 1 of 2 files was uploaded23:54
lifelessconstraints *should* have different versions for py2 and py3 in that sort of case23:54
lifelessworking-as-designed23:55
dhellmannthat happened to be a py2 wheel23:55
dhellmannright, but it got "fixed" and the fix passed our tests, which means we can constrain to a thing we can't install under python 323:55
lifelessdhellmann: yah, it would be wonderful if the pypi upload process could be transational23:55
fungiso presumably the generated constraints file said use new oslo.messaging on py2 and older oslo.messaging on py3, and dims "fixed" it?23:55
dhellmannfungi : right23:55
funginow i understand why23:55
dhellmann[Thu 03:40:11 PM]  <dims_>fungi : dhellmann : this should have triggered doubts in my head - https://review.openstack.org/#/c/276058/23:56
lifelessits the issing unit tests thing23:56
fungidhellmann: thanks, i looked at the change but it didn't click that it was a modified change23:56
* dims peeks23:56
fungiand then i got dragged into other fires23:56
lifelessif we can start running unit tests of nova on requirements checks, we'd catch things like that23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!