| *** armax has joined #openstack-release | 00:10 | |
| *** lbragstad has quit IRC | 00:45 | |
| *** whoami-rajat has joined #openstack-release | 01:12 | |
| *** mriedem has quit IRC | 01:18 | |
| *** openstackgerrit has quit IRC | 01:30 | |
| *** ekcs has quit IRC | 02:13 | |
| *** hongbin has joined #openstack-release | 02:29 | |
| *** armax has quit IRC | 03:17 | |
| *** udesale has joined #openstack-release | 03:26 | |
| *** ricolin has joined #openstack-release | 03:31 | |
| *** hongbin has quit IRC | 03:42 | |
| *** dave-mccowan has quit IRC | 04:15 | |
| *** diablo_rojo has quit IRC | 04:29 | |
| *** openstackstatus has quit IRC | 04:35 | |
| *** openstackstatus has joined #openstack-release | 04:36 | |
| *** ChanServ sets mode: +v openstackstatus | 04:36 | |
| *** ykarel|away has joined #openstack-release | 04:39 | |
| *** ykarel_ has joined #openstack-release | 04:44 | |
| *** ykarel|away has quit IRC | 04:47 | |
| *** ykarel_ is now known as ykarel | 04:56 | |
| *** pcaruana has joined #openstack-release | 05:06 | |
| *** ykarel_ has joined #openstack-release | 05:09 | |
| *** ykarel has quit IRC | 05:11 | |
| *** ricolin has quit IRC | 05:30 | |
| *** ykarel_ is now known as ykarel | 05:50 | |
| tonyb | tosky: That shoudl be fine as long as it's already in global-requirements and (not that it shoudl) sahara-tests doesn't have the stable tag | 06:23 |
|---|---|---|
| *** e0ne has joined #openstack-release | 06:28 | |
| *** ifat_afek has joined #openstack-release | 06:39 | |
| *** e0ne has quit IRC | 06:48 | |
| *** e0ne has joined #openstack-release | 06:57 | |
| *** tesseract has joined #openstack-release | 07:09 | |
| *** tesseract has quit IRC | 07:10 | |
| *** tesseract has joined #openstack-release | 07:10 | |
| *** tosky has joined #openstack-release | 07:28 | |
| *** e0ne has quit IRC | 07:33 | |
| *** pcaruana has quit IRC | 07:34 | |
| *** pcaruana has joined #openstack-release | 07:35 | |
| *** amoralej|off is now known as amoralej | 07:35 | |
| *** e0ne has joined #openstack-release | 07:40 | |
| *** jpich has joined #openstack-release | 07:48 | |
| *** e0ne has quit IRC | 07:48 | |
| *** e0ne has joined #openstack-release | 07:53 | |
| *** hberaud|gone is now known as hberaud | 07:54 | |
| *** ykarel is now known as ykarel|lunch | 07:59 | |
| *** priteau has joined #openstack-release | 08:05 | |
| *** evrardjp has quit IRC | 08:18 | |
| *** evrardjp has joined #openstack-release | 08:19 | |
| evrardjp | o/ | 08:20 |
| ttx | o/ | 08:22 |
| priteau | Hello. I noticed some issues in the "Stein Screenshots" section of https://www.openstack.org/software/stein: image/caption are in the wrong place, and there is an Ironic text instead of the Neutron text. | 08:26 |
| *** ricolin has joined #openstack-release | 08:30 | |
| *** e0ne has quit IRC | 08:47 | |
| evrardjp | good catch priteau | 08:50 |
| ttx | checking | 08:52 |
| ttx | ok we'll get it fixed before the go-live | 08:53 |
| priteau | Thanks Thierry! | 08:56 |
| ttx | the web masters need to get up first :) | 08:56 |
| *** dtantsur|afk is now known as dtantsur | 08:58 | |
| *** e0ne has joined #openstack-release | 08:58 | |
| *** openstackgerrit has joined #openstack-release | 09:01 | |
| openstackgerrit | dharmendra kushwaha proposed openstack/releases master: heat-translator 1.3.1 https://review.openstack.org/651467 | 09:01 |
| *** ykarel|lunch is now known as ykarel | 09:04 | |
| *** e0ne has quit IRC | 09:06 | |
| openstackgerrit | Carlos Goncalves proposed openstack/releases master: Release Octavia Queens 2.0.5 and Rocky 3.0.3 https://review.openstack.org/651475 | 09:26 |
| *** electrofelix has joined #openstack-release | 09:28 | |
| *** ifat_afek has quit IRC | 09:56 | |
| *** ifat_afek has joined #openstack-release | 09:57 | |
| *** cdent has joined #openstack-release | 10:33 | |
| fungi | priteau: good catch. i got them to stop calling those "dashboard screenshots" but didn't think to vet the individual captions | 10:42 |
| fungi | ttx: there's a surprising amount of gate activity for this time of day, but everything seems to be flowing smoothly so far | 10:44 |
| fungi | i'm still catching up on e-mail but suspect we're all clear for release activities | 10:45 |
| ttx | ok, plan is still to run at 1200z | 10:47 |
| * ttx lunches | 10:47 | |
| openstackgerrit | Dirk Mueller proposed openstack/releases master: Release a bugfix update for renderspec https://review.openstack.org/651494 | 10:51 |
| *** hberaud is now known as hberaud|lunch | 10:54 | |
| *** e0ne has joined #openstack-release | 11:12 | |
| *** e0ne has quit IRC | 11:15 | |
| *** e0ne has joined #openstack-release | 11:19 | |
| *** e0ne has quit IRC | 11:20 | |
| fungi | all caught up on e-mail and irc scrollback, no emergency situations apparent, still all clear for mashing the big red button at 12:00z i think | 11:22 |
| *** udesale has quit IRC | 11:24 | |
| *** e0ne has joined #openstack-release | 11:24 | |
| *** udesale has joined #openstack-release | 11:24 | |
| smcginnis | Calm before the storm? :) | 11:27 |
| *** cdent_ has joined #openstack-release | 11:29 | |
| openstackgerrit | Sean McGinnis proposed openstack/releases master: Mark Stein as released https://review.openstack.org/650365 | 11:30 |
| *** cdent has quit IRC | 11:31 | |
| *** cdent_ is now known as cdent | 11:31 | |
| smcginnis | ttx: I should send the release announcement 30 minutes to an hour before the PR, right? So 14:00-14:30z? | 11:37 |
| smcginnis | Or is it fine to send as soon as all our patches go through? | 11:37 |
| ttx | Ideally around 14:30 | 11:39 |
| ttx | but can be done earlier if you need | 11:40 |
| ttx | If we are all here I propose we do the +2a now | 11:40 |
| ttx | smcginnis: I'll let you do the honors | 11:40 |
| fungi | i'm standing by | 11:43 |
| fungi | though should pour myself some coffee | 11:43 |
| ttx | I'll push it at 1200z if smcginnis does not beat me to it | 11:44 |
| smcginnis | Sounds good, I'll hit the big ol' button. | 11:44 |
| ttx | yay | 11:44 |
| ttx | I like to start early since we get more gate activity later | 11:45 |
| smcginnis | ++ | 11:45 |
| ttx | Also I have a meeting tunnel starting at 1300z, so the more we do before then, the better | 11:45 |
| smcginnis | Now to watch zuul... | 11:46 |
| ttx | Go zuul go | 11:46 |
| ttx | Step 1 - gate for the openstack/releases change | 11:47 |
| fungi | https://www.youtube.com/watch?v=j3Uy9wsfkok ~2:20 | 11:48 |
| smcginnis | We didn't have a review soundtrack this cycle. That's too bad. | 11:50 |
| *** hberaud|lunch is now known as hberaud | 11:52 | |
| ttx | I'm trying to find that diagram that shows the flow of release jobs, for reference | 11:52 |
| ttx | but can't remember where it landed | 11:52 |
| fungi | mick smiley: magic (ghostbusters soundtrack) | 11:52 |
| smcginnis | I'm not sure I recall that diagram. | 11:52 |
| dhellmann | o/ | 11:53 |
| smcginnis | We can queue up this classic next: https://www.youtube.com/watch?v=b0tsZB_LEQk | 11:54 |
| ttx | dhellmann: do you remember where we put the job flow diagram? | 11:54 |
| smcginnis | Morning dhellmann | 11:54 |
| fungi | though in actuality i was listening to (mystic knights of the) oingo boingo: dead man's party | 11:54 |
| dhellmann | hmm | 11:55 |
| dhellmann | let me see if I can find that diagram | 11:55 |
| *** bobh has joined #openstack-release | 11:55 | |
| dhellmann | ttx: is this it? https://docs.google.com/drawings/d/1B9u2VpLeuzj-1T1VJrpgaE7_uM9T7cRlEXLFf56ggKE/edit | 11:56 |
| ttx | sure but we did an ascii version of it for doc | 11:57 |
| dhellmann | oh, that I have no memory of | 11:57 |
| ttx | we used it in training not so long ago | 11:57 |
| smcginnis | Hmm, the drawing does look familiar, but I don't think we ever added it to our docs. | 11:58 |
| smcginnis | Docs job done. Validate takes awhile. | 11:58 |
| *** udesale has quit IRC | 11:59 | |
| smcginnis | Looks like it's in the "s"'s though. | 11:59 |
| ttx | I wonder if we did not lose tat diagram when we removed release-tools | 12:00 |
| smcginnis | Hmm, that could be. | 12:00 |
| smcginnis | We've found a couple little things that got lost there. | 12:00 |
| * fungi just notices he marked this on his calendar as "release the stein" | 12:01 | |
| smcginnis | :) | 12:01 |
| ttx | fungi: is there a way to access the old release-tools? | 12:01 |
| smcginnis | Fill the stein. Empty the stein. | 12:01 |
| fungi | sure, i just have to find them ;) | 12:02 |
| fungi | gimme a sec | 12:02 |
| dhellmann | I don't see any diagrams in the presentations in the release-tools repo | 12:02 |
| openstackgerrit | Merged openstack/releases master: Stein final releases for cycle-with-rc projects https://review.openstack.org/650362 | 12:02 |
| smcginnis | \o/ | 12:02 |
| dhellmann | smcginnis had a newer version of the onboarding presentation somewhere else, could it be in that copy? | 12:02 |
| ttx | Step 2: Release-Post job doing all the tagging | 12:03 |
| fungi | https://opendev.org/openstack-infra/release-tools/src/commit/0abf645cc36aadce365ba08328e98864272d64ed | 12:03 |
| fungi | that's the last commit before the retirement commit which deleted all the content | 12:03 |
| fungi | i should say, browsing the file tree as it existed at the time | 12:04 |
| smcginnis | Doesn't look like I had that: https://gitpitch.com/stmcginnis/presentations/master?p=Berlin2018/ReleaseTeam#/ | 12:04 |
| ttx | bah, maybe I dreamt it | 12:05 |
| smcginnis | Electric sheep and release diagrams. :) | 12:06 |
| ttx | No, I'm pretty sure that was thoroughly documented | 12:06 |
| ttx | With steps and all | 12:06 |
| ttx | in ASCII form | 12:06 |
| smcginnis | Maybe local branch that you forgot to push up? | 12:07 |
| dhellmann | I know we had a task on the tracker to do that for a while. I don't think I would have created an ascii diagram, though. | 12:07 |
| dhellmann | unicode, maybe ;-) | 12:07 |
| ttx | i remember creating it for sure. | 12:07 |
| *** e0ne has quit IRC | 12:09 | |
| ttx | Ah! | 12:09 |
| smcginnis | Found it? | 12:09 |
| ttx | yes :) | 12:09 |
| *** amoralej is now known as amoralej|lunch | 12:10 | |
| ttx | https://opendev.org/openstack/releases/src/branch/master/CONTRIBUTING.rst | 12:10 |
| ttx | wow that's ugly | 12:10 |
| ttx | http://git.openstack.org/cgit/openstack/releases/tree/CONTRIBUTING.rst | 12:10 |
| dhellmann | aha | 12:10 |
| dhellmann | maybe that should move into the reference docs | 12:11 |
| evrardjp | I still think K should have been named kraken | 12:11 |
| fungi | https://opendev.org/openstack/releases/raw/branch/master/CONTRIBUTING.rst | 12:11 |
| ttx | I'm not crazy. The government is onto us | 12:11 |
| smcginnis | Here come the tags. | 12:11 |
| * fungi braces for impact | 12:11 | |
| ttx | Step3 - as each tag is created, triggers the release pipeline for each tagged repo | 12:12 |
| fungi | anyway, see link above, gitea does have a raw mode where the rst parser doesn't kick in | 12:12 |
| *** priteau has quit IRC | 12:12 | |
| *** e0ne has joined #openstack-release | 12:12 | |
| smcginnis | We have some pydot diagrams that mostly work well in the Cinder docs. | 12:12 |
| evrardjp | love watching blue bars | 12:12 |
| fungi | though i think very minor format edits would cause those diagrams to render in a monospaced typeface | 12:12 |
| ttx | dhellmann: we probably should. | 12:13 |
| dhellmann | yeah, they're just not indented in code blocks | 12:13 |
| evrardjp | we can probably use *diag | 12:13 |
| dhellmann | evrardjp : then you have come to the right place! | 12:13 |
| evrardjp | that should look clean in the docs. | 12:13 |
| *** e0ne has quit IRC | 12:13 | |
| evrardjp | dhellmann: so it seems | 12:14 |
| evrardjp | super sad for colorblind ppl I guess | 12:14 |
| fungi | well, upside to ascii art in the readme is that the readme more than the rest of the docs is ideally also entirely usable as plain text | 12:14 |
| *** priteau has joined #openstack-release | 12:14 | |
| ttx | Step 3 subtlety: release-announce job skips announcing every single RC->Release promotion, in order to keep the PTL email annoucnement thunder intact | 12:14 |
| evrardjp | oh | 12:15 |
| evrardjp | makes sense | 12:15 |
| smcginnis | Tagging has made it to the Ns | 12:15 |
| ttx | Turning release day into an education process | 12:15 |
| smcginnis | ++ | 12:15 |
| smcginnis | Maybe we should have made this a meeting to capture it all in a nice log. | 12:16 |
| fungi | we do still have it in a log | 12:16 |
| smcginnis | Yeah, just not a nice isolated one. | 12:16 |
| evrardjp | you mean in a documentation? | 12:16 |
| fungi | complete with my various inane commentary | 12:16 |
| evrardjp | #mindblown | 12:16 |
| evrardjp | fungi: we shouldn't do otherwise | 12:16 |
| *** dave-mccowan has joined #openstack-release | 12:17 | |
| evrardjp | ttx: how does the announce-release job skip the promotion? as it seems the job still runs | 12:18 |
| *** bobh has quit IRC | 12:18 | |
| evrardjp | I mean shouldn't we rely on zuul's feature to return from jobs early in those cases? | 12:18 |
| evrardjp | (curious) | 12:18 |
| ttx | evrardjp: it's like a test on cycle-with-rc | 12:18 |
| mnaser | release pipeline is full, woo | 12:19 |
| ttx | which may or may not have been updated since we moved from cycle-with-milestones | 12:19 |
| smcginnis | It needs to run to inspect what the change is. | 12:19 |
| evrardjp | smcginnis: oh | 12:19 |
| ttx | oops\ | 12:20 |
| smcginnis | Are we not skipping those announcements? | 12:20 |
| dhellmann | I think ttx has always wanted to skip them, but we send them. http://lists.openstack.org/pipermail/release-announce/2019-April/thread.html | 12:20 |
| ttx | hmmm | 12:20 |
| dhellmann | we never quite got around to implementing that | 12:21 |
| ttx | ok, so maybe the subtlety is that we send RCs to -discuss and final to -announce | 12:21 |
| ttx | moving on... | 12:21 |
| evrardjp | maybe we should just moderate the list, and it would make it simple to determine what to announce, and only a few clicks away | 12:21 |
| dhellmann | yeah, that we do do | 12:21 |
| ttx | Historical trivia: Did you know that running release jobs used to take 7-ish hours, so that we would close every Launchpad bug usnig its very slow API ? | 12:22 |
| smcginnis | Looks like all tags complete. | 12:22 |
| smcginnis | ttx: Oh wow. | 12:22 |
| evrardjp | smcginnis: indeed! | 12:22 |
| dhellmann | masakari release note job failed | 12:22 |
| evrardjp | ttx: that seems to be a painful experience | 12:22 |
| ttx | I would statr them really early in the morning so that they would complete in time for 10am CT PR | 12:22 |
| smcginnis | Is that why we switched from marking Fix Committed to Fix Released right away. | 12:22 |
| dhellmann | http://logs.openstack.org/31/3191a411fb823913ddfed43e9c44d4e163992ebb/tag/publish-openstack-releasenotes-python3/66005d9/ | 12:23 |
| *** pabelanger has joined #openstack-release | 12:23 | |
| dhellmann | smcginnis : yes | 12:23 |
| dhellmann | ah, their release notes are still using oslosphinx | 12:23 |
| evrardjp | mmm oslosphinx | 12:23 |
| dhellmann | are we tracking failures somewhere already? | 12:23 |
| smcginnis | Masakari still using oslosphinx. | 12:23 |
| evrardjp | yeah | 12:23 |
| ttx | evrardjp: also despite all our retries it would still fail to update some particularly-busy bugs. Those would be logged into a file and closed manually | 12:23 |
| ttx | using (hang on)... | 12:24 |
| ttx | The launchpad Email interface, which had a different timeout | 12:24 |
| dhellmann | that I didn't know | 12:24 |
| ttx | That would catch 99% of the cases | 12:24 |
| ttx | Some bugs would just stay open because LP would always timeout if you tried to midify them | 12:25 |
| smcginnis | Adding release issue notes to the tracking etherpad. | 12:25 |
| *** ifat_afek has quit IRC | 12:25 | |
| evrardjp | dhellmann: maybe a weird question but where did you catch that job running? | 12:25 |
| ttx | wow that sphinx thing really messed up the process this time | 12:25 |
| *** ifat_afek has joined #openstack-release | 12:26 | |
| smcginnis | Merging a switch from oslosphinx to openstackdocstheme will get the release notes published at least. | 12:26 |
| dhellmann | evrardjp : there's an email list for release job failures, let me get you the link | 12:26 |
| evrardjp | oh | 12:26 |
| evrardjp | dhellmann: I was looking at zuul's status | 12:26 |
| dhellmann | http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures | 12:26 |
| evrardjp | I have those emails (open mailbox now) | 12:26 |
| * dhellmann nods | 12:27 | |
| evrardjp | s/open/opening/ | 12:27 |
| ttx | dhellmann: now I remember... we did keep release announcements because they are no longer sent to openstack-announce. | 12:27 |
| dhellmann | ttx: ah, yes! | 12:27 |
| ttx | leaving full openstack-announce thunder to our fearless leader | 12:27 |
| fungi | evrardjp: another option is http://zuul.opendev.org/t/openstack/builds?pipeline=release | 12:27 |
| evrardjp | ttx: I see | 12:28 |
| evrardjp | fungi: ofc | 12:28 |
| fungi | or even http://zuul.opendev.org/t/openstack/builds?pipeline=release&result=FAILURE | 12:28 |
| evrardjp | I was just wondering why I didn't catch it on zuul current run things, but that's probably because it's in the past | 12:28 |
| *** e0ne has joined #openstack-release | 12:28 | |
| evrardjp | I didn't pay enough attention so it seems | 12:29 |
| ttx | Oh, Step 4 - Tag pipelines also react to the tagging, publishing release notes | 12:29 |
| evrardjp | fungi: but that one doesn't seem to be there? | 12:29 |
| fungi | though i guess those failures were in the "tag" pipeline rather than release | 12:29 |
| fungi | so better still is http://zuul.opendev.org/t/openstack/builds?pipeline=release&result=FAILURE&pipeline=tag&pipeline=release-post | 12:29 |
| evrardjp | I see | 12:29 |
| dhellmann | it's interesting to see the release announcements come through in an apparently random order. I guess building packages for some projects are faster than others | 12:30 |
| evrardjp | well that explains WHY I was not seeing it :) I didn't check the tag pipeline, I didn't think it was a tag job for this | 12:30 |
| ttx | That step 4 is not captured in the doc. | 12:30 |
| ttx | Not sure why that stands separate from the release pipeline job... | 12:30 |
| ttx | (which is also a tag-triggered pipeline) | 12:31 |
| evrardjp | ^ that would be interesting to know | 12:31 |
| dhellmann | evrardjp : the "release" and "prerelease" pipelines are configured to run jobs based on patterns in the tag. The "tag" pipeline runs jobs for any tag. We used to have different artifact building jobs for pre-releases vs. releases, but the announcement works the same | 12:31 |
| evrardjp | ttx: it's not in your trivia box? :) | 12:31 |
| evrardjp | dhellmann: I see, but that could be different jobs in the same pipeline though? | 12:32 |
| ttx | release is given extra priority iirc | 12:32 |
| evrardjp | I see | 12:32 |
| dhellmann | yeah, pipelines can have priorities. I think today the 2 release pipelines have the same priority, but that may not always have been the case | 12:32 |
| ttx | Looks like it's going smoothly so far | 12:33 |
| dhellmann | we reworked the pipeline priorities in the last year because the gate was backing up when we had a lot of jobs in the check queue | 12:33 |
| evrardjp | (I am just wondering where all of this can be simplified, now that we have zuul v3) | 12:33 |
| dhellmann | well, I think we no longer run different jobs for different types of releases, so we might be able to get rid of one of those pipelines | 12:34 |
| fungi | there was a time when we published release artifacts to tarballs.o.o and pypi but published pre-release artifacts only to tarballs.o.o (owing to old versions of pip happily installing prereleases by default) | 12:34 |
| fungi | possible we could combine them now | 12:34 |
| fungi | as we no longer avoid publishing pre-releases to pypi | 12:35 |
| evrardjp | sorry to have so many questions... why did we avoid doing that in the past? | 12:35 |
| fungi | old versions of pip would install prereleases when you asked them to upgrade | 12:35 |
| fungi | newer versions of pip require an optional flag or config setting to enable installing pre-releases, or require you to specify the exact pre-release version you want | 12:36 |
| smcginnis | Masakari fix - https://review.openstack.org/651516 | 12:36 |
| *** udesale has joined #openstack-release | 12:36 | |
| fungi | note however that the tag pipeline also matches other sorts of tags which don't look like either release or pre-release version patterns (for example, "eol" tags) | 12:37 |
| evrardjp | fungi: oh yeah | 12:37 |
| dhellmann | right, so we still need different pipelines for releases and tags, but maybe not for releases and prereleases | 12:38 |
| *** zul has joined #openstack-release | 12:38 | |
| evrardjp | I have follow up question on why these would be different, but I can keep that for later :p | 12:39 |
| dhellmann | we're just watching jobs, you might as well ask now | 12:40 |
| fungi | i see no reason to wait until later to ask | 12:40 |
| smcginnis | masakari-monitors fix - https://review.openstack.org/651518 | 12:42 |
| ttx | One release job left | 12:42 |
| ttx | evrardjp: all tarballs are supposed to have been uploaded. Time to run missing-releases | 12:43 |
| ttx | Are you with me? | 12:43 |
| evrardjp | I am now | 12:43 |
| ttx | on a recent checkout, tox -e venv -- missing-releases --series stein | 12:43 |
| evrardjp | already running | 12:44 |
| evrardjp | :) | 12:44 |
| dhellmann | https://tmate.io | 12:44 |
| evrardjp | dhellmann: oh didn't know that | 12:44 |
| smcginnis | Oh, handy. | 12:45 |
| evrardjp | for next time :) | 12:45 |
| dhellmann | I've been using that a bit within the new team I'm on. Great for collaboration, esp. if you combine it with video chat so you can talk and type at the same time | 12:45 |
| smcginnis | Jobs are complete. | 12:45 |
| smcginnis | Raise your steins to stein. | 12:46 |
| evrardjp | smcginnis: :) done! | 12:46 |
| ttx | NB: as an extra check, I also usually run a link check on https://releases.openstack.org/stein/index.html | 12:46 |
| fungi | ooh, that's a new one on me. the infra team has used several solutions for sharing and/or streaming terminals among a group | 12:46 |
| ttx | using https://validator.w3.org/checklink/ | 12:46 |
| fungi | but i hadn't seen tmate yet. and as a long-time tmux user (once i got over using ctrl+b instead of ctrl+a) it's quite compelling | 12:47 |
| evrardjp | so what does "did not find python 2 wheel https://tarballs.openstack.org/aodh/aodh-8.0.0-py2-none-any.whl | 12:47 |
| evrardjp | " mean with the next line being "found python 2/3 wheel" | 12:47 |
| dhellmann | fungi : besides the ease of opening a connection, the other big benefit is that everyone sees the same version of the tmux session, so there's no need to tell people to change screens | 12:47 |
| smcginnis | Probably once the validation is done, looks like we are ready for https://review.openstack.org/#/c/650365/ | 12:47 |
| evrardjp | is the second just saying _a_ wheel has been found? | 12:47 |
| ttx | patience, young padawan | 12:47 |
| dhellmann | evrardjp : yes, it looks for a python-2-only wheel and a python-2-or-3 wheel | 12:47 |
| smcginnis | It will report at the end any issues. | 12:48 |
| evrardjp | dhellmann: very interesting :) | 12:48 |
| dhellmann | yeah, a lot of these release tools log what they're doing as they do it and then a summary at the end | 12:48 |
| dhellmann | not very unixy, but then you aren't sitting there looking at a blank terminal wondering if the thing is doing any work at all | 12:48 |
| evrardjp | it seems that's what I am doing right now. | 12:49 |
| dhellmann | at some point we will want to update that check to only support the 2-or-3 wheels, or maybe even just 3 wheels | 12:49 |
| evrardjp | maybe I should enable multi-threading | 12:49 |
| fungi | of course nova's releasenotes job is the last one standing | 12:49 |
| evrardjp | switch flag in /proc/brain | 12:49 |
| *** ricolin has quit IRC | 12:50 | |
| smcginnis | :) | 12:50 |
| *** ricolin has joined #openstack-release | 12:51 | |
| evrardjp | fungi: so yeah, why the difference between eol tags and release pipeline? | 12:55 |
| dhellmann | we don't build packages of the eol or em tags | 12:55 |
| evrardjp | I didn't really realise that, I wonder what's the reason | 12:56 |
| dhellmann | they aren't releases :-) | 12:56 |
| dhellmann | they're just informative markers on the repo | 12:56 |
| evrardjp | EM I can get that | 12:56 |
| evrardjp | but EOL sounds like a good thing to publish as release... but that's already enough for separation :) | 12:57 |
| smcginnis | EOL isn't a "release" though, just a tag. | 12:57 |
| dhellmann | well, EM comes before EOL and at the point when the branch switches to extended maintenance we stop doing releases | 12:58 |
| evrardjp | yeah like what dhellmann said | 12:58 |
| *** lbragstad has joined #openstack-release | 12:58 | |
| dhellmann | EM and EOL also aren't "versions", so if we did want to tag a release at those points we would use a regular version and then add the other tag | 12:58 |
| dhellmann | I wonder if that nova release notes job is ok? the stream just shows "END OF STREAM" over and over | 12:59 |
| dhellmann | http://zuul.openstack.org/stream/cf2f6940f4c2443f99bc65f1ab43195d?logfile=console.log | 12:59 |
| evrardjp | I think my head can agree on the fact it makes sense to not see something on pypi named projectA:version-eol :) | 13:00 |
| jrist | seems not good dhellmann | 13:00 |
| evrardjp | dhellmann: isn't that what happens at the end of the job? | 13:00 |
| dhellmann | I would expect it to just stop showing output | 13:00 |
| ttx | END OF STREAM | 13:00 |
| ttx | END OF STREAM | 13:00 |
| dhellmann | I wonder if it's hung starting? or crash-looping? | 13:00 |
| evrardjp | ttx: stop your end of stream :) 9 errors | 13:01 |
| * ttx switches to meeting tunnel, but will be around to report on the linkchecks | 13:01 | |
| *** udesale has quit IRC | 13:01 | |
| evrardjp | http://paste.openstack.org/show/749110/ | 13:01 |
| fungi | currently zuul's status api provides links to console streams for jobs which have not yet finished initializing, before the stream becomes available | 13:01 |
| *** mriedem has joined #openstack-release | 13:02 | |
| smcginnis | So a few signatures, a few wheels and a couple not uploaded to pypi. | 13:02 |
| fungi | at some point i expect it will grow some way of either indicating when the stream becomes ready so the status page can avoid hyperlinking to it early, or will just delay including the stream details until there is one available | 13:02 |
| *** udesale has joined #openstack-release | 13:03 | |
| evrardjp | smcginnis: correct | 13:03 |
| evrardjp | smcginnis: so what is the next step from that on? | 13:03 |
| smcginnis | fungi: Have you just manually taken care of those in the past? ^ | 13:03 |
| *** irclogbot_0 has joined #openstack-release | 13:04 | |
| fungi | smcginnis: depends on the situation | 13:04 |
| fungi | are those all for the current release? | 13:04 |
| evrardjp | I will check | 13:05 |
| ttx | linkcheck still running but I expect it to report the same | 13:05 |
| fungi | we'll want to find the job which was responsible for generating/publishing the missing bits reported there and see why each failed | 13:05 |
| smcginnis | kuryr-kubernetes is this current release. | 13:06 |
| evrardjp | smcginnis: sure? | 13:06 |
| smcginnis | Same with monasco-kibana-plugin. | 13:07 |
| evrardjp | I should refresh my eyes | 13:07 |
| smcginnis | Actually, these are cycle-with-intermediary ones. | 13:07 |
| *** altlogbot_2 has joined #openstack-release | 13:07 | |
| ttx | so could be old fails | 13:07 |
| fungi | this is my suspicion, which is why i asked | 13:08 |
| evrardjp | yeah all of them seems to not be from https://review.openstack.org/#/c/650362/ | 13:08 |
| fungi | some of the package names and versions looked familiar | 13:08 |
| smcginnis | Same for puppet-mistral. | 13:08 |
| evrardjp | so it's a \o/ then ? | 13:08 |
| smcginnis | Old fails, but for the latest releases. | 13:08 |
| fungi | odds are we could find where we discussed each of them and concluded they were unfixable without tagging a new release | 13:09 |
| evrardjp | the question still stands | 13:09 |
| evrardjp | what to do if it was something happening in this one (and what to do with the old ones?) | 13:09 |
| smcginnis | Same with tripleo-ui. | 13:09 |
| evrardjp | fungi: I see | 13:09 |
| evrardjp | maybe we should mark them in the tooling as part of the failure history | 13:10 |
| fungi | if it was something happening from the current run we would probably have also seen a job failure for it, but the action to take depends a lot on how and why it fails | 13:10 |
| dhellmann | it would be useful if that script reported the series associated with each missing thing | 13:10 |
| evrardjp | fungi: so it's something like re-enqueue some of the jobs for those repos? | 13:10 |
| dhellmann | aside from the masakari release notes failures and the odd thing with nova's release notes, I haven't seen any failures today | 13:10 |
| fungi | evrardjp: maybe, or manually copy artifacts from one place to another, or tag a new release | 13:11 |
| smcginnis | Tripleo-ui failure was here http://lists.openstack.org/pipermail/release-job-failures/2019-March/001104.html | 13:11 |
| evrardjp | I see | 13:11 |
| fungi | evrardjp: as i said, how we deal with different failures depends on how and why things failed, there's not just one answer | 13:11 |
| smcginnis | http://logs.openstack.org/b5/b5dbb90fb347db2876f61cae8c2623f9ce767dfe/release/release-openstack-javascript/1c22748/job-output.txt.gz#_2019-03-14_19_36_52_223039 | 13:11 |
| evrardjp | fungi: yeah I understand | 13:12 |
| fungi | some of those jobs are not idempotent for various reasons of practicality and pragmatism | 13:12 |
| smcginnis | So do we care about these missing c-w-i? | 13:12 |
| fungi | it's likely some/most of those could stand to have a new release tagged on stable/stein, but that then gets into constraints updates | 13:14 |
| fungi | so probably better to just plan to have very early stable point releases of them? | 13:14 |
| smcginnis | Works for me. | 13:14 |
| ttx | I have 37 404 on the stein page | 13:14 |
| evrardjp | ttx: :( | 13:14 |
| smcginnis | 404's on what links? Release notes | 13:15 |
| ttx | paste in progress | 13:15 |
| smcginnis | Gotta grab a quick breakfast with my wife. Be back in a bit. | 13:15 |
| ttx | http://paste.openstack.org/show/749111/ | 13:16 |
| ttx | some are irrelevant and coming from the template | 13:16 |
| ttx | some others are coming from old releases | 13:17 |
| ttx | Like https://tarballs.openstack.org/oslo.policy/oslo.policy-1.39.0.tar.gz | 13:17 |
| ttx | was redone as 1.39.1 | 13:17 |
| ttx | so probably just needs a clean up so that 1.39.0 is not listed | 13:18 |
| evrardjp | same for triple-ui logically :) | 13:18 |
| evrardjp | ttx: should we go through each of those ? | 13:19 |
| ttx | probably yes, can't while I'm in my meeting | 13:19 |
| ttx | should be done in 40min | 13:19 |
| *** ifat_afek has quit IRC | 13:20 | |
| evrardjp | is that normal I don't see ironic in there? | 13:20 |
| *** e0ne has quit IRC | 13:20 | |
| dhellmann | don't see ironic where? | 13:21 |
| *** amoralej|lunch is now known as amoralej | 13:21 | |
| evrardjp | ttx: it seems all of those deadlinks are not from https://review.openstack.org/#/c/650362/ | 13:21 |
| dhellmann | I think the scan he was running just checks all of the links on the site with a web crawler | 13:22 |
| evrardjp | yes | 13:22 |
| evrardjp | so that would explain why old failures could still be linked in that crawler | 13:22 |
| *** ifat_afek has joined #openstack-release | 13:22 | |
| fungi | evrardjp: going back to an earlier question about why we have a release pipeline which has a strict regex match on the tag name, separate from a pipeline which matches any tag... one big reason was people pushing incorrectly/inconsistently-formatted tags. by having our release artifact publishing jobs in a pipeline with a strict version pattern match we catch and avoid accidentally releasing | 13:23 |
| evrardjp | so I mean I don't believe those issues are due to this release | 13:23 |
| fungi | artifacts for anything which doesn't match, but there are some projects which are relevant to other ecosystems (java, npm, dockerhub, puppetforge...) where versioning conventions might be different than for the python packaging ecosystem/pypi | 13:23 |
| evrardjp | fungi: Aaaah! that makes sense | 13:23 |
| fungi | for example, we've had folks push tags like v1.2.3 instead of 1.2.3 for python projects, and release automation doesn't run because that pattern doesn't match | 13:24 |
| fungi | that way they still have to go back and add a separate 1.2.3 tag before they get release artifacts generated and published | 13:25 |
| *** e0ne has joined #openstack-release | 13:25 | |
| fungi | it's not as crucial for projects using the releases repo because there's preliminary checking and review, but there are lots of unofficial projects in opendev which use the same jobs triggered by manually pushing a tag | 13:25 |
| evrardjp | dhellmann: my question was why don't I see ironic in there: https://review.openstack.org/#/c/650362/ ... but it's because it's not cycle-with-rc. | 13:26 |
| fungi | yes, ironic is cwi | 13:26 |
| dhellmann | right | 13:26 |
| fungi | though that's been called into question a few times since they weren't tagging more than one release a cycle anyway | 13:27 |
| evrardjp | I was just a little surprised :) | 13:27 |
| *** e0ne has quit IRC | 13:54 | |
| *** e0ne has joined #openstack-release | 13:57 | |
| *** mlavalle has joined #openstack-release | 13:58 | |
| smcginnis | I think we probably want to approve https://review.openstack.org/650365 before I send out the announcement? | 14:01 |
| *** ifat_afek has quit IRC | 14:02 | |
| ttx | yes | 14:05 |
| ttx | smcginnis: go go go | 14:07 |
| * ttx is back from call | 14:08 | |
| ttx | once that merges, feel free to send the announcement | 14:09 |
| smcginnis | Queued up and ready to go. | 14:10 |
| ttx | I'll propose a change to clean up dead links... we can discuss how much of a good idea it is on the review | 14:10 |
| ttx | but that's not a release blocker | 14:10 |
| smcginnis | ++ | 14:11 |
| *** ykarel is now known as ykarel|away | 14:11 | |
| *** openstackgerrit has quit IRC | 14:14 | |
| *** openstackgerrit has joined #openstack-release | 14:19 | |
| openstackgerrit | Thierry Carrez proposed openstack/releases master: Clean up failed releases https://review.openstack.org/651549 | 14:19 |
| ttx | That leaves tripleo-ui but that one always failed. Should probably be no-tarball-link | 14:21 |
| ttx | I mean artifact-link-mode: none | 14:21 |
| smcginnis | That makes sense. | 14:22 |
| * ttx tries that | 14:22 | |
| openstackgerrit | Thierry Carrez proposed openstack/releases master: Do not generate tarball links for tripleo-ui https://review.openstack.org/651551 | 14:24 |
| openstackgerrit | Merged openstack/releases master: Mark Stein as released https://review.openstack.org/650365 | 14:24 |
| * smcginnis waits on release-post | 14:25 | |
| ttx | OK, I'll tell Jimmy to flip the release link to the Stein page when that post-job finishes | 14:25 |
| ttx | then you can send email | 14:25 |
| ttx | feel free to review and approve the two cleanups above, before we open the floodgates. I doubt people will frantically click on those links but meh | 14:26 |
| openstackgerrit | Rico Lin proposed openstack/releases master: release heat 11.0.2 https://review.openstack.org/651557 | 14:34 |
| *** ykarel|away has quit IRC | 14:35 | |
| smcginnis | A watched publish-tox-docs-static never finishes. | 14:35 |
| smcginnis | ttx: Job completed. Can I send our should I wait for Jimmy? | 14:43 |
| ttx | smcginnis: i asked him to flip the switch. So you can send | 14:43 |
| smcginnis | Sent! | 14:45 |
| smcginnis | Thanks everyone for working on the Stein release! | 14:45 |
| smcginnis | Doh, needs moderator approval. :) | 14:45 |
| ttx | ah! can fix that | 14:45 |
| ttx | done | 14:46 |
| evrardjp | ttx: could you check my comment on https://review.openstack.org/#/c/651549/ ? | 14:46 |
| ttx | http://lists.openstack.org/pipermail/openstack-announce/2019-April/002019.html | 14:46 |
| smcginnis | Woot! | 14:47 |
| ttx | you can send to discuss | 14:47 |
| evrardjp | woot :) | 14:47 |
| smcginnis | Yep, just waiting for it to actually show up in my inbox to forward it. | 14:47 |
| ttx | Congrats team! | 14:47 |
| evrardjp | congrats everyone! | 14:47 |
| *** e0ne has quit IRC | 14:50 | |
| *** e0ne has joined #openstack-release | 14:54 | |
| *** ianychoi has quit IRC | 14:56 | |
| smcginnis | Still didnt show up in my inbox for some reason. Guess I'll just send the link out. | 14:56 |
| ttx | I got it | 14:57 |
| smcginnis | Oh, there it is. | 14:59 |
| *** diablo_rojo has joined #openstack-release | 15:02 | |
| *** ykarel has joined #openstack-release | 15:07 | |
| * ttx opens a beer | 15:09 | |
| smcginnis | Might be good to get this one through: https://review.openstack.org/#/c/650173/ | 15:11 |
| smcginnis | Would open a beer too, but that might be frowned upon given my timezone. :) | 15:11 |
| cdent | pshaw! | 15:12 |
| cdent | https://twitter.com/anticdent/status/1115981337717084161 | 15:12 |
| smcginnis | I do work from home, so not like anyone would really knwo. | 15:12 |
| *** armax has joined #openstack-release | 15:12 | |
| ttx | cdent: you inspired me, I must say | 15:12 |
| * cdent drinks double for smcginnis | 15:12 | |
| smcginnis | cdent: And well deserved. | 15:13 |
| cdent | I often struggle to like beer, but today is a) a long time coming, b) rather sunny and warm so... | 15:13 |
| smcginnis | Perfect | 15:13 |
| smcginnis | My excuse would be it's going to snow again. | 15:14 |
| cdent | ah yes, my nephew (who lives around there somewhere) reports potential blizzards | 15:14 |
| smcginnis | Yep | 15:15 |
| smcginnis | (sadness) | 15:15 |
| cdent | runs the rochester fresh thyme | 15:16 |
| cdent | at leat I think it is the rochester one | 15:16 |
| openstackgerrit | Sean McGinnis proposed openstack/releases master: More stein release note links https://review.openstack.org/651578 | 15:18 |
| *** dave-mccowan has quit IRC | 15:23 | |
| *** tesseract has quit IRC | 16:03 | |
| openstackgerrit | Brian Rosmaita proposed openstack/releases master: Release python-brick-cinderclient-ext 0.5.1 https://review.openstack.org/651595 | 16:14 |
| *** dtantsur is now known as dtantsur|afk | 16:15 | |
| openstackgerrit | Brian Rosmaita proposed openstack/releases master: Release cinder 11.2.2 https://review.openstack.org/651596 | 16:16 |
| cdent | "155 changes per day" that's impressive. what's the max day? and total for the cycle? | 16:38 |
| smcginnis | ttx: You got all of that from a script, right? ^ | 16:40 |
| *** altlogbot_2 has quit IRC | 16:45 | |
| *** e0ne has quit IRC | 16:55 | |
| openstackgerrit | Rico Lin proposed openstack/releases master: release heat 11.0.2 https://review.openstack.org/651557 | 16:58 |
| *** jpich has quit IRC | 17:00 | |
| *** ykarel is now known as ykarel|away | 17:03 | |
| *** amoralej is now known as amoralej|off | 17:22 | |
| *** udesale has quit IRC | 17:23 | |
| *** priteau has quit IRC | 17:31 | |
| *** ekcs has joined #openstack-release | 17:45 | |
| *** ricolin has quit IRC | 17:49 | |
| *** tosky has quit IRC | 18:21 | |
| *** electrofelix has quit IRC | 18:24 | |
| *** hberaud is now known as hberaud|gone | 18:27 | |
| *** zul has quit IRC | 18:28 | |
| *** cdent has quit IRC | 18:54 | |
| *** ykarel|away has quit IRC | 19:00 | |
| -openstackstatus- NOTICE: Restarting Gerrit on review.openstack.org to pick up new configuration for the replication plugin | 19:05 | |
| *** priteau has joined #openstack-release | 20:33 | |
| *** priteau has quit IRC | 20:48 | |
| smcginnis | tonyb: I was going to leave setting up the train tracking etherpad to you. | 20:49 |
| smcginnis | tonyb: There's a script - tools/list_weeks - that we've been running to get the output and paste into an etherpad. | 20:49 |
| smcginnis | tonyb: Then some edits to add/change a few things. | 20:50 |
| tonyb | smcginnis: funny you shoudl say that, I have a TODO item to workout where the tool is to do that ;P | 20:50 |
| tonyb | smcginnis: Thanks! | 20:50 |
| smcginnis | :) | 20:50 |
| smcginnis | Any questions with it, just let me know. | 20:50 |
| tonyb | smcginnis: will do | 20:50 |
| * tonyb needs to add an item for tomorrows meeting | 20:50 | |
| tonyb | done | 21:00 |
| tonyb | evrardjp: Any chance you can review https://review.openstack.org/#/c/651443/ if you +1 it we can 'make it so' for next week | 21:02 |
| *** bobh has joined #openstack-release | 21:39 | |
| *** mriedem has quit IRC | 21:40 | |
| *** bobh has quit IRC | 21:47 | |
| *** bobh has joined #openstack-release | 21:48 | |
| *** bobh has quit IRC | 22:14 | |
| *** mlavalle has quit IRC | 22:19 | |
| openstackgerrit | Brian Rosmaita proposed openstack/releases master: Release cinder 11.2.2 https://review.openstack.org/651596 | 22:19 |
| *** tosky has joined #openstack-release | 22:26 | |
| *** tosky has quit IRC | 22:47 | |
| *** whoami-rajat has quit IRC | 22:51 | |
| tonyb | I think I've asked this before but for want of a better place would y'all mind if I created a stable-process.rst in the reference directory so I can have a place to drop 'stable/EM tribal knowledge' | 23:38 |
| tonyb | I s'pose some of it might end up int he project-teams-guide eventually | 23:38 |
| diablo_rojo | No complaints from me, though I expect I am not really the one you are asking. | 23:43 |
| *** ianychoi has joined #openstack-release | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!