Thursday, 2025-11-13

opendevreviewDavid proposed openstack/watcher-tempest-plugin master: Add test for skipped actions  https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/96686008:54
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Fixed incorrect use of status_choices in statetable  https://review.opendev.org/c/openstack/watcher-dashboard/+/95918909:39
jgilaberhi all watcher meeting starting shortly, sorry forgot to send a reminder today!12:00
jgilaber#startmeeting watcher12:00
opendevmeetMeeting started Thu Nov 13 12:00:47 2025 UTC and is due to finish in 60 minutes.  The chair is jgilaber. Information about MeetBot at http://wiki.debian.org/MeetBot.12:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.12:00
opendevmeetThe meeting name has been set to 'watcher'12:00
chandankumaro/12:01
rlandy_o/12:01
*** rlandy_ is now known as rlandy12:01
morenodo/12:01
jgilabercourtesy ping: dviroel amoralej sean-k-mooney 12:01
amoralejo/12:02
sean-k-mooneyo/12:02
dviroelo/12:02
jgilaberok let's get started with today agenda, here's the link in case someone has a last minute topic https://etherpad.opendev.org/p/openstack-watcher-irc-meeting12:02
jgilaber#link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting12:02
jgilaberwe have some topic already12:03
jgilaberI added the first12:03
jgilaber#topic 2026.1 status etherpad review12:03
jgilaber#link https://etherpad.opendev.org/p/watcher-2026.1-status12:03
sean-k-mooneyi need to step a way for 5-10 minues sorry brb12:04
jgilaberwe have two specs ready for review12:04
jgilabertwo active blueprints12:04
jgilaberand two patches12:04
jgilaber*three actually12:04
jgilaberdoes anyone want to highlight something? 12:05
amoralejI added a topic to discuss about one of the blueprints in the agenda12:05
jgilaberack thanks amoralej 12:06
jgilaberI wanted to quickly point out that two of the patches are action items from the PTG12:06
jgilaber1. deprecate prometheus datasource in favor of aetos12:06
jgilaber#link https://review.opendev.org/c/openstack/watcher/+/96667212:06
jgilaberand 2. remove monasca datasource12:06
jgilaber#link https://review.opendev.org/c/openstack/watcher/+/96678612:07
dviroel++ thanks jgilaber 12:07
jgilaberIIUC before proceeding with the deprecation we need to announce it in the ML12:07
jgilaberdo we need to do the same for the removal?12:07
dviroelI am not sure, would need to check governance docs, but I don't think we need12:09
jgilaberack thanks dviroel, I can check that later12:09
jgilaberif there are no more thoughts on this topic we can move to the next12:09
dviroelone thing 12:09
dviroeljust to mention, in the same line12:09
amoralejfor monasca, i think it's fine given that monasca is marked as inactive12:10
dviroelthat yesterday I sent another call for MAAS maintainers12:10
amoralejfor prometheus, it may be good to ask in the ML12:10
dviroel#link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/VIY3V2XD5H2KSSBQXLBS2QHKG24JKCQZ/12:10
dviroeljsut saw that we have an reply on that topic, but still, dmitriy doesn't plan to work on that soon12:11
dviroeli will reply after our meeting12:11
jgilaberis keeping the MAAS deprecated a hard blocker on the eventlet removal?12:11
jgilabercould we for example keep it as only working with evenlet?12:12
dviroelwe need to remove MAAS code since it has evenlet-dependent code there12:12
dviroelso we would deprecate it this cycle12:12
dviroelwe can't modify that code since we can't validate it12:13
sean-k-mooneyback12:13
dviroelso the final move in eventlet-removal is to really remove eventlet code :) 12:13
dviroeland MAAS code will need to go together unless someone fix it12:14
jgilaberack I think I got confused because the proposal here is to mark as deprecated not removal in this cycle12:14
dviroelcorrect12:14
jgilaberso +1 for me to continue with the plan despite the reply12:14
dviroelwe need to deprecate in a SLURP release12:14
dviroeland remove in a future release12:14
dviroelwe don't need to remove all code in 2026.112:15
sean-k-mooneytechnially we do not need to annouch deprecation on the mailing list but we can for more visisbality. however as part of the deprecation of the direct backend we shoudl provde upgrade instuction on how to adopt aetos we may want to do that in the deprecation patch but it shoudl happen ebfore the end of the cycle in either case12:15
sean-k-mooneyjgilaber: its not a hard blocker as maas is an optional integration12:15
opendevreviewDavid proposed openstack/watcher-tempest-plugin master: Add test for skipped actions  https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/96686012:15
jgilabersean-k-mooney, thanks I did include a migration guide in the patch12:16
sean-k-mooneyin threaded mode it just wont work properly12:16
sean-k-mooneywhat it is a blocker for woudl be removal of eventlet entirly unless we prot it or remove it12:16
sean-k-mooneybut we shoudl be able to achive this cycle goal of being able to run all compent s in threaded mode12:17
sean-k-mooneyi decided not to reply12:17
jgilaberthe final deadline for the evenlet removal is 2027.1?12:17
dviroelabout eventlet-removal schedule12:17
dviroel#link https://governance.openstack.org/tc//goals/selected/remove-eventlet.html#completion-criteria12:17
dviroel2027.212:17
sean-k-mooneybut even without the eventlet depency we are not currently planning to use ascynio12:17
jgilaberthanks for the link dviroel 12:17
sean-k-mooneyso the fact the mass code does use that is also a problem form my point of view12:18
sean-k-mooneymeaning it may need to be rewritten even if we ketp the integration12:18
sean-k-mooney2027.1 for services12:18
sean-k-mooney2027.2 for oslo12:19
sean-k-mooneyour goal is 2026.212:19
sean-k-mooneyfor removal and defaultign to threaded shoudl ideally happen this cycle12:19
sean-k-mooneywe have a full cycle of buffer so it ok if our schdule gets delayed a bit12:20
jgilaberso we should deprecate MAAS in this cycle and remove in the next?12:20
sean-k-mooneyyep12:20
sean-k-mooneythat woudl be my perfernce in any case12:20
dviroelthat's the pan12:20
dviroels/pan/plan12:20
jgilaberwfm, thanks!12:20
jgilaberany other thoughts on this topic?12:21
dviroellets move :) 12:21
jgilaberok, moving on chandankumar looks like the next topic is yours12:21
jgilaber#topic Updates on Pytest vs PTI discussion 12:21
chandankumarAs we know, watcher-dashboard tests are broken, needs rewriting.12:22
chandankumarI sent an email <  https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/3V3CNPQLB77SKFVLZ6LXJ5NPNYWW4QFD/ > needing TC guidance on using use Horizon's pytest fixtures in PTI doc.12:22
jgilaber#link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/3V3CNPQLB77SKFVLZ6LXJ5NPNYWW4QFD/12:22
chandankumarPTI recommends using stestr as a testrunner via tox and use unittest module for tests.12:22
chandankumarBelow is the summary of the thread.12:22
chandankumarmany people supports addition of pytest in the PTI doc as the tool has evolved over time. Some does not.12:22
chandankumarBefore modifying PTI doc we also need to emphasize on the importance of understanding why rules exist before changing them.12:22
chandankumarkeypoints discussed:12:22
chandankumarHorizon is using pytest as tests are more maintainable and less flaky in CI. Horizon team were not aware about PTI.12:22
chandankumar Horizon has never used stestr for running integration tests after 2018. Some exception made in past around nosetest usage in horizon < https://opendev.org/openstack/governance/commit/759c42b10cb3728f5549b05f68e826b1c62a968c >12:23
chandankumarRally is using pytest-fixtures module and pytest as a test runner.12:23
chandankumar pytest lacks parallel test execution but seems to be solved by pytest-xdist, But someone needs to verify does it works with shared resources.12:23
chandankumar pytest framework fixture dependency injection mechanism is not standard and locks tests into requiring pytest as the runner.12:23
sean-k-mooneythe test being more maintainable and less flaky is not because of pytest its because of how they were re written12:23
chandankumar If we start using pytest, devs will add multiple pytest plugins, which might create a headache for packagers to package it.12:23
chandankumarNot all the current unittest based integration tests are runned via pytest(I have not verified it).12:23
chandankumar[other topic from this thread] replacing Horizon's local_settings.py with oslo.config-based configuration discussed long time back, no-one is working on that12:23
chandankumarthat's the summary12:23
chandankumarI need some help to draft further response with following points12:24
chandankumar[1]. Should we use Horizon's pytest work(they will not going to rewrite it into unittest) OR build unittest tests for consistency with main watcher project(will avoid learning new test framework)? We will be needing an spec for integration test.12:24
chandankumaror Since we started the thread, should we propose a patch to update the PTI doc around using pytest? or let TC decide on this.12:24
sean-k-mooneyright now i am -1 on useing there test framewrok12:24
sean-k-mooney-2 if it not in the pti12:24
jgilaberthanks for the summary chandankumar, looks like there are a few related thread12:24
chandankumaryes, there was one similar thread related to keystone.12:25
sean-k-mooneydo you mean the rust one?12:26
chandankumaryes12:26
sean-k-mooneybecause that is very diffent12:26
jgilaberre the packaging, I'm not sure I understand the problem if it's already in Horizon12:26
sean-k-mooneyya so that is a cause of the peoron wanting to use rust activly not following the process to have it propsed as a new language12:26
chandankumarjgilaber: that was raised by debian packager.12:27
sean-k-mooneyjgilaber: so it might be in horizon but that does not mean its in rdo or in other distros12:27
sean-k-mooneyjgilaber: there was an explcit rule that it shoudl nto be allowed in the requriement repo (pytest)12:27
chandankumarWe also need to choose selenium/playwright for integration test.12:27
sean-k-mooneythe test runner was allowed but it was expclity not12:27
jgilaberbut if those distros pacakge horizon they must have dealt with it in some way, no?12:28
chandankumarI need help on deciding next course of action here12:28
dviroelis it going to be to hard to write our own tests and not depend on  horizon? don't seems to be too much code in there tbh12:28
chandankumarnothing is hard, whatever we write, we need to maintain it for longer term12:28
jgilaber+1 to what dviroel said, looking at the amount of replies in the mailing list, this looks like a complicated topic12:29
sean-k-mooneychandankumar: so we could do a poc of both approches and review however i thikn we also shoudl have a spec for this topic12:29
sean-k-mooneyadopting playwright aslo has packaging implication but unless unittests the borwser test faramework is not specified in the pti12:29
sean-k-mooneyso we can sleect the one that is best suited12:30
chandankumarone question? for spec do we want to do poc first then spec12:31
chandankumarpoc with selinium with unittest and poc with playwright with unit test?12:31
jgilaberI think it this case it would be helpful to have pocs while reviewing the spec12:31
sean-k-mooneyso normally you do the spec first to agree on the requirement and probelm statement and you amy do a very simple poc to supprot the spec12:31
sean-k-mooneybut obvioulsy you would not start the main bulk of the implemeiton until after the spec is approved12:32
chandankumarok, I will take a look at smallest bit and do the poc and with spec for both selenium and playwright12:33
chandankumartaking this small bit as a example https://review.opendev.org/c/openstack/watcher-dashboard/+/959189: Fixed incorrect use of status_choices in statetable12:34
chandankumarfor testing via poc12:34
chandankumaror do we need a different example?12:35
jgilabernot sure about that, but I think that any example will do to show the pattern the tests should follow12:36
sean-k-mooneythat could work bu it not what i woudl start with12:38
sean-k-mooneyi woudl start with the basics fo naviagting to a pannel for exampel the action plan templeate o r goal or similar12:39
sean-k-mooneyand show hwo we will test that the relevent element are tehre.12:39
sean-k-mooneyfor example we coudl start with creat a audit then check that an action plan is created12:40
chandankumarsean-k-mooney: that sounds like a good example, thank you!12:40
jgilaberthanks chandankumar, any more thoughts on this topic?12:40
chandankumarNow I have the next course of action.12:40
sean-k-mooneyi might spend a day and try my own poc just to get a feel for it depending on how much time i have next week12:41
jgilaberack, moving to the next topic from dviroel 12:41
jgilaber#topic Issues with Taskflow parallel engine in threading mode12:41
dviroeltks12:41
sean-k-mooneywhat i really want to see is what is the ux fo creatign a test, and what output we get and howt this works end to end12:41
sean-k-mooneybut ya lets move on12:42
chandankumarthank you everyone for input on this thread12:42
dviroelok, let me add some background on that issue/topic12:42
dviroelwhile working on eventlet-removal12:42
dviroel but first adding support to native thread mode, we start to see sql error in our unit tests, when executing action plans, e.g:12:42
dviroel#link https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f82/openstack/f827ad85b1294c43bae471abcbb69d2d/testr_results.html12:42
dviroel"error: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) cannot start a transaction within a transaction"12:42
dviroelsomething that doesn't reproduce in the integration tests12:43
dviroelbut still reproducible in unit tests12:43
dviroelit happens that our applier works with multiple threadpools today12:43
dviroelthe processing of starting action plans is handled by a threadpool executor, where each action plan creates a new taskflow engine that when working in parallel mode creates another threadpool12:44
dviroelin addition to that we also have a thread spawn for each action, so it can be killed when running in eventlet mode12:44
dviroelnot an easy thing to debug or follow12:45
dviroelwhile investigation the issue, i asked some help from sean-k-mooney (tks!), and we start to dig into details, and how watcher currently implements the db access using oslo.db12:45
dviroelthe current implementation in our db api, using a threading.local object as context may not be the best approach12:46
dviroeland we may need to rework everything there, but still no guarantee that this would solve our concurrency problems12:46
dviroelso we want to propose a short-term solution for now to unblock the eventlet-removal work12:47
dviroelwhich is implemented in12:47
dviroel#link https://review.opendev.org/c/openstack/watcher/+/96622612:47
dviroelwhere we configure taskflow engine to work with the "serial" mode when native thread is configured12:48
dviroelwith that, we would lose the parallellism across actions of a action plan, when running in native thread mode (for now)12:49
sean-k-mooneyprocess mode woudl also likely work but that feels to heavy weight if we are going to allow more then 1 action plan to be proceed at once per applier12:49
dviroelsean-k-mooney: agree12:50
sean-k-mooneythe fact that we have potenitaly 2 levels of concurancy (concurrent actions and concurrent action plans) 12:50
dviroelso with that we can continue working on a better solution to enable parallellism, which may need some rework on the db api and in the applier12:50
sean-k-mooneymakes the scaling and resource usage harder to reason about12:50
amoralejlosing parallelism in an actionplan is a bad thing, tbh12:51
amoralejbut if there are no alternative ...12:51
sean-k-mooneymy curent thinking is as follows12:51
dviroelamoralej: yes, but this is the short-term solution while we improve watcher12:51
sean-k-mooneyi thiks that we need ot factor out the management of the action plan and the concurance of actions within that into a seperate compoent12:52
sean-k-mooneyand then an applier will be able to execute multipel actions but each action woudl be a result fo an rpc call12:52
sean-k-mooneyso we woudl not use tasks flows engine to manage the concurancy 12:53
sean-k-mooneyat least not the way we do today12:53
amoralejso having an action-dispatcher12:54
dviroelthis could be considered as part of the scaling topic that amoralej is working with too, since it may solve more problems12:54
sean-k-mooneyyes, each node in the graph woudl effectivly jsut be dispatching an rpc to be proceed by the applier12:54
amoralejyes, that's what i'm thinking about12:54
amoraleji think that's a good solution, but also it will take time to develop12:55
dviroeland that's why we wouldn't be fixing now in eventlet-removal effort12:55
sean-k-mooneyright the other thing is if we can run 50 appliers12:55
amoralejthat's why i'm afraid that the short-term solution may stay for some time ...12:55
sean-k-mooneyand load balcne action plans over applier12:55
amoraleji think that's better actually, as a short term12:56
sean-k-mooneywe may or may not need the 2 levels fo concurance we have today even without the action dispatcher12:56
amoralejif each applier can only run an actionplan at a time, that would also fix the issue?12:56
sean-k-mooneyno12:56
amoralejah, I missunderstood then :)12:57
sean-k-mooneythe probelm is that each action is sharign a singel db connection12:57
sean-k-mooneywe effectivly need one connection per thread12:57
sean-k-mooneyif we want to re enabel the threadpool engine12:57
sean-k-mooneythat or have all db oepratiosn done by the top level thread12:57
amoralejok, that's the change in db access that dviroel mentioned before12:57
sean-k-mooneythat the actual probelm we have today12:58
sean-k-mooneyya there is maybe oen other hack we coudl do in the short-medium term12:58
sean-k-mooneywe coudl intoduce a transaction lock12:58
sean-k-mooneyeffectivly to serialise all db transactions12:59
sean-k-mooneymy concern with taht is obviously again perfaomce and possibel deadlocks12:59
amoralejto be clear, i don't mind having a short-term solution that reduces concurrency if we are able to do a better fix during the G releasse12:59
sean-k-mooneywell for now this woudl only be enabled if you disable eventlet12:59
amoralejthat's right.... good point13:00
sean-k-mooneyso there is actully no regression unless you opt into threaded mode13:00
sean-k-mooneyso we really have until the end of 2026.213:00
dviroelack, and we don't need to enable threading mode for the applier as default in this release13:00
amoralejright13:00
sean-k-mooneyi.e. when we remove the eventlet option even 2027.1 if we really need it13:00
sean-k-mooneyso to summerise the general propsal13:01
amoralejthat makes sense13:01
sean-k-mooney1 in thread mode use the serial engine for now13:01
sean-k-mooney2 continue to supprot the greenpool when using eventlet13:01
sean-k-mooney3 before we remvoe eventlet supprot we shoudl develop a way to do parallel action execution in threaded mode13:01
sean-k-mooney4 defer making it the default until 3 is done13:02
amoralejwfm +113:02
dviroelyeah, sounds like a good plan13:02
jgilaber+113:02
jgilaberthat was a good discussion, but we're overtime and have one more topic to cover13:03
sean-k-mooneyack lets move on13:03
jgilaberwe can continue in irc or the patch13:03
dviroelack, yes jgilaber 13:03
jgilaberlast topic, we can leave the bug triage for next week13:03
jgilaber#topic  new blueprint for automatic skip actions on pre_condition13:03
jgilaberfrom amoralej 13:03
amoralej#link  https://blueprints.launchpad.net/watcher/+spec/skip-actions-in-pre-condition13:03
amoralejas discussed I've created the blueprint for automatic actions skip13:04
amoralejand send initial patch for migrate action https://review.opendev.org/c/openstack/watcher/+/96669913:04
amoraleji plan to send a review per-action13:04
amoralejI'm not sure what's the review process for blueprints, given that there is no peer review13:05
amoralejlemme know if i should add something13:05
sean-k-mooneyamoralej: the normal proces is to present them in a team meeting13:05
sean-k-mooneywhere we would agree it can proceed without a spec or ask for one13:05
amoralejok, so, that was my plan for today :)13:05
sean-k-mooneyand then if we agree we mark it as approved for the cycle and leave a comment in the blueprint13:05
amoralejwe can discuss on next mtg anyway13:05
dviroel+!13:06
dviroel+113:06
sean-k-mooneyok we can come back to it then13:06
amoralejsure, no problem13:06
amoralejI'll add the topic to the agenda13:06
jgilaberack, thanks amoralej13:07
jgilaberlast topic 13:07
jgilaber#topic Volunteers to chair next meeting13:07
jgilaberany volunteer?13:07
dviroeli can't :( - local holiday next thursday13:08
rlandyI'l do it13:08
jgilabernp dviroel thanks rlandy 13:08
jgilaberthat's it for today, thanks all for participating!13:09
dviroelthank you all13:09
jgilaber#endmeeting13:09
opendevmeetMeeting ended Thu Nov 13 13:09:12 2025 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:09
opendevmeetMinutes:        https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-11-13-12.00.html13:09
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-11-13-12.00.txt13:09
opendevmeetLog:            https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-11-13-12.00.log.html13:09
dviroelthanks jgilaber o/13:09
amoralejthanks jgilaber++13:09
morenodthanks jgilaber++!!13:10
amoralejbtw, while working in the pre_condition checks for migrate i found an issue in host_maintenance, https://review.opendev.org/c/openstack/watcher/+/966777 is independend on the skip13:11
amoralejit was not causing any real issue until now, as the source_node was only used in abort which is never called13:11
sean-k-mooneywe have talked about that before13:13
sean-k-mooneyits not a new issue but i dotn know if we have an existing bug to track that 13:13
sean-k-mooneyi think jgilaber or dviroel  orgianly say this when testing the host_maintiance stragey13:13
sean-k-mooneybut yes we shoudl fix that13:14
sean-k-mooneymaybe this is another instanfce of the same proble13:14
sean-k-mooneyi.e. maybe we fixed that for zone migration?13:14
sean-k-mooneyand it exits again for host maintaince13:15
amoralejhttps://bugs.launchpad.net/watcher/+bug/2122149 i will update the commit message13:15
sean-k-mooneythis is oen of the places where using typign woudl help13:15
sean-k-mooneyah yep that looks like the same underlying issue13:16
amoralejactually, my fix is duplicated https://review.opendev.org/c/openstack/watcher/+/95988913:16
amoralejI will abandon mine13:16
sean-k-mooneyalso yes :)13:16
sean-k-mooneyi knew that felt familar13:16
amoraleji missed it, sorry13:17
opendevreviewAlfredo Moralejo proposed openstack/watcher master: Skip migrate actions in pre_condition phase  https://review.opendev.org/c/openstack/watcher/+/96669913:18
dviroelsorry,  i was getting a coffee13:21
dviroelI totally forgot that fix 13:22
dviroelbut yeah, I was falling on the revert :) 13:22
chandankumarsean-k-mooney: sorry I got confused about two pocs? It was poc of playwright with unittest and pytest na?13:53
sean-k-mooneythre are two thigns to decsidn unittest vs pytest and selenium vs playwrigt13:54
chandankumaryes correct13:54
sean-k-mooneyif we assume we wont be using pytest then the poc woudl be betwen selenium vs playwright13:54
sean-k-mooneyspecifcly playwright-python13:54
sean-k-mooneyrahter then the javscript version 13:55
chandankumarSince PTI does not recommend pytest that's why we are going with unittest poc for selenium and playwright-python13:56
chandankumarOr we want to have poc with pytest with selenium and playwright also? 13:59
chandankumarthen decide13:59
sean-k-mooneyits not about recommends13:59
sean-k-mooneyit does not allow13:59
sean-k-mooneytechnially the horizon code shoudl not be in the repo13:59
sean-k-mooneythe only useage of pytest that is allowed in openstack si as an optioanl test runner. it is not allowed to be imported in any test code14:00
chandankumarok,14:02
chandankumarIt is much clearer now.14:03
chandankumarI will be going with unittest based poc on selenium and python-playwright and integration spec.14:03
opendevreviewDouglas Viroel proposed openstack/watcher master: Consolidate and improve Zuul CI job definitions  https://review.opendev.org/c/openstack/watcher/+/96694214:52
opendevreviewDouglas Viroel proposed openstack/watcher master: Consolidate and improve Zuul CI job definitions  https://review.opendev.org/c/openstack/watcher/+/96694216:04
opendevreviewDouglas Viroel proposed openstack/watcher master: Adds support for threading mode in applier  https://review.opendev.org/c/openstack/watcher/+/96622617:29
opendevreviewMerged openstack/watcher master: Fix source_node value in host maintenance strategy migrate action  https://review.opendev.org/c/openstack/watcher/+/95988917:47
opendevreviewDouglas Viroel proposed openstack/watcher master: Consolidate and improve Zuul CI job definitions  https://review.opendev.org/c/openstack/watcher/+/96694218:02
opendevreviewDouglas Viroel proposed openstack/watcher master: Consolidate and improve Zuul CI job definitions  https://review.opendev.org/c/openstack/watcher/+/96694218:07
-opendevstatus- NOTICE: The OpenDev team will be restarting Gerrit at approximately 2130 UTC in order to pick up the latest 3.10 bugfix release.20:32
*** haleyb is now known as haleyb|out22:56

Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!