Wednesday, 2021-04-14

*** mlavalle has quit IRC00:02
ianwi really don't want to roll back the sdk version on bridge, i feel like there might be other things we need that for00:23
ianwi'm thinking a manual run of the cloud launcher out of a venv with an old sdk might be the best way to get this done this once00:23
fungithat seems reasonable00:38
*** tkajinam has quit IRC00:55
*** tkajinam has joined #opendev00:56
ianw is alive, yay!01:05
openstackgerritIan Wienand proposed opendev/system-config master: Fix typo on OSU OSL password template
*** artom has quit IRC01:56
openstackgerritMerged opendev/system-config master: Add redirect to static
ianwwhen ^ deploys i'll switch the dns02:08
openstackgerritMerged opendev/system-config master: Fix typo on OSU OSL password template
openstackgerritIan Wienand proposed opendev/system-config master: Add OSU OSL to nodepool configuration
openstackgerritIan Wienand proposed openstack/project-config master: Add OSUOSL resources to nodepool
*** hamalq has quit IRC02:29
openstackgerritIan Wienand proposed opendev/system-config master: Add OSU OSL to nodepool configuration
ianwfungi: ^ thanks; i just renamed the var to "id" match what it is02:34
fungiahh, cool02:35
fungialso i left a comment on the other one02:36
ianwyeah, that seemed to come in via but didn't update the autogen scripts02:38
fungiaha, makes sense02:44
fungilooks like it was an experiment we never cleaned up02:44
openstackgerritMerged opendev/system-config master: Stop managing
*** ykarel has joined #opendev04:14
ianw#status log redirected to via static.o.o, server removed and dns entries cleaned up04:32
openstackstatusianw: finished logging04:32
ianwvale planet, and our RSS based future that wasn't to be04:34
openstackgerritMerged opendev/system-config master: Add OSU OSL to nodepool configuration
*** marios has joined #opendev04:59
*** whoami-rajat_ has joined #opendev05:20
*** ralonsoh has joined #opendev05:21
*** cenne-away is now known as cenne05:39
*** tinwood has quit IRC06:22
*** hemanth_n has joined #opendev06:24
*** ysandeep|holiday is now known as ysandeep06:24
*** tinwood has joined #opendev06:25
*** zbr has quit IRC06:30
yoctozeptofungi: \o/ for jeepyb, thanks!06:31
*** zbr has joined #opendev06:32
*** eolivare has joined #opendev06:32
*** knikolla has quit IRC06:34
*** ildikov has quit IRC06:35
*** sboyron has joined #opendev06:35
*** knikolla has joined #opendev06:36
*** ildikov has joined #opendev06:36
*** lpetrut has joined #opendev06:57
*** amoralej|off is now known as amoralej06:59
*** amoralej is now known as amoralej|off06:59
*** andrewbonney has joined #opendev07:14
*** marios has quit IRC07:21
*** DSpider has joined #opendev07:34
*** AJaeger has joined #opendev07:35
AJaegerHi, Is publishing to broken? The promote job in run 30 mins ago and;O=D shows that is not updated.07:36
AJaegerinfra-root ^07:38
* AJaeger tries to push the Wallaby release to docs.o.o early and run into this07:39
*** tosky has joined #opendev07:50
AJaegerAll good, docs are finally pushed - didn't remember that it takes that long07:51
*** rpittau|afk is now known as rpittau07:55
*** jpena|off is now known as jpena07:57
*** AJaeger has quit IRC08:07
hrwyay! moar aarch64 nodes!08:12
hrwand it started with a tweet:
*** slaweq_ has joined #opendev08:19
*** slaweq has quit IRC08:19
*** marios has joined #opendev08:21
*** slaweq_ is now known as slaweq08:51
*** klonn has joined #opendev09:16
*** amoralej|off is now known as amoralej09:20
*** dtantsur|afk is now known as dtantsur09:45
ttxhrw: nice!10:04
*** fresta has joined #opendev10:17
*** whoami-rajat_ is now known as whoami-rajat10:17
*** snapdeal has joined #opendev10:26
*** snapdeal has quit IRC10:26
*** klonn has quit IRC11:04
*** snapdeal has joined #opendev11:05
*** avass has quit IRC11:07
*** avass has joined #opendev11:08
*** eolivare_ has joined #opendev11:12
openstackgerritJeremy Stanley proposed opendev/system-config master: Revert "Revert "Temporarily serve static sites from AFS R+W vols""
fungiinfra-root: we should maybe consider merging that ^11:13
*** tkajinam has quit IRC11:14
fungiAJaeger's comment earlier got me looking, and indeed there was a lag in publishing static site content at the time he observed, owing to a nonzero content update in the tarballs volume11:14
*** eolivare has quit IRC11:15
fungithe openstack release activity today represents a decidedly nontrivial update to the tarballs volume by comparison11:15
fungiit's possible that could hold up publication of updates to all our static sites for hours11:15
*** avass has quit IRC11:31
*** jpena is now known as jpena|lunch11:32
*** avass has joined #opendev11:32
*** eolivare_ has quit IRC11:35
*** ykarel_ has joined #opendev11:37
yoctozeptohrw: well, that was quick!11:39
*** ykarel has quit IRC11:40
*** zoharm has joined #opendev11:45
*** ykarel_ is now known as ykarel11:47
*** hrw has quit IRC11:50
fungii'm setting 786205 to wip for now, apparently whatever caused the 45-minute vos release of the tarballs volume earlier (someone publishing a set of container images maybe?) was an anomaly. the full openstack release was barely a blip on the radar by comparison11:51
*** eolivare_ has joined #opendev12:16
*** amoralej is now known as amoralej|lunch12:21
*** marios is now known as marios|call12:23
*** hemanth_n has quit IRC12:24
*** AJaeger has joined #opendev12:28
*** jpena|lunch is now known as jpena12:29
*** snapdeal has quit IRC12:30
*** snapdeal has joined #opendev12:39
*** klonn has joined #opendev12:43
*** ysandeep is now known as ysandeep|afk12:46
openstackgerritMerged openstack/project-config master: Allow delete permissions in Gerrit ACLs
*** amoralej|lunch is now known as amoralej13:02
openstackgerritMerged openstack/project-config master: Add puppet-cinder-core and puppet-glance-core
*** AJaeger has quit IRC13:04
*** snapdeal has quit IRC13:13
*** tkajinam has joined #opendev13:14
*** ysandeep|afk is now known as ysandeep13:25
*** artom has joined #opendev13:36
*** marios|call is now known as marios13:36
*** hemanth_n has joined #opendev13:47
*** hemanth_n has quit IRC13:52
*** arxcruz has quit IRC13:57
*** arxcruz has joined #opendev13:59
clarkbfungi: heh reading scrollback I was all ready to merge a chnage then kep reading and was happy to see it was unnecesasry14:10
clarkbfungi: and ya I suspect that the actual tarballs may not be an issue, but some other artifact14:10
clarkbfungi: anything else to look at? based on scrollback here and in -infra no, but figured I would ask14:11
funginah, seems we're all good14:11
fungithe final change to mark wallaby as released will be approved in another ~15 minutes and then the press release goes out an hour after that14:12
fungier, half an hour after that14:12
fungirelease notes jobs are taking a long time because we serialize them to avoid rsync fighting over the same doctrees, but that's not a blocker14:13
*** slaweq has quit IRC14:16
*** slaweq has joined #opendev14:20
*** ykarel_ has joined #opendev14:20
*** ykarel has quit IRC14:22
clarkbfungi: if you have a moment can you review as part of the firehose cleanup?14:26
clarkbonce that lands and applies I can go and make sure the workers are restarted on new configs.14:26
fungisure, i have a sec14:26
fungithough need to take a break for a shower soon, been going for ~4 hours but wanted to make sure release stuff completed smoothly14:27
clarkbthank you for being around for that14:27
clarkbOnce I feel awake enough I'll try to do lp bug subscription investigation again14:28
clarkbto get the other input to firehose disabled14:28
*** ykarel_ is now known as ykarel14:29
fungigetting 782830 merged soon would also be cool so we can start experimenting with acl inheritance in openstack fairly soon, now that the release is basically done14:31
fungiand the open changes for topic:lp-integration before we restart gerrit at the end of the week14:32
clarkb++ I'll look at it14:32
clarkbI was going to followup on the zuul status plugin change this morning too to see where that was at14:32
clarkblooks like davido is in favor and +2'd it but waiting on luca's feedback14:33
fungiawesome, that's great news14:33
clarkbif we restarted before that change lands we would probably want to land ianw's existing change on our side then manually update all-projects14:33
fungithere's no urgency for restarting gerrit as far as lp-integration work is concerned14:34
fungiit's hand-patched into the currently running container (to make sure it actually works)14:34
clarkbcool, and ya we don't want to restart until friday anyway so have time to sort out a solution14:35
fungiif we down and up the container then having those changes merged first would be good, right14:35
clarkbfungi: re the meta-config project the jeepyb project creation process will create that repo with a master branch and a .gitreview file. The repo will inherit from all-projects by default. I think this means it may be in a weird spot if anyone wants to merge code to master. Maybe we half manually drop a readme in there explaining the repo's purpose and force merge it and then leave it be?14:36
clarkbjust calling that out to see if there is any concern with that extra branch laying around14:36
clarkbAnother thing to consider is replication. The repo will be replicated but not its config branch iirc14:36
fungii expected that, yeah, hopefully the project description is sufficient for people to understand there's little point in proposing changes directly for that repo14:37
clarkbI don't think any of these concerns are necessarily issues. Just things to be aware of as they straddle the behavior of existing config projects and existing openstack repos14:37
clarkbI have +2'd it and will let you +A if you decide the concerns are merely that14:38
mtreinishI've got a random gitea question, I was looking at one of my commits which was signed and it says my key wasn't found in the database:
mtreinishis there a way I can fix that?14:38
clarkbmtreinish: not currently as we don't expose user auth on gitea14:38
*** zoharm has quit IRC14:39
clarkbmtreinish: I believe the gitea mechanism for that is you create a user and then add your gpg pubkey14:39
fungiright, making that work would mean we'd need to have user accounts in gitea14:39
mtreinishah, ok14:39
mtreinishI was just wondering if there was something I forgot to do :)14:39
clarkbmtreinish: gitea operates as 8 separate installations currently so doing user management is largely a non starter14:39
clarkbif we manage to get it into a single gitea cluster then we could start to consider the implications of adding users for things like this14:40
openstackgerritMerged opendev/system-config master: Stop publishing subunit worker data to mqtt
*** lpetrut has quit IRC15:00
*** mlavalle has joined #opendev15:17
*** d34dh0r53 has quit IRC15:20
*** artom has quit IRC15:21
*** d34dh0r53 has joined #opendev15:22
*** ysandeep is now known as ysandeep|away15:25
clarkbthe openstack release has completed. One down and one to go before we start making bigger changes15:33
clarkblooking at firehose I'm not sure that anything but the exim and imap services are running15:52
clarkbwe really should be ok to clean this up in that case15:53
clarkbI've restarted the subunit workers for completeness though it appears that may not have been necessary15:55
clarkbfungi: the openstack bugs link was the ticket on the lp side too. I went to "edit bug mail" and unsubscribed from "everything openstack"15:59
clarkbI'm tailing the exim log now to see if there is any indication for other subscriptions16:01
clarkbfungi: do you think we should send annoucnement of turning off firehose? As far as I can tell the service has been effectively off already16:05
clarkbalso slightly confused around why certcheck didn't fail against firehose16:11
clarkbbut certcheck is accurately reporting survey right now so not too worried16:11
*** zoharm has joined #opendev16:16
mtreinishclarkb: we did announce it on the list when it was brought up:
clarkbmtreinish: ah in that case probably worth a small note to the list now that it is going away (even if it has been away)16:18
* clarkb starts writing one16:18
*** zoharm has quit IRC16:19
*** rpittau is now known as rpittau|afk16:21
clarkband sent16:26
*** hamalq has joined #opendev16:28
clarkbmtreinish: re haskell you should write more :) though anymore the only haskell I dabble with is my xmonad config16:30
clarkbI did fix a bug in it within the last year though so thats something16:31
*** marios is now known as marios|out16:32
*** marios|out has quit IRC16:37
fungisorry, back now. i disappeared for a post-release shower16:37
fungialso minor farming drama, most of our onions have decided to bolt in their first season, probably warm wet winter to blame16:37
fungibut hey, fresh onions in dinner tonight!16:38
mtreinishclarkb: heh, I know it has a lot of fans. I might try it for a small project at some point16:38
mtreinishmost of my new projects have been in rust, which has been a lot of fun16:38
fungiif my programs develop rust, i sandblast and powdercoat them16:40
mtreinishhah, well I've been oxidizing slow python libs. Different philosophy I guess :)16:42
*** ralonsoh has quit IRC16:45
clarkbthe last email received by the host was over half an hour ago. Though it did receive an email about 2 minutes after I unsubscribed (lag in the system maybe?)16:48
clarkb is probably ready for review now though as things appear stopped on the host16:48
fungilaunchpad api not synchronous, details at 2216:48
*** eolivare_ has quit IRC16:48
*** klonn has quit IRC16:56
clarkbfungi: als not sure if you want to go ahead and approve after our earlier conversation16:56
*** ricolin has joined #opendev16:57
clarkbfungi: re onions, if you have dry weather you should be able to dry them out and keep them edible in a cool dark spot for later too17:01
fungiclarkb: normally yes, but if they bolt early the tops are still green and very wet and that becomes much harder17:01
fungiideally you let them go brown and fall over, but they only do that in the second season17:02
fungiwe'll try to preserve them as we can, but probably going to make a lot of onion jam, caramelized onion, chop and freeze a bunch, et cetera17:02
*** clarkb has quit IRC17:08
*** clarkb has joined #opendev17:08
clarkbapparently I ping timed out17:09
clarkbdid I miss anything good?17:09
*** jpena is now known as jpena|off17:10
fungictcp pings17:10
fungiwhether those are good is for you to decide though17:10
*** avass has quit IRC17:16
*** amoralej is now known as amoralej|off17:24
*** klonn has joined #opendev17:30
*** artom has joined #opendev17:34
*** ykarel has quit IRC17:35
*** dtantsur is now known as dtantsur|afk17:44
*** andrewbonney has quit IRC17:59
*** zimmerry has joined #opendev18:00
clarkbfungi: any chance I can get a review on ? then maybe can land that this afternoon and delete the server18:10
fungisure thing18:18
openstackgerritMerged openstack/project-config master: Add an empty project for an OpenStack base ACL
*** avass has joined #opendev18:26
*** whoami-rajat has quit IRC18:38
openstackgerritMerged opendev/system-config master: Remove
*** fressi has quit IRC19:00
*** sboyron has quit IRC19:30
*** klonn has quit IRC20:16
ianwre gerrit plugin, it looks like everyone is happy with switching the default, so i think let's merge that20:38
clarkbianw: did luca end up responding? /me refreshes the chagne20:39
ianwoh, there's a question for luca20:39
ianwheh, yeah, i guess let's see if any more comments20:39
clarkbya I think davido wants luca to weigh in if possible20:40
clarkbI'm about to eat some lunch but will followup on firehose after since that change merged20:41
clarkbshould mostly be a matter of cleaning up the server at this point20:41
clarkbI have also been reminded that we should all register for the ptg if we plan on attending20:43
clarkbI had missed doing that (I think or maybe I'm double registered now, I'm registered at least once :) )20:43
openstackgerritMerged openstack/project-config master: Add OSUOSL resources to nodepool
*** zimmerry has quit IRC20:57
ianwdoes anyone else have an opinion on the openEuler mirroring changes at ?20:58
ianw"Currently openEuler repo site doesn't support anonymous rsync."20:59
ianwi mean it's not a blocker i guess, but it just feels ... odd20:59
*** zimmerry has joined #opendev21:01
*** zimmerry has quit IRC21:02
clarkbis that the official method for mirroring that distro? if so I guess we can live with it? but it does feel odd to do something special. In particualr I don't want to become a mirror people rely on outside of CI (whcih seems far more probably if we're mirroring it when others cant)21:05
fungiclarkb: also cleaning up dns records... maybe skim the list of records and remove any others we've obsoleted recently (e.g., nodepool launchers)21:07
clarkb786126 hasn't gotten to the run puppet step yet, I wanted that to finish before I remove the server. But once that is done I think I should be good to delete firehose01.openstack.org21:07
clarkbfungi: ++ can do once the server is removed21:07
clarkbddd5b4c1-37af-4973-b49c-b2023582b75f is the uuid of the firehose server if anyone else wants to double check that21:08
ianwfungi: i cleared out the nlXX entries the other day after i got myself confused why the host key had changed for them :)21:09
fungiianw: oh, awesome thanks! yeah i kept trying and failing to ssh into them too21:09
fungibut hadn't gotten around to logging in to delete them21:09
clarkbsorry about that (I should've done that when I replaced them)21:10
fungii'm sure there's still plenty in there i've forgotten to delete too21:10
*** zimmerry has joined #opendev21:11
ianwit's easier to see things in /var/lib/rax-dns-backup than their webui21:12
openstackgerritIan Wienand proposed opendev/system-config master: rax-dns-backup : fix cron output capture
ianwi don't know what i was thinking there ... must have copy pasted a command line21:16
fungiclarkb: ddd5b4c1-37af-4973-b49c-b2023582b75f looks right to me, yep21:17
clarkbthanks for checking21:19
clarkbafs is running then puppet then I can double check empty syslogs on firehose then do cleanups21:25
clarkband after these runs the runs to enable the osu arm resources should happen21:25
clarkban exciting afternoon after an exciting morning21:25
ianwyep i'll keep an eye on those to make sure images get in place, etc.21:28
clarkbremote puppet else did end up failing, but there are no new syslog entries for ansible or puppet that correspond to that run. I think that means I'm clear to delete the server. Doing that now21:41
* clarkb double checks the puppet run log first though21:42
ianwyeah i think that has been mostly working but also failing for a while21:42
clarkbthe failures were ask, logstash.o.o, and openstackid-dev21:43
clarkbnone of that should be related to firehose so I will proceed21:43
clarkb#status log Deleted (ddd5b4c1-37af-4973-b49c-b2023582b75f) as its deployment was unmaintained and it was never used in production21:44
openstackstatusclarkb: finished logging21:44
clarkband dns has been cleaned up too. Going to skim the list as fungi suggested and see if any other cleanups stnad out21:48
clarkbthe only thing that jumped out is there is a backup server in the openstack domain. I think we may have moved all of those to now? I don't want to break backups so I will leave it alone for now until we determine it is safe21:51
*** jaicaa has quit IRC21:52
fungiianw: oh, i meant to follow up, seems like the openstack release was barely a blip for syncing the tarballs volume... i have a feeling those longer sync times we're seeing are projects uploading large container images and the like21:52
clarkbnow to rerun the gerrit externalid conflict audit script to prepare for tomorrows cleanups21:52
*** jaicaa has joined #opendev21:53
ianwfungi: yeah, that sounds likely as you wouldn't think a few tarballs would be that big.  it's a bit of a conundrum what to do ...21:53
ianwthe best thing would be if it could replicate at more than 1.3mb/s21:54
fungiwell, in the case of the openstack wallaby coordinated release it wasn't "a few" tarballs... more like 60-7021:54
fungiplus as many wheels21:55
fungiplus signatures21:55
fungibut yeah, all of that together is probably smaller than a kolla layer21:55
clarkbmaybe we should move sub artifacts onto a different volume?21:59
clarkbbasically have tarballs be top layer release artifacts, then secondary things like container images go elsewhere?22:00
clarkb(also I'm not quite sure why we put containers on tarballs at all since docker hub and quay etc seem to work ok?)22:00
fungisome projects were using that as "scratch space" for emulating the pattern which eventually became our container promotion workflow22:05
clarkbmight be a good time to push towards the intermediate registry22:06
openstackgerritIan Wienand proposed openstack/project-config master: OSU OSL : add diskimages to nb03
clarkbianw: I think that chagne is not quite right22:09
clarkbneed to change the cloud and provider name on one of the two blocks22:09
mordredclarkb: I think it's definitely time to push towards the intermediate registry22:09
ianwclarkb: ahh yes, copy-paste issue22:09
openstackgerritIan Wienand proposed openstack/project-config master: OSU OSL : add diskimages to nb03
ianwi'm not actually sure if we need those boot flags for this cloud22:10
ianwor indeed, at all any more; i think nova got smarter about starting arm64 nodes over time22:10
ianwhrw would know for sure :)22:10
clarkbyes I recall nova got smarter. Is the issue with sdk doing security groups that this cloud is older though?22:11
clarkbalso I don't think it hurts to have the extra flags22:11
ianwyeah i didn't actually trace back what release the sdk issues would have come in22:13
*** DSpider has quit IRC22:17
ianwactually the nova changes only got added for wallaby, so "older" is maybe too strong a term :)22:17
clarkboh wow22:18
fungithat's been around for, like, hours man22:24
fungiget with the times22:24
openstackgerritMerged openstack/project-config master: OSU OSL : add diskimages to nb03
clarkbthe audit looks good and I've got my list of accounts to cleanup external ids for tomorrow all ready to go22:35
*** tkajinam has quit IRC22:43
ianw# grep 'kazoo.client: Connection dropped: socket connection error: Connection reset by peer' nodepool-builder.log | wc -l22:43
ianwnb03 seems to drop out of zookeeper about every 2 minutes22:43
clarkbianw: I think the zk config is ipv4 specific (we use ips not names)22:45
corvuswhere is nb03?22:45
ianwcorvus: in linaro-us22:45
clarkband we recently had ipv4 issues with the mirror there right?22:45
clarkb(just wondering if that could be related)22:45
ianwi don't think we're actively having issues, but it certainly doesn't seem to keep a reliable connection to ZK at least22:46
corvuspotential mitigation: a dib-compatible shell script that does the work over ssh and copies results back.  make nb03 a vm in rax and ssh into linaro to do the builds.22:48
clarkbmaybe check if the ipv4 connectivity out to something else is relatively stable?22:48
clarkbcorvus' idea is neat but would probably suffer issues too (though I guess ssh is very resilient to issues)22:49
corvus(i mean, obvs "make network good" is better :)22:49
corvusyep; put my idea near the bottom of the list22:49
corvusbut ssh can have pretty long timeouts22:49
clarkbour zk timeout is 2 minutes iirc22:49
clarkbwhich isn't very short either22:50
ianwif we find OSU is more stable and perhaps faster, we could move the builder there22:50
ianwit has 3ms pings and no packet loss on ping to zk0122:51
clarkbthat makes me wonder if it isn't able to service the zk pings in a timely manner (its all running on one cpu and if busy may not be able to do zk and normal duties?22:52
*** tkajinam has joined #opendev22:52
clarkb(that is essentially what happens with the scheduler when it ooms and swapps22:53
ianwit does say "Connection reset by peer"22:54
ianwdoes that perhaps suggest ZK decided there was nothing on the other end and closed the connection?22:54
clarkbianw: ya that typically indicates that zk hit its internal ping timeout to the node and disconnects it from the zk side22:54
corvusor could be bad firewall/nat issuing a gratuitous RST22:55
clarkbif you look at syslog on the zk leader (zk02 last I checked) you will see it make those decisions22:55
ianwopenstack.exceptions.SDKException: Image creation failed: Unable to establish connection to ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')22:55
ianwthat is unfun22:55
ianwand indeed points to local issues even more22:56
*** hamalq has quit IRC22:57
*** hamalq has joined #opendev22:57
openstackgerritMerged opendev/system-config master: rax-dns-backup : fix cron output capture
ianwthey all just failed to upload again23:00
ianw--upload-workers 8 porbably too much for this host23:03
ianwprobably even23:03
clarkboh that could be23:08
clarkbmaybe turn it down to 2 and see if that helps?23:08
openstackgerritIan Wienand proposed opendev/system-config master: nodepool-builder: provide for configuration of upload works, reduce nb03
openstackgerritIan Wienand proposed opendev/system-config master: nodepool-builder: configure upload workers, reduce nb03
ianwImage build debian-stretch-arm64-0000100981 (external_id 47944e6a-c9a3-4c58-bc74-055b90eab1ec) in osuosl is ready23:13
ianwsome are getting through23:14
ianwaliasByNode(stats.timers.nodepool.task.$region.ComputePostServers.mean) ... these stats no longer seem to be there23:26
ianw in theory the client should be setup to send out nodepool.task results23:28
*** gry has joined #opendev23:29
*** tosky has quit IRC23:29
ianw0x0010:  68ef f0a7 b535 1fbd 003e 5343 6f70 656e  h....5...>SCopen23:38
ianw0x0020:  7374 6163 6b2e 6170 692e 636f 6d70 7574  stack.api.comput23:38
ianw0x0030:  652e 4745 542e 7365 7276 6572 732e 6465  e.GET.servers.de23:38
ianw0x0040:  7461 696c 3a38 3733 2e30 3030 3030 307c  tail:873.000000|23:38
ianwweird, it looks like they're coming out with a generic "openstack.api.compute" prefix23:38
ianwi think it's not impossible openstacksdk has broken setting the prefix23:40
ianwrelocating for a bit ... bib23:41
*** irclogbot_2 has quit IRC23:50
*** irclogbot_1 has joined #opendev23:55

Generated by 2.17.2 by Marius Gedminas - find it at!