| f0o | Good morning; Edited my /etc/openstack/upper_constraint/* with the new ovs version that was updated in 2023.1 and everything seems to be working fine so far (os-neutron-install passed, now running os-nova-install so fingers crossed) | 07:23 |
|---|---|---|
| f0o | did a repo-install prior ofc - forgot to mention | 07:23 |
| f0o | cycled the ovn-controllers and the reconnect-loop of the metadata-agent is now working correctly after bumping ovs to 2.17.2 | 08:00 |
| noonedeadpunk | o/ | 08:19 |
| noonedeadpunk | ah, ok, so you've faced the same issue.... | 08:20 |
| noonedeadpunk | it's a shame that folks rejected patches to update ovs version in u-c we've proposed | 08:21 |
| f0o | why did they reject it? it seems that all OS vendors packaged it as high sev bug | 08:23 |
| noonedeadpunk | f0o: oh, wait, it was merged o_O https://review.opendev.org/c/openstack/requirements/+/892892 | 08:25 |
| f0o | yeah to 2023.1 not .2 | 08:25 |
| noonedeadpunk | just like - after half a year after being proposed | 08:25 |
| f0o | that patch should be backported to .2 | 08:26 |
| noonedeadpunk | so then it should be in some of latest releases I assume | 08:26 |
| f0o | it is not, 2023.2 is still using the bugged ovs version | 08:26 |
| noonedeadpunk | that;s stupid | 08:26 |
| f0o | master is having a super new one so the Next Release will likely fix it but 2023.2 is broken although 2023.1 is fixed | 08:26 |
| noonedeadpunk | https://review.opendev.org/c/openstack/requirements/+/892891 | 08:27 |
| noonedeadpunk | so this is why it's not in 2023.2.... | 08:27 |
| noonedeadpunk | /o\ | 08:27 |
| f0o | I mean... I can understand that a major bump is rejected but the minor bump from 2.17..1->.2 should still be backported | 08:28 |
| noonedeadpunk | well, dunno if "backport" is applicable to the newer branches, as that's basically a regression | 08:30 |
| f0o | correct just lack of better wording | 08:31 |
| noonedeadpunk | fair enough | 08:31 |
| f0o | reversed backport? | 08:31 |
| f0o | :D | 08:31 |
| noonedeadpunk | forwardport | 08:31 |
| noonedeadpunk | dunno | 08:31 |
| f0o | forwardport. let's coin it | 08:31 |
| f0o | IETF forwardport - a backport that was made to an ancient branch but forgotten to apply to the newer ports thus breaking lineage | 08:32 |
| f0o | idk | 08:32 |
| noonedeadpunk | :D | 08:32 |
| noonedeadpunk | yeah, sounds sweet | 08:32 |
| f0o | I need more coffee - breaking my head over how to get FIPs to both gateway nodes which are our edge/TO-Routers so both can resolve it without flushing to the wire | 08:33 |
| f0o | I mean worst case they flush it to the wire and done, they got beefy MLAG interconnects for those sort of spillages | 08:34 |
| noonedeadpunk | but, um, gateway nodes are the ones who held routers? | 08:34 |
| noonedeadpunk | or you mean core routers | 08:35 |
| f0o | but both are connected to the OVN/OVS so they have end-to-end connectivity to everything so flushing to wire is just one hop | 08:35 |
| f0o | yeah OVN Gateway Nodes hold routers and make N/S traffic happen | 08:35 |
| f0o | not doing Distributed FIPs because it seems a bit messy to drag the public VLAN to all compute nodes and have it resolve in arp for all FIPs (very linux bridge) | 08:36 |
| f0o | so my idea was to consolidate to the top-of-rack switches which BGP the FIP range to the rest of the network and then just go BGP<>OVS<>Compute | 08:36 |
| f0o | but OVS only applies the flow to one of the two and the kernel has no way of knowing which so routing only works on one of the nodes despite both annoucning the prefix | 08:37 |
| f0o | either need to get the OVS flow on both routers and do VRRP on the br-ext subinterface for L3 to work internally - or teach the kernel's routing table that FIPs are onlink on an OVS | 08:38 |
| f0o | Or bridge the OVS to an interconnect and let standard ARP resolve the whole thing adding in worst case one additional wire | 08:38 |
| f0o | _OR_ my decaffeinated brain cannot see the forest from all the trees and the solution is super obvious somewhere | 08:39 |
| jrosser | good morning | 08:56 |
| f0o | Morning | 08:57 |
| *** Guest6972 is now known as starkis | 09:17 | |
| nixbuilder | I read fOo posting about Full Deployment Example for OVN and the response. I am working on the same thing (I think) and am having similar issues. I was wondering if you had made any progress. Anyway I would like to help with documentation and my configuration for an example when I get this figured out and deployed. | 10:58 |
| jrosser | part of the complexity is that there are many possible scenarios with OVN | 11:02 |
| jrosser | so if we are able to describe particular scenarios well, that would be excellent | 11:02 |
| nixbuilder | jrosser: OK... well I will work on it. | 11:11 |
| noonedeadpunk | I personally got OVN working quite nicely in our full-scale sandbox, except ovn-bgp agent part which is slightly /o\ | 13:07 |
| opendevreview | Merged openstack/openstack-ansible stable/2023.2: Bump SHAs for 2023.2 https://review.opendev.org/c/openstack/openstack-ansible/+/911943 | 13:39 |
| opendevreview | Aleksandr Chudinov proposed openstack/openstack-ansible-os_nova master: fix apparmor profile for non-standard nova home https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/912583 | 13:52 |
| f0o | I'm looking at /var/log/journal and see some very large journals from the LXC containers (from the crashloops over weeks) - is there a quick way to tell ansible to run something by the lines of `journalctl --vacum-size=1G`? Looking at https://docs.openstack.org/openstack-ansible-lxc_container_create/latest/ it's not super obvious if I can use lxc_container_extra_commands | 14:19 |
| ThiagoCMC | jrosser, noonedeadpunk, check this out: https://github.com/ceph/ceph-ansible/issues/7496 | 14:23 |
| jrosser | f0o: you can use an ansible ad-hoc command for that i think? | 14:27 |
| jrosser | something like this i think https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks.html#running-the-shell-module | 14:28 |
| noonedeadpunk | f0o: and then we also have `openstack_hosts_journald_config` variable which you can use to configure journald on lxc/host | 14:45 |
| noonedeadpunk | ie we do smth like this for CI machines: https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L39-L45 | 14:46 |
| jrosser | ThiagoCMC: /o\ https://github.com/ceph/ceph-ansible/commit/9c467e41b39af63dec9a0b237b550edb9c57da85 | 14:46 |
| noonedeadpunk | wow | 14:46 |
| jrosser | i really wonder what is going on there | 14:49 |
| ThiagoCMC | Let's make some noise in there! | 14:51 |
| NeilHanlon | fyi https://www.openwall.com/lists/oss-security/2024/03/12/5 -- CVE in OVN | 14:53 |
| ThiagoCMC | I can deploy Ceph Reef with `stable-7.0`, but only works if using `ceph-ansible` alone, not via OSA. I tried to force OSA with `ceph_stable_release: reef` but it creates a APT pinning which locks it down to Ceph Quincy anyway. | 14:53 |
| jrosser | ThiagoCMC: there is an argument to say we should adjust all that to deploy Reef for the upcoming OSA release | 14:53 |
| jrosser | i asked in the ceph slack where the roadmap for the changes to ceph-ansible is | 14:54 |
| ThiagoCMC | Yeah, I need it. But the thing is that `stable-8.0` alone is failing. This should be fixed first. Then integrate the new version with OSA. | 14:55 |
| noonedeadpunk | `ceph_stable_release: reef`? | 14:55 |
| jrosser | ThiagoCMC: why would we try to fix stable-8.0 first? they broke so much already | 14:56 |
| ThiagoCMC | noonedeadpunk, yes, if we use `stable-7.0` but "force" ``ceph_stable_release: reef`, it deploys Ceph Reef on Ubuntu 22.04 with Bobcat UCA. | 14:56 |
| ThiagoCMC | jrosser, well, I'm asking `ceph-ansible` to fix it (and also to bring back what OSA needs). | 14:58 |
| noonedeadpunk | jrosser: well, looking in the patch, probably nothing too bad happened | 14:59 |
| noonedeadpunk | probably, from their prespective/intended focus containing openstack bits explicitly is an overkill | 14:59 |
| jrosser | didnt they also take out the config overrides stuff earlier too? | 14:59 |
| noonedeadpunk | so if they wanna reduce codebase and have just good working core - it's reasonable thing to do | 14:59 |
| noonedeadpunk | they did, yes | 14:59 |
| ThiagoCMC | Yes, they did removed the overrides. I mentioned this in the Github issue. | 15:00 |
| noonedeadpunk | but again - I think config today is not in ceph.conf mainly anyway? | 15:00 |
| noonedeadpunk | it's like injected in daemon or smth like that - ending up in raft? | 15:00 |
| jrosser | it doesnt have to be in ceph.conf, thats true | 15:00 |
| jrosser | certainly those openstack specific tasks could be put into openstack-ansible-plugins collection | 15:01 |
| ThiagoCMC | I'm okay with that =P | 15:01 |
| noonedeadpunk | what they really dropped right now - is explicit variables for openstack | 15:01 |
| noonedeadpunk | so we just need to define more things now explicitly leveraging "native" vars | 15:02 |
| noonedeadpunk | jrosser: I don't think we need even that? | 15:02 |
| jrosser | perhaps not? i didnt look too closely | 15:02 |
| noonedeadpunk | these tasks were just creating pools/generating cephx. I think they do that anyway somewhere else? | 15:03 |
| jrosser | here ? https://github.com/ceph/ceph-ansible/blob/main/roles/ceph-client/tasks/create_users_keys.yml | 15:06 |
| noonedeadpunk | yeah, for instance | 15:08 |
| noonedeadpunk | can't find where to define pools to create.... | 15:08 |
| noonedeadpunk | but potentially, they would be fine with more "generic" naming for that feature | 15:08 |
| noonedeadpunk | oh, damn | 15:14 |
| NeilHanlon | oh.. is it meeting time? noonedeadpunk | 15:14 |
| NeilHanlon | fucking timezones | 15:14 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:14 |
| opendevmeet | Meeting started Tue Mar 12 15:14:42 2024 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:14 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:14 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:14 |
| noonedeadpunk | #topic rollcall | 15:14 |
| NeilHanlon | o/ | 15:14 |
| noonedeadpunk | o/ | 15:14 |
| noonedeadpunk | sorry folks for being late | 15:14 |
| jrosser | o/ hello | 15:15 |
| NeilHanlon | i got a notification 5 mins before and then got distracted, so... | 15:15 |
| noonedeadpunk | yeah, I did pretty much the same | 15:15 |
| noonedeadpunk | #topic office hours | 15:17 |
| noonedeadpunk | Just wanted to ask, if things got any more clear regarding occupancy during the PTG week | 15:17 |
| noonedeadpunk | (which timing is slightly unfortunate this time around) | 15:17 |
| NeilHanlon | i think my availability is largely the same, that is, pretty open | 15:18 |
| noonedeadpunk | Ok, seems that our highlights were published for the upcoming release | 15:20 |
| noonedeadpunk | #link https://releases.openstack.org/caracal/highlights.html#openstackansible | 15:20 |
| noonedeadpunk | I guess out of the list, we still have octavia ovn driver not merged | 15:33 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 | 15:33 |
| noonedeadpunk | as well as ovn-bgp-agent | 15:34 |
| noonedeadpunk | I'm still playing with this though, so it's in WIP | 15:34 |
| noonedeadpunk | but should be really ready in a week or so | 15:34 |
| noonedeadpunk | I've also proposed new minor releases today | 15:34 |
| noonedeadpunk | #link https://review.opendev.org/q/topic:%22release_osa%22+status:open | 15:35 |
| jrosser | would be nice to have a zuul job for he ovn octavia provider | 15:35 |
| noonedeadpunk | yeah, I can recall promising that | 15:35 |
| noonedeadpunk | it also feels, that on top of these highlights, we may finally be able to have skyline support | 15:36 |
| jrosser | should we make it so horizon and skyline can co-exist? | 15:37 |
| jrosser | i.e you might want to test skyline alongside | 15:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_skyline master: Re-add Zuul testing to the project https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912333 | 15:38 |
| jrosser | thats perhaps just a case of making sure that the port numbers are programmable and don't assume 443 for either | 15:38 |
| noonedeadpunk | yeah | 15:39 |
| noonedeadpunk | I was actually thinking about some logic | 15:39 |
| noonedeadpunk | Like what we agreed on PTG was - whoever runs last - wins | 15:39 |
| noonedeadpunk | but I was going to look into more complex logic frankly speaking | 15:39 |
| noonedeadpunk | also I guess that having nginx is probably least concern, especially since we'd need to serve staticfiles somehow anyway... | 15:40 |
| noonedeadpunk | so these weekends I had plans to look into haproxy maps | 15:40 |
| noonedeadpunk | and make skyline to co-exist with horizon | 15:40 |
| jrosser | apache would be a good move for metal deploys | 15:41 |
| jrosser | so that it drops in as another thing alongside keystone | 15:41 |
| noonedeadpunk | I guess idea was to make /horizon and /skyline when both exist with redirect of / to /horizon by default | 15:41 |
| noonedeadpunk | actually. we have 1 thing left on nginx - repo server | 15:41 |
| noonedeadpunk | so metal deploys still have it | 15:41 |
| jrosser | we do, that should ideally be tidied up too | 15:41 |
| noonedeadpunk | yeah | 15:42 |
| noonedeadpunk | agree | 15:42 |
| noonedeadpunk | ok | 15:42 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_skyline master: Add EL distro support https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912370 | 15:42 |
| noonedeadpunk | but otherwise at least role/playbook do pass here: https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912370 | 15:42 |
| noonedeadpunk | so thing is not super noop at least | 15:43 |
| jrosser | its very nice to see some work on that | 15:43 |
| noonedeadpunk | have no idea if that would be used by me, but as I got it working at least - feels that we should finilize it | 15:44 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_skyline master: Re-add Zuul testing to the project https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912333 | 15:52 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_skyline master: Add EL distro support https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/912370 | 15:52 |
| noonedeadpunk | I also still not sure how I feel about skyline UX at all | 15:53 |
| noonedeadpunk | Looks very modern... But slightly weird | 15:53 |
| noonedeadpunk | It took me a while to understand where admin UI is at all | 15:56 |
| noonedeadpunk | anyway | 15:56 |
| noonedeadpunk | #endmeeting | 15:57 |
| opendevmeet | Meeting ended Tue Mar 12 15:57:13 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:57 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-03-12-15.14.html | 15:57 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-03-12-15.14.txt | 15:57 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-03-12-15.14.log.html | 15:57 |
| ThiagoCMC | jrosser, not sure if you know this but, I'm deploying OSA_AIO 2023.2 branch with SCENARIO='aio_lxc_ceph', plus the following variables set: 'install_method: distro', 'rabbitmq_install_method: distro', 'ceph_origin: distro', 'ceph_stable_release: reef', and 'ceph_pkg_source: distro'. But it still deploys Ceph Quincy! I noticed if I remove the `/etc/apt/preferences.d/openstack_hosts_pin.pref` file, then `apt upgrade` will try to upgrade | 16:56 |
| ThiagoCMC | Ceph to Reef (inside of Ceph Mon container, for example). What would be the proper way of avoiding or setting `openstack_hosts_pin.pref`? Also, it creates another file called `ceph_community_pin.pref` which is not needed when `distro` is used. | 16:56 |
| noonedeadpunk | ThiagoCMC: you probably need to add/override `ceph_apt_pinned_packages` as well | 17:03 |
| noonedeadpunk | but that should be adding different file - `ceph_client_pin.pref` | 17:04 |
| ThiagoCMC | Cool, I see that `ceph_apt_pinned_packages` belongs to the `ceph_client` role, not `ceph-ansible`. But I'll take a look into it too! | 17:10 |
| jrosser | noonedeadpunk: are you in the ceph slack? theres a ceph-ansible channel just been made | 17:14 |
| noonedeadpunk | I'm not | 17:16 |
| * noonedeadpunk not having slack installed even | 17:16 | |
| jrosser | think i managed to get slightly frustrated talking to them :/ | 17:17 |
| noonedeadpunk | aren't they in irc anymore.... | 17:17 |
| noonedeadpunk | how to get to slack.... | 17:17 |
| noonedeadpunk | jsut to double-check - slack is not mirrored to the irc? | 17:19 |
| jrosser | for the #ceph channel yes | 17:19 |
| jrosser | though i guess there is a bot of some sort does that | 17:20 |
| nixbuilder | noonedeadpunk: What is neutron_is_first_play_host for? I see it in a patch that I think your wrote??? | 17:29 |
| noonedeadpunk | it's to run things only once | 17:30 |
| nixbuilder | My current neutron install is failing on that variable. | 17:30 |
| noonedeadpunk | like db creation or rabbitmq creation | 17:30 |
| noonedeadpunk | it should not | 17:30 |
| noonedeadpunk | likely you're missing smth from inventory | 17:30 |
| nixbuilder | https://paste.openstack.org/show/b8vLMtRLTLuFYVrEoIqj/ | 17:30 |
| nixbuilder | It's hard for me to filter through all that mess of output... trying though. | 17:31 |
| noonedeadpunk | ugh | 17:33 |
| noonedeadpunk | yeah, not easy to read for sure. | 17:34 |
| noonedeadpunk | `'dict object' has no attribute 'vlan,local,geneve'` | 17:34 |
| noonedeadpunk | but that is really smth off with defenition of some var, as this should not be an attribute, but rather a value | 17:37 |
| ThiagoCMC | Another quick question: is OSA_AIO 'aio_lxc' 'master' branch good to start playing with Ubuntu 24.04 with OpenStack Caracal? | 17:43 |
| noonedeadpunk | no | 17:48 |
| noonedeadpunk | or well | 17:48 |
| jrosser | it's the best place to start development work to make that supported :) | 17:48 |
| noonedeadpunk | I mean, we never started looking into that | 17:48 |
| noonedeadpunk | But it's good place to start playing | 17:49 |
| noonedeadpunk | yeah, exactly | 17:49 |
| ThiagoCMC | Cool! I'll give it a try. | 17:49 |
| noonedeadpunk | nixbuilder: so I would check places in your configs where you mention "vlan,local,geneve" and verify punctuation/mapping/values around that | 17:49 |
| jrosser | ThiagoCMC: Caracal isnt released yet? | 17:50 |
| noonedeadpunk | neither is ubuntu 24.04 ;) | 17:51 |
| noonedeadpunk | But I guess timing to start testing them is not bad | 17:51 |
| nixbuilder | nonedeadpunk: Well I removed the 'local' so now I have just "vlan,geneve" but still get the almost the same error... minus the 'local'. Still checking though. | 17:51 |
| noonedeadpunk | My main concern if py3.12 will work with openstack | 17:51 |
| noonedeadpunk | nixbuilder: I really think it's either wrong variable or smth off with delimiters | 17:52 |
| noonedeadpunk | as it's should be value, but role treats it as a key | 17:52 |
| ThiagoCMC | jrosser, damn, sorry... My bad. I meant that I'd like to try OSA_AIO with Ubuntu 24.04 (and whatever OpenStack version will be on it by default, I assumed it would be Caracal). | 17:53 |
| ThiagoCMC | Anyway, are you folks trying anything on Ubuntu 24.04 already, or not yet? | 17:53 |
| noonedeadpunk | not yet | 17:53 |
| noonedeadpunk | at least not me | 17:54 |
| jrosser | I might take a look as soon as there is a C release, which would be next month | 17:54 |
| jrosser | and i think a bunch of work has been done on python 3.12 for debian | 17:54 |
| jrosser | so thats fingers-crossed, but at the same time it could all just be a big bonfire | 17:55 |
| ThiagoCMC | Okdok, I plan to build a Private Cloud for a customer using Ubuntu 24.04 and Ceph Reef (without cephadm Docker crap). Let's hope for the best! :-D | 18:02 |
| noonedeadpunk | sounds like a plan | 18:04 |
| ThiagoCMC | ^_^ | 18:04 |
| noonedeadpunk | I also having a pet project around, and was thinking about same | 18:04 |
| noonedeadpunk | Just not going as extereme with full-distro source | 18:04 |
| ThiagoCMC | I really like Debian packages! I even run `apt upgrade` inside of the containers, no problem at all. | 18:05 |
| jrosser | we need to make some adjustments anyway in OSA | 18:05 |
| jrosser | ideally we would be deploying reef in the next release | 18:05 |
| ThiagoCMC | I'm ultra interested on that! | 18:06 |
| ThiagoCMC | Could OSA also provide a way of installing `ansible-core` Ubuntu package (`install_source: distro`), instead of `pip`? Including all of its "requirements.txt" (assuming they are all available as Ubuntu packages, leaving to `pip install` only what doesn't exist in Ubuntu itself)? | 18:13 |
| ThiagoCMC | It seems that `scripts/bootstrap-ansible.sh` could just `apt install ansible-core` and be done with it. =P | 18:15 |
| jrosser | but how would you get the version we want? | 18:15 |
| jrosser | and on modern python you cannot co-exist distro python packages with pip ones | 18:16 |
| ThiagoCMC | Well, Ceph Ansible can easily leverage Ansible's Ubuntu PPA, works great. | 18:17 |
| ThiagoCMC | I didn't know about the co-exist issue with Python packages and pip ones. I personally almost never use `pip` in production (for non-OSA stuff). If the package doesn't exist on Debian (Sid) or Ubuntu rolling, I do my best to use the `stdeb` utility and make a Debian package out of a source Python project. | 18:19 |
| spatel | anyone have seen this error before - ForbiddenException: 403: Client Error for url: https://openstack01.example.com:9292/v2/images, You are not authorized to complete get_images action.<br /><br /> | 18:19 |
| spatel | when I run openstack image list | 18:19 |
| noonedeadpunk | actually, we had related report for horizon | 18:22 |
| noonedeadpunk | and I think i saw same happening there randomly in logs | 18:22 |
| noonedeadpunk | ThiagoCMC: but does ubuntu ppa has all ballpark of ansible versions? at least by major releases? | 18:23 |
| noonedeadpunk | let me check... | 18:23 |
| noonedeadpunk | ThiagoCMC: so Jammy has only 2.16? https://launchpad.net/~ansible/+archive/ubuntu/ansible?field.series_filter=jammy | 18:23 |
| noonedeadpunk | default repo is 2.14 | 18:24 |
| jrosser | i just looked and it did not seem to have them all https://launchpad.net/~ansible/+archive/ubuntu/ansible?field.series_filter=jammy | 18:24 |
| noonedeadpunk | How we are to install 2.15 we're currently using for 2023.2 and 2024.1? | 18:24 |
| jrosser | so this is basically not usable | 18:24 |
| ThiagoCMC | There's the `ppa:ansible/ansible-2.10` - But I'm unsure if they are maintaining the versioned PPA repositories. | 18:32 |
| ThiagoCMC | It's okay... I was just curious (I love Debian packages lol) | 18:32 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!