Friday, 2020-05-29

*** cshen has joined #openstack-ansible00:04
*** cshen has quit IRC00:09
spatelfolks, i am trying to install OSA on centOS801:24
spatelWhat python version OSA going to continue with CentOS8?01:25
spatelc8 has default python301:25
*** cshen has joined #openstack-ansible02:05
*** cshen has quit IRC02:10
prometheanfireanyone able to help me debug this? for some reason lxc isn't creating containers02:11
prometheanfirefstype, changing that to xfs (using lvm backing store) seems to help02:20
prometheanfiremight have to do with chaing my default fs everywhere to xfs, mkfs.ext4 was still available02:27
*** threestrands_ has joined #openstack-ansible02:47
*** threestrands_ has quit IRC02:48
*** threestrands_ has joined #openstack-ansible02:49
*** threestrands_ has quit IRC02:50
*** threestrands has quit IRC02:50
*** threestrands_ has joined #openstack-ansible02:50
*** threestrands_ has quit IRC02:51
spatelprometheanfire: are you running ubuntu or centOS8?02:52
*** threestrands_ has joined #openstack-ansible02:52
prometheanfiredebian 1002:53
*** threestrands_ has quit IRC02:53
*** threestrands_ has joined #openstack-ansible02:53
*** threestrands_ has quit IRC02:54
spatelno worry i am dealing with CentOS 8.x02:55
*** threestrands_ has joined #openstack-ansible02:55
*** threestrands_ has quit IRC02:56
*** threestrands_ has joined #openstack-ansible02:56
*** threestrands_ has quit IRC02:57
*** threestrands_ has joined #openstack-ansible02:58
*** threestrands_ has quit IRC02:59
*** threestrands_ has joined #openstack-ansible02:59
*** threestrands_ has quit IRC03:00
prometheanfirespatel: not sure what python version os uses, what version of osa?03:01
*** threestrands_ has joined #openstack-ansible03:01
spateli am using latest version master branch03:01
prometheanfireif master it SHOULD be python3.6 iirc as that's what ussuri is on in general for openstack03:01
spateli think its train03:01
prometheanfiretrain might be py2 still, not sure03:01
spatelI believe 20.0.0 branch is train03:02
prometheanfireussuri isn't released yet, so, probably03:02
*** threestrands_ has quit IRC03:02
*** threestrands_ has joined #openstack-ansible03:02
*** threestrands_ has quit IRC03:03
*** threestrands_ has joined #openstack-ansible03:04
*** threestrands_ has quit IRC03:05
*** threestrands_ has joined #openstack-ansible03:05
*** threestrands_ has quit IRC03:06
*** threestrands_ has joined #openstack-ansible03:07
*** threestrands_ has quit IRC03:08
*** threestrands_ has joined #openstack-ansible03:08
spatelhttps://www.google.com/search?safe=active&source=hp&ei=7HzQXrLbCvivytMPyISE6Aw&q=openstack+ussuri+release+date&oq=ussuri+release+da&gs_lcp=CgZwc3ktYWIQAxgAMgYIABAWEB46BQgAEIMBOgIIADoECAAQClCuA1jSM2CyOmgBcAB4AIAB9wKIAbkLkgEIMTEuMi4wLjGYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab03:09
*** threestrands_ has quit IRC03:09
spatelussuri release date 2020-05-1303:10
*** threestrands_ has joined #openstack-ansible03:10
*** threestrands_ has quit IRC03:11
*** threestrands_ has joined #openstack-ansible03:11
*** threestrands_ has quit IRC03:12
*** threestrands_ has joined #openstack-ansible03:13
*** threestrands_ has quit IRC03:14
*** threestrands_ has joined #openstack-ansible03:14
*** threestrands_ has quit IRC03:15
*** threestrands_ has joined #openstack-ansible03:16
*** threestrands_ has quit IRC03:17
*** threestrands_ has joined #openstack-ansible03:17
*** threestrands_ has quit IRC03:18
*** threestrands_ has joined #openstack-ansible03:19
*** threestrands_ has quit IRC03:20
*** threestrands_ has joined #openstack-ansible03:20
*** threestrands_ has quit IRC03:21
*** threestrands_ has joined #openstack-ansible03:22
*** threestrands_ has quit IRC03:23
*** threestrands_ has joined #openstack-ansible03:23
*** threestrands_ has quit IRC03:24
*** threestrands_ has joined #openstack-ansible03:25
*** threestrands_ has quit IRC03:26
*** threestrands_ has joined #openstack-ansible03:26
*** threestrands_ has quit IRC03:27
*** threestrands_ has joined #openstack-ansible03:28
*** threestrands_ has quit IRC03:29
*** threestrands_ has joined #openstack-ansible03:29
*** threestrands_ has quit IRC03:30
*** threestrands_ has joined #openstack-ansible03:31
*** threestrands_ has quit IRC03:32
*** threestrands_ has joined #openstack-ansible03:32
*** threestrands_ has quit IRC03:33
*** threestrands_ has joined #openstack-ansible03:34
*** threestrands_ has quit IRC03:35
*** threestrands_ has joined #openstack-ansible03:35
*** threestrands_ has quit IRC03:36
*** threestrands_ has joined #openstack-ansible03:37
*** threestrands_ has quit IRC03:38
*** threestrands_ has joined #openstack-ansible03:38
*** threestrands_ has quit IRC03:39
*** threestrands_ has joined #openstack-ansible03:40
*** threestrands_ has quit IRC03:41
*** threestrands_ has joined #openstack-ansible03:41
*** threestrands_ has quit IRC03:42
*** threestrands_ has joined #openstack-ansible03:43
prometheanfirethat's for non-cycle-trailing projects, openstack-ansible is cycle-trailing (as a deployment project)03:43
* prometheanfire is trying to find it, but libsystemd-dev is needed on debian containers03:44
*** threestrands_ has quit IRC03:44
*** threestrands_ has joined #openstack-ansible03:44
*** threestrands_ has quit IRC03:45
*** threestrands_ has joined #openstack-ansible03:46
*** threestrands_ has quit IRC03:47
*** threestrands_ has joined #openstack-ansible03:47
*** threestrands_ has quit IRC03:48
*** threestrands_ has joined #openstack-ansible03:49
*** threestrands_ has quit IRC03:50
*** threestrands_ has joined #openstack-ansible03:50
*** threestrands_ has quit IRC03:51
*** threestrands_ has joined #openstack-ansible03:52
*** threestrands_ has quit IRC03:53
*** threestrands_ has joined #openstack-ansible03:53
*** threestrands_ has quit IRC03:54
*** threestrands_ has joined #openstack-ansible03:55
*** threestrands_ has quit IRC03:56
*** threestrands_ has joined #openstack-ansible03:56
*** threestrands_ has quit IRC03:57
*** threestrands_ has joined #openstack-ansible03:58
*** threestrands_ has quit IRC03:59
*** threestrands_ has joined #openstack-ansible03:59
*** threestrands_ has quit IRC04:00
*** threestrands_ has joined #openstack-ansible04:01
*** threestrands_ has quit IRC04:02
*** threestrands_ has joined #openstack-ansible04:02
*** threestrands_ has quit IRC04:03
*** threestrands_ has joined #openstack-ansible04:04
*** threestrands_ has quit IRC04:05
*** threestrands_ has joined #openstack-ansible04:05
*** cshen has joined #openstack-ansible04:06
*** threestrands_ has quit IRC04:06
*** threestrands_ has joined #openstack-ansible04:07
*** threestrands_ has quit IRC04:08
*** threestrands_ has joined #openstack-ansible04:08
prometheanfireit's needed for systemd-python btw04:08
prometheanfiremaybe it was a requirements thing04:09
*** threestrands_ has quit IRC04:09
*** threestrands_ has joined #openstack-ansible04:10
*** cshen has quit IRC04:11
*** threestrands_ has quit IRC04:11
*** threestrands_ has joined #openstack-ansible04:11
*** threestrands_ has quit IRC04:12
*** threestrands_ has joined #openstack-ansible04:13
*** threestrands_ has quit IRC04:14
*** threestrands_ has joined #openstack-ansible04:14
*** threestrands_ has quit IRC04:15
*** threestrands_ has joined #openstack-ansible04:16
*** threestrands_ has quit IRC04:17
*** threestrands_ has joined #openstack-ansible04:17
*** threestrands_ has quit IRC04:18
*** threestrands_ has joined #openstack-ansible04:19
*** threestrands_ has quit IRC04:20
*** threestrands_ has joined #openstack-ansible04:20
*** threestrands_ has quit IRC04:21
*** threestrands_ has joined #openstack-ansible04:22
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-ansible04:33
*** LuckyClover has left #openstack-ansible04:46
*** cshen has joined #openstack-ansible04:49
*** cshen has quit IRC04:54
*** spatel has quit IRC04:59
*** cshen has joined #openstack-ansible05:02
*** jamesdenton has quit IRC05:05
*** cshen has quit IRC05:06
*** williampiv has joined #openstack-ansible05:07
prometheanfireneat pkg-config is missing too05:11
prometheanfireand a few other packages...05:20
*** jamesdenton has joined #openstack-ansible05:22
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: install libs for needed for keystone on debian  https://review.opendev.org/73165005:23
prometheanfire^ so I don't forget, seems odd...05:23
*** williampiv has quit IRC05:24
*** udesale has joined #openstack-ansible05:25
*** udesale_ has joined #openstack-ansible05:45
*** udesale has quit IRC05:45
*** williampiv has joined #openstack-ansible05:58
*** cshen has joined #openstack-ansible06:00
jrosserprometheanfire: you should check the 'devel' package list i os_keystone for those things, not the lxc roles06:15
prometheanfiresure, I'm still kinda surprised it broke in train between when I deployed a month or two ago and now06:19
noonedeadpunkI'm wondering if it's image issue, our or rax one? https://zuul.opendev.org/t/openstack/build/a85dfdb9485e4f5d824c5c0b17a4cfc9/log/logs/host/lxc-cache-prep-commands.log.txt#17906:27
noonedeadpunkconsiderring it's only for lxc jobs, it's not image...06:28
*** poopcat has quit IRC06:29
noonedeadpunkand it's both for train and master....06:30
*** poopcat has joined #openstack-ansible06:31
*** williampiv has quit IRC06:33
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Rewrite CI provided apt sources to be http instead of https on bionic  https://review.opendev.org/73166206:36
jrossernoonedeadpunk: ^ this is my attempt to fix06:37
jrosseri don't understand right now why its only bionic breaking on this06:37
*** jbadiapa has joined #openstack-ansible06:42
*** jbadiapa has quit IRC06:42
noonedeadpunkwe won't be able to test this......06:48
noonedeadpunkso probably let's just merge....06:48
noonedeadpunkonly bionic with lxc lol. as distro and metal pretty ok with that....06:49
prometheanfireI guess I'm at the point of asking if debian 10 is supposed to be usable as a host os for openstack for train06:49
noonedeadpunkmaybe because of our script or dunno...06:49
noonedeadpunkprometheanfire: I beieve it is06:49
noonedeadpunkwe have deployment with debian 10 and train06:50
* prometheanfire isn't sure why keystone is failing to pip install itself in that case06:51
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Remove retired congress  https://review.opendev.org/73159406:51
prometheanfirethe keystone containers seem to be missing a few deps at the very least https://review.opendev.org/73165006:51
prometheanfireI'd expect the rest of the containers to have the same problem, given the packages involved06:52
prometheanfirealmost up to setup-openstack again06:52
noonedeadpunkprometheanfire: but all of them should be installed with keystone role as far as I see06:52
noonedeadpunkthe thing is, that devel packages are installed only for repo container - where wheels are build iirc06:53
prometheanfirenoonedeadpunk: ya, that's part of what has me confused, I may share a redacted user_variables and openstack_user_config when I wake up06:53
noonedeadpunkbut with wheels ready you should have no issues with installing package in venv06:54
prometheanfireand yet...06:54
prometheanfireya, I see the systemd-python wheel built at least06:56
openstackgerritamolkahat proposed openstack/openstack-ansible-os_tempest master: Added tempest ironic resources setup.  https://review.opendev.org/72070506:57
prometheanfirehmm, may have something to do with it06:57
prometheanfire2020-05-29T06:57:03,669 Could not fetch URL http://1.2.3.4:8181/os-releases/20.1.2.dev2/debian-10-x86_64: 404 Client Error: Not Found for url: http://1.2.3.4:8181/os-releases/20.1.2.dev2/debian-10-x86_64/ - skipping06:58
prometheanfire404, ya, parrent dir shows this though07:00
prometheanfire<a href="debian-10-x86_64/">debian-10-x86_64/</a>                                  29-May-2020 06:56                   -07:00
prometheanfireso maybe the wheel builder has a perms issue07:00
*** fghaas has joined #openstack-ansible07:03
*** cshen has quit IRC07:03
prometheanfirehmm, only one of the three backends succeeds I think07:04
*** rpittau|afk is now known as rpittau07:04
prometheanfireguess I have another bug to track down07:04
jrossersyncing the content between the repo containers may be broken07:06
jrosseruse curl and do that at least 3 times07:06
jrosser(if this is multinode)07:07
prometheanfireya, that's what I'm thinking ( curling 3 times is what tipped me off )07:11
prometheanfirebut I'll look into it tomorrow07:11
*** tosky has joined #openstack-ansible07:12
noonedeadpunkjrosser: ok, good, your patch works07:12
*** cshen has joined #openstack-ansible07:13
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Create Ussuri  https://review.opendev.org/72925807:13
*** jbadiapa has joined #openstack-ansible07:53
*** jbadiapa has quit IRC08:09
*** jbadiapa has joined #openstack-ansible08:34
*** itandops has joined #openstack-ansible08:46
*** rh-jelabarre has quit IRC09:02
openstackgerritMerged openstack/openstack-ansible-os_swift master: Update docstheme for style  https://review.opendev.org/72970709:30
*** yolanda has joined #openstack-ansible09:46
*** threestrands_ has quit IRC09:48
openstackgerritMerged openstack/openstack-ansible-ops master: Ensure that the beat setup role runs when elk_beat_setup=true  https://review.opendev.org/72965409:56
*** rpittau is now known as rpittau|bbl10:07
*** udesale_ has quit IRC10:16
*** udesale has joined #openstack-ansible10:18
*** itandops has quit IRC10:22
*** itandops has joined #openstack-ansible10:23
*** udesale has quit IRC10:28
*** udesale has joined #openstack-ansible10:32
*** cshen has quit IRC11:03
*** rh-jelabarre has joined #openstack-ansible11:04
*** cshen has joined #openstack-ansible11:30
*** cshen has quit IRC11:34
noonedeadpunkjrosser: your vote on https://review.opendev.org/#/c/729258/ is really appreciated12:07
noonedeadpunkI'd love to merge this asap12:07
jrosserdone12:08
noonedeadpunkcool. mnaser - would be great if you could check it as well (with all underlying patches)12:12
noonedeadpunkor guilhermesp12:12
guilhermespsure noonedeadpunk12:18
*** cshen has joined #openstack-ansible12:20
*** rpittau|bbl is now known as rpittau12:35
dmsimardjrosser: I found a way to improve mobile experience on 1.x btw :p12:40
dmsimardstill very much WIP but I know it's important to you12:41
*** rh-jelabarre has quit IRC12:47
*** rh-jelabarre has joined #openstack-ansible12:47
jrosserdmsimard: oh cool, interesting, though the current work from home situation has somewhat put an end to my reviewing patches on the train.... :)12:47
*** rh-jelabarre has quit IRC12:47
jrosserbut yes, the 0.x was surprisingly functional on a tiny screen12:48
*** rh-jelabarre has joined #openstack-ansible12:48
dmsimardI put the WIP on https://api.trunk.demo.recordsansible.org if you want to see12:48
jrosseriOS never understood the pop up windows for a task, always had to click the link to that as a whole page12:49
dmsimard"desktop view": https://i.imgur.com/UlLpLQ7.png "mobile view": https://i.imgur.com/IssXZuC.png12:51
dmsimardthe CSS supported it natively all along but I found that we weren't setting viewport so it didn't trigger and always rendered in dekstop view12:52
dmsimardI'm not going to prioritize a "mobile first" interface but if it's at least readable and rendered for mobile it's an improvement12:52
dmsimardthe other pages will get a similar treatment12:53
dmsimardplanned for 1.5 (1.4.1 should be out next week)12:54
noonedeadpunklooks pretty readable12:59
dmsimardI am hesitating on the compactness of the table13:05
dmsimardThis is compact: https://i.imgur.com/IDUW4Es.png13:07
dmsimardI think compact would be better (shows more in less space) and would allow to show more characters of the playbook name/path13:08
jrosserI think generally I am liking the more dense layouts better13:10
openstackgerritMerged openstack/openstack-ansible master: Rewrite CI provided apt sources to be http instead of https on bionic  https://review.opendev.org/73166213:11
dmsimardjrosser: dense scales better, more whitespace makes for some very long pages when you have a lot of results :p13:13
dmsimardah damn it, now I plug in my widescreen high res monitor and see all that lateral whitespace T_T13:26
mgariepynoonedeadpunk, https://github.com/facebookincubator/rpm-backports/issues/1213:35
mgariepycentos8 have 23913:35
noonedeadpunklooks exactly like what we see....13:37
mgariepyyep13:37
noonedeadpunkso this is kinda... know not to work13:37
noonedeadpunkso we need to look for some replacement of machinectl for centos 8 or smth like that....13:38
noonedeadpunkso it's either lxd, or just nspawn....13:38
noonedeadpunk(not sure it will work though)13:39
mgariepyswitching to lxd would be awesome and the same accross all repos..13:39
noonedeadpunkjust kinda.... can hardly imagine upgrade path to lxd at the moment....13:39
mgariepylol yep, just need to sync it to OS swtich 113:40
mgariepyswitch.13:40
mgariepyfor ubuntu when you need to upgrade from 18.04 to 20.04 that's a good moment.13:40
mgariepysame wtih centos7 > centos813:40
mgariepynot sure about debian tho.13:40
noonedeadpunkyeah, but what I mean - we should probably leave lxc as well, not to do this as drop-in replacement13:41
noonedeadpunkso like make 20.04 and centos 8 use lxd13:41
mgariepyyep13:41
noonedeadpunkbut all distros before - lxc13:41
mgariepynot sure how much work it would be tho. but would be a lot better for the future.13:42
mgariepywould be easier in the for the future.13:42
noonedeadpunkwe might carry debian for a while, but yeah...13:42
mgariepyis it possible to have both installed at the same time ?13:43
noonedeadpunkdunno tbh... Never looked into that...13:43
mgariepyif so we can probably write a playbook to move slowly the system to lxd.13:44
noonedeadpunkyeah we can13:44
mgariepyit's a one time thing anyway.13:44
noonedeadpunklet's probably write that down for ptg :p13:44
mgariepyhaha13:44
mgariepywhen is that btw ?13:44
mgariepyquick one, how do you guys feel about snapd ?13:45
*** cshen has quit IRC13:46
noonedeadpunkwe have booked slots on tuesday and wednesday http://ptg.openstack.org/ptg.html13:47
* noonedeadpunk not sure what snapd is13:48
noonedeadpunklike aware about snap packages13:48
noonedeadpunkbut tbh not really like them13:48
mgariepyis cpu cycle that expensive not to enable ssl everywhere.. ;)13:48
mgariepyi use them for lxd on prod server and it works great imo.13:48
noonedeadpunkjust every snap package bringhs the whole envirnoment with it...13:49
noonedeadpunkand  uses too much diskspace in case you have some of them13:49
noonedeadpunkcomparing to regular deb/rpm13:50
mgariepyyep all the libs are packaged together. it does ensure it works tho.13:50
*** cshen has joined #openstack-ansible13:51
mgariepylxd snaps are about 70 mb.13:52
*** cshen has quit IRC13:55
*** cshen has joined #openstack-ansible14:09
jrosserwe use lxd via snap, which is mostly ok14:11
jrosserbut there is one massive issue that you can’t disable auto update of the snaps14:12
jrosserand that really sucks14:12
prometheanfirenoonedeadpunk: jrosser missing quote (on train too I'm sure) https://github.com/openstack/openstack-ansible-repo_server/blob/master/templates/lsyncd.lua.j2#L62114:38
prometheanfireyep14:38
noonedeadpunkyeah.....14:39
noonedeadpunkWe changed that lately....14:39
prometheanfirehttps://github.com/openstack/openstack-ansible-repo_server/commit/53a8add55b480166ce5ded7df4e196f70d613c1e14:39
*** spatel has joined #openstack-ansible14:40
noonedeadpunkit's missing even a bit more that quote...14:40
spatelnoonedeadpunk: jrosser ^^14:41
spatellast night i was trying to build master brach of OSA on CentOS8 and i badly hit the wall...14:41
spatellots of work need to be done here..14:41
noonedeadpunkspatel: tell that mgariepy :p14:41
noonedeadpunk(in terms of hitting the wall)14:42
mgariepyyes we know spatel :)14:42
spatelCentOS8 doesn't have btrfs, bridge-utils and many more stuff14:42
openstackgerritMatthew Thode proposed openstack/openstack-ansible-repo_server master: add missing quote to lsyncd.lua template  https://review.opendev.org/73177514:42
prometheanfirenoonedeadpunk: oh?14:42
mgariepyhaha we hit a wall after every 2 kb of software run..14:42
spatelmgariepy: what is the next step? should i open ticket for all the issue i am seeing?14:42
mgariepywell it's WIP, if you can fix issue it would be better to have patches.14:42
noonedeadpunkprometheanfire: I mean it should probably be {"{{ repo_build_global_links_dirname }}"} ?14:43
spatelI gave up last night :)  inventory is biggest pain (manage.main() )14:43
spatelI am not python expert so i definitely need help.. without fixing manage.main()   i can't move on14:44
noonedeadpunkspatel: I'm pretty sure inventory should be pretty ok14:44
prometheanfirenoonedeadpunk: it's running with the review above14:44
noonedeadpunkok then:)14:45
spatelnoonedeadpunk: i got this error - http://paste.openstack.org/show/794152/14:45
noonedeadpunkso it probably more about openstack_user_variables.yml14:46
noonedeadpunkor maybe smth from https://review.opendev.org/#/c/689629/ needs to be patched as well..14:46
openstackgerritMatthew Thode proposed openstack/openstack-ansible-repo_server stable/train: add missing quote to lsyncd.lua template  https://review.opendev.org/73177914:47
noonedeadpunkbut feels more like some issue with openstack_user_variables14:47
spatelnoonedeadpunk: hmm let me check these patches, how do i get latest patches via git pull?14:48
spatelI am using master branch at present14:48
noonedeadpunkthere's download option in top right corner14:48
spatelnoonedeadpunk: oh! i can see that now..14:49
spatellet me play more around this.. i really want to build my production OSA on centOS8 if we can get this out in next 2 month :)14:50
noonedeadpunkbut its still not ready, just believe:)14:50
noonedeadpunkbut hope that in 2 month we'll have it on our hands...14:51
spatelnoonedeadpunk: sure,  is 20.0.0.0rc1 ussuri or train ?14:52
noonedeadpunktrain14:52
spateli didn't see any announcement so not sure when we are going to add ussuri?14:52
noonedeadpunkussuri will be 2114:52
noonedeadpunkI hope to place branching patch today14:52
spatelnice!14:53
noonedeadpunkonce https://review.opendev.org/#/c/729258/ merge14:53
noonedeadpunkbut it failed in gates14:55
noonedeadpunk(╯°□°)╯︵ ┻━┻14:56
jrossernoonedeadpunk: did we release train with broken lsync? i was wondering if a few folks coming here in the last week with issues were related to that15:03
noonedeadpunkyeah, I think we did.....15:03
*** ebbex has quit IRC15:03
noonedeadpunk(I'm pretty sure we did in 20.1.115:03
*** born2bake has joined #openstack-ansible15:24
born2bake Hi, I am trying to configure designate...once I am trying to set dns domain to network "openstack network set 830fcb47-1229-428c-922b-283d1aadb6fa --dns-domain sample.openstack.org.", I am getting - openstack network set: error: unrecognized arguments: --dns-domain sample.openstack.org. Any ideas why openstack network set doesnt support --dns-domain flag?15:24
noonedeadpunkjrosser: also now it seems mistake using repos for rabbit instead of files... as there's a conflict in gates....15:32
*** also_stingrayza has joined #openstack-ansible15:50
*** stingrayza has quit IRC15:53
*** cshen has quit IRC16:06
*** rpittau is now known as rpittau|afk16:21
*** udesale has quit IRC16:30
*** cshen has joined #openstack-ansible16:37
*** cshen has quit IRC16:41
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq and erlang versions  https://review.opendev.org/73180917:13
noonedeadpunk^ this should fix rabbitmq17:13
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq and erlang versions  https://review.opendev.org/73180917:15
*** cshen has joined #openstack-ansible17:16
*** aedc has quit IRC17:17
*** cshen has quit IRC17:21
mgariepyjrosser, do you have issues with the lxd snap ? i only saw issue once on the stable one17:27
mgariepyyou can always switch back to an older version in that case.17:27
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq and erlang versions  https://review.opendev.org/73180917:30
jrossermgariepy: we had it upgrade the snap and dnsmasq failed to restart, leaving one of the interfaces broken17:40
jrosserthough it wasnt obvious becasue the containers were serving things on other interfaces which were still up, there was just strange brokenness due to no dns and missing default route17:41
jrosseri'd much rather just be able to switch the updates off17:41
*** rmcall has joined #openstack-ansible18:09
mgariepyi read somehwere that they didn't want to add this because it would lead to apps never get updated18:12
*** born2bake has left #openstack-ansible18:16
*** arkan has joined #openstack-ansible18:49
spatelmgariepy: what branch / tag i should be using for centOS8 ?19:10
spatelor should be using https://review.opendev.org/#/c/689629/  which noonedeadpunk mentioned19:12
spatellet me give it a try19:12
noonedeadpunkjrosser: arg, functional tests are also failaing with https://zuul.opendev.org/t/openstack/build/26ac3f739fd144adbdf7f036d8e7f767/log/logs/host/lxc-cache-prep-commands.log.txt#17919:12
jrossernoonedeadpunk: how is that happening :(19:13
jrosseroh right we need the same in the tests repo right?19:13
noonedeadpunkyeah19:13
noonedeadpunkbut dunno where to place19:14
noonedeadpunkwhat also bothers me, that apt worked fine with https19:14
noonedeadpunklike https://zuul.opendev.org/t/openstack/build/26ac3f739fd144adbdf7f036d8e7f767/log/job-output.txt#136019:14
jrosserwell, i've not looked at it but i expect ca-certificates is not installed into the container image19:14
noonedeadpunkor https://zuul.opendev.org/t/openstack/build/26ac3f739fd144adbdf7f036d8e7f767/log/job-output.txt#150919:14
noonedeadpunkah, which we eventually download....19:15
noonedeadpunkfair19:15
jrosserbut if during container creation or openstack_hosts on the created container we put it there, then everything is ok19:15
jrosseroh hahah19:16
jrosser_lxc_cache_distro_packages:19:16
jrosser  - apt-transport-https19:16
jrosser  - ca-certificates19:16
jrosserliterally the first thing we do in setting up the container image19:16
noonedeadpunkso....19:17
*** cshen has joined #openstack-ansible19:17
noonedeadpunkwhy it's not working ;(19:17
jrosserbecause the CI image sets up sources.d to be https i think, as of a day or so ago19:18
jrosserthe the first apt update in image cache prep fails becasue it can't validate the cert on the https mirror19:18
noonedeadpunkargh19:18
jrosseryes this is a very recent change19:18
* noonedeadpunk worried if real images still have http19:18
noonedeadpunkwhat a timing, what a timing19:19
jrosserapt repos have been "secure" for a long time without https because of signing19:19
jrosserbut the CI repos are not signed19:19
jrosserso i think that we need to put a similar task in the tests repo somewhere19:20
noonedeadpunkyah, already looking for the best place for this19:20
noonedeadpunkoh19:21
jrossertest-prepare-host.yml has the right kind of name19:21
noonedeadpunkeventually...19:21
*** cshen has quit IRC19:21
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests master: Rewrite CI provided apt sources to be http instead of https on bionic  https://review.opendev.org/73183719:22
noonedeadpunk^19:22
jrosseroh cool we have the same cleanup stuff there too19:22
noonedeadpunkI probably should add co-owner but already can't recall how to correctly add this19:22
noonedeadpunkyeah:)19:22
noonedeadpunkso we merge this and that https://review.opendev.org/#/c/731809/ and after gate will be unlocked....19:23
spatelDid you guys see this error on centOS8? - http://paste.openstack.org/show/794156/     during scripts/bootstrap-ansible.sh  run19:26
noonedeadpunkI think we don't....19:27
noonedeadpunkbut it is _really_ not ready yet...19:27
noonedeadpunkLike at the moment we can't create lxc on centos8 because of missing btrfs19:28
spateli did install btrfs from third-party repo19:28
noonedeadpunkSo if you know the way how to mount btrfs fs (with mount command) that would be really cool... Or at least how to make machinectl just work with dir (at it's just stuck)19:29
noonedeadpunkyeah, but can you mount ?19:29
noonedeadpunkas it do not contain kernel module for that anymore19:29
noonedeadpunkyou can format as btrfs but can't mount19:30
spateloh!! i need to check that for mounting stuff.. let me check19:30
noonedeadpunkand this one occurs https://github.com/facebookincubator/rpm-backports/issues/1219:30
noonedeadpunklike we can workaround and create ext4 or xfs, but machinectl got stuck for some reason during upackaging of the image19:31
noonedeadpunkso that' where we stuck at the moment...19:31
spatelRedhat pushing for ZFS instead of btrfs19:32
-spatel- [root@aio1 openstack-ansible]# modprobe btrfs19:34
-spatel- modprobe: FATAL: Module btrfs not found in directory /lib/modules/4.18.0-147.8.1.el8_1.x86_6419:34
spatellook like we need to compile kernel :(19:34
noonedeadpunkso it's really pita19:35
spatelThis kernel provided by Redhat why don't i grab one from outside redhat which has btrfs support19:36
noonedeadpunktbh.... like I think if we work with btrf out we got stuck on https://github.com/facebookincubator/rpm-backports/issues/12 as well19:38
noonedeadpunkso we used a workaround of it like this https://review.opendev.org/#/c/728108/19:39
noonedeadpunkbut it just runs machinectl import-tar forever despite import is already finished...19:40
spatel+1  so are we planning to ditching btrfs?19:45
noonedeadpunkfor centos 8 we'll have to I think19:46
noonedeadpunkSo point is if we could prepare lxc image without it so machinectl is happy19:47
noonedeadpunkif we can't than we have to replace lxc with smth as well....19:47
noonedeadpunkwhich is smth not really easy to do.19:47
noonedeadpunkI guess we would discuss options on ptg....19:48
jrosserreally we could simplify19:50
jrossertake machinectl out19:51
jrosserbecasue all we do is download a tgz, unpack it, chroot into it, run the prep script, pack it back up and drop in /var/cache/lxc/<...>19:51
spatelnoonedeadpunk: i installed kernel 5.6.15-1.el8.elrepo  and i got btrfs support in kernel19:51
-spatel- [root@aio1 ~]# lsmod | grep btrfs19:51
-spatel- btrfs 1486848 019:51
spatelits hack.. but just trying to do POC19:52
jrosserafaik machinectl was a later addition19:53
*** arkan has quit IRC19:53
jrosserso it would be interesting to look at the patch which added that and see what was there before19:54
noonedeadpunkhuh maybe this kernel fix machinectl as well...19:57
spatelnoonedeadpunk: Here you go if you want to upgrade kernel - https://www.osradar.com/upgrade-the-kernel-on-centos-8-rhel-8-oracle-linux-8/19:58
spatelnoonedeadpunk: tell me if i am doing something stupid - i download tarball from - https://review.opendev.org/#/c/689629/20:01
spateland trying to install from it..20:01
spateldo we need to do anything else?20:01
*** cshen has joined #openstack-ansible20:02
*** gary_perkins has quit IRC20:06
*** cshen has quit IRC20:07
*** gary_perkins has joined #openstack-ansible20:09
noonedeadpunkjrosser: damp, circular dependency :(20:40
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq and erlang versions  https://review.opendev.org/73180920:44
*** itsjg has quit IRC20:58
*** mgariepy has quit IRC21:01
*** KeithMnemonic has quit IRC21:50
*** spatel has quit IRC21:56
*** fghaas has quit IRC21:59
*** cshen has joined #openstack-ansible22:03
*** cshen has quit IRC22:07
*** rh-jelabarre has quit IRC22:32
*** weifan has joined #openstack-ansible23:11
*** tosky has quit IRC23:38

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!