Thursday, 2018-03-08

osnayaalready checked the validity of syntax for user_secrets.yml and user_variables.yml syntax thru YAML Lint00:00
osnaya@evrardjp can you please give any clues on the error above?00:03
osnaya@evrardjp can you please give any clues on the error above?00:05
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Resolve nsenter fork exception  https://review.openstack.org/55033100:06
cloudnullosnaya: is there a "---" at the top of those files ?00:08
osnaya@cloudnull yes ... is that a problem?00:13
*** gunix has left #openstack-ansible00:17
osnaya@cloudnull I believe --- in the beginning is expected though optional. What about ... at the end of the file (is that optional or must), currently I don't have that in both of my files...00:23
*** chyka has quit IRC00:28
osnaya@cloudnull passed that error... found a typo.... fixed. will continue... thx00:36
*** lbragstad has quit IRC00:39
openstackgerritMerged openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026800:42
*** zerick has quit IRC00:56
*** zerick_ has joined #openstack-ansible00:56
*** SerenaFeng has joined #openstack-ansible00:57
*** SerenaFeng has quit IRC01:01
*** osnaya has quit IRC01:01
*** woodard has joined #openstack-ansible01:16
*** udesale has joined #openstack-ansible01:56
*** udesale_ has joined #openstack-ansible02:00
*** udesale has quit IRC02:02
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory  https://review.openstack.org/55066502:03
openstackgerritJimmy McCrory proposed openstack/openstack-ansible master: [TEST] Update Ansible to 2.5.0rc1  https://review.openstack.org/50181402:04
*** ANKITA has joined #openstack-ansible02:41
*** jamesdenton has joined #openstack-ansible02:42
openstackgerritShangXiao proposed openstack/openstack-ansible-openstack_hosts master: Update the old http links in docs  https://review.openstack.org/55067303:06
*** udesale_ has quit IRC03:31
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory  https://review.openstack.org/55066504:03
*** fusmu has joined #openstack-ansible04:08
*** fusmu has quit IRC04:13
cloudnullevenings all04:15
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Resolve nsenter fork exception  https://review.openstack.org/55033104:17
*** dave-mccowan has quit IRC04:26
*** radeks has joined #openstack-ansible04:38
*** jappleii__ has joined #openstack-ansible04:42
*** fusmu has joined #openstack-ansible04:43
*** jappleii__ has quit IRC04:43
*** threestrands_ has quit IRC04:44
*** udesale_ has joined #openstack-ansible04:45
*** fusmu has quit IRC04:47
*** lhinds has quit IRC04:58
*** portante has quit IRC04:58
*** pabelanger has quit IRC04:58
*** threestrands has joined #openstack-ansible05:00
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Resolve nsenter fork exception  https://review.openstack.org/55033105:07
*** poopcat1 has quit IRC05:09
*** karthi has joined #openstack-ansible05:14
*** threestrands has quit IRC05:25
cloudnulllogan-: re the "__libc_fork THREAD_GETMEM" issue we've recently been seeing05:33
cloudnullI recently came across https://lists.linuxcontainers.org/pipermail/lxc-users/2017-June/013493.html05:33
cloudnulland then http://manpages.ubuntu.com/manpages/bionic/man7/namespaces.7.html05:33
cloudnullIDK if this will come across as double, so sorry for the spam if so... bouncer just rebooted.05:35
cloudnulllogan-: re the "__libc_fork THREAD_GETMEM" issue we've recently been seeing05:35
cloudnullI recently came across https://lists.linuxcontainers.org/pipermail/lxc-users/2017-June/013493.html05:36
cloudnulland then http://manpages.ubuntu.com/manpages/bionic/man7/namespaces.7.html05:36
cloudnullfrom bionic which shows quite a few new namespace params we could tune / manage in the kernel05:36
cloudnullonce we get to bionic, however it would seem the THREAD_MEM issue is tightly related to clone05:37
*** ansmith has quit IRC05:37
cloudnullso to resolve this in earlier kernels we might be able to simply bump the pid_max (/proc/sys/kernel/pid_max) to 64k on infra nodes (nodes running containers) to resolve this issue.05:38
cloudnullall that said, it looks like a bug in glibc https://sourceware.org/bugzilla/show_bug.cgi?id=1539205:53
openstacksourceware.org bug 15392 in nptl "Linux, setns, PID namespaces, fork: assert() about pid inequality hit sporadically" [Minor,Resolved: fixed] - Assigned to unassigned05:53
cloudnullso there may just be no way around it, which may have also been the source of the lxc-attach hang we've seen before and now using nsenter we're just seeing the assertion error instead of silently failing.05:55
*** pcaruana has joined #openstack-ansible05:57
cloudnullfixed in 2.25 we have 2.23 in xenial http://paste.openstack.org/show/694602/05:58
*** karthi has quit IRC06:03
*** masber has joined #openstack-ansible06:03
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Resolve nsenter fork exception  https://review.openstack.org/55033106:06
cloudnull^ added a retry when the bug is encountered06:08
*** pcaruana has quit IRC06:08
cloudnulllogan-: thanks again for finding and sharing the link to https://sourceware.org/bugzilla/show_bug.cgi?id=21386 I think that closes the loop on that for now :)06:09
openstacksourceware.org bug 21386 in nptl "Assertion in fork for distinct parent PID is incorrect" [Normal,Resolved: fixed] - Assigned to fweimer06:09
*** karthi has joined #openstack-ansible06:20
*** chyka has joined #openstack-ansible06:23
*** chyka has quit IRC06:27
*** karthi has quit IRC06:37
*** karthi has joined #openstack-ansible06:37
*** jamesdenton has quit IRC06:44
*** jwitko_ has quit IRC06:44
*** masber has quit IRC06:55
*** eumel8_ has joined #openstack-ansible06:57
*** jwitko_ has joined #openstack-ansible07:00
*** eumel8_ has quit IRC07:01
*** clone99 has joined #openstack-ansible07:03
*** SimAloo has quit IRC07:04
*** Fritzmen has quit IRC07:06
*** gus has quit IRC07:14
cloudnullodyssey4me: mnaser: looks like the changes for gate clean up has worked07:15
cloudnullhowever it also looks like changes upstream in the ceph ansible roles is breaking things07:15
cloudnullhttp://logs.openstack.org/42/550542/2/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/1b57d55/job-output.txt.gz#_2018-03-07_21_01_12_63664407:15
*** gus has joined #openstack-ansible07:15
cloudnull-cc andymccr ^07:15
*** sar has joined #openstack-ansible07:19
*** Fritzmen has joined #openstack-ansible07:23
*** masber has joined #openstack-ansible07:23
*** karthi has quit IRC07:31
*** admin0 has joined #openstack-ansible07:39
*** pmannidi has quit IRC07:49
*** holser__ has joined #openstack-ansible07:50
*** ANKITA has quit IRC07:53
*** armaan has quit IRC07:56
*** armaan has joined #openstack-ansible07:56
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info  https://review.openstack.org/55054207:59
*** pcaruana has joined #openstack-ansible08:04
*** gaoyan has joined #openstack-ansible08:12
*** gaoyan has quit IRC08:12
*** chyka has joined #openstack-ansible08:13
*** karthi has joined #openstack-ansible08:17
*** mbuil has joined #openstack-ansible08:17
*** chyka has quit IRC08:17
*** gkadam has joined #openstack-ansible08:27
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Add ceph-ansible library path  https://review.openstack.org/55074708:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Add ceph-ansible library path  https://review.openstack.org/55074908:33
odyssey4meandymccr cloudnull evrardjp ^ should do the trick - will wait for gate confirmation before combining them08:33
*** lkoranda has quit IRC08:42
*** pcaruana has quit IRC08:43
*** fusmu has joined #openstack-ansible08:46
*** holser__ has quit IRC08:49
*** pcaruana has joined #openstack-ansible08:52
*** lkoranda has joined #openstack-ansible08:52
*** radeks has quit IRC08:58
*** fusmu has quit IRC08:59
*** holser__ has joined #openstack-ansible08:59
*** fusmu has joined #openstack-ansible09:00
*** pcaruana has quit IRC09:02
*** epalper has joined #openstack-ansible09:02
*** lkoranda has quit IRC09:02
*** shardy has joined #openstack-ansible09:03
*** holser__ has quit IRC09:03
*** holser__ has joined #openstack-ansible09:03
*** holser__ has quit IRC09:07
*** holser__ has joined #openstack-ansible09:07
*** lkoranda has joined #openstack-ansible09:12
*** pcaruana has joined #openstack-ansible09:15
*** SerenaFeng has joined #openstack-ansible09:21
*** SerenaFeng has quit IRC09:26
*** lkoranda has quit IRC09:30
*** jwitko_ has quit IRC09:31
*** pcaruana has quit IRC09:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Add ceph-ansible library path  https://review.openstack.org/55074709:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Add ceph-ansible library path  https://review.openstack.org/55074909:36
Taseerodyssey4me: Hi ! is zuul back to normal ?09:36
*** lkoranda has joined #openstack-ansible09:39
*** pabelanger has joined #openstack-ansible09:41
*** lhinds has joined #openstack-ansible09:41
*** portante has joined #openstack-ansible09:41
odyssey4meTaseer no, the tests are still broken for the integrated gate... some upstream changes in ceph-ansible have broken us, so we've got two issues at once. Working on a fix.09:43
Taseerodyssey4me: ok, thanks..09:44
*** ANKITA has joined #openstack-ansible09:45
*** pcaruana has joined #openstack-ansible09:45
*** Sha0000 has joined #openstack-ansible09:59
*** shardy is now known as shardy_afk09:59
*** jwitko_ has joined #openstack-ansible09:59
*** epalper has quit IRC10:03
*** epalper has joined #openstack-ansible10:03
*** Sha0000 has quit IRC10:08
*** karthi has quit IRC10:10
*** radeks has joined #openstack-ansible10:14
andymccrodyssey4me: good catch, thats pretty sucky10:21
*** electrofelix has joined #openstack-ansible10:28
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and include ceph-ansible library path  https://review.openstack.org/55054210:33
*** karthi has joined #openstack-ansible10:40
*** karthi has quit IRC10:44
*** karthi has joined #openstack-ansible10:44
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054210:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Unpin ceph-ansible and add ceph-ansible library path  https://review.openstack.org/55074910:51
gokhan_hi folks, is it possible to create multiple venvs in  a role ? or I am trying wrong things:(  I tried it and then run repo-build.yml but it does not create any venvs.10:52
odyssey4megokhan_ no, not with the current tooling10:53
odyssey4meI mean, the role could create more than one - but the repo build can't10:53
evrardjpgokhan_: I think the right answer depends on your use case11:03
evrardjpwhat are you trying to achieve?11:03
*** epalper has quit IRC11:04
*** epalper has joined #openstack-ansible11:05
gokhan_odyssey4me, ok thanks.  evrardjp,  I want to install my notification service on horizon containers. I tried it with changing horizon role and on there also tried to create a new venv for my notification service11:06
odyssey4megokhan_ better to create a role specifically for that, then just have it execute against the horizon containers11:06
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054211:08
evrardjpgokhan_: there is an alternative to what odyssey4me said11:08
evrardjpHave a way to override the packages that need installing into the horizon venv11:09
evrardjpthis way you'll build them into the repo build, and use them into the venv11:09
evrardjpbut that needs to be pip installable11:09
gokhan_odyssey4me, ok you mean create a role for myservice and then in env.d directory define it belongs to dashboard containers11:11
*** kysse_ is now known as kysse11:12
evrardjpgokhan_: yes11:12
gokhan_evrardjp odyssey4me, ok thanks I will try first way :)11:14
*** chhagarw has joined #openstack-ansible11:15
*** chhavi__ has joined #openstack-ansible11:15
odyssey4megokhan_ you won't need an env.d entry at all, unless you need to create more groups for the service11:15
evrardjpfair point11:16
odyssey4meif you just need to install something into the horizon containers, then your playbook just needs to use the horizon group11:16
odyssey4mefor the repo build to prepare a venv, you'll need this in your role: https://github.com/openstack/openstack-ansible-os_horizon/blob/master/defaults/main.yml#L331-L33511:17
odyssey4meand you'll need to ensure that the role name is prefixed by os_11:17
gokhan_odyssey4me, ok now I exactly got it :) thanks a lot :)11:21
openstackgerritwangqi proposed openstack/openstack-ansible-os_sahara master: Update links in README  https://review.openstack.org/55078411:24
*** udesale__ has joined #openstack-ansible11:25
*** udesale_ has quit IRC11:27
*** karthi has quit IRC11:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054211:36
*** udesale__ has quit IRC11:36
mnasermorning11:38
*** karthi has joined #openstack-ansible11:39
evrardjpmorning mnaser11:39
* mnaser is still sick from Dublin :(11:40
evrardjpmnaser: same. Fever is gone though, but I still feel a little dizzy11:40
evrardjpyou?11:40
evrardjptoo much guiness? :p11:41
mnaserI didn’t drink at all actually!  It was just a flu or cold or whatever it is11:43
mnaserThroat, congested, always tired11:43
mnaserAnd I can’t escape the snow. https://usercontent.irccloud-cdn.com/file/LfZ1JNn5/1520509421.JPG11:43
evrardjphaha it seems so11:44
evrardjpnothing is blocked in your place though :p11:44
*** karthi has quit IRC11:44
evrardjpcloudnull: could you ping me when you're here?11:45
hwoarangseems like the latest openstack sdk has issues with self-signed certs11:45
hwoaranghttps://bugs.launchpad.net/python-openstacksdk/+bug/175430211:45
openstackLaunchpad bug 1754302 in OpenStack SDK "openstacksdk-0.12.0 breaks with self-signed certificates" [Undecided,New]11:45
evrardjpoh :(11:45
evrardjpgreat11:45
hwoarangfyi... we may have to adjust requiremetns or something11:45
evrardjpyeah11:45
evrardjpadd a pin11:45
evrardjpis that bumped into requirements ?11:45
mnaserI don’t think that will take too long to fix11:47
evrardjpfor us it's a one liner.11:49
evrardjp:)11:49
evrardjpbut not worth having a one liner if that gets fixed quick upstream11:49
evrardjphwoarang: where did you see that?11:50
hwoarangin the opnfv xci with queens11:50
hwoarangbut it should be visible in AIO/queens too i think11:50
hwoarangsince we have self-signed certificates there11:50
hwoarangyou can try 'openstack network list' for example11:51
mnaserThe reason I think it’ll be fixed quick is that I only can assume every project is broken right now.11:51
evrardjphwoarang: we have other issues in queens right now11:51
evrardjpthat's why we didn't see it11:52
evrardjphwoarang: will check because queens should be following the proper u-c, and therefore shouldn't bump like that11:52
evrardjpmnaser: yes that's what I understood with your comment. But in stable branches, u-c are computed, and we don't bump things litely11:53
evrardjplightly*11:53
*** ANKITA has quit IRC11:53
evrardjphwoarang: https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt#L41111:53
evrardjpthat's not normal we are pulling 0.1211:54
evrardjpin q11:54
evrardjpwhere did you see that?11:54
evrardjp:D11:54
evrardjpmaybe you found THE thing I was looking for a while.11:54
hwoaranghmm11:54
hwoarangi wonder why we pullsed 0.12.011:55
hwoarang*pulled11:55
*** shardy_afk is now known as shardy11:59
evrardjpexactly12:00
evrardjpif you have a log, that would be great12:00
*** jamesdenton has joined #openstack-ansible12:01
*** jwitko_ has quit IRC12:04
evrardjpthanks mbuil for being core on os_neutron, os_tacker, and os_congress12:06
evrardjpI should say congratulations!12:08
*** radeks has quit IRC12:12
hwoarangthat's great news!12:13
odyssey4mehmm, we won't see it in our AIO/tests because we use the internal endpoint - not public (where the cert is)12:14
evrardjpodyssey4me: maybe but it's still bad if you exceed upper constraints of the branch :p12:14
mbuilevrardjp: thanks. I will try to help as much as possible12:17
hwoarangyep no clue why the upper constraints is not being respected12:17
hwoarangwill look into it12:17
*** Sha0000 has joined #openstack-ansible12:24
* fdegir congratulates mbuil as well12:24
*** SmearedBeard has joined #openstack-ansible12:25
evrardjpthanks ofc to previous ones I didn't advertise on the channel: mugsie now core on openstack-ansible-os_designate, and mnaser now core on the roles of the telemetry stack12:26
*** portante_ has joined #openstack-ansible12:27
mnaser:)12:27
*** masuberu has joined #openstack-ansible12:27
*** pabelanger_ has joined #openstack-ansible12:27
*** lhinds|out has joined #openstack-ansible12:27
*** pabelanger has quit IRC12:29
*** Smeared_Beard has joined #openstack-ansible12:29
fdegirmnaser: it was nice meeting you last week12:30
*** portante has quit IRC12:30
*** portante_ is now known as portante12:30
fdegirmnaser: and I must say I'm impressed with the level of support you are providing to ODL folks12:30
*** masber has quit IRC12:31
*** lhinds has quit IRC12:31
*** SmearedBeard has quit IRC12:32
mnaserfdegir: oh, I was hoping to ask about you, we didn’t get a chance to formally meet!12:32
fdegirmnaser: I was the guy who stood during foundation infra ci/cd and complained all the time12:33
mnaserfdegir: thank you :) if you ever need anything at all, please feel free to reach out!12:33
mnaserfdegir: I figured just as much! :p12:33
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/newton: Update all SHAs for 14.2.17  https://review.openstack.org/55081312:34
fdegirmnaser: will keep your offer in mind, thanks12:34
mnaserfdegir: we’re very “if our customers stuff isn’t working, then our stuff isn’t working either” .. no problem, anytime :)12:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054213:08
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054213:09
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054213:10
*** admin0 has quit IRC13:10
*** pabelanger_ is now known as pabelanger13:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054213:13
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info and pin ceph-ansible SHA  https://review.openstack.org/55054213:14
*** udesale has joined #openstack-ansible13:24
*** woodard has quit IRC13:25
*** woodard has joined #openstack-ansible13:25
*** holser__ has quit IRC13:27
*** holser__ has joined #openstack-ansible13:28
*** karthi has joined #openstack-ansible13:29
*** udesale has quit IRC13:31
*** udesale has joined #openstack-ansible13:31
*** gameon has joined #openstack-ansible13:36
*** SerenaFeng has joined #openstack-ansible13:36
*** karthi has quit IRC13:36
hwoarangmnaser: do you plan to submit a spec for binary installations?13:41
mnaserhwoarang: need to find a bit of time to do that, if you'd like to draft up the first steps of it, please feel free!13:41
hwoarangthe topic highly interests me but i am super busy so i wanted to check. i will prob ping you end of next week again and start drafting something up if you haven't done it already13:42
mhaydenbuenos dias13:47
*** dave-mccowan has joined #openstack-ansible13:48
openstackgerritRamon Orru proposed openstack/openstack-ansible-ops master: Fixing typo  https://review.openstack.org/55083813:49
openstackgerritRamon Orru proposed openstack/openstack-ansible-ops master: Using cluster_metrics host as default output  https://review.openstack.org/55083913:49
openstackgerritRamon Orru proposed openstack/openstack-ansible-ops master: Moving telegraf-plugins to drop them properly  https://review.openstack.org/55084013:49
*** gameon has quit IRC13:50
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Remove kernel modules: scsi_dh + x_tables  https://review.openstack.org/54975413:53
*** lbragstad has joined #openstack-ansible13:54
*** kstev has joined #openstack-ansible13:57
*** karthi has joined #openstack-ansible13:57
*** kstev1 has joined #openstack-ansible14:01
*** kstev has quit IRC14:01
*** karthi has quit IRC14:02
*** Sha0000 has quit IRC14:03
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts stable/queens: Remove kernel modules: scsi_dh + x_tables  https://review.openstack.org/55084514:06
mhaydenhwoarang: still trying to figure out why https://review.openstack.org/#/c/550102/ fails on SUSE14:07
mhaydensomething about the /etc/cron.d/sysstat is not working right -- i thought maybe the permissions were wrong14:08
mhaydenit keeps trying to set something for that cron file other than the content of the file itself, because the checksums match14:08
mhaydenfor centos, it was that it expects different file perms14:08
mhaydenbut i'm using 0644 for suse14:08
hwoaranghmm14:10
mhaydenhwoarang: i'm spinning up an opensuse instance on my cloud to gander14:11
mhaydenmaking openstack_hosts idempotent has been -- less than fun14:12
odyssey4memhayden hwoarang it seems prudent to port https://review.openstack.org/#/q/topic:bug/1750665+(status:open+OR+status:merged) back to queens/pike - your thoughts?14:13
hwoarangmhayden: ExecStart=/bin/ln -fs /etc/sysstat/sysstat.cron /etc/cron.d/sysstat14:13
hwoarangsneaky14:13
hwoarangthat's in the systemd service file14:13
mhaydenodyssey4me: agreed14:13
mhaydenhwoarang: oh wait what now?14:13
mhaydendo we even need the cron file for opensuse?14:13
hwoarangodyssey4me: yep14:14
mhaydenodyssey4me: you want to mash buttons or should i?14:14
odyssey4meif mhayden can review that galera_server one, then we can port them back in a batch14:14
odyssey4meI'm happy to port them combined once the galera_server one is merged14:15
mhaydendone14:15
*** SerenaFeng has quit IRC14:15
*** throwsb1 has joined #openstack-ansible14:15
hwoarangmhayden: probably not because we symlink it from elsewhere and we delete the symlink when the service is stopped14:15
hwoarangdon't know why it's like that14:15
hwoarang:/14:15
hwoarangmhayden: maybe you can change the task a bit to 'state: link' for suse?14:16
*** SerenaFeng has joined #openstack-ansible14:16
hwoarangpossibly a new task...14:16
hwoarangmhayden: wait, you can put the template file in /etc/sysstat/sysstat.cron14:17
hwoaranginstead of /etc/cron.d/. then it should work fine14:17
*** Sha000000 has joined #openstack-ansible14:17
mhaydenhwoarang: so something in systemd does the linking for you?14:19
mhaydenwow, that would have taken me a lot of digging to find14:19
hwoarangyep the systemd service symlinks the cron file from /etc/sysstat/ to /etc/cron.d/14:19
mhaydenokay, i can do that14:20
hwoarangthat doesn't seem right but it is what it is ;p14:20
*** ansmith has joined #openstack-ansible14:21
* mhayden scurries off to test14:22
mhaydenhwoarang: what's the default permissions on /etc/sysstat/sysstat.cron on your box?14:27
hwoarang64414:27
*** armaan has quit IRC14:33
*** armaan has joined #openstack-ansible14:33
mhaydenhwoarang: Idempotence test: pass14:38
mhaydenAND THERE WAS MUCH REJOICING14:38
hwoarangsweet :)14:38
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010214:38
mhaydenhwoarang: ^^14:38
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Fix openstack_host_module_file typo  https://review.openstack.org/55054914:39
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts stable/queens: Fix openstack_host_module_file typo  https://review.openstack.org/55085214:39
*** jwitko has joined #openstack-ansible14:39
evrardjpmhayden: :)14:42
openstackgerritMerged openstack/openstack-ansible-galera_server master: vars: Switch to galera HTTP server on all distros  https://review.openstack.org/54917114:44
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/queens: vars: Switch to galera HTTP server on all distros  https://review.openstack.org/55085314:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client stable/queens: vars: Switch to galera HTTP server on all distros  https://review.openstack.org/55085414:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/pike: Allow Galera package downloads over HTTP  https://review.openstack.org/55085614:50
mhaydenevrardjp: just replied in https://review.openstack.org/55010214:50
*** sar has quit IRC14:50
mhaydencloudnull: if you get time to look at https://review.openstack.org/550135 -- that'd be great :)14:51
mhaydenevrardjp: i am happy to break up the openstack_hosts idempotency review into multiple patches if it makes things easier14:52
*** ThomasS has quit IRC14:52
evrardjpmhayden: I am just surprised14:52
mhaydensurprised at the poor patch quality?14:53
mhayden:)14:53
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client stable/pike: Allow Galera package downloads over HTTP  https://review.openstack.org/55085714:53
evrardjpmhayden: I am surprised we can't do this in an idempotent manner14:53
evrardjpwe should have a thing that's omitting giving filename14:54
*** masber has joined #openstack-ansible14:56
evrardjpmhayden: my understanding was: If you remove repo X you have multiple choices:14:57
evrardjpwithout filename it would remove it from anywhere14:57
evrardjpwith filename it would remove it from filename14:57
evrardjpthe second item of the _package_repos list is maybe wrong14:58
*** masuberu has quit IRC14:59
evrardjpso you're telling me with filename it removes it from anywhere?14:59
evrardjpthat would explain the break of idempotency14:59
evrardjplet me read14:59
*** lhinds|out is now known as lhinds15:02
evrardjpthat's not what I wanted to say15:02
odyssey4meit may be better to stats the file, then use the conditional to say delete any source by that name unless it's the right file name15:02
evrardjpI wanted to say15:02
evrardjp"please allow me to read the apt_repository code real quick"15:02
*** hamzy has quit IRC15:05
*** esberglu has joined #openstack-ansible15:10
mhaydenhaha15:11
mhaydenevrardjp: the problem i had is that the task was removing the queens UCA repo no matter what filename it was in15:11
mhaydenso if you say "remove queens repo from file called com_ubuntu_whatever", it will remove that repo from any file it finds15:12
mhaydenso the task was removing the queens repo from the new uca file15:12
mhaydenthen adding it right back15:12
mhaydenso it always showed as changed15:12
mhaydenperhaps that's a bug in apt_repository15:12
odyssey4memhayden can we make it smarter using a stat to check whether the right filename exists, then to disable the removal task if thee right file exists? or something to that effect15:13
mhaydenodyssey4me: i just added a task that removes those files :)15:13
mhaydenso if they're not there, it gets skipped15:13
odyssey4meotherwise perhaps we should do the complicated thing we do in the galera_client repo15:13
mhaydenand the right repo gets added right after15:13
*** kstev1 has quit IRC15:14
evrardjp[m]exactly what i meant :)15:15
cloudnullevrardjp: ping15:16
cloudnullmhayden: looking15:16
cloudnullodyssey4me: those prs look good to me, are you going to combine them ?15:17
hwoarangmhayden: should ests/roles/bootstrap-host/vars/redhat.yml in the integration repo switch to Queens?15:18
hwoarang*tests/roles/bootstrap-host/vars/redhat.yml15:18
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Simplify SELinux check  https://review.openstack.org/55021515:19
odyssey4mecloudnull the pike patches are combined - the queens patches already have the first set merged, so no combining required15:21
*** masber has quit IRC15:22
evrardjpmhayden: I think your patch is nice, and I won't block it based on the fact we can make it better.15:25
odyssey4mehow gracious ;)15:26
evrardjplol15:27
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: tests: bootstrap-host: Switch to Queens repo for SUSE and Red Hat  https://review.openstack.org/55087015:27
evrardjpdo I speak like an *** all the time, and you are just very patient/friendly with me?15:28
evrardjpI am trying you know15:28
*** woodard has quit IRC15:28
mhaydenevrardjp: i just read your comments with a french accent and then i'm all good15:29
odyssey4meroflmao15:29
*** woodard has joined #openstack-ansible15:29
evrardjpmhayden: :)15:30
*** hamzy has joined #openstack-ansible15:30
d34dh0r53lmao15:30
* mhayden hugs evrardjp15:31
mhaydeni'm going to miss you guys :|15:31
mhaydeneven d34dh0r5315:31
mhaydenyes, even d34dh0r5315:31
d34dh0r53awww :(...15:32
* mhayden still needs to write that openstack-dev email but he keeps putting it off15:32
mhaydenmy eyes get wet when i try to write it15:33
*** kstev has joined #openstack-ansible15:33
mhaydenevrardjp: but no, you don't talk like an ***15:33
mhaydenyou're just very direct, and that frightens some americans :)15:33
*** Sha000000 has quit IRC15:34
mhaydenbut i like direct people15:34
mhaydenlike when hughsaunders reviews my patch and says "this part here is rubbish"15:34
mhaydenit drives the point home -- i can feel what he's feeling ;)15:35
*** hamzy has quit IRC15:35
mhaydenhwoarang: opensuse's red bash prompt for root is intimidating ;)15:36
shananigansAnyway I can get someone else to check this one out?  https://review.openstack.org/#/c/550539/   I just had someone else hit me up with networking issues on the gateway network for osa-ops mnaio deploys.15:38
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: test-prepare-host: Switch package state to 'present' on openSUSE  https://review.openstack.org/55087815:39
openstackgerritMerged openstack/openstack-ansible-galera_client stable/queens: vars: Switch to galera HTTP server on all distros  https://review.openstack.org/55085415:40
*** hamzy has joined #openstack-ansible15:41
hwoarangmhayden: lol ;p15:41
* hwoarang senses that mhayden is going somewhere...15:42
spotzshananigans: looking15:42
shananigansThanks15:42
spotzshananigans: done15:43
*** Smeared_Beard has quit IRC15:43
*** chhagarw has quit IRC15:44
*** chhavi__ has quit IRC15:44
*** chhagarw has joined #openstack-ansible15:44
openstackgerritMerged openstack/openstack-ansible-plugins stable/queens: Reduce interactions by nearly 50%  https://review.openstack.org/55015615:44
*** chhavi__ has joined #openstack-ansible15:44
*** woodard has quit IRC15:47
*** woodard has joined #openstack-ansible15:47
openstackgerritMerged openstack/openstack-ansible-ops master: Fix broken linuxbridge-agent config for the flat network in osa-ops mnaio  https://review.openstack.org/55053915:48
*** Leo_m_ has quit IRC15:49
*** SmearedBeard has joined #openstack-ansible15:52
*** electrofelix has quit IRC15:52
*** woodard has quit IRC15:54
*** woodard has joined #openstack-ansible15:54
*** SmearedBeard has quit IRC15:58
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: [WIP] Simplify nodepool variable gathering  https://review.openstack.org/55088115:59
*** kstev has quit IRC15:59
*** udesale has quit IRC15:59
*** kstev has joined #openstack-ansible16:02
*** pcaruana has quit IRC16:06
mhaydenlogan-: you need a few of these in the DC now https://www.servethehome.com/lenovo-thinksystem-sd650-water-cooled-dense-hpc-applications/16:08
*** ThomasS has joined #openstack-ansible16:12
*** SmearedBeard has joined #openstack-ansible16:14
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: test-prepare-host: Switch package state to 'present' on openSUSE  https://review.openstack.org/55087816:18
*** woodard has quit IRC16:18
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common SystemD init role  https://review.openstack.org/46617916:21
hwoarangmhayden: can you merge https://review.openstack.org/#/c/548641/ with https://review.openstack.org/#/c/550870/ ?16:22
hwoarangmine does redhat + suse at once ;p16:22
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Simplify nodepool variable gathering  https://review.openstack.org/55088116:23
mhaydenhwoarang: looking16:23
mhaydenhwoarang: why don't i just abandon mine16:24
*** vnogin has joined #openstack-ansible16:24
hwoarangyeah whatever works16:26
mhaydenhwoarang: i just abandoned mine16:28
openstackgerritMerged openstack/openstack-ansible-galera_server stable/queens: vars: Switch to galera HTTP server on all distros  https://review.openstack.org/55085316:29
*** Leo_m has joined #openstack-ansible16:30
*** Smeared_Beard has joined #openstack-ansible16:31
*** RandomTech has joined #openstack-ansible16:32
*** SmearedBeard has quit IRC16:32
mnaserodyssey4me: wanna recheck https://review.openstack.org/#/c/550542/ perhaps?16:37
*** Leo_m has quit IRC16:41
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Simplify nodepool variable gathering  https://review.openstack.org/55088116:41
*** Leo_m has joined #openstack-ansible16:42
*** chyka has joined #openstack-ansible16:43
*** prometheanfire has quit IRC16:43
mhaydenso if i download ansible.sqlite.txt.gz, sqlite3 complains about it being correupt16:50
*** RandomTech has quit IRC16:50
mhaydentrying to generate an ARA report from a completed job16:51
openstackgerritzhongshengping proposed openstack/openstack-ansible-os_trove stable/pike: Remove Trove's support for Heat  https://review.openstack.org/55090516:52
*** ygl has joined #openstack-ansible16:55
yglhi all16:55
ygli am facing a problem when doing a AIO install on a vm16:55
yglcan someone help me please16:55
*** armaan has quit IRC16:55
*** RandomTech has joined #openstack-ansible16:56
cloudnullygl: what are you seeing ?16:56
yglcloudnull: when I do a bootstrap like this "scripts/bootstrap-aio.sh" it is throwing this error16:57
yglcloudnull: TASK [bootstrap-host : Fail if there is not enough space available in /]16:57
yglcloudnull: fatal: [localhost]: FAILED! => { assertion": "(host_root_space_available_bytes | int) >= (host_data_disk_min_size_bytes | int)\n",16:57
yglcloudnull: however the scripts/bootstrap-ansible.sh is scucessful16:58
yglcloudnull: but there is 13 GB of space left in '/' partition16:58
yglcloudnull: is this a bug ?16:59
*** woodard has joined #openstack-ansible16:59
*** woodard has quit IRC16:59
cloudnullI believe the base requirement is 64G of space for an AIO16:59
* cloudnull looking16:59
yglcloudnull: is it ?17:00
*** woodard has joined #openstack-ansible17:00
yglcloudnull: but they haven't mentioned here https://docs.openstack.org/openstack-ansible/queens/contributor/quickstart-aio.html17:00
cloudnullhttps://github.com/openstack/openstack-ansible/blob/3ce37a347adb343fe6f493724b22c4846a1163a1/tests/roles/bootstrap-host/tasks/check-requirements.yml#L69-L8217:02
cloudnulland17:03
cloudnullhttps://github.com/openstack/openstack-ansible/blob/25b671e344f0f71a99406eedccf749f42e8fd9ec/tests/roles/bootstrap-host/defaults/main.yml#L21317:03
cloudnulllooks like 50G17:03
yglcloudnull: ok let me increase the disk size and try again17:03
openstackgerritMerged openstack/openstack-ansible-openstack_hosts stable/queens: Remove kernel modules: scsi_dh + x_tables  https://review.openstack.org/55084517:03
*** chyka_ has joined #openstack-ansible17:04
*** vnogin has quit IRC17:07
*** chyka has quit IRC17:07
*** gkadam has quit IRC17:12
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Archive ARA report on successful jobs  https://review.openstack.org/55095617:16
*** fusmu has quit IRC17:17
*** eumel8 has joined #openstack-ansible17:22
*** epalper has quit IRC17:23
*** holser__ has quit IRC17:24
*** mbuil has quit IRC17:26
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Archive ARA report on successful jobs  https://review.openstack.org/55096217:29
*** ivve has joined #openstack-ansible17:31
*** SerenaFeng has quit IRC17:45
RandomTechhey is anyone here able to help me troubleshoot an issue with running the playbooks17:46
RandomTech"galera_server : Restart mysql fall back" is failing and im not sure why17:46
*** ivve has quit IRC17:55
openstackgerritMerged openstack/openstack-ansible-tests master: Update the test path for plugins  https://review.openstack.org/54920417:56
*** armaan has joined #openstack-ansible17:56
*** radeks has joined #openstack-ansible17:59
*** holser__ has joined #openstack-ansible18:03
*** jwitko_ has joined #openstack-ansible18:06
*** jwitko_ has quit IRC18:08
*** jwitko_ has joined #openstack-ansible18:08
odyssey4mecloudnull mind giving https://review.openstack.org/#/q/topic:bug/1750665-stable/pike+(status:open+OR+status:merged) a nudge?18:08
RandomTechanyone here familar with the galera install?18:09
odyssey4meygl https://docs.openstack.org/openstack-ansible/queens/contributor/quickstart-aio.html says pretty clearly in the requirements that 50G is required18:09
odyssey4me50G free18:09
*** jwitko has quit IRC18:10
*** radeks has quit IRC18:13
*** jwitko_ has quit IRC18:13
*** chhagarw has quit IRC18:15
*** chhavi__ has quit IRC18:15
*** radeks has joined #openstack-ansible18:18
*** hamza21 has joined #openstack-ansible18:18
*** Smeared_Beard has quit IRC18:18
openstackgerritMerged openstack/openstack-ansible-tests master: Improve idempotency  https://review.openstack.org/55013518:22
*** ygl has quit IRC18:26
*** Leo_m_ has joined #openstack-ansible18:29
*** Leo_m has quit IRC18:32
*** Leo_m_ has quit IRC18:33
*** Fritzmen has quit IRC18:47
*** jwitko has joined #openstack-ansible18:49
RandomTechhey odyssey4me do you have a moment?18:55
odyssey4meRandomTech some, what's up?18:55
RandomTechim having an issue with the galera container during setup-infrastructure.yml that im having trouble figuring out18:56
*** poopcat has joined #openstack-ansible18:56
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common systemd init role  https://review.openstack.org/46617918:59
RandomTechheres a link with the failed task and the error i found in galera_server_error.log  odyssey4me : http://paste.openstack.org/show/695625/19:00
RandomTechi seperated them with a dashed line19:00
*** woodard has quit IRC19:00
RandomTechif you could take a look it would be really appriciated19:00
*** Fritzmen has joined #openstack-ansible19:00
RandomTechweve been having trouble with this error for a few days now19:01
odyssey4meRandomTech what tag/release/branch are you using?19:02
RandomTechwe are attempting to use pike19:02
RandomTechfollowing the instructions here: https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/19:02
RandomTechweve been at this for a few months now honestly19:03
odyssey4meRandomTech there was a version of mariadb which was borked, but we patched that up real quick - you might have been caight in the middle somewhere19:04
RandomTechwill we need to reinstall the os? or will just pulling down osa again work?19:04
odyssey4meif you do 'git log' in /opt/openstack-ansible then what's the top-most SHA?19:04
*** woodard has joined #openstack-ansible19:05
odyssey4mealso, do "grep -r '^openstack_release' * and let me know the version it gives19:06
RandomTechwhat does SHA stand for again?19:07
odyssey4methe SHA is the long hex, something like: commit f94c4b90629bb34638035e575596feea0b14fe6319:08
RandomTechhere you go: http://paste.openstack.org/show/695640/19:09
RandomTechi sperated tem by a dashed line again19:09
odyssey4meRandomTech yep, you're hitting this issue: https://github.com/openstack/openstack-ansible/commit/0e0819bb1c3782045c8a539bf04f4e92d019224d19:11
odyssey4meRandomTech better to checkout 16.0.9, then rebootstrap-ansible19:12
RandomTechso pull down 16.0.9 and then rerun bootstrap?19:12
odyssey4meyou'll have to destroy the galera_server containers if there's no data in there you care about... otherwise you'll have to uninstall the existing packages and remove the repositories19:13
odyssey4mebasically the mariadb version used is very broken when used in a cluster19:13
RandomTechim fine with running lxc-containers destroy19:13
RandomTechthis things never been setup fully19:13
odyssey4meit does take some time to get the initial config right for your use-case19:14
RandomTechi definately figured that out19:14
odyssey4methat's why we recommend having a good test environment where you can wipe and rebuild quickly19:14
RandomTechsadly we do not19:14
RandomTechit can take us a few hours to reinstall but its fairly doable19:15
odyssey4meif you have a single host, with 64GB RAM and some disk - then you can use https://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio19:15
*** SmearedBeard has joined #openstack-ansible19:15
odyssey4memore memory is better, of course19:15
RandomTechsadly im having to reuse old blades that fell out of a previose cluster19:17
RandomTechyour saying we should reinstall though?19:17
RandomTechif so we can do that itll just take time19:17
odyssey4meRandomTech no, just ditching the galera_server containers and rebuilding them will be fine19:18
RandomTechoh okay19:18
odyssey4meassuming the install stopped there, even the client won't be installed anywhere else19:18
RandomTechoh yay odyssey4me i noticed the documentation is having me pull 16.0.8 should i pull 16.0.9 instead?19:18
odyssey4meRandomTech odd, I guess the docs didn't publish quite right - where're you seeing that?19:19
RandomTechhere: https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/deploymenthost.html19:19
RandomTechthe page is listed as 16.0.9 but says to pull 16.0.819:20
RandomTechive noticed this with previose versions as well19:20
RandomTechheres a better link odyssey4me : https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/deploymenthost.html#install-the-source-and-dependencies19:21
odyssey4meRandomTech hmm, that's a bug if it's been consistent like that19:21
RandomTechi think ive brought it up before but was told to just use the one listed19:21
odyssey4meRandomTech ah ok - I understand how this happened19:23
odyssey4methis was released in the last 24 hours, and not a single patch has merged since19:23
odyssey4methe docs are published after any merge19:23
odyssey4meso this is usually a brief state19:23
RandomTechi should be fine pulling 16.0.9 then?19:24
odyssey4meyes, absolutely19:24
RandomTechjust destroyed containers, deleted /opt/openstack-ansible/ and pulled the new one19:26
RandomTechrunning the new bootstrap now19:27
RandomTechthanks again odyssey4me. hopefully it works correctly now19:28
* odyssey4me hopes so too :)19:28
RandomTechnow that ive run bootstrap i should be able to start running setup-hosts.yml again correct?19:29
odyssey4meRandomTech yep19:30
RandomTechsounds good :)19:30
odyssey4mecloudnull I'm still seeing disk space problems on some tests, quite randomly it seems - the host has 80GB, and it seems the fail is when doing things in containers19:31
odyssey4medo we have any machinectl info for the integrated build job logs19:31
*** woodard has quit IRC19:35
*** woodard has joined #openstack-ansible19:36
*** prometheanfire has joined #openstack-ansible19:41
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Archive ARA report on successful jobs  https://review.openstack.org/55096219:43
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Archive ARA report on successful jobs  https://review.openstack.org/55095619:44
jrosserany ideas on this? local test failure in os_keystone 'vagrant up ubuntu1604' http://paste.openstack.org/show/695690/19:46
evrardjpjrosser: your role is named openstack-ansible-plugins instead of plugins? It depends on your git clone I guess?19:48
jrosserwell - i just cloned the repo off github, did a little tweak and tried to test it in vagrant19:49
jrosserjust to be super clear its the openstack-ansible-os_keystone test i'm trying19:50
*** holser__ has quit IRC19:51
cloudnulljrosser: if testing master this was a recent issue  we just resolved. https://review.openstack.org/#/c/549204/19:52
evrardjpjrosser: yup see ^19:52
odyssey4mejrosser https://review.openstack.org/549204 just merged - remove tests/common and re-run run_tests19:52
evrardjpvagrant destroy; vagrant up19:52
jrosserooh right yes that looks like it19:53
jrosserthanks!19:53
*** radeks_ has joined #openstack-ansible19:57
*** radeks has quit IRC19:59
RandomTechwell i need to get going odyssey4me, if i remember ill tell you how it went20:01
odyssey4meRandomTech hope it turns out bettwr this time - have a great evening!20:02
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common systemd init role  https://review.openstack.org/46617920:04
*** RandomTech has quit IRC20:05
*** radeks__ has joined #openstack-ansible20:07
*** radeks_ has quit IRC20:09
*** flemingo has joined #openstack-ansible20:15
*** admin0 has joined #openstack-ansible20:18
*** radeks__ has quit IRC20:22
*** flemingo_ has joined #openstack-ansible20:25
*** flemingo has quit IRC20:25
*** flemingo_ has quit IRC20:25
*** flemingo has joined #openstack-ansible20:26
*** hw_wutianwei has quit IRC20:31
*** hw_wutianwei has joined #openstack-ansible20:32
mhaydencloudnull / odyssey4me thanks for those quick looksees on those ARA patches ;)20:32
*** Leo_m has joined #openstack-ansible20:35
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common systemd init role  https://review.openstack.org/46617920:40
*** Leo_m has quit IRC20:40
*** Leo_m has joined #openstack-ansible20:40
*** woodard has quit IRC20:46
*** lucasxu has joined #openstack-ansible20:46
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Simplify nodepool variable gathering  https://review.openstack.org/55088120:46
*** throwsb1 has quit IRC20:47
*** sar has joined #openstack-ansible20:47
*** sawblade6 has joined #openstack-ansible20:50
openstackgerritMajor Hayden proposed openstack/openstack-ansible-galera_client master: [WIP] Make galera_client role idempotent  https://review.openstack.org/55100120:51
*** ansmith has quit IRC20:53
openstackgerritMajor Hayden proposed openstack/openstack-ansible-galera_client master: [WIP] Make galera_client role idempotent  https://review.openstack.org/55100121:04
*** masber has joined #openstack-ansible21:05
*** lucasxu has quit IRC21:09
*** prometheanfire has quit IRC21:12
*** prometheanfire has joined #openstack-ansible21:17
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_keystone master: Only try to stop services that exist  https://review.openstack.org/55101021:18
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010221:21
*** eumel8 has quit IRC21:24
*** Leo_m_ has joined #openstack-ansible21:24
*** eumel8 has joined #openstack-ansible21:25
*** Leo_m has quit IRC21:26
*** holser__ has joined #openstack-ansible21:27
openstackgerritMajor Hayden proposed openstack/openstack-ansible-galera_client master: Make galera_client role idempotent  https://review.openstack.org/55100121:28
*** holser__ has joined #openstack-ansible21:29
*** eumel8 has quit IRC21:33
logan-dang mhayden. wonder how much those weigh re: https://www.servethehome.com/lenovo-thinksystem-sd650-water-cooled-dense-hpc-applications/21:35
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Simplify nodepool variable gathering  https://review.openstack.org/55088121:36
mhaydenlogan-: 5 pounds?21:40
*** sar has quit IRC21:45
*** threestrands has joined #openstack-ansible21:47
*** hamzy has quit IRC21:57
*** hamza21 has quit IRC22:00
*** kstev has quit IRC22:03
*** vnogin has joined #openstack-ansible22:08
*** holser___ has joined #openstack-ansible22:08
*** jra has joined #openstack-ansible22:10
jraSo OSA creates rsyslog containers for the service containers to syslog to; if I already have a nice highly-available infrastructure-grade syslog concentrator service set up, could I not define log_hosts and cause the service containers to log directly to that instead?22:11
*** vnogin has quit IRC22:12
*** holser__ has quit IRC22:12
*** holser___ has quit IRC22:15
jrosserjra: https://docs.openstack.org/openstack-ansible-rsyslog_client/latest/22:22
jrajrosser: Thanks!22:22
*** masber has quit IRC22:25
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common systemd init role  https://review.openstack.org/46617922:28
openstackgerritMerged openstack/openstack-ansible-tests master: test-prepare-host: Switch package state to 'present' on openSUSE  https://review.openstack.org/55087822:31
openstackgerritMerged openstack/openstack-ansible-tests master: test-log-collect.sh: Collect networkd information  https://review.openstack.org/54976422:31
*** admin0 has quit IRC22:33
*** armaan has quit IRC22:39
*** armaan has joined #openstack-ansible22:39
*** esberglu_ has joined #openstack-ansible22:40
*** ansmith has joined #openstack-ansible22:41
*** Leo_m_ has quit IRC22:41
*** esberglu has quit IRC22:43
*** armaan has quit IRC22:49
*** armaan has joined #openstack-ansible22:50
jrajrosser: so I know these are very elementary questions, but I'm just getting my bearings with OSA: once I've added an rsyslog_client_user_defined_targets stanza to user_variables, should I re-run setup-everything.yml with --tags=rsyslog to have that take effect?22:54
*** masber has joined #openstack-ansible22:54
*** Leo_m has joined #openstack-ansible22:56
*** esberglu_ has quit IRC22:59
*** esberglu has joined #openstack-ansible23:00
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_keystone master: Convert role to use a common systemd init role  https://review.openstack.org/55102823:00
jrosserjra: rather than give you possibly bogus advice, i'd start here http://codesearch.openstack.org/?q=rsyslog_client&i=nope&files=&repos=23:02
*** esberglu has quit IRC23:04
jrosseryou can see there all the places rsyslog_client gets used, and also that the tag 'rsyslog' might help you target an ansible run more accurately that just rerunning everything23:05
jrosserso it looks like you are on the right track23:06
jraYeah, I found that tag; I guess I'm just wondering whether there's a general pattern in OSA for applying these kinds of targeted updates as opposed to just re-running everything. Running with tags=rsyslog still runs a pile of stuff, and doing the setup-hosts.yml run with tags=rsyslog seems to want to also restarge the lxc containers, which I don't really want to do.23:07
jrosserthere are also rsyslog_client-install and rsyslog_client-config tags23:09
jrosserhowever others are better placed than me to comment on a general pattern in OSA23:09
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Convert role to use a common systemd init role  https://review.openstack.org/46617923:13
cloudnulljra: that tag should work to update only rsyslog globally.23:14
cloudnullin general I will use a tag like that on a specific service when updating the user_variables23:15
cloudnullsomething like `openstack-ansible repo-server.yml --tags=rsyslog`23:15
cloudnulletc23:15
cloudnullif the change your making will impact everything you can run setup-everything with the tag and it'll do the tagged actions in all of the required places.23:16
jracloudnull: that's how I expected things to work based on my non-OSA ansible experience, but it seems like there are an awful lot of "always"-tagged tasks that also run23:17
cloudnullyes23:17
jraso if I do "openstack-ansible --tags=rsyslog setup-hosts.yml", the playbook actually fails outright23:17
cloudnullthere are quite a few always tasks23:17
cloudnullwhat is it failing on ?23:18
jraLXC image staging, it looks like23:18
cloudnullah yea. i see.23:18
*** esberglu has joined #openstack-ansible23:18
* cloudnull just confirmed that23:18
jrafatal: [host04]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'lxc_images' is undefined\n\nThe error appears to have been in '/etc/ansible/roles/lxc_hosts/tasks/lxc_cache_prestage.yml': line 66, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# download23:19
cloudnullit does fail in lxc-hosts.yml. that's a bummer.23:19
jraI feel like that shouldn't be an always task23:20
cloudnullif i comment that playbook out it seems to go23:20
jradang23:20
* cloudnull looking23:20
cloudnulli think there's a missing fact that needs to be passed on23:20
* cloudnull trying setup-infrastructure.yml with that tag now23:21
jrayeah, this is always the hazard with tag-isolated runs, it's non-trivial to design stand-alone play paths23:21
cloudnull++23:21
jrabut a full OSA run in my dev environment takes literally 3.5hrs, so I'm kinda trying to avoid that... :)23:22
cloudnullthe tag works in infra23:23
jranice23:23
*** esberglu has quit IRC23:23
cloudnullgranted my test cluster is only 5 nodes23:23
cloudnullbut it should convey the same23:24
* cloudnull trying setup-openstack.yml23:24
cloudnullsetup-openstack.yml worked too23:26
cloudnullso we have a bug in setup-hosts.23:26
cloudnulljra: idk if you have a moment, but mind raising an issue for that ?23:30
jracloudnull: definitely, I'll do that23:31
jrathanks for your help!23:31
*** threestrands_ has joined #openstack-ansible23:33
*** jra has quit IRC23:36
*** threestrands has quit IRC23:36
*** jra has joined #openstack-ansible23:36
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts master: Fix role when running with isolated tags  https://review.openstack.org/55103223:37
cloudnulljra: ^23:37
cloudnullwith that conditional tags in isolation works with setup-hosts23:37
*** chyka has joined #openstack-ansible23:47
*** chyka_ has quit IRC23:48
jraaha, so it's already in the works23:54
jracloudnull: oh I see, and you've just authored the change. Thanks!23:54
cloudnullyea it should make it go , if you want to pull that in to try it out and review the pr it'd be appreciated23:55
cloudnullbut local tests seem to be all ok now23:55
jraI'll give it a go23:55
cloudnullthat was the only change i need to apply to get `openstack-ansible setup-everything.yml --tags rsyslog` working23:56
cloudnullbut YMMV given that my test cloud is not all that big or complex23:56
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_keystone master: Convert role to use a common systemd init role  https://review.openstack.org/55102823:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!