Tuesday, 2018-04-17

*** Leo_m has quit IRC00:04
mmercerbah... haproxy logging is not enabled, talk about annoying00:05
*** cjloader has quit IRC00:07
*** csmart has quit IRC00:08
*** keithmnemonic[m] has quit IRC00:09
*** weezS has quit IRC00:09
*** evrardjp[m] has quit IRC00:09
*** RossSchulman[m] has quit IRC00:09
*** flaviosr has quit IRC00:09
*** flaviosr has joined #openstack-ansible00:10
*** flaviosr has quit IRC00:17
mmercerheh, looks like it might have been my own fault, i seemed to have forgotten to stop and mask firewalld00:18
*** flaviosr has joined #openstack-ansible00:20
mmercerwell, its definitely taking longer to fail =D  lol00:21
*** cshen has joined #openstack-ansible00:32
*** cshen has quit IRC00:37
*** yifei1 has joined #openstack-ansible00:47
*** jbadiapa has quit IRC00:51
*** jbadiapa has joined #openstack-ansible00:53
*** geb has quit IRC00:55
*** jbadiapa has quit IRC01:01
*** cjloader has joined #openstack-ansible01:02
*** jbadiapa has joined #openstack-ansible01:02
*** cjloader has quit IRC01:06
*** esberglu has quit IRC01:16
*** cshen has joined #openstack-ansible01:26
*** cshen has quit IRC01:31
*** jbadiapa has quit IRC01:36
*** jbadiapa has joined #openstack-ansible01:38
*** portante_ has joined #openstack-ansible02:01
*** pabelanger_ has joined #openstack-ansible02:02
*** lhinds|out has joined #openstack-ansible02:02
*** cjloader has joined #openstack-ansible02:02
*** portante has quit IRC02:02
*** lhinds- has joined #openstack-ansible02:03
*** pabelanger has quit IRC02:03
*** pabelang- has joined #openstack-ansible02:03
*** portante has joined #openstack-ansible02:04
*** pabelang- is now known as pabelanger02:05
*** lhinds has quit IRC02:05
*** portante_ has quit IRC02:06
*** pabelanger_ has quit IRC02:06
*** lhinds|out has quit IRC02:06
*** cjloader has quit IRC02:07
*** cjloader has joined #openstack-ansible02:08
*** cjloader has quit IRC02:13
*** cshen has joined #openstack-ansible02:21
*** cshen has quit IRC02:25
*** yifei1 has quit IRC02:25
*** dave-mcc_ has quit IRC02:45
*** jbadiapa has quit IRC02:46
*** jbadiapa has joined #openstack-ansible02:49
*** mma has joined #openstack-ansible02:52
*** nicolasbock has quit IRC02:55
*** mma has quit IRC02:56
*** cjloader has joined #openstack-ansible02:58
*** cjloader has quit IRC03:06
*** cshen has joined #openstack-ansible03:15
*** cjloader has joined #openstack-ansible03:16
*** cshen has quit IRC03:19
*** cjloader has quit IRC03:21
*** jbadiapa has quit IRC03:41
*** jbadiapa has joined #openstack-ansible03:42
openstackgerritmichael rice proposed openstack/monitorstack master: Adding vg_check from the maas plugins  https://review.openstack.org/47019603:54
*** cjloader has joined #openstack-ansible03:57
*** poopcat has quit IRC04:03
*** ildikov has quit IRC04:03
*** mrhillsman has quit IRC04:03
*** cjloader has quit IRC04:03
*** mrhillsman has joined #openstack-ansible04:04
*** andymccr has quit IRC04:05
*** andymccr has joined #openstack-ansible04:05
*** cshen has joined #openstack-ansible04:09
*** udesale has joined #openstack-ansible04:11
*** cshen has quit IRC04:13
*** asettle_ has joined #openstack-ansible04:14
*** dcdamien has quit IRC04:16
*** asettle has quit IRC04:16
*** NostawRm has quit IRC04:16
*** NostawRm has joined #openstack-ansible04:17
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible master: Improve Congress scenario testing  https://review.openstack.org/56083904:28
*** markvoelker has quit IRC04:37
*** geb has joined #openstack-ansible04:51
*** cjloader has joined #openstack-ansible04:57
*** cjloader has quit IRC05:07
*** mrhillsman has quit IRC05:29
*** mrhillsman has joined #openstack-ansible05:30
*** keithmnemonic[m] has joined #openstack-ansible05:30
*** csmart has joined #openstack-ansible05:32
*** RossSchulman[m] has joined #openstack-ansible05:37
*** evrardjp[m] has joined #openstack-ansible05:38
*** markvoelker has joined #openstack-ansible05:38
*** jafeha__ has quit IRC06:00
*** mma has joined #openstack-ansible06:01
*** cjloader has joined #openstack-ansible06:02
*** DanyC has joined #openstack-ansible06:06
*** DanyC has quit IRC06:12
*** cjloader has quit IRC06:12
*** DanyC has joined #openstack-ansible06:12
*** pcaruana has joined #openstack-ansible06:16
*** DanyC has quit IRC06:17
*** DanyC has joined #openstack-ansible06:19
*** tobberydberg has quit IRC06:21
*** harryhuang_ has joined #openstack-ansible06:35
*** ianychoi has joined #openstack-ansible06:38
*** gokhan has quit IRC06:39
*** DanyC has quit IRC06:41
*** DanyC has joined #openstack-ansible06:41
*** esteghlal has joined #openstack-ansible06:42
esteghlalhi! i've deployed openstack ansible stable/pike. i 've used exactly same setup with test environment example in documentations. but when i try to launch an instance it got an error about can binding port !! could any one help me please?06:44
esteghlalthe exact error is :06:44
esteghlal2018-04-17 20:09:46.255 2403 ERROR neutron.plugins.ml2.managers [req-5b6f9ea2-1310-4cf8-859e-432286e3f012 5be3ce9c92db469596ccd49f16d162e7 d48a5793110f47e0801b5f15c32c0660 - default default] Failed to bind port 03a9ce0b-13a9-46ba-9fc4-1cf2c7bb7028 on host compute for vnic_type normal using segments [{'network_id': '6e7c4b53-6a5e-4ef6-969c-9f10c354d24c', 'segmentation_id': 38, 'physical_network': None, 'id': 'e56657ff-db06:45
esteghlalalso i have another question can i just use one bridge as br-vlan and br-vxlan ?06:45
*** DanyC has quit IRC06:46
*** maybebuggy has joined #openstack-ansible06:51
*** radeks has joined #openstack-ansible06:56
*** cjloader has joined #openstack-ansible06:56
*** cjloader has quit IRC07:01
*** jbadiapa has quit IRC07:01
*** jbadiapa has joined #openstack-ansible07:01
*** jbadiapa has quit IRC07:05
niraj_singh I have this erro "Failed to find required executable rabbitmqctl in paths: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" in TASK [Ensure Rabbitmq vhost]07:05
*** jbadiapa has joined #openstack-ansible07:06
*** mardim has quit IRC07:11
*** mbuil has joined #openstack-ansible07:22
*** tobberydberg has joined #openstack-ansible07:23
*** esteghlal has quit IRC07:26
*** epalper has joined #openstack-ansible07:31
*** hamza21 has joined #openstack-ansible07:34
*** foutatoro has joined #openstack-ansible07:43
*** shardy has joined #openstack-ansible07:54
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible master: Improve Congress scenario testing  https://review.openstack.org/56083907:56
*** electrofelix has joined #openstack-ansible07:57
*** cjloader has joined #openstack-ansible07:57
evrardjpniraj_singh: did you ensure rabbitmq was installed in your role tests? Are the groups okay?08:00
evrardjpesteghal that doesn't okay08:00
evrardjpwell, now you left... but physical_network: None seems a problem.08:01
evrardjpPlease check at your configuration.08:01
evrardjpour last run in the gates of Pike ran tempest perfectly fine, so it's on your configuration.08:01
*** DanyC has joined #openstack-ansible08:03
*** ianychoi has quit IRC08:04
jrosserevrardjp: this looks wierd https://github.com/openstack/openstack-ansible-os_tempest/blob/master/tasks/tempest_resources.yml#L194 <- i think that "type" should actually the neutron label for the interface mapping, not the type08:07
*** cjloader has quit IRC08:07
jrosseri think this works currently because osa uses vlan:vlan and vxlan:vxlan everywhere, but it doesnt look right for an interface mapping like physnet1:eth<n>08:08
*** ianychoi has joined #openstack-ansible08:10
evrardjpwhat do you propose to replace it with?08:20
jrosserwell the naming in there is a bit obtuse at the moment08:20
evrardjpthat's not answering my question :)08:21
jrosseri know :)08:21
jrosserright, so the reason i didnt just patch it was there is already tempest_public_net_name08:21
jrosserand i'm not sure what the proper term for the LHS of the interface_mappings entries are, ie the physnet in physnet:eth108:22
jrosserperhaps it should be tempest_public_net_physical_net08:24
evrardjphttps://github.com/openstack/openstack-ansible-os_tempest/blob/master/defaults/main.yml#L94-L105 that's not good that's what you mean?08:29
jrosseryes08:29
jrosserline 100 is called _type and is then driven into os_network to select the *name* of the pysical network08:30
jrosserwhich also isnt necessarily the name you give the newly created network in line 9408:31
evrardjpI am fine with the rename to _name if you put a release note08:31
jrosseron the cli you would pass --provider:physical_network=<blah>08:32
*** udesale_ has joined #openstack-ansible08:32
jrosserso _net is a closer match to that08:32
evrardjpalternatively making it just a var08:32
evrardjpand split on :08:32
foutatoroHi all, I've an issue when integrating OSA with ceph http://paste.openstack.org/show/719350/08:33
evrardjphey foutatoro08:33
foutatoroevrardjp: good moning08:34
*** udesale has quit IRC08:34
evrardjpfoutatoro: please paste your openstack_inventory.json -- we shouldn't deploy something on the ceph nodes, as there is no ceph_mon|osd into your conf.d/openstack_user_config08:35
*** TobbeCN has joined #openstack-ansible08:35
*** erolg has joined #openstack-ansible08:36
foutatoroevrardjp: http://paste.openstack.org/show/719352/08:39
foutatorohere is the inventory file08:39
evrardjpdid you also read: https://docs.openstack.org/openstack-ansible-ceph_client/pike/configure-ceph.html ?08:40
odyssey4meevrardjp the deliverables split was back when how we released was based on the governance file - that's all changed now... each deliverable had its own cadence for releasing - so the 'openstack-ansible' deliverable was cycle-trailing, but the others were independent08:40
erolgevrardjp, hey you said that maybe, pypi being in maintenance. If you can see logs. pip can not read requirement.txt because of unsupported archive format. http://paste.openstack.org/show/719068/08:41
evrardjpfoutatoro: I am sorry but the inventory.json is not in line with what you gave me as openstack_user_config/conf.d08:41
evrardjplook at L820 and above08:41
evrardjpthere are definitely stuff there08:41
evrardjpabout ceph installation08:41
evrardjpmaybe your inventory is not clean and you should use inventory manage08:41
evrardjpodyssey4me: ok08:42
evrardjperolg: sorry I had to leave yesterday, so I didn't do a in-depth analysis. Nobody got the chance to help you?08:42
evrardjpodyssey4me: we have a problem right now08:42
evrardjpabout the releasing. So why was the independant thing? Should we release everything as the same kind of deliverable, for ease of use?08:44
foutatoroevrardjp: sorry here is the inventory http://paste.openstack.org/show/719353/08:44
evrardjpfoutatoro: same thing08:45
odyssey4meevrardjp not everything was ready for releasing yet - anything incubated was kept out of the mainline deliverable so that it could be released independently08:45
odyssey4meif we put them into one deliverable, then they had to be release at the same time with the same tag08:46
odyssey4meI don't know whether that rationale still stands today though08:46
evrardjperolg: http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/stable/queens/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/d5aaac4/logs/openstack/aio1_repo_container-80e41045/repo/venv_build_heat-17.0.2-x86_64.log08:47
evrardjperolg: not sure what you are doing, but here is an extract of a latest queens run heat venv build ^08:47
evrardjpit works just fine08:47
evrardjpNot sure you repo build was fine08:47
evrardjpmaybe re-run it with the options to clean stuff08:48
erolghmm08:48
evrardjperolg: there are flags you can pass to force the repo to rebuilt08:48
evrardjprebuild*08:48
evrardjpodyssey4me: that's a good question08:48
evrardjpodyssey4me: basically releases have a problem with our "one deliverable" system08:49
evrardjpthey want all of them separated.08:49
evrardjpI would like to discuss on the reasons for it, right now it's a technical reasons: There are timeouts in the jobs08:49
odyssey4mewhy is it a problem?08:50
odyssey4meoh, interesting08:50
evrardjpI don't know yet08:50
evrardjpbut I'd like to understand the rest of the history first08:50
evrardjpand what can we do08:50
odyssey4mewell, our infra roles could release independently now as we discussed at the PTG08:50
evrardjpI don't mind having all those repos as separate commits and separate deliverables08:50
evrardjpwhat's the impact of this?08:51
odyssey4mewe may have to add more tests to ensure that they are supported by a broader range of ansible versions08:51
odyssey4methat will mean that the way we test is going to have to change08:51
evrardjpwe'd have to branch ourselves, and do many things ourselves, right?08:51
*** armaan has joined #openstack-ansible08:51
odyssey4meno, I don't think so08:51
erolgevrardjp, actually i just follow docs. i will try it with force rebuild.08:52
evrardjperolg: mmm interesting08:52
erolgevrardjp, thanks. :)08:52
odyssey4meit would just mean that the submission for a single release would now become many releases08:52
evrardjperolg: well tell me if it works better after a force rebuilt. Then we might have to dig and file a bug08:53
evrardjpodyssey4me: you mean you'd put every single repo into its own thing in projects.yaml right?08:53
*** jwitko has quit IRC08:53
evrardjplet's talk real code08:53
evrardjpall the things here:08:54
evrardjphttps://github.com/openstack/governance/blob/b570b77afc4efc502b7c76e260a86525f70b67b3/reference/projects.yaml#L245308:54
evrardjpwould be like this: https://github.com/openstack/governance/blob/b570b77afc4efc502b7c76e260a86525f70b67b3/reference/projects.yaml#L2494-L249608:54
odyssey4meI don't know if that's what they're asking you to do - and I don't know the implications of having them as one deliverable or many in governance.08:55
evrardjpok08:55
evrardjpbut the initial idea, for us, was to have ONE deliverable, because one release was supposed to be an atomic thing08:55
evrardjpright?08:55
odyssey4meI'm happy to talk through it with them to get more clarification if you'd like my perspective?08:55
evrardjpI have an email ready08:56
evrardjpI have started to discuss with them on releases too08:56
odyssey4meThe general idea was that when something graduated into the integrated build, it became part of the main deliverable.08:56
evrardjpbut now I think it's better if we communicate08:56
*** TobbeCN has quit IRC08:56
odyssey4meUntil then, it was independently released.08:56
evrardjpwhat's the impact?08:56
odyssey4meimpact of what?08:56
*** TobbeCN has joined #openstack-ansible08:56
evrardjpwhat's this whole graduation process? What's the purpose of it?08:57
niraj_singhevrardjp: i resolve earlier issue regarding rabbitmq but now getting this fatal: [aio1 -> 172.29.238.76]: FAILED! => {"changed": false, "msg": "The MySQL-python module is required."}. Earlier all was working fine. My playbook included the task to install Mysql. I also tried using pip.08:57
evrardjpI think the line is blurried now08:57
evrardjpwe are using things from the outside08:57
odyssey4meevrardjp sure, things have changed from back then08:58
evrardjpMy question is what's the purpose of graduation now, and what's the purpose of having one deliverable08:58
odyssey4methe purpose is whatever you make it to be08:58
evrardjpok08:58
evrardjpwell for me we come with all bolts included anyway, because they are wired in a-r-r08:58
odyssey4methe single deliverable was really just to keep things simple08:58
odyssey4mewe release it regularly and everything has the same tag - it's simple08:59
evrardjpnobody cares about the openstack deliverables if it's one, or many, if it's graduated or not, if it's independant or x08:59
odyssey4mereleasing independently makes it harder to manage because there are many things that have their own schedule, or no schedule08:59
evrardjpok so everything as NOT independant would be fine09:00
odyssey4methat's how it is now09:00
evrardjpI am comparing with triple-o now09:01
evrardjphttps://github.com/openstack/releases/tree/master/deliverables/rocky09:01
evrardjpthey have multiple yaml files, one per repo09:01
evrardjpall cycle-trailing09:02
*** cjloader has joined #openstack-ansible09:02
odyssey4meyes, and?09:03
evrardjpso currently they seem to have bumped the timeout for openstack-tox-jobs09:07
*** cjloader has quit IRC09:07
evrardjpodyssey4me: and that's something we can do09:07
evrardjpI really don't care as long as it's easy to release. If that makes it easier, good.09:07
evrardjpwhat I take from this, is that we had something, but it doesn't matter anymore, we can change it.09:08
evrardjpI will speak with dhellmann and smcginnis to decide our next steps then.09:08
evrardjpbecause we are talking about stable branches.09:09
evrardjpthanks for your time.09:09
odyssey4meyeah, what was done in the past doesn't necessarily matter - it's more important to understand what we want from now on and to drive towards that09:10
odyssey4meit's up to the release team to guide us how to achieve that09:10
evrardjpyes. That's why a meeting with them will help.09:10
evrardjpfor now, let's continue on what we have.09:11
evrardjpthe future can change if required.09:11
evrardjpwe have other areas to focus our energy09:11
odyssey4mechanging the release model is something that we should choose to do, not be forced to do if the tooling can't cope09:11
evrardjpyes. But it's all about impact. I might be willing to change if tooling _helps us_ more09:12
odyssey4meyep09:12
evrardjpbecause our customers won't see any impact, as far as I am aware09:12
evrardjpso the only things that mattered to me was the outside processes: what is the full impact on the openstack namespace, and others09:13
evrardjpanyway09:13
odyssey4meI think it makes sense for us to moving towards releasing the infra roles independently, but the openstack service roles will have to stick to the same cycles as their upstream counterparts09:13
evrardjpodyssey4me: I think it's time we split the a-r-r09:13
evrardjpfor clarity09:13
evrardjpbecause that's our main integration point09:14
evrardjpthat's what we use for releasing too09:14
erolgevrardjp, http://paste.openstack.org/show/719358/ when i compare logs, realize that there is no any whl package in repo. should it be? can i use origin?09:14
evrardjpa-r-r should now become multiple files09:14
odyssey4mebut to switch to that model we will need to figure out how we do tests for multiple ansible versions, and we'll have to decide how we do branching/releasing09:14
evrardjpone for infra (one sort of deliverables), one for external roles, one for openstack validated things (following cycle-trailing), one for openstack pending things (cycle trailing)09:14
evrardjpodyssey4me: splitting the a-r-r doesn't impose us anything09:15
odyssey4meif you want, sure - I'm not sure I see the value in splitting them out09:15
evrardjpbut sets the clarity09:15
evrardjpit also brings the ability for a deployer to override the variable by appending their a-r-r instead of replacing it09:16
odyssey4meoh sure, but we can do that without splitting them out09:16
odyssey4mewe can also just add notes in the a-r-r file09:16
evrardjpyes we could09:16
odyssey4methere are many ways to add annotations if that's what you're after09:16
evrardjpwell09:17
evrardjpI thought that was clearer but you're maybe right, comments and parsing properly would suffice.09:17
odyssey4mewe could easily patch the ansible bootstrap script to treat https://github.com/openstack/openstack-ansible/blob/master/scripts/bootstrap-ansible.sh#L26 as a list, instead of a single file09:18
evrardjpit's just easier to see if it's a different file09:18
evrardjpodyssey4me: yes I know that part09:18
evrardjpI am just taking your position right now: "What do we gain"09:18
evrardjpI have the impression it's more readable to have more files09:18
evrardjpand to rely less on parsing stuff.09:19
evrardjp(if it's a comment telling it's an infra role, do x, else if it's an openstack role do y, else do z)09:19
evrardjphere it's just: open that file (convention), do x09:19
evrardjpbut I am torn between the two, we're gonna see a proliferation of those files09:20
* hwoarang senses there are is an important discussion in progress..09:20
evrardjphwoarang: maybe I should put it into today's agenda then :)09:21
odyssey4meso yes, from a tooling standpoint it's probably easier to have multiple files09:21
odyssey4mebut from a human standpoint that means I have to work through multiple files to find what I want, which is not always nice09:21
evrardjpodyssey4me: maybe there is a mix and match happening there09:22
evrardjpwe have those special files09:22
evrardjpand we have a merge tool09:22
evrardjpputting everything into a file09:22
evrardjpno that's even worse.09:22
odyssey4meyeah09:23
evrardjpmaybe we stop pinning.09:23
evrardjphaha, who doesn't love living dangerously09:23
odyssey4mewe did discuss changing how we do releases at the PTG, we did agree that the infra roles should move towards independent releasing... but to make that shift is going to take some work, and I'm not sure that we have enough motivation to make that switch right now09:24
odyssey4mewe have a lot of initiatives on the table right now, many of which are only part way done09:24
odyssey4meI need to finish the two on my table - the build process simplification, and the use of the embedded files instead of templates09:25
odyssey4methe nspawn jobs are failing consistently, so I think we need to fix whatever's broken there and then make that voting09:26
evrardjpI will finish the inventory lib in the next days, and we can use that to reform the inventory09:26
evrardjpyes I agree09:26
odyssey4methe config template split out seems to be stalled - I don't know what the delays are there09:26
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Do not log passwords  https://review.openstack.org/56184709:29
evrardjpit's split and we have an extra repo09:31
evrardjpnow we need to include it09:31
evrardjpand remove it from the plugins repo09:31
*** cshen has joined #openstack-ansible09:31
evrardjpI will work on that too09:31
*** udesale__ has joined #openstack-ansible09:32
*** udesale_ has quit IRC09:35
hwoarangevrardjp: if you are curious the keystone distro tempest failures are here http://logs.openstack.org/08/560308/2/check/openstack-ansible-keystone-distro_install-opensuse-423/61d673a/job-output.txt.gz#_2018-04-17_09_07_43_64515409:35
hwoarangi will try to make some sense of them09:35
evrardjpthanks hwoarang09:36
evrardjp50209:37
evrardjpthat sounds bad09:37
evrardjphwoarang: http://logs.openstack.org/08/560308/2/check/openstack-ansible-keystone-distro_install-opensuse-423/61d673a/logs/openstack/keystone1/keystone/keystone.log.txt.gz#_2018-04-17_08_58_00_53709:38
evrardjphwoarang: that probably doesn't help09:38
hwoarangi am not very familiar with this area so maybe tempest tries do use an API that queens keystone doesn't support09:38
hwoarangdon't know09:39
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: templates: keystone-uwsgi: Explicitly load the required uwsgi plugins  https://review.openstack.org/56030709:39
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: [RFC] Add support for using distribution packages for OpenStack services  https://review.openstack.org/56030809:39
odyssey4mejamespage oh dear, it looks like the lxd driver broke again - or perhaps we're doing something dumb... http://logs.openstack.org/54/559354/8/check/openstack-ansible-lxd-ubuntu-xenial/74d9b24/logs/host/nova/nova-compute.log.txt.gz#_2018-04-16_19_27_02_22709:39
evrardjpodyssey4me: maybe we need a bump09:40
jamespagenope I expect the nova team moved something around and nova-lxd needs to catchup09:40
jamespagetinwood: any chance you can check ^^ ?09:40
evrardjpah... nova.09:40
odyssey4meevrardjp this is on a role test, so no bumping required09:40
tinwoodjamespage, sure.  I'll take a look.09:41
odyssey4mejamespage yeah, I suspect they might have finally removed the nova network code?09:41
hwoarangoh scary OSA email09:41
evrardjphwoarang: haha.09:41
evrardjphwoarang: did you see my link with the permissions?09:41
evrardjphwoarang: I didn't want to be scary09:41
hwoarangi didn't read it yet it's just long ;p09:42
hwoarangi need to wait for the coffee to be ready and then go through it09:42
evrardjphaha09:42
evrardjptake a double!09:42
tinwoododyssey4me, is this on master?09:42
odyssey4metinwood yep09:42
tinwoododyssey4me, kk -- I'll take a look and see what's moved.09:42
odyssey4mesomething has changed since last week, because this all worked last week09:43
*** hamza21 has quit IRC09:43
odyssey4meyep, the last job passing for our role was on friday09:44
odyssey4meso something has merged to break things between then and now09:44
tinwoododyssey4me,  commit d2166ed072 on nova is the change.  I'll sort something out.09:45
tinwoododyssey4me, looks really simple (I hope)!09:45
odyssey4meah yeah, hopefully just a rename of some things09:46
erolgevrardjp, http://paste.openstack.org/show/719358/ when i compare logs, realize that there is no any whl package in repo. should it be? can i use origin?09:48
odyssey4meerolg hmm, that's odd - although that's 17.0.0... can you update to a later tag?09:52
erolgodyssey4me, to 17.0.2 right ?09:54
evrardjperolg: yup09:54
odyssey4meevrardjp ah yes, that reminds me of the bug we found last week where the wheel build failed, but the task didn't fail... I need to follow that up again and figure out what's going on09:56
erolgevrardjp, can i destroy all of deployment by using OSA cli like kolla-ansible destroy?09:56
odyssey4meerolg you shouldn't need to - just checkout the new tag and run the playbooks09:57
evrardjperolg: we have a playbook.09:57
odyssey4mebut yes, you can wipe out the containers using lxc-containers-destroy.yml09:57
evrardjpnot sure what kolla-ansible does.09:57
evrardjpodyssey4me: or filing a bug09:57
evrardjp:)09:57
*** cjloader has joined #openstack-ansible10:00
odyssey4mecores, can I get some reviews for https://review.openstack.org/#/q/owner:jesse-pretorius+status:open+label:Verified%253E%253D1%252Czuul+NOT+label:Workflow%253C%253D-1+NOT+label:Code-Review%253C%253D-1 please10:00
odyssey4meevrardjp those patches for not logging passwords - do you intend to port them back into the stable branches?10:01
evrardjpodyssey4me: I don't know yet I want the lint test to merge first10:02
evrardjpoh it's merged10:03
evrardjpI will do the backports10:03
evrardjpodyssey4me: talking about cherry picks: https://review.openstack.org/#/c/559725/ ?10:04
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible master: Improve Congress scenario testing  https://review.openstack.org/56083910:06
tinwoododyssey4me, fyi https://review.openstack.org/#/c/561864/ -- just need it to pass gate, etc.10:08
*** cjloader has quit IRC10:12
evrardjptinwood: that's how it starts ... and then down the rabbit hole? :)10:15
evrardjptinwood: thanks for the patch :D10:15
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/queens: Add a rule to prevent passwords to be logged  https://review.openstack.org/56186510:16
tinwoodevrardjp, np :)  ... feel free to ping on future nova-lxd things too. :)10:16
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-galera_server stable/queens: Do not log passwords  https://review.openstack.org/56186610:16
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-ops stable/queens: Do not log passwords  https://review.openstack.org/56186710:16
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_monasca-agent stable/queens: Do not log passwords  https://review.openstack.org/56186810:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_molteniron stable/queens: Do not log passwords  https://review.openstack.org/56186910:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_cloudkitty stable/queens: Do not log passwords  https://review.openstack.org/56187010:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_cloudkitty stable/queens: Do not log passwords  https://review.openstack.org/56187110:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_watcher stable/queens: Do not log passwords  https://review.openstack.org/56187210:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_tacker stable/queens: Do not log passwords  https://review.openstack.org/56187310:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/queens: Do not log passwords  https://review.openstack.org/56187410:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_rally stable/queens: Do not log passwords  https://review.openstack.org/56187510:17
odyssey4meawesome, thanks tinwood and jamespage :)10:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone stable/queens: Do not log passwords  https://review.openstack.org/56187610:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/queens: Do not log passwords  https://review.openstack.org/56187710:18
tinwoodnp10:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_barbican stable/queens: Do not log passwords  https://review.openstack.org/56187810:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Do not log passwords  https://review.openstack.org/56187910:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Do not log passwords  https://review.openstack.org/56188010:18
evrardjpodyssey4me: apparently I made up my mind and backported to queens.10:20
evrardjpmaybe I should do P too10:21
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/pike: Add a rule to prevent passwords to be logged  https://review.openstack.org/56188210:21
odyssey4meyeah, I think to P will do10:21
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Do not log passwords  https://review.openstack.org/56188310:21
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_barbican stable/pike: Do not log passwords  https://review.openstack.org/56188410:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/pike: Do not log passwords  https://review.openstack.org/56188510:22
odyssey4meI'd rather be encouraging anyone using ocata to upgrade to P. Q is still quite new, so it'll take another month or two before usage starts picking up.10:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone stable/pike: Do not log passwords  https://review.openstack.org/56188610:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_rally stable/pike: Do not log passwords  https://review.openstack.org/56188710:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/pike: Do not log passwords  https://review.openstack.org/56188810:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_tacker stable/pike: Do not log passwords  https://review.openstack.org/56188910:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_watcher stable/pike: Do not log passwords  https://review.openstack.org/56189010:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_molteniron stable/pike: Do not log passwords  https://review.openstack.org/56189110:23
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Do not log passwords  https://review.openstack.org/56189210:23
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-ops stable/pike: Do not log passwords  https://review.openstack.org/56189310:23
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Do not log passwords  https://review.openstack.org/56189610:26
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-galera_server stable/pike: Do not log passwords  https://review.openstack.org/56189710:29
*** armaan has quit IRC10:31
*** armaan has joined #openstack-ansible10:31
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-specs master: Allow non-priviledged users to configure OSA  https://review.openstack.org/55910510:34
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-specs master: Allow non-priviledged users to configure OSA  https://review.openstack.org/55910510:35
hwoaranghow did this  'not log password'  error appear?10:37
hwoarangdid we bump ansible-lint or something?10:37
*** nicolasbock has joined #openstack-ansible10:37
odyssey4mehwoarang https://review.openstack.org/56040510:42
hwoarangok so we did it ;p10:42
openstackgerritAndy McCrae proposed openstack/ansible-config_template master: Set py3 job to be voting  https://review.openstack.org/56190010:44
*** armaan has quit IRC10:46
*** armaan has joined #openstack-ansible10:46
*** armaan has quit IRC10:46
*** armaan has joined #openstack-ansible10:47
openstackgerritMerged openstack/openstack-ansible-ops stable/pike: Do not log passwords  https://review.openstack.org/56189310:48
openstackgerritMerged openstack/openstack-ansible-os_molteniron stable/pike: Do not log passwords  https://review.openstack.org/56189110:50
openstackgerritMerged openstack/openstack-ansible-os_molteniron stable/queens: Do not log passwords  https://review.openstack.org/56186910:50
openstackgerritMerged openstack/openstack-ansible-ops stable/queens: Do not log passwords  https://review.openstack.org/56186710:51
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Do not log passwords  https://review.openstack.org/56189210:58
evrardjphwoarang: where do you see it? You have an issue somewhere?11:02
*** cjloader has joined #openstack-ansible11:02
evrardjpmaybe I missed some of those11:02
evrardjpandymccr: did you edit the jobs.yaml? :)11:03
hwoarangevrardjp: no i was curious why we are seeing this lint warning now11:05
hwoarangsince we didn't bump ansible-lint11:05
hwoarangi forgot that we have our own rules as well11:06
openstackgerritMerged openstack/openstack-ansible-plugins master: Add test for conditional delegation  https://review.openstack.org/56059511:06
*** cshen has quit IRC11:07
*** cjloader has quit IRC11:07
*** mgariepy has quit IRC11:09
evrardjpyup I'd rather we add rules for consistency11:09
evrardjpelse we'll always forget11:09
openstackgerritMerged openstack/openstack-ansible-galera_server stable/queens: Do not log passwords  https://review.openstack.org/56186611:10
openstackgerritMerged openstack/openstack-ansible-os_rally stable/queens: Do not log passwords  https://review.openstack.org/56187511:14
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty stable/queens: Do not log passwords  https://review.openstack.org/56187111:15
*** TobbeCN has quit IRC11:16
erolgevrardjp, i got fail again while setup infra but this time wheels are installed. But playbook want to achieve requirements by using internal_vip, but the ip in utillity container and repos are in repo_container :) you guys do you have any opinion abut that?11:16
*** TobbeCN has joined #openstack-ansible11:17
*** TobbeCN has quit IRC11:17
*** TobbeCN has joined #openstack-ansible11:17
erolghttp://paste.openstack.org/show/719371/11:18
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: DNM - Testing ARA upgrades  https://review.openstack.org/53736011:19
hwoarangevrardjp: https://review.openstack.org/#/c/560382/ this is not enough11:21
hwoarangthere are a bunch of other playbooks that fail11:21
hwoarangfor example http://logs.openstack.org/08/560308/3/check/openstack-ansible-linters/26596fb/job-output.txt.gz#_2018-04-17_09_46_11_41266511:21
hwoarangwas that on purpose? or do you have another WIP patchset?11:21
DimGRadmin0 sorry i was away11:21
hwoarangotherwise i can fix that11:21
openstackgerritMerged openstack/openstack-ansible-os_tempest stable/queens: Do not delegate to omit  https://review.openstack.org/56105911:22
*** mgariepy has joined #openstack-ansible11:23
*** armaan has quit IRC11:25
*** armaan has joined #openstack-ansible11:25
openstackgerritMerged openstack/openstack-ansible-os_watcher stable/pike: Do not log passwords  https://review.openstack.org/56189011:28
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/ocata: Update tests a-r-r to use newton-eol tag  https://review.openstack.org/56115211:30
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty stable/queens: Do not log passwords  https://review.openstack.org/56187011:30
openstackgerritMerged openstack/openstack-ansible-os_neutron stable/ocata: Update tests a-r-r to use newton-eol tag  https://review.openstack.org/56115011:30
*** ansmith has quit IRC11:31
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/queens: Do not log passwords  https://review.openstack.org/56186811:31
*** mgariepy has quit IRC11:31
openstackgerritMerged openstack/openstack-ansible-os_rally stable/pike: Do not log passwords  https://review.openstack.org/56188711:31
admin0DimGR, welcome back11:32
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/queens: Do not log passwords  https://review.openstack.org/56187311:34
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/pike: Do not log passwords  https://review.openstack.org/56188911:34
*** udesale__ has quit IRC11:34
odyssey4mehwoarang yeah, we're going to see many more pop up and will have to deal with them11:34
odyssey4meit's a good rule, but of course it's going to cause lint failures now where things were not cleaned up11:34
openstackgerritMerged openstack/openstack-ansible-os_watcher stable/queens: Do not log passwords  https://review.openstack.org/56187211:36
openstackgerritMerged openstack/openstack-ansible-os_nova stable/ocata: Fix kernel post-installation script  https://review.openstack.org/56121911:38
*** smorant has joined #openstack-ansible11:38
*** TobbeCN has quit IRC11:39
*** TobbeCN has joined #openstack-ansible11:40
admin0DimGR, so can u give me a brief rundown on how to use this magnum11:41
admin0 i have it installed ..  have the core-os image, created a template as well ( gui failed, cli worked )11:41
admin0when you have time, do buzz me11:41
openstackgerritMerged openstack/openstack-ansible-os_swift stable/ocata: Update tests a-r-r to use newton-eol tag  https://review.openstack.org/56115111:41
*** neith has joined #openstack-ansible11:42
*** TobbeCN has quit IRC11:42
*** TobbeCN has joined #openstack-ansible11:43
*** dcdamien has joined #openstack-ansible11:43
dcdamienHi11:43
dcdamienWhat's the reason that VPNaaS dashboard isn't deployed with OSA?11:43
openstackgerritMerged openstack/openstack-ansible-os_nova stable/ocata: Add always tag where it's needed  https://review.openstack.org/55904311:44
*** armaan has quit IRC11:44
*** armaan has joined #openstack-ansible11:45
odyssey4medcdamien probably because no-one ever pushed up a patch to make it deploy11:45
*** cshen has joined #openstack-ansible11:45
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: Do not log passwords  https://review.openstack.org/56191711:48
openstackgerritMerged openstack/openstack-ansible-tests stable/queens: Do not log passwords  https://review.openstack.org/56187411:50
erolgodyssey4me, i got fail again while setup infra but this time wheels are installed. But playbook want to achieve requirements by using internal_vip, but the ip in utillity container and repos are in repo_container :) you guys do you have any opinion abut that? http://paste.openstack.org/show/719371/11:56
admin0anyone has magnum installed and has seen this type of error before: Create for cluster web-cluster failed: Cluster type (vm, core-os, kubernetes) not supported (HTTP 400)12:01
*** cjloader has joined #openstack-ansible12:02
*** blinkiz has joined #openstack-ansible12:04
*** cjloader has quit IRC12:06
blinkizHello. Am trying to install 17.0.2 for the second time on Ubuntu 16.04. Again it hangs on "TASK [pip_install : Install PIP]" when running "openstack-ansible setup-hosts.yml". Looking at the process with strace and I can see that it just waits for something, timeout.. Known problem? How to resolv? Yesterday I tried with 17.0.1 and it stopped at the same task.12:07
blinkizI think it is installing PIP inside the LXC containers. I did "export SCENARIO='aio_lxc'" before running scripts/bootstrap-aio.sh12:07
odyssey4meblinkiz that's the default, so you didn't have to do that - and sure, that task is trying to install pip using the get-pip.py script.... if it's hanging then there's most likely some sort of networking issue, like a bad MTU or something12:09
blinkizhere is the last lines visible. https://paste.ubuntu.com/p/GPtQJxbq27/12:09
blinkiz"get-pip.py" I can find in /opt/get-pip.py.12:13
*** armaan has quit IRC12:14
*** armaan has joined #openstack-ansible12:14
*** exodusftw has quit IRC12:14
odyssey4meblinkiz yes, but that's a very small file - the files it will try to download are larger, which is why I suspect an MTU issue somewhere in the traffic path12:14
*** dave-mccowan has joined #openstack-ansible12:15
blinkizodyssey4me, here is strace on this process: https://paste.ubuntu.com/p/kpC3vhdwYs/12:16
odyssey4methat doesn't tell me anything - does it tell you anything?12:16
blinkizHow can I troubleshoot network issues? What is it trying to download?12:16
odyssey4methis is the task it's running: https://github.com/openstack/openstack-ansible-pip_install/blob/master/tasks/install_source.yml#L1712:17
odyssey4mehere's an example of what it's doing: http://logs.openstack.org/88/560388/1/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/e050be5/logs/ara-report/result/4b611e21-c6ee-4640-9fa9-8d181da6693f/12:18
*** armaan has quit IRC12:19
odyssey4menot that yours will be different - you'll be able to see the command if you increase the output verbosity when executing the playbook12:20
blinkizodyssey4me, thank you for the information. Running get-pip.py and it has problem connecting to something. Firewall with ufw is enabled so disabling it will probably solve it. Lets try...12:20
blinkizhttps://paste.ubuntu.com/p/2V4pHwGsSx/12:20
blinkizodyssey4me, yep, solved it. Thanks for helping me find the problem12:21
*** exodusftw has joined #openstack-ansible12:21
odyssey4mesure, happy to help12:23
*** markvoelker has quit IRC12:25
*** markvoelker has joined #openstack-ansible12:25
*** mgariepy has joined #openstack-ansible12:26
admin0DimGR, thanks .. its working now12:26
admin0i had my keypair  and image issue12:26
admin0magnum is working :)12:26
DimGRhappy to help , magnum is very powerfull12:26
admin0DimGR, how to price this stuff :) how is usage calculated .. is it less than a normal vm .. or more than a normal vm ?12:27
admin0though company is private, i still need to do some internal billing for accounting12:27
DimGRfor that , i can not help , just say magnum is priceless :)12:28
*** armaan has joined #openstack-ansible12:29
admin0:D12:29
*** Tbone has joined #openstack-ansible12:32
TboneAnyone happy to assist me with Neutron networking ? Really struggling to figure out why my external network is not working. Internal networks are working 100% with DVR12:33
admin0DimGR, normally how long does a cluster take to create ?12:34
Tboneusing kolla-ansible for deployment12:34
DimGRin my hardware to fire up 4 instances it took 15min i think12:34
DimGRif it takes a lot of time  ,  check heat logs12:34
admin0oh12:35
admin0then i wait :D12:35
odyssey4meTbone perhaps you should ask in the kolla channel then.12:35
Tboneodyssey4me: its not really a kolla-ansible issue. It feels like a Neutron, openvswitch issue12:39
Tboneand i'm not sure if there are additional ML2 configs which I need to update12:39
*** koukou has joined #openstack-ansible12:44
*** ansmith has joined #openstack-ansible12:48
*** sawblade6 has quit IRC12:52
mgariepymorning everyone12:54
odyssey4memgariepy ah, long time no see!12:58
*** sawblade6 has joined #openstack-ansible12:58
mgariepyodyssey4me, yep indeed.12:59
mgariepyhow are things on your side ?12:59
odyssey4menot too bad, things are moving as always12:59
mgariepywould be boring otherwise12:59
mgariepyare you going to vancouver ?13:00
*** cjloader has joined #openstack-ansible13:02
odyssey4meI won't be. I do enjoy Vancouver as a city, so I will miss it.13:02
*** smorant has quit IRC13:03
mgariepyha :(13:03
odyssey4meit's near enough for you, so I guess you'll be going?13:03
*** armaan has quit IRC13:04
*** armaan has joined #openstack-ansible13:04
mgariepy5k km. yeah close enough13:04
mgariepyyep i can go, my boss want to send me. and i'll have some colleague going as well.13:06
mgariepyjust need to plan this, with plane hotel and stuff13:06
matttmgariepy: who do you work for in quebec, if you don't mindm e asking?  :)13:06
mgariepyuniversity of sherbrooke, research cluster13:07
*** cjloader has quit IRC13:07
matttmgariepy: oh wow, that's pretty cool :)13:07
mgariepyyep.13:07
*** ianychoi_ has joined #openstack-ansible13:07
mgariepywhat's about you ?13:07
mnasermgariepy: the foundation has some crazy discounts going on right now13:08
matttmgariepy: i work w/ jesse and evrardjp13:08
*** ianychoi has quit IRC13:10
dcdamienI'm testing installation of VPNaaS dashboard. I've added git repo and specified package in horizon_optional_pip_packages variable. After running `repo-build -e repo_build_wheel_rebuild=yes` there is no wheel built. Any ideas?13:14
dcdamiengit clone task works fine - repo is cloned13:14
*** cmart has joined #openstack-ansible13:17
evrardjpnot sure if that's in optional_pip_packages13:18
mgariepymattt, didn't know rs was hiring remote workers :)13:18
mgariepymnaser, are you going to vancouver ?13:18
mnaserYep13:18
mnaserWe’re sponsoring the summit too!13:19
matttmgariepy: oh yeah, most of our engineering team is remote13:19
mgariepyha didn't knew that.13:20
dcdamienevrardjp is there something else required to build additional package?13:21
TboneIs this still required  enable_neutron_provider_networks: "yes"  ???13:22
*** armaan has quit IRC13:24
*** armaan has joined #openstack-ansible13:24
*** armaan has quit IRC13:25
*** armaan has joined #openstack-ansible13:25
*** armaan has quit IRC13:27
*** armaan has joined #openstack-ansible13:28
evrardjpTbone: not sure what you mean13:30
evrardjpdcdamien: it's vpn aas?13:31
evrardjphttps://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L141-L14313:31
evrardjpit should be alright13:31
Tboneevrardjp: https://docs.openstack.org/kolla-ansible/latest/reference/networking-guide.html says that in order to make use of provider networks you need to enable it in globals.yml13:31
evrardjpdcdamien: oh the dashboard13:32
dcdamienevrardjp: yep ^^13:32
Tbonehowever, its not there by default13:32
evrardjpdcdamien: well, as you can see it's missing something like that in the integrated: https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L226-L22813:32
evrardjpso if you have cycles, you should add it13:32
evrardjp:D13:32
odyssey4meTbone as suggested earlier, it's probably better discussed in the kolla channel - this channel is attended by contributors and consumers of the openstack-ansible project, not kolla13:32
evrardjpthat would allow the wheels to be built in the repo dcdamien13:33
Tboneodyssey4me: no problem13:33
evrardjpthen you can say in horizon "install this package"13:33
hwoarangevrardjp: after a bit of investigation i found it that it seems to be a haproxy issue (keystone+tempest). The haproxy backend is at maintenance mode when tempest is running13:33
evrardjpdcdamien: for that we have this: https://github.com/openstack/openstack-ansible-os_horizon/blob/master/vars/main.yml13:33
hwoarangonce it's up, if you run tempest again, everything works ok13:33
hwoarangnot sure how to 'trigger' haproxy to notice that the backend is up13:34
hwoarangi am looking into it13:34
evrardjpdcdamien: look at this for example: In horizon you have to wire the newly installed package, like for fwaas.13:34
odyssey4meevrardjp well, if the package is available on pypi that's not necessary - assuming you're happy to use the one from pypi13:34
evrardjphttps://github.com/openstack/openstack-ansible-os_horizon/blob/master/vars/main.yml#L10013:34
evrardjpthat needs doing too13:34
evrardjpdcdamien: look at what odyssey4me  just said. It's valid too :)13:35
evrardjpbut wiring still need to happen in the role. Please check at the dashboard's documentation, because they all do something different.13:35
evrardjphwoarang: it should be done automatically by default13:35
evrardjphwoarang: however, we have some handlers for putting things in maintenance mode13:36
*** armaan has quit IRC13:36
evrardjpthat might be a notify you should add13:36
dcdamienevrardjp: I've already added repo. It's cloning correctly in repo-build but wheel is not building for some reason.13:36
*** armaan has joined #openstack-ansible13:36
dcdamienwondering why13:36
*** esberglu has joined #openstack-ansible13:36
evrardjpdcdamien: I think this deserves a more analysed thing. Could you file a bug? With the user overrides and your repo build logs?13:37
odyssey4melooks like it's published to pypi, so what dcdamien did should be fine I think... dcdamien can you output the result of running this on a pastebin or something: /opt/ansible-runtime/bin/python /etc/ansible/roles/plugins/lookup/py_pkgs.py /opt/openstack-ansible /etc/ansible/roles /etc/openstack_deploy13:37
hwoarangevrardjp: can you give me some pointers on where these handlers are?13:38
*** jwitko has joined #openstack-ansible13:38
evrardjphwoarang: you're working on role tests, right?13:38
hwoarangyes13:38
hwoarangthe backend comes back to life eventually but not soon enough for tempest to succeed13:38
evrardjplet me double check because what I had in mind was for integrated13:39
evrardjpoh13:39
*** Tbone has left #openstack-ansible13:39
evrardjphwoarang: https://github.com/openstack/openstack-ansible-tests/blob/master/test-install-keystone.yml#L49-L6413:40
hwoarangoh hmm13:40
hwoarangthanks13:40
evrardjptempest runs just after, so maybe you want to include a delay here:13:41
evrardjphttps://github.com/openstack/openstack-ansible-os_keystone/blob/master/tests/test.yml#L3313:41
evrardjphowever I do not think a delay should be required13:41
evrardjpinstead if would be interesting to know when it gets back online13:41
evrardjpif the raise value shouldn't be lowered.13:42
*** jwitko has quit IRC13:43
*** asettle_ is now known as asettle13:47
odyssey4mehwoarang evrardjp for the keystone role test it's a cluster of 3, so it shouldn't matter if one is offline13:47
odyssey4methe playbook runs in serial: https://github.com/openstack/openstack-ansible-tests/blob/master/test-install-keystone.yml#L3913:47
evrardjpodyssey4me: I guess it can hit a dead node and not retry ?13:48
evrardjpmaybe the haproxy parameters could be changed too13:48
evrardjpI don't know13:48
odyssey4mein the keystone test it uses haproxy, so if haproxy thinks it's offline then haproxy shouldn't be passing data to it13:49
odyssey4meyeah, there may be a config issue there of some kind13:49
evrardjpodyssey4me: that's what I meant by config13:50
evrardjpit's maybe not detecting the downtime fast enough13:50
evrardjpbut I doubt it's that. It's probably something else :)13:50
evrardjpoccam's razor right?13:51
*** udesale has joined #openstack-ansible13:51
*** cjloader has joined #openstack-ansible14:01
*** vakuznet has joined #openstack-ansible14:01
*** rickflare has quit IRC14:02
*** udesale has quit IRC14:02
tomtom001hello14:03
evrardjpDear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle,14:05
evrardjpcolinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok14:05
evrardjpHere is our agenda for today: https://wiki.openstack.org/wiki/Meetings/openstack-ansible14:05
evrardjpthe meeting starts in an hour14:05
evrardjp2 hours?14:05
evrardjp2 hours14:05
evrardjplet me double check14:05
evrardjp(switch to a new agenda)14:06
evrardjp2 hours.14:06
*** cjloader has quit IRC14:06
*** vakuznet_ has joined #openstack-ansible14:10
tomtom001odyssey4me; my galera cluster crashed do you know any recovery procedures you can share with me?14:10
*** cjloader has joined #openstack-ansible14:11
*** alanmeadows has quit IRC14:11
*** johnsom has quit IRC14:11
*** AndyWojo has quit IRC14:11
*** xgerman_ has quit IRC14:11
*** dxiri has joined #openstack-ansible14:11
*** johnsom has joined #openstack-ansible14:12
*** alanmeadows has joined #openstack-ansible14:12
*** xgerman_ has joined #openstack-ansible14:12
*** vakuznet has quit IRC14:13
cloudnullmornings14:16
*** cristicalin has joined #openstack-ansible14:16
niraj_singhevrardjp: i am getting error in TASK [Create DB for service] fatal: [aio1_masakari_container-2f18b752 -> 172.29.238.60]: FAILED! => {"changed": false, "msg": "The MySQL-python module is required."}14:18
niraj_singhNot able to resolve this issue any clue?14:18
*** cjloader has quit IRC14:18
*** AndyWojo has joined #openstack-ansible14:19
evrardjpniraj_singh: https://github.com/openstack/openstack-ansible-os_keystone/blob/master/defaults/main.yml#L418-L425 that's what's done in keystone. Maybe you could adapt it to your needs?14:21
*** cmart has quit IRC14:21
niraj_singhok14:22
*** dmsimard|off is now known as dmsimard14:23
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible master: Improve Congress scenario testing  https://review.openstack.org/56083914:27
*** Leo_m has joined #openstack-ansible14:32
*** cmart has joined #openstack-ansible14:34
odyssey4meevrardjp we'll need to get https://review.openstack.org/#/c/561847/1 merged - the nova-lxd patch has gone in, so I've just rechecked it14:35
odyssey4metomtom001 not personally, we have some docs in the admin guide14:36
odyssey4menot sure how good they are14:36
evrardjpodyssey4me: do you mind if I add a depends-on for clarification in the commit message?14:39
evrardjpit's been only 5 minutes.14:39
odyssey4meevrardjp I don't really see the point14:39
odyssey4methis change doesn't depend on that one14:39
evrardjpoh my bad.14:40
evrardjpI misread.14:40
hwoarangevrardjp: odyssey4me: got the bottom of it. haproxy is fine just wusgi from distro package segfaults ;p14:40
evrardjpyeah no point14:40
evrardjpsorry14:40
evrardjpbrain busy to something else :)14:40
hwoarangand is restarted automatically that's why only half of tempest fails...14:40
evrardjpI think I missed a part of your message hwoarang14:41
mgariepytomtom001, this one should be ok-ish .https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks.html14:42
mgariepywith systemd you need to use the "systemctl set-environment" command stuff  which is not present in the docs for other release14:44
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/queens: [DOC] rectify galera recovery for systemd  https://review.openstack.org/56196714:45
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/pike: [DOC] rectify galera recovery for systemd  https://review.openstack.org/56196814:45
*** cjloader has joined #openstack-ansible14:47
hwoarangevrardjp: ok so what i said was that haproxy is fine but uwsgi from distro packages segfaults14:47
hwoarangand restarted midway through the tempest run14:47
*** cjloader has quit IRC14:48
hwoarangso i am looking into it14:48
*** cristicalin has quit IRC14:48
*** cjloader has joined #openstack-ansible14:48
*** cristicalin has joined #openstack-ansible14:49
openstackgerritMerged openstack/openstack-ansible-os_barbican stable/pike: Do not log passwords  https://review.openstack.org/56188414:50
evrardjphwoarang: OOMing?14:53
hwoarangdoesn't look like it14:53
evrardjpor just segfaults14:53
evrardjpok14:53
evrardjp:(14:53
hwoarangyeah...14:53
evrardjpso it's not the folder permissions?14:53
evrardjpsad14:53
hwoarangdont know. it starts, it serves some requests and then dies14:54
hwoarangwill see14:54
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Do not log passwords  https://review.openstack.org/56188514:55
openstackgerritMerged openstack/openstack-ansible-tests stable/queens: Do not log passwords  https://review.openstack.org/56187714:55
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Do not log passwords  https://review.openstack.org/56188814:55
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_ironic master: update for depricated driver definition  https://review.openstack.org/56127714:57
prometheanfireevrardjp: ^ not sure I did the 'or' correcly in the when statement in the set fact, but I guess gate will tell14:57
*** cshen has quit IRC14:59
*** f5guy has joined #openstack-ansible15:00
*** weezS has joined #openstack-ansible15:04
*** TobbeCN has quit IRC15:04
mmercermorning15:05
evrardjpmorning15:05
*** TobbeCN has joined #openstack-ansible15:05
mmercermore activity in the past 12 hours than ive seen in two days =D15:05
f5guyHi, I'm looking for guidance on a test pike installation using OSA, running the last commit as of this morning. All playbooks complete however the single compute node is never discovered when listing the nova services. I have attempted adding the compute node again without success. I thought that I may have hit bug #176272, I redid the deployment on an infra node with the same behavior. For the discovery process to complete, is15:05
f5guythe only requirement the ability to reach the api container?15:05
openstackbug 176272 in Ubuntu "[needs-packaging] Freeverb3" [Wishlist,Confirmed] https://launchpad.net/bugs/17627215:05
*** armaan has quit IRC15:05
evrardjpprometheanfire: let's say we agree to disagree on the cleanup of the code :) I think it's better to have less variables, and more 'profiles' but I guess my mind can be changed15:06
evrardjpmmercer: not many people work on the week-ends :)15:06
*** armaan has joined #openstack-ansible15:06
prometheanfireevrardjp: I'd like things to be more flexible15:07
*** jwitko has joined #openstack-ansible15:07
* prometheanfire hates how zoom steals focus15:07
prometheanfireand less busywork :P15:07
mmercerso i have an oa-aio install,  and im working on the second part (infrastructure.yml),  originally it was crashing because i must have overlooked stopping and masking firewalld,  now it gets to galera_client,  and repo container fails to be able to connect to mariadb.  i attached to the console,  it has a 10.255.255.77 as its primary address with default route attached,  but i dont appear to be able to reach any external15:08
mmercerinterfaces, so i decided to check iptables and similar.... what should be forwarding the nat traffic for the aio container ?15:08
mmercerevrardjp: wasnt yesterday monday everywhere ?15:08
evrardjpf5guy: I don't know. We've changed things recently, maybe worth asking to the the latest patch people :)15:08
evrardjpmnaser: mondays!15:08
evrardjpwoops15:08
evrardjpmmercer: mondays*15:08
*** ashak_ has quit IRC15:09
evrardjpit was quite a busy monday for me, I wasn't just active on the channel15:09
mmercergotcha15:09
mmercerits irc, i expect it to fluctuate,  was just surprised by the level of quietness yesterday for such a big project =D15:10
evrardjpmmercer: yes15:10
evrardjpthe lxc containers, by default, need the dnsmasq + nat to reach internet15:10
evrardjpthat's standard lxc15:10
mmercerright, which firewall is supposed to be handling the nat ?15:10
evrardjphowever, to reach internal networks, you can do whatever you want15:10
mmerceriptables?  ebtables?15:10
evrardjpthat would be iptables I guess.15:11
mmercerhrm15:11
evrardjpI haven't checked into what lxc is doing behind the scenes, but that must be it.15:11
evrardjpelse it's maybe nftables?15:11
mmercerdoesnt seem like thats handled in infrastructure, must be part of the bootstrap,  will check15:11
mmercernftables15:11
mmercer?15:11
mmercerno nftables present15:11
evrardjpmmercer: I think that's not an ansible task15:11
f5guyevardjp: Are these patch people you speak of in a different channel or is the mailing list perhaps a better place to ask? The only interesting log I see anywhere is on the nova-api-metadata container complaining about connection resets from the infra1 node. I'm guessing this is actually haproxy monitoring...15:12
evrardjpmmercer: I mean I am sure it's not an ansible task15:12
evrardjpmmercer: the dnsmasq part is when installing lxc15:12
evrardjpthat's default15:12
evrardjpcomes for free15:12
evrardjp:)15:12
evrardjplike you would on docker15:12
mmercerevrardjp: the openstack scripts cover all of that as far as i know,  there is the bootstrap machine, bootstrap openstack, etc15:12
*** TobbeCN has quit IRC15:12
mmercereither that or the docs are missing some pretty important steps =D15:12
evrardjpmmercer: I am confused.15:13
odyssey4meevrardjp mmercer we also drop this script onto the lxc hosts which helps fix things up: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/templates/lxc-system-manage.j215:13
evrardjpoh its our script then15:13
evrardjpgood15:13
evrardjpthat's the script.15:13
mmercergot it, looking to see which part executes that15:14
evrardjpmmercer: the docs isn't describing every single task. That would be getting out of date very quick.15:14
evrardjpf5guy: please file a bug that's the easiest15:14
mmercerevrardjp: im not expecting it to document every task,  but the steps are pretty minimal -- bootstrap, bootstrap, yml, yml, yml15:14
evrardjpmmercer: I still don't understand15:15
evrardjphttps://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/overview.html15:15
evrardjpthe deployment guide: https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/  should be good enough to understand the big steps15:15
evrardjpwhat's wrong with that documentation?15:16
*** DanyC has quit IRC15:16
mmercerevrardjp: its missing a few steps, and as a result, the playthrough broke,  so im going back to find *where* it broke15:16
mmercerthis is the one im following (linked from that very page) https://docs.openstack.org/openstack-ansible/latest/user/aio/quickstart.html15:17
odyssey4memmercer those steps work for most people, and are the tested steps we use for validating all code - however, ymmv based on extra things you have on your hosts15:17
evrardjpmmercer: I doubt there is step missing. But please file a bug with the steps you think missing.15:17
odyssey4meso yeah, it'd be interesting to understand what caused a problem and what you suggest be added to cater for that for anyone that hits it15:18
evrardjpmaybe we should say "if you want more understanding on how a deploy works, have a look at the deploy guide"15:18
mmercerevrardjp: there is another page for aio installation that has more steps that are not present in the main list, including stopping for instance NetworkManager, etc,  the new one does not include that15:19
f5guyevardjp: OK, I've filed a few already, it seems like they are ignored even though valid. Someone has had to have run into this, especially because I'm using Pike. Perhaps a better question, is there a way to do a discovery of a compute node manually without an ansible task involved? The old nova-manage binary doesn't seem present on my utility container...15:19
evrardjpmmercer: what do you mean?15:19
evrardjpmmercer: could you link me to it?15:19
mmercerone sec, im looking for it15:19
evrardjpnetwork manager? That's not shipped by default15:19
*** kstev has joined #openstack-ansible15:20
odyssey4mef5guy nova-manage is only on your nova hosts - you should be using the openstack cli in the utility container15:20
evrardjpyou had to install network manager. We assume a clean node.15:20
evrardjpminimal node.15:20
odyssey4memmercer the aio is more of a developer tool, which is why it describes more options15:21
evrardjpmmercer: what would be nice, is that you'd say all the trouble you went through in a bug. We can then decide what needs improvement in the docs.15:21
odyssey4methe deploy guide actually does share the same steps, but they're less specific because it has to be general... the aio is a very well known config implementation15:21
evrardjpall input is good to take15:22
mmercerevrardjp: im working on it,  give me a second, jeesh.  i already asked the part i need help with.  the nat rules are not present on my system, it rebooted at some point (i think one of the it guys knocked power out),  it came back up and had firewalld running, i stopped that,  it got me past the pip issues i was having15:22
mmercerbut as of right now, i am not seeing any nat forward rules in iptables15:22
odyssey4mef5guy what exactly do you mean by 'discovery of a compute node'? for cells?15:22
mmercerwhich would explain why the galera steps are failing (since there is no nat forward, the machine cannot reach outside the network),  now that i know the lxc template step,  i need to dig in and see where that is invoked in the roles so i can re-execute that template15:23
*** RandomTech has joined #openstack-ansible15:25
f5guyodyssey4me: OK, makes sense and I have been. Yes, maybe I am misunderstanding the relation here.. When listing my compute services, I am missing the nova compute instance, all others (conductor, scheduler, consoleauth) are shown and working fine. I thought the cell discover is the process of adding the compute host to this list and making it available for users.15:25
odyssey4mef5guy have you checked whether the nova compute service is running on your compute host?15:26
f5guyodyssey4me: Yes, it is. There are no errors listed in the logs either. Systemctl output looks good.15:27
odyssey4methe playbooks do the discovery for you, and the service is set to auto register new nodes too... so if it's not showing up then there's a fair likelihood that you've got a config issue that's causing nova-compute not to start15:27
*** pcaruana has quit IRC15:28
RandomTechmorning everyone15:28
odyssey4mef5guy if you connect to the nova/conductor container, you can use nova-manage from the venv folder ... /openstack/venvs/nova-.../bin/nova-manage15:29
f5guyodyssey4me: Sure, and I'm able to get this all working fine in my other test lab when not using OSA. I of course prefer OSA for deploying these environments as I will be deploying them monthly. If the nova compute service is running (it is) and I can reach all of my API endpoints, how can I dig further and provide a proper bug report?15:31
f5guyFor instance the only statement in my nova-compute log is: 2018-04-17 10:27:58.443 23395 INFO os_vif [-] Loaded VIF plugins: ovs, linux_bridge15:31
odyssey4mef5guy you might want to turn debug logging on, then restart the service to see if it helps you figure out what's going on15:32
f5guyIt is already on :/15:32
odyssey4methe compute service should be talking through conductor to the placement service if I understand it correctly15:32
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/pike: Do not log passwords  https://review.openstack.org/56188615:32
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Do not log passwords  https://review.openstack.org/56191715:32
odyssey4meyou should see the chatter all the time in the log, especially with debug enabled15:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/queens: Do not log passwords  https://review.openstack.org/56197815:33
f5guyOK I only turned debug logging on the compute node itself15:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/pike: Do not log passwords  https://review.openstack.org/56197915:33
*** tosky has joined #openstack-ansible15:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Re-order database migrations  https://review.openstack.org/55935415:35
evrardjpmmercer: re-run the playbooks. Everything should be done as part of the playbook, technically no step is required on that level.15:36
evrardjpI mean no manual intervention is required.15:36
openstackgerritMerged openstack/openstack-ansible-os_barbican stable/queens: Do not log passwords  https://review.openstack.org/56187815:39
f5guyodyssey4me: Turned on debug logging on all nova-related hosts, nothing. I suppose I could try to do the discover manually to see what happens. I really do appreciate the help.15:41
*** vinsh has joined #openstack-ansible15:43
odyssey4mef5guy that's weird - no activity in the log at all? I wonder if the process isn't hanging or something for some reason15:44
openstackgerritMerged openstack/openstack-ansible-ops master: Add bind mount for logstash and ensure java is installed  https://review.openstack.org/56164415:44
*** foutatoro has quit IRC15:44
f5guyNothing on the compute host; I see logs at startup on the placement container, but no logs after either service has restarted.15:44
*** smorant has joined #openstack-ansible15:45
RandomTechhey, im about to pull openstack pike down, however, the documentation is labeled as 16.0.11 but says to pul 16.0.10. Which should i end up pulling?15:45
vinshHello openstack-ansible folks... working on our OSA pike 16.0.10 deployment here at viasat..  we are hung up on getting glance working.  When I attempt to upload an image to glance.. I can see that glance does a PUT then almost immediately a DELETE of the image before it is created.  I see this same thing when using file or swift as the store for glance.15:46
odyssey4meRandomTech whatever the latest tag is that you see in the git repo15:46
vinshfrom the client side I get CommunicationError: Error finding address for https://use1.viasat.cloud:9292/v2/images/5c1b194f-fbc8-40d9-b0b9-8696fc4a8815/file: (104, 'ECONNRESET')15:46
vinshany ideas?15:46
RandomTechim honestly not sure how to check that15:46
f5guyodyssey4me: Should the compute host be able to ping the api-placement container directly for some reason? It should be going through the haproxy VIP.15:46
vinshglance was up and working in newton.. for us btw. then we rolled ocata and right to pike and glance has been dead-ish since.15:47
odyssey4mef5guy it could, but api interations will go through haproxy - although I can't recall exactly how that communication flows for that particular service15:48
vinsh@RandomTech "git tag" from the osa repo.. and find the latest 16.x tag in that list.15:48
odyssey4mevinsh not sure, not seen that happen except when either haproxy has a different timeout to the back-end service, or the client has a different timeout... or the services are unable to communicate15:49
admin0vinsh, is it NFS /15:49
hwoarangevrardjp: fyi I will miss the beginning of the meeting15:49
hwoarangsorry about that\15:49
vinshadmin0: no, no nfs15:50
*** cristicalin has quit IRC15:50
vinshglance with file store or even swift. you see in swift the glance_images/<imageid>/file is put. then glance just sends a delete right away and bombs out15:50
vinshsame for file store15:50
RandomTechvinsh: would i need to clone the repo to ruun git tag in it?15:51
RandomTechwouldnt*15:51
vinsh@RandomTech Yes, clone it then "cd" into it then "git tag"15:52
f5guyodyssey4me: Weird, from the compute host tcpdump shows it sending a crap-ton of SYNs with no response to the rabbitMQ container but nothing to anything else. I don't understand what impact or why the compute host would be talking directly to this container.15:52
admin0vinsh, debug helps ?15:52
RandomTechvinsh: then clone the latest?15:52
vinshadmin0: debug seems like its ok/happy.  POST /v2/images then.. PUT /v2/images/5c1b194f-fbc8-40d9-b0b9-8696fc4a8815/file ... then DELETE /v2/images/5c1b194f-fbc8-40d9-b0b9-8696fc4a881515:53
vinshon the client side its a lot of that "error finding address..."15:53
vinsh@RandomTech "git clone https://git.openstack.org/openstack/openstack-ansible" "cd openstack-ansilbe" "git tag"15:54
vinshthen maybe 'git checkout 16.0.10" or whatever you want15:55
*** koukou has quit IRC15:55
*** sew has joined #openstack-ansible15:55
*** maybebuggy has quit IRC15:56
odyssey4mef5guy hm, I always thought that compute spoke to conductor, and conductor uses rabbit - if rabbit is being used by the service then it will talk directly to the containers as it handles its own failover15:56
*** johnduarte has joined #openstack-ansible15:56
evrardjpmeeting in 3 minutes.15:57
odyssey4mef5guy if rabbit's not responding, then that'll be a serious problem which will need to be figured out15:57
odyssey4meyou might want to verify whether time is in sync15:57
odyssey4mevinsh does the glance store debug logging help at all?15:58
vinshdoes glance store just run on demand?15:59
vinshnever see any logs from it15:59
vinshdoesn't appear to be running. just glance-api is15:59
*** smorant has quit IRC15:59
mnaser<evrardjp> mnaser: mondays! << even if it was a miss highlight15:59
mnaser"mondays!" is a thing for me too15:59
odyssey4mevinsh glance_store is a library used by glance15:59
odyssey4mevinsh it's not its own service16:00
vinshOk, how do you enable logging for that guy?16:00
evrardjphaha16:00
evrardjpok sorry to interrupt your conversations everyone16:00
evrardjpbut it's meeting time16:00
evrardjpDear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle,16:01
evrardjpcolinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok16:01
evrardjpHere is our agenda for today: https://wiki.openstack.org/wiki/Meetings/openstack-ansible16:01
evrardjp#startmeeting openstack_ansible_meeting16:01
openstackMeeting started Tue Apr 17 16:01:07 2018 UTC and is due to finish in 60 minutes.  The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot.16:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:01
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)"16:01
openstackThe meeting name has been set to 'openstack_ansible_meeting'16:01
evrardjpevrardjp reminded on the ML we need more help on the releases/bug handling/monitoring jobs.16:01
prometheanfiresup16:01
evrardjpwoops forgot the topic16:01
spotzo/16:01
prometheanfireo/16:01
evrardjp#topic what happened since last meeting16:01
*** openstack changes topic to "what happened since last meeting (Meeting topic: openstack_ansible_meeting)"16:01
evrardjpevrardjp reminded on the ML we need more help on the releases/bug handling/monitoring jobs.16:01
*** ethfci has joined #openstack-ansible16:01
evrardjpevrardjp reminded we have pending specs to review16:01
evrardjpevrardjp says he didn't ask for newton repo EOL yet16:01
evrardjpevrardjp reminds there is an etherpad for vancouver forum: https://etherpad.openstack.org/p/YVR-openstack-ansible-brainstorming16:02
evrardjpevrardjp and odyssey4me discussed the releases, as we now have issues with them16:02
f5guyodyssey4me: I will look into it. Dumping on the rabbitmq containers vinterface shows no traffic reaching it, looks like a possible bridging issue between the br-mgmt and vinterface. Tons of other traffic from all other containers and the infra host itself.16:02
evrardjpI will let you a few minutes to process these16:02
evrardjpif you have any other news to add, please add it to the wiki page, as usual.16:03
evrardjp#topic bug triage16:03
*** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)"16:03
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176447016:03
openstackLaunchpad bug 1764470 in openstack-ansible "upgraded get-pip does not install pip in venvs in repo container" [Undecided,New]16:03
*** TobbeCN has joined #openstack-ansible16:04
evrardjpthe 17.0.0 in 17.0.1 is my mistake but it's not fixable, so people should check out next version16:04
evrardjpodyssey4me: do you think there is a place where we don't constraint properly? the first part of the bug seems that it's installing 10.0.016:05
evrardjpit doesn't show the usage of -c either16:06
evrardjpmaybe that an old bug16:06
evrardjpI think that's what we fixed16:06
odyssey4meevrardjp that's likely the issue that was resolved in later patch to constrain get-pip properly?16:07
evrardjpyes16:07
evrardjpthat's what I'd say16:07
d34dh0r53yep16:07
f5guyodyssey4me: fixed it! I deployed the compute host in a VM (for testing), I needed to enable promiscuous mode on the vswitch. Thanks for the guidance.16:07
openstackgerritMerged openstack/openstack-ansible stable/pike: Do not log passwords  https://review.openstack.org/56188316:07
openstackgerritMerged openstack/openstack-ansible stable/pike: Do not log passwords  https://review.openstack.org/56189616:07
openstackgerritMerged openstack/openstack-ansible stable/queens: Do not log passwords  https://review.openstack.org/56188016:07
openstackgerritMerged openstack/openstack-ansible stable/queens: Do not log passwords  https://review.openstack.org/56187916:07
evrardjpI mark it as invalid for now16:08
odyssey4mequeens: https://review.openstack.org/#/c/558127/ should be in 17.0.1 and 17.0.216:08
*** sew has quit IRC16:08
mnasero/16:08
andymccro/16:08
evrardjpbut that looks like repo build16:08
evrardjp/tmp/openstack-venv-builder/venvs/keystone/bin/python16:09
*** TobbeCN has quit IRC16:09
evrardjpI think that's confusing and we don't see the task in question16:09
*** tomtom001 has quit IRC16:10
evrardjpok let's continue16:10
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176401616:10
openstackLaunchpad bug 1764016 in openstack-ansible "openstack-venv-builder Fails PIP not found" [Undecided,New]16:10
evrardjpdo we have dual stack openstack-ansible?16:11
evrardjpmaybe we should just drop python2, and deal with the fallout16:11
evrardjpI am tired of these python2 and python3 issues16:12
prometheanfireevrardjp: you see the py3 thread on -dev?16:12
evrardjpor have multiple jobs16:12
prometheanfirejust this morning16:12
evrardjpyes I saw it16:12
andymccrlike throughout the whole stack?16:13
andymccrjust drop py2 everywhere16:13
prometheanfireswift is py2 only16:14
andymccryeah there are a few i think16:14
evrardjphere I think the python thing is a red-herring though16:14
andymccrprobably16:14
evrardjpok found the problem16:15
evrardjplinked to previous bug16:16
evrardjphttps://github.com/openstack/openstack-ansible-repo_build/blob/master/templates/venv-build-script.sh.j2#L8616:16
evrardjpwe don't constraint there16:16
evrardjpor I missed something in the PIP_INSTALL_OPTIONS16:16
*** tomtom001 has joined #openstack-ansible16:16
evrardjpanyone to confirm there?16:17
odyssey4meok, I'll check into that constraint - it might be right16:17
odyssey4meI've commented in the bug16:17
evrardjpok thanks.16:17
evrardjpWe might need to step into pip10 sooner rather than later, and deal with all we're gonna see.16:17
*** pcaruana has joined #openstack-ansible16:18
evrardjpbut right now I think it's best to constraint if a constraint is required.16:18
evrardjpI do not see your comment odyssey4me did you comment on the previous bug?16:19
odyssey4meyeah, I'll see if I can replicate the issue - py3 issues have come up quite a bit lately16:19
odyssey4meyes, I commented on the previous bug - these look similar though so I'll look into them both16:19
evrardjpok16:19
evrardjpcan you assign them both to you?16:19
mnaser++ about py3 only (hopefully)16:19
odyssey4meja, will do that now16:19
evrardjpit looks like it's an easy fix to add constraints there, and we can bump to pip10 when we're ready16:19
evrardjpcool16:20
evrardjpthanks16:20
evrardjpnext16:20
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176342216:20
openstackLaunchpad bug 1763422 in openstack-ansible "config_template is not fully python3 compatible" [Undecided,New]16:20
*** epalper has quit IRC16:20
evrardjpandymccr: you started to work on this, right?16:20
andymccrevrardjp: si si, i believe it is though16:21
andymccrbut that bug says otherwise :)16:21
andymccrleave that with me and i'll try recreate16:21
evrardjphttps://review.openstack.org/#/c/561900/16:21
evrardjp:)16:21
andymccryeah i need to fix that up16:21
andymccrbut the py3 test is currently passing16:21
andymccrwhcih could just mean we are missing some test case16:21
evrardjpprobably missing a lot of those16:21
evrardjpbut with a separate repo it would be hopefully easier to deal with16:22
evrardjpand faster to iterate16:22
*** sew has joined #openstack-ansible16:22
evrardjpI assign this bug to you andymccr16:22
andymccryeah please16:23
evrardjpI am not marking it as confirmed, as the job will help us discover if it's true or false, and if extra coverage is necessary16:23
evrardjpimportance medium, okay for you?16:23
evrardjpfor everyone I mean16:23
odyssey4mesure16:23
evrardjpok next16:24
jrosseryes we have a worksround, it's ok16:24
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176336116:24
openstackLaunchpad bug 1763361 in puppet-sahara "Sahara: add support for the new [trustee] configuration section" [Undecided,New] - Assigned to zhongshengping (chdzsp)16:24
*** tomtom001 has quit IRC16:24
toskythat's me o/16:24
*** tomtom001 has joined #openstack-ansible16:24
evrardjpwoot tosky !16:24
evrardjpthanks for being there for your bug16:24
tosky(bug both for puppet-sahara and openstack-ansible_os_sahara16:24
odyssey4meyeah, seems sensible - should be a pretty straightforward patch as the same mechanism is used for neutron/nova IIRC16:25
toskyI think that I put all the details in the bug, tl;dr we changed a bit the configuration, and while the old one works with some magic code, it's time to change16:25
evrardjptosky: is that only for Rocky?16:25
toskyevrardjp: technically it could be backported to Queens, but not a strong requirement16:26
evrardjpor does Queens will get this backported?16:26
toskythe new configuration works with Queens too, so it's up to you16:26
evrardjpnope but if it's a better practice, more future proof, let's do it16:26
evrardjpthanks for the heads up16:26
evrardjpI'd like to mark this as confirmed16:26
toskythat's what I wanted to ask in fact - I see that it's assigned already, but it was not marked yet16:27
odyssey4meI've never seen a patch from the person who self--assigned it.16:27
evrardjpodyssey4me: https://review.openstack.org/#/c/558378/1/templates/sahara.conf.j216:28
evrardjpyou reviewed it though :p16:28
evrardjpI am fine with that.16:28
odyssey4meah ok :p clearly I need more coffee16:28
evrardjpthese are morning patches, before you have your second one. :D16:29
evrardjpanyway thanks tosky it's really nice to have your feedback16:29
evrardjpanyone wants to pick this one up?16:29
toskyI will keep an eye on the patch - we receive notifications for new patches on openstak-ansible-os_sahara on #openstack-sahara16:29
evrardjpoh great!16:29
*** cshen has joined #openstack-ansible16:29
odyssey4me:)16:30
evrardjplet's talk about that after the bug triage then16:30
evrardjpnext16:30
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176333616:31
openstackLaunchpad bug 1763336 in openstack-ansible "stale tempest config remains in the workspace after re-running os_tempest role" [Undecided,New]16:31
evrardjpI haven't got the chance to try that on my machine16:31
openstackgerritAndy Smith proposed openstack/openstack-ansible-tests master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55912716:32
evrardjpbut it looks like the task https://github.com/openstack/openstack-ansible-os_tempest/blob/540a6be329d78df6a195bf1749364c6f514cefe9/tasks/tempest_post_install.yml#L4916:32
odyssey4meoh wow, nice find there jrosser16:32
evrardjpshould maybe happen after https://github.com/openstack/openstack-ansible-os_tempest/blob/540a6be329d78df6a195bf1749364c6f514cefe9/tasks/tempest_post_install.yml#L6116:32
evrardjpand yes good find16:32
jrosseryeah i ws wtf for a bit till i found that16:32
evrardjpI don't know in which order does those file resolve though?16:33
odyssey4meyeah, I can't recall - it's been a while since I've read the tempest docs16:34
odyssey4methings may have changed16:34
evrardjpprobably worth simplifying and putting them on the same file16:34
evrardjpmaking sure these pesky errors don't show up :)16:34
evrardjpconfirmed high/16:34
evrardjp?16:34
odyssey4meyep16:35
evrardjpanyone up for it?16:35
evrardjpok let's continue then16:35
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176305816:35
openstackLaunchpad bug 1763058 in openstack-ansible "openstack_user_config.yml.example says log_hosts is required" [Undecided,New]16:35
*** ashak has joined #openstack-ansible16:36
evrardjpfair16:36
evrardjpconfirmed and low16:36
evrardjpand low-hanging-fruit16:36
evrardjpfine for everyone?16:36
odyssey4meyep16:36
mnasergood thing for a first time contributor to hack on :>16:36
evrardjpyup :)16:37
evrardjplow-hanging-fruit are the deal people, enjoy! :)16:37
evrardjpok moving on16:37
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176274216:37
openstackLaunchpad bug 1762742 in openstack-ansible "Some delegated tasks are failing when deployer is node part of a deployment (eg. controller node)" [Undecided,New]16:37
evrardjpshould we consider this completely done now?16:38
odyssey4meI think that's done, yes.16:39
mnaserlooks solved to me16:39
odyssey4meare we waiting for a pike release?16:39
*** DanyC has joined #openstack-ansible16:39
evrardjpnope16:39
evrardjplast week everything was released16:39
evrardjpor did I miss that patch for inclusion before release?16:39
*** DanyC has quit IRC16:39
odyssey4meah https://review.openstack.org/#/c/560920/ didn't make it into the release16:40
*** f5guy has quit IRC16:40
*** DanyC has joined #openstack-ansible16:40
odyssey4meyep, it looks like https://review.openstack.org/#/c/561167/ was missed16:41
evrardjpwant me to re-issue a release?16:42
evrardjpI am fine with that, or we wait till EOM16:43
odyssey4mewell, the next sha bump is about to merge - I guess you could do a release immediately from that16:43
odyssey4mepike is still what a lot of people are using, so perhaps that's a good idea16:44
evrardjpup to you people :)16:44
evrardjpok16:44
evrardjpwill do an extra release for it16:44
evrardjpnext16:44
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/175814416:44
openstackLaunchpad bug 1758144 in openstack-ansible "resolv.conf in containers set too late" [Undecided,New] - Assigned to Logan V (loganv)16:44
evrardjpI will let logan work on this one more week16:45
evrardjpunless someone else wants to help right now :)16:46
evrardjpnext16:46
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/175582116:46
openstackLaunchpad bug 1755821 in openstack-ansible "config_template fails to parse template if it contains a comment with leading spaces" [High,New] - Assigned to Kevin Carter (kevin-carter)16:46
evrardjpsame I will leave it cloudnull for now16:46
evrardjpand last one is for cloudnull too:16:46
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/174303216:46
openstackLaunchpad bug 1743032 in openstack-ansible "Galera cluster maintenance in OpenStack-Ansible" [Undecided,New] - Assigned to Kevin Carter (kevin-carter)16:46
evrardjplet's keep them one more week, and then unassign next week16:47
evrardjpthat's all for bug triage16:47
evrardjp#topic open discussion16:47
*** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)"16:47
evrardjpanything to add?16:47
toskyjust a curiosity: I was checking the jobs executed for openstack-ansible-os_sahara16:48
toskyI see that tempest is executed, but it's not clear which tests were executed16:49
spotzodyssey4me: COA exam is moving to pike soon16:49
toskydid I miss them, or otherwise, would it be possible to collect them too? iirc there are few useful roles in the tempest repository16:49
evrardjptosky: generally each role defines its own whitelist16:50
evrardjphttps://github.com/openstack/openstack-ansible-os_sahara/blob/master/tests/os_sahara-overrides.yml#L2416:50
evrardjpspotz: good16:51
evrardjpspotz: queens would be even better! :)16:51
evrardjpis there anything we can do to help making it move to P or Q ?16:51
evrardjptosky: any increased coverage is welcomed16:51
toskyevrardjp: just to be clear, I'm talking about the testr* files here http://logs.openstack.org/85/560985/6/check/sahara-tests-tempest/8794f37/16:52
toskywhich are not visible for example in http://logs.openstack.org/78/558378/1/check/openstack-ansible-functional-ubuntu-xenial/cedddae/16:52
evrardjphttp://logs.openstack.org/78/558378/1/check/openstack-ansible-functional-ubuntu-xenial/cedddae/logs/openstack/infra1/16:53
toskyoh, ok16:53
toskythanks!16:53
toskyI didn't dig too deep16:53
*** shardy has quit IRC16:53
evrardjpthe real location depends on where tempest is installed (so check the inventory to find it)16:53
evrardjpok I think we are good for today's meeting16:53
evrardjpthanks everyone!16:53
odyssey4methanks evrardjp16:54
evrardjp#endmeeting16:54
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs"16:54
openstackMeeting ended Tue Apr 17 16:54:10 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-17-16.01.html16:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-17-16.01.txt16:54
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-17-16.01.log.html16:54
*** erolg has quit IRC17:07
*** mbuil has quit IRC17:09
*** DanyC_ has joined #openstack-ansible17:11
Taseerevrardjp: I finally got congress scenario passed, is this all that is needed ? => https://review.openstack.org/#/c/560839/ ?17:14
*** DanyC has quit IRC17:14
*** admin0 has quit IRC17:17
*** electrofelix has quit IRC17:18
*** admin0 has joined #openstack-ansible17:18
prometheanfireevrardjp: am I running test incorrectly or something?17:21
prometheanfireopenstack-ansible-os_ironic/tests/common/test-pep8.sh: No such file or directory17:21
prometheanfirewhen running tox -e linters17:21
odyssey4meprometheanfire did you do run_tests.sh first to ensure that the tests repo is cloned?17:23
odyssey4me./run_tests.sh linters17:23
prometheanfireodyssey4me: nope, guess I should17:23
* prometheanfire is used to direct tox17:24
prometheanfire+ echo 'unsupported distribution: gentoo'17:24
prometheanfireunsupported distribution: gentoo17:24
prometheanfireLOL17:24
prometheanfireugh, it tries to install stuff even if already installed17:25
prometheanfirevia eval at that17:25
odyssey4mego ahead an make it better17:25
prometheanfireheh17:26
* prometheanfire wonders why we don't use bindep17:27
odyssey4mewe do17:27
odyssey4mebut for bindep to work, you need some packages installed17:27
odyssey4merun_tests does those first packages17:27
prometheanfirechicken and egg, ok17:27
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_nova master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55862017:28
prometheanfirewell, we can do what devstack does17:28
prometheanfiresee https://review.openstack.org/56149217:28
openstackgerritMerged openstack/openstack-ansible-os_nova master: Do not log passwords  https://review.openstack.org/56184717:28
odyssey4meprometheanfire Sorry, I don't follow how that applies.17:29
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_ironic master: update for depricated driver definition  https://review.openstack.org/56127717:29
prometheanfireodyssey4me: emerging git every time I want to run tests doesn't seem like a good use of time17:29
prometheanfirethere should be a checker, seeing if git is installed already17:29
prometheanfirejust checking if 'which git' returns 0 should be sufficient17:30
odyssey4meas I mentioned earlier, please go ahead and make a patch - just make sure it's done to the tests repo, and the change (once merged) will propogate from there17:31
odyssey4meof course you could also just do what it's doing, then use tox17:31
prometheanfireI'm working on it17:31
prometheanfirethat's what I ended up doing17:31
vinshodyssey4me: admin0: Oh wow.  Far too long banging my head against that glance issue :). I had glance_v2_api: true.  and of course I had glance_v1_api false. I figured.. why wouldn't the v2 api work on its own?17:37
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: templates: keystone-uwsgi: Allow uwsgi to autoload required plugins.  https://review.openstack.org/56030717:37
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: [RFC] Add support for using distribution packages for OpenStack services  https://review.openstack.org/56030817:37
vinshre-deployed with glance_v1_api: true.. and it all works now.17:37
vinshis that expected?17:37
odyssey4mevinsh is that pike?17:38
vinshYes, Pike17:38
vinshThis doc says that v1 is deprecated for pike https://docs.openstack.org/openstack-ansible-os_glance/pike/17:38
vinshBUT without v1.. things don't seem well for us17:38
odyssey4mevinsh interesting, because we test with v2 only in the gate: https://review.openstack.org/#/c/557373/17:39
vinshis v2_registry always enabled?17:40
odyssey4meI'm not quite sure I understand the question.17:41
vinshin gate.. do you always have the v2_registry enabled?17:41
vinshit is optional for v2 I guess17:41
odyssey4meno, we're testing in several different ways17:41
odyssey4meone is the defaults: https://github.com/openstack/openstack-ansible-os_glance/blob/stable/pike/defaults/main.yml#L81-L8617:42
odyssey4meone is with glance_enable_v1_api disabled17:42
vinshthis has to do with the versions that glance sends back to a client I bet.. it was telling us no address found.17:43
odyssey4meand one is with glance_enable_v2_registry enabled and v1 disabled17:43
*** DanyC has joined #openstack-ansible17:43
odyssey4mehmm, it's plausible that there may be a client issue with understanding the v2 response17:43
admin0vinsh, how did you solve it ? what setting did you changed ?17:43
*** cshen has quit IRC17:43
vinshadmin0: I deployed glance_v1_api17:43
vinshand it all worked17:43
vinshwith only glance_v2_api enabled. and v1 api/registry disabled AND v2 registry disabled.. no good.17:44
admin0" I deployed glance_v1_api"  - how :D ?17:45
vinshdo you test with glance_enable_v2_api true.. and glance_enable_v1_api false. AND glance_v2_registry: false?17:46
vinshadmin0: I set "glance_enable_v1_api: true" and ran os-install-glance play17:46
odyssey4meadmin0 these flags enable/disable the v1/v2 api: https://github.com/openstack/openstack-ansible-os_glance/blob/stable/pike/defaults/main.yml#L81-L8617:46
*** DanyC_ has quit IRC17:46
openstackgerritAntony Messerli proposed openstack/openstack-ansible master: Ensure zfsutils-linux is installed by default  https://review.openstack.org/56201017:46
odyssey4mevinsh yes we do, that's what runs in the v2_api_only test17:47
vinshWell dang. I wish I knew what we did wrong17:47
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert rsyslog to an include_task with group_vars  https://review.openstack.org/55600217:47
*** stuartgr has left #openstack-ansible17:48
vinshWell thanks for the confirmation on the gate tests odyssey4me17:50
*** hamza21 has joined #openstack-ansible17:51
vinshGoing to have to experiment with disabling glance_v1_api once things cool off here.17:51
*** sew has left #openstack-ansible17:55
*** hachi has joined #openstack-ansible18:00
*** hamza21 has quit IRC18:03
cloudnullanyone around have a moment to review https://review.openstack.org/#/c/560750/18:05
cloudnullodyssey4me: ^ thanks already18:05
cloudnullalso https://review.openstack.org/#/c/558227/18:05
cloudnullwith that I can carry on with the common systemd roles18:05
*** foutatoro has joined #openstack-ansible18:05
*** RandomTech has quit IRC18:06
foutatorohi all, someone knows how to solve this issue : FAILED - RETRYING: Ensure that the LXC cache has been prepared (60 retries left) ?18:06
odyssey4mecloudnull did you see that the cinder one got reverted - I think the role needs to have support for a service to reload18:06
openstackgerritMatthew Thode proposed openstack/openstack-ansible-tests master: Add support for testing on Gentoo.  https://review.openstack.org/56201418:07
odyssey4mefoutatoro that'll retry for 20 mins, it's not a failure unless it times out18:07
tomtom001odyssey4me thanks for the help! I ended up editing the grastate.dat file and setting the safe_to_bootstrap to 1 and then starting a node and getting the other two to join.18:08
foutatoroodyssey4me: Ok thanks18:08
odyssey4metomtom001 glad to hear that you came right!18:08
*** poopcat has joined #openstack-ansible18:09
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Fix repo_build_venv_rebuild toggle effectiveness  https://review.openstack.org/56201618:11
openstackgerritMerged openstack/openstack-ansible stable/queens: Update all SHAs for 17.0.3  https://review.openstack.org/56128818:13
openstackgerritMerged openstack/openstack-ansible-ops master: Add APM Server  https://review.openstack.org/56123818:13
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Constrain the pip/setuptools/wheel versions in the venvs  https://review.openstack.org/56201718:14
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Constrain the pip/setuptools/wheel versions in the venvs  https://review.openstack.org/56201718:15
*** cshen has joined #openstack-ansible18:15
*** mma has quit IRC18:17
tomtom001odyssey4me, horizon doesn't come up after the upgrade to pike, any errors in particular that you know I should look for?18:17
cloudnullodyssey4me: yea I need to figure out why the reload wasnt working with cinder.18:24
cloudnullthe role has reload functions, however they may need changing.18:24
*** cshen has quit IRC18:25
openstackgerritMatthew Thode proposed openstack/openstack-ansible-tests master: Add support for testing on Gentoo.  https://review.openstack.org/56201418:26
openstackgerritMerged openstack/openstack-ansible-os_nova master: Re-order database migrations  https://review.openstack.org/55935418:26
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone master: Only replace python when re-initializing the venv  https://review.openstack.org/56202018:27
vinshodyssey4me: Interesting other data point on glance api versions.. it looks like v2 glance uses uwsgi and v1 doesn't. maybe uwsgi requires some special care in our setup.18:28
odyssey4metomtom001 nope, sorry - no idea... it should be coming up18:28
odyssey4mevinsh it shouldn't as it should be a drop-in replacement... but yes if it's v2 api only, then it'll be using uwsgi18:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/queens: Do not log passwords  https://review.openstack.org/56202118:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Do not log passwords  https://review.openstack.org/56202218:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/queens: Re-order database migrations  https://review.openstack.org/56202318:30
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Re-order database migrations  https://review.openstack.org/56202418:30
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/queens: Do not log passwords  https://review.openstack.org/56187618:31
*** cshen has joined #openstack-ansible18:31
vinshodyssey4me: Glance team suggesting to NOT use uswgi in production settings yet-> https://github.com/openstack/glance/blob/master/releasenotes/notes/queens-uwsgi-issues-4cee9e4fdf62c646.yaml18:31
vinshanyway. sorry to bother. I'll file a bug or a review if something helpful comes up that isn't covered.  Keep up the amazing work on OSA. Thanks again.18:32
cloudnulltomtom001: using haproxy ?18:34
odyssey4mevinsh yeah, that's quite interesting - because we have run with glance under uwsgi for queens and have not had negative feedback at all...18:35
*** cshen has quit IRC18:35
vinshI hear ya.. and it worked in our lab.. but buckled in production18:36
vinshIN pretty much the exact way they warn of.18:36
vinshthis is uwsgi/pike for us.  queens soon.. and we will attempt it there with uwsgi and have to have in the standalone if its still busted.18:37
vinsh*have to hack18:37
odyssey4meyeah, it looks like the noted bug was resolved for queens18:40
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_keystone master: (WIP) Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/56202618:41
mnaseryeah it works fine for queens18:41
mnaser(i can confirm first hand, ran tempest against a prod cloud we delivered with osa)18:42
vinshhow about on pike? :)18:42
mnaseryou'll have to carry that patch18:42
mnaserbecause glance doesnt want to merge it18:42
vinshGood to hear with queens. will let this run v1+v2 in pike and then look forward to v2 only in queens +uwsgi.18:43
vinshWe have no real pressure to disable glance v1 api right now so good enough.18:43
odyssey4mevinsh yeah, for queens that's the default - you won't have to set a thing, it'll just magically happen ;)18:43
vinshmnaser: good point.18:43
odyssey4mevinsh thanks for sharing your findings - it's useful to know :)18:44
vinshodyssey4me: that bug that is closed.. is closed as wont fix basically btw.18:44
vinshoh for pike. n/m18:45
odyssey4mevinsh which bug is that?18:45
odyssey4meI was looking at this one: https://bugs.launchpad.net/glance/+bug/170385618:45
openstackLaunchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Fix released] - Assigned to Matthew Treinish (treinish)18:45
*** mma has joined #openstack-ansible18:45
openstackgerritMerged openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.12  https://review.openstack.org/56129918:46
vinshodyssey4me: Yep that one. I miss-read the fix for queens/pike.18:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Extend linters time out  https://review.openstack.org/56202718:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Extend linters time out  https://review.openstack.org/56202718:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/ocata: Extend linters time out  https://review.openstack.org/56202818:49
*** mma has quit IRC18:50
vinshThe last comment on this one is a gem: https://review.openstack.org/#/c/525994/18:52
odyssey4mesjoe18:55
*** cshen has joined #openstack-ansible18:57
*** cshen has quit IRC19:02
*** TobbeCN has joined #openstack-ansible19:09
*** TobbeCN has quit IRC19:13
*** cshen has joined #openstack-ansible19:19
tomtom001cloudnull, yes using haproxy19:20
DimGRmaybe rerun the horizon role tomtom001 ?19:21
tomtom001ok, can do19:21
*** jrosser has quit IRC19:22
*** jrosser has joined #openstack-ansible19:23
mmercerok, so ive gone through several of the varying document pages for installing the aio (quickstart),  and none of them mention stopping firewalld, networkmanager, etc....19:24
mmercerwhich is weird because i swear i was following one yesterday that had stop networkmanager, use network service (legacy) instead, etc...19:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Only replace python when re-initializing the venv  https://review.openstack.org/56203319:26
cloudnulltomtom001: does haproxy-stat show that the end point is up ?19:27
mmercerso.... aside from a literall 'minimal install',  and getting git on the machine to pull down things, is there any other step necessary on an aio c7 install ?19:27
cloudnullsorry kinda bouncing around these last few days.. .19:27
mmercerahhh, nvm,  i know where the confusing came from.  hah19:28
mmerceri had been messing around with packstack originally, and it was packstack that mentioned disabling firewalld, networkmanager, etc19:28
odyssey4memmercer heh, ok - to my knowledge for an AIO it's literally just that, nothing special - I do AIO's using a stock cloud image in a public cloud19:29
odyssey4meif you're wanting to do an AIO for production, then there's more to it because you'll have to do all the work we do in the bootstrap-aio by hand (create the bridges, setup your storage).19:30
mmercerodyssey4me: nah, its my own 'workstation' that im using for purposes of developing more reasonable ansible automation tooling around some of our stuff19:34
mmercerits just a beefy workstation19:34
mmercerwell... quasi beefy.... 16 core 144g ram.... 9tb disk19:35
mmerceri wish i could get another 2 procs in it, lol19:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Do not log passwords  https://review.openstack.org/56203419:37
odyssey4memmercer you can try using virtualbox - at least it'll isolate the mess a bit more19:37
mmercerno way, lol.  nested id lose soooo much power =D19:38
mmercerim being greedy19:38
tomtom001cloudnull no command haproxy-stat exists.  was it haproxy status or something?19:38
odyssey4meI'm out for the night - cheerio all!19:40
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_keystone master: (WIP) Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/56202619:45
*** kstev has quit IRC19:46
mgariepytomtom001, if you have socat installed: echo show stat |socat stdio /var/run/haproxy.stat19:50
*** cjloader has quit IRC19:51
*** cjloader_ has joined #openstack-ansible19:51
*** cjloader_ has quit IRC19:52
*** radeks has quit IRC19:53
*** kstev has joined #openstack-ansible19:54
*** hachi has quit IRC20:01
*** cjloader has joined #openstack-ansible20:02
openstackgerritKevin Carter (cloudnull) proposed openstack/monitorstack master: Adding vg_check from the maas plugins  https://review.openstack.org/47019620:03
*** throwsb1 has joined #openstack-ansible20:15
*** cjloader has quit IRC20:20
*** cjloader has joined #openstack-ansible20:21
*** vakuznet_ has quit IRC20:33
*** ansmith has quit IRC20:38
*** pcaruana has quit IRC20:42
*** cshen has quit IRC20:50
*** oneswig has joined #openstack-ansible20:51
*** cjloader has quit IRC20:57
*** cshen has joined #openstack-ansible20:58
*** cshen has quit IRC21:03
*** cjloader has joined #openstack-ansible21:08
*** cjloader has quit IRC21:08
*** cjloader has joined #openstack-ansible21:08
*** cshen has joined #openstack-ansible21:10
*** kstev has quit IRC21:14
*** cshen has quit IRC21:15
*** foutatoro has quit IRC21:15
*** kstev has joined #openstack-ansible21:16
*** cshen has joined #openstack-ansible21:24
*** jwitko_ has joined #openstack-ansible21:26
*** johnduarte has quit IRC21:27
*** cshen has quit IRC21:28
*** jwitko has quit IRC21:29
*** ansmith has joined #openstack-ansible21:31
*** jwitko_ has quit IRC21:32
*** cshen has joined #openstack-ansible21:35
*** cmart has quit IRC21:36
*** cshen has quit IRC21:40
*** openstacking_123 has joined #openstack-ansible21:41
DimGRcloudnull  hey21:48
*** esberglu has quit IRC21:51
*** openstacking_123 has quit IRC21:52
*** cshen has joined #openstack-ansible21:53
*** cjloader has quit IRC21:53
*** tomtom001 has quit IRC21:55
*** jwitko has joined #openstack-ansible21:55
*** tomtom001 has joined #openstack-ansible21:55
*** admin0 has quit IRC21:57
*** cshen has quit IRC21:58
*** oneswig has quit IRC22:02
*** cmart has joined #openstack-ansible22:09
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage  https://review.openstack.org/55508422:12
*** DanyC has quit IRC22:13
*** DanyC has joined #openstack-ansible22:14
*** cshen has joined #openstack-ansible22:14
*** cshen has quit IRC22:19
*** DanyC_ has joined #openstack-ansible22:19
*** DanyC has quit IRC22:23
*** cshen has joined #openstack-ansible22:25
*** DanyC_ has quit IRC22:26
*** cshen has quit IRC22:30
*** cshen has joined #openstack-ansible22:36
*** cshen has quit IRC22:40
*** cmart has quit IRC22:41
*** cshen has joined #openstack-ansible22:48
*** cshen has quit IRC22:52
*** cshen has joined #openstack-ansible22:59
*** cshen has quit IRC23:04
*** TobbeCN has joined #openstack-ansible23:11
*** cshen has joined #openstack-ansible23:14
*** TobbeCN has quit IRC23:15
*** cshen has quit IRC23:19
*** cshen has joined #openstack-ansible23:25
*** throwsb1 has quit IRC23:25
*** cshen has quit IRC23:29
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Do not log passwords  https://review.openstack.org/56208123:31
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage  https://review.openstack.org/55508423:32
*** cshen has joined #openstack-ansible23:36
*** tosky has quit IRC23:41
*** Leo_m has quit IRC23:42
*** cshen has quit IRC23:43
*** weezS has quit IRC23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!