Wednesday, 2017-10-18

*** weezS has quit IRC00:00
*** esberglu has quit IRC00:01
*** yifei has joined #openstack-ansible00:19
*** smatzek has joined #openstack-ansible00:20
*** jwitko has quit IRC00:24
*** smatzek has quit IRC00:25
*** jwitko has joined #openstack-ansible00:25
-openstackstatus- NOTICE: due to unscheduled restart of zuulv3.o.o you will need to 'recheck' your jobs that were last running. Sorry for the inconvenience.00:31
*** rmcallis has joined #openstack-ansible00:33
*** pbandark has quit IRC00:37
*** woodard_ has joined #openstack-ansible00:42
*** woodard has quit IRC00:42
*** markvoelker_ has quit IRC00:53
*** markvoelker has joined #openstack-ansible00:54
*** thorst has joined #openstack-ansible00:57
*** markvoelker has quit IRC00:58
*** smatzek has joined #openstack-ansible01:02
*** thorst has quit IRC01:02
*** thorst has joined #openstack-ansible01:03
*** thorst has quit IRC01:08
*** smatzek has quit IRC01:25
*** smatzek has joined #openstack-ansible01:26
snowman48391is the the ceph setup in OSA only for redhat based distros?01:31
*** smatzek has quit IRC01:50
*** thorst has joined #openstack-ansible02:03
*** thorst has quit IRC02:08
*** pramodrj07 has quit IRC02:29
*** MasterOfBugs has quit IRC02:29
*** gouthamr has joined #openstack-ansible02:32
*** gouthamr has quit IRC02:42
*** gouthamr has joined #openstack-ansible02:43
*** dave-mccowan has quit IRC02:44
*** duonghq has quit IRC02:47
*** duonghq has joined #openstack-ansible02:48
*** dave-mccowan has joined #openstack-ansible02:48
*** duonghq has quit IRC02:56
*** duonghq has joined #openstack-ansible02:57
*** thorst has joined #openstack-ansible03:04
*** thorst has quit IRC03:09
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273903:16
*** openstackgerrit has quit IRC03:22
*** hwoarang has quit IRC03:26
*** andressanchez has quit IRC03:31
*** hwoarang has joined #openstack-ansible03:31
*** rmcallis has quit IRC03:37
snowman48391so I actually wiped all 6 machines, reinstalled ubuntu, and ran the scripts fresh on the machines and everything ran great up until that keystone problem in task TASK [os_keystone : Ensure service tenant]. It still fails with gateway timeout 504. Anyone have an idea?03:43
*** smatzek has joined #openstack-ansible03:51
*** gkadam has joined #openstack-ansible03:55
*** smatzek has quit IRC03:55
*** kiennt26 has quit IRC03:57
*** kiennt26 has joined #openstack-ansible03:57
*** Jeffrey4l has quit IRC04:02
*** Jeffrey4l has joined #openstack-ansible04:03
*** chyka has joined #openstack-ansible04:06
snowman48391so when the script POST http://172.29.237.33:35357/v3/auth/tokens, it just times out though I get a response using curl on 5000 and 3535704:08
SamYaplesnowman48391: whats the keystone logs say?04:08
SamYaple(aslo do you mean you can get a token with curl?)04:08
*** udesale has joined #openstack-ansible04:09
*** chyka has quit IRC04:10
*** rmcallis has joined #openstack-ansible04:11
snowman48391the keystone logs just show haproxy activity and the ansible scripts making requests for tokens and them timing out04:13
snowman48391I haven't tried to get a token with curl. I'm haven't looked up what goes in the request04:14
*** udesale has quit IRC04:14
*** udesale has joined #openstack-ansible04:14
snowman48391I can pastebin keystone or nginx logs04:16
snowman48391so I can curl it and it takes about 60 seconds for a response but I do eventually get one. I'm running 6 machines. 3 compute, 3 everything else. Would the slowness of replication in the galera cluster be the reason it's taking so long?04:28
SamYaplesnowman48391: token backed is fernet or uuid?04:28
snowman48391the default. I believe it's fernet04:29
SamYapledefault changed with different versions of OSA04:29
snowman48391pike04:29
SamYapledefinetley fernet04:29
SamYaplehmmm databsae would be where i would start04:29
SamYapledo queries take really long time?04:29
snowman48391yup04:29
SamYaplesnowman48391: is your database like really really large?04:34
SamYapledu -sh /var/lib/mysql04:34
snowman48391it has nothing in it. this I haven't gotten through the OSA install yet04:35
SamYapleoh. hmmm. without additional info, i would guess networking04:35
SamYaplelogs should tell more04:35
snowman48391it's 3GB04:35
SamYaplehmmm just started, fresh db, and its 3GB already04:36
SamYapleis the keystone database itself 3GB?04:36
snowman48391the /var/lib/mysql showed 3GB04:36
snowman48391I can log into it and see that it created the keystone db04:36
SamYaplethat might be cache, innodb files can preallocate alot depending on settings04:37
*** rmcallis has quit IRC04:44
*** kiennt26 has quit IRC04:47
*** dave-mccowan has quit IRC04:47
*** armaan has joined #openstack-ansible04:48
*** armaan has quit IRC04:49
snowman48391yeah I'm just watching the galera and the keystone containers on tcpdump go back and forth04:55
snowman48391it's just obscenely slow04:55
snowman48391it's just 6 pretty nice desktops with gigabit ethernet but it's not pegging their networking, ram, or cpu04:55
*** thorst has joined #openstack-ansible05:05
*** thorst has quit IRC05:10
*** armaan has joined #openstack-ansible05:13
*** pmannidi has quit IRC05:15
*** yolanda has quit IRC05:15
*** shardy has quit IRC05:23
*** shardy has joined #openstack-ansible05:25
*** snowman48391 has quit IRC05:33
*** gouthamr has quit IRC05:34
*** markvoelker has joined #openstack-ansible05:35
*** markvoelker_ has joined #openstack-ansible05:37
*** vnogin has joined #openstack-ansible05:38
*** markvoelker has quit IRC05:39
*** vnogin has quit IRC05:44
*** markvoelker_ has quit IRC05:46
*** markvoelker has joined #openstack-ansible05:47
*** snowman4839 has joined #openstack-ansible05:48
*** chyka has joined #openstack-ansible05:48
*** markvoelker has quit IRC05:50
*** armaan has quit IRC05:50
*** markvoelker has joined #openstack-ansible05:50
*** markvoelker has quit IRC05:51
*** vnogin has joined #openstack-ansible05:51
*** chyka has quit IRC05:53
*** vnogin has quit IRC05:56
snowman4839looks like the clean install of galera was just borked06:05
snowman4839should therre need to be any configs done to galera beforehand besides choosing the hosts?06:06
*** Oku_OS-away is now known as Oku_OS06:06
*** vnogin has joined #openstack-ansible06:17
*** kiennt26 has joined #openstack-ansible06:17
*** vnogin has quit IRC06:18
*** vnogin has joined #openstack-ansible06:19
evrardjpsnowman4839: ceph is supported on ubuntu too06:20
snowman4839in the os_ceph roles, all i saw were tasks that checked for dnf and yum06:21
*** jvidal has joined #openstack-ansible06:21
evrardjpsnowman4839: technically if your networking is fine (follow the docs) there is only one config: giving which host does what06:21
evrardjpcheck the docs06:22
evrardjpwe don't have os_ceph roles06:22
evrardjpour client is openstack-ansible-ceph_client06:22
evrardjpand we are using ceph ansible to deploy (if you want to deploy ceph with ceph-ansible)06:22
snowman4839sorry. was shorthand. I know it's a seprate project06:22
evrardjpwell, we can't help on something we don't know :p06:23
snowman4839well I figured someone would've used it given the popularity of ceph. I know ceph is supported on ubuntu. I was just wondering if that role that OSA included for ceph supported ubuntu.06:24
*** drifterza has joined #openstack-ansible06:24
snowman4839and I can't get my galera cluster started with the default configs. Been at it for 12 hours now :-/06:26
snowman4839guess it's a problem for tomorrow06:26
*** vnogin has quit IRC06:27
*** openstackgerrit has joined #openstack-ansible06:27
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_ceilometer master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51277506:27
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_neutron master: Update upgrade role for Queens from P  https://review.openstack.org/51181606:33
*** DanyC_ has quit IRC06:34
evrardjphwoarang: are you there?06:34
evrardjpsnowman4839: ceph is tested for ubuntu and centos06:34
evrardjpmaybe suse06:35
evrardjpI have to check06:35
evrardjpwe have users06:35
evrardjpI don't know your issue... Like I said, I want to help, but I have 0 details.06:35
evrardjpno openstack_user_config, no network configuration...06:36
evrardjpodyssey4me: I don't see haproxy server on https://review.openstack.org/#/q/topic:zuulv3-jobs+(status:open+OR+status:merged)+project:%255Eopenstack/openstack-ansible.*06:43
*** snowman4839 has quit IRC06:43
*** snowman4839 has joined #openstack-ansible06:45
*** pcaruana has joined #openstack-ansible06:45
*** markvoelker has joined #openstack-ansible06:45
evrardjpit's probably one git review away from your script :p06:46
*** vnogin has joined #openstack-ansible06:51
*** muran has joined #openstack-ansible06:51
*** muran is now known as muran_cn06:51
*** thorst has joined #openstack-ansible06:58
*** chyka has joined #openstack-ansible07:00
*** chyka has quit IRC07:00
*** thorst has quit IRC07:03
*** chyka has joined #openstack-ansible07:07
*** mbuil has joined #openstack-ansible07:09
*** chyka has quit IRC07:11
*** vnogin has quit IRC07:11
*** vnogin has joined #openstack-ansible07:12
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273807:21
*** shardy is now known as shardy_afk07:26
*** hw_wutianwei has joined #openstack-ansible07:33
openstackgerritMarcus Murwall proposed openstack/openstack-ansible-os_cinder master: Release notes added  https://review.openstack.org/51295407:35
*** vnogin has quit IRC07:35
*** rodolof has joined #openstack-ansible07:40
*** udesale has quit IRC07:55
*** DanyC has joined #openstack-ansible07:59
*** shardy_afk is now known as shardy08:00
*** andressanchez has joined #openstack-ansible08:01
evrardjphwoarang: I will move all the suse to non voting. It's been 2 weeks it's broken now, for many different reasons.08:05
evrardjpWe'll wait for it to stabilize before moving back to voting08:05
evrardjpexcept if you have a fix08:06
*** portante has quit IRC08:08
*** portante has joined #openstack-ansible08:09
*** muran_cn has quit IRC08:13
*** pbandark has joined #openstack-ansible08:14
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Queens from P  https://review.openstack.org/49667908:25
*** asb47_ has joined #openstack-ansible08:26
*** clayton_ has joined #openstack-ansible08:27
*** john51 has quit IRC08:28
*** clayton has quit IRC08:28
*** bauruine has quit IRC08:28
*** asb47 has quit IRC08:28
*** john51 has joined #openstack-ansible08:28
*** clayton_ is now known as clayton08:28
*** kiennt26 has left #openstack-ansible08:29
*** bauruine has joined #openstack-ansible08:29
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Update upgrade role for Queens from P  https://review.openstack.org/51181708:30
*** strattao has quit IRC08:30
*** admin0 has quit IRC08:30
*** strattao has joined #openstack-ansible08:31
evrardjpwoot cinder merged!08:31
*** admin0 has joined #openstack-ansible08:34
*** Trident has joined #openstack-ansible08:36
*** mmm_cn has joined #openstack-ansible08:37
mmm_cnSo I abandoned my previous commit as it appeared to end up in the wrong place (separate commit instead of added to this one: https://review.openstack.org/#/c/504015/1)08:37
mmm_cnMy question now is how do I add the release notes so they get added properly to the existing commit above?08:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_molteniron master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274108:43
*** chyka has joined #openstack-ansible08:44
odyssey4memmm_cn you do a commit --amend08:44
*** electrofelix has joined #openstack-ansible08:45
mmm_cnYeah trying to but first I forgot the amend part which ended up in a new commit. I abandoned that commit and reverted my branch back to head, just made a commit with amend now and the changeid look correct.08:45
mmm_cnCurrently it freezes on git review but that is probably some issue on my end. Will try and sort out.08:46
odyssey4mewhen you do git review it does a rebase against the head of the branch via the gerrit port08:47
odyssey4meso it freezes if your computer can't reach gerrit08:47
odyssey4meby the way, next time you can just do an interactive rebase and squash your two commits08:48
*** chyka has quit IRC08:48
odyssey4meevrardjp ah yes, I left haproxy off by mistake08:48
odyssey4melemme do that quickly08:49
evrardjpodyssey4me: thanks08:50
openstackgerritbhujay kumar proposed openstack/openstack-ansible-os_aodh master: Closes-Bug:#1724462  https://review.openstack.org/51297208:50
evrardjpmmm_cn: yeah one patchset is one commit. you can squash your commits together etc. What counts for gerrit is the change-id too. New change-id for the same branch, means new commit.08:50
evrardjpand new patchset08:50
*** bhujay has joined #openstack-ansible08:51
evrardjpso we generally amend our previous patchset, when we want to iterate08:51
evrardjpI think it's all explained somewhere, and we link it from our developer/contributor docs08:51
mmm_cnevrardjp: odyssey4me ok, I currently can't rech gerrit for some reason. (running this from a VM originating in tokyo), not sure why as it appears to have connectivity to the internet as it should. What host/port is the review trying to request?08:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-haproxy_server master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51297308:52
evrardjpmmm_cn: ssh maybe?08:53
evrardjpI don't remember which port08:53
odyssey4memmm_cn look at .gitreview in the repo and you'll see08:53
mmm_cnSSH to review.openstack.org:29418. And that works perfectly to reach using telnet. Very strange.08:54
odyssey4methen your git database might be broken - could you clone the repo again, then copy your changes over to the fresh clone?08:55
evrardjpyeah that should do the trick08:56
mmm_cnA bit of GIT newbie here, can I clone my specific branch that I created?08:58
mmm_cnI mean, I have this: https://review.openstack.org/512973 .   Can I get a clone from that? Or do you mean clone openstack-ansible-os_cinder, create a new branch and start over?08:59
*** thorst has joined #openstack-ansible08:59
bhujayodyssey4me: That day replacing pipeline.conf  for some stupid reason populated the gnocchi metric list but failed to capture any new resources. Finally the problem was solved  by taking two actions.  The back end storage cepg for gnocchi was showing many degraded pages and rectified. The 2nd action I took was dropped the containers , reinstalled gnocchi first and then installed ceilometer. Vanilla installation from OSA wor09:01
evrardjpmmm_cn: probably clone a new one09:02
evrardjpthen doing git review -d <your patchset>09:02
bhujaymanual installation doc says gnocchi to be pre installed before ceilometer and is wondering why the order of installation in OSA is differenent , ceiloemeter first and gnocchi next ?09:02
evrardjpmmm_cn: then after that, adding your extra bits09:02
evrardjpthen git add <extra bits>09:02
evrardjpthen git commit --amend09:02
evrardjpthen git review09:03
evrardjpbhujay: The Telemetry stack need more love... Which branch are you trying to deploy?09:03
bhujaystable/pike09:03
odyssey4meevrardjp yeah, bhujay has been figuring out how ot get it working for pike IIRC09:04
odyssey4menicely done bhujay09:04
odyssey4meis it all working now?09:04
evrardjpthat is great09:04
bhujayyeah its working stable  since last two days09:04
*** thorst has quit IRC09:04
evrardjpbhujay: what do you have in your telemetry stack for now?09:04
odyssey4meoh awesome - so now, is that patch to fix it?09:05
*** Guest31788 has quit IRC09:05
odyssey4mebhujay heh, now that I see it - it makes sense, that's to enable keystone v3 auth09:05
evrardjpbhujay: if we need to re-order the playbooks we can do that ! :D09:05
evrardjpodyssey4me: but couldn't we do the gnocchi install just after keystone, but before ceilometer?09:06
evrardjpI think it would be great to have gnocchi + ceilometer + heat autoscaling as a scenario :D09:07
evrardjpI don't know what's needed on top of that... aodh ?09:07
bhujayevardjp:  i  have deployed pike with  layer 3 routed config ( thanks to oddeysey4me for providing the hints) ,  multi os set up with kvm on ubuntu and redhat. for Telemetry I have ceilometer , aodh and gnocchi with ceph .09:07
evrardjpwow that's quite complex!09:08
* jrosser interested too09:09
odyssey4methe playbooks already do that for swift - we'd have to modify it to do it for ceph too: https://github.com/openstack/openstack-ansible/blob/master/playbooks/setup-openstack.yml#L2809:09
evrardjpbhujay: how is the experience?09:09
*** pmannidi has joined #openstack-ansible09:09
odyssey4mealthough it seems, at least at the time of newton, that all it needed was a keystone user for the config09:09
odyssey4meI'm not sure if that's the same for ceph.09:09
bhujayyeah i know , thats the demand from organization , i was worried  if it would be possible but  thankfully all is ok in poc , i am going to replicate it in prod soon09:09
evrardjpbhujay: wow that's great09:10
evrardjpbhujay: do you think we should shuffle the playbooks?09:10
*** seba has quit IRC09:10
odyssey4mewell, ideally the services should handle missing a connected service more gracefully09:10
evrardjpthat's for sure :)09:11
odyssey4meI would think that we need to be celeverer in the group_vars with the ceph implementation side of things09:11
evrardjpAgreed there too.09:12
odyssey4meie if you have ceph, not swift, then use it as a default store for gnocchi and do all the right things09:12
evrardjpyeah, and ppl can still override that behavior if need be09:12
odyssey4meyep09:12
odyssey4mewe've done it for swift, but now we should do it for ceph09:12
odyssey4meit should be easier to implement than it is right now09:13
odyssey4mebut yeah, fantastic news that it's all actually working bhujay - it'd be great if you could keep an eye on those roles and help maintain them09:13
odyssey4meit's possible that they won't take too much work - we mostly just need someone around who understands how they fit together to help troubleshoot when patches fail09:14
bhujayas far as experince is concerned ,    the dash board response  or even the cli response is little slow . So far I ahve tested two  applications one being in .Net on wondows . Despite old hardwares the response from the application is very good . I am trying to get more business application for test cases. Security and support  two things that my organization will look forward before bringing more applications, lets see how09:14
evrardjpbhujay: what's your use case? Is that pure telemetry for apps in openstack, or for other reasons like billing/scaling?09:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_molteniron master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274109:16
bhujayevardjp: auto scaling and billing are vital to convince my folks that i dont need  costly software like  vrealive or  rightscale etc09:16
bhujayNow moving towards aodh , i have filed a bug https://bugs.launchpad.net/openstack-ansible/+bug/172446209:17
openstackLaunchpad bug 1724462 in openstack-ansible "wrong service credential in aodh.conf for pike leading authorization faliure" [Undecided,In progress] - Assigned to bhujay kumar (bhatta)09:17
bhujayand ventured to submit a change through gerrit , hope will get some guidence from you guys . Its overwhelming to me to catch up so many things but facinating too09:18
evrardjpyeah... openstack developer workflow is a little different09:21
evrardjpbut I find it great at the end09:21
evrardjpdon't hesitate to ask questions. Not sure we can help in everything telemetry related, but we can help you on the rest :)09:21
*** gokhan has joined #openstack-ansible09:26
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Queens from P  https://review.openstack.org/49667909:35
*** stuartgr has joined #openstack-ansible09:36
*** hw_wutianwei has quit IRC09:38
openstackgerritMarcus Murwall proposed openstack/openstack-ansible-os_cinder master: Adds support for specifying GID AND UID for cinder system user  https://review.openstack.org/50401509:41
mmm_cnOk there. Think I finally got it working. I sincerely hope that I managed to commit to the correct change this time.  On a side note, it was my dev env that for some reason decided to no longer have a working GIT client. Fixed now so problem solved. Thanks for help09:44
evrardjp:)09:47
evrardjpmmm_cn: I think this release note will not pass linters, due to length of each line... Want me to edit it?09:48
mmm_cnevrardjp: Ah I see, let me fix it as I need to learn.09:49
mmm_cnI think I got a +1 from Zuul on the same comment in my prior commit on the incorrect changeid though09:49
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181009:56
odyssey4memmm_cn yeah, releasenotes will pass, but it's hard to read - if you could limit the width of each line to less than 80 chars that'd be nice09:58
mmm_cnSure09:59
*** pmannidi has quit IRC10:02
openstackgerritMarcus Murwall proposed openstack/openstack-ansible-os_cinder master: Adds support for specifying GID AND UID for cinder system user  https://review.openstack.org/50401510:05
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Queens from P  https://review.openstack.org/49667910:10
*** mmm_cn has quit IRC10:11
*** andressanchez has quit IRC10:14
*** yifei has quit IRC10:15
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181010:23
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181010:24
*** andressanchez has joined #openstack-ansible10:24
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181010:24
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181010:26
evrardjpI am gonna get there when my brain allows it.10:27
*** duonghq has quit IRC10:34
bhujayodyssey4me: how do i add the subject line as suggested , is it through git -a  --amend -m 'subject line'   and then git review to push it once again ?10:35
evrardjpjust git amend to have your text editor10:39
evrardjpif you do -m I suppose it will override everything, including the change id10:39
*** andressanchez has quit IRC10:40
bhujayok thanks evrardjp ,  follow that10:41
openstackgerritbhujay kumar proposed openstack/openstack-ansible-os_aodh master: Switch auth config to keystone v3  https://review.openstack.org/51297210:47
evrardjpwoot, thanks for your first contribution bhujay!10:48
evrardjpsmall detail for readability in my review10:49
pylot15.1.9 im getting RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa ... filing a bug report or stop me ;)10:51
pylot(os_gnocchi)10:51
odyssey4meyeah, thanks bhujay - good work! although that commit body needs to be no wider than 80 chars ideally - for reference :) https://chris.beams.io/posts/git-commit/10:58
*** thorst has joined #openstack-ansible11:00
*** thorst has quit IRC11:05
*** chyka has joined #openstack-ansible11:08
*** mmm_cn has joined #openstack-ansible11:09
*** vnogin has joined #openstack-ansible11:10
*** poopcat has quit IRC11:10
*** poopcat has joined #openstack-ansible11:10
TahvokIs zuul fixed? I would really like to see my changes merged already :)11:11
*** chyka has quit IRC11:12
odyssey4meTahvok is the repo you submitted a patch to in this list? if so, not yet: https://review.openstack.org/#/q/project:%22%255Eopenstack/(monitorstack%257Copenstack-ansible%257Cansible-).*%22+status:open+topic:zuulv3-jobs11:16
odyssey4mewhich review is it?11:16
TahvokThere are 2:11:16
Tahvokhttps://review.openstack.org/#/c/510650/11:16
Tahvokhttps://review.openstack.org/#/c/510654/11:16
TahvokWhy is a dependency of the other11:16
odyssey4merechecked them, they should be fine11:18
TahvokHolding my fingers11:18
TahvokThanks!11:18
odyssey4mealthough the role patch shouldn't need to depend on the integrated build - if it does, you're missing something11:18
Tahvokodyssey4me: Please read here to understand why it's needed: https://bugs.launchpad.net/openstack-ansible/+bug/168759411:19
openstackLaunchpad bug 1687594 in openstack-ansible "no configuration for osa neutron to configure external dns yet" [Wishlist,In progress] - Assigned to Albert Mikaelyan (tahvok)11:19
*** smatzek has joined #openstack-ansible11:19
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273811:22
bhujaythanks a lot evrardjp and odyssey4me , doing it in few minutes11:23
*** ansibler has joined #openstack-ansible11:23
Tahvokodyssey4me: so is it ok, or some other way is preferred?11:24
openstackgerritbhujay kumar proposed openstack/openstack-ansible-os_aodh master: Switch auth config to keystone v3  https://review.openstack.org/51297211:29
evrardjpwe should have proposal bot visit soon!11:29
ansiblerHi, still cannot install pike using openstack-ansible, failing on "openstack-ansible setup-infrastructure.yml" https://pastebin.com/f88gjCpC11:31
*** dave-mccowan has joined #openstack-ansible11:31
ansiblerI wonder where from does content of /etc/ansible/roles come? I have tried to start from sratch, gut for that i deleted /etc/openstack_deploy and /opt/openstack-ansible. Does content of /etc/ansible/roles comes with ansible itself?11:33
bhujayansibler: the roles get installed when you run bootstarp11:34
*** dave-mcc_ has joined #openstack-ansible11:35
*** rodolof has quit IRC11:36
*** dave-mccowan has quit IRC11:37
ansiblerSo it should be safe to remove /etc/ansible/roles and after bootstrapping it should be again created. Will try making backup of it just in case.11:37
*** acormier has joined #openstack-ansible11:39
bhujaynot sure why u r deleting them . But the way it works is your confuration is all kept in /etc/openstack_deploy folder .  You mentioned commenting out a repo host from there . It wont work because after the first run you will have the inventory.json file string all your host info . If there is mismatch between openstack_user_config.yml and inventory it wont go through11:40
odyssey4meTahvok reviewed11:41
bhujaybefore removing a host or container u shd destroy the container , remove it from inventory and then modify  the yml files .See if you list your inventory by running  this command from playbook directory11:43
bhujay../scripts/inventory-manage.py -l11:43
*** acormier has quit IRC11:44
*** acormier has joined #openstack-ansible11:44
ansiblerright, forgot to mention that I removed inventory.json as well. The problem is I do not understand is it a bug that I mentioned or it is because of some previous versions config that triggers this error. That is why I am trying to start from scratch. I can reinstall HW nodes if needed, but have to clean deployment host without reinstall.11:45
ansibleras the error mentions /etc/ansible/roles/... I guess there is something messed in that so I am going to remove it and try to recreate.11:46
odyssey4meansibler it's more likely that there's a config problem than an issue in the roles11:48
odyssey4mebut, considering you've wiped your inventory you're going to have to re-kick the hosts, or clean them up manually now11:49
odyssey4mewhen you run the playbooks next time, use -vvv in the CLI to deliver verbose output, then pastebin the last playbook that gives the error and share that pastebin link here - then others can inspect and help11:49
*** thorst has joined #openstack-ansible11:52
Tahvokodyssey4me: I'm not sure how to answer your questions. Everything I've done was according to this docs: https://docs.openstack.org/neutron/latest/admin/config-dns-int.html#config-dns-int-ext-serv11:53
TahvokI have actually asked some of your question myself. See here: https://bugs.launchpad.net/neutron/+bug/172236711:53
openstackLaunchpad bug 1722367 in neutron "designate extension reporting errors and failing upon delete" [Undecided,Confirmed]11:53
TahvokAnd have not received an answer11:53
TahvokWonder how should I proceed in that case?11:54
ansiblerodyssey4me, thanks, will do that! The thing is I am reusing config files from previous newton release where those worked. The error does not include any clue to what can be wrong with config - therefore confusion. https://pastebin.com/f88gjCpC11:55
odyssey4meTahvok you can never assume upstream docs are right ;)11:55
odyssey4meTahvok I would recommend, if you can, to try the alternative I've proposed and see if it works - if so, then update the patches, but also submit patches to the docs to update them :)11:56
Tahvokodyssey4me: guess that's what I'll do11:56
Tahvokodyssey4me: one more question: 'neutron_designate_enabled' is false, however if a user wants to enable it, I don't thing he should provide the variables, when they can be provided by group_vars/all/11:58
TahvokThat's why I made a dependency11:58
odyssey4meansibler ok, that's a repo-build error, specifically the py_pkgs lookup - can you provide, perhaps better in a gist, the openstack_user_config/user_variables and the output of executing: /opt/ansible-runtime/bin/python /etc/ansible/roles/plugins/lookup/py_pkgs.py /opt/openstack-ansible /etc/ansible/roles /etc/openstack_deploy11:59
odyssey4meTahvok yes, but the role is usable independently of the integrated build12:00
odyssey4meand that patch stands alone - it requires absolutely nothing from outside of the role12:00
Tahvokodyssey4me: the default role yes - but not if one wants to enable this option12:00
odyssey4methe integrated build may give a value to the role, but the role has a value without it12:00
TahvokI see12:00
TahvokOk, I understand now what you mean12:00
odyssey4meand for the integrated patch to work as intended, it actually depends on the patch to be implemented in the role... so the dependency is the wrong way around12:01
TahvokI need to start thinking differently12:01
TahvokThanks for the help! Will update the patches12:02
odyssey4meno worries, that's why we're a community - it takes a village to make awesome :)12:02
*** acormier has quit IRC12:03
odyssey4meTahvok can you reply in gerrit to my comments with that docs reference so that we have history to go back on?12:04
odyssey4mespecifically mention the upstream docs you followed12:04
odyssey4megood work, by the way -hopefully we can kill a few birds with one stone, however12:05
*** Dinesh_Bhor has quit IRC12:09
ansiblerodyssey4me, have updated same bin https://pastebin.com/f88gjCpC  with output, but not sure what "gist" is, if you could point me.12:11
odyssey4meansibler gist.github.com - you can put notes up easily there, and update them12:11
odyssey4meansibler ok, try just running /opt/ansible-runtime/bin/python /etc/ansible/roles/plugins/lookup/py_pkgs.py on its own12:13
odyssey4methen try running it with each of the parameters on their own12:13
mhaydengood morning12:16
evrardjpmorning mhayden12:20
*** albertcard1 has joined #openstack-ansible12:21
*** poopcat has quit IRC12:24
*** dave-mcc_ is now known as dave-mccowan12:29
ansiblerodyssey4me, have updated bin with output, maybe easier to see in raw format: https://pastebin.com/raw/f88gjCpC12:30
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Zuulv3 migration  https://review.openstack.org/51261812:31
odyssey4meansibler aha, ok - so one of the roles there has something breaking things - can you update with a list the roles in /etc/ansible/roles and each of their SHA's12:32
odyssey4methe output from the plugin could really use some sort of debug output - not sure if that's possible cloudnull / evrardjp ?12:33
*** gkadam has quit IRC12:34
evrardjpwell it's technically possible to write that into a temp file12:35
evrardjpbut lookup will still behave the same way12:35
odyssey4meja, I just wonder whether we can add a --debug or something to let it output the steps it's doing to show the data as it's being processed12:36
openstackgerritMerged openstack/openstack-ansible-os_molteniron master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274112:37
evrardjpoh yeah technically we could do that12:38
evrardjpthe call by ansible will just always be the same12:38
*** jhesketh_ has joined #openstack-ansible12:38
*** armaan has joined #openstack-ansible12:39
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51270912:40
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51297312:40
odyssey4mecloudnull - see the new jobs in https://review.openstack.org/512709 - it seems that btrfs isn't working, but in the fix for that we might want to promote those jobs to voting :)12:41
odyssey4mejrosser - you might be interested to see that the various back-ends are now being tested12:42
*** jhesketh has quit IRC12:43
ansiblerodyssey4me, updated bin with role list. There are folders under /etc/ansible/roles/, not sure about how to sha those12:44
odyssey4meansibler if you're able to - can you move the non-OSA roles out of that folder, then re-execute again12:45
odyssey4meif it works, then add them back in one at a time to find the one that's breaking things12:45
ansiblerok12:45
openstackgerritMerged openstack/openstack-ansible-plugins master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51272812:45
*** clayton has quit IRC12:45
odyssey4meevrardjp cloudnull it looks like we might need to figure out some sort of thing to ignore non-OSA roles :/12:46
evrardjp?12:46
evrardjpoh12:46
evrardjpin the lookup?12:46
evrardjpmaybe we should simply put the roles inside tree back :p12:47
evrardjpodyssey4me: maybe we should be more resilient to external roles instead12:49
*** clayton has joined #openstack-ansible12:49
odyssey4meyeah, although doing that doesn't help stable releases12:49
odyssey4mewe need to fix it for the future, but also for Pike at the very least12:49
evrardjpagreed12:50
*** mmm_cn has quit IRC12:51
*** hw_wutianwei has joined #openstack-ansible12:52
odyssey4mealright, we should be getting lint checks back now: https://review.openstack.org/51281612:54
*** woodard_ has quit IRC12:55
*** woodard has joined #openstack-ansible12:55
*** rodolof has joined #openstack-ansible12:56
openstackgerritMerged openstack/openstack-ansible-os_trove master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274812:57
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273212:58
openstackgerritMerged openstack/openstack-ansible-os_zaqar master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51275012:58
openstackgerritMerged openstack/openstack-ansible-os_watcher master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274912:59
openstackgerritMerged openstack/openstack-ansible-os_barbican master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273112:59
*** shardy has quit IRC13:00
*** shardy has joined #openstack-ansible13:00
*** alanmeadows has quit IRC13:02
*** alanmeadows has joined #openstack-ansible13:02
*** armaan has quit IRC13:03
*** fdegir has quit IRC13:04
*** mmm_cn has joined #openstack-ansible13:07
*** fdegir has joined #openstack-ansible13:07
*** shardy has quit IRC13:11
*** SerenaFeng has joined #openstack-ansible13:11
*** shardy has joined #openstack-ansible13:13
ansiblerodyssey4me, running openstack-ansible -vvv setup-infrastructure.yml. Meanwhile suggestion from user perspective - it would be really usefull to have short description in docs or some OSA included script how to clean out all OSA install and start from scratch. Probably additional chapter in Appendices.13:13
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Update upgrade role for Queens from P  https://review.openstack.org/51181613:13
ansiblerCurrently for that people need to gather bits of info from deployment docs and search for infolved folders/files. Have found some useful hints in https://cunninghamshane.com/tips-for-managing-openstack-with-openstack-ansible/ but that does not cover all, for example it turns out in infra hosts you can find /tmp/index-system file created by OSA, probably other similar things.13:13
odyssey4meansibler we used to have a thing to do that, but it gets dated to easily13:14
odyssey4meour current recommendation is to re-kick13:14
odyssey4mewe also have https://docs.openstack.org/openstack-ansible/latest/contributor/quickstart-aio.html#rebuilding-an-aio which can be updated - but no-one ever does because it's easier to rekick13:14
ansiblerthat means reinstall everything?13:15
*** rmcallis has joined #openstack-ansible13:15
odyssey4meyes, kill the hosts and start again13:15
odyssey4methe only time you do this sort of thing is in a lab, one would hope - and labs should be implemented in such a way that it's easy to start from scratch again13:15
odyssey4methat said, one would hope that we could possibly contain where we do things a bit more so that our stuff isn't littered all over the place13:16
*** lbragstad has joined #openstack-ansible13:16
odyssey4methat said, if you could write up a bug to update the information in https://docs.openstack.org/openstack-ansible/latest/contributor/quickstart-aio.html#rebuilding-an-aio with some stuff you found was missing, then someone can pick that up and do it13:17
openstackgerritLogan V proposed openstack/openstack-ansible stable/pike: Add ceph-ansible v3 support  https://review.openstack.org/51265513:18
ansiblerwell, if you have >5 hosts it is not so convenient :) At least as it turns out deployment host can be shared and can not be reinstalled, but of course that can be stated as policy - do not use OSA deployment host for other ansible roles.13:18
openstackgerritMerged openstack/openstack-ansible-os_swift master: Update upgrade role for Queens from P  https://review.openstack.org/51181913:19
*** rmcallis has quit IRC13:19
*** SerenaFe_ has joined #openstack-ansible13:21
odyssey4meansibler there's a nifty tool in the openstack-ansible-ops repo which allows you to build out a lab, and trash and rebuild it easily13:21
odyssey4mehttps://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio13:21
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: lxc_cache_preparation: Convert the LXC cache preparation to async task  https://review.openstack.org/51181013:22
openstackgerritMerged openstack/openstack-ansible-os_ironic master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273613:22
odyssey4mebut yeah, point taken - if you can register a bug with details of what you'd like to see - hopefully someone can pick it up, if you can't :)13:22
*** SerenaF__ has joined #openstack-ansible13:23
*** SerenaFeng has quit IRC13:24
*** gouthamr has joined #openstack-ansible13:24
*** SerenaFe_ has quit IRC13:25
mhaydenodyssey4me / evrardjp: Zuulv3 for ansible-hardening13:27
mhaydenhttps://review.openstack.org/51261813:27
evrardjpthanks !13:27
odyssey4memhayden you're amazing 0 love your work!13:30
*** SerenaF__ has quit IRC13:32
*** smatzek has quit IRC13:33
*** smatzek has joined #openstack-ansible13:33
*** esberglu has joined #openstack-ansible13:36
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51272113:37
*** smatzek has quit IRC13:38
openstackgerritMerged openstack/openstack-ansible-os_glance master: Update upgrade role for Queens from P  https://review.openstack.org/51181513:43
evrardjpwoot13:45
openstackgerritMerged openstack/openstack-ansible-os_rally master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274313:45
openstackgerritMerged openstack/openstack-ansible-os_aodh master: Switch auth config to keystone v3  https://review.openstack.org/51297213:48
*** drifterza has left #openstack-ansible13:50
odyssey4meevrardjp did you see my q in https://review.openstack.org/#/c/511549/ ?13:53
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273813:54
evrardjpoh yeah ok13:54
evrardjpgreat to see things merging13:54
openstackgerritMerged openstack/ansible-hardening master: Zuulv3 migration  https://review.openstack.org/51261813:55
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests master: Remove useless and confusing file  https://review.openstack.org/51154913:56
*** yifei has joined #openstack-ansible13:56
openstackgerritMarkus Zoeller (markus_z) proposed openstack/openstack-ansible master: docs: add a decision help  https://review.openstack.org/50408813:57
*** yifei has quit IRC13:57
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Update paste, policy and rootwrap configurations 2017-10-14  https://review.openstack.org/51204313:57
odyssey4memhayden you gonna do the zuul v3 job configs for monitorstack?13:57
evrardjpmmm kay. https://review.openstack.org/#/c/512631/13:57
odyssey4meotherwise I can do it if you like13:58
evrardjpso we definitely need to have our jobs in each branch13:58
odyssey4meevrardjp you need to review https://review.openstack.org/51277113:59
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Only build ARA report when test fails  https://review.openstack.org/51192214:00
evrardjpmay I edit it?14:00
evrardjpto remove what's not needed?14:00
evrardjpor are you busy with it?14:00
odyssey4meeverything is needed14:00
odyssey4mewhat is it you think isn't?14:00
evrardjpthe func is needed?14:00
evrardjpin tox.ini14:00
odyssey4meleave that be rather, we can clean that up in master and backport14:00
evrardjpit's already clean in master I think14:00
odyssey4meno it isn't14:01
odyssey4mealso, we'll probably ditch most of the tox config at some point once we figure out a better way to run all thise14:01
*** bhujay has quit IRC14:01
evrardjpmy bad it isn't14:01
evrardjpstill I don't understand14:01
evrardjpisn't the repo supposed to take what's in master branch?14:02
odyssey4meperhaps - would you like to query that and wait for a response, or get working gating?14:02
evrardjpodyssey4me: I am confused for also this patch14:06
evrardjphttps://review.openstack.org/#/c/512773/14:06
evrardjpvs this patch14:06
evrardjphttps://review.openstack.org/#/c/512771/14:07
evrardjpI now have voted for both14:07
*** andressanchez has joined #openstack-ansible14:07
evrardjpI voted on https://review.openstack.org/#/c/512773/  earlier, thought it was good enough :p14:07
odyssey4methey depend on each other14:07
evrardjpand now I realize14:08
evrardjpyeah14:08
evrardjptop on each other14:08
odyssey4methe one includes the initial work to allow both zuul v2 and zuul v3 to work in terms of fetching the roles, the other includes the jobs and fixes things that were discovered later14:08
evrardjpyeah, we could have done into one single patch to avoid confusion...14:09
evrardjpbut doesn't matter14:09
odyssey4meperhaps, but the one was pushed to the branch ages back14:09
evrardjpas long as we understand it's one body of work later14:09
evrardjpit's fine14:09
evrardjp:)14:09
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Implement zuul v2/v3 role fetch shim  https://review.openstack.org/51277114:10
odyssey4mewe'll likely still need to do some work to get ocata/newton gating too - but we have no depends-on support there, so it should be simpler14:10
evrardjpnot so where what you mean here.14:11
evrardjpoh you mean we shouldn't have to carry this shim ?14:11
evrardjpwe can still backport it14:11
evrardjpit doesn't hurt14:11
odyssey4meall the repo cloning and stuff may not work with ocata/newton - but we're not trying to merge any patches for those branches yet so we'll see14:11
*** woodard has quit IRC14:12
evrardjpwell we do need to merge patches on those branches for neutron14:12
odyssey4melet's get master and pike fixed up, then move on to those - unless you have something urgent for ocata/newton to handle?14:12
evrardjpthis way we can re-enable the master branch14:12
evrardjpmaster branch upgrade*14:12
evrardjpprio is still master/pike14:13
odyssey4meok, but surely that's only for pike -> master ?14:13
odyssey4meor are there patches for pike not going in ?14:13
evrardjpyeah but pike upgrade is broken too, and all the way back14:13
*** andressanchez has quit IRC14:14
odyssey4meok, I'll switch context to help out a bit later - put things on the etherpad, then prioritise what you want help with and I'll work through them14:14
evrardjpwoooot https://user-images.githubusercontent.com/26403/31723399-59674f14-b3e4-11e7-9e25-0080ce5378a4.png14:14
odyssey4mea new task result status?14:14
*** andressanchez has joined #openstack-ansible14:15
evrardjpyup14:16
evrardjphttps://github.com/ansible/ansible/issues/31245#issuecomment-33760444614:16
*** galstrom_zzz is now known as galstrom14:16
*** yolanda has joined #openstack-ansible14:16
evrardjpwe're gonna need to change our strategy14:16
evrardjpprobably14:17
evrardjpbut still, good thing14:17
*** hachi has joined #openstack-ansible14:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_ceilometer master: [DNM] Quick Attempt to fix ceilometer  https://review.openstack.org/51305114:26
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_ceilometer master: [DNM] Quick Attempt to fix ceilometer  https://review.openstack.org/51305114:27
*** smatzek has joined #openstack-ansible14:29
evrardjpodyssey4me: fun: we're gonna have our git cloning issues in our gates now too14:29
evrardjphttp://logs.openstack.org/49/511549/2/check/openstack-tox-linters/9ea3964/job-output.txt.gz#_2017-10-18_14_12_53_70781314:29
odyssey4meevrardjp yeah, I have a solution for that - add it to the etherpad14:30
evrardjpok great14:31
*** DanyC has quit IRC14:33
*** DanyC has joined #openstack-ansible14:33
*** pbandark1 has joined #openstack-ansible14:33
*** smatzek has quit IRC14:34
*** pbandark has quit IRC14:34
*** pbandark1 is now known as pbandark14:34
*** smatzek has joined #openstack-ansible14:36
*** DanyC has quit IRC14:37
*** weezS has joined #openstack-ansible14:38
*** phalmos has joined #openstack-ansible14:38
*** smatzek has quit IRC14:40
*** smatzek has joined #openstack-ansible14:42
cloudnullmornings14:42
evrardjpmorning14:42
*** mmm_cn has quit IRC14:45
*** smatzek has quit IRC14:46
odyssey4meohai cloudnull - how's it going?14:47
*** smatzek has joined #openstack-ansible14:48
odyssey4mefresh from #openstack-dev: PTG confirmed in Dublin, Ireland the week of Feb 26, 201814:50
evrardjpwooot14:52
*** smatzek has quit IRC14:52
evrardjpthe wild rover14:52
evrardjpodyssey4me: I have to go, I am still trying to find out what's wrong in the integrated repo.14:53
evrardjpI will handle that today or tomorrow14:54
evrardjpthere will always be more fires :p14:54
odyssey4meevrardjp which patch are you seeing problems?14:56
odyssey4meI have a few hours I can spend looking into things.14:57
*** hachi has quit IRC14:57
*** woodard has joined #openstack-ansible14:59
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent master: Add OpenStack-Ansible metadata  https://review.openstack.org/51049815:02
*** acormier has joined #openstack-ansible15:04
openstackgerritMerged openstack/openstack-ansible-os_heat master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51277815:05
dmsimardAnsible 2.4.1 pushed to next week, they'll spin a rc215:05
*** Oku_OS is now known as Oku_OS-away15:07
*** acormier has quit IRC15:08
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Initial OSA zuul v3 role jobs  https://review.openstack.org/51277315:09
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Add OpenStack-Ansible metadata  https://review.openstack.org/51047415:09
openstackgerritMerged openstack/openstack-ansible-os_glance master: Add OpenStack-Ansible metadata  https://review.openstack.org/51048715:11
openstackgerritMerged openstack/openstack-ansible-plugins master: Add OpenStack-Ansible metadata  https://review.openstack.org/51145215:11
evrardjpwe just merged tests and no proposal bot15:13
evrardjpdarn15:13
evrardjpyeah http://logs.openstack.org/periodic/propose-updates/ didn't run15:13
*** andressanchez_ has joined #openstack-ansible15:13
*** andressanchez has quit IRC15:14
evrardjpodyssey4me: the failure I was looking for is https://review.openstack.org/#/c/512046/15:15
mhaydenugh, no proposal bot?! :P15:15
*** weezS has quit IRC15:15
evrardjpbut ofc I was checking on master, so it doesn't help finding out what's wrong.15:15
evrardjpso it's simple, we need to backport things in those branches too.15:16
odyssey4meevrardjp stable/pike integrated it broken because https://review.openstack.org/512475 has not merged yet15:17
evrardjpyeah.15:17
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: Provide support for SFC deployments  https://review.openstack.org/51090915:17
evrardjpthat's what I said.... stupid me.15:18
odyssey4mehaha15:18
odyssey4meok, fun fact - we download the cirros image twice to different file system locations in os_tempest15:18
odyssey4meso I'll fix that, but also make it possible to use the nodepool cached image15:19
*** pcaruana has quit IRC15:19
cloudnullodyssey4me: sorry was getting coffee15:20
cloudnulldoing well, hows it going ?15:20
cloudnullanything that I can go smash on ?15:20
*** qcrivera has joined #openstack-ansible15:20
odyssey4mecloudnull we have zuul v3 jobs in most repositories for master only at this time15:20
odyssey4meso we're cool, for the most part to merge any patches in master15:21
odyssey4melxc-container-create has an error in the new btrfs check... you might want to fix that, and promote those check to voting :)15:21
evrardjpok I am gone for today. I leave you the keys!15:22
cloudnullodyssey4me: I'll have a look15:22
cloudnullit'd be great to promote those checks15:23
*** andressanchez_ has quit IRC15:23
odyssey4meyeah, agreed15:23
*** rmcallis has joined #openstack-ansible15:27
*** rmcallis has quit IRC15:27
*** germs has joined #openstack-ansible15:30
*** vnogin has quit IRC15:31
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: L3 Support for ODL deployments  https://review.openstack.org/49745815:33
openstackgerritMerged openstack/openstack-ansible-os_sahara master: Add OpenStack-Ansible metadata  https://review.openstack.org/51050415:41
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Add OpenStack-Ansible metadata  https://review.openstack.org/51049215:41
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Add OpenStack-Ansible metadata  https://review.openstack.org/51048815:41
*** smatzek has joined #openstack-ansible15:50
*** DanyC has joined #openstack-ansible15:55
*** DanyC_ has joined #openstack-ansible15:56
*** chyka has joined #openstack-ansible15:56
*** vnogin has joined #openstack-ansible15:56
*** cshen_ has joined #openstack-ansible15:57
*** trevormc has joined #openstack-ansible15:57
*** DanyC has quit IRC16:00
*** gkadam has joined #openstack-ansible16:02
*** cshen_ has quit IRC16:02
*** cshen_ has joined #openstack-ansible16:04
*** aaron has joined #openstack-ansible16:04
*** aaron is now known as Guest9112016:04
*** cshen_ has quit IRC16:06
*** hw_wutianwei has quit IRC16:06
Guest91120hey all - if it normal for heat api container to consume 3.5GB of RAM straight out of the box? before anything has even been configured?16:06
*** cshen_ has joined #openstack-ansible16:08
cloudnullodyssey4me: do you know if the zuul v3 work has happened in the ops repo ?16:11
odyssey4mecloudnull I think it's only doing docs/lint tests right?16:11
odyssey4meif so, it should have been16:12
cloudnullit is but its failing still.16:12
*** qcrivera has quit IRC16:12
odyssey4melooking now16:12
cloudnullhttps://review.openstack.org/#/c/51243116:12
cloudnullas an example.16:13
odyssey4meyeah, I see the problem16:13
odyssey4mehmm, we have no functional tests for this one - so I guess we'll have to modify this upstream for now16:13
odyssey4meI'll push a patch shortly16:14
cloudnullcool16:15
cloudnullthanks odyssey4me16:15
odyssey4mehmm, although - lemme do a quick patch to that repo which will resolve this I think16:15
cloudnulldo we need to make the zuul v3 change to pike / ocata too ?16:16
odyssey4meyeah, it will need to be16:16
odyssey4meevrardjp and I are working through them slowely16:16
odyssey4mefirst getting master right, then porting back16:16
cloudnullsounds good.16:16
*** cshen_ has quit IRC16:19
*** weezS has joined #openstack-ansible16:19
*** gkadam has quit IRC16:22
*** mbuil has quit IRC16:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Add openstack-ansible-linters job  https://review.openstack.org/51309116:28
odyssey4mecloudnull that adds the job definition - I'll do an ops patch now to use it16:28
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: Add openstack-ansible-linters test  https://review.openstack.org/51309316:31
odyssey4meif that works, I'll replicate it to the other repositories16:32
odyssey4meand pull out the project-config one16:32
*** markvoelker has quit IRC16:32
*** markvoelker has joined #openstack-ansible16:33
odyssey4mecloudnull  mind working through any passing jobs in https://review.openstack.org/#/q/project:%22%255Eopenstack/(monitorstack%257Copenstack-ansible%257Cansible-).*%22+status:open+topic:zuulv3-jobs and reviewing them ?16:35
*** markvoelker has quit IRC16:38
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Only build ARA report when test fails  https://review.openstack.org/51192216:38
openstackgerritMerged openstack/openstack-ansible-os_heat master: Add OpenStack-Ansible metadata  https://review.openstack.org/51048916:38
odyssey4meok, it's working - lemme do a bunch of those16:44
*** drifterza has joined #openstack-ansible16:45
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-apt_package_pinning master: Add openstack-ansible-linters test  https://review.openstack.org/51309916:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ceph_client master: Add openstack-ansible-linters test  https://review.openstack.org/51310016:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310216:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-haproxy_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310316:46
*** bhujay has joined #openstack-ansible16:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: Add openstack-ansible-linters test  https://review.openstack.org/51309316:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_container_create master: Add openstack-ansible-linters test  https://review.openstack.org/51310416:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts master: Add openstack-ansible-linters test  https://review.openstack.org/51310516:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-memcached_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310616:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Add openstack-ansible-linters test  https://review.openstack.org/51310716:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_openrc master: Add openstack-ansible-linters test  https://review.openstack.org/51310816:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_aodh master: Add openstack-ansible-linters test  https://review.openstack.org/51310916:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_barbican master: Add openstack-ansible-linters test  https://review.openstack.org/51311016:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder master: Add openstack-ansible-linters test  https://review.openstack.org/51311116:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Add openstack-ansible-linters test  https://review.openstack.org/51311216:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Add openstack-ansible-linters test  https://review.openstack.org/51311316:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Add openstack-ansible-linters test  https://review.openstack.org/51311416:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_gnocchi master: Add openstack-ansible-linters test  https://review.openstack.org/51311516:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_heat master: Add openstack-ansible-linters test  https://review.openstack.org/51311616:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_horizon master: Add openstack-ansible-linters test  https://review.openstack.org/51311716:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Add openstack-ansible-linters test  https://review.openstack.org/51311816:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone master: Add openstack-ansible-linters test  https://review.openstack.org/51311916:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Add openstack-ansible-linters test  https://review.openstack.org/51312016:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_molteniron master: Add openstack-ansible-linters test  https://review.openstack.org/51312116:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron master: Add openstack-ansible-linters test  https://review.openstack.org/51312216:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally master: Add openstack-ansible-linters test  https://review.openstack.org/51312316:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara master: Add openstack-ansible-linters test  https://review.openstack.org/51312416:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Add openstack-ansible-linters test  https://review.openstack.org/51312616:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove master: Add openstack-ansible-linters test  https://review.openstack.org/51312716:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_watcher master: Add openstack-ansible-linters test  https://review.openstack.org/51312816:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_zaqar master: Add openstack-ansible-linters test  https://review.openstack.org/51312916:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Add openstack-ansible-linters test  https://review.openstack.org/51313016:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-plugins master: Add openstack-ansible-linters test  https://review.openstack.org/51313216:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Add openstack-ansible-linters test  https://review.openstack.org/51313316:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Add openstack-ansible-linters test  https://review.openstack.org/51313416:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Add openstack-ansible-linters test  https://review.openstack.org/51313516:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rsyslog_client master: Add openstack-ansible-linters test  https://review.openstack.org/51313616:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rsyslog_server master: Add openstack-ansible-linters test  https://review.openstack.org/51313716:51
odyssey4meI've proposed a patch to be rid of the upstream linters test: https://review.openstack.org/51312516:51
odyssey4meonce that's in we should be good16:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_freezer master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273316:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_karbor master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273716:55
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_almanach master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51272916:55
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_searchlight master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274516:56
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51277516:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274616:58
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274716:58
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Queens from P  https://review.openstack.org/49667916:59
xdfilHello, anyone working on OpenDaylight integration into OSA: I'd like to assist, I will be coming into some Nexus 9k  equipment soon. I need to absorb some knowledge before I can be of use but I promise to contribute what i learn back to the project.17:01
*** weezS has quit IRC17:03
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Add openstack-ansible-linters test  https://review.openstack.org/51314217:04
odyssey4mexdfil from the looks of the current patches up for review, it would appear that mbuil is working on ODL17:05
odyssey4mehttps://review.openstack.org/#/q/owner:mbuil%2540suse.com+status:open17:05
xdfilThanks odyssey4me , I need to improve my review stalking skills17:06
*** bhujay has quit IRC17:07
xdfilIn its current state, there is no creation of an ODL LXC container in OSA right? and the ODL bits that do exist seem to be P+17:07
SamYaplewow. you really did push 40 patches odyssey4me17:07
odyssey4meSamYaple bash - doing my work for me for approximately the last 15 years17:09
SamYapledont i know it17:09
*** pbandark has quit IRC17:14
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51309317:17
odyssey4meum, my patch storm broke zuul - please hold...17:22
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing reduced packages  https://review.openstack.org/50745117:25
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing reduced packages  https://review.openstack.org/50745117:25
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Queens from P  https://review.openstack.org/49667917:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274717:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274617:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51277517:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_searchlight master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274517:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_almanach master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51272917:34
SamYapleodyssey4me: 30 second timer. maybe thatlly be better17:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_karbor master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273717:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_freezer master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51273317:34
odyssey4meSamYaple yeah, that's actually a good idea - however, it's all done for now17:36
odyssey4mebut next time I do this, that's a good thing to add ot the mix17:36
SamYapleyea how often are you going to need to modify all the zuul.yamls17:36
odyssey4mehopefully not often, but now that they're in-repo it's plausible that it'll need to be done17:37
* odyssey4me adds a comment in his horrible bash script used for these sorts of things17:38
*** electrofelix has quit IRC17:41
*** hachi has joined #openstack-ansible17:46
*** rodolof has quit IRC17:54
*** rodolof has joined #openstack-ansible17:55
*** lbragstad has quit IRC17:56
*** albertcard1 has quit IRC17:59
*** mwynne has joined #openstack-ansible17:59
openstackgerritMerged openstack/openstack-ansible-galera_client master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51271118:03
*** poopcat has joined #openstack-ansible18:03
evrardjpodyssey4me: it would be good to share the shell script :)18:06
odyssey4meevrardjp heh, you don't want to see this nasty bunch of hacks18:06
evrardjpI've seen bash hacks before.18:06
evrardjpnasty ones even18:07
odyssey4meevrardjp I just added a linters test to the in-repo job definitions because not all repos have functional tests, so thelinters was failing without the tests repo18:08
odyssey4meso we now have our own linters which specifies that the tests repo is needed18:08
*** albertcard1 has joined #openstack-ansible18:08
evrardjpok18:09
evrardjpwhere is this definition?18:09
evrardjpoh in repo18:09
evrardjpI don't see it, maybe I am not looking at the right thing.18:10
*** poopcat has quit IRC18:10
evrardjplet me check at your patches18:11
evrardjphttps://review.openstack.org/#/c/512816/ ?18:11
odyssey4meevrardjp there's a patch being tested for the tests repo with the job definition, then patches to all the others to use it18:11
*** stuartgr has quit IRC18:11
evrardjpok18:11
evrardjpgreat18:11
odyssey4meevrardjp no that one's being reverted: https://review.openstack.org/#/c/513125/18:12
evrardjpI don't know how this works though18:12
odyssey4meit's causing linters tests to fail for openstack-ansible-ops18:12
evrardjpoh so18:12
evrardjpopenstack-ansible-linters18:12
odyssey4meyup18:12
evrardjpso we are sharing a single namespace18:12
evrardjpat all times18:13
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Fix BTRFS package name  https://review.openstack.org/51315718:13
evrardjpeven when things are in repo18:13
odyssey4mehttps://review.openstack.org/51309118:13
evrardjpthat is freaking scary18:13
cloudnullodyssey4me: ^ i think that's all that was broken with the btrfs gate job18:13
SamYapleevrardjp: yes, it compiles a global job list18:13
SamYapleevrardjp: even secrets (which are only accessible from the same repo) share global names18:14
evrardjpSamYaple: OMG OMG OMG18:14
*** trevormc has quit IRC18:14
odyssey4meyeah, they spoke a bit about perhaps auto-prefixing the jobs with the repo name in the future18:14
SamYaplethe fact ti can scan 6000 repos and reload a config in seconds is impressive to me18:14
evrardjpthat is awesome and scary at the same time18:14
odyssey4mebut for now, it's a global list - much like jenkins18:14
evrardjpSamYaple: yes indeed18:14
evrardjpodyssey4me: well how are we gonna trigger the keystone upgrade now?18:15
evrardjpbecause the job name would have to be unique18:15
evrardjpand right now it's quite generic18:15
odyssey4meevrardjp the keystone repo upgrade test?18:15
evrardjpyes18:15
odyssey4meit'll use our defined job18:15
evrardjpnot sure how this would work.18:16
evrardjphow does the job know what to check out etc?18:16
odyssey4mewe define the job, they use it as-is - we *might* have to change the load ordering for zuul though to ensure our job loads ahead of their repo18:16
odyssey4mewe define the job with the required repositories18:16
*** lbragstad has joined #openstack-ansible18:16
odyssey4meso it's likely it won't work exactly as-is now18:17
evrardjpodyssey4me: oh you mean defining a new job then18:17
evrardjpok defining a new job I get it.18:17
odyssey4memaybe - for now that's not a big concern18:17
*** vnogin has quit IRC18:17
odyssey4meI'd prefer that we modify our job, and they use it - but we may not have a choice.18:17
SamYapleyea i did a cross-repo job with good success18:18
SamYaplethat would totally be possible18:18
evrardjpSamYaple: how was the job definition?18:18
evrardjpBecause here we have a generic job name18:18
*** vnogin has joined #openstack-ansible18:18
odyssey4meit's not that generic18:18
SamYapleevrardjp: defined the job in loci repo, merged, call the job from cinder pipelines by the loci-job-name18:18
evrardjpanyway I am off again... I will think about it tomorrow.18:18
evrardjpSamYaple: yes, that sounds easy and makes sense, that's how I'd do it18:19
odyssey4meah I see what evrardjp means - our job name is openstack-ansible-upgrade-ubuntu-xenial... and we have several of those18:19
SamYaplethe job even has a loci specific secret, so it all works well18:19
evrardjpso in our case, defining a test in the os_keystone repo that's derived from our upgrade job18:19
evrardjpodyssey4me: indeed.18:19
evrardjpI have to go though18:20
evrardjpsorry18:20
odyssey4mebut they all work the same way, so any one of them could be used actually - it's only which repositories need to be there that make it special18:20
SamYapleevrardjp: bye!18:20
odyssey4mebut yeah, we'd likely have to do some sort of globally usable job in openstack-ansible-tests (as that repo loads very early)18:20
*** vnogin has quit IRC18:21
odyssey4memhayden if you're around, it'd be good to get https://review.openstack.org/513093 in to unblock some of the patches submitted there18:26
*** vnogin has joined #openstack-ansible18:29
*** RandomTech has joined #openstack-ansible18:29
RandomTechhey everyone, I have a problem.18:29
*** newmember has joined #openstack-ansible18:30
RandomTechI have a single interface to use and when i go to configure the etc/network/interfaces file to set up the bridges it tells me i cant have multiple bridges on an interface18:30
RandomTechanyone know how to solve this?18:31
*** weezS has joined #openstack-ansible18:31
dmsimardSamYaple: 6000 repos * >=3 branches each18:31
dmsimardSamYaple: :P18:31
dmsimardyou can have branch specific configuration inside each branch18:31
SamYapledmsimard: how dare you correct me!18:32
SamYaplebut thats even more impressive18:32
odyssey4meRandomTech the appendices have example network interface configs to show you how18:32
dmsimardSamYaple: at least unless I'm mistaken18:33
RandomTechodyssey4me: thanks i missed that one18:34
mhaydenlookin18:35
mhaydendone, odyssey4me18:35
odyssey4methanks mhayden - that'll help antonym and cloudnull get on with their day18:36
odyssey4meit'll take a bit of time for the merge to actually happen though18:36
*** cshen_ has joined #openstack-ansible18:39
*** hachi has quit IRC18:42
*** vnogin has quit IRC18:42
*** markvoelker has joined #openstack-ansible18:44
*** cshen_ has quit IRC18:48
openstackgerritMerged openstack/openstack-ansible-tests master: Add openstack-ansible-linters job  https://review.openstack.org/51309119:00
openstackgerritMerged openstack/openstack-ansible-galera_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310219:00
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310319:00
openstackgerritMerged openstack/openstack-ansible-ceph_client master: Add openstack-ansible-linters test  https://review.openstack.org/51310019:00
openstackgerritMerged openstack/openstack-ansible-apt_package_pinning master: Add openstack-ansible-linters test  https://review.openstack.org/51309919:00
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Clarify keepalived overrides  https://review.openstack.org/51230719:00
*** hachi has joined #openstack-ansible19:07
*** hachi has quit IRC19:14
*** hachi has joined #openstack-ansible19:17
*** cshen_ has joined #openstack-ansible19:17
*** hachi has quit IRC19:23
*** trevormc has joined #openstack-ansible19:35
*** yolanda has quit IRC19:36
*** vnogin has joined #openstack-ansible19:39
*** vnogin has quit IRC19:41
*** albertcard1 has quit IRC19:47
RandomTechanyone here able to help me troubleshoot networking setup?19:50
*** vnogin has joined #openstack-ansible19:54
woodardhey guys this the right place to ask a question on ceph-ansible openstack setup?19:57
mhaydenRandomTech: sure, what's up?19:59
mhaydenwoodard: quite possibly ;) but if you have something ceph-ansible specific, #ceph-ansible on OFTC would be your best bet ;)19:59
mhaydenwoodard: i know that logan- is well-versed in things ceph-ansible19:59
woodardon the ceph-config: generate the ceph config running into an error fatal: [ceph01]: FAILED! => {"failed": true, "msg": "'dict object' has no attribute u'ansible_br-storage'"}20:00
woodardhttps://github.com/ceph/ceph-ansible/blob/master/roles/ceph-config/templates/ceph.conf.j2#L67-L9620:01
woodardlooks like it is defaulting to containerized deploy20:01
woodardwas running v2.2.12 prior and had no issue20:01
woodardthink this is something new on master20:01
woodardhttps://github.com/ceph/ceph-ansible/blob/v2.2.12/roles/ceph-common/templates/ceph.conf.j220:01
*** cshen_ has quit IRC20:02
*** cshen_ has joined #openstack-ansible20:03
mhaydenif you do 'ansible -m setup ceph01', do you see ansible_br-storage in there?20:05
*** trevormc has quit IRC20:05
mhaydenin my case, i see things like ansible_br_mgmt (note the underscores)20:05
mhaydenand ansible_br_vlan20:05
mhaydenyou may have to put your interface as br_storage20:06
*** qcrivera has joined #openstack-ansible20:07
spotzRandomTech: You're in good hands with mhayden:)20:07
mhaydenwoodard: might be a bug to raise with the ceph-ansible folks20:09
openstackgerritFrank Zhang proposed openstack/openstack-ansible-ops master: Exclude elasticsearch from container-destroy task  https://review.openstack.org/51243120:09
Guest91120hey, im spinning more than AIO for the first time.. is br-vxlan for tenant networks ie Cloud Networks at Rackspace?20:12
mhaydenGuest91120: sorta -- br-vxlan is used for private networking between VMs20:14
mhaydenit allows for lots more networks than VLANs20:14
*** esberglu has quit IRC20:14
mhaydenVLANs stop at 4096 but you can get 16M vxlan networks on the same L2 segment20:14
*** esberglu has joined #openstack-ansible20:15
woodard@mhayden thanks, I was looking up a few bug reports from 2016 that noted the same thing.20:15
Guest91120ah OK that makes sense thank mhayden20:16
*** openstackgerrit has quit IRC20:17
woodardhrmm I see ansible_br_storage for br-storage20:19
mhaydenwoodard: yeah, you'll need to change that in your group_vars to use underscores20:19
*** esberglu has quit IRC20:19
*** openstackgerrit has joined #openstack-ansible20:19
openstackgerritMerged openstack/openstack-ansible-ops master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51309320:19
woodardthe monitor-interface?20:20
*** qcrivera has quit IRC20:20
RandomTechspotz: i got it thanks20:21
RandomTechspotz: i used .45 in the auto and .35 in the iface so it couldnt find the .45 interface20:21
*** acormier has joined #openstack-ansible20:22
*** acormier has quit IRC20:22
*** acormier has joined #openstack-ansible20:22
spotzRandomTech: Yeah typos will get you all the time:)20:23
woodard@mhayden that seems to have sorted it, updated monitor_interface to br_storage20:23
woodardwill see if this works20:24
woodardthanks20:24
spotzRandomTech: Glad you found it. As you're working through things please feel free to ask for help here. :)20:24
*** smatzek has quit IRC20:24
*** smatzek has joined #openstack-ansible20:25
mhaydenwoodard: i slid in a potential fix -> https://github.com/ceph/ceph-ansible/pull/207820:25
RandomTechspotz: we are going to create a playbook to automate the setup for us. this was just a test to make sure stuff would configure correctly manually20:25
*** woodard has quit IRC20:26
openstackgerritMerged openstack/openstack-ansible-galera_client master: Add openstack-ansible-linters test  https://review.openstack.org/51314220:26
spotzRandomTech: You might want to look in the ops-scripts directory/repo cloudnull has put a lot of helpful things in there20:26
*** smatzek_ has joined #openstack-ansible20:27
RandomTechspotz: we have a template to go off of we just need to change some things to work with the bridges and virtual interfaces correctly20:27
spotzRandomTech: Cool20:27
*** weezS_ has joined #openstack-ansible20:28
*** weezS has quit IRC20:28
*** weezS_ is now known as weezS20:28
*** smatzek has quit IRC20:29
*** smatzek_ has quit IRC20:31
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert the AIO from using overlayfs to machinectl  https://review.openstack.org/51017520:33
*** esberglu has joined #openstack-ansible20:34
snowman4839for anyone wondering, the fix to galera not responding in a timely manner to keystone was to change "keepalive_timeout   70;" to "keepalive_timeout   10;" or so and restart nginx and the keystone-wsgi* services20:36
openstackgerritMerged openstack/openstack-ansible-os_searchlight master: Initial OSA zuul v3 role jobs  https://review.openstack.org/51274520:38
openstackgerritMerged openstack/openstack-ansible-rsyslog_server master: Add openstack-ansible-linters test  https://review.openstack.org/51313720:38
openstackgerritMerged openstack/openstack-ansible-rsyslog_client master: Add openstack-ansible-linters test  https://review.openstack.org/51313620:38
jrossercloudnull: i ran a zfs+dedup aio, and got all the containers in just over 3G20:39
*** esberglu has quit IRC20:39
cloudnullwow!20:39
cloudnullthat's amazing20:40
jrosseryeah, 6.6 ratio20:40
*** esberglu has joined #openstack-ansible20:40
*** poopcat has joined #openstack-ansible20:40
jrosseryou could see it creeping up and up the further it got theough the deploy20:40
jrosserwhich says that a to  of whats in each container is the same20:40
jrosser*ton20:40
*** esberglu has quit IRC20:41
*** esberglu has joined #openstack-ansible20:41
cloudnullyea!20:42
jrossertheres other loopbacks in the aio20:43
*** gouthamr has quit IRC20:43
jrosserwith a following wind you might get the whole deploy ina ramdisk on a 16G laptop20:43
cloudnull++20:45
cloudnullhttp://paste.openstack.org/show/623843/20:45
cloudnullthis is my setup with btrfs (machinectl)20:45
cloudnullI've a deployment at home running on 5 nodes.20:46
cloudnullhowever in the next few days I intend to re-kick one of my infra nodes zfs20:46
jrosserinteresting, i did two side by sideand the no dedup one took 20G on the zfs20:47
jrosseri wondered if youd converged someof your containers?20:48
snowman4839I say that but running setup-openstack broke it again. wtf20:48
cloudnullI'm super curious what the performance will be between the two and what the operator experience differences are20:48
cloudnullbut the recent changes in lxc-{hosts,container-create} are pretty exciting20:48
jrossertempest didnt pass though20:48
jrosserbut i didnt have chance to dig at that20:49
cloudnulljrosser: something related to a master based deployment or do you suspect something FS related?20:49
cloudnullsnowman4839: what are you seeing ?20:50
mhaydendmsimard: AWWWW i've always wanted to try poutine https://www.openstackcanada.com/20:51
jrossercloudnull: i can dig out the report tomorrow, but it was something along the lines of "minimal test scenario" that failed. about 20 tests before that were ok20:51
RandomTechspotz: i think i have it20:51
spotzRandomTech: cool20:52
dmsimardmhayden: Vancouver after Sydney ?20:52
RandomTechjust compiled my first ever template for ansible :p20:52
mhaydendmsimard: i won't be in sydney, but i hope to be in vancouver20:52
cloudnullmhayden: poutine is amazing!20:52
cloudnullRandomTech: woot!20:52
dmsimardmhayden: us both20:52
dmsimardcloudnull: in Quebec there is poutine everywhere. Even at McDonald's.20:53
mhaydenmattt: what's your take on poutine?20:53
mhaydendmsimard: is poutine a thing in vancouver?20:53
cloudnullI had it in Vancouver, fantastic !20:53
dmsimardEven super fancy restaurants will have poutine, but it'll be like fancy poutine20:53
dmsimardI dunno, EmilienM is poutine a thing in Vancouver ?20:53
cloudnull++ it was when we were there for the Vancouver OS summit a while back20:54
dmsimardCanada and Quebec keep arguing where poutine comes from :)20:55
dmsimardIt's the reason why Quebec wants Independence (just kidding)20:55
*** cshen_ has quit IRC20:56
EmilienMdmsimard: yeah, in some pubs21:03
EmilienMI had one last night :D21:03
snowman4839cloudnull: responses from keystone take 60+ seconds or fail when I have memcached enabled21:04
*** thorst has quit IRC21:04
snowman4839I spent all day yesterday thinking it was my galera cluster but just turning off cacheing makes everything work21:04
*** weezS has quit IRC21:05
*** Guest91120 has quit IRC21:08
*** RandomTech has quit IRC21:08
snowman4839so my reponse time gets worse linearly with every memcached host I add to keystone21:17
snowman4839It works very fast with no memcached, ok with 1 memcached in the config, and basically times out with 3 memcached21:18
*** gouthamr has joined #openstack-ansible21:20
*** yolanda has joined #openstack-ansible21:20
*** smatzek has joined #openstack-ansible21:23
*** newmember has quit IRC21:27
*** newmember has joined #openstack-ansible21:27
*** acormier has quit IRC21:27
*** phalmos has quit IRC21:29
*** chyka has quit IRC21:32
*** smatzek has quit IRC21:36
cloudnullsnowman4839: are you using memcached for tokens ?21:37
cloudnullor is that just for caching ?21:37
openstackgerritAntony Messerli proposed openstack/openstack-ansible-galera_client stable/newton: Find and remove existing galera sources files  https://review.openstack.org/51319121:46
SamYaplecloudnull: confirmed yesterday he is using fernet21:47
SamYaplepike release21:47
SamYaplesnowman4839: you said that running queries directly against the database was slow for you21:47
SamYaplesnowman4839: is that no longer the case?21:47
*** openstackgerrit has quit IRC21:48
*** acormier has joined #openstack-ansible21:50
*** acormier has quit IRC21:54
*** markvoelker_ has joined #openstack-ansible21:57
snowman4839now it is not no21:59
snowman4839destroyed galera containers and rebuilt and now galera is working great21:59
snowman4839SamYaple: it's just keystone stuff is super slow with memcached enabled22:00
*** markvoelker has quit IRC22:00
snowman4839I edited the template for keystone to just use one of my memcached servers and it's slower than no caching but allowing the install to continue22:01
*** lbragstad has quit IRC22:03
*** acormier has joined #openstack-ansible22:05
*** acormier has quit IRC22:10
*** dave-mccowan has quit IRC22:11
*** acormier has joined #openstack-ansible22:16
*** acormier has quit IRC22:19
*** acormier has joined #openstack-ansible22:20
*** openstackgerrit has joined #openstack-ansible22:21
openstackgerritMerged openstack/openstack-ansible-memcached_server master: Add openstack-ansible-linters test  https://review.openstack.org/51310622:21
*** idlemind has quit IRC22:22
*** chyka has joined #openstack-ansible22:22
*** lbragstad has joined #openstack-ansible22:22
*** Guest34657 has quit IRC22:23
*** brad[] has quit IRC22:23
*** mfisch has joined #openstack-ansible22:23
*** brad[] has joined #openstack-ansible22:23
*** mfisch has quit IRC22:23
*** mfisch has joined #openstack-ansible22:23
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Add openstack-ansible-linters test  https://review.openstack.org/51310722:24
openstackgerritMerged openstack/openstack-ansible-openstack_openrc master: Add openstack-ansible-linters test  https://review.openstack.org/51310822:24
*** acormier has quit IRC22:24
SamYaplesnowman4839: that sounds like networking to me22:27
SamYaplesnowman4839: still22:27
SamYaplesnowman4839: possibly the keystone containers cannot talk to *all* the memcached hosts22:27
snowman4839they're all on the same vlan and not maxing bandwidth but I'll check into it more22:29
SamYaplesnowman4839: echo stats | nc <memcached ip> 11211 from the keystone containers22:30
*** esberglu has quit IRC22:33
lbragstadare they failing on a timeout?22:33
*** esberglu has joined #openstack-ansible22:33
lbragstad60 seconds feels like timeout territory22:33
*** galstrom is now known as galstrom_zzz22:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Use the deployment host to download images for tempest  https://review.openstack.org/51320022:36
snowman4839the token request to keystone was timing out22:38
snowman4839and I didn't use netcat to test but I could telnet to all of them on port 1121122:38
openstackgerritMerged openstack/openstack-ansible-repo_build master: Add openstack-ansible-linters test  https://review.openstack.org/51313422:40
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Add openstack-ansible-linters test  https://review.openstack.org/51310422:40
*** gouthamr has quit IRC22:41
*** esberglu has quit IRC22:43
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Use the deployment host to download images for tempest  https://review.openstack.org/51320022:44
*** rodolof has quit IRC22:45
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Add openstack-ansible-linters test  https://review.openstack.org/51313322:45
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: Add openstack-ansible-linters test  https://review.openstack.org/51310522:45
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty master: Add openstack-ansible-linters test  https://review.openstack.org/51311222:45
*** vnogin has quit IRC22:48
*** vnogin has joined #openstack-ansible22:50
*** vnogin has quit IRC22:56
*** woodard has joined #openstack-ansible23:05
*** dave-mccowan has joined #openstack-ansible23:07
*** lbragstad has quit IRC23:09
*** chyka has quit IRC23:31
*** thorst has joined #openstack-ansible23:32
*** thorst has quit IRC23:36
*** smatzek has joined #openstack-ansible23:37
*** smatzek has quit IRC23:41
*** acormier has joined #openstack-ansible23:44
*** smatzek has joined #openstack-ansible23:46
*** acormier has quit IRC23:49
*** electrocucaracha has left #openstack-ansible23:50
*** vnogin has joined #openstack-ansible23:52
*** acormier has joined #openstack-ansible23:55
snowman4839if I want to destroy a service and make OSA act as if it never created it, do I just need to destroy the container and remove it from the /etc/openstack_deploy/openstack_inventory?23:56
*** vnogin has quit IRC23:56
*** yifei has joined #openstack-ansible23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!