Friday, 2016-03-18

*** xinwu has quit IRC00:14
*** yamahata has joined #tripleo00:20
*** morazi has quit IRC00:33
*** saneax is now known as saneax_AFK00:33
openstackgerritKeith Schincke proposed openstack/tripleo-heat-templates: Add support for deploying RGW on controllers.
*** dmacpher-afk has quit IRC00:44
*** shivrao has quit IRC01:02
*** ramishra has quit IRC01:07
*** rlandy has quit IRC01:09
*** ramishra has joined #tripleo01:09
*** thrash is now known as thrash|g0ne01:22
*** shivrao has joined #tripleo01:26
*** mannidi has joined #tripleo01:33
*** lazy_prince has joined #tripleo01:39
*** dmacpher has joined #tripleo01:41
*** rhallisey has quit IRC01:42
*** lblanchard has joined #tripleo01:55
*** colonwq has quit IRC01:57
*** lblanchard has quit IRC02:07
*** shivrao has quit IRC02:07
*** lazy_prince has quit IRC02:12
*** weshay has quit IRC02:13
*** dtantsur|afk has quit IRC02:15
*** dtantsur has joined #tripleo02:17
*** links has joined #tripleo02:40
openstackgerritMichael Chapman proposed openstack/puppet-tripleo: Add keystone and db sync profiles
*** TSCHAK_ has quit IRC04:12
*** TSCHAK has joined #tripleo04:14
*** Erming__ has quit IRC04:23
openstackgerritIan Wienand proposed openstack/diskimage-builder: centos-minimal: Disable creation of eth0/eth1 interface scripts
openstackgerritIan Wienand proposed openstack/diskimage-builder: centos-minimal: can be used with base
*** Erming_ has joined #tripleo04:29
*** rasca has quit IRC04:30
*** rasca has joined #tripleo04:30
*** masco has joined #tripleo04:40
*** akuznetsov has joined #tripleo04:45
*** akuznetsov has quit IRC04:50
*** ansiwen has quit IRC04:53
*** larsks has quit IRC04:54
*** yamahata has quit IRC04:55
*** lifeless has quit IRC04:55
*** jayg|g0n3 has quit IRC04:55
*** SlickNik has quit IRC04:55
*** lifeless has joined #tripleo04:56
*** SlickNik has joined #tripleo04:57
*** ansiwen has joined #tripleo04:59
*** larsks has joined #tripleo05:00
*** jayg|g0n3 has joined #tripleo05:00
*** bswartz has quit IRC05:02
*** yamahata has joined #tripleo05:07
*** mannidi has quit IRC05:08
michchapCan anyone tell me what version of puppet-redis we use? It looks like a change has broken the gate05:28
*** saneax_AFK is now known as saneax05:29
*** bswartz has joined #tripleo05:29
*** tzumainn has quit IRC05:41
*** bswartz has quit IRC05:46
*** akuznetsov has joined #tripleo05:56
*** jaosorior has joined #tripleo05:57
*** ebalduf has quit IRC05:59
*** rasca has quit IRC06:02
*** rasca has joined #tripleo06:04
*** shivrao has joined #tripleo06:06
*** shivrao_ has joined #tripleo06:08
*** shivrao has quit IRC06:11
*** shivrao_ is now known as shivrao06:11
*** pcaruana has quit IRC06:18
*** jprovazn has joined #tripleo06:30
michchapI think the HA job will be fixed by this:
*** rcernin has joined #tripleo06:37
*** dmacpher has quit IRC06:38
*** marios_ has joined #tripleo06:39
*** marios_ has quit IRC06:39
*** yamahata has quit IRC06:40
*** aufi has joined #tripleo06:44
*** devvesa has joined #tripleo06:46
*** akuznetsov has quit IRC06:57
*** yamahata has joined #tripleo07:07
*** akuznetsov has joined #tripleo07:13
*** onovy has joined #tripleo07:14
*** mikelk has joined #tripleo07:16
*** Marga_ has quit IRC07:18
*** yamahata has quit IRC07:20
*** shivrao has quit IRC07:31
*** Marga_ has joined #tripleo07:32
*** shivrao has joined #tripleo07:32
*** Marga_ has quit IRC07:35
*** Marga_ has joined #tripleo07:35
*** Marga_ has quit IRC07:36
*** Marga_ has joined #tripleo07:36
*** shivrao has quit IRC07:36
*** pcaruana has joined #tripleo07:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Make cipher suite and SSL options configurable
*** akuznetsov has quit IRC08:08
*** akuznetsov has joined #tripleo08:10
*** ccamacho has joined #tripleo08:12
*** dshulyak has joined #tripleo08:14
*** mkovacik has quit IRC08:15
*** akuznetsov has quit IRC08:18
*** paramite has joined #tripleo08:20
*** tesseract has joined #tripleo08:20
*** stendulker has joined #tripleo08:21
*** tesseract is now known as Guest5995408:21
*** apetrich has joined #tripleo08:23
*** shivrao has joined #tripleo08:24
*** Guest59954 is now known as new08:25
*** new has quit IRC08:26
*** tesseract- has joined #tripleo08:26
jaosoriormarios: I think didn't land because it has a depends-on... and even though that depends-on landed on master, it hasn't landed in stable/liberty08:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Redirect to https for horizon
jaosoriormarios: I temporarily abandoned the stable/liberty one though. Can you workflow ?08:36
*** ohamada has joined #tripleo08:38
mariosjaosorior: hmm ok thanks. so i guess it just needs a new workflow (maybe it hadn't landed in master when gfidente +A yesterday)08:42
mariosthe dependency i mean08:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Redirect to https for horizon
jaosoriormarios: Nah, the issue is that if there is a depends-on, and there is also something in stable/liberty (Even though this one is for master) then it thinks the stable/liberty one has to land also08:43
jaosoriorthis Depends-On mechanism needs branch awareness08:44
mariosjaosorior: oh i hadn't realised that (i thought it would just check master)08:44
*** ohamada has quit IRC08:45
jaosoriormarios: Yeah, I had thought that too initially, and stumbled upon this several times :P08:46
mariosjaosorior: but what if we don't want to land it in stable/liberty  ... i mean this.patch depends on that.patch and that.patch lands both stable and master but this.patch only master.08:46
jaosoriormarios: How's it going dude?08:46
mariosjaosorior: anyway that is hypothetical...08:46
*** ifarkas has joined #tripleo08:46
jaosoriormarios: Yep, it gets messy. That's why I abandoned the stable/liberty one for this to land. So then I'll re-propose it08:47
mariosjaosorior: ok, i think there is a new issue, seen on the ha job this morning on couple of review like Error: Could not find dependent Service[redis] for Exec[cp -p /etc/redis.conf.puppet /etc/redis.conf] at /etc/puppet/modules/redis/manifests/config.pp:90\u001b[0m\n", "deploy_status_code": 1}08:47
*** adarazs has quit IRC08:48
jaosoriormarios: Although... I would really argue for having in stable/liberty. cause those default credentials are a pretty bad security hole08:48
jaosoriormarios: Wha...08:48
mariosjaosorior: oh nm i just realised my example doesn't work anyway (like that.patch will land on both stable/master.) anyway is good to know thanks08:48
mariosjaosorior: going to file a bug for it08:49
jaosoriormarios: Do you know how to use this elastic search? Might be useful to see how many occurrences of that error you see are on-going. And when did it start08:49
*** adarazs has joined #tripleo08:50
*** ohamada has joined #tripleo08:50
jaosoriormarios: Where did you see that error?08:51
mariosjaosorior filed as placeholder08:51
openstackLaunchpad bug 1558962 in tripleo "Error: Could not find dependent Service[redis] for Exec[cp -p /etc/redis.conf.puppet /etc/redis.conf] at /etc/puppet/modules/redis/manifests/config.pp" [Undecided,Triaged]08:51
mariosjaosorior: sec08:51
jaosoriormarios: Fuck, I think that workflow didn't trigger a merge08:52
mariosjaosorior: this is one
jaosoriorlets see08:52
mariosjaosorior: check a controller (I randomly picked 1)08:52
mariosjaosorior: this is another
mariosjaosorior: i think anything from this morning will hit it08:53
jaosoriormarios: On a side-note. I think there might be something wrong with our mongo configuration08:53
jaosoriorI also see a bunch of errors related to mongodb08:53
jaosorioruhm... funky, that it's failing for that service Redis08:55
jaosoriormarios: Do you know if redis is managed by pacemaker?08:55
jaosoriormarios; There was a guy saying that there was a fix incoming for redis:
mariosjaosorior: yeah the pacemaker::resource::ip { 'redis_vip'08:57
mariosjaosorior: looking08:57
mariosjaosorior: and indeed we have service_manage => false,09:00
mariosfor redis09:01
marios("Prevent catalog failures in cases where service_manage is09:01
mariosnot enabled"09:01
*** shivrao has quit IRC09:02
jaosoriormarios: Yeah, so only thing that we need to make sure now is that the title tag for redis is not matching $::redis::service_name09:02
*** mbound has joined #tripleo09:03
michchapI fixed it09:04
michchapI think?09:04
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Set default locale to image in debootstrap element
mariosmichchap: o/ thanks, maybe it just isn't picked up by ci yet09:05
jaosoriormichchap: Well, that maybe, but on the other hand, I think we're using the default $::redis::service_name for redis. So we might needs to change that?09:06
jaosoriorBecause if that's defined, then it will find it in the catalog and will still notify the redis Service resource09:07
michchapjaosorior: is that a bad thing?09:07
jaosoriorwell, as far as I understand that's what your change does, no? It checks if the default is being used, and then notifies the Service resource09:08
*** mkovacik has joined #tripleo09:08
jaosoriormichchap: I'm referring to this
michchapjaosorior: yep, is a restart bad for some reason?09:10
jaosoriormichchap: In our case, yes, because we are not using systemd to manage redis. We're using pacemaker, which still doesn't have a proper Service provider09:11
jaosorioruhm...wait up09:12
jaosoriormichchap: Nevermind haha, it should fix it09:13
jaosoriormichchap: I got confused. So yeah, since we're using service_manage = false, then that service wouldn't be in the catalog, and it won't call the notify... so yeah, your change will fix this stuff as soon as it's taken into the CI09:13
jaosoriorsorry for the confusion09:13
*** mcornea has joined #tripleo09:15
*** mgould has joined #tripleo09:18
*** gfidente has joined #tripleo09:22
*** derekh has joined #tripleo09:22
jaosoriorderekh: Hey dude, is there a way to...unstuck a commit?09:23
derekhjaosorior: whats stuck?09:23
jaosoriorderekh: It has a depends-on on a commmit that has merged on master, and another one that was abandoned in stable/liberty09:24
jaosoriorderekh: Or should I just remove that Depends-On, now that the commit in master has merged?09:25
derekhjaosorior: going to add anouther +A to see if anything happens, if nothing happens just resubmit with no depends on and I'll +A immediately09:26
*** jistr has joined #tripleo09:28
derekhjaosorior: nothing happening, remove the depends on and we can just merge it I think09:28
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Remove default for the RabbitMQ password
jaosoriorderekh: Done ^^09:29
derekhjaosorior: done, all going well, it should merge in a few minutes09:31
jaosoriorderekh: Thanks dude09:31
derekhjaosorior: no prob at all09:31
gfidenteguys I don't like how the ha job is failing this morning09:35
gfidenteError: resources.ControllerNodesPostDeployment.resources.ControllerLoadBalancerDeployment_Step1.resources[1]09:35
jaosoriorgfidente: I think it's the same error michchap fixed earlier today. Hopefully this change will be taken into CI soon09:35
*** links has quit IRC09:37
openstackgerritMerged openstack/tripleo-heat-templates: Remove default for the RabbitMQ password
michchapjaosorior: I'm surprised we're tracking master of puppet deps tbh09:37
michchapjaosorior: are all modules tracking master or do we pin some of them?09:38
jaosoriormichchap: Honestly I'm not sure09:38
*** panda has quit IRC09:40
openstackgerritDougal Matthews proposed openstack/tripleo-docs: Ignore both the AUTHORS and ChangeLog files
*** panda has joined #tripleo09:41
openstackgerritDougal Matthews proposed openstack/tripleo-docs: Correct a link the basic_deployment document
*** apetrich has quit IRC09:42
*** apetrich has joined #tripleo09:42
*** ohamada has quit IRC09:57
*** mbound has quit IRC09:59
*** ohamada has joined #tripleo10:01
*** Marga_ has quit IRC10:03
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Enable heat-manage purge_deleted cron job
snecklifterI need to enable some kernel module parameters which are disabled by default in deployed images10:11
snecklifterwhats the best way to achieve this?10:11
snecklifterspecifically vxlan offload in mlnx adapters10:11
*** jaosorior has quit IRC10:14
*** jaosorior has joined #tripleo10:15
mariossnecklifter: can you use (sysctl?)10:16
sneckliftermarios: thanks, just looked but i'm not sure it will do mellanox-specific stuff10:27
snecklifterlike at the moment I achieve it with adding options to /etc/modprobe.d/mlx4.conf10:27
snecklifteralthough it looks like i can do general ipv4 tuning with that so thanks10:28
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Remove GlanceRegistry from EndpointMap
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Redirect to https for horizon
mariossnecklifter: np - just rang a bell with something i looked at recently ( so thought it may apply10:32
*** links has joined #tripleo10:34
chemhi guys, how do I install the delorean repo for fedora 23 ?10:37
chemhum wrong channel I think, sorry10:38
openstackgerritGiulio Fidente proposed openstack/tripleo-docs: Update list of pkgs in includepkgs
*** tosky has joined #tripleo10:46
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Update wanted_ci_job to reflect current status of things
gfidentederekh, just to make sure, in
gfidenteI see  No valid host was found. There are not enough hosts available.11:03
gfidentebut NODECOUNT should be enough there right?11:04
derekhgfidente: looking11:05
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Use --overcloud-update in the upgrades job
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: puppet pacemaker module upgrade.
derekhgfidente: weird, we're setting --ceph-storage-scale 111:08
derekhgfidente: but its deployed two11:09
gfidentewait ... derekh shit11:09
gfidentepuppet-ceph-devel does that :(11:09
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: puppet pacemaker module upgrade.
derekhgfidente: it does what?11:09
gfidentecheck yourself puppet-ceph-devel.yaml11:10
gfidenteyou will notice :)11:10
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: puppet pacemaker module upgrade.
gfidenteI'd remove the count from there and rely on the client11:10
gfidentejust for consistency with the other env files11:10
gfidenteand because it's bad to override user provided --args anyway11:11
derekhgfidente: ok, will do, thanks11:11
*** rdopiera has joined #tripleo11:11
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates: Remove CephStorageCount
*** devvesa has quit IRC11:15
openstackgerritQasim Sarfraz proposed openstack/tripleo-heat-templates: PLUMgrid Neutron integration
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job
*** jprovazn has quit IRC11:16
openstackgerritAttila Darazs proposed openstack/python-tripleoclient: Fix no_proxy list generation for overcloudrc
*** leanderthal|afk is now known as leanderthal11:20
*** rhallisey has joined #tripleo11:21
rhalliseyderekh, container job is working again11:23
sneckliftermarios: fyi, after some research, i think guestfish will do it, allows mounting of images and modification of files11:26
derekhrhallisey: great, looking now11:26
snecklifterbut thanks for responding earlier11:26
gfidentesnecklifter, so you don't need to edit the image though11:30
gfidentemarios pointed you to the right place
gfidenteyou just need to add the name of the module there, for example11:30
*** thrash|g0ne is now known as thrash11:31
gfidente  nf_conntrack: {}11:31
gfidente  mlx4: {}11:31
gfidenteand that should do it11:31
sneckliftergfidente: ah ok, and that will allow me to pass something like:11:32
snecklifteroptions mlx4_core debug_level=1 log_num_mgm_entry_size=-111:32
gfidenteoh the module options can go in the {}11:33
sneckliftergfidente: ok, thanks, so something like:11:33
gfidentesec I need to figure the syntax11:33
snecklifter  mlx4:{options mlx4_core debug_level=1 log_num_mgm_entry_size=-1}11:33
sneckliftergfidente: ok, thanks, yes that would be easier than modifying image11:34
*** mgould has quit IRC11:34
sneckliftergfidente: its not clear to me...11:37
*** dtantsur is now known as dtantsur|brb11:37
gfidenteso the puppet module handles the module options in a different way, it expects to write those down into modprobe.conf before loading the module11:37
gfidentebut when loading the module it just gives the entire key to modprobe so i think you can trick it like this11:38
*** stendulker has quit IRC11:38
gfidente  nf_conntrack: {}11:38
*** dshulyak has quit IRC11:38
adarazsderekh: hi! can you tell me how you change the type of job you run on that jenkins env? I don't see it in the logs. I want to run the "ha" job instead of others.11:39
gfidente  "mlx4_core debug_level=1 log_num_mgm_entry_size=-1": {}11:39
gfidente  "mlx4": {}11:39
gfidenteyou can try without passing any option first to make sure the module is correctly loaded into the kernel11:40
snecklifterits getting loaded fine at the moment, its just not enabling the vxlan offload11:40
gfidenteso what you needed was the actual feature we miss about dumping module options into modprobe.conf11:41
sneckliftergfidente: exactly11:41
gfidenteI'll try to track this with a BZ, but the workaround is worth trying imho11:41
gfidentecause editing images is not easily maintenable11:42
snecklifterso something like:11:42
gfidentesnecklifter, yep11:42
sneckliftergfidente: ok, thanks, i'll give that a go and let you know11:42
snecklifterno, thank you. :)11:42
derekhadarazs: whats the IP address again, I'll log into the screen session and show you11:43
derekhadarazs: its an env variable called TOCI_JOBTYPE or something like that11:43
adarazsderekh: ssh jenkins@
adarazsderekh: oh cool, thanks :)11:44
derekhadarazs: no prob, that should be it11:44
adarazsderekh: don't run the job yet, I'll have to update my patch.11:44
*** masco has quit IRC11:45
derekhadarazs: no prob, fire ahead yourself when ready11:45
adarazsthanks, will do.11:45
adarazsderekh: the env helped me a lot yesterday, I figured out that we can't use IPv6 floating IPs. :)11:45
adarazsderekh: will the Depends-On stuff be pulled in or that's done by the wrapper script?11:46
adarazsaround toci_gate.sh11:46
*** rdopiera has quit IRC11:47
*** rdopiera has joined #tripleo11:47
derekhadarazs: it wont get pulled in, I'll have to set that up manually if you want to test it11:48
adarazsderekh: where should I clone the extra repo for it to work in relation to /opt/stack/new/tripleo-ci ?11:49
derekhadarazs: it shouldn't take long, let me know when your ready and I'll do it on tab 311:49
*** mgould has joined #tripleo11:49
adarazsderekh: okay, I have the proper tripleo-ci patch in place, but we need this:
*** rdopiera has quit IRC11:51
adarazsso outside out tripleo-ci and delorean will find it. thanks :)11:51
*** rdopiera has joined #tripleo11:51
derekhadarazs: has to be in that dir,11:51
derekhadarazs: then we have to set ZUUL_CHANGES so tell tripleo-ci it needs to be built11:52
derekhadarazs: hmm, I's lost my connection, or its frozen one sec11:53
adarazsderekh: froze for me as well11:53
adarazsso it should be ZUUL_CHANGES="openstack/python-tripleoclient:master:refs/changes/26/293626/5^openstack-infra/tripleo-ci:master:refs/changes/45/289445/16"11:54
*** saneax is now known as saneax_AFK11:54
derekhadarazs: Yup, when it comes back if you set that it should do the right thing11:54
adarazsat least that's what I see from my previous "" in the gate job :)11:54
*** dshulyak has joined #tripleo11:55
adarazsderekh: this is not needed? ZUUL_CHANGE_IDS="293626,4 289445,16"11:55
derekhadarazs: I've lost my connection to a baremetal host on the rack aswell so we may need to wait until its back11:55
adarazsderekh: now it works for me.11:56
adarazsI can type11:56
derekhadarazs: nope that's what tells the infra tools to do the cherry-pick that I just did11:56
derekhadarazs: you may need to export it11:57
derekhadarazs: opps you did11:57
adarazsall righty, all seems to be set. I'll try the job :)11:57
derekhcarry on11:57
derekhgo for it11:57
*** weshay has joined #tripleo11:58
derekhadarazs: then the undercloud comes up just double check that it has the tripleoclient patch11:58
adarazsgood idea11:59
*** mbound has joined #tripleo11:59
adarazsderekh: not sure how you logged in to the undercloud machine, I had to make a file with the key of the seed machine and use that. I hacked myself in, but I think you did it with some default password that I don't know.12:01
derekhjenkins public key should have been added to roots authorized_keys on the undercloud node12:03
slaglemichchap: hi, any chance this commit, could be causing this failure in tripleo ci:
openstackgerritRyan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again
slagleEmilienM: oh hi :) maybe you could have a look too ^12:07
EmilienMsure, CI broken?12:07
slaglelooks like it might be12:07
EmilienMok, looking12:07
jaosoriorslagle: I think the CI is passing again already12:08
slaglejaosorior: oh is it?12:08
jaosoriorslagle: That commit you pointed out was meant to fix it12:08
*** jcoufal has joined #tripleo12:08
EmilienMthat is Terrible code this puppet-redis commit, Terrible with a T12:08
EmilienMand slagle makes me read that at 8am12:08
slaglejaosorior: thx, i'll keep an eye on the running jobs12:09
gfidenteyour fault for waking uo12:09
*** mcornea has quit IRC12:09
slagleEmilienM: st. patty's day hangover?12:09
adarazsEmilienM: one less cup of coffee needed to wake up :)12:09
EmilienMslagle: don't tell anyone.12:09
EmilienMoh michchap wrote that oops12:10
openstackLaunchpad bug 1558962 in tripleo "Error: Could not find dependent Service[redis] for Exec[cp -p /etc/redis.conf.puppet /etc/redis.conf] at /etc/puppet/modules/redis/manifests/config.pp" [Undecided,Triaged]12:11
*** mcornea has joined #tripleo12:13
adarazsEmilienM: what's the state of that facter error causing problems in IPv6? do we have some workaround that I could try in the IPv6 gate job patch?12:14
*** jayg|g0n3 is now known as jayg12:14
EmilienMadarazs: chem has a patch:
adarazsEmilienM: excellent, I add it to my patch.12:15
EmilienMslagle: I'm writting a patch, in the meantime I suggest you to pin CI12:15
EmilienMjaosorior: what is the commit that broke CI?12:16
jaosoriorEmilienM: AFAIK the CI has been fixed12:16
slagleEmilienM: i think is the fix actually12:16
slaglethat's what people are telling me :)12:16
*** mbound has quit IRC12:16
jaosoriorat least an hour ago some commits started passing12:16
EmilienMok so broke OOO CI12:17
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs
EmilienMdoing a "cp" in Puppet... /me sigh12:17
EmilienMand michchap patch is fixing it, perfect12:17
*** dtantsur|brb is now known as dtantsur12:19
michchapEmilienM: do we chase master on all puppet module deps outside of puppet-openstack?12:22
michchapslagle: That's the one that fixes it I hope12:23
slaglemichchap: yep, thanks :)12:23
EmilienMmichchap: *all* puppet modules are on master12:23
michchapslagle: EmilienM is right it's bad in many ways :(12:23
*** dprince has joined #tripleo12:23
EmilienMand for stable/liberty, AFIK we use OPM12:24
*** jprovazn has joined #tripleo12:24
michchapEmilienM: ah, so we get random breakage when puppetlabs decide to change interfaces for things12:24
gfidentemichchap, yes we hit these from time to time12:25
*** lucasagomes is now known as lucas-hungry12:25
gfidentebut I am not sure it's a bad thing, it's an alarm to react, we'd hit that anyway just later in the process I think12:26
*** paramite has quit IRC12:26
*** paramite has joined #tripleo12:27
*** ohamada has quit IRC12:30
*** bswartz has joined #tripleo12:31
*** pradk has joined #tripleo12:32
*** akuznetsov has joined #tripleo12:32
*** ohamada has joined #tripleo12:33
*** ohamada has quit IRC12:35
*** Marga_ has joined #tripleo12:37
*** Goneri has joined #tripleo12:37
*** morazi has joined #tripleo12:43
*** MaxPC has joined #tripleo12:44
*** rlandy has joined #tripleo12:44
*** openstackgerrit has quit IRC12:48
*** openstackgerrit has joined #tripleo12:48
*** jistr has quit IRC12:51
*** jistr has joined #tripleo12:51
*** tesseract- has quit IRC12:59
openstackgerritMichele Baldessari proposed openstack/python-tripleoclient: Clarify error when a template is missing
gfidentebandini, oh yeah it's only overcloud.yaml and overcloud-without-mergepy.yaml missing :P13:00
bandinigfidente: :)13:00
bandinigfidente: not when you are drunk lik eme and mistype everything :P13:00
sneckliftergfidente: no luck with the mlx4 deployment sadly13:00
*** lblanchard has joined #tripleo13:00
gfidentesnecklifter, hmm as in it fails?13:01
snecklifteryeah, dependency? on what I wonder?13:02
bandinibecause a kernel module named mlx4 does not exist?13:05
bandinimlx4_en if you need the ethernet one, or mlx4_ibfor the infiniband?13:05
* bandini guessing wildly13:05
gfidentesnecklifter, no it's sysctl which depends on kmod and failed to load because kmod failed13:05
gfidenteExec[modprobe mlx4] has failures: true13:05
bandinigfidente: the mlx4 kernel module does not exist13:06
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Make cipher suite and SSL options configurable
snecklifterbandini: excellent point sit13:06
bandinisnecklifter: are you trying to make it work on infiniband or ethernet?13:07
bandinijust out of curiosity13:07
snecklifterbandini: ethernet13:07
*** tesseract has joined #tripleo13:08
*** tesseract is now known as Guest8468813:08
gfidenteshadower, ovs-agents still gets oom for me from time to time, did you get to find anything?13:09
shadowergfidente: not really. When I redeployed everything, it stopped. But I also set it up with 2 cores and 9GB of mem13:10
snecklifterbandini: gfidente: redeploying again, will keep you posted13:11
shadowergfidente: I mean, I found that swift-proxy-server was taking up a ton of memory during machine boot (image upload) but that was it13:13
shadower*where machine = overcloud node13:14
*** ohamada has joined #tripleo13:19
*** links has quit IRC13:22
*** akshai has joined #tripleo13:22
openstackgerritMerged openstack/tripleo-common: Upgrades: Add post-upgrade stack update
*** mgould has quit IRC13:28
*** rbrady has joined #tripleo13:30
*** mbound has joined #tripleo13:31
*** mbound has quit IRC13:31
*** lucas-hungry is now known as lucasagomes13:31
*** mbound has joined #tripleo13:33
*** rdopiera has quit IRC13:34
*** panda has quit IRC13:40
*** panda has joined #tripleo13:41
*** shivrao has joined #tripleo13:45
chemhi, I've got a failing pingtest on gate-tripleo-ci-f22-ha, while everything seems in good order .  Does it ring a bell to anybody ?13:47
*** shivrao has quit IRC13:47
chemshould I just do a recheck or something is wrong ?13:48
*** jaosorior has quit IRC13:54
*** tzumainn has joined #tripleo13:56
*** rdopiera has joined #tripleo13:57
*** mbound has quit IRC13:59
*** rcernin has quit IRC14:01
openstackgerritMerged openstack/python-tripleoclient: Fix no_proxy list generation for overcloudrc
*** mbound has joined #tripleo14:06
openstackgerritMarius Cornea proposed openstack/tripleo-heat-templates: Comment out the ManagementPort in external-loadbalancer-vip.yaml
*** trozet has joined #tripleo14:08
*** links has joined #tripleo14:09
*** akuznetsov has quit IRC14:13
*** leanderthal has quit IRC14:14
bnemecchem: Not sure what happened there, but I would try a recheck first.14:17
chembnemec: oki, thanks14:17
*** leanderthal has joined #tripleo14:17
openstackgerritMarius Cornea proposed openstack/tripleo-heat-templates: Comment out the ManagementPort in external-loadbalancer-vip.yaml
EmilienMcan someone review this blocker for ipv6 please?
*** pradk has quit IRC14:24
*** pradk has joined #tripleo14:24
EmilienMchem: did you send a PR to facter upstream about the regex issue?14:26
chemEmilienM: no, I though pena should have the code, but as he's not there for some time, I may do the PR as well14:27
chemEmilienM: even If I changed the regexp slightly ... :)14:27
*** yamahata has joined #tripleo14:32
openstackgerritRyan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again
pradkjistr, slagle,  hi, the aodh tripleoclient patch is now passing CI .. just needed a depends on, i tested this yesterday and everything is looking good14:34
*** ebalduf has joined #tripleo14:35
jistrpradk: so just this one is red on -upgrades job14:35
jistrand the rest is green now14:35
* jistr looks at the CI log14:35
pradkjistr, ah ok, are we running upgrade jobs? i thought that was same as nonha currently?14:36
pradkah doesnt look like it even kicked off the install on that job, fails pretty early14:36
jistrpradk: for now it's just called that way. It doesn't test upgrades, but i think it tests Ceph still (it used to be a -ceph job, now its renamed according to future plans)14:36
jistrshould be green in general
pradkso that job did pass once on Mar 10.. i'll recheck and see14:38
*** pcaruana has quit IRC14:38
jistrpradk: yeah just saw that14:38
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Split out controller manifest steps
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Configure ControllerServices via resource chains
jistrpradk, slagle: ok let's see how the recheck goes. Given that we have the patch on top of it which is all green, and even this one passed on Mar 10, i wouldn't personally object against merging the patch even if one of the jobs comes up red again after recheck.14:40
*** yamahata has quit IRC14:45
*** Guest84688 has quit IRC14:48
snecklifterbandini: gfidente: deployment finished successfully but offload not enabled unfortunately14:50
sneckliftercurrently debugging14:50
*** ifarkas has quit IRC14:52
d0ugalAnyone got time to review this?
*** links has quit IRC14:54
*** links has joined #tripleo14:56
*** links has quit IRC14:57
*** links has joined #tripleo14:57
*** yamahata has joined #tripleo14:58
ayoungSo I Hit this again:
*** tesseract has joined #tripleo15:00
*** tesseract is now known as Guest7688315:00
*** akuznetsov has joined #tripleo15:13
*** jprovazn has quit IRC15:13
*** dtantsur is now known as dtantsur|pto15:16
*** sanjayu has joined #tripleo15:18
*** sanjayu has quit IRC15:18
*** chem has quit IRC15:19
*** links has quit IRC15:19
EmilienMgfidente: I don't think is causing failure on HA job15:24
*** mgould has joined #tripleo15:24
EmilienMsounds like ping test issue15:25
*** Marga_ has quit IRC15:27
*** trozet has quit IRC15:29
*** trozet has joined #tripleo15:29
*** paramite is now known as paramite|afk15:31
*** paramite|afk is now known as paramite15:31
*** paramite is now known as paramite|afk15:32
*** david-lyle has quit IRC15:32
*** david-lyle has joined #tripleo15:33
openstackgerritBen Nemec proposed openstack/instack-undercloud: Switch to package-installs
*** jcoufal has quit IRC15:41
*** akuznetsov has quit IRC15:42
*** akuznetsov has joined #tripleo15:47
*** paramite|afk is now known as paramite15:51
*** michchap has quit IRC15:51
*** rdopiera has quit IRC15:56
*** paramite is now known as paramite|afk15:56
ayounglarsks, tripleo quickstart commits the same sin as instack.  192.0 is not a legal network space to use.16:00
larsksayoung: yes, I know.16:00
ayoungand,. since it is an undercloud sin16:00
*** michchap has joined #tripleo16:00
ayoungI need to tear down my whole thing...again16:00
ayounglarsks, OTOH the quickstart works very well16:00
*** david-lyle has quit IRC16:00
ayoungI'm kindof in love with it16:00
larsksayoung: fixing the address ranges is on my list :)16:00
larsksglad it is working otherwise.16:00
ayounglarsks, I randomly chose 10.149.16:00
ayounglarsks, question on Horizon16:01
*** david-lyle has joined #tripleo16:01
larsksany chance you can open a github issue on that topic?16:01
ayounghow do I get to see it"?16:01
*** mbound has quit IRC16:01
larsksWell, that's a good question! :) We were just discussing solutions to that at the bootcamp this week.  At the moment, you would need to port-forward through the undercloud.16:02
ayounglarsks, its two layers deep.16:02
ayoungand I need 2 ports/ 80 and 44316:02
ayoungplus I am setting up a second server that I also need to see.16:03
ayoungso...that is really painful16:03
ayoungand...its seems like that is just denying basic access to the overcloud16:03
ayoungwhat is the right answer?16:03
ayounglong term larsks16:03
larsksWell, first, as you discover problems or have feature requests, log them at so we don't lose track of them.16:04
larsksPossible solutions including giving all the overcloud nodes interfaces on the "external" libvirt network, so that they can be accessed from the virt host.16:04
larsksThis at least removes a layer of indirection.16:04
ayounglarsks, so I would do that by hacking the setup of the overcloud vms?16:05
*** leanderthal is now known as leanderthal|afk16:05
*** chem has joined #tripleo16:06
larsksMaybe.  If we are binding services to particular ip addresses that might not work (e.g., even with an interface on the external network, haproxy on the controllers might not listen to it).16:06
larsksThat's something we'll need to explore a little more.16:06
larsksIdeally we want a solution that doesn't require deviating too much from a standard deploy.16:06
ayounglarsks, the standard deploy has this problem too16:07
ayoungwe put everything on an internal network, and provide not access to it.16:07
larsksAn actual tripleo deploy on baremetal hardware?  That sounds crazy.16:07
larsksI mean, like, people using tripleo to deploy in "production"?16:08
openstackgerritMerged openstack-infra/tripleo-ci: Provide fallback mirrors for centos pin
*** mbound has joined #tripleo16:14
ayounglarsks, wacky, right?  We need a way to make the overcload available, and it should probably be an "after install" thing, so you could keep it isolated up to a point?16:16
ayounglarsks, so I was using sshuttle, but that is python, and does not play nicely with the quickstart way of putting new keys into ~/.quickstart/16:17
ayoungdoes not honor the -i option from ssh16:17
larsksayoung: I may need to find some actual hardware and see what a non-virtual install looks like myself.16:18
ayounglarsks, well, we can kindof figure that out, can't we?16:19
ayoungI mean, you had to have the machiens PXE bootable, or at least booted and ready for Ironic16:19
ayoungassume all of the undercloud machines are baremetal.16:19
larsksayoung:  maybe.  But I don't buy that we're deploying at customer sites without any overcloud, presumably this is solved for some use cases.16:19
ayounglarsks, right, I am saying that it probably works fine16:20
ayoungI think this is a side effect of how we build the vms for this deploy16:20
*** saneax_AFK is now known as saneax16:20
*** yamahata has quit IRC16:20
ayoungwhen we set up an overcloud controller, we put that on a vm for a live deploy, right?  Or, no, I guess we use Ironic, so it is on a no network setup needed16:21
ayoungI mean, physica network is already in place16:21
larsksSo, again, there are various options for making it easier to access the overcloud nodes from the virt host with the quickstart.  I'm going to be looking at those soon.  If there aren't github issues on the topic, it may get overlooked in favor of other things.16:22
larsksI think the quickstart may always require some sort of port forwarding for overcloud access, because we can't assume that ports 80 and 443 are free on the virt host, and there may even be multiple quickstart deployments on the same host.16:22
*** penick has joined #tripleo16:23
*** penick_ has joined #tripleo16:24
*** akuznetsov has quit IRC16:25
*** dprince has quit IRC16:26
ayounglarsks, gah!  I just edited all of the template files to change 192.0 to 10.149 and it still deployed using 192.016:27
*** penick has quit IRC16:28
*** penick_ is now known as penick16:28
larsksayoung: there are some patches under review right now that make it easier to set address ranges from variables without requiring any template editing.16:28
ayounglarsks, link?16:28
ayounglarsks, we using gerrit for these, I take it, as there is only one pull request16:29
larsksE.g., and
*** mkovacik has quit IRC16:30
*** akuznetsov has joined #tripleo16:31
*** Hunner has left #tripleo16:32
*** rodrigods has quit IRC16:32
*** rodrigods has joined #tripleo16:33
*** sanjayu__ has joined #tripleo16:33
*** aufi has quit IRC16:33
ayounglarsks, so, I have some requirements beyond the basics. I need to create an additional overcloud VM, and throw a bunch of identity stuff on there.16:34
ayoungI tried creating a new flavor (idm) but ironic would not accept it.  Couldn't boot, even though it was identical to compute16:35
ayoungI was able to boot compute, though.  So, work around.16:35
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs
*** sanjayu__ has quit IRC16:35
ayoungbut I assume the flavor is used to differentiate between the size of the vms used for tasks, and we should be able to vary from the compute flavor16:35
*** sanjay__u has joined #tripleo16:35
ayounglarsks, also...trying to add a "" that just calls the teardown playbook16:36
*** dshulyak has quit IRC16:36
ayoungits alittle wonky...I'll get there16:36
pradkslagle, bnemec,  seeing this fail os-cloud-config python jobs this was passing until yesterday  .. did some package get updated on ci node?16:38
*** trozet has quit IRC16:38
slaglepradk: the python27/python34 jobs don't run on the tripleo-ci cloud16:41
pradkslagle, ah ok, wonder what changed then16:42
slaglei think all the requirements.txt are pip installed16:42
bnemecThat looks like a bug in OpenSSL, or some sort of version incompatibility.16:42
slaglemaybe something in pyOpenSSL?16:43
slaglelet's check if they had a new release16:43
slaglei don't see one16:43
slaglebut it could be in another dep16:43
*** mikelk has quit IRC16:44
slaglepradk: you can run those jobs locally via tox. do they pass?16:44
pradkslagle, yea locally tox passes for me16:44
slagledid you run with force recreate?16:44
*** mcornea has quit IRC16:45
bnemecLooks like lbaas had the same problem recently:
*** sanjay__u has quit IRC16:46
*** sanjay__u has joined #tripleo16:46
pradkrunning with force lets see16:46
pradkah yep see the error locally too in master16:47
*** yamahata has joined #tripleo16:47
*** pcaruana has joined #tripleo16:47
pradkpyOpenSSL>=0.14 is whats in requirements.txt16:48
*** paramite|afk is now known as paramite16:48
*** Marga_ has joined #tripleo16:55
pradkhmm i see the same in master without this patch.. but dont see any recent changes to requirements17:01
bnemecpradk: There doesn't have to be a change in requirements.  Anything that uses >= is uncapped, so a new release will get used automatically.17:02
bnemecIf there wasn't a pyOpenSSL release in the past few hours, there was probably one of something it depends on that caused this.17:03
pradkyea but thought slagle said he dint see any new release?17:03
pradkyea i guess some dependency on pyopenssl is barfing here17:03
pradkcould one of you confirm if you see the same locally?17:04
*** saneax is now known as saneax_AFK17:05
bnemecpradk: It's happening in the gate and locally for you. I'm reasonably certain this isn't environment-specific.17:05
*** ccamacho is now known as ccamacho|afk17:07
bnemecLook what just released today:
* bnemec would like to put money on cryptography-1.3 being what broke this17:10
pradkyea thats quite possible.. if i run with my system deps all works (cryptography 1.1)17:13
*** Nisha has joined #tripleo17:23
openstackgerritMerged openstack/instack-undercloud: Remove unused scripts
NishaHello tripleo17:24
*** dshulyak has joined #tripleo17:25
NishaI work on ironic module of openstack. We use DIB to build images using different elements. The ironic-agent element requires the packages to be added to the ramdisk for the Ironic-python-agent to work without any issue for the new feature in which we support partition images for agent drivers17:25
NishaThat feature is partially merge in Ironic and the rest of the patches will also be merged by next week when we cut stable Mitaka17:26
NishaThis patch is required for ironic to work flawlessly for this feature17:27
Nishajroll, see above17:27
*** pblaho has quit IRC17:30
*** ohamada has quit IRC17:36
*** panda has quit IRC17:40
*** alop has joined #tripleo17:40
*** panda has joined #tripleo17:41
*** lucasagomes is now known as lucas-afk17:44
*** david-lyle_ has joined #tripleo17:44
*** david-lyle has quit IRC17:45
*** mbound has quit IRC17:45
*** david-lyle_ is now known as david-lyle17:46
*** akshai has quit IRC17:49
*** myoung has joined #tripleo17:50
pradkbnemec, should we just add cryptography<1.3 in requirements.txt ? seems to help locally at least17:51
pradkwe use global req in tripleo?17:51
*** dshulyak has quit IRC17:53
*** derekh has quit IRC17:55
openstackgerritRyan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again
bnemecpradk: That might be a temporary workaround.  We need a fix though.  I've opened
openstackgerritPradeep Kilambi proposed openstack/os-cloud-config: Add a cap on cryptography version
pradkbnemec, ^^ referred your link17:59
bnemecpradk: It needs to be proposed to global-requirements first, then the bot will sync it over.17:59
pradkok thats what i thought and asked above, will do17:59
pradkactually would be a good test to see if ci passes18:02
pradki'll leave it there as a test18:02
bnemecpradk: You'll want to reference
openstackLaunchpad bug 1559241 in os-cloud-config "cryptography-1.3 breaks unit tests" [Critical,Triaged]18:02
*** shivrao has joined #tripleo18:03
bnemecThis is why we don't release on Fridays. :-)18:04
pradkindeed :)18:04
openstackgerritBen Kero proposed openstack/diskimage-builder: Add new posix element.
*** tosky has quit IRC18:10
openstackgerritBen Kero proposed openstack/diskimage-builder: Add new posix element.
larsksI've hit a problem with the undercloud image we're generating in tripleo-quickstart; swift-proxy is failing to start because: IOError: [Errno 2] No such file or directory: '/etc/swift/account.ring.gz'18:13
larsksI am curious if this is an issue anyone has seen before.18:13
larsksIt looks like puppet may be starting the service before that file gets created.18:14
larsks(At the conclusion of the 'undercloud install' process, that file does exist)18:14
*** Guest76883 has quit IRC18:15
*** akshai has joined #tripleo18:19
*** paramite has quit IRC18:22
*** dprince has joined #tripleo18:28
*** jistr has quit IRC18:33
*** mbound has joined #tripleo18:38
*** mbound has quit IRC18:39
*** mbound has joined #tripleo18:39
openstackgerritBen Kero proposed openstack/diskimage-builder: Fix add-apt-repository package for debian
openstackgerritBen Kero proposed openstack/diskimage-builder: Fix add-apt-repository package for debian
*** trozet has joined #tripleo18:54
*** mgould has quit IRC18:58
*** ibravo has joined #tripleo19:02
EmilienMcould we have a review on please? It's really helping to have IPv6 in our CI19:04
*** dprince has quit IRC19:08
*** dprince has joined #tripleo19:08
*** chlong has quit IRC19:12
*** Marga_ has quit IRC19:16
*** yamahata has quit IRC19:17
*** morazi has quit IRC19:21
*** rpothier has joined #tripleo19:26
*** rpothier has left #tripleo19:27
*** trozet has quit IRC19:27
*** mbound has quit IRC19:29
*** akshai has quit IRC19:30
*** akshai has joined #tripleo19:43
*** yamahata has joined #tripleo19:59
openstackgerritGonéri Le Bouder proposed openstack/instack-undercloud: ironic: turn on ipxe timout
*** Nisha has quit IRC20:00
*** Nisha has joined #tripleo20:00
*** akshai has quit IRC20:14
*** akshai has joined #tripleo20:18
*** dprince has quit IRC20:22
*** Marga_ has joined #tripleo20:22
*** akshai has quit IRC20:23
*** Nisha has quit IRC20:23
*** Nisha has joined #tripleo20:24
*** Nisha_away has joined #tripleo20:25
*** Nisha has quit IRC20:29
*** mbound has joined #tripleo20:30
openstackgerritGonéri Le Bouder proposed openstack/instack-undercloud: ironic: turn on ipxe timout
*** mbound has quit IRC20:35
*** morazi has joined #tripleo20:37
*** akshai has joined #tripleo20:42
*** jayg is now known as jayg|g0n320:46
*** jdob has quit IRC20:53
*** Nisha_away has quit IRC20:54
*** Nisha has joined #tripleo20:55
*** mbound has joined #tripleo20:56
*** Nisha_away has joined #tripleo21:06
*** david-lyle has quit IRC21:06
*** david-lyle has joined #tripleo21:07
*** gfidente has quit IRC21:07
*** Nisha has quit IRC21:07
*** trozet has joined #tripleo21:08
*** zaneb has quit IRC21:08
*** Nisha_brb has joined #tripleo21:09
*** Nisha_away has quit IRC21:09
*** Nisha_away has joined #tripleo21:10
*** lblanchard has quit IRC21:11
*** Nisha_brb has quit IRC21:13
*** Nisha_brb has joined #tripleo21:14
*** Nisha_away has quit IRC21:14
*** ccamacho has joined #tripleo21:22
*** akshai_ has joined #tripleo21:23
openstackgerritJeff Peeler proposed openstack/tripleo-common: Upload docker image files from yaml config file
openstackgerritJeff Peeler proposed openstack/tripleo-common: Build image files from definitions in yaml
*** Nisha_brb has quit IRC21:24
*** ryansb_ has joined #tripleo21:24
*** ryansb_ has joined #tripleo21:24
*** Nisha has joined #tripleo21:24
*** bhunter_ has joined #tripleo21:25
*** bhunter has quit IRC21:25
*** bhunter_ is now known as bhunter21:25
*** Nisha has quit IRC21:25
*** ryansb has quit IRC21:25
*** greghaynes has quit IRC21:25
*** Tristitia has quit IRC21:25
*** ryansb_ is now known as ryansb21:25
*** ccamacho|afk has quit IRC21:25
*** andreaf has quit IRC21:26
*** akshai has quit IRC21:26
*** mordred has quit IRC21:26
*** wendar has quit IRC21:26
*** dhellmann has quit IRC21:26
*** wendar has joined #tripleo21:26
*** dhellmann has joined #tripleo21:26
*** mordred has joined #tripleo21:27
*** greghaynes has joined #tripleo21:27
*** andreaf has joined #tripleo21:28
*** Tristitia has joined #tripleo21:29
*** rlandy has quit IRC21:36
*** Goneri has quit IRC21:38
*** panda has quit IRC21:40
*** panda has joined #tripleo21:41
*** Marga__ has joined #tripleo21:45
*** Marga__ has quit IRC21:48
*** Marga_ has quit IRC21:48
*** Marga_ has joined #tripleo21:48
*** Marga_ has quit IRC21:56
*** pradk has quit IRC21:58
*** akuznetsov has quit IRC22:08
*** alop has quit IRC22:11
*** Marga_ has joined #tripleo22:13
*** trozet has quit IRC22:15
*** MaxPC has quit IRC22:28
*** r-mibu has quit IRC22:30
*** r-mibu has joined #tripleo22:30
*** Goneri has joined #tripleo22:31
*** ebalduf_ has joined #tripleo22:33
openstackgerritMerged openstack/tripleo-common: Upgrades: Add initialization to StackUpgradeManager
*** mbound has quit IRC22:44
*** Goneri has quit IRC22:46
*** Goneri has joined #tripleo22:56
*** akshai_ has quit IRC23:04
*** jtomasek has quit IRC23:09
*** Marga__ has joined #tripleo23:11
*** Marga_ has quit IRC23:15
*** Marga__ has quit IRC23:23
*** Marga_ has joined #tripleo23:24
*** mbound has joined #tripleo23:44
*** mbound has quit IRC23:49

Generated by 2.14.0 by Marius Gedminas - find it at!