Wednesday, 2014-11-26

*** rwsu has joined #tripleo00:02
*** zhenzanz has joined #tripleo00:05
*** zhenzanz_ has joined #tripleo00:06
*** zhenzanz has quit IRC00:09
*** zhenzanz_ is now known as zhenzanz00:09
*** daneyon_ has quit IRC00:09
*** ChuckC_ has quit IRC00:18
*** tzumainn has quit IRC00:20
*** ChuckC_ has joined #tripleo00:25
*** julim has joined #tripleo00:40
*** vinsh has joined #tripleo00:49
*** NobodyCam has quit IRC00:56
*** NobodyCam has joined #tripleo00:58
*** juice has quit IRC00:58
*** juice has joined #tripleo01:01
*** yamahata has joined #tripleo01:01
*** daneyon has joined #tripleo01:02
*** vinsh has quit IRC01:04
SpamapSgreghaynes: yes I know upstart quite well. Reading bug now.01:23
greghaynesSpamapS: The change I posted is also pretty explanatory - basically neutron-ovs-agent forks a few times and then errors, it also takes a while to setup state beofore doing this01:23
greghaynesso coming up with a way for upstart to be aware of when it should consider start completed is... not straightforwar01:24
SpamapSgreghaynes: we shouldn't really be relying on upstart to know it is "started"01:25
SpamapSgreghaynes: systemd will have slightly different ideas about it01:26
greghayneswell we need to either have our own check in o-r-c scripts or make the service start <thing> exit with 101:26
SpamapSgreghaynes: or right after start/restart we poke neutron until it works.01:26
greghaynesand I think the service script failing when the service doesnt start is preferred01:26
greghaynesyea01:27
greghaynes(the check in o-r-c)01:27
SpamapSThat's the only way we get things to work in the Ansible update01:27
SpamapSwith wait_for on the ports01:27
SpamapSfor start and stop01:27
greghaynes:(01:27
SpamapSdaemons suck.01:27
greghaynestrue story01:27
SpamapSI've spent basically 4 years trying to make them not suck with upstart, sysvinit, and now a bit in systemd. They all suck.01:28
greghaynesI wonder if we could make neutron-ovs-agent do something a bit easier to detect when it things its started...01:28
SpamapSThe few daemons that don't suck do this:01:28
SpamapSopen a pipe, fork, read the pipe from the parent, initialize the daemon from the child. write to the pipe and close it in the child, parent returns from read and exits01:29
SpamapSthe problem is, upstart _TOTALLY_ misses that parent dying thing01:29
SpamapSand just thinks "oh it forked, IT IS READY"01:29
greghaynesah!01:29
SpamapSwhich basically only one daemon ever written does, dbus01:29
greghaynesthat is the key that I think explains why this totally fails in upstart :)01:29
SpamapSbut since Keybuk was writing upstart with early plumbing like udev and dbus in mind.. he thought that was how things should work.01:30
SpamapSwell without 'expect fork' upstart doesn't even wait for the fork01:30
greghaynesIs there not a directive for an upstart config of "expect normal_daemon"01:30
SpamapSas soon as the forked process exec's, upstart will consider that pid "started" unless there's a post-start to delay the 'started' event.01:31
greghayneshrm01:31
SpamapSexpect daemon waits for _two_ forks.01:31
SpamapSbecause old school daemons did two01:31
greghaynesyep, that is what I was messing with. maybe you could get clever in post-start and ps | grep for parent PID01:31
greghaynesfor now, my patch is basically01:32
SpamapSps grep isn't necessary01:32
greghaynes"sleep 10"01:32
SpamapSin post-start you know the pid of the main process01:32
greghaynesYes, but seems like the issue is we need to delay in post-start until the parent exits?01:32
SpamapSparent will exist after 'exec /opt/stack/venvs/openstack/bin/neutron-ovs-agent'01:33
SpamapSit just won't be doing anything _useful_ yet.01:33
SpamapSanother thing well behaved old school daemons would do is not write their pid file until they were in operational mode.01:33
greghaynesyep, I read about that a bit when debugging this01:34
SpamapSso that is sometimes effective01:34
greghaynesok, so that gives me some stuff to poke at01:34
SpamapSI think somebody may have landed systemd compatability in oslo.service.. it does something with dbus or signals IIRC01:34
clarkbits not signals. signals are bonghits and are what upstart uses iirc01:35
clarkbinstead they have a socket you write to or some such01:35
SpamapSupstart also has an explicit mode where you can say 'expect stop' and then upstart will wait on the forked process until it sends itself a SIGSTOP, at which point upstart with SIGCONT it and consider it startd.01:35
SpamapSclarkb: right thats it, a socket01:35
greghayneshahaah01:35
SpamapSclarkb: the signals are interesting because it becomes atomic. You an SIGSTOP yourself and know that after that point upstart knows you're started. I forget the single practical example I was given that made this compelling 4 years ago when I learned of it.01:36
clarkbSpamapS: but its a massive abuse01:37
clarkbplease read man 7 signal :)01:37
clarkbsigstop means stop process01:37
clarkbnot I am ready01:37
greghaynesIt does stop it01:38
greghaynesbut I could see how that would be hilarous for a process that expects to actually stop01:38
SpamapSclarkb: Well it is stopping the process you're signalling. :)01:38
SpamapSclarkb: the abuse is the ptrace. :)01:38
SpamapSclarkb: a socket is far simpler and makes me happy.01:39
greghaynesSo, i discovered a reason to be using expect daemon even though we work without it while I was debugging this - bad things happen if upstart is tracking a pid which can eventually be re-used by the OS01:39
greghayneslike upstart can go and kill that pid when it gets reused later on01:40
*** pradeep has joined #tripleo01:47
*** wuhg has joined #tripleo01:52
*** marun has quit IRC01:55
*** pradeep has quit IRC01:55
*** rwsu has quit IRC01:58
SpamapSgreghaynes: yes that is an insidious bug that I've never understood fully01:59
SpamapSgreghaynes: that goes hand in hand with upstart using weird ptracey things to track pids01:59
*** vinsh has joined #tripleo02:02
*** vinsh has quit IRC02:07
*** nosnos has joined #tripleo02:14
*** cwolferh has quit IRC02:15
*** NobodyCam has quit IRC02:26
*** NobodyCam has joined #tripleo02:27
*** sdake_ has joined #tripleo02:31
*** sdake_ has quit IRC02:31
*** sdake_ has joined #tripleo02:31
*** vipul has quit IRC02:33
*** jerryz has joined #tripleo02:35
*** vipul has joined #tripleo02:37
openstackgerritRichard Su proposed openstack/tripleo-image-elements: Run restorecon against actual rabbitmq directories  https://review.openstack.org/13725902:39
*** pradeep has joined #tripleo02:39
*** yuanying_ has quit IRC02:47
*** vinsh has joined #tripleo02:51
openstackgerritRichard Su proposed openstack/tripleo-image-elements: RabbitMQ SELinux policy update  https://review.openstack.org/13726002:52
*** pradeep1 has joined #tripleo02:53
*** pradeep has quit IRC02:55
*** ramishra has joined #tripleo02:59
*** nosnos has quit IRC03:01
*** Marga_ has quit IRC03:03
*** pradeep has joined #tripleo03:03
*** Marga_ has joined #tripleo03:04
*** yuanying has joined #tripleo03:04
*** pradeep1 has quit IRC03:05
*** vinsh has quit IRC03:09
*** pradeep1 has joined #tripleo03:14
*** pradeep has quit IRC03:16
*** pradeep has joined #tripleo03:22
*** killer_prince has quit IRC03:23
*** Marga_ has quit IRC03:24
*** pradeep1 has quit IRC03:25
*** pradeep1 has joined #tripleo03:34
*** jerryz1 has joined #tripleo03:35
*** pradeep has quit IRC03:35
*** jerryz has quit IRC03:37
*** pradeep has joined #tripleo03:37
*** pradeep1 has quit IRC03:39
*** pradeep1 has joined #tripleo03:40
*** pradeep has quit IRC03:42
*** pradeep has joined #tripleo03:45
*** pradeep1 has quit IRC03:46
*** pradeep1 has joined #tripleo03:47
*** pradeep has quit IRC03:49
*** pradeep has joined #tripleo03:52
*** noslzzp has quit IRC03:53
*** pradeep1 has quit IRC03:54
*** pradeep1 has joined #tripleo03:55
*** pradeep has quit IRC03:56
*** noslzzp has joined #tripleo03:56
*** pradeep has joined #tripleo03:57
*** noslzzp has quit IRC03:58
*** pensu has joined #tripleo03:58
*** pradeep1 has quit IRC03:59
*** untriaged-bot has joined #tripleo04:00
untriaged-botUntriaged bugs so far:04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/139121804:00
uvirtbotLaunchpad bug 1391218 in tripleo "Configuration Dependent on HAProxy" [Undecided,In progress]04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/138706104:00
uvirtbotLaunchpad bug 1387061 in tripleo "deploy under cloud failure due to pacemaker " [Undecided,New]04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/139364304:00
uvirtbotLaunchpad bug 1393643 in tripleo "haproxy start failed loop with 'Starting proxy horizon: cannot bind socket'" [Undecided,New]04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/139141204:00
uvirtbotLaunchpad bug 1391412 in tripleo "devtest  deploy overcloud successfully,but there have one baremetal in  Maintenance state" [Undecided,New]04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/138748704:00
uvirtbotLaunchpad bug 1387487 in tripleo "tripleo-image-elements/elements/tuskar-ui /os-refresh-config /post-configure.d/101-tuskar-ui  needs authentication from Keystone" [Undecided,New]04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/139330404:00
uvirtbotLaunchpad bug 1393304 in tripleo "create-nodes failure for customised pool" [Undecided,New]04:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/139165604:00
uvirtbotLaunchpad bug 1391656 in diskimage-builder "There is no way to customize the installation of the bootloader" [Undecided,New]04:00
*** untriaged-bot has quit IRC04:00
*** pradeep has quit IRC04:02
*** pradeep has joined #tripleo04:02
*** pradeep1 has joined #tripleo04:07
*** pradeep has quit IRC04:08
*** yuanying_ has joined #tripleo04:14
*** nosnos has joined #tripleo04:16
*** yuanying has quit IRC04:17
*** killer_prince has joined #tripleo04:18
*** killer_prince is now known as lazy_prince04:18
*** vinsh has joined #tripleo04:28
*** vinsh has quit IRC04:28
*** pradeep has joined #tripleo04:28
StevenKstevebaker: Do you have a sec to talk about bug 1390112?04:29
*** pradeep1 has quit IRC04:29
uvirtbotLaunchpad bug 1390112 in heat "Invalid parameter defaults can't be overriden" [Medium,Triaged] https://launchpad.net/bugs/139011204:29
*** Marga_ has joined #tripleo04:35
*** weshay has quit IRC04:37
*** Marga_ has quit IRC04:39
*** pcrews has quit IRC04:48
*** pradeep has quit IRC04:48
*** pensu has quit IRC05:04
*** pradeep has joined #tripleo05:05
*** rushiagr_away is now known as rushiagr05:10
openstackgerritMerged openstack/diskimage-builder: Warning when using pypi element without a mirror  https://review.openstack.org/13531605:27
*** jerryz1 has quit IRC05:32
*** michchap has quit IRC05:41
*** pensu has joined #tripleo05:48
tchaypogreghaynes: yes05:55
greghayneskk05:57
greghaynes:( hit the spurios node getting in state error deleting bug http://logs.openstack.org/94/136894/3/check-tripleo/check-tripleo-ironic-overcloud-precise-nonha/2278484/console.html05:58
*** rushiagr is now known as rushiagr_away05:59
*** k4n0 has joined #tripleo06:03
greghaynesour meeting is in 1hr?06:03
*** edmund has quit IRC06:06
tchaypohaving met keybuk long before i knew what upstart was06:14
tchaypothis is all very amusing06:14
tchaypogreghaynes: no, 2 hours06:15
tchaypoat least according to the ical feed on https://wiki.openstack.org/wiki/Meetings06:16
tchaypohowever https://wiki.openstack.org/wiki/Meetings#TripleO_team_meeting says 0700UTC, and that’s in an hour06:16
tchaypoor is it?06:18
tchaypo0700 London is in an hour, but are they running on UTC right now?06:18
*** dshulyak_ has joined #tripleo06:23
tchaypoyes, yes they are.06:25
tchaypo> Because of the whining^W weekly-expressed preferences[1] of the06:27
tchaypo> Europe-based folks, the latter meetings are going to be moved by +1 hour.06:27
tchaypo>06:27
tchaypo> So the new meeting times are:06:27
tchaypo>06:27
tchaypo> * Tuesdays at 19:00 UTC (unchanged)06:27
tchaypo> * Wednesdays at 8:00 UTC (1 hour later)06:27
tchaypofrom the mailing list06:27
tchayposo it seems as though the new time was announced on list, and thierry updated the ical, but the wiki was not updated06:27
* tchaypo fixes wiki06:27
tchaypoah, the meeting page was updated, but not the meetings page06:33
* tchaypo talks too much sometimes06:35
tchaypoNov 26 06:42:19 undercloud-undercloud-gxpandxta2cg ironic-conductor: 2014-11-26 06:42:19.785 6915 WARNING ironic.conductor.manager [-] During sync_power_state, could not get power state for node 4b73150c-1c67-4465-baf2-1d09593f6c38. Error: f() takes exactly 1 argument (2 given).06:45
tchaypowhee06:45
*** pcrews has joined #tripleo06:48
tchaypohttps://www.irccloud.com/pastebin/Kgrfk8hq06:48
*** pcrews has quit IRC06:53
*** jtomasek has joined #tripleo06:55
*** jtomasek has quit IRC07:00
*** jprovazn has joined #tripleo07:02
*** mrunge has joined #tripleo07:04
*** rushiagr_away is now known as rushiagr07:07
*** pcrews has joined #tripleo07:23
*** pcrews has quit IRC07:31
*** jtomasek has joined #tripleo07:36
*** penick has joined #tripleo07:38
*** jtomasek has quit IRC07:47
*** gfidente has joined #tripleo07:51
gfidenteare we having the meeting today?07:52
GheRiveroI think so07:53
GheRiveromorning @TripleO07:53
tchaypomorninge07:56
*** athomas has joined #tripleo08:01
*** jtomasek has joined #tripleo08:04
*** michchap has joined #tripleo08:11
greghayneshttps://review.openstack.org/#/c/136894/ should unbreak devtest if another core is around to +A08:12
GheRiveroon my way08:12
greghaynes\O/08:12
*** dtantsur|afk is now known as dtantsur08:13
openstackgerritMerged openstack/tripleo-heat-templates: Fix empty local_ip in ml2_conf.ini on undercloud  https://review.openstack.org/13689408:13
*** lucasagomes has joined #tripleo08:14
*** penick has quit IRC08:14
greghaynestyty08:15
GheRiverogo to bed08:15
greghayneshaha08:15
tchaypoor join the meeting08:16
*** ifarkas has joined #tripleo08:16
StevenKshardy: When is the Heat meeting, so I can bleat about bug 1390112?08:18
uvirtbotLaunchpad bug 1390112 in heat "Invalid parameter defaults can't be overriden" [Medium,Triaged] https://launchpad.net/bugs/139011208:18
*** yamahata has quit IRC08:18
shardyStevenK: It alternates between 12.00UTC and 20.00UTC on Wednesdays - you can nag me directly about it if you like though08:18
StevenKshardy: *nag* ? :-)08:19
shardyit's on my todo list, but I was unclear if we agreed to fix it in heat, or refactor to remove the parameter defaults from the templates in favour of a default environment containing the parameters (which won't have the same problem)08:19
tchaypogreghaynes: i think i hit that exact error this morning, I’m trying to track down what’s causing it. i know where a quick error handler might help, but I don’t understand the cause of the error yet08:19
shardyStevenK: we also have a nearly-landed "parameter_defaults" section for the environment which I think will solve this issue08:20
greghaynestchaypo: the no ip set one?08:20
shardyStevenK: I'm happy to submit the template-level fix for review, if you think that sounds sane08:20
tchaypono, the “could not delete” one08:20
greghaynesah08:20
StevenKshardy: So I can put up a patch to remove the defaults, but my concern with that is if the flavor is in template, we fail fast with a nice error message08:21
shardyStevenK: https://wiki.openstack.org/wiki/Meetings/HeatAgenda08:21
*** dshulyak_ has quit IRC08:21
shardyStevenK: Yeah, I agree, solving it in heat too would be good08:21
shardyStevenK: I started looking into it and it's not that simple, hence considering other approaches ;)08:21
shardyStevenK: Isn't the problem that is *does* fail fast?08:22
shardye.g we should pass validation but fail at runtime when using custom constraints, if the default isn't overriden by a user-provided parameter value?08:22
StevenKshardy: I mean without this issue -- if the default flavor exists, but the one passed in doesn't exist, the stack-create will fail fast with an error message about the flavor. If we don't have that in the template, then the stack create will just fail and you need to read logs.08:24
shardyStevenK: I'm not sure that's correct, if you pass in a flavour which doesn't exist it should fail in the exact same way as if the default doesn't exist08:25
shardyStevenK: Could you paste an example of such a scenario, or some sort of reproducer into the bug?08:26
shardyStevenK: The only difference not having the default should cause is the parameter becomes mandatory (either via passing parameters or putting it in the environment)08:27
shardyStevenK: If it fails slow then it's definitely a bug because the custom constraint on the parameter is not validating the value08:27
*** ifarkas has quit IRC08:28
StevenKshardy: I meant without the custom constraint08:28
*** ifarkas has joined #tripleo08:28
shardyStevenK: Ah, ok, yeah I think we want this to work with the custom constraints but without the default values in the template08:28
shardyThat should make the templates portable but maintain the fail-fast validation08:28
StevenKAh ha08:29
StevenKWe should be a position to drop the default, I think everything is now passed in08:29
shardyStevenK: Ok, that's good :)08:29
*** jcoufal has joined #tripleo08:29
StevenKshardy: I'll test with that tomorrow08:30
shardyStevenK: Sounds good, let me know how it goes08:33
*** greghaynes changes topic to "Using OpenStack to deploy OpenStack; meetings Tuesday 1900/0800 UTC in #openstack-meeting-alt""08:33
*** greghaynes changes topic to "Using OpenStack to deploy OpenStack; meetings Tuesday 1900/0800 UTC in #openstack-meeting-alt"08:33
openstackgerritMerged openstack/tripleo-heat-templates: Don't replace OS::Neutron::Port on update of undercloud  https://review.openstack.org/13343508:35
*** bvandenh has joined #tripleo08:43
*** zz_avozza has quit IRC08:56
*** zz_avozza has joined #tripleo08:57
*** zz_avozza is now known as avozza08:57
*** jistr has joined #tripleo08:59
*** jp_at_hp has joined #tripleo08:59
*** zhenzanz has quit IRC08:59
*** jp_at_hp has quit IRC08:59
*** jp_at_hp has joined #tripleo08:59
*** jprovazn has quit IRC09:01
*** rdopiera has joined #tripleo09:01
*** rdopiera has quit IRC09:03
*** rdopiera has joined #tripleo09:03
*** derekh has joined #tripleo09:12
*** jprovazn has joined #tripleo09:15
openstackgerritMerged openstack/tripleo-image-elements: Add a delorean-repo element  https://review.openstack.org/13686809:24
Ngmornin'09:27
*** gfidente has quit IRC09:28
*** gfidente has joined #tripleo09:28
openstackgerritMerged openstack/tripleo-incubator: Set neutron ovs local-ip in seed correctly  https://review.openstack.org/13638909:32
openstackgerritMerged openstack/tripleo-image-elements: Modify sysctl-set-value script to support multi-value parameters  https://review.openstack.org/13461609:33
openstackgerritMerged openstack-infra/tripleo-ci: Updates to the CI doc  https://review.openstack.org/13371509:34
derekhLooks like ubuntu are updating repositories again09:43
derekh2014-11-26 09:05:12.310 | W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/i18n/Translation-en  Hash Sum mismatch09:43
*** pblaho has joined #tripleo09:43
derekhcausing some CI tests to fail09:43
derekhbecause different mirrors are in a different state http://paste.openstack.org/show/138720/09:44
*** yamahata has joined #tripleo09:44
tchaypolo derekh09:45
derekhlast time this happened we pinned DNS on the CI proxy to a particular IP address until everything had settled down09:45
tchaypowhat would you say to the idea of making the weekly CI/CD update an email instead of an item on the meeting agenda?09:46
tchaypoderekh: seems like a decent solution09:46
derekhtchaypo: Yup, can do and have been doing something like the for the last while (although emails havn't been a regular as weekly)09:47
*** lazy_prince has quit IRC09:48
tchaypoWould it help if you could share the workload instead of having to do it all yourself?09:49
derekhtchaypo: I guess this came up because I wasn't at the meeting ? sorry neither times suit me very well at the moment but I'll09:50
derekhtchaypo: try and make some of them09:50
derekhtchaypo: yes it would, the main thing is reacting to problems and keeping this file uptodate https://etherpad.openstack.org/p/tripleo-ci-breakages09:51
tchaypoderekh: well, one of the things that come up at the meeting09:52
tchaypowas that the current so-called euro-friendly meeting time seems equally unfriendly to both EU and APAC people09:52
tchayposo I’m busy drafting an email suggesting we do something about that09:53
tchayposuch as, for instance, seeing if we can move some of the things we’ve been doing out of the meeting, and having a more focussed meeting09:54
derekhok, sounds good, ya my problem is that its bang in the middle of getting the kids ready for the morning09:54
tchaypoand the other meeting used to be in the middle of ng getting his kids ready for bed09:55
tchaypoi think dst fixed that for him, but now the euro-friendly meeting time is in the middle of APAC dinner time...09:55
openstackgerritstephen-mulcahy proposed openstack/tripleo-image-elements: Add new nscd element  https://review.openstack.org/13709909:55
derekhtchaypo: the other metting used to be ok for me because I could have them off to bed and then attend meeting, now its moved an hour earlier so clashes now09:56
*** jcoufal_ has joined #tripleo09:57
derekhtchaypo: so basically we've leaned, Ng sends his kids to bed an hour later then me09:57
derekh*learned09:57
*** akrivoka has joined #tripleo09:59
*** jcoufal has quit IRC10:00
*** untriaged-bot has joined #tripleo10:00
untriaged-botUntriaged bugs so far:10:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/138706110:00
uvirtbotLaunchpad bug 1387061 in tripleo "deploy under cloud failure due to pacemaker " [Undecided,New]10:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/139165610:00
uvirtbotLaunchpad bug 1391656 in diskimage-builder "There is no way to customize the installation of the bootloader" [Undecided,New]10:00
untriaged-bothttps://bugs.launchpad.net/tuskar/+bug/138748710:00
uvirtbotLaunchpad bug 1387487 in tuskar "tripleo-image-elements/elements/tuskar-ui /os-refresh-config /post-configure.d/101-tuskar-ui  needs authentication from Keystone" [Undecided,New]10:00
*** untriaged-bot has quit IRC10:00
derekhOk, this is now added to the squid server10:00
derekh91.189.91.14 archive.ubuntu.com10:00
derekh91.189.91.14 nova.clouds.archive.ubuntu.com10:00
derekh91.189.91.14 security.ubuntu.com10:00
tchaypo<3 derekh10:00
derekhremind me later to check if it can be removed ;-)10:01
*** rlandy has joined #tripleo10:05
Ngderekh: 7pm every day!10:06
derekhNg: looks like I did that maths the wrong way round, so its my kids that go to bed an hour later10:08
*** subscope has quit IRC10:08
tchaypomy bedtime was 8:30 every night10:11
tchaypountil I went to uni at 1710:12
*** avozza is now known as zz_avozza10:13
*** zz_avozza is now known as avozza10:15
gfidentederekh, thanks :) rechecking10:23
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Add element for hardware discovery ramdisk for ironic-discoverd  https://review.openstack.org/12215110:31
openstackgerritGiulio Fidente proposed openstack/tripleo-incubator: Add support for CEPHSTORAGESCALE to devtest  https://review.openstack.org/13703110:33
*** subscope has joined #tripleo10:34
*** yamahata has quit IRC10:43
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add ceph-source.yaml to the overcloud.yaml Makefile target  https://review.openstack.org/13702810:44
gfidente^^ both don't have deps anymore so they're easier to review10:44
*** yamahata has joined #tripleo10:45
*** killer_prince has joined #tripleo10:56
*** killer_prince is now known as lazy_prince10:56
*** jerryz has joined #tripleo10:58
gfidentetchaypo, derekh so the ceph/ci email is into the wild11:25
tchaypoAre you scared?11:25
gfidenteyeah11:25
gfidentetchaypo, you didn't tell me if germany was sunnyt11:25
gfidenteI was going to suggest you get down here as long as some sun is still there11:26
*** blinky_ghost has joined #tripleo11:28
blinky_ghostHi can anybody help me with network part in overcloud controller? I have a bridge br-ex created through the interface em2 that binds openstack services. The problem is that when I reboot the controller br-ex doesn't get ip anymore.11:31
*** pensu has quit IRC11:41
*** avozza is now known as zz_avozza11:44
*** pradeep has quit IRC11:48
tchaypoDoes os-collect-config put it back?11:53
blinky_ghosttchaypo: br-ex never comes up. apparently it should get ip from dhcp11:57
blinky_ghosttchaypo: is it possible to deploy the overcloud without br-ex? I don't to use br-ex only vlan provider networks. But I can't figure it out how to do this11:59
*** pradeep has joined #tripleo12:12
*** dshulyak_ has joined #tripleo12:15
blinky_ghostlsmola: good morning, did you had the chance to test baremetal deployment?12:15
*** morazi has quit IRC12:16
lsmolablinky_ghost: hello, not yet, still waiting for machines12:16
lsmolablinky_ghost: have you succeed with your test?12:16
blinky_ghostlsmola: I can reach overcloud horizon, I had to change ALLOWED_HOSTS = ['*']. But I cannot login, it reports a authentication problem. So I set selinux=permissive and rebooted the node. The problem is that when I reboot I cannot get ip in br-ex interface anymore.12:19
*** pradeep1 has joined #tripleo12:21
*** pradeep has quit IRC12:23
lsmolablinky_ghost: have you rebooted using ironic console?12:23
*** bvandenh has quit IRC12:24
lsmolablinky_ghost: hm for the authentication error I would need to see you keystone log12:24
blinky_ghostlsmola: no, I tried to reboot it normally12:25
lsmolablinky_ghost: I don't thing selinux is the problem here, otherwise it would not even show you the login screen12:25
lsmolablinky_ghost: please reboot it only via ironic or nova, it should boot it correctly then12:25
lsmolablinky_ghost: didn't know about the allowed host, but it make sense12:26
blinky_ghostlsmola: I've activated debug in keystone but nothing happens12:26
lsmolablinky_ghost: you didn't see anything there?12:26
lsmolablinky_ghost: what about horizon log?12:26
blinky_ghostlsmola: /var/log/httpd/error_log ?12:27
*** dshulyak_ has quit IRC12:27
blinky_ghostlsmola: Login failed for user "admin".12:27
lsmolalsmola: I think there is horizon_error log?12:27
lsmolalsmola: hm, might be good to run debug mode with horizon too12:28
lsmolablinky_ghost:  but I think should be in local settings, either in /etc/ or under the package12:29
lsmolablinky_ghost: damn why I am writing to myself :-D12:29
lsmolablinky_ghost: if you search for local_settings.py under /opt12:29
lsmolablinky_ghost: I think there was a bug and it didn't take the one under /etc last time I checked12:30
lsmolablinky_ghost: you are using the password you see in the UI ?12:30
blinky_ghostlsmola: yes I am12:30
lsmolablinky_ghost: also could you try to ssh into the controller node12:30
lsmolablinky_ghost: do, sudo -i12:30
lsmolablinky_ghost: source stackrc12:31
lsmolablinky_ghost: and try some CLI commands12:31
lsmolablinky_ghost: optionally check if the admin password in stackrc is the same you are using12:31
lsmolablinky_ghost: might be that you have a bad password given the error log I see12:31
blinky_ghostlsmola: yes the password is the same12:32
lsmolablinky_ghost: and the CLI is working?12:32
blinky_ghostlsmola: yes12:32
lsmolablinky_ghost: hmm12:32
blinky_ghostlsmola: If I run openstack-status12:32
blinky_ghostlsmola: I can see that everything is work12:33
blinky_ghostworking12:33
lsmolablinky_ghost: og try e.g. nova list, or glance image-list, or nova hypervisor-list12:33
blinky_ghostlsmola: [root@ov-tox5cniho3l-0-a4sicw6qkntj-controller-5iiawyyoe5za ~]# nova list12:33
blinky_ghost+----+------+--------+------------+-------------+----------+12:33
blinky_ghost| ID | Name | Status | Task State | Power State | Networks |12:33
blinky_ghost+----+------+--------+------------+-------------+----------+12:33
blinky_ghost+----+------+--------+------------+-------------+----------+12:33
blinky_ghostlsmola: [root@ov-tox5cniho3l-0-a4sicw6qkntj-controller-5iiawyyoe5za ~]# nova hypervisor-list12:34
blinky_ghost+----+-----------------------------------------------------------------+12:34
blinky_ghost| ID | Hypervisor hostname                                             |12:34
blinky_ghost+----+-----------------------------------------------------------------+12:34
blinky_ghost| 1  | ov-hjokdompzj-0-vld4jkylg56c-novacompute-ecndpatd7xxv.novalocal |12:34
blinky_ghost+----+-----------------------------------------------------------------+12:34
lsmolablinky_ghost: ok that looks good, please paste it to external file :-)12:34
blinky_ghostlsmola: sorry12:34
lsmolablinky_ghost: seems like only the UI is miss configured12:34
lsmolablinky_ghost: ok, could you check what OPENSTACK_HOST you have in local_settings?12:35
blinky_ghostlsmola: I still believe is something with selinux I think I saw this error before12:35
lsmolablinky_ghost: and if you can curl the address?12:35
lsmolablinky_ghost: hm, it could be I guess12:35
blinky_ghostlsmola: OPENSTACK_HOST = "127.0.0.1"12:36
lsmolablinky_ghost: I would need to see from the log if it really talks to keystone12:36
*** pradeep has joined #tripleo12:36
lsmolablinky_ghost: I think it's fine if you have keystone on localhost:500012:37
lsmolablinky_ghost: yo uare using fedora 20? or centos?12:37
blinky_ghostlsmola: nopes12:37
*** pradeep1 has quit IRC12:37
lsmolablinky_ghost: so what host do you have in stackrc?12:37
blinky_ghostlsmola: keystone is running on public_bind_host = 192.0.2.6612:37
blinky_ghostlsmola: I have http://192.0.2.64:5000/v2.012:38
lsmolablinky_ghost: ok fill the same to local settings  OPENSTACK_HOST12:38
*** bvandenh has joined #tripleo12:38
blinky_ghostlsmola: it's working :D12:41
lsmolablinky_ghost: cool12:41
blinky_ghostlsmola: thanks :) how do I change this? on heat-template? on the image?12:42
lsmolablinky_ghost: this is in horizon image-element12:42
lsmolablinky_ghost: hm damn https://github.com/openstack/tripleo-image-elements/blob/master/elements/horizon/os-apply-config/etc/horizon/local_settings.py#L6212:44
lsmolablinky_ghost: so it's the bug that it actually does not take the local_settings from /etc12:44
lsmolablinky_ghost: cause the element has it correct12:44
blinky_ghostlsmola: ok the bug is already open, right?12:45
lsmolablinky_ghost: I don't think so :-)12:45
blinky_ghostlsmola: OK I'll file a bug :)12:45
*** rushiagr is now known as rushiagr_away12:46
lsmolablinky_ghost: I just noticed it while ago and wanted to investigate further, haven't have time yet :-)12:46
lsmolablinky_ghost: ok, thank you12:46
lsmolablinky_ghost: yo uare using source install, right?12:46
lsmolablinky_ghost: I think package install has this working12:46
blinky_ghostlsmola: yes12:46
lsmolablinky_ghost: so the bug is most probably here https://github.com/openstack/tripleo-image-elements/blob/master/elements/horizon/install.d/horizon-source-install/100-horizon12:47
*** jcoufal_ has quit IRC12:47
*** jcoufal has joined #tripleo12:48
blinky_ghostlsmola: another question, I know that I'm a pain in the a** :) Is it possible to deploy the overcloud without br-ex? I don't want to use that. I only want to use provider networks.12:48
lsmolablinky_ghost: I am glad to help, I am actually glad you are helping me to test this :-)12:49
lsmolablinky_ghost: we will have less work when QA will take this :-)12:49
lsmolablinky_ghost: hm, that question goes probably beyond me :-) really my networking skills sucks12:50
blinky_ghostlsmola: well I can say it's working now. If you want to hire me just for doing this I don't mind :)12:50
blinky_ghostlsmola: OK, I'll ask dprince when he's around :)12:50
lsmolablinky_ghost: well I think we are hiring :-)12:51
lsmolablinky_ghost: ok, I think he is the right person to ask12:51
blinky_ghostlsmola: OK so now I'll try to reboot the controller through nova to see if it comes up :)12:52
lsmolablinky_ghost: ok :-)12:52
* lsmola will be back in 30mins12:52
*** morazi has joined #tripleo13:01
*** pradeep1 has joined #tripleo13:02
*** pradeep has quit IRC13:04
*** noslzzp has joined #tripleo13:04
*** zz_avozza is now known as avozza13:06
*** tzumainn has joined #tripleo13:07
*** pradeep has joined #tripleo13:10
*** avozza is now known as zz_avozza13:12
*** pradeep1 has quit IRC13:13
*** zz_avozza is now known as avozza13:13
*** pradeep1 has joined #tripleo13:14
*** avozza is now known as zz_avozza13:14
*** pradeep has quit IRC13:14
tchaypoblinky_ghost: I’m perversely glad I’m not the only person who can’t answer your question13:15
tchaypoit sucks for you, but it makes me feel less dumb13:15
*** zz_avozza has quit IRC13:16
*** zz_avozza has joined #tripleo13:18
*** zz_avozza is now known as avozza13:18
*** julim has quit IRC13:19
*** nosnos has quit IRC13:20
*** avozza is now known as zz_avozza13:21
*** zz_avozza is now known as avozza13:22
openstackgerritMerged openstack/python-tuskarclient: Updated from global requirements  https://review.openstack.org/13599013:27
*** weshay has joined #tripleo13:31
*** bvandenh has quit IRC13:32
*** rushiagr_away is now known as rushiagr13:42
*** bvandenh has joined #tripleo13:47
*** julim has joined #tripleo13:49
*** yamahata has quit IRC13:50
*** yamahata has joined #tripleo13:51
openstackgerritMerged openstack/tuskar: Remove outdated note from the install docs  https://review.openstack.org/13643013:59
*** jehb_ has joined #tripleo14:07
*** avozza is now known as zz_avozza14:13
*** zz_avozza is now known as avozza14:14
*** tchaypo has quit IRC14:15
openstackgerritLadislav Smola proposed openstack/tripleo-incubator: Switch to os-cloud-config register-endpoints  https://review.openstack.org/11384114:16
*** tchaypo has joined #tripleo14:17
openstackgerritMerged stackforge/kolla: Fix to not fork mongod so mongodb container works as expected  https://review.openstack.org/13723014:24
*** pradeep has joined #tripleo14:27
blinky_ghostthcaype: ok :) thanks anyway14:29
*** pradeep1 has quit IRC14:30
*** ramishra has quit IRC14:33
*** pradeep has quit IRC14:34
*** pradeep has joined #tripleo14:34
*** Hazelesque has quit IRC14:40
*** Hazelesque has joined #tripleo14:40
*** pradeep1 has joined #tripleo14:41
*** mikeit has joined #tripleo14:42
*** pradeep has quit IRC14:42
*** Hazelesque has quit IRC14:44
*** pradeep has joined #tripleo14:45
*** pradeep1 has quit IRC14:46
*** mikeit has quit IRC14:47
*** Hazelesque has joined #tripleo14:47
*** mikeit has joined #tripleo14:47
*** pradeep1 has joined #tripleo14:49
*** pradeep has quit IRC14:52
*** pradeep has joined #tripleo15:02
*** pradeep1 has quit IRC15:03
sdakemorning15:03
openstackgerritJerry Zhao proposed openstack/tripleo-incubator: use lowercase name to check for user/tenant existence  https://review.openstack.org/13647915:03
blinky_ghostlsmola: https://bugs.launchpad.net/tripleo/+bug/139663815:05
uvirtbotLaunchpad bug 1396638 in tripleo "Controller overcloud horizon doesn't allow login after deployment" [Undecided,New]15:05
*** wuhg has quit IRC15:05
*** wuhg has joined #tripleo15:05
*** pradeep1 has joined #tripleo15:07
*** pradeep has quit IRC15:09
*** jcoufal has quit IRC15:10
*** avozza is now known as zz_avozza15:11
*** zz_avozza is now known as avozza15:15
*** lazy_prince is now known as killer_prince15:21
*** avozza is now known as zz_avozza15:27
*** edmund has joined #tripleo15:28
openstackgerritRyan Brady proposed openstack/diskimage-builder: Add Activation Key Support For Customer Portal  https://review.openstack.org/13737815:30
*** zz_avozza is now known as avozza15:31
*** marun has joined #tripleo15:31
*** radez is now known as radez_g0n315:32
*** WebSeb has joined #tripleo15:32
WebSebHello15:34
*** rwsu has joined #tripleo15:34
*** WebSeb has quit IRC15:34
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Add element for hardware discovery ramdisk for ironic-discoverd  https://review.openstack.org/12215115:36
openstackgerritNicholas Randon proposed openstack/tripleo-image-elements: Split RabbitMQ actions out of startup script  https://review.openstack.org/13424915:36
*** pradeep has joined #tripleo15:37
*** pradeep has quit IRC15:37
*** pradeep1 has quit IRC15:39
*** avozza is now known as zz_avozza15:43
*** zz_avozza is now known as avozza15:43
*** jcoufal has joined #tripleo15:46
*** akrivoka has quit IRC15:46
*** avozza is now known as zz_avozza15:54
*** vinsh has joined #tripleo15:54
*** pcrews has joined #tripleo15:56
*** untriaged-bot has joined #tripleo16:00
untriaged-botUntriaged bugs so far:16:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/138706116:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/139663816:00
uvirtbotLaunchpad bug 1387061 in tripleo "deploy under cloud failure due to pacemaker " [Undecided,New]16:00
uvirtbotLaunchpad bug 1396638 in tripleo "Controller overcloud horizon doesn't allow login after deployment" [Undecided,New]16:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/139165616:00
uvirtbotLaunchpad bug 1391656 in diskimage-builder "There is no way to customize the installation of the bootloader" [Undecided,New]16:00
untriaged-bothttps://bugs.launchpad.net/tuskar/+bug/138748716:00
uvirtbotLaunchpad bug 1387487 in tuskar "tripleo-image-elements/elements/tuskar-ui /os-refresh-config /post-configure.d/101-tuskar-ui  needs authentication from Keystone" [Undecided,New]16:00
*** untriaged-bot has quit IRC16:00
*** pblaho_ has joined #tripleo16:01
*** mikeit has quit IRC16:04
*** mrunge has quit IRC16:04
*** pblaho has quit IRC16:04
openstackgerritRyan Brady proposed openstack/tripleo-image-elements: Migrate horizon element to svc-map  https://review.openstack.org/13408116:06
tteggelanyone able to look at https://review.openstack.org/#/c/131795/ ?16:07
*** jerryz has quit IRC16:08
*** pblaho_ is now known as pblaho16:16
*** zigo has quit IRC16:16
*** akrivoka has joined #tripleo16:17
*** zigo has joined #tripleo16:27
*** k4n0 has quit IRC16:30
*** pblaho_ has joined #tripleo16:31
*** rdopiera has quit IRC16:32
*** jcoufal has quit IRC16:33
*** pblaho has quit IRC16:35
*** Marga_ has joined #tripleo16:46
*** akrivoka has quit IRC16:47
*** pblaho_ has quit IRC16:49
*** daneyon has quit IRC16:51
*** eghobo has joined #tripleo16:54
*** Marga_ has quit IRC16:57
*** jsidhu has joined #tripleo16:58
*** Marga_ has joined #tripleo16:58
*** Marga_ has quit IRC16:59
*** Marga_ has joined #tripleo16:59
*** Marga_ has quit IRC16:59
*** Marga_ has joined #tripleo17:00
*** eghobo has quit IRC17:04
*** ChuckC_ has quit IRC17:05
*** ChuckC has joined #tripleo17:06
*** eghobo has joined #tripleo17:08
*** eghobo has quit IRC17:09
*** Marga_ has quit IRC17:24
*** Marga_ has joined #tripleo17:26
openstackgerritMerged openstack/tripleo-image-elements: Increased protection to prevent vip  https://review.openstack.org/13549417:28
*** Marga_ has quit IRC17:30
*** Marga_ has joined #tripleo17:30
*** jang1 has joined #tripleo17:32
*** daneyon has joined #tripleo17:36
*** bvandenh has quit IRC17:46
*** Marga_ has quit IRC17:52
*** Marga_ has joined #tripleo17:53
*** Marga_ has quit IRC17:54
*** Marga_ has joined #tripleo17:55
*** Marga_ has quit IRC17:55
*** jistr has quit IRC17:56
*** Marga_ has joined #tripleo17:56
*** derekh has quit IRC17:58
*** Marga_ has quit IRC18:06
*** Marga_ has joined #tripleo18:06
*** Marga_ has quit IRC18:07
*** Marga_ has joined #tripleo18:07
*** dshulyak_ has joined #tripleo18:10
*** Marga_ has quit IRC18:12
*** Marga_ has joined #tripleo18:12
*** dtantsur is now known as dtantsur|afk18:13
*** Marga_ has quit IRC18:13
*** Marga_ has joined #tripleo18:14
*** Marga_ has quit IRC18:18
*** Marga_ has joined #tripleo18:18
*** Marga_ has quit IRC18:19
*** Marga_ has joined #tripleo18:19
*** Marga_ has quit IRC18:21
*** Marga_ has joined #tripleo18:22
*** athomas has quit IRC18:24
*** Marga_ has quit IRC18:25
*** Marga_ has joined #tripleo18:25
*** ifarkas has quit IRC18:27
*** zz_avozza is now known as avozza18:28
*** lucasagomes has quit IRC18:28
openstackgerritNicholas Randon proposed openstack/tripleo-image-elements: Split RabbitMQ actions out of startup script  https://review.openstack.org/13424918:28
*** jp_at_hp has quit IRC18:29
*** Marga_ has quit IRC18:32
*** spredzy has left #tripleo18:32
*** Marga_ has joined #tripleo18:32
*** jsidhu has quit IRC18:33
*** wuhg has quit IRC18:33
*** pradk has joined #tripleo18:35
*** Marga_ has quit IRC18:35
*** Marga_ has joined #tripleo18:36
*** avozza is now known as zz_avozza18:36
*** Marga_ has quit IRC18:39
*** Marga_ has joined #tripleo18:39
*** rushiagr is now known as rushiagr_away18:45
*** jsidhu has joined #tripleo18:47
*** rwsu has quit IRC18:50
*** zz_avozza is now known as avozza18:57
*** yamahata has quit IRC18:59
*** avozza is now known as zz_avozza19:07
*** zz_avozza is now known as avozza19:08
*** penick has joined #tripleo19:09
blinky_ghostlsmola: are you there?19:11
*** jprovazn has quit IRC19:20
*** blinky_ghost has quit IRC19:26
*** blinky_ghost has joined #tripleo19:32
rbradyblinky_ghost: it's likely lsmola is afk for the evening19:36
*** blinky_ghost has quit IRC19:37
*** dshulyak_ has quit IRC19:45
*** dshulyak_ has joined #tripleo19:45
*** dshulyak_ has quit IRC19:49
*** jang1 has quit IRC19:59
*** radez_g0n3 is now known as radez20:01
openstackgerritMerged openstack/tripleo-image-elements: Fix syntax error in deploy-ci-overcloud  https://review.openstack.org/13571620:03
openstackgerritMerged openstack/os-cloud-config: Deal with uppercase MAC address in register-nodes  https://review.openstack.org/13580920:10
*** penick_ has joined #tripleo20:11
*** dshulyak_ has joined #tripleo20:12
*** penick has quit IRC20:13
*** penick_ is now known as penick20:13
openstackgerritChris Jones proposed openstack/diskimage-builder: Continue past dependency ordering diffs.  https://review.openstack.org/13746520:24
*** dshulyak_ has quit IRC20:25
*** jsidhu has quit IRC20:27
*** avozza is now known as zz_avozza20:36
*** penick has quit IRC20:37
*** dshulyak_ has joined #tripleo20:40
openstackgerritgreghaynes proposed openstack/tripleo-image-elements: Longer upstart sleep for neutron-ovs-agent  https://review.openstack.org/13719520:51
openstackgerritMerged stackforge/kolla: Horizon container implementation  https://review.openstack.org/13577320:55
openstackgerritgreghaynes proposed openstack/diskimage-builder: Add new package-installs system  https://review.openstack.org/13491720:55
*** weshay has quit IRC21:12
*** radez is now known as radez_g0n321:17
*** pradk has quit IRC21:23
*** amitpp has joined #tripleo21:30
*** weshay has joined #tripleo21:31
*** gfidente has quit IRC21:36
*** dshulyak_ has quit IRC21:38
openstackgerritgreghaynes proposed openstack/diskimage-builder: Migrate to new package-installs  https://review.openstack.org/13747821:38
*** jehb_ has quit IRC21:42
openstackgerritgreghaynes proposed openstack/diskimage-builder: Migrate to new package-installs  https://review.openstack.org/13747821:49
openstackgerritgreghaynes proposed openstack/diskimage-builder: Migrate to new package-installs  https://review.openstack.org/13747821:50
*** zz_avozza is now known as avozza21:53
*** ifarkas_ has joined #tripleo21:56
*** amitpp has quit IRC21:58
*** ifarkas_ has quit IRC21:59
*** jsidhu has joined #tripleo22:00
*** ccrouch has quit IRC22:21
*** morazi has quit IRC22:24
*** rlandy has quit IRC22:25
*** daneyon has quit IRC22:35
*** weshay has quit IRC22:42
*** EmilienM has quit IRC22:47
*** EmilienM has joined #tripleo22:47
openstackgerritBen Nemec proposed openstack/tripleo-image-elements: Add heat sudo enablement from dib  https://review.openstack.org/13664323:11
NobodyCamgood afternoon TripleO I have a question about the best way to have an element modify a setting in another element. As an example I am working on adding support Ironic-python-agent in to devtest. IPA requires swift-temp-url support, so I am looking at a way to change the backend setting in seed-stack-config element23:16
NobodyCamor have a devtest it self detect the element and affect the change from the devtest_* script it self23:18
* greghaynes doesnt totally understand23:18
NobodyCamgreghaynes: seed-stack-config holds the config.json for the seedvm23:18
NobodyCamwhich has the glance backend set to file.23:19
greghaynesso if its a feature you would like to turn on that has effects the elements included and some settings then it probably will need to go in similar to how we did with ironic where we had the USE_IRONIC=123:19
NobodyCamI would like to clean;y switch to swift23:19
greghaynesah23:19
greghaynesso, does that need to be configurable / can we make swift be a requirement there when using ironic?23:20
NobodyCamI can add settings that will not be used by other elements but not sure how to change a current value23:20
NobodyCamthe patch I am working will add a $IRONIC_DEPLOY_PROVIDER var23:21
NobodyCamso the ironic driver can be switched23:21
NobodyCamI'm not sure what the cost to the seed would be to make swift a requirment if its only used for one ironic driver23:22
NobodyCamwhich i am thinking will not be the default one23:23
greghaynesyea, so you generally have to do two things to add something like that. for the undercloud (and overcloud if it applies but I dont think it does here) you parameterize that property in the heat templates and give it a reasonable default, then you also have to add a default to the seed local.json youre talking about and add the jq in devtest_seed.sh so a user can set something else via env var or somesuch23:24
greghaynesif its something where the default is pretty reasonable for most users you can get away with not doing the env var + jq stuff though since a user can just modify the seed config.json manually23:25
greghaynesthis does kind of seem like a thing where you can add the property to config.json and a parameter to the undercloud heat template and be done23:26
greghaynessince I dont think most users will want to mess with that value23:26
NobodyCamyea ... I have an Idea, let me ponder on it for a bit :)23:28
NobodyCamTy greghaynes23:28
greghaynesnp23:28
openstackgerritMerged openstack/tuskar: Updated from global requirements  https://review.openstack.org/13438523:28
*** avozza is now known as zz_avozza23:46
*** marun has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!