Tuesday, 2017-04-11

*** apearson has joined #openstack-powervm00:40
*** mdrabe has quit IRC00:57
thorstadreznec: need to figure out https://review.openstack.org/#/c/452220/01:01
thorstis the bot running now or do we just need to update the manual requirements again?01:01
adreznecthorst: Ah, I see01:02
adreznecI bet it's the pypowervm version01:02
adreznecWe moved it to 1.1.1 in requirements but that patchset is out of date01:02
adreznec1 sec01:02
thorstadreznec: you weren't supposed to respond tonight...01:02
adrezneclol01:02
openstackgerritAdam Reznechek proposed openstack/networking-powervm master: Manual sync with upstream requirements  https://review.openstack.org/45222001:06
openstackgerritAdam Reznechek proposed openstack/ceilometer-powervm master: Manual sync with upstream requirements  https://review.openstack.org/45222201:07
thorstadreznec: these ceilometer-powervm ones will be tricky01:16
thorstgiven that we didn't merge them when gordc proposed them, I think we're going to have to wrap many of them into a single change set01:16
adreznecYeah... kind of a pain. We could probably rework them into a patch series if we wanted01:19
adreznecBut probably not worth the effort01:19
thorstwell, I'm just picking up gordc's first01:20
thorstand then working from there01:20
thorstI should connect up with him and thank him01:20
adreznecYeah... downside to not having a more thorough CI on ceilometer-powervm01:21
thorstwe'll get there someday01:21
thorstok...I think I have them sorted.01:24
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm master: use new instance polling interface  https://review.openstack.org/44813401:28
*** thorst has quit IRC01:30
*** tjakobs__ has joined #openstack-powervm01:30
*** thorst has joined #openstack-powervm01:30
*** thorst has quit IRC01:34
*** edmondsw has joined #openstack-powervm01:39
*** tjakobs__ has quit IRC01:39
*** edmondsw has quit IRC01:43
*** esberglu has joined #openstack-powervm01:58
*** esberglu has quit IRC02:03
*** apearson has quit IRC02:13
*** thorst has joined #openstack-powervm02:31
*** thorst has quit IRC02:51
*** esberglu has joined #openstack-powervm03:47
*** esberglu has quit IRC03:52
*** shyama has joined #openstack-powervm04:24
*** tjakobs__ has joined #openstack-powervm04:40
*** thorst has joined #openstack-powervm04:48
*** thorst has quit IRC04:52
*** tjakobs__ has quit IRC04:54
*** jay1_ has joined #openstack-powervm05:32
*** esberglu has joined #openstack-powervm05:37
*** esberglu has quit IRC05:41
*** thorst has joined #openstack-powervm05:48
*** jay2__ has joined #openstack-powervm05:50
*** jay2__ has quit IRC05:50
*** thorst has quit IRC05:53
*** k0da has joined #openstack-powervm06:11
*** thorst has joined #openstack-powervm06:49
*** thorst has quit IRC06:54
*** thorst has joined #openstack-powervm07:50
*** thorst has quit IRC08:09
*** openstackgerrit has quit IRC09:03
*** thorst has joined #openstack-powervm09:06
*** thorst has quit IRC09:11
*** esberglu has joined #openstack-powervm09:14
*** esberglu has quit IRC09:19
*** k0da has quit IRC10:43
*** smatzek has joined #openstack-powervm11:07
*** thorst has joined #openstack-powervm11:08
*** thorst has quit IRC11:13
*** shyama has quit IRC11:26
*** thorst has joined #openstack-powervm11:47
*** edmondsw has joined #openstack-powervm12:12
*** shyama has joined #openstack-powervm12:13
*** jpasqualetto has joined #openstack-powervm12:24
*** nbante has joined #openstack-powervm12:58
*** apearson has joined #openstack-powervm13:00
*** esberglu has joined #openstack-powervm13:00
efriedless dudiss13:01
esberglu#startmeeting powervm_driver_meeting13:01
openstackMeeting started Tue Apr 11 13:01:14 2017 UTC and is due to finish in 60 minutes.  The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot.13:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:01
openstackThe meeting name has been set to 'powervm_driver_meeting'13:01
efried\o13:01
thorsto/13:02
esberglu#topic In Tree Driver13:02
thorstefried: review this in background13:02
thorsthttps://review.openstack.org/#/c/448134/13:02
efriedack13:02
esbergluefried: Need anything from us in-tree?13:03
esbergluOther than reviewing once the rebases all go through?13:03
efriedesberglu I don't think so.  Assume CI will be a separate topic.13:03
esbergluyeah13:03
*** jay1_ has quit IRC13:04
esberglu#topic OOT Driver13:04
esbergluthorst: Are there any reviews that we are still trying to push through CI OOT?13:05
esbergluI think at least most of them have a CI +1 now13:05
thorstesberglu: tons13:05
thorstwe need to do a full on scrub of things13:05
esbergluYeah I think that's the best bet13:06
esbergluGonna be a pain to redeploy until that packaging stuff while stacking is fixed though13:06
thorstwell, I think that we have a few things that we should chat through and then once we're relatively solid...we can then go ahead and redeploy OOT13:07
thorstOOT CI...13:07
thorstnet, there are about 6 or so change sets we need to scrub through.13:07
*** mdrabe has joined #openstack-powervm13:07
esbergluLinks? I will keep an eye and try to recheck as soon as any fail13:09
thorstwell, some will take effort.  But here's a first pass (and they're not just blocked by CI, but also reviews)13:09
thorsthttps://review.openstack.org/44813413:10
*** newbie|0 has joined #openstack-powervm13:10
thorsthttps://review.openstack.org/44824913:10
thorsthttps://review.openstack.org/44776213:10
thorsthttps://review.openstack.org/45302413:10
thorsthttps://review.openstack.org/45271213:10
thorsthttps://review.openstack.org/44776313:10
thorsthttps://review.openstack.org/41666713:11
*** mdrabe has quit IRC13:11
thorstI think that's it13:12
thorstfrom my end13:12
esbergluOkay. I'm kicking off rechecks on the ones that haven't had a run yet13:12
*** nbante has quit IRC13:12
*** newbie|0 has quit IRC13:13
esbergluOkay is that it for OOT (CI separate topic)13:14
*** nbante has joined #openstack-powervm13:14
thorstlonger term...13:14
adreznecThe other two that will need reviews are https://review.openstack.org/#/c/452220/ and https://review.openstack.org/#/c/452222/13:14
thorstI would like to see us start to look at PCI direct attach13:14
adreznecthe latter is gated behind those ceilometer changes thorst posted13:14
thorstin case anyone has spare cycles.13:14
adreznecBut the former should be good, though it appears to have failed tempest with different tests than before the recheck...13:15
esbergluAre you saying it failed different tests? Or ran different tests?13:15
adreznecFailed13:15
esbergluWhew okay good13:16
adreznecHaha yeah, the latter would definitely be more confusing to deal with13:16
adreznecesberglu: Do you know if the CI was down/didn't run on a patch when it first came in and I do a powervm: recheck comment on it, will CI run on that patch?13:17
esbergluYeah it will13:17
adreznecTalking https://review.openstack.org/#/c/454506/ in particular13:17
adreznecOk13:17
esbergluthorst: You have anything else you wanted to say about PCI direct attach, or just for awareness at this point?13:19
thorstjust awareness...start thinking about it13:20
efriedYeah, that was a bit of a bomb there, thorst13:20
efriedWe talking queens?13:20
thorstyeah13:20
thorstlong term game13:20
efriedight13:20
thorstI suspect the code for that right now is very kvm centric13:21
efriedOh hell yeah.13:21
thorstso I'd like our team to build an understanding of what's there...does it fit...do we have changes needed in the core driver...etc...13:21
efried#agreed PCI direct attach is a big fish, long term.13:21
thorstcore driver --> manager13:21
efried(been looking for an opportunity to use #agreed)13:22
thorstheh13:22
efried(oh, it only works for the chair.  Rats)13:22
thorstanywho, that's all I had13:22
esbergluOkay lets move on13:22
esberglu#topic CI13:22
esbergluSo we already talked about pushing the backlog through13:22
esbergluAnd then at that point I think the plans is to wipe the mgmt node and redeploy from scratch13:23
esbergluCuz this whole saving a sinking ship with a bucket thing isn't working13:23
esbergluOther than that we have efrieds local2remote changes13:25
esbergluI know you want to have that installed as part of the base image template13:25
efriedWell, that's open for discussion, but I think that's the right place to do it.13:25
esbergluBut we would still have to reinstall during the prep_devstack part13:25
thorstand we have a long term plan to go 'patchless'?13:25
efriedWe shouldn't have to reinstall it at all then.13:25
esbergluWe don't know which version of pypowervm we want until then13:26
efriedYou mean because newton and ocata are still on the table at that point?13:26
efriedDoesn't matter.13:26
efriedIf we diddle /etc/environment in the base image, and have the .py present, pre-1.1.2 will simply ignore, and we'll still have to do the old style local2remote patching if newton/ocata.13:27
efriedBut 1.1.2 will just work.13:27
efriedTo summarize: the new stuff I'm proposing can be part of the base image template; and the old stuff will need to change to be conditioned on pre-pike-only once 1.1.2 is released.13:28
esbergluWell we can't pull the version from requirements then13:28
efriedWhen requirements says 1.1.2, we can.13:28
efriedCause we should be pulling from the requirements of the branch we're testing.13:29
esbergluWhat if requirements changes between the nightly build and the CI run?13:29
efriedYeah, we still pull from requirements of whatever branch we're testing.13:29
esbergluWe don't know which branch we are testing until way after the image build13:29
esbergluMaybe I'm missing something here13:30
efriedDoesn't matter if we flipflop back and forth between 1.1.2 and pre-1.1.2.13:30
esbergluYeah but we still have to reinstall at that point is what I'm saying13:30
efriedOnly if we're testing pre-pike.13:30
efriedWhole point of this is we don't have to patch pypowervm at 1.1.2.13:30
efried...which should be concomitant with pike.  And pre-1.1.2 with pre-pike.13:31
esbergluMy point about the reqs changing between the image build and the run still stands13:31
esbergluEdge case, but still13:32
efriedreqs of pypowervm specifically?13:32
efriedI'm still not seeing the problem.13:32
esbergluYeah, what if we bump the reqs up to 1.1.3? We would then have to wipe all of the nodes out13:32
esbergluAnd build from scratch13:32
efriedno, we shouldn't.13:32
efriedCause 1.1.3 will then get pulled in when we stack.13:33
efriedAnd 1.1.3 will still just work when the env var & .py are present.13:33
esbergluOhhh duhhhh13:33
*** smatzek has quit IRC13:33
efriedWhole reason we're doing things this way.13:33
esbergluYeah my b13:33
efriedCool13:33
*** tjakobs__ has joined #openstack-powervm13:34
esbergluI still owe you a patch / review on that stuff13:34
efriedYeah, but this is background work, as there are lulls in the flushing-CI-backlog business.13:35
efriedAs excited as I am about trying it out and getting it working...13:35
esbergluYep. I would like to get it in for when we do the full CI redeploy13:35
esbergluSo we don't have to redeploy again right after13:36
efriedFor the sake of having it documented, we will ultimately need: o) patches to deploy the .py and /etc/environment change; o) image template rebuild; o) 1.1.2 released; o) g-r and nova project requirements bumps13:36
efriedI think in that order.13:37
efriedthe image template rebuild & CI redeploy could happen at any point after the neo-os-ci change is merged.13:38
esbergluYeah I think so13:39
efriedoh, and o) separate neo-os-ci (or powervm-ci?) change to condition manual local2remote patch on pre-pike.  I think that needs to come in sync with the reqs bumps.13:39
efriedwhich will be fiddly.13:39
efriedCause the manual local2remote patch will simply fail on 1.1.2.13:40
efriedmerge conflict there.13:40
efriedI guess we could change that right now to condition on the pypowervm version.13:40
efriedrather than the openstack branch.13:40
efriedThen no fiddly.13:40
esbergluSure I like that13:41
efriedWill probably have to use pbr to do version compare, groan.13:41
efriedOkay, let's move on.13:41
esbergluI didn't have anything else base CI13:41
esbergluthorst: Did you want to talk OSA CI here? Or separate discussion on that?13:42
thorstlets have a separate discussion on that when we stabilize13:42
esbergluOkay13:43
esbergluThat's it from my end then13:43
*** tjakobs__ has quit IRC13:43
esberglu#topic Other Discussion13:44
esberglunbante: You have anything you are working through?13:44
efried#action thorst As able, take a look at https://review.openstack.org/#/c/438729/12..13 (tasks/* bits - the others are rebase) and let's move toward getting this back to nova-powervm for https://bugs.launchpad.net/nova-powervm/+bug/168094713:44
openstackLaunchpad bug 1680947 in nova-powervm "Use PrintingDurationListener, get rid of PowerVMTask" [Wishlist,New]13:44
efriedLow priority.13:44
nbanteI tried basic tests with tempest  and those are passed successfully. Now working to configure external storage then we can try deploy test cases13:45
thorstit looks like I need to figure out how to get our v7k on a private IP address for nbante13:46
thorstit'll take some wiring  :-/13:46
nbantemeanwhile I'll check,what are configuration files need to modify and once all changes done. I'll redploy to get cinder configured13:47
esberglunbante: Cool sounds good13:48
esbergluDoesn't look like jay is on13:48
esbergluAny final topics before I close?13:48
esberglu#endmeeting13:50
thorstefried: do you know if we've fixed the arping thing?13:50
openstackMeeting ended Tue Apr 11 13:50:50 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-04-11-13.01.html13:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-04-11-13.01.txt13:50
openstackLog:            http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-04-11-13.01.log.html13:50
efriedthorst Sylvain has proposed the fix.  Hopefully it goes through today.13:51
thorstlets raise hell to make sure it does13:51
efriedthorst 511713:51
thorstrabble rabble rabble rabble13:51
efriedLooks like it got +2 from nvcastet.13:51
efriedLet's merge that sucker.13:51
efriedSeems to be having trouble in jenkins.13:52
efriedSo we may have to rabble rabble jfoliva13:52
efriedI kicked off a recheck ~45m ago.13:52
thorstk13:52
*** jpasqualetto has quit IRC13:57
*** smatzek has joined #openstack-powervm14:02
*** jpasqualetto has joined #openstack-powervm14:09
*** k0da has joined #openstack-powervm14:16
-openstackstatus- NOTICE: latest base images have mistakenly put python3 in some places expecting python2 causing widespread failure of docs patches - fixes are underway14:27
*** mdrabe has joined #openstack-powervm14:44
-openstackstatus- NOTICE: we have rolled back centos-7, fedora-25 and ubuntu-xenial images to the previous days release. Feel free to recheck your jobs now.14:47
esbergluefried: One thing we will be giving up with the local2remote change is the ability to add additional pypowervm patches without rebuilding the image template14:48
esbergluNot sure if you were aware of that14:48
esbergluActually I have an idea to get around that14:53
esbergluStay tuned14:54
*** nbante has quit IRC14:56
*** tjakobs__ has joined #openstack-powervm15:03
*** openstackgerrit has joined #openstack-powervm15:16
openstackgerritEric Berglund proposed openstack/networking-powervm master: Switch to manual service enablement for devstack plugins  https://review.openstack.org/41666715:16
*** tjakobs__ has quit IRC15:21
*** tjakobs has joined #openstack-powervm15:21
efriedesberglu Not sure how we lose that ability.  We just need to keep the pre-pike local2remote patch separate.15:32
esbergluBecause we aren't reinstalling pypowervm for master. So if we want to test a pypowervm patch quick we can't just add it to the jenkins job parameter and reload the jenkins job15:35
esbergluI have a solution I think hold tight15:35
efriedight15:39
esbergluI'll walk you through my solution quick15:42
esbergluLeave the -p flag in the prep_devstack script (pypowervm_patch_list) so we can still pass in additional patches from the jenkins job15:44
esbergluIn prep_devstack15:44
esbergluIf on newton or ocata15:44
esberglu     pypowervm_patch_list+=238215:44
esberglufi15:44
esbergluif pypowervm_patch_list isn't empty15:44
esberglu    checkout the correct pypowervm from requirements15:44
esberglu    apply all patches from the patch list15:44
esberglu    install pypowervm15:45
esberglu    wipe pypowervm from requirements so stack won't overwrite15:45
esberglufi15:45
esbergluif the pypowervm_patch_list is empty we just proceed as normal and don't worry about anything15:45
efriedesberglu That sounds okay.15:48
efriedLong as "from requirements" is branch-gnostic.15:48
esbergluYeah we checkout the correct branch for all projects before any of this happens15:49
esbergluWe just need to make sure that the env. var. for pypowervm either survives to this point or set it again15:50
efriedesberglu If it's in /etc/environment in the image template, it should survive just fine.15:55
efriedJust need to make sure the .py is present if the env var is set - else kablooey.15:56
efriedWhich is as it should be - I don't think we want to ignore a failed import there.15:56
efriedthorst Has this ceilometer-powervm stuff been tested live?15:57
esbergluthorst: Most of the reviews from that first list you posted have passed CI now excluding16:01
esbergluhttps://review.openstack.org/#/c/416667/  -- This needs a local.conf change in neo-os-ci16:01
esbergluhttps://review.openstack.org/#/c/452712/ -- Has been running for 3 hours, will check out logs when complete16:02
openstackgerritEric Fried proposed openstack/ceilometer-powervm master: Manual sync with upstream requirements  https://review.openstack.org/45222216:18
openstackgerritMerged openstack/ceilometer-powervm master: use new instance polling interface  https://review.openstack.org/44813416:22
thorstefried: gautam has been testing I believe16:31
openstackgerritMerged openstack/nova-powervm master: Remove INSTALL_PYPOWERVM  https://review.openstack.org/45434517:05
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm master: Remove INSTALL_PYPOWERVM  https://review.openstack.org/45434417:18
openstackgerritMerged openstack/networking-powervm master: Manual sync with upstream requirements  https://review.openstack.org/45222017:29
openstackgerritMerged openstack/ceilometer-powervm master: Manual sync with upstream requirements  https://review.openstack.org/45222217:29
openstackgerritMerged openstack/ceilometer-powervm master: Remove INSTALL_PYPOWERVM  https://review.openstack.org/45434417:31
efriedHowzat backlog looking?17:34
thorstI was just looking17:36
thorstits much better17:36
thorstbut I'm debugging one of adreznec's changes17:36
thorstit looks like it failed to start networking-powervm17:36
thorsto duh, because his change turns that off17:38
thorstadreznec: https://review.openstack.org/#/c/41666717:38
thorstthis won't pass CI unless we have a CI change?17:38
thorstesberglu: ^^17:38
esbergluWe would need to add the17:40
esbergluenable_service pvm-q-sea-agt pvm-q-sriov-agt17:40
esberglubit to the local.conf17:40
thorstyeah, I'm not sure I even buy the change though17:40
thorstif you're running networking-powervm...you'd want those services enabled I would think17:40
esbergluPerhaps the thinking was that you would only want 1 or the other enabled instead of both?17:41
thorstmeh17:43
thorstmaybe...I know we had a reason for it...but it's escaping my mind right now17:43
thorstso I -1'd it17:43
esbergluK. Just let me know if you change your mind and I will update CI accordingly17:43
*** jay1_ has joined #openstack-powervm17:44
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm master: Optimize the link address for fetching git code  https://review.openstack.org/44776217:45
thorstefried: you finally OK with that one above???17:46
thorst:-)17:46
openstackgerritDrew Thorstensen (thorst) proposed openstack/networking-powervm master: Optimize the link address for fetching git code  https://review.openstack.org/44776317:48
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm master: Remove support for py34 for ceilometer-powervm  https://review.openstack.org/43061917:50
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm master: Remove support for py34 for ceilometer-powervm  https://review.openstack.org/43061917:51
*** nbante has joined #openstack-powervm17:59
*** nbante has quit IRC18:19
*** shyama has quit IRC18:32
*** jay1_ has quit IRC18:54
efriedthorst I'm confused about those "optimize link address" ones.18:55
efriedBasically, why tf do we need to do that?18:56
thorsthttps is better than http18:56
thorstbut to me...its a little thing18:56
thorstI'm indifferent.18:57
efriedmeh, whatever.18:57
efriedesberglu thorst As for enabling the agents, we need the SEA agent for networking to work in the CI env.  Having the SR-IOV one enabled should be unnecessary, but harmless18:58
efriedAt least, I'm assuming we have SEA set up on our VIOSes in the CI env.  (I know we don't have SR-IOV.018:59
thorstyeah...but I'm wondering why we need to change it19:00
esbergluI think thorst was just confused about why this change needs to go in at all19:00
esbergluyeah19:00
efriedwhy, what is it now?19:01
thorstif you install networking-powervm, and set the virt driver to powervm, you get the SEA19:01
thorstif you set the virt driver to powervm, and don't install networking-powervm...you can use OVS or something19:01
thorstmaybe it was to bypass the case where you had installed networking-powervm, but wanted to flip a future devstack to OVS...19:01
openstackgerritMerged openstack/ceilometer-powervm master: Optimize the link address for fetching git code  https://review.openstack.org/44776219:02
efriedIf you install networking-powervm and set the virt driver to powervm, you get the SEA?19:02
efriedHow?19:02
thorstthat code looks at the nova compute virt driver attribute, and if set to powervm, it turns on SEA19:02
openstackgerritMerged openstack/networking-powervm master: Optimize the link address for fetching git code  https://review.openstack.org/44776319:04
efriedthorst "that code" what code?19:04
thorstugh...of course you ask for details19:05
efriedof course.19:05
thorsthttps://review.openstack.org/#/c/416667/2/devstack/settings19:05
thorstjerk wad19:05
efriedOkay, right, so with that part deleted, we need to enable those services explicitly via local.conf19:06
efriedright?19:06
efriedAm I missing something fundamental?19:06
thorstcorrect19:06
thorstand I'm just asserting..do we really need to make them jump through that hoop?19:07
efriedI would think so.  They should need to decide whether to enable one, both, or neither.19:07
efriedAnd ultimately if we have others, those too.19:07
efriedIf for no other reason than that they shouldn't have services running that they don't need.19:08
thorstok...well, we can't merge until CI has that setting done19:08
efriedI'm okay with that.19:08
esbergluI will put a neo-os-ci patch quick19:09
openstackgerritOpenStack Proposal Bot proposed openstack/nova-powervm master: Updated from global requirements  https://review.openstack.org/45450619:14
*** jpasqualetto has quit IRC19:16
*** jpasqualetto has joined #openstack-powervm19:29
*** jpasqualetto has quit IRC19:41
*** smatzek has quit IRC19:53
*** jpasqualetto has joined #openstack-powervm19:54
openstackgerritMerged openstack/nova-powervm master: Updated from global requirements  https://review.openstack.org/45450620:37
*** thorst has quit IRC21:07
*** apearson has quit IRC21:11
*** esberglu has quit IRC21:19
*** esberglu has joined #openstack-powervm21:20
*** esberglu has quit IRC21:24
*** thorst has joined #openstack-powervm21:30
*** edmondsw has quit IRC21:34
*** thorst has quit IRC21:34
*** edmondsw has joined #openstack-powervm21:35
*** esberglu has joined #openstack-powervm21:35
*** edmondsw has quit IRC21:39
*** esberglu has quit IRC21:53
*** dwayne has quit IRC22:00
*** edmondsw has joined #openstack-powervm22:04
*** mdrabe has quit IRC22:04
*** edmondsw has quit IRC22:08
*** jpasqualetto has quit IRC22:27
*** tjakobs has quit IRC22:43
*** jpasqualetto has joined #openstack-powervm22:44
*** k0da has quit IRC23:07
*** thorst has joined #openstack-powervm23:20
*** thorst has quit IRC23:23
*** thorst has joined #openstack-powervm23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!