Efforts towards fixing the lag

[Carefor]

Hello - First post over here on the LotRO boards; a shame it is in this topic.

I am experiencing crushing lag as well since U6. It has become utterly unplayable, with skill delays, hitching, ineraction delays, stalled travel, micro-teleporting at locations such as workbenches (as in the video in the thread showed), etc, etc. Short of repeating many things above or capturing a video to try to make you see what we are seeing, I just do not know what else I can say to convince you how frustrating this has become.

I can pretend to appreciate excuse-this or technical-that, but honestly, the wife just gave up and left the keyboard, and I logged as well in disgust. We coughed up for 3 months right before the update as we were enjoying the game, and liked what we were playing. This experience is NOT the same, and I am tired of waiting for weeks for a resolution, or being told it is my problem. I am guessing it has been somehow unclear, so I will try my best to be concise when I say:
LotRO before U6: Playable, fun, like it. Pay money.
LotRO post U6: Can be played, but not enjoyable. It pains me to say 'playable', but by definition it is, just abysmal and not something I would have ever, EVER, paid money toward.
[With no change on my part; If I could have know there would be problems, I could not even opt out of the update now could I?]

So while we wait, what happens to the money we paid forward? I'm not trying to be petty, but I feel really burned here, and more than a lot disappointed that we have lost our game time with no recourse, and worse yet, are heavily remorseful for paying in for this mess.
I hate to even attempt to play the 'bait and switch' card but, well, selling me on a product that worked, and handing me one that does not (or at least not in the same capacity) seems a bit over the line, does it not?

-- Carefor - Brandywine Server

[monteeburns]

I'm afraid your chances of getting your money back are slim to nil. You'll just have to chalk it up to experience. It may be fixed in a few days, it may be a few months, or longer. That's how things roll around here, have done for a while now. Sad but true.

[Lorr]

We do believe we're getting closer to the root cause, if only by process of elimination of what it isn't.

Your comment made me think of this old definition of how a missile guidance system works. Sorta off topic, but then again could be spot on to trying to pinpoint any problem. If nothing else, good for a quick chuckle...

The missile knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation. The guidance subsystem uses deviations to generate corrective commands to drive the missile from a position where it is to a position where it isn't, and arriving at a position where it wasn't, it now is. Consequently, the position where it is, is now the position that it wasn't, and it follows that the position that it was, is now the position that it isn't.In the event that the position that it is in is not the position that it wasn't, the system has acquired a variation, the variation being the difference between where the missile is, and where it wasn't. If variation is considered to be a significant factor, it too may be corrected by the GEA. However, the missile must also know where it was.
The missile guidance computer scenario works as follows. Because a variation has modified some of the information the missile has obtained, it is not sure just where it is. However, it is sure where it isn't, within reason, and it knows where it was. It now subtracts where it should be from where it wasn't, or vice-versa, and by differentiating this from the algebraic sum of where it shouldn't be, and where it was, it is able to obtain the deviation and its variation, which is called error.

[Wilkens]

The root cause was update 6. The problem that so many people are experiencing didn't happen until update 6 went live.

That's like saying the problem with your favorite team losing is that they didn't scroe enough points....sure its obvious but that doesn't actually help you solve the issue with what SPECIFICALLY went wrong and how you need to fix it.

[probitas]

Well for most people, the only thing that changed recently is the addition of U6. I would concentrate on that as a starting point, then branch out to the server/client code. Nothing else users may have done besides turning off modems and unplugging routers could account for that, unless they also have a PC downloading torrents at the same time.

I do have one idea. For those with wireless routers, whether you use the wireless part or not, get the mac addresses of all the computers you use internally, turn on mac address filtering in the router, and set up an allow only list, and add your computers to it. Then change the master password (not the key!!) on the router, to prevent easy access. It may be that people are having others piggy back their routers for some reason. I know it's a stretch to conclude it's all at the same time, but still, just for security it would be a good idea.

[Rainothon]

Please do not reinstall. Maybe I did not make things clear enough. We have seen that it helps in some cases and not in others. That means it is not the root cause. If it were, then it would resolve the issue 100% of the time. It doesn't.

What we do know, for a fact, at this point, based on all our research, is:

• It is not server side.
• It is not in our datacenter.
• It is client based.

We have some theories we're testing and should know more once we run them all down.

Thank you, I knew it wasn't my hardware nor my ISP. Now I know you know this too and are on the right path to fix the freezes. Because that is what they are. Not network lag. Freezes/stutters.

Here's another tidbit: the effects (not the cause) are reduced with better hardware, I suspect a SSD helps to reduce the effects. I think this had to do with loading data 'to catch up' after a freeze. A stutter is essentially the same as a freeze, it's just that those with slow setups experience a freeze, and those with fast setups experience a stutter. The stutter lasts as about long as the glitch, the freeze lasts longer as your setup works frantically to catch up. Anyway, that's my guess so far.

Thank you for the update Sapience. Here are some waffles - he! not just for you, also for the techy gerbils fighting the freezes and stutters. After you're done flogging them for what they stuffed in those lootboxes, of course

[Hurin]

I find this incredibly simplistic and frankly, insulting.

How can it be client based when entire raids lag AT THE SAME TIME? I am on one side of the country, we have someone from South America, Alaska, west coast and points in between -- and we all lag SIMULTANEOUSLY. I admit to not knowing much about the technical side of things, but I cannot accept that this is client based.

Please don't insult us further with such explanations.

Your attitude sucks.

Perhaps you shouldn't be so quick to feel insulted when you so readily admit that you don't have the expertise with which to adequately evaluate or comprehend what you are being told.

[shann81]

I hate this lag. I play Lotro on 3 different PC setups (Linux, W7+SSD, Wxp) and lag is still there. For me there are 3 occasions when lag occurs:
1.) Using stable horse (horse stops sometimes)
2.) Traveling a country.. I suppose there is new data load which lags game a little (before U6 it was more smooth)
3.) Solo fights, group fights..
I noticed that I can reproduce this problem more often if I ride a stable horse and then I press Alt+Tab or click outside game window (windowed mode). This switch-over generates some synchro-hole with server and horse usualy stops on place.. (never happened before U6)

[Lyzau]

I noticed that I can reproduce this problem more often if I ride a stable horse and then I press Alt+Tab or click outside game window (windowed mode). This switch-over generates some synchro-hole with server and horse usualy stops on place.. (never happened before U6)

I can confirm that. I used to alt+tab all the time during a regular stable travel (non-swift), like from Bree to Forsaken Inn or worse, from Rivendell to Echad Candelleth. Never had any problems with that before U6, but whenever I do that now the horse stops precisely where I alt+tab. From what I have heard, relogging helps.

[enginekid]

travelling cross country...before U6 it was more smooth

I notice this too. When i walk around a town, or ride around any area, the game stutters, or hitches, as some call it. This never happened to me pre-U6. Now it happens no matter where I go. I even reduced my GFX settiing in an vain attempt to fix it. Apparently it has nothing to do with settings. It simply hitches and stutters for no apparent reason when you travel.

[sensyi]

the hot fix did help some... ppl are not lagging left and right anymore.
But lag spikes still occur much more often than before U6.

And i personally have this mysterious freezes ever since U6. Twice or so per hour char gets frozen for 30secs -2 mins...
Cannot move, use skills, etc... Incomng traffic is ok, se other chars, chat . But outgoing traffic just about stops...

Same issue, In end boss in ToO 2 min freeze no internet issues or connection logs on routers it was pure server lag, also the amount of crashes people get when they enter isengard raids is just stupid we spend 45min-1 hour waiting for people to all get in.

[Rainothon]

I hate this lag. I play Lotro on 3 different PC setups (Linux, W7+SSD, Wxp) and lag is still there. For me there are 3 occasions when lag occurs:
1.) Using stable horse (horse stops sometimes)
2.) Traveling a country.. I suppose there is new data load which lags game a little (before U6 it was more smooth)
3.) Solo fights, group fights..
I noticed that I can reproduce this problem more often if I ride a stable horse and then I press Alt+Tab or click outside game window (windowed mode). This switch-over generates some synchro-hole with server and horse usualy stops on place.. (never happened before U6)

It is not lag. It is stuttering or freezing.

Alt-tab causing issues is another sign that something's off at the client-software level, not at your router, ISP or Turbine's servers. Maybe the techy-gerbils can use this info to reproduce the problem.

[Darmokk]

The alt-tab thing, if it can be confirmed on a larger scale, would lead me to believe that it is a problem with the client-side mechanism for exchanging network packets (I mean the one in the game, obviously the OS isn't involved in the problem).

[jcmanda]

The alt-tab thing, if it can be confirmed on a larger scale, would lead me to believe that it is a problem with the client-side mechanism for exchanging network packets (I mean the one in the game, obviously the OS isn't involved in the problem).

Alt-Tab switching between programs has nothing at all to do with exchanging network packets. At all. It's an OS or API layer issue with swapping video memory, if the alt-tab is at fault.

[Wolfhardt]

Ok i´ll show my situation in short:

I use two diffrent pc´s for playing.

PC 1 for High-Res Playing and Raiding. Its a high-end setup (i5 2500k, geforce 580GT, 8GB DDR3 RAM, Lotro intsalled on SSD Harddrive, os: windows 7 ultimate, service pack 1)

PC 2 for low res gaming, questing and so on ( dual core, geforce 8400m, 3gb ddr2 ram, Lotro installed on sata driv, os: windows xp service pack 3)

Lags occur on both systems.
Internet connection is VDSL 50 and also checked gaming on another connection at a friends home. also re-installed the whole PC1 setup completly (a week ago, complete new windows and programs/drivers)

i have NO lags in times till 6 PM (GMT +1) but then i have micro-lags and skill lagging till 11-12 PM...and then...no lags anymore...and you say it´s client based?

That makes no sense for me....problems occur since Update 6, only occur in *main* playtimes off all timezone, occur on BOTH systems....and so on.....

so what should we do, if reinstalling a complete system or client doesn`t help? my kinship acutally lost two players, who due to these lags could not play anymore ?

[Mirarian]

A couple of things other folks have said resonated with me and maybe are worth adding to the data gathering along with system specs, OS, and connection.

Someone mentioned they could fair reliably repro the stalled traveling horse bug by alt tabbing out on a long ride. It occurred to me as I thought about it that though I have only experienced this bug once I was indeed surfing the web during the trip. I don't alt tab though I just launch my browser from my bottom bar on windows.

The other thing someone mentioned was they play in windowed mode not full screen. I too play in windowed mode and have very few problems with lag comparatively to what is described in this thread.

Might be worth looking to see if there is a pattern there.

[shann81]

Alt-Tab switching between programs has nothing at all to do with exchanging network packets. At all. It's an OS or API layer issue with swapping video memory, if the alt-tab is at fault.

Don't think so.. My stable-horse stays in place and I have to click on Dismount a go back to Stable-master. There is no graphic freeze, just horse stays in one place. Maybe Alt-tab switching takes some time when client misses to send stable-horse location-data, server "forget"s about that horse and therefore horse stays still. But again there wasn't such problem before U6.
And right after U6 my stable-horse was doing more crazy stuff.. running impossible routes, right into palisade or down the cliff. :-)

[Grimfellows]

Every time I use a stablemaster mount I think of that line from Aliens "Nobody touch nothin!". If I pop up my map, the horse stalls, if I alt tab or start an app the horse stalls etc. I haven't been playing very long but this definately started with U6, it never happened to me once before that.

[Eartholloth]

I been playing on Brandywine to see if I have more problems then what I been experiencing on other servers & I have allot more problems playing on this server.

[Sapience]

I find this incredibly simplistic and frankly, insulting.

How can it be client based when entire raids lag AT THE SAME TIME? I am on one side of the country, we have someone from South America, Alaska, west coast and points in between -- and we all lag SIMULTANEOUSLY. I admit to not knowing much about the technical side of things, but I cannot accept that this is client based.

Please don't insult us further with such explanations.

I apologize if you were offended. Let me try to explain what "client side' means. It certainly does not mean 'your fault' as it seems you took it to me. It means the game client itself seems to have some sort of issue. Because this is an MMO and grouping requires cooridination of clients to some extent, grouping will actually make the issue more pronounced, causing all clients in the group to react the same way. So in fact, if you were not seeing a whole group suffer the same issue at the same time then the more likley culprit would be something unique to those player suffering the issue and not the client, or even possibly the server.

Given all the accidental rollbacks on fixes from pre-U6, I'm not at all surprised. I hope rolling out 6.1 changes won't further obscure the issue.

One client side issue causes these two?

• All members of a raid in the same boss fight having for the same number of seconds at the same time
• You getting unhorsed on a stable horse

Sounds far-fetched. The only possible thing would be if the client-side networking code has been screwed up (e.g. it now is broken when handling some network conditions).

Non-networking client side issues also offer little explanation for Turbine's employees doing better. Unless of course they didn't really test the things that were lagging for the users.

Your LOTRO client talks to the LOTRO server.

If your LOTRO client doesn't know how to talk to the LOTRO server properly, because it's having a bad day, or it's sad because its dog just died, or because one of its files got corrupted or something, then you might notice a problem.

Now, if there was a problem in one of the LOTRO client files that got PUSHED to every user's machine in a big update, and that problem only manifested itself in a certain (common) environment (network card type X, video card type X, etc), and furthermore only when certain SERVER-side events triggered that bit of CLIENT-side code to load....

See? More stone than sand.

In short, correct.

It is not lag. It is stuttering or freezing.

That's pretty close, and why tracking it down is hard. Lag means specific things. Unfortunately anything impacting game play is usually listed as lag or 'server issues' by player, even when it's something else. You'd be surprised how many times we get 'lag' reports from people who are running 7 year old machines on HIGH and have massive hitching. Turning down the graphics settings to medium immediately makes it go away but they report the 'lag' is back as soon as they put the game back to HIGH.

Clientside networking issues (especially packet handling) can cause hitching locally as well with no apparent cause or impact anywhere else in the connection (thus why we are looking at servers that are performing as expecting with no issues, but users are experiencing the issue).

The alt-tab thing, if it can be confirmed on a larger scale, would lead me to believe that it is a problem with the client-side mechanism for exchanging network packets (I mean the one in the game, obviously the OS isn't involved in the problem).

The client side networking code is part of what we're investigating. There seem to be several changes that together are impacting performance in very unexpected ways. What we call "halo" effect.

Halo effect can be pretty crazy sometimes. Imagine someone down the road from you plants an apple tree and your toilet backs up. The two things shouldn't be related, but somehow they are. That's halo. trying to find what connects the two things is not an easy task.

[Osgril]

I'm starting to believe that a major trigger for lag is proximity to other players, as has been hinted at before.

I do have experienced skill lag and freeze-ups when being completely alone however so it's not the whole truth. Yesterday I was running around finishing up some deeds in Trollshaws, and according to /who I was alone in the zone. Still had a couple of skill lags and some stuttering.

But, today I've been crafting a lot, running around in Galtrev and other towns with plenty of other players around. It's been a lot worse - stuttering when entering and exiting buildings, rubber-banding while running around. Almost as if the game client is unable to load textures quickly enough and goes down to a crawl while they're loading. But I do run the game off a fast SSD so that should be a non-issue. It's not "fragmented" or degraded either, I've run some benchmarks on the drive and it's at its peak performance. Not to mention that I never had these issues pre-U6, on the same hardware.

I usually run the game at Very High settings at a stable 60fps (vsync on). Tried playing with settings, from Low to Ultra and found no difference except dropping to 40-50fps sometimes on Ultra, which reminded me why I turned it down a notch.
I run at 1920x1200 resolution by the way.

Tried removing, not just disabling, add-ons (CombatAnalysis, Buffbars) - no difference.

Tried repeatedly running in and out of the same building to see if I still had stuttering, and I had. That should eliminate disk I/O as a bottleneck I would presume, since textures would've been cached?

System specs:

Core i7 6-core CPU @ 3.9GHz
EVGA X58 mobo with built-in Realtek network adapter
12GB RAM
Dual AMD 5870 1GB (tried both with and without Crossfire), Catalyst 12.3 drivers
2x Intel Gen2 160GB SSD, one for OS and apps, the other dedicated to games.
Windows 7 64-bit, up-to-date.
Clean install - no apps, no antivirus or firewall used (this is a dedicated gaming PC now, got a new work machine that I do all internet stuff on)
My ISP is a cable company (Comhem) and I'm rated at 100Mbps down, 10Mbps up. No issues with internet connectivity.
Internet speed tests while experiencing lag in-game show up fine, close to my rated speeds.
Location: Sweden

[Armitas]

My PC
Radeon5800
processor quad core
Graphics are at medium
8GB ram, with a raptor drive

My PvP experience.
In the moors I am getting 100fps. When I "lag" my fps and latency is unchanged.

Every player around me will freeze and then return to their actual positions. My character does not lag, only the other players.

If there are lots of creeps and freeps within visibility it tends to lag when a creep approaches melee range and not while they are far away. This is not always the case, but when it is there will be no "lag" until I reach melee range, then it releases once they are out of range. This is the case the majority of the time I experience "lag". I do not always lag if 1 creep is in melee range and no other players are in sight.

The other majority is when I am riding horseback with freeps. I am not in melee range, but around 10m from freeps.

My PvE experience. Unlike pvp, my fps is volitile here. I can't confirm if the lag is the same as in pvp because my fps does drop. We usually aren't moving that much so I can't confirm that the "lag" is exactly the same. What I see is a tank go from full health to dead. Skirms/acid were never a problem before U6 for me at these settings.

I am at around 40-60fps in pve, when I lag in pve my frames are dropping to 10 or lower on medium graphics. I lag (fps lag) more in skirmishes.

I always lag in acid wing boss. We all lag, $2500 computers going from 60fps to 9fps as soon as the boss fight starts.

[Thorcar]

We had a lot of problems last night in Orthanc. Twice, the whole raid had a good 2-3 second lag at the same time during the Fire & Frost boss fight. Two of our group got stuck in inductions while out of combat and had to relog. One of our guards lost the ability to put protection on another player.

[Arodion]

It takes SO LONG to log in, like at least 5 minutes on my 75 lm. Lower level characters are better but still take a few minutes. Maybe it's because they're outside the great river area, idk, I can test this later. But when I'm logging, the little glowy circle above the 'loading...' screen will freeze and if I click the window it will say 'not responding' but then recover after a few seconds.

My lower levels don't lag as much but I still have problems with skills not going off. It's very frustrating. My 75, I can't even move hardly. I just get frozen in place. It was okay last night but the second I walked into Stangard, I started getting hitching and rubber banding and the NPCs weren't showing up. I had to leave and return for them to appear. But even when I left that zone it didn't get better. Sometimes I'd be able to walk but the NPCs would freeze, other times I'd be frozen but everything else still moved and if I was in combat, I'd still be taking damage and therefore die because it would take so long to recover, 5+ seconds every time.

Stable horses are awful, like everyone else said. As to windowed vs full screen, I play in windowed move and have lots of problems.

I can't raid or group at all anymore so I haven't really been playing. =/

I did submit a tech report and got a response but just wanted to add my experience to the thread.
Thanks Sapience for helping track this down.

[Vescran]

Hi Guys.

Please don't laugh at me or insult me because of what I am writing here now. I am not that technical superman that has enourmous insight in all the details but I just want to share my experiences with you.

I had huge lag problems after U6. I am a player from Germany and I had some delay before that update, too, but they weren't as severe as they are now.

First some detail regarding my system:

Intel I5 2500k 3,30Ghz
MSI C45-B3 Board
8 GB Corsair Vengance 1600MHz
Zotac 570GTX Amp!
Intel 510 Series SSD 120GB
Win 7 professional 64 bit

All at standard clock - no overclocking. All drivers up to date

The reason why I chanced something today was that my Windows Index only gave a 6,8 for my SSD drive. So I did a little research (google is great) and I read something about AHCI-mode. So I check up in my bios and I had to realize that there was IDE mode set. So I changed a registry entry within my windows and changed the sata-mode to AHCi in bios.

Now my Windows index gives a 7,8 to my SSD and strangely enough my lags in Lotro are gone. I will observe this the next time. But there was no lag since nearly 2 hours.

As I already said: I am no technique guru. So do not take this too serious. Just sharing my experiences.

Regards
Vescran