PDA

View Full Version : World Restarts: Monday, March 21st 3:00 PM Eastern (-5 GMT)



Cordovan
03-21-2016, 01:23 PM
We will be bringing down Argonnessen and Thelanis at 3:00 PM Eastern (-5 GMT) today, Monday, March 21st, for diagnostic testing and maintenance work. We will keep you updated. No immediate ETA to announce.

Lagin
03-21-2016, 01:44 PM
We will be bringing down Argonnessen and Thelanis at 3:00 PM Eastern (-5 GMT) today, Monday, March 21st, for diagnostic testing and maintenance work. We will keep you updated. No immediate ETA to announce.


I knew it!! The Thelanis issue was/is those kobolds from House C challenges getting out and reaking havoc.
And people thought I was crazy.......oh wait....:rolleyes:

FifthTime
03-21-2016, 02:08 PM
I guess this saves me the frustration of actually playing the game.

Now the fanbois can truly claim that there is no lag on Thelanis and actually be telling the truth.

Malex
03-21-2016, 02:11 PM
please DONT fix the lieutanants getting stuck in walls in shroud part2.

With this lag it's the only chance to shatter the crystal if you get fire or cat + lag

drwho1985
03-21-2016, 02:18 PM
Thanks for the communication!! Hope you find the gremlins responsible and punish them appropriately.

Enoach
03-21-2016, 02:40 PM
This is the type of communication we like to see. Thank you.

So Argonnessen/Thelanis are causing LAG for the rest of us? I know it is only wishful thinking :)

Hobgoblin
03-21-2016, 02:46 PM
is this going to happen toallservers?

do you have any ideas about how to fix the lag?

GlassCannon
03-21-2016, 02:56 PM
We will be bringing down Argonnessen and Thelanis at 3:00 PM Eastern (-5 GMT) today, Monday, March 21st, for diagnostic testing and maintenance work. We will keep you updated. No immediate ETA to announce.

Thanks! Love the updates. Please keep us informed.

Argo was having some seriously major issues. A party of 3 couldn't complete Epic Elite (at level) Lesson in Deception and getting to the guy in the entrance to talk to him and initiate the first fight took nearly 5 minutes. The fight itself took another 3 minutes (it normally takes 20-30 seconds with that amount of AoE damage and Crowd Control) and the others left. I finished the quest solo with only several major lag incidents. The rest of the chain went smoothly (with the occasional reminder that lag was lurking around the corner waiting to strike).

sleepingferret
03-21-2016, 02:59 PM
Guess this means I get to take a nap, before I go to my tabletop group this evening.

Nédime
03-21-2016, 03:03 PM
Yesterday, on Argo, while running an Epic (normal or hard) Inferno of the Damned, I saw a CR 0.00 kobold in a corridor with ~ 7k hp.

I have a screenshot but I'm too lazy to create an account on a free uploading site.


Thanks! Love the updates. Please keep us informed.

Argo was having some seriously major issues. A party of 3 couldn't complete Epic Elite (at level) Lesson in Deception and getting to the guy in the entrance to talk to him and initiate the first fight took nearly 5 minutes. The fight itself took another 3 minutes (it normally takes 20-30 seconds with that amount of AoE damage and Crowd Control) and the others left. I finished the quest solo with only several major lag incidents. The rest of the chain went smoothly (with the occasional reminder that lag was lurking around the corner waiting to strike).

Hah - I even had lag on our guild ship

DagazUlf
03-21-2016, 03:18 PM
And people thought I was crazy.......

Past tense?

Cocomajobo
03-21-2016, 03:19 PM
Both Argonnessen and Thelanis are now open to players!

We'd love any before/after performance reports encountered on these two worlds that players are able to provide.

Lerincho
03-21-2016, 03:23 PM
Both Argonnessen and Thelanis are now open to players!

We'd love any before/after performance reports encountered on these two worlds that players are able to provide.

Specifically tracet? Specific locations? What are you needing to be able to isolate issue.

Cordovan
03-21-2016, 03:24 PM
Argo and Thelanis are now open. We made some adjustments to these two worlds that we will be keeping an eye on over the next 24 hours.

Cocomajobo
03-21-2016, 03:32 PM
Specifically tracet? Specific locations? What are you needing to be able to isolate issue.

General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

SithKhan
03-21-2016, 03:35 PM
General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

My guild airship is missing all the amenities after the startup. It would be nice if this was fixed. Thanks

Clemeit
03-21-2016, 03:39 PM
My guild airship is missing all the amenities after the startup. It would be nice if this was fixed. Thanks

Seconded. But if this restart actually reduced the lag, I'm willing to just forget about the missing amenities.

Cordovan
03-21-2016, 03:41 PM
My guild airship is missing all the amenities after the startup. It would be nice if this was fixed. Thanks

This is typical behavior; your amenities should re-populate within an hour or so.

Haplo
03-21-2016, 03:42 PM
Both Argonnessen and Thelanis are now open to players!

We'd love any before/after performance reports encountered on these two worlds that players are able to provide.

Well, our guild airship is completely empty and missing every buff station, vendor, and so on on it....does that count as a "performance report", because I would say that is a problem.

Lerincho
03-21-2016, 03:44 PM
Well, our guild airship is completely empty and missing every buff station, vendor, and so on on it....does that count as a "performance report", because I would say that is a problem.

Negative, this is behavior that occurs with restarts as Cordovan stated. They will reappear, sometimes insuring you the first to log in does not require patience.

Clemeit
03-21-2016, 03:46 PM
Sounds good! Will check back later.

Drwaz99
03-21-2016, 03:48 PM
First character I logged on with (on Argo), checked the mail, went to log off to another toon, bar filled and disappeared (the log off bar) and it took about 90 seconds to actually log off and return to character screen. I could run around and do things, just not log off. Second toon, took about 30 seconds longer and that pattern repeated (adding time to actually log off) itself through 4 toons. Last one I didn't wait to see how long, I just alt-f4'ed to log off.

xSeverinax
03-21-2016, 03:48 PM
This is typical behavior; your amenities should re-populate within an hour or so.

I am trying to figure out when this was posted, as there is several hours between the initial downtime post and this and yet if you took the game down at 3pm that would make it 8pm my time and it is currently 8:37pm gmt. The reason I ask is that not only are my amenities missing, when I stand on the hookpoint to put them back several of my amenities were actually gone and asking me to buy them again. As a guild of 1 it is rather expensive to buy all those astral shards again! So will they all return?

SithKhan
03-21-2016, 03:49 PM
This is typical behavior; your amenities should re-populate within an hour or so.

Thanks Cordovan, I will let you know if it does not repopulate

Eryhn
03-21-2016, 03:49 PM
General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

last night I pugged EE stormhorns 1-3 in a group of 4 and had static raid group for LN TS, Hound and EN Peaks with a group of 10-12. it was quite horrible on all though nothing wiped. major lockups in lines of supply and breaking the ranks, constant stutters in TS, Hound went best but was hitchy, too with a major lagout at completion. Peaks was nasty. Party chat broke for 2 people who had restarted client after zoning to estar and one latecomer, had to reform group to make that work. Lots of lots of stutters and 1-4 sec freezes on the trash, and at the end the server failed to take notice that the dragons were dead and kept spawning trash (chest was lootable though). Several people had persisting fire walls dmg in public area after peaks, and so on and so forth.

Looking forward to any improvements after the reboot, will report.

p.s. on argo

Lerincho
03-21-2016, 04:00 PM
So will they all return?
Yes they will. They always do.

SithKhan
03-21-2016, 04:20 PM
My guild airship is missing all the amenities after the startup. It would be nice if this was fixed. Thanks

Amenities have repopulated, thanks

Lerincho
03-21-2016, 04:22 PM
Amenities have repopulated, thanks
So the panic was unjust?

RydeaNikkna
03-21-2016, 04:27 PM
I did red fens wilderness with no lag, got into the pit and it was fine, until i tried to wand heal my party member. Just me and her in here, no hires/pets. The wand heal didn't go off, the 2nd try did but lagged. I get to the first furnace valve and use it, wait a second then it animates.

Opened the shrine in the 1st furnace, and had a slime fall all the way down off 2 ledges.

Did have a couple trogs jump off the ramps in the security center.

Did not see rubberbanding, or stuttering in the toons.

Toon: Laralyn Antaghar

Manganakus
03-21-2016, 04:30 PM
Please do the same on Ghallanda, since for many people lag is worse than before (hard to believe) since DC move...

Maykoe
03-21-2016, 04:33 PM
Stress tested DoJ at 4CST.. 11 members~ Failed due to lag
Not sure how to make the pic show up in post but here is the link



http://oi65.tinypic.com/4glpco.jpg

Shadow_Flayer
03-21-2016, 04:34 PM
Didn't have time to do much so I ran one of the Underdark arena challenges. I mostly solo, these days on a level 21 barbarian with one of two hirelings. In the last week lag has been consistent for me, with lots of rubber banding, dps lag (cleave and wait 5 seconds for numbers to fly), bank lag, zoning lag (hirelings showing up 30 seconds after I was in when going from a landscape to a quest).

Entered the challenge, let mobs build up, cleave cleave cleave. No noticeable dps lag. No jump lag (hanging in the air for no reason). It's only one datapoint, but things seemed to be smoother than in the morning. I'll see how things go later this evening as population builds on the server.

This was on Argo.

Jomee
03-21-2016, 04:47 PM
Both Argonnessen and Thelanis are now open to players!

We'd love any before/after performance reports encountered on these two worlds that players are able to provide.


Could only run a short period this evening but the lag seems to be much improved from the weekend. I had actually given up playing on Sunday due to the lag.

Tentaki
03-21-2016, 04:50 PM
Argo player here. I mostly solo, and previously there was usually a fair amount of lag. Not enough to make it unplayable, but pretty noticeable and it did affect running through the dungeon. In groups, though, the lag would blow up and make most of the run unplayable.

After the reboot, I've been playing for about an hour and no real issues to speak of. Everything seems mostly back to normal. However, it's not peak hours yet and I haven't tried a raid, for context.

Nédime
03-21-2016, 04:51 PM
Stress tested DoJ at 4CST.. 11 members~ Failed due to lag
Not sure how to make the pic show up in post but here is the link



http://oi65.tinypic.com/4glpco.jpg

Hey I was there too Chopsley ! Note that one guy was disconnected all the way so only 10 ppl really online. Began with a small lag on first fight at door then seemed to go normal then after 3rd or 4th archon it became unbearable. Lasted till wipe.

Before that I ran a solo EE Thrill of the hunt, went no problem.

Then just after the DoJ wipe I ran EE Friends in low place solo, there were some short lag spikes at each large mob spawns nothing that cannot be managed but still very noticeable, each time a group of bad guys appeared.


Edit : in both solo quest, a 2-3 seconds between the boss death and the DM's message, then 2-3 seconds before xp and quest completion are displayed

Kristii
03-21-2016, 04:51 PM
So are other servers going to get some love too? Cannith is in shambles. Having any more than 6 players in TH wilderness this morning and last night made it unplayable because no one in the party could move. Even with 6, 15-second lag spikes after any rare was killed.

Cocomajobo
03-21-2016, 04:56 PM
So are other servers going to get some love too? Cannith is in shambles. Having any more than 6 players in TH wilderness this morning and last night made it unplayable because no one in the party could move. Even with 6, 15-second lag spikes after any rare was killed.

This work was done as a test case to attempt to isolate issues causing the lag. When we determine that we have found the root of the issues through these test cases we will, of course, implement the changes on all servers.

Cordovan
03-21-2016, 04:58 PM
Heads up: We will be restarting Ghallanda at 6:30 PM Eastern (-5 GMT) for maintenance and to put in place the same monitoring we have on Argo and Thelanis at the moment. We've been getting a moderate number of lag reports from Ghallanda specifically today, so we want to take the world down for 60-90 minutes to make some adjustments and see what kind of impact it has.

RistoffDervish
03-21-2016, 05:18 PM
I am trying to figure out when this was posted, as there is several hours between the initial downtime post and this and yet if you took the game down at 3pm that would make it 8pm my time and it is currently 8:37pm gmt. The reason I ask is that not only are my amenities missing, when I stand on the hookpoint to put them back several of my amenities were actually gone and asking me to buy them again. As a guild of 1 it is rather expensive to buy all those astral shards again! So will they all return?

Don't try to put them back. I think it can make the situation worse if I recall correctly. Just wait it out and if they don't come back or got overwritten somehow, customer service will help you.

FifthTime
03-21-2016, 05:32 PM
Still getting stuttering, rubber banding, and freezing on Thelanis in public areas and quests.

I will try a raid later to see if they are also still crippled.

Duana
03-21-2016, 05:53 PM
Still there, but IMO much improved. Will know more tonight when we do the TS.

In my mind this has gone from ****ing irritating to on the right track.

Knobull
03-21-2016, 06:03 PM
Have you looked at the BIG-IP appliance(s) configuration(s) and logs yet?

Eryhn
03-21-2016, 06:08 PM
argo

tentative improvement but definetly not something that totally fixed it. still, might be on the right track.

loged in and saw an LFM of 6 for thunderholm slayers: lag test. seemed just right. in total, while i was in party, cleared the instance 4 times or so.

the first run was with 7 people, and 2 more joining half way, and was by and large great. everybody seemed to think it had improved and was running smooth. you could run into a group of 20 dwarves, swing away, and not lag out one bit.
by the time we got to forge, and some people still were above, the first slight stutters showed. when the first 5 of us reched bottom, I charged ahead right into the red named and her escort of 15-20 trashs and successfully managed to lag myself out and die.

the following runs were with group of 10, then 12, and much more laggy on all levels of the dungeon. averse to the green icon lag of the last weeks, on spikes and mid air freezes i would on occasion see my netgraph go yellow, spike to 500 latency and settle down again within 5-10 seconds to a stable level. lagouts on later runs affected full group, usually lasting around 10 sec to 2mins or so before some mobs or players died and it then slowly cleared up again.

now, it could be the first run was best cause manageable group size with 8 or so. or that at the time not so many folks had logged into the server yet. or, if you have some issue causing lag that stacks itself up over longer time, after the reboot it was good cause it hadnt become an issue yet, idk. or any combination of these i suppose.

bottom line: tentative thumbs up in the sense of *something is different and it isnt worse* however we are not in the green yet by some margin

as it stand now i would expect groups of 4-6 to perform somewhat less laggy if not perfect, but definetly less aggravating. raids still would expect to be a mess, especially on the ones with large spawns.

all this, as mentioned, based on 1h testing JUST in thunderholm slayers. will give more feedback tomorrow after more grouping.

Giedon
03-21-2016, 06:15 PM
Hope this fixes it. I bailed on two DOJ attempts due to raid members not being able to make it to the first gate to start the fight. I have noticed for some reason when lag hits and can't move, doing an evasive move (block+direction) gets me moving again. But with the lag I mostly just solo lately, seems to limit the lag issues.

Codar
Sauthe

Elsbet
03-21-2016, 06:15 PM
I'm just now logging in to see how things are going, but I appreciate this level of communication and response. Would still love to see something from Sev, but serious thanks for the thread!

Cocomajobo
03-21-2016, 06:29 PM
I'm just now logging in to see how things are going, but I appreciate this level of communication and response. Would still love to see something from Sev, but serious thanks for the thread!

Unlike me, Sev has the skill set to actually help implement the work being done. Thus I'm talking while he's fixing stuff. =]

Rog
03-21-2016, 06:42 PM
argo

tentative improvement but definetly not something that totally fixed it. still, might be on the right track.

loged in and saw an LFM of 6 for thunderholm slayers: lag test. seemed just right. in total, while i was in party, cleared the instance 4 times or so.

the first run was with 7 people, and 2 more joining half way, and was by and large great. everybody seemed to think it had improved and was running smooth. you could run into a group of 20 dwarves, swing away, and not lag out one bit.
by the time we got to forge, and some people still were above, the first slight stutters showed. when the first 5 of us reched bottom, I charged ahead right into the red named and her escort of 15-20 trashs and successfully managed to lag myself out and die.

the following runs were with group of 10, then 12, and much more laggy on all levels of the dungeon. averse to the green icon lag of the last weeks, on spikes and mid air freezes i would on occasion see my netgraph go yellow, spike to 500 latency and settle down again within 5-10 seconds to a stable level. lagouts on later runs affected full group, usually lasting around 10 sec to 2mins or so before some mobs or players died and it then slowly cleared up again.

now, it could be the first run was best cause manageable group size with 8 or so. or that at the time not so many folks had logged into the server yet. or, if you have some issue causing lag that stacks itself up over longer time, after the reboot it was good cause it hadnt become an issue yet, idk. or any combination of these i suppose.

bottom line: tentative thumbs up in the sense of *something is different and it isnt worse* however we are not in the green yet by some margin

as it stand now i would expect groups of 4-6 to perform somewhat less laggy if not perfect, but definetly less aggravating. raids still would expect to be a mess, especially on the ones with large spawns.

all this, as mentioned, based on 1h testing JUST in thunderholm slayers. will give more feedback tomorrow after more grouping.
If they can get thunderholme slayer running smooth. in a 12 man. that's something that never done at lest on Khyber. when the party is splits apart there is to many spawns.
I will keep my hopes up.

Cocomajobo
03-21-2016, 06:49 PM
Ghallanda is now open to players! Just as with Argonnessen and Thelanis we'd love any before/after performance reports encountered on these two worlds that players are able to provide.

As to what type of things that means we are looking for:

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

Cordovan
03-21-2016, 06:49 PM
Ghallanda has reopened. Thank you!

Giedon
03-21-2016, 06:55 PM
LFM posted for LN Shroud - Codar

Elsbet
03-21-2016, 06:55 PM
Unlike me, Sev has the skill set to actually help implement the work being done. Thus I'm talking while he's fixing stuff. =]

Understandable, but not really the point I was going for. Thank you for keeping us up to date. Haven't died due to lag yet thus far.

Shadow_Flayer
03-21-2016, 07:11 PM
Didn't have time to do much so I ran one of the Underdark arena challenges. I mostly solo, these days on a level 21 barbarian with one of two hirelings. In the last week lag has been consistent for me, with lots of rubber banding, dps lag (cleave and wait 5 seconds for numbers to fly), bank lag, zoning lag (hirelings showing up 30 seconds after I was in when going from a landscape to a quest).

Entered the challenge, let mobs build up, cleave cleave cleave. No noticeable dps lag. No jump lag (hanging in the air for no reason). It's only one datapoint, but things seemed to be smoother than in the morning. I'll see how things go later this evening as population builds on the server.

This was on Argo.

After wandering the Fen's and doing a quest there (protect the camp/town or whatever) it appears things aren't much better. Things were fine until the 3rd round where a bunch of mobs appear at once casting spells. Things were back to slide show mode, jumping and hanging in the air, no hits registering, rubber banding. Sadly, not much improvement after all.

LrdSlvrhnd
03-21-2016, 07:24 PM
This is typical behavior; your amenities should re-populate within an hour or so.

You really should just auto-add this to the "The worlds are open again" announcement, because it seems like there's always half a dozen people who didn't get the memo the last 482 times lol

Steve_Howe
03-21-2016, 07:27 PM
You really should just auto-add this to the "The worlds are open again" announcement, because it seems like there's always half a dozen people who didn't get the memo the last 482 times lol

Wouldn't matter. People are that dumb.

Glenalth
03-21-2016, 07:32 PM
Did the Stormhorns chain on Argonessen with just a couple small hitches that didn't seem out of the normal.

Did the same yesterday and it was comically bad.

Giedon
03-21-2016, 07:36 PM
Testing LN Shroud so far so good. Playable but not perfect, lag spikes here and there but no lag deaths haven't wiped and playable by all. Typical lag spikes after completing the stage. Moving to DOJ after this.

Sho-sa
03-21-2016, 07:37 PM
Same problem I've been having since this began.

VIP
Trying to enter Gland.
Server connection timed out...over and over again....

Hobgoblin
03-21-2016, 08:08 PM
tomb of the forbidden ghallanda.

all 3 of us are frozen

5 minutes and counting

Aelonwy
03-21-2016, 08:12 PM
General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

While I do appreciate, and am very encouraged that you guys are working on the lag problem whatever you did seems minimal on my end.

Thelanis (tested lag in Heroic Wheloon, Friends in Low Places HE, and Heroic Stormhorns)
Lag is slightly less frequent than before but by no means as unobtrusive as before data center move.
Lag is lasting about a minute or so at a time.
Lag is affecting everyone in the party but to slightly different degrees. That is, the delay for the game to recognize that something has occurred like a hit, swapping gear, the usage of a shrine for instance may be 10 or so seconds for me but twice as long for my husband who plays beside me. As the play session continued the lag got worse such that the delay increased over time. We also saw some intermittent rubber banding mostly in the heroic wheloon wilderness zone but also some in the harbor.
We did not see mobs bungee jumping but we gave up playing after only an hour.

Thank you for your efforts to fix the game.

xSeverinax
03-21-2016, 08:18 PM
Don't try to put them back. I think it can make the situation worse if I recall correctly. Just wait it out and if they don't come back or got overwritten somehow, customer service will help you.

All up and running now, many thanks.

Giedon
03-21-2016, 08:20 PM
Made it past the first gate/fight but LAG making it unplayable. Latency is about 39 Sent is 590 received is 10K+ with no loss. Raid failed, Latency stayed about 35 with Data Sent about 590 Data Received 10K+. DOJ unplayable with 12 people. Attempting again with 8.

Edit: standing in Amrath I'm running about 150 latency / 350 sent / 1100 received waiting for one more person to try EN DOJ 8 man

8 man attempt DOJ EN - failed - latency jumping from 30 - 500 / sent from 70 - 700 / received 3k - 10K+ never made it past the first gate.

gothboy
03-21-2016, 08:38 PM
Running TH ... all players lagging right out of the starting gate and continue to lag.

Eryhn
03-21-2016, 08:39 PM
Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

forgot that part on earlier comment on argo thunderholm:

yes, dragon cultists from the 3 level mirror room near lower boss still will fall of their walkways.

another thing that was very weird: on one run, after we had cleared to bottom and where getting ready to recall one party member was left dead in upper level where the netherese are. i went back there, through the forge, my invis scroll failed and i was in a hurry so i aggroed about two dozen salamanders. didnt think much of it, until 2 mins later, the whole bunch of them came around the corner on the netherese level where we had just raised our party member.

0.o

those buggers followed me all the way up there, stubbornly. THAT, I havent seen in there before, though, once pondering it, mobs in general seem to leash back to their spot far less and more often follow you all accross the map in quests lately, spiders, too i.e. ...

Thunder-Monkey
03-21-2016, 08:53 PM
- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Ghallanda, group of 7, Thunderholme wilderness:

- Are you seeing lag as often as you were before? Most encounters are playable, but larger groups of mobs are causing some minor stuttering that reminds me of playing on my old OLD pc. This IS an improvement but its still worse than before the move.

- When you do encounter lag, how long does it last? However, when we hit the stairs room on the 1st level the whole party lagged for 5+ minutes while being shot at by many skelly archers. Movement was barely possible during this time and all spells/heals did not proc until we had killed enough mobs and then all the spells/heals caught up in one go. One of our party was monitoring net traffic during this lag spike and reported it was fairly constant (ie; it didnt flat-line)

- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)? i did notice a spider trying to reach me and dropping straight off the path into the water bewtween us.

Hope this helps.

TM

Hue_Dye
03-21-2016, 09:00 PM
- Are you seeing lag as often as you were before?

Yes, it is a constant choppy presence, that will sometime spike. Spikes occur when large mobs spawn or when multiple AoE spells are cast

- When you do encounter lag, how long does it last?

As above it is constant choppy performance with spikes lasting 2 to 10 seconds on average by my estimation. I did not run raids, but was with a full party.

- When you do encounter lag, is it affecting everyone in your party?

By census after running several chains, yes, although to certain degrees of effect.

- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

I only rarely encountered this previously, and have not seen it since the restart of Argonnesson.

Rog
03-21-2016, 09:33 PM
Since I have been complaining ,and your asking help for pre and post fix.
here are some of the lag and bug issues I notice. this is on Khyber not fixed yet
lag> 1.I lagged in to the lava on the way to the rust monster while climbing the ladder.
2. the spot where you get blown up by the wind is a wild ride to say the lest made it up to the second bridge before rubber banding 2 and landing below wind billow.
Bugs to note
1. people are still in voice chat after leaving the raid party.
2. 2 drawfs fell off into the spire down that leads to the talking head. there used to be only undead down there.
the next is a different quest: into the deep on ee
the rezz shines failed to remove all stat damage.
after the fix I will post any changes I see.

Elsbet
03-21-2016, 09:41 PM
Okay, here goes because y'all earned it. :)

Woohoo!!!

I saw considerable improvement on Argo tonight. I only had a couple of small hitching episodes in Demon Assault. The GH slayer area was MUCH better. Before tonight, we couldn't even make it to a quest if we could load in at all. It was lag free tonight. One or two little bouts of lag in Cabal.

I didn't notice any pathing issues, but I wasn't in any of the quests where I saw them before (VoN3 or the Spinner Prison one where whatsherface talks too much).

A few people did mention having issues, but my experience was much better, so

THANK YOU!

Now, about that communications plan... ;)

GlassCannon
03-21-2016, 09:43 PM
The lag in Friends in Low Places caused huge groups of mobs to spawn one at a time and typically not spawn any more AIs until the current ones had died. I was killing what would normally be groups of 5-20 in strings of 1-2 at a time.

The lag also caused certain spell effects (e.g. Exalted Angel's Quickened Maximized Empowered Divine Vengeance or whatever that big heal/damage spell is called) to not appear or do anything for 30 seconds to 3 minutes while lag kept everyone frozen.

This was the previous experience. I may have updated experience tomorrow morning.

That Accounting 4303 final took a lot out of me.

Thunder-Monkey
03-21-2016, 09:46 PM
Ghallanda, Tor, 6 in group:

Main quest: smooth, no lag spikes, minor stuttering, no pathing issues that I noticed.

Dragons: Lag spike the instant they died, lasting 5-10 seconds, on EVERY dragon/giant pair. We experienced the same in FoT a few days ago.

Hope this helps.

TM

HastyPudding
03-21-2016, 10:31 PM
I just did tempest's spine and vault of night raids in heroics on Argo server. Both raids had about a maximum capacity of 8-9 people.

1. Lag is noticeably better than before, but still not like it was before the data center move.

2. Lag comes in short stutters. Before today's maintenance, it basically made the game unplayable for about 2-3 minutes about once every few hours; the longer you played, the worse it got. Right now, it's playable, with short stutters, but nothing lasting more than a few seconds.

3. Lag always affects everyone in the dungeon when it occurs. Sometimes during really bad lag spikes, the kind that last several minutes, people talking in channels confirm that it's more or less widespread or server-wide.

4. Yes, the pathing is still off. I did tempest's spine and one of the drow blackguards needed 3 tries to walk up a ramp because they kept walking off the edge. It was quite comical, really. In vault of night, a warforged walked off the edge of the path and into the lava twice before he was aggroed enough to not wander away.

Krelar
03-21-2016, 10:35 PM
Argo:

Ran ToEE Part 2 solo, did all nodes. No noticeable lag. The previous two days I didn't even make it into the nodes before lagging out completely.

Full party ran haunted halls extended on EH and EE 3BC. Almost no lag. (Just a little hitch near the end of ghost and a chance after pulling the lever after the trap)

Haven't tried any raids yet.

hp1055cm
03-21-2016, 10:46 PM
Thelanis: Played on and off for about 3 hours tonight. Overall lag wasn't very noticeable, except for getting stuck in Von 2 (not sure if that is a result of lag though).
Monster pathing seemed normal.

(all on HE - except Relic on HH)

Taming the flames (Solo) - no lag
VON 1 (2 players) - no lag
VON 2 (Solo w/hire) - slight lag [Did not complete - wind pushed me into ether and stuck falling. Log off and on did not resolve. Recalled, but items I had collected already missing.]
VON 2 (solo w/hire) - slight lag

Cursed Crypt (6 man party) - no lag
Relic of a Sovereign Past (Solo w/hire) - slight sporadic lag
Siegebreaker (6 man party) - no lag

*Slight sporadic lag while running around House K and Marketplace.

caleneth
03-21-2016, 10:51 PM
On Thelanis, My group did lvl 13 quests tonite, and the lag was much better. Also, ran a Grim and Barrett, hardly any lag. Will say the monsters are still screwed up at times regarding falling off cliffs, etc. There are still little hiccups, here and there, but nothing like it has been.

AlreadyDead
03-21-2016, 10:55 PM
Ghallanda is now open to players! Just as with Argonnessen and Thelanis we'd love any before/after performance reports encountered on these two worlds that players are able to provide.

As to what type of things that means we are looking for:

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

5 toon party GH Tor HE and EH multiple runs (still not will to risk EE lag wipe)
lag much less than last night, still occurred in every run, and much worse than before move
when ever there is more than 5 or some monsters especially if they cast spells, it affects the entire party
didn't see the broken pathing like last night, giants would get stuck on corners and jump into the lave pits, tonight giants still struggled a little with corners
5 to 10 second lag (frozen in place) whenever a boss dies, giant in end fight and each dragon/giant pair. Dragon/giant pairs had to be killed exactly at the same time or the lag from first death would prevent the second

game is much closer to playable now but the fun is still broken.
Thanks for the update and the effort.
AD

Seikojin
03-21-2016, 11:13 PM
Ghallanda is now open to players! Just as with Argonnessen and Thelanis we'd love any before/after performance reports encountered on these two worlds that players are able to provide.

As to what type of things that means we are looking for:

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

I logged in today around 6pm pacific on sarlona, and ran some of the depths quests.
I experienced as much lag today as I have since the data center move.
I get 3 different intensities of lag. Some small stuttering in motion, affecting me and mobs and other players. A more intense wave with rubberbanding. That case, the mobs don't rubberband, but they do damage the players. Everyone is affected. Then there is freezing in conjunction with rubberbanding. Mobs don't seem affected by this and most of the time combat buffers in their favor as their hits land and ours are not received.
When I get these waves of lag, the can last a few seconds up to a minute or two (tonight at least).
I did see a few mobs wander off. No ledges, but they just 180 and go away for a long while before spinning around and stopping in the distance.

Hobgoblin
03-22-2016, 12:21 AM
so i had one instance that i said earlier in tomb of the forbidden

others i had less lag then usual

Blivit
03-22-2016, 12:29 AM
My first quest of the night was a 12-man Legendary Hard Tempest Spine on Argonessen. Lag improvements were mixed. Some people said they were rubberbanding. Some said it was just as bad as before. I, on the other hand (in West Florida), suffered only minor lag. My lag was infrequent, and only lagged for 1-2 seconds at a time. This was the "stutter" kind of lag, where everything freezes momentarily, then everyone will have move several feet from where they were and everything goes back to normal. I did not notice any lag that was specifically triggered by mob spawn/death, etc. (which was rampant prior to today's changes). From my perspective, lag was much improved, even if others in the party were still rubberbanding. Just the fact that different people in the same party and quest were experiencing different levels of lag is an improvement over before (when everyone experienced the same level of severe crippling lag at the same time, frequently).

My guild mates (1 in California, 1 in Nevada, 1 in Texas) also had improved lag. In our 2-4 man runs for the rest of the evening, there was no lag at all for any of us. This was in various Devil's Gambit quests on Epic Elite. Contrast this with last week where, in these same quests, with the same small party sizes, we would freeze for several seconds at a time, frequently, on every single mob spawn and frequently on monster death, and my guildmate in Nevada would be near death before he even saw the monster spawn and appear on him. I'd lag solo. I'd have crippling lag solo *on the guild ship*. None of us experienced any of this tonight.

So, although some people are still having bad lag, it's *different* than before, and for at least for 4 people in my guild, lag was much improved over last week. My lag was minimal in a 12-man raid, and non-existent elsewhere, which is a gigantic improvement for me.

ThePrincipal
03-22-2016, 01:11 AM
This is my first complaint post regarding lag. I played today on Agro, about 10pm PST. I was in Tears on Hard, with just me and a hire. Lag has never been so bad. For example, it took 10 tries to open a chest (the animation would complete but then it would rubberband me and say the chest was out of range). About 70% in, lag monster finally caught up with me and I died. Logged out.

bartharok
03-22-2016, 01:40 AM
My lag went from minor to none. But i have no idea if it helped those with more lag any.

LightBear
03-22-2016, 02:40 AM
Ghallanda is now open to players! Just as with Argonnessen and Thelanis we'd love any before/after performance reports encountered on these two worlds that players are able to provide.

As to what type of things that means we are looking for:

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

Was in the 10th anniversary quest yesterday and these are my findings:
- Lagg was still there, in marketplace, house p and inside that quest.
- Mobs kept agro unusually long and kept standing in corners and shooting through walls or stared blankly past you.

Manganakus
03-22-2016, 03:45 AM
Heads up: We will be restarting Ghallanda at 6:30 PM Eastern (-5 GMT) for maintenance and to put in place the same monitoring we have on Argo and Thelanis at the moment. We've been getting a moderate number of lag reports from Ghallanda specifically today, so we want to take the world down for 60-90 minutes to make some adjustments and see what kind of impact it has.

Thanks Cordovan.

Jiirix
03-22-2016, 03:48 AM
Did the legendary raids yesterday and it was far better than the days before lag wise. We had a few seconds of collective freeze two times in Tempest Spine but the rest was OK. Lag seemed back to normal for me but I had hoped to see an improvement after the server move.

sudzz
03-22-2016, 04:52 AM
General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

I am seeing lag significantly more often than previously, every single raid I have completed or been lag wiped in for the last 2 weeks has experienced many bouts of frozen to the spot unable to do anything while watching mobs roam about killing people, this was very rare previously and could go from month to month without a lag wipe in raid.

When lag is encountered it varies from a second or so ie stuck for a brief moment but then catches up with commands and all good to long enough for mobs to close ground and beat toons to death resulting in suddenly going from alive to dead with mob stomping away.

When lag is experienced it does seem that all party mebers are experiencing it but that also guild chat frequently comes alive with lag comments even from folk not in parties or even in hostile territory, I have experienced lag while standing/moving about our guild ship as a prime example.

I have noticed the occasional mob who will leap off a ledge into water and then re-spawn, mainly in Meridia explorer area in the Gnoll area.

The only reason I can come up for you guys not seeing in-game lag yourselves is that you are connecting locally whereas most of us are having to come in through a gateway to your server and it is here that the likely blockage is, maybe the connection to the outside world is not large enough to accommodate the number of connections bear in mind many of us are paying cash for the time to play the game it would therefor fall to we are not getting the service we are paying for or perhaps your ISP is not supplying what you are paying them for?

Eternal_Newbie
03-22-2016, 05:09 AM
General performance.

- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Those types of things are useful. Thank you in advance for any insight you can provide.

Hello,

After testing the change I'd say that Lag is really mitigated, but root cause seems till hidden somewhere. TY anyway for this mitigation.

Now anser to your questions:

1) Are you seeing lag as often as you were before?:

Yes more lag than before Data Center move even if it is really better since this night settings.

2) When you do encounter lag, how long does it last?

Lag duration is variable, could be very long (like 5 mn where noone could move in instance) before the setting modification. Now it is like 5 to 10 sec lag spikes.
Interresting point is that each time a script is triggered to let DM talk (with yellow text) or each time option XP are granted, the spike (at least 3 sec lagging if not freeze) is systematic for all the players, at least in full groups.
This is still very noticeable after the new settings.
Seemseasy to reproduce, pretty sure that if you fix that it will show you root cause and let you save DDO worlds from this lag monster (before Datacenter move we had this kind of systematic lag only in rare situation, like at the end of shroud part 1, when going to the shrine).

3) When you do encounter lag, is it affecting everyone in your party?

Yes, all are affected (systematic). For what I see seems not client <-> server network lag.
Look much more like saturation of instance wide ressources (ie: ram --> swap, threads or file system saturation etc...) or issue with server to server services interaction (db call etc...).

4) Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

Didnt checked since the mitigation but just after DC move was nearly not possible to finish accute delirium quest cause nealy all beds & shelves pathed on the wall and were blocked.
Will test it again if can find time.

Really happy to see real investigations on that huge problem that ruined completly DDO, the worst experience, by far since i play the game.

Hope you can find and kill the root cause.

Regards,

Deadlock
03-22-2016, 05:11 AM
Heads up: We will be restarting Ghallanda at 6:30 PM Eastern (-5 GMT) for maintenance and to put in place the same monitoring we have on Argo and Thelanis at the moment. We've been getting a moderate number of lag reports from Ghallanda specifically today, so we want to take the world down for 60-90 minutes to make some adjustments and see what kind of impact it has.

Did I miss a message on the loader to say this was happening last night?

Or did I miss a world announcement that the worlds were coming down?

The time we all froze up coincides with this. Tell me that they didn't just take down Ghallanda without some ingame announcement.

lebuffalo
03-22-2016, 05:57 AM
Did I miss a message on the loader to say this was happening last night?

Or did I miss a world announcement that the worlds were coming down?

The time we all froze up coincides with this. Tell me that they didn't just take down Ghallanda without some ingame announcement.

Was playing before the down. We had an announcement 45mn before. And others one each 10mn as usual.

Gonna log today to see if it changed something

Giedon
03-22-2016, 07:33 AM
Did I miss a message on the loader to say this was happening last night?

Or did I miss a world announcement that the worlds were coming down?

The time we all froze up coincides with this. Tell me that they didn't just take down Ghallanda without some ingame announcement.

Yes there was a broadcast every 5 minutes for about 30 minutes that the server was coming down.

Basura_Grande
03-22-2016, 08:46 AM
Whatever was going on for about the first 30 minutes of Ghallanda being rebooted was fine. Completely lag-free LH TS with a full group.

Reports from other people on TS and guild reflected the same.

Then all the lag came back, pretty much hitting everyone at the same time according to reports on the channels.

Don't know if that helps at all.

Franghasea
03-22-2016, 09:55 AM
First, thank you for the maintenance. It has greatly improved my playing experience on Argo. It is not quite as smooth as pre-data center move, but it is close.

- Are you seeing lag as often as you were before?
-----Not nearly as often. While anecdotal, during my two 3+ gaming times since the maintenance, the issues while still seldom, seem to occur more frequently the longer I stay logged in (without alting at any time).

- When you do encounter lag, how long does it last?
-----It seems to be isolated 1-3 sec lag spikes; very sporadically.On the rare times it occurred, they varied. Once was running through the Epic Gianthold Slayer area, once was while fighting the Orc round in an EH Breaking the Ranks, and another time was while attempting to open the quest menu to enter Lines Of Supply.

- When you do encounter lag, is it affecting everyone in your party?
-----Yes.

- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?
-----Two previous quests of contention seem to be corrected. First, In Old Tomb, New Tenants (Epic 3BC), Kyrnxala, the red name that you fight just before going underwater to the end fight, no longer dives off the bridge repetitively. Likewise in Murder by Night, the 8 villagers that you have to escort to the central room no longer get caught on corners.

Thanx!
Frang

DagazUlf
03-22-2016, 10:03 AM
- Are you seeing lag as often as you were before?
- When you do encounter lag, how long does it last?
- When you do encounter lag, is it affecting everyone in your party?
- Do enemies still appear to be pathing strangely (specifically: walking off of ledges)?

On Thelanis last night:
- Much less lag
- Quick little bouts of lag now
- Only 3 and 4 in party, but all were affected.
- We had a few instances of enemies attacking each other, which might have been pathing-related? I dunno. But we'd come upon a group and they would already be fighting amongst themselves. This was both in quest instances and wilderness areas.

Note: most of the lag encountered revolved around dialogs; either in-quest NPCs, end rewards NPCs, or during quest granting.

EazyWeazy
03-22-2016, 10:24 AM
Both Argonnessen and Thelanis are now open to players!

We'd love any before/after performance reports encountered on these two worlds that players are able to provide.

I would say it was much improved. 60% better. I was still getting "lag" where I couldn't interact with NPCs, or get quest windows to open up, but my character wasn't frozen up while this was going on which is an improvement. Kill more Lagitaurs!!!!

We also had mobs attacking each other in "The End of the Road" out in Epic High Road.

Cordovan
03-22-2016, 10:51 AM
Closing this thread to keep the conversation organized. There's a new thread where we are restarting the worlds that didn't get this improvement made to them yesterday.