Page 2 of 6 FirstFirst 123456 LastLast
Results 21 to 40 of 107
  1. #21
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by KoobTheProud View Post
    It's a 15 year old game that likely has been in development for close to 20.

    It's a miracle that the thing runs at all given all the changes it has had to accommodate over the years.
    It's not a miracle. I've had systems run thirty years. I've had them run thirty years successfully because they went through parallelization change when mgmt finally got serious to task someone to finally figure out you couldn't shove any more stuff in the lone pipe.

    If they truly wanted to make a bet on another 10 years, they figure out how to multiply the pipes because the gain from that if done right will be to save the current customer experience and eliminate the need to rework years of updates that wired effects into so much of the game in addition to hopefully giving them back some capacity on the top end.

    Or we can look forward to rework that if it is coming, like someone astutely said, is absolutely going to be felt.
    Last edited by myliftkk_v2; 04-14-2021 at 11:47 PM.

  2. #22
    Community Member
    Join Date
    Sep 2009
    Posts
    2,209

    Default

    I don't think the DDO software uses more than one core. I could be wrong on this but I think I read somewhere that when DDO was in development the producers bet on single core processing continuing to be the standard. Instead we got multi-cores as the standard but the engine was already built for just one.

    This would jibe with other games in development in 2000-2002. Everybody knew multi-core processing was going to be a thing but who could afford to devote development time to them when none of the processors in play at the moment, the processors most of your customers would be using at launch, would get anything out of dev time devoted to the extra cores?
    Last edited by KoobTheProud; 04-15-2021 at 12:03 AM.

  3. #23
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by KoobTheProud View Post
    I don't think the DDO software uses more than one core. I could be wrong on this but I think I read somewhere that when DDO was in development the producers bet on single core processing continuing to be the standard. Instead we got multi-cores as the standard but the engine was already built for just one.

    This would jibe with other games in development in 2000-2002. Everybody knew multi-core processing was going to be a thing but who could afford to devote development time to them when none of the processors in play at the moment, the processors most of your customers would be using at launch, would get anything out of dev time devoted to the extra cores?
    Multicore at the consumer level is not the same thing as scalable architecture (software and hardware) at the server level. Not relevant for a server side problem like was described. It is also true that conversions from a kind of single threaded system into parallel processing systems has/is done actually quite frequently. Especially since the patterns for doing it now are pretty commonly known and understood in development circles.

    It's also somewhat irrelevant that the engine even "knows" the number of cores. There's clearly internal messaging infrastructure which is how one has a "effects queue" that is shared across instances. Whether that "queue" exists as a single solitary logical unit on a single core, or is separated by a messaging bus that traffics messages with many "queues" each on there own hardware is far enough away from the hardware that the physical hardware underneath is rather unimportant. Software can be "single" core and still scale wide, as that's what messaging allows for.

    Why a decision was made nearly 20 years ago doesn't really carry a whole lot of weight to making the more correct decision now. Trying to cater to that decision though, that's going to turn out far less optimal for the consumer than dealing with the reality that the industry long ago made a decision in a different direction and created countless examples how to do it.

  4. #24
    Community Member Mercureal's Avatar
    Join Date
    Sep 2009
    Location
    Somewhere that's far too cold right now.
    Posts
    1,365

    Default

    Quote Originally Posted by myliftkk_v2 View Post
    Sure, it is "tougher" theoretically to solve a serialized process problem by converting it to scaling parallel processing with resource guardrails. But, the solution being proposed, actually isn't a good one.

    It doesn't fix the resource theft that hammers instances entirely unrelated to queue stuffing. That gaping hole still exists, ready for every subsequent update to drive right into it and viola, we are back at square one.

    What they're proposing is re-jiggering the entire game at the consumer level, as it's been played for years, to fix a problem that technically speaking, in software terms, probably shouldn't even exist if the development was done with an eye toward parallelization.

    Maybe they'll re-jigger it without blowing the game up. See bridge I have for sale.
    Assuming I correctly understand what you're saying, I agree. My guess has to do with their decision-making environment.

    What you're talking about sounds like it would require money, either for equipment, expertise, or both, money that is almost certainly not included in their current approved budgets. It's unlikely that the parent company allows SSG to budget discretionary funds, or to retain much of a reserve; any new expenditure would mean going to whoever in the structure oversees their division and saying something like:

    • "Uh, we have some costs that weren't included in the reports you looked at when you were doing the pre-acquisition diligence review last year, but they're really important."
    • "No, I'm not sure why those costs weren't included."
    • "Could we have some more money?"


    And that's probably not a conversation anyone calling the shots at SSG wants to have.

  5. #25
    Community Member Oxarhamar's Avatar
    Join Date
    Sep 2009
    Posts
    6,491

    Default

    Quote Originally Posted by Mercureal View Post
    Assuming I correctly understand what you're saying, I agree. My guess has to do with their decision-making environment.

    What you're talking about sounds like it would require money, either for equipment, expertise, or both, money that is almost certainly not included in their current approved budgets. It's unlikely that the parent company allows SSG to budget discretionary funds, or to retain much of a reserve; any new expenditure would mean going to whoever in the structure oversees their division and saying something like:

    • "Uh, we have some costs that weren't included in the reports you looked at when you were doing the pre-acquisition diligence review last year, but they're really important."
    • "No, I'm not sure why those costs weren't included."
    • "Could we have some more money?"


    And that's probably not a conversation anyone calling the shots at SSG wants to have.
    Except that the devs have indicated that they just fired outside team to help so that pretty much negates that theory

  6. #26
    Founder & Super Hero Arkat's Avatar
    Join Date
    Feb 2006
    Location
    Wyoming
    Posts
    9,530

    Default

    "fewer" effects.
    "A wise person chooses the right road; a fool takes the wrong one." - Author unknown
    *12609*
    Quote Originally Posted by Lynnabel View Post
    Hi Welcome

  7. #27
    Community Member Mercureal's Avatar
    Join Date
    Sep 2009
    Location
    Somewhere that's far too cold right now.
    Posts
    1,365

    Default

    Quote Originally Posted by Oxarhamar View Post
    Except that the devs have indicated that they just fired outside team to help so that pretty much negates that theory
    Not really.

    Yes, I'm just speculating, but hiring external consultants to work on a fix is different than overhauling a system. Use of consultants is common in IT, and they very likely already have an annual amount in their budget they're allowed for consultants that they could be using for this outside team.

    There's a lot of financial control over consultant work - you have a fixed amount allocated for it in advance, and you get an estimate from the consultants of what they expect to bill for any specific task. You can say "no" or say, "reduce the scope" if the estimate is over the limit. What the OP was saying sounds like a project that would be a lot bigger than a simple consulting engagement.

    Of course, they may have already requested the funding for a big overhaul, already had it approved, and already be putting a workplan together - but just not talking about it now because it's a longer-term fix. Possible, yes. Likely?

  8. #28
    Community Member Oxarhamar's Avatar
    Join Date
    Sep 2009
    Posts
    6,491

    Default

    Quote Originally Posted by Mercureal View Post
    Not really.

    Yes, I'm just speculating, but hiring external consultants to work on a fix is different than overhauling a system. Use of consultants is common in IT, and they very likely already have an annual amount in their budget they're allowed for consultants that they could be using for this outside team.

    There's a lot of financial control over consultant work - you have a fixed amount allocated for it in advance, and you get an estimate from the consultants of what they expect to bill for any specific task. You can say "no" or say, "reduce the scope" if the estimate is over the limit. What the OP was saying sounds like a project that would be a lot bigger than a simple consulting engagement.

    Of course, they may have already requested the funding for a big overhaul, already had it approved, and already be putting a workplan together - but just not talking about it now because it's a longer-term fix. Possible, yes. Likely?
    Different sure but it’s evident that there was s work being done and funds being allocated to this new funds or existing

  9. #29
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by Mercureal View Post
    Not really.

    Yes, I'm just speculating, but hiring external consultants to work on a fix is different than overhauling a system. Use of consultants is common in IT, and they very likely already have an annual amount in their budget they're allowed for consultants that they could be using for this outside team.

    There's a lot of financial control over consultant work - you have a fixed amount allocated for it in advance, and you get an estimate from the consultants of what they expect to bill for any specific task. You can say "no" or say, "reduce the scope" if the estimate is over the limit. What the OP was saying sounds like a project that would be a lot bigger than a simple consulting engagement.

    Of course, they may have already requested the funding for a big overhaul, already had it approved, and already be putting a workplan together - but just not talking about it now because it's a longer-term fix. Possible, yes. Likely?
    I think rebalancing the very many parts of game, where effects are no small part, is a even more massive and riskier undertaking than decoupling an internal single threaded system and converting it into a modern messaging bus architecture and where there is near zero reason to impact the consumer.

    It is, yes, more challenging, if someone has never done this kind of work. It's also challenging if inhouse devs are basically not keeping up with what's happening in modern development circles.

    There are methods to solve technical problems that force the problem impact onto the consumer, and there are methods that force the challenge onto the developers to look out for the consumer. Yes, the slowdown was affecting all consumers, but having them consuming less, or having less consumers, isn't really a forward-looking solution. It's basically a band aid over a wound that's not going to heal.

  10. #30
    Community Member Telekinesis's Avatar
    Join Date
    Nov 2020
    Posts
    49

    Default

    Quote Originally Posted by myliftkk_v2 View Post
    You're thinking of scaling in game terms not software terms. I am speaking of something very different.

    Let me break this down in an area where I've done this for years, financial transaction processing.

    Let's say I have 100 cashier terminals doing electronic transactions. I can have one piece of software in the middle that handle the details of every transaction from every terminal. Ok great, but what happens if one terminal has a black friday situation all the time? Well, if I serialize all the processing through my one "transaction processor" then guess what, if that terminal dumps in tons more transactions than the physical hardware can support in a serialized process, everyone's terminals back up. This is the "effects queue" problem distilled to the simplest example.

    How do you solve this? Easy SSG says the company stops selling merch at all terminals and that means the problem terminal does fewer transactions and all terminal go back to normal throughput and everyone is happy. Ummm, obviously not, and this is readily apparent to most businesses.

    What should happen is there should not be an "effects processor" singular. There should be a flexible bank of "effects processors" that sit around and numerically scale wide across hardware as needed, that allow for terminals to grab them as they need them. Even in a situation where a single terminal outstrips the capacity of the effects processor assigned to it (let's say only one can be assigned to a terminal), because of the parallelization, the other terminals have no idea because they still run at peak speed. And of course the customers relying on all other terminals are happy.

    There are in reality, no hard physical reasons software systems cannot be parallelized, all the way down to the database level. When you encounter throughput issues, the answer is not to ask the customer to put less work in the pipe, but to figure out how to multiply the pipes that do the work.

    Or, you could just look inside your machine and see the multiple cores on your silicon.
    My guess is alot people on the forums and even ssg have considered these things. It would most likely be a massive undertaking to rewrite and separate code and functions. If you owned the company what would you do? Spend a few years rewriting the entire data bases to reduce the load. Or spend 1 month to combine combat logs. Cheapest quick fix always.
    Look at history, they can't even change the simplest of things with out it bugging the game with something to us or them seems to have anything at all to do with the change. Look at the bigger picture, they are servicing the old tractor till it kicks the bucket.

  11. #31
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by Telekinesis View Post
    My guess is alot people on the forums and even ssg have considered these things. It would most likely be a massive undertaking to rewrite and separate code and functions. If you owned the company what would you do? Spend a few years rewriting the entire data bases to reduce the load. Or spend 1 month to combine combat logs. Cheapest quick fix always.
    Look at history, they can't even change the simplest of things with out it bugging the game with something to us or them seems to have anything at all to do with the change. Look at the bigger picture, they are servicing the old tractor till it kicks the bucket.
    You would think, but that's not how the software industry works (first off, there are not "decaying parts"). Shortcuts that don't keep up with development trends happen all the time. Largely because the gap between the people who are top tier, write O'Reilly books, and say the average game developer approaches a Jacque Cousteau level crevasse. It's akin to the difference between the 4th quarter scrub and James. Technically they are both professionals, but one can solve almost any problem elegantly and the other just punches the clock for some cash. But, there are people out there who can do the work, do it fast, and do it elegantly.

    The people who "justify" these kinds of decisions are the people who understand the least about how software works top to bottom. Mostly they run off misconceptions or experience as poor developers themselves. That or they get snowed by poor devs as to what's possible and what's not. The "floats to the top" metaphor in software is as true as anywhere else.

    The number one reason consumers don't expect better from software is because because they think its a magic box and make bad metaphors about it. It's not magic, and the machine does exactly what the developer tells it to do. If it doesn't work right, that's all on the devs (hardware specs have really outstripped most prior physical constraints).

  12. #32
    Community Member redoubt's Avatar
    Join Date
    Mar 2006
    Posts
    6,780

    Default

    Quote Originally Posted by droid327 View Post
    It's more like if NASCAR kept canceling races because the cars all kept blowing their head gaskets mid race, and they figured out the cars were going faster than a head gasket could take, so they slowed down to what the engines could actually do, so they could finish races.

    Effect throttling is not an arbitrary limit being imposed just as a band aid. It's literally what the problem is...regardless of whether or not you liked it, previous proc rates were not something the game could actually support.
    Did we not try this back when TWF nerfed all those years ago... and we found out it did not fix the problem...
    Should a reaper see me? I think Death itself should have to make a spot check when I'm rolling up behind him. -- Krimsonrane

  13. #33
    Community Member
    Join Date
    May 2010
    Location
    Near Buffalo, NY
    Posts
    729

    Default

    I dont nerfs, but I can abide by a nerf that helps eliminate the lag. The main problem here is with the dated game engine and I would imagine that a total rewrite of the game is pretty much out of the question as it would cost many, many $$$. The "proof will be in the pudding" as to how effective these changes will be in reducing lag. Another question is: will these changes also help eliminate the porting issues? To me, just standing there waiting to port for 3-5 minutes then having to relog to do it is more annoying than the lag. I did a ping test recently while playing the game and the test showed intermittent instances of pinging DDO's server and not getting a response. I assume this is to do with server lag?. I can play another game (ESO) without this happening so I assume its just a DDO issue.

  14. #34
    Community Member Chai's Avatar
    Join Date
    Aug 2008
    Location
    Wisconsin, birthplace of D&D
    Posts
    27,789

    Default

    Quote Originally Posted by droid327 View Post
    It's more like if NASCAR kept canceling races because the cars all kept blowing their head gaskets mid race, and they figured out the cars were going faster than a head gasket could take, so they slowed down to what the engines could actually do, so they could finish races.

    Effect throttling is not an arbitrary limit being imposed just as a band aid. It's literally what the problem is...regardless of whether or not you liked it, previous proc rates were not something the game could actually support.
    Yet games like EQ in older tech eras supported them just fine.

    The idea that this is a problem with no solution other than proposed makes those of us with experience on these issues lol.

    I'd be fine to be proven wrong on this if/when they lower the procs and lag disappears, but we all know what kind of dream that is. We will see the same explanation provided as before: We eliminated one source of lag, but since others remain, there appears to be no change/minimal change in the symptoms.
    Quote Originally Posted by Teh_Troll View Post
    We are no more d000m'd then we were a week ago. Note - This was posted in 10/2013 (when concurrency was ~4x what it is today)

  15. #35
    Community Member Chai's Avatar
    Join Date
    Aug 2008
    Location
    Wisconsin, birthplace of D&D
    Posts
    27,789

    Default

    Quote Originally Posted by redoubt View Post
    Did we not try this back when TWF nerfed all those years ago... and we found out it did not fix the problem...
    In that era, many of the physics checks were changed to procs, and the explanation was that procs needed less overhead than physics checks on every swing/shot did.

    Now in this era, procs are the issue?
    Quote Originally Posted by Teh_Troll View Post
    We are no more d000m'd then we were a week ago. Note - This was posted in 10/2013 (when concurrency was ~4x what it is today)

  16. #36
    Community Member Paladin_of_Power's Avatar
    Join Date
    May 2016
    Posts
    266

    Default

    Quote Originally Posted by Mercureal View Post

    • "Uh, we have some costs that weren't included in the reports you looked at when you were doing the pre-acquisition diligence review last year, but they're really important."
    • "No, I'm not sure why those costs weren't included."
    • "Could we have some more money?"


    And that's probably not a conversation anyone calling the shots at SSG wants to have.
    Legacy Technical Debt that has snowballed into a real big problem is always a fun conversation. Hope is not a strategy.

  17. #37
    Community Member Mercureal's Avatar
    Join Date
    Sep 2009
    Location
    Somewhere that's far too cold right now.
    Posts
    1,365

    Default

    Quote Originally Posted by Oxarhamar View Post
    Different sure but it’s evident that there was s work being done and funds being allocated to this new funds or existing
    Yes, anyone paying attention knows they've acknowledged it and have said they're working on it. The only point I'm making is about resources. SSG probably has a specific amount of resources that they're allowed to spend on IT stuff, which would be what they're using now. IF those resources are insufficient to fix the problem, then SSG would have to go to it's parent company and ask for additional funds, or instead implement a partial or short-term fix.

    What I am saying is that SSG's executive leadership may be (in fact, is likely) reluctant to make that request for additional, unplanned-for resources. Especially so, given the short time that has passed since the new parent company took over. This could explain why they are not talking about the kind of fix that myliftkk_v2 suggests.

    We're not disagreeing, just talking about two different elements of the same issue.

  18. #38
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by Chai View Post
    In that era, many of the physics checks were changed to procs, and the explanation was that procs needed less overhead than physics checks on every swing/shot did.

    Now in this era, procs are the issue?
    I'm sure last time it saved the "physics queue"...

    Now the answer is to simply game less... fine SSG, imma leave my DDO icon un-clicked for a while longer. Y'all let me know when you can handle more traffic than a Popeye's lunch rush now once you've chopped the menu down to like three items.

    Trinity here we come.

  19. #39
    Community Member
    Join Date
    Feb 2021
    Posts
    111

    Default

    Quote Originally Posted by Oxarhamar View Post
    Except that the devs have indicated that they just fired outside team to help so that pretty much negates that theory
    Actually, the folks they just hired were in fact fired from Turbine in 2015, not all that long before SSG was formed out of a tumor taken out of Turbine. They literally took a brand new MMO with huge name recognition set in the DC Universe, and managed to kill it outright in less than 6 months.

    So, hiring them as outside advisors is actually pretty hilarious, and so on-brand for SSG.

  20. #40
    Community Member
    Join Date
    May 2013
    Posts
    1,861

    Default

    Quote Originally Posted by Mercureal View Post
    What I am saying is that SSG's executive leadership may be (in fact, is likely) reluctant to make that request for additional, unplanned-for resources. Especially so, given the short time that has passed since the new parent company took over. This could explain why they are not talking about the kind of fix that myliftkk_v2 suggests.
    I think this is being too generous by eleven donuts in a dozen.

    It assumes SSG knows both the cost and scope to the optimal choice and is instead deciding on a suboptimal choice which they also know the cost and scope to because of external factors a, b, c, etc.

    I don't believe that, nor do I believe there is any evidence for that. They are flying by seat of pants here. They just "discovered" (or opened their eyes, po-tay-to po-tah-to) the "effects queue" is a single threaded chokepoint. Properly analyzing the parallelization of that isn't some 1 week adventure.

    They dropped the changes into Lam with zero announced plans on how they were going to adjust procs to compensate (thus see all of the back and forth on that), so they hadn't bothered to scope that downstream halo or collateral damage either. They basically just chopped at effects with an axe with any considering on the net. Now they are on the fly trying to balance. I would have never dropped a major change like chopping procs off at the knees without at the very same time rolling out a plan of how the major effects will compensate. Unless of course, I didn't have a plan.

    If this is not evidence they are riding the lag tiger unleashed through the zoo, not sure what other evidence could be stronger. The players are going to ride the lighting on this. It won't be pleasant, especially for those of us with effects based builds who, sorry to say, ain't got time to do SSG's QA for them. Chances are, in the flurry of fat fingers to deal with complaints, it will also create heretofore opportunities for great power exploits as they again, test nearly nothing about the end result.

    Or, we know, they could spend the time to do it right (and some of that sweet, sweet $3M+/yr profit).
    Last edited by myliftkk_v2; 04-16-2021 at 02:56 PM.

Page 2 of 6 FirstFirst 123456 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

This form's session has expired. You need to reload the page.

Reload