Reply To: Star Citizen – Musings

Forums › Main › Star Citizen – Musings › Reply To: Star Citizen – Musings

May 24, 2017 at 9:34 am #5263
dsmart
Keymaster

    STAR CITIZEN MULTIPLAYER INSTANCING

    In my Irreconcilable Differences blog, I wrote extensively about the instancing issue and how they stand very little chance of ever getting past the broken underlying architecture that they currently have in StarEngine. In a Feb 2017 interview, Erin Roberts made the following comment:

    “So with the next big release a lot of the underlying game is there and then we can look at transferring people between servers so we can have hundreds of thousands of people maybe in one instance, but that doesn’t come online until later.“

    Yesterday, a new post, How many people can be in an instance?, popped up on Spectrum (Reddit thread also) in which one of the devs has made several statements which lead me to believe that :

    1. They have no clue what they’re doing.
    2. When they do get a clue,  it would be revealed to them that they have to gut their entire networking layer to implement what they are aiming for.
    3. They’re fucked. Completely.

    QUESTION:

    “so we were reading that this dynamic local instancing will try it’s best to put you in the same instance as friends and people/ things of interest. so if you were a pirate, and were following your prey.. are you guaranteed to jump into the same instance or is there a chance it’s all in vein and you lose them? based on yall’s dynamic local instancing system?“

    ANSWER:

    “In a single server instance we can currently have up to 40 players in Area18 or 24 players in Crusader(1). Matchmaking tries to put you in the same instance as your friends, but beyond that it is luck of the draw which instance you will end up in(2). However @H0wland is correct in that our goal is that eventually everyone will be in the same instance(3).
    There quite a few engineering hurdles we need to overcome before this can happen. Server performance needs to improve a lot, so there are several tasks to address this that are either currently underway or in the schedule(4). This will only get us so far though, and won’t be enough to fill a solar system with players and NPCs. To go further we are going to have to connect multiple servers together in something we’re calling a “server mesh.” Each server will take on the processing load for a region of space, and these regions will adjust their boundaries to best balance that load with their neighbors. You will be able to see (and fire) across the boundary from one server to another, and, as you fly through space, will move seamlessly from one server to another(5). We will also be able to dynamically add and remove servers to suit the current level of demand. This technology will allow us to scale almost without limit while keeping everyone in the same instance(6).
    The problem we still need to figure out is how to handle everyone heading to the same place at the same time. I’m not sure there’s an engineering solution to that one, so it may require some game mechanic to prevent too many players congregating in the same place(7).
    TL;DR – yes, once all the pieces are in place and the kinks have been worked out, you’ll be able to stalk your prey, and should always be in the same instance.”

    Let me break it all down:

    1. I know for a fact, as do most backers who are actually playing this right now, that the server can’t handle more than 8 clients within the same locale without falling over. Let alone anywhere near 24 clients in Crusader (introduced in 2.0 released in Q4/2015), which is the core of the Star Citizen that started the “Persistent Universe”. Area 18, a glorified shopping center, can handle more players because, well, there’s nothing to do there except move around, look at, buy stuff etc.
    2. This is a glaring Red flag. There are lots of games, even those built with SteamWorks, that allow some form of grouping agnostic matchmaking, even for instanced games. For six years, since they started using cloud servers, they didn’t think that implementing the ability for clients to group, then all launch in the same instance, was a priority. Elite Dangerous, which also uses instancing, had this same issue during alpha and beta cycles. They address it with features such as Wing Beacons, nav-lock, private grouping etc. In fact, read this Elite: Dangerous’ 3,000-player battle royale article.
    3. This one is a head-scratcher.  I hope that his use of “everyone” means those wanting to group with their friends in the same instance. If that’s not the case, then we’re back to the “they have no fucking clue” part, because there is no way they can get “everyone” in the same “instance”.
    4. Whatever that schedule is, it’s not public. The current schedule which goes all the way to 3.2, has no mention of anything related to any of what he wrote. In fact, the entire schedule page has 12 instances related to network implementation and/or revision; and none of those entries mentions anything like that. Not. One. Thing.
    5. This is all wishful thinking. If six years into the development of an MMO, you don’t have this stuff already completed or in progress, chances are it’s either never going to get done in the short-term (delays cost money, and when money runs out, the project is dead), or there was never an intention to actually do it. Make no mistake, everything he said there, are things that both Chris and Erin have said in the past.The reality is that it is simply not possible with their current networking framework which was built around CryEngine 3.7. And LumberYard (based on CryEngine 3.8), isn’t going to give them that because it too does not have support for any of that. They would have to build it themselves. Just like how Frontier did it for Elite Dangerous, and how we did it (FYI we don’t use instancing; so our server-to-server hops are live) for Line Of Defense.For one thing, they have touted this whole “seamless” 64-Bit space, which is one large “scene”. For them to do any sort of population control, they would have to split it up into boundaries. And each of those would then need to have a set of criteria that determines how many clients are allowed in there. And that involves a significant amount of work involving proxy server connections, data aggregation & collection, etc. The way we did pop-loc in LoD, is similar to how Planetside did it. You set a limit on the number of clients in a scene, then don’t allow any further connections until someone dies, drops off etc. And this is possible when you have low-level control – right from the start – of the scene management structure. In our Wide Span Global architecture we built this from the start so that each space or planet “scene” is controlled by a server connection. And that server is the arbiter that controls how clients can enter via jump anomalies (Dynamic Jump Pad, Jump Gate, high altitude insertion from space). If you try to enter a scene (e.g. Heatwave planetary base from Lyrius space) that has reached it’s server configured client limit, you’re stuck in Lyrius, and will have to keep trying. The messaging is all done from the connection interface for the jump anomaly which talks directly to the server. And this was all done right from the start and before we even had complete dynamics for fps, space craft, and vehicles in the game.And if they do manage to actually build all of that, they have a different problem. Players can EVA. So if they allow, say 64 clients per instance, guess what happens when you have 64 ships and 64 players in EVA. And that’s just assuming 1 player crew per ship. Imagine the hilarity if you have passengers, and cargo. And ships are shooting, EVA players are shooting. LMAO!! I can’t even.Let’s not even get into the whole issue with localized physics grids, which allow players to move around inside their space chariots in fps mode. That’s got it’s own performance and networking issues which are currently part of the problem they are faced with.
    6. Yeah, this is the part where any developer would start laughing. Basically, “scaling” implies “limits”. And when it comes to networking architecture, there is no such thing as “without limits”.
    7. And therein lies the rub that negates everything he said previously. Note the use of the phrases “will allow”, “still need to”, “how to”, “not sure” etc.If you have your pol-loc sorted out, there is no requirement to figure out how to handle “everyone heading to the same place at the same time”. The fact of the matter is that if you allow 64 clients per instance/shard, you should be prepared for the inevitable scenario that all of them are likely to end up in the same place at some point. Elite Dangerous planned for this, right off the bat. Which is why sessions in which over 900 players journeyed to Sagittarius A, was possible.Saying “I’m not sure there’s an engineering solution” simply means that, as I said, they really have no clue what they’re doing with this game. There is absolutely no way to prevent all server allowed players from being at the same area at the same time. Which is why, even though they claim that Crusader can support 24 players theoretically, all it takes is for more than 8 players to be in the same local area for the server to croak and it all becomes unplayable.

    This game was never supposed to be an MMO. And it wasn’t pitched as one. And Chris has gone on the record several times, even after all the stretch goals funding were met back in Nov 2014 at $65 million, saying that it wasn’t. And the stretch goals have no such indication or implication that they were building an MMO. Somewhere along the line, because of scope creep and promises made as they pulled every trick in the book to keep raising money from gullible backers, it morphed into an MMO because that’s the only game model that would support some of the things they were promising. And they’re doing all this despite the fact that they neither have the tech, nor the talent, or the time and money to pull it off.

    At this point, as I’ve shown above, if they don’t have the framework for their future networking model already in and working in some fashion, there is absolutely no way they’re going to have time to gut what they have now, and implement a proper solution. Something they should have done from the very start. Now it’s too late. And they are still making promises they can’t keep, even as they continue to defer* promised features into a post-release schedule.

    My guess is that the current networking layer is going to remain as-is for quite some time, as they continue to build other features and systems on top of it. Then if by some miracle they survive (they won’t) long enough to actually get around to it, all that stuff they are building on top of a network layer they have to replace, will either have to be ripped out, or modified to support whatever it is they need to do in order to support their long term goals.

    All of this means that even if they are around long enough for a 4.0 schedule to go live, and it does include the major networking features they need to make what they plan work, until then, backers are still going to be stuck with 8 player clients in Star Citizen. I can’t wait to see what happens when 3.0 goes live with the two moons. It’s going to be hilarious. Maybe they’ll shock everyone and have 6 clients running in Crusader without problems.

    Anyone who still has hopes that this project is ever going to be completed, let alone as promised, is delusional. Meanwhile, it’s Sandi’s birthday, and apparently they’re in Monaco again this year. Paid for with backer money of course.

    *  Modding is out. Private servers are out. And a litany of other things are either not in progress, or have been deferred. The latest being the docking collar support for ships (e.g. Cutlass) that have that feature, and which were sold with it during the 2012 Kickstarter campaign if you pledged $110 or more. So instead of having two modes of docking, one which was a big draw for backers who bought the ships that were designed to support it, they will now have only one, whereby you have to EVA in order to board another ship. So if you’re looking to fulfill your dreams of boarding another ship, weapons armed, like in The Expanse, Interstellar or similar movies, ain’t gonna happen. Like ever.

    UPDATE: Shortly after this article went live, some backers were trying to say that “building an MMO” out of Star Citizen, was the $3m stretch goal because it says:

    “Citizens with appropriate packages will receive access to the Star Citizen universe with 40 star systems for persistent online play upon release.”

    That’s the single most ridiculous thing I have ever read about this issue. People listen: “persistent online play” does not, and never did, imply that the game will be an MMO. Heck, even CIG themselves proved this point when they released Star Citizen 2.0 in Q4/15 and called it “Persistent Universe”, when in fact, nothing about the game is persistent, other than player stats stored and retrieved from a database. By this definition, they are implying that games with leaderboards, stats saving, are all MMO games because they have persistent stats save/restore features. Which would make every Call Of Duty or Battlefield game an MMO. The Star Citizen universe isn’t persistent. It’s an instance. When the instance closes, everything shuts down. I wrote about this extensively in my Star Citizen – Condition Red blog from May 2016.