Greetings former and future denizens of Agon.

With last week’s turmoil, we realized that not many people were familiar with reverse engineering  and what it entails.
Our quick statement to reassure people may have even blurred the truth for some instead of setting it straight. This is why this week we have decided to go more in depth into the subject, with a mixture of dev diary/technical blog and an update on our plans to move forward.

We’re afraid we won’t be able to make it for our ETA of early June, instead pushing release back to the end of June.
The good news is that some issues have already been reported to us through our security@ub3rgames.com hot line and we have started working on solutions. We already did changes that makes our release slightly more secure than DF 2012, and we’re taking the time to address the new issues that were revealed to us.
Our Stress test launch will come with our first batch of security upgrades, and perhaps even some extra surprises.

We have also decided that, rather than interrupting InDev for our hacking weeks, we should invest into an extra server dedicated for it.
This will improve the experience for everyone and not impact play time for normal players.

Skip to the end if you want to know more about our short term security plans, otherwise, let’s dive into the subject at hand:

Addressing incomprehension:

First, we’ll go over some incomprehension we’ve noticed about last week’s events.
It seems like some are under the impression that the breach that occurred was something we would have released with as well and is inherent to Darkfall, but that is not the case.

The issue was about client packaging, something we’ve talked about multiple times over our road to release posts. We have had an obfuscated delivery ready for a while. It was a tedious process, and one of our team members nearly lost his sanity over it, but everything necessary to accomplish a delivery was given to us in working order. It was just a matter of recreating a proper build process from sources to installer.

There has always been hacks and flaws in Darkfall, just like in any complex software, but what was made public last week really created a new context.
Releasing the source code doesn’t immediately make Darkfall more hackable but it makes it much easier to analyze and find eventual flaws or confirm previous assumptions. It also makes any future efforts to solve issues much harder.
The rest of this post should explain why.

What is reverse engineering?

In the programming field, reverse engineering is the act of discovering the behavior of a program without having access to the source code.
It is generally considered that there are three types of reversing:

• Static analysis:
  • When the source code is available, it isn’t reverse engineering anymore but simply static analysis of code.
  • When the source code is not available but can be disassembled, then the resulting assembly/IL/Bytecode can be analyzed. It is a very tedious process.
• Dynamic analysis:
  • Through debugging the program and following its flow step by step. If it is in a VM like java, then you can always debug the VM itself.
  • No matter how protected a program is, it will always be possible to follow what it does and it is only a temporary safety.
• Network analysis:
  • If the protocol is human readable, then it is very easy to study and understand.
  • If the networking is encrypted, Dynamic analysis can help by discovering where the encryption key stored in memory.
  • If the protocol is in binary code, then it is more tedious. Understanding the packet structure is needed to understand the communication flow.

As a developer, no matter how much protection you implement, your software is going to be reverse engineered in one, two or all three of these above domains.
If a program is executed on your processor and communicates through your network card, it can be reversed.
It is only a matter of time.

Why even attempt to protect programs?

The main goal is to set a barrier of entry, a set cost in time that will either discourage lesser skilled people or make it not worth the time for skilled reverser that are doing it for the money.
A second goal is to gain time to make changes to your program that makes the knowledge gained from the previous version obsolete. The barrier of entry may be worth climbing once, but maybe not at each update.
And finally, just gaining time allows for flaws to be discovered faster by trusted parties with access to the source code rather than being discovered by ill intentioned third parties.

You will always have people that reverse for the challenge, these you cannot deter and will most likely attract with advanced security, but usually these people are well intentioned and are what is commonly referred as “white hat” and are a useful tool in improving security.

The story of how Alan Turing and his team reverse engineered the enigma machine to help win WWII is a great example of how reverse engineering and code protection impacted real life events. Essentially, the Allies were exploiting.

What are the consequences of releasing the source code of a game?

In a technical point of view:
It allows reversers to confirm anything they had deducted in the past and allows them to no longer work on assumptions.
This means that if the reversing was meant to create hacks, reversers will not have to understand the complete architecture of the program before being able to impact the part they wanted to impact. They will just have to browse the sources to where they want to do a change, and just do it.
It also opens the gate to new types of cheats that would not have been reasonably discoverable before. The time investment required to discover a flaw being cut down so drastically that even the most obscure one can be found and exploited for a hack.

In a project management and business point of view:
It all comes down to harmed reputation and the consequences for all parties involved. Either through flaws being made public by a white hat or through the sad observation that the game is much more hackable later, reputations will be harmed eventually.
This leads to less sales and the need to recruit to compensate the flaws, both of which taking away from the budget planned for operating the service and improving it, which in turns reduce sales even more

In Darkfall’s case:
A lot of things have been reversed over the years, but there were always some very crucial parts that never got fully discovered. Now these parts are not only discovered but confirmed. Some entirely new and pretty alarming possibilities have already been discovered by friendly reversers that we’ve consulted. Reversers that never worked on Darkfall before, which illustrate the ease of reversing now.
This means that Darkfall will inevitably end up having a reputation of being a broken game at whole new level of magnitude. Much more than in the past.

What can be done to delay reversing and prevent hacking?

• Obfuscation: This has never stopped reversing or hacking, but combined with other security steps can add up to the barrier of entry. It is also essential to make finding what has changed between versions harder and is the most basic step to protect intellectual property.
• Encryption of protocol and data: Again, a good time saving tool, but has an impact on runtime performances.
• A mine field of annoyance: ASLR, DEP, hide threads, overwrite call stacks, bomb registries or break every 10 lines in case of debuggers being attached, instruction causing crashes on disassembler, polymorphic code, remote execution and so on.
• Rootkit: pretty much hack your end users yourself to control the environment they execute the program in. Too intrusive just for a game.
• Move authority server-side: increase infrastructure cost and latency issues, but completely prevents cheating for the concerned parts.
• Change portions of the program from Java to native code. Java being pretty easy to reverse, this would be a great improvement.
• And much more.

What we will be doing:

After studying further our options, and hearing the feedback from the community, here is what we’ve decided to do:

• In the short term, our work in progress:
  • Deliver a different client for users, with none of the code from GM/Admin commands.
  • Prevent an outdated/altered client from connecting to the server.
  • Encrypt critical network exchanges between client and servers.
  • Invest in a new server infrastructure dedicated for hacking weeks, to not impact InDev play time at all.
• In the medium term:
  • Completely change the archive format to invalidate all current knowledge.
  • Switch to block chain encryption, and other fancy tricks.
  • Lay a minefield of annoyance that will drive hackers-for-hire away to other games.
• In the long term:
  • Move as much authority as possible to the server-side.
  • Only send relevant data client-side.
  • Over time, convert all of the Java to C++.

Most of these things were always planned, we just did not expect to need to work on them so soon. This is obviously impacting our roadmap and how we prioritize things.

The good news is that the short term goals are already being worked on and we expect to finish them in a few weeks. We’ll release the stress test once done.
Our first step here is to make sure that our client is secure enough that all of the future changes and improvement we do are done on a stable base that cannot easily be avoided. We are also setting up the infrastructure necessary to benefit fully from the intervention of the white hats we’ve reached out to.
This will give the opportunity for community members to help out in our anti-hacking efforts and be rewarded with game time should they find any flaws.

Hopefully, most of the people with no previous technical background now understand better the situation in which we are.
This new context will require drastic measures to adapt to, but we feel confident that we are well equipped to deal with it and are prepared to do the necessary changes.
If you have further questions, feel free to ask them in the discussion thread. In the mean time, see you next week!

Link to discussion thread.