Deutsche Version anzeigen
Show english version

---
Github: https://github.com/Flamefire

---

Vorwort

Ich werde den Beitrag noch erweitern, das ist jetzt alles nur das, was mir spontan in den Kopf kam.


list statt std::list

Es sollten die std::-Dinge verwendet werden statt der eigenen Implementierung (src/list.h). Ich weiß nicht, warum es die andere überhaupt gibt oder gab, aber sie hat schon zu dem ein oder anderen Problemchen geführt.


Parallelisierung

Momentan ist RttR weder parallelisiert noch wäre dies möglich, da statt einer Trennung viele Klassen direkt ineinander "verknotet" sind. Eine Parallelisierung ist sinnvoll, da im Moment pro Grafik-Frame maximal ein GF durchgeführt wird. Eine Entkoppelung des Ganzen würde dafür sorgen, dass man das Spiel wesentlich schneller laufen lassen könnte. Diejenigen, die das Spiel tatsächlich spielen, werden bestätigen können, dass eine niedrige FPS-Begrenzung nicht nur das Spiel an sich ausbremst, sondern auch die Bewegungen mit der Maus etc.

Die KI neigt auch dazu, das Spiel bei gewissen Suchen auszubremsen. Umgehen könnte man auch dies, indem die KI parallel zum eigentlichen Ablauf der Logik ihre Entscheidungen trifft, ganz wie der Mensch.

Aufgrund der Probleme mit NAT, die immer wieder auftreten, wäre es auch toll, einen dedizierten Server zu haben. Bei lokalen Spielen könnte man einen Server, NWF etc. ganz umgehen und so für ein flüssiges Spielerlebnis sorgen. Noch schöner wäre es dann natürlich, wenn die KI auf dem Server mitlaufen würde.


Modularisierung

Viele Werte sind in RttR einfach fest eingebaut. Dazu zählen nicht einmal unbedingt die Enums wie Nation, Species, GoodType, Job etc. und Konstanten wie z.B. MAX_PLAYERS, sondern auch die vielen Stellen, an denen wirklich feste Werte im Code stehen. Um nur mal ein Beispiel zu nennen:

Code:

Die Deklarationen sind nicht besser, auch da ist die 5 fest im Code drin. Es wäre also in dem Fall z.B. ungeheuer aufwändig, einen neuen Soldaten-Rang hinzuzufügen.

Terrain sollte auch nicht, wie es jetzt überall passiert, nur nach einer ID unterschieden werden und da dann nicht mit festen Werten. Beispiele hierfür finden sich vor allem im TerrainRenderer, wo man sogar sowas wie

Code:

findet. Wenn nun jemand auf die Idee käme, zusätzliches Terrain einfügen zu wollen, gäbe das eine Menge Probleme, mit denen niemand rechnet.

Gerade bei Terrain wäre es auch schön, wenn es Eigenschaften gäbe wie 'walkable', 'buildable', 'swimmable', ... Aufgrund dieser Eigenschaften könnte man dann entscheiden, was auf dem Terrain möglich ist und käme von IDs ganz weg. Man müsste lediglich die alten S2-Karten beim Laden entsprechend konvertieren.

Neben dem Terrain wäre es auch ganz toll, wenn Gebäude und Berufe modular wären und sich einer allgemeinen API bedienen könnten. Abhängigkeiten könnte man als Liste in die Module integrieren, so dass z.B. ein Bäcker nur gemeinsam mit einer Mühle, einem Bauernhof und einem Brunnen geladen werden kann. Oder etwas abstrakter: ein Gebäude kann nur dann geladen werden, wenn es Produzenten für die benötigte(n) Ware(n) gibt.


Addons

Die jetzigen Addons sind alles andere als schön. Auch hier wären Module schön, gegebenenfalls müsste man im Core die ein oder andere umstellbare Option einbauen, die man über ein Interface verändern kann.

Im Falle von Addons wie den Katapult-Grafiken könnte man einfach ein Katapult-Modul bauen, das entsprechende Einstellungen mit sich bringt. Zusätzliche Gebäude wie den Köhler, das Ausbildungslager etc. würde man direkt als Gebäude-Modul integrieren, das man optional laden kann.


Modernisierung der Formate

Es wäre meines Erachtens nach sinnvoller, die Originaldateien einmalig in ein gängiges Format (z.B. PNG, da lossless und komprimiert) konvertieren und dann damit zu arbeiten. Neue Grafiken könnten mithilfe einer Palette in modernen Formaten erstellt werden und die Konvertierungsprobleme entfielen. Es wäre wesentlich leichter für einen "normalen" Menschen, neue Grafiken, Landschaftsobjekte etc. zu erstellen.

Dasselbe gilt auch für das Map-Format, da das alte Format sehr unflexibel ist und das dynamische Hinzufügen neuer Objekte (z.B. eines Schatzes, den man in Missionen finden könnte) nicht erlaubt. Ein modernes Format könnte Deko-Objekte bzw. Skripting integrieren und somit Missionen ermöglichen.


Desktop-/Window-Manager

Der Window-Manager müsste komplett überarbeitet werden. Das Einbauen des Beobachtungsfensters hat mir gezeigt, dass es da einiges zu tun gibt. Im Spiel darf man z.B. gerne mal versuchen, das Beobachtungsfenster mittels Rechtsklick auf den Rahmen zu schließen. Das wird nicht funktionieren, da der Window-Manager beim Rechtsklick nicht zwischen Rahmen und Fensterinhalt unterscheidet. In dem Zusammenhang wäre es auch toll, zumindest eine Stencil Maske zu verwenden, um dort, wo Fenster sind, nicht dennoch alles andere zu rendern.


Renderer

Abgesehen von den oben bereits angesprochenen Unzulänglichkeiten im Quellcode besteht zusätzlich das Problem, dass viele Funktionen von OpenGL noch verwendet werden, die seit vielen Jahren als deprecated markiert wurden, also als abgelehnt bzw. überholt.

Im Moment wird immer alles gezeichnet, angefangen vom Terrain über die Terrain-Übergänge, Wege, Landschaftsobjekte, Gebäude, Figuren, Fenster, Mauszeiger, ..., was nicht nur äußerst ineffizient (Fill-Rate!) ist, sondern auch genau die falsche Reihenfolge. Eigentlich müsste man oben anfangen (erst Mauszeiger, dann Fenster etc. und zuletzt die Lücken mit dem Terrain füllen) und dafür sorgen, dass möglichst kein Pixel doppelt angefasst wird. Man wird es nicht unbedingt immer vermeiden (z.B. im Falle von durchscheindenen Grafiken wie Schatten), aber zumindest auf ein notwendiges Minimum reduzieren können.

Es bringt wenig, den Support für alte Grafikkarten aufrecht zu erhalten, wenn das Spiel darauf dennoch unspielbar langsam wird.


Überläufe, implizite Typen-Konvertierung etc.

Es sollten für Objekteigenschaften grundsätzlich Integer-Typen fester Größe genutzt werden, damit es nicht bei 32-Bit zum Überlauf kommt, aber 64-Bit weiterläuft. Auch Array-Indizes sollten an kritischen Stellen nicht gerade ein 'char' sein.

Es sollte außerdem vermieden werden, dass bei Aufrufen (insbesondere im Pathfinding) etliche Male implizite Typen-Konvertierungen zwischen x verschiedenen Integer-Typen stattfinden. Abgesehen davon, dass die Ausführung hierdurch verlangsamt wird, führt das zu potentiellen Fehlern und Async-Quellen bei 32- und 64-Bit.

Einige nette Links hierzu:

• http://www.viva64.com/en/a/0004/


• http://www.viva64.com/en/a/0050/

Außerdem sollten wir uns grundsätzlich ein paar Gedanken über die Effizienz des geschriebenen Codes machen:

• http://www.open-std.org/jtc1/sc22/wg21/docs/ESC_Boston_01_304_paper.pdf


• http://www.open-std.org/jtc1/sc22/wg21/docs/ESC_San_Jose_98_401_paper.pdf


• http://www.open-std.org/jtc1/sc22/wg21/docs/ESC_SF_02_405_&_445_paper.pdf

Man müsste sich zusätzlich noch Wege überlegen, wie man verhindern kann, dass z.B. ein Objekt stirbt, aber Events hinterlässt, die dann mit nicht-initialisiertem Speicher durchgeführt werden.


Vorschläge für einen Rewrite bzw. ein neues Design

Hier ein kleiner Entwurf:
https://dl.dropboxusercontent.com/u/11244516/design.pdf

Der Server ist im Grunde dumm und macht nicht viel mehr als die GameCommands zu verteilen, die er erhält. Der Server in Kombination mit der Simulation sorgt für ein Interface, das für alle exakt gleich ist, egal auf welchem Client sie laufen. Es ist essentiell, dass die Player die Simulation nie direkt ändern können, sondern immer über den Server gehen müssen. So wird schon per Design sichergestellt, dass das Spiel überall synchron läuft. Ob nun an einer Simulation ein menschlicher Spieler, ein Zuschauer, eine KI oder alle drei hängen, ist im Grunde egal. Damit wäre es sogar möglich, eine Simulation an einen dedizierten Server zu hängen, an der wiederum einige KIs hängen. Damit wäre es aber auch möglich, einen Server mit einer KI zu starten und mehrere KIs mit Simulation über das Netzwerk dranzuhängen, um eine Art Testing gegen Asyncs oder für die KI zu bauen. Als Zuschauer könnte man sich jederzeit einklinken und beobachten, was im Spiel passiert.

Eine Möglichkeit, im Async-Fall den kompletten Status erneut abzugleichen, wäre toll, braucht allerdings viel Bandbreite und bedingt, dass man einen schönen Weg hat, das Chaos erst mal aufzuräumen. Momentan gibt es auch da Probleme, so dass nicht alles sauber entsorgt wird, wenn man ein Spiel beendet.

Klar, das ist alles noch unvollständig und es gibt noch keine Implementierungen, aber ich denke, dass wir hiermit einen Weg gehen, der etwas für die Zukunft ist. Die aktuelle Nightly ist aus meiner Sicht aufgrund der vielen Async-Fälle nur im Singleplayer sinnvoll spielbar und es müsste zumindest eine Art Feature-Stop geben, bis man die Ursache für den Async gefunden hat. Letzteres erachte ich aber beim aktuellen Design als eine sehr schwere Aufgabe...

---

More about addons, we need ability of adding buttons to given windows, e.g. all buildings and/or specific building, along with action. Moreover, the tricky part comes with construction builder. Right now I'm working on military building upgrade addon and implementing it requires a significant amount of code added to current builder logic, so he can distinguish construction from upgrading. Personally, I have no idea how could you simply extend his functionalities in such way with some kind of addon system.

As much as I looked into rttr code, it seems that there is a possibility with some work to tear off server code from client and put it in separate app. I mean, there is need to implement communication with such server, but object-oriented model of current code somehow should allow to do this without much pain. So how about this, let's create separate server app, then add to our codebase additional packet checks and change client to be more of receiver of simulation data.

I'd like to point out that parallelization isn't cure for every fault in RTTR code - there are lots of games, which work properly without it. If we depend on it, then we lose all compatibility with single-core CPUs or even older (retro maybe?) platforms. It's not that it's something scary nowadays, it's more like a question of cutting off our roots :)

---

Separating server and client is, although possible, rather difficult. It starts with the nightly's
approach of starting games, then there are calls to the GameServer object from both client and
especially KI, as it is tied to the host or server. One would have to rebuild the way network
communication is working atm.

Of course parallelization isn't the cure for everything, but it helps. Threading even works with single-
core CPUs, you just have to tell the threading library where to interrupt the flow of one thread and
continue with another. They just won't run parallel. The problem we are currently facing is that of one
part of the game slowing down another one due to delays of doing nothing but waiting for the right time
to render the next graphics frame. I doubt by the way that RttR would be able to run on retro platforms.
Even on high end computers it's kinda slow.

RttR is grown code and for what I know has first been some kind of proof-of-concept. A lot of problems
we found during development weren't that easy to solve, since some wrong decisions were made in the
first place.

I personally think that a rewrite would be a much better idea and other members of the team think alike.
I'm not that kinda guy who renovates dilapidated houses. I'd rather demolish the building and rebuild a
new one. After renovating a house, what is nicer than finding out that the foundation is crumbling? ;-)

For those of you who want to continue the current nightly branch, please keep in mind that it doesn't
make much sense to implement more and more features without getting rid of the async bugs that keep
following us. As I am the one who has been running the debug server, I can tell you that asyncs are more
often than we think. Even those revisions that were considered stable generated numerous async reports,
so this should be the main objective for those continuing to develop the nightly branch.

---

What I don't like about full rewrite is that right now after all these years we have somewhat complete game and I feel that much time will pass before we'll have stable enough build to play a simple match. But on the other hand we can still play using current version.

What I don't like about parallelization or multithreading is like, using atomic bomb to get rid of a cockroach. I mean, S2 remake with additions should be able to work like a charm on a 500MHz machine without compatibility with shader model. Guys, it's completely 2d game with simple graphics, what the hell? Remember system requirements of S2, S3 or even S4. RTTR shouldn't need any more resources as far as I understand what is needed to calculate.

I suggest that we should try to simplify a code A LOT in small steps, so we have more efficient codebase. If you don't like idea of fixing old and bloated code then at least let's do proper, extensive audit and find at least one thing to change in particular parts of code, mark them as fixable or not and then let's start taking some decisions.

In about a week I'll have access to my old Thinkpad (Pentium M 1.5GHz, 1GB RAM, integrated gfx) and older Dell workstation (swappable P2-266 to P3-500MHz, some SD-RAM megs), then I'll try to do some benchmarks regarding amount of CPU time taken by game logic and renderer. Perhaps after just optimising drawing routines we could cut asyncs in half?

Nonetheless, if you decide to do a full rewrite, you can count me in.

---

I agree with KaiN that a full rewrite would take a huge amount of time. However maybe it is an idea.
What I would prefer is cutting a part of and rewrite it, than add it again and change what is neccessary. Like KaiN suggested: Improve the current code in small steps.
E.g. Introduce a new class structure for buildings, so every building is split in working (simulation) and drawing part. Each building knows what it needs and stuff. Rewrite all the existing buildings to use that new model.
Well...wait. You are right. At the end a complete rewrite is faster. And we can copy large parts of the code.
But we should make the design first.

Multithreading is not an atomic bomb. It's a solution for slowness problems. Rendering and calculating should be done in separate threads. Even single cores are able to multithread: It automaticly(!!!) switches to a different thread when one thread is waiting/sleeping. Thats how the whole system works!
And changing the drawing part won't fix any async (as long as drawing does not change the game state)

---
Github: https://github.com/Flamefire

---

I am an opponent of multithreading because I think for RTTR it would be more of a way chosen by developers to eliminate timing problems in quick way, because there is a 95% chance that it will solve the case. Quick for them, but not necessarily efficient, not for all machines, if we look especially at the older ones. If it is the one and only way, then why there are lots of multiplayer games with splendid net code and able to keep itself together in one thread? Just look at starcraft, age of empires, or even s3/s4. I could never complain about their net code quality. I mean, if multithreading is so cool, then why most of games done since 2000 aren't using it and it's now-emerging trend?

There are main questions to be answered, and one of them is: do we need to support older machines? If not, let's set some minimal system requirements.

Okay, I'm going to put this straight: I came here from Amiga community with plans to eventually port RTTR to 68k. But I guess such project should have game client specially built for this purpose, not just dedicated renderer.

---

Well. Multithreading is emerging with the spread of multiprocessors. I can say that 95% of the computers, laptops etc. do have at least 2 cores!
The reason why this was not used earlier are actually 3:
1) Multithreading is hard! It's not just "yeah let's get another thread done". You have to be very careful using threads as you have to sync all accesses.
2) With only 1 core available the benefits are not that great. As mentioned: the OS switches to a different thread every now and then and especially when one thread is sleeping. But only with more than 1 core threads are running parallel.
3) Graphic rendering used to be slow. It was acceptable to use only 1 thread as there was no reason to stress graphics rendering when nothing has happened yet. Today however graphics are fast and can be rendered more efficiently. So you can e.g. move the mouse more often than the game frame rate. With only one thread FPS=game frames and you would notice a lacky mouse cursor (e.g.)

It won't solve timing problems. It will just make the game smoother. Just imagine a simple gameloop:
a) Calculate positions
b) Get AI cmd
c) Render
d) Wait for next game frame

Alright. a) is fixed to a GF, but what if b) or c) take a bit longer? It will result in a lack in GFs.
Other way round: You move your mouse cursor just after a) you won't see any changes till b), c) and d) are done although this would be possible because you are doing nothing in d)

Threaded solution with e.g. 3 Threads (could add AI into thread 1):
a1) Calculate game
b1) Wait for next GF (~30ms -> 30GF/s)

a2) Get AI cmd (maybe based on event-queue)

a3) Render
b3) Wait for next (rendering) Frame (~15ms -> 60 FPS)

About your port: A dedicated server would lower the system requirements. If Game logic (validation) and AI are running on server the game client won't do to much.
However I don't know if we can split the logic that apart.

---
Github: https://github.com/Flamefire

---

There are two parts that could be considered logic:
1) everything the AI does,
2) the simulation.

The key behind the concept RttR is using for networking is to do everything on all clients to avoid network traffic. Only GameCommands are sent, everything else can be computed everywhere. Every single client is aware of the game state. And that's exactly what makes it rather hard to port RttR, since a client has to be attached to a simulation somehow. Otherwise every single path a ware takes, every step a figure makes, ... would have to be transmitted via network, requiring really fast network connections.

My idea is to distinguish between clients and players. One client (or simulation) can have one or more players attached to it. Everything a player does will, maybe after a short plausibility check, be sent to the server to be distributed to all other clients. The client itself handles local distribution between the players.

You may want to have a look at http://web.cs.wpi.edu/~claypool/courses/4513-B03/slides/BT01.pdf
These slides describe the concept RttR is supposed to be using and used by Age Of Empires.

My threading approach would be:
1) logic thread
   - simulation/logic
   - non-blocking networking
2) one thread per player
   - human players/spectators: rendering and user io (mouse/keyboard responsiveness!)
   - AI players/bots: decision making (involves a lot of pathfinding)
3) (optional) server thread
   - in addition to running a dedicated server
   - makes it easier, less overhead than whole new process
   - server responsiveness (reduce lags)

Atm, all clients will appear laggy if the server is not able to receive data fast enough. Even worse, with the current design, the server is always the same 'client' that is running the AI, so it makes things even more laggy.

As Flamefire already said, threading has nothing to do with solving the async bugs in any way. It's just a way to make the game more fluent on systems with more than one core. OpenGL with VSync enabled just waits for a vsync on flipping buffers, blocking everything else. This can be avoided using threads. Disabling VSync may lead to tearing when scrolling.

---

Yes the client-Players approach is good. However every command has to be sent by the server in case of lost packets! (I guess that's an issue right now...)
As the whole simulation is run on a client, it should be possible to have "views" on that simulation, which is "players" in your terms. If you set a server validation on the commands (player only changes own stuff) this would even allow splitscreen gaming. And is the base for the observer window.
AI would really be a player attached to any client.

1) thats ok. However got to be careful with network protocol. In Simulation you have to wait for next GF and might miss network packets there.

2) One thread per player is ok as rendering and input got to be apart from rest. That's critical in my opinion.

3) Server HAS to be in a different thread. I suggest making the server a library which can run in a thread or as a separate process (dedicated server) There should be NO connection to the server from any other program code besides start/stop

---
Github: https://github.com/Flamefire

---

Thanks for the slides, they've enlightened some aspects of the problem, but also I have some new doubts.

So, if current RTTR net code concept has been proven correct and doable by similar implementation in AOE then perhaps it is possible to fix asyncs by thoroughly revising net code or even reimplementing same concept in current codebase? I'd like to remind you that it's not like the concept is bad as it's working correctly until about 100k gf has been reached. Then hell breaks loose.

---

I don't think that we will have a lot of dropped or lost packets when (still) using TCP. For UDP, this may be more of a problem and will best be dealt with in yet another thread to wait for udp packets, send acks etc. Incrementing the receive buffer size should work, too, as we do not have many/big packets. Linux has a limit of 4 MB for send/recv each, so this should suffice to hold ~20ms of packets (about 1.6 GBit/s).

Split screen should be possible using this approach. Every GameCommand contains the player id. The distribution should work as follows:

Client A: Player 1 (human), Player 2 (KI)
Client B: Player 3 (human)
Client C: Player 0 (spectator - there may be more than one player with that special ID, spectators cannot send GameCommands anyway)

1) Player 2 (attached to Client A) sends a GameCommand
2) The client part relays it locally (to the local simulation) and sends it to clients B and C via our server.
[...]
n) Player 3 (attached to Client B) sends a GameCommand
n+1) The client part relays it locally and sends it to A and C.

The server should have two possible commands:
- send to one (for chat messages)
- send to all but self

The client part would expose
- send to one
- send to all (local relaying + send to all but self on server)

Due to local relaying, local games would work without any networking at all, avoiding the whole timing problems. A few years ago it was a pita to build a flag in RttR, as it took ages even in single player. I used to triple-click to build a flag and start building a road, but that was impossible in RttR. :-) To avoid those problems, the player part (the MVC view) should be able to remember the flag built and already display it and make interaction possible, as there is no reason for it to fail. If it should, for some reason, fail, this visual aid should just be removed.

Here's another paper about 'A Review on Networking and Multiplayer Computer Games': http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.16.2565&rep=rep1&type=pdf

KaiN: The problem with the current codebase is not only that. There's also the problem of different sizes of used integer types (32 vs 64 bit) that could explain late asyncs. See my first post for some information and articles on that topic. I think there are a lot of things one has to keep in mind to produce code that will run synchonous on every platform. There has to be some way to control that objects won't disappear while still being accessed and array bounds checking becomes more important than ever. That's why we should make use of std::vector, ... instead of static sized arrays that may be compiled in a debug release to do bound checking.

Just consider the following:

Code:

something_else would be set to 4 in this case. This can happen in a lot of places of the current code, especially since many loop/array bounds are hardcoded. We would have to change every single instance. We'd have to get rid of the old list as it is buggy, ... As Flamefire already noticed, it's easier to rewrite everything than changing stuff over and over again for every single step to a complete renewal. The reason I suspect something like that is that Spike told me about asyncs only happening with iirc four or more players. I think the implementation of the concept is working well, although there are some suspicions about it not working correctly. It's more likely that there is some access to uninitialized data, some overflow, calling member functions of already destroyed objects or out-of-bound array access, for whatever reason. All these cases have one thing in common: they are really hard to find.

---

Do not use local relaying. Just imagine a packet sent to server gets lost. The local simulation has it and wont wait for the server to sent it. -->Async

So it should rather be that you make the simulation send commands to the server. So basicly Simulation.queueCmd()
And the server distributes all the commands. The Network part of the simulation takes care about the order and the frequency of sent packets and NWFs. It also filters double commands (dont't try to build a flag twice)
The server on the other side takes care of NWF length (dynamic adjust to current pings and lacking players) and sends all commands to everyone except chat commands.

Well that the game runs fine till 100kGF doesn't mean it isn't asynced earlier. It's just not detected by then.
E.g. the deer mentioned: A deer is placed on x coordinate 10.0000023 instead of 10.0 on 1 client. It takes ages for that to have an effect where the hunter will miss that very deer and the async get's detected.

BTW: How would you implement the building class? Should every building have an array with e.g. costs/nation, Produced stuff,... Or would you make that global?
What would you do when you add a new nation? You would have to change all classes...

---
Github: https://github.com/Flamefire

---

Your explanation is making case for me even more clear. But I still have some slight doubts. What if we try to reimplement game in the same concept, but bearing in mind known mistakes of previous code? I think that by taking this approach, there is much greater chance to evade additional design mistakes. It's a matter of experience.

It's not that I'm going to infinitely defend current code model. Bah, I must confess that I don't understand it in every aspect, I haven't read through most of game code and I'm also not some kind of long-term or important dev over here. What am I trying to do here is asking some questions, which I hope could help in finding and taking right decision.

---

Sidenote: Tcp doesnt lose packets, it ensures you receive them (however late). If you lose lots of packets, the game starts lagging, it doesnt go async.

If the async problem were that simple, i'd have fixed it a long time ago ;)

---

I don't think this has already been said, so:

every dev is important!

Just to make this clear, there is some need for project management etc., but I think that every single person wanting to contribute to this is important.

Local relaying is no problem. We just have to make sure that packets arrive. We would have to do so otherwise, too. Missing packets have to be resent anyway, otherwise the player would for example attack an enemy building, move somewhere else, the packet containing the attack command would be lost and the player wouldn't even notice. There are ways to make sure that packets arrive. With a possiblity of losing packets, there could also be packet loss between the server and one of the clients only, making the game async. Therefore local relaying is not a problem at all. The concept is just as described in that paper on AoE. Order is important, so every single packet should have a sequence number and maybe even carry the current GF number. As there are not many commands, there is no need for them to be collected locally, I'd rather prefer to send them out as soon as possible to minimize delay. We'll have to see about the details, but this is fine-tuning then.

As we are not using floating point numbers, x coordinates like 10.0000023 are not possible. Of course async state may not even be detected sometimes, but that's another thing.

There are different building classes. Some buildings may just be producers, i.e. they need nothing, but produce wares regularly. They will just register an Event every x GF that will tell the core that a new ware is ready. Processing buildings will have a list of certain types of wares they are accepting and will request those. The logic will then try to organize the wares for the building. Then again, the building tells the core whenever a new ware is ready. Warehouses are another base class for buildings as well as military buildings. It is possible for buildings to provide more than one base functionality.

A new nation would just contain new images for specified buildings. All images used by a module should be known by name and therefore be overrideable. Once again, details will have to be defined later.

One of the key classes I am planning to use is some kind of an Enum implementation like:

Code:

This is serializable as well, so could be transmitted and synchronized over the network. The special value of (uint32_t) 0 is reserved for 'not assigned'. This way it is possible to use integers internally and only use the names at load time, making things really fast. As the numbers are assigned ascendingly, it is possible to manage arrays (rather std::vectors) of e.size(), once all possible values have been assigned. Tests assure that the enum implementation is working as expected. We should add tests for other data structures and algorithms we implement, too.

---

Alright. I just removed the custom list class from the code and (maybe) fixed some possible bugs that could not be noticed. Now assertions will be thrown there instead.

What I consider important is that GF# are included in the network packets, so I'll look into that next.

About your enum class: What do you want to use it for? The problem there is that typos won't be seen during compile and you can get in conflict with names of other stuff.

About local relaying: You are right. However those commands shall not be executed before the server issues the next command where they would be in.

A new nation will also (probably) contain other building costs and also the offsets are different (smoke etc.). Additionally it could be possible that a military building contains different numbers of soldiers... How would you handle that?

---
Github: https://github.com/Flamefire

---

The GF# does not have to be included in every packet. It is enough to send a packet inbetween, just telling that the GF number has been changed. As long as the order stays the same, this should be no problem. Using UDP, we can make sure that the order is correct using a sequence number. The server can send a tiny ack packet using the same sequence number to tell the client the original packet has been received. The client could resend its packet after it has not received an ack in time. The server would ignore duplicates. Again, we'd have to specify details. It's about the same as in the AoE article and already working, so that shouldn't really bother us. Keeping the classes as independent from others as possible, it should be possible to write tests to make sure networking is working as expected.

There could be another version of get that would return 0 if the string does not exist and the id otherwise. Of course it does not make things easier, but what else would you do? Make a global registry of buildings and everybody has to get an id from there? How would you manage a list of buildings in-game? And even more important: how would you manage ties between different modules like requirements? How would scripting be able to disable certain buildings?

I doubt that there will be conflicts with names of other stuff, as there may be more than one instance of that class. So you can have one Enum for each buildings, terrains, favourite food etc. Just use "Enum buildings, terrains, favourite_food;".

Graphics offsets won't be a problem, as they should be merged, anyway. I do not want to work with thousands of tiny textures, as this is really inefficient using OpenGL. There will be some class for textures and/or animated textures.

A military building with different building costs or containing a different number of soldiers is another type of military building. Although it may carry the same caption, it would just be a different building. However this is not that important right now, as the focus is on recreating the original settlers 2 experience before starting to develop any extensions.

---

Zitat von Marcus:

Here's a question: why use UDP to create an exact replica of whats provided by TCP?

---

It's not an exact replica.

The UDP header is fixed 8 bytes long, a TCP header is at least 20 bytes long. I did some sniffing on my local network and tcp headers are 32 bytes long, as tcp headers may contain options (like in this example timestamps). A simple sequence number in addition to the packet would be 4 bytes longer, so our header would be 12 bytes for UDP and at least 20 for TCP. Also, there are protocols like STUN for UDP NAT traversal. Then there are funny things like nagle's algorithm that needs to be disabled in order to make TCP "responsive".

Again, I did some sniffing of starting a game + 10.000 GF of just letting 2 AIs build a bit:

There were 2945 packets carrying 111996 bytes of data.

For me, this has been:
20 (IP header) + 32 (TCP header here) bytes headers/packet
  => 153140 bytes headers
  => 1.37 header bytes/data byte

Minimum would be:
20 (IP header) + 20 (min. TCP header)
  => 117800 bytes headers
  => 1.05 header bytes/data byte

For UDP + 32-bit sequence number:
- 20 (IP header) + 12 (UDP + seq) bytes headers/packet
  => 94240 bytes headers
  => 0.84 header bytes/data byte

Please keep in mind that this contains some bigger startup packets and the difference will become more visible later in the game.

One last thing: I am not against using TCP. It's just an answer to your question.

---

I don't think we should go with UDP. There is a reason why newer games ONLY support TCP.
The guaranteed delivery is one of them. We just have to keep the order.

Just say you send an packet every 10ms with an average size of 100B (20B IP + 32 B TCP + 48B Data) than this will result in 100KB/s Upstream and <=800KB/s Downstream which is acceptable in todays connection speed.
Now assume a NWF of 100ms and a packet send every half (50ms) than this will cut down to 20KB/s Upstream. Its ridiculous small compared to about 768KB/s Upstream speed from a normal DSL connection.

But yes, you are right and we may or may not implement a driver that may use UDP. But I think UDP does have some decent drawbacks also I'm not certain about them.

In TCP the client should send a packet with GF# to tell: I passed GF# x so the server knows, if a client is lacking. Also it should be in a way, that the command sent in GF# x is executed in GF# x + y for all players. (where y = NWF frame length or some server determined value)
The server in turn should send a packet: GF# x is over, execute commands a-z. Then everything is fully synced.

I would use a normal enum to handle that. I know you have to change that every time you add a building but if you want to access the well from the bakery you need the string from the well anyway. So you can keep it in an enum. Otherwhise imagine something like: IsAvailable(enum.get("Well")) but "well" was stored, than the building will fail to load or work or whatever.
Or even worse:
init(){...IsAvailable(enum.get("well"));}
work(){...getBuilding(enum.get("Well"));} // BOOM

You would have to define constants for that. Well. Make a normal enum in the first place instead.

About offsets: What about different positions of smoke? You HAVE to have some offsets there. And graphics are merged into one texture anyway. Wouldn't make 1 picture in the first place as that is hard to edit.

I know it is not a question for now, but if we don't ask it now we will have the same problems later...

---
Github: https://github.com/Flamefire

---

Network-Stuff
100 Byte every 10ms would be ~78 KBit/s. My internet connection (DSL 1000 - yaay!) has 128 KBit/s up and 1.024 KBit/s down. I'm quite sure this will be problematic, as TCP is not designed to transfer many small packets and there is other traffic as well. The rate needs to be lower and it could be nice to be able to reduce the number of packets send and make them bigger instead. TCP (with Nagle's algorithm) has a higher throughput than UDP, as it will combine a lot of small packets to bigger ones, but also introduces latency. As we do not need to send that many packets (there's no reason to send one packet each GF), we should use ways to keep the latency low (like disabling Nagle's algorithm for TCP). Once again, as I said, ordering a few packets containing sequence numbers is not magic. Making sure all packets arrive is not magic as well. It's just more time-consuming to code and more prone to bugs.

My suggestion would be to use TCP first, as it is easier to use without making mistakes, and provide an additional UDP protocol implementation later. UDP allows the use of STUN and therefore NAT traversal on both ends.

Btw, I don't really understand why you think that modern games won't use UDP and prefer TCP. As far as I see, they use both:

FIFA 2013 (https://help.ea.com/uk/article/what-online-ports-should-i-open-for-fifa-13)
- UDP: 3659; 9565; 9570; 9000 - 9999
- TCP: 3569; 9946; 9988; 10000 - 20000; 42124

Battlefield 3 (https://help.ea.com/article/online-ports-for-battlefield-3)
- TCP: 80, 443, 9988, 20000-20100, 22990, 17502, 42127
- UDP: 3659, 14000-14016, 22990-23006, 25200-25300

Settlers 5 (http://ubisoft-de.custhelp.com/app/answers/detail/a_id/11882/~/what-ports-does-settlers-5-need-for-online-play%3F)
- TCP: 6668, 40000-42999 , 7073
- UDP: 41006, 44000, 45000, 45001, 7073

Call of Duty - Black Ops II (http://portforward.com/cports.htm)
- TCP: 3074, 27014-27050
- UDP: 3074, 3478, 4379-4380, 27000-27030

Civilization IV
- TCP: 2033, 2056, 3783, 6500, 6667, 28900, 29900, 29001
- UDP: 2302-2400, 6500, 6515, 13139, 27900

Normal enums won't work.
We are talking about modules here. Parts of the game that may be added dynamically. Missions may want to add special landscape objects that may be found (like the ruins in s2). They may want to add additional buildings (tent as a headquarter). People will enjoy the possibility to become creative and extend the game with their own buildings (see Spikeone's strawberry farm or even the charburner) and they won't be ugly hacks. That's the very idea of making things modular.

To your 'even worse' example: it's not. It will crash, trying to do something with a NULL pointer. You will run the game in a debugger, it will serve up the error on a plate. That's it.

And no, defining constants for things that should be dynamical is not a good idea.

Smoke on the water... :)
There is no need for smoke to be on a different texture. It's rather contra-productive. There's no problem with the module building one texture out of a few images, but the game should not have to handle unnecessary (tiny) textures while it's running.

---

Network: Yeah I agree with you. So stay with TCP disabling Nagle. And the 10ms was just the very worst case. So you see: Even your (very very bad) connection gets only used half ;) I think 6k is now the standard so even that won't be a problem. Latency however is.

Enums: Ok I get it. However it should be common practice to use them like this: wellID=buildingModule.get("Well"); in the constructor and then use only the wellID so no messing with strings afterwards! You may even define them as consts in the header of the module that uses it.
Alright. .getBuilding() can then even produce an assert where it mentions the wrong string or smth like that.

Smoke: So you really want to say that additionally to the fucking lot of images that are needed for a single building you also want to add smoking variants of that building? That is contra productiv IMO. And if the images don't have the smoke you need offsets.
Tiny textures: Well they are commonly used and take not much memory. Currently RttR is combining all the images in 1 texture. I think this is the best solution as it leaves space for more textures. Combining the buildings with the smoke will increase the size of that texture too much.

---
Github: https://github.com/Flamefire

---

Enums: Yes, that's exactly the point. The strings are only used once and replaced by numeric ids afterwards. Reverse lookups (id to string) are possible, too, but will take longer. They come in handy for debugging.

Smoke: Maybe it is not always possible, but drawing over one pixel again and again is one of the worst things you can do (fillrate is limited!). A lot of tiny textures will bring in a lot of overhead. The textures may carry more meta data then data. It's somehow compareable to the TCP/UDP thing, only that there's no reliability issue. I'm talking about the texture size itself here, not about how much space it takes on the screen once it's rendered. Switching textures can be expensive. One big texture is a solution to that, but takes some time to be built (I implemented the big textures used in RttR). All the information about texture coordinates and coordinates has to be sent to the graphics card. The size of that meta data stays the same for a big texture and a smaller one, so we would double the amount of data to be sent if we draw two (size does not matter) quads instead of one. Trust me, I did most of the optimizations, OpenGL and C++ profiling in RttR. That's the reason why shadows are combined on one texure with the figures now. I would improve the fill rate, but translucency makes it difficult to do so without using shaders.

---

Well, then combine shadows with the buildings and stuff. But combining buildings with smoke...
The transformation to 1 texture takes already a huge amount of time after I added the new nation at least in debug mode just because the number of textures increased.

---
Github: https://github.com/Flamefire

---

That is one of the reasons why I would convert the data that belongs together ONCE after installing the game. See 'Modernisierung der Formate' in my first post. There are a lot of other things that take a lot of time in total atm, like converting thousands of indexed and rle compressed bitmaps to RGBA and creating textures for every single one of them.

Building mega textures is not that much effort once it does not have to include merging of textures and recalculating the dimensions for every single bitmap.

---

Network Protocol

It all boils down to

• All GCs sent have to be executed.
• The order of execution has to be the same on all clients.
• Everything has to be executed simultaneously, in the same GF.
• Clients should be treated fairly.

For a client, it is not important to know what NWF a GC it is about to sent is executed in. This means clients can just send GCs as they are produced.

The server can decide what GCs will be put into what NWF and therefore manage the NWF length. However, clients have to be aware of that NWF length, since the GF everything will be executed in has to be the same everywhere.

Every GC the server receives in NWF n will be executed in NWF n+2.
NWF n will be used for collecting all GC to be executed.
NWF n+1 will be used for sending them to the clients.
NWF n+2 will then be the time the clients should be able to execute them.

There will be lag if NWF n+2 is reached on a client without all GC from NWF n being received.

We could, instead of collecting the GCs in NWF n, just forward them to the clients as soon as they are received, sending an END OF NWF packet after that. We have to try what solution is the faster, less laggy one.

The server may change the NWF duration, but this cannot be effective immediatelly, as the clients need to be able to compute a GF number that may not be overstepped without receiving the necessary END OF NWF packet.

Example:

• [S] Initialization
• [2] send GC 2.1
• [1] send GC 1.1
• [S] END OF NWF 1 (contains all GCs of that NWF)
• [3] send GC 3.1
• [2] send GC 2.2
• [S] END OF NWF 2 (contains all GCs of that NWF)
• [C] execute all GCs from NWF 1
• ...

---

Whatever you do, keep on going guys - we belive in you :D

---

Wow, this thread urges me to my first post since years or so. I haven't been playing any of the more recent nightly builds, but rather stuck to one of the older versions, with less performance, less bugs (but still a version after the time the game was unplayable because of crashes). What I noticed already years ago: the game has a lot of performance issues (shouldn't cost a lot to have some 2d graphics displayed and also the game logic ran on old 166MHz machines) and with all the addons people worked on, a *lot* of new bugs have been introduced, even affecting parts of the game that previously worked fine.

I am really happy to see that some programmers noticed as well that the game probably needs to be rewritten or at least completely refactored. If I ever found some time I would gladly join in programming something, but I fear I am too busy programming and work to even do more programming in my freetime... I'll however follow the discussion from time to time and see what you are doing/planning to do ;)

---

Yes, the rewrite is currently beeing planned. Everything should be modular and therefore extendable.
However some more coders with time will be required. Then it will be a great game with huge possibilities.

---
Github: https://github.com/Flamefire

---

Sounds like a good plan. Make repo on github :P

---

We will work on Launchpad and therefore use Bzr. This eases the work of contributors for translations and bug reports.
Although git is better for developers.

---
Github: https://github.com/Flamefire

---

There is currently no one really working on rttr anymore. What we need are more developers for the rewrite that is planned or
for rttr itself.
But for any further development on rttr we need access to the build server that is based on an internal svn. Or at least base
the build server on the Bzr from Launchpad and have some more people access the build branch to allow effective merging
of new features. Otherwise there is no point in creating anything new if it is that complicated to get it in.
There would also be a need of huge refactoring of the code base to fix some hacks introduced over time...

---
Github: https://github.com/Flamefire

---

Just maybe one should focus on getting the original settlers II gold running smoothly - or in other words get to version 1.0 - before creating an additional culture, buildings, options, and so on...

---

In my opinion one of the problems of this project is the language barrier. Many of posts, source code comments and
talks on IRC are in German, not English. There is no roadmap, no code documentation. This effectively limits the
potential people which can be interested in joining to developers team of that project.

---

There are actually 4main problems: language and the to early implemented and to many addons,  the mix of Server, client
and drawing, undocumented and inconsistent constants and the difficulty to add changes due to the build server structure.

---
Github: https://github.com/Flamefire

---

agreed. there is also a rttr facebook side but it is pretty dead - maybe we could get more developers by creating more awareness. crowd funding might holds the key to get money and thereby more commitment plus speed in development.

---

Yes.  However there are multiple design problems. That is the reason why wanted to create a total rewrite. The changes
would be very huge. And the new system should be really expandable... Not just with "hacks" as in the current one.

---
Github: https://github.com/Flamefire

---

Die Idee mag zwar nicht schlecht sein, aber was bringt es uns denn, wenn jetzt neue Coder wieder von vorne anfangen und irgendwann, noch bevor das ganze den 1.0 er Status erreicht, dann auch keine Lust/Zeit mehr haben ? Ich würde sagen nichts. Dann hätten wir 2 halbe Projekte. Ich weiß zwar nicht wie der Code aus RTTR aussieht und ob der nun gut oder schlecht ist, aber jedenfalls kann man es spielen ;) Wie lange würde es wohl dauern, bis die erste Fehlerfreie(Spielbare trifft es eher) Version von den neuen Codern rauskommt ? Wahrscheinlich auch ewig. Vielleicht sogar Jahre. Wenn man mal bedenkt wie alt das Projekt jetzt schon ist und wie lange die hier schon gebraucht haben ein vernünftig aussehendes (Code)Remake zu entwerfen, dann bezweifel ich doch stark, das es Sinn machen würde. Ich kenne keinen von den Codern hier, die sich daran beteiligen wollen, aber wenn man mal darüber nachdenkt, könnte man sich die Frage stellen: "Ist das wirklich ernst gemeint ?" Habt Ihr überhaupt die Zeit dafür ? REDEN können doch alle. Die Frage ist ob Ihr das auch durchziehen wollt. Denn wie gesagt ist jeglicher Gedanke daran verschwendete Zeit, wenn angefangen wird und nach einem Monat schon die Luft raus ist. Ich weiß zwar nicht genau wieviele Coder an dem alten Projekt beteiligt waren, aber ich denke, viele können es nicht gewesen sein. Es gibt immer einen im Team, der das meiste von allen macht.

---

Achja, der Text ist extra in deutsch, da ich mich nicht blamieren möchte. Denn English ist nicht meine Stärke. Verstehen ist zwar kein Problem bei mir, aber selber schreiben schon ;)

Zur Sprachbarriere:

1. Das Spiel ist in Deutschland entstanden.
2. Fast alle deutschen kennen es.
3. Im Ausland kennen das nur sehr wenige (Ist das überhaupt in anderen Ländern erschienen ?!?).
4. Daher ist die Sprache auch bequem in Deutsch gewählt. :P
5. Wenn ein Deutsches Spiel von einem Deutschen neugeschrieben wird, ist das Projekt in Deutsch und auch das Forum, sowie Anleitungen und ähnlich. Ich würde das nicht anders machen. Das spart einfach Zeit.
6. Learn German :P

---

There should not be any kind of crowd funding or similar. The reason is simple: we don't want to produce anything that may be considered a commercial product. I don't think anyone here is able and willing to pay a herd of lawyers... ;-)

RttR is playable, yes, but only in single player mode. There are some bugs that would take ages to fix. And, as you all can imagine, this is not exactly what motivates a developer to stay active. I don't know why many of you seem to think that there are only new devs involved in the remake. It was my idea to do a rewrite. I think I know the "old" source code and its problems quite well.

It may take some time. I'm in the middle of my thesis and final exams, therefore I am not exactly available. Let's face it, RttR is something we contribute to in our leisure time. There are things more important than RttR and RttR will have to wait. There will be progress, but rewriting is a lot of planning and work. We do not want to make design decisions that we will regret later. I'd rather wait another five years for something really working than do everything as quickly as possible and afterwards realize that things just don't work that way. That would just be history repeating itself...

---

Nur mal angenommen es käme jetzt einer auf die Idee wirklich ein Kommerzielles Produkt daraus machen zu wollen, denn dann würde ich dafür keinen müden Euro ausgeben. Warum auch ? Zur Not spiele ich halt wieder das Original via Dosbox, denn das habe ich schon gekauft oder halt das momentane RTTR für die schönere HD Optik. Generell finde ich die geldgeile Einstellung mancher gerade zu abstoßend. Zudem wäre das sowieso nicht möglich, denn keiner von Euch hat die Rechte an dem Spiel. Bluebyte würde Euch was anderes erzählen. Man kann doch froh sein, das man überhaupt einen MOD (Was es in dem Fall nicht ist), wenn man das so ausdrücken kann, dafür machen darf. Denn selbst das ist mittlerweile schon in vielen Spielen verboten. Kann ich natürlich auch verstehen. Wenn jetzt mehrere so denken und sich ne goldene Nase mit der Idee/Gerüst eines Spiel verdienen wollen und lediglich ein paar Skins austauschen oder es halt nur neuschreiben und die alten Grafiken verwenden zum Beispiel ist das durchaus nachvollziehbar. Ich würde das auch nicht toll finden ;)

Alle Dinge die ich bis jetzt für die Allgemeinheit getan habe, waren alle umsonst. Ich habe nie etwas dafür verlangt, auch wenn es wirklich lange gedauert hat. Ich habe zwar keine Ahnung von C++, dafür aber in anderen Scriptsprachen. Nein, mich könnt Ihr trotzdem nicht für das Projekt gebrauchen :P Und ich sags auch direkt heraus: "Selbst wenn ich es könnte... ich bin einer, der energisch an einem Projekt sitzt bis es fertig ist. Und Jahre kämen für mich nicht in Frage. Vorallem wenn mehrere Coder daran arbeiten... Dann müsste man wieder auf irgendeinen warten, weil er noch nicht fertig ist oder keine Lust hat oder oder und das ginge mal garnicht". Es gibt auch einen passenden Spruch dafür: Viele Köche verderben den Brei." Und seien wir mal ehrlich, das Chaos so wie Ihr es ausdrückt was in dem Code wohl herrschen soll ist doch darauf zurückzuführen, eben weil mehrere daran gearbeitet haben. Einige hatten wohl weniger Ahnung, andere mehr und genau das ist auch schädlich. Das ist meine Meinung dazu. Jeder hat einen anderen Schreibstil.

---

+1 Maqs

As you said some of you are afraid that it will take a long time until something really playable is aviable or something
but the point is: maqs wants to do everything modular means you can add addons easy and change things when they are
finished quite fast.

Since I know that also wow Windows are made with XML (since I'm learning something about LUA for RttR²) I also think using
this way for windows would be really great. Still, I would rather wait for a working project than extending one that wont
work correctly. Maybe Wusel you don't know it but really BIG changes haven't been made since 0.3 I think and thats already
7 years ago. Everything later was mostly quick and dirty (as far as I can judge). So, you had to wait nearly 10 years for
the current version which is not really great - means when maqs starts his rewrite (and I think he will do it anyway) you
may also have to wait, 3 - 4 maybe 7 years until something is finished but when it's ready, it will be already better than
RttR could ever be I guess.

Means I also have not that large knowledge of C++ or any other language but I help him as good as possible. If somebody is
still not sure if it'S better to fix RttR or to rewrite RttR he may should think about that part first and then start to
change RttR or help maqs with RttR² - just my opinion.

That's what I do, I stopped working on the babylonians or any other RttR related project to contribute graphics for RttR².

---

---

@Wusel: Don't know why you are complaining about money-hungry bastards. Nobody was talking about any plans of taking money for a s2-remake. We were doing it for free and always will be.
The point with many devs will make it worse is a real point though. However not necessarily. There will always be good devs and better devs. But everyone can learn from each other (yes, both ways) and someone might have solutions for problems the others can't think of. In the end: Many hands make small work! We need lot's of devs if we want to get something done (refactoring OR rewrite, although I think it will be the rewrite) because it's a load of work.
For ETAs follow CyanogenMod: It's done when it's done! ;)

As mentioned: The rewrite is the currently best solution as using the current version would mean lot's of refactoring that break a lot of things for some time. And in the end it will result in almost a rewrite anyway. But the point is: It is almost impossible to contribute to rttr with the current infrastructure.
This might have been one of the problems that the project took that long to get to the current point. A real opensource and free2contribute version is necessary. Although "gatekeepers" will be a good idea. But plural!

About language: Yes it is a project made by Germans (mostly) but writing everything in German limits the number of devs. And as you will see if you search the forums: There are many international players here...

It would be great to spread the word and get more devs in. With good work distribution and high usage of the Repos with frequent commits it is possible for about 6-8 active coders to work on the project and some contributors.
I also like to get things done ASAP but that's only possible with a lot of hands.

Last point: Crowd funding is not a good idea as it makes it commercial. However one could think about donations to get a good build server, web site etc. It would be great to have the "old" team with us to, but that would need some more rights for a couple of people. Otherwise we have our hands bound...

---
Github: https://github.com/Flamefire

---

Naja, ich wünsche Euch jedenfalls viel Glück dabei. Was war eigentlich mit der Seite los ? Gestern oder Vorgestern ging die eine Zeit lang nicht.

Schönes WE.

---

Wusel, das soll jetzt kein Flame werden, aber du machst dich ziemlich zum Affen hier.

Zitat von Wusel:

Jau, abstoßend und ekelhaft, für jahrelange Arbeit bezahlt werden zu wollen. Abgesehen davon hat hier
niemand vorgeschlagen, RTTR zu verkaufen.

Zitat von Wusel:

Die Rechteinhaber haben keine Möglichkeit, Moddern und Anbietern von 3rd-Party-Software die Entwicklung
zu verbieten. Es ist übrigens DIE Mod.

Zitat von Wusel:

Lass mich raten, das war alles unprofessionelles Zeug, das du an einem Wochenende hingeklatscht hast.
Du Superheld.

Zitat von Wusel:

Oh man was eine gequirlte Scheiße. Siehe oben, du bist vermutlich ein Kiddie ohne Ahnung von
professioneller Softwareentwicklung. Was meinst du, wie heutzutage Software entsteht? Nein, nicht viele
Köche verderben den Brei, sondern bei einem ordentlich geführten Projekt ist jeder zusätzlicher
Entwickler eine Bereicherung für die Repo.

Nur weil deine Arbeit als (vermutlich noch jugendlicher) Autodidakt nichts wert ist, hast du nicht das
Recht, dich derart anmaßend und dennoch uninformiert über die ausgezeichnet arbeitenden Jungs von RTTR
zu stellen. Ich verstehe gar nicht erst, wie du auf die Idee gekommst, dass deine Grütze hier irgendwen
interessieren würde oder ein Beitrag zu diesem Thema wäre. Verpiss dich bitte einfach.

---

Stimme da zu wobei der Tonfall zu wünschen lässt. Ergänzen möcht ich bitte noch:

Wegen der Gelddiskussion: Also für die Entwicklungsarbeit - warum nicht? Nochmal das Stichwort: Crowdfunding.
Manche verkaufen sogar Siedler 2 Addons: http://www.jnsoftware.de/Siedler2/home.htm Das müsste nicht unbedingt sein.

Es wurde auch DosBox als Alternative angesprochen. Dazu möcht ich sagen dass es mit den heutigen Bildschirmen ned angenehm ist bei max. 1024x768 zu spielen. Die Möglichkeit die Auflösung anzupassen ist für mich persönlich der Hauptgrund für rttr.

---

Zitat von Gast:

Sieht aber für mich ziemlich nach einem Flame aus. Zusätzlich noch als anonymer Gast. Kein guter Stil.

Zitat von Gast:

Ist das wirklich rechtlich so eindeutig? Ich dachte Mods bewegen sich allgemein in einer rechtlichen Grauzone und sind abhängig von der Gutmütigkeit bzw. Gleichgültigkeit der Hersteller. Gibt es da verlässliche (rechtliche) Quellen?
Auf RTTR bezogen haben wir lediglich die mündliche Aussage von BlueByte, dass es ok ist, solange zwei Bedingungen eingehalten werden:

1) Es darf nicht den Anschein erwecken, dass BlueByte irgendwas mit uns oder dem Projekt zu tun hat. Daraufhin wurden hier auch alle Vorkommnisse des Namens "Die Siedler 2.5" getilgt ;) "Die Siedler" ist ein Markenname von BlueByte.

2) Es darf nichts von den Originaldateien "mitgeliefert" werden.
Wenn mich meine Erinnerung nicht trügt, war da sonst nichts.

Zitat von Gast:

Man sollte bei diesen "Aussagen" daran denken, dass die aktuelle Code-Basis eben zum größten Teil von einer Person ist, Oliver. Und als Oliver angefangen hat, war er (mit deinen Worten) "ein Kiddie ohne Ahnung von professioneller Softwareentwicklung". Zusammen mit Flo hat er wohl >90% des Codes geschrieben. "Bergab" ging es eigentlich erst als Oliver sich mehr und mehr zurückgezogen hat und immer mal jemand anders ein bisschen am Code gewurschtelt hat. Mal mit mehr, mal mit weniger Ahnung.

Also, Wusel hat in einigen Punkten sicher nicht ganz unrecht.

---

http://www.golem.de/news/echtzeitstrategie-0-a-d-sammelt-spenden-fuer-vollzeitentwickler-1310-101900.html

---

Es ist schon immer wieder interessant, wenn Leute Vermutungen anstellen und total daneben liegen. Am besten hat mir aber der Teil gefallen, wo es hieß, ich sei ein Kiddie. Hmmm, wenn ich ehrlich bin höre ich sowas meist nur von Leuten, die selber noch welche sind. Ich bin 28 Jahre alt und ganz bestimmt kein Kiddie mehr. Nur weil wir zwei eine andere Meinung haben, finde ich die Art und Weise wie du dazu Stellung nimmst nicht berauschend. Denn auch wenn du schreibst, du wolltest hier jetzt nicht flamen: Du tust es. Und weißt du auch warum ? Weil du deine Argumente mit Beleidigungen schmückst. Das ist Flamen ;) Hättest du jetzt sachlich wie alle anderen auch dazu Stellung genommen hätte man dir das glauben können, das du nicht flamen wolltest. Im Endeffekt ist es mir auch egal, was jemand über mich schreibt, denn ich kenne denjenigen nicht und er mich auch nicht. Trotzdem wünsche ich dir noch ein schönes WE.

---

Hi,

I'm not sure what the current state of the rewrite idea is, but please take the time to read this (if you do not already know about it): http://www.joelonsoftware.com/articles/fog0000000069.html

I did not have a look at the source code yet, but I think it is rather risky to start from scratch. Refactoring can be painful, yes, but when I look at how much effort has been put into RTTR over the last years, it would just be a waste of precious developer time.

You have built something great here! Don't rush things.

---

Hi,

ich weiß auch nicht, wie weit eure Rewrite Idee gekommen ist, aber ich würde mich auch dagegen aussprechen wollen.
Die Idee die Teile, an denen es momentan hakt, sauber neuzukompilieren halte ich für besser. Allerdings ist mir natürlich auch klar, dass dies hier freiwillige Arbeit ist und bei einem mühsamen Einarbeiten eine Menge Zeit verloren geht.

Andererseits würde ich auch sagen, dass man mit einem Rewrite wieder bei null anfängt und noch dazu alle Bugfixes verliert, bzw. neue Bugs erzeugt. Und dies kostet meiner Meinung nach mehr Zeit, als die sorgfältige Umarbeitung des aktuellen Codes.

Soviel zu meinem Senf.

Grüße,

---

Just maybe the hole concept of having to use the original files is not the most efficient way of creating a more "nowadays-pc" version. i mean if you want to play something close to settlers2, you could play widelands - like wormux.
if you want something way closer there is only rttr but you still need the original files -- unlike hedgewars which is a very damn close copy of worms 2/armageddon (let's face it: as close as one could possibly get).
I admit that I have no idea what soever about programming but using an thereby rely the old files might not be the most efficient way to do it. the very commit team around the new people (babylonians) has proofen that there is enough manpower here to develop different peoples for the game so why not create an independent version and maybe improve the graphic a bit. doesn't have to be a 3d-version like in the 10th anniversary edition from a few years ago...
the damn security code picture of this freakin' self written forum is giving me a very hard time now (can't decipher and there is no reload button for a new graphic) - why not use a standard solution like phpforum?

---

Close to s2 and widelands in one sentence is just wrong. The only thing that is close to s2 is the way you build buildings in
my opinion. Also is ottd a Game which runs good with the original files and an open gfx. The main problem of rttr was,that
two people wrote nearly everything.

The "team" around the Babylonians is also small, two people at all and one is completely inactive. Also in fact I was working
since now 3 years which is a long time and the main part is still missing. If everything was more open since the beginning
and not "only German" for the first years, maybe then things could have been different.

Well,the hp, you may see our winter header graphics which were first made around a year ago, implemented this year
February and since then we got those graphics - not regarding if summer or winter.

That's the Main reason why maqs had the idea of a rewrite. More people who are active and easy access to code.

---

---

I had a look at the code some time ago, because i wanted to help with bugfixing a little bit (by running the game in debug mode). However, i have to admit, it was an annoying barrier for me to use bazzar without access to the mirror SVN respositories. It's understandable that the bug-reporting features are nice to have, but not to the costs of having less active developers in my opinion. Also, github has bugtracing today.

In case some other developers are interested to perform the rewrite which is under disscussion here, i'm happy to join. Anyway, then we might discuss the programming environment before start programming to avoid former mistakes. Some "project managment" might not hurt (small roadmaps and task distribution for motivation purpose), without overdoing it.

---

I agree. The Launchpad/bzr vendor lock-in effects work for Canonical/Ubuntu, but it is poisonous for you here. Even the
internally used svn is not state of the art nowadays. The best way to get more patches and contributors that help you get
things fixed and improved is to switch to git. http://www.youtube.com/watch?v=qT90jZP58jM ;)

---

I don't get why "git" should solve all problems... it's just "Uh git is soooo cool, I would contribute
something just to use git" just my thougts about this. Not regarding, that there already is a git version and
nobody ever was interested.

---

---

I currently package rttr as RPM in openSUSE and the bzr overhead is terrible. The initial checkout took ages and required
lot's of bandwidth. Would love to grab it from https://github.com/Return-To-The-Roots/Return-To-The-
Roots/archive/0.8.tar.gz instead, but that repositories looks dead like an unused one-time SVN import that is not supported
by the development team.

Actually git is the new de-facto standard in the Open Source community. You are not forced to use it sure, but everyone
else does. See a talk by Linus Torvalds http://www.youtube.com/watch?v=4XpnKHJAok8 about git.

---

So there's one version on github? Didn't know that ...
Sounds bit wired to me, changing from subversion/bazaar to git (at least if you want to maintain the history), but if it worked, good job! Is there any developer who can update the git-repository to the current nightybuilt?

Also, a tiny introduction into the source code might be useful for me. Last time i tried to get into it, it's been quite difficult, i have to admit (and problably forgot most stuff until now). An outline of the rough structure would be helpful/save me some time.

---

Also um hier auch mal kurz meinen Senf dazuzugeben. Die Idee ein Remake des Remakes zu erstellen, scheint ja ganz lustig zu sein, aber ich möchte ja gerne diejenigen sehen, die das wirklich durchziehen. ;-)

Wie schon jh richtig angedeutet hatte, war die ganze Aktion damals sicherlich nicht professionell geplant worden und vor allem nicht darauf ausgelegt, beliebig erweiterbar zu sein und für Leute aus aller Welt offen zu sein. Damit hatte damals auch glaube ich keiner (zumindest ich nicht) gerechnet, dass es plötzlich auch viele Leute aus Polen, Tschechien, Finnland usw. spielen oder dass Leute Babylonier und Erdbeerfarmen einbauen wollen. Es ging lediglich darum, eine weitestgehende übereinstimmende Kopie des Originals mit einem Multiplayermodus und einigen kleinen Verbesserungen zu erstellen, wobei Verbesserungen/Erweiterungen dann eher so einschneidend waren, dass Modularität hier nichts bringt, z.B. erweiterte Schiffsfahrt, Handel etc.

Da sollte man sich halt fragen, was man eigentlich genau will. Will man nur die bisherigen akuten Probleme beseitigen (Asyncs, Performance-Probleme), kann man das denke ich auch in dem aktuellen Code tun. Will man dagegen eine Art vollständig modularen Baukasten a la Widelands, wird es wohl bedeutend aufwändiger.

Ich weiß nicht genau, was die Quelle der Asyncs ist, aber wenn ich mich nicht falsch erinnere, waren diese "früher" (also damals, als ich noch wesentlich beteiligt war) nie ein so gravierendes Problem und wenn es welche gab, ließen sich diese immer noch irgendwie lokalisieren. Deswegen weiß ich nicht, ob das wirklich ein Problem des Designs von meinem alten Code ist (wie die von Marcus angesprochenen integer-Größen usw.) oder ob eher Veränderungen in der "neueren" Zeit nicht eher eine Rolle spielen, sicherlich zugegebenermaßen auch durch die eher schlechte Erweiterbarkeit des Codes.

Was die ganzen Erweiterungen betrifft, würde ich einfach in Frage stellen, ob diese wirklich notwendig sind. Also sprich ob es wirklich so wichtig ist, 6 Soldatenränge oder neue Terrainarten einzuführen, ob man geskriptete Missionen braucht (Wer will das spielen?) usw.

Um nochmal auf ein paar Punkte konkret einzugehen:

Zitat von Marcus:

Damals wollte man glaube ich die STL umgehen, da diese eben als vergleichsweise langsam galt und die executables aufblähte. Ist natürlich angesichts der jetzigen Entwicklung des Projekts vollkommmen übertrieben, aber ich denke, die Listen zu ersetzen, wäre jetzt kein so großer Aufwand, jedenfalls rechtfertigt das keinen Rewrite. ;)

Zitat von Marcus:

Ich halte Multithreading einzubauen für eher übertrieben. Wenn man das so amateurmäßig einbaut, würde das wohl die Bug-Büchse der Pandora öffnen. Wie schnell willst du das Spiel dann laufen lassen? Dass der Mauszeiger sich langsamer bewegt, liegt dann wohl eher an einer falschen Ansteuerung der GUI-Bibliotheken.

Wenn Multitrheading, dann vielleicht nur den Server und die KI, das fände ich noch halbwegs realistisch.

Zitat:

Da Siedler kein Shooter ist, also ohnehin auf (fast) alle Kommandos der Spieler verzögert reagiert, würde es wohl genügen, diese Verzögerungen entsprechend zu verstecken. Bei dem Wegebau ist das ja schon eingebaut worden. Im Prinzip fehlt es nur noch bei den Flaggen- und Gebäudebau? Oder gibt es da noch irgendwo ein Problem?

Zitat:

Ja, die festen Werte, die noch im Code rumschwimmen, müsste man finen. Aber das halte ich jetzt nicht unmöglich. Ansonsten ist das halt wie gesagt eine Aufwand-Nutzenrechnung. D.h. lohnt es sich z.B. ein LUA-Skriptinterface für Gebäude, Animationen etc. zur Verfügung zu stellen, um dann nur eine Köhlerei und eine Erdbeerfarm einzubauen?

Zitat:

Ja, sicherlich ganz nett, aber das wäre denke ich auch nicht so viel Arbeit, das im aktuellen Code zu verändern.

Zitat:

Ja, hier könnte man einiges tun. Aber letztendlich muss man das sowieso zusätzlich zum aktuellen Loader-Code irgendwie machen, da uns verboten wurde, die Originaldateien mitzuliefern.

Zitat:

Nunja, hier ist wieder die Frage, ob man für solche Sonderfälle wie das Beobachtungsfenster ein neues Windows-API erstellen will, oder ob dann nicht doch ein kleiner "Hack" genügt, zumal ja der Großteil der GUI im Prinzip fertig ist?

Zitat:

Und wie willst du genau garantieren, dass "kein Pixel doppelt angefasst wird"? Der Großteil der Grafiken ist ja nicht rechteckig?

---
Warum heißt der Staatsbürger "Staatsbürger"?
-> Weil er für den Staat bürgt.

---

Uh, Oli hat was gesagt, dann muss ich aus Spielersicht etc. auch was sagen bzw einfach was ich mitbekomme
damit man alle Aspekte hat :)

Zitat:

Also ich weiß wie viele (gut ich weiß es nicht, aber es waren einige) versucht haben die Asyncs zu fixen -
und es hat niemand geschafft, vielleicht fehlt dann einfach das genaue verständnis vom code, den du damals
geschrieben hast, ich weiß es nicht. Ich weiß nur, dass Asyncs Problem #1 von RttR im Moment sind. (#2 sind
abstürze und #3 performance) Wäre natürlich unheimlich geil wenn das irgendwann gefixt würde (wovon aber
denke ich leider niemand mehr ausgehen kann).

Zitat:

Meiner Ansicht und meinen Ideen nach: Ja 100% lohnt es sich

Zitat:

Man braucht a) einen guten PC und b) eine gute Verbindung. Ping 200+ sorgt meist schon für massive lags die
das spiel beeinflussen und worauf man nicht wirklich lust hat. Auch ist die performance wie du ja schon
sagtest nicht so gut, dass man riesige karten spielen könnte und wenn man dort über einer industrie ist, hat
man schon mal nur noch so 20-40 fps auch wenn man sonst 1000 hat und dann lagts einfach auch wenn nix
passiert - das ist ärgerlich.

Abschließend will ich nur dazu mein Gefühl äußern: Ich glaube, es gibt eben viele, die gerne etwas machen
würden aber es entweder nicht können weil es schwer ist an den Code zu kommen, keinen Spaß daran finden sich
in Code einzuarbeiten wo alles irgendwie zusammenhängt und man nichts finden oder, und das ist wohl der
wichtigere Punkt meiner Ansicht nach, eigentlich nie irgendeinen Verantwortlichen mehr zu sehen und dadurch
fast sicher zu wissen, dass Änderungen eh nie in die Nightly gelangen und getestet werden.

---

---

Absolutely right. Even if there were people wanting to contribute anything: They simply don't want to do it anymore as it is to hard to get changes in the code. Putting all of that stuff to GIT would help.

Maybe it is possible to fix the current code. Assuming that it is run on non-ancient graphic cards and switching to a newer SDL version would give quite some performance.

Also multithreading will give some more performance if you split some parts off (KI, server, networking...)

There are a couple of possibilities to fix the asyncs or at least find them. But we need a code cleanup and comment/docu translations first. Who is willing to do that? ;)

I consider extensions useful but only to a certain point. E.g. a better map format that allows more terrains, or better buildings with descriptions of their data (xml,...) If there were to many: Just switch to widelands, create the races, buildings and materials there and you are done ;)

---
Github: https://github.com/Flamefire

---

Zitat:

I would guess, that adding some more features would be really
cool - like adding goods, controlling a ship, starting events
(controling normal map objects etc.)

I still ask myself if you could fix the babylonian code that we
can add this race in a build to gain more popularity I guess :>

---

---

Hmm, maybe Babylonians + http://settlers2.net/the-settlers-2/texture-set-new-greenland/ and you have
enough for a basic free replacement graphics to make this work as a standalone like OpenTTD?

---

We would not even be close, thats maybe 5-10% I guess

---

---

Like Matthias said, it would be great to have a complete pack of textures (world, settlers, map-textures) to be able to give out a package without the need of the orignial files.
If there is nobody to build that in the game, maybe the first step could be to have a DATA + GFX pack with the new race map and setlers as a fall back option for all people without the original game.
What about the menu grafic I think its also the original one, but it could be enougth to just do some kind of standard windows 98 like buttons.

---

I think the special feeling of RTTR comes along with the usage of the original files.
So I think using new/own buttons will destroy a part of this feeling.

And honestly, it is no problem to get a copy of the settler II game disk. Amazon, ebay, Saturn and how they are all called still have gamecopys of this game. So there is no need for changes to get the game more like OpenTTD.

---
http://www.danta-spiele.de

"Der Mensch spielt nur, wo er in voller Bedeutung des Wortes Mensch ist, und er ist nur da ganz Mensch, wo er spielt." - Schiller

---

Well - I guess adding a way to use default files to be independent would be great, there was also a small
attemt of creating all thos files but... well https://code.launchpad.net/~rttr/rttr/graphics

---

---

I know that it´s still easy to get the original game.
But it could be both, to be the new race and the possibility to be independent of the original files.

---

Sorry to open up this thread again, but I'm at the moment reading the code and feel like sharing the current perspective of an outsider attempting to get into development...

* I would not go as far as to radically try and rewrite everything, but rather incrementally improve piece by piece in a fork->stabilize->backmerge fashion. This ensures still the best of current status for the players, as well as takes the codebase a step closer to something actually nice to look at.

Problems I encountered so far that need addressing to make it possible for people to contribute;
* Translate code comments.
* Get rid of over documentation and sparsity (huge headers for nothing since they basically only repeat the name of the function/method)
* Get everything into a uniform and convened upon shape. Suggest going for a slightly modified code style directive of idTech4 code base (its just beautiful to look at)
* Cleanup the whole build environment... its set up like you are expected to be wiring stuff around before getting anything build. On every platform it should be nothing but a simple; "cmake ." no special out-of-build-stuff.
* Cleanup preprocessor directives... they make parts of the code really unpleasant to look at. Encapsulate in interface to the different OS'es and take the one you're building against at build time. Should keep out 99% of macros from anywhere.
* Get rid of the usage of global state. Singletons are not a feature, they provide an escape for bad architecture in many cases... like GameManage... who would instantiate twice or access it from below top level? Its the Game Manager that manages the game not the other way around.
* Own types for Directory and String (to use OS interface)... more pruning of ugly macros.
* Internationalization should be supported on single binary contrary to compile time... there is no way to handle a build per language without getting tired of it and messy along the way of automating.
* Debian and derivate packaging must be agnostic of anything from automated builds, over structure, to build process. No jenkins changelog versioning... I'm running under Debian for instance and had the pleasant experience to find a combination of quilt-native format + versioning like: "20140724-9485.339". It just does not work and having automated changelog updates defeats its purpose in the first place.
* Get rid of custom types that could come from the stdlib. There is a lot of stuff reinventing the wheel.

These are the ones I want to tackle right away. Would be nice to have testing support for cross-platform compliance, since getting at the build system and OS specifics is bound to break things across the landscape. Also I don't have much extra time and experience to take care also for the Windows and Apple builds. If successful I promise though to seriously attempt to get an Android build going.

Further down the line;
* Purge entities from hard coded behaviour... maybe externalize to Lua? Saw the lua lib headers in the source tree. Btw there are specific lua headers for C++ no need to work around to get the C definitions into the C++ codebase.
* Maybe going hand in hand with (1), make entities agnostic,

Things I need to figure out short to mid term:
* What basis does the game tick work on.
* Is the game state deterministic?
* Is UI running separate from Game loop?. At first glance they are hanging together.

Thoughts on Multithreading;
* Not a bad Idea but only if you can segregate into subsystems and thread safe well defined data structures in-between. Otherwise things get really messy, really quickly.
In case of the UI/Game segregation for instance it does make perfect sense. Or between Game and Networking. Don' know though if segregations like this already exist though... just running my mouth on this one ;).

Thats it ...for now... :D
LG,
~Arlias

---

Zitat:

Just time. A game tick = game frames (GF) length is basically a set time for whatever game speed you choose, in 10ms increments. Normal speed is 40ms, fast 30ms.

Additionally, there are network frames (NWF) happening every nth GF (n depending on game speed), its only during those the user interacts with the game simulation (=game commands (GC), e.g. "player P places building B at coordinates x, y"). Network play works by sending each players GCs and executing them during a specific NWF.

Zitat:

Its supposed to be, and mostly it is. If it isnt, you're getting those dreaded asyncs.

Zitat:

They're not seperate, RTTR is completely single-threaded: game logic, UI, game server, AIs as well as networking are handled sequentially in the same thread.

---

There's already a large internal list (@Maqs) for a whole (multithreaded) rewrite of rttr.

---
mfg
Flo

---

Cool. Any chance to have a look at it?

---