my thoughts about engo and systems as packages

[scottlawsonbc]

I recently learned about EngoEngine and had a chance to spend a few hours reading the code.
It was a fun experience and I learned a lot about how EngoEngine is implemented. I also had some ideas that I would like to share and get feedback on, especially from those who know a lot more about the EngoEngine internals than myself. I'd like to use this issue as a space to share some of my thoughts about EngoEngine, and I also want to highlight some areas of the Go standard library that may be of interest to EngoEngine developers. Please share your feedback by commenting on this issue!

For context, I am not a game developer but I am interested in game engines. I am a mechatronics engineer with a background in physics; keep in mind that I am missing a lot of knowledge about game engines and have an outsider perspective.

Things I really like:

• Installation. No problem setting up dev environment to run EngoEngine. Minutes vs >1 hour for Unity.
• Repo structure. I could understand at a glance how the engine is structured.
• Tests. It was reassuring to see a lot of tests, especially for some really important functions like TMX loading / saving.
• Documentation. There's a website and I can read tutorials and see code examples and explanations. Awesome!
• Demos. Handy collection of demo programs doing different things. I was able to learn a lot from context by going through the demo programs.
• Coherent source files. Code related to ECS and TMX loader (and others) have no dependencies on the rest of the system. Individual source files have high cohesion and loose coupling.
• Composition. Go structs and interfaces can be composed and the EngoEngine implementation of ECS takes advantage of this in a way that feels very natural in Go. In particular, structs feel very natural as entities.
• Simple. So simple even me, someone who isn't a game developer, could read through most of the codebase in a few hours and understand it at a general level.

Pain points:

• common is well organized but does too many things for one package. Perhaps common could wrap multiple subpackages. TMX code could be a standalone subpackage, for example.
• Entities are elegantly modeled as structs but systems feel awkward.
  • System.AddByInterface is an improvement over System.Add, but they both feel a bit awkward.
  • Most systems (>75%) store entities, but the minority which do not must still implement System.Remove.
• EngoEngine interface names are sometimes confusing and unidiomatic. Animtationable[sic], Drawable, Mouseable, Audioable, NotAudioable, NotRenderable, NotCollisionable. Although these names may be idiomatic in another context, such as Unity, they stand out as particularly unidiomatic in Go. Consider using Drawer instead of Drawable, Animator instead of Animationable, Renderer instead of Renderable. Consider inferring whether something can collide by testing whether Collider is implemented, instead of implementing two separate interfaces Collisionable and NotCollisionable. My overall impression is that interfaces are powerful but EngoEngine isn't using them to their full potential.
• Goroutines and channels are powerful. EngoEngine doesn't utilize them to their full potential. Systems are not concurrent and communicate sequentially with the engine. Systems communicate with each other by writing events to a custom mailbox implementation. Could systems be goroutines that communicate by sending messages over channels instead?

The awkwardness of systems leads me to wonder about different ways ECS can be modeled in Go.

In ECS, entities are data and systems are data transformations (behaviour).

In EngoEngine, entities are structs and systems are structs which implement a System interface. Systems do two things:

• Manage a list of entities that the system is interested in.
• Transform entities.

In Go, a struct is a sequence of fields. A field has a name and a type. Two structs are comparable if they have the same sequence of fields. Fields are components. Structs are entities. Structs with different fields are different entities. Structs with the same fields are the same kind of entity.

In Go, a type may have a method set associated with it. The method set of an interface type is its interface. The method set of a struct is the set of all methods declared with receiver type of that struct. Is a method set the behaviour part of a system? The method set associated with a struct (entity) feels to me like a really important part of systems, but another piece is missing.

Other thoughts:

• Imagine if each system was a Go package. Systems could register themselves using a plugin mechanism similar to how the Go SQL driver works. Go analysis package is another example of an idiomatic plugin registration system using go packages.
• Packages are singletons, meaning if packages are systems then only one instance of a system can exist at a time.
• Packages initialize themselves. Structs don't initialize themselves. If packages are systems then there is no need to call NewCustomSystem(), the package (system) will already have been initialized when you import it.
• Packages have clear dependencies which can be inspected by looking at the import statements in a package. If packages are systems, inter-system dependencies would be explicit and easy to understand, and circular dependencies between systems would be impossible.
• Packages are inherently modular and pluggable. So are systems.
• Analysis tools and static checkers can automatically verify whether packages adhere to custom naming / method conventions. You could say "a system is a package which has an update method" and then write a simple go tool which verifies at initialization time that all imported systems meet this requirement.

In summary, key benefits of modeling systems as packages:

• packages initialize themselves
• clear dependency relationship by looking at import statements
• circular dependency impossible
• guaranteed to only have one instance (singleton)
• packages are uniquely identified by name
• packages can have versions
• packages easy to inspect using built-in go analysis tools

[Noofbiz]

Hello! Thank you for your interest in engo! A lot of your points are 100% spot-on. We have issues currently about optimizing systems by running Update on their own loop. That would considerably speed up the update loop.

The AddByInterface stuff makes engo act like a normal ecs setup, but the issue is that we store the entities inside the systems instead of storing them at the world level. I believe the choice was made because that was much more performant than storing all entities in the world and letting systems filter them.
The CollisionAble and NotCollisionAble are necessary because by default engo does assume if it implements CollisionAble it’ll go in that system, so if you do implement everything in CollisionAble but for some reason don’t want that entity in the system (it’s kinda just for rare edge cases), then you’d use NotCollisionAble.
If we wanted a more traditionall ecs setup where the world only passes on the entities to the systems, it could still be done with the way engo works relatively easily, the world would just have a map[System]entities that it uses to pass the right entities to the systems every time without the need to filter every update (yay speedy and effective!).

The Systems as Packages part is an interesting idea. I think that could be a cool way of implementing Systems! You could just call in the system’s init method something to add the system to the world, as well as set things up. I’ll definitely think on that, but that’s a pretty neat thought!.

[Asday]

I'm currently knee-deep in understanding DoD, but from what I understand, the most important part of ECS is that components are iterated over in L1 cache branchlessly. This means no pointers (as they generally necessitate a cache miss), and it means pre-sorting of data. Here's the book I'm reading most attentively.

Given the above, I'm surprised to hear that the way Engo suggests doing it is less performant, but then there's a lot about DoD that I'm still thinking hard about to try to understand.

[Noofbiz]

The way engo does it is performant, and even follows ecs (what difference is there, really, between the world maintaining a list of entities vs a system keeping them itself), it's just not standard. The only issue is doing it a true separation, where the world just passes entities based on some filter function, wasn't as performant as doing it the way it ended up.

[Asday]

The difference is that the System keeps only pointers to the data, in my opinion. Obviously things get confusing if they keep the actual data because, say, which System gets SpaceComponent?

My biggest sticking point on it all so far is that while a given system can guarantee linear access to one Component, how does it deal with matching components? I had an epiphany at some point and shouted out loud in real life in my bedroom "it's .select_related()!" But that didn't really get me any closer...

[scottlawsonbc]

Thanks for your thoughts! Really interesting to hear some of the reasons for the design decisions, it helps me understand why it is structured as it is.

Overall, I feel like this engine has a lot of potential. The ECS system as implemented here makes use of some of the unique strengths of Go, like struct composition.

I also wonder what roles goroutines might play in the future of EngoEngine. In theory, channels and goroutines should be able to do as much if not more than the current event message system. I feel this area is a bit unexplored as far as game engines in Go are concerned.

[airtonix]

Consider using Drawer instead of Drawable, Animator instead of Animationable, Renderer instead of Renderable. Consider inferring whether something can collide by testing whether Collider is implemented, instead of implementing two separate interfaces Collisionable and NotCollisionable. My overall impression is that interfaces are powerful but EngoEngine isn't using them to their full potential.

A Naming BikeShed !!!! this is someting i can get behind 😆

👍🏻

• Drawable
• WithAnimation (or just Animates)
• Renderable

The AddByInterface stuff makes engo act like a normal ecs setup, but the issue is that we store the entities inside the systems instead of storing them at the world level. I believe the choice was made because that was much more performant than storing all entities in the world and letting systems filter them.

problem with this is that now we have to manually manage adding and removing entities from all the potential systems it might be a part of.

I came here from ExcaliburJs, where systems have a list of component types that entities must have in order to be operated on. Other frameworks I've used(like ecsyjs) have a more expressive query language.

If systems filtered entities based on which components they had, then we would avoid more tight coupling.