Disclaimer 2: This is not a tutorial about writing Rust in any way - I am just sharing the experience of using Rust for the first time in a professional environment.

At my current job, my work oscillate between C++ and Typescript. Then we have this new project that came, fairly self-contained, with clear specifications and relatively simple - we decided to jump on the occasion.

My team was/is responsible for this project, but we synchronized with 2 other teams working on low level software as we all had an interest in trying Rust.

A word on the project

The project was pretty simple: We needed to develop a library (to be loaded in a huge C++ Software), that would launch a WebSocket server to accept connection from multiple clients. This library should forward all messages coming from the clients to the embedder and allow the embedder to send messages either in broadcast to all clients or only to a specific targeted client.

The software is running on embedded devices with low resources so performance is definitely a critical component. Finally, the server should allow to be started and stopped when the embedder sees fit.

A quick search returned that there are 3 WebSockets libraries in Rust that seemed to be "active enough" and with a few stars on Github:

• websocket: https://crates.io/crates/websocket
• ws: https://crates.io/crates/ws
• embedded-websocket: https://crates.io/crates/embedded-websocket

So it looks like we are covered on that front. Making a library that can be loaded from C++ (from C actually) is "trivial":

In the cargo.toml you just add this:

[lib]
crate-type=["cdylib"]

And in your rust code, you make callable function pretty easily:

#[no_mangle]
pub extern "C" fn hello() {
    println!("Hello World!");
}

In the C++ code:

extern "C" {
    void hello();
}

int main() {
    hello();
    return 0;
}

As you guessed, this will print our famous Hello World!.

Ok, Rust seems to fit all of our "requirements", let's see how it went!

Not quite there yet

It was actually easy (contrary to popular believes) to get something pretty quickly working and passing all the tests and requirements for this project. It took less than a week to get that working, and integrated into a sample C++ program mocking the functionality of our production code.

However, getting it working wasn't all - We want the software to run on hundreds of thousands of devices worldwide. We don't want it to break, cause problems and be difficult to maintain.

Our initial solution contained a thread to run the WebSocket message pump independently from the rest of the software. Fundamentally, this isn't a bad idea, but we have - at Airtame - the rule that "if you can do it without a thread, then do it without a thread". I will not go into the details of the why, but let's just say that threads bring their own set of issues that are more difficult to reason about and tends to make software more complicated.

Ultimately, most of us come from a background working with C, so we all were thinking the same thing: can we get something like the select function, so we can have our own loop and handle the incoming and outgoing messages whenever we like? The answer is yes, kind of. The libraries that we tried don't allow that by default, but a couple of them allowed to plug our own loop handling. Great! But digging in that direction shown that it was actually not as simple and straightforward as one would have thought.

Finally, have the Rust library share some data with C++ is unsafe. Rust is amazing at memory-safety, thread-safety and the likes by removing those issues at compile time. All those advantages vanish when passing data between C++ and Rust.

Overall, we decided to drop the project in Rust for the time being, we found an easier way, arguably safer and easier to maintain with our current technology stack. But this is not the end of Rust at Airtame. Rust is young and needs to evolve, and we have many incoming projects that will be much more suited for Rust (or Rust suited for them) in the near future.

Working with Rust

Ok, but how was it to work with Rust. The answer was: Pretty amazing. Understand that not everything was an amazing ultra smooth sailing, otherwise we would have stuck with it for production. In this section, I want to go through a couple of things that I think are worth mentioning when talking about Rust.

Correctness

One of the Rust main focus is correctness. Through a really rich type system and the ownership model, Rust guarantees memory-safety and thread-safety.

When I started to write rust code and play around with it, the compiler/borrow-checker/etc... would be screaming at me constantly. The fact that everything is const unless you explicitly state otherwise, the fact that everything is moved unless you explicitly state that you want to borrow the ownership of a resource, and the fact that every time you borrow the ownership, then the said resource cannot be read or modified anywhere else makes it very different from what I used to... in a good way.

In our jobs, most of us spend long hours writing code, sometimes with deadline that are difficult to meet, with some stress, maybe distractions from our personal life, and so on. None of us is flawless. Having a set of tools supporting you when writing software is priceless.

Sometimes, it feels like the tool is fighting against you - but trust me, this is for the best :)

Anecdote time:

A few years back, I started the movement to move our Javascript codebase to Typescript. I come from the strongly/statically typed languages world and Javascript always frustrated me by being the literal opposite: a loosely and dynamically typed language. I remember that time, when I stumble upon that function in a code review. That function was taking an ID, great. And there were some operations on the ID in the code of the function. Strings and arithmetic. I was confused. So I asked, the developer, "ok what does the function takes exactly? An Id. Ok, but what is the type of the Id? A number I guess. What do you do this hashing operation that takes a string then? Javascript will convert it to string. Are you sure? Euh.... kinda yes." He was wrong in that case.

In this example, it is not a case of right or wrong, but more of thinking about the code we write. You - as a developer having to work with the compiler and spend more time merging the code is much better than the code breaking in production and cost tens (hundreds) of thousands of dollars to you or your company.

I managed to convince the different parties that it would be beneficial on the long run. Typescript it was. And for the next 2 years, I heard complaints about how Typescript was stupid because "Of course this variable cannot be undefined, I set it here". Frustration growing but what I found is in 98% of cases (made up number to support my anecdote ;) ), Typescript just caught a potential disastrous production error.

My point here is that the "compiler" (in case of Typescript, this term is debatable) is here to help and catching the errors as early as in development is the best case scenario for everyone. The QA doesn't have to spent time on a piece of software that has obvious flaws, the business doesn't have to put off fires with clients threatening to leave and so on. (Ultimately, everyone is more than happy to use Typescript today in our teams)

Rust is amazing at that. Maybe too amazing. I exactly heard that from people. The frustration of not being able to write clunky code just to test out something. Because Rust is not like that. Rust is not for quick and dirty prototyping. Javascript and Python are much more suited for that. But when it comes to code correctness, I grew to love Rust.

3 sections on the subject from the Rust documentation:

• https://doc.rust-lang.org/1.8.0/book/ownership.html
• https://doc.rust-lang.org/1.8.0/book/references-and-borrowing.html
• https://doc.rust-lang.org/1.8.0/book/lifetimes.html

Dependency management

Rust comes with a dependency management tools builtin. It might seem like a small feat to some, but I love that.

I used to code in Ruby that have a pretty nice package manager: Bundler. Today I use Javascript a lot with NPM and Elixir/Erlang with Hex. The only language that I use on a daily basis not having a package manager/dependency manager today is C/C++ (they have a couple of project attempting to solve the issue, but this is not the point here).

I must say that Cargo is at least as good as the best of the ones on the list mentioned previously. It is pretty fast, simple to use, has all the features that I needed and it seemed to do the job properly. I played with much bigger projects as well where Cargo did a fantastic job.

You can access the list of packages to use with cargo here. The packages are called crates.

Tests

Rust comes with builtin tests functionality. Again, this might sound like a picky thing but I love that I don't have to spend my time evaluating what is good or not. I have one thing, that works and that is used by tons of people. This is all I need to know.

A small example:

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

We attempt to write tests as much as we can before going into QA. Having a set of tools at our disposition to do so easily is actually priceless.

A bit on the subject: https://doc.rust-lang.org/book/ch11-01-writing-tests.html

Conclusion

To conclude this rather wordy article I would just say that while we decided not to go with Rust for production this time, I can guaranty that I (we) will use Rust in the future. There is a great potential and it was actually fun and enjoyable to write code with it. I totally recommend playing with it.

If you would like to hear more about writing in Rust, like examples or even a short "tutorial-like", lemme know in the comments!

Some links:

• https://rust-lang.org: Official Rust website
• https://blog.discord.com/tagged/engineering: Discord talks about some of the real life challenges they have to overcome. They have a couple of articles about Rust that are worth reading.
• https://gist.github.com/brson/a324c83a6af6a8a78dfaa9d33eb9b48e: A collection of Rust notable blog posts.

Over the last couple of weeks I worked on multiple projects which a large React codebase spanning from a couple thousand line of codes to more than 100k for one of them.

Something that stroke me while going through the code was the way the key prop was used when rendering an array of elements. In way too many cases the "index" of the array was used. It really felt like it was here more to silence the warning produced by React more than for its actual job: Keys help React identify which items have changed, are added, or are removed.

The React documentation itself says:

We don’t recommend using indexes for keys if the order of items may change. This can negatively impact performance and may cause issues with component state.

I want to focus on the last part of this sentence: may cause issues with component state..

The problem

The following section will demonstrate a component that has a list of numbers. Each number will be passed to child component that will take the number and will expose a button that each time it is click will multiply the number by 2:

const Parent = () => {
  const [numbers, setNumbers] = useState([0, 1, 2, 3, 4]);
  
  useEffect(() => {
    // After 5s, we will remove the midle element of the array.
    setTimeout(() => {
      setNumbers([0, 1, 3, 4]);
    }, 5000);
  }, []);
  
  return (
  	<main>
      {numbers.map((el, index) => (
        <Multiply num={el} key={index} />
      ))}
    </main>
  )
};

const Multiply = ({num}) => {
  const [state, setState] = useState(num);
  
  return (
  	<section>
      <div>Initial number: {num}</div>
      <div>New number: {state}</div>
      <button onClick={() => setState(state * 2)}>Multiply</button>
    </section>
  );
};

The trick part here is the setTimeout. After 5 seconds, we mutate our arrays which triggers re-render of the Multiply components. You can see in the example bellow the issue:

The new and initial component are not synchronized anymore. During the re-render, React in an attempt to optimize the operation will actually shift the prop and delete the last element of the array. This is where the key is extremely important. It helps react keep tracks of the different components and order so you can safely have operations like this one within your components.

The solution

How can we then avoid this kind of sneaky issue? As a rule of thumb, I usually generate a unique ID for my objects. Thankfully, in most application of this kind (be it a todo app, a pet management app or a game), when you get elements from your backend API you can get the ID.

You will notice here that I use shortid to generate a short and simple unique ID for my elements. And you can see that after 5s, my array is still looking good.

The exception

There is an exception to the rule. Basically, if the array and its elements follow a certain set of rules, you can safely use the index as the key:

1. The elements in the array do not change - they are static elements
2. The array itself is static: no new elements is added, removed and replaced
3. And finally the elements don't have unique IDs.

Only - and only if - those 3 conditions are met, then using the index as the key will be safe.

See the previous example but following those rules:

(Notice that in that case we do not mutate the array).

Conclusion

As a rule of thumb, I never use index as a key. But if you end up in a situation where the array that you are about to display exposes elements without unique identifiers, I would refer you to the exception rules. If the first 2 points are met, feel free to use the index safely - otherwise generate ids for your object.

Some more documentation on the subject:

• https://reactjs.org/docs/lists-and-keys.html
• https://medium.com/@robinpokorny/index-as-a-key-is-an-anti-pattern-e0349aece318

This article with be the first of a long list about the development of my own Game Engine. This series of posts will document the progress made during development, re-transcribe the struggles, but also progresses, the thinking process and the cool stuffs found along the way.

This first article will set the base for this series: Where does this idea come from? Why doing this? And what will be the goal?

The beginning

Since I started programming, I have had in mind to learn how to build video games. Every single game that I play, I am obsessed with find which engine it uses, how is it made, etc... . Around 4 years ago (might be 5 or 6), I started to think about investing time in learning how to create video games. But as everything related to programming, I was more obsessed with the low level functioning of the systems that make games possible more than having a clear project in mind.

Don't get me wrong, I have a couple of ideas of video games I want to produce - one being written and partially scripted already. But my need for understanding of the subsystem was winning at the time. And still is.

I then started to develop my own game engine in C++, starting with the 3D engine, following development tutorial for OpenGL. I came to a point where I could load 3D models, render textures, skybox, having a basic Phong lighting model, a basic system of plugins and a "fairly" abstracted 3D rendering engine.

But as I was doing that in parallel of a full time job, sometime motivation faded out and eventually I basically stopped committing.

For around 1.5 years, I barely committed to the repo and improved what was in place. Until 2 weeks ago.

The new beginning

Behind this catchy section title is actually the truth.

2 weeks ago, I decided to start a refactoring (more of a rewrite) of the engine, with more defined goals and a slightly different approach.
First, let me list the points and objectives that will stay the same and where I will be focusing my next few years:

• Cross platform: I want the engine to run on Windows, Linux and MacOs. Hopefully I will add more platforms to it in the future but this on those 3 platform that my experience is.
• Abstracted: I want a sufficient level of abstraction that it will be easy to replace the Audio engine, the rendering engine, the physics engine, add plugins, etc...
• 2D and 3D: I want to be able to choose rendering pipeline for both 2D and 3D, and having the subsystems working with it.
• Editor: I still want to have a Game/Level editor coming with the engine.
• Open Source
• There are many many more ideas that will be listed on the repo of the project. Those are just the main lines of thinking, and what will drive the overall architecture and process around the Engine.

What will change now:

• I want to build games with it. Meaning that throughout development, I want to produce examples demoing and proving that the engine works and is made for building games (example: Tetris, R-Type, small 3D puzzles, ...)
• I want to divide the project development into small challenges: for example cool module system, plugins, UI renderer, Bloom effect, ... And document them, write about them, ...
• Implement only features that are understood and used in an example: in the previous project, I implemented some small bits because some tutorial said it was nice but never dug into what was the real purpose and all.
• Finally I want to be less picky. I am learning, this is a fun and learning experience. I used to be stressed not to do the right thing. But this is ok, I will do mistake and I will fix them.

Next steps

I started a refactoring-0.1 branch that intends to rewrite the code up until a similar state as before.

I will spend a lot of time in playing around with an architecture that will enable:

• A good module system that enable hot reload of some subsystem (I had this on my todo list of things to play with for years)
• A good plugin system to write proper plugins to load resources, enable algorithm, etc...
• Abstractions of the platforms and the rendering API/Driver
• And finally of the rendering pipelines: 2D/3D/UI.

As mentioned, this project is a learning experience, mistakes will be made, non optimal code will be made (do not forget, no code review, no peer help... yet, and no experience in that domain...).

Conclusion

The project is named CodeHero (this is up to change), it is hosted on github.com: CodeHero.

I hope to make great progress and share experience as often as possible on this "devlog".