Automatic bitrate and redundant encoding against bugs and freezes: what is the peculiarity of the Loudplay technology

This is how the debugging process of the streaming technology usually happens. As you can see, it happens on a very old and weak PC.

What do we see on the monitor?

In the background is a game client that receives a video stream from the server. The client has graphics with debugging information, which includes video stream bitrate, FPS, video packetlosses, encoding and decoding time, network delays, etc.

On the right is the task manager, where you can track the CPU load of a weak PC.

Next are network analyzers and shaders, with which you can emulate various network states (losses, delays, etc.)

What are we trying to achieve?

We try to achieve high picture quality, high FPS and low response delays.

What problems are we trying to solve?

The fact is that streaming in real time does not work like the well-known YouTube or any online cinema, where you can cache part of the data on the client device and smoothly play the video stream while the following data is being downloaded. We don't have that luxury. With any buffering in cloud gaming, delays grow to such an extent that it becomes impossible to control a character in the game.

So, if we remove this buffering, then we see that our commonly used Wi-Fi routers and Wi-Fi networks are not as good as we think.

Here is what I mean: Most Wi-Fi routers (and wired routers too) have rather poor performance. As soon as you start downloading a lot, the router processor stops coping with such a speed, the buffer begins to accumulate and when overflowing happens, all packets are reset. As a result, in cloud gaming, first we feel an increase in the control delay, and then the gameplay is interrupted altogether, and we are waiting for a new portion of data. YouTube, as we remember, stores part of the data on the client "in advance" and we do not notice the process of overloading the router. For the gameplay, this overload is significant, and it is uncomfortable to play with it.

Usually there are a lot of Wi-Fi routers and a lot of Wi-Fi networks installed in an office or apartment building. Often the frequencies of the networks overlap, and we get the phenomenon of the radio wave interference. This means that some of the data may suddenly be lost without warning.

How do we solve this problem?

We have developed and use the technology of automatic bitrate change. It works like a smart cruise control in a car: as soon as we notice that the router is not coping with the data processing speed, the buffer begins to fill up and delays in milliseconds increase, then we begin to reduce the bitrate of the video stream until the delays stabilize. Next, we start increasing the bitrate again until the buffer starts to overflow and reduce it again. This process occurs continuously, several dozen times per second. As a result, we utilize all the useful bandwidth of the router and do not bring it to an accident. That is, we support the router at the limit of its capabilities. As a result, we always have consistently low delays and maximum quality within the existing conditions.

And that's not all. You may ask, how to deal with sudden losses on the radio network when the router is operating normally? For that, we have developed a redundant coding system. Along with the useful data, redundant data is also transmitted from the server to the client, with the help of which we can restore lost data on the user device. The process of restoring losses occurs instantly and imperceptibly for the end user. With this technology, we can restore the image on the channel with losses within 40%. Thus, the gameplay remains smooth, the picture quality and bitrate of the video stream are high and without reduction of the FPS.

Many companies that develop cloud gaming technology usually stop at the implementation of an automatic bitrate algorithm and are not able to effectively deal with sudden losses on the network. When they have losses, they have a big "lag", and then either the picture is very "squared", or you are informed that it is impossible to play on your network.

Only few companies go further in the technology development and cope with these losses. Loudplay is among them.

In the following articles we will tell you about the technological onboarding of the user and the GMOS system, which we are developing to improve the quality of the gameplay.