The Sony PlayStation 5 game console is under development and has been in the news a lot lately. The PS5 will be released at the end of next year, the manufacturer has since confirmed. Sony Interactive Entertainment is still developing a headset for AR (Augmented Reality) gaming.
There are some significant differences between Augmented Reality and Virtual Reality. With AR, images are added to the real world, so you remain aware of the environment in which you find yourself.
This is in contrast to VR, where you enter a virtual world and you no longer have any real awareness of the actual environment. Then there are some Mixed Reality headsets, these are suitable for both AR and VR applications.
Sony already has considerable experience with VR, the Japanese manufacturer is also not completely new in AR. You can also play simple AR games with the PlayStation Vita, just like with the Nintendo 3DS. Various AR games have also been developed for the smartphone, such as Pokémon Go, which was a particularly popular AR game three summers ago.
Although Augmented Reality games are still in its infancy, expectations for the future are promising. The following patent shows that Sony also has a professional solution in development.
Sony wireless AR headset with stereoscopic cameras
In Japan, Sony Interactive Entertainment has filed a utility patent with the JPO (Japan Patent Office). The patent entitled ‘Image processing device’ was published on 11 July 2019. A headset is described that is equipped with 3 stereoscopic cameras or 3 × 2 cameras.
The HMD (head-mounted display) has 2 cameras (6 in total) on the front and on both sides. These stereoscopic cameras serve to take photos of the user’s environment. With these 3D photos, the distance is calculated between different objects that are located in the room where the user is in. These images are then used to visualize the space in real-time.
If necessary, the user is shown a so-called ‘guide image’, in which the user is instructed in a playful manner to assume a certain position, so that the cameras can correctly photograph the object and a full depth map of the environment can be created.
As an example, a staircase is shown, which the camera cannot possibly look around at once. The user is then shown a binocular, which he must walk to and look through (see patent illustrations). At that time the stereo cameras will take extra photos to be able to display the entire space in 3D. A motion sensor can be used to accurately monitor the movements of the user and the headset.
The Sony headset can be used wirelessly and can be connected via Bluetooth to a PlayStation console or a computer. The operation is via the game controller or a keyboard, depending on the medium (console or PC) that the gamer uses.
In addition, the patent description states that the headset can also be used in combination with a smartphone or tablet. So you can also play mobile AR games via the headset. By implementing the AR headset wirelessly, the user retains much more freedom of movement. Furthermore, the patent speaks about the possible addition of a USB and an HMDI connection.
In this patent, incidentally, only the technology is recorded and not so much the design. The images are for illustrative purposes only, so Sony could opt for a completely different design. In addition, the description states that the number of stereoscopic cameras used may differ. This will be related to, among other things, the cost price, the more cameras the more expensive such as an AR headset or AR glasses.
Sony AR prototype glasses & Ghostbuster in the Park AR Project
Whether this technology is used in the prototype AR glasses that Sony currently uses for its ‘Ghostbusters in the park’ project is unknown. This week Sony started with a major AR project that takes place in Japan in the Ginza Sony Park.
‘Ghostbusters in the park’ is a location-based AR game where gamers compete against virtual spirits. Gamers can register via the website. Two to four sessions take place every day, each lasting one hour. Per session 50 to 160 people can participate in the AR experience.
The project lasts from October 12, 2019, to December 8, 2019. At the same time, Sony has also announced prototype AR glasses, which will be used for this. The video above shows a trailer for this special project. It is obvious that we will also encounter similar applications in amusement parks in the future.
The future of AR gaming
AR gaming can be realized in two different ways. Headsets such as the Microsoft Hololens and the Magic Leap use a transparent screen. This technology is cheaper than using stereoscopic cameras. With transparent AR, however, it is difficult to accurately visualize dark materials or surfaces, as Zero Light describes, a company that also deals with real-time 3D visualizations.
Moreover, stereoscopic cameras have the advantage that an extremely low latency can be realized, so that users do not become disoriented as quickly, due to the delay that occurs. A much more immersive AR experience can be created in this way, taking into account all objects that are in the actual environment.
The specifications regarding the prototype AR headset from Sony are as yet unknown. The results from the extensive ‘Ghostbusters in the park’ will undoubtedly be used to refine the experience and optimize the headset.
Sony is certainly not the only manufacturer that sees a future in AR gaming. As mentioned, other game manufacturers such as Microsoft and Nintendo are also involved in AR gaming. But Google and Apple also have openly expressed the future in Augmented Reality. Huawei also has smart AR glasses in development, the company announced earlier this year.
Yet it will certainly take several years before AR gaming becomes generally good. Finally, this new way of gaming will also require an entirely new way of game development. As more developers start working on this and therefore more game content is available, the popularity will also increase, according to the general expectation of analysts.