Meshroom

Photogrammetry has been around for as long as there have been photos. It's the process of taking measurements from images and using those measurements to reconstruct models of the items and spaces within the images. But modern photographic and processing technology has taken this process to a whole new level, allowing you to build 3D textured models from a carefully taken collection of 2D photos. To reconstruct a scene, for instance, you need to take hundreds of shots with similar exposure, focus, and range, while making sure all elements within the scene are covered from multiple angles. The same is also true for scanning objects, although physical props like a turntable can make it considerably easier to attain good results. And the results can be incredible. Entering a reconstructed scene in virtual reality, for example, lets you wander around an exact to-scale recreation of a cathedral, a museum, or mountain top, captured perfectly for all time. Similarly, there doesn't seem to be a modern 3D game that hasn't incorporated photogrammetry for its models and locations, and it's likely to be a huge growth area in the future as our photos and Raspberry Pis become ever more capable.

But much of the software used to turn photos into 3D is proprietary, with RealityCapture and PhotoScan being two popular choices. There is an open source pipeline, using VisualSFM, MeshLab, and Blender, but it's difficult to use and takes plenty of work to create good results. Which is why it's brilliant to see a new piece of open source software in the 3D space, which is exactly what the excellently named Meshroom is. The application itself is easy to understand. This is because it's still only in the early phases of development, but also because of great design that doesn't overwhelm you with options. You start by importing your stack of images, which can then be refined to make sure you're making the best use of the available data. You can then just click Start to initiate the intense processing required to build the 3D mesh.

If you've imported hundreds of photos, which you need for a complex scene, processing can take hours and tens of gigabytes of memory. Simpler models can be generated from a dozen and 20 minutes of processing, but there is an inescapable requirement in the current version. You will need an Nvidia GPU and CUDA installed, as this does all the number crunching. Other GPUs are being evaluated, so hopefully more options will become available. Either way, even though processing can still take hours, the 3D view will become populated by 3D colored points as they're derived from your image set, along with triangulated viewports for the location where each photo was taken. The graph editor lets you select and edit each step in the process, and, cleverly, each step logs its own output to a separate folder, complete with any generated assets, such as depth maps or objects. This means when you change something small, these elements won't need to be calculated again. The final result can then be imported into your modelling or mesh reduction application, either as an object, as a full texture, or both.

Project Website

https://github.com/alicevision/meshroom

For best results, take your photos in RAW mode and lock the exposure. If possible, use a DSLR with low depth of field.

Meshroom's output is a fully textured model that can be loaded into Blender and refactored and refined for final use in your scene or game.

Game streaming

Moonlight

If you're into games, you may have heard of Valve's low cost Steam Link hardware. This was a wonderful little embedded (proprietary) box that on one side is connected to your network – ideally, via Ethernet – and on the other side is connected to your television and a games controller. You could then stream your games from a game-capable PC running Steam to the television via Steam Link, letting you play your favorite games from anywhere with good network connectivity. The most impressive part of all this was that it actually worked remarkably well, even at 1080p and 60 frames per second. There was no noticeable latency, allowing you to free yourself of an often noisy and constrained PC environment to play games on the sofa.

As you've likely guessed, the past tense of the first paragraph means that Valve's Steam Link is no more; Valve stopped selling them in late 2018. To mitigate this, alongside iOS and Android clients, Valve released its Steam Link proprietary streaming protocol via an app on the Raspberry Pi, helping you replace the old hardware with a newer and cheaper option. It works well, but it doesn't integrate well with other software, such as Kodi, and doesn't have an x86 build. If you want the same facility on your Linux box, you need to install, sign-in, and run the entire Steam stack. Which is where Moonlight comes in. Moonlight is an open source client for Nvidia's GameStream, another proprietary service for Nvidia's hardware that does the same thing Steam Link did, except Moonlight is better. It has an even lower latency, supports 4K resolutions and 120fps, and works on many different devices. It integrates well with Kodi (with an add-on), a Raspberry Pi, x86, and even an unlocked PlayStation Vita. Because it's open source, it can be easily built and run on almost any Linux system.

Project Website

https://moonlight-stream.org/

The main app has a Qt UI, but a separate version, Moonlight Embedded, runs headless for media centers.