An interview with Nathan Wright,
Digital Technology Manager at ULA Group
As published in CX Magazine
Why are hi-def LED screens replacing green screens in TV and film production?
Utilising LED screens within TV studios has been a trend for some time now. Providing the flexibility of dynamic content helps create that extra visual element as well as that ease of quickly changing content on large surfaces.
LED screens are nothing new to anyone in this sector, however the bigger changes that have been happening over the last couple of years are the graphics engines driving the content to them. This in turn has grown the market for virtual studios.
There are many types of virtual studios in today’s COVID environment. What we are focusing on here are some of the main things associated with studios used for the creation of films, TV commercials and other content, not a webinar type of setup, which has been called a ‘virtual studio’ as well.
Recently the production of the TV series ‘The Mandalorian’ used an extensive ‘digital stage’ comprising an almost 360-degree LED wrap around with a celling to create a ‘true’ scene.
This was driven by the Unreal Engine, which has been the backbone of many of today’s computer games.
So, what’s so special about this? Using this set-up lets you create 3D worlds on the fly with the ability to move objects around as needed. The biggest feature being the motion tracking of the content with the camera.
This creates a much more realistic visual on screen, a necessity to make it all work. It’s now no longer a static look; the camera and content are moving in harmony.
One of the major advantages of LED screens is the fact there is none of the old green screen ‘spill’ from reflective surfaces that was always a challenge to deal with in post-production.
If your content is created right, you can get all the ‘real’ reflections and lighting on your subject, which was also traditionally done in a post-production environment.
Another advantage is that your actors (and everyone else) can see in real-time what the environment is going to look like, and if they are referencing something that is in the virtual world, they can see it rather than just taking a guess.
How do virtual studio LED walls save money in film production?
Doing all of the clean-up work in post-production is very time consuming and costly. Even more costly re-shooting a scene if it didn’t work.
The initial outlay for a virtual studio is a large investment, but in the long run, a lot of the work that used to be done in post-production or on location can be done faster and more efficiently in a virtual studio.
What are the important factors when designing a LED wall for virtual production?
The main questions to ask is what are you trying to shoot, and what are everyone’s expectations?
Obviously, pixel pitch is one main factor to be considered when you purchase a screen, but of course the finer the resolution, the higher the cost will be.
Also, what is the size of the digital platform you need to build in your studio? Can a mix of high and lower resolution panels work? Do you need to create a digital ceiling? Rigging and installation of the screen is also an important factor.
You might have a grid, but can it take the weight of a large-scale LED wall or would it need a combination of ground support with some of it flown? Most studios will also prefer a curved LED wall, so you need to make sure that the screen can support curving, or at least the rigging can.
Most of the time the screen will be convex, which means the panels don’t necessarily need to have the curve option (however it will make your line-up better if they do). Our VuePix Infiled DB Virtual Studio series supports +/- 5-degree curving on every 500mm facet as standard, so it’s perfect for this type of application.
What are scan and refresh rates, and why are they so important in film production?
Scan and refresh rates seem to be the biggest cost factor when it comes to LED screens on the market today. Like everything, you get what you pay for.
There are many ways to cut costs on the production of LED screens. The easiest way is to use low quality LEDs, driver ICs and PCBs. We are often asked to match or beat prices from other vendors, and the first question we usually ask is “what are the components of the screens and what level of components would you be looking for?”
Most of the time the panels aren’t of the same build quality nor are the components, which creates a significant price difference. We do provide budget offerings and these differences in quality will be clearly outlined, so you will know what to expect.
There are two main factors that determine the performance of an LED screen. These are driver ICs and receiving cards.
I’m not going to dive into the science here as it starts to hurt your head if you’re not an electronics engineer but, in a nutshell, the driver ICs tell the LED (RGB) what intensity it should be.
The amount of LEDs that are driven off an IC is translated to the scan rate. It’s a balancing act of ICs vs. scan rate and performance. When brightness isn’t an issue (i.e. – indoors) the scan rate can vary from 1:8 or 1:32 and anywhere in between.
So why not have everything at the best scan rate? Simple – cost! This all comes back to ‘you get what you pay for’.
Normally, outdoor screens will have a better scan rate than indoor screens, because they run brighter and they need more IC power to drive the LEDs. Indoor screens don’t need that level of brightness, so they will generally have a higher scan rate.
So why is scan rate important in film production?
Working with LED walls on camera can be very challenging, from rolling of content, to Moiré effect, to just bad calibration or nasty setups.
One factor to help with this is a better scan rate, so instead of a driver IC refreshing a group of LEDs (i.e. – 24), it might have the option to do 12 or 8, which is much more friendly to the sensor in the camera and will give you a smoother image when it comes to moving the camera in the field of view.
Our VuePix Infiled DB Virtual Studio series offers a scan rate of 8 which is one of the best on the market in this class.
The other important factor is the refresh rate. This is how often the ICs, receiving cards, and processor are refreshing the LEDs. A lot of people relate this back to the way TVs work, but LED screens are different.
The refresh rate of LED displays in the simplest terms refers to the frequency at which the LED display’s hardware draws data from its source. This frequency provides a measure of frame rate per second; however, it is important to know the difference between refresh rate and frame rate per second.
While refresh rates include the drawing of identical frames every second, frame rate specifically measures the frequency at which an entire frame of new data is fed into a display. When you have lower refresh rates, cameras will often pick this up, as will the human eye. The higher the refresh rate, the better the screen will look. Most of the major receiving card vendors automatically suggest the appropriate selectable refresh rates. Most screens produced within the last couple of years should have a refresh rate above 1920Hz. High performing screens will normally be around 3840Hz.
On top of the above technical aspects, the type and quality of the driver IC will have a big impact on the image quality on the screen, mainly around greyscale and how it deals with bright and dark images. With cheaper ICs you may have some issues with greyscale and colours just not looking right. You could be running the best receiver cards, but if you have low quality ICs, you will have bad visual performance.
MBI (macroblock) are one of the market leaders in the LED driving IC space, but there are more expensive and cheaper ones out there. It’s just about defining your needs and making sure your vendor can deliver. We use macroblock in the majority of our VuePix Infiled range. All of the above factors will vary the cost of your LED screen. At VuePix Infiled we offer a special version of our Deep Black series with a 1:8 scan rate, angled cabinets, black LED and special masks to reduce the Moiré effect. Our DB Virtual Studio series is available in a 2.8mm and 2.6mm option.
What do filmmakers need to take into account when creating content for virtual studio LED walls?
There are really no limitations with the content that can be shown, as it is just a giant video wall. However, you need to make sure that your content isn’t too bright, to the point that it overshadows your main subject.
What needs to be considered when using or creating HDR/10bit content?
We are seeing more and more specifications asking to support 10bit and HDR content. While it’s an easy tick box on paper, in reality it’s a bit more complex.
There are a few things you need in order to support both, and normally if your system supports HDR then 10bit will also be available. But on the flip side, your system may support 10bit, but your hardware may not support HDR.
The first and most important thing is to have quality content. I’m not going to get into colour spaces here, but true 4K HDR 4:4:4 content isn’t readily available.
There are only a few cameras that can actually record in this native format, and if you want to generate content of this quality, it isn’t cheap to produce. On top of this, the file sizes are extremely large. The other factor, as mentioned earlier, is content getting generated on the fly by virtual engines.
Depending on your needs (and client needs), most high-end render systems can create and export in a variety of formats. Once again this comes back to the question of what is needed vs. what is wanted.
The other element is the program chain to get the source to screen. There is definitely a lot more to HDR and all the variants, which is for another discussion.
At ULA Group, we are the distributor for NovaStar and Brompton in Australia and New Zealand, so we have a fair bit of knowledge and expertise on both product lines.
One thing that is common across these two platforms is that if you do decide to use 10bit processing, you will halve the loading capacity of your system! This needs to be taken into consideration when it comes to your system designs, as most of the pixel load charts are based on 8bit processing.
Brompton has been the leader in this space for some time now. Their SX40 processors and the new Hydra calibration technology make 10bit and HDR a pleasure to work with.
Low latency (two frames end to end), free rotation and chroma tune are supported on most of the Brompton range, and all of their processors allow advanced colour adjustments on the fly.
One of the other unique Brompton systems is their dynamic calibration. This is a recent addition to their software. It will change the values of colours and brightness depending on the content and variables that have been user defined to deliver a superb end user experience (you can read more about this on Brompton’s website).
We will have a Hydra system here in Australia in the coming months for you to re-calibrate your existing screens. If your screen is running R2 cards, then your screen will support Hydra calibration.
NovaStar also supports HDR and 10bit in their MCTRL 4K, H series and Ultra Pro HD processors. When you have supporting receiving cards (standard in our DB series), you can view 10bit HDR content on your panels.
With the new range of processors driving large format screens (4K & above) one unit is making things much simpler, that’s the H9, supporting a massive 52 million pixels in one controller.
The NovaStar range offers four frames end to end with scaling on, or two frames with low latency mode.
NovaStar are also working on a new range of receiver cards using 5Gb Ethernet, which allows for five times the current pixel loading on a single cable – COEX. This tech was launched at ISE this year and has just become available. It is geared towards the 8K screen market.
NovaStar is the market leader in LED video processing with over 60% of global LED screens running their processing systems.
Both vendors offer processing and calibration that support Rec.709, DCI-P3, Rec.2020 as well as custom options (you need special LED chips to reach Rec.2020).
How are these technologies likely to develop?
LED screens are constantly being refined. At VuePix Infiled, apart from our new DB series specifically developed for virtual studio environments, we offer a whole range of other products suited to various markets – from the large outdoor mesh series designed to withstand the elements, to the superfine pixel pitch indoor 0.9mm panels.
New technologies always arise from our customers asking “can it be done?”, and our team always works with the client to tailor our technology to perfectly suit their specific project.
The main advantage we have in this space is that our factory is highly flexible and our products can be fully customised if needed.
Where is the world of media servers heading?
I think we will continue to see further development in the processors and software, creating and optimising the content for the new high-tech platforms.
With the consumer release of 8K TVs, this means big changes to a lot of program chains and cameras. But as with every new technology, this will come at a cost, as most companies have just gone to a 4K chain recently, so this will happen when the market is ready.
