The Hiperwall software development team had a busy and productive 2022 with an outstanding video wall software release, and we’re preparing amazing stuff for 2023. Each member of the team designed, built, integrated, or tested complex software components that form the Hiperwall distributed video wall software product. Each team member expanded their capabilities by learning new techniques and skills to do a great job making highly capable and reliable software.

The big version release of 2022 was Hiperwall video wall software version 8.0. This version includes new features strongly desired by our customers, including HiperZones, the new HiperSource Streamer+, a new component to drive the video wall called HiperView+, and an enhancement to make HiperCast connections easier and more powerful with HiperCast Pull. Each of these major new capabilities and all the smaller features that go along with them add value to the product and give our customers more power to show their content in the most productive way possible. Customers with maintenance agreements can upgrade to version 8 where most of these features are included with no extra charge. Our Premium customers automatically get the incredibly powerful HiperZones when the upgrade, which is yet another reason the Premium package is the best deal around.

HiperZones

The crown jewel of the Hiperwall version 8 release is HiperZones, which allows a customer to define virtual video walls on their physical video walls to make separate zones of control and responsibility that help maintain content integrity. Different users can be assigned to control each zone and they cannot interfere with content on other zones. This means a large LED video wall can be split into different zones monitoring different regions of the country, for example, or different business elements. Changes to content in each zone are isolated from the other zones, so critical content is never obscured by another user’s content.

HiperZones is very useful for customers with large video walls, especially seamless LED walls, but it is also perfect for customers with multiple video walls or satellite displays, so content from one does not impinge on any of the others. The technology behind HiperZones is impressive – when customers learn that zone boundaries do not have to be display boundaries but can be defined to be nearly anywhere, they are impressed. Hiperwall software’s ability to manage and display content keeps getting better and better, and HiperZones is a huge step above traditional video wall capabilities.

HiperView+

Until recently, Hiperwall systems have typically used one small form factor computer per display to provide scalable performance as the system grew. In high-end systems with multiple LED controllers, we use HiperView Quantum to drive several LED controllers per computer and synchronize each computer to get frame-by-frame accuracy on content playback. We recognized the need to drive multiple displays or LED controllers for smaller systems from a single PC, so we developed HiperView+. This software is based on the powerful HiperView Quantum technology that optimizes video performance on NVIDIA GPU-based display computers but applies it to smaller systems that do not need multiple display computer synchronization. HiperView+ provides top-notch performance with multiple display output and is compatible with NVIDIA Mosaic technology to layout the displays.

Streamer+

The original HiperSource Streamer software is powerful and easy to use but is based on a Windows technology infrastructure that has not kept up with 4K video performance. The development team reimagined the Streamer concept and created Streamer+ based on new API standards that glean significant performance from the same hardware and take advantage of newer GPU hardware. It also uses a new more secure and more capable interface to the HiperController to allow for future enhancements. This new Streamer+ turned out to be a huge benefit for our customers and integrators during the pandemic supply chain constraints, because it allowed more and better streams from a single source PC, thus reducing cost and required hardware.

HiperCast Pull

HiperCast is a powerful capability to securely share source content between Hiperwall systems across town or across the world, but it was tricky for sysadmins to configure because of port forwarding and more. The software team developed a pull-based solution where each HiperController connects to the HiperCast server and subscribes only to needed sources. This new approach makes configuration of each Hiperwall system trivial yet adds significant power over content choice. HiperCast Pull increases the available audience for HiperCast’s sharing capabilities.

What’s next?

The Hiperwall software development team has not slowed down since the release of version 8.0. We have been working towards an exciting and powerful new release that builds upon HiperZones and the other great version 8 features. Beyond that, 2023 will see even more terrific new features and products added to the Hiperwall video wall software and beyond. We hope everyone who reads this has a terrific 2023 and we look forward to shaping the future with you.  

Hiperwall video wall software has always been great at accepting and displaying lots of sources of several types, but sources and their uses have changed over the years as what they were showing evolved. This post contains my experiences with the changing landscape of sources used in Hiperwall video wall systems around the world, particularly for control rooms and similar, rather than signage applications.

Early days

Our first source type was the Sender, followed closely by the Streamer. These standalone apps send data to the video wall computers in very different ways making their uses unique.

Sender

The Sender software (now called HiperSource Sender) captures the screen or part of the screen of the computer it runs on and sends that to the Hiperwall system. Sender can run without even being installed on the computer (for environments that restrict software installation). The encoding of the Sender’s video stream is completely CPU-based, so it runs on nearly any PC, Mac, or Linux box, and the performance scales with the CPU and network speed. Sender can send its data directly to the video wall if it is on the LAN or can use the HiperController as an intermediate via an encrypted channel. This allows Sender sources to come from anywhere on the Internet. We extended this capability with a product now called HiperCast (was Share) that can deliver multiple Sender sources to multiple Hiperwall systems around the world.

Our clients use Sender for monitoring dashboards, social media feeds, desktop sharing, and more. Sender provides multiple captures on a single machine, runs in VMs, and supports KVM control of the source machine. The flexibility and utility of Sender means customers want to use it in many situations where high frame rate is not required. Sender also scales and resizes handily, so if a user changes the resolution or orientation of the screen of a Sender PC, it can adjust automatically.

While Sender is our oldest source type, it has undergone many performance and capability improvements over the years and remains the simplest and perhaps most flexible of our sources. Sender was built at a time when most of our customers were running custom applications to monitor and control their systems, so it was perfect for delivering those output screens to the video wall. Since the nature of such applications has changed to be more web-based, other solutions, especially HiperSource Browser, discussed later, grew in popularity.

Streamer

The HiperSource Streamer software is designed to deliver high frame rate, high quality video streams to the Hiperwall video wall. It uses hardware accelerated video capture, compression, and encoding to send a bandwidth-efficient video stream. Streamer uses our patented approach to synchronize playback on different display computers to make the frames sync across display boundaries for a seamless experience even if the video spans multiple display tiles. Streamer provides display capture, like Sender, but also supports capture cards, so it can stream video content captured from HDMI, SDI, or analog video sources to the wall. For display capture, Streamer also supports KVM control of the Streamer PC, which means content and applications on that PC can be interacted with while being shown on the video wall. Because Streamer requires hardware support to encode the stream, it must run on a moderately powerful PC on the Hiperwall LAN.

Since Streamer is great for high frame rate video streams, our customers use it to play videos, show presentations that have artistic transitions, share VMS consoles or other applications where high frame rate matters, and to show TV stations to monitor news and weather. Streamer has evolved over the years to support more hardware types and to manage capture cards in a flexible manner. Streamer continues as the go-to source type for video-style streaming.

Streamer+

With version 8 or the Hiperwall video wall software, we added a complete re-imagining of the Streamer idea in a product caller Streamer+. This new product was built from the ground up for performance, allowing more streams, more desktop captures, more capture card inputs, and higher resolution and frame rate than the original Streamer on supported hardware. The new Streamer+ has been the go-to choice for new installations and even upgrade customers since it was released.

Streaming Evolution

While Sender and Streamer are well suited to sending desktops and captured streams to the Hiperwall, our customers also wanted to send networked video streams from IP Cameras, video encoder boxes, and VMS gateways to their Hiperwall video walls. Since we already had a powerful synchronized streaming protocol from the Streamer, we adapted that to make HiperSource IP Streams. The IP Streams source software can ingest many types of network streams from ONVIF cameras, RTSP encoders, RTP and HTTP sources, and more. It then wraps the streams with our patented approach to synchronize playback across the multiple displays of the Hiperwall video wall and delivers the streams to the display computers. Because the IP Streams software is so efficient, a single moderate PC can ingest and process around 75 streams for simultaneous delivery and playback on a Hiperwall system.

With the addition of HiperSource IP Streams, and to provide more flexibility to our customers, we combined all the source license types into a single HiperSource license type. Therefore, customers could easily switch sources as their needs changed and no further license updates were needed. This simple source interchangeability has worked very well for our customers who may not know exactly what they want to use when they are defining their system. Now they can pick and choose and change as needed.

IP Streams has become a significantly popular source type for many of our clients’ applications. Some need to display streams from IP cameras, possibly via a VMS gateway, such as those provided by Milestone and Genetec. Some customers in secure or otherwise restricted environments use video encoder boxes to take the HDMI output of a computer and convert it to an RTSP stream that the IP Streams software delivers to the Hiperwall. Thus, their secure, mission-critical computers never have to be on the same network as the Hiperwall system, and the only interface to the Hiperwall network is a video cable. IP Streams also delivers content from TV decoder boxes, now that MPEG 2 streams are supported.

Large scale revolution

While Sender and Streamer have been able to capture web pages and send them to the video wall from the start, we wanted to take web content to the next level, so we developed HiperSource Browser. It is a real web browser, based on the Chromium engine, so it supports current web technologies, but it scales in amazing ways. Like most web browsers, it supports tabs with different content (web pages or PDFs, etc.), but each tab is actively rendered and sent to the Hiperwall video wall simultaneously. This means one PC can deliver several web page sources at once while the user does something else entirely. HiperSource Browser also scales in size, allowing a single web page capture to be huge. A control room customer with an enormous video wall uses HiperSource Browser to send several dashboards with 10s of millions of pixels each to their giant Hiperwall. That isn’t a typo – tens of millions pixels worth of custom dashboard data each! Other customers use Browser for normal-size web content.

Because the world has migrated from custom applications to web-based application and dashboards, HiperSource Browser has an incredibly bright future as more and more customers switch over to modern infrastructure. Its flexibility to send multiple content items of differing, possibly enormous, size makes it strongly in demand. And because it works in virtual machine environments and doesn’t interfere with other uses of the PC, it is very friendly to the staff and operators that use Hiperwall video walls. Browser is built on the Sender protocol, so it can send across the Internet via an encrypted channel and HiperCast can deliver Browser streams to multiple Hiperwall systems at once.

The future

The current broad range of sources described here support our customers’ current use cases, but technology is always changing. Performance and feature improvements are obvious next steps, but we are always examining new use cases and customer needs. New streaming protocols are becoming popular for both audio and video, so we are watching market acceptance of those. Integration with collaboration or other products could also be in the cards as the world recovers from disruptions over the last years. We have great sources to get your content to your Hiperwall video wall system, but we’re far from done.

Hiperwall video walls have supported playing content with audio since the beginning, but we had avoided adding audio controls to the system because of the complexities described in this article. That changed with a product version we released several months ago. This article describes the journey the development team took to make the robust and powerful audio controls in the current product. Customers with maintenance contracts can upgrade to the current version and get these new audio management capabilities for no additional cost.

Why Audio?

Hiperwall makes video wall software, so why is there a need for audio in such a visual medium? Many of our customers use their Hiperwall video wall in a control room environment, where audio might be a distraction. Therefore, whatever audio features we add must allow operators to easily mute the system to avoid disturbing the people monitoring and using the control room video wall. Many other customers, however, use their Hiperwall video wall in some sort of public (or employee) facing application where audio is commonly used. In the past, we advised such customers to use a multi-input audio mixer to manage their audio with fine-grain control.

Why is Audio Management a Challenge?

The Hiperwall video wall software is a distributed and parallel computing system that uses multiple display computers to draw content on video wall displays, including LCD, projector, and LED walls, which allows it to scale from very small systems to enormous systems with hundreds of displays or thousands of square feet of LED tiles. Each of those display computers can have audio output and perhaps even speakers attached. Beyond that, a Hiperwall video wall can have tens or hundreds of sources and content items, many of which can have audio channels. Therefore, video wall audio control not only has to manage display computer output volume, but also volume levels for perhaps hundreds of content items, all in an easy-to-use and scalable manner.

Project Beginnings

One of our significant partners expressed the need for volume control for the display computers, because some of their customer projects could use those capabilities. Having looked into volume controls in the past, I thought, “I know how to do that. I’ll do it.”

In the early days of Hiperwall software development, I made the Hiperwall Daemon, a small program that runs on each of the display computers to manage them, transfer content and issue commands, and make sure the display software is running. Having the Daemon control the output volume of its display computer is easy but communicating to make it do so was the challenge. I enhanced the state transfer protocol to allow individual volume control for each of the possibly many displays in the system. The protocol needed to be concise, yet scalable, so the mechanism I added extended an existing control scheme to add volume values when needed. I also added a volume control slider and status reporting to the “Walls” tab in the controller software so volume levels can be quickly and easily changed for one display or an entire wall at a time. The volume values also had to be consistent between fault-tolerant controllers, so I made sure the controllers coordinated.

Content Audio Control Development

After discussing this new capability with the team, we decided that we couldn’t just stop at device volume control, but needed to address content volume as well. Managing volume of individual content items is more technically challenging than just changing device output volume. Because each display computer can have many contents playing audio simultaneously, each object needs separate audio controls both on the controller and built into the display software. Managing audio levels for multiple items is a lot like using an audio mixer like the Windows mixer (right-click on the speaker in your system tray and choose “Open Volume Mixer” to see what I mean). Each object can have a volume setting, but the display computer volume limits that volume to its maximum.

The development team members who make the display software developed mechanisms to adjust the volume level for each object open on their display(s), and that worked very well. We added object volume to the extensive list of properties we manage for each object (like size, position, rotation, transparency, etc.). We added a volume slider to the controller so the operator could change the volume of a selected content object. We the made mechanisms to propagate the volume properties throughout the system and between fault-tolerant controllers. We also extended the existing Environment mechanism so volume levels are saved with each Environment and restored when the Environment is loaded. We even added volume control to our XML-based web services style interface so third-party programs can control content volume. But we weren’t done yet…

Audio Muting Policies

Being able to quickly and easily mute audio content is critical – we do it all the time with our TVs and radios and phones, but when you have many outputs and potentially hundreds of audio sources, it becomes a lot more complicated. We first had to decide which operations made sense. Of course, we needed the ability to mute all content, but we had to decide what it meant to mute audio. Was it just setting the volume of all the content items to 0? That’s easy, but it doesn’t remember the old value in case we want to unmute some or all of the content. Your TV remembers the volume it was before it was muted, so we decided customers would expect that kind of behavior, even though a TV is only playing one thing at a time, while we can show many items at once. Therefore, we had to make muting reversible rather than just setting an object’s volume to 0. This meant maintaining a bit more state and a lot more logic to mute and unmute properly. Of course, muted objects can be unmuted as a group or individually.

Beyond muting everything, we also devised a “mute all but this” mechanism, which allows the operator to define an object that should be the focus of attention by eliminating any potentially distracting audio from other source. While this is not something TVs would have, it is very beneficial in a video wall environment with many sources that include audio.

Convenient and Simple Controls

Because audio has the potential to be disruptive and distracting, we had to make sure the controls to mute everything were easy to find and always available. In addition to the “Mute All” capability as part of the audio controls, we added very obvious system Mute button to the controller so it can be used quickly and easily. Like its “Mute All” counterpart, this Mute button can mute or unmute all the content at once. It has intuitive icons to show its state rather than words like the button in the audio controls.

In addition to the fault tolerance controller software, we have another way of controlling the video walls called HiperOperator. This is a very easy-to-use graphical application that several possibly remote operators to manipulate content on the video wall simultaneously. In keeping with the simplicity of HiperOperator, we had to add very simple audio controls to it. HiperOperator makes it very easy to manipulate properties of individual objects, including applying filters, rotating, and even making the transparent using a simple menu. We added audio volume and muting to the menu, making it extremely convenient to set audio properties of content items. We also added the object’s volume level to the descriptive label shown with each object in HiperOperator.

As with the controllers, HiperOperator needed a very simple way to mute and unmute all audio in the system, so we added a button/icon in the corner of the display that toggles the muted state when clicked. All of these volume and mute states need to be coordinated across all controllers and HiperOperators, too, so if one user performs a volume control operation, it is reflected everywhere. As anyone who builds distributed systems knows, such coordination is easier said than done, but because we were building on already robust protocols and communications links, it turned out very nicely.

Development of audio control capabilities was a very interesting and challenging activity, but more for policy reasons than technical ones. The underlying technology to change display computer output volume or individual object volume is not particularly difficult. Rather, defining behaviors and capabilities was by far more challenging. Since multi-input, multi-output audio is rare except in audio mixers, we had to do lots of prototyping and debating of how things should work. We worked with our partners to get feedback on our designs and made some changes based on their suggestions. The new audio features are the result of a great collaborative effort among the dev team and with our technical services group and our partners.