Beyond's ultra-clear return is not just for feelings

On the evening of July 3, a unique concert was launched on Douyin, Xigua Video, Toutiao, and Xianshi TV, causing 140 million music fans to collectively nostalgic. This is a selected re-release of the "Beyond Live 1991 Life Contact Concert" and commemorative concert. Beyond is a shining pearl in the history of the development of Chinese rock music and a youthful memory of a generation.

And this time when hundreds of millions of viewers were immersed in the nostalgia of DNA awakened, many friends may not know that the pictures and sounds they saw and heard that they had traveled through 31 years were actually repaired through ultra-clearity. It’s just that many people haven’t discovered it under the authentic restoration of the original flavor.

The one who completed this ultra-clear repair is Volcano Engine, a cloud service platform under ByteDance.

Ultra-clear repair is not only to improve the clarity of the sound and picture

If you have watched the original video of Beyond Live 1991 Life Contact Concert, you can truly feel the improvement of this ultra-clear repair version. For example, the following comparison picture of the effect before and after repair is a visible improvement.

It is indeed a blessing to still see such a clear version of this concert after many years. I believe some friends will be curious about what magic did Douyin cast? How did the volcanic engine be repaired?

I might as well explain it to you here.

First of all, because it is a concert repair, it will inevitably be divided into two parts: picture quality repair and sound quality repair.

In terms of image quality repair, let’s first look at the difficulties.

The first is that the blurring of the picture is damaged. In the early days, due to poor technical conditions, the picture will be blurred and noise-free during the entire process of shooting, production, compression, transmission, etc. of the concert. For example, the videos at that time were generally interlaced, and the encoding and display would bring flaws such as interlaced stripes.

The second is the unique color, brightness and stage atmosphere of the concert. For example, how to balance the contradiction between brightness adjustment and noise floor amplification is also a difficult point.

The third is that the portrait face needs to be finely processed, which is difficult to achieve with ordinary image quality repair algorithms, and adjusting the difference between portrait effects and background is also a difficult point.

These are the core difficulties in concert repair. For these difficulties, Volcano Engine has developed special algorithm technology to overcome them. For example, the problem of blurred picture is damaged. They have clarity enhancement and defect repair algorithms. These are deep learning algorithms trained based on a large amount of data to improve resolution.At the same time, it can also generate rich details in areas with missing textures.

At the same time, deep learning multi-frame interleaving algorithm based on motion compensation can also be used to solve the brushing and transverse patterns of videos in various rows of scanning formats.

The photo below shows the contrast between before and after the repair. You can clearly see that there are horizontal lines and blurred hands on the guitar before the repair. After the repair, the whole whole is obviously clearer and sharper.

While improving the picture quality, we must also pay attention to the preservation of the artistic atmosphere of the concert. Volcano Engine provides an adaptive color brightness enhancement algorithm based on aesthetic scores. It will enhance the color, brightness, contrast, saturation, portrait ROI and background areas according to the effect of the film source, so that both the bright and dark areas can achieve the best performance effect.

For example, the comparison before and after the restoration of the picture below, the colors after the restoration are obviously fuller, and the overall style does not lose the atmosphere of the concert lighting.

In terms of portrait restoration, the Volcano Engine uses its self-developed adaptive portrait enhancement algorithm to reconstruct the characters' high-definition facial features. It has a good repair and enhancement effect for faces of different sizes (close/medium/long vision), different postures (side face/down/upward), and different occlusion levels.

There is also similar in sound quality repair. The original concert video naturally has poor sound quality caused by poor equipment conditions in the early stages, as well as interference caused by environmental noise. In addition, the equipment and compression algorithms at that time were immature, which would also lead to low audio cutoff frequency. Excessive reverberation of the concert affects the quality of the sound, and the hearing discomfort caused by uneven loudness. These are the difficulties in sound quality repair.

For the noise of the original audio, Volcano Engine has provided a self-developed noise reduction algorithm for multi-element scenes, which can achieve compatible AI noise reduction for music scenes and vocal scenes. Ordinary noise reduction algorithms are mainly aimed at vocals, which inevitably damage some sounds when reducing noise in concerts. The volcano engine's algorithm can retain people to speak/sing/music while suppressing other noises.

For example, in the comparison picture below, the original audio spectrum carrying noise is very turbid, and the noise (purple part) exists throughout the concert. After processing by the AI audio noise reduction algorithm, the spectrum graph is much clearer.

Volcano Engine also has an audio overscore algorithm to solve the problem of insufficient audio bandwidth in early devices. It can expand the frequency band of the vocal part in the concert, enrich high-frequency information, improve sound quality, and will not introduce other distortions during the processing to make the experience worse.

In addition, there is a loudness algorithm, which extracts the singing vocal part in the concert and then adjusts the loudness to make the overall vocal more comfortable.

Of course, after the re-screening of this concert, IT Home also learned from Volcano Engine that the restoration work is not entirelyThe parts that are processed by AI algorithms will also be processed manually, but very little is involved. For example, when restoring the image quality, some manual adjustments will be added to color-related parts, while some manual details are mainly adjusted to adjust the loudness.

In general, most of the restoration of this concert was completed through AI algorithm technology, and we can also see that the restoration of the music and picture clarity is not only as simple as improving the clarity of the sound and picture, but also involves the preservation of the artistic atmosphere, noise, details selection, color adjustment, recoding, etc., which is a systematic and complex process. As for this process, volcanic engines can already use technology to complete most of the process without manual participation.

What was finally presented to us was a classic concert with full rejuvenation of sound and painting, and an authentic and artistic sense, which created a carnival of emotion.

What is more important than feelings is empowerment

In terms of effect, the ultra-clear repair of this Beyond concert is amazing. After reading the above introduction, I believe that friends will naturally think that these repair technologies are so powerful and rarely require manual participation. So if they can be opened up so that more subjects can participate in this meaningful repair, wouldn’t it be better?

In fact, these are what Volcano Engine Video Cloud is doing, and they provide more capabilities.

I believe everyone can also feel the importance of video in today's Internet media era. IDC released a report "China Video Cloud Market Tracking (First Half of 2021)" at the beginning of the year, which showed that the size of China's video cloud market in the first half of 2021 reached US$4.37 billion, a year-on-year increase of 38.7%. We are welcoming a brand new "super video era": whether it is from enterprises to users or users to users, more than 50% of the information transmission in an industry is carried through video, which can be considered as the "super video era".

In short, in the context of a wide range of business scenarios moving from graphics to video, video is becoming a new productivity, and all walks of life are thinking about how to build their own video capabilities.

Volcano Engine, a cloud service platform under ByteDance, just brings complete video capabilities to all industries through video cloud. The relevant technologies involved in this repair of Beyond concert are actually just the "tip of the iceberg" of the volcano engine video cloud capabilities.

According to the latest video cloud product matrix released by Volcano Engine in February this year, this is a core product including pan-Internet, games, finance, radio and television scenarios, video on demand, veImageX, real-time audio and video, etc., to form a complete solution. At the lowest levelTaiwan is the technical capability accumulated and accumulated by Volcano Engine in serving excellent applications such as Douyin and Xigua Video, covering the full link of network transmission, intelligent production and intelligent processing.

As far as subdividing each capability, Volcano Engine Video Cloud also has corresponding technical advantages. For example, in terms of encoding, Volcano Engine's BVC series encoder won 17 championships in the world's top video encoder event MSU2020.

In terms of video playback experience, Volcano Engine has the original "zero first frame" optimization, which achieves the start of short video first frame less than 100ms and the start of long video first frame less than 400ms, bringing a smooth playback experience without perception.

Their VQScore image quality evaluation algorithm system has also won the first place in the world-class academic conference in multimedia field ICME2021, and in terms of stability, it can reduce the application crash rate to less than 1/100,000.

Precisely because of these technological advantages, Volcano Engine Video Cloud can provide the ultimate video experience for different industries. In addition, in terms of video experience, Volcano Engine has also established its own data system standards, and has made detailed optimizations from four aspects: interactive experience, playback experience, picture quality experience, and performance experience, so that the user experience can be quantified and optimized.

For example, many "super new Generation Z" now like to watch "comic dramas" on Kuaikan. Behind this new video content form is the help of Volcano Engine Video Cloud. Volcano Engine Video Cloud provides powerful video editing for Kuaikan’s “comic drama” creation, promoting the prosperity of UGC content, and also provides Kuaikan with relevant optimization experience with zero first frames, improving the user’s viewing experience.

Quick Watch World Technology VP Anchangsi

For example, in May this year, Pico held an interactive live broadcast of Zheng Jun’s “We Are” VR private singing and chat conference in the form of 8K 3D VR. Volcano Engine Video Cloud provides Pico with full link capabilities from VR acquisition, VR encoding and streaming, VR cloud director, virtual reality special effects superimposed to VR players.

Zheng Jun’s “We Are” VR private singing and chat meeting

In general, Volcano Engine Video Cloud has served many industries, covering various fields such as games, e-commerce, finance, and education. Including the various technologies we mentioned earlier used in the repair of Beyond concerts, they have already been very mature in all walks of life. Perhaps, it is very emotional to repair the Beyond concert and have a free nostalgic carnival with everyone, but the editor thinks that being able to open up these abilities and empowering thousands of subjects who need to use video to build productivity is more important than sentiment.

Conclusion

Beyond Live 1991 The re-release of the Life Contact Concert has ended, but the editor believes that the sentiment of using technology to restore culture will not stop. Last year, Douyin and Xigua Video and Volcano Engine have restored more than 100 classic animations for us. This time is Beyond’s classic concert. In the future, perhaps more works that have been eroded by time will regain their vitality.

Such things are far from enough to rely on a few companies to complete. On the contrary, more companies or entities need to participate. The empowerment of Volcano Engine Video Cloud will undoubtedly play a very important role in promoting such valuable things.