IEEE Journal on Emerging and Selected Topics in Circuits and Systems

This special issue aims to present recent technical advances in screen content video coding and applications. Screen content video has recently evolved from a niche to the mainstream due to the rapid advances in mobile and cloud technologies. Real-time, low-latency transport of screen visuals between devices in the form of screen content video is becoming prevalent in many applications, e.g., wire... View full abstract»

This paper presents recent advances in screen content video coding, with an emphasis on two state-of-the-art standards: HEVC/H.265 Screen Content Coding Extensions (HEVC-SCC) by ISO/IEC Moving Picture Experts Group and ITU-T Video Coding Experts Group, and Display Stream Compression (DSC) by Video Electronics Standards Association. The HEVC-SCC enhances the capabilities of HEVC in coding screen co... View full abstract»

With the emerging applications such as online gaming and Wi-Fi display, screen content video, including computer generated text, graphics and animations, becomes more popular than ever. Traditional video coding technologies typically were developed based on models that fit into natural, camera-captured video. The distinct characteristics exhibited between these two types of contents necessitate th... View full abstract»

Palette mode is a new coding tool included in the HEVC screen content coding extension (SCC) to improve the coding efficiency for screen contents such as computer generated video with substantial amount of text and graphics. It is observed that a local area in screen content typically has a few colors separated by sharp edges. To exploit such characteristics, palette mode represents samples in a b... View full abstract»

Palette mode and intra block copy are two new coding tools that have been adopted in the screen content coding extensions of high efficiency video Ccoding (HEVC SCC). Palette mode can represent color clusters for screen content efficiently. Intra block copy (IBC) facilitates block prediction within the same frame. Both tools have demonstrated promising benefits for screen content coding. In additi... View full abstract»

The screen content coding (SCC) Extensions of high efficiency video coding (HEVC) employs in-loop adaptive color-space transform (ACT) technique to explore the inter-color-component redundancy, i.e., statistical redundancy among different color components. In ACT, the prediction residual signal is adaptively converted into a different color space, i.e., YCgCo. The rate-distortion criteria is emplo... View full abstract»

The Video Electronics Standards Association (VESA) Display Stream Compression (DSC) standard is widely accepted as a low-cost, visually lossless codec for use in display links. However, as high dynamic range (HDR) and wide color gamut (WCG) technologies become more prevalent, several new features are needed to address television and cinema applications, including support for greater bit depths and... View full abstract»

Today, many existing types of video transmission and storage infrastructure are not able to handle UHD uncompressed video in real time. To reduce the required bit rates, a low-latency lightweight compression scheme is needed. To this end, several standardization efforts, such as Display Stream Compression, Advanced DSC, and JPEG XS, are currently being made. Focusing on screen content use cases, t... View full abstract»

In the latest large-screen TVs, an RGBW color domain is widely used to enhance the brightness of liquid crystal display (LCD) panels. As the correlations among RGBW color components are different from those among conventional RGB components, a compression algorithm developed for an RGB image may not be efficient for RGBW compression. For high-fidelity RGBW image coding, a novel prediction-based co... View full abstract»

Screen content sequences are ubiquitous type of video data in numerous multimedia applications like video conferencing, remote education, and cloud gaming. These sequences are characterized for depicting a mix of computer generated graphics, text, and camera-captured material. Such a mix poses several challenges, as the content usually depicts multiple strong discontinuities, which are hard to enc... View full abstract»

The challenge in data compression is to transmit or store a minimum number of bits per sample that should be as close as possible to the number of bits defined by the actual average information content. The latter can be approximated by the contextual entropy. The dilemma is generally that the optimal estimation of contextual symbol probabilities is a kind of artificial-intelligence problem and re... View full abstract»

This paper presents a fast mode and partition decision framework for screen content coding (SCC) based on machine learning. Extensive statistical studies and complexity evaluations are conducted to explore the distribution of different coding modes and their complexities. The proposed encoder scheme is designed based on the results from these studies. Firstly, a CU is classified as either a natura... View full abstract»

Objectively accessing the quality of screen content images (SCIs) is a challenging problem, as SCIs may not always have identical properties as natural scenes. Here we conduct comprehensive studies on the subjective and objective quality assessment of the compressed SCIs. Firstly, we build a database that contains the distorted SCIs generated by the high efficiency video coding standard as well as... View full abstract»

With the growing adoption of mobile devices as the primary device for information and media consumption, and recent growth in availability of mobile devices with auto-stereoscopic 3D displays, the time has come to enable rich 3D virtual immersive experiences on mobile devices, like 3D display gaming, 3D virtual classroom, 3D virtual art gallery, etc., which are currently enabled on PC platforms. I... View full abstract»

This paper proposes a low cost hardware decoder design and architecture that supports general string matching (SM) using both a primary reference buffer (PRB) and a secondary reference buffer (SRB). An SM decoding module mainly performs string copy operation. The SM decoding module can also support intra block copy and palette decoding because both are actually two special cases of general string ... View full abstract»

This paper considers how to separate text and/or graphics from smooth background in screen content and mixed document images and proposes two approaches to perform this segmentation task. The proposed methods make use of the fact that the background in each block is usually smoothly varying and can be modeled well by a linear combination of a few smoothly varying basis functions, while the foregro... View full abstract»