Processing math: 100%
Nikos Nikoleris - IEEE Xplore Author Profile
Subscribe
ADVANCED SEARCH

Author details

< Back

Nikos Nikoleris

Publications
18
Citations
174
Publications by Year
20112024
Co-Authors:
Show All Co-Authors (32)
Headshot silhouette indicating that no photo is available for this author

Nikos Nikoleris

Affiliation

Arm

Publication Topics

Biography

Nikos Nikoleris is a senior principal engineer at Arm, Cambridge, CB1 9NJ, U.K. His research interests include concurrency and system architecture. Nikoleris received his Ph.D. degree in computer science from Uppsala University. Contact him at nikos.nikoleris@arm.com.(Based on document published on 9 July 2024).
Publications
18
Citations
174
Publications by Year
20112024
Co-Authors:
Show All Co-Authors (32)
Author's Published Works

Showing 1-25 of 130 results

  • Sort

Filter Results

Show

Sort By

Results

Fast Dependent Quantization Using Trellis Pruning, Forward Context Adaptation and Vectorization

Adam Wieckowski;Christian Stoffers;Heiko Schwarz;Benjamin Bross;Detlev Marpe

2024 32nd European Signal Processing Conference (EUSIPCO)
Year: 2024 | Conference Paper |
Dependent quantization (DQ) is one of the key coding tools in the Versatile Video Coding (VVC) standard. DQ employs two scalar quantizers, with the per-coefficient quantizer selection being governed by a parity-driven 4-state state machine. As the design is normatively enforced, usage of DQ requires a rate-distortion optimized quantization (RDOQ) with a per coefficient decision and state update, e...Show More

Fast Dependent Quantization Using Trellis Pruning, Forward Context Adaptation and Vectorization

Adam Wieckowski;Christian Stoffers;Heiko Schwarz;Benjamin Bross;Detlev Marpe

Year: 2024 | Conference Paper |

Optimizing Learned Image Compression On Scalar and Entropy-Constraint Quantization

Florian Borzechowski;Michael Schäfer;Heiko Schwarz;Jonathan Pfaff;Detlev Marpe;Thomas Wiegand

2024 IEEE International Conference on Image Processing (ICIP)
Year: 2024 | Conference Paper |
The continuous improvements on image compression with variational autoencoders have lead to learned codecs competitive with conventional approaches in terms of rate-distortion efficiency. Nonetheless, taking the quantization into account during the training process remains a problem, since it produces zero derivatives almost everywhere and needs to be replaced with a differentiable approximation w...Show More

Optimizing Learned Image Compression On Scalar and Entropy-Constraint Quantization

Florian Borzechowski;Michael Schäfer;Heiko Schwarz;Jonathan Pfaff;Detlev Marpe;Thomas Wiegand

Year: 2024 | Conference Paper |

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding

Philipp Merkle;Martin Winken;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

IEEE Transactions on Image Processing
Year: 2024 | Volume: 33 | Journal Article |
Cited by: Papers (1)
This paper presents a convolutional neural network (CNN)-based enhancement to inter prediction in Versatile Video Coding (VVC). Our approach aims at improving the prediction signal of inter blocks with a residual CNN that incorporates spatial and temporal reference samples. It is motivated by the theoretical consideration that neural network-based methods have a higher degree of signal adaptivity ...Show More

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding

Philipp Merkle;Martin Winken;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2024 | Volume: 33 | Journal Article |

Nonlinear Transform Coding for VVC Intra Coding

Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2024 Picture Coding Symposium (PCS)
Year: 2024 | Conference Paper |
Cited by: Papers (1)
Modern hybrid video codecs like Versatile Video Coding (VVC) heavily rely on transform coding tools. Given a prediction signal at the encoder, the residual is transformed using trigonometric transforms. Rate-distortion-optimized quantization (RDOQ) and entropy coding of the transformed residual is well-understood due to the orthogonality and the energy compaction of these transforms. Within this s...Show More

Nonlinear Transform Coding for VVC Intra Coding

Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2024 | Conference Paper |

Deep Video Compression with Conditional Feature Coding

Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2024 Picture Coding Symposium (PCS)
Year: 2024 | Conference Paper |
In the last years, deep video coding has attracted a lot of research interest. Usually, it employs the concept of inter coding by transmitting features in a latent space that represent a motion field or a residual. However, in such a setting there are still redundancies between the features of consecutive frames. In previous approaches, these redundancies are exploited for compression by adding an...Show More

Deep Video Compression with Conditional Feature Coding

Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2024 | Conference Paper |

Simplified CNN In-Loop Filter with fixed Classifications

Wang-Q Lim;Björn Stallenberger;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2024 Picture Coding Symposium (PCS)
Year: 2024 | Conference Paper |
In this paper, we present a novel in-loop filter for video coding which is based on a convolutional neural network (CNN). For that, the adaptive loop filter (ALF) of Versatile Video Coding (VVC) is generalized to define the model architecture for a CNN-based in-loop filter which requires significantly lower computational complexity compared to other existing CNN-based in-loop filters. Experimental...Show More

Simplified CNN In-Loop Filter with fixed Classifications

Wang-Q Lim;Björn Stallenberger;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2024 | Conference Paper |

Neural Network Coding of Difference Updates for Efficient Distributed Learning Communication

Daniel Becking;Karsten Müller;Paul Haase;Heiner Kirchhoffer;Gerhard Tech;Wojciech Samek;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

IEEE Transactions on Multimedia
Year: 2024 | Volume: 26 | Journal Article |
Cited by: Papers (2)
Distributed learning requires a frequent communication of neural network update data. For this, we present a set of new compression tools, jointly called differential neural network coding (dNNC). dNNC is specifically tailored to efficiently code incremental neural network updates and includes tools for federated BatchNorm folding (FedBNF), structured and unstructured sparsification, tensor row sk...Show More

Neural Network Coding of Difference Updates for Efficient Distributed Learning Communication

Daniel Becking;Karsten Müller;Paul Haase;Heiner Kirchhoffer;Gerhard Tech;Wojciech Samek;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2024 | Volume: 26 | Journal Article |

Block-Based Motion Estimation for Deep-Learned Video Coding

Sophie Pientka;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2023 IEEE International Conference on Image Processing (ICIP)
Year: 2023 | Conference Paper |
Cited by: Papers (1)
The research on deep-learned end-to-end video compression has attracted a lot of attention over the course of recent years. A central component of many approaches is to perform motion-compensated prediction by using convolutional neural networks (CNN) which determine a compressed representation of the motion field as features. Often, this task is divided into searching motion vectors by one networ...Show More

Block-Based Motion Estimation for Deep-Learned Video Coding

Sophie Pientka;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2023 | Conference Paper |

Bitrate-Performance Optimized Model Training for the Neural Network Coding (NNC) Standard

Paul Haase;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2023 IEEE International Conference on Image Processing (ICIP)
Year: 2023 | Conference Paper |
In August 2022, ISO/IEC MPEG published the first international standard on compression of neural networks, namely Neural Network Coding (NNC, MPEG-7 part 17). It compresses neural networks to about 5% to 15% in size at virtually no performance loss. In NNC, the model weights are usually quantized and then encoded into the bitstream using DeepCABAC entropy coding. In order to improve the coding eff...Show More

Bitrate-Performance Optimized Model Training for the Neural Network Coding (NNC) Standard

Paul Haase;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2023 | Conference Paper |

A Study on Data-Driven Probability Estimator Design for Video Coding

Heiner Kirchhoffer;Christian Rudat;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2023 Data Compression Conference (DCC)
Year: 2023 | Conference Paper |
Cited by: Papers (1)
Data-driven optimization is employed to study alternative approaches [1] to the probability estimator of the the Enhanced Compression Model (ECM) (which includes additional coding tools on top of the Versatile Video Coding standard). In ECM, each context model uses a weighted sum of two hypotheses for probability estimation with different associated adaptation rates. Four alternative approaches ar...Show More

A Study on Data-Driven Probability Estimator Design for Video Coding

Heiner Kirchhoffer;Christian Rudat;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2023 | Conference Paper |

Region-Based Template Matching Prediction for Intra Coding

Gayathri Venugopal;Karsten Müller;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

IEEE Transactions on Image Processing
Year: 2023 | Volume: 32 | Journal Article |
Cited by: Papers (4)
Copy prediction is a renowned category of prediction techniques in video coding where the current block is predicted by copying the samples from a similar block that is present somewhere in the already decoded stream of samples. Motion-compensated prediction, intra block copy, template matching prediction etc. are examples. While the displacement information of the similar block is transmitted to ...Show More

Region-Based Template Matching Prediction for Intra Coding

Gayathri Venugopal;Karsten Müller;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2023 | Volume: 32 | Journal Article |

Performance-Complexity Analysis of Adaptive Loop Filter with a CNN-based Classification

Wang-Q Lim;Bjöorn Stallenberger;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2022 Picture Coding Symposium (PCS)
Year: 2022 | Conference Paper |
Cited by: Papers (2)
Recently, convolutional neural network (CNN)-based in-loop filters have been introduced for video coding and they show huge coding gains. However, one of the main issues of this approach is the high computational complexity of these filters. In this paper, we present various settings for CNN-based in-loop filters targeting on the reduction of their decoder complexity and describe the corresponding...Show More

Performance-Complexity Analysis of Adaptive Loop Filter with a CNN-based Classification

Wang-Q Lim;Bjöorn Stallenberger;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

Deep video coding with gradient-descent optimized motion compensation and Lanczos filtering

Sophie Pientka;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2022 Picture Coding Symposium (PCS)
Year: 2022 | Conference Paper |
Cited by: Papers (1)
Variational autoencoders have shown promising results for still image compression and have gained a lot of consideration in this field. Recently, noteworthy attempts were made to extend such end-to-end methods to the setting of video compression. Here, low-latency scenarios have been commonly investigated. In this paper, it is shown that the compression efficiency in this setting is improved by ap...Show More

Deep video coding with gradient-descent optimized motion compensation and Lanczos filtering

Sophie Pientka;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

History Dependent Significance Coding for Incremental Neural Network Compression

Gerhard Tech;Paul Haase;Daniel Becking;Heiner Kirchhoffer;Karsten Müller;Jonathan Pfaff;Heiko Schwarz;Wojciech Samek;Detlev Marpe;Thomas Wiegand

2022 IEEE International Conference on Image Processing (ICIP)
Year: 2022 | Conference Paper |
Cited by: Papers (1)
This paper presents an improved probability estimation scheme for the entropy coder of Incremental Neural Network Coding (INNC), which is currently under standardization in ISO/IEC MPEG. More specifically, the paper first analyzes the compression performance of INNC and how the bitstream size relates to the neural network (NN) layers. For the layers requiring the most bits, it analyzes the coded N...Show More

History Dependent Significance Coding for Incremental Neural Network Compression

Gerhard Tech;Paul Haase;Daniel Becking;Heiner Kirchhoffer;Karsten Müller;Jonathan Pfaff;Heiko Schwarz;Wojciech Samek;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

Trellis-Coded Quantization for End-to-End Learned Image Compression

Karsten Sühring;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2022 IEEE International Conference on Image Processing (ICIP)
Year: 2022 | Conference Paper |
Cited by: Papers (1)
The performance of variational auto-encoders (VAE) for image compression has steadily grown in recent years, thus becoming competitive with advanced visual data compression technologies. These neural networks transform the source image into a latent space with a channel-wise representation. In most works, the latents are scalar quantized before being entropy coded. On the other hand, vector quanti...Show More

Trellis-Coded Quantization for End-to-End Learned Image Compression

Karsten Sühring;Michael Schäfer;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

Intra-Inter Prediction for Versatile Video Coding Using a Residual Convolutional Neural Network

Philipp Merkle;Martin Winken;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2022 IEEE International Conference on Image Processing (ICIP)
Year: 2022 | Conference Paper |
Cited by: Papers (5)
This paper presents a convolutional neural network (CNN) based solution for inter prediction in Versatile Video Coding (VVC). Our approach aims at improving the motion-compensated prediction signal of inter blocks with a residual CNN that incorporates spatial and temporal reference samples, i.e., intra-inter prediction. It is motivated by two considerations. Firstly, incorporating intra reference ...Show More

Intra-Inter Prediction for Versatile Video Coding Using a Residual Convolutional Neural Network

Philipp Merkle;Martin Winken;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

Adaptive Loop Filter with a CNN-Based Classification

Wang-Q Lim;Jonathan Pfaff;Björn Stallenberger;Johannes Erfurt;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2022 IEEE International Conference on Image Processing (ICIP)
Year: 2022 | Conference Paper |
Cited by: Papers (5)
In this paper, a data-driven generalization of the adaptive loop filter (ALF) of Versatile Video Coding (VVC) is presented. It is shown how the conventional ALF process of classification and FIR filtering can be generalized to define a natural model architecture for convolutional neural network (CNN) based in-loop filters. Experimental results show that over VVC, average bit-rate savings of 3.85%/...Show More

Adaptive Loop Filter with a CNN-Based Classification

Wang-Q Lim;Jonathan Pfaff;Björn Stallenberger;Johannes Erfurt;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2022 | Conference Paper |

Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning

Daniel Becking;Heiner Kirchhoffer;Gerhard Tech;Paul Haase;Karsten Müller;Heiko Schwarz;Wojciech Samek

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Year: 2022 | Conference Paper |
Cited by: Papers (6)
Federated learning (FL) scenarios inherently generate a large communication overhead by frequently transmitting neural network updates between clients and server. To minimize the communication cost, introducing sparsity in conjunction with differential updates is a commonly used technique. However, sparse model updates can slow down convergence speed or unintentionally skip certain update aspects,...Show More

Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning

Daniel Becking;Heiner Kirchhoffer;Gerhard Tech;Paul Haase;Karsten Müller;Heiko Schwarz;Wojciech Samek

Year: 2022 | Conference Paper |

Rate-Distortion Optimized Encoding for Deep Image Compression

Michael Schäfer;Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

IEEE Open Journal of Circuits and Systems
Year: 2021 | Volume: 2 | Journal Article |
Cited by: Papers (4)
Deep-learned variational auto-encoders (VAE) have shown remarkable capabilities for lossy image compression. These neural networks typically employ non-linear convolutional layers for finding a compressible representation of the input image. Advanced techniques such as vector quantization, context-adaptive arithmetic coding and variable-rate compression have been implemented in these auto-encoders...Show More

Rate-Distortion Optimized Encoding for Deep Image Compression

Michael Schäfer;Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2021 | Volume: 2 | Journal Article |

Encoder Optimizations For The NNR Standard On Neural Network Compression

Paul Haase;Daniel Becking;Heiner Kirchhoffer;Karsten Müller;Heiko Schwarz;Wojciech Samek;Detlev Marpe;Thomas Wiegand

2021 IEEE International Conference on Image Processing (ICIP)
Year: 2021 | Conference Paper |
Cited by: Papers (6)
The novel Neural Network Compression and Representation Standard (NNR), recently issued by ISO/IEC MPEG, achieves very high coding gains, compressing neural networks to 5% in size without accuracy loss. The underlying NNR encoder technology includes parameter quantization, followed by efficient arithmetic coding, namely DeepCABAC. In addition, NNR also allows very flexible adaptations, such as sig...Show More

Encoder Optimizations For The NNR Standard On Neural Network Compression

Paul Haase;Daniel Becking;Heiner Kirchhoffer;Karsten Müller;Heiko Schwarz;Wojciech Samek;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

Rate-Distortion-Optimization for Deep Image Compression

Michael Schäfer;Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2021 IEEE International Conference on Image Processing (ICIP)
Year: 2021 | Conference Paper |
Cited by: Papers (7)
Given the capabilities of massive GPU hardware, there has been a surge of using artificial neural networks (ANN) for still image compression. These compression systems usually consist of convolutional layers and can be considered as non-linear transform coding. Notably, these ANNs are based on an end-to-end approach where the encoder determines a compressed version of the image as features. In con...Show More

Rate-Distortion-Optimization for Deep Image Compression

Michael Schäfer;Sophie Pientka;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

CNN-based parameter selection for fast VVC intra-picture encoding

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

2021 IEEE International Conference on Image Processing (ICIP)
Year: 2021 | Conference Paper |
Cited by: Papers (6)
This paper presents two new methods for fast VVC intra-picture encoding. Both are based on an approach that uses a CNN for blockadaptive parameter estimation. The parameters restrict the multitype-tree (MTT) partitionings tested by the encoder. The methods aim for an improvement of the approach by further constraints with additional parameters. Adding parameters increases the time required for tra...Show More

CNN-based parameter selection for fast VVC intra-picture encoding

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

Extension of Matrix-Based Intra Prediction to 4:4:4 Chroma Formats

Björn Stallenberger;Michael Schäfer;Philipp Merkle;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

2021 Picture Coding Symposium (PCS)
Year: 2021 | Conference Paper |
Cited by: Papers (1)
The final version of the Versatile Video Coding (VVC) standard incorporates the Matrix-Based Intra Prediction (MIP) tool. It consists of additional intra prediction modes which were derived from a data-driven training. These modes, in general, are applied to the luma component only. This paper describes how to apply the MIP modes to the chroma components in certain cases. In the generic case, if a...Show More

Extension of Matrix-Based Intra Prediction to 4:4:4 Chroma Formats

Björn Stallenberger;Michael Schäfer;Philipp Merkle;Jonathan Pfaff;Heiko Schwarz;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

Rate-Distortion-Time Cost Aware CNN Training for Fast VVC Intra-Picture Partitioning Decisions

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

2021 Picture Coding Symposium (PCS)
Year: 2021 | Conference Paper |
Cited by: Papers (6)
This paper presents a new method for fast VVC intra-picture encoding using a CNN. The CNN operates on the original samples of \mathbf{32}\times \mathbf{32} blocks. Given a current block, it derives for each of the block's multi-type trees (MTTs), which are nested in quad-tree (QT) nodes, a parameter pair. The parameter pairs constrain the minimum width and height of the sub-blocks in their MTTs....Show More

Rate-Distortion-Time Cost Aware CNN Training for Fast VVC Intra-Picture Partitioning Decisions

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

Fast Partitioning for VVC Intra-Picture Encoding with a CNN Minimizing the Rate-Distortion-Time Cost

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

2021 Data Compression Conference (DCC)
Year: 2021 | Conference Paper |
Cited by: Papers (16)
This paper presents a CNN to reduce the encoding time of a VVC-based intra-picture encoder. For encoding a 32 x 32 block, the CNN estimates two partitioning parameters that restrict the allowed coding block width and height. To estimate them such that the encoder skips testing inefficient partitioning modes, we train the CNN as follows: First, we generate training data by encoding sequences withou...Show More

Fast Partitioning for VVC Intra-Picture Encoding with a CNN Minimizing the Rate-Distortion-Time Cost

Gerhard Tech;Jonathan Pfaff;Heiko Schwarz;Philipp Helle;Adam Wieckowski;Detlev Marpe;Thomas Wiegand

Year: 2021 | Conference Paper |

IEEE Account

Purchase Details

Profile Information

Need Help?

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.