FPGA-based multiprocessor systems are viable solutions for stream-based embedded applications. They provide a software abstraction which enables coarse-grained parallel deployment on an FPGA chip. A widely used model for such a deployment is the class of Kahn process networks despite their limitation to pure FIFO communications. In this paper, a new mechanism denoted as windowed FIFO is introduced, extending the functionality for data transfer. The new concept allows non-destructive read, reordering, and skipping of data within a communication channel. We present the behavior, the software interface and the hardware design of this mechanism. We introduce our abstraction of WFIFO process network which is suitable for systematic and automated synthesis while still inheriting the nice property of Kahn process networks, i.e. being determinate. Also, we present illuminating examples to demonstrate the practicality of the outlined approach.