The omni-directional image has more advantages in that it provides a wider view than a single directional camera and able to realize various new video applications including flexible TV conferencing system, surveillance video system, Web camera systems. In this paper, a new video streaming system by high definition flexible middleware for omni-directional video transmission is introduced, we describe system architecture and functions of the middleware for high-definition omni-directional image control and effective video transmission system using DV and HDV (1080i format). Qos control function is also introduced to dynamically control the frame rate of video depending on the changes of network and CPU load. The prototype system of a TV conference and surveillance system is constructed to evaluate the performance of our suggested high-definition omni-directional system. Through the performance evaluation of the prototyped system, we could verify the usefulness of our proposed system.