The architecture of an application specific instruction set processor (ASIP) for an orientation tracking system to be interfaced with 6D inertial measurement units was developed. A sensor fusion algorithm with a simplified Kalman filter was implemented. The algorithm was modeled with MathWorks Simulink and the architecture of an application specific signal processor was specifically designed to process the filter algorithm. The ASIP was mapped on TSMC 0.18 μm technology to be integrated within the conditioning logic of a 6D IMU sensor. A proof-of-concept demonstrator on FPGA technology was implemented and successfully verified.