Profiling program and user behaviors is an effective approach for detecting hostile attacks to a computer system. A new model based method by non-negative matrix factorization (NMF) is presented in this paper to profile program and user behaviors for anomaly intrusion detection. In this new method, the audit data streams obtained from sequences of system calls and UNIX commands are used as the information source. The audit data is partitioned into segments with a fixed length. Program and user behaviors are, in turn, measured by the frequencies of individual system calls or commands embedded in each segment of the data, and NMF is applied to extract the features from the blocks of audit data associated with the normal behaviors. The model describing the normal program and user behaviors are built based on these features and deviation from the normal program and user behaviors above a predetermined threshold is considered as anomalous. The method is implemented and tested with the system call data from the University of New Mexico and the Unix command data from AT&T Research lab. Experiment results show that the proposed method is promising in terms of detection accuracy, computational expense and implementation for real-time intrusion detection.