Sensor management in real time and a camera control application for air vehicles

Camera systems are increasingly utilized in Intelligence, Surveillance and Reconnaissance (ISR) missions on manned and unmanned aircraft. For manned missions, the automatic control of the camera field of view and orientation through pan-tilt-zoom (PTZ) commands frees the pilot to perform higher cognitive functions. For unmanned missions, automatic PTZ commands enable higher degrees of autonomy. This paper addresses the problem of controlling the field of view and orientation of cameras mounted on air vehicles so that the resulting images of commanded areas of interest (AOIs) have specified ground sampling distances (GSDs). The problem is complicated by several factors: First, while the instantaneous positions of the vehicles are assumed to be known, their routes, including their altitude profiles, are not known in advance, which precludes preplanning. Second, the sizes of the AOIs are arbitrary, implying that a single look of the camera may not be sufficient to cover them. Third, the slew times of the cameras may be too slow, and the cameras may not operate while PTZ parameters are changing. Finally, given an AOI, the time intervals when the desired GSDs are achievable varies for different cameras. In the paper, we first show that successful collection of an AOI image (i.e. collection with a prescribed GSD) is only possible if the aircraft lies inside a spherical cap depending on the position of the AOI, and the desired GSD. Given a stream of requested AOIs, with corresponding GSDs and values characterizing their importance to a user, we then develop algorithms to service the most valuable ensemble of requests, while satisfying the requested GSDs. Three algorithms are developed: (1) a scheduling algorithm, for deciding which request to service at a given instant of time, (2) an admission control algorithm, enabling a graceful degradation in case of system overload (more requests than available timeline) and (3) an algorithm for deciding which of the platforms (air vehicle and camera) services a given request. We examine each of the three algorithms in turn. The scheduling algorithm is centered on the earliest deadline first (EDF) scheme from the real-time systems literature. Deadlines for servicing AOI requests are determined by computing the intersection between the extrapolated aircraft trajectory and the spherical caps defined earlier. The admission control algorithm is developed based on schedulability tests for EDF. Finally, the algorithm for routing the requests among platforms are based on machine shop scheduling problems, which essentially promote load balancing among the platforms. We introduce a heuristic called min-max utilization, inspired on the makespan minimization procedures developed for machine shop scheduling problems. Extensive simulation results are presented, illustrating the superior performance of the algorithms in contrast to other alternatives.