Deep Neural Network-Based Landmark Selection Method for Optical Navigation on Lunar Highlands

Spacecraft that rely on self-localization based on optical terrain images require suitable landmark information along their flight paths. When navigating within the vicinity of the moon, a lunar crater is an intuitive choice. However, in highland areas or regions having low solar altitudes, craters are less reliable because of heavy shadowing, which results in infrequent and unpredictable crater detections. This paper, therefore, presents a method for suggesting navigation landmarks that are usable, even with unfavorable illumination and rough terrain, and it provides a procedure for applying this method to a lunar flight plan. To determine a good landmark, a convolutional neural network (CNN)-based object detector is trained to distinguish likely landmark candidates under varying lighting geometries and to predict landmark detection probabilities along flight paths attributable to various dates. Dates having more favorable detection probabilities can be determined in advance, providing a useful tool for mission planning. Numerical experiments show that the proposed landmark detector generates usable navigation information at sun elevations of less than 1.8° in highland areas.


I. INTRODUCTION
A landmark is a perceptually distinctive geographical feature of interest at a particular location and date [1]. In terms of navigation, a landmark can be defined as a recognizable natural or artificial feature. For space flight, an observed topographical feature on an object's surface during navigation is often the means of landmark localization far outside the range of the earth's Global Navigation Satellite System (GNSS).
Historically, all past lunar landers (i.e., Luna, Ranger, Surveyor, Apollo 11,12,14,15,16,and 17 (at 23 • N, 23 • S, 17 • S, 4 • N, 15 • S, and 31 • N, respectively), and Chang'e 3 and 4 (at 44 • N, 45 • S, respectively) arrived at low latitudes of less than approximately 45 • [2]. Most landed around large flat plains (e.g., maria and basins). Such environments at low-latitude areas provide a wide range of visibility The associate editor coordinating the review of this manuscript and approving it for publication was Szidónia Lefkovits . and solar-altitude angles. Easy visibility allows a probe to obtain its own absolute position directly from the earth's ground-tracking station or from additional tracking spacecraft, such as the Queqiao satellite in a Lissajous orbit around earth-moon Lagrange point 2 [3]. Additionally, the range of choices of solar elevation angle can be used to ensure the best illumination conditions for maximizing position estimation performance via optical camera landmark recognition. Intuitively, impact craters on the moon are identifiable natural landmarks. Thus, a variety of studies have focused on how to accurately detect abundantly distributed craters via template imaging for self-localization purposes. Some of these advancements resulted in segmentation-based crater detection algorithms [4], region pairing method [5], and Hough Transform-based method [6] to achieve geometrical matching by comparing detected craters with an onboard map [7]- [10]. However, crater-based methods have critical limitations, owing to the assumption that well-shaped craters will be present in the imagery and that a minimum number of craters will be obtained under good illumination conditions.
Recently, regions of interest on the moon have shifted toward the shadowy highlands around the poles. In particular, there has been a growing interest in creating sustainable human habitats on the moon. For example, NASA is preparing to send the first female astronaut to the lunar south pole in 2024 via the Artemis program [11] to help establish of sustainable human presence on the moon. The program incorporates a construction plan for the first lunar space station, the Gateway [12], which will orbit the moon. Furthermore, regions of permanent shadow created by rough terrain and low solar altitude around the poles, where cold traps with water ice are present, have been located and are considered to be suitable places for a lunar base with human habitation [13]- [16]. However, such areas can adversely affect onboard self-localization functions that require visual landmarks. Unfortunately, there have been no relevant studies or practical applications for these purposes.
For a Mars landing, Mourikis et al. conducted a study to adapt a space-proven algorithm of image-based velocity estimation [17] to an experimental application of position estimation [18] for all types of terrain, regardless of the type of landmark or illumination condition. This was based on template matching (TM), which measures the similarity between a template image and the same sized area on the map. The goal of the TM method is to find landmarks distributed on the surface in template images expressed as unit-image fragments for comparison. When starting the TM process, configuration work for the template is required to determine its size, shape, and the number candidate landmark areas. In [18], the sizes and total numbers of small templates were initially set 50 -100 and 15 x 15 pixels for searching a given map. If the number of unnecessary target landmarks could be reduced while maintaining performance, the computational burden could be alleviated.
Motivated by such research, we propose a landmark selection method based on deep learning. The idea is to search for landmark alternatives of specific sizes using a trained deep neural network (DNN). Research into excellent DNN object detection capabilities that can recognize individual objects in an image having severe noise and environmental changes has enjoyed tremendous progress [19]- [21]. Thus, the selection of highly rated landmarks among many, according to a flight plan, will make possible the use of optical navigation systems under the worst of conditions. To obtain such landmarks, a massive dataset based on real lunar-surface data is used to train a DNN to maximize the discrimination between local areas of the moon. Next, candidate local regions are scored using the DNN, which demonstrates significantly higher recognition performance than other methods. Finally, the performance for each local area is arranged according to the position of the sun during the year. The performance summary provides selection recommendations for good templates within the flight path according to the flight scenario and the movements of the moon and the sun. The validity of this approach is verified using the numerical results of flight simulations with highly ranked templates.
The rest of this paper is organized as follows. Sections II and III provide a brief overview of the related work and background, respectively. The proposed method is described in Section IV. In Section V we provide descriptions and results of the performed experiments. Conclusions are finally presented in Section VI.

II. RELATED WORK
There have been few related studies about locating geographically distinctive landmarks prior to an extraterrestrial flight. On earth, where position information can be readily computed with the help of GNSS, there have been related studies for indoor navigation. Alt et al. [22] proposed a template selection method for indoor object tracking using a support vector machine. However, it did not address the issue of landmark selection, nor did it consider natural characteristics and environmental changes. Penate-Sanchez et al. [23] used a neural network to predict the matchability of a template for solving a panorama-stitching problem of natural scenery.
For nearby navigation of asteroids and comets, landmark characterization approaches have been developed for whole-landmark maps, such as the L map method [24], the silhouette-carving method [25] and Maplet [26], which have been tested and implemented with a combination of stereophotoclinometry and stereophotogrammetry techniques. In their cases, the asteroids and comets were small such that the entire image model was reconstructed from the earth by taking images covering the entire surface rather than by selecting landmarks.
Johnson [27] studied landmark selection to recommend efficient areas having low rates of incorrect matching before a flight. The core of the algorithm was the spin-image generation method of representing landmarks. However, it was based on 3D data from a highly accurate laser rangefinder rather than an optical camera. Shilane and Funkhouser [28] explored a selection method of the most important regions of an object's surface based on the harmonic shape descriptor.
The commonalities of the aforementioned methods intuitively suggest the existence of a metric or a method of maximizing distinctions among image regions. Nevertheless, it is difficult to apply such things to the lunar highlands at extremely low elevations. This study orders the relatively distinguishable landmarks that appear in this template.

III. CHARACTERISTICS OF LUNAR SURFACE
Images captured by optical cameras are directly affected by natural factors, such as the roughness state of the physical surface of the moon, the ephemeris of the sun, etc. Unlike the earth's soil, the moon, which has a surface of fine dust [29] (i.e., lunar regolith) can affect measured images. Noticeably different from the earth, the topography in the highlands of the moon is rough and steep with large differences of elevation [30]. Statistical analysis of the roughness of the VOLUME 8, 2020 lunar surface of the moon was performed in [31], resulting in synthesized Lunar Orbiter Laser Altimeter (LOLA) data.
The median slope of the highlands covering most of the moon accounts for 7.5 • , and the maria covering less than 20% of the surface is in a smooth region having a median slope of 2.0 • . This means that more than 80% of the entire surface is rough highland terrains. Moreover, the slope of the southern pole area in the highlands is characterized by a sharp terrain of up to 7.6 • . Thus, the steep slopes create an irregularly shadow-casting terrains that cause complex factors for landmark recognition.
The area containing the most maria at the near-side of the moon tends to have relatively large and smooth plains with a high crater density. This area widely covers the nearside of the moon and provides geological benefits for the TM method, which uses craters as landmarks. L-1 and L-2 in Fig. 1 are images taken by cameras mounted on the Chang'e 3 and 4 landers, respectively. They were moving toward their landing targets when the pictures were taken. The craters in both images indicate that appropriate shadowing around the rim caused by the angle of incidence of light at approximately 45 • aided crater recognition. On the other hand, it can be seen that it was not easy to distinguish between small image areas in a monotonously flat plain having few craters. For this reason, past studies concentrated on well-located and well-lit craters under appropriate lighting and distribution conditions. For the highland areas near the poles, the altitude of the sun is too low to enable the extraction of navigational information from a dark image, because the angle of the lunar obliquity to the ecliptic plane is less than 1.543 • . This small angle results in increasing difficulties in designating explicit landmarks. H-1 and H-2 show that the image texture in the same region can be significantly different according to the azimuth angle of the sun. Additionally, the situation cyclically produces large shadows of hundreds of kilometers, as shown in H-3, rendered using LOLA data from aboard the Lunar Reconnaissance Orbiter (LRO).
In summary, the polar regions of the moon have the worst conditions for determining or screening locations and sizes of natural landmarks via template imaging.
The basic assumption underlying this study is that the natural characteristics of each region can show unique differences, similar to human fingerprints, when arbitrarily dividing the area of the lunar surface without artificially planned roads and milestones. The local surface might be unique from place to place, because there have been no weathering processes, no crustal movements, no active volcanos, no water erosion, and no anthropomorphic alterations to the surface. Infrequent meteorites and lunar probes make small but insignificant changes, however. Furthermore, when the geometric position relationship between the moon and the sun changes, the landmarks in the template image reflect cyclical shadow changes.
The features of undesirable landmark areas on the moon are described below: (a) Landmarks in large monotonous areas: uniform surfaces, such as flatlands, lead to difficult discrimination. (b) Landmarks large dark or white areas: the lack of information about landmark features and low contrasts lead to difficult discrimination for variances of noise and illumination. (c) Landmarks in similar landforms: it is difficult to discriminate when parts of the natural terrain are similarly continuous.

IV. PROBLEM FORMULATION
In this paper, it is assumed that the results of evaluating performance of a landmark in a template image depends primarily on the inherent geographical characteristics of the region within the field of view of the camera, the position and attitude of the spacecraft, and the illumination environment of the lunar surface at a specific time. Therefore, any candidate area can be placed at any position on the map, because the position of the best landmark is not known in advance.

A. CONSTRUCTION OF CANDIDATE LANDMARK
To find mutually distinguishable areas within image templates, it is necessary to describe the geometrical representation of those regions. A candidate for a regional object (CRO) is defined as an image object (i.e., a block of pixels). Such a block is likely to contain landmark data that can be compared using the TM method. Coordinate systems, as shown in Fig. 2, are introduced to describe the geometrical relationship between the CRO, the moon, the sun, and a lunar probe having a mounted camera: the moon-centered moon-fixed frame {F}, the surface local frame {N}, the lander's body frame {B}, the camera frame {C} and the image-centered frame {I}. The selenographic coordinate on the global moon for the regional object (CRO), as shown in the left of Fig, 2, is used for preparing experimental image data that refer to specific latitudes and longitudes. Unlike the earth, the moon's prime meridian lies in the mean direction of the central axis between the moon's and the earth's centers (i.e., the mean earth/polar axis lunar referencing system [36].  CROs can be represented in the frame {I}, as shown in Fig. 3, and can be located within the i th image. In this paper, the CRO is a landmark in a lunar surface image, as shown in (1).
where D is a collection of observed imagery datasets consisting of training images x and its ground truth t. The images denotes the i th image. M is the total number of CROs in an image. N is the total number of CROs in an image. CRO k is the k th potential region of landmark, where k is the index number of the CRO corresponding to each absolute position having a specified region size in an area of the moon. F p CRO k indicates the known position of CRO k in the {F} frame. If any particular CRO turns out to be the most distinctive one, it will be relatively easy for it to be identified in the landmark map. Then, the ultimate position of the lunar probe, F p Body will be obtained by (2), using the recognized position in the {C} frame, the its alignment angles, its range to the landmark surface, and the attitude knowledge.
where F C q is a quaternion from the {C} to the {F} frame. The left side of Fig. 3 shows an example of CRO configuration expressed in the image frame {I }, assuming that the CROs will rank as landmarks via each distinguishing score against other scores. The shape of CRO is defined as a circular form to simply express the property of each without overlapping another. In particular, CRO 70 occupies the largest area, including those of the other CROs. It is added for comparison with others. It is assumed that the CRO would be the most recognizable with its rich texture and large surface. The 69 small CROs are evenly spaced without crossing each other.

B. PERFORMANCE METRIC OF CRO RECOGNITION
The final recognition performance of a CRO should be expressed as a value that can be distinguished from other CROs for quantitative comparison. For evaluating CRO recognition performance, classification performance, which reflects the detection rate, and regression performance, which indicates the position accuracy of the detected ones, are used. The right side of Fig. 3 shows that the CRO metric is defined by its radius and size for the performance evaluation. For a regression measure, the L 2 norm is used in (3) for expressing the error of the localized position.
where P i cro k and p i cro k denote the positions of the true and a detected CRO k , respectively, and D i cro k and d i cro k are the radii of the true and a detected CRO k , respectively. σ i cro k is the tolerant size error of a detected CRO k , and p i cro k and δ i cro k are the position and size errors of a detected CRO k , respectively. For a classification measure, an f 1 -score (or an f -measure) is used to evaluate the rate of correct prediction VOLUME 8, 2020 of CRO's index, k. This performance determines whether the region of the CRO is matched with a landmark, which is more critical than the regression performance that quantifies how close it is. The f 1 -score [37] considers both the recall, indicating detection performance, and the precision, indicating the correctness of the true-positive decision. As shown in (6), the f 1 -score represents the harmonic average of the precision and recall in (4) and (6), respectively.
where tp i , fp i , and fn i denote the true positive, true negative, and false negative for each CRO, respectively.

C. DNN-BASED CRO DISCRIMINATOR
To identify CROs that are likely to be good landmarks, we develop an object detector (i.e., LunarNet) based on a convolutional neural network (CNN). Fig. 4 shows LunarNet's architecture, whose backbone is based on a widely used detector (i.e., YOLO9000 [38]), which uses a modified GoogLeNet architecture. The Lunar-Net can be trained using images and known CRO positions via supervised learning. This consequently reduces losses, so that results will be closer to the highly discriminated state. The general loss function can be expressed as in (7).
where L is a loss function for evaluating the LunarNet given n th sub-datasets of the training data, x n , and the ground-truth data, t n . For a sub-dataset, f θ is the predicted output of the LunarNet with hyperparamter θ consisting of weights and biases.
The loss function of YOLO9000, which embodies L, consists of five sub-items for the rectangular shape surrounding the object. Considering that the CRO's form is circular, the first two items are modified to be concise using (8) and (9).
where S is the size of grid of an input image, B is the total number of circles to be predicted in a cell, I

1) CRO CONFIGURATION & LABELING (CCL) PROCESS
This step is a process for the deployment and image preparation of CROs leading to ultimately massive image data as inputs to LunarNet learning. The size and shape of each CRO, the placement of all CROs, and the ephemerides of the sun and a camera are configured as the main factors affecting the final selection of the CRO as a landmark. Initially, the CRO is set to a fixed shape and position within the image frame, and the image itself can be prepared as either real and/or synthesized images having sampling intervals within the dynamic ranges of the camera and the sun. The position of the sun depends on its natural ephemeris, corresponding to the range of flight time. The position and attitude of the camera depend on a flight trajectory of the lunar probe upon which the camera is mounted. Therefore, image datasets should be prepared to cover the range of the sun and camera movement as much as possible. A large amount of image data can be divided into three datasets: training, validation, and test. These comprise a combination of the image and its truth information. In other words, all images are tagged with true CRO information, including size, position, and index number.

2) CRO DISCRIMINATOR LEARNING (CDL) PROCESS
This step is the main process of the entire procedure. The process consists of training the LunarNet to discriminate a 99014 VOLUME 8, 2020 CRO from others via the supervised learning method. Lunar-Net learns the difference between CROs and can make correct predictions in a training set. The validation set is used to tune hyperparameters. At first, the loss value computed using the loss function of (8) and (9) is large. However, LunarNet adjusts the hyperparameter values by using a finite training set and dozens of learning iterations until its inference capability of distinguishing between CROs is maximized. Thus, LunarNet is forced to apply minimum epochs of learning iterations such that all CROs can be sufficiently categorized, and their positions can be predicted after satisfying the minimum performance for the validation set. However, if the LunarNet diverges or if its loss value is not met, the configurations of CRO and data-sampling intervals are reconfigured to demand an additionally larger dataset in the level of details using the CCL process in a loop.

3) CRO PERFORMANCE MAPPING (CPM) PROCESS
This step is the final process of preparing the performance map and determining favorable landmark alternatives. Via the trained LunarNet, as the result of the CDL process, the final score per each CRO is evaluated using the test dataset independently of the training and validation sets. However, its purpose is not to tune the hyperparameters, but it is to VOLUME 8, 2020  measure the final performances of classification and regression, as described in (4) and (6). At the end of testing, both performances of each CRO are summarized and visualized in the CPM. Then, the CPM can be utilized for the configuration of target landmarks during the TM design phase. Finally, each CRO can be sorted by rank at a specified flight date, and the required CROs can be selected among the highly ranked list. However, if there are not enough CROs to become alternatives to landmarks, repetitive operations are performed after changing the flight time.

V. NUMERICAL SIMULATION A. TEST CONFIGURATION
This section defines a practical example of the problem wherein a lunar lander operates via a navigation camera. The exemplary scenario is set such that a lander descends over the highland areas on the moon. The flight direction is polar, so that the lowest altitude of the sun is seen from the lunar surface makes the worst environment for landmark-based navigation. The flight trajectory during the descent phase is computed to minimize the fuel consumption while taking into consideration the lander's six degrees-of-freedom dynamics equations. The mass property, the thruster's efficiency, and the start and end conditions of the flight are described in Table 1.
The flight trajectory is depicted as the relationship between the altitude and the range with regard to the landing site, which presents a parabolic shape, as shown in 6. An experimental image acquisition plan is initially set, considering the cumulative error rate of the system. In this flight track, the areas to be observed are denoted as A, B, C, D, and E, passing through flight points a, b, c, d, and e, respectively, at an altitude of 3 km or more. In the CCL process of Fig. 5, a total of 70 CROs are placed within the camera's field of view. This arrangement applies equally to the five regions.
A large set of image data for each CRO are required for LunarNet learning. A digital elevation model derived from LRO's LOLA instrument data [39] is used to render the massive images. The parameters for the image production for the experiment are summarized in Table 2. The solar conditions are set in a range of the azimuth and elevation angles that assume the worst case. The azimuth and elevation angles are divided into 15 and 20 intervals, respectively. The numbers are determined at the stop condition of the CCL and CDL process loop shown in Fig. 5. All images applied to training, validation, and testing are assigned at a ratio of 10:3:160. The test set consists of eight sets of 92,160 images per CRO. For each test set, the position of the CRO viewed by the center line in the camera's field of view is built with Gaussian noise by setting the mean position and variance.

B. LunarNet LEARNING FOR DISCRIMINATION
The well-trained LunarNet yields a smaller loss in the next epoch (or iteration) from every back-propagation task. Ultimately, the LunarNet stabilizes at the minimum loss value, and the CROs in a given training set are clearly distinguished via the CDL process. The graphs in Fig. 7 show the variation in the loss values of the learning time for the five regions. The criteria for stopping LunarNet learning adopt 0.1 as the limit of the loss values reliably converged in the 15-epoch Thus, it is judged that the hyperparameter space of LunarNet is successfully searched such that its distinction from CROs is as high as possible. The solid line represents the training loss before the back-propagation task of the mini-batch, and the dotted line represents the interpolation curve of the validation loss values, calculated at each epoch.
While traveling from 85.39 • to the polar region, images from the five regions show shadows from extremely low sun altitudes, including monotonous and rough shapes. In particular, the loss curves for A and B areas show that the validation losses vary exceptionally during the learning process. A visual analysis of the images from each region show that the effects of such fluctuations are caused by environmental impacts in the area. The level of fluctuations during the learning process is related to the geographical characteristics that suddenly darken the entire region or make the surface monotonous. In other words, some CROs placed in the image may have strong similarities with other CROs, making a distinction difficult. Figure 8 shows the performance results with respect to the position of the sun, particularly at a detailed interval in region A. As shown in the graph on the left side, the effect of the sun's azimuth on two performances result in a unique feature of region A. When the sun's azimuth is between 130 • and 270 • , many areas of region A are covered by shadows, indicating poor performance. The black dots represent the top-five solar positions in the average performance of all CROs. In contrast, the increase in the sun's altitude leads to a monotonous increase in performance. It can be seen that the position and identification performance of the CRO changes by approximately 0.78% when the angle changes by up to 1.568 • .

C. CRO PERFORMANCE MAP
Each CRO in the test set was detected or/and identified using LunarNet. The test results can be expressed using a 3D matrix by mapping with respect to the sun's azimuth and elevation VOLUME 8, 2020 angles. Ultimately, the matrix can be visualized as a CPM to make it easier to characterize the regional features for image recognition performance. Figure 9 shows the CPM map for CRO 70 in area A. The CRO has the largest group of pixels among the others. Therefore, it is conceivable that it will be recognized with the best performance under the same configurations, such as the camera's field of view and resolution of other CROs. The thick pink line in the figure represents the sun's ephemeris over the performance map in 2020, which can help determine the time window for planning a flight initiation. The two performance distributions show a similar pattern for the position of the sun. Overall, it can be seen that the performances are more affected by the azimuth than by the altitude of the sun. Criteria for determining a valid area for CRO selection among the large areas of the CPM are required. For example, the minimum selection criterion for f 1 -score as the classification performance is set to more than 85% and the minimum criterion for the position error as the regression performance is set to less than one-half the size of the CRO as described in ρ 1 and ρ 2 , respectively in Table 3. It can be seen that the yellow area on the f 1 -score side and the blue area on the position-error side can provide a wide range of selections under the selection constraints. Additionally, to determine the superiority of the two criteria, the classification performance is given higher priority as a more important factor than the regression performance to minimize false detections. Even if the regression performance is relatively high, such that the position of the image coordinate is well predicted, it is useless if the classification performance is too low to detect a sufficient number of circles or to identify the correct index  number. The weak identification of CRO becomes less robust to subtle variances of the sun. The larger the size of the available area, the greater the acceptability of changes in the position of the sun over time. Thus, the availability of CROs to changes in flight time increases. Therefore, the CROs that can be successfully matched with a circular area in an onboard map during flight is able to be determined by simply inspecting the performance at a flight time on the pink line in its CPM map.
As a result, CRO availability in the area A with relatively high variability in the value of loss during learning is confirmed among the five regions, and the unique characteristics of CROs in the rest of the regions can be identified via each CPM. If the flight time is specified, the process of selecting CROs via CPMs can be easily performed visually. For example, when 2020-03-09 14:50:00 [UTC] is confirmed as the landing initiation time, the position of the sun corresponding to this time can be fixed and displayed with a red triangle on the line of the CPM. Fig. 10 shows an example of the CPM in each region based on the type of performance with the marked date. Specifically, the classification and regression performances of CRO 34 observed in area A are shown to have an excellent evenness over a wide azimuth range at that time. However, the classification performance of CRO 13 is shown to be poor within a wide range. This map shows the CROs that are suitable for use within a year. Apart from the flight time passing through the blue (f 1 -score) and yellow (position error) areas in A, all times can become a flight time window when using CRO 34 . In particular, CRO 12 in region B shows a poor performance throughout the year and is highly affected by the Sun's altitude. In other words, a good CRO for use as a template should not only satisfy a performance appropriate for the flight date of concern, but also cover a wide area over time to achieve the appropriate performance. Then, according to certain criteria, some CROs can be selected or rejected, as indicated by ''Accept'' and ''Reject,'' shown in the figure. After a flight plan changes, the required CROs should be selected again within the available range after sorting CPMs repeatedly in descending order.

D. FLIGHT SIMULATION USING TOP-5 LANDMARKS
We verify the effectiveness of the proposed LunarNet learning and landmark selection algorithm by comparing   Table 3.
performances of the top-five landmarks with the bottom-five during the descent flight. Table 4 shows the CROs enumerated in descending order for all areas at a specific time. It also shows the results of indexed top-five and bottom-five CROs by their CPMs. Given the low priority of the regression performance, the performance's threshold is only used for providing the criteria to avoid CROs that produce only the worst position error, and the classification performance is used to provide the acceptance criteria of CROs to sequentially select from the performance list. Unexpectedly, CRO 70 , ranked 34 th in terms of classification performance, did not achieve the best results for any performance. Such results were the same in other areas except for area E. This is because a small part of the large CRO easily disappeared from the camera's field of view and underwent relatively large area changes, compared with the other CROs. Figure 11 shows successfully specified landmarks in the top-five CROs. The top-five results for the areas A and C show that the landmark placement patterns are located mainly on sides of hills or mountains. The most important landmark alternatives are found in the rough highland having irregular shaped craters in area E. Moreover, in case of areas B and D, the landmarks in the dark images re successfully selected, even in mostly shadowed environments. Figure 12 illustrates snapshots of areas B and C to visualize recognition performances on flight simulation using the top-and bottom-five CROs, respectively. Area B has dark shadows, and area C has the extreme condition of ridges and high mountains. A closer look at the detection of the landmarks in the dark image shows that some of the slightly illuminated peaks correspond to the locations shown in the figure. Nevertheless, it is confirmed that the top-five CROs are well detected in both places, whereas the bottom-five are located in the complete shadow regions, making it difficult to detect. Figure 13 shows performance assessments using the five selected landmarks for the five areas during the landing. Each graph (a, b, c, d, and e) in the figure indicates the averaged position errors and the total number of correct identifications VOLUME 8, 2020 for the five best and worst CROs, respectively. For example, one of the top-five CROs in graph (a) is detected and recognized correctly from the 304-km point at the beginning of the pass over area A, whereas the bottom-five CROs are shown to have an interval of nearly 10 km, which was not detected at all. This is an expected result, owing to the high priority of classification performance, and a sharp difference in performance is confirmed. For comparison, CRO 70 has difficulty maintaining its fully original shape in the camera's field of view, as described above, and it can be seen that the undetected interval extending to the 8-km section appears.
With regard to the position error, there is an interval in which the mean position error of the top-five is rather higher than that of the bottom-five, such as with area A, given the priority order of performance criteria. Furthermore, it is confirmed via the mean difference of position error, ξ (a)-(b) , in Table 5 between the top-and bottom-five is not large like areas B and E. It can be seen that area A, compared with other regions, is difficult to infer with regards to location because of the unique characteristics from the terrain itself, where the monotonous plane is widespread. In case of areas B and E, each ξ (a)-(b) shows a small difference between the top-and  bottom-five. However, the value of area E moves close to CRO 70 without much variability, indicating that each local region in the entire area performs well. This result means that the E region is suitable for the small regions where the CRO is located and has unique characteristics. The important point is the fact that its valid interval is too narrow when using the bottom-five to obtain required samples of CROs, given that the speed of the lander is approximately 1.5 km/s. As a result, area C is the most beneficial area from the proposed landmark selection and can be applied for an approximately 13-km interval with low position error while reducing unnecessary matching time if only the top CROs, such as the top-five, are selected. Looking at the images of each area again in Fig. 11, it is easy to discern that the shadow regions in region C are relatively well distributed, less monotonous, and have fewer similar or continuous shapes. The relatively

VI. CONCLUSION
In this paper, we proposed a landmark selection method that applies a DNN (LunarNet) to characterize unique landmarks on the moon's surface. The key part of the proposed methodology is the LunarNet learning, which is used for constructing a visualized performance map for candidate landmarks (i.e., CROs). Ultimately, the map broadens the various options of known landmark alternatives under variable flight conditions. With a margin and/or availability of a flight planning based on the map, landmark alternatives were successfully extracted from the worst dark highlands near the south pole. Numerical experiments were presented to demonstrate the applicability and validity of the landmark selection method by comparing the relative recognition performances of the top-and bottom-five CROs. SUJIN CHOI received the M.S. degree in aerospace engineering from Seoul National University, in 2007. He has been a Senior Research Engineer with the Korea Aerospace Research Institute (KARI), since 2007. He was involved in mission control center for flight dynamics and space debris analysis for Korea Multi-purpose Satellite (KOMPSAT). He was also involved in mission design and analysis for Korea Pathfinder Lunar Orbiter (KPLO). His current research interests include mission design and analysis of a lunar orbiter, lunar lander mission, and artificial intelligence. VOLUME 8, 2020