A 79.7g Manipulator Prototype for E-Flap Robot: A Plucking-Leaf Application

The manipulation capabilities of flapping-wing flying robots (FWFRs) is a problem barely studied. This is a direct consequence of the load-carrying capacity limitation of the flapping-wing robots. Ornithopters will improve the existent multirotor unmanned aerial vehicles (UAVs) since they could perform longer missions and offer a safe interaction in proximity to humans. This technology also opens the possibility to perch in some trees and perform tasks such as obtaining samples from nature, enabling biologists to collect samples in remote places, or assisting people in rescue missions by carrying medicines or first-aid kits. This paper presents a very lightweight manipulator (79.7g) prototype to be mounted on an ornithopter. The distribution of the mass on the flapping-wing robot is sensitive and an extra lumped mass far from the center-of-mass (CoM) of the robot deteriorates the flight stability. A configuration was proposed to avoid changing the CoM. Flight experiments show that adding the arm to the robot only moved the CoM 6mm and the performance of the flight with the manipulator has been satisfactory. Plucking leaf is chosen as an application to the designed system and several experimental tests confirmed successful sampling of leaves by the prototype.


I. INTRODUCTION
I N the last decade, the physical interaction between aerial robots and their environment, particularly for manipulation, has attracted great interest in academia [1]. Recent aerial platforms [1]- [4], have demonstrated to be very efficient solutions for remote sensing for mapping, inspection in industrial environments, detecting natural disasters, and cooperative free-flying for assembly and structure constructions. There are several devices for manipulation, ranging from one-degree-of-freedom (DoF) grippers [5]- [7], vacuum-cups to grab an object [8], to multi-DoF manipulators [9]- [12]. The reported aerial manipulation tasks have been successfully performed [1]. Nowadays by multirotor unmanned autonomous vehicles (UAVs), users could carry lightweight manipulators; however, this technology still suffers from some limitations such as autonomy or safety to collaborate with humans.
In the last years, a new generation of aerial robots with flapping wings has aroused interest because of their prominent advantages. Flapping wing robots are bio-inspired so-lutions to produce lift and thrust in aerial robots, leading to a quieter, safer, and more efficient alternative [13]. Adding manipulation capability to ornithopters is the next step after fly and perch to justify the usage of the robot in different applications. However, the "lightweight" manipulators for multirotor UAVs are still "heavy" for ornithopters. As a result, the scale of weight must be reduced more than the conventional one for aerial manipulators [4]. There are some limitations in aerial manipulation with ornithopters such as a reduction in the accuracy since it is difficult that the system holds a position long enough for the manipulator to converge. That limit is due to the incapability of stationary flight of the FWFR in contrast with multirotor systems. Then the manipulation in this work is done after perching on a branch and after that, the manipulation starts. The leg must hold the bird in a proper position.
The animal flapping flight has attracted enormous interest in the last few years [14], [15]. Many flapping-wing aerial platforms have been developed that motivate the use of this generation of aerial robots. The current effort is concentrated on increasing the load capacity [16], improving flight performance, and adding manipulation capabilities. The hard challenge of adding manipulation capabilities to aerial robots such as flapping-wing birds has been barely explored [17], [18]; the control problem of manipulation was addressed while the system was perched maintaining the equilibrium.
Small manipulators for surgeries [19], micromanipulation [20], etc. have been proposed in the literature. A small robotic manipulator was built for intrauterine fetal surgery in an open magnetic resonance imaging; the robot possessed 5 DoFs and the weight was 500g (major part was dedicated to electronics) [19]. A 535g two-DoF manipulator was designed for carrying a load up to 200g [21]. The design and development of new prototypes of very lightweight manipulators is currently a research topic of interest since there is not a standard or commercial solution suitable for their integration into aerial platforms. The hard requirements of aerial robots and in particular flapping-wing robots make us investigate new designs to perform some manipulation tasks in high-altitude or dangerous places that cannot be easily accessed from the ground. We propose an ultra-lightweight two DoF robot manipulator with a gripper at the end-effector to carry out tasks like sample collection. Moreover, this work is analyzed in-depth in terms of the integration of the manipulator in a way not to influence the flight of the system and perform some manipulation tasks.
An interactive application with plants has been aimed, specifically, to pluck a leaf from a branch or a tree for post-analytical laboratory work. The analysis of plant diseases and pests is important for determining the yield and quality of plants. Plant diseases are responsible for major economic losses in the agricultural industry worldwide, in particular in dangerous places that humans can not access easily. Sampling leaves is an application of robotics using visual servoing and computer vision technique [22]- [25]. The platforms for moving the sampling device and the vision system are usually mobile robots or industrial manipulators.
The monitoring systems for leaves and also agricultural applications using aerial systems were mostly vision-based monitoring systems without interaction with the plant [26]- [28]. The problem associated with automatic plant disease identification using visible range images has received considerable attention in the last two decades [29]. Monitoring plant health and detecting pathogen, at early stages, are essential to reduce disease spread and facilitate effective management practices [30]. However, the detection of diseases using vision is challenging because of the complex lighting conditions and multiple morphologies. The ultimate longterm vision is to substitute the traditional manual plucking methods of leaves which not only are inefficient, but also require a great deal of labor, leading to a high-cost problem. For example, the research and development of automated tea harvesting robots have attracted great interest in the last years [24].
Perching and manipulation could be viewed as the same ability for flapping-wing robots since both have actuators and add additional degrees of freedom to the system. Considering two legs, one could balance the bird on a branch or the ground and the other could help to manipulate an object. However, here in this work, manipulators are considered as an additional mechanism possibly on top of the robotic birds for sampling, carrying small objects, etc.; and the legs installed below the robot. In the literature, the legs served for take-off [31], [32], bipedal locomotion [33], or perching [34], [35]. The preliminary works for leg design [34], [35] focused on bio-inspiration and soft robotics, and for the safety of the bird, a supporting-safety rope was used. They used two legs for adding manipulation capability as well, Bellow the robot or for moving the bird on the branch. Here the motivation is to increase the workspace and have the manipulator for sampling or manipulation on top at the tip of the bird. The scale is ultra-lightweight to show the flight demonstration without a safety rope, presented in the video file as supplementary material.
The main contribution of this work is to design a 79.7g manipulator (including the driver, micro-controller, and electronics), mounted at the tip of a flapping wing robot to sample leaves from trees, as a post-perching study. The results have been validated and tested experimentally: flight capability with the manipulator on top in indoor tests, and leaf-plucking tests in outdoor experimentation. Also, it is important to highlight that to the best of the authors' knowledge, it is the first time that a flapping-wing robot with a manipulator on top, automatically flies without a radio control transmitter. The flight control repeatedly regulates the FWFR to the desired set-point. The prototype has autonomous capabilities onboard and the validation has been done by closing the loop with the motion capture system. This article is organized as follows. Section II presents the description of the flapping bird robot. Section III describes the mechanical design of the arm. Section IV describes the actual prototype and hardware characteristics of the manipulator, and experimental results are reported in Section V. Finally, concluding remarks and proposals for future studies are given in Section VI.

II. E-FLAP ORNITHOPTER: FEATURES AND BACKGROUND
This section presents the most important features of the flapping-wing robot, E-Flap [16], on which the manipulator design is integrated. A photograph of the E-Flap (flying outdoor) is shown in Fig. 1. The E-Flap has been developed within the GRIFFIN 1 Advanced Grant of the European Research Council, and the features considered in the design have been: 1) Flapping wing robots with autonomous capabilities, which includes onboard electronics for perception and control activities, 2) Manipulation capabilities, which include devices to be able to interact with the environment. VOLUME 4, 2016 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3184110 The perception part including the hardware and computational power needed to perform onboard reliable perception, including an event camera has been addressed by [36]. E-Flap prototype has a NanoPi companion computer directly plugged into the PCB (Printed Circuit Board) with the microcontroller and a WiFi module, used to receive flight instruction from an external PC when flying. The NanoPi is connected to the VectorNAV VN-200 board which is a highperformance GNSS-Aided Inertial Navigation System. This allows autonomous navigation with the E-Flap ornithopter. More information about the hardware architecture of the E-Flap can be found in [16]. Some control techniques have been proposed for improving the autonomous flight of the system [37]. E-Flap prototype weighs 510g and it has been demonstrated that it can carry a payload up to 520g [16]. However, a payload bigger than 300g degrades the performance in flight enormously reducing the maneuverability and the climb rate of the system (flight angle). This forces us to design a manipulator with the hard requirement of 100g, that is the 30% of the payload capability leaving the rest of the payload for the perception and other devices that are required. Another requirement is to establish an adequate position for the installation of the manipulator. The E-Flap's body consisted of two carbon fiber plates of 1.5mm thickness which create a protected space to add the transmission, the motors, and so on.

III. DESCRIPTION OF TWO-DOF ROBOT MANIPULATOR A. DYNAMIC MODEL OF THE MANIPULATOR
The robotic manipulator in this work is a two-DoF planar arm, set up in the Y Z plane and the gravity affects the system in the Y axis direction. It should be noted that once the robot landed on a branch, the manipulator starts working, then the dynamic equation of the robotic arm is in the form of fixed base robots. The dynamics of the manipulator possess the common form of [38]: where q(t) ∈ R 2 includes a generalized coordinate vector, M(q) : R 2 → R 2×2 is an inertia matrix, c(q,q) : R 2 × R 2 → R 2 includes the Coriolis and centrifugal terms, and g(q) : R 2 → R 2 is a gravity vector. The inertia matrix, gravity, and Coriolis and centrifugal vectors are presented as: where the details are presented in [39]: in which a i (m), m i (kg), I zz,i (kgm 2 ) are the length, mass, and moment of inertia of the i-th link, m p is mass of the load, and g = 9.81(m/s 2 ) is gravity constant.
Considering the state vector of the system as x = [q T ,q T ] T , the state-space representation of the system (1) is found:

B. DESIGN DECISION MAKING
The load-carrying capacity of the flapping-wing robot is limited, hence, reducing the weight in any possible aspect has been always demanded. To reduce the weight, the use of a carbon-fiber plate was preferred rather than 3D printed polylactide (PLA) material in joints and holders. On the one hand, the flapping wing bird is stable in flight, and adding any mass to the system changes the center-of-mass (CoM) of the whole body and deviates the bird from stability conditions in flight. On the other hand, the proper position for the arm is at the tip of the bird for manipulation purposes. Mounting the manipulator at the tip of the bird has several advantages: it can be placed on the wings during the flight, see Fig. 2. It should be noted that on top of the wing is the safest place in case of a possible landing. It also has access to the workspace VOLUME 4, 2016 in front of the bird after perching on a branch. The leg of the bird could move the arm for extending the workspace (the leg adds another DoF to the bird and it can move the whole body while it is on the branch).
workspace FIGURE 2. The bird, holding the arm on the wings.
An added lumped mass at the tip of the bird changes significantly the distribution of the mass on the whole body. To compensate for the imbalanced mass, the leg could be moved back from the CoM. An end-effector gripper is required for holding/moving a lightweight object during the manipulation and a lightweight mini servomotor is chosen for this task. To reduce the weight of the gripper, the side surface of the servo holder was chosen as the fixed jaw of the gripper. The design of the manipulator is presented in Fig. 3. The total weight of the arm with electronics weighs 79.7g, see Fig. 4.
The center of gravity of the E-Flap before adding the leg and manipulator was reported 218.215mm from the tip of the bird (alongside the longitudinal axis) [16]. Adding the leg moved it back to 221mm, measured experimentally by finding a point that puts the bird in a horizontal and balanced state in the OptiTrack testbed, see Fig. 5, the top picture. Finally, adding the manipulator at the tip of the bird and spreading it on the wings, will change the CoM to the final value of 215mm. The change in the CoM is not significant then the flight parameters have remained the same for the experimentation. The CoM measurement method is as follows. Searching a position alongside the body axis and placing a pin under the main rod of the body. When the bird is stable and the rod is horizontal, that point is the CoM which was found experimentally. The obtained data before adding the manipulator was also close to the data received from the CAD model [16]. The same approach defined the CoM position after the installation of the manipulator and the electronics.  controlled using pulse-width modulation signals through an L293D quadruple half-H driver. Angular positions of the motors are measured using magnetic encoders, which provide the rotatory position of the motors. The gripper of the manipulator is actuated by a servomotor BlueBird BMS-115HV, which weighs 11.3g, with 4.3kgcm torque at 6.0V.
Processing is performed on a Arduino Micro single-board computer, responsible for data acquisition, control implementation, and communications. Also, it has been designed a PCB to mount all the electronic components on a board and connect the different sensors and actuators in a working circuit. The hardware/software architecture of the system and the different components are represented in Fig. 6. The figure shows the weight of all the electronic components around 52g in total which means that the weight of the structure of the manipulator and the wiring is only 28g.

IV. NONLINEAR CONTROL DESIGN
To control the system (1), a feedback linearization term combined with a PID controller is used. This control technique cancels out all the nonlinearities so that the output dynamics become linear. In essence, it transforms the original system into an equivalent linear form by change of coordinates and feedback. Thus, since the whole state x = [q T ,q T ] T is measurable, the system can be linearized utilizing the following fictitious control input v, a PID input: where v is supplied into:

VOLUME 4, 2016
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and  where M(q), c(q,q), g(q) are reported in Eq. (2). By taking Laplace transform of (4), the transfer function of the system is G(s) = diag(1/s 2 , 1/s 2 ), that relates the angular position of each joint with the fictitious input, v. Then, by using a PID controller for each angular position in the form of: each link is controlled successfully, provided that the PID gains are well-tuned. The controller (6) with the linearization (5) cancels out all the nonlinearities of the system ensuring good trajectory tracking and eliminating the error due to friction of the motors and unmodeled disturbances. On the other hand, the plucking leaf experimentation has been done outdoors on a real tree. We highlight that the topics to perform the complete process with flapping-wing robots have been addressed in previous literature such as flying [16], and stabilization after perching [18]. The other necessary step is perching on a branch, which is not considered in this work. Then, the focus of this paper is to address the post-perching application of the ornithopter and its manipulator in outdoor scenarios. The robot manipulator must be capable of plucking leaves from the branches of a tree. The ornithopter is placed in an appropriate position with accessibility to a leaf. The manipulator is programmed to go towards the leaf's position and pluck it from a tree, then put it back in a box on top of the wings. Then, the process is not fully autonomous at this stage since there is no camera to report the relative position between the leaf and the manipulator.

A. FLIGHT EXPERIMENTS WITH MANIPULATOR
We test the flight performance of the ornithopter including the manipulator. This is important to verify: a) the manipulator is lightweight enough to be transported by the flappingwing robot, b) the decision of mounting the manipulator at the tip and spread it above the wings is a good choice and c) the increase in weight with a slight modification of the center of gravity (6mm) does not influence the performance in flight. The ornithopter was launched from its initial point using a motorized launcher which sets the initial speed of the ornithopter. The ornithopter was commanded to a target point with a height of 2m and followed a straight line in the plane XY using the PID controller based on the previous work [37]. The snapshots of the flying bird carrying the manipulator are shown in Fig. 7. The manipulator was spread on top of the wings to distribute the mass alongside the main axis of the bird, the longitudinal one. Figure 8 shows the robot's trajectory in the XZ plane.
The trajectories are obtained via a visual infrared tracking system with very high precision. The figures report the data of four experiments. The first experiment was performed VOLUME 4, 2016 5 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and without the manipulator using it as a flight performance reference because the controllers were adjusted for this case. The other three experiments were performed with the manipulator. It can be seen that the performances of the four experiments are similar and the influence of the manipulator in the flight is almost negligible. The figure also shows that the controller performs correctly, with a maximum error of 10cm in the Z axis.

B. PLUCKING-LEAF APPLICATION RESULTS
Experiments have been carried out with the prototype described in Section II to demonstrate the possibility to perform manipulation tasks. We validate the proposed design and demonstrate that the hardware architecture allows us to control the manipulator. The tests verify the reliability of the designed gripper and the control position of the DC micro motors using the control strategy described in Section IV. Also, it is demonstrated that the micromotors are powerful enough to perform the desired tasks. The PID gains of the controller are well-tuned experimentally to achieve almost perfect trajectory tracking. The gains are set to the values k p = 500, k d = 25 and k i = 5 for each link. Figure 9 shows the results of the experiment. Figure 9-(a) shows the references of the DC motors (dashed line), their angular positions measured by the encoders of the motors (solid lines), and the commanded servomotor position. We can see that the position of the servomotor varies between the opened claw position (58 • ) and the closed claw position (98 • ) to keep the gripper opened and closed, respectively. Moreover, it shows that the DC motors perform good trajectory tracking using the proposed nonlinear controller of Section IV. The reference trajectories are second-order polynomial curves to avoid moving the motors abruptly or reaching the motor saturation bounds. These soft references are important in the DC micromotors to avoid any possible damage to the gears.     in the range of [−100, 100]. A signed PWM duty cycle is used for clearer comprehension. It can be noticed that the signals are smooth and they are not reaching the values of the motor saturation. Also, it can be seen that in the stage of plucking the leaf, the control signal is twice bigger than in the stage of moving towards the commanded point because the trajectory is faster and the system is moving against gravity. Even so, the motors do not produce saturation which means that they are powerful enough for this task despite their ultralightweight characteristics.

A. CONCLUSIONS
This work presented the design of a very lightweight manipulator for flapping-wing flying robots to sample/pluck leaves from a branch or tree after perching. The arm consists of a two-DoF planar robot and a servo gripper for taking the sample. The load capacity of the ornithopter is quite limited, defined as 100g for the manipulation device, which forces us to reduce any extra part from the manipulator. The system designed considering the reduction of plastic/aluminum parts and adding carbon-fiber components, resulted in a 79.7g robot including electronics, drivers, and processor board. The experiment, plucking a leaf from a tree was shown the success of the design and the prototype is still 20g below the load carrying limitation. Moreover, this approach could also be used to carry some medicine to humans in rescue missions perching next to the injured person. Other applications can be addressed easily with this approach by changing only the end-effector of the system. The FWFR flew with the arm freely spread on top of the bird, and the motors were turned off to avoid damaging the gears caused by the oscillation of the flapping. We believe if we lock the arms in a specific position, the motors will be damaged. VOLUME 4, 2016 7 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3184110

B. FUTURE STUDY
Tacking a sample and restoring it with only one arm is a challenging task. It is suggested to increase the number of arms to two to perform dual arm manipulation, which will allow more sophisticated sampling and delivery applications. The other topic for studying is visual servoing and adding a camera to the robot for online monitoring and providing the position of the leaf for plucking to the arm. This approach will present an autonomous platform for the task. Moreover, the application of the aerial flapping-wing robot could be extended to the delivery of the medicine for people in an emergency in out-of-reach places, sensing and measurement from nature, and monitoring.
SAEED RAFEE NEKOO is a senior postdoctoral research fellow with Robotics, Vision, and Control Research Group (GRVC) at the University of Seville in association with AICIA. He is currently involved in AERIAL-CORE H2020 project in the control of inspection robots for high-voltage power lines; and Advanced Grant GRIFFIN ERC in the design and prototyping of bird-shaped legs for perching; and lightweight cooperative manipulator design, prototyping, and control for a flapping bird bio-inspired robot. Saeed's research interest includes: robotics, nonlinear and optimal control, control engineering, manufacturing, cooperative robotics, flexible joint manipulators, observer, and estimator design, path planning, wheeled mobile robots, control of autonomous underwater vehicles, free-floating space manipulator design, and control, digital implementation of continuous-time nonlinear controllers, and design, manufacturing, and control of mechatronic systems, aerial robotics, multirotor UAV, variable-pitch-rotor quadcopter control, motion control of aerobatic maneuver, sliding mode control, the soft landing of drones, lightweight cooperative manipulators, gravity compensation. ANIBAL OLLERO is full Professor and Head of GRVC at University Seville, and Scientific Advisor of the Center for Aerospace Technologies (CATEC) also in Seville. He has been full professor at the Universities of Santiago and Malaga (Spain) and researcher at the Robotics Institute of Carnegie Mellon University (Pittsburgh, USA) and LAAS-CNRS (Toulouse, France). He authored more than 750 publications, including 9 books and 200 papers in journals and has been editor of 15 books. He has delivered plenaries and keynotes in more than 100 events including IEEE ICRA 2016 and IEEE IROS 2018. He has been supervisor or co-supervisor of 45 PhD Thesis that have received many awards. He led more than 160 research projects, participating in more than 25 projects of the European Research Programmes being coordinator of 7 and associated or deputy coordinator of 3, all of them dealing with unmanned aerial systems and aerial robots. From November 2018 he is running the GRIFFIN ERC-Advanced Grant with the objective of developing a new generation of aerial robots that will be able to glide, flapping the wings, perch and manipulate by maintaining the equilibrium, and from December 2019 he is the coordinator of the H2020-AERiAL-CORE project with the participation of 15 universities, research centers and companies dealing with aerial robotic manipulators and applications to inspection and maintenance He has transferred technologies to more than 20 companies and has been awarded with 25 international research and innovation awards, including the recent Rei Jaume I in New Technologies (Spain), the Challenge 3 of the MBZIRC 2020 International Robotics Competition, the Overall Information and Communication Technologies Innovation Radar Prize 2017 of the European Commission, and has been also elected between the three European innovators of the year being candidate to the European personalities of the year 2017. VOLUME 4, 2016