4.52. Irrespective of whether operations were carried out by large international organizations or NGOs, on-the-spot monitoring by Commission staff still needs to be improved. The monitoring of current operations and the results actually obtained is a high-risk area, and the Commission must devote more attention to it. If, in accordance with the mandate given to ECHO, the Commission wishes to be more than a simple provider of funds for humanitarian purposes and is to be credible in the long run by playing an active role, in coordination with the Member States and the UN agencies, and with its own programmes of actions, then it has to ensure that, in addition to having clearly defined policies and programmes (see paragraphs 2.26 and 3.2), it has adequate resources, properly organized, on the spot.
EVALUATION OF THE AID
Internal evaluations by ECHO
5.1. When ECHO was established, an evaluation adviser was included in the organigramme. In 1995 a second person was added and in 1996 a separate Evaluation Unit was created. Between 1992 and the end of 1995, a total of 109 evaluations were carried out, at a cost of 1,3 Mio ECU (65). All the evaluations are carried out by consultants, who are selected directly by ECHO in each case. ECHO has built up a list of potential evaluators from its own experience, names proposed by Member State humanitarian aid agencies, and information provided by other Directorates General and organizations involved in humanitarian aid. By the end of 1995, a total of 48 different evaluators had worked for ECHO.
5.2. In the interests of a standard approach, ECHO first developed a set of terms of reference with specific headings/questions, and then, in 1995, with the help of external expertise, prepared an operational manual, which was introduced in 1996.
5.3. While most evaluation reports completed up to the end of 1995 contain useful information and data, the following weaknesses were identified:
some of the evaluation reports are superficial, and consultants have not carried out the terms of reference in a rigorous or thorough manner; this indicates weaknesses in the selection and/or supervision of the consultants;