An independent evaluation was completed in December 2007 to ‘assess the usefulness of the All in Diary and determine the degree to which it achieved the project objectives.'   The evaluation methodology included a review of documentation, survey of 61 / 189 trial participants and in-depth interviews with a number of key informants and 29 trial participants in Sri Lanka.

Evaluation findings

• Effectiveness: the All In Diary project delivered most of its production outputs, and made early progress toward achieving primary expected outcomes. It appeared to be a well conceived project admired by its stakeholders.

• Relevance: the All In Diary concept and formats (hard copy, CD Rom and website) seemed very relevant to the target group of less experienced field-based staff in rapid onset disasters, who needed broad comprehensive humanitarian information and sometimes struggled to access information at local level. However, time availability and language would be barriers for the target group, and the All In Diary seemed equally relevant to a wider group of humanitarian workers.

• Utility: the All In Diary was generally perceived as useful or 'very useful' by trial participants and stakeholders. The information provided was considered useful, comprehensive and presented concisely. Utility problems could be resolved in relation to: design and layout, language barriers  (a Tamil version has now been developed and a Sinhala version is underway), summary formats (that presume knowledge but may be addressed through clearer links to the supplementary resources) and the recognised absence of context-specific information (which was not developed for the global trial).

• There were mixed views about the utility of the All In Diary as a diary and to assist handover.  Generally the space for note taking was considered to be insufficient. For many, the All In Diary personal organiser format was too large for daily use in the field. The electronic formats were perceived as useful and many people were able to access them without difficulty. Whereas participants and organisations might be prepared to pay towards the All In Diary, additional funding should be sought to subsidise its production and increase its impact.

• Utilisation: a narrow majority of participants appear to have used the All In Diary and in diverse ways. Overall the impact of All In Diary seemed most likely in:

i)    raising awareness and learning
ii)   providing access to information
iii)  supporting improved practices (more than changing practices).

Use of the All In Diary, it appears, will result primarily in increased awareness, but it may also contribute to improving All In Diary quality by reinforcing good practices over time. The All In Diary, it appears, is more likely to help learning as a training tool than through diary notes and handover support.

o        The steering group  review the All In Diary aim statement so that it specifically reflects commonly articulated and recognised goals for humanitarian action;

o        The All In Diary is further developed and tested in a second trial / set of trials (e.g in a disaster, a conflict and a sudden disaster);

o        The producers embrace a research and development approach that relies on small and frequent utility tests.

 The next phase should:

• target a broader category of  'field-based humanitarian staff';
• focus on how to increase and reinforce awareness of humanitarian standards and good practice;
• review the function and design of the diary pages;
• enable use of the All In Diary as a multifunctional information tool such that use of the information does not depend on use of diary pages;
• reduce the size of the wallet;
• develop a marketing strategy, integral to the production process, aimed at organisations as purchasers.

The evaluation highlighted specific synergies between the aim of the All In Diary and work of the Quality & Accountability Initiatives and recommended that:

 ‘the Quality and Accountability Initiatives analyse the opportunity presented by the All In Diary with regard to their own efforts to raise awareness; provide access to information; and support improved practices. They should also consider supporting the electronic All In Diary formats in the context of their commitments to work together, and provide information in a comprehensive, concise and impartial manner’.