Back to Published Articles
Journal:
Eval Clin Pract. 2003 Nov;9(4):391-400
Author(s):
Pagliari C, Clark D, Hunter K, Boyle D, Cunningham S, Morris A, Sullivan F.
Contact:
Tayside Centre fir General Practice, University of Dundee, Dundee, UK.
OBJECTIVE:
Failure to engage in user-informed evaluation of emergent health informatics tools can have negative consequences for future implementation, related both to poor usability or clinical utility and to suboptimal stakeholder buy-in. This paper describes a formative evaluation in primary care of a multifaceted, web-based resource for diabetes management. The primary aims were to assess the usability and utility of the prototype in order to inform system refinements prior to implementation, and to investigate barriers and facilitators to system use so as to aid the development of a tailored implementation plan.
METHODS:
A mixed-method approach involving survey, remote observation, semi-structured interviews and electronic feedback.
CONTEXT:
One Scottish local health care co-operative comprising five general practice surgeries and their staff.
RESULTS:
A survey following temporary exposure to a dummy site revealed high levels of computer familiarity, welcoming attitudes and positive ratings of usability, format and utility. Comments mainly addressed content accuracy, feature suggestions and usability issues. Key barriers and facilitators to use included time and training. Remote observation following access to live clinical data enabled profiling of usage by individual and professional group. In the 3 month observation period administrators were the most frequent users, followed by GPs and nurses, with clinical and data entry screens accessed most often. Semi-structured interviews with key respondents sampled by professional group and usage frequency provided richer qualitative information on barriers and facilitators to use, patterns of system integration into work routines and system usability, content and utility. Content analysis of electronic feedback revealed mainly technical queries and general expressions of satisfaction.
CONCLUSIONS:
Evaluation informed a number of important and unforeseen improvements to the prototype and helped refine the implementation plan. Engagement in the process of evaluation has led to high levels of stakeholder ownership and widespread implementation