The design of a user satisfaction survey instrument and methodology is determined by the objective of the survey, the users' characteristics and the resources available. These three inputs will inform decisions about the survey options. The option chosen will in turn influence the layout of the survey, methods of maximising response rates and methods of analysis. These are all interrelated and it will be necessary to review earlier decisions when viewing the survey project from each of these aspects. Edited version of a paper given at the Performance measures for libraries and information services conference Sydney May 2004
User satisfaction survey design is not a linear process. This paper cannot therefore be a simple progression. Background research needs to identify the objectives of the survey, the user group or groups, and the resources available. The information determined in this research phase will then inform the instrument development, administration and analysis phases. A well designed survey
* will not just be a very subjective measure of user satisfaction, but will provide a measure of the value of the services provided by a library
* provides library managers with priorities for effort and a basis on which to report the return on investment, in qualitative terms, to stakeholders
* is constructed, run and analysed quantitatively to ensure that the results can be relied upon and generalised
User survey objectives
User satisfaction is a complex concept. It may simply be how good users feel after dealing with a library, it may include their likelihood to return to that library when next they need information. It may also include their perception of how well the experience answered their information problem, improved their productivity or the quality of their own output.
As the complexity of the user satisfaction concept increases so does the measurement method. In the case of measuring how good a user feels after dealing with the library a simple complaints or feedback system, planted shoppers, a suggestion box, or a point of sale smiley face flick and tick survey could be employed.
User satisfaction may be affected by previous encounters with a library, current information use environment, the characteristics of the information product or service, or their current information gap. The characteristics of the products and services that may affect satisfaction are the level of customisation, the subject matter, up to datedness, complexity, reliability, organisation, language, and format.
User satisfaction is related to the benefit that is perceived to be gained in using the library service. This benefit could be measured in actual cost savings, in terms of the willingness of users to pay in terms of real price, cost savings in dollars or time, decrease in uncertainty, decrease in duplication of work or potential loss of productivity. A cost/benefit analysis can then be used to determine the value of the services. These methods require stringent instrument design to reduce bias and inaccuracies.
The value of information services can also be measured after the information has been used and after decisions have been made. Tracing the affect of the information can be used to determine the actual use made of each product or service. This is time consuming and difficult, but can provide a rough quantitative measure of value.
Gallagher, in an innovative study, attempted to show that client satisfaction is an equivalent measure of value based on the cost benefit ratio. He compared an estimated dollar value measure with a qualitative multi item semantic differential technique to measure the value of a management information system report. The results showed a small statistically significant correlation between the two value measures. (1) His approach has been criticised on the basis that the correlation was too low to conclude that the two measures were measuring the same phenomenon. …