Search Results

This chapter suggests how the different evaluation techniques used in research evaluation systems (RES) affect their informational yield for the exercise of authority by university management. It compared three peer review-based and two indicator-based systems according to their informational yield and to the use to which the information is put by university management, and uncovered clear differences between the informational yields of varied RES for universities and between the ensuing usability of information for different purposes. The clear superiority of peer reviews was demonstrated by the range of changes it enabled in the German state of Lower Saxony, as well as by the distortions resulting from the internal use of quantitative indicators in Australia. Further indirect confirmation stems from the widespread ad hoc use of peer reviews in structural decisions, and decisions about promotions to professorial levels in Australia.

A research instrument is defined as a systematic and standardized tool for data collection. It includes all types of research questionnaires and standardized scales. There are three ways in cross-cultural research instrument development: adopting an existing instrument, adapting or modifying an existing instrument, and developing a new instrument. To develop a cross-culturally valid questionnaire or instrument, the concepts or constructs selected for the investigation must be clearly defined and bear the same meanings across the selected cultural groups. No good questionnaire can be developed without clear definitions. This is a matter of utmost importance for all levels of cultural comparative research and evaluation, whether it is a gender or racial/ethnic comparison within one society or across nations. Chapter 2 describes the process of cross-cultural instrument development, from formulating the research aims to the assessments of cross-cultural measurement properties.

This chapter describes the common approaches to evaluation, which is the fourth step in the Total Process Planning (TPP) framework. As well as setting out the theory behind designing and conducting evaluation research, it provides case studies that show how evaluation works in practice. By the time the evaluation phase is reached, you should have completed the scoping report, and developed and implemented your marketing strategy. However, evaluation is not a distinct stage that only needs to be addressed when you have implemented your intervention. It must be considered from the scoping stage onwards.

This chapter highlights the gaps in the evidence base of palliative care and attempts to provide some pointers on the future direction of research in the evaluation of palliative care. It discusses the strengths and weakness of evaluation research and proposes different ways to enhance the effectiveness of evaluation studies. The chapter also explores the policy process concerning palliative care planning and provision.

The lengthy observational period required for many experimental and observational studies in child welfare has lead to greater interest in formative evaluations for guiding quality improvement. This chapter offers practical advice for opening up the proverbial “black box” of evaluation research and for developing data-tracking systems for continuous monitoring of service implementation, program outputs, and longer-term outcomes. It discusses the importance of fostering a results-oriented organizational culture that communicates the attitudes, behaviors, and values of results-oriented accountability (ROA).

In this concluding chapter, the commonalities and differences among the books’ contributors are discussed. In particular, the various meanings given to “justice” and “restoration” by the authors are considered. If justice is seen as, in part, an experience, the question is raised as to where, in the practices described in the book, justice is thought to happen. The chapter ends with recommendations concerning new justice practices that address violence against women, with an emphasis on screening for abuse, advocate-researcher collaborations, and evaluation research.

This chapter discusses evaluation research and how it can be informed by the theory of change research. More specifically, it considers how community-based practice must increasingly document effectiveness through research that systematically evaluates whether community-based programs and services produce intended results, especially when funders demand this type of accountability. Although accountability is undoubtedly a legitimate goal, the chapter argues that the type of evaluation research that is demanded in the name of accountability often leaves a great deal to be desired. It also highlights the problems with documenting effectiveness via evaluation research as well as the criticisms against measurement associated with performance management, such as disconnecting social work researchers from their community partners to the point of suspicion and distrust instead of promoting the synergy and increased effectiveness that can come from combining forces to revise and improve practice.

The aim of this chapter is to identify the major methods of research in the social sciences and to examine the relative strengths and weaknesses of each method with respect to their relevance and usefulness in the arenas decision making, social policy, and social problems. The different research methods are classified as laboratory-experimental, evaluation research, statistic methods and survey research, comparative-historical analysis, case studies, counterfactual reasoning and mental experiments. These methods are treated as variations—but identical in aim—in the process of varying and holding relevant variables constant in order to arrive at adequate causal explanations. The authors assess strengths and weaknesses of each as they are applied to “real” situations in the “real” world.

Broad spectrum interventions aimed at reaching the general population make use of mass communication approaches such as TV, radio, press, billboard posters, and leaflets. This chapter examines the difficulties inherent in evaluating mass media campaigns. Topics discussed include the scope of interventions, evaluation research, process evaluation, outcome evaluation, selection of indicators/outcome measures, and measuring unintended consequences.

This chapter examines the need for a new initiative, the evidence for this, the content of the provision, the design of the programme, and the research evaluation and design, and concludes with an overview. It brings out positive elements of the initiative, as well as making critical comments. The chapter also focuses on effects on child and family functioning rather than on the longer-term and more indirect effects on child poverty. It questions whether the government has learnt any lessons from the experience of the implementation and evaluation of SSLPs.

This chapter examines how wider social forces, including globalization, neo-liberal economics, and widening participation in higher education shape government policies, and how much space these allow for academics to pursue meaningful and socially useful research. It assesses how various research evaluation schemes can be made more effective and meaningful, and how they may encourage more relevant research in the social sciences. The chapter addresses the use of bibliometrics (citations indices) and measurements aimed at assessing the impact of academic research. It also examines how teaching may be restored as in its rightful place as a core meaningful activity for social science academics. Teaching must once again anchor the identities and practices of academics, whether they see themselves as research-active or inactive, as long as they seek to maintain active scholarly identities.

The principles of design, conduct, analysis and reporting of clinical trials, including trials conducted for regulatory purposes, are sufficiently advanced that drugs and other therapeutic interventions can be put to fair tests that provide valid evidence of efficacy and safety. However, clinical research—especially research that can make a product a winner or a loser—is not a value-free human endeavor. The rosiglitazone story exemplifies how medical product manufacturers can manipulate regulatory evidence by designing trials to ensure certain answers are—or are not—obtained, how companies can try to intimidate scientists whose research yields unfavorable evidence about a product, and how ambiguous can be the evidence from clinical trials. Federally-mandated comparative effectiveness research does not generate evidence used in product regulation, but it generates evidence that has market implications. It therefore will always be at risk for the same kinds of manipulation observed in the rosiglitazone saga.

This chapter reviews a range of concepts appropriate for evaluating oral history research. It presents strategies researchers can employ in their own research in order to address each dimension on which oral history may be judged. It discusses concepts such as explicitness, thoroughness and congruence, ethical practice, and validity and related concepts.

The epilogue, after reminding readers of the fundamental problem of massive iatrogenic deaths and injuries in the American healthcare system (by transposing the incidence into the context of air travel) and noting the inability of the healthcare industry and of traditional legal responses to improve patient safety, encourages the following of Dr. Seuss’s counsel: “If you can see things out of whack, then you can see how things can be in whack.” The book has explored a number of law-driven and law-related developments, and speculated on new possibilities. Its larger aim is to fuel a wide-ranging conversation directed at finding effective legal innovations able to improve patient safety. The book also encourages experimentation and evidence-based evaluations of those innovations: abandon what does not work, disseminate and replicate what does work.