All posts by Eileen Munro

EIP Signs of Safety Research: Statement by Munro, Turnell and Murphy

The England Innovations Programme (EIP) has published the evaluation reports on the MTM Signs of Safety project with 10 local authorities (Baginsky et.al 2020) and the summary evaluation report on all EIP projects (FitzSimons and McCracken 2020).

Evaluation of MTM’s Signs of Safety Pilots – Evaluation report
Evaluation of MTM’s Signs of Safety Pilots – Evaluation report appendices

The reports conclude that the MTM project had no impact. MTM accepts that the outcomes we set ourselves — transformation of the organisations and their child protection case results — were not matched by our interventions. 

A more nuanced picture of the MTM EIP impacts is reflected through an analysis of Ofsted reports and the surveys of staff and families in the local authorities (Munro and Turnell 2020). Some did very well and others made no progress or in fact went backwards. Whereas these mixed results wash out to show no impact for the whole set of local authorities, Munro and Turnell demonstrate that the results depended on what the local authorities actually did, particularly in their leadership and the extent to which they aligned organisational systems with the practice. 

MTM takes this as a sobering reminder that a poor implementation of Signs of Safety could make things worse. This should be no surprise as introducing another set of demands into a busy and complex environment without clear commitment and making it fit and thus allowing space for the work, will produce as much contention as traction.

MTM also notes that the Partner in Practice Lincolnshire project that included embedding Signs of Safety and the Partner in Practice Richmond and Kingston Achieving for Children project that included implementing Signs of Safety both showed “some positive impact”. Additionally, the key messages from the summary evaluation report on all EIP projects (FitzSimons and McCracken 2020) describing what is effective in practice accord with the key features of the Signs of Safety model. 

The evidence from both the independent study and MTM’s own (Baginsky et al, 2020 and Munro and Turnell, 2020) shows that Signs of Safety was only partially and to varying degrees implemented in the ten local authorities and so the studies tell us about the challenges of implementation not the impact on families of receiving a Signs of Safety service.   

The research is testament to the assertion by Pawson (2006) that “social interventions (such a practice framework) are complex systems thrust into complex systems”. While MTM is confident that Signs of Safety can make powerful differences for particular cases, particular practitioners and teams, we take lessons from the research as to the multiple factors that affect adoption of Signs of Safety and the difficulties of achieving the desired depth and scale of practice in a sustainable way.

MTM takes strong note of reported confusion among some staff about what the Signs of Safety actually is. It accepts that as Signs of Safety is a comprehensive framework and methodology for child protection practice comprising strong principles, practice methods and disciplines for applying those methods, setting a process for the practice rather than prescribing the content, there is a lot to grasp. Moreover, MTM recognises the challenge, as Baginsky et al (2020) reported: “Supporting families to take responsibility, and to work in partnership with them to do so, is at the heart of Signs of Safety. While it was viewed as a strength of the model it was not always seen to be compatible with statutory social work in England or with the high level of risk involved in many child protections cases. It was agreed that MTM could have given more guidance on how to achieve this using Signs of Safety. The concern was that Signs of Safety in the hands of an experienced child protection social worker was very different from that used by an inexperienced worker, where it could become superficial and the line between the model and statutory responsibilities become blurred.”

Addressing both the clarity of the practice and implementation issues is imperative as Baginsky et al (2020) have reported that around a third of local authorities in England use Signs of Safety as their model of practice and another third also do so in combination with other approaches. 

In offering the Signs of Safety, MTM considers that we come to the complex systems of child protection agencies with values, aspirations and a vision of collaborative practice and partnership with families that the vast majority of child protection professionals want to embrace. We offer methods and tools within a framework for this practice. Additionally we look to get alongside agencies and their leadership to work on whole system change that involves detailed attention and slow thinking and discerning judgement. While MTM has a well-developed implementation framework with good methods, and some local authorities did well in the EIP, it unreservedly accepts that it has not sorted out all the implementation issues and is on a journey with child protection agencies in doing so. MTM’s strongest challenges lie in building a shared understanding and working in partnership with agencies and researchers of how significant the paradigm shift is, how complex the systems are that we are trying to change, how hard it is to do so, and how likely failure always is. 

MTM and the the Signs of Safety community has learned an enormous amount from the England Innovation Project. The Signs of Safety practice has evolved in the key areas of analysis and safety planning, the implementation framework has been refined and further developed to be more focussed and include practical methods that support the agency to own the approach, measures for practice breadth, depth and impact have been developed, and the Signs of Safety IT Solution that fit with agencies’ existing systems to deliver essential alignment and substantial efficiencies is available.

Profound change can be achieved. Some EIP local authorities did so. And another recent Ofsted assessment of “outstanding” for North Tyneside, having worked on whole system implementation over some years including a “learning lab” for the Signs of Safety IT Solution, attests:

The way in which the local authority’s preferred method of social work has been rolled out across early help and children’s social care, and embraced by partners, has had a transformational impact. It provides a common language with which to talk about and explore issues and concerns, needs and risks, dangers and protective factors in a way that is easy to understand for parents, professionals and partners. Particularly impressive is the way in which the local authority’s electronic case recording system has been adapted to ensure that it helps rather than hinders this approach. Equally impressive is the way in which senior leaders are leading by example, using the same simple methodology in reports and policy documents.

A comprehensive summary of Signs of Safety research on family outcomes and parent perceptions (Pecora 2020) is available on the Signs of Safety Knowledge Bank.


Professor Eileen Munro and Professor Andrew Turnell with Marie Devine and Dr Jack Cunliffe, You Can’t Grow Roses In Concrete Part 2 Action Research Final Report, Signs of Safety England Innovation Programme, Elia International Ltd, 2020

Mary Baginsky, Ben Hickman, Jess Harris, Jill Manthorpe, Michael Sanders, Aoife O’Higgins, Eva Schoenwald and Vicky Clayton, Evaluation of MTM’s Signs of Safety Pilots Evaluation report, October 2020. UK Department for Education

Dr Ana FitzSimons and Katie McCracken, Opcit Research Children’s Social Care Innovation Programme, Round 2 Final Report, September 2020, UK Department for Education

Professor Peter Pecora, Signs of Safety Research Summary of Family Outcomes and Parent Perceptions, 2020, Casey Family Progams

EIP2 Action Research Final Report Released

Brief overview – Eileen Munro

'You Can't Grow Roses In Concrete' Part 2 cover

How can similar child protection agencies implement the same Signs of Safety approach and receive assistance from the same trainers and consultants yet show vastly varied progress (and, for some, regress)? This is the question explored in the second report on the England Innovations Programme: ‘You can’t grow roses in concrete’ Part 2.

In all, Munro, Turnell and Murphy Child Protection Consultancy (MTM) received two grants from the UK government to work with ten English local authorities implementing whole system change to support Signs of Safety practice, the first for 18 months work in 2014–16 (EIP1) and the second for two years in 2017–19 (EIP2).

During the five-year period of the work, all the authorities were inspected twice by Ofsted, the national inspection body. Three authorities progressed to the highest rating of ‘outstanding’, four showed some progress, and three either showed no progress or deteriorated to the lowest judgment ‘inadequate’. At one level, the disparities are not surprising: introducing change into complex social systems like child protection agencies will have different effects depending on how the change interacts with other parts of the system. But what other parts of the system are most influential? This report pulls together the evidence on how each authority implemented the sections of the Signs of Safety implementation framework to see whether progress on the implementation predicts progress in Ofsted judgments.

The behaviour of leaders seems to be crucial especially in demonstrating commitment to the reforms and having close contact with direct work with families. Slowness in aligning administrative processes and IT to the practice caused additional work for front line staff and can be seen as sending mixed messages about the commitment to implementing Signs of Safety completely. Monitoring what was happening and looking for evidence of quality not just quantity was challenging in a sector embedded in a compliance-with-process culture. Providing training was possibly the easiest section of the implementation framework but progress was clearly linked to how extensively the authority instigated methods for on-going learning and support. Group supervision was highly valued in those places where it was established as a core, not a peripheral, support.

In EIP2, a new version of the staff survey used the Safety Attitudes Questionnaire developed and extensively researched in aviation and, subsequently, in health. This captures measurement of dimensions of work culture that are associated with the number of mistakes, slips, and poor practice. Between the two surveys carried out, the best performing authorities showed good progress and the moderate performing showed some progress. Only one authority in the weakest group conducted two surveys and that showed deterioration on all dimensions except the recognition of stress.

An independent evaluation is being carried out by a team from Kings College, London and that will explore more of the impact on children, young people and families. It also studies implementation and includes a contrast study of two local authorities using Signs of Safety and two who are not.


The full report and linked documents can be accessed from https://knowledgebank.signsofsafety.net/you-cant-grow-roses-in-concrete-part-2.