The social worker I spoke to sounded weary. “Everyone thinks they would have spotted what was going on with the Doncaster boys and that they should have been taken away long ago. Well, maybe they’re right. But among the cases I work with, that family wouldn’t stand out. Violence, drugs, chaos: that’s what I deal with every day. I can’t take children into care just for that. There’s not the space for them. There are so many families like that. I don’t think the public has any idea.”

You can call this broken Britain, as David Cameron does, or see it as the ugly consequence of a profoundly unequal society like Ed Balls. The result is the same: 100,000s of children are growing up in disorder and neglect, and our system is prepared to deal with only a fraction of them. This woman works in a northern town and says that over the past 20 years the families on her case list have become increasingly difficult to handle. Those with multiple problems are now so numerous that only the most complex cases are looked at; and they are hard for her to judge.

“I go into houses and they don’t live like we’d live. There can be dog faeces all over the floor, and beer cans, and so much rubbish that there’s only a narrow path from the front door to the kitchen. There might be a lot of swearing and aggression, and the children are missing school, and no one works. But it’s not my job to make value judgments on how adults live their lives. We're not allowed to do that.

“Our only concern is whether the children are at substantial risk of harm, and that’s the criterion we use to ask them to change. And they’re often a bit sullen, but next time you go there’s a bit less muck on the floor, and the kids are in school half the week, and the health visitor’s been, and you think things are getting better. You hope it is, because the pressure’s on you to keep that family together. Then a couple of months later it’s all gone back again, and the kids might be in a bit of trouble with the police — but it’s hard to tell when that all crosses the line and when you should be thinking, ‘Hang on, this really isn’t working and these children shouldn’t be here.’ ”

Every day, social workers are being asked to make sophisticated judgments. They’re expected to make accurate assessments of motivation, psychology, character and capacity for change. When they get those judgments badly wrong, as happened in Doncaster, they risk being mercilessly and publicly criticised. Yet they aren’t given the training, the time, the freedom or the resources to produce the results we want.

Over two decades the reaction of governments to every high-profile tragedy in social work has been to create ever more restrictive systems in which staff must operate. The belief is that human error will be reduced if only all the proper procedures are followed. The lives of social workers are governed by the integrated children’s system (ICS), a computer model that tells staff when they must file reports, communicate with other agencies and complete assessments.

Responding to the computer timetable has become a much higher priority than responding to the complicated, messy needs of the humans it is meant to serve. Staff now spend far more time on forms and in meetings than they do with their clients. Twenty years ago 30% of their time was spent with families; now it is just 11%.

Eileen Munro is a reader in social policy at the London School of Economics who is highly critical of this bureaucratic approach. The starting point for any social work is observation and understanding, and yet that now has a low priority. The gradual realisation that a child is frightened of its father or that a mother has deep psychiatric problems requires repeated visits, time and thought. Good social work often requires a constant reassessment of initial judgments, and yet that’s something all human beings find hard to do. We prefer to stick with the first story we tell ourselves. So a social worker who at first thinks a family is responsive is going to be reluctant to recognise the gathering evidence that something is wrong.

Munro says questioning a caseworker’s decisions used to be at the core of what social work supervision was all about. Now managers tend to concentrate on whether the boxes are being ticked on time. That is, after all, what local authorities’ children’s services are judged by, and what determines their star rating, their directors’ jobs, their budgets, and whether they are named and shamed. Worse still, the computer system discourages staff from changing their minds. Altering the initial assessment means staff must re-enter the entire 40 pages again.

Two recent reports from the Commons children’s committee have echoed and added to those criticisms of how children’s social work now operates. Too much monitoring impedes action. Social workers are often powerless to get children specialist help. Care is too often a last resort. Managerialism has replaced professionalism as in much of the public sector. Above all, the quality of social workers needs to be raised. Neither the entry requirements nor the training are producing staff who can do what the job now demands. Many are simply not up to scratch.

The MPs called for a complete government overhaul of the system. Some authorities aren’t waiting for that. In Hackney, east London, a borough where children’s services were in special measures five years ago, the council has radically rethought its work. New units have a psychologist or a clinician to advise them, and all have an administrator, so that 70% of staff time can be spent with families. The ICS is being ignored in favour of a simpler system that Hackney is developing itself. The greatest emphasis is on recruiting bright people. Just as Cameron wants to raise the bar for teachers’ qualifications, so social workers need to be of higher calibre too.

There must also be a readiness to intervene far earlier and more forcefully in vulnerable children’s lives. We used to think children were resilient. Now we know that by the age of three the damage done to the brain of a brutalised or neglected child is almost irreversible. It is no longer defensible to leave children such as the Doncaster boys in such chaos to see how they turn out. We should either support them intensively or remove them, fast. Neither solution is easy and both are expensive. But if we really don’t want to live with the consequences of these traumatic childhoods, we have no choice but to act.

jenni.russell@sunday-times.co.uk

Minette Marrin is away