Speaking exclusively to BBC Breakfast on Sunday, Burnham said he had applied “common sense” on IPPs.
“I was faced with a situation where people on those sentences who have more than served their time, not being released when others who’ve committed very serious crimes being released early,” he said.
He said the government would table legislation to terminate IPPs by the end of this Parliament.
He conceded that of those offenders, there would be those who would “pose a risk to communities”, and a cross-party oversight group would be established.
And there are others who have a more favourable outlook about releasing some prisoners serving IPPs.
Former Conservative justice secretary Sir Robert Buckland told the Today programme that while there are people serving these indefinite sentences who are a “real danger to the public”, these are “dwarfed” by instances where people end up back in prison for things like breaching their strict licence conditions.
One such person is Shaun Lloyd, who served eight years following a conviction for street robbery.
Once released, because of his IPP, he remained on indefinite licence. People on these could be recalled for behaviour that, while not criminal, would be in breach of those rules.
His mother, Shirley Debon, told the Today programme that he had never committed a violent crime and called his latest recall “totally unfair”.
When Lloyd was called back to prison, he had become drug-free and was near to completing an apprenticeship, she said.
Debon is campaigning more broadly for an inquiry into IPPs, under which she says people are “living in limbo”.
Sir Robert said that looking at recall cases like Lloyd’s could be a solution for the government.
He added that there was “a very good policy reason” why IPPs were abolished: “It did create uncertainty, not just for the prisoners but actually for the victims”.
On the impact overall of changes to the early release scheme for prisons, Davies-Jones told the BBC: “We will have capacity within the system with those numbers but it will be run very tight.”
She said plans to deport foreign nationals on sentencing and to re-purpose women’s prisons would help tackle overcrowding.
“Our plan is to deport [foreign nationals] on sentencing so that they don’t serve a day at his Majesty’s and at the taxpayers pleasure.”
Asked if these offenders would serve sentences for their crimes once deported, she said “that would be for the determination of their country of origin”.
On 3 August, the prison population in England and Wales was 86,495, around 97% of useable capacity.
On Tuesday, Justice Secretary Alex Norris is expected update Parliament with further details on the changes to the early release scheme.
