Labour’s deputy leader Tom Watson has said now is “not the time” for a leadership contest, but the party must “do better” at winning over voters.
Speaking at the Scottish Labour conference in Perth, he said the party’s by-election loss in Copeland had been “hugely disappointing”.
Jeremy Corbyn has said he will stay on as leader, after Labour held Stoke Central in a separate by-election.
David Miliband has said Labour is at its weakest point in half a century.
The former foreign secretary, who lost out on the Labour leadership in 2010, told the Times that he was “deeply concerned” about the party’s future.
Mr Watson said he would not seek to “sugar coat” the loss in Copeland.
The Conservatives won the Cumbrian seat in the first by-election gain by a sitting government in 35 years.
In the other by-election this week, Labour retained its seat in Stoke-on-Trent Central, with newly elected MP Gareth Snell winning 37% of the vote – slightly down on the 39.3% Labour got in 2015.
Mr Watson told delegates: “All of us with leadership roles in the Labour Party need to have a long hard look at ourselves at what’s not working.
“I have said it a lot recently – this is not the time for a leadership election, that issue was settled last year.”
Mr Miliband, who now works as the head of the International Rescue Committee in New York, said: “I don’t think this is just a repeat of the 1980s.
“We have to really understand the historic nature of the challenge that we have to face.”
But when asked if he would come back to politics, he said: “I honestly don’t know what I am going to do. It’s hard to see – but what’s the point of saying never?”
BBC political correspondent Chris Mason said some in Labour saw Mr Miliband as the best leader they never had, but it was not the first time he had been “a public doom-monger” about his party under Mr Corbyn.
Labour had held the seat in Copeland for 80 years. PM Theresa May said the “astounding” victory there showed her government was “working for everyone”.
Trudy Harrison got 44.3% of the vote, increasing the Conservatives’ vote share by more than 8%. She overturned a Labour majority of more than 2,564 to take the seat by 2,147 votes – a swing of more than 6%.
The leader of trade union Unison, Dave Prentis, told the Guardian that the loss was “disastrous”.
“The blame for these results does not lie solely with Jeremy Corbyn, but he must take responsibility for what happens next,” he said.
Gerard Coyne, who is standing to become general secretary of the Unite union, said there was “no doubt” Copeland was a “meltdown” for Labour.
Labour must “face up” to the “disconnect” between Westminster and ordinary Labour voters, he said.
But he stopped short of calling on Mr Corbyn to step down, saying it was a matter for the party and its MPs.
Mr Corbyn said Labour’s “message was not enough to win through in Copeland” but he hailed victory in Stoke as a “decisive rejection of UKIP’s politics of division and dishonesty”.
Behind Labour’s 37.1% share of the vote in Stoke, UKIP took 24.7% to come second, narrowly beating the Conservatives on 24.4%. Both UKIP and the Tories did slightly better than in 2015.