Raphinha converted his spot-kick in the third minute of added time after Leeds skipper Liam Cooper’s header had struck Palace defender Marc Guehi on the hand in the area.
It was a dramatic finale to what looked like being another frustrating stalemate for Bielsa’s side, who sealed just their third Premier League win of the season.
“It was a relief, we can breathe again and it was very satisfying,” said Bielsa, who was clearly delighted his players responded positively to last week’s disappointing display in a goalless draw at Brighton.
“The team fought the whole game,” he said. “That is true, it’s necessary. Either way, our team gives everything in every game.
“In that aspect, the team was very similar to the game against Leicester. Always, a victory brings happiness and in a human group happiness is indispensable.”
Leeds have come under increasing pressure after failing to reproduce the exhilarating form they showed last season and Bielsa revealed what pleased him most about a win which lifted his side five points clear of the bottom three.
“The intensity with which the team played, the constant search for a goal, even if it was difficult to get it,” he added. “The closeness, the togetherness of all the team to defend together and attack together.”
The late penalty was cruel on Palace, who had lost for the first time in eight league matches at home against Aston Villa on Saturday.
Patrick Vieira’s side always looked capable of snatching victory themselves on the counter at Elland Road, with Jordan Ayew and Wilfried Zaha a constant threat.
Substitute Christian Benteke spurned one of the game’s best chances when heading Conor Gallagher’s cross wide at the back post and went agonisingly close with another stabbed effort.
Vieira said: “It’s a tough one, a real tough one. I thought we were the better team tonight, we created the most dangerous situations and chances. To lose the game that way is hard to take.”
Referee Kevin Friend awarded Leeds’ penalty after consulting the pitch-side monitor and Vieira had no complaints about that decision.
“I was expecting it (the decision). If I was on the other side I would expect the referee to give the penalty. I’ve had the chance to look at it again and it was a penalty.”
Palace lost back-to-back league games for the first time under Vieira, but the Frenchman said he could not fault his players.
“The performance tonight is more positive,” he added. “Obviously the outcome of the game wasn’t something that we wanted.
“But the performance was what I wanted from the squad, from the team, because we defended well as a team, we didn’t give them so much space, they didn’t create too many chances.
“I think we were the better team, we just didn’t take our chances and when you don’t at this level, you get punished.”