Lawyers for reputed gangster James “Whitey” Bulger got Thursday what they were looking for: the removal of the federal judge presiding over the case, U.S District Judge Richard Stearns. Bulger’s defense team had argued that Stern’s background as a former federal prosecutor serving at the height of the crime lord’s reign in Boston would create the appearance of bias, and the U.S. Circuit Court of Appeals agreed. The decision is not sitting well with some who worked on tracking the long elusive fugitive.

Last year, attorneys for Bulger twice asked Stearns to recuse himself. When he declined, Bulger’s public defender, J.W. Carney, Jr. filed a formal request for removal with the federal appeals court. Here’s the thrust of Carney’s argument: Stearns should be removed, he said, because he was a federal prosecutor in the 1980s, at the same time that Bulger was working undercover as an FBI informant. Carney said there was no way that Stearns -- the former chief of the criminal division of the U.S. Attorney’s Office here -- could be fair and impartial under those controversial circumstances. The appeals court agreed in a decision written by retired U.S. Supreme Court Judge David Souter.

This is a significant victory for the 83-year-old Bulger, according to two attorneys not connected to the case. And it is infuriating to one of the special agents assigned to track him down over many years.

“It’s frustrating, and it’s really unsettling for all of the law enforcement officials who worked on the Bulger case,” said Pamela Hay, a retired U.S. Drug Enforcement Agency agent, worked from 2000 to 2005 to try to locate Bulger, then one of the FBI's most wanted.

“I feel that he has to face trial,” Hay said. “He’s been charged with 19 murders, and that’s charges that we just know about, and that the government is aware of. But I can say that the drama in this case continues to escalate and and I can’t wait for the truth and evidence to come out, so the public can see what kind of a monster this man really is.”

I also asked Hay if there was reason to believe that some in the government actually welcome a delay in the trail of the former head of the Winter Hill Gag because of what the reputed gangster might reveal in terms of his relationship with the FBI.

“I don’t know if it will hurt the government, but I certainly think that one particular agency may be shaking in their shoes,” she said.

Hay -- who now teaches at Boston University -- says she is looking forward to Bulger taking the stand and “facing the music,” as she puts it. Bulger’s attorney says he may now call Stearns to the witness stand to comment on the government’s role in Bulger’s alleged reign of terror in the 1980’s.