The letter, which was signed by the department's top civil rights attorney, criticized Mayor Bill de Blasio for limiting the number of people who could attend religious gatherings to prevent coronavirus spread, but failing to impose similar restrictions on individuals protesting the death of George Floyd.
The Wall Street Journal first reported Berman's refusal to sign the letter, citing his concern the letter was a political stunt.
"The message to the public from New York City’s government appears to favor certain secular gatherings and disfavor religious gatherings," the June 19 letter said.
A spokesman for the US Attorney's Office for the Southern District of New York declined to comment.
US Attorney General William Barr announced on Friday that Berman would be resigning next month as US Attorney for the Southern District of New York to pave the way for the appointment of Jay Clayton, who is currently chair of the US Securities and Exchange Commission.
Berman refused to step down, only to have US President Donald Trump fire him on Saturday.
Berman's office, which is known for its high-profile prosecutions of terrorists and white collar criminals, has not shied away from probing people in Trump's inner circle.
Justice Department spokeswoman Kerri Kupec said Barr did not realize Berman had not signed the letter to de Blasio before he announced Berman's resignation.
Berman's office oversaw the prosecution of Trump's former personal lawyer Michael Cohen, indicted two of Rudy Giuliani associates and launched a probe into Giuliani's efforts to dig up dirt on Trump's political adversaries in Ukraine.
Berman ultimately agreed to resign after Barr pledged to install Berman's hand-picked No. 2, Deputy US Attorney Audrey Strauss, as Acting US Attorney until a permanent replacement is put in place.