Residents reported heavy gunfire in Aden's central Crater district that houses government headquarters and the central bank.
The museum, located in Tel Aviv, tells the story of a small but long-lasting Jewish community who were located in the Gulf for almost 1,000 years.
It is unclear what comes next in the war-torn country.
Coalition air strikes have killed thousands, including many civilians, over the course of the war.
Saudi Arabia has hosted indirect talks between them to rebuild the coalition fighting against the Iran-aligned Houthi movement that expelled Hadi's government from the capital Sanaa five years ago.