"This is an illegitimate decision and this so-called referendum has no legal grounds at all. That's the reason why we urge the Russian government not to support those who support separatism in the Ukraine," Yatseniuk told a news conference in Brussels following talks with EU leaders.
"Crimea is was and will be an integral part of Ukraine."
He also urged Russia on Thursday to withdraw its military from Crimea and said the current crisis must be resolved only through peaceful means.
The crisis in Ukraine escalated on Thursday after the parliament in Crimea, which has effectively been seized by Russian forces, voted to join Russia.