The forces, composed of fighters mainly from Misrata, say they are exercising caution to limit casualties to civilians who may still be trapped with Islamic State fighters.
Islamic State took full control of Sirte, a city of some 80,000 inhabitants, in early 2015. Its loss would leave the jihadist group without any territorial control in Libya.
The Libyan forces have the remaining militants in Sirte surrounded in part of the Ghiza Bahriya neighbourhood. The civilians who left on Sunday were Libyans and mainly women and children, the media office of the Misrata-led operation said.