Jack Merritt, 25, and Saskia Jones, 23, both former students active in a program on prisoner rehabilitation, were killed on Friday when Usman Khan went on the rampage with kitchen knives at a conference about the program beside London Bridge.
At the vigil, London's mayor, Sadiq Khan, said the British capital had gathered "in a spirit of defiance to say that London will never be cowed or intimidated by terrorism."
"The best way to defeat this hatred is not by turning on one another but is by focusing on the values that bind us to take hope from the heroism of ordinary Londoners and emergency services who ran towards danger."
Friday's attack has become a political issue ahead of the Dec. 12 election. Prime Minister Boris Johnson has promised a tougher stance on the release of prisoners if his Conservatives win a majority, criticizing the opposition Labour Party's leader Jeremy Corbyn as soft on terrorism.
Labour says cuts to policing and social services under a decade of Conservative leadership have made it harder to fight militancy, to track former prisoners and to reintegrate them.
Confronted by bystanders, including a Polish man brandishing a narwhal tusk and others with fire extinguishers, the attacker wearing a fake suicide vest was wrestled to the ground. He was then shot dead by British police.
Johnson and Corbyn both attended the vigil in Guildhall Yard, in the heart of the City of London, held to honor the dead, those injured, the emergency services and the members of the public who intervened.
The attacker had been sentenced to a minimum of eight years in 2012 for offenses including plotting to blow up the London Stock Exchange and British parliament.
He was released a year ago without an assessment from the parole board of whether he was a threat to the public, despite a warning from the sentencing judge that such an assessment should be made.
Repeating a promise to end such 'automatic' early releases, Johnson said during a campaign visit later in the day that for some militants like Khan, de-radicalisation may never be possible:
"There are some cases that appear to be too tough to crack, and, alas, he appears to have been one of them."
Corbyn said Conservative cuts to community policing, probation, mental health, youth and social services could "lead to missed chances to intervene in the lives of people who go on to commit inexcusable acts."
A veteran peace campaigner, Corbyn has also said convicted terrorists should not necessarily serve their full prison terms, and has long said that British foreign policy in the Middle East has played a role in motivating militants.
Amid the political and media rhetoric, the father of one of the dead called on the media and politicians not to use his son to justify division or hatred.
"Don't use my son's death, and his and his colleague's photos - to promote your vile propaganda," David Merritt said in a tweet above images of the front pages of right-wing tabloids, the Daily Mail and Daily Express, both of which described a government plan for a "blitz on freed jihadis."
"Jack stood against everything you stand for - hatred, division, ignorance," he said.
Both the dead were involved in the University of Cambridge's Learning Together program to help educate and rehabilitate prisoners, which was holding the event where the attacker began his rampage.
Islamic State said the attack was carried out by one of its fighters, the group's Amaq news agency reported on Saturday. The group did not provide any evidence.