Pakistani authorities "must have had" a hand in the deadly Mumbai siege, India's prime minister said Tuesday, stopping just short of directly accusing Islamabad of aiding the gunmen. Prime Minister Manmohan Singh repeated India's allegations that the attack was carried out by the Pakistan-based militant group Lashkar-e-Taiba. But in his most forceful speech since the November attacks, he also said "there is enough evidence to show that, given the sophistication and military precision of the attack, it must have had the support of some official agencies in Pakistan." The careful phrasing seemed aimed at keeping tensions between the bitter rivals at a low burn, and reflects the widespread belief that there are multiple power centers in Pakistan. The overwhelmingly Muslim nation's civilian government is relatively new and weak, while its military-controlled spy agency, the Inter-Services Intelligence, is thought to have a high degree of independence. Singh did not directly name any Pakistani officials, but India has blamed the ISI for being involved in attacks against India in recent years. Singh also charged Pakistan with "whipping up war hysteria," and criticized what he called their reluctance to crack down on militants operating on their territory. The speech came one day after India handed Pakistan evidence that New Delhi says proves the attacks were plotted in Pakistan. The dossier included details from the interrogation of the lone surviving gunman, recovered weapons, and intercepted communications with the suspected handlers back in Pakistan. The investigation into the attack, which left 164 dead, showed the 10 gunmen could not have been working on their own, Singh said. "Unfortunately, we cannot choose our neighbors," Singh said. "Some countries like Pakistan have in the past encouraged and given sanctuary to terrorists and other forces who are antagonistic to India." He spoke before a meeting with senior officials from around the country to discuss security concerns in the latest in a recent ratcheting up of Indian rhetoric. In response, Pakistan Information Minister Sherry Rehman called for an end to "the blame game," and said the two nations needed to work together to fight terror. "Scoring points like this will only move us further away from focusing on the very real and present danger of regional and global terrorism," she said in a statement. "It is our fond resolve to insure that non-state actors to do not use Pakistan's soil to launch terrorist attack any where in the world." Calls for war in India have been largely muted with even conservative opposition politicians, who endorse a hard line toward Pakistan, adopting a fairly conciliatory approach. While Pakistan's own rhetoric has been fairly quiet in recent days, it has also moved some of its soldiers toward the Indian border and away from the Afghan border, which Islamabad is battling militants. India, which is largely Hindu, and Pakistan have fought three wars against each other since they gained independence in 1947. On Tuesday, Singh was highly critical of how Pakistan has handled the investigations into recent attacks, indicating Pakistan has been unwilling - or perhaps unable - to crack down on terrorists operating on its soil. "The more fragile a government, the more it tends to act in an irresponsible fashion," he said. On Monday, Foreign Secretary Shivshankar Menon called for Pakistan to hand over any suspects to New Delhi to be brought "to Indian justice." Pakistan has said it would try any suspects in its own courts. But Singh also criticized India's own security and intelligence operations, which have been excoriated as ineffective and threadbare since the attacks. He called for better maritime security, improved intelligence sharing and stronger local police forces. Analysts say the speech was at once intended for several different audiences: the Indian electorate, who will be heading to the polls in the coming months and have in the past supported confrontations with Pakistan; leaders in Islamabad, who India wants to crack down on the terror network it says operates across the border; and the international community, which New Delhi hopes will help pressure Pakistan into action. "The idea is to make it more than an Indo-Pak issue," said Mahesh Rangarajan, a prominent political expert in New Delhi.