It is time to take drastic action against the Filipino armed men after their intrusion turned deadly in Lahad Datu and Semporna, Sabah, said former prime minister Tun Dr Mahathir Mohamad.

"There is no way out other than launching a counter-attack to eliminate the Sulu group.

"Although many of them will be killed, this cannot be avoided because they had attacked Sabah, and not the other way round," he told reporters after addressing a gathering organised by the Perak branch of Perkasa, a Malay right group, here today.

However, Mahathir cautioned the authorities to be prepared for retaliations from remnants of the group after they were decimated or got rid of Sabah.

He said the Royal Malaysian Navy needed to tighten its surveillance on Sabah waters and check on boats from Tawi-Tawi and Jolo islands that could be smuggling in intruders or arms.

"We cannot underestimate them, even though at first we think that their attempt to capture Sabah is a futile act, but they can become guerrillas and launch sporadic attacks in future," he said.

On adverse reaction over the government's decision to allow the police to move in first, instead of the armed forces, Dr Mahathir said the country's security threats were based on whether they were internal or external threats.

"External threats are handled by the armed forces while internal threats are dealt with by the police.

"At first we did not think the intrusion was as an external threat for they were on our shore. However, when it is clear that the invasion is an attack from outside, the military was ordered to move in," he added.

On claims that the government appeared to be soft in dealing with the intruders, he said the government did not want to kill them because they also have families and friends in Sabah.

"That's why we acted cautiously at first. After all, they are Muslims, and we are Muslims. We do not want to be hasty, but they acted in bad faith by gunning down two of our policemen. Now, the stalemate can no longer be resolved through negotiation," he said.

On foreign media reports that intruders made their way to Lahad Datu at an "invitation" of an opposition politician, Dr Mahathir said the politician promised a lot of things for Sabah including giving autonomy if the opposition were to win the next general election.