As Tony Abbott met with Muslim leaders on Monday, respected community figure Dr Jamal Rifi was one of several leaders to lash out at the government's "nonsensical" approach of cancelling passports without any properly resourced community programs to supplement it.

"All this does is keep these individuals in the local community," he said in a letter to the Prime Minister. "[They're] given a badge of honour to parade and thus in turn creating for them a fan base, without the availability for any pathways or programs for re-education."

His comments come amid revelations the federal Attorney-General's department has quietly let its community deradicalisation program lapse.

The Building Community Resilience program, the cornerstone of the department's plan to counter violent extremism, allocated $1.5 million annually since 2011 to community-run programs targeting youths at risk of radicalisation but no funding was allocated in the recent budget.

Dr Rifi, who has known the Sharrouf family for years, is so frustrated by the lack of funds for community deradicalisation programs that he has formulated his own five-page plan, called the Clear Hearts, Clear Minds initiative, to send to Mr Abbott.

He is one of four Muslim leaders that have been threatened and intimidated by radicals in Sydney in the past week.

"When we see a young person become inclined to radical ideas, we need to be able to tap into resources within community," he said. "We have been doing this on our own volition on a smaller scale but it's time to do it on a much bigger scale."

Former counter-terrorism officer Peter Moroney, who spent 18 months monitoring Sharrouf's every move, said cancelling the passports of radicals without any community measures to back it up was like "pulling the pin on a hand grenade and leaving it in a confined space".

Sharrouf, a 33-year-old father of five from Punchbowl, slipped out of the country on his brother's passport last year and has been fighting in Syria and Iraq with terrorist group Islamic State, formerly known as ISIL.

He was isolated and left with untreated schizophrenia for most of his four-year prison term for supplying materials for the Pendennis terrorist plot.

Mr Moroney said the Pendennis members knew they were being monitored. "We raided their houses almost religiously each year and it didn't deter them," he said. "So what's the answer? You don't not do that but it's more than a law enforcement problem. It's a community problem too."

Wissam Haddad, owner of the hardline Al-Risalah prayer centre in Bankstown and a close friend of Sharrouf's, said wannabe jihadists should be allowed to leave the country and revoke their citizenship, a view supported by Dr Rifi.

"You can't put a wild bird in a cage and just hope it's going to be domesticated," he said.

"Khaled was the bird in the cage but he found a way out. It just goes to show how much people are willing to risk just to get out of the country. Cancelling passports is not going to deter anyone from jihad. You can't kill an idea."

The Australian National Imams Council have opposed tough counter-terrorism laws proposed by the Abbott government, saying the "trend towards unilateral aggression and infringement on basic human rights ... will only aggravate the state of global fear and violence".

ASIO has cancelled 69 passports in the past two years.

The Attorney-General's department failed to provide a response to questions about deradicalisation efforts.