Gavin Williamson was sacked as defence secretary by Theresa May
Theresa May also reportedly blocked Mr Williamson from sending British troops "to invade Africa".
Mr Williamson told Sky's defence and security correspondent Alistair Bunkall: "It is absolutely crazy, on both counts.
"Of course no one has ever suggested it. Classic PM/Sedwill smear because they don't have any evidence."
According to various reports citing government sources, Mr Williamson had told fellow Tories Mrs May's health makes her unfit to be prime minister.
A Downing Street source said the briefings did not come from No 10.
Mr Williamson was sacked by the prime minister after an inquiry found there was "compelling evidence" he was responsible for the unauthorised disclosure of information from the National Security Council to a newspaper.
He has accused Theresa May and her cabinet secretary Sir Mark Sedwill of badly mishandling the inquiry and called for a probe into it.
However, it is reported that Mrs May was already considering Mr Williamson's position because of his alleged desire to send British troops to Africa.
The Sunday Times reported it had seen a National Security document suggesting Mr Williamson drew up plans for troops to be sent to "at least five African countries, including Zimbabwe, Nigeria, Kenya and Egypt."
According to the paper, when Mrs May refused to allow him to send Royal Navy ships to the South China Sea, Mr Williamson wrote "f*** the prime minister" on paperwork.
A source from the Ministry of Defence is quoted as saying: "He wanted to invade Africa."
Mr Williamson and those close to him deny the newspaper reports and believe he is the victim of a "witch hunt".
Fellow Tory MP Peter Bone told Sky News: "It is just not the way we do justice in this country.
"Everyone is innocent until proved guilty but it seems the former defence secretary has been found guilty by a secret court on evidence he hasn't seen.
"How can that possibly be right?"
On the briefing against Mr Williamson, he agreed it appeared "a perfect piece of spinning or black arts" by Downing Street.
Mr Williamson has called for Mrs May to release details of the Huawei leak investigation "so everyone can make a judgement".
He has previously said he would welcome a criminal investigation as a way to clear his name, but the Metropolitan Police said the disclosure did not amount to a criminal offence.
Comment: Also see: