I am sorry you have been dealing with this with your cat, I know this must be upsetting and you likely feel on edge a lot of the time.
The issue that you are experiencing is actually not as uncommon as you may think and is a reason that many owners will seek assistance. This is particularly true of cats that are rescues or were feral or semi-feral as they often did not receive adequate socialisation and integration with humans when they were in their key developmental stages in their first couple of months of life. However, this sort of aggression can be seen in any cat.
Our aim is to minimise and eventually eliminate these events.
We want the home to be a calm and predictable place. He should have a daily routine (e.g. fed at x o'clock, litter tray gets changed at y o'clock) so he knows what is coming.
Be sure all in the home are reading his body language. At the first sense of an attack coming on, it is best the person remove themself from the situation. Watch for things like: Holding ears down, freezing, tail wagging, becoming vocal, hairs raised etc. Some cats give lots of clues, while others do not.
If he seems like he is about to attack it is best to say 'No' firmly and, of course, leave and close the door. It is best not to create too much anxiety or to tell him off, as this will heighten his stress levels.
It is useful to build his confidence and sense of security so he feels less inclined to lash out in general. This is done by enriching his environment with interactive toys such as laser pointers and battery operated mice, as well as food puzzles, cat mazes and cat trees. Some cats also enjoy catnip toys.
Playing together can also really help to build bonds and trust.
Another consideration would be stress-reducing calming supplements in his food each day (e.g. Nutracalm or Yucalm) as well as a Feliway plug-in.
I would definitely consider getting a routine vet check for him if he has not had one recently. This would just be a nose to tail check to ensure we are not missing a source of discomfort or discontent such as a bad tooth, chronic cystitis or any joint pain.
For cats who we are not seeing improvements with, it is worth discussing the possibility of some prescription anxiolytic medicine with your vet. Medicines like Fluoxetine or Sertraline can be prescribed and often make a real difference.
I'd also encourage you to contact a local feline behaviourist; your vet should have a good local contact. They can come to assess him in his home, providing a more tailored plan.