I believe it's enough to have [Final] keyword set in deployed mode to give a developer a hint that this class should not be extended.
If you want to enforce this behaviour, I would add a check into each method as a first line , something like
if $this.%ClassName(1)'="My.Class" quit $$$ERROR(50000,"don't extend this class")
Since all code will be deployed, developers will not be able to remove this check easily.
You can also try to add a method-generator, I believe when you have a deployed class with method generator it will not be able to compile a subclass without method generator's source (though I'm not sure).