It works fine that way. No need to fool with spacers and shims.
If you want to leave the scrub brake for looks that is cool, but seriously considering adding a better brake to it. Scrub brakes are mostly useless in my opinion. Wet tire = no brakes.
Clutch brakes are simply to install, cheap, and will LOCK up the rear wheel easy.