Except for Boba U4 (my tolerance for scratchy feel is apparently higher than most) I share your disappointment with these and most silent switches on the market. Silent is such a misnomer anyway, they should be called dampened, or quiet at best. And maybe they really are just not as well made as popular non-dampened switches. I wouldn’t know because I got in to the keyboard nonsense wanting something opposite of MX Greens, the only other mechanical board I have used.
What I’ve learned so far is firstly what Deadeye has already put well. Once you remove the housing collisions, your ear adjusts and all the little ugly sounds start sticking out. The stem slaps against the housing once your finger touches the keycap, the stabilizer has a click you never heard before and you can even hear that lube sounds wet.
The other thing is that the rest of the keyboard has a lot to say about the little ugly sounds, but I’m only scratching the surface of what is to know there. I soldered Midnight Pro on an FR4 plate on my Sonnet when I just got it only to find out that it was a completely different and much worse experience from having them on a PC plate with silicone plate foam. My top-mounted aluminum works well with smooth and thuddy switches but amplifies scratch and ping. The favorite setup is the stack-mount in my sonnet with a POM plate and foam. It’s the closest thing to a library instead of a percussion instrument. With more money and time I would try to find out what makes a good keyboard for quiet switches and I would start by resting the entire PCB on something soft and dense and not using a plate.
The only two switches I completely accept in their stock conditions are Anthracite and Gecko (linear), which are both lubed from the factory.