I’m quite new to building custom mechanical keyboards and now I think I know at least enough to build my first keyboard. I had a really hard time finding all required information which is cluttered around different forums, youtube, reddit etc.
Even more of a pain was getting all the parts together. I just discovered the thread which lists all the vendors. That’s really great! And now I’m just thinking about creating a website which basically lists most parts which are available (with some kind of nice search) on the market and directly displays a list with vendors who sell them and actually have them in stock. This would have helped me a lot when searching for switches, pcb’s, cases etc.
Would you guys see value in such a thing? Or would it be a waste of time? Maybe there is something already? I saw switches.mx, which is great. Think of a more comprehensive version of it, not only limited to switches.
That’s the best part! I get it though. On the bright side, the more time you put into the hobby the less this will be an issue. I ended up just buying spares of parts/tools I knew I’d need for future builds.
Neat! Cool idea. I took a swing at this but ran into a few roadblocks. Most notably, was how to data mine only “in stock” items when a vendor’s inventory is rendered server side? Ugh, Shopify is getting tricky. Let us know if we can help.
Yep it certainly would not be an easy thing to do but I still think the amount of vendors is still manageable.
What I have in my head is a place where it’s easy to find for example a specific switch which then lists all vendors including availability (that’s what matters to me). So that I can pretty certain if sth is not available by looking just at this site instead of searching through tons of stores.
Of course managing this manually would be hell. I’m thinking to automate getting the data and availability on a regular basis within short intervals. I’m having a deep web and software development background so I would imagine something like this is achievable.
Thanks for your feedback already. Excited for a few more opinions. I really dont want newcomers to struggle like I did with spending hours on finding a few parts.
Certainly its also a nice part of discovering new parts, but I also think that you also might miss out on things. At least I found it quite daunting, especially when you do not have a lot of time to spare
I’m curious, how did you try to mine the pages? I have a few ideas in mind how to achieve that.
Oh, I wrote a small R program leveraging the rvest library. Did the job, but normalizing the data schema was ridiculously over complicated and error prone. I’m sure I was doing something wrong. I then went and got a subscription with ParseHub which was well worth the money, but still lacked the bot controls Shopify vendors are implementing these days. For example, when NK_ restocked the Entry Edition this last go around, the URL was new and totally different than the one used for Google indexing. This tactic prevents bots from crawling indexed webpages automatically at designated times. Forcing the bot to reindex the entire NK_ site before becoming aware of the new page, thus nullifying it’s ability to work faster than it’s human counterpart.
Feel free to PM me if you need any further tech details. Good luck!
