The Leyden Jar, a tentative PCB replacement for the Brand New Model F Keyboards

Rico · April 9, 2023, 9:51am

Hello all!

Long time no see in this thread, did you thought it was a dead project ?
Far from it
It fact I continued slowly working on it, mostly on the firmware development side.

As I had little visibility on how this controller would work on other boards than mine, I needed someone with other variations of capacitive keyboards to help me.
And after discussing with @Ellipse he kindly accepted to give me some help on that task.

After receiving the Leyden Jar PCB he decided to solder it to one of it’s Brand New Beamspring Round 2 Full Size keyboards, our goal was then to try to make the Leyden Jar controller PCB work with this hardware.

Here are some shots of the Leyden Jar PCB assembly on the Beamspring Full Size, courtesy of @Ellipse.

As I didn’t have the board on hand, making the controller work on it was a complicated matter.

First I had to figure out how the switch matrix was organized and did a complete visual reverse engineeing of the beamspring PCB plate based on images provided by @Ellipse.

And also had to find the correct values for the calibration process.

After quite a few back and forth exchanges and firmware revisions we finally managed to do it
There is now at @Ellipse workspace a Brand New Beamspring Full Size Round 2 keyboard with a Leyden Jar controller working nicely with solenoid and leds support, than you very much for your great support Joe!

Now is this controller perfect ?
Of course not, even if the most important feature, the capacitive sensing technology, is working very reliably.
So it is now time to make a revision 2 of this PCB and try to make it perfect.
The plan is also to add two additional columns, if possible, for future keyboards with a lot of keys like 122 keys battleships.

See ya people!

Halfling_Barista · April 10, 2023, 5:10pm

This is interesting to me, as I’m becoming more interested recently in the new Model Fs after a surface-level look a while back. Thanks for the update.

Rico · April 10, 2023, 5:47pm

And with recent Joe announcment for selling IBM Model M like keyboards with Model F technology, this makes the appeal even stronger

Halfling_Barista · April 11, 2023, 4:04pm

Whaaaa?

Rico · April 11, 2023, 5:04pm

Yes, look at model f website

Rico · April 30, 2023, 10:46pm

I have big news people

The Leyden Jar design and software is now officially open source under the GPLv3 licence!

The Github repository is available here.

Have a good dowload and reading session !

jtl · May 15, 2023, 1:00pm

Amazing work.

As a prospective owner of one of the newer Model F keyboards and someone whose experimented with embedded development from time to time I have a few questions about this project.

How’s the consistency of the matrix scanning and USB report frequency compared to the xwhatsit controller? As I understand with other keyboards, not all MCUs have consistent performance when running QMK with stricter timing requirements.
The xwhatsit controller has a signal level monitoring utility to diagnose capacitive sensing. Does analogous functionality exist for the Leyden Jar, either as a utility or existing in concept but needing a utility written for it?

Best regards

JTL

Rico · May 15, 2023, 3:46pm

Hello @jtl !

The matrix scanning is entirely done using PIO hardware of the RP2040, it is very consistent in term of timing and gives very stable results during use.
The matrix scanning time is also very short (at 640us) and as the USB polling rate is set as 1ms you can expect around 2ms latency from the keyboard to the PC.

I already wrote one although it is very basic and not user friendly at all, it is just enough for me to look at useful data when troubleshooting.
It would need a lot of time to make a monitoring tool like for the xwhatsit (make by Pandrew), not a priority for now but maybe in the future ?

Best Regards,

Rico

jtl · July 8, 2023, 8:19am

Excellent. That’s one less problem to worry about.

I don’t know when I’ll have the time to potentially work on it, but I wouldn’t mind seeing what you have so far as a starting point.

Best regards

JTL

Rico · July 9, 2023, 11:16am

I may in the future add the Windows executable version of the tool in the Github repository.
In the Leyden Jar firmware source code there is some code to communicate with the tool, could be easily extended in the future if needs be (the communication protocol runs on top of the VIA/VIAL protocol, both are usable at the same time).

Rico · August 24, 2023, 5:25pm

I just received the Rev3 Leyden Jar PCBs from JLCPCB, and also PS2 AT to USB daughterboards.

I’ll have to find time to solder this new controller to my F77 in order to check that everything works and tune the firmware code.

Rico · September 9, 2023, 9:49am

Hello everyone,

A little bit of news about the Leyden Jar Revision 3 PCB.
And before going to explain the validation status of this new PCB maybe explain a bit why another revision of the Leyden Jar PCB (after all the Revision 1 is already working nicely) and a bit of news on the Revision 1 itself.

Revision 1 has been up until now the center of interest of both @Ellipse an I to fully validate the basics of the design.
For the validation this PCB has been tested and proved to work reliably on several keyboards:

The Brand New Model F F77, firmware development and testing/validation done by me.
The Model F Labs Beam Spring 104, firmware development done by me, testing/validation done by @Ellipse.
The Model F Labs F104, firmware development done by me, testing/validation done by @Ellipse.

Parts of the validation process have been:

Check that keys are all working.
Check correct solenoid support.
Check correct status LED support.
Check correct VIAL support.

For people asking were is the Revision 2 ?
Well this revision is purely an internal development milestone in-between Rev 1 and Rev 3, so no need to advertise much on it.

As for why a Revision 3, both @Ellipse and I wanted to refine the design a little bit and add a few extra features:

Remove the external EEPROM chip (QMK has now RP2040 support for EEPROM emulation into flash storage).
Drive the solenoid with dedicated IO pins (no need to make QMK tricks to make solenoid work).
Add support for potential two additional columns, @Ellipse future BS122 and F122 keyboards will use those additional columns.
Add future support for a PS2 AT to USB converter board(firmware development not started yet).
Add another 6 pins header for both SWD debugging and future I2C device connection of any kind that may suit the user if wanted (you’ll have to work on your own firmware code for that).

Now is the Revision 3 working ?
I recently installed it on my own F77 and everything worked first time: key activations, solenoid support and VIAL integration.
While the support of the LED status and the two additional columns still need to be validated, I am stocked to see that most of the design is working flawlessly

It is now time to work on the firmware code again, add support of the two additional columns and make the controller work on F122 and BS122 boards.

Rico · September 10, 2023, 9:04am

Debugging setup ready !

The board on the top left is a standard Raspberry Pi Pico controller, configured as an SDW probe with picoprobe software installed on it.
The board on the bottom right is the Leyden Jar PCB Rev 3.
The SWD probe allows to:

Load and run an executable on the Leyden Jar PCB.
Reset the Leyden Jar PCB.
Step in/out firmware code.
Inspect variable and memory values.

For the best experience I compile my code in debug mode with no optimizations and with debugging symbols.

As you see the location of the SWD connector has changed location (now on the top left of the PCB) and has been improved in the Revision 3:

There is now a ground pin (where the white wire is connected) and it is also merged with an I2C device extension port.
The 3 pins on the top of the connector are for ground and SWD pins, the 3 pins on the bottom are for 3.3V output and I2C protocol pins.

Without SWD debugging support it would have taken me ages to have a working Leyden Jar firmware working.
Not only it helped me tune the low level code on my test application, but also has been very usefull during QMK firmware development (yes you can also debug your QMK firmware with it).

nathana · September 10, 2023, 10:43pm

I am quite excited that there will finally be a capsense controller board based on something more powerful than an AVR micro. Great work…I can’t imagine how much time + effort + knowledge + skill goes into something like this.

Will you be leaving the SWD header on the final production boards? I think it would be a good idea to do so, even if it adds a little in cost. There are times when it would have been nice to have easy access to JTAG or ISP on the AVR Xwhatsit boards. It looks like there is space for one to stay in place, so why not keep it there and have it populated with pins installed by the factory on all shipping boards?

I’ve also been very curious to know what the story is / what your plans are for the PS/2 daughterboard. Maybe you aren’t even sure yet what strategy you will pursue? The dilemma I foresee is that QMK apparently has some limited PS/2 host-mode capabilities, but has zero support for / implementation of PS/2 device-mode on any platform. When it comes to being a client & presenting itself to a host, QMK only knows how to do USB (at least at this point). There is the ps2avr firmware project, but 1) use of this or a similar project would require users to re-flash their controller with non-QMK firmware everytime they wanted to be in “PS/2 mode”, and then re-flash back to QMK to return to USB mode, which is…not ideal, and 2) naturally ps2avr itself is specific to the AVR platform, so you’d have to basically rewrite from scratch anyway in order to do something similar on RP2040.

So are you actually planning to implement PS/2 device firmware from scratch yourself?

Rico · September 11, 2023, 6:05am

Hello @nathana !

Nice to see you in this forum
Heard you made good progress on putting XWhatsit firmware into official QMK repository, congratulations !

The project has been announced in June 2022, but I worked on it far before this date, so at least 1 year and a half of work (part time). This is indeed a lot of work …

The cost may be an issue for @Ellipse as for any production process every dollar count.
As an examble the two reset buttons (1$ each) are not soldered by default for cost reason.
Maybe ask him in person to have this header installed by default ?

Just like the buttons, the end user will probably never see the need to use this feature.
And in the case they do for experimentation, the soldering job is not out of reach of someone that knows how to use a soldering iron: just unscrew the controller from the plate to give you more fiddle room and voila !

I can confirm that there is enough place for the header to be soldered while the controller is installed on a board (tested on my F77).
Although to plug some wires you may need to unscrew the controller and move it away from the plate to give you a little room for this operation.

Yes you are totally right, I have absolutely no idea on how this could be done for the moment

My strategy would be first to see how QMK firmware reacts when the firmare is running with no USB connectivity. Will it hang or with it execute its logic while not sending data to a non existent USB bus ?

Then look at QMK source code to find ways to hook into the system for the PS2 logic.
Will we need a custom QMK source code for that or can it be done just in the keyboard specific code ?

If none of the attempts are succesfull this would be the path to take, although this is a huge task to to and we could question the validity of the approach.
Ideally we’d want the PS2 connected keyboard to use most of the QMK logic to benefit from the layers, macros, and other nice features already available into QMK.
You’d want to configure your keyboard with VIAL while the keyboard is connected to USB, then when it is done plug it using PS2.

nathana · September 11, 2023, 6:29am

Actually no, I’m not working on that. My project is to make a stable and full-featured default firmware for Xwhatsit-based Model F LLC boards, including Vial support and all the trimmings (that I can manage to fit into 32K of ROM and 1K of RAM; heh!). Others have been working on upstreaming the pandrew capsense code. In my case, at least for the moment, the goals of getting pandrew’s code into the official QMK repository run counter to my short-term goals, because the current ‘master’ QMK branch is too bloated for me to be able to squeeze it into the constraints of the poor ATmega32U2 micro with pandrew + Vial + extra features like Mousekeys and such. Once the older QMK branch I am working off of is able to support all of the various Model F Labs keyboard models, then I’ll start looking at what can be done to move to more current QMK and Vial versions.

But working on this has definitely made me cheer your project on, because the AVR platform’s limits are really starting to show themselves!

I personally think this is a huge mistake, even bigger than leaving off the SWD header. I have needed to open up my New F77 keyboard many times into order to short the “PROG” pads in order to engage AVR USB-DFU bootloader, when a bad firmware has “soft-bricked” my keyboard. It can take me 2-3 attempts sometimes, and whenever I have to do this, I curse the lack of a simple button!

Having an actual button instead of exposed pads is a huge user experience upgrade for when a firmware update goes wrong. And similarly, having JTAG/ISP/SWD headers available would make rescuing a controller from more of a “hard-brick” scenario actually possible, whereas for 99% of customers, if the bootloader gets corrupt or overwritten, they can’t honestly be expected to break out the soldering iron. Their options at that point are limited to sending the keyboard for repairs to somebody who knows what they’re doing, which is a HUGE inconvenience, not to mention waste of shipping $, risk of damage to keyboard in transport, and so on.

Extremely short-sighted in my opinion to shave off a few pennies…the trade-off is nowhere worth it. Perhaps I will talk to Ellipse about this. Perhaps at the very least he might consider to stock some Leyden Jar controllers that have both reset buttons and SWD headers on them for extra $, and maybe even offer an “upgrade” option to one of these at the time of keyboard purchase, so it already comes pre-installed in the keyboard. At least that would be better than not having the option at all!

Wish I had some ideas for you on the PS/2 front! Sadly I haven’t studied the USB side of QMK intimately enough to be able to tell you how easy or difficult it will be to plug-in a different device protocol mode or not.

Rico · September 11, 2023, 7:12am

Arg, my mistake …

Looks like the limited ATmega32U2 is giving more difficulties that I was expecting.
It is great that you are maintaining this part for the many users that have a Model F keyboard with an XWhatsit controller installed in it.

I have 5 Revision 3 PCBs at home, much more than I’ll ever need.
I can send you one if you want to play with in the future.

Coming from an industry that makes hardware it is not surprising to me to see an effort to lower the manufacturing cost.
For example 2$ for 2 buttons on a board that may cost 10$ to make is quite significant.
Usually companies are chasing cents in order to optimize their production cost.

Normally if the firmware is well written you should not have to touch one of those buttons.
There is the bootmagic feature activated on the Leyden Jar, just keep ESC pressed when plugging in the keyboard and you go into the bootloader.
And as long as you flash a firmware that is known to work on the board there should be no issue.
Moreover the RP2040 is virtually unbrickable as the bootloader code is in ROM.

But yes, if you play with your own firmware development then having buttons and SWD header can be of great use. This what tinkerers would need, now would normal end users need this (meaning you’d at least have to disassemble the board)?

Does not hurt to ask indeed.
And having more features is allays good, even in this case for more advanded users.

nathana · September 11, 2023, 7:39am

Very generous offer! I may take you up on that!

Of course, I understand that. And I could understand it in this case if we were talking about a $100-200 product. But in this case we are talking about low-production-run, boutique keyboards that have a sticker price of several hundred dollars. If I’m already paying USD$400-500 for a board, do you think I’m going to blink at a few more dollars, especially if it more solidly guarantees future serviceability of such an expensive board? If I am paying hundreds of dollars for a board like this, I want it to last AND be as easy to service as possible, both in terms of hardware and software! Let’s say 2 buttons costs $2 extra per controller board. So tack $10 extra onto the price of the whole product to cover that extra cost, and call it a day.

Since virtually all of the firmware development for these boards is community-driven & largely by hobbyists, I can expect there to potentially be more problems on average than perhaps there might be if a well-staffed (and well-lawyered!) company was backing the product and wanted to avoid having to swallow the cost of paying for board repairs out of their own pocket in the event that they release a buggy firmware update.

Plus, since it is all open-source, there are tons of different forks of the stock firmware that different developers are releasing for community members to try. (Such as my Vial-enabled firmwares, for example!) And regular (non-developer) users might decide that they want to try to use one of those firmwares instead of the one that their board shipped with. But…oops! Turns out the one that they decided to try wasn’t tested well by the developer that released it, and it is so buggy that now they can’t kick their keyboard back into firmware update mode to undo the mess and get it back to working the way it was before!

Also, there are many of us tinkerers and more advanced users who may know their way around software development, but are not so skilled when it comes to hardware. My own soldering “skills” are infamous(ly bad), for example!

Interesting. Bootmagic is disabled in pandrew QMK firmwares because the capsense code runs a self-calibration routine at start-up, and that routine assumes all keys are unpressed. So you can’t hold down a key when plugging in the keyboard or the calibration will fail.

(EDIT to add: with ATmega32’s 1K SRAM, it is very very easy to unintentionally overflow the stack. If that happens during QMK initialization, then all bets are off. Even if Bootmagic is enabled in the build and normally works, in a buggy firmware build that crashes the keyboard with a stack overflow, it may not even be possible to trigger Bootmagic before the crash occurs.)

Arkku’s enhanced calibration code that he wrote for his own firmware will store the results of the last calibration in EEPROM so that it doesn’t need to re-run calibration every single time it boots. Are you doing something similar for Leyden Jar? Unfortunately, my Vial firmware is already bursting at the seams, and changing out the pandrew capsense code for the Arkku code puts the firmware over-budget for the poor ATmega!

Rico · September 11, 2023, 11:57am

It could be a good solution, but it is up to @Ellipse to decide that

I didn’t thought about that, it makes sense now.

There is a new calibrating routine on the Leyden Jar that allows keys to be pressed and still have accurate results.
To be more technical I use the median value of the key levels, that is I’m sorting all levels and I only take the value in the middle of the pack. So if you press keys during the calibration phase this will have no effect as their value will be on the sides of the sorted values and ignored.
Of course if a psychopath presses all keys during the calibration sequence it will not work

I don’t store anything in EEPROM/Flash.
The calibrating routine is very fast(at most 200 ms) and allows you to press keys during the process, this is an almost transparent process for the user.

nathana · September 14, 2023, 8:07am

As if to prove my point, as of this evening, I now have an Xwhatsit controller that is totally bricked.

It took a partial flash of a new firmware image, and now the AVR USB bootloader refuses to write to the flash anymore. Whatever (bad) application code is in the flash, is the only thing it can execute now. Interestingly, the USB bootloader can still write to EEPROM, just not to flash.

I have spent several hours researching this, and the symptoms I am experiencing line up with what would be expected if the application lock fuse was blown. This should NOT be possible to do without using ISP/JTAG. But according to a handful of threads I found on AVR Freaks forum (and even QMK Toolbox Github issues), this has apparently been known to spontaneously happen on occasion, for unknown reasons.

The only fix for this, is to wire up an ISP programmer to the MCU, wipe the flash, re-set the fuses, and then re-flash the USB bootloader.

But as has already been established, there is no ISP header on the Xwhatsit board design. And in the case of this board, trying to add ISP connections means soldering to 6 pins on a tiny surface-mount QFN package.

I am so angry right now, I cannot think straight. It will likely be weeks before I can get around to even attempting to fix this.

As the English proverb goes, leaving off such headers is “penny-wise, but pound-foolish”.