Yet another interesting topic has arisen this week from the MLB LED Scoreboard project – delving into the almost 30-year-old BDF 2.2 font specification from 1993 to figure out how to add Unicode A4D8 (ꓘ) support for all fonts supported on the scoreboard.
As mentioned in my previous post, a very generous community member merged in a massive refactor of the entire project after MLB (not-so-) suddenly deprecated the old data source the scoreboard previously consumed. Because hardware is expensive and hard to source post-pandemic, the new version of the scoreboard was built with only a handful of matrix resolutions in mind, namely 32x32 and 64x32. (Shameless plug, that’s what RGBMatrixEmulator was designed to solve.)
It’s important to note that the different resolutions supported all can potentially use different fonts, as you would expect. It’s quite hard to fit lots of rows filled with 7x13 rectangular characters on a low-resolution matrix like 32x32, so you might choose to pick a smaller font like 4x6. Luckily for us, the matrix driver library includes plenty of font choices for us to use right out of the box. It’s as simple as reaching into the submodules/fonts directory and grabbing what you need.
Background on Baseball – Why We Need This
If you’re familiar with baseball symbolism, it does have one strange symbol that’s not really found anywhere else. When striking out, normally the play is scored as a K, allegedly because S was taken for “sacrifice”. There’s one wrinkle – when striking out on a called 3rd strike (so called out without swinging or “striking out looking”), the play is scored as a backwards K or ꓘ. It’s usually written by hand, but here this character is Unicode A4D8, and it’s close enough to an actual backwards K that we co-opt it for our purposes
As you can imagine this is not very uncommon as batters strike out plenty of times, often with multiple strikeouts per team per game.
Unbeknownst to me, the new refactor included a couple of patched fonts that were used by the smaller supported resolutions, and the play-by-play renderer now attempted to render called 3rd strikes with this character. Anything other than the patched fonts simply had no way to render this character, so they either displayed a generic missing character from the font or nothing at all in its place.
BDF Fonts
In order to fix the issue, I needed to dive into the BDF 2.2 specification.
This is an old bitmap font format from 1993, so, while old, it works great for the scoreboard’s purposes as it’s a mature format and very straightforward. Given the age of the format (and the creation date comment fields from the fonts I patched), it’s really no surprise they didn’t support this character. I love the fact that the specification mentions the fonts are typically distributed on tape.
Patching a BDF Font
BDF font files are sort of like a markup language in that they have open/close tags for lots of things and some “self-closing” tags for others. They’re also human-readable for the most part, which makes them easy to work with manually (lucky us!). This last part is great because the age of the format means the tooling is old, missing, nonexistent, or incompatible with current OSes (that I’ve seen).
Encapsulating the font are STARTFONT/ENDFONT tags. Within those tags are essentially two sections, the first of which is a “metadata” section which tells the font the name of the creator, number of characters, global size of characters, and other properties (with properties being enclosed by their own STARTPROPERTIES/ENDPROPERTIES). For our purposes the global properties are fine, so we will gloss over them. The second is a list of characters, again enclosed with their own STARTCHAR/ENDCHAR tags. Within the character demarcations are local properties for a glyph (overrides to the global metadata) and the bitmap data for the font.
STARTFONT 2.1
FONT -Raccoon-Fixed4x6-Medium-R-Normal--6-60-75-75-P-40-ISO10646-1
SIZE 6 75 75
FONTBOUNDINGBOX 4 6 0 -1
STARTPROPERTIES 25
FONT_NAME "Fixed4x6"
. . .
ENDPROPERTIES
CHARS 206
STARTCHAR space
. . .
ENDCHAR
ENDFONT
A couple of things I didn’t call out explicitly before:
- The
FONTBOUNDINGBOXtells the font what the global bounding box for each glyph is, so this one is 4x6 with 0 X offset and -1 Y offset. Pretty much what you’d expect for a 4x6 font. - The font name is in the properties node, but it’s not incredibly important other than for helping the developer know which font they’re working with.
CHARStells the font how many glyphs there are. It’ll need to be adjusted later.
Anatomy of a Glyph
Let’s hone in on a glyph to figure out how the font gets rendered. We’ll start with “K”, since that’s also the baseline for creating our own backwards K glyph by horizontally mirroring it.
STARTCHAR K
ENCODING 75
SWIDTH 1000 0
DWIDTH 4 0
BBX 3 5 0 0
BITMAP
A0
A0
C0
A0
A0
ENDCHAR
Notice the BBX override from the FONTBOUNDINGBOX above as well as the width overrides from the global font properties. Not particularly useful here, just a good callout. The first tag tells the font to start character K, and the ENCODING is the character code for the font – akin to ord("K") in Python or similar languages.
The BITMAP block is the meat of the glyph, and as we can see it’s a literal bitmap encoded in hex. Conversion to binary looks like:
10100000
10100000
11000000
10100000
10100000
And in pixel form:
A little wide, but the font trims (or pads, if there wasn’t enough data) that for us.
Mirroring a Bitmap
So the next question is… How do we mirror the bitmap? It seems like a simple question but it is deceptive. Consider the following approaches that I tried:
1. Reverse the nibbles, lookup the reversed nibble in a table:
I created a reverse lookup table for individual nibbles (4 bits) in hex could quickly be looked up, such as 1 -> 8, 2 -> 4, 3 -> 7, etc.
Flip the nibble:
A0 -> 0A
Reverse the bits using lookup table:
0A -> 05
This truly mirrors the bitmap, but isn’t actually what we want because we also mirror the padding. No good! This means the font renders a bunch of nothing because the first 4 columns of the glyph are padding bits.
Result:
00000101
00000101
00000011
00000101
00000101
2. Don’t reverse the nibbles, lookup each nibble in the table:
Reverse the bits using lookup table:
A0 -> 50
This works, with a small problem for this font and a much larger one for wider fonts. For this font, there’s a bit of extra noticeable padding:
01010000
01010000
00110000
01010000
01010000
Passable, but not great. However, larger fonts are INCREDIBLY wonky, such as 5x7. The problem with this font size is that its bitmap isn’t nibble-aligned, meaning unlike 4x6, we care about 5 contiguous bits, and not 4. Therefore, if we don’t flip the nibbles, we can drop or add data when we don’t mean to:
Reverse the bits using lookup table:
F8 -> F1
We dropped a bit:
F 8 -> F 1
1111 | 1000 -> 1111 | 0001
Font sees 11110 instead of 11111
The same thing happens even more noticeably with fonts 9 pixels wide and higher, as they cross 3 nibble boundaries:
F880 -> F110
F 8 8 0 -> F 1 1 0
1111 | 1000 | 1000 | 0000 -> 1111 | 0001 | 0001 | 0000
Font sees 111100010001 instead of 000100011111
As you can imagine, this gets worse and worse as the font grows wider until this method no longer works.
3. Reverse the order of nibbles, look up reversed nibbles, logical AND all bits, bitshift left by minimum left 0 padding
An attempt to combine the two approaches was to perform the actual lookup and then shift the whole bitmap left (literally bitshift each byte) until there was no padding. We’ll go back to 5x7 as a simpler example. Let’s build a checkered box:
A8
50
A8
10101000
01010000
10101000
When reversed, this should be the same as the forward version.
Flip the nibbles:
A8 -> 8A
50 -> 05
A8 -> 8A
Reverse the bits using lookup table:
8A -> 15
05 -> 0A
8A -> 15
AND the bytes:
15 & 0A & 15 -> 31
00011111
Bitshift left till ANDed value has no 0 pad (3 times):
15 -> 2A -> 54 -> A8
0A -> 14 -> 28 -> 50
15 -> 2A -> 54 -> A8
Perfection! Or is it? There’s actually two subtle problems.
Firstly, some fonts do want a left pad on the glyph, and this approach will wipe that out indiscriminately. Secondly, this looks horrible on some bolded fonts and all italic fonts. Another solution is needed to do this programmatically, but at this point, this seems good enough.
The Final Solution
In the end, I went with approach #3 as it gave the most consistent results on the approximately 20 fonts I applied the patch to. For the fonts that didn’t work, good ol’ fashioned manual effort did the trick nicely. It wasn’t too much of an effort to fix the badly formed ones to the best of my ability, and if they aren’t right, I’m sure we’ll accept a PR down the line.
With mirrored bitmaps in hand, it’s time to actually patch the character in. First, for each font I copied over the K glyph and changed the name to uniA4D8 (standard across the fonts I found) and encoding to 42200 (A4D8 in decimal notation). Next, just copy the bitmap over from the K glyph, and we’re good to go:
STARTCHAR uniA4D8
ENCODING 42200
SWIDTH 1000 0
DWIDTH 4 0
BBX 3 5 0 0
BITMAP
A0
A0
60
A0
A0
ENDCHAR
Finally, I needed to increment the CHARS total for the font. I tested on an emulator and I don’t think anything would go wrong if this was not correct, but it never hurts.
Once that was done, it was time to make some commits.
Commit Strategy
The font files were huge, so in order to (try) to get an easy diff on them I attempted to first apply the unpatched fonts to a new directory fonts/patched so that new commits on top of the old one should just be the parts that I edited. Unfortunately, it wasn’t quite enough and the diff failed to render both locally and in GitHub. The total change was almost 1 million LoC, by far the largest PR change I’ve ever intentionally committed:
With all said and done, here’s an example of all the patched fonts running!
That’s all for now. Thanks for reading!