The Battle Royale: benchmarking old computers

▼ On his Youtube channel, Matt Heffernan has a series of 8-bit Battle Royale videos to see which 8-bit computer is fastest. For this, he uses a simple program to calculate the world's lowest resolution mandelbrot set. Still, the Commodore 64 and ZX Spectrum take minutes to do this in BASIC. It's much faster in assembly.

(Aside: I do believe the Z80 CPU has some advantages over the 6502 due to all the arithmetic. I'd expect in a sort test like the one I did some time ago, the 6502 would do better relative to the Z80. I'd probably stick to just the insertion sort, as it's so simple to implement: something like 20 lines of BASIC should do it. The prime calculation test that I used to see which programming language is fastest is even shorter and simpler, but again math-heavy.)

Watching the videos, I started wondering how 16 or 32 bit computers like the Amiga would perform. Or even the C64 with a C version of the same program. So I made a C version, taking advantage of the fact that you can change text colors with ANSI escape codes, so no need to work with real graphics.

Just like the BASIC version, the initial version of the C code used floating point math. I added an 8.8-bit fixed point version to match the 6502 and Z80 assembly versions, but I'm not sure if I got that completely right as the output looks noticeably different.

This is the code. I compiled two Amiga versions that use the ixemul.library: floating point and fixed point as well as a version that doesn't require ixemul.library. That last one won't report its run time, so it's less useful on very fast systems where a stopwatch won't provide accurate results.

On my Amiga 1200 with its 50 MHz 68060 accelerator, the floating point version runs in 0.74 seconds and the fixed point version in 0.64 seconds. With the accelerator disabled so using the stock 14 MHz 68EC020 and just chip RAM, it's 4.4 and 2.04 seconds, respectively. On my Amiga 3000 with its 25 MHz 68030 the results are 2.12 and 1.4 seconds.

If you try this on any other machines, I'd love to hear your results. I'm especially interested in results from 7 MHz 68000 Amigas and the C64.

Permalink - posted 2022-03-29

Making of the PETSCII typer

▼ I thought it might be interesting to talk a bit about how my PETSCII typer web tool came together.

It all started with the Drop + Matt3O MT3 retro keycap set. I ordered this set because I like the old style key shape that's replicated here. And then as a bonus you get the PETSCII characters we know and love from the Commodore 8 bit computers printed on the front of the keycaps.

Not sure how I ended up there, but I found the Style64 C64 TrueType fonts, which replicate Commodore's take on the ASCII character set, usually referred to as PETSCII because the Commodore PET computer from 1977 introduced it.

So now I can have PETSCII graphical characters on my keyboard and PETSCII on my screen. The one missing thing: pressing a key and having the PETSCII character on that key show up on the screen. That's what the PETSCII typer does.

PETSCII, fonts and Unicode

PETSCII is actually two (four?) sets of 128 printable characters. One set (mostly) has the 95 printable ASCII characters and then about 30 graphical characters. The other set turns the lower case letters into upper case, and replaces the upper case letters with additional graphical characters. The two sets are also included in inverted (reverse video) form.

Style64's C64 Pro Mono font will display regular text in the Commodore 64's font, including Unicode characters that match PETSCII graphical characters. So when you change fonts, the characters will look different, but you still see the same characters. In addition to that, the four sets of 128 characters are also mapped into the Unicode private use space. Text that uses these Unicode code points will only show properly with the C64 font. If you change fonts, you'll just see a lot of "unknown character" boxes.

When using the C64 Pro Mono font, I try to keep most ASCII characters as ASCII and use the private use area mapping for the graphical characters. I also built in the ability to switch to a different font, which then requires additional character remapping. There are still some bugs there, but I decided to publish the tool now and do some more bug hunting at a later date.

To make things even more interesting, fairly recently additional "symbols for legacy computing" were added to Unicode, which makes it possible to map almost all non-inverted PETSCII characters to Unicode. However, my systems don't implement this version of Unicode (13 or 14) yet, requiring additional filtering/mapping options.

Javascript: mapping keyboards and Unicode

Javascript lets you run some code when the user presses a key. Your code then sees which physical key was pressed, which letter that key produced, and whether any modifier keys such as shift, control or alt were also pressed at the time.

The PETSCII typer keyboard event function then checks if there is a mapping from the key being pressed along with shift/control/alt to a PETSCII character. If so, it puts the mapped PETSCII character in the text area, and then uses event.preventDefault() to prevent the browser from performing the normal action associated with that keypress. Easy peasy. (If the key doesn't map, I just let the browser handle it the way it sees fit.)

But... I'm using the alt key for typing the PETSCII characters, so it's possible to type regular letters without alt. However, on the Mac the alt key (often called option there) can also be used to invoke dead keys for typing accented letters. And for some reason, in that case event.preventDefault() doesn't do its job, and an extra character appears. Turns out there is absolutely no way to get around that. Javascript actually has several events that let you catch dead key character composition, but no help there.

Eventually I did two things: allow the user to use control rather than alt for typing PETSCII characters, and catching the compositionupdate event, then noticing the size of the text in the text area, and then when a keydown event happens, if the text is now longer, remove the extra characters. On Safari this solves the problem, but not on Chrome.

And remember, Javascript handles characters as 16-bit values. So Unicode characters that don't fit into 16 bits (such as the new legacy characters, but not just those) are actually two code points in Javascript. So I had to handle that myself in order to be able to map to/from those Unicode characters.

Pasting rich text

The idea behind the PETSCII typer was that you can use it to type PETSCII and then copy and paste the results into another application, such as a text editor or word processor. To make that work as seamlessly as possible, I set things up such that when you copy text from the PETSCII typer text area (but not cut text), I catch that event and then put the plain text on the clipboard and also an HTML version of the same text with the C64 Pro Mono font specified. So in most other applications, the PETSCII will paste looking the way it should. Although often, it's necessary to tinker with the line height a bit to make sure everything lines up nicely.

(By they way, designers of modern fonts have no idea how box drawing characters and other graphical characters are supposed to work. The amount of whitespace above and below is all over the place so PETSCII graphics converted to fonts like Courier look pretty bad, even if all the characters could be mapped to Unicode.)

Loading files

I thought it would be a nice addition to be able to load PETSCII text files, as I'm not aware of an obvious way to convert those otherwise. The Javascript FileReader event makes the browser ask the user to select a local file, which is then made available to the Javascript script. So even though it looks like you're uploading a file, it's not transmitted over the network, it stays within the browser.

Mapping the characters from a text file to PETSCII in the text area was of course simple enough. Then I thought: what about Commodore BASIC programs? Those use line numbers in binary and they map all the BASIC keywords to "tokens". Again some mapping, so that was fairly easy, too. Although my code doesn't handle control codes inside strings in BASIC.

C64 colors

I couldn't resist the temptation to copy the C64 color scheme along with the PETSCII font. Turns out that there's many takes on how the C64 colors should look. I had a look at Secret colours of the Commodore 64 and Commodore VIC-II Color Analysis, but both delivered very dark and drab colors. (But both interesting reads anyway.) Especially the red is barely red. Eventually I used an HDMI capture dongle to capture the output of my THEC64 but then still increased the saturation a bit for several colors.

Of course today it seems weird that we can't agree on the objective color output specs of a computer. But the C64 and computers like it output NTSC or PAL directly, with no RGB values anywhere. Also, build variations between chip and board revisions and between individual computers made for significant differences. And we all set the saturation on our monitors or TVs to whatever we found most pleasing. But for sure, a Commodore 64 doesn't produce some drab brownish output when you ask for "red".

You can change the text color using control 1 - 8 and shift control 1 - 8, and the border and background colors by entering a number from 0 to 15 the "poke" boxes. And you can enable reverse video (background color becomes text color, text color becomes background color) using control 9 and turn it back off with control 0.

But that adds a new complication: what if the text cursor is on some text in reverse video? So the text cursor ("caret" in Javascript speak) needs to contrast with both the foreground and background colors. So I needed to keep track of (roughly) the brightness of each color, and then either use a bright cursor when text and background are dim, a dark cursor when both are bright, or a middle gray when one is dark and the other is bright.

It's always the small things that take an unexpected amount of extra time. Another example: when you click the "choose file" button, and select the last file you loaded, then no Javascript events trigger. So additional complexity is needed to be able to load the same file again.

Permalink - posted 2022-03-26

PETSCII typer: type your favorite C64 characters

▼ A few months ago, I ordered the Drop + Matt3O MT3 retro keycap set:

These keycaps have the cool/retro "PETSCII" characters printed on the front. But how do you actually type those PETSCII characters?

The first part of the solution is the C64 Pro Mono font by style64.org. This font replicates all the characters from a Commodore 64.

My web-based PETSCII typer tool connects everything together by producing the correct characters as you type as if your keyboard is a C64 keyboard, or using the layout of the MT3 keycap set. You can even load PETSCII text files or C64 BASIC programs which will show up more or less correctly. Or convert to current fonts to the degree possible.

When using the C64 Pro Mono font, if you copy some PETSCII text and then paste that text, it will use the C64 Pro Mono font if that font is installed on your system. This way, the PETSCII looks as intended in a word processing document.

There are still a few minor issues to be figured out, but I wanted to get this out sooner rather than later. Send me an email or a Twitter message if you have any comments or questions.

Permalink - posted 2022-03-23

Longevity of recordable blu-ray discs (BD-R / BD-RE)

▼ So now I got that new optical drive and can start burning CDs, DVDs, BDs (regular 25 and 50 GB blu-ray discs) and BDXLs (newer 100 and 128 GB BDs). But how long will those last?

For each of these types of optical disc, there are three types: mass produced, recordable and rewritable. The mass produced ones are made by stamping physical pits and lands into polycarbonate, cover this with a reflective aluminium layer, and then a lacquer (CD) or polycarbonate (DVD/BD) top layer. DVDs and BDs can have two data layers, BDXLs have three and sometimes four, with semi-transparent reflective layers.

Recordable CDs and DVDs (CD-R, DVD-R and DVD+R) have a layer of photosensitive dye below the reflective layer. The CD/DVD writer uses its laser at higher power to "burn" into the dye layer. With the result that even early CD drives can play CD-Rs without issue. DVD+R improves somewhat on the burn process used by DVD-R.

I'll cover BD-Rs/BDXL-Rs below.

Rewritable or recordable/erasable CDs (CD-RW), DVDs (DVD±RW), BDs (BD-RE) and BDXLs (BDXL-RE) all use a phase change material layer to hold the data. Depending on the intensity of the laser, the phase change material may end up in an amorphous or a crystalline state, which each having a different optical properties.

Longevity: the reflective layer

Obviously all types are sensitive to scratches from poor handling, but the advantage of optical discs over vinyl records and tape is that playing/reading them doesn't involve any contact with the part of the disc that holds the data. So mass produced discs can be played an unlimited number of times without any degradation. For the recordable types, there is a possibility that over time, the reading laser will affect the data layer. But unless you're playing one of these on repeat for months on end, that shouldn't be a problem in practice.

It's widely recommended to store discs in jewel (CD) cases or similar cases where the disc is only held by the hole in the middle, so there is nothing touching it or exerting pressure. And store them vertically.

For mass produced discs, the main issue is that sooner or later, oxygen will reach the aluminium reflective layer, which will then start to corrode and eventually render the disc unreadable. This disc shows lots of small holes in the reflective layer, but so far it plays just fine.

Recordable/rewritable discs often use silver or a silver alloy for their reflective layers. Silver won't corrode immediately just from being exposed to oxygen, but it will in the presence of pollutants like sulfur oxides.

According to Verbatim, a maker of such discs, it's possible to tell the difference between an aluminium and silver blank discs by holding them up to the light and seeing if there's any light coming through the disc. If there is, the reflective layer is silver. If there isn't, it's aluminium.

Of course this doesn't work if there's an opaque layer on top of the disc, for instance, if it's a printable disc. These are pretty nice, by the way:

Longevity: the data layer(s)

The photosensitive organic dye layer in CD-Rs and DVD-Rs is also somewhat of a liability: it will degrade over time. CD-Rs come in various colors that may or may not indicate the type of dye used, with better ones being developed later on.

DVD±Rs always look purple but of course came along when better dyes had been developed, anyway. In practice, I found that out of about a dozen CD-Rs that I burned between 2000 and 2004, one just wouldn't be recognized at all and two others had problems reading a few of their files. For DVD-Rs and DVD+Rs that I burned between about 2003 and 2010, I had issues with about 5% of the discs I tried to read recently.

Rewritables already have the issue that the difference between the two states of the data layer material are much less obvious, but it must also be able to change in two directions, making the addition of room for error much harder. In practice I've had relatively poor results with CD-RWs and DVD-RWs. Sometimes they won't even verify after a burn, and ones from the 2000 - 2010 timeframe had a failure rate of more than 75% when I tried to read them recently.

No problems so far with a box of fresh 25 GB BD-REs, though.

BD-R HTL vs LTH

Which brings us to recordable BDs/BDXLs. Unlike their predecessors, those originally didn't use an organic dye layer to record data, but a composite material, which is more stable. These discs are referred to as high to low (HTL), which gets at the reflexivity of the disc: a blank disc has high reflectivity, which becomes lower when burning.

However, a few years in the disc manufacturers brought the organic dye approach to BD-R, which has the property that the blank disc has low reflectivity, and then gets more reflective after burning, so "low to high" (LTH). This way, the manufacturers could reuse their existing CD-R/DVD+R production lines with only small modifications. HTL discs, on the other hand, require a more complex sputtering production method.

The way I see it, the only reason to still use optical for storage is because of its longevity. The good part is that you can replace a drive and then still read all your existing discs. With hard disk drives and flash, once one part fails (mechanics? electronics? the data?) then you're pretty much left without any useful course of action. In that light, the adoption of LTH BD-R is a step back.

A Reddit contributor felt much the same, but also concluded that LTH BD-Rs had completely replaced HTL BD-Rs in 2018. (Worth reading just for the first comment: "This is one of those posts that shows that it's entirely possible to be 100% correct, and also 100% bat-shit crazy.")

Fortunately, things don't seem to be quite this bad. Yes, it's annoying and perplexing that disc manufacturers don't label their products with important information like whether the discs are HTL or LTH. But, it does seem HTL BD-Rs are still being made, as I ordered a spindle of 10 25 GB Verbatim 6x speed inkjet-printable BD-Rs before I did any research... and it looks like those are HTL. Using Apple's System Profiler, we can see the following under Disc Burning:

We can then look up the ID over at the Disc Manufacturer ID & Media Type ID Licensee List and find out that the CMCMAG BA5 disc type is actually HTL.

Also, if you're careful (it's easy to get confused), you can tell by looking at a burned disc whether it's HTL or LTH. The two discs at the top are a blank DVD+R and a used DVD-R. The ones at the bottom a blank BD-R and a used BD-R.

If you look carefully, you can see that the unused part along the edge on the top right DVD-R is slightly darker than the burned part. For the BD-R at the bottom, it's the other way around, which is easiest to see around the middle. The first ring is very dark, but that one isn't writable. The next ring is writable material, but outside the writable area. This doesn't show up all that well on photos, and it's also easy to misinterpret. But there is a visible difference.

M-DISC

M-DISCs are supposed to last for 1000 years. The accelerated aging tests aren't proving them wrong, but of course also not proving them right. (Wait another 990 years for that.) But at least M-DISCs are making a claim about longevity, which is more than the competition does.

What to do?

If you have data that you want to stay around for a long time, without having to copy it from one HDD or SSD to another every few years, optical is probably the best choice. Still, just writing your data to disc once and then assume it will remain readable for decades is too optimistic. So I'd say:

• If possible, check the error levels for every disc you burn. Unfortunately, this information is generally not available.
• Store everything at least twice, preferably three times. Apply optical disc storage best practices.
• Use different types of discs and/or storage for the different copies.
• Also try to keep the different copies in different locations, both to accommodate for large scale disasters such as fire and floods, but also more mundane stuff such as less than ideal temperature and relative humidity.
• Check your discs once every decade or so, and make fresh copies when in doubt.
• Single layer (25 GB BD) is probably a safe bet, with less to go wrong compared to multiple layers.
• BD-R M-DISC is probably best, with BD-R HTL in second place. Whether BD-RE is better than BD-R LTH is unclear. So when in doubt, use both.

Permalink - posted 2022-01-09

Optical storage and media in the '20s

▼ Last year I wrote posts about listening to the radio, watching TV and having a landline phone in the 2020s (in Dutch). Today, I want to talk about the place left for optical storage in the 2020s.

Do I actually want or need those shiny optical discs in my life?

The short answer: most people probably don't. Software is now always available online, so you don't need an optical drive to install applications or drivers. For music, you can get pretty much everything through streaming services such as Spotify. And if you bought CDs back in the 1990s, you'll certainly have ripped those by now. I think you can watch pretty much any movie on one online service or another. Current TV shows are also generally available on a streaming service.

Transitioning to the disc-free lifestyle

But what if you bought movies and TV shows on DVD in the past, and used to burn some of your data to CDs and/or DVDs? You really want to go over all your data CDs and DVDs and copy those to a harddrive while you still have a working optical drive. That optical drive will die at some point (or become unreliable to the point of being unusable). You can of course then spend less than $/€ 50 for a DVD drive and handle this chore then, but why wait? Those CD-R and DVD±R organic dye layers aren't getting any better as they age.

Ripping movies: meh

I'm not sold on the option to rip all my movies and TV shows on DVD and blu-ray to my NAS or an external HDD. That's just way too much data. For movies, I don't have too much of an issue with simply using my blu-ray player (the one connected to my TV) to watch them. Yes, finding the right disc and then waiting for the drive to do its thing and having to deal with unskippable warnings and annoying menus isn't great, but that's only a minute or so vs a movie that's at least 90 minutes.

Ripping TV shows the hard way

For TV shows, this is quite different. Especially when I want to watch a 22 minute sitcom episode (and boy, do I love those turn-of-the-millennium sitcoms!), having to wait for the player to do its thing and then the warning and menus and me forgetting which episode was next, it's all quite annoying.

So what I used to do before my the Apple USB Superdrive that I got in 2011 kinda-sorta died recently, was rip TV shows from DVD and then use Handbrake to convert them to H.264 format, put in all the iTunes metadata using Subler, and store those on my NAS. However, that is quite a lot of work and eats up quite a bit of storage.

Ripping TV shows the easier way

Then I found an easier way to handle TV shows on DVD (and, to some degree, blu-ray): MakeMKV. MakeMKV is an app that, well, makes MKV files. You can use it to rip DVDs (easy), regular blu-ray (a bit more difficult) and UHD/4K blu-ray (when the stars align). My new blu-ray burner mentioned below didn't give MakeMKV any trouble when ripping DVDs, even region 1 ones on this region 2 drive.

For DVDs with TV shows, it's relatively straightforward to select the right tracks and turn those into .mkv files. However, MakeMKV doesn't do any transcoding. So the video is still MPEG 2, and the audio is still the same (usually) AC3. This means the files will be unplayable in many places, including iTunes and the iPhone/iPad.

However, my LG TV is more than happy to play these files. And, being a TV, it handles the intricacies of framerates and deinterlacing well automatically. fSo I just rip one DVD at a time, put the MKV files on a USB flash drive, and stick that in one of the USB ports of my TV. Yes, the files are three times as large as compared to the handcrafted H.264 versions, but I don't have to think about any settings and just delete the files after watching. Should I want to rewatch the same show a few years from now, I can always rip the same DVDs again.

My new optical drive

Like I said, my Apple USB Superdrive is no longer working properly, and the idea of having a big box full of music CDs and a smaller box full of data CDs and DVDs that I can no longer read bothers me.

So I decided to get a new external USB optical drive. One that can read and write blu-ray discs in addition to CDs and DVDs, of course. I also had a look at UHD blu-ray, which is used for 4K content on blu-ray. However, reading those requires firmware modifications, if possible at all. Dive into the MakeMKV forums to figure that out.

So I ended up ordering a Hitachi-LG BP55EB40 "Slim Portable Blu-ray WRITER" for a little under € 100. Optical drives used to have some issues with the amount of power provided by the USB2 bus in some cases, but that doesn't seem to be a problem anymore. You'd think USB3 would solve that, but no, as the datarates required for 6x speed BD, 4x speed BDXL (100/128 GB blu-ray), 8 x DVD and 24 x CD reading and writing fall well below what USB 2.0 can handle, that's what we get. The drive pretty much handles all the formats:

It will also write M-DISC discs. Those are recordable DVDs or BDs that use a special material for the recordable data layer(s) that should make them last for a thousand years.

Blu-ray for archiving

I'm not sure if my data still needs to be available in the year 3022, and it probably won't be, even on an M-DISC, but given that my failure rate from the DVD-Rs that I burned between about 2003 and 2013 is quite low (maybe 1 in 25 is no longer readable), an improved type of recordable disc seems like a good bet for things that need to last a few decades.

The advantage of an optical disc as an archival medium is that it only includes the data, not any electronics or logic. It's also fairly small and especially flat, so you can mail them or store them in a folder or safe deposit box and not worry about shocks or drops. A couple of years ago, I wrote about using flash for archiving, purely based on price. However, as flash capacities have gotten larger, especially for USB thumb drives and SD cards, their reliability has dropped. This is not a technology appropriate for long time archival storage.

Also, unlike storing data on a (possibly archival) hard disk drive (HDD), burning a disc is a one-time thing, so it encourages careful consideration of what needs to be archived in what format.

Conclusions

Although I think it's possible to leave the optical past behind, I do think keeping an optical drive around to rip TV shows and burn archival copies of my data is a good idea.

Permalink - posted 2022-01-02