Tag - microcontroller

1
on recent disk reading results
2
more real world tests
3
working FPGA version of the amiga floppy project
4
SX EOL
5
FPGA eval board
6
logic analyzer on Amiga 500 Paula chip
7
data rate vs throughput
8
March 21st Status
9
too many variables
10
SX microcontroller with 74HC132

on recent disk reading results

(this was posted to Classic Computer mailing list, please disregard if you’re on the list.  I think this is an important topic)

The last two nights I’ve been busy archiving some of my Amiga floppy collection.  Most disks were written over 20 years ago.

On a sample size of about 150 floppies, most of them were perfectly readable by my homegrown usb external amiga floppy drive controller.

I paid very close attention to the failures or ones where my controller struggled.

Without sounding too obvious here, the time between the pulses (which more or less define the data) were grossly out of spec.  The DD pulses should nominally be 4us, 6us, and 8us apart before pre-write compensation.  Most good disks are slightly faster, and normal times for these ranges are:

4us: 3.2-4.2us.  Many around 3.75us
6us: 5.5-6.2us.
8us: 7.5-8.2us

(notice margins around 1-1.3us)

My original microcontroller implementation was 3.2-4.2, 5.2-6.2, and 7.2-8.2.

When my current FPGA controller would have a problem, I’d notice that there were problems right on a boundary.  So maybe pulses were coming in at 3.1us apart instead of 3.2.  Or maybe 4.3 instead of 4.2.  So I kept bumping the intervals apart, making a larger range of pulse times acceptable — the XOR sector checksums were passing, so I was likely making the right choices.  The bits were ending up in the right buckets.

But as I went through some of these disks, I ended up with the difference between ranges(and basically my noise margin) being reduced smaller and smaller.  Some to the point where an incoming pulse time might fall darn smack in the middle of the noise margin.  Which bucket does THAT one go into?

My approach has been very successful(easily 95%+), but it makes me wonder about Phil’s DiscFerret dynamic adaptive approach where a sample of the incoming data defines the ranges.

Some disk drives and controllers might be faster or slower than others, and if you create custom ranges for each disk (each track?), perhaps you’ll have better luck.

more real world tests

So tonight, I put my new FPGA implementation of the amiga floppy project to good use.  I read some more of my collection of amiga floppies.

GREAT NEWS.

It’s working like a champ.  As a matter of fact, I selected disks which I could not previously read with my microcontroller solution.  I could read 90% of the previously unreadable ones.  Most of the unsolvable problems were related with HD disks (which, based on my earlier posts, some drives handle better than others)  Note this is just temporary until I try other drivers to read the disks — and try covering the HD hole.

I have better visibility on the PC as to the “problem” delta T’s.  So pulses that are slightly too far apart, just on the boundary of what I consider to be valid, if I adjust my boundary accordingly, and now consider them valid, everything is peachy-keen.  I want to add a real-time option in my java client to allow this to be adjusted on the fly.  See problems? Adjust the slider, and problems go away.  Pretty neat.

I didn’t have this visibility when the microcontroller was interpreting the delta T’s.  The microcontroller had no easy feedback method to tell me what was happening.  Having high-level debugging of the information on the PC makes this all possible.

Nice to see the software purring.  There is still plenty of improvements to make, usability to be improved, etc.  But its working very nicely.

working FPGA version of the amiga floppy project

So, I’ve been working on the FPGA version of the amiga floppy project for some time.  I just recently had a fair bit of free time, and so everything came together rather quickly!

I’m now able to read amiga floppy disks in using the same Java client software I had developed for use with the Parallax SX microcontroller board.  There were a few minor changes in the software — most notably the read data routine from the hardware.

I’ve written the code in Verilog on a Xilinx Spartan-3e evaluation board.

The various hardware parts I’ve described:

  • UART: Written from scratch, a transmitter and a receiver.   Simple to use, variable baud rates.
  • FIFO: Generated from Xilinx’s CoreGen. This connects the floppy interface to the USB interface. 32k bytes
  • FSM to always empty the FIFO to the PC.  Once something goes in the FIFO, it immediately gets sent to the PC
  • Read-floppy-FSM: Stores 225ms of Delta T’s (aka time between edges) as 8-bit integers into the FIFO.
  • Command FSM: Receives single-character commands from the java software to execute (R for read, U for upper head, L for lower head, etc)
  • Transmit test pattern routine: Sends 32k of data to the PC to test for reliable communication

A couple advantages with the FPGA solution:

  • We transmit the data to the PC as soon as it’s available.  I want to characterize the actual latency, but it should be pretty small.  This is different from my load->FRAM, and then FRAM-> PC method.  This method should be much faster and we’re just not idling for 225ms.
  • Instead of transmitting the bit-sync’d raw MFM to the PC, I’m sending the delta T’s.  While this requires a little more processing on PC, the PC can more easily determine why a particularly sector can’t be read.  For instance, is the time between pulses too small? Too large?  On a fringe edge?  Plus, since the Java decodes these, I can now add sliders for “acceptable delta T’s” for each 4, 6, or 8us windows.  Before that would require modifying the firmware on the microcontroller.  I can also start to do statistical analysis on the pulse times.

I am currently doing about 430ms per track.  This sucks.  I was beating that by 100ms with my microcontroller.  However, the problem is that because a variable amount of data is going to the PC, the PC receiving code does not know when exactly to stop receiving, so there’s a wait-timer which I have to optimize.  Once I receive the minimum amount of data, I wait 100ms since the last received data, and then exit.  I’ve got to put my logic analyzers in place and figure out how to optimize it.

Denis@h3q can read a disk in 43s, which is pretty darn good.  He is using tokens like I mentioned here and here and here.  I’m transferring much more data though, which gives me more information.  I like his time, and maybe that would be a nice goal to beat!  Wow. That’s only 269ms/track.  Hrrrmm…. that’s pretty good.

SX EOL

I hate when this happens. 🙂

The Parallax SX microcontroller line has reached production EOL. The owner of the SX design
(www.ubicom.com) has given Parallax final notice that we are to place a lifetime buy of wafers. We
recognize this announcement will be difficult for customers who have designed the SX into their products.
We share your disappointment.

http://www.parallax.com/Portals/0/Downloads/docs/prod/sx/SX_EOL_Notice.pdf

FPGA eval board

One of these arrived today.  Very cool.  I’ve only done minor things with it so far — but I’m learning.

I’ve got FPGAs by Example for Verilog coming within a couple days.  Looks like a good book.

The only thing I’ve managed to do so far was modify the assembly for the picoblaze microcontroller to change the text scrolling on the lcd — you’ve got to start some place.

So is this amiga floppy project rev 3 now? heh. I dunno.

(I want to get the afp stable, working perfectly, and move on — I’ve spent too much time on it already, although I’ve enjoyed every minute)

logic analyzer on Amiga 500 Paula chip

So I’ve attached my Intronix LA1034 to the Paula chip, the 16-bit wide data bus, and the DMA Request line. I triggered the logic analyzer on 0x4489, of course.

Here’s the results:

(click for a full size version)

So I’ve spent most of my time sort watching it come out of a floppy drive, but never from Paula’s perspective. It’s always before Commodore’s custom LSI floppy controller Paula could get ahold of it. So notice that the data is still RAW MFM, and not processed in any way, because Paula doesn’t perform this function. The MFM decoding is done in the trackdisk.device, with the help of the bit blitter.(part of Agnus)

So the normal floppy sync pattern is 0xAAAA 0xAAAA 0x4489 0x4489, but why do we see 0x2AAA ??

So, 0x2AAA is 0010 1010 1010 1010. right? We’re missing the first bit. It turns out there was a bug in the MFM encoding routine. It was fixed March 16th 1990 at 1:08am in the morning, in revision 32.5 of the trackdisk.device.

Also, most times, the sync word used was 0x4489 — which is exactly what I use to find sync in the firmware I wrote for the microcontroller.

Oh and here’s Paula with the great EZ hooks on her leads

(click for full size)

data rate vs throughput

Although I’m transmitting from the Parallax SX microcontroller to the PC via USB at a data rate of 2mbps, the actual throughput is much less.  First, there is some overhead, namely start and stop bits, which is 25%.  Next, I actually have to read the data from the FRAM, and this takes time.

It takes approximately 1.7us to read 1 byte, and then 5us to transmit that byte.  The 5us is 500ns (1/2mbps) * 10 bits (8 start + 1 start + 1 stop).  So 6.7us per byte.  This doesn’t include the time it takes to findsync().

So my throughput is approximately ~800 kbps on a data rate of 2mbps.

Kind of sucks, but getting to 2mbps is impossible unless I integrate the reading/findsync’ing into the transmit routine.  And I think that’s generally a bad idea.  I really want to protect my bit times so I have quality transmission.  I don’t want to get into the uh-o situation where processing something else overruns the time slot, etc.

Yeah, so right now it looks like <READDATA aka pause><SEND BYTE><READ><SEND><READ><SEND> and so on.  To get to 2mbps, I’d basically have to <send><send><send><send>.  Now, if I could utilize the “dead time” between bits to actually read the data….. well… then I might be closer.  Remember, too, that I’m bit-banging so doing something interrupt driven is out of the question.

I’m not 100% sure the PC isn’t introducing some of this delay.  Which is why I’ve been looking at revamping the read routines.  First, they are butt-ugly, and second, they don’t handle error cases well.  Actually, they hang in error cases.

I’m still floating around the idea of error CORRECTION by taking advantage of the inherent structure of MFM.  I really think that there is something here.

Next steps are to try to work out a better read routine, and then implement a retry system for bad tracks.

March 21st Status

Well, here’s where I’m at:

The external amiga floppy drive is now hooked directly to my SX microcontroller.  The code within the microcontroller waits for a command.  If it get’s a “B” command via USB, it turns the motor on the drive, stores the contents into FRAM, and then turns the motor off.  It’s grabbing one track at a time.  The SX then goes into command mode again.  If my PC software sends it an “A” Command, then the SX dumps to contents of FRAM back to the PC, then the software proceeds to process the track by checking checksums, and ensuring everything is ok.

As far as speed: The motor is on for approximately 225ms.  It takes roughly 83ms to transfer the track from SX to PC.  So really only ~308ms per track.  Now there will be PC processing time, and everything still has to be automated, but I’m pretty happy with the times.

Absolute minimum time would be 160 tracks @ 308ms per track, 49 seconds.  But I really think the end result would be closer to 80 seconds(500ms per track).  I guess that’s not bad for reading a whole disk straight into an .ADF file.

What’s left is automate everything, like the sending of the STEPS, change from one side of the disk to the other, etc.

Everything will be commands to the SX via USB, so the commands might be:

“C” – STEP forward

“D” – STEP backward

“E” – Switch to upper head

“F” – Switch to lower head

So a command string might be:

“EBA FBA C EBA FBA” and so on.

too many variables

I think that’s really the problem I’ve been facing with this floppy project. Trials and tribulations indeed. Or at least trial and error, with stress on the error part. There are probably less than half-a-dozen people worldwide that have the very specific knowledge that I need to get this off the ground. These would be people from CAPS, one person who sells an amiga (as well as other floppy drives) floppy controller card, and that’s it.

I’ve talked to some really nice ex-Commodore employees who have been friendly, and helpful.

The problem, of course, is that these machines were created a long time ago. The programmers of the original OS, freeware/shareware software, hardware, have all long forgotten the details. And you know what they say, “the devil’s in the details.”

To give you an idea of the variables with which I’m working:

1. Am I reading the hardware signal right? Are there *really* only three possibilities coming out of the drive, a “10”, “100”, and a “1000”? This is all I’ve been able to observe…

2. Is the software I’ve written on the SX microcontroller properly sampling this data at the right time? I have two interrupts occuring, one for the edge detection and one for the 2us timeout which clocks a zero in (up to 3 of them in a row before going idle). So far, sometimes this works, and sometimes it doesn’t. Why and how is this getting out of sync?

3. Is the SX transmitting faster than the PC can handle this? So far, my observations say no, and I implemented a software FIFO to help out.

4. Is my software FIFO working?

5. Is my PC software, that’s designed to receive all this, working properly? I’m now storing the transmitted bytes in a memory array, and later storing to disk, to prevent any speed issues associated with accessing the hard drive.

The REAL problem here is that I simply don’t know what is actually leaving the drive in terms of data. The only thing I’ve been able to figure out is that the MORE data is leaving the drive than is showing up on the PC. This is a bad sign. Where’s the data being lost? My guess, the sampling isn’t working properly. Something is slipping. But how the heck can it slip when a transition resets it, and transition is occuring on a very regular basis (minimum every 4us, maximum every 8us). The thing only has to run freely for a few cycles.

Add the fact that the only real pattern that I can VISUALLY look for is the 0xAAAA 0xAAAA 0x4489 0x4489 raw sync sector in raw MFM. Any words, or real data has to be decoded before, properly aligned on a byte, etc. PAIN IN THE BUTT to figure out if everything is actually working. Most of my programming projects, whatever they might be, are straight-forward. Run it, see if it works, and then go back and fix it if it doesn’t. Even the indication that this thing is working is obscured.

There are so many variables, and I’m constantly changing all of them, that who knows when one part is working? Of course, you say, only change one thing at a time. OK, makes sense, but this just isn’t practical. If I change how the SX xfers to the PC, then I have to modify the PC software accordingly…. So if I make a mistake in coding on one side or the other, who knows?

Maybe I’m lacking the necessary basic project management, coding, microcontroller, hardware experience required to get this off and running?

BTW: my title of this entry reminds me of “Too many secrets” or “Cootys Rat Semen.” from Sneakers. If you don’t know what I’m talking about, please disregard! 🙂

SX microcontroller with 74HC132

Here’s a picture of the prototype coming together.

From the top down on the breadboard, you see a pull-up resistor, on the input that comes in on the yellow wire. The output comes out of the grey wire on the right hand side(and is going to a scope on the green wire heading off the bottom of the board.

The RC network that makes up this one-shot negative going oscillator is on the right hand side. This controls the width of the output pulse, which in this is case is 1.5us.

74HC132