Tag - Parallax

working FPGA version of the amiga floppy project
further development on the AFP
Halloween 2007 AFP updates
Flow control
heat problems
data rate vs throughput
java client works
new face of the amiga floppy project
mid-january status

working FPGA version of the amiga floppy project

So, I’ve been working on the FPGA version of the amiga floppy project for some time.  I just recently had a fair bit of free time, and so everything came together rather quickly!

I’m now able to read amiga floppy disks in using the same Java client software I had developed for use with the Parallax SX microcontroller board.  There were a few minor changes in the software — most notably the read data routine from the hardware.

I’ve written the code in Verilog on a Xilinx Spartan-3e evaluation board.

The various hardware parts I’ve described:

  • UART: Written from scratch, a transmitter and a receiver.   Simple to use, variable baud rates.
  • FIFO: Generated from Xilinx’s CoreGen. This connects the floppy interface to the USB interface. 32k bytes
  • FSM to always empty the FIFO to the PC.  Once something goes in the FIFO, it immediately gets sent to the PC
  • Read-floppy-FSM: Stores 225ms of Delta T’s (aka time between edges) as 8-bit integers into the FIFO.
  • Command FSM: Receives single-character commands from the java software to execute (R for read, U for upper head, L for lower head, etc)
  • Transmit test pattern routine: Sends 32k of data to the PC to test for reliable communication

A couple advantages with the FPGA solution:

  • We transmit the data to the PC as soon as it’s available.  I want to characterize the actual latency, but it should be pretty small.  This is different from my load->FRAM, and then FRAM-> PC method.  This method should be much faster and we’re just not idling for 225ms.
  • Instead of transmitting the bit-sync’d raw MFM to the PC, I’m sending the delta T’s.  While this requires a little more processing on PC, the PC can more easily determine why a particularly sector can’t be read.  For instance, is the time between pulses too small? Too large?  On a fringe edge?  Plus, since the Java decodes these, I can now add sliders for “acceptable delta T’s” for each 4, 6, or 8us windows.  Before that would require modifying the firmware on the microcontroller.  I can also start to do statistical analysis on the pulse times.

I am currently doing about 430ms per track.  This sucks.  I was beating that by 100ms with my microcontroller.  However, the problem is that because a variable amount of data is going to the PC, the PC receiving code does not know when exactly to stop receiving, so there’s a wait-timer which I have to optimize.  Once I receive the minimum amount of data, I wait 100ms since the last received data, and then exit.  I’ve got to put my logic analyzers in place and figure out how to optimize it.

Denis@h3q can read a disk in 43s, which is pretty darn good.  He is using tokens like I mentioned here and here and here.  I’m transferring much more data though, which gives me more information.  I like his time, and maybe that would be a nice goal to beat!  Wow. That’s only 269ms/track.  Hrrrmm…. that’s pretty good.


I hate when this happens. 🙂

The Parallax SX microcontroller line has reached production EOL. The owner of the SX design
(www.ubicom.com) has given Parallax final notice that we are to place a lifetime buy of wafers. We
recognize this announcement will be difficult for customers who have designed the SX into their products.
We share your disappointment.


further development on the AFP

I’m about to bring a new member of my family into this world.  As a new proud parent, I’m completely unsure about what this means in terms of time, dedication, and so on.  I’m sure I’ll have less free time to spend on projects like this.   As a result, updates and improvements are unlikely to happen in the short term.

Something to remember is that I’ve already achieved my goal, which was loosely, “to create a device that sits in between a PC and a floppy drive that can read amiga floppies and create emulator .ADFs in a reasonable period of time.”  You know, crossing the finish line was rather anti-climactic, mainly because (as you’ve seen) there are so many smaller, but no less important, steps that all add up to a working product.

I’ve learned a fair bit on this project, and would like to go back and identify the specific problems I faced, how I eventually solved them, and why they presented such a challenge.  There were some really fundamental mistakes I made which frustrated things, and made this task harder than it should have been.  I think other beginners can learn from my mistakes, and I’d like to make a post/(maybe a paper?) on some basic do’s and dont’s.

In any event, there are a few things I want to touch up, but then I’ll be posting a copy of the java source, compiled java with instructions on how to execute, and the SX firmware.

Note that I’m not going incommunicado — I’ll be monitoring the blog and email as usual.

Last but not least, there are a number of things I really want to do with this thing, namely build a custom circuit board.  I’d like to add the writing capability, but that is a fairly large task.  There are other minor to-do’s that eventually need done.  So I’m not finished yet here.

You know, I could never have gotten as far as I have without the help of people here, especially Tim and David, and people over at the Parallax forums(Bean, pjv, Michael C, Guenther), who I’m pretty sure are tired of my posts and my frustration.

Halloween 2007 AFP updates

  • DTR-based PC to SX flow control is now enabled and working, thanks to jd2xx. It has been tested, and SX responds appropriately to a STOP request. Strangely, with the new jd2xx code, it doesn’t appear that flow control is even needed now, where it was CLEARLY needed with rxtx.
  • The heat problems I was worried about was no big deal. The slow boat from China arrived with my IR thermometer. My SX goes from 74 degrees F to 100F when idling, and 118F or so when operating. I’m going to continue to monitor it, but Parallax tech support says 120-130 is perfectly normal. Chip’s max is +185F.
  • Performance update: From start to finish on the PC side, my java code is reporting 141 ms per track to transfer it. My logic analyzer says 122 ms or so, but I’ve found the reason for the gap. As mentioned in earlier posts, my USB2SER device uses a USB transfer size of 4096 bytes, or 3968 after 128 bytes of overhead(2 status bytes per 64 data). It has to transfer 3*3968 byte blocks + 1*3334 byte block for a total of 15238 bytes. The last block is subject to a latency timer of 16ms (adjustable to 1ms), where the USB2SER is waiting for another 634 bytes before xmitting to the PC. Sooooo you get 122 + 16 + 3ms scheduling delay and voilà, you are at 141ms. Maybe it makes sense to just pad out to the full buffer. (Not to be rhetorical, but it does, 634*8us per byte (see earlier posts) = 5ms instead of 16. I’d have to get fairly precise, don’t want to send 1 extra character. I could also just drop the latency timer value.
  • I posted a schematic (now slightly outdated, already) here. Also added it on the right hand side blog menu.

Things are progressing nicely now that I’ve put the required time into it. I have a something on the way that is going to consume a lot of my time in the future, so I’m trying to bang this out.

Flow control

I’ve gotten away thus far without any flow control. I just started noticing that my software was locking up occasionally when reading a disk. It was locking up in the “read” portion of the code because it wasn’t receiving enough data. When I rewrote the read routine (which is much cleaner now, btw), I implemented a small receive timeout, because I really should be getting all the data within 150ms(timeout currently set at 300ms). With my new read routines, I noticed times right around 125ms which is almost exactly the time it should be taking —- meaning that there isn’t any inherent dead time with my new routine. This is good.

In any event, I don’t know how or where exactly this “lost data” problem has cropped up, but I’m almost positive it’s related to the lack of flow control on my serial interface. Some people have really stressed that I must have flow control, but I haven’t seen any dropped data, until now. So I haven’t really worried about it. It’s really worked flawlessly in the past, but I’m now getting a short read every now and again.

I think flow control is easy enough to implement. I have a spare pin, although I’m starting to get short. The Parallax USB2SER ties the DTR pin of the FTDI FT232BM usb-to-serial chip to Parallax’s RES pin. What this means is that if FTDI’s VirtualComPort(VCP) drivers support hardware DTR flow control, then the drivers should drop DTR if the PC/USB chip is getting overrun.

Inside my sendusb() routine, I’ll have a small check that looks like:


if RC.5 = 1 then waitforDTR

And this will be checked before each byte is transmitted. The reaction time should be instant, if DTR is not high, then my code will wait for it to come high. I’m pretty sure that the FT232BM warns at 32-bytes, so I actually have plenty of time.

I don’t pretend to understand USB. The USB reads have to be scheduled by the OS, and I guess if the OS gets too busy, it doesn’t have time to service the USB port, and the data falls on the floor. While I haven’t seen this before, perhaps I’m running too many background apps etc to properly service the USB requests.

heat problems

As I alluded in an earlier post(s), my SX28 has been pretty warm. After doing a number of different tests, I’ve now ordered an infrared thermometer from amazon for $29.99. I want to find out exactly how hot this thing is getting. It may very well be a normal operating temperature, and I’m able to keep my finger on it for an extended period of time(sort of on the comfortable side of pain.) I’m pretty sure the temperature is under 120 degrees, and the chip is rated for 185. It is running at 50mhz, and is in a small SSOP package, so who knows.

I’ll probably make a forum post over on forums.parallax.com and find out what others’ SXs are running.

Might be concern over nothing, but I’ve emailed support on Saturday, and they are checking into it.

My IR thermometer is on the free-super-saver-shipping-slow-boat-from-China, so it will probably be another
week and a half before I see this thing. It’s definitely the el-cheapo model, with a field of view of 1:1, but I don’t care. It should be accurate enough to get me in the ballpark. I just need some type of absolute reference instead of saying, “this feels hot to me.” I can also monitor it as I’m using it as well.

My biggest concern is that I fry the SX28, which is surface-mount, and not easily replaceable. Either soldering a new SX28 on there or re-doing my circuit board does not sound like fun to me. So, it’s better to be safe than sorry.

data rate vs throughput

Although I’m transmitting from the Parallax SX microcontroller to the PC via USB at a data rate of 2mbps, the actual throughput is much less.  First, there is some overhead, namely start and stop bits, which is 25%.  Next, I actually have to read the data from the FRAM, and this takes time.

It takes approximately 1.7us to read 1 byte, and then 5us to transmit that byte.  The 5us is 500ns (1/2mbps) * 10 bits (8 start + 1 start + 1 stop).  So 6.7us per byte.  This doesn’t include the time it takes to findsync().

So my throughput is approximately ~800 kbps on a data rate of 2mbps.

Kind of sucks, but getting to 2mbps is impossible unless I integrate the reading/findsync’ing into the transmit routine.  And I think that’s generally a bad idea.  I really want to protect my bit times so I have quality transmission.  I don’t want to get into the uh-o situation where processing something else overruns the time slot, etc.

Yeah, so right now it looks like <READDATA aka pause><SEND BYTE><READ><SEND><READ><SEND> and so on.  To get to 2mbps, I’d basically have to <send><send><send><send>.  Now, if I could utilize the “dead time” between bits to actually read the data….. well… then I might be closer.  Remember, too, that I’m bit-banging so doing something interrupt driven is out of the question.

I’m not 100% sure the PC isn’t introducing some of this delay.  Which is why I’ve been looking at revamping the read routines.  First, they are butt-ugly, and second, they don’t handle error cases well.  Actually, they hang in error cases.

I’m still floating around the idea of error CORRECTION by taking advantage of the inherent structure of MFM.  I really think that there is something here.

Next steps are to try to work out a better read routine, and then implement a retry system for bad tracks.

java client works

Well I’m producing correct .ADF’s with the new java client.

The terminal mode is also finished and working.

I have some various odds and ends to wrap up with it, namely allowing the user to save his preferences, actually putting the preferences to use(I’m currently ignoring them) and using the correct filenames, etc. Bunches of little stuff really.

Besides those, the other major thing that needs implemented, written, etc, is proper error handling.  I’m not reporting or taking many error conditions into consideration.  I don’t save “bad data”, but I also don’t retry, etc, to hope to fix the problem.

I *am* currently having a heat problem where my SX28 is heating up way too much. I tried posting on the parallax forums, but that was unfruitful. I think I must be asserting something, and that same lead is being connected to ground, but heck if I can figure out what it is.

I’m going to try disconnecting the floppy and see if it continues to heat up. and then I’m going to check all the pins into the floppy and see if any go straight to +5v or GND.

new face of the amiga floppy project

You’d think that with the lack of posts lately that I haven’t been doing anything.

BUT, in fact, I’ve been quite busy.

I now have the entire floppy project on one single circuit board, the SX28 protoboard.

See http://www.parallax.com/detail.asp?product_id=45302 for a link to it.

What’s really nice is that the protoboard has pads for surface mount stuff, and my ram fit perfectly on it. I think it’s designed for slightly wider surface mount stuff, but it worked.

Old recent pictures:

Old Floppy pic 1

Old Floppy pic 2

BUT, the new floppy project looks like this:

New floppy pic 1

What’s nice is that everything is integrated into one board, and there is no extra SOIC8 to DIP adapter, etc. Much cleaner.  Most of the wiring is below the board, and I’ll take better pictures in a little bit.

Best part of this whole thing is that the new board actually fully functions.  A minor problem here or there, but I fixed it in short order!

mid-january status

So here’s where I’m at:

I know my memory read and write routines are good.  I calculate a checksum as I’m writing the data, and on the output of that data to the PC, I also calculate the checksum.  They match, no problems there.

I know my USB to PC routines are good.  I calculate a different byte-based checksum (8-bit checksum)  from the data I get from the FRAM, and then I have the PC software calculate it.  They always match and I’m using Parallax’s UART for the time being.  Mainly for reliability.

I’m using a new basic ISR routine, which I posted a post or two back.  It’s simple, it doesn’t force any particular type of encoding.  What comes off the drive goes into the memory.  There are some drawbacks, for instance, I don’t support any type of idling.  For the initial data, I wait to see a transition, and then I turn on interrupts and start recording.  I don’t check double 1’s now, and I don’t check more than (3) 0’s.  The data SHOULD be coming out of the drive correct, and force fitting it into correct form just doesn’t work, and while it does fix SOME situations, I *REALLY* have to get to the bottom of why this happens.

My MFMSanityCheck software is telling me .3% of the data is bad, which I think 99.7% still isn’t anything to complain about, but I really have to find the source of the problems.

All .3% of at least one sample file is a double 1’s situation.  And I’ve seen this before.  And its NEVER triple one’s, and it’s NEVER too many zeros.  Just double 1’s.  Two 1’s back to back.

So now that I’ve tested my memory, the problem is either the drive is REALLY spitting out two 1’s (and I have no clue how to fix that problem), or my ISR is triggering twice on the same edge.

I’m leaning towards the second choice but I really have to figure out how that is happening.