C64 Preservation - Late Activision titles

Status
Not open for further replies.
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Greetings everyone,

I have run across several later Activision titles (~1990) that I am not
able to image *at all*. They pretty much are detected as blank tracks by
all my routines. So far I have Afterburner, F-16 Combat Pilot, Project
Stealth Fighter, and Skyfox 2. All seem to lack a proper track cycle at
any density, much like certain tracks on later Epyx titles.

Anyone that has any insight into Activision's later protections (beyond
the Pirateslayer years), please contact me. It looks like maybe density
changes within the tracks or something.

Pete Rittwage
C64 Preservation Project
http://rittwage.com/c64pp
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

On Sun, 24 Jul 2005 22:41:33 -0400, Pete Rittwage <peter@rittwage.com>
wrote:
>
>Anyone that has any insight into Activision's later protections (beyond
>the Pirateslayer years), please contact me. It looks like maybe density
>changes within the tracks or something.

I was thinking about this several days ago, and came up with an idea
that is not trivial, but I believe would be the be-all-end-all way to
address intratrack density changes, as well as other very difficult
protections such as timed sync lengths, missing short syncs, sync-less
tracks, etc.

Addressing with G64 just a single game such as you mention would
involve the format supporting density changes in-track, assuming the
program reads only certain parts at certain densities, and figuring
out which parts to sample at which density. Such a process I think
will have to have manual intervention per program, at least as tedious
as hacking it, and hard to be quite confident it is right.

The problem we have with even current mnib+emulator format is the 1541
is doing too much processing, so we do not have pure enough
information. By the time we pull a byte from the shift register, the
info on the read signal itself has already been lost, going through
the edge sensing to generate 1s, density-dependent generation of 0s,
and sync sensing. The ideas here would instead require upfront work
of a new even lower-level emulator format, trivial hardware change to
small hardware circuit, and mnib updates, but would offer very high
confidence of really solving the problem for arbitrary games.

Background

Info is based on 1541 Repair and Maintenance Manual and Inside
Commodore DOS. Any references to chips assume the original 1541 'long
board', not the 1541 II 'short board' which coalesced some chips into
PLDs.

Read signal from the 1541 is amplified and converted to TTL, available
on pin 7 of UH4 (LM311 voltage comparator). This is then used in
conjunction with UG2 (74LS86 XOR), UG3 (9602 one-shot), and UF6
(72LS74 D flip-flop) to detect read signal transition. The result is
UG3 pin 10 output is a 1 pulse to indicate read signal transition, 0
for no transition. Thus far the operation is density independent.

UE7 (74LS193 counter) divides the 16Mhz clock by a value from 13-16,
depending on the density setting in the VIA PB6:5. This then
determines the rate at which 0's will be clocked out to the shift
register if no transition occurs. At this point we're already
dependent on what density the program expects, so no need to look
higher into counter to resync to each transition, the shift register
and sync detection.

1) Synchronous read signal sampling

Sampling the read signal itself takes any density, sync, etc. out of
the picture. Given that density determines a 13-16 divider of a 16Mhz
clock, sampling in a density independent fashion requires sampling at
16Mhz. In theory, just hook one of those parallel port input pins to
UH4 pin 7, and sample it at >16Mhz sampling rate. This would provide
raw data capable of reading anything a 1541 disk track could have used
for protection. (in theory, a read signal glitch of sub-66ns could be
missed by this but still have been caught by the 1541 transition
detection, but this would have been too short to have been reliably
written by drives or possibly even represented on the media).

In practice, 16Mhz sampling may be no problem for modern logic
analyzers, but no standard PC hardware is going to be capable of
achieving it. Latencies of FSB->PCI or proprietary NB/SB link->ISA
likely mean an IN opcode will take 100s of ns to a couple us.
Additional hardware would be needed to make true 16Mhz sampling
feasible.

a) Shift register(s)

To reduce the necessary sampling rate by a factor of 8, add a small
circuit with shift register such as 74164 or 74595, where UH4 pin 7 is
clocked into the shift register on each transition of the 1541 16Mhz
clock (TBD - can such a load be placed on the oscillator?). A counter
such as 74193 to count to 8 clocks and latch the shift register would
also be needed, as well as a T flip-flop which would be toggled after
each 8 clocks (delay flip-flop transition slightly from latching shift
reg). Parallel output of the shift register goes to parallel data,
flip-flop output goes to a handshake. Mnib16Mhz polls the parallel
handshake, and when it has transitioned, samples the 8 bits of data.
This is much less intensive than direct 16Mhz sampling, but for
reliable operation still requires guaranteeing 2 IN opcodes in <500ns
- not very likely on a parallel port on the other side of an ISA
bridge.

An alternative which could make that sampling feasible though is to
not use parallel port for sampling. IDE also operates at TTL levels,
is 16 bits wide, one hop closer to the CPU, and was intended for
higher speed data. Cascade 2 shift registers together. Capture 15
bits at a time, use 1 bit for the toggle flip-flop (optionally use 14
bits and also add 1 bit for an index hole sensor - might as well solve
track synchronization protection for good too). Dedicate an IDE
channel to this since it will definitely not be a real IDE device and
would interfere with anything else on the channel. Setup the channel
for PIO mode 4 (via the BIOS or chipset-specific registers) to get
shortest cycle times. Mnib16Mhz polls IDE data register (0x01f0 or
0x0170) for T flip-flop transition, on transition stores the 14 or 15
bits. 14 bit sampling in this manner would require a single IN from
the data port take < 1000/(16/14) =~870ns. PIO mode 4 has a 120ns
cycle time on the IDE bus, plus FSB, PCI, etc. delays. I believe most
modern hardware can achieve this a fast enough IN for this.

This all assumes no other sources of significant delay. A high speed
CPU with instructions cached running with ints disabled would be
necessary. Data cache misses would not be critical - misses would be
stores, which will likely be buffered on modern CPUs and not delay
execution. No other memory delays could be tolerated though. No XMS
copies, no BIOS calls for EMS, no hard or even soft protected mode
page faults. The only >1M memory access methods under DOS that may be
fast enough would be RealMem, non-paged protected mode if you can get
it, or possibly EMS without BIOS usage.

b) Counter

Since we know the 16Mhz rate is much higher than real bit rate of any
density used, we expect to generally have runs of 13*4, 14*4, 15*4, or
16*4 of the same read signal (this amount of data could later be
trivially RLEed). The assumption here is that we'll always have runs
of such sizes (a safe assumption unless a disk used sub-bit-time
glitches), so why not reduce necessary sampling rate by a hardware
counter to encode runs.

Would need an 8-bit counter clocked off 16Mhz, a way to buffer/latch
counter output (TBD), and a read signal transition detection (UG3 pin
10 probably usable) to latch the counter value and toggle a flip-flop
to indicate new data ready. A D flip flop would be used as well to
latch the read signal level. To deal with counter overflow on very
long runs, another counter could trigger at 64 to also latch counter
output and toggle the T flip flop (but not change the D flip flop).
This would provide a 6 bit run counter, run level bit, and bit which
transitions on each new count ready. Assuming no sub-bit-time
glitches, sampling just parallel port data in this fashion would be
safe at an IN latency of <~3us. Using handshakes for level and T flip
flop could get an 8 bit run but require another IN, moving to IDE
instead of parallel could allow 8 bit run and other 2 bits in a single
IN.

Same concerns about no other meaningful delays such as memory paging
would still apply.

2) Asynchronous read-signal sampling

Just sampling (and time stamping) as fast as we can *might* still be
good enough as long as we can significantly beat 3.25us. The trick is
to achieve >4Mhz sampling (since in the 1541 a counter counts to 4 and
resyncs on each read signal transition, so this could buy us a factor
of 4 breathing room). This is not as guaranteed as 16Mhz sampling and
requires slightly more complex software, but if this fast of sampling
is possible, there is no need for circuitry to provide a hardware
assist.

Connect read-signal UH4 pin 7 to the fastest reasonable TTL input on
the PC - ie. a dedicated IDE channel operating in PIO mode 4. Poll
IDE data as fast as possible. Along with the read signal bit level,
store a timestamp. The legacy 8254 takes at least 2 8 bit ISA reads
to read, so it is right out. The only timestamp source fast enough is
the free-running clock counter present on CPUs since Pentium,
accessible via RDTSC opcode (assumes 32 bit mode, this gets timestamp
to edx:eax). The timer has oddities, and may be read out of order
with other opcodes, will not advance when halted, etc. As long as it
advances when blocked waiting for an IN (I believe it does), the rest
can be dealt with though.

Post processing would then need to construct a simulated 16Mhz sampled
track based on the samples, timestamps, and time for a track rotation
(easier with that index hole sensor). (RDTSC values would need
converted into normal time using CPU clock frequency, which can be
tough to calculate accurately) As long as IN sampling is <250ns, no
transitions would be missed, though the time of transition may be off
from reality by some sub-bit-time. This timing requirement is very
difficult, but IDE on the right hardware might make it feasible. It
would be worth a shot to check if 4Mhz sampling can be achieved before
considering extra hardware for true 16Mhz sampling.

Reading a raw disk

If the arduous timing requirements can be met, the rest is easy by
comparison. Raw disk data would be stored out as quickly as possible
in >1M memory, storing multiple rotations. Checking/triming track
match would have to be fuzzy by at least +/-1 count of a given read
signal run to account for reading it the second time through sampling
starting at a slightly different point, and variations in rotation
speed. Index hole sensing could narrow down point to start looking
for track repeat.

Raw .16mhz emulator disk

This lower level data would mean yet another disk format, though a
conceptually simple one. For a track, it would contain # of bits
(need not be multiple of 8), and store raw bit data for signal level.
Size would vary based on how many samples were picked up when reading
(ie vary by drive RPM), but at 300RPM would be
16000000/5bits=400KB/track, so a 16MB image for a 40 track disk. Of
course this could be trivially compressed with RLE. A simple fixed
1bit value/7bit run would get down to <1-2MB, slightly smarter storage
methods could still improve significantly on that.

Emulator support would need added for this. I do not know how complex
or pluggable the disk image model is in VICE or CCS64, but again this
is at least conceptually straightforward. A track pointer would
advance through the signal level at a fixed rate simulating 300RPM,
and transition detection, count-to-4/resynch on transition, and
clocking 0 bits based on current density settings would need emulated.
I'm assuming emulation of byte-ready/sync/shift register port are
already there, the new module would just be the means to supply their
values.

The format could likewise be translated to G64 by running through the
tracks at given densities.

Effectiveness

It's a dangerous statement to make, but if 1541 disks were sampled and
emulated in this manner, I can't think of a disk track based
protection that could not be handled. Sync lengths would be
faithfully reproduced, short syncs wouldn't cause problems, syncless
tracks wouldn't matter. Custom formats and changing density within a
track would be non-issues.

I've heard rumors of protections using >2 0 (non-transition) bits in a
row and exploiting some bug where a false 1 bit could be introduced,
so the protection is to reread the track multiple times and see that
the 1 bit positions don't always match. I cannot confirm the validity
of that rumor, but even if true, with the read signal level stored
this could be emulated by randomly introducing false transitions to
the emulator during a long non-transition run.

The only disk protection I know of which couldn't be addressed by this
is relying on inter-track synchronization, but having a defined track
start point, emulating track position+time for stepper movement, and
reading from the index hole would address that.

Development

This idea is pretty involved, requiring high speed sampling, probably
extra hardware, and a new emulator disk format. There may be smarter
ways to achieve the kind of low-level emulation I'm talking about, but
I don't believe there is a higher level alternative you could be
confident in applying to an arbitrary protected disk.

If you want to discuss this or have other ideas and want to bounce
them off me, that would be great. If you decide to implement
something here, I could help with logic design of the circuits, but am
not the one to help design in discretes such as resistors or
capacitors if needed, let alone touch a soldering iron (though this
*might* be breadboardable). I'd be up for tackling VICE usage of a
raw format too (I'm assuming CCS64 is not open source).

Aaron
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> On Sun, 24 Jul 2005 22:41:33 -0400, Pete Rittwage <peter@rittwage.com>
> wrote:
> If the arduous timing requirements can be met, the rest is easy by
> comparison.

Since this all will require a breadboard, why not dump the info straight
to a 16mb battery backed ram module on the board? The only timing
constraints would be how fast your chips could latch and ram speed.

Or something along those lines...

Great post!

CL
 
G

Guest

Guest
Archived from groups: rec.games.video.classic (More info?)

Thanks, Aaron. (I met Aaron yesterday, incidentally- great guy).

You have gone way over my head with your knowledge of digital
electronics- I only have knowledge of this stuff through reverse
engineering the code for Burstnibbler that Markus already had done most
of and improving upon the idea. With that said, I am certainly able to
solder and am willing to help however I can.

I wonder if it's worth it to do this using the old Commodore hardware or
if there is something else out there that's more readily available?
There is a piece of software on OldSkool called Disk2FDI. The author
gave me a version to test, but I was never sure I was able to get my old
Teac drive to rotate at 300rpm, so I never got it to work. I believe
that it needed a slower PC than what I had lying around at the time, but
I may revisit that idea as well. I think a PC disk drive can only
sample MFM at 4MHz, though, right? I was told by another contact this
would not be enough to get a better image than what we had with NIBs.

Pete

--
Sent via Gamer Newsgroups
http://www.gamernewsgroups.com
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: rec.games.video.classic (More info?)

Hi Pete,

I didn't offer the registered version of Disk2FDI in order for you to
use the shareware functions ;-)
Drive rotation of 300 rpm is not needed for what you want to do, that
is create low-level FDI files of games using the Disk2FDI cable.
With the games you describe, the FDI format is clearly the best way to
go. You just need to find a good-enough PC (Pentium 233MHz would
probably work best) with a PCI parallel port card (can be found at
around $15 on the Net) and create the simple 2-wire Disk2FDI cable.

Please let me know if you need help to configure a computer for
Disk2FDI, but your drive needs not be modified.

And keep up the good work!

Vincent Joguin.
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Hi Aaron,

You may be glad to learn most of the work is already completely done
and functional. I have created and released Disk2FDI a year ago that
allows exactly what you describe (with some differences of course). It
will output the raw track images to a file called an FDI (Formatted
Disk Image) file. Sampling is done at 33 MHz (speed of the PCI bus), so
it's even better than your requirements!

The only remaining task is support of the FDI format within VICE. So
far, I have found no one willing (or able) to add that support to VICE.
If you are willing and able to do this, that would be so great!

You can find the specifications of the FDI format here:
http://www.oldskool.org/disk2fdi/files/FDISPEC.pdf
Only the "low-level" format is really useful in this case. Code to
decompress and process raw FDI streams is already available with the
source for WinUAE (an Amiga emulator. The Amiga machine supports GCR,
so everything needed is there). I can also provide sample FDI files of
some C64 games and programs to test. We can of course arrange even more
if you are willing to jump in.

Please let me know what you think. You can contact me directly at
disk2fdi _at_ joguin.com

Vincent Joguin.
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

I need to try this again. Time to dig for an old Pentium in the pile at
work. :)


ApH wrote:
> Hi Aaron,
>
> You may be glad to learn most of the work is already completely done
> and functional. I have created and released Disk2FDI a year ago that
> allows exactly what you describe (with some differences of course). It
> will output the raw track images to a file called an FDI (Formatted
> Disk Image) file. Sampling is done at 33 MHz (speed of the PCI bus), so
> it's even better than your requirements!
>
> The only remaining task is support of the FDI format within VICE. So
> far, I have found no one willing (or able) to add that support to VICE.
> If you are willing and able to do this, that would be so great!
>
> You can find the specifications of the FDI format here:
> http://www.oldskool.org/disk2fdi/files/FDISPEC.pdf
> Only the "low-level" format is really useful in this case. Code to
> decompress and process raw FDI streams is already available with the
> source for WinUAE (an Amiga emulator. The Amiga machine supports GCR,
> so everything needed is there). I can also provide sample FDI files of
> some C64 games and programs to test. We can of course arrange even more
> if you are willing to jump in.
>
> Please let me know what you think. You can contact me directly at
> disk2fdi _at_ joguin.com
>
> Vincent Joguin.
>
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

On Sun, 31 Jul 2005 00:00:29 -0500, Christian Lott <c65@cox.net>
wrote:
>Since this all will require a breadboard, why not dump the info straight
>to a 16mb battery backed ram module on the board? The only timing
>constraints would be how fast your chips could latch and ram speed.

True, going even more sophisticated is certainly possible if you're up
for it. I was trying to stick to something that could be designed and
built with 74xx parts and minimal glue, hopefully even on breadboard.

For 16M buffer, you'd need a micro for it to do buffering to RAM and
then deliver from RAM to PC. Probably the easiest to find cheapest
micros with a good PC interface already would be USB slaves, eg.
something like Cypress FX2. You'd need a high clock rate one to do
the sampling fast enough, and a port to bank >64K of external RAM.
And that is assuming 16M of SRAM.. DRAM interfacing to make the RAM
cost effective would require RAS/CAS and refresh handling, so even
more complex micro or glue.

Unless you'd be planning to spin a board and sell it in at least 100s,
I don't think going more complicated than breadboardable logic would
be worth it. For a single unit, the cost where it's worthwhile making
hand-built hardware is bounded by cost of low-end external logic
analyzers. eg. http://www.usbee.com/sx.html looks like a $300 device
that would suffice - max 24Msamples/s, up to 8 signals, USB2 interface
to PC, storage bounded by PC memory.

Aaron
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> On Sun, 31 Jul 2005 00:00:29 -0500, Christian Lott <c65@cox.net>
> wrote:
>
>>Since this all will require a breadboard, why not dump the info straight
>>to a 16mb battery backed ram module on the board? The only timing
>>constraints would be how fast your chips could latch and ram speed.
>
>
> True, going even more sophisticated is certainly possible if you're up
> for it. I was trying to stick to something that could be designed and
> built with 74xx parts and minimal glue, hopefully even on breadboard.
>
> For 16M buffer, you'd need a micro for it to do buffering to RAM and
> then deliver from RAM to PC. Probably the easiest to find cheapest
> micros with a good PC interface already would be USB slaves, eg.
> something like Cypress FX2. You'd need a high clock rate one to do
> the sampling fast enough, and a port to bank >64K of external RAM.
> And that is assuming 16M of SRAM.. DRAM interfacing to make the RAM
> cost effective would require RAS/CAS and refresh handling, so even
> more complex micro or glue.
>
> Unless you'd be planning to spin a board and sell it in at least 100s,
> I don't think going more complicated than breadboardable logic would
> be worth it. For a single unit, the cost where it's worthwhile making
> hand-built hardware is bounded by cost of low-end external logic
> analyzers. eg. http://www.usbee.com/sx.html looks like a $300 device
> that would suffice - max 24Msamples/s, up to 8 signals, USB2 interface
> to PC, storage bounded by PC memory.
>
> Aaron

I was thinking you may be able to get away without having a full
processor for the thing. You only need to store data on the bus. You get
a 16M counter and count up for your address sequence. Just a read/write
signal twice the speed of your desired sampling rate.

As for getting the data out of the darned thing, could you dma the
sampled 16m file back out the disk drive through an x1541 cable?

You could probably get a swift upload with that.


Christian Lott/
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

On 1 Aug 2005 02:11:39 -0700, "ApH" <bidulecornu@hotmail.com> wrote:
>You may be glad to learn most of the work is already completely done
>and functional. I have created and released Disk2FDI a year ago that
>allows exactly what you describe (with some differences of course). It
>will output the raw track images to a file called an FDI (Formatted
>Disk Image) file. Sampling is done at 33 MHz (speed of the PCI bus), so
>it's even better than your requirements!

Very interesting. I encountered Disk2FDI before, but it was a page
with very scarce info. Once I saw I needed a 360K floppy drive I
admit I didn't look any further. Do you have available any tech info
on how you sample an old XT-era floppy drive at 33Mhz? I'm assuming
some hardware mods were in order.

>The only remaining task is support of the FDI format within VICE. So
>far, I have found no one willing (or able) to add that support to VICE.
>If you are willing and able to do this, that would be so great!

So you have been sampling disks, but never been able to test them?

>so everything needed is there). I can also provide sample FDI files of
>some C64 games and programs to test. We can of course arrange even more
>if you are willing to jump in.

If this is really a viable way to capture disks, it sounds like a good
place to start. Your low level streams are more complex than raw
signal level. Is the 'average' stream essentially a list of n 33Mhz
clocks between read signal transitions? Setting aside the optional
min/max streams, any more info (timing diagram would be good) for it
and index stream would be very helpful to clarify what data is really
available and how it will need emulated.

Aaron
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> On 1 Aug 2005 02:11:39 -0700, "ApH" <bidulecornu@hotmail.com> wrote:
>
>>You may be glad to learn most of the work is already completely done
>>and functional. I have created and released Disk2FDI a year ago that
>>allows exactly what you describe (with some differences of course). It
>>will output the raw track images to a file called an FDI (Formatted
>>Disk Image) file. Sampling is done at 33 MHz (speed of the PCI bus), so
>>it's even better than your requirements!
>
>
> Very interesting. I encountered Disk2FDI before, but it was a page
> with very scarce info. Once I saw I needed a 360K floppy drive I
> admit I didn't look any further. Do you have available any tech info
> on how you sample an old XT-era floppy drive at 33Mhz? I'm assuming
> some hardware mods were in order.
>
>
>>The only remaining task is support of the FDI format within VICE. So
>>far, I have found no one willing (or able) to add that support to VICE.
>>If you are willing and able to do this, that would be so great!
>
>
> So you have been sampling disks, but never been able to test them?
>
>
>>so everything needed is there). I can also provide sample FDI files of
>>some C64 games and programs to test. We can of course arrange even more
>>if you are willing to jump in.
>
>
> If this is really a viable way to capture disks, it sounds like a good
> place to start. Your low level streams are more complex than raw
> signal level. Is the 'average' stream essentially a list of n 33Mhz
> clocks between read signal transitions? Setting aside the optional
> min/max streams, any more info (timing diagram would be good) for it
> and index stream would be very helpful to clarify what data is really
> available and how it will need emulated.
>
> Aaron

I looked over the file specifications for FDI 2.0 and it looks like the
track data is stored as GCR?

I figured it would be stored in some kind of sampled version of the flux
transitions on the disk that we would then have to interpret at the
known Commodore bitrates and then translate into GCR for the purpose of
emulation. (The original sampled data would be quite large). I guess
if the XT drive hardware and Disk2FDI is capable of interpreting this
bitstream exactly without the limitations we have with a 1541, then it's
not an issue. Can you send me some examples of disks imaged with Disk2FDI?

There are only a handful of disks that can't be imaged properly with a
1541 and MNIB, which is itself a Burstnibbler/21-second-backup style
parallel backup system. It would be nice to be able to image these
other 1% of disks, though, even if we can't run them on the emulators.

Pete Rittwage
C64 Preservation Project
http://rittwage.com/c64pp
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> Very interesting. I encountered Disk2FDI before, but it was a page
> with very scarce info. Once I saw I needed a 360K floppy drive I
> admit I didn't look any further. Do you have available any tech info
> on how you sample an old XT-era floppy drive at 33Mhz? I'm assuming
> some hardware mods were in order.
>

Not much, but you can read about the Disk2FDI cable here:
http://disk2fdi.joguin.com/D2FCABLE.htm
Anyway, sampling a brand new HD 3.5" drive, an XT 5.25" drive or an 8"
drive is the same. All drives return pulses on the Read-Data line that
correspond to flux transitions on the disk surface. These pulses go
through the parallel port's interrupt line, and the resulting data is
processed by Disk2FDI.

> So you have been sampling disks, but never been able to test them?
>

Not C64 disks. I have tested some Amiga and Apple 2 disks. And Disk2FDI
internally detects any standard sector it may find in the signal.

> If this is really a viable way to capture disks, it sounds like a good
> place to start.

It is! I can assure you.

> Your low level streams are more complex than raw
> signal level. Is the 'average' stream essentially a list of n 33Mhz
> clocks between read signal transitions?

Yes, exactly!

> Setting aside the optional
> min/max streams, any more info (timing diagram would be good) for it
> and index stream would be very helpful to clarify what data is really
> available and how it will need emulated.
>

Each pulse coming from the disk drive is represented by a collection of
data. The most important one is the average value which represent the
average time from the previous pulse to the current one. The minimum
and maximum values are the same, corresponding to the min and max time
from the previous pulse to the current one. The index values are the
number of times the index was read as a '0' and as a '1' when this
pulse was read. Additionally, when you have found the maximum sum of
those 2 values in the whole track, you know how many passes the track
was initially read. If, for a given pulse, the sum is below this
absolute maximum, it means that the pulse "disappears" sometimes, and
is therefore weak.

I have sent you an e-mail with a sample image and more information.
Please let me know if you didn't receive this e-mail.

Vincent.
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

> I looked over the file specifications for FDI 2.0 and it looks like the
> track data is stored as GCR?

No, not currently. Track data stored as GCR are not yet created, and
will be created only (if I can finish it) when using the shareware
version and 2 drives. These track images will of course be of a far
lesser quality compared to those created with the registered version
and the Disk2FDI cable.

The track type of interest to you is the last one, the "low-level" one.

> I figured it would be stored in some kind of sampled version of the flux
> transitions on the disk that we would then have to interpret at the
> known Commodore bitrates and then translate into GCR for the purpose of
> emulation. (The original sampled data would be quite large).

Yes, this is exactly it!

> I guess if the XT drive hardware and Disk2FDI is capable of interpreting this
> bitstream exactly without the limitations we have with a 1541, then it's
> not an issue. Can you send me some examples of disks imaged with Disk2FDI?

I have just sent you an e-mail with a sample image. Please let me know
if you didn't receive it.

> There are only a handful of disks that can't be imaged properly with a
> 1541 and MNIB, which is itself a Burstnibbler/21-second-backup style
> parallel backup system. It would be nice to be able to image these
> other 1% of disks, though, even if we can't run them on the emulators.

Yes, I agree. Although it would be even better to be able to run them
on an emulator ;-)

Vincent.
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

OK, so it sounds like disk2fdi is the same concept as my 2) approach,
but with:
- PC floppy with index line instead of 1541 for raw data
- parallel port to poll instead of ATA

This approach should be as viable as the 1541 approach, and share its
intense timing requirements. If you are having problems with .d64s
not working that you made from a .fdi you captured, I strongly suspect
it is since this approach isn't be able to sample quickly enough.

Parallel port read on many chipsets will still involve a PCI->ISA
bridge. Parallel port accesses would be limited to theoretical max
8Mhz, real-world would be lucky to be much better than 1Mhz. No
chance your average user's PC parallel port would be fast enough.

The Lava PCI card Pete said you recommended is a true PCI parallel
port. It might be fast enough to beat 4Mhz sampling, but only with
optimal conditions that may not hold. If the chipset is very good and
introduces only minimal delays, all PCI settings are optimized, and
the Lava device itself can always respond to an IO read immediately,
then I believe you could complete a target IO read each 4 PCI clocks,
for just over 8Mhz sampling. If any of those assumptions don't hold
then it will be slower.

I'm assuming you have one of these Lava PCI parallel cards - what real
sampling rate do you get polling it? If you haven't already measured
it, do you mind checking, eg.

__uint64 ParallelAccessClocks()
{
__asm
{
rdtsc
mov esi, eax
mov edi, edx ; save 1st time stamp to edi:esi
mov dx, 0x0378 ; pick another IO reg address as appropriate
in al, dx
in al, dx
in al, dx
in al, dx ; n times reading parallel port
cpuid ; needed to prevent 2nd rdtsc from executing out-of-order
rdtsc
sub eax, esi
sbb edx, edi ; subtract old timestamp from new
; edx:eax is standard for x86 64bit C return value
}
}

The return value would contain the time, in CPU clock cycles, for all
those port accesses. Divide by # of in opcodes, then divide by CPU
clock rate to determine real time per port input. If you actually are
getting sub-250ns it will be very impressive, and that means the Lava
card (with whatever chipset you use at least) would be viable.

Aaron
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> OK, so it sounds like disk2fdi is the same concept as my 2) approach,
> but with:
> - PC floppy with index line instead of 1541 for raw data
> - parallel port to poll instead of ATA
>
> This approach should be as viable as the 1541 approach, and share its
> intense timing requirements. If you are having problems with .d64s
> not working that you made from a .fdi you captured, I strongly suspect
> it is since this approach isn't be able to sample quickly enough.
>

A built-in parallel port (provided it works at all) could be a little
too slow for the outer tracks (higher densities). Any PCI parallel port
card should be fast enough. But Disk2FDI will let you know about it.

> Parallel port read on many chipsets will still involve a PCI->ISA
> bridge. Parallel port accesses would be limited to theoretical max
> 8Mhz, real-world would be lucky to be much better than 1Mhz. No
> chance your average user's PC parallel port would be fast enough.
>

Yes, a built-in port is indeed usually around 1MHz, most often below
850KHz.

> The Lava PCI card Pete said you recommended is a true PCI parallel
> port. It might be fast enough to beat 4Mhz sampling, but only with
> optimal conditions that may not hold. If the chipset is very good and
> introduces only minimal delays, all PCI settings are optimized, and
> the Lava device itself can always respond to an IO read immediately,
> then I believe you could complete a target IO read each 4 PCI clocks,
> for just over 8Mhz sampling. If any of those assumptions don't hold
> then it will be slower.
>

The best I have ever achieved is 4MHz, which is far above the
necessary, even for high-density disks (hey, it would even be quite
enough for extra-high-density disks! ;-) ).

> I'm assuming you have one of these Lava PCI parallel cards - what real
> sampling rate do you get polling it? If you haven't already measured
> it, do you mind checking, eg.
>

I don't have a Lava PCI card, but probably something similar (Pete
didn't yet update me on his PCI cards finds). The best card I have is
based on the SUN1888 chip.

As for the speed of this card, please look at the log file for the
front side (which was created using this card). The log file for the
back side shows the speed of another card, one based on an Oxford
OX12PCI840 chip.
I hope you received my e-mail? If not, please contact me directly and
I'll resend it.

> The return value would contain the time, in CPU clock cycles, for all
> those port accesses. Divide by # of in opcodes, then divide by CPU
> clock rate to determine real time per port input. If you actually are
> getting sub-250ns it will be very impressive, and that means the Lava
> card (with whatever chipset you use at least) would be viable.
>

For C64 disks (not high-density IBM disks), I'm quite sure any PCI card
is OK. The speed also depends on the motherboard and processor, for
some reason (I'm not really a hardware guy, although I have some
general understanding).

Vincent Joguin.
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

On 5 Aug 2005 05:22:42 -0700, "ApH" <bidulecornu@hotmail.com> wrote:
>
>The best I have ever achieved is 4MHz, which is far above the
>necessary, even for high-density disks (hey, it would even be quite
>enough for extra-high-density disks! ;-) ).

To handle 1541 disks with arbitrary density would ideally take 16Mhz
sampling, but its 2 bit counter which resets to 0 on read signal
transition (to allow for slight speed variations) allows us the slop
such that 4Mhz should suffice. Below 4Mhz sampling it's possible to
construct timing diagrams of a read signal and sample points where
errors could occur. (these frequencies assume 300rpm 1541 speed when
sampling; increased rpm when sampling requires linear increase in
sampling rate, eg. >=4.8Mhz at 360rpm of a high density PC drive) If
there's enough similarity in the tracks to correlate them across
enough multiple reads and do some smart reconstruction I could see you
dealing with error cases from too low sample rate. Your capturing
does this?

You noted the .fdi converted to .d64 didn't work, but as Pete
mentioned, that may be due to protection .d64 couldn't represent.
Unless you are dealing with one of those
density-changing/short-sync/no-sync/etc. especially hard cases,
though, .g64 should be able to represent it. Have you ever converted
a captured .fdi to a .g64 to test?

Aaron
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Aaron Bilger wrote:
> You noted the .fdi converted to .d64 didn't work, but as Pete
> mentioned, that may be due to protection .d64 couldn't represent.

Yes, the sector data matches perfectly with an original disk, but it
checks for protection by looking in the gaps on track 21. This isn't
stored in a D64, but if he can get this far, he should be able to do a
perfect G64 that works.

--
-
Pete Rittwage
http://rittwage.com
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

> To handle 1541 disks with arbitrary density would ideally take 16Mhz
> sampling, but its 2 bit counter which resets to 0 on read signal
> transition (to allow for slight speed variations) allows us the slop
> such that 4Mhz should suffice.

Ah, we're not talking about the same frequency here. 4MHz is the
maximum frequency between 2 pulses. In the case of a 1541 disk, this
frequency is between 250Kb/s (so time between 2 pulses can be no less
than 1/250000 second) and maybe 350Kb/s, depending on speed zone
(didn't really calculate). So we're very far from 4 MHz, which is more
than excellent.
On the other hand, Disk2FDI samples time between 2 pulses at the speed
of the processor, although the real time "grain" is that of the PCI bus
(33 MHz). So we're above the necessary 16 MHz you mentioned.

Don't worry, everything works fine ;-) I know you at least partially
have to trust me, and this may be a problem, especially for people like
you who know a lot about those issues.
But if you want further explanations, I'm always ready to provide them.

Thanks for this interesting thread!

Vincent.
 

aph

Distinguished
Aug 11, 2004
8
0
18,510
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

Oops, forgot to answer this:

> Have you ever converted a captured .fdi to a .g64 to test?

No. I don't have such code ready. Pete told me privately that such code
could be the first step to support FDI files in the C64 community, and
I do agree. Do you think you could create such a convertor?

Vincent.
 
G

Guest

Guest
Archived from groups: comp.sys.cbm,comp.emulators.cbm,rec.games.video.classic (More info?)

ApH wrote:
> Oops, forgot to answer this:
>
>
>>Have you ever converted a captured .fdi to a .g64 to test?
>
>
> No. I don't have such code ready. Pete told me privately that such code
> could be the first step to support FDI files in the C64 community, and
> I do agree. Do you think you could create such a convertor?
>
> Vincent.
>

Hi Vincent,

If you got as far as extracting the sectors, you were already past what
we need for a G64. :)

If you have a way to get me the raw GCR data from each track, I can get
it into the G64 file format with no problem.

--
-
Pete Rittwage
http://rittwage.com
 
Status
Not open for further replies.

Similar threads