Denise C64 mode problems!

[amilo3438]

Decided to stop testing the existing g64 collection from 2015! The reason for this is the frequent finding of something that looks like it works but is actually problematic.

As a last example: "pacmania[grandslam_1988]"... it will start and then after a while in the game the emulator will go crazy!? And the first thing is that something is wrong with the emulator, but in fact the problem is in the tested file!
(If I convert nbz from that file with e.g. -an -C -r -f options, I will get something that does not cause problems.)

So, now there remains the possibility of creating instead of looking for problems, and that in a way.... i.e. generating a new g64 from nbz using the latest version of nibconv and testing them in denise with checking problematic ones (ie those that doesn't seem to work) in vice.

-------------------

Well, this is my last test where I converted the contents of the Cyan folder (without alt versions) from the group of nbz files and tested them in denise and then in vice.

In doing so, I created a batch file to make the job easier:

for %%f in (*.nbz) do (
  nibconv %%f "%%~nf.g64"
  del %%f
)

So, that was about 90 files to test. Of those, I got 64 that work in denise (default), and those 26 that didn't work, I checked in vice3.8(x64sc) and found another 11 files that worked! I then checked those same 11 again in denise but now with new settings in disk wobble, and here are the results...

Below didn't work in denise with default disk wobble 0.5 rpm but worked fine in vice(x64sc)!

Then tested again in denise but with adjusted disk wobble to work! (wobble 0.5 rpm +- ~0.1 rpm)
------------------------------------------------------------------
bionic_commando[capcom_1988](pal)(!) .................. 0.6 rpm
black_tiger[us_gold_1989](pal)(!)...................... 0.4 rpm
crack_down[us_gold_1990](pal)(!)....................... 0.4 or 0.6 rpm
dynamite_dux_s1[activision_1988](pal).................. 0.4 rpm
forgotten_worlds_s1[us_gold_1989](pal)(!) ............. 0.6 rpm
indiana_jones_temple_of_doom[us_gold_1988](pal)........ 0.6 rpm ?(very close but not start)?
mega-apocalypse[martech_1987](cyan2)(pal).............. 0.7 rpm
metro_cross[us_gold_1987](pal)......................... 0.4 rpm
out_run_s1[us_gold_1988](pal)(!)....................... 0.4 or 0.6 rpm
rolling_thunder[tengen_1989](pal)(cyan2)............... 0.6 rpm
trantor[probe_1987](pal)..............................  0.4 rpm
-------------------------------------------------------------------------------------
--> 11 more working in vice(x64sc) and in denise but only with adjusted disk wobble

EDIT: Even micro64 had no problem running all 11 listed, while hoxs64 managed to run only the first one! (Note: 0/11 as for denise with default disk drive 1541-II ... and if it were 1541 disk drive then the ratio is 5/11 for denise!)

Is it normal that a difference of +-0.1 rpm in disk wobble can have such an impact on whether something works or not in denise!?

Emu compatibility after the test:
-----------------------------------
64/90 -> ~71% denise (default disk wobble 0.5 rpm and 1541-II disk drive)
69/90 -> ~77% denise (default disk wobble 0.5 rpm but 1541 disk drive)
75/90 -> ~83% vice3.8(x64sc) EDIT: Just an assumption (quite incorrect) because not all but only 11 problematic ones in denise have been tested!
-----------------------------------

And the rest that didn't work probably require a new conversion with some of the options turned on! (to be tested later)


EDIT: And the following 15 did not work in denise or vice! The nibconv (even with options) did not helped so it was necessary to process them with remaster-utility!

armageddon_man[martech_1987](cyan2)(pal)
avenger[gremlin_1986](cyan1)(pal)
deceptor[us_gold_1986](pal)
gauntlet_ii[us_gold_1987](pal)(!)
highway_encounter[vortex_1985](pal)(!)
indiana_jones_last_crusade[lucasfilm_1989](pal)(!)
italy_1990[us_gold_1990](pal)
jack_the_nipper[gremlin_1986](cyan1)(pal)
last_mission[data_east_1986](cyan2)(pal)
muncher_eats_chewits[gremlin_1988](pal)(!)
out_run_europa_s1[us_gold_1992](pal)(!)
rainbow_warrior_s1[micro_style_1989](pal)
shadow_dancer_s1[us_gold_1991](pal)(cyan2)(bad)
supersports_s1[gremlin_1988](pal)
zyron[kingsoft_1986](pal)

Some were recognized by the utility itself as problematic, while for some others it was necessary to turn on patching manually!

P.S.
It seems that denise is only "sensitive" when it comes to nibconv created g64 files. With the remaster-utility created g64 files, denise is not at all sensitive to rotation speed and disk wobble settings!? (then what's the problem?)
After all, I believe that there is nothing more to test or compare! So that would probably be it, as far as this testing is concerned!

[amilo3438]

One note about...

shadow_dancer_s1[us_gold_1991](pal)(cyan2)(bad) ... tested to work in denise after nibconv -aa -r -f nbz g64!
shadow_dancer_s1[us_gold_1991](pal)(cyan2)(alt)(bad) ... tested to work in denise after nibconv -r -f nbz g64!

It may take a few tries to get it to run as it sometimes stops at track 19! (it's best to make save state at the title screen and then try from there)
It seems that in vice does not want to start! (always stops at track 19)


---------------------------------
An addition to the test from the previous post...

For this test was used other batch file and then they were all tested to find which one of previously problematic works with 0.5 rpm wobble!

for %%f in (*.nbz) do (
  nibconv -aw %%f "%%~nf(-aw).g64"
  nibconv -ag %%f "%%~nf(-ag).g64"
  nibconv -a0 %%f "%%~nf(-a0).g64"
  nibconv -as %%f "%%~nf(-as).g64"
  nibconv -aa %%f "%%~nf(-aa).g64"
  nibconv -an %%f "%%~nf(-an).g64"
)

Previously 11 problematic files in denise now works fine with disk wobble 0.5 rpm if other alignment was used than default:

Now works with denise disk wobble 0.5 rpm if converted with other alignment!
----------------------------------------------------------------------------
bionic_commando[capcom_1988](pal)(!) .................. -an, -as, -aw
black_tiger[us_gold_1989](pal)(!)...................... -a0, -aa, -an, -aw 
crack_down[us_gold_1990](pal)(!)....................... -a0, -an
dynamite_dux_s1[activision_1988](pal).................. -a0, -as, -aw
forgotten_worlds_s1[us_gold_1989](pal)(!).............. -aw, -ag -r -f
indiana_jones_temple_of_doom[us_gold_1988](pal)........ -as
mega-apocalypse[martech_1987](cyan2)(pal).............. -a0, -as
metro_cross[us_gold_1987](pal)......................... -aa
out_run_s1[us_gold_1988](pal)(!)....................... -an
rolling_thunder[tengen_1989](pal)(cyan2)............... -aa -r -f
trantor[probe_1987](pal)..............................  -a0
----------------------------------------------------------------------------
Note: Some of above needed also additional options to work! (-r -f)

Seems -ag is default when no options is used in nibconv! (as was in previous post)

EDIT: Checked above 21 test also in vice3.8(x64sc) and only rolling thunder didn't work, but worked in micro64! (hoxs64 fails to work in almost all)
A more detailed test would require checking all 11x6 combinations, actually 11x5, because we know from the previous test that vice had no problems with -ag, i.e. the default setting.

From the nibtools_readme.txt:

-aw: Align all tracks to the longest run of unformatted data.
-ag: Align all tracks to the longest gap between sectors.
-a0: Align all tracks to sector 0.
-as: Align all tracks to the longest sync mark.
-aa: Align all tracks to the longest run of any one byte (autogap).
-an: Align all tracks to the raw data as found (not normally used).

I know it might be interesting to see how other emulators would react to the new alignment change, like vice, but that would require a bit too much work and time.

The conclusion of this test might be that you don't need to change anything in the emulator, but you need to choose the right option when converting, so then there won't be any problems!

[PiCiJi]

@amilo3438
thank you for this much work. I'll take a look at it soon.

I'm almost done with the Metal Graphics API. I've been planning to integrate Metal since 2018.

[amilo3438]

After today's testing, I think that it really makes no sense to touch or change anything in the emulator for the reasons that I will state below.
Any change in the emulator would actually be just an adaptation to the g64 created with nibtools/nibconv, but is that really necessary anymore!?

Here is an example where a g64 file was created with nibconv and then passed through remaster-utility:

From the nbz collection of the the radwar folder: beam[magic_bytes_1989](!).nbz

File 1: beam[magic_bytes_1989](!)(-aw-r-f).g64 --> created with nibconv -aw -r -f nbz g64
File 2: beam[magic_bytes_1989](!)(-aw-r-f)_ReMaster.g64 --> file 1 passed through remaster-utilty (no options used)

Regarding the file 1:
- it will work on denise with d'n'd on power button or on vice with d'n'd on the screen or in micro64 with load":*",8,1!
- will work if the disk drive is 1541-II or 1571 (tested in denise)
- won't work in denise or vice if trying to load it manually with: LOAD"*",8,1 or LOAD":*",8,1

Regarding file 2:
- will work in denise, vice, micro64 or hoxs64
- will work with all disk drives
- will work if trying to manually load it with: LOAD"*",8,1 or LOAD":*",8,1

So, the file 1, which used to cause problems, after passing it through remaster-utility no longer causes problems! (magic)
Therefore, is there any reason to change anything inside the emulator if we know where the problem is!?
I think there is no more, only in terms of better adaptation to g64 created with nibconv, but you see and decide.

EDIT: Only if the emulator could recognize the problematic g64, as the remaster utility recognizes it, and then "process" it internally before it was loaded!? (that would be an improvement)


@r.cade
Guys, thanks for the latest version of the utility!


Note: I don't want to downplay the importance of nibconv when converting, as even the remaster utility doesn't handle all situations! (like for example with pacmania conversion) Both play their role depending on the situation.

[PiCiJi]

Ok, let's start with this.

Originally Posted by amilo3438

EDIT:
Same happens for 2 more files! (The Zone! updated)
--> gauntlet[mindscape_1987](vmax2)(newer)
--> harrier_combat_simulator[mindscape_1986](vmax2)

nightly (is currently building)
This has nothing to do with the drive emulation (relieved), but with uninitialized memory.
This is an annoying problem. As is well known, the authors of some programs did not expect that not every C64 here has the same values after a cold start.
Unfortunately, the pattern is so similar on the same devices that the authors didn't notice it. Of course, if the pattern were purely coincidental, it would have been noticed.
Under Configuration -> memory there are preconfigurations that set special values in the selection boxes above to work for a few problematic programs. There is now a new pre-configuration 1. The other two are now 2 and 3.
This new pre-configuration 1 is the same as VICE's. I guess through many tests this has proven itself and will adopt it.
If a new user uses Denise, i.e. there is no config yet, the values under pre-configuration 1 would be active.

[amilo3438]

Yep, new configuration/memory/pre-configured 1 fix the problem. Great.

[amilo3438]

Originally Posted by amilo3438

Note: I don't want to downplay the importance of nibconv when converting, as even the remaster utility doesn't handle all situations! (like for example with pacmania conversion) Both play their role depending on the situation.

And here is another example: wicked[electric_dreams_1989].nbz

When converted using the remaster utility it stops at track 38! But when converted with nibconv with -r option, in any possible alignment, it works fine in any emu!


Other example is: andy_capp[mirrorsoft_1987)(pal).nbz

When converted using the remaster utility (latest version), it works in any emu! But when converted with nibconv with the -r option, in any possible alignment, it might work, but in which emu, it depends on the alignment used!
(-an -r = denise, -aa -r = vice, -as -r = micro64 ... sometimes it takes several tries in vice)

[amilo3438]

On the example from the post #62: black_tiger[us_gold_1989](pal)(!)

Originally Posted by amilo3438

Now works with denise disk wobble 0.5 rpm if converted with other alignment!
----------------------------------------------------------------------------
black_tiger[us_gold_1989](pal)(!)...................... -a0, -aa, -an, -aw 
----------------------------------------------------------------------------

When converted with nibconv in all alignments + remaster... (In The Zone!)

- In vice and micro64 works all versions!
- In hoxs64 work only remaster version!
- In denise works -a0, -aa, -an, -aw + remaster versions if drag'n'drop on the power button, but -ag, -as versions works only if loaded manually with LOAD"*",8,1 !?
(Note: -a0, -aa, -an, -aw + remaster work also in denise if load manually with LOAD"*",8,1)

PS. A similar test could be applied to the other problematic ones listed in post #62!

You remember post #1?

Originally Posted by amilo3438

Here is the list of problems found while testing some of protected disks in Denise C64 mode:
01) out_run_s1[us_gold_1988](pal)(!) ... loading problem!? (fine in vice)

Originally Posted by PiCiJi

- out_run_s1[us_gold_1988](pal)(!)
shouldn't work or only sometimes on real device(Denise, Ulimate II+)
try it multiple times for success

Well, same case as above... it will work in denise only if loaded manually with LOAD"*",8,1 !!!

EDIT: The conclusion of this would be...
If something works in vice with d'n'd to the screen but not in denise with d'n'd to power button, then try to load it manually with LOAD"*",8,1 = there is a high probability that it will then work in Denise too!

Now the question arises: What causes the difference that in one case it works ie when it is manually started and in the other it does not ie d'n'd on the power button!?
(I guess there might be some minor differences in "disk wobble" between the two modes!?)

[PiCiJi]

nightly
fixed:
out_run_s1[us_gold_1988](pal)(!)
black tiger
and probably more

One cycle is too many in Denise G64 emulation. This is only noticeable when the image is very close to the tolerance limit.
With your many test results, you put me on the right direction. Before that, I would be sure this are broken G64. Possibly they are, because they are still too close to the tolerance limit. However, it should be possible to load the programs (at least some of them) with 300 RPM with or without a small wobble.

Originally Posted by amilo3438

Now the question arises: What causes the difference that in one case it works ie when it is manually started and in the other it does not ie d'n'd on the power button!?

Different positions of the write/read head. If it is very close to the limit, it can be enough for the program not to load. The change in this nightly should now make it so far from the limit that this should now work, doesnt matter if D'n'D or loaded from Basic screen.

Please deactivate the mechanics emulation for all further tests. This is only necessary for very few programs.
Motor deceleration: Star Trekking
Stepper Seek Time: inertia drive emu check, Star Trekking 2, Exoneration