Zelda Majora's Mask FLASHRAM Save Editor

[buckchow]

Originally Posted by HatCat

Only problem is that the location of the 16-bit checksum is different when saved/accessed by the Japanese Zelda Majora's Mask than the USA/PAL release.

Has anyone been able to convert between JP and US/PAL saves? I've been trying to do a JP -> US conversion without luck so far.

I've done the following:
- Use zs to change the name so that valid characters are used for the US/PAL versions
- Copied 8kb block from the start of the JP save to a US save file (of the same endianess)
- Duplicated that 8kb at 0x2000 to complete save slot 0 (File 1)
- Run zs to update the checksum

The end result is always that File 1 appears as unused. File 2 always shows up fine, so I know the .fla endianess is correct in PJ64.

Any help would be appreciated.

Thanks!

[HatCat]

Originally Posted by buckchow

Has anyone been able to convert between JP and US/PAL saves? I've been trying to do a JP -> US conversion without luck so far.

I've done the following:
- Use zs to change the name so that valid characters are used for the US/PAL versions

This is optional. If you leave the file names stored under the code page that old Japanese versions interpret names through, then the US/PAL versions will still recognize the save blocks as valid. (The file names will just have some gibberish in them.)

Though, if you've read the doc, you know that digits 0 through 9 share the same code points in the US/PAL code page as with the JP one, so you could just keep the file name under stuff like "13371337" or whatever for consistency. I think it has to be the full 8 characters wide, though, because the spaces ' ' used to null-terminate file names shorter than 8 characters has a different code point in JP versions.

Originally Posted by buckchow

- Copied 8kb block from the start of the JP save to a US save file (of the same endianess)

This should work. Though I would just rename/copy the JP save file to the file name Project64 looks for when loading the US/PAL versions. Ideally it should be possible to convert the "File 1" (File 2 looks like it should work just as well.) portion of any JP version save file to US/PAL without even having to use a hex editor or copy any blocks.

Originally Posted by buckchow

- Duplicated that 8kb at 0x2000 to complete save slot 0 (File 1)

This is also optional. The new file format will use &file[0x2000] as a 8-KiB backup of &file[0x0000] if &file[0x0000] fails to validate. The 8-KiB block starting at &file[0x2000] is completely ignored if we did everything right to get &file[0x0000] working.

Originally Posted by buckchow

- Run zs to update the checksum

Should be the only step needed I believe.

Originally Posted by buckchow

The end result is always that File 1 appears as unused. File 2 always shows up fine, so I know the .fla endianess is correct in PJ64.

Any help would be appreciated.

Thanks!

Yeah, endianness should never be a factor. Actually, the trick is that the .fla endianness is always wrong in PJ64, but my utility just analyzes what endian the save file is already stored in and just maintains that byte order. The only possible way you should ever have a mix-up between .fla endians is if you're getting the save file off of a real Nintendo 64 cartridge save chip, which is going to have the correct endian that Project64 never uses. (Again, my utility will detect and maintain storage in this byte order, but it will be incompatible with Project64 only in cases like that.)

If it's not working, can I have a sample .fla save file that you are using and see if I can get it to work myself? Maybe there is some detail I accidentally left unchecked in the editor.

[buckchow]

Originally Posted by HatCat

...
If it's not working, can I have a sample .fla save file that you are using and see if I can get it to work myself? Maybe there is some detail I accidentally left unchecked in the editor.

Thank you very much for the very detailed response.

The first thing I tried (and forgot to mention) was just copying the save to ZELDA MAJORA'S MASK.fla, swapping endianess, and doing a CRC fix. I just tried it again since you mentioned it, but it's not working.

For reference, I am working with saves from carts. ED64-Saveswap.exe as well as my own swapping tool both have the same output with the several games I've tested, and all of the saves work in emulators.

I've attached the unmodified cart save and renamed the .fla for use with the US version of the game.

I very much hope that you'll have more success with this save file than I did. My old US cart save was lost several years ago, so getting my old JP cart save converted would be the next best thing.

Thanks again.

[HatCat]

Aha! Got it working. Sorry about the wait.

One problem here may or may not have been that you were using the latest built version of my save editor that I had released a binary download for, but there was an issue someone reported with VC save files that I committed a fix to since the time I did the binary release. (If you compiled my utility from the latest source, then this does not apply and should be disregarded.)

That aside, I found that your save file wasn't even loading on the Japanese version of Zelda MM (never mind the US or PAL versions) to begin with, after using my save editor to change the byte order from N64-native to 32-bit LE. It seems to have been a long time since I've actually tested updating the checksum correctly for JP saves, so I as of tonight (well this morning since it's 2:30 AM here) have committed a couple fixes to the save editor.

https://github.com/cxd4/zs-flash/com...e45ad1a6c33cee
and
https://github.com/cxd4/zs-flash/com...464bf2f5a74fb5

In the process, I added a -J (true/false) switch that tells the save editor whether you want to format the resulting save file to be loadable with ROM versions using the old save format (JP 1.0, JP 1.1) or the new save format (US, PAL 1.x, JP collector's edition / GC Version, and everything else) that supports owl-statue-saving.

So here's how I got your attached .fla file to work (I renamed your fla to "zelda.fla" to keep my typing down):

# All in one command:
zs zelda.fla -J true "-=" 3

# Convert the byte order from MIPS-native endian to Project64's endianness.
zs zelda.fla "-=" 3
# Keep the checksum, Pictograph Box, etc. all accessed in the old save format.
zs zelda.fla -J true

After that I just renamed zelda.fla to what Mupen64 for Linux looks for with the US version of the game. (Note that this is different from the file name Project64 uses, so you'll have to rename this file from what I got Mupen64 to load since I don't use Project64 or Windows much these days.)

The resulting file is attached to this post.

BTW, one last thing. Your save file seems to have a Pictograph Box snapshot stored to it. This suggests you did not save your game using the Song of Time previously (nor did you use an owl statue since the JP versions don't support saving by owl statue). It doesn't look like you saved the game by beating the final boss either (acts as if you played the Song of Time) since you don't even have Twinmold's remains, so either you used cheats to beat the final boss or you did indeed save using the Song of Time and I must be missing something, because whenever I do that it always wipes the Pictograph Box frame buffer image data.

Anyway the point is, the Pictograph Box picture can't be converted out of your JP save into a US/PAL one due to the different file format (unless I manually construct an owl statue save out of it, in which case then you have space to store it), so I just extracted it from your save file and uploaded it here :

[buckchow]

Originally Posted by HatCat

Aha! Got it working.

Hi! That's great news. Thank you very much for your help with this.

I tested the save, and while it shows up and the game starts there are some problems right away such as my sword not working. Please take a look at the attached screenshots (JP ROM with the original save on the left, US ROM with the converted save on the right).

Aside from the serious inventory problem, I found that changing the name using zs before using the converted save doesn't work. The game is apparently loading the backup save on the first load, so I assume it isn't seeing the main save data as valid after the conversion. If I wait to change the name after the game has overwritten the main data with the backup the name is changed successfully. The inventory is corrupt either way though.

A few details for reference related to various points in your post:
- I was using my own build from the latest source (before your new updates in the past day of course)
- Swapping endianess was done as necessary and I had been fully able to use the save with the JP ROM in PJ64.
- I didn't beat the final boss in that save.
- There was definitely no cheating going on.
- I'm happy to see the Deku guy's picture again

I haven't looked at the updated source yet, but I'm guessing that the new -J option is doing more than just setting "is_old_JAP" to TRUE. Over 7,000 bytes are different between the save you attached and the one produced by using a build from the now-old source with that flag set.

Thanks again for your help with this. Hopefully the inventory problem can be solved without too much trouble.

[HatCat]

Yes, that is to be expected. I know that the mask and item ID numbers are different in the JP version than they are in the US and PAL versions.

https://github.com/cxd4/zs-flash/wiki/item_register

This difference is caused by the fact that the US version removed several "beta" items and legacy cruft from the index database in ROM, which conflicts with offsets the JP version uses. (The new collector's edition / GC Version JP ROM fixes this and uses the table proposed by the US version along with the new save format, just fyi.)

So it's an easy fix:

# usual conversions to get JP N64 save working on US ROM in Project64
zs zelda.fla -J false "-=" 3

# B-button icon assignment:  Gilded Sword (4F in US/PAL)
zs zelda.fla -e 0 4F

# C-button left, down, and right assignments (only C-right needs change)
zs zelda.fla -e 1 01 -e 2 00 -e 3 39

# The entire Masks subscreen (except blanks) needs to be fixed with this.
zs zelda.fla -I M 0 0 3E             -I M 2 0 47 -I M 3 0 45 -I M 4 0 40 -I M 5 0 32
zs zelda.fla -I M 0 1 3A -I M 1 1 46 -I M 2 1 39 -I M 3 1 42 -I M 4 1 48 -I M 5 1 33
zs zelda.fla                         -I M 2 2 37             -I M 4 2 36 -I M 5 2 34
zs zelda.fla -I M 0 3 43             -I M 2 3 3B -I M 3 3 44

# Finally (optional), change Link's name from the Hiragana "AAA" to "Link".
zs zelda.fla -N 152C312E3E3E3E3E

(Hmm...doing this, I realize I should probably update the -I command to read mask and item subscreen offsets supplied by the user as (row horizontal, column vertical) or (y, x), not (x, y)...little bit of a counter-intuitive detail I noticed. This isn't some OpenGL program; it's just a memory array of bytes that the (row, column) coordinate system should fit better.)

Your save file now looks like this (at least on my end):


I probably should have done that for you before uploading the .fla file, sorry. When I saw that I got your save working I was pretty quick to just leave it at that and upload the solution, but I did notice the Bunny Hood appeared then instantly disappeared off Link's head. This happens when player_mask is set to a mask ID that doesn't match what's on any of the 3 C-button mappings. (It was supposed to, but again the C-button IDs you have stored were compatible with the JP version and not US, which we have fixed with the command above.)

Please let me know if you see any other incompatibilities besides the mask ID numbers and the file name code page for naming Link. Those (and owl statue save formatting or lack thereof, of course) have always been the only two differences I had stumbled on between JP and non-JP save files, but maybe you'll see another one.

Originally Posted by buckchow

I haven't looked at the updated source yet, but I'm guessing that the new -J option is doing more than just setting "is_old_JAP" to TRUE. Over 7,000 bytes are different between the save you attached and the one produced by using a build from the now-old source with that flag set.

Correct. Those 7,000+ bytes are coming from the Pictograph Box picture, which is a 5-bit-per-pixel (i.e., I5) 160x112 frame buffer image. 160 px * 112 px * 5/8 fractional byte pixels = 11,200 bytes (which adds up to the difference in end - start offsets for the photo documented in the wiki for the repository).

So because the US and PAL versions cannot store the Pictograph Box in regular Song-of-Time saves like the JP version can (they have to use the second 8-KiB half of the 16-KiB save block as a backup duplication), at most 11,200 bytes should be different as a result of XOR'ing old_JAP_save.

This (consequential of not having support for owl-statue-saving) and the item and mask ID enumerations, are the only two differences I've discovered between old-JP-formatted save files and the newer format.

[buckchow]

Thank you very much again. My perusal through the Wiki several days ago didn't get into many of the individual pages for items in the "game state reference" column, so I was completely unaware that conversion might be necessary for the inventory.

Originally Posted by HatCat

So it's an easy fix:

If you happen to know the save format inside and out, sure thing.

When I have more time I think I'll experiment with adding inventory conversion support to zs based on your item-register details. I have a second JP cart with two more saves on it that aren't mine but it would be fun to get those working in English anyway.

Thank you once again. It doesn't look likely that there will be other major issues now and I'm very happy.

[HatCat]

When I say that it's an easy fix, I mean that the commands are easy to paste in and run, without having to understand the save file format. The hard part was trying to get the save file to validate from JP over to US to begin with. That was the hard fix on my end...checksum bugs are no fun. But with that out of the way, adjusting the inventory item offsets with a few additional commands was straightforward.

Originally Posted by buckchow

My perusal through the Wiki several days ago didn't get into many of the individual pages for items in the "game state reference" column, so I was completely unaware that conversion might be necessary for the inventory.

Yeah, converting save files between the old and new formats (particularly in cases where the user has an older (J) ROM with the save data) was not something I talked much about at all on the index of the Wiki. I kind of left discussion of the differences between the two formats down to just the "sectional allocation" part of the File Format wiki documenting whether owl statue saves play a part, as well as the offsets in item and mask ID codes in the item_register page for people who clicked on that specifically.

It's kind of one of those "bugs" (like seeing the Razor Sword or Fisherman's Pole instead of the Zora Mask or Great Fairy's Mask) that sometimes people intentionally want to happen just to make their save file more unique from common ones, or they're just curious about discovering beta goodies while testing. Take the Circus Leader's Mask for example...most useless out of all 24 masks IMHO, except you can cry while running with it on. I'd probably be fine with whatever replaced it due to taking the raw JP save and loading it on US.

But yeah, the option to fix the masks' IDs is definitely there for those that want/need it. (It's less of a problem when your game is relatively new and doesn't have that many masks to replace or just start over and re-collect again to fix the subscreen that way.) Certainly having your B-button icon set to an unusable item that will crash your game was no fun, but again sometimes people want to test low-level functionality of the game by breaking things on purpose.

Originally Posted by buckchow

When I have more time I think I'll experiment with adding inventory conversion support to zs based on your item-register details. I have a second JP cart with two more saves on it that aren't mine but it would be fun to get those working in English anyway.

You're certainly welcome to give it a shot.

Automating the conversion between the old and new formats is something I have given a little thought to in the past. I have thought about it...has a few quirks that made me feel discouraged from decisively putting that feature in.

There are ways to reliably guesstimate whether a save was written by a JP ROM or not, but there is no definitive way to prove it was absolutely beyond the shadow of a doubt. Yes you can examine the Masks sub-screen, see if all the masks collected are 0x1C larger than the mask ID for US ROMs. This technically could be a coincidence, though...people can use the save editor to put any ID they want in any mask's position, even items like Bombchu or Fierce Deity Link's sword.

The most reliable estimation is probably by checksum location. Compute the checksum and see if it's stored at the JP ROM's preferred offset or the new format's offset. Although, it is possible to have both offsets store 2 correct checksums in an attempt to keep the save file bi-compatible with all ROM versions.... So again, no definitive logic here.

So yeah, you can automate the conversion of other things besides the checksum. Welcome to implement it...it takes some being careful with though; I would be sure to have the automation optional in case the user wants to preserve custom (but inappropriate) IDs for mask and item positions they were testing with for beta study.

Also, I don't know for 100% sure that Link's game name and the Masks subscreen (Items too with certain bottled items, but Song of Time saves don't preserve those) are the only 2 things you'd need to implement in your automation of the JP-to-US conversion. They happen to be the only 2 things I've ever known about to be different in a JP ROM, but maybe in your testing you'll happen upon others. That's probably part of why I was too lazy to attempt a preliminary implementation of automating the save file conversion.

[buckchow]

Originally Posted by HatCat

Also, I don't know for 100% sure that Link's game name and the Masks subscreen (Items too with certain bottled items, but Song of Time saves don't preserve those) are the only 2 things you'd need to implement in your automation of the JP-to-US conversion. They happen to be the only 2 things I've ever known about to be different in a JP ROM, but maybe in your testing you'll happen upon others. That's probably part of why I was too lazy to attempt a preliminary implementation of automating the save file conversion.

I just plan to add a old -> new conversion option that fixes items/masks using what you've documented. I'm not worried about if a few behind-the-scenes things have been missed since I won't conceivably ever notice.

I don't intent to try to automatically detect the save type for conversion. I know where my saves are from and where they're going.

Thank you again for all of your time and help.

[HatCat]

Sure thing, not a problem.

And yeah, if you're adding this feature in just for yourself then that makes it a little easier to implement. One good thing about open-source.

The masks are easy btw. As you'll probably find from the page, it's just if (*mask_position == 0xFF) { /* empty slot */ } else subtract 0x1C from the old/JP system to get the mask ID for the new system. So just subtract 0x1C from every Mask subscreen entry unless it's blank or 0xFF.