Pixel-Accurate N64 Software Video Plugin

[ReyVGM]

Originally Posted by RPGMaster

Doesn't sound like he's using the latest version .

What the friggin' hell?? How could I have missed that release? I've been using version 4 all this time!

[HatCat]

lol. well at least someone's got some catching up to do with testing, though I can't really use that as an excuse for putting off the remaining work to an even newer release.

[ReyVGM]

Alright, I'm testing the display of the 5th version.

There seems to be an issue with the resolution. It is understood that the N64's regular resolution is 320x240. For the sake of simplicity I'm going to drop the "320x" part and just mention "240".

Anyway, on version 4 of the plugin, the window display would be 240. 237 of that would be actual graphics, and 3 would be a black bar at the bottom.
In version 5, if I select "DP Framebuffer", which I assume is supposed to make the window the same size as the native resolution, then I get a window size of 237 including the 3 for the black bars.

So somewhere along the line, the plugin is eating 3 lines of pixels.

Take a look at the MK64 title screen. Look at the name, and at the "W" in Wario's hat. (Image is a resized gif, so it will look worse than normal).






And I have a question. Have there been any advances/changes in how the plugin handles the filters? Because I compared a title screen at 640x480 and the plugin v5 seems to have more filtering now.

I also tested Goldeneye with the "DP Framebuffer". Amazing, the screen actually resizes itself automatically right before my eyes! Thank you! :P

Anyway, for some reason the text still looks wrong even if the window is showing the "correct" resolution and even if I disable the filters. So there must be something wrong with the resolution. Either the native resolution is not matching the size of the window, or vice versa.

[HatCat]

"DP frame buffer" for "Screen Resolution Formula" in plugin config looks like this in my current DLL:



Though since you brought it up, I think I remember changing the frame buffer height guesstimation-and-round formula to give it a better chance of using 240 pixels instead of 239 or 241 or something (no correct formula for it really...all you can do is generalize game settings and FP round it), so that might explain it if it looks any better in my to-be-released version. I have not kept a very good mental changelog at all of all the things I've done since the DLL posted to this thread.

Originally Posted by ReyVGM

And I have a question. Have there been any advances/changes in how the plugin handles the filters? Because I compared a title screen at 640x480 and the plugin v5 seems to have more filtering now.

No, game at 640x480 on this build should look the same as it did in 640x480 at the previous build you were using.

Of course VI filters with gamma and dither always have pseudo-random results...pixels can be +/- 1 ore more RGB channel points with random variations to them, so you do not necessarily get the same exact image twice, even if the frame buffer native image has not changed at all. Still, that being said, no changes to VI filtering were made.

However I might have flipped whether "Force NN" or "GPU-accelerated scaling" was on/off by default which can use bilinear filtering to remove the pixilated effect (you should really be turning Force NN on if you're going to disable VI DAC filters), and I think I corrected clamping over mirroring of the OpenGL screen texture which might have also improved the result on my current dll.

Anyway, for some reason the text still looks wrong even if the window is showing the "correct" resolution and even if I disable the filters. So there must be something wrong with the resolution. Either the native resolution is not matching the size of the window, or vice versa.

Neither. It's that the native N64 resolution isn't matching your preference of GoldenEye's frame buffer resolution
640x480 != 440x330, and 640 divided by 440 is not a very balanced floating-point ratio to multiply the GL scale to. The result is still hardware-accurate.

[ReyVGM]

Originally Posted by HatCat

"DP frame buffer" for "Screen Resolution Formula" in plugin config looks like this in my current DLL:



Though since you brought it up, I think I remember changing the frame buffer height guesstimation-and-round formula to give it a better chance of using 240 pixels instead of 239 or 241 or something (no correct formula for it really...all you can do is generalize game settings and FP round it), so that might explain it if it looks any better in my to-be-released version. I have not kept a very good mental changelog at all of all the things I've done since the DLL posted to this thread.

Ok yes, that image looks just like version 4. So I guess it's a problem with version 5. No problem, I'll wait for version 6 then.

No, game at 640x480 on this build should look the same as it did in 640x480 at the previous build you were using.

Of course VI filters with gamma and dither always have pseudo-random results...pixels can be +/- 1 ore more RGB channel points with random variations to them, so you do not necessarily get the same exact image twice, even if the frame buffer native image has not changed at all. Still, that being said, no changes to VI filtering were made.

I understand the image will not be necessarily the same, but even if the filtering is not always the same, it is at least consistently very similar.

Take a look at this. Check the text on both images, one looks considerably more filtered than the other. On version 4 I have filters on. On version 5 I have "VI register layout" on, "Disable VI filters" off and "GPU scaling" on.

However I might have flipped whether "Force NN" or "GPU-accelerated scaling" was on/off by default which can use bilinear filtering to remove the pixilated effect (you should really be turning Force NN on if you're going to disable VI DAC filters), and I think I corrected clamping over mirroring of the OpenGL screen texture which might have also improved the result on my current dll.

Sorry, I don't know what "NN" means and I don't see any option over here for that. And by the way, I never turn filters off :P

Neither. It's that the native N64 resolution isn't matching your preference of GoldenEye's frame buffer resolution
640x480 != 440x330, and 640 divided by 440 is not a very balanced floating-point ratio to multiply the GL scale to. The result is still hardware-accurate.

You said the framebuffer has one resolution (which changes depending on the game), and the VI has a set resolution of 640x480. I understand that part.

But if Goldeneye is throwing a non-standard resolution, and the window is resizing accordingly, and I have the VI filters OFF. Then shouldn't I get the text/graphics displayed correctly then?
I mean, if I take a native screenshot with the 440x330 resolution and the text looks perfect in that image, then shouldn't the resized 440x330 window, and with the VI filters OFF also look perfect?
After all, the native resolution is not being downscaled or upscaled to fit the window (like older versions), so it should be showing on the screen the same thing I'm seeing on the native screenshot.

But then again, that might be showing up correctly in your unreleased version.

Side note: Why not change "Disable VI DAC Filters" to "Enable VI DAC Filters"? All the other options are to turn something on, only that one is to turn something off.

[ReyVGM]

By the way, thanks for the GUI RPGMaster

I don't know if you have implemented this already, but is it possible to include a short explanation of what each option does? Maybe as a popup or whatever.

For example, I really don't know what "DP frame buffer" means, I figured it out when I noticed the window resized itself automatically to fit the game's internal resolution. But other people might not even pick that option because they wouldn't know what it means.

[HatCat]

Originally Posted by ReyVGM

"GPU-accelerated scaling"*, then.

Always forget the way the build in this thread looks. It was changed to "Force NN" in the dll on my disk. Back then I was a bit too zealous about introducing people to the alternative TV output effect which resembled bilinear filtering, which somewhat complicated the analysis of the pixels in a pixel-accurate image.

So leave it on if you don't mind smoothening out the low-res look in games; turn it off if it bothers you.

I will say that the bilinear filter in v5 is shit. I was using glDrawPixels back then...it should look better in the next release.

Originally Posted by ReyVGM

You said the framebuffer has one resolution (which changes depending on the game), and the VI has a set resolution of 640x480. I understand that part.

But if Goldeneye is throwing a non-standard resolution, and the window is resizing accordingly, and I have the VI filters OFF. Then shouldn't I get the text/graphics displayed correctly then?
I mean, if I take a native screenshot with the 440x330 resolution and the text looks perfect in that image, then shouldn't the resized 440x330 window, and with the VI filters OFF also look perfect?
After all, the native resolution is not being downscaled or upscaled to fit the window (like older versions), so it should be showing on the screen the same thing I'm seeing on the native screenshot.

I guess it depends on your perspective of what really are the "VI filters".

Part of what the VI does is scale the smaller 320x240 (or in this case 440x330) frame buffer up to fit the 640x480 DAC resolution, through its own pixel-exact specification. So if I use OpenGL's texture scaling feature to fit the 440x330 screen texture into the current Project64 window size (maybe 640x480, maybe also 440x330 as well) then the result is inconsistent with real hardware.

However, since disabling VI filters itself is also inconsistent with real hardware, I kind of understand your request. It's tricky, but I will carefully try to look into what you said provided that VI filters are turned off...maybe by "VI filters = off" this should also refer to disabling the VI's pixel-accurate scaling algorithm, which does seem to be exactly what you're looking for in GoldenEye's case.

Originally Posted by ReyVGM

Side note: Why not change "Disable VI DAC Filters" to "Enable VI DAC Filters"? All the other options are to turn something on, only that one is to turn something off.

Because the default setting is to keep them enabled.

If I made the default settings to have VI filters emulation bypassed or omitted, that would be encouraging people to see games without complete emulation of the VI's pixel-accurate filtering, which would be less accurate.

I'm sure your next question after this, then, is, "In that case, why not force the checkbox to be checked by default AND rename it from Disable to Enable VI DAC Filters?" The answer is that I could, but I don't really feel like it. I like 0's more than 1's with defaults. Just seems more intuitive with systems programming.

Originally Posted by ReyVGM

By the way, thanks for the GUI RPGMaster

I don't know if you have implemented this already, but is it possible to include a short explanation of what each option does? Maybe as a popup or whatever.

If that excites him, I wouldn't stop him.
I can't remember how he felt about that. I **think** he was hesitant to learn tooltips.

The problem with tooltips and the like is that we're also merging the size of English language documentation with the program binary...when I think documentation should be kept separate. So I wouldn't exactly encourage him to add it, but if he wants, I guess it would look kind of cool for some people.

In the event he's not interested, you think I should make some HTML or whatever to document the advanced specifics of each option?

Originally Posted by ReyVGM

For example, I really don't know what "DP frame buffer" means, I figured it out when I noticed the window resized itself automatically to fit the game's internal resolution. But other people might not even pick that option because they wouldn't know what it means.

The accurate setting is "VI register layout" which is the default, and it should be.

People don't exactly "need" to know about the "DP frame buffer size" option, but I'm sure if they disagree with the 100% accuracy look of always playing in a 640x480 window every time, they will encounter it and see that for themselves on their own. (The "Custom" option right below that makes this easier.)

[RPGMaster]

Originally Posted by ReyVGM

By the way, thanks for the GUI RPGMaster

You're welcome .

Originally Posted by ReyVGM

I don't know if you have implemented this already, but is it possible to include a short explanation of what each option does? Maybe as a popup or whatever.

Idk, I'll have to think about it. I have a habit of procrastinating. I may consider it, but can't promise anything.

Originally Posted by HatCat

I **think** he was hesitant to learn tooltips.

It just seemed like too much hassle for what it's worth.

Originally Posted by HatCat

The problem with tooltips and the like is that we're also merging the size of English language documentation with the program binary...when I think documentation should be kept separate. So I wouldn't exactly encourage him to add it, but if he wants, I guess it would look kind of cool for some people.

In the event he's not interested, you think I should make some HTML or whatever to document the advanced specifics of each option?

I figured you'd prefer it not being in the binary anyway. I think having an HTML would be simpler.

[ReyVGM]

Originally Posted by HatCat

"GPU-accelerated scaling"*, then.

Always forget the way the build in this thread looks. It was changed to "Force NN" in the dll on my disk. Back then I was a bit too zealous about introducing people to the alternative TV output effect which resembled bilinear filtering, which somewhat complicated the analysis of the pixels in a pixel-accurate image.

So leave it on if you don't mind smoothening out the low-res look in games; turn it off if it bothers you.

Ok so the "Force NN" adds filtering, much like old TVs did?
Yeah I wouldn't want that. The only filters I want are the ones the N64 hardware had by default.

And guess what? I retook that Hybrid Heaven title screen shot again with the "NN" stuff disabled (and VI enabled), and it looks exactly the same as version 4. And I mean, pixel perfect accurate 100% bing bang boom. So I guess the N64 filters behavior is not so "random" after all.

Part of what the VI does is scale the smaller 320x240 (or in this case 440x330) frame buffer up to fit the 640x480 DAC resolution, through its own pixel-exact specification. So if I use OpenGL's texture scaling feature to fit the 440x330 screen texture into the current Project64 window size (maybe 640x480, maybe also 440x330 as well) then the result is inconsistent with real hardware.

However, since disabling VI filters itself is also inconsistent with real hardware, I kind of understand your request. It's tricky, but I will carefully try to look into what you said provided that VI filters are turned off...maybe by "VI filters = off" this should also refer to disabling the VI's pixel-accurate scaling algorithm, which does seem to be exactly what you're looking for in GoldenEye's case.

I understand that it's tricky. Because on real hardware, even if a game is 320x240, it still passes through the VI before you see it on the TV. And like you said, disabling the VI is not something you, the player, could do on the hardware.

But, you once said that in your opinion the "real" N64 graphics are the ones that come out the framebuffer, i.e.: the non-filtered ones. So, with the VI disabled, using a non-standard resolution window, then the graphics should look like how the native screenshot does: perfectly.

However, and I might be asking something impossible... but you mention that what I am asking is possible as long as the VI filters are off, but what if you achieve that and have Goldeneye looking all perfect in that non-standard resolution window. What if I turned on the VI and the filters themselves downscaled or upscaled to fit the screen? Wouldn't that solve the issue?

You once said that the VI filters are not always 'pixel accurate' anyways, so I guess applying 440x330 filtering wouldn't be that much different from 640x480 filtering. Or is that bordering on 'hacking'?

I mean, as long as you're adding additional things like that "Force NN", then why not add this 'feature' too? Assuming it is possible of course. Don't get angry, I'm just asking :P


RPGMaster

If that excites him, I wouldn't stop him.
I can't remember how he felt about that. I **think** he was hesitant to learn tooltips.

The problem with tooltips and the like is that we're also merging the size of English language documentation with the program binary...when I think documentation should be kept separate. So I wouldn't exactly encourage him to add it, but if he wants, I guess it would look kind of cool for some people.

Well, it doesn't have to be tooltips. He could add an extra bit of space to the bottom of the GUI screen, and whenever someone clicks an option, the explanation appears in that empty space. This is what the cheat window does if you click a cheat that has a Note.

In the event he's not interested, you think I should make some HTML or whatever to document the advanced specifics of each option?

Yeah sure. It could be included on the readme.txt too.

People don't exactly "need" to know about the "DP frame buffer size" option, but I'm sure if they disagree with the 100% accuracy look of always playing in a 640x480 window every time, they will encounter it and see that for themselves on their own. (The "Custom" option right below that makes this easier.)

Regular people will probably not care or learn what that option does. But people that are looking for the same thing as I am (accurate screens), will be very interested to know what it does right from the get-go. Remember not all games have different resolutions while playing, thus, they might not notice it right away. So if there's an explanation of what it does, somewhere, that would be great.

[HatCat]

Originally Posted by ReyVGM

Ok so the "Force NN" adds filtering, much like old TVs did?

Actually renamed it from "GPU-accelerated scaling" to refer to the opposite of "Force NN" due to a debate over which was more accurate and whether I should default it to on/off.

So "Force NN" enabled = disabled "GPU-accelerated scaling".
The NN means "nearest neighbor". The upscale is point-sampled and pixilated.

And it doesn't really add any filtering; it's just one of various filtering methods.

Originally Posted by ReyVGM

Yeah I wouldn't want that. The only filters I want are the ones the N64 hardware had by default.

Sure, then you most likely would want to always have GPU-acc turned off (or, in the next release, "Force NN" turned on).

But I want to explain as well, that the N64 hardware does not go beyond taking Super Mario 64, and scaling it up thru the VI from 320x240 to 640x240 (and of course adding the filters).

Once it's at 640x240, how it is stretched to 640x480 is not up to the N64...it's a different visual effect to have either bilinear or NN/pixilation. It's just that if you're neutral and really don't care one way or the other, then you would prefer NN so that's what you want. I personally alternate between the two every now and then...it isn't too bad to hide the low-res look a little by using the effect that many TV's have on the image, even if it does mostly just blur it I think it smoothens out the jaggedness in some games better than native hi-res would sometimes. So I think calling it "blur" is somewhat pessimistic though accurate.

Observe my awesome ASCII artwork of a 320x240 screenshot of Mario64:

A C E G
I K M O
Q S U W

granted it's 4x3 characters, not 320x240 pixels, but my aspect ratio still matches.

When the VI upscales it to 640x240 and adds things like anti-aliasing, we get something more like this:

A B C D E F G H
I J K L M N O P
Q R S T U V W X

But how should the FINAL 640x480 image look?

Should it look like this:

A B C D E F G H
A B C D E F G H
I J K L M N O P
I J K L M N O P
Q R S T U V W X
Q R S T U V W X

... or maybe something like this ...

A B C D E F G H
a b c d e f g h
I J K L M N O P
i j k l m n o p
Q R S T U V W X
q r s t u v w x

... where the lowercase letters represent the unison of themselves and their uppercase counterparts as part of an image transformation done by future interference?

As far as angrylion and myself are aware, neither final result is dictated by N64 hardware, though marshallh himself prefers the former with the point-sampled, standard pixel resize that you prefer. Naturally most people who are only interested in what the N64 by itself does would agree. On some of angrylion's video cards, though, the DirectDraw code in his original plugin code produces the latter with a bilinear-ish sort of effect. This is because scaled blitting has GPU-defined/undefined behavior in Microsoft DirectDraw which is one of the foremost reasons that I have rewritten the plugin to OpenGL.

Hope this clears up a thing or two.

Originally Posted by ReyVGM

And guess what? I retook that Hybrid Heaven title screen shot again with the "NN" stuff disabled (and VI enabled), and it looks exactly the same as version 4. And I mean, pixel perfect accurate 100% bing bang boom. So I guess the N64 filters behavior is not so "random" after all.

If you use an eyedrop tool (or w/e it's called) in GIMP/Photoshop to check the exact RGB value of some pixels you may find that they have a very small deviation constantly changing each vertical interrupt.

If you are not finding this...it is because the interfering VI filters are not currently enabled within the ROM itself (n64 can choose which, if any, VI filters to enable). Only some of the VI filters yield pseudo-random results; the others are pixel-exact every time. Just depends the current part of the game you play.

Originally Posted by ReyVGM

But, you once said that in your opinion the "real" N64 graphics are the ones that come out the framebuffer, i.e.: the non-filtered ones.

They are the RDP's "real N64 graphics", but not the VI's "real" N64 graphics.
The RDP is the video card, and the VI is the transmitter.

Which one is real N64 gfx, the RDP's result or the VI's result, is somewhat debatable, though if there is a correct answer, then it's the VI's result after filtering. Some games don't even execute on the RDP, which means the graphics plugin has absolutely nothing at all to emulate except only the VI. Correct emulation of all VI registers invariably requires this filtering.

Originally Posted by ReyVGM

However, and I might be asking something impossible... but you mention that what I am asking is possible as long as the VI filters are off, but what if you achieve that and have Goldeneye looking all perfect in that non-standard resolution window. What if I turned on the VI and the filters themselves downscaled or upscaled to fit the screen? Wouldn't that solve the issue?

Once complete VI emulation is restored through the settings to the plugin, it must do all future DAC with the scaling done by the VI, first, before the filters (put bluntly, at least).

The filters themselves can never downscale or upscale...they are simply applied to each pixel in the 640x480 result. Since the method that OpenGL uses to downscale it back to 440x330 again is nothing at all like what real N64 does to scale it up FROM that to 640x480, it is something of a redundant mess really. It would only make sense to hack around it when VI filters are disabled through the plugin. Calling it "VI filtering" after doing what you said (using OpenGL by itself and not scaling the image up to 640x480 in the first place) would be lying. Those are not VI filters because SGI designed them to apply to a finished 640x480 result, in effect.

The only reason these filters even exist is because of television screens or CRT differences. I somewhat doubt that any of these VI filters would even exist, if all the Nintendo 64 ever did as a gaming system was output graphics to a modern flatscreen monitor. So it should be more than satisfactory to have this hack done only when VI filters are being bypassed, not when they are not.

Originally Posted by ReyVGM

You once said that the VI filters are not always 'pixel accurate' anyways, so I guess applying 440x330 filtering wouldn't be that much different from 640x480 filtering. Or is that bordering on 'hacking'?

I said only that they are not always pixel-exact. Some (but not all) of the filters can have some random amount of deviation.

Basically, think JPEG (with a very minimal amount of compression, not typical Internet crappy quality JPEG). It's not totally out of the question for keeping some VI-filtered screenshots in.

And the scaling thing between 440x330 and 640x480 does way many more changes beyond just the type of filter being done...

Originally Posted by ReyVGM

I mean, as long as you're adding additional things like that "Force NN", then why not add this 'feature' too? Assuming it is possible of course. Don't get angry, I'm just asking :P

I will look into it.

Thinking about it, you are probably right about it as long as VI filters are off. So I will see how much friction I get when trying to implement it.

Originally Posted by ReyVGM

Regular people will probably not care or learn what that option does. But people that are looking for the same thing as I am (accurate screens), will be very interested to know what it does right from the get-go. Remember not all games have different resolutions while playing, so they might not notice it right away.

No, but they will notice it eventually.

They will see the pixilation of Mario64 being stretched from 320x240 and look under the "Options" menu, and see "Configure Graphics Plugin" isn't grayed out. They will see "Screen Resolution Formula" and suppose that it does just what it implies...changes the resolution for deducing the resolution of the screen. If they are at all curious to see the kind of result you're after, it's almost certain that they will find out about it sooner rather than later.