Pixel-Accurate N64 Software Video Plugin

[GPDP]

The "black lines" are merely a result of only the odd or even lines being used every frame. They are not inserted in any way. They are literally just unused space in the raster.

The reason they do not appear on LCD screens is because of the fact that they are fixed-pixel displays and do not operate in terms of lines like CRTs do. What they typically do with 240p output (if they understand or expect it at all) is they line-double it, scale it up to their native resolution through some kind of algorithm (I suspect they typically use bilinear interpolation, but I haven't looked into it all that thoroughly), and just for shits and giggles, run it through the deinterlacer anyway because they are dumb and treat it like it's 480i.

[HatCat]

Yeah I know that. If it was just drawing black "scanlines" over the would-be actual scanlines of a 480p/480i then they wouldn't actually be the void gaps of absence in-between the even/odd scanlines constituting the benefit of sacrificing pitch for performance/throughput while maintaining the same height of the TV display.

Even I'm not so fundamentally naive as to ponder the possibility that the TV just draws black scanlines over an image, because that would only succeed in costing not only performance but also picture information.

Anyway, I'm finishing up the letterboxing implementation. I don't like letterboxing, but I don't like stretching either and pretty much there's no good solution to my tastes other than to just have a 4:3 monitor. I'll probably add some option in a later version about stretching it to fill the whole TV screen...could this be pixel-accurate for some games? Or do they all use letter-boxing to the aspect ratio defined by Xscale/Yscale and not the TV monitor. IDK really I haven't paid attention to most newer TVs, been a while since I played on a real N64 on any TV really.

[ReyVGM]

I know it's never going to look the same on an LCD, but it's not supposed to. It's supposed to look how the N64 originally output its graphics. In other words, pixel-acurate emulation is the goal, not how it looked on a CRT.

[MarathonMan]

Originally Posted by GPDP

Let's get some terminology straight.

240p/Non-interlaced mode = ~240 lines give or take = only the odd or even lines are displayed, with black gaps between lines (i.e. what is commonly and erroneously referred to as "scanlines").

480i/Interlaced mode = ~480 lines give or take = every frame only shows half the lines, alternating between odd and even fields.

....

So why interlacing? Bandwidth and hardware concerns, basically. In NTSC land, old displays only synced up to 15.7 KHz horizontally and at 60Hz vertically. This only allows ~240 lines to be displayed per frame. To achieve a higher resolution, this necessitated interlacing, and letting the human brain do the rest. However, most consoles were too weak to output in 480i and still achieve playable framerates, so until the Dreamcast arrived, the vast majority of console games were in 240p.

Thank you. I've been trying to understand all that for awhile and that was a great concise write-up.

Originally Posted by ReyVGM

I know it's never going to look the same on an LCD, but it's not supposed to. It's supposed to look how the N64 originally output its graphics. In other words, pixel-acurate emulation is the goal, not how it looked on a CRT.

marshallh/retroactive is designing a HDMI converter that attaches directly to the RCP pins, so pixel-accuracy will soon be verifiable as all that will need to be done is comparing binary data. As for the CRT approach, I'd think it's best to stick to a pixel-accurate VI implementation and then apply filters on top of that to mimic CRT effects (this is how higan does it, no?)

[HatCat]

Originally Posted by ReyVGM

I know it's never going to look the same on an LCD, but it's not supposed to. It's supposed to look how the N64 originally output its graphics. In other words, pixel-acurate emulation is the goal, not how it looked on a CRT.

It's fine. I'm not interested in implementing TV-specific filters (we already have monitor screens to handle that for us at the moment I think) either for CRT or LCD; I think he's just illustrating a conclusion that there cannot be one fixed, 640x480 resolution that the VI outputs to the TV in every single case. It's something I'll have to ask about, but it doesn't affect what I'm doing before the release.

[GPDP]

From what I have seen on my tests, games with smaller resolutions, such as DK64 and Star Fox 64, do letterbox on real hardware, but back in the day this was ok because most TV sets had considerable overscan area, and most letterboxing would have fallen within that area, so it was typically not seen.

[HatCat]

I see. So stretching instead of letterboxing would probably just be an option of user preference...easier for me to implement tho.

Well here is currently the problem I am facing with the final letterbox implementation.

    if (is_full_screen)
    {
        get_screen_size(dimensions);

        aspect_ratio_FB = (GLfloat)(hres)/2.f / (GLfloat)(vres);
        aspect_ratio_PC = (GLfloat)dimensions[0] / (GLfloat)dimensions[1];

So in Mario64's case, we know that it's 320x237.
N64 FB aspect ratio = 320. / 237.

The user's current PC monitor aspect ratio is easily calculated (I wrote my own get_screen_size function for this.) and in my case is 1024x768, so aspect_ratio_PC = 1024. / 768. = +4:3

Now here's the problem. In case people haven't noticed, for proper scaling what is done is there are 3 inactive video lines (just black pixel row gaps) below the 237 active scanlines, so we still have an effective 320x240 rendering window canvas which could easily be multiplied into 640x480 etc. with GL calls.

I have sufficient information from the VI registers to tell me that the viewport width should be 320, but not that the height should be 240...I only know that the active FB video is 237 tall. So 320/237 != 1024/768...I can try rounding the 237 up to 240 but this would not work for games with unique aspect ratios like Star Wars and Resident Evil 2. So I am currently investigating possible patterns to interpret the VI registers in a way to deduce the canvas height. (I had one trick going for RE2 successfully! But with Mario it fails, so more investigation is needed for perfect letterboxing.)

That aside though, the rest of the implementation is very easy to do. A simple average formula "1/2 * (m + n)" for the bounding endpoints gives the amount of space to yield on both the left and right of the GL viewport for letterbox centering.

[HatCat]

Letterbox implementation was successful. My approach was to calculate whether the FB res is wider than it is tall, or taller than it is wide. So if its shorter dimension is the width (like with RE2 sometimes), then use the fact that there is more pixel space in the height dimension to calculate letterboxing around the top and bottom of the VI display.

    sync |= (hres != hres_old) | (vres != vres_old);
    if (sync & is_full_screen)
    { /* GL context calls apparently fail when directly in ChangeWindow. */
        GLint viewport[4];
        int dimensions[2];
        GLfloat aspect_ratio, aspect_ratio_PC;

        get_screen_size(dimensions);
        aspect_ratio = (GLfloat)(hres)/2.f / (GLfloat)(vres);
        aspect_ratio_PC = (GLfloat)dimensions[0] / (GLfloat)dimensions[1];

        if (aspect_ratio < 1.f) /* narrow, tall image */
        {
            viewport[2] = dimensions[0];
            viewport[3] = (GLint)((GLfloat)dimensions[0] / aspect_ratio);
            viewport[0] = 0;
            viewport[1] = (dimensions[1] - viewport[3]) / 2;
        }
        else /* wide, short image */
        {
            viewport[3] = dimensions[1];
            viewport[2] = (GLint)((GLfloat)dimensions[1] * aspect_ratio);
            viewport[1] = 0;
            viewport[0] = (dimensions[0] - viewport[2]) / 2;
        }
        glViewport(viewport[0], viewport[1], viewport[2], viewport[3]);
        sync = FALSE;
    }

It scales to my 1024x768 monitor just fine in GPU-accelerated GL render mode, just not glDrawPixels. I'm fixing the software version for the regression before releasing.

[HatCat]

New attachment of new version to OP (which contained text that is no longer relevant with the intervention of recent developments, so was shortened into a smaller post). See OP for new release.

Anyone have a monitor with a narrow, tall screen (like 3:4 or w/e)? I'd be curious how the full screen results are with this OpenGL beta. Well either way try it people, and let me know. Also does this work on Windows XP

[RPGMaster]

Nice! I just tested it out. I keep getting this messagebox error saying "swapping buffers seems to fail. Hope your device doesn't multi-buffer."

Man I procrastinated and didn't finish the gui yet ;/ . It should be done by tomorrow though. I want to spend extra time to make sure my GUI has no bugs. Also fullscreen is messed up for me. I see this weird purple. I'm wondering if I need to update drivers or something.

Edit: Here's a picture although unfortunately, I couldn't get the purple bar that blocks part of the gamescreen and constantly moves down, then goes back to the top part of the screen once its offscreen. I must say, I love how fast it is to switch to full screen though .