Sound Plugin Development
 

Hello, I'm trying to develop a sound plugin and would like some advice :D . So far I've looked at Zilmar's Basic Audio and 1964 Audio. I've tried changing things around to see what happens. I'm understanding more and more of it.

One thing I need help with is buffers. I tried adding more buffers, but the sound was kinda weird. I probably messed up with the algorithm somewhere. Strange thing is, adding an extra buffer to Zilmar's audio, made SSB64's intro more in sync, but that didn't work for 1964 Audio.

I also tried seeing how to reduce crackling. 1964 Audio's sync audio does the job well, but at the price of fps. Lowering the sleep time from 10ms to 8 sorta helped, but it's not a perfect solution. I thought about implementing SSE for the ucode stuff. Problem is, the main part I need to optimize has too many if statements ;/ . If I were to get the cpu usage of the audio plugin down, I might be able to get stable 60fps with sync audio.

Here's a macro that is a bottleneck.
In the meantime, I'll try learning the API for DSound and XAudio2.

Also, which do you guys think is more accurate for your typical commercial game, using 1964 HLE audio, or LLE using PJ64's RSP recompiler? I know the HLE one doesn't support every game yet.

Now that is unfortunate. HLE was based on interpreter ops from pj64 rsp? Seems like a waste.

You knew that's what VMULF is, right? An RSP opcode?

It does kind of seem like a waste, but perhaps maybe it's as accurate as an interpreter? Idk how I'll even be able to tell the difference in accuracy.

Ya I saw VMULF in your rsp plugin. 1964 Audio's HLE is only a little slower than RSP recompiler, if not approximately equal in speed, but if I was able to use sse for VMULF and some other slow ones, it would be much faster.

Lol everything made by 1964 has optimizations disabled. Sound, Video, and Emulator all had significant improvements, when I turned on optimization. The bad thing about these products is that they pretty much only work for MSVC's compiler. Good thing I'm used to MSVC though :D .

That macro I posted is used 8 times consecutively. Would there be any way to do it less than 8 times? It has too many if statements for me to figure out how to use SSE to do at least 4 at a time. I should practice SSE algorithms.

It seems like sound plugin performance is heavily dependant on your hardware too, because if I use sync audio and disable video, I always get 60 VI/s, so that means I just don't have enough cpu power to use sync audio and still get 0 crackling sounds. I'm sure there are better implementations, because Jabo's sound does 60fps in some games at least, but the timing is sometimes more messed up with sync audio on.

So far the only ideas I have for actually improving the sound plugin, aside from optimization related stuff, is to add more options. Not enough sound plugins have adjustable buffering, and also for sync audio, it would be nice to select the exact millisecond sleep time. At the very least, I wouldn't have to constantly compile it, just to change the timing and buffer size xD.

I"ve briefly looked at Azimer's old source, it's pretty good. His HLE approach is much different than 1964's.

I wouldn't even use that algorithm at all, never mind repeating it.

Neither the recompiler nor interpreter RSP ops in PJ64 accurately represent the behavior of the VMULF operation.

Rofl thanks for clarifying. I had a feeling that algorithm was weird. Too many if statements :D . So basically the HLE implementation needs a rewrite. I kinda do think it's cool the way they did it, but at the same time, it's inefficient the way they mixed assembly with c. I guess at the time they wrote it, gcc wasn't that good? Who knows lol. Good thing the inefficient stuff had hardly any impact on performance. The only expensive stuff were macro's like VMULF, and I forgot the other ones. I think VMACF was one of the other expensive ones.

Sigh, I wouldn't want to spend a lot of time on learning RSP stuff, just to implement HLE. Maybe for now, I will attempt to fix the algorithms though. I'll need to find documentation.

I was hoping this current HLE implementation was at least more accurate than PJ64's RSP recompiler.

Lol now I'm looking at Pj64 2.1's RSP.

While I cannot give coding advice...I would add a gausain filter to the mix to allow for less harsh sounding midtones that blast speakers to death. Prime example...Zelda OOT pressing "C-Left"....

Thanks. I appreciate your advice. I'm not very knowledgable when it comes to sound, but I'm willing to learn. Lol i can't remember the last time I've seen that in sound options, since ZSNES.

Even learning API could help me improve the plugin, even if I can't improve the main code base quite yet.

the fuck is he talking about


Yeah I can't emulate the RSP worth a damn, so I'll just add a Gaussian blur filter so no one can make out my source code.

No it's because PJ64 Team are Win32 developers.
Don't ask me why nobody uses GCC. Everyone doing emulators for Windows just uses Visual Studio all the time. Even I do sometimes; it depends whether I'm making a plugin an emulator or what.

Yeah, VMULF is the most common in ABI1 audio ucodes I think. It's the most commonly encountered RSP opcode in audio...definitely not graphics though.

Also, VMADN and VMADH. I know VMADH is extremely populated in both audio and gfx ucodes, but VMADN ... it's extremely populated (even more so sometimes than VMADH) in either graphics or audio, can't remember which one.

Then don't dive into HLE? LLE is simpler. If you don't like the extra work why worry?

Audio HLE is documented but I have no idea where. Maybe in some functions reference manuals or sdk of some sort.

Not only that, but HLE is more accurate than PJ64's RSP interpreter too, because the interpreter ops are full of extraneous, unnecessary virtual overhead and excessive steps, partly because of great segmentation of the 128-bit vr's active in operation and also memory endianness with the way unions are employed. It's actually rather hazard-prone.

k, not much to learn though

The Default RSP doesn't handle High Level Audio very well outside the realm of DirectSound...which is why when I use an audio plugin with XAudio2...I use RSP Plugin versionm 1.7.0.9.

I don't have much faith in MSVC. Only reason I'll use it is if I'm writing some generic program, or if I'm stuck with it when working on other people's programs.
Good to know :D .
I'm not deadset on doing HLE, but I think it would be better if I did do HLE. If I can get the help I need with the Ucodes, then I will keep HLE. If not, then I will just stick to LLE. I do want to practice SSE :D .
Lol, it's just out of curiousity. I thought maybe I could modify it. As you pointed out, it has flaws, so I should take your advice and not bother reading it. I'm just eager to improve the performance for LLE ;/ . It would be nice if the RSP recompiler was fixed.

Smoke weed every day.

I am curious though. Would you like to tell a little about what issues you're having when using a DirectSound plugin for HLE audio? :D

RPGMaster: Yeah, but with LLE you just have audio quality to fix. With HLE you have multiple things to fix all at once. I would rather fix and then add HLE in later on.

I use shuayan's HLE audio because it has a good quality surround sound capability...usually, under the default DirectSound plugin High Level Audio stays silent under the default RSP plugin. If I use Zilmar's RSP plugin it seems to go to LLE...but when I use Shuayan's HLE Audio, the emulation process crashes unless I use RSP Plugin 1.7.0.9.

ok thanks for your responses

It does yeah, since it is an XAudio2 implementation, perhaps the one you showed me off the stuff spacy51 came up with on VBA-M. XAudio2 is like an enhanced version of DirectSound, so naturally it can give some better audio.

This is because the default DirectSound plugin (you mean by Jabo right?) is intended for LLE audio. It does not aim for a HLE audio approach. You need to use a LLE RSP plugin with it.

Shunyuan's LLE audio plugin has a hack to work with a HLE RSP, but it is still LLE. That might be why you have the crash.

Which is faster? Do you get more speed from the emulation if you tell Project64 to use LLE while using Shunyuan's audio, or if you say use High-level audio when using it

True, I just want to use 1964 Audio as a base now, because Zilmar's source has too much UB. I don't feel like going through hoops to fix something other people have taken care of. Lol I could just leave HLE the way it is and just select LLE :D . I just wanted to improve the speed for HLE even more to squeeze out a little more performance. With the sleeping method for Audio sync, it only works good if your computer is fast enough, otherwise, the FPS will flux. I'll try testing out a game that's less resource intensive to see if my theory is correct about getting 60fps with audio sync if your computer is fast enough.

For now I'm going to focus on learning how to deal with the common issues. First thing I need to do is master the API. If there's no reason to use Direct Sound 8 over XAudio2, then i will use XAudio2.

No not really, go ahead and use XAudio2. In this day and age it'd be rather masochistic not to. We need less DS plugins more others. :p

Alright, Xaudio2 it is.

Well looks like my theory was incorrect anyway. Using 1964 1.1 recompiler with Jabo d3d6 1.52, sync audio still slowed the fps in SM64. So that method is not a good solution imo. Sure if you don't mind fps fluxing, then use it, but I like having a stable fps. One interesting thing though is, I can play around with the ms sleep time and reduce crackling (not completely), but at least keep the stable fps.

Anyone have advice on buffers? I think that 4 buffers would work better than 3. Doing 4 helped with the timing in Zilmar's basic audio for SSB64's intro, but I think i didn't do the algorithm 100% correct.

I had a big old response typed out but the site logged me out mid-message. I recommend emulating the length register more accurately and making sure you emulate the status register to prevent overruns. Best you can do with an inexact plugin spec. This should make 60/50 fps possible without audio issues.

Yes... I am oversimplifying.

Yeah hate it when that happens (site log-out while posting), if you use FireFox and you still have the same tab open then pressing Back enough times till you're at the page where you were replying should normally retrieve a cache of the attempted reply.

Alright, thanks for the advice. I noticed your plugin plays certain games like Quest64 well, while most require audio sync to play the music at its normal pace. Yet if I turn the audio sync on for 1964 Audio or Jabo, the fps slows down significantly. Before, I was actually convinced that those games just naturally played below 60fps.

It's a side effect of PJ 2.1 trying to fix audio incorrectly by introducing a hack. If you turn both audio options in the rom settings, it will sound much better with v0.60 WIP 2. Or use PJ 1.6 (my target at the time). The plugin spec is a lot of messing around with timing since you can't sync it with the speed of cpu emulation.

I usually have both those options unchecked in pj64 2.1. Fixed Audio definitely messes up the audio for some games. Although for some reason, I need it on if I want to use AQZ.

What's AQZ?

Some people call it "AQZ's" input plugin or whatever.
http://forum.pj64-emu.com/showthread.php?t=1973

It's some closed-source implementation of NetPlay which substitutes KailleraClient and uses TCP/IP instead of UDP I think, via the Controller #1.0 plugin spec. I'm guessing RPGMaster means "Fixed Audio Timing" in Project64 affects the ability to initialize some sort of synchronization that this plugin assumed was true for Project64 1.7 (it was meant only as a temporary solution until zilmar added NetPlay officially).

Lol sorry, I have a bad habit of shortening names. And ya I meant the Fixed Audio Timing option in PJ64 2.1 for using AQZ's plugin. That's a shame I'd have to mess up the audio, for netplay lol. I wonder if Kaillera also messes up the Audio. I'll have to test that sometime.

Aw, so that's why games like Clay Fighter 63 1/3 and Ridge Racer get jacked up framerates when sync game to audio is enabled. I've always wondered that.

That's interesting that you use PJ 1.6 as your target. I haven't done a variety of tests, but Mupen64 seems to have the best audio timing. I guess it's good that you targetted 1.6 because your plugin has the best timing for SSB64 out of any plugin I've tested. If I do CF = 3 for SSB64, I rarely if ever hear any crackling either. What's interesting is that when using your plugin, the audio timing is relatively good on Pj64, even better on 1964, and very good on Mupen64. The odd thing is that other plugins that had bad timing on pj64, have good audio timing on Mupen64.

Being able to make the sound plugin work well for all 3 of the major emulators is amazing accomplishment tbh. Most don't work well on pj64.

Actually, you have to only uncheck the "Fixed Audio Timing" so it syncs properly without frame drop..in all the ROM settings in PJ64 2.1. The people that are complaining that the timers in Zelda:MM are out of sync are either forgetting that they have to play the song of time backwards and have Link learn that from the Scare Crow in the base of the Astronomy Tower, or never played the US version of the game.

You don't learn it from the scarecrow; he just tells you about it. I typically play the reversed song of time not too long after first getting the ocarina back, without talking to any of them scarecrows! Furthermore, it doesn't have to be the one in the Asstral Observatory.

looooooool

...What? I was just correcting his typo. He called it the "Astronomy Tower". I was just pointing out it's the ASSTRAL OBSERVATORY.

Man, what an ass.

I just come up to the guy and ask him if I can use his telescope, and he just undresses himself and takes this big shit all over my fucking face. Couldn't even look for the Moon's Tear if I wanted to now.

Anyone have advice on buffers? I need to figure out the ideal number and how to implement more buffers. I doubt that 3 is optimal. I think I'm going to try an HLE implementation too.

Seeing Azimer's audio fix for pj1.4 has inspired me to try and improve audio by modifying the emulator itself. That's one of the reasons I'm focusing on the emulator core atm. That way when most emulators have good audio, it will be less of a pain making the sound plugin work good on multiple emulators.

HLE has no effect on the XAudio2 sound output quality or its buffers.

lol, audio buffers is about where I took a break off the OpenAL version. I got it to play N64 audio while running a game, but it was massively crackled to the point where the audio was hardly legible, because I had no buffers set up for queuing it directly at all. :p

I read in the readme for Winamp's oal sound output plugin that the author got the best experimental output quality by using a high number of small-size sound buffers. Can't you just experiment with it yourself until it sounds better?

I know HLE has no effect on sound output. I'm just interested in this microcode stuff. Of course I will implement the important stuff first before implementing HLE, but it's just something for me to think about if I get bored lol.

The thing is, I tried adding buffers, but I'm guessing I did the algorithm wrong lol. Like when I doubled the number of buffers in Zilmar's Basic Audio, the sound would stop for a split second quite often. I think I got 4 buffers to work good, but I think I remember some undesired effect. I should learn XAudio2 before messing around with buffers again I guess.

Here's a piece of code I that confuses me a bit.Why is it & 3 when there's only 3 buffers? Couldn't that cause UB? I know that I declared certain variables like this Reason I added the +1 was to remove some UB, because there was code like this What was interesting was, when I implemented 4 buffers instead of 3, I had to increase the buffer size for better timing or reduced crackling (i forgot which one lol).

I think you should probably change that to mod 3 ("(count + offset) % 3", not ... & 3). If he set count = 3 that probably was his way of breaking out of the loop signifying that there is only a buffer 0, 1, and 2 but not a buffer 3 so count = 3 should be invalid. So it should be % 3, not & 3.

Lol silly me. Sometimes I just get confused easily :) . Thanks for clearing that up. I thought %3 would make more sense, but he prolly did & 3 for speed. Which is why I decided to try 4 buffers instead of 3, so that &3 would be correct :D . I forgot about the fact that the loop exits when count = 3.

What I'll do is try converting Basic Audio Plugin to XAudio2 and see how it sounds. Then I'll mess around with the buffers. Lol I'm curious whether an even number is better than odd number for buffers. My theory is that an even number is better. Maybe I'm looking too deep into things.

The number of sound buffers matters whether it is even or odd? That has me curious. Why would you suspect that?

Probably just me messing up the algorithm tbh xD. Also I thought maybe even numbers were better for picking a buffer size since it divides evenly. I tend to have these random ideas lol.

4 buffers is certainly better than 3. Also no need to use the % operation :) .

It sounds to me like you're just thinking about performance. If that's the case then 3 buffers should be better than 4 of the same size to promote less memory overhead, not the other way around as you say.

Also being able to & by (power_of_two - 1) instead of resorting to modulus does not promote sound quality.