Project64 2.0 is now available and open source!

[HatCat]

Originally Posted by RPGMaster

Lol I ended up following your advice before you even replied . Turns out I was able to do what I wanted in C. I guess I should read up on C because it seems like some things are just easier to do in assembly. I'm sure some of them are easier for me to do in assembly simply because I haven't learned how to do it in C.

I got that feeling.

I learned assembly languages before I learned C.
Never did read up any books/texts to learn C (aside from the occasional inquiries here and there), just read the official processor manuals to jot down what I needed to know about assembly and, once you get used to the somewhat arbitrary syntax, you'll teach yourself C just as well.

Originally Posted by RPGMaster

I just figured out how to treat a variable any way I want. What I mean is do things like set the 3rd bit of an int to a value or set 4 bytes of a c-string to a specific value. I was just confused how things like *(int*)&variable works. I think I got the hang of it now though.

The former: (expression >> shift_amount) & 1;
... will obviously extract the 3rd bit of an int to a Boolean value, if shift_amount is set = 3.

The latter, you could do it that way.
I think you've figured it out already.

Instead of: /* char string[9] */
string[4] = 'A';
string[5] = 'B';
string[6] = 'C';
string[7] = 'D';

... you'd probably do:
*(int *)(string + 4) = "ABCD" (except obviously this is not valid syntax, the ASCII string needs to be rewritten as a 32-bit hexadecimal).

But, the former is probably readable enough that the compiler would figure it out for you and generate 1 32-bit write of the 4 characters without you having to explicitly code that. Another advantage C has over many assembly languages.

Originally Posted by RPGMaster

I definitely prefer intel syntax over AT&T. I can't stand looking at assembly code if it's not Intel syntax. Honestly if it weren't for google, I'd be much worse at coding. I keep learning useful things that I don't see in any of the books I've read. Is there a manual or book where I can learn more about things like *(int*)&?

lmao, agreed, AT&T syntax is just fugly to me.
I think MarathonMan likes it cause it's more explicit with data types than Intel syntax.

Still, whether I generate assembly output using GCC (AT&T) or Microsoft compilers (Intel syntax), as far as I'm concerned for any specific debugging purposes they're about the same readability to me, with the major difference being that semi-retarded right-to-left principle. (Then again it's probably good practice for Hebrews.)

& just means take the address of a variable.
&(expression) will resolve to the memory address of where something is stored.

Since char array[4]; actually, on the low-level, declares an ADDRESS (called "array") where 4 bytes are stored, you don't need to say *(int *)&array (that's taking the address of an address and you can't do that); it's just *(int *)array.

Pointers are the last thing I stopped hating with C, I tell ya.
But when they finally make sense, you'll find the '&' operator isn't as frequently used as you might suspect.

Originally Posted by RPGMaster

Lol I'm bad at explaining things. My problem with inline assembly is that sometimes when I mix ASM code with C code, the compiler changes code outside of my inline code, which is annoying because it defeats the purpose for what I am try to do. So my solution in those situations where the compiler messes things up, is to either convert more C code into assembly or write all of it in C. What I meant by ignoring is making the compiler not change other areas of code because of my inline assembly. I don't think there is a way to fix it. Funny thing is, I tried using the emit macro, and that made the compiler do different things.

The compiler probably does that because it is an optimizing compiler.
If the compiler sees the inline assembly code you wrote, it might try to maintain legal, non-program-breaking changes on the exterior of the inline assembly you wrote if it means even MORE optimization. It's just trying to do its job; that's all.

And yeah, I would just try to write it all in C. Have never needed to use inline assembly yet.

[RPGMaster]

Rofl, I really do need to try out different compilers. The reason I wanted to try out different compilers is because I keep getting conflicting information when I google things. Some people say MSVC is the best for windows, some say GCC is the best, and others say Intel. Now I bet Intel is probably the best (at least for intel cpu's), but I really got to try GCC now. Part of the reason I got into inline assembly is because the compiler I use (MSVC2010) seems pretty inefficient sometimes. When I do

string[0] = 'A';
	string[1] = 'B';
	string[2] = 'C';
	string[3] = 'D';

the assembly output looks like this

mov         byte ptr [_string],41h  
mov         byte ptr [_string+1],42h 
mov         byte ptr [_string+2],43h 
mov         byte ptr [_string+3],44h

. I also happen to enjoy writing code in assembly, so i do it for fun sometimes. My issue earlier was that I didn't understand how to do arrays with the *(int*). I initially knew that if I wanted to deal with just the 2nd byte of an int, I'd do ((char*)variable)[1]. The number in the [ ] would be multiplied by the size of the variable in the ( *). My problem was, I didn't want to size of that number based on the variable type inside ( *), then I just randomly tried *(char*)variable[i] and that worked. I also barely learned today, that you can do *(int*)(variable +2) to start at a different location. The reason I put & is that I was getting errors sometimes, and I never got errors when I put the &. I think it has to do with what language I compile as in MSVC. If that's not the case, then I'm just confused lol.

Ya I understand that it tries to optimize and reads the inline code as well. I was just getting frustrated that it keeps switching the registers it uses. For readability I just comment my code although it would still be a pain to write out the ASCII values in hex lol. I'm starting to use Macros now, which to me is amazing.

Edit: I just tried *(int*)(string+4) = 'ABCD' and it actually compiled. Only problem was that it stored the letters in reverse since I'm using a little endian PC. I made a macro that puts it in order for me . Now i don't have to write hex for certain things anymore lol.

[HatCat]

I'd actually like to try out Intel compiler myself someday.
No reason not to, just that me I haven't gotten around to it. (Edit: I do have to pay for it, right?)

Originally Posted by RPGMaster

Rofl, I really do need to try out different compilers. The reason I wanted to try out different compilers is because I keep getting conflicting information when I google things. Some people say MSVC is the best for windows, some say GCC is the best, and others say Intel.

Yeah, well, essentially every programmer has opinions. It can get pretty religious at times. -.-

Fact is they're all good at different things. I'd say it depends what kind of project you're trying to do. Most of the time though GCC usually gives much more optimized output, but, there are again exceptions (like with bit-field decodes as some of us seem to have discovered).

Originally Posted by RPGMaster

When I do

string[0] = 'A';
	string[1] = 'B';
	string[2] = 'C';
	string[3] = 'D';

the assembly output looks like this

mov         byte ptr [_string],41h  
mov         byte ptr [_string+1],42h 
mov         byte ptr [_string+2],43h 
mov         byte ptr [_string+3],44h

.

Ah, guess I was partly wrong.
VS should be capable of figuring this out, but I guess it doesn't want to.

It could be also cause MSFT expects you to use an intrinsic of some sort to accomplish this, based on strcpy (the string copy function).
http://www.cplusplus.com/reference/cstring/strncpy/

strncpy(string, "ABCD", 4);

... should do the job more readably and maintainably. See what that outputs?

I personally have never tried strncpy before, but this I believe is correct. (I usually work with the plain strcpy() since all strings in C tend to use "null termination" as a safeguard for the rest of the C stdio.)

Ideally, it should actually be string[5], not string[4].
string[0,1,2,3] hold ABCD, and string[4] is the terminating null byte '\0' character you might read about.

But, that's not as clear of an example, because:

char string[5];
strcpy(string, "ABCD"); /* This writes FIVE bytes, not four. */

...is probably uglier output for the sake of this example, as it can't copy 5 bytes with a 32-bit write.

Originally Posted by RPGMaster

I also happen to enjoy writing code in assembly, so i do it for fun sometimes.

I'm curious.

This isn't impossible (as it applied to me too), but, you're new to C, yet you're more familiar with sticking to an assembly language often.

Are there any other languages in your programming history besides assembly?

[oddMLan]

Originally Posted by BatCat

I'd actually like to try out Intel compiler myself someday.
No reason not to, just that me I haven't gotten around to it. (Edit: I do have to crack it, right?)

(unless you want to shell out $700 bucks only for the C/C++ compiler)

[RPGMaster]

Originally Posted by BatCat

I'd actually like to try out Intel compiler myself someday.
No reason not to, just that me I haven't gotten around to it. (Edit: I do have to pay for it, right?)

Yeah, well, essentially every programmer has opinions. It can get pretty religious at times. -.-

Intel has a 30 day trial. My problem with the Intel compiler is that I heard they purposely make the performance worse for AMD computers, so I don't think it would be good for releasing public binary files, unless you have separate versions. I tried out gcc and I must say, the output seems to be a lot more efficient, at least when it comes to number crunching. I can see why people stick with MSVC though. I think MSVC is a great IDE and I liked it's features better. The only thing I really am not too happy with is the compiler itself. Maybe I'm a noob, but MSVC seems a lot easier to use than GCC. I'm glad to know that now I can learn from multiple compilers!

Originally Posted by BatCat

Ah, guess I was partly wrong.
VS should be capable of figuring this out, but I guess it doesn't want to.

I tried the string[0] = 'A' code in GCC and it also did 1 byte at a time. I'll check out that strncpy function.

Originally Posted by BatCat

I'm curious.

This isn't impossible (as it applied to me too), but, you're new to C, yet you're more familiar with sticking to an assembly language often.

Are there any other languages in your programming history besides assembly?

Sadly I have almost 3 years of experience with C++. I wasn't a serious programmer until a year ago though. Also I didn't do much practicing outside of school until I started getting serious. I also took 1 semester of C# and 2 semesters of Visual Basic. I suck at C# and VB lol. This goes to show how little you should rely on school to learn stuff. I learned a lot more on my own than I did in school when it comes to programming. I started teaching myself C a few months ago and Assembly about a year ago. For some reason the lower level languages just feel more natural to me (C and Assembly) compared to Java, C#, VB, etc. I used to think C++ was cool, until I started doing C. Now I don't even like C++ that much anymore. When I say I don't like C++ much, I just mean the libraries, not the syntax of the language. I much rather prefer using stdio over iostream.

I really wish I didn't listen to brainwashed people. In school I didn't even learn about bitwise operations (like OR, XOR, AND) or debugging until I took an assembly class (sadly it was some simulator language called PEP8). I should have been studying on my own from day 1 instead of just relying on school to teach me. Also, learning assembly really helped me get better at C/C++. Lately I've just been focusing a lot more on assembly, but I've also been learning stdio and winapi. I will probably start learning intrinsics soon, so that I can rely less on inline assembly. I would say my best language is C/C++, because I could write more complex algorithms easier, but when it comes to writing efficient code, sometimes I only know how to efficiently do it in assembly. One thing I could never find out how to do in C was branching off of a variable. What I mean is, lets say I had a variable named num which had the value of 401030, I can't just write goto num, to jump to 401030. Lol I know this might seem obscure, but I wanted to see how it's done because it's like that in Super Smash Bros 64, where the game jumps to a specific address, based on the value of a variable.

I finally started getting the hang of compiling PJ64. I still don't know why I had some of the problems I had earlier, but now everything works fine . The reason the display list was messed up was because I didn't put the rdb file in the same folder as my new pj64.exe lol. So now I'm just trying to see what good changes I can make.

[HatCat]

Originally Posted by RPGMaster

Intel has a 30 day trial. My problem with the Intel compiler is that I heard they purposely make the performance worse for AMD computers, so I don't think it would be good for releasing public binary files, unless you have separate versions.

Somehow I don't feel the blame on them for that.
AMD does screw up a few things with conformance to the modern-day benefits of the Intel architecture, and code would have to get optimized in a different way for Athlon CPUs than Intel ones in some cases, like SSE.

Originally Posted by RPGMaster

I tried the string[0] = 'A' code in GCC and it also did 1 byte at a time. I'll check out that strncpy function.

Well that's surprising. Neither compiler did it?

Have you tried passing -O3 to GCC for full optimization?
I thought Visual Studio did 32-bit-write conversions from separate char writes I did in some of my basic plugins...maybe you're not using the optimized project settings like /O3 or /Ox in Visual Studio (I think it was). It's somewhere in the Code Generation settings.

Still, strncpy should do it.

Originally Posted by RPGMaster

Sadly I have almost 3 years of experience with C++. I wasn't a serious programmer until a year ago though. Also I didn't do much practicing outside of school until I started getting serious. I also took 1 semester of C# and 2 semesters of Visual Basic. I suck at C# and VB lol. This goes to show how little you should rely on school to learn stuff. I learned a lot more on my own than I did in school when it comes to programming. I started teaching myself C a few months ago and Assembly about a year ago. For some reason the lower level languages just feel more natural to me (C and Assembly) compared to Java, C#, VB, etc. I used to think C++ was cool, until I started doing C. Now I don't even like C++ that much anymore. When I say I don't like C++ much, I just mean the libraries, not the syntax of the language. I much rather prefer using stdio over iostream.

Oh yeah I only took one programming class in school, and it was about Java.
I probably learned less than I ended up teaching. -_-

I tried learning C++ I think *before?* C? I downloaded some books for both, but, I got really impatient learning by reading. I gave it up for a couple maybe 3 years, then through a spontaneous series of events started following my own example.

I don't know a thing about C# or VB though, lol.

I used to think the stdio in C was more annoying than the iostream in C++.
But after I taught myself all that jazz, I'm like, WUT?
C++ is frikkin' hideous to me now. (It's probably the best choice for people who plain and simple are way better at science/math than they'll ever be at computer science. But, nah, stick to HTML. )

Originally Posted by RPGMaster

I really wish I didn't listen to brainwashed people. In school I didn't even learn about bitwise operations (like OR, XOR, AND) or debugging until I took an assembly class (sadly it was some simulator language called PEP8). I should have been studying on my own from day 1 instead of just relying on school to teach me. Also, learning assembly really helped me get better at C/C++. Lately I've just been focusing a lot more on assembly, but I've also been learning stdio and winapi. I will probably start learning intrinsics soon, so that I can rely less on inline assembly. I would say my best language is C/C++, because I could write more complex algorithms easier, but when it comes to writing efficient code, sometimes I only know how to efficiently do it in assembly. One thing I could never find out how to do in C was branching off of a variable. What I mean is, lets say I had a variable named num which had the value of 401030, I can't just write goto num, to jump to 401030. Lol I know this might seem obscure, but I wanted to see how it's done because it's like that in Super Smash Bros 64, where the game jumps to a specific address, based on the value of a variable.

If you learn stdio, then you probably learn everything I know about APIs, haha.
I never did get into WINAPI stuff. Microsoft mostly supports the C++ world of things.

Do you mean like a switch statement?

if (x == 0)
    do_case0();
else if (x == 1)
    do_case1();
else if (x == 2)
    do_case2();
else if (x == 3)
    do_case3();

... being replaced with:

switch (x)
{
    case 0:
        do_case0();
        break;
    case 1:
        do_case1();
        break;
    case 2:
        do_case2();
        break;
    case 3:
        do_case3();
        break;
}

Here modern compiler theory will convert the logical if/elseif/elseif nature into an unconditional, "indexed" branch using a hash table or jump table.

It might convert the value (x) into an instruction memory offset, and use that offset to create the new location in code memory where the program jumps to, to avoid the problem of going through too many if'ss/else's to eventually find the code to jump to.

But, if you're trying to jump to a numeric address in C (not referenced by label of a function or other symbol), then, you can't. C doesn't really let you pick the numbers and exact numeric addresses where code begins because the compiler handles those matters for us.

Originally Posted by RPGMaster

I finally started getting the hang of compiling PJ64. I still don't know why I had some of the problems I had earlier, but now everything works fine . The reason the display list was messed up was because I didn't put the rdb file in the same folder as my new pj64.exe lol. So now I'm just trying to see what good changes I can make.

oh lol, sorry to hear actually because I was *thinking* maybe your ROM browser issue might have just been a newb issue with hte placement of the RDB file, but, I kind of felt bad bringing that up. Glad you figured it out though.

It's possible zilmar might really appreciate your help, but he's too busy dealing with the usual transitional channels in his career to log on here for months sometimes. So if you think you might be able to make a difference, you've got plenty of time to figure out how.

[RPGMaster]

Well about Intel, ya I understand that AMD doesn't support all of the Intel instructions, but I have heard of people tricking the CPUID thing and improve performance on AMD cpu's. Maybe Agner's blog is outdated, but I was still disappointed when I read about it.

For the string thing, I did max speed for MSVC, but I did highest optimization for GCC. Funny thing is, I saw a piece of code like that in pj64 1.6

PIF_Ram[36] = 0x00;
PIF_Ram[37] = 0x06;
PIF_Ram[38] = 0x3F;
PIF_Ram[39] = 0x3F;

I know about switch statements, but I was curious about that method I described because I was wondering how they did it in Super Smash Bros 64. They have a structure of data that contains the addresses for the player states of each action. The game then jumps to the address loaded from the data structure. I guess I should learn more about compiler theory. I'm interested in this stuff is because I may make a game one day, and I heard that finite state is an efficient way to make games. So I guess the compiler is did all that fancy work for them. It just sounds impressive considering that I heard compilers were bad back then.

My problem with pj64, was that it somehow was very buggy when I first compiled it. So I assumed that the problem for the display had to do with the compiler issue, rather than files/settings. I still don't get how erasing the project and starting fresh fixed the bugs I was getting and then finally yesterday while I was debugging, it just hit me that I didn't put the rdb file in the folder. Lol that would also explain why it says bad rom for my patched roms since I never updated the file.

Right now, I don't know what I can do to contribute. You think it's best that I work with 1.6 instead of 2.x? All I can do right now is read through the code and try to understand it. For me, it's hard to work with other people's code, especially when there's so much to read. I'd say my best bet is to find the most important things to work on and go from there. It will take a while for me to learn enough to do anything significant. I have too little knowledge about emulators at this point in time.

[HatCat]

Originally Posted by RPGMaster

Funny thing is, I saw a piece of code like that in pj64 1.6

PIF_Ram[36] = 0x00;
PIF_Ram[37] = 0x06;
PIF_Ram[38] = 0x3F;
PIF_Ram[39] = 0x3F;

Right. This sort of code, while "algorithmically" unoptimized, is meant for maintainability and stability.

It's written like that in PJ64 source code because, you can change the bytes easier, and the byte addresses with 8-bit precision.

Sure, zilmar could have written it as *(int *)(PIF_Ram + 36) = 0x00063F3F; instead of those 4 1-byte write lines, but then it wouldn't be as..."flexible". His intention was probably to let the compiler figure out the 32-bit write for him. (I'm rather surprised that so far you haven't been able to reproduce this easy compiler strategy though.)

Originally Posted by RPGMaster

I know about switch statements, but I was curious about that method I described because I was wondering how they did it in Super Smash Bros 64. They have a structure of data that contains the addresses for the player states of each action. The game then jumps to the address loaded from the data structure. I guess I should learn more about compiler theory. I'm interested in this stuff is because I may make a game one day, and I heard that finite state is an efficient way to make games. So I guess the compiler is did all that fancy work for them. It just sounds impressive considering that I heard compilers were bad back then.

Indeed so. Compilers today are much better than what people had to explicitly lay out for compilers in the 90s.

(One very simple example of this is bit shifts. I'm sure you're aware of shifting integers like the x86 SHL instruction. Multiplying ax * 2 is slower than doing a SHL ax, 1, but, if you type out * 2 in C source code today, compilers will convert it to shifts for you. This is something you could not so easily count on compilers to do for you in the 90s.)

And what you described about the table of jump addresses in Super Mario 64 is exactly what I meant to model by the switch.

Whenever you type a huge switch statement with loads of cases, the optimizing compiler will not (or should not) generate thousands of conditional jumps. In the x86 output you should see it create a "look-up table" of addresses to jump to. (GCC will show this better I think...I'm not sure if you can see this just as well in Visual Studio output.)

const unsigned long table_of_addresses[4] = {
    0x3FC496,
    0x3E1298,
    0x101101,
    0x543210
};

... is an example of a look-up table containing addresses to jump to.

A switch() statement with 4 cases (case 0:, 1:, 2: and 3 could get compiled to the assembly code where the assembler after that is able to generate the absolute addresses and created table like the one above. (Except this can't be managed on the C level since you can't force start addresses in C.)

Originally Posted by RPGMaster

Right now, I don't know what I can do to contribute. You think it's best that I work with 1.6 instead of 2.x? All I can do right now is read through the code and try to understand it. For me, it's hard to work with other people's code, especially when there's so much to read. I'd say my best bet is to find the most important things to work on and go from there. It will take a while for me to learn enough to do anything significant. I have too little knowledge about emulators at this point in time.

I think you should read through the code that makes most sense to you.

You think Project64 source code is most sensible to you? Practice with that.
You prefer Mupen64 or 1964 source code? Try experimenting with those.
1.6 source to PJ64 looks cleaner to you than 2.1? Check that out (or the older 1.4 source code).

The point is, at least in my experience, it is challenging sometimes to not let our ambitions get the better of our plans. If we start out right away with the "best" code, or the project we really want to improve (in your case PJ64), we might be over-encumbered by new programming experiences that will constantly distract us while working with huge, colossal code bases (especially a systems critical thing like an emulator).

So while 2.1 or 1.6 or even 1.4 PJ64 source code might be best, sometimes it's the smallest (even most incomplete and not finished) [emulator] source code bases that we learn the most out of experimenting with. But then again, we all have different methods of learning. I can't pretend to know what's best for most people.


I will say this much.
I'd rather evade talking about 1964/Mupen64 source code in this thread because it's kind of disrespectful to this topic and this forum (which is about PJ64 source). Sure I'd like an imaginative way to get banned, but there are better ways than that.

[RPGMaster]

Well with the current compiler settings I was using (SSE2 enabled), it just ended up using the xmm0 register to transfer the values, so it wasn't as bad in this case. Another advantage to doing each byte separate is letting the compiler deal with endianess.

What I will do is just learn how to get the compiler to recognize exactly what I want to do. When I first started debugging n64 games, I was really impressed with the formulas they were using. Then after studying some more assembly, I decided to debug my C code and compare the difference between things like num *= 2 and num <<= 1. With MSVC, I saw that sometimes it would switch the num *= 2 to num += num instead of switching to num <<= 1. I feel silly, all this time I've been underestimating the compiler considering I've been using MSVC which has some obvious flaws. I honestly don't know how anyone can say MSVC produces faster code... It didn't even generate efficient loops (especially if you use global variables). I guess compilers generally make code less efficient when you use global variables? I will say, I love how compilers generate magic numbers when doing math with constant values (like num *= 7). I used to laugh at people who said "you can't beat compilers", but now it makes a little more sense after seeing the performance difference between MSVC and GCC.

Alright, I guess I'll stick with 1.6 but maybe look at 1.4 as well. Now I just need to find the hotspots so that I can see what I'm capable of doing.

Edit: I was looking at some code in the dma.c file. Is there any reason why they wrote mov edx,0 with inline assembly? I thought it was better to do xor edx,edx? Also I just went ahead and installed MSVC2013 since 2010 kept crashing anytime I right clicked a variable or register while debugging. I noticed it optimized the strings a lot better . I hate how they made the disassembly view look worse while debugging in 2013. It no longer shows variable names, and it only tells you the address that it pushes and not the variable. Lol either I initially had the wrong settings enabled or installing MSVC 2013 changed the compiler for 2010 as well, because I don't remember seeing the optimized string [0] = 'A', but now I see it even in 2010.

[HatCat]

I'm really not sure how MSVC works with different memory types in that regard.

I figure it might possibly be doing ...
add ax, ax;
... instead of ...
shl ax, 1;
... because they aren't really all that different performance-wise. (Though if anything I'd say the SHL by 1 is preferable.)

Or it could do a LEA instruction, where it handles both writing the function of the source memory operand (multiplying it by 2 with an internal shift of the source operand) TO a destination memory operand which effectively kills two birds with one stone (multiply and store in one instruction).

No doubt about it, though: There are cases where the compiler just never figures out some of the shit you're really trying to do. I wouldn't blame you for sticking to assembly code anyway, but even so, I'd keep such resorts for only the performance-critical code, those within the main bottlenecks.
Compilers will also continue to improve over time, over years and over decades, so other times all you have to do to fix it is keep upgrading till it works.

Originally Posted by RPGMaster

Edit: I was looking at some code in the dma.c file. Is there any reason why they wrote mov edx,0 with inline assembly?

Heh. It is, actually, but a few people out there are rebellious against the Intel optimization manual.

I didn't know zilmar might be one of them.
Many people say XOR reg,reg is full of shit, and that it should just be MOV reg,0.
After all the latter is more readable, natural and somewhat maintainable.
(The other optimized way to say it is SUB reg,reg instead of XOR reg,reg.)

Of course, if this is the worst case of non-conformance with PJ64 inline assembly code, I doubt the performance degradation from solely this amounts to much.