Font Size

Profile

Layout

Cpanel
Jake Stine_avatar

VirtualAlloc on Linux

Yes, there is a way to simulate Microsoft's VirtualAlloc behavior on Linux. Much searching of the internet did not reveal a satisfactory answer; only hints that when combined with some applied tests of my own yielded the following result:

Code:
// to RESERVE memory in Linux, use mmap with a private, anonymous, non-accessible mapping.
// The following line reserves 1gb of ram starting at 0x10000000.

void* result = mmap((void*)0x10000000, 0x40000000, PROT_NONE, MAP_PRIVATE | MAP_ANON, -1, 0);

// to COMMIT memory in Linux, use mprotect on the range of memory you'd like to commit, and
// grant the memory READ and/or WRITE access.
// The following line commits 1mb of the buffer.  It will return -1 on out of memory errors.

int result3 = mprotect((void*)0x10000000, 0x100000, PROT_READ | PROT_WRITE);


When using mmap, you can create a simple uncommitted reservation of memory simply by specifying PROT_NONE on any anonymous mapping (in the world of mmap, anonymous means it has no associated file/pipe -- it's just a memory block). This is sufficient for reserving a large contiguous address range from being fragmented up by the likes of malloc. Granting the memory read and/or write privileges tells Linux to commit the memory (equivalent to VirtualAlloc with MEM_COMMIT). If there is not enough system memory to complete the call, it returns -1.

Oddly enough, though, Linux makes it so that it isn't even necessary to bother with the above solution, via a strange little hacky technique called...

Over-committing Memory

This 'feature' is enabled by default in most modern Linux kernels (anything 2.6 or newer). Basically all this means is that Linux will let programs commit a lot more RAM than is actually available to the operating system! Instead of performing a "strict contract" on commit that says "oh yes we absolutely have this much ram available", Linux looks at the ram and looks at the request, and makes some arbitrary judgement call on if the program will actually use that much ram or not. In other words, just because your call to malloc returned a valid non-NULL pointer doesn't mean there's actually anywhere near that much memory available to your app. It just means that Linux doesn't think you're going to use that much.

Instead, as a program references its allocated memory, Linux commits the memory on-demand. Most of the time, programs that malloc huge amounts of ram only use a wee bit of it, so that's fine. By using overcommitted memory management, Linux avoids the dreaded "Low on virtual memory!" error that can sometimes plague Windows. This is actually highly ideal for apps like PCSX2 and the Java virtual machine, for example. Kudos!

.. oh but things do get fun if apps over-step their bounds!

Thanks to over-committing, Linux programs that run out of memory do not get error codes or NULL pointers. Instead they will typically be KILLED INSTANTLY by the kernel. They do not get out of memory errors, and they don't even get SIGSEGV or anything else that can be handled or logged. They just DIE -- because doing anything else would risk system stability. So in the long run, its still a good idea to use the Reserve/Commit management strategy even on Linux (mmap / mprotect as described above); because your app will be more likely to get proper out-of-memory errors instead of just causing itself (and possibly other processes on the system) to die suddenly and without warning or error.

Another positive for the the above mmap / mprotect example is that it will also work well on Linux systems that have over-commit disabled, since it basically does what over-commit does but without the hacky "programs die instantly without error" part if the system runs out of physical memory.

Jake Stine_avatar

Advanced memory management

Being an emulator of a fairly robust system (the PS2), PCSX2 typically consumes a lot of system RAM. It needs multitudes of caches and buffers for various things. Just to give an idea, I'll list some of the larger stuff and their current defaults:

  • PS2 main memory [32mb]
  • IOP memory [2mb]
  • EE/IOP BIOS roms [6mb]
  • Scratchpad, Hardware registers, VU memory, DMA buffers, etc [4mb]
  • VTLB indexes, lookups, and protection tables [8mb]
  • EE/R5900 recompiler cache [16mb]
  • EE/R5900 recompiler block/pc translation table [48mb]
  • R5900 memory protection mirror [32mb]
  • IOP/R3000A recompiler cache and translation table [10mb]
  • microVU recompiler code caches [16mb * 2]
  • superVU recompiler code caches [8mb]

Read more: Advanced memory management

Jake Stine_avatar

Why MSI sucks...

I just spent five weeks waiting for MSI to mail me a check back for an RMA'd HD 5770. I mailed my card in on May 28th, after getting an RMA number from MSI tech support. The check showed up today on July 7th. The amount refunded naturally doesn't cover shipping costs, neither for the initial purchase or for mailing the piece of crap back to MSI. And when I consider time spent troubleshooting the GPU, dealing with tech support, packing and mailing the card back, and now shopping for a replacement (I hate shopping for GPUs, and not surprisingly, whatever I buy this time around will not be an MSI brand), I'm really feeling steamrolled.

(More gory details of the ordeal are posted here: Why MSI sucks )

In other news, we recently freed PCSX2 of the shackles of MMX an XMM register freezes. This is a godsend for Linux and Mac users, as it means PCSX2 can finally be compiled with gcc optimizations enabled and be more stable at the same time. Thusly Linux and Mac users can most likely expect some really good things in the near future.

I may do a more detailed blog on the nightmares that plagued the Linux/Mac builds for so many years, and how we went about fixing them. But for now I'm too busy trying to pick out a new HD 5770, because MSI was too cheap to send me a replacement. -_-

Jake Stine_avatar

The return of the Commandline!

After its absence for many moons, the Commandline functionality will finally be restored to PCSX2. Third-party frontend and config-manager authors rejoice! ... and hopefully stop hating my guts, too.

To paraphrase Darth Vader: "Witness the power of this fully armed and operational Command line-driven battlestation."

(we all know a command line-driven death star would have been way cooler than some click-and-drag crap.)

The new PCSX2 command line should be functional in our next beta release, which should be out pretty soon, and it will work as follows:

Syntax: pcsx2 [IsoFile] --toggle --option=value ... etc

  • IsoFile - optional ISO image to load and run on startup; uses the PCSX2 internal ISO loader.


General Options :

  • --cfg=[file] {specify a custom configuration file to use instead of PCSX2.ini (does not affect plugins)}
  • --cfgpath=[dir] {specifies the config folder; applies to pcsx2 + plugins}
  • --help {display this help text}
  • --forcewiz {forces running of the First-time Wizard (selection of docs folders and what-not)}


Auto-Run Options :

  • --elf=[file] {executes an ELF image}
  • --nogui {disables display of the gui on exit (program auto-exits)}
  • --nodisc {boots with an empty dvd tray; use this to boot into the PS2 system menu}
  • --usecd {uses the configured CDVD plugin instead of IsoFile}


Compatibility Options:

  • --nohacks {disables all speedhacks}
  • --gamefixes=[fix,fix] {Enable specific gamefixes for this session. Valid fixes in 0.9.7 are: VuAddSub, VuClipFlag, FpuCompare, FpuNegDiv, XGKick, IpuWait, EETiming, SkipMpeg }
  • --fullboot {disables the quick boot feature, forcing you to sit through the PS2 startup splash screens}


Plugin Overrides (specified dlls will be used in place of configured dlls):

  • --cdvd=[dllpath] {override for the CDVD plugin}
  • --gs=[dllpath] {override for the GS plugin}
  • --spu=[dllpath] {override for the SPU2 plugin}
  • --pad=[dllpath] {override for the PAD plugin only}
  • --dev9=[dllpath] {override for the DEV9 plugin}
  • --usb=[dllpath] {override for the USB plugin only}



Jake Stine_avatar

SPU2 is more than just sound!

The SPU2 is the Sound Processing Unit for the Playstation 2, and works a lot like the sound card in your own PC; albeit still quite unique in its approach to mixing sounds/voices and the programmable interface it provides for that. But the SPU2 is more than just sound. It's one of the more reliable timing mechanisms on the PS2 and games tend to use it as such. Without at least basic SPU2 emulation, no games will boot at all. This isn't too surprising if you understand how console hardware typically works, but what might be surprising is realizing how many games won't boot even with what appears to be fairly competent SPU2 emulation.

Until SPU2-X 1.4, no SPU2 plugin had gone the distance on implementing IRQs (Interrupt Requests). IRQs are scheduled via specific SPU2 memory addresses. When a marked memory address is accessed anywhere in SPU2 memory (either read or write), the IRQ is signaled to the IOP. The most important IRQs on DMAs and audible voice playback have been supported for eons; without these no games would boot, period! Meanwhile, many of the lacking IRQ checks were known, but glossed over because of overhead required for the checks (a couple other checks were simply overlooked). The three main culprits for causing emulation errors were as follows:

1) the "free run" feature of SPU2 voices.
2) the write-back areas for each core's mixed output.
3) Reverb Processing, which uses a series of overlapping buffers to generate feedback.


Free Running Voices

The SPU2 has 48 total voices (24 voices for each core), plus two dedicated streaming audio input sources. Each voice can play a sound effect or stream audio, and can either be stopped, looping, or 'free running.' Free running voices typically zero out their volume rather than stopping or looping, and continue to 'play' forever (albeit silently). These free running voices access inaudible areas of SPU2 memory and thus trigger IRQs unexpectedly -- except, of course, some games are cleverly designed to expect these unexpected IRQs!

Because of the overhead required to free-run otherwise silent voices, all other SPU2 plugins (until now!) have opted to ignore processing them. This is the feature that fixes Fatal Frame 2 (Project Zero 2) and a dozen more games.

Output Write-back Areas

The SPU2 defines a handful of special areas of memory where it writes back sound data at various stages of the mixing process. It's perfectly legal for a game to set an IRQ address within these buffers, and then expect it to trigger when the SPU2 does its write-back to that address. The write-back areas are mapped as follows:

Code:
0x0400 - 0x05FF  :  Core 0, Voice 1
0x0600 - 0x07FF  :  Core 0, Voice 3
0x0800 - 0x09FF  :  Core 0 Output (Left) [includes Wet/Dry/ADMA sources]
0x0A00 - 0x0BFF  :  Core 0 Output (Right) [includes Wet/Dry/ADMA sources]
0x0C00 - 0x0DFF  :  Core 1, Voice 1
0x0E00 - 0x0FFF  :  Core 1, Voice 3

// Following are results of mixing all 24 voices for the given Core.

0x1000 - 0x11FF  :  Core 0, Dry Mix (Left)
0x1200 - 0x13FF  :  Core 0, Dry Mix (Right)
0x1400 - 0x15FF  :  Core 0, Wet Mix (Left)
0x1600 - 0x17FF  :  Core 0, Wet Mix (Right)
0x1800 - 0x19FF  :  Core 1, Dry Mix (Left)
0x1A00 - 0x1BFF  :  Core 1, Dry Mix (Right)
0x1C00 - 0x1DFF  :  Core 1, Wet Mix (Left)
0x1E00 - 0x1FFF  :  Core 1, Wet Mix (Right)


In specific, some games set an IRQA for Core0's write-back area. The IRQ can either be used as a timing mechanism, or as a synchronization point for post-processing audio effects. Most SPU2 plugins properly handled the write-backs, but overlooked the necessity of doing IRQ checks for them.

Reverb Processing

The SPU2 employs a clever reverberation algorithm that utilizes multiple overlapping read and writeback buffers within SPU2 memory to generate feedback. Each step of the reverb process accesses memory and must test against the IRQ address; for a grand total of 24 IRQ tests per Core. Fortunately, all reverb activity occurs within a specified area of SPU2 memory, so for most games a single simple test can be used to exclude the IRQ test.


And It All Applies to SPU2null!

This is the boring part that I'm going to look to implementing soon: In order for SPU2null to be fully emulation-compliant, it must properly simulate all of these things, which basically means it needs to have a complete sound mixer implemented; including reverb buffering/addressing logic. It probably seems silly, but SPU2null would still be without any platform dependent code or sound drivers, making it an ideal base for emulation analysis and as a base for future plugins.

You are here: Home Developer Blog