Archived

This topic is now archived and is closed to further replies.

Noel

Breaking the 4Gb Barrier w/ a Ram Drive

Recommended Posts

Just for fun I used the nice IMDisk freeware Ram Drive software driver to create a 8Gb ram drive and assigned my swap file to it w/ a fixed size of 7gb.  By paging to the ram drive I noticed maximum memory use went over 4030Mb, whereas when I left the page file on my C:/ drive SSD maximum memory committed would peak out at about 3780Mb.   The thing I haven't yet tried is disabling the page file.  Methinks WIndows would allocate even less to FSX in that instance, however I may not understand how it functions.  Some have conjectured using a swap file in ram can help reduce writes to your SDD if used correctly, which might matter some for longevity, or not.

 

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

Of course your total memory consumption can go over 4GB with 64-bit OS. Problem is within FSX itself and as it is a 32-bit program, it can't address more than 4GB of memory. Using 32-bits you can't just make any bigger number than 4,294,967,296. Law of the mathematics apply here. Period. Besides, more important is FSX process virtual size, which you can't see from task manager. It reaches 4GB limit far more earlier, than the process itself. Then boom, and you have yourself an OOM error.

 

What is possible though, is that 32-bit OS can use over 4GB using so called physical address extension, where tables are used to map 4GB memory areas to a larger memory chunk up to 64GB. In 32-bit desktop Windowses 4GB limit is imposed and PAE is not supported, but I've read somewhere that desktop Windowses can be hacked too to utilize up to 64GB. Never tried that though, because I've been running 64-bit OSs since 2006 when 1-2GB of memory was pretty common in home use and problem didn't exist. PAE doesn't magically make 32-bit programs act like they would be 64-bit programs and 32-bit OS with PAE can't run 64-bit programs. It is just remapping 32-bit memory addresses in processor level.

Share this post


Link to post
Share on other sites

Yes, but it was interesting to note the difference, albeit small, when I used the ram drive.  Next will try no pagefile which could be the best option of all unless OOM's occur.  That the system doesn't have to allocate resources to page seems like the best option of all.

Share this post


Link to post
Share on other sites

Next will try no pagefile which could be the best option of all unless OOM's occur.  That the system doesn't have to allocate resources to page seems like the best option of all.

Disabling pagefile doesn't most likely affect in anything when you have plenty of RAM. If there is room for the program to fit in the main memory, no data is written in page file. Only exception may be some very, and I mean very, legacy programs which can require a pagefile for it to function at all. In these cases program was written to write some data straight to page file instad of the memory. I would say, that it is very unlikely that you use anything like that.

 

Page file is relevant only when your amount of RAM is lower than address space and your program actually uses more than that amount. Page file is understandably slow, even if it is on SSD and makes using something like modern game completely useless and most probably it would end up in a crash or program may be even written so that it refuses to work if there is not enough physical RAM. As a general rule of thumb, page file is quite useless with todays common RAM amounts of 8GB or 16GB with 64-bit OS: there is plenty of room for (at least) one large 32-bit application, like game, and still more than enough for all backround programs to run in their own address spaces.

 

If you see more memory consumption with ramdisk one reason is the ramdisk software itself, which requires some memory. Total memory consumption varies also a lot depending what your OS and all backround processes are actually doing. By the way, I just noticed that in the screen shot your total memory consumption is 4030MB and even that is not over 4GB.

 

As I said, it is pointless to look FSX memory consumption through task manager, because you can't see its virtual address space and usage of virtual memory it actually needs. When I once suffered from OOMs with my FSX I noticed, that in general, when VAS-usage hit 4GB and program crashed, FSX process memory usage was around 3.7GB. Those few hundred MBs is consumed mainly by all the drivers and essential OS components that FSX requires to function. They need to be in the same virtual address space of FSX, which is bound to 4GB.

 

One more thing, whether you are talking of main memory or page file, adding page file doesn't help in any way for the program address space. Let's say, that you have a situation where you have 2GB of consumed physical RAM and 2GB of page file for the program. That is a 32-bit address space consumed, totalling 4GB. In that sense, page file is exactly similar memory compared to RAM from software and memory address perspective. You can't add to address space by using page file.

Share this post


Link to post
Share on other sites

That the system doesn't have to allocate resources to page seems like the best option of all.

 

...

 

Some have conjectured using a swap file in ram can help reduce writes to your SDD if used correctly, which might matter some for longevity, or not.

Except that without the page file the OS can no longer move lower priority data out of RAM (including things like the registry). Many background tasks that get loaded at system start, for example, don't actually do anything until they get called upon, in which case the OS moves their data to the page file. If you have far more RAM than anything you do with the system will require, you can probably get away without the page file, but in most cases it is best to just leave it alone (or at most just set a fixed size).

 

Trying to save your SSD by moving the page file to RAM also doesn't really make much sense. If your system is heavily using the page file, it means you need more RAM, period. Moving the page file to RAM would just exacerbate the situation. This holds true with hard drives as well. SSDs are also designed to last many years and aren't going to wear out from page file usage. One would have to do something like delete and write to every sector in an SSD 24/7 for a year or more before the drive began to wear out. Under normal usage, an SSD will last a few years. They wouldn't offer 3+ year warranties if the drives were going to die after one or two years.

 

Moving the page file to a RAM disk doesn't really make much sense either as the point of the page file is to free up RAM, however, if the page file is moved to RAM, it can't really do this. It would be far better to make as much RAM available as possible for applications to use directly.

 

Here is a link to some Microsoft articles on how Windows memory is managed.

 

http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx

Share this post


Link to post
Share on other sites

If you have far more RAM than anything you do with the system will require, you can probably get away without the page file, but in most cases it is best to just leave it alone (or at most just set a fixed size).

This. Every now and then you see on some forums extremely long discussions about what to do with the page file. SSD drives becoming more common have lifted this subject up again and some are claiming that pagefile writes are considerably taking hours away from SSDs and better way is to disable it or set it to fixed size. Whole discussion is moot IMO, as you said. SSD drives are used succesfully in far more demanding environments than regular home desktops, where writes occur more often than OS page filing. OP with 32GB can run his computer fully without any page file. I once ran my Core i7 920 with 6GB of RAM for a while without any issues, untill I realized the uselessness of all that to my new SSD. Fixed size? Yeaaah, but I'd still keep it OS managed because then you have in every possible scenario enough of page file, if it is needed. Keeping it fixed doesn't take away any writes to that allocation, so if PF is used, writes will occur whether the size is fixed or not.

Share this post


Link to post
Share on other sites

This. Every now and then you see on some forums extremely long discussions about what to do with the page file. SSD drives becoming more common have lifted this subject up again and some are claiming that pagefile writes are considerably taking hours away from SSDs and better way is to disable it or set it to fixed size. Whole discussion is moot IMO, as you said. SSD drives are used succesfully in far more demanding environments than regular home desktops, where writes occur more often than OS page filing. OP with 32GB can run his computer fully without any page file. I once ran my Core i7 920 with 6GB of RAM for a while without any issues, untill I realized the uselessness of all that to my new SSD. Fixed size? Yeaaah, but I'd still keep it OS managed because then you have in every possible scenario enough of page file, if it is needed. Keeping it fixed doesn't take away any writes to that allocation, so if PF is used, writes will occur whether the size is fixed or not.

A fixed just keeps the OS from using up 16GB of hard drive space for a page file that doesn't get used much (especially if you using 128GB or smaller SSD as your boot drive). I had one system where Windows allocated a huge page file on an SSD for some reason. Guess I must have run something that requested a lot of address space. However, for most people, I agree, just leave the page file settings alone unless you really do have a reason to change them.

Share this post


Link to post
Share on other sites

goates, on 19 Jul 2013 - 07:54 AM, said:

Trying to save your SSD by moving the page file to RAM also doesn't really make much sense. If your system is heavily using the page file, it means you need more RAM, period.

What's the best way to evaluate paging activity?

 

What value has a page file in a 64-bit system running a 32-bit app (FSX), where there is ample physical RAM? I'd like to see exactly how often memory is paged using either a SDD or RAM based pagefile.sys. I ran this evaluation 5 years ago or so in a system w/ 4GB of RAM in Vista 64. At that time when FSX was running paging was virtually constant. I think it was perfmon that I used to observe this.

 

Right now I have a 16GB page file set up in RAM using imdisk. Works well, but may not be worth the trouble. The key for me would be watching the write activity going to the page file. I understand some applications (maybe FSX) were designed to page more actively than others, though that was internet hearsay.

Share this post


Link to post
Share on other sites

 

 


Except that without the page file the OS can no longer move lower priority data out of RAM (including things like the registry). Many background tasks that get loaded at system start, for example, don't actually do anything until they get called upon, in which case the OS moves their data to the page file. If you have far more RAM than anything you do with the system will require, you can probably get away without the page file, but in most cases it is best to just leave it alone (or at most just set a fixed size).

 

This supports my hypothesis, however irrelevant it may be in the real world, unless as I say FSX tends to page more frequently be design.  Putting the page file on a RAM drive in a box w/ mega amounts of RAM should ultimately be the best theoretical solution:

  • Frees up all available ram by being occupied by 'lower priority items'--unless this is obviated in 64 bit memory management w/ lots or RAM installed
  • Eliminates any paging to the SSD, which can't hurt the SSD
  • Is faster than paging to an SSD or HDD, though as I say, may be completely irrelevant in the system described

Share this post


Link to post
Share on other sites

Theoretically, having a page file (on disk) would allow the OS to page out unused data, allowing RAM to be used for stuff you actually run. The system doesn't just page to disk because it has run out of physical RAM, it might page out data that hasn't been used for ages even if you have plenty of free RAM.

However, computers today have such ridiculous amounts of RAM (8GB is minimum, many have 16GB or even 32GB) that it really doesn't matter. A couple of years ago (ie. when Windows 7 was released), a system might have had 4GB of RAM plus a 6GB paging file or 2GB plus a 3GB paging file (using the arbitrary 1.5x RAM rule for the paging file) - total address space would be 10GB or 5GB - much less than the 16GB of actual RAM many systems have today. I don't think anybody back then foresaw computers with 16GB of RAM.

Basically, it's not worth it to mess with the default settings unless you're running desperately low on disk space. If you are, it's probably time to upgrade your primary SSD or HDD anyway, and the new one will no doubt have at least 2x the capacity of your current drive, removing any concerns about disk space. Leaving the paging file alone will guarantee you won't run into weird compatibility problems with poorly coded applications and gives you the benefit of the crash dump, which can be analyzed to find the offending driver or software.

 

RAM disks are, of course, a total waste of time regardless of your setup. It's like trying to pull yourself up by your bootstraps.

Share this post


Link to post
Share on other sites

This supports my hypothesis, however irrelevant it may be in the real world, unless as I say FSX tends to page more frequently be design.  Putting the page file on a RAM drive in a box w/ mega amounts of RAM should ultimately be the best theoretical solution:

  • Frees up all available ram by being occupied by 'lower priority items'--unless this is obviated in 64 bit memory management w/ lots or RAM installed
  • Eliminates any paging to the SSD, which can't hurt the SSD
  • Is faster than paging to an SSD or HDD, though as I say, may be completely irrelevant in the system described
Just turn the page file off alltogether if you are going to test "all in RAM solution". Using some ram drive for page file really isn't theoretically fastest solution because OS has to write the data to page file format and then you have ram drive software somewhere in the middle handling that data. Optimal way to get everything 100% in the memory is just to turn it off. Just a couple of mouse clicks and you are good to go.

Share this post


Link to post
Share on other sites

RAM disks are, of course, a total waste of time regardless of your setup.

 

How so?  As long as paging activity occurs, and as long as you have enough RAM so that dedicating a portion to become the location for a page file doesn't pull address space away from all applications & processes running, I don't see how RAM disks are a total waste.   As I mentioned when I tested this out many moons ago in a 4Gb system I saw continuous write to disk operations happening when FSX runs, using perfmon I believe it was.   I moved the page file to a 3rd HDD that had only data on it, w/ OS running on #1 HDD and FSX running on #2 HDD.  What I witnessed was virtually constant write activity in chunks from 20 or 30 kb up to 50-100mb per write.   When I then moved the page file to #2 the write activity commenced on #2 HDD and stopped on #3 HDD as expected.  I'm assuming resources to do this are greater on a physical HDD than in RAM as a RAM drive.  I plan to do the same test now but need to find the correct tool to identify page file write activity.  Perfmon may work I haven't looked to see if it's in WIn 7.

Share this post


Link to post
Share on other sites

You may be interested in this thread over on FlightSim.com.

 

http://www.flightsim.com/vbfs/showthread.php?257691-RAMDisk-vs-SSD-tests&highlight=ssd+ramdisk

 

The conclusion was that the RAM disk wasn't worth the trouble, at least in his setup.

As I mentioned when I tested this out many moons ago in a 4Gb system I saw continuous write to disk operations happening when FSX runs, using perfmon I believe it was.

With 4GB I would expect there to be lots of paging activity as FSX could easily use up to 4GB on its own. Add in the OS and a couple background tasks or other programs, and 4GB would fill up pretty quick. With 8GB of RAM there is plenty of room for FSX, Windows and a few other tasks or programs. FSX will still be limited to 4GB itself, but the rest of the RAM would be free for Windows and whatever else is running.

 

Perfmon is still included in Windows. One of the enhancements to Windows 8 was an improved task manager as well, though it still doesn't quite match perfmon or some of the Sysinternal utilities.

Share this post


Link to post
Share on other sites

How so?  As long as paging activity occurs, and as long as you have enough RAM so that dedicating a portion to become the location for a page file doesn't pull address space away from all applications & processes running, I don't see how RAM disks are a total waste.

If you simply take the page file away, there is nothing for the OS to page anymore! It is all in the memory even wihtout ram drive. From software standpoint PF is same memory as main RAM and OS handles what and if it puts something in there, usually when main memory gets full. Programs read and write to the page file without "knowing" which one it is. Thus, if you disable PF, you have only your RAM as memory.

 

But if you don't want to listen what other people here are trying to say, by all means use the ram drive. It is useless nonethless.

Share this post


Link to post
Share on other sites

 

 


But if you don't want to listen what other people here are trying to say

 

I like listening to what people try to say, but that doesn't mean I accept it without questioning or clarifying.  The argument I am making hinges on comments I've read elsewhere that certain applications themselves can affect the OS' paging activity and therefore if that is true and those sorts of apps were built during a time when large amounts of physical ram weren't even considered, and paging activity occurs in an environment that doesn't even look for RAM over 4GB, as is the case in 32-bit OS, then ram drives absolutely provide value because they utilize physical RAM not even visible to that 32-bit OS. As I've already concluded in a 64 bit OS that is likely not the case, but even that argument hinges on whether or not an app such as FSX forces paging activity even when there is ample physical ram.

Share this post


Link to post
Share on other sites

 

 


You may be interested in this thread over on FlightSim.com.

http://www.flightsim...ght=ssd ramdisk

The conclusion was that the RAM disk wasn't worth the trouble, at least in his setup.

 

Here is his summation:

 

While the SSD matched the RAMDisk's FPS performance, Panning, AI movements, and changing Views, did not seem to be quite as smooth. Awesome to say the least, just not quite as good.
As I saw when moving from a spinning drive to an SSD, the faster RAMDisk cut FSX launch, and flight loading times tremendously.

Was it worth it? Yeah, I had a riot doing it. biggrin.png I satisfied my need-to-know, and in all actuality it was not that expensive.


If the results would have been more promising, I was ready to pop for a GSKill DDR3-2400 64GB set. No way, now!

 

It appears he noticed a difference for certain, but that it wasn't worth the effort.  He actually installed FSX completely into the RAMdrive so this test was really much bigger than looking only at paging activity.

 

What we are all tacitly hoping to see some day is a robust simulator engine that exploits modern hardware better than the clunker does:  multicore/multithreading, 64-bit native, DX11+ support, etc.  

Share this post


Link to post
Share on other sites

???

 

Why would you want to emulate a Disk Drive using memory, (RAM,) so you can the use the said emulated HD to emulate memory, (RAM.)

 

Isn't that sort of like robbing Peter to pay Paul?

 

Chris.

Share this post


Link to post
Share on other sites

???

 

Why would you want to emulate a Disk Drive using memory, (RAM,) so you can the use the said emulated HD to emulate memory, (RAM.)

 

Isn't that sort of like robbing Peter to pay Paul?

 

Chris.

+1  It's a dog chasing it's tail

Share this post


Link to post
Share on other sites

???

 

Why would you want to emulate a Disk Drive using memory, (RAM,) so you can the use the said emulated HD to emulate memory, (RAM.)

 

Isn't that sort of like robbing Peter to pay Paul?

 

Chris.

Let's say you were using a 32-bit OS as I was years ago when I first looked into using a ram drive.  My thinking was, since have 8Gb of RAM installed but my OS can only access 4Gb I can trick the OS into using the extra 4Gb of physical ram because it sees it as a disk drive.  So in this scenario, if when I run an app like FSX loaded fully, my 32-bit OS will write to the page file.  This write activity is much faster to a ramdrive compared to a fast SSD.   This is the basis of my assumption and I'm open to other interpretations for sure.   I've mentioned here it may truly be meaningless now in the 64-bit OS domain anyway.  

Share this post


Link to post
Share on other sites

And if you left the RAM alone, the OS would not need to generate all of the unnecessary page faults to access redundant RAM masquerading as a dirve. A RAM Drive is a DRIVE, not RAM, just a rather quick one.

 

All you end up doing is giving the system more work and the only thing you will achieve is to slow the system down.

 

Chris.

Share this post


Link to post
Share on other sites

Here is a link to some Microsoft articles on how Windows memory is managed.

 

http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx

 

I read that series by Mark Russinovich a few years back.  It convinced me to keep a pagefile (which I ran without for a time). 

 

A couple of points come to mind:

  1. A ramdisk will reduce the amount of physical RAM available to the system cache and to applications.
  2. "1." may not matter once you've exceeded a certain threshold of physical RAM.  

Consider three scenarios:

  • You have 8GB of physical RAM and a 2GB pagefile on a hard drive, your commit is 6GB, that will leave 2GB for cache and/or other processes in physical RAM plus the 2GB for swap.  
  • You have the same 8GB of physical RAM, but you've allocated a 2GB ramdisk for the pagefile.  That 2GB ramdisk will now count as committed memory and be added to the previous 6GB, leaving you with 0 cache and 0 available RAM, but a fast 2GB swap file.  It's probably not going to end well.
  • You have 16GB of physical RAM and you've allocated a 4GB ramdisk for the pagefile.  Your 4GB + 6GB (previous) commit is now 10GB, leaving you 6GB for cache and other applications (more than you had in the first scenario).

So the affect of a ramdisk pagefile will depend on how much total physical RAM is in your system, and how much headroom you had before allocating it.  If your available physical memory was never below 10GB, then allocating some of it to a ramdisk shouldn't hurt and might help.  If your minimum available was 2GB, then it might not be a good idea.

 

As with most things, I could be wrong (as my wife says) and YMMV (not an airport in OZ).

Share this post


Link to post
Share on other sites

 

 


All you end up doing is giving the system more work and the only thing you will achieve is to slow the system down.
 
Chris.

 

Not when the alternative is to page to a HDD or even an SDD.


 

 


So the affect of a ramdisk pagefile will depend on how much total physical RAM is in your system, and how much headroom you had before allocating it.  If your available physical memory was never below 10GB, then allocating some of it to a ramdisk shouldn't hurt and might help.  If your minimum available was 2GB, then it might not be a good idea.

 

This, of course, was where I was going.  I have 32Gb of fast ram so the question was worth posing.  From what I gather though, and depending on how much influence the application has on paging activity in the OS, it may be quite moot in the 64 bit OS where the memory over 4Gb can be directly accessed by the OS for other system processes, even though FSX will only use what it can as 32-bit.  I think also even though RAM is way faster for read/write than an SDD even that may not offer much since the SSD isn't holding back much anyway.  I decided to create a fixed page file of 1Gb min/max and while using performance monitor I saw no paging during a longish flight in FSX, so indeed, it's moot in terms of a page file, and like almost moot even when loading other parts (or the entire FSX install) on a big ram drive.

Share this post


Link to post
Share on other sites