Jump to content


Photo

uGridsToLoad Limit: maximum allocated RAM on 64bit < 3GB?


  • Please log in to reply
49 replies to this topic

#1 PatimPatam

PatimPatam

    Prisoner

  • Members
  • 10 posts

Posted 31 May 2012 - 03:57 PM

Preface: Hello there, i've got a tricky question (i think) about HW + modding, probably aimed only to people with high-end rigs (8GB or more of RAM + 3GB or more VRAM [non-splitted] + SDD + Windows 7 64bit) that have tried to push Skyrim to the limit.. (for reference i've got a i7-2600K 16GB DDR3 + GTX580 3GB slightly OC + windows n game installed on 128GB SDD). I'm using UGRIDS=9 and about 60+ mods (mostly texture mods from STEP), including a combination of Skyrim HD, Realistic Overhaul and Serious HD, all of them at the maximum quality available. Still running on Skyrim 1.4.27 since NVidia Ambient Occlusion for me is a must and apparently is still not solved. Short version: As i understand since v1.3 Skyrim should be able to use up to 4GB of RAM memory (the maximum for a 32bit application on a 64bit OS).. has anyone actually managed to have it working with anything above 3GB?? for instance TESV.exe using something like 3.5GB according to windows task manager? if so did you do anything special to archive that?? Long version: As we all know there are lots of myths and theories regarding the value ugrids and that can lead to "instability", same thing about using lots of high-res texture mods.. Well i've got the theory that the only real problem when increasing this value or the size of the textures is either lack of VRAM or lack of RAM.. I've got a very stable game using UGRIDS=9 and only recently i found a few specific outside locations where the game would CTD after walking from another outside location (would not CTD if i load a savegame already on the location).. I'm pretty sure VRAM is not a problem (using always less than 2GB - 65%). Total amount of memory is obviously not a problem either (16GB).. However checking my main memory usage i realized CTDs always happens when the game tries to use more than 3GB of RAM. I think that happens when "usable" RAM is already almost filled and when you move to a new area with lots of new resources that have to be loaded there's just no space to put them. I have spend quite some time googling and checking both HW and Skyrim forums with no luck.. have tried using Game Booster 3, changing boot options from the command line "bcdedit /set increaseuserva 4096", changing Skyrim.ini iMaxAllocatedMemoryBytes=4294967295 (4GB) and still the same problem. So is it really an issue of software-limited maximum allocated RAM per process? If so is there a way to use more than 3GB? Or am I totally wrong and it's just a problem of the GPU/CPU not being able to cope with so much information? I'm a bit annoyed because i have the feeling that if i could really use the whole 4GB as advertised then all my CTD's would be gone and I could even possibly run with all textures mods + UGRIDS=11. I know there are other ways i could go about this like optimizing the textures, reducing settings to use less memory, etc but that's not the point of the question for now.. Any ideas?? Thanks in advance!
  • 0

#2 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,313 posts

Posted 31 May 2012 - 04:00 PM

I am guessing that Windows memory management may be the culprit ....

#3 stoppingby4now

stoppingby4now

    Sleepy

  • Site Founders
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,281 posts

Posted 31 May 2012 - 07:07 PM

There aren't any inherent memory limits placed on applications. If they request memory and it's available, they will get it. Game engines themselves aren't very big, it's all the textures and resources that they load. In general, the only things that will reside in system RAM for a game is pre-cached data, or data that was recently used but being kept around "just in case". I highly doubt that its Windows fault, but more likely a momentary spike of the game needing to load a large amount of textures to the video card and it not managing the process very efficiently to deal with what is already loaded.
  • 0

#4 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,313 posts

Posted 01 June 2012 - 12:49 AM

I am guessing that Windows memory management may be the culprit ....

Sorry, I should have been more specific: I mean if Skyrim is never utilizing more than 3 gigs, then it could be Windows memory manager not playing nicely with certain processes and graphics hardware software memory managers. Then throw in the game's memory management, and we have several memory handlers that don't necessarily work efficiently together, resulting in paging out to disk and increasing chances of head-on collisions ... = CTD.



Just speculation though. s4n is probably right, alluding to the issue being with the game's memory management limitations (hence the uGrids bugs with increasing values above 5)

#5 PatimPatam

PatimPatam

    Prisoner

  • Members
  • 10 posts

Posted 01 June 2012 - 03:45 AM

Thanks a lot for the feedback guys, what s4n said makes sense.. i guess it could be just a coincidence that this always happens when it goes exactly above 3GB. Will have to either save often and deal with a CTD from time to time or suck it and cut down on some settings.. will see. Still would be nice to hear if someone else with similar or better specs than mine can confirm this!
  • 0

#6 frihyland

frihyland

    Dwemer Lord

  • Site Founders
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,087 posts

Posted 01 June 2012 - 07:01 AM

By default a 32 bit windows application will never ask for more than 3GB of memory from the system, memory management above that would have to be specially coded. One of the primary differences between 32 and 64 bit applications is the radical increase in memory supported, from 3GB to 64GB but until consoles go 64bit we won't see any games that do. Since skyrim was coded for consoles and only ported to pc later and it only supported 2GB on launch, its more than a little unlikely that this special above 3GB code has been written.

Actually getting some use out of that much ram is rather a problem, one solution is to buy a ramdisk application that turns your unused memory into a virtual harddisk that is roughly 8000 times faster than the fastest ssd. You can copy all of your skyrim directory here and launch skyrim, needless to say load times will be a thing of the past and so should any ctd's or slowdowns involving any type of memory management or disk management. FYI if you encounter any type of drm issues you can probably circumvent them by judicious use of symbolic links, ask again if it arises and many here could help with that.

Additionally there are special ini settings which should increase ram usage for specific game uses. Many of these have been tested fairly exhaustively back in November with little to no effect on memory usage or gameplay excepting uGridsToLoad and uExterior Cell Buffer. Posted merely in case you want to try them out of curiosity to see if something has changed since then.

In skyrim.ini


[General]
uGridsToLoad=9
uExterior Cell Buffer=200
uInterior Cell Buffer=18
fMasterFilePreLoadMB=200.0000
iPreloadSizeLimit=419430400

[Papyrus]
iMaxMemoryPageSize=16384
iMinMemoryPageSize=4096

[BudgetCaps]
uActorMemoryBudgetCap=20971520
uLoadedAreaNonActorMemoryBudgetCap=371195904
uWaterMemoryInterior=20971520
uTextureMemoryInterior=20971520
uGeometryMemoryInterior=20971520
uWaterMemory=10485760
uTextureMemory=20971520
uGeometryMemory=10485760
  • 0

#7 PatimPatam

PatimPatam

    Prisoner

  • Members
  • 10 posts

Posted 01 June 2012 - 09:27 AM

Thank you frihyland, looks like some good advice in there. Will try both things as soon as i can and report back!
  • 0

#8 Vypir

Vypir

    Prisoner

  • Members
  • 47 posts

Posted 01 June 2012 - 10:50 AM

I know this doesn't answer your question, but Tom's Hardware had a good piece about using large amounts of RAM for gaming. It looked at using RAM disks for swap and temp files. You might want to give that a try and see if you can eke out some more performance with all that extra RAM available.
  • 0

#9 stoppingby4now

stoppingby4now

    Sleepy

  • Site Founders
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,281 posts

Posted 01 June 2012 - 06:32 PM

By default a 32 bit windows application will never ask for more than 3GB of memory from the system' date=' memory management above that would have to be specially coded. One of the primary differences between 32 and 64 bit applications is the radical increase in memory supported, from 3GB to 64GB but until consoles go 64bit we won't see any games that do. Since skyrim was coded for consoles and only ported to pc later [b']and [/b]it only supported 2GB on launch, its more than a little unlikely that this special above 3GB code has been written.

A 32 bit windows application can reference a full 4GB of RAM, and thus it can also request more than 3GB. But requesting and actually being granted are two different things. The supposed 3GB limit on Windows 32-bit systems is constantly misunderstood and thus misrepresented and has to do with hardware addresses being "memory mapped IO reservations" in Windows. It's a horrible design.

To make matters worse, 32-bit Windows allocates up to 2GB of system memory for the kernel, leaving the rest for user space. Applications requesting more memory than is available in user space can still be granted the extra memory that is reserved by the kernel, but it's not guaranteed. Microsoft introduced the /3GB switch in order to help special cases by forcing the 32-bit kernel to only reserve 1GB, leaving 3GB available for applications (more like ~2.3GB after hardware addresses eat up what's left).

Point being, on a 64-bit system with sufficient RAM, a 32-bit application should have no problem using 4GB of RAM.
  • 0

#10 frihyland

frihyland

    Dwemer Lord

  • Site Founders
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,087 posts

Posted 02 June 2012 - 08:45 AM

Correct, I simplified the problem slightly for readability. Since we are getting techie 64 bit windows also reserves memory for OS and hardware but since it can address more than 4GB as long as you have at least 6GB RAM on your motherboard your 32 bit applications should be able to access their full 4GB alottment (on a 32 bit system this can never happen without special software), in practice this doesn't really ever happen though that I have seen.
  • 0

#11 stoppingby4now

stoppingby4now

    Sleepy

  • Site Founders
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,281 posts

Posted 02 June 2012 - 04:32 PM

Exactly. Even in games with much higher resolution textures, I never see any large amount of RAM usage approaching anywhere near 4GB. The majority of the textures should be in VRAM except when cached by the application. Even then, there shouldn't be any necessary case where it would need to cache that much data.
  • 0

#12 Alister

Alister

    Prisoner

  • Members
  • 15 posts

Posted 07 June 2012 - 09:18 AM

i have a amd phenom x2 555 unlocked to a quad at 3.2 a vertex 2 ssd and a radeon hd 6970 2G running at 930mhz interested in trying to get the most of skyrim was the other persons issues due to I/O speed of the ssd and would a ramdisk help with higher ugrids? would 16 Gb or ram be enough for step most high res textures and mods ect my pc using about 1GB of 4 after boot with win 7 x64 any constructive input appreciated im sorry that you dont read posts without punctuation yet do take the time to make posts that dont serve much point there was a point and it was kinda rude but i think staying on a high horse isnt a good thing but you might disagree with me
  • 0

#13 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,313 posts

Posted 07 June 2012 - 01:21 PM

Sorry Alister. I just don't read posts without punctuation. This is a BB after all and not a chat ;)

#14 Alister

Alister

    Prisoner

  • Members
  • 15 posts

Posted 08 June 2012 - 04:49 AM

thought better of it and removed
  • 0

#15 Vond

Vond

    Nyarlathotep

  • Members
  • PipPipPipPipPipPipPipPipPip
  • 678 posts

Posted 08 June 2012 - 05:06 AM

Well your post was very hard to read due to the fact that you didn't use puncuation. And you did it again in the reply aswell. Please, just edit the original post with punctuation so that it's easier to read and I'm sure you'll get constructive input.
  • 0


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users