Jump to content


Photo

Benchmarking Methodology


  • Please log in to reply
22 replies to this topic

#1 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 10 July 2013 - 10:59 AM

Please refer to the Performance Benchmarking article to lean how to contribute benchmarks. This forum may be used to post suggestions, questions, data and anything related to STEP benchmarking.

#2 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 15 August 2013 - 01:06 AM

I updated the OP with some suggested testing protocol that we have up in the MT admin area. These are just suggestions that I am using myself. I'll post the performance data for STEP:Core on my system shortly.

#3 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 23 August 2013 - 01:38 AM

OP updated with link to reliable benchmark data showing performance cost (in VRAM) of STEP:Core and the effects of DDSopt on vanilla performance.

#4 MallaVelve

MallaVelve

    Citizen

  • Members
  • Pip
  • 52 posts

Posted 07 October 2013 - 08:19 PM

This is the first place I have seen Scenic Carriages as a STEP:Core Mod. Perhaps it should be put on the Core page. I will install it now.
  • 0

#5 WilliamImm

WilliamImm

    Legendary Blue Dragon

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1,570 posts

Posted 08 October 2013 - 10:08 AM

It seems to be left over from earlier testing. Plus, I'd recommend Touring Carriages instead.

#6 Slaughtz

Slaughtz

    Prisoner

  • Members
  • 1 posts

Posted 29 November 2013 - 05:38 AM

EDIT: my apologies answer was found here: http://forum.step-pr...ad.php?tid=3081
although, i still don't understand the difference between the two and how i should be reporting them.

GPU Load
GPU Temperature
Memory Usage (Dedicated)
Memory Usage (Dynamic)

What should I be looking at for the Dynamic/Dedicated Memory usage?

Posted Image
Date , GPU Core Clock [MHz] , GPU Memory Clock [MHz] , GPU Shader Clock [MHz] , GPU Temperature [°C] , Fan Speed (%) [%] , Fan Speed (RPM) [RPM] , Memory Used [MB] , GPU Load [%] , Memory Controller Load [%] , Video Engine Load [%] , VDDC [V] ,
Thanks
  • 0

#7 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 05 December 2013 - 02:48 AM

EDIT: my apologies answer was found here: http://forum.step-pr...ad.php?tid=3081
although, i still don't understand the difference between the two and how i should be reporting them.

GPU Load
GPU Temperature
Memory Usage (Dedicated)
Memory Usage (Dynamic)

What should I be looking at for the Dynamic/Dedicated Memory usage?

Posted Image
Date , GPU Core Clock [MHz] , GPU Memory Clock [MHz] , GPU Shader Clock [MHz] , GPU Temperature [°C] , Fan Speed (%) [%] , Fan Speed (RPM) [RPM] , Memory Used [MB] , GPU Load [%] , Memory Controller Load [%] , Video Engine Load [%] , VDDC [V] ,

Thanks

Your video card apparently either does not have the sensors or GPU-z does not support them. You may need to use another utility or determine if you have the most updated version of GPU-z.

Newer cards usually report on-card and system-swapped GPU memory. Ideally, the swapped should not increase much during gameplay. If it does, that indicates that VRAM is maxing out and paging more to system RAM to compensate (this leads to stuttering).

#8 karavenstone

karavenstone

    Prisoner

  • Members
  • 1 posts

Posted 10 October 2014 - 12:20 AM

These are my benchmarks from my recent loading of STEP v2.2.9. I hope they provide useful data for someone. ::):

 

Attached File  STEP v2.2.9 Installation Benchmarks.xlsx   188.1KB   7 downloads


Edited by RavenstoneMods, 10 October 2014 - 12:22 AM.

  • 0

#9 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 13 October 2014 - 08:53 PM

Thanks for submitting the data!



#10 Xaviien

Xaviien

    Commander

  • Contributors
  • PipPipPipPip
  • 232 posts

Posted 01 February 2015 - 12:17 PM

Your video card apparently either does not have the sensors or GPU-z does not support them. You may need to use another utility or determine if you have the most updated version of GPU-z.

Newer cards usually report on-card and system-swapped GPU memory. Ideally, the swapped should not increase much during gameplay. If it does, that indicates that VRAM is maxing out and paging more to system RAM to compensate (this leads to stuttering).

I definitely have the most updated version of GPU-z. Any idea how I can find out if my card has the sensors?

The previous update of GPU-z states: "Added full support for GTX970" but I seem to only have the same options available as Slaughtz, closest being "Memory Used [MB]".

 

Do you have a suggestion on another monitor to try? Googling isn't helping much. Apparently the other most recommended monitors don't log.

 

Where do we post benchmark info? I know it will probably be useful to myself while modding, but this page says "We greatly appreciate any help from our users in accumulating benchmarks according to the procedures outlined here." and I'm interested in helping with STEP as much as I can.

The fact that I haven't even made it to STEP 2 yet is worrying though. :ermm:


  • 0

#11 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 01 February 2015 - 01:54 PM

It is tough to say why some cards report certain numbers and other cards report other numbers. I use AMD cards, and they all report dedicated and dynamic VRAM usage. I'm not sure what is typical for Nvidia, but is is most useful if you can monitor on-card versus swapped. If you can't then try some other utility for measurement (I don't know of any off the top of my head, but try Google)

 

Without knowing exactly what numbers you card is reporting, I can't say what is what. GPU-z has options that you need to select for logging, else the data won't be reported.

 

You can post your stats right here on this thread as a file attachment. You can also link to your data, but if it is an image, it won't be of much use.

 

I have not begun aggregating any performance data yet accross systems. That is a to-do after we get STEP 2.3.0 up and running.



#12 GrantSP

GrantSP

    The antipodean

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,012 posts

Posted 01 February 2015 - 05:38 PM

You might try HWInfo. For quite a while I had trouble getting correct details about my laptop's dedicated GPU, only the Intel GPU would show up, HWInfo provided me with all the background info on my entire machine.



#13 Xaviien

Xaviien

    Commander

  • Contributors
  • PipPipPipPip
  • 232 posts

Posted 01 February 2015 - 09:44 PM

Thanks for the fast responses.
I'll try HWinfo, and I'll try to find a GPU-z forum to see if I'm just missing something.
 
EDIT: HWinfo only gives the option for "Memory Usage" as well. I'm guessing it's a limitation of the graphics card or something.
I've posted in the GPU-z forum, so maybe the W1zzard can give me some answers!
 
Here are screenshots of my GPU-z UI. I'm assuming you get options for Memory Usage (Dedicated/Dynamic) where I only have the option to log Memory Used?
I guess for now, I'll just need to log Memory Used and see if it peaks above 4GB.
 
bs5.pnge2d.png
 
 
EDIT: Another point: The Benchmarking Guide says the first benchmark should include SKSE, but in the STEP Guide the first

Benchmark32.png Benchmark icon comes just before SKSE installation. At which point should the first Benchmark be done? Before or after SKSE?
 
 

Vanilla

  • Base vanilla game
  • All DLC
  • SKSE
... no more & no less please.

 

Also, do we need to get the FPS log to line up with the GPU-z logs?


Edited by Xaviien, 01 February 2015 - 11:44 PM.

  • 0

#14 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 03 February 2015 - 04:15 PM

Before/after SKSE does not matter much. Your choice. We are just trying to get people to measure and share.

 

FPS and GPU logs will not line up perfectly unless you use the same app to measure both. This is fine, as long as it is the same run.



#15 Xaviien

Xaviien

    Commander

  • Contributors
  • PipPipPipPip
  • 232 posts

Posted 04 February 2015 - 01:02 AM

Ok no problems.

I actually worked out how to line up the FPS with the GPU-Z stats, which also helps clear the noisy data. See the below screenshot if this doesn't make enough sense.

  1. Import GPU-Z data to spreadsheet. (Sort ascending by column A, if you have issues with extra empty lines)
  2. Open FRAPSLOG.txt from the FRAPS Benchmarks, and take note of the time you began logging FPS.
  3. Copy the data from the TESV.....fps.csv and paste it into you the spreadsheet from step 1, ensuring the first FPS entry lines up with the corresponding time from the GPU-Z Data.
  4. Delete all the GPU-Z data rows which do not have a corresponding FPS entry. In the example below, this would be row 2 through row 17, and row 266 through 273 (not shown)

You now have a clean benchmarking log, which runs from the time you hit F11 to start logging, until you hit F11 to end logging.

Benchmarking%20Guide_zpsvoz2lwhd.png
 

 

 

I also received a reply from W1zzard, the creator of GPU-Z regarding the Dedicated/Dynamic memory usage issue:

from http://www.techpower...9/#post-3232410
On most NVIDIA cards there is no Dedicated/Dynamic (which is obtained through DirectX), instead GPU-Z uses the memory usage reported by the NVIDIA driver, which seems more accurate.
AMD has no such feature in their drivers, so GPU-Z always uses the DirectX memory usage.


Edited by Xaviien, 05 February 2015 - 12:26 AM.

  • 0


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users