Sign in to follow this  
Guest

Framerate Benchmark

Recommended Posts

Seeing the debate about framerates in this thread:http://ftp.avsim.com/cgi-bin/dcforum/dcboa...orum=DCForumID8I wonder if there is (or we should create) a benchmark test scenario so that framerate claims can be compared legitimately.For example, we could have the "Meigs Standard". Something like:- Default Meigs- Default Cessna 172 on the active at 36- Display Scenary/Aircraft Sliders maxed. All boxes checked.- Performance - Resolution 1024*764*16, Target Framerate Unlimited, Sliders maxed, boxes checked, Trilinear Filtering.- Realism - Hard- MS Default Clouds Broken - that defaults to 5/8 cumulus from 5000' - 8000'- Visibility Unlimited- Summer / Day SettingsOnce fired up, switch to External View (ie hit S 3 times on entry). Wait for the camera angle to settle down.Shift Z for the frame rate, and print screen for the proof.You should get something like the shot below. Yep - I using a 400Mhz Compaq Armada E500 laptop, Rage Mobility Card, and 320Mb RAM :).Anyway, the beauty of this is that the set up is pretty much the standard initial set up when you enter FS - you just have to max everything out and make a few tweaks to the weather. Also, I guess FS9 will have this as the default setup too. We would have to state which version of FS we using when the time comes.Thoughts?RichardPS Avsim, please don't scrap this 'cos of the shot, I'm trying to make a point here.

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

How's this...Just something to prove a point.

Share this post


Link to post
Share on other sites

Seems to work with a totally different system. But where are your aircraft on the ramp? You should get a whole line of them with everything maxed.

Share this post


Link to post
Share on other sites

Hi Richard,Unfortunately, that wouldn't work very well. As you can see, there are already differences in setup with just one reply: the zoom setting, resolution, ini tweaks, etc. (from what it looks like).The only real way to do an apples to apples comparison is with a benchmark that is setup exactly the same between systems. Unfortunately, there is none in FS. There used to be a project to do exactly that with FS at a sim site, but it seems to have fallen off the radar.What is needed is for FS9 to include its own built-in benchmark like most other games today - one that controls all variables. Sadly, this has been taboo in the sim genre for some reason (I have my ideas), that is: until Comanche came along. FS desperately needs this functionality in the next version, not just for us FS or hardware junkies but to give the sim genre and FS in particular a much wider focus on the extremely popular hardware sites. Until that happens, sims will always stay a declining niche genre. And that hurts us all.To benchmark FS properly today, you need a lot of time and some specific tools: FRAPS to get minimum, average and maximum frames, a set of recorded flights to play back bench specific general situations and scenery, the exact same addon list (or revert to stock FS which is hard to do for many with hundreds of addons - unlike most FPS type games), an included .ini file that keeps settings exactly the same across all systems and a custom front end scripting utility that runs specific video card settings. Thats a pretty tall order for most in this hobby, even if all the pieces were in place.What you suggest doesn't hurt of course, but it just won't be apples to apples - pretty much defeating the purpose.Take care,Elrond---Not enough bandwidth to display this signature! Don't reformat hard drive? (Y/N)

Share this post


Link to post
Share on other sites

Valid points as ever, Elrond,Few of us have the time or inclination to set up such exhaustive test conditions for true benchmarking.Which is why I still think my suggested approach (OK Benchmark may be too strong a word), would go someway to alleviating some of the extremes that we come across when claiming framerates. Paul's examples in the other thread exemplify my point. If FPSFREAK's screenshot had been of the "Meigs Standard" or whatever we choose, more credence may have been given to the claim, if you get my point.Happy to drop it, it's just a suggestion. What do others think?Richard

Share this post


Link to post
Share on other sites

I notice you're not using anti-aliasing, anios-tropic filtering, I have mine all maxed to 4X AA/8X AF/32Bit colours @ 1024x768. I am using Chris Willis FS Sky World SE with ActiveSky wxRE.Anyways enough jabbering.

Share this post


Link to post
Share on other sites

I just joined so bear with my ignorance. Exactly what is the 'Miegs Standard"?. That screenshot was taken at 1024x768x32/6X AA/2x AF. All sliders except for weather were set to max. I'm all for a simulated Benchmark for Flight Sims. Maybe I'm reading into the post but is there something in your point you want to make but won't come out and say it. I was just trying to proving a point about being able to carry 50+ FPS. I took the shot while flying as this is surely a more intensive test than sitting idle on the runway. No offense to those who used that as the situation.Come on guys, with all the brilliance on this forum we need a group to get together and work on this.Any suggestions on settings that maybe I could have changed?Bobby

Share this post


Link to post
Share on other sites

I did have AA / AF on - suspect is is a limitation of the Rage Mobility. 16 FPS is impressive given your sub 1ghz, sub 512 Mb RAM machine. Goes to show what a decent graphics card can do (GeForce 4 Ti4200).

Share this post


Link to post
Share on other sites

No offense taken. Meigs Standard - I made it up today :). Read my first post. Basically, it is a simple way of having a very similar environment to test FPS's for all users. Elrond makes it clear that this is nowhere near a clinical Benchmark Test. However, it provides some form of relative assessment. I am not trying to establish a killer environment to challenge your system. At the moment, I am just trying to guage support for this. If people believe it is too unscientific - then so be it. However, in the absence of anything else...Bobby, why don't you follow the guidelines in the first post, and show us what your Framerates look like?Richard

Share this post


Link to post
Share on other sites

There is a very nice FS2002 benchmark program in th AVSIM library by Maurizio Losso.http://library.avsim.com/search.phpSearchT...=Root&Go=SearchI've found this to be an excellent tool for tuning my system. I can change resolution, increase the level of AA or anisotropic filtering, change the in-game settings, or whatever -- and the program will give me some hard data on the effect of this change (rather than just eyeballing the fps counter). When the benchmark has run, there is a very nice graph that shows the average sustained rate and all of the peaks and valleyes.The benchmark comes with an autopilot flight in a heavy over New York, but you can just as easily set up a flight in your favorite plane in your favorite area so that the results are reflective of your normal flying. Or, maybe set up a worse-case scenario like flying on approach to LAX at night with thunderstorms. Once you run this benchark and make some adjustments as appropriate, you'll have a very good idea of where to set your max fps slider (mine is at 23).As someone eles mentioned, comparing fps between systems is really difficult. There are just too many variables. Not only do we have video card settings, resolutions, all of the in-game sliders, but things like the AI slider will have different results for people who have installed custom AI flight plans and AI aircraft. Then there is add-on scenery. custom clouds, it just goes on and on.Nevertheless, it would be fantastic if MS offered a benchmkark utility in Century of Flight. Something like the Unreal Tournament benchmarks would be great. This would allow the hardware sites to integrate this benchmark into their reviews (increasing the visibility of the product), and it would give each of us a good, standardized litmus test for what happens when we change hardware or make a setting adjustment.In the meanwhile, I recommend the benchmark program in the link above. It's a great little tool, and has allowed me to quit staring at the fps counter.

Share this post


Link to post
Share on other sites

16FPS is pretty good, but it gets better, I had a few programs running in the background, actually CFS2 (lol), It is suppose to be 512MB ram but the Ares I have is not coming up as 256, trading it in for Acer saturday.

Share this post


Link to post
Share on other sites

Rich, Glad you didn't take offense. LOL This is Meigs set up to exactly what you stated. The only difference is I'm running 6X AA and 16X Anis. Also running 8 Lights.(You said all sliders maxxed LOL) No its not 50 FPS and before someone posts it's not 75 LOL. But for what I want it more than puts a smile on my face. I'm going to try that other benchmark mentioned in this thread.Bobby

Share this post


Link to post
Share on other sites

Most of what you said is valid Elrond but I don't see the need for an actual flight.A benchmark, IMHO, is not to show who has the biggest or fastest but to offer a comaprison between RAM, CPU & graphics cards and I submit this can be done with a static view as long as each person uses the EXACT same settings.Some years ago, in the land of Compuserve, a number of people were involved in setting up a standard FS setting and asking people to run FS with this setting and posting their specs and results. This was organised into a spreadsheet by one guy so we could all see which combination of h/w got best results.The standardised settings were done using a modified fs cfg file and a situation file (for some weather effects) with instructions on how to create an icon to run this setup for testing and leave your own fs settings intact. So there was no need to keep changing FS config each time you wanted to compare fps.As you point out, the two screen shots attempting to compare fps do look to have different setups so the fps figures are meaningless as comparision figures.Rgds

Share this post


Link to post
Share on other sites

1280X1024X32, windowed mode maximizedP42.66@3.13, dual channel DDR, GeForce Ti4600On my machine Fs2002 windowed mode FPS is higher than in full screen even in lower resolutions. For best performance window should be stretched over whole screen and then maximized. I noticed big FPS drop when restore window is significantly smaller than maximized window.Cheers, Petar

Share this post


Link to post
Share on other sites

Hi Vulcan,You bring up good points: indeed, for setups to be comparable all FS variables must remain the same.While a static shot would work to show some hardware differences (CPU mostly, video card a distant second), it fails to stress the other parts of a machine. A few flying videos played back in strategic areas of FS start to show the overall difference between machines because it stresses CPU, video card, memory and drive performance all at once.When it comes down to it, a good benchmark needs to stress individual pieces of hardware on their own (such as your static shot for the mentioned video card and IQ) as well as total system performance. As long as your guidelines are rigid as far as setup and cfg, doing so would give the best comparison results from box to box.The final piece is to create a front-end that controls external FS variables such as AA, AF, LOD, resolution and bitdepth on multiple runs. If it was made with scripting abilities and could control FS automatically between runs - and even more important to graph and provide final results from all aspects, all the better. If all of those pieces were put together, you'd have a great FS benchmark suite.Take care,Elrond---Not enough bandwidth to display this signature! Don't reformat hard drive? (Y/N)

Share this post


Link to post
Share on other sites

Well ... here we go. 1024x768x16 is a bit of a grimy image IMHO. Much prefer the 1600x1200x32.

Share this post


Link to post
Share on other sites

6X AA with those jaggies in your screenshot? Compare your shot to shane737400CDN's above, which depicts 4X AA.No offence, but it looks like AA is off to me.

Share this post


Link to post
Share on other sites

Performance figures for my rig at the specified settings are 32.2 fps for Quincunx AA (what I use normally) and 40.0fps sans AA. Interestingly, I don't usually see fps this high in normal use of FS. A healthy weather pattern compliments of Activesky WxRE and complex scenery textures in areas like Oahu usually keep me in the 15 -> 25fps (capped) range.System Specs:P4 2.4GHz @ 2.7GHzAsus P4PEGF3 Ti200 at stock GF3 speedsOnboard sound

Share this post


Link to post
Share on other sites

Here is mineWinXP HomeXP 2100+756 High Performance CAS2 mem (pc133...lol)Radeon 9700pro (Omega drivers 3.1)Gigabyte MoboCarlos

Share this post


Link to post
Share on other sites

Shane,I noticed you have a GeForce 4 TI4200 installed in your P3 866Mhz system. I also have a P3 but with 933Mhz. I wanted to install the TI4200 also but was concerned my 200W power supply would be insufficient. Assuming you have the same power supply, have you run into any problems ?ThanksJohn

Share this post


Link to post
Share on other sites

Hi Elrond,Point taken about stressing various parts of the system with a flight, also back then most people didn't bother with AA, AF, & LOD so that made the set-up a little easier :-)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this