post-process triple-monitor stretch/fish-eye distortion fix

[denali 74]

I am working on a post-process to fix the fisheye/stretching distortion that plague 3 monitor views.  I already have satisfaction, but it does not stretch all the way across all three monitors, more like 1/2 of the side monitors, and it uses lazy math (it's a prototype).  There is no noticeable affect on framerate.   I am working on the correct math and filling all three monitors (my math ain't so good since I quit dealing drugs).

 

I would like to see the strip on my base leg or even downwind, and not have the geometry distorted from what I'd see IRL.  That and a readable scanable panel are my goals.

 

Here is a quick summary of how to get what I have already functioning.

 

Open notepad, but with administrator rights (right click / run as administrator).  

 

From here on "context menu" will mean "right click".

 

Paste this into notepad:

<?xml version="1.0" encoding="UTF-8"?>

 

<SimBase.Document Type="AceXML" version="1,0">

 

    <PostProcessEffects.PostProcessDefinition>

        <EffectName>AntiStretch 155</EffectName>

<ShaderFileName>AntiStretch155.psh</ShaderFileName>

        <ShaderEntryPoint>PsAntiStretch155Main</ShaderEntryPoint>

        <ColorRequest>TRUE</ColorRequest>

        <DepthRequest>FALSE</DepthRequest>

    </PostProcessEffects.PostProcessDefinition>

 

</SimBase.Document>

 

Save in C:\Program Files (x86)\Lockheed Martin\Prepar3D v2\ShadersHLSL\PostProcess as

AntiStretch155.xml  (adjust for your installation location) ( I'll move/re-install mine to my separate F: drive when I get P3D2 stable.  I think I missed that choice during the installation.)

 

Be sure to save as type "all files" at the bottom of the save dialog, or it will have a txt extension.

 

(comments are left out for brevity, but I'll put them in with a finalized function if/when I get to it.  I have a 7-month pregnant wife making unreasonable demands of me lately)

Clear the contents of notepad and paste this: 

 

#include "Quad.sh"

 

Texture2D<float4> srcTex;

 

float4 PsAntiStretch155Main(PsQuad vert) : SV_Target

{ 

float normFactor = 0.5f;

float normX = vert.texcoord.x - normFactor;

float normY = vert.texcoord.y - normFactor;

float X = vert.texcoord.x;

float Y = vert.texcoord.y;

 

    uint2 uTDim;

    srcTex.GetDimensions(uTDim.x,uTDim.y);

 

float colorUa = normX * normX * normX * uTDim.x;

float colorUb = colorUa * 3;

float colorU = colorUb +  uTDim.x * X;

 

float colorV = normY + normFactor;

 

    int3 iTexCoord = int3(colorU, uTDim.y * colorV, 0);

 

float4 color = srcTex.Load( iTexCoord );

 

return color;

} 

 

"Save as" in the same location as above (C:\Program Files (x86)\Lockheed Martin\Prepar3D v2\ShadersHLSL\PostProcess) as

AntiStretch155.psh .  Again, Be sure to save as type "all files" at the bottom of the save dialog, or it will have a txt extension, and P3D2 will not be forgiving.

 

So that is a "High Level Shader Language" shader.  It's c programming language with some DirectX additions, only scripted, not compiled.  You have to re-start P3D2 for every change, AFAIK so far.  This is one of what I believe the Oculus Rift will need to work, but there is probably plenty of code to source from for the math already.

 

Now to load it into P3D2:

 

Quick summary, right click (on the view you wish to augment) / custom camera / save location.  While you're experimenting, put a short name, like "as" (one handed) in the Name box.  Then override FOV and play with the Field of View settings.  There are FOV calculators available on the web.   Settings will have a stretch/compress effect on your view.  Play with the zoom and other settings.  

 

At this point because of the bad math, precise FOV isn't important, or even really possible.  But you will need a wider view.  I've used 155 x 45, but you can really do whatever you want and maybe do much better.

 

Now attach the shader.  Under the camera effects / effects dropdown, look for AntiStretch155 (did you save that notepad file as "all"?).  Leave the excludes out.

 

You can preview the view, and even grab the edges of the preview to fill your whole monitor/s.  Save and desire to exit the dialog.  Then  right click  / custom camera / [the name you gave] to use the view.  To edit the view: Views / Edit Custom Cameras / [your view name]

 

For me it is important to have the just enough panel visible for a full scan, even if it's small.  If I can read that first then I go for the external view.  After you load the view, you'll see black on the sides.  I've placed radios and switches in there for now (even though the final file will span the fill view). 

 

There are some issues when you change from view to view.  You have to reload the view (right click / custom camera / [view]) again to bring it back.  And I haven't figured out how to save the shader view from session to session.  I believe this will be part of an aircraft.cfg, which is probably a good idea because every panel and cockpit size is different, needing different dimensions (my tests are with the Bonanzas).

 

For reference, open the Prepar3d Learning Center, and follow the Contents Tree to Prepar3D Product/Getting Started/ View System/ Custom Camera UI and Post Process right below that.

 

Grey Skies

 

[gboz 157]

I am working on a post-process to fix the fisheye/stretching distortion that plague 3 monitor views.

 

 

Great work Denali.

 

I just tried it out.

 

The first two pictures show the cream building at centre staying the same size.

I then moved the cream building to the centre of my right hand monitor.

There is slight vertical stretching in the custom view but a lot less horizontal.

There is obvious reduction in the HUD distortion too.

 

If only you could get the black areas to fill.

 

gb.

[denali 74]

Thanks.  The vertical stretching is more pronounced because all I've done so far is compress the sides in a little logarithmically (sp?)  The edges will need to be compressed in a tapered manner.  

I fell back a few steps because I deleted a bunch of pseudocode I'd thought through and then saved the file.  My backup wasn't as complete; the new system I built doesn't have that windows previous version system enabled.  oops.

 

It's difficult to work it through because you can't do a buffer for this; as we know all memory used is very expensive.  The operations need to be pixel by pixel, original position to new.  When you consider that it needs to be somewhat three steps, expansion in the middle, compression on the sides, and clipping off the edges to clean them up after the distortion, doing it without a buffer makes it daunting.  The process needs to be thought of in reverse, first considering clipping, then expand/compress.  And there's some nasty math in there, trig or something like that.  (there, that's basically what my pseudocode said).  

 

I've gathered some bookmarks regarding the math.  This one gives a clear idea that it can be done:  http://www.altesc.net/2013/05/06/eyefinity-guide-an-in-depth-how-to/4/  And we need it more than the racing sims.

 

First I need to understand the distortion that is programmed into these single monitor views since the days of Doom.  I need to undo that, and then redo into what is needed.  At least that's how I think of it now (that's probably wrong)  I'm finding that the best thing is to dive into DirectX programming head on and see if I can come up with an elegant solution.  This is a great resource for that:   http://www.rastertek.com/tutdx11.html    (I don't have my IDE set up on this thing either, and I replaced the drive in my laptop with an SSD, so I'm kinda lame right now)

 

If someone out there has a life preserver (a torus if that's what I need to call it for you math types), now would be a good time.

[DarkstarF16 72]

Will take a look at this - interesting

[denali 74]

Other things to work on along with the view, from one of the links I posted earlier: bezel correction and monitor angle compensation.  Those will actually be easy after the stretch problem is fixed.  There will need to be a configuration in a few variables for different monitors and angles, obviously.  Maybe putting in even a FOV calculator, auto-adjusting for measurements, if it doesn't ream framerates.  

Unfortunately I might not be able to do much on this until Saturday night Tejas time.

[styckx 1,869]

Why not reach out to the widescreenfixer or flawless widescreen devs and throw some sweet language talk at them and find out what hooks they are using to correct the problem on other games.. 

[denali 74]

Haven't you noticed, I'm not that good at sweet talk?  

 

I'm considering that.  But being in the field I am very familiar with the responsibility to do my best first, or RTFM/GIYF as it is said.  That makes things much sweeter in my experience.

 

That, and I actually have my old calculus book from 25 years ago sitting on a shelf behind me staring down at me with a shaming look.

 

They use DirectX.  I think how it is done is, just like SweetFX, they interecept DirectX calls with their own d3d.dll file.  That is why I'm going through the tutorial.  Also, widescreenfixer does not fix the stretch/fisheye distortion, AFAIK.  (If you're sure it does please let me know)

 

The end might be a DirectX injection.  From the link about about Assetto Corsa it seems like it might need to be done with three coupled views.  If you create independent views now you will get an independent and disorienting horizon tilt in each view.   

Actually, that reminds me I think there was source code available for one of those said to be available, the dev gave up the project.  I didn't bother looking far for it, but it might come in handy now.

[styckx 1,869]

Haven't you noticed, I'm not that good at sweet talk?  

 

I'm considering that.  But being in the field I am very familiar with the responsibility to do my best first, or RTFM/GIYF as it is said.  That makes things much sweeter in my experience.

 

That, and I actually have my old calculus book from 25 years ago sitting on a shelf behind me staring down at me with a shaming look.

 

They use DirectX.  I think how it is done is, just like SweetFX, they interecept DirectX calls with their own d3d.dll file.  That is why I'm going through the tutorial.  Also, widescreenfixer does not fix the stretch/fisheye distortion, AFAIK.  (If you're sure it does please let me know)

 

The end might be a DirectX injection.  From the link about about Assetto Corsa it seems like it might need to be done with three coupled views.  If you create independent views now you will get an independent and disorienting horizon tilt in each view.   

Actually, that reminds me I think there was source code available for one of those said to be available, the dev gave up the project.  I didn't bother looking far for it, but it might come in handy now.

 

Some games they do eliminate the fish eye, some not, others they fix the menus and HUD (from stretching out across 3 screens)..  It's mostly FOV adjustments they're accomplishing with these as far as the main render goes.. The menu and HUD stuff they fix is the best.. Skyrim is nearly unplayable in surround without these apps because the menus and HUD are streched across 3 monitors instead of being constrained to the center monitor.. 

[gboz 157]

Now attach the shader. Under the camera effects / effects dropdown, look for AntiStretch155 (did you save that notepad file as "all"?). Leave the excludes out.

 

 

You can preview the view, and even grab the edges of the preview to fill your whole monitor/s. Save and desire to exit the dialog. Then right click / custom camera / [the name you gave] to use the view. To edit the view: Views / Edit Custom Cameras / [your view name]

 

 

Just like to point out to anyone trying this that you have to click the "Add" button after you have selected

AntiStretch155 from the drop down list. You will then see it listed in the effects box. Then you can click the "Save" button.

 

Interesting to note that if you have FXAA turned on in-game it will be listed in the effects box too.

 

gb.

[gboz 157]

First I need to understand the distortion that is programmed into these single monitor views since the days of Doom.

 

 

Wide-screen distortion has been asked about several times on the LM forum and I have yet to see a response from them.

 

The last poster I read there suggested that the game projection would have to be changed

from "onto a flat plane" to "onto a cylinder". Not being a programmer I would not have a clue if this is possible.

 

gb.

[denali 74]

LM has responded.  They opened access to post-process shaders.  It's not their fault; the convention to put a little fish-eye on first person views is so old and runs so deep that it probably is not easily retractable from the libraries and thinking around views.  

 

There are at least a handful of companies now that specialize in geometry correction for projection systems.  The user base for very large display views was up until lately was very high end and very small, so it's probably not in the core of their development path.  It's more likely that the answer will come from others, especially with the opened shaders. So I'm going to have a go at it.

 

The problem is with the field of view and the focal length of the camera taking the view.  The focal length is too short for the FOV.  I think that's how it works.  What I'm trying to do in this first iteration is basically make a post-process shader lens to fix the distortion.  It might work just fine, except there may be a little problem with resolution in some portions of the new image.  The other way would be to get farther into the camera if possible and adjust the focal length (right now I'm not even sure if I know what I'm talking about on that part).

[denali 74]

Just like to point out to anyone trying this that you have to click the "Add" button after you have selected

AntiStretch155 from the drop down list. You will then see it listed in the effects box. Then you can click the "Save" button.

 

Thanks for adding that.  

 

I just noticed that if you save a flight with a custom view configured it will be there under the custom camera menu the next time it's loaded.  It worked for the raptor anyways, haven't tried it with any other aircraft yet.  

 

I've loaded a bunch of photoreal blueskyscenery over California and Nevada, and the lack of most of the distortion makes the flight much more fun.

 

I also didn't realize how kick-butt the raptor is for getting around.  I skipped away from that to a "real" airplane (one I can actually fly someday) when I first loaded P3D2.

 

And then to think LM made the raptor, and now the sim I'm flying it in.

[DarkstarF16 72]

A really good beginning its one of those things that will eat into your soul (I'm hoping hehe) until you find the perfect solution. I foresee lots of sleepless nights, both from programming and for what's to come by the sound of it.

 

Have fun! and do share any progress.

 

Bri

[denali 74]

So there was a layer of dust up to a millimeter thick on my calculus book (Tejas is dusty, even though It's humid here in the jungle climate of the Gulf Coast)  But it still functions.  The Battery seems like it's still going, or however these things work, I don't remember clearly right now.  Anyways, the display still works.

 

 

 

its one of those things that will eat into your soul (I'm hoping hehe) until you find the perfect solution. I foresee lots of sleepless nights, both from programming and for what's to come by the sound of it.

 

I'm trying to get this all settled before, so I can just enjoy the view.  I can handle sleeplessness fairly well, mostly because I don't tend to get sleep much if I've got something eating my brain.

 

I'm starting to get an idea of what the equation is for this distortion.  I'm drawing it out as a graph of how much displacement is needed.  Then maybe I can find the equation; my friend google might even have an equation calculator out there.  I'm not sure this will work optimally as moving the pixels around may cause noticeable decrease in resolution in the areas towards the middle of the view that I enlarge.  But it's worth a try to see if I can get rid of the margins.

 

After this I'm going to dig into the depth variable that is exposed in the post-processes.  Right now I don't really have a clue what that is.  I'm hoping it's going to be relative to a lens effect.  What I'm doing now is just the color.

 

Actually, I'm going to take a look at the fish-eye example in the Learning Center.  There is expansion done in that image ... wondering if the decrease in effective resolution is noticeable.

[DarkstarF16 72]

Actually, I'm going to take a look at the fish-eye example in the Learning Center. There is expansion done in that image ... wondering if the decrease in effective resolution is noticeable.

I took a look at the fisheye file earlier this afternoon and I'm just taking a wild guess that its calls to directX11?