Ortho4XP External Raid5 solutions that work well? Port multipliers, etc?

[theskyisthelimit 48]

I'm having a hard time tracking down a solution for a 4 drive hardware raid solution for my setup. I want something that will at least give me the throughput of a single 7200 rpm sata III drive, but hopefully with added speed and fault tolerance of raid5, granted hopefully with raid5 performance increases, but i think this may depend on the interface speed.   This ultimately will be to host about 12TB of storage for Ortho4xp for XP11.. currently just on a single 8TB drive that is 7200.

I have a hero maximus X motherboard and already tight on space in my main case.
I started looking at some standalone (plugs in via esata 6 Gbps max) solutions, but there arent any or many that are raid 5 capable and at least 6 Gbps throughput.. there is one or two that do usb 3.1 at 10 gbps but dont have good reviews at all.

So i figure maybe a $100 box plus esata card that does multiplying and raid and likely a pciex4 card, to go in the 3rd pciex16 slot on this board (i assume without slowing down the video).

Originally i was budgeting around $200 for all of this (not counting drives).

As far as non raid external bay solutions i have found this one for around $99
There are other models which supports usb 3.1 10gbps for a little more, however, i assume these that dont mention raid are jbod, no raid level support especially since usb.

There is this mobius which does raid5, but its not clear what interface speed the esata is, i assume maybe as low as 3, which is too low.  Edit: this one is definitely 3 Gbps on the esata.

So far its been slim pickins (especially on a raid card multiplier at a reasonable price and good reviews).. can anyone recommend a solution or two that works and performs well, especially at raid 5?  I'm pretty sure i've seen others who mentioned using similar setups for ortho, I just cant pinpoint the right combo of hardware, yet.

Edit.. I guess one other option (but with mixed reviews) might be a drobo 5c unit, though a little more costly and proprietary raid. ($364ish)

And another, maybe the winner here, this TerraMaster D5-300 For around $250 , downside it may be 5 Gbps, lacking usb-c

[gpb500 10]

Something to consider regarding performance.  The files being read from the drive are rather small, so thinking in terms of what large-file sequential-read performance is for a given drive (some approaching 200MB/s), well, you probably won't attain anything close to that for this application.  You might try to establish a baseline, when launching XP, look at what throughput you are currently getting from the drive itself (use a free program like HwInfo64 which captures the min/max/avg).  For my system, ortho reading peaks at 30MB/s.  So at 1Gb/s network speed (of which you'd expect between 110 and 120 MB/s), you'll still have plenty of bandwidth available.  My tests are done on a WD Red 5TB 5400 drive.  

Cheers.

-- Greg

[theskyisthelimit 48]

17 hours ago, gpb500 said:

Something to consider regarding performance.  The files being read from the drive are rather small, so thinking in terms of what large-file sequential-read performance is for a given drive (some approaching 200MB/s), well, you probably won't attain anything close to that for this application.  You might try to establish a baseline, when launching XP, look at what throughput you are currently getting from the drive itself (use a free program like HwInfo64 which captures the min/max/avg).  For my system, ortho reading peaks at 30MB/s.  So at 1Gb/s network speed (of which you'd expect between 110 and 120 MB/s), you'll still have plenty of bandwidth available.  My tests are done on a WD Red 5TB 5400 drive.  

Cheers.

-- Greg

Thats a great point, thanks for reminding me of this.. bottleneck being the size of the files and or XP11 coding on loading itself.. i appreciate that stat.. ill test with HWinfo64 (i have that one) and see, i'm sure ill see similar numbers, none of which should saturate even the raid units that offer 3Gbps ability.. i guess it may just boil down to which setup (either onboard addon card or external raid enclosure with its own raid) gives me the best hardware raid not so much the connection speed.

[Humpty 751]

RAID 5 will get you 5 TB of the 8tb but you will need minimum of 3 disks  , one is for fault tolerance and RAID 5 will be slow , why not look at RAID 10 ? you just need mirroring  , RAID 2 can be used too with minimum of two disks. I think RAID 2 let's say 2 x 4TB should get 3 TB iirc

Did read somewhere that Samsung is launching a 30TB HDD

 

[theskyisthelimit 48]

2 minutes ago, HumptyDumpty said:

RAID 5 will get you 5 TB of the 8tb , one is for fault tolerance , why not look at RAID 10 ? you just need mirroring 

 

Not sure where those numbers came from.. but raid 5, with single drive parity.. 4, 4TB drives, is 16TB total.. take away 1 for parity leaves 12TB usable space..

Raid10 is striping with mirroring.. you need twice as many drives to achieve the same thing, granted faster.  I use them on our business servers configured in raid 10.. such as 8 or 12 drive raid 10 arrays that are lightning fast, but only single drive fail protected in a span.

[Humpty 751]

1 hour ago, theskyisthelimit said:

Not sure where those numbers came from.. but raid 5, with single drive parity.. 4, 4TB drives, is 16TB total.. take away 1 for parity leaves 12TB usable space..

Raid10 is striping with mirroring.. you need twice as many drives to achieve the same thing, granted faster.  I use them on our business servers configured in raid 10.. such as 8 or 12 drive raid 10 arrays that are lightning fast, but only single drive fail protected in a span.

Hi , I have installed a RAID 5 using 4 x 2TB ENterprise drives for a client  a couple of years back for Windows 2012 AD and that gave sorry around 5 TB is what was achieved on it of the 8TB storage . with parity iirc sorry it was long back and had left IT during that period but was just helping them out. The RAID card was the 3ware 4 port

That same client is now using RAID 2 with 2x4TB drives and this is giving around 3TB usage but i can check it tomorrow and let you know how much  space is RAID 2 giving . This one is using a LSI with 8 ports i  think. The RAID 5 had a failed 0 UNIT and was mission critical to get the server up. They were planning on connecting SAS drives in RAID.

Yep RAID 10 is faster. I have never used RAID 10 as yet. 

[theskyisthelimit 48]

Update here.. i decided to go with the Terramaster D5-300 unit.. built in Raid.. so far i think its mostly a win.  Again my setup is OS on SSD, my Xplane main directory / structure on SSD.. and the Ortho was on 8TB 7200.. which had very good read and writes to begin with (i think my basic windows large file tests were in the 200+ MB/sec range).. despite hwinfo showing Xplane loading the ortho at a max of 30 Mb/sec, i knew what my basic target should be.. actually, i redid some ortho load tests on the old drive using hwinfo and suddenly i did see 200 MB/sec, so maybe not entirely an XP bottleneck on loading.

At first i had some troubles with the Terramaster.. i used their USB-C to usb 3.0 cable.. tried various ports on the computer.. CrystalDisk Benchmark wasnt showing great numbers.. i at first did a single drive in their unit to get a baseline read/write.. i was getting maybe 150 MB/sec for reads and writes only 34 MB/sec (the drive when tested on sata that is now in the unit was getting more like 217.6 MB/sec read and 214.9 MB/sec.. strangely a few of my 4TB drives were showing DMA errors (check cable etc) via the terramaster.. their reads were more like 170 and 148.. while the other 4TB units in the terramaster were showing lower numbers like 105 and 43 etc.. I thought maybe this was just a crystal benchmark issue.. so i tried some file transfers.. i was getting a max of 40 MB/sec when copying to the terramaster drive or raid5 (5 drives) as well.. in either case.

I then decided to put the cable into a usb 3.0 to usb-c adapter i have going into my motherboards usb-c connection.. I was using all 5 drives in raid 5 at this point.. tested the file copy process.. it was giving me up to 195 MB/sec.. and when taking the same file From the raid array (reads) it was around 220 MB/sec or better.   CrystalDisk Benchmark still shows low Reads (140) and super low writes (40) even via the usb-c, but i get those consistent windows file transfers

So at this point i've left it at this.. happy with the results.. though i may consider a dedicated usb 3.0/ usb-c pcie card.. either using this usbc adapter or maybe getting a usb c to usb c cable and trying that.. 

The reason i'm wondering if maybe i need a dedicated card, is i swear i may be getting some fps fluctuations.. starts off lower for some things then a few minutes in sitting at the same scene jumps up, plus some stuttering.. not sure that its really usb c related (terraraid related) or not.. just a consideration.

Overall happy so far.  In terms of load times in the sim, nothing has changed much.. about the same.. so not as good as having a true blue hardware raid that would really boost the loads but at least as good or slightly better than single disk.

I have two drives inside my tower that are 8TB.. i may do a storage spaces on them, combining into one big drive.. and use them as a backup for the ~15TB raid ortho data.

 

[theskyisthelimit 48]

Update:  so the issue with this terramaster i thought was just a glitch with the benchmark software isnt that.. at times it does drop to 20-40 MB/sec on the writes on the drive(s).. tried every usb port on the system, updated drivers you name it nothing.. what does fix it (maybe temporarily) is turning the unit off and back on, then its back to 200 MB/sec.. so there is a definite issue going on here.

[theskyisthelimit 48]

Well now the terramaster seems to drop the drive regularly.. so i'm moving on to test the Mobius 5 drive 5gbps usb 3.0 solution (also has 3gbs esata).. hoping it will fair better.

Though the more i look at this the more i think maybe i should go with a NAS solution.. get at least 10gbe ethernet on each end and on a switch.. though 3gbe would probably be enough to keep speeds at least single drive speed i think.. but my NAS media 2016 server is already physically out of space for new drives (using flexraid now).. i'd have to go to a larger case, get a switch that had spf x 4 for these faster gbe modules etc.. could be a tad costly in the short term but a better long term space fix.

 

[theskyisthelimit 48]

Update again.. so i went with the 5 drive mobius esata (6gbps) / firewire / usb 3.0 (5 gbps) solution found here

Very good results now a few days in.. getting consistent results with crystal benchmarks.. 242 MB/sec on reads and around 200 on writes (for a 2 run 1GB test).. On par or slightly better than the single drive performance test results.. no disconnections or lost drive issues.. i'm also using a startech esta 6 gbps 2 port card (pciex1) and its working great.

Short term i also need to find a way to combine my two 8tb existing internal drives (storage spaces maybe) into one 16GB unit so i can backup the content of the 16GB mobius array (ortho4xp photoscenery).

 

I think down the road ill probably get some sort of NAS solution and wire up fiber runs to a 20+ port gigabit switch which also has 10gbe ports (fiber).. i could either go dual ports on the server i already have in my basement (dual 10gbe) or maybe just 10gbe by itself there and up on my flight sim rig.. tied into the 10gbe capable switch.. this would give me the option of using this type of das solution on my existing media server.. or getting a bigger chassis + bigger raid card in the media server and going with more drives internally.. or just a nas external solution that can hit at least 10gbe.. longer term, but for now this workstation + das external 5 drive bay works..