How Much is One Petabyte of Storage?

[TechguyMaxC 516]

6 hours ago, nickhod said:

In one of the recent YouTube videos, one of the Asobo devs states that on each of the press demo PCs there's a 60GB cache of the Renton / (South?) Seattle area and the ground was cached at ZL19.

It's a lot of data for a small-ish area, but that is at very high quality.

The "petabtye" thing is largely irrelevant, as you say. No one is going to try to cache the entire earth at ZL19.

 

I'm a sys admin with access to a fair amount of retired server hardware.  My home media server is already 120TB using off-the-shelf consumer hardware, 1PB isn't out of the realm of possibility since a Petabyte is only 1000 Terabytes.  In fact, I just picked up a 24-bay 3U server chassis and 6-port SAS RAID card for use in a potential storage repository home server build yesterday.  Granted, given modern HDD sizes one would need either a much larger chassis or multiple chassis to reach a total size of 1PB of storage, but this could be done.

Were I to undertake such a task I don't think I would actually attempt to cache scenery for the whole earth since I don't fly outside of the U.S. often.  If it takes a couple hundred TB of storage capacity to cache the U.S. I would be up for that.  

[nickhod 425]

9 minutes ago, TechguyMaxC said:

Were I to undertake such a task I don't think I would actually attempt to cache scenery for the whole earth since I don't fly outside of the U.S. often.  If it takes a couple hundred TB of storage capacity to cache the U.S. I would be up for that.  

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

If your home broadband is fast, I can't see the point. Just stream it as MS will be constantly improving satellite imagery and autogen.

[TechguyMaxC 516]

1 minute ago, nickhod said:

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

If your home broadband is fast, I can't see the point. Just stream it as MS will be constantly improving satellite imagery and autogen.

I have gigabit symmetrical fiber at home, with no data caps.  I would still cache though.  Same reason I have a media server - you can't guarantee that a 3rd party service will:

1) always be accessible

2) continue to operate indefinitely, at a price that is justifiable

3) offer the content you desire

In other words, if MSFS' satellite scenery goes offline, for an hour, a day, or forever, I'll still have the scenery.

[snglecoil 1,828]

(Scans Newegg for petabyte SSDs)

Nope, I got nothing...😉

Edited October 4, 2019 by snglecoil

[Dominique_K 10,138]

Just now, nickhod said:

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

 

Petabytes, schmetabytes, if the home broadband is slow how do I download 100 Gb per flight into the cache ?

Actually MS should sell the sim on 10 Tb SSD 😊. 

[zalox 54]

A tech question.

What about TBW (degradation by bytes written) on a SSD drive?

SSD have limited writes. My 512GB Samsung Evo850 can "only" write 75TB before reach his limit.

If the proces is "download escenery">"erase escenery">"download another escenery"..lots of GB erased over a year...a lot of degradation.

[TechguyMaxC 516]

Just now, snglecoil said:

(Scans Newegg) for petabyte SSDs...

Nope, I got nothing...😉

Now that's a whole different ballgame.  Technically SSDs of greater capacity than even the largest HDDs exist, but they cost tens or even hundreds of thousands of dollars.  

SSDs are only semi-effective at reducing load times for scenery.  The initial simulator startup doesn't load any scenery, so it definitely makes sense to install the sim itself on an SSD as the size (and subsequent cost) of an SSD sufficient for this purpose are not prohibitive.  However, outside of the initial scenario loading, scenery will be loaded in a streaming fashion, i.e. as needed to match the speed of the aircraft in the sim.  The initial scenario load will no doubt take longer if scenery data is stored on an HDD rather than an SSD, but the required disk read speed in flight with even the highest quality LZ19 satellite scenery is unlikely to exceed a few, perhaps tens of MB/s, a rate traditional spinning disks long ago exceeded.  Modern HDDs can read, especially continuously, well in excess of 100MB/s.  

[ca_metal 2,015]

7 hours ago, domkle said:

The recent Orbx debate ZL16 vs 17 was interesting and a majority of customers was, with good reasons, againts the ZL16 pushed by John Venema. On another trhead, a fellow simmer gives a link to Simhaven which carries a comparison between ZL16, 17,18 and 19. Very telling.

My question is really about the cache mode. At ZL19 what kind of SSD storage would be needed for Paris/Geneva or Miami/Orlando. Also if I have to download first 300 Gb, I'd better plan my flights a week in advance. 

I mean, is the hybrid mode the panacea for bandwidth challenged people ?

 

The cache concept is different from the storage concept. Caching the scenery isn't supposed to be something the sim will do in a unlimited way. The way I see it, if you fly every day around cote d'azur region, it will be cached on your PC, this way you won't to download it every new flight you load on that said area.

Now, let's say you are flying a long haul, everytime the cache limits are reached, the sim will delete the old scenery (the area you left behind) and will download and load the new scenery (the area you are going to. They don't want you to store every area you flew. If you fly a different area everyday, your scenery cache is supposed to delete areas you are not using anymore and replacing them for the areas you are currently flying. 

As far as I know, caches have a limit of size (they are not supposed to use all your storage) and they are meant to save only things you use often or is currently using at the time. I have no idea what size the scenery cache will have by default, but you will probably be able to set it larger or smaller.

 

[TechguyMaxC 516]

2 minutes ago, ca_metal said:

The cache concept is different from the storage concept. Caching the scenery isn't supposed to be something the sim will do in a unlimited way. The way I see it, if you fly every day around cote d'azur region, it will be cached on your PC, this way you won't to download it every new flight you load on that said area.

Now, let's say you are flying a long haul, everytime the cache limits are reached, the sim will delete the old scenery (the area you left behind) and will download and load the new scenery (the area you are going to. They don't want you to store every area you flew. If you fly a different area everyday, your scenery cache is supposed to delete areas you are not using anymore and replacing them for the areas you are currently flying. 

As far as I know, caches have a limit of size (they are not supposed to use all your storage) and they are meant to save only things you use often or is currently using at the time. I have no idea what size the scenery cache will have by default, but you will probably be able to set it larger or smaller.

 

MS/Asobo stated that users have the ability to choose which regions to cache.  The limit is your storage capacity, which itself is limited by your budget.

[ca_metal 2,015]

21 minutes ago, TechguyMaxC said:

MS/Asobo stated that users have the ability to choose which regions to cache.  The limit is your storage capacity, which itself is limited by your budget.

Sure, but I think the auto-cache system will have some sort of a default limit and when reached the sim will delete old areas to download new areas, avoiding people to unadvertently filling up with SSDs/HDDs with sceneries they are not using anymore. Let's say I will do a tour around the world, never flying the same airport twice, I think at some point the sim will start to delete the sceneries I already flew to give some space to the sceneries I will need ahead in my flight.

I'm just talking about the auto-cache the sim might do when you start a flight. Not the pre-cache.

Edited October 4, 2019 by ca_metal

[TechguyMaxC 516]

5 minutes ago, ca_metal said:

Sure, but I think the auto-cache system will have some sort of a default limit and when reached the sim will delete old areas to download new areas, avoiding people to unadvertently filling up with SSDs/HDDs with sceneries they are not using anymore. Let's say I will do a tour around the world, never flying the same airport twice, I think at some point the sim will start to delete the sceneries I already flew to give some space to the sceneries I will need ahead in my flight.

I'm just talking about the auto-cache the sim might do when you start a flight. Not the pre-cache.

I'm not sure we have that level of detail on the scenery caching system yet.  I imagine it will have a default behavior, with default settings, in line with what you describe.  I have heard the behavior will be configurable by the user, though, and the only limit of the caching system's ability to cache scenery "tiles" will be the capacity of one's storage.  

[ca_metal 2,015]

1 minute ago, TechguyMaxC said:

I'm not sure we have that level of detail on the scenery caching system yet.  I imagine it will have a default behavior, with default settings, in line with what you describe.  I have heard the behavior will be configurable by the user, though, and the only limit of the caching system's ability to cache scenery "tiles" will be the capacity of one's storage.  

That's what I think also. I was just trying to explain, by the concept of caches, in a long haul flight the system, by default, should delete old areas and download new areas as you fly. 
That's my point of view, of course. Like you said, we don't know the details of the system yet.

[TechguyMaxC 516]

1 minute ago, ca_metal said:

That's what I think also. I was just trying to explain, by the concept of caches, in a long haul flight the system, by default, should delete old areas and download new areas as you fly. 
That's my point of view, of course. Like you said, we don't know the details of the system yet.

I see what you're saying now, thanks for expounding.

[Kilo60 660]

1 hour ago, zalox said:

A tech question.

What about TBW (degradation by bytes written) on a SSD drive?

SSD have limited writes. My 512GB Samsung Evo850 can "only" write 75TB before reach his limit.

If the proces is "download escenery">"erase escenery">"download another escenery"..lots of GB erased over a year...a lot of degradation.

Ouch...!!!!

I plan on flying all over the world upon release!

 

I better start stocking up on throw away SSD’s!

[goates 576]

Might have to look into getting one of these to cache things:

https://www.backblaze.com/blog/open-source-data-storage-server/

They sometimes put old ones up for sale, if you happen to live nearby.