Jump to content

BIGSKY

How Much is One Petabyte of Storage?

Recommended Posts

Posted (edited)

This Pb thing is overrated. Google Earth is about 20 Pb and we all run it on our machine without even thinking of it.

The real question is how large will be the chunk to be downloaded in hybrid (cache) mode for a flight like, for instance, Paris to Geneva or Miami to Orlando at ZL19.

Edited by domkle

Share this post


Link to post
Share on other sites
Posted (edited)

The whole flight we saw in the demo used about 60GB

Edited by Wobbie

Share this post


Link to post
Share on other sites
2 hours ago, domkle said:

The real question is how large will be the chunk to be downloaded in hybrid (cache) mode for a flight like, for instance, Paris to Geneva or Miami to Orlando at ZL19.

In one of the recent YouTube videos, one of the Asobo devs states that on each of the press demo PCs there's a 60GB cache of the Renton / (South?) Seattle area and the ground was cached at ZL19.

It's a lot of data for a small-ish area, but that is at very high quality.

The "petabtye" thing is largely irrelevant, as you say. No one is going to try to cache the entire earth at ZL19.

 

  • Upvote 1

Share this post


Link to post
Share on other sites
Posted (edited)

Thanks for the info, Nick. Don't you think that would bring a medium haul flight at the Tb level ("cache") ?

Edited by domkle

Share this post


Link to post
Share on other sites

As long as many consecutive 16+ hour flights are possible in very complex airliners (B787/777/A340/350) without CTD or performance loss, I'm good.👨‍✈️😁

Share this post


Link to post
Share on other sites
29 minutes ago, domkle said:

Thanks for the info, Nick. Don't you think that would bring a medium haul flight at the Tb level ("cache") ?

I think it'll depend on the satellite imagery quality you want. Drop down to ZL16 - 17 and you can cover the whole of Britain with tens of GB.

Remember each zoom level is four times more detail than the previous.

The OrbX TrueEarth GB packs are around 25GB compressed. There's 3 of them, and I think that's ZL16 plus all the millions of houses and trees.

  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Also I should add, my understanding of the modes so far (with a bit of guess work based on recent videos) are:

Full online Mode: Streams all data, maybe does some pre-caching in your direction of travel

Cached Mode: Uses pre-downloaded cache first, then streams non-cached areas relative to available bandwidth. (Good for slow connections)

Offline Mode: Only uses pre-downloaded cache and does not attempt to stream. I'm guessing non-cached areas are shown as very low resolution satellite imagery, or maybe land-class if it's still there as a fall back.

Edited by nickhod

Share this post


Link to post
Share on other sites
3 hours ago, domkle said:

This Pb thing is overrated. Google Earth is about 20 Pb and we all run it on our machine without even thinking of it.

The real question is how large will be the chunk to be downloaded in hybrid (cache) mode for a flight like, for instance, Paris to Geneva or Miami to Orlando at ZL19.

Indeed cache and internet connection will be the key for high detail and of course the local hardware stuff  🙂

Share this post


Link to post
Share on other sites

The recent Orbx debate ZL16 vs 17 was interesting and a majority of customers was, with good reasons, againts the ZL16 pushed by John Venema. On another trhead, a fellow simmer gives a link to Simhaven which carries a comparison between ZL16, 17,18 and 19. Very telling.

My question is really about the cache mode. At ZL19 what kind of SSD storage would be needed for Paris/Geneva or Miami/Orlando. Also if I have to download first 300 Gb, I'd better plan my flights a week in advance. 

I mean, is the hybrid mode the panacea for bandwidth challenged people ?

 

Share this post


Link to post
Share on other sites
Posted (edited)

Someone here should buy one or two of these petaracks and download the entire world when msfs is released. We need to have some backup inside the community in case MS shuts down the servers

Edited by cepact
  • Like 1

Share this post


Link to post
Share on other sites
14 minutes ago, domkle said:

The recent Orbx debate ZL16 vs 17 was interesting and a majority of customers was, with good reasons, againts the ZL16 pushed by John Venema. On another trhead, a fellow simmer gives a link to Simhaven which carries a comparison between ZL16, 17,18 and 19. Very telling.

My question is really about the cache mode. At ZL19 what kind of SSD storage would be needed for Paris/Geneva or Miami/Orlando. Also if I have to download first 300 Gb, I'd better plan my flights a week in advance. 

Yep, I was wrong; TrueEarth is ZL17 but it's compressed to within an inch of its life. I was fairly underwhelmed by the quality up close to be honest.

Paris to Geneva at ZL19 is going to be huge (TBs). I'd doubt you need it all at ZL19 though. You'd want the area around the Paris and Geneva airports at ZL19. The Paris and Geneva metro area at ZL 18, then the 'cruise' part of the flight at ZL17 or ZL16.

How much flexibility MSFS will give to achieve that is another matter though.

Share this post


Link to post
Share on other sites

Venema said this week that the bgl format allows much more compression than the XP dds format. How much I don't know.

 

Share this post


Link to post
Share on other sites
1 hour ago, cepact said:

Someone here should buy one or two of these petaracks and download the entire world when msfs is released. We need to have some backup inside the community in case MS shuts down the servers

MS will support the game for a MINIMUM of 10 years

Share this post


Link to post
Share on other sites
6 hours ago, nickhod said:

In one of the recent YouTube videos, one of the Asobo devs states that on each of the press demo PCs there's a 60GB cache of the Renton / (South?) Seattle area and the ground was cached at ZL19.

It's a lot of data for a small-ish area, but that is at very high quality.

The "petabtye" thing is largely irrelevant, as you say. No one is going to try to cache the entire earth at ZL19.

 

S4ccJgX.jpg

I'm a sys admin with access to a fair amount of retired server hardware.  My home media server is already 120TB using off-the-shelf consumer hardware, 1PB isn't out of the realm of possibility since a Petabyte is only 1000 Terabytes.  In fact, I just picked up a 24-bay 3U server chassis and 6-port SAS RAID card for use in a potential storage repository home server build yesterday.  Granted, given modern HDD sizes one would need either a much larger chassis or multiple chassis to reach a total size of 1PB of storage, but this could be done.

Were I to undertake such a task I don't think I would actually attempt to cache scenery for the whole earth since I don't fly outside of the U.S. often.  If it takes a couple hundred TB of storage capacity to cache the U.S. I would be up for that.  

Share this post


Link to post
Share on other sites
9 minutes ago, TechguyMaxC said:

Were I to undertake such a task I don't think I would actually attempt to cache scenery for the whole earth since I don't fly outside of the U.S. often.  If it takes a couple hundred TB of storage capacity to cache the U.S. I would be up for that.  

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

If your home broadband is fast, I can't see the point. Just stream it as MS will be constantly improving satellite imagery and autogen.

Share this post


Link to post
Share on other sites
1 minute ago, nickhod said:

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

If your home broadband is fast, I can't see the point. Just stream it as MS will be constantly improving satellite imagery and autogen.

I have gigabit symmetrical fiber at home, with no data caps.  I would still cache though.  Same reason I have a media server - you can't guarantee that a 3rd party service will:

1) always be accessible

2) continue to operate indefinitely, at a price that is justifiable

3) offer the content you desire

In other words, if MSFS' satellite scenery goes offline, for an hour, a day, or forever, I'll still have the scenery.

Share this post


Link to post
Share on other sites
Posted (edited)

(Scans Newegg for petabyte SSDs)

Nope, I got nothing...😉

Edited by snglecoil

Share this post


Link to post
Share on other sites
Just now, nickhod said:

If your home broadband is slow, I can see why it would be useful, but you'll never download that much data.

 

Petabytes, schmetabytes, if the home broadband is slow how do I download 100 Gb per flight into the cache ?

Actually MS should sell the sim on 10 Tb SSD 😊

Share this post


Link to post
Share on other sites

A tech question.

What about TBW (degradation by bytes written) on a SSD drive?

SSD have limited writes. My 512GB Samsung Evo850 can "only" write 75TB before reach his limit.

If the proces is "download escenery">"erase escenery">"download another escenery"..lots of GB erased over a year...a lot of degradation.

Share this post


Link to post
Share on other sites
Just now, snglecoil said:

(Scans Newegg) for petabyte SSDs...

Nope, I got nothing...😉

Now that's a whole different ballgame.  Technically SSDs of greater capacity than even the largest HDDs exist, but they cost tens or even hundreds of thousands of dollars.  

SSDs are only semi-effective at reducing load times for scenery.  The initial simulator startup doesn't load any scenery, so it definitely makes sense to install the sim itself on an SSD as the size (and subsequent cost) of an SSD sufficient for this purpose are not prohibitive.  However, outside of the initial scenario loading, scenery will be loaded in a streaming fashion, i.e. as needed to match the speed of the aircraft in the sim.  The initial scenario load will no doubt take longer if scenery data is stored on an HDD rather than an SSD, but the required disk read speed in flight with even the highest quality LZ19 satellite scenery is unlikely to exceed a few, perhaps tens of MB/s, a rate traditional spinning disks long ago exceeded.  Modern HDDs can read, especially continuously, well in excess of 100MB/s.  

Share this post


Link to post
Share on other sites
7 hours ago, domkle said:

The recent Orbx debate ZL16 vs 17 was interesting and a majority of customers was, with good reasons, againts the ZL16 pushed by John Venema. On another trhead, a fellow simmer gives a link to Simhaven which carries a comparison between ZL16, 17,18 and 19. Very telling.

My question is really about the cache mode. At ZL19 what kind of SSD storage would be needed for Paris/Geneva or Miami/Orlando. Also if I have to download first 300 Gb, I'd better plan my flights a week in advance. 

I mean, is the hybrid mode the panacea for bandwidth challenged people ?

 

The cache concept is different from the storage concept. Caching the scenery isn't supposed to be something the sim will do in a unlimited way. The way I see it, if you fly every day around cote d'azur region, it will be cached on your PC, this way you won't to download it every new flight you load on that said area.

Now, let's say you are flying a long haul, everytime the cache limits are reached, the sim will delete the old scenery (the area you left behind) and will download and load the new scenery (the area you are going to. They don't want you to store every area you flew. If you fly a different area everyday, your scenery cache is supposed to delete areas you are not using anymore and replacing them for the areas you are currently flying. 

As far as I know, caches have a limit of size (they are not supposed to use all your storage) and they are meant to save only things you use often or is currently using at the time. I have no idea what size the scenery cache will have by default, but you will probably be able to set it larger or smaller.

 

  • Like 1

Share this post


Link to post
Share on other sites
2 minutes ago, ca_metal said:

The cache concept is different from the storage concept. Caching the scenery isn't supposed to be something the sim will do in a unlimited way. The way I see it, if you fly every day around cote d'azur region, it will be cached on your PC, this way you won't to download it every new flight you load on that said area.

Now, let's say you are flying a long haul, everytime the cache limits are reached, the sim will delete the old scenery (the area you left behind) and will download and load the new scenery (the area you are going to. They don't want you to store every area you flew. If you fly a different area everyday, your scenery cache is supposed to delete areas you are not using anymore and replacing them for the areas you are currently flying. 

As far as I know, caches have a limit of size (they are not supposed to use all your storage) and they are meant to save only things you use often or is currently using at the time. I have no idea what size the scenery cache will have by default, but you will probably be able to set it larger or smaller.

 

MS/Asobo stated that users have the ability to choose which regions to cache.  The limit is your storage capacity, which itself is limited by your budget.

  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)
21 minutes ago, TechguyMaxC said:

MS/Asobo stated that users have the ability to choose which regions to cache.  The limit is your storage capacity, which itself is limited by your budget.

Sure, but I think the auto-cache system will have some sort of a default limit and when reached the sim will delete old areas to download new areas, avoiding people to unadvertently filling up with SSDs/HDDs with sceneries they are not using anymore. Let's say I will do a tour around the world, never flying the same airport twice, I think at some point the sim will start to delete the sceneries I already flew to give some space to the sceneries I will need ahead in my flight.

I'm just talking about the auto-cache the sim might do when you start a flight. Not the pre-cache.

Edited by ca_metal

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...