The workings of the terrain engine and how textures are loaded

[markturner 0]

Hi, I still struggling with understanding exactly how this works and how we are able to influence what it does and thus the appearance of the scenery in FSX. Recently i started this: http://forums1.avsim.net/index.php?showtopic=250294 thread in which I asked similare questions. I found out a lot of useful stuff, but fundamentally, was unable to improve the situation despite all the tips.I am posting here instead, because as scenery developers, i feel you guys may have more in depth knowledge of how the scenery works and we may be able to turn up some more answers. Essentially, I use photoscenery, no autogen. My "problem" is getting the hi res textures loaded. For example, using Gottfried Razek's bluesky scenery, I would find that within a few minutes, I was flying over an area of out of focus scenery, this was usually at a tile junction, so i could be straddling an area of perfect scenery and out of focus scenery as illustrated here: Luis pointed me to a thread in my first post where he explained how the LOD radius affects texture loading. 2 questions arise from this in the light of what you see above. As you near the edge of the area with high res textures loaded, why does FSX not start laoding the hi res textures quicker, so that as you near that spot, it looks in focus? And why, despite me using a LOD radius setting of 8.5, does it take only a short time to reach the boundary of loaded hi res textures? is this something to do with the way the scenery tiles are made? If i press the scenery refresh button as i near out of focus areas, it will all load again, usually takes about 1 minute and hey presto !!! all the out of focus areas in front of me are all loaded with hi res sharp textures. Why does the Sim engine not do this, surely it needs to keep the hi res textures displayed around where you are in the sim? I also think it must be something to do with the scenery tiles, for 2 reasons. Firstly, using my megascenery, with all settings the same, this problem is not so noticable ( in fact, it is so noticable in Gottfried's scenery, that the first time I used it, i was getting nearly all textures badly displayed and just though it was a bda product. Now i have found out that you can "force" the sharp textures by using the scenery refresh button). Secondly, if you look in the following screenshot, you can cleary see where the photoscenery tile I am flying over ends and the standard generic tiles start. however, the tile i have just flown over is displaying the blurred low res textures, while the tile in the distance, is crisp and clear! This should surely be the other way round, yes? :I have also experimented with changing the following config files entries: ( my current settings are in brackets after) lod detail radius ( 8.5)MipBias (6)TBM (90)Bufferpools ( 400000000 )None of these seem to have any dramatic effect on how quickly FSX loads the hi res textures nearby. Which is a bit wierd. When you make a photoscenery tile, is there anything in the process that determines how FSX uses that tile and its textures? What if you made it with ONLY hi res textures? What would that do? This is leading me to think that there must be some way that the game determines which textures to load. If you can force it by using the scenery refresh button, surely you adjust the way the game does it, to get more pleasing visuals? In my flight last night round Colorado using Gottfried's Blue sky scenery, I was having to stop every 2 or 3 minutes and press refresh scenery, which kind of spoiled it. When it loaded though, it looked great!! so, essentially, the holy grail question is: How can we force FSX to keep high RES textures loaded around the area where you are ? Hope you guys can give me some input,many thanks, Mark

[LuisFelizTirado 33]

Hello Mark,The graphics engine is just code, and in many ways there is nothing complex about it. It will load as many high resolution textures as are needed to the graphics card. To do this, it must continuously determine where you are and where you are headed, what textures are in the video memory and which must be unloaded and which must be loaded. The code is solely dependent on hardware limitations, that is, if the hardware can execute quickly, then you will not see the dread Blurries, but if for any reason the hardware cannot keep up, then the Blurries appear.So, the problem is definitely not due to the graphics engine at all. It is just code and will be executed well or badly, quickly or slowly according to hardware capacity.So, why do you see the Blurries in the instances you mentioned? For example, Gottfried does not use FS X techniques to make custom terrain (I think not anyway, but I could be wrong). You can easily check this by looking in the relevant scenery folder. If there are any bitmaps in the local texture folder (and there would probably be thousands of them!), then the ground textures were made using FS 2000-2002-2004 techniques. The new FS X method does not have any bitmaps in the texture folder and the image of the ground is all in one piece and integrated in a very large bgl file.This old method for making custom ground is particularly inefficient and will strain hardware resources to the limit, particularly as the computer struggles to load those thousands of textures in a timely fashion. When it has trouble doing so, the low resolution MIP maps that were in memory will not be unloaded in a timely manner and replaced by high resolution MIP maps, thus resulting in the Blurries.In the second instance that you mentioned, there are a few possibilities, but since I do not know anything about that scenery package at all, they are only guesses:1. the custom scenery has a lower maximum resolution than the default ground textures so it looks "blurrier" than the default ground, or;2. the custom ground textures do not fill an entire Cell. A Cell in FS terrain is a basic unit and when it is not complete, the graphics engine must take processor time to create transparencies for the empty areas (so as to display the default ground) instead of for loading and unloading the video memory. It is generally difficult to always ensure that an entire Cell will be full because a designer does not necessarily want to cover more areas than planned with his scenery, or;3. this scenery package also uses FS 2000-2002-2004 techniques that are very inefficient.But, as mentioned, since I know nothing of that package, it is impossible for me to affirm anything concerning it.In any case, when you refresh the scenery, you are taking the time to do what the graphics engine cannot do in flight, and that is to unload low resolution MIP maps and load all the required high resolution MIP maps. If your hardware were not struggling to do this in flight, then you would never get the Blurries.It is a difficult conclusion to accept, but for what you want to display, your hardware is not adequate, even though it may be quite up-to-date and extremely powerful. The custom ground requires more resources than your machine provides. You can, of course, try making trade-offs and you already do so by not displaying autogen. Perhaps you could try turning off all shadows (very demanding on hardware), lens flare, bloom, water effects, and even all traffic, vehicle and aircraft. Then attempt your flights again and see if the display improves.Best regards.Luis

[markturner 0]

Hi Louis, thanks for your quick and imformative reply. While you were posting, I was trying out my freshly installed MegasceneryEarth washington scenery. The difference between the way the 2 sceneries perform is very noticable. I have virtually none of the problems I described with the bluesky scenery, , just small areas here and there and they usually pop into focus within 5 10 seconds, in fact it is not really very noticable at all.But with Gottfried's scenery, its very noticable. His scenery seems to be a mix of old and new types, some areas come with a texture file containing bitmaps, some with just BGL files. i guess this is what the problem must be. I also noticed, that using scenery refresh in the washington area takes just seconds, whereas it takes at least a minute using Gottfried's Scenery. here is a screeny from the washington area:here you can see clearly how well focussed and crisp everything is.Thx, Mark

[markturner 0]

I just wonder having read your reply again and also your tutorial on the Blurries, wether there is room for improvement here. Gottfried has emailed me and stated the .bmp files are just water.The thing that bugs me is the difference in performance between the 2 sceneries. I know my PC can perform, it runs washington and california very well, as I said, with virtually no problems with out of focus areas. So I can discount the performance of my PC as the problem here. It would be interesting to compare the files for the megascenery and gottfried's blue sky stuff, to see the difference ( maybe file size, area covered, number of detail levels etc etc) and perhaps this would shed important light on how and why FSX loads and uses the textures and what other factors affect this. The problem is I do not know how to even begin doing this and what to look for. perhaps the .bgl files could be better matched to the settings in FSX to achieve maximum performance. If we knew this, we could make adjustments accordingly to achieve better results. I know it is possible, as it it took me several weeks of playing around with settings to get my Megascenery looking as good as it does.I also know that I did not experiment a lot with settings such as the TBM, and that perhaps there is even better performance to be achieved.I do know that I am definitely not having a problem with "Blurries" as defined by Luiz in his article. I am seeing something different that I think is related directly to the way the scenery is made and utilised by FSX.I have just got my new graphics card GTX 285 OC, so I will do some more detailed messing with TBM etc and see if I can improve anything.cheers, Markcheers, Mark

[scott967 62]

I don't think the answer to your question is known (at least publicly), but we know that in FSX, resample takes the source image and creates image data that is stored in the bgl with various compression techniques. Looking at MegaSceneryX Hawaii using the tmfviewer tool, I see that the way the photos are organized, is that the day photos are gridded (I assume to reduce file size) and contain the photo resampled at LOD 4-15. Then the night photos cover the entire island but at only a single LOD for each bgl, except the LOD7 bgl covers LOOD4-7.Internally I don't know how the data are organized, but I assume that each LOD is stored as a separate entry, rather than as a mipmap.As Luis indicated, you also have the issue of transparency in the photo causing FSX to also load the landclass terrain tiles (including the cvx vectors) and blending them in.scott s..

[markturner 0]

So theoretically, you could make the scenery with only the high lod tiles, then there would be no low res ones to display? Is there any way to force the sim to only display the higher res tiles? That way, as long as your PC was capable of running it, you could tailor the scenery to your own machines capabilities?rgds, Mark

[LuisFelizTirado 33]

Sorry, Mark, but you will probably find that if the bgl only has one LOD level (for example, the highest resolution), then that custom ground will only display when the user is in range for that LOD. The rest of the time only the default ground will display.Perhaps you should try making some custom ground - this will help you to understand the process. The scenery developer only provides one source with the highest resolution desired. It is the Microsoft tool, BGLComp, that creates the different LOD levels as indicated by the developer, who can specify that the scenery contain only one or more LOD levels as desired. As soon as the Avsim library comes back up, you could also try the following document:Make photo-real ground textures in Flight Simulator Xhttp://library.avsim.net/esearch.php?CatID...amp;DLID=112595It will give you a good idea of the methods and constraints.The principal reason why the new method of making custom ground is more efficient than the old is probably because there is only one (very large) texture to load. Loading thousands of textures is very demanding on any system, even yours as your experience proves. (By the way, you have probably made things even worse by increasing your Level of Detail Radius by such a large amount, as you are trying to force your hardware to load exponentially more high resolution textures than the default setting.) For the last few years, it has become urgent among developers to reduce the number of draw calls precisely because of the large cost to performance. There might be some benefit to be gained from the new DirectX 10 and Shader Model 4.0 (texture arrays, occlusion culling, instancing) which would improve performance drastically, but unfortunately FS X only uses Shader Model 2.0 (for maximum backwards compatibility). And anyway SM4 was not available during development. In any case, by the time FS 11 is ready for release, it will once again strain hardware resources to the maximum as all previous versions have done.So, one could say that your best solution is just not to use that old type of scenery or to turn the LOD Radius down to levels that your hardware supports. If there is no equivalent scenery made with FS X methods, you can always make it yourself; it is not particularly difficult.Best regards.Luis

[markturner 0]

Hi Luiz and thx again for the interesting reply. Now I have my system rebuilt, I will indeed start to investigate the making of the scenery. I was going to start with FSEarth tiles. I can see a big learning curve however!!Regarding the LOD radius, my reason for increasing this is to make the distance textures clearer. Obviously, I do not mind the far away stuff being out of focus, as you could not see detail on it in real life. However, I feel that certainly stuff within 2 -3 miles should not be an indistinct blob, which it can be if you set the lod radius too low. Do you know exactly how far out the different settings work too Luiz? Eg, base setting is 4.5, at what distance do high res textures stop being loaded at this setting? If I am understanding your explanation in the wiki correctly, this setting also loads an area of mesh and textures almost the size of europe. Is this so? If yes, then what size is the area on higher settings? Quite obviously we do not want an area any bigger than this loaded initially, as it would take forever to fly to the edge, but we do want high res textures loaded further out. You also refer in the wiki, to setting the lod radius to small, medium and large and again, if understand correctly, a setting of 2.5 will load 1.2 metre textures ( or the highest res) out to approx a mile from the centre. How far out does it load them on 4.5 setting and say 7.5 setting? Would that be 4.5 cells and 8.5 cells respectively? If so, then that is not really an huge area to display the high res textures in - i estimate from the pictures you embedded that one cell equates to approx .5km. So using 8.5 would be pretty realistic really, it is a shame that at this etting the game also loads such a correspondingly huge area of mesh and texture data as well. I wonder if the 2 could be separated? Incidentally Luiz, my machine does not really struggle using 8.5, it loads quick and the high res textures snap in pretty quickly. Since running up my new system, I have noticed that its not quite so crisp, even using the same settings, but everything is new including my graphics card. I did notice that the high res textures snap in to focus much quicker. Using a graphics card with 1GB of onboard memory and 4GB of system memory, what setting of TBM and lod radius to think will be most efficient? Which of these 2 variables is likely to have the most effect in this case? I think there is probably a bit more testing and learning to be done with these settings, I just am not sure where to start, then there is the bufferpools twaek, which complicates things further. is there any easy way to benchmark these settings in game, or is it just a question of repeatedly flying over the same terrain and judging the effect by eye? Cheers, mark

[markturner 0]

I have been doing some experimenting and I think I have found the solution to my problem. It flies in the face of some recommended advice, but what I did was:As there were at least 5 or 6 possible variables to work with, i decided to try one at a time, starting with texture bandwidth multiplier, as it sounded like it was the one that would most directly affect what I was experiencing. I reduced it to 30 , then basically went upwards in increments of 30. Essentiallly the lower the number the worse the problem. I have it now at 300 and all the high res textures are loaded nearly instantly as you near them. I have no stutters and frame rates of around 50 FPS. I tried it out on Gottfried's colorado scenery and its like a different product, all looks crisp and clear, fantastic.How much of this is down to my new hard rive set up, I dont know, but for my video card and system specs, this setting has eliminated the blurry textures and slow loading of the high res textures. The sim looks and flies fantastic.

[LuisFelizTirado 33]

It is perhaps not so surprising, Mark. With your multiple cores handling the terrain texture loading threads and your very fast graphics card with such a large amount of memory, you are able to do what was impossible when FS X was released. Good for you.Best regards.Luis