INTRODUCTION

I've written this QBasic "hack" to convert Digital Elevation Model (DEM) files to Flight Sim Toolkit FTD files.  I was inspired by the information I found on the web by both Forest Pearson and Daniel Berlin metioning that it could be done (but with the use of some very expensive software).

The program I created is rather "vertical" in that it was created to meet a specific need.  I want to build in FST, not spend lots of time dressing up a program.  However, I have written a rather detailed description of the code, and anyone with a grasp of fundamental programming should be able to make modifications if necessary.  I suppose if I were motivated, I could make this program into something really cool.  The longer I worked on it, the greater the potential I could see for creating modeling software for this kind of thing.  A nice GUI with a command line interface would be great (also, using something like Borland Turbo Pascal or C++ would be better).

Anyway, this written explanation should help anyone who's interested in creating terrain data files for FST.  This write-up fully explains the problem I wanted to solve, the approach I took to the problem, assumptions, trade-offs, and the solution.

BACKGROUND: What's a DEM file?

A DEM file (in particular, a 1 degree DEM) is a file that represents a one degree square area of the earth.  The DEMs I used sample the area with 1200 south to north profiles, each marking the elevation at 1200 points.  For more information on DEM files, visit the USGS web site and then download their DEM Data Users Guide.

	http://edcwww.cr.usgs.gov/doc/edchome/ndcdb/ndcdb.html

If you just want to salivate at the possibilties for creating terrain for the contiguous 48 states of the USA, just go to this site:

      http://edcwww.cr.usgs.gov/glis/hyper/guide/1_dgr_demfig/index1m.html

I recommend you spend some time learning about DEMs before you do any conversion.  They are simple, but you should be able to graphically model them in your mind so you can develop some verification strategies along the way.  (Verification strategies would be built into a sophisticated program.)

BACKGROUND: The Problem

For my simulation, I wanted to create FTD files for a portion of the state of Colorado.  I wanted to include the Eastern plains and some of the Rocky Mountains around the city of Colorado Springs.  I selected the area between these four points on the map.

Northeast corner	40N/104W
Southeast corner	38N/104W
Southwest corner	38N/106W
Northwest corner	40N/106W

This area encompasses slightly more than 12 FTD squares of 250 kilometers.

BACKGROUND: The Approach

Looking at the area I chose, I marked off the 12 FTD squares I would create. The area is represented by 4 DEM files from the USGS repository:  The southwest portion by Pueblo West, southeast by Pueblo East, northwest by Denver West, northeast by Denver East.  I also marked off the areas covered by each of the DEMs.  

	NOTE:  I suggest to anyone doing this, that you mark off both your areas of DEM coverage and your FTD squares.  You will need to know this to set the program indices in a couple of places.

The program, then runs from the southwest, up to the north for each profile of both the lower (Pueblo) and upper (Denver) DEMs, processing profiles from west to east.  This is, of course, taking into account that the FTD file formats are 90 degrees orthogonal to DEMs.  FTDs run from left to right, from the bottom to the top.


IMPLEMENTATION: The Assumptions

I selected 1 degree DEMS over the other DEM formats.  The spacing was small enough to allow me to select elevations that were no more than about 10% of the distance between FTD points of 500 meters.  I also assumed that the linear (not angular) quality of the sampling would be easier to work with than the 7.5 minute DEMs even though the separation between points is closer in the 7.5 minute DEMs.

The DEM files I downloaded were taken as exact.  The first data record of the DEM contains information describing the accuracy of the elevation information (horizontal and vertical).  For my needs, I was not about to quibble over a couple of 10 meters or so.  I assumed the elevation information was good enough.  Because of that, I do no filtering for an occasional "bad" elevation value.  But, I've not found any in the terrain I've created for Colorado.

IMPLEMENTATION: The Trade-offs

I own a somewhat small (by today's standards) PC.  It's got 4MB RAM, but usually over 100MB free on the drive.  I wrote a special set of boot files so I could run a RAM drive to keep I/O to a minimum.  I just didn't want to beat up on my drive after the first successful runs to the lower left FTDs.
This is due to the orthogonal perspective between DEMs and FTDs.

Why Qbasic?  Why not?  It's already on my machine.  At work, I've got access to just about anything imaginable, but I opted for something I thought I could more readily share.  Qbasic isn't fast (I could really tell what a dog it is when I direct I/O to RAM instead of spinning media), but it's EVERYWHERE.  Another language would have better facilities for creating internal data structures for optimal processing.  Writing in Qbasic was fun, if not sometimes frustrating.

THE SOLUTION: Code Explanation

SUBROUTINE DECLARATIONS:

The top of the program code contains two subroutine declarations.  One for outputting to the FTD files.  The other building the variable bigstring$.  Each of these subroutines is described in more detail later.

SEPARATION CONSTANTS:

These constants are used to calculate the closest profile to a vertical row of FTD points and for each profile, the closest points to the horizontal rows of FTD points.

Horizontal Point Separation:  Horizontal separation at the northernmost edge of the area is not as great as it is at the southernmost edge.  That's because the separation of meridians (lines of longitude) grow as you approach the equator from the North Pole.  I acknowledged this, but didn't program for it.  The error isn't that great for my purposes.

I found the distance between the lines of longitude in the middle latitude of my area.  For this, I used the distance calculator at:

	http://www.indo.com/distance/

The distance is 87 kilometers.

That means the distance between profiles in the DEMS (on average between the two) is found by dividing 87,000 meters by 1200 samples.  Then, to account for how many samples would have to passed to meet the 500 meter distance of the FTD, divide 500 by the distance between samples.  This value is used to tell us which profile to process.

Vertical Point Separation:  Lines of latitude are separated by equal distances for our purposes.  Again, I got the distance from the indo.com web site and it came out to 111,000 kilometers.  The same calculation applies to vertical point separation, and it is used to tell us which points along the profile to process.

HARDCODE THE FTD FILES

Daniel Berlin's explanation of FTD files came in handy here, but I didn't trust myself to do this alone.  I created a 3x4 world and looked at the file naming FST used.  I didn't feel like writing an algorithm to convert the x,y coordinates of FTDs to the 0 and _ encoding of the FTDs.  That is left to you, dear reader, as an experiment for yourself.  I used good old brute force to do this.  I did use the array, however, because of its ease of use when it comes to indexing and calling the proper places for output.

BUILD THE FTD HEADER

Taken out of Daniel Berlin's explanation.  The FTD file header contains some ASCII characters describing some FTD file attributes and 90 null characters (probably for some future use).  I built strings, assigned them to string variables, then PUT them into the files.  Because the FTD file points are written at 200 byte offsets for the terrain data, I opted to open the files for BINARY operation and work to specific byte locations.  This is what makes this program so I/O bound.

I also opened the first four FTD files along the left side of the area and initialized two FTD file set tracking variables to "F" (for false).  The tracking variables are used to open and close the other two sets of FTD files later in the program execution.

PREPARE TO READ THE DEMs

The profilesample variable is dimensioned as shared because it is used by the buildbig procedure, and all DEM files are opened.  I read the first record and do nothing with it.  The DEM Data Users Guide explains the contents of the Type-A record, but this being a simple program, I'm not too concerned about its contents.  The real purpose for reading the Type-A record is to set the file pointers to the first profile record.  The DEM files are read sequentially, so keeping track of the file pointers is important.

I initialize the demfileset variable to 1.  This variable is used later in reading the files.

I dimension the variable bigstring$ as shared, because it, too is shared with the buildbig subroutine.

GETTING READY TO RUN THE MAIN PROCESS LOOP

There are three variables to initialize:  The profilebase variable which is used to calculate the profile to process.  The profiletoprocess variable which tells us which profile to process next. The number of profiles processed which controls the main loop.  Because my area is 3 FTD squares wide, each FTD square containing 100 points, My loop runs out at 300. 

If you wanted to process all the data, index the loop on all the end of file condition for the last file to read.  You might end up with some portion of an FTD containing real world data, and the rest ocean.  But, hey! You've gotten this far, and maybe one day you would like to do the next adjacent DEM.

'-------------------------------------------------------------------------
'Initialize the profile processing variables

LET profilebase = 1     'for calculating the profile to process
LET profiletoprocess = profilebase
LET profilesprocessed = 0

DO WHILE profilesprocessed < 300
   
   INPUT #demfileset, profilesample

   IF VAL(LTRIM$(MID$(profilesample, 4, 4))) > 1200 THEN
      CLOSE #1: CLOSE #2
      LET profilebase = 1
      LET profiletoprocess = profilebase
      LET demfileset = demfileset + 2
      INPUT #demfileset, profilesample
   END IF

   IF VAL(LTRIM$(MID$(profilesample, 4, 4))) <> profiletoprocess THEN
     
      INPUT #demfileset + 1, profilesample
      FOR lridx = 1 TO 7
      INPUT #demfileset, profilesample
      INPUT #demfileset + 1, profilesample
      NEXT lridx

   ELSE                               'process the points on a profile
      LET profilesprocessed = profilesprocessed + 1
      PRINT "Processing DEM Set ", demfileset, "Profile ", profilesprocessed
     
      CALL buildbig(demfileset)       'build bigstring$
     
      LET pointsprocessed = 0  'reset the number of points processed
      LET pointctr = 1  'reset the point counter
      LET pointbase = 1  'reset the point base
      LET pointtoprocess = pointbase   'start with point one.
      LET pointvalue$ = LEFT$(bigstring$, 6)
     
      DO WHILE LEN(MID$(bigstring$, (pointctr * 6))) > 6
         IF pointctr = pointtoprocess THEN
            LET pointsprocessed = pointsprocessed + 1
            CALL outproc(profilesprocessed, pointsprocessed, pointvalue$)
            LET pointbase = pointbase + vertpointsep 'These two lines
            LET pointtoprocess = INT(pointbase)  'caluclate the next point
         END IF
        
         LET pointctr = pointctr + 1 'increment point counter/pointer
         LET pointvalue$ = MID$(bigstring$, (pointctr * 6), 6) 'pick point
      LOOP
        
      INPUT #demfileset + 1, profilesample
      CALL buildbig(demfileset + 1)

      LET pointctr = 0
      LET pointbase = pointbase - 1200 'starting in a new DEM
      LET pointtoprocess = pointtoprocess - 1200

      DO 'until pointsprocessed = 400
         LET pointctr = pointctr + 1
         LET pointvalue$ = MID$(bigstring$, (pointctr * 6), 6)
         IF pointctr = pointtoprocess THEN
            LET pointsprocessed = pointsprocessed + 1
            CALL outproc(profilesprocessed, pointsprocessed, pointvalue$)
            LET pointbase = pointbase + vertpointsep
            LET pointtoprocess = INT(pointbase)
         END IF

      LOOP UNTIL pointsprocessed = 400
      LET profilebase = profilebase + horpointsep
      LET profiletoprocess = INT(profilebase)
   END IF
  
   IF ((profilesprocessed = 100) AND (middleftd$ = "F")) THEN
      'switch to middle FTDs
      PRINT "Middle FTDs"
      LET middleftd$ = ""
      FOR idx = 1 TO 4
      CLOSE #(50 + idx)
      NEXT idx
     
      FOR idx = 5 TO 8
      OPEN ftd$(idx) FOR BINARY AS (50 + idx)
      NEXT idx
      CLS
   END IF

   IF ((profilesprocessed = 200) AND (rightftd$ = "F")) THEN
      'switch to right FTDs
      PRINT "Right FTDs"
      LET rightftd$ = ""
      FOR idx = 5 TO 8
      CLOSE #(50 + idx)
      NEXT idx
     
      FOR idx = 9 TO 12
      OPEN ftd$(idx) FOR BINARY AS (50 + idx)
      NEXT idx
      CLS
   END IF
LOOP
CLOSE

SUB buildbig (filenum)

   LET bigstring$ = MID$(profilesample, 142, LEN(profilesample)) + "  "
   FOR lridx = 1 TO 7    'process filenum
   INPUT #filenum, profilesample
   LET bigstring$ = bigstring$ + profilesample + "  "
   NEXT lridx

END SUB

SUB outproc (profilenum, pointnum, pointval$)
'profilenum is between 1 and 300
'pointnum is between 1 and 400
'pointval$ is the string of elevation data from the DEM
'
IF ((profilenum > 300) OR (pointnum > 400)) THEN STOP
LET filenum = 51

'determine the filenumber for the output by first calculating the base

IF profilenum > 100 THEN filenum = filenum + 4
IF profilenum > 200 THEN filenum = filenum + 4

'now calculate the actual file number
IF pointnum > 100 THEN filenum = filenum + 1
IF pointnum > 200 THEN filenum = filenum + 1
IF pointnum > 300 THEN filenum = filenum + 1

'calculate the byte location in the file for the output of the two bytes
LET location = 0
LET locoffset = 0
LET locbase = 0

LET locbase = pointnum
IF locbase > 300 THEN locbase = locbase - 100
IF locbase > 200 THEN locbase = locbase - 100
IF locbase > 100 THEN locbase = locbase - 100

LET locoffset = profilenum
IF locoffset > 200 THEN locoffset = locoffset - 100
IF locoffset > 100 THEN locoffset = locoffset - 100

LET location = 128 + (((locbase - 1) * 200) + (locoffset * 2)) - 1

'determine the two byte values for the elevation
LET pointval = VAL(pointval$)
LET byteone = pointval MOD 256
LET bytetwo = INT(pointval / 256)

LET outbytes$ = CHR$(byteone) + CHR$(bytetwo)
PUT #filenum, location, outbytes$
filenum = 0

END SUB
