Perl P2/91 & P2/94 fathometer data correction script (code nostalgia)

This is the last posting of a series of postings initiated by me finding an old backup hard disk containing some scripts written when I was working in the Marine Seismic industry.

I remember hacking these scripts in the ed editor on HP-UX, with no version control apart from manually maintaining the versions in different directories. Code wise the script is not something to look at for code structure etc... but it worked well at the time over a period of almost 3 years.

Perl 5 did have full support for classes but for some reason I did not implement this using OO.

The help information in the script states:

DESCRIPTION:
------------
This script comes in handy when your echosounder data in the P291/P294
file either is extremly noisy or non existing. The script can:
- correct noisy data, by deleting corrupt data and inserting new data
- insert new data where data is missing
The script needs the P291/P294 file with the bad data, also needed is a
"WD" file where each line contains a data pair of (SP/Water depths).


WHERE TO GET THE WD FILE:
-------------------------
SEISMIC data processors. To see more help type: "correct_P2_fatho HELPSEISMIC"
at the command prompt.

This version the script allows multiple SP ranges in the WD file.
What happens is that when the SEISMIC data operators pick the water bootom for the
SP range that are bad, the normally pick all SP ranges at one time so you
get at WD file consisting of several SP range-datasets like this (SP/depth):

 992      1285    (start first SP range)
  .         .
 999      1022    (end first SP range)
1245      1000    (start second SP range)
  .         .
1255       995    (end second SP range)   
The script will insert or remove the values for the SP ranges in the WD file
(in this example: 992-999, 1245-1255). The SP range in between the 2 ranges will
not be changed and will retain the original values from the echosounder.


IMPORTANT:
----------
The water depth file cannot have any text/empty lines in the beginning
of the file. The last line must be the last data set (no empty lines).
The script does not care about spaces or TABs on a line. Leading spaces
in a line is no problem. Some times the file from SEISMIC data processing can have double
values for the same shotpoint, this is not a problem.

 992      1285    (double values, not a problem)
 992      1285
      993              1285    (different spacing, not a problem)

and for seismic data processors the help states the following

             correct_p2_fatho v1.3
                  by
                Kenneth Thorman
             SEISMIC data - HELP
                     
1. Migrate the Near Trace Gather
2. Pick the water depth of migrated near trace gather in QCTOOL
3. Output formatted listing of DBU (this will be shotrec/time pairs)
4. In DBU examine file created in QCTOOL

EXAMPLE DBU FORMATTED OUTPUT LISTING

  Enter project name [exnor] :
  Enter access mode (r[eadonly], u[pdate]) [readonly] : u
  Enter selection (h|...|q) [l] : e 17awbpick
  Enter examine option (h|l|o|d|f|g|...|q) [l] : o
  Enter output listing file name (or "stdout") [stdout] : 17wblist.txt
  Enter examine option (h|l|o|d|f|g|...|q) [l] : f
  Enter filename of an existing format [] :
  Enter maximum line length for the list (max 255) [79] :
  Enter number of chars for instance number (0=none) [0] :
  Enter repeat factor for header line (0=no header) [50] : 0
  Select fields to tabulate :
  Enter field (q to quit, \ to list) [KEY1] :
  Enter number of characters to display field data in [4] : 8
  Enter field (q to quit, \ to list) [key1] : time
  Enter number of characters to display field data in [4] : 8
  Enter number of decimal places to display [2] : 4
  Enter field (q to quit, \ to list) [time] : q
  Enter filename to save format (leave blank for none) [] :
  Enter first instance to list (0 to quit) [1] :
  Enter last instance to list [21] :
  Enter first instance to list (0 to quit) [1] : 0
  Enter examine option (h|l|o|d|f|g|...|q) [l] : q
  Enter selection (h|...|q) [e] : q


5. run the following command from the command line:
SYNTAX:

awk '{printf "%s \t %.2f\n", $1+SPRec->SP, $2*Water_Vel}' inputfile > outputfile       

Variables:
SPRec-SP    -     difference from shot record to SP (1000)
Water_Vel   -     water velocity used / 2       ( 750)
inputfile   -     the DBU list
outputfile  -     file to give to the QC

Example:
awk '{printf "%s \t %.2f\n", $1+1000, $2*750}' EX00-22.WD.dbu > EX00-22.waterdepth

.
And finally here is the script

#!/opt/perl5/bin/perl

#################################################################################
#   GENERAL COMMENTS FOR THIS SCRIPT   #
#################################################################################
#           #
# THE TOP LINE YOU SEE IN THIS SCRIPT IS THE LINE THAT TELLS THE COMPUTER WHERE #
# TO LOOK FOR THE PERL - EXECUTOR. (NO IT HAS NOTHING TO DO WITH HANGING :-) #
# IF YOU DO NOT KNOW WHERE PERL IS LOCATED/INSTALLED ON YOUR COMPUTER THEN TRY  #
# TYPING THE FOLLOWING AT THE COMMAND PROMPT.     #
#           #
# whereis perl         #
# which perl         #
#           #
# IMPORTANT:  THIS SCRIPT WAS WRITTEN FOR PERL VERSION 5. SINCE THERE WAS A  #
#   BIG RE-WRITE/CHANGE FROM V4 TO V5 I DOUBT IT WILL RUN ON PERL 4 #
#           #
# 17 Dec 1999,          #
# Kenneth Thorman        #
#################################################################################




 

#################################################################################
#    SUB ROUTINE DECLARATION     #
#################################################################################
sub GetFileBaseName;
sub CheckCommandLineSyntax;
sub OpenFiles;
sub DecideDepthOutput;
sub PutWaterDepthFileIntoArray;
sub ExtractSP;
sub ExtractWD;
sub FormatEchosounderTimeStamp;
sub CreateEchosounderDataCard;
sub Presentation1;
sub Presentation2;
sub Presentation3;
sub Presentation4;
sub DisplayOnlineHelp;
sub DisplayHELPSEISMIC;

#################################################################################
#   VARABLE DECLARATION     #
#################################################################################
$wd_file = $ARGV[1];
$outfile = "$ARGV[0]" . ".wd";
$log_dir = "./";



#################################################################################
#   MAIN SCRIPT FLOW     #
#################################################################################
CheckCommandLineSyntax;
OpenFiles;
$filebasename = GetFileBaseName($ARGV[0]);
$wd_line = <WDFILE>;
$firstsp = int (ExtractSP ($wd_line));
PutWaterDepthFileIntoArray;
$fields_in_array = @wd_file;
$wdc = 0;

$min_sp = 100000;
$x = $fields_in_array;
while ($x >= 0) {    # Loop through the array backwards
  if (($min_sp > $x) and (@wd_file[$x] ne "")) {
    $min_sp = $x;
  }
  $x--;
}

$x = $min_sp;
while ($x <= $fields_in_array) {
  if ((@wd_file[$x] eq "") and (@wd_file[$x-1] ne "")){
    @wd_sp_string[$wdc] = $x-1;
    $wdc++;
  }
  elsif ((@wd_file[$x] ne "") and (@wd_file[$x-1] eq "")){
    @wd_sp_string[$wdc] = $x;
    $wdc++;
  }  
  $x++;
}
print (@wd_sp_string);

$x = 1;
while (@wd_sp_string[$x] ne ""){
  if (@wd_sp_string[$x+1] eq ""){
    $wd_sp_ranges = $wd_sp_ranges . $wd_sp_string[$x-1] . "-" . $wd_sp_string[$x] . ".";
  }
  else{
   $wd_sp_ranges = $wd_sp_ranges . $wd_sp_string[$x-1] . "-" . $wd_sp_string[$x] . ", ";
  } 
  $x = $x + 2;
}
 
$fields_in_array = @wd_file;
$lastsp = $fields_in_array - 1;
$T14101_Cards_Encountered = 1;
Presentation1;


$sp_count = 0;
$line = <P291FILE>;
while ($line ne ""){
  if (($line =~ /E1000/) and ($line =~ /$filebasename/)){
    $p291sp = int(substr($line,24,8));
    if ($sp_count == 0){
      Presentation2;
      $sp_count = 1;
    }
    elsif (($p291sp % 500) == 0){
      Presentation3;
    }
    if (($T14101_Cards_Encountered == 0) and ($wd_file[$p291sp] ne "")){
      $ETimeStamp = FormatEchosounderTimeStamp($line);
      $wd = @wd_file[$p291sp];
      $string = CreateEchosounderDataCard($wd, $ETimeStamp, "                                                                     ");
      print OUTFILE ($string);
      print OUTFILE ($line);
      $T14101_Cards_Encountered = 0;
    }  
    else{
      print OUTFILE ($line);
      $T14101_Cards_Encountered = 0;
    }
  }
  elsif (($line =~ /T14101/) and ($wd_file[$p291sp] ne "")){
    $wd = @wd_file[$p291sp];
    $rest_str = substr($line,12,68);
    $string = CreateEchosounderDataCard($wd, $rest_str);
    print OUTFILE ($string);
    $T14101_Cards_Encountered = 1;
  }
  else{
    print OUTFILE ($line);
  }
  $line = <P291FILE>;
}
$date = `date -u +%x`;
chop ($date);
print LOGFILE ("$date     $outfile     $wd_sp_ranges\n");  
Presentation4;





#################################################################################
#   SUB ROUTINES      #
#################################################################################

#################################################################################
#COMMENTS: "CheckCommandLineSyntax"      #
#-------------------------------------------------------------------------------#
#          #
# ASSIGN THE NUMBER OF CMD. LINE ARG. TO $arg_cnt    #
# IF NUMBER OF CMD. LINE .ARG. NOT EQUALS 2     #
# EXIT PROGRAM AND DISPLAY THE FOLLOWING LINE     #
# "SYNTAX: correct_p291_wb <P291 File> <Waterdepth file> ..."   #
#################################################################################

sub CheckCommandLineSyntax{
  $arg_cnt = @ARGV;   
  if ($ARGV[0] eq "HELPSEISMIC"){
    DisplayHELPSEISMIC;
    die ("SYNTAX: correct_p2_fatho <P2 File> <Waterdepth file> ...\n\n");
  }
  elsif ($arg_cnt != 2){
    DisplayOnlineHelp;
    die ("SYNTAX: correct_p2_fatho <P2 File> <Waterdepth file> ...\n\n");
  }
}








sub DisplayOnlineHelp{
  $~ = "HELP";
  write;
}
format HELP =

     correct_p2_fatho v1.3
      by
       Kenneth Thorman
          
DESCRIPTION:
------------
This script comes in handy when your echosounder data in the P291/P294
file either is extremly noisy or non existing. The script can:
- correct noisy data, by deleting corrupt data and inserting new data
- insert new data where data is missing
The script needs the P291/P294 file with the bad data, also needed is a 
"WD" file where each line contains a data pair of (SP/Water depths).


WHERE TO GET THE WD FILE: 
-------------------------
SEISMIC data processors. To see more help type: "correct_P2_fatho HELPSEISMIC"
at the command prompt.

This version the script allows multiple SP ranges in the WD file.
What happens is that when the SEISMIC data operators pick the water bootom for the 
SP range that are bad, the normally pick all SP ranges at one time so you 
get at WD file consisting of several SP range-datasets like this (SP/depth):

 992   1285 (start first SP range)
  .         .
 999    1022 (end first SP range)
1245   1000 (start second SP range)
  .         .
1255   995 (end second SP range) 
The script will insert or remove the values for the SP ranges in the WD file
(in this example: 992-999, 1245-1255). The SP range in between the 2 ranges will 
not be changed and will retain the original values from the echosounder.


IMPORTANT: 
----------
The water depth file cannot have any text/empty lines in the beginning 
of the file. The last line must be the last data set (no empty lines). 
The script does not care about spaces or TABs on a line. Leading spaces 
in a line is no problem. Some times the file from SEISMIC data processing can have double
values for the same shotpoint, this is not a problem.

 992   1285 (double values, not a problem)
 992   1285
   993     1285 (different spacing, not a problem)
   
.






sub DisplayHELPSEISMIC{
  $~ = "HELPSEISMIC";
  write;
}
format HELPSEISMIC =

     correct_p2_fatho v1.3
      by
       Kenneth Thorman
    SEISMIC data - HELP
          
1. Migrate the Near Trace Gather
2. Pick the water depth of migrated near trace gather in QCTOOL
3. Output formatted listing of DBU (this will be shotrec/time pairs)
4. In DBU examine file created in QCTOOL

EXAMPLE DBU FORMATTED OUTPUT LISTING

  Enter project name [exnor] : 
  Enter access mode (r[eadonly], u[pdate]) [readonly] : u
  Enter selection (h|...|q) [l] : e 17awbpick
  Enter examine option (h|l|o|d|f|g|...|q) [l] : o
  Enter output listing file name (or "stdout") [stdout] : 17wblist.txt
  Enter examine option (h|l|o|d|f|g|...|q) [l] : f
  Enter filename of an existing format [] : 
  Enter maximum line length for the list (max 255) [79] : 
  Enter number of chars for instance number (0=none) [0] : 
  Enter repeat factor for header line (0=no header) [50] : 0
  Select fields to tabulate :
  Enter field (q to quit, \ to list) [KEY1] : 
  Enter number of characters to display field data in [4] : 8
  Enter field (q to quit, \ to list) [key1] : time
  Enter number of characters to display field data in [4] : 8
  Enter number of decimal places to display [2] : 4
  Enter field (q to quit, \ to list) [time] : q
  Enter filename to save format (leave blank for none) [] : 
  Enter first instance to list (0 to quit) [1] : 
  Enter last instance to list [21] : 
  Enter first instance to list (0 to quit) [1] : 0
  Enter examine option (h|l|o|d|f|g|...|q) [l] : q
  Enter selection (h|...|q) [e] : q


5. run the following command fron the command line:
SYNTAX:

awk '{printf "%s \t %.2f\n", $1+SPRec->SP, $2*Water_Vel}' inputfile > outputfile  

Variables:
SPRec-SP    -  difference from shot record to SP (1000)
Water_Vel   -  water velocity used / 2    ( 750)
inputfile   -  the DBU list 
outputfile  -   file to give to the QC

Example:
awk '{printf "%s \t %.2f\n", $1+1000, $2*750}' EX00-22.WD.dbu > EX00-22.waterdepth

.




#################################################################################
#COMMENTS: "GetFileBaseName"       #
#-------------------------------------------------------------------------------#
#          #
# INPUT  : FILENAME (TEXT STRING) EX. BR98-092.0.P291    #
# FUNCTION: READ THE STRING FROM THE START WHEN ENCOUNTING A "." THROW THE REST #
#    AWAY         #
# OUTPUT  : FILE BASE NAME (LINENAME) EX. BR98-092    #
#################################################################################

sub GetFileBaseName{
  my ($input_str) = @_;
  my ($end_pos, $temp);
      
  if ($input_str =~ /\./g)
  {
    $end_pos = pos ($input_str);
  }
  
  $temp = substr ($input_str, 0, $end_pos-1);
  return ($temp);
}








#################################################################################
#COMMENTS: "OpenFiles"        #
#-------------------------------------------------------------------------------#
#          #
# OPENS ALL NECCESARY FILES (P291 ORIGINAL, WATER DEPTH FILE, OUTFILE)  #
# IF A PROBLEM IS ENCOUNTERED OPENING ANY OF THE FILE AN ERROR MESSAGE IS  #
# DISPLAYED.         #
#################################################################################

sub OpenFiles{
  open (WDFILE, "$wd_file") or die ("Could not open infile $ARGV[1].\n");
  open (P291FILE, "$ARGV[0]") or die ("Could not open infile $ARGV[0].\n");
  open (OUTFILE, ">$outfile") or die ("Could not open outfile $outfile.\n");
  $log = $log_dir . "correct_P2_fatho.LOG";
  if (-e $log){
    open (LOGFILE, ">>$log") or die ("Could not open logfile.\n");
  }
  else{
    open (LOGFILE, ">$log") or die ("Could not open logfile.\n");
    print LOGFILE ("\n\n\n     Echosounder data\n");
    print LOGFILE ("Date:      Filename:   corrected for SP range:\n");
    print LOGFILE ("-----------------------------------------------------------\n");
  }
}







#################################################################################
#COMMENT: "Presentation 1-3"       #
#-------------------------------------------------------------------------------#
#          #
# THIS SUB ROUTINE PRESENTS IMPORTANT INFORMATION TO THE USER IN A EASIER #
# READABLE WAY THAN IF IT HAD BEEN TRHOWN OUT ON THE SCREEN.   #
#################################################################################

sub Presentation1{
$~ = "MYFORMAT1";
write;
}

sub Presentation2{
$~ = "MYFORMAT2";
write;
}

sub Presentation3{
$~ = "MYFORMAT3";
write;
}

sub Presentation4{
$~ = "MYFORMAT4";
write;
}


format MYFORMAT1 =




***************************************************************************
*      FILE       |  FIRST SP  |  LAST SP  |  NOW PROCESSING  *
***************************************************************************
* Water depth correction file: @<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<*
           $wd_sp_ranges
***************************************************************************
.

format MYFORMAT2 =
* @<<<<<<<<<<<<<<<<<<<<<<<<<< |  @|||||||  |           |      *
  @ARGV[0]     $p291sp
.

format MYFORMAT3 =
* @<<<<<<<<<<<<<<<<<<<<<<<<<< |       |           |@|||||||||||||||||*
  @ARGV[0]          $p291sp
.

format MYFORMAT4 =
* @<<<<<<<<<<<<<<<<<<<<<<<<<< |       |  @||||||  |    *
  @ARGV[0]        $p291sp
***************************************************************************
* STATUS: FINISHED PROCESSING            *
* CORRECTED P291 FILE: @<<<<<<<<<<<<<<<<<<<<<<      *
   $outfile
***************************************************************************

Author  : Kenneth Thorman        
Script  : correct_p291_fatho v1.3      
Date    : 19/6 - 1999        


*********************************************************************
* This script is shareware, if you find this script usefull a small *
* contribution would be appreciated.                                *
* It encourages to write more usefull scripts and programs.         *
*********************************************************************
       
.








#################################################################################
#COMMENT: "ExtractSP"        #
#-------------------------------------------------------------------------------#
#           #
# THIS SUB ROUTINE EXTRACTS THE SHOTPOINT NUMBER IN A TEXT FILE   #
# THE FILE CAN HAVE LEADEING SPACES, IT CAN BE UNEVENLY SEPARATED BY SPACES #
# BUT IT MUST CONTAIN 2 NUMBERS PER LINE, SHOTPOINT NUMBER / WATERDEPTH  #
# EXAMPLES: 993   1285       #
#    994       1285   #
#          #
# BASICALLY IT DOES NOT CARE ABOUT SPACES, AT THE TOP OF THE FILE THE CAN BE #
# NO TEXT, AND THERE CAN BE NO BLANK LINES INBETWEEN LINES   #
#          #
# INPUT:  ONE LINE OF TEXT      #
# RETURN VALUE: SHOTPOINT NUMBER      #
#################################################################################

sub ExtractSP{

  my ($temp_line) = @_;
  my (@temp_array, $SP);
  
  @temp_array = split (/[\t ]+/, $temp_line);
  if (@temp_array[0] eq ""){
    $SP = @temp_array[1];
  }
  else{
    $SP = @temp_array[0];
  }
  return ($SP);
}








#################################################################################
#COMMENT: "ExtractWD"        #
#-------------------------------------------------------------------------------#
#           #
# THIS SUB ROUTINE EXTRACTS THE WATER DEPTH NUMBER IN A TEXT FILE  #
# THE FILE CAN HAVE LEADEING SPACES, IT CAN BE UNEVENLY SEPARATED BY SPACES #
# BUT IT MUST CONTAIN 2 NUMBERS PER LINE, SHOTPOINT NUMBER / WATERDEPTH  #
# EXAMPLES: 993   1285       #
#    994       1285   #
#          #
# BASICALLY IT DOES NOT CARE ABOUT SPACES, AT THE TOP OF THE FILE THE CAN BE #
# NO TEXT, AND THERE CAN BE NO BLANK LINES INBETWEEN LINES   #
#          #
# INPUT:  ONE LINE OF TEXT      #
# RETURN VALUE: WATER DEPTH       #
#################################################################################

sub ExtractWD{

  my ($temp_line) = @_;
  my (@temp_array, $WD);

  @temp_array = split (/[\t ]+/, $temp_line);
  if (@temp_array[0] eq ""){
    $WD = @temp_array[2];
  }
  else{
    $WD = @temp_array[1];
  }
  return ($WD);
}








#################################################################################
#COMMENT: "PutWaterDepthFileIntoArray"      #
#-------------------------------------------------------------------------------#
#          #
# THIS SUB ROUTINE DOES EXACTLY WHAT THE NAME IMPLIES. IT USES THE 2 ABOVE  #
# MENTIONED SUB ROUTINES TO EXTRACT THE SP NUMBER AND THE WATER DEPTH FROM A  #
# FILE AND THE IT PUTS THESE VALUE INTO AN ARRAY BUILT UP AS FOLLOWS  #
# SP NUMBER (INDEX VALUE) / WATER DEPTH (FIELD VALUE)    #
#          #
# SO NOW YOU CAN REFERENCE THE ARRAY BY SHOTPOINT NUMBER AND THEN GET THE  #
# WATER DEPTH AT THE SPECIFIC SHOTPOINT      #
#################################################################################

sub PutWaterDepthFileIntoArray{
  while ($wd_line ne ""){
    $sp = int (ExtractSP ($wd_line));
    $wd = int (ExtractWD ($wd_line));
    @wd_file[$sp] = $wd;
    $wd_line = <WDFILE>;
  }
}







#################################################################################
#COMMENT: "FormatEchosounderTimeStamp"      #
#-------------------------------------------------------------------------------#
#          #
# THIS SUB ROUTINE EXTRACTS AND FORMATS THE TIMESTAMP FROM THE E1000 GENERAL    #
# CARD  IN THE P291 FILE AND SUBTRACTS 0.1 SECOND.     #
# THIS IS DONE SO THE ECHOSOUNDER CARD CREATED LATER        #
# WILL HAVE A SEQUENTIAL INCREASING TIMESTAMP COMPARED TO THE CARDS LOCATED     #
# BEFORE AND AFTER IT IN THE P291 FILE. THE TIME STAMP IN THE GENERAL CARD IS   #
# NOT IN THE SAME FORMAT AS THE TIMESTAMP NEEDED FOR THE ECHOSOUNDER CARD #
#          #
# GENERAL CARD: HHMMSS.S       #
# ECHO CARD   : HHMMSSS        #
#          #
# INPUT  : E1000 - GENERAL LINE FROM P291 FILE     #
# OUTPUT : TIMESTAMP IN CORRECT FORMAT FOR A T14101 CARD IN THE P291 FILE #
#################################################################################

sub FormatEchosounderTimeStamp{
  
  my ($temp_line) = @_;
  my ($string);
  
  $time = substr ($temp_line,58,9);
  $string = sprintf("%7.1f", ($time - 0.1));
  $string =~ s/\.//;
  return ($string);
}







#################################################################################
#COMMENT: "CreateEchosounderDataCard"      #
#-------------------------------------------------------------------------------#
#          #
# THIS SUB ROUTINE CREATES/OR MODIFIES THE ECHOSOUNDER CARD DEPENDING ON WHICH  #
# ROUTINE IT IS CALLED FROM, AND WHAT VARIABLE THAT IS PASSED TO IT.  #
#################################################################################

sub CreateEchosounderDataCard{

  my ($wd, $TimeStamp, $end_string) = @_;
  my ($temp_string);

  if ($wd < 100){      
    $water_depth = sprintf("  %5.1f", $wd);
  }
  elsif ($wd < 1000){
    $water_depth = sprintf(" %5.1f", $wd);
  }
  else{
    $water_depth = sprintf("%5.1f", $wd);
  }

  $temp_string = "T14101" . "$water_depth" . "$TimeStamp" . "$end_string";
  $string = substr($temp_string,0,80) . "\n";
  return ($string);
}








# Version History:

# 1.3:  The online help and the comments in the script clarified
# plus check for compatibility with the new P294 format
# 1.2:  Seperate SP-ranges in the WD file allowed
# 1.1:  The output to a log file was created.
# 1.0:  Script created