User's Guide

marXperts GmbH
Werkstr.3
D-22844 Norderstedt / Germany
Tel. +49 (40) 529 884 0
www.marxperts.com

Output

1. Overview

Program mar345dtb produces a number of output files:

Table 1: mar345dtb output files.

File type	File name	Description
Lock file	$MARLOGDIR/dtb.lock	A file indicating that another session of program mar345dtb is still active.
Message file	$MARLOGDIR/mar.message	File used at startup by program marstart
Log file	$MARLOGDIR/log/dtb.log.N	Log file with slightly more extensive output than printed on the terminal screen (stdout)
SPY file	$MARLOGDIR/spy/dtb.spy.N	Log file with native messages of the dtb controller
SPY file	$MARLOGDIR/spy/mar.spy.N	Log file with native messages of the mar345 controller
Image statistics	$MARLOGDIR/lp/mar.lp.N	Log file with statistical information about produced images
Current params	$MARLOGDIR/mar(345)dtb.dat	Saved parameters from "Edit"-page
Status file	$MARLOGDIR/dtb.status	Optional status file that is updated regularaly
Carousel file	$MARLOGDIR/csc/CSC.csv	Carousel parameters from "CSC"-page
Beam history	$MARLOGDIR/beam/dtb.time.N	Binary file containing a history of the ionization chamber readings
Motor scan	$MARLOGDIR/beam/dtb.scan.N	ASCII file containing ionization chamber readings as function of a motor positions
Beam profile	$MARLOGDIR/beam/dtb.profile.N	Binary file containing the result of a beam profile determination
Beamplot	$cwd/beamplot.xwd	Screen dump of plot in "Alignment"-window saved in X system window dump format
Crystal plate data	$MARLOGDIR/xps/XPS.dat	Stored data of currently mounted crystal plate
Crystal drop photos	$MARLOGDIR/xps/*.jpg	Stored photos from assigned drops of current plate
Thumbnails	.jpg	Thumbnail image of data image produced during data collection
Crystal photo	.jpg	Crystal photo in jpg format produced during data collection
Data Images	.marXXXX \| .cbfXXXX	Image files in mar or cbf format produced during data collection
Summary (ASCII)	.SUMMARY	ASCII file with current data set parameters produced during data collection
Summary (HTML)	.html	HTML file with current data set parameters produced during data collection
Errors	.ERRORS	List of images that contained some sort of errors during data collection
Empty image	$MARLOGDIR/xtal/empty.jpg	CSC only: latest background picture of goniometer environment without sample mounted
Loop image	$MARLOGDIR/xtal/find.jpg	CSC only: latest picture to be used for automatic crystal finding
XPS file	$MARLOGDIR/xps/XPS.dat	Parameters of crystallization plate from crystal plate scanner (XPS)

### denotes a 3-digit serial number and usually is defined as $MAR_SCANNER_NO.

ASCII file $MARLOGDIR/dtb.lock will be created at startup of program mar345dtb. Existence of this file indicates that a session is or might be active. The file will be deleted by the program when exiting the program. A program crash will leave the file where it is. If the file already exists at program startup, the program will ask you to confirm wether you really want to start another instance of it. The main reason for this procedure is that the dtb and the mar345-controller reuse the network sockets for talking to the computer, so any attempt to open a new connection to the controllers will make previous connections useless!

3. Message File

ASCII file $MARLOGDIR/mar.message will be created at startup of program mar345dtb. This file reports wether the program is able to establish connection with the controllers. This message file will be read by program marstart which presents this information in a window to be acknowledged. In case of failure, the window allows to terminate the current mar345dtb process. Program marstart is called automatically by program mar345dtb and has no other functions.

4. Log File

ASCII file $MARLOGDIR/log/dtb.log.N will be created at startup of program mar345dtb. It contains essentially the same information as printed on the terminal where you run the program.

It is generally a good idea to keep some versions of this file for backtracing hardware problems. The number of file versions to be kept is configurable (keyword VERSION LOG <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

The program automatically creates a soft link $MARLOGDIR/dtb.log pointing to the current log file $MARLOGDIR/log/dtb.log.N, so for looking into the file currently in use you don't have to descend to directory $MARLOGDIR/log and look at the creation dates of all log files.

5. Spy Files

At startup, program mar345dtb optionally creates ASCII files $MARLOGDIR/spy/dtb.spy.N and $MARLOGDIR/spy/mar.spy.N. These files contain native controller messages of both the dtb and the mar345-detector. The controllers produce very detailed information about all hardware processes.

It is generally a good idea to keep some versions of this file for backtracing hardware problems. The number of file versions to be kept is configurable (keyword VERSION SPY <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

The program automatically creates a soft link $MARLOGDIR/dtb.spy pointing to the current spy file $MARLOGDIR/spy/dtb.spy.N and $MARLOGDIR/mar.spy pointing to the current spy file $MARLOGDIR/spy/mar.spy.N.

Note: the files may become quite large in size. It is therefore advisable to have sufficient disk space!

6. Statistics File

At startup, program mar345dtb optionally creates ASCII files $MARLOGDIR/lp/mar.lp.N. This file contains some statistical information of the images produced during data collection. This file is mostly used for calibration purposes and of little importance for the user.

The number of file versions to be kept is configurable (keyword VERSION STATS <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

The program automatically creates a soft link $MARLOGDIR/mar.lp pointing to the current file $MARLOGDIR/lp/mar.lp.N.

7. Parameter Files

Program mar345dtb continuously saves parameters whenever they change within the program. When starting the program, the saved parameters are read back so the user always finds the latest changes after quitting a mar345dtb session. The parameter file read at startup and updated during program execution is called $MARLOGDIR/mar(345)dtb.dat. The parameter file is a keyworded ASCII-file which may be edited. For syntax, consult chapter Parameter File in section Input.

Parameter files can be explicitely created from within the program. This is useful when working on a project where you always apply the same set of parameters. The parameter files carry the extension ".set" and may be saved using the "Edit -> Save Parameters ..." menu option in the "Edit"-page ( see chapters 2.3 in section Edit).

8. Status File

Program mar345dtb can be configured to periodically save current status information to file $MARLOGDIR/dtb.status. This ASCII-file contains information about motor positions and about progress of current tasks of the detector or dtb. The purpose for this is to give any other program a chance to exchange information with the mar hardware.

9. Beam History File

At startup, program mar345dtb creates the binary file $MARLOGDIR/beam/dtb.time.N. This file of fixed size (approx 1 MB) stores information about beam history as plotted in the "Time"-plot in the "Align"-window. Depending on how long time program mar345dtb has been running there may or may not be entries for displaying beam history in the 5, 10 or 30 minute units.

The number of file versions to be kept is configurable (keyword VERSION TIME <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

10. Beam Scan File

When doing beam scans, i.e. digitization of ionization chamber readings as a function of some motor movement (translations, rotations, slits), program mar345dtb creates an ASCII file $MARLOGDIR/beam/dtb.scan.N for each scan. This file stores information about current motor positions in a header. The header is followed by a variable number of lines containing readings of chamber 1 and chamber 2, motor position of the moving motor and time (in Unix computer time notation). Stored files can be loaded and displayed in the plot area of the "Align"-window.

The number of file versions to be kept is configurable (keyword VERSION SCAN <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

11. Beam Profile File

Program mar345dtb is capable of establishing a beam profile by driving the horizontal and vertical translation motors around the beam at small slit sizes. Beam profiles are stored in binary format as $MARLOGDIR/beam/dtb.profile.N. This file stores information about current motor positions in a header. The header is followed by a digital array of data containing normalized chamber 1 readings for a given range of movement in x and y. Stored files can be loaded and displayed in the plot area of the "Align"-window.

The number of file versions to be kept is configurable (keyword VERSION PROFILE <max> in the Configuration File). At startup the program determines automatically which version number "N" to use. If the current "N" exceeds "max", N restarts with 1.

12. Beamplot File

All plots displayed in the plot area of the "Align"-window can be saved into file beamplot.xwd (no other choice of filename) of the current working directory. Existing beamplot.xwd files will be overwritten without warning. The file format is "XWD" (X Windows system window dump file). There are utilities for converting this format into many other image formats (png, jpg), the most useful and popular being the ones from the ImageMagick suite available for all Unix flavours, in particular program convert.

13. mar2thumb: Thumbnail Conversion Script

It is possible to produce data image thumbnails with the help of a conversion program. The command to use can be specified in the configuration file and defaults to "mar2thumb" which is a shell script that calls other programs. The shell script shown below calls program marcvt to convert a data image into a full size tiff file. The next step is reduction to a suitable thumbnail size and conversion into a compressed graphics format in order to save space. The shell script implies that the called programs (marcvt and convert) are in the executable search path otherwise, the thumbnail creation will fail. The shell script given here will be called during data collection each time an image has been stored on disk. The shell script gets that image file name as argument. The resulting thumbnail will be stored in subdirectory "thumb" of the data directory:


#!/bin/csh -f
#
# Parse command line
#
set done 	= 0
set v		= 0
set show	= 0
set q		= 50
set p		= 20
set D		= ( none )

while ( $done == 0 )

 switch ($1 )
   case '-o':
   case '-d':
   case '--directory':
	set D 		= $2
	set argv 	= ( $argv[3-] )
	breaksw

   case '-q':
   case '--quality':
	set q 		= $2
	set argv 	= ( $argv[3-] )
	breaksw

   case '-p':
   case '--percent':
	set p 		= $2
	set argv 	= ( $argv[3-] )
	breaksw

   case '-s':
   case '--show':
	set show	= 1
	set argv 	= ( $argv[2-] )
	breaksw

   case '-v':
   case '--verbose':
	set v 		= ( 1 )
	set argv 	= ( $argv[2-] )
	breaksw

   case '-h':
   case '--help':
	goto usage
	breaksw

   default:
	set done 	= 1
	breaksw
 endsw     
end

#
# Remaining arguments on command line?
#
# First argument: complete image name 
# Second argument: directory where to store thumbnail
if ( $#argv < 1 ) then
	goto usage
endif
set A = $1 
#
# Split path into directory and filename 
#
set DIR  = ( `dirname $A` )
set BASE = ( `basename $A` )
if ( $D == 'none' ) then
	set D	 = ( ${DIR}/thumb )
endif
#
# Strip off suffix from BASE name
#
set B = ( `echo "$BASE" | awk '{ split( $1, a, "." ); print a[1] }'` )
#
if ( $#argv > 1 ) then
	set D	= $2
endif
#
# Check if input file exists
#
if ( ! -f ${A} ) then
	echo "ERROR: Input file ${A} does not exist"
	goto usage
endif
#
# Check if output directory exists
#
if ( ! -d ${D} ) then
	mkdir ${D}
	if ( ! -d ${D} ) then
		echo "WARNING: Can't create output directory ${D}. Using current one"
		set D = ( . )
	endif
endif
#
# Convert mar image into tiff file and deposit in directory D
#
if ( $v == 1 ) then
	echo "Transform $A info tiff in: ${D} "
	marcvt -v -tiff --force $A -o ${D} 
else
	marcvt -tiff --force $A -o ${D} > /dev/null
endif
#
# Convert uncompressed tiff files with N x N pixels into something smaller
# using standard Unix tools, e.g. convert from the ImageMagick package
#
cd ${D}
set IN	= ( ${B}.tiff )
set OUT	= ( ${B}.thumb.jpg )
if ( $v == 1 ) then
	echo "Converting ${IN} info ${OUT} "
endif
if ( -f ${IN} ) then
	convert -quality $q -geometry ${p}% ${IN}  ${OUT}
	echo "          Thumbnail: $OUT"
else
	echo "ERROR: marcvt didn't work on file ${A}"
	goto usage
endif
/bin/rm -f ${IN}
if ( $show == 1 ) then 
	display ${OUT}
endif
exit
#
#
#
usage:
	echo "Usage: mar2thumb [OPTIONS] input_mar_image [output-directory]"
	echo "Options:"	
	echo "       -h  --help             show usage"
	echo "       -v  --verbose          show messages                [dont]"
	echo "       -s  --show             display thumbnail            [dont]"
	echo "       -q  --quality N        quality for jpg              [50]"
	echo "       -p  --percent N        shrink original to N percent [20]"
	echo "       -d  --directoy D       output directory             [thumb]"
	EXIT

14. Data Images

mar345dtb produces images in 3 formats during data collection. See chapter 4. Directories, Names and Formats of section Edit for more details about file formats.

Table 2: Image formats produced by mar345dtb

Format	Usual extension	Description
mar345	.marXXXX	Images in mar345 format
cbf	.cbfXXXX	Images in CBF format
spiral	.sXXXX	Raw spiral images

15. ASCII Summary File

During data collection, the program optionally produces an ASCII file containing all relevant information about parameters of the current data collection. The file will be saved into the current data directory and is called "Image root".N.SUMMARY. N is a number starting at 1 and is incremented if the file already exists. This file is useful for keep track of what has been done during the current data collection.

16. HTML Summary File

Same as the ASCII summary file, but in html format.

17. Automar Sync File

ASCII file that may be used by program marProcess to synchronize with processing. When configured (option 'USE AUTOMAR' in configuration file), the program writes a file rootname.sync into the current data directory. During data collection, this file is being updated with every image. Program marProcess can make use of this file to determine how many images to process. The syntax follows the requirements of the marProcess control files. See the automar documentation for further details.

18. Errors File

If an image has been recollected during a data collection because of some hardware problem, there will be a file written to the data directory containing the number of the images affected by an error. The file is called "Image root".N.ERRORS where N is a number starting at 1 and is incremented if the file already exists.

19. Images used for Crystal Finding

When using the sample changer, there is an option of automatically finding a crystal after it has been mounted. The method requires 2 images: one that contains an object ("find.N.jpg), i.e. a crystal to be located, and another that provides an empty background ("empty.jpg). Since the illumination of the sample is a critical parameter, the background file will be updated every time a new sample is mounted. This is done automatically by the mar345dtb program without user intervention. The images will be stored in directory $MARLOGDIR/xtal.

When calling the automatic crystal finding routines from within the program, the program loopfind will be executed. This program is called with 3 arguments:

$MARLOGDIR/xtal/find.N.jpg
$MARLOGDIR/xtal/empty.jpg
$MARLOGDIR/xtal/xhair.jpg

Please note, that N is a number running from 1 to 99 (configurable). I.e. after 99 versions have been created the photos will be cyclically overwritten.

For reasons of flexibility loopfind is a shell script that calls the program to actually do the work (typically program marloop):


#!/bin/csh -f
#
# Set some defaults
#
set N           = ( $#argv )
set dir         = ( ${MARLOGDIR}/xtal )
set skip        = ( )
set F1          = ( )
set F2          = ( )
set F3          = ( )
set done        = 0
set verb        = 0
set method      = 0
#
if ( $N < 1 ) then
        goto USAGE
endif
#
#
# Command line args
#
while ( $done == 0 )

        switch ( $1 )
# Verbosity
                case '-v':
                case '-verb':
                        @ verb++
                        set argv = ( $argv[2-] )
                        breaksw
# Method
                case '-m':
                case '-method':
                        set method = $2
                        set argv = ( $argv[3-] )
                        breaksw
# Directory
                case '-d':
                case '-dir':
                        set dir = $2
                        set argv = ( $argv[3-] )
                        breaksw
# Help
# Help
                case '?':
                case '-h':
                case '-help':
                        goto USAGE
                        breaksw
# Skip
                case '-s':
                case '-skip':
                        set skip = ( "-skip" )
                        set argv = ( $argv[2-] )
                        breaksw
#
#       End of cases
#
                default:
                        if ( ${F1} == '' ) then
                                set F1 = $1
                                if ( ${F1} == '' ) then
                                        set done = 1
                                else
                                        set done = 0
                                endif
                        else if ( ${F2} == '' ) then
                                set F2 = $1
                                set done = 0
                        else if ( ${F3} == '' ) then
                                set F3 = $1
                                set done = 0
                        else
                                set done = 1
                        endif
        endsw
end
#
# End of command line
#
if ( ${verb} > 0 ) then
        echo "Input file 1: ${F1}"
        echo "Input file 2: ${F2}"
        echo "Input file 3: ${F3}"
        echo "Directory   : ${dir}"
        echo "Method      : ${method}"
        echo "Skip        : ${skip}"
endif
#
# Argument is a filename containing crystal photos
#
if ( $N < 2  || ${method} == 1 ) then
        if ( ! -f ${F1} ) then
                goto USAGE
        endif
#
        cd ${dir}
        lis2txt FIND.lis XC.in
        if ( -f XC.log ) then
                /bin/rm XC.log
        endif
        XCentering XC.in XC.log
        if ( -f XC.log ) then
                CosFit -i XC.log -l CF.log
                echo ======================================================================
                if ( -f CF.log ) then
                        awk '{print "Result:    ", $2, $3, $4, $1}'  CF.log
                else
                echo "Result:    error in CosFit"
                endif
        else
                echo "Result:    error in XCentering"
        endif
        echo ======================================================================
#
# Arguments are: 1. xtal photo 2. empty photo 3. xhair photo
#
else if ( ${F1} != '' && ${F2} != '' && ${F3} != '' ) then
        echo "Running marloop"
        marloop ${skip} $1 $2 $3
else
        echo "ERROR running marloop "
        goto USAGE
endif
#
exit
#
USAGE:
echo "usage: loopfind   FIND.lis | find.N.jpg   empty.jpg  xhair.jpg"
exit

20. Carousel File

Program mar345dtb continuously saves carousel data whenever they change within the program. When starting the program, the saved parameters are read back so the user always finds the latest changes after quitting a mar345dtb session. The carousel file read at startup and updated during program execution is called $MARLOGDIR/csc/CSC.csv. The parameter file is a ASCII-file in Excel-type format using 8 columns:

Vial no.: 1 to 19
Status: one of "carousel", "mounted" or "empty"
Barcode: barcode on sample, e.g "A18B0001"
Identifier: arbitrary sample identification string, e.g "lysozyme"
Pin: Pin length in mm, either 14,16,18,21,24 or 0
x y z:Positions of xyz-translations of goniometer head where the crystal is located

All columns are separated by a colon (;). It is legal to leave a column empty, e.g. if the barcode is unknown or the xyz-coordinates are not yet determined.

During operation, the corresponding fields will be updated automatically. A barcode is usually updated in the loading process of a sample. The xyz-coordinates will be updated as a result of an automatic or manual centering process of a mounted sample. The identifier is an additional string that does not have further relevance for the program, but may help to identify samples.

All fields may be edited by hand, even the status field. When replacing an entire carousel manually, the information stored in the "Carousel status"-area of the "CSC"-page will become void. All positions should either be cleared by choosing menu option "Clear carousel data" in the "CSC"-menu of the main window. Otherwise, it is possible to manually compose a carousel file using the syntax as described above, deposit that file in directory $MARLOGDIR/csc and load the file using option "Load carousel data" in the "CSC"-menu of the main window.

The following contains an example of the contents of CSC.csv file:



! Generated by program mar345dtb on Fri Feb 27 11:09:59 2004
! and contains current settings of the mar CSC carousel
!
! Vial ; Status ; Barcode ; Identifier ; Pin ; x ; y ; z
1;carousel;B18A0153;Trypsin;18;-0.337;0.970;-4.713
2;mounted;A18B0020;Lysozyme;18;0.482;0.228;-3.460
3;empty;A18B0026;Myoglobin;18;0.482;0.228;-3.460
...

21. XPS File

Program mar345dtb continuously saves crystal plate scanner (XPS) parameters whenever they change within the program. When starting the program, the saved parameters are read back so the user always finds the latest changes after quitting a mar345dtb session. The XPS file read at startup and updated during program execution is called $MARLOGDIR/xps/XPS.dat. It is an ASCII-file and contains geometric parameters of a given crystallization plate and coordinates for drop positions within the plate together with status indicators. The file is not supposed to be edited manually. If it is missing,defaults will apply and the file will be regenerated. The worst thing that might happen is that all stored drop positions are lost.