Using Client-Side CGI to Implement a PDS Search and Display System

by Dr. Eliot Young (Southwest Research Institute) and Sky Coyote (Intergalactic Reality)

April 1998

1. CSCGI: What is it?

CGI stands for "Common Gateway Interface", which is a World Wide Web protocol by which a web server can call and pass data to and from external programs on a remote computer. Usually this involves calling either CGI scripts (often written in Perl) or binary programs (often written in C). To communicate with a CGI program, the web server uses a set of environment variables, and typically passes HTML form data via the stdin and stdout file handles of the program. That is, the CGI program reads its standard input to get a set of variables to process, and returns results to the web server via its standard output.

Client-side CGI (CSCGI) differs from CGI in that it does not involve a remote system, a web server, an Internet connection, or local Internet software such as TCP or PPP. In CSCGI, HTML form data and results can be passed directly between the web browser and an external CGI program on the local system:

In CGI, the flow of data goes from the calling HTML page, to the web browser, to the Internet software and hookup, to the remote web server, and then to the remote CGI program. In CSCGI, the flow of data goes from the calling HTML page, to the web browser, to a Netscape plugin, and then directly to the CGI programs on the local system.

Some of the important features of this architecture are:

  1. By using CSCGI, component applications consisting of several independent C programs can be coordinated and managed as a single program on a local computer.
  2. One or more HTML web pages act as the user front-end for such a component system, so that all graphical I/O and control is performed through the standard operations of a web browser. This means that a graphical user interface can be implemented with standard form elements available in HTML.
  3. This architecture is transparent to the user, and the overall performance is comparable to that of a stand-alone C program.
  4. CSCGI provides access to legacy code written in Fortran, C, etc..., from a browser.

To enable CSCGI, a feature of Netscape 3-4 called LiveConnect is used, which essentially allows the creation of a Java interface for a C plugin. Unfortunately, this means that CSCGI applications currently run only under Netscape 3-4 (version 4 strongly recommended). At present, a Netscape plugin and associated CSCGI programs are written for the Windows95 operating system, although Macintosh support is expected soon.

A CSCGI program (whether it is written in C, Perl, Fortran, etc...) must parse its CGI variables, perform some specific operation, format the result as an HTML page or specific text or binary data, and return the result via stdout. A Netscape CSCGI plugin provides the data interface from the browser, Javascript, and Java, to the external CGI program:

During typical use, the browser displays an HTML page which contains form elements. Javascript handlers and functions package this form data (including URL escape encoding), as if it were to be sent to a remote CGI program using the POST method. The CSCGI plugin writes a few CGI environment variables (such as CONTENT_LENGTH), launches an external program by name, and opens pipes to its standard input and output. Nearly every scripting and programming language (e.g. Fortran, Basic, C, Perl, TCL/TK, etc...) can communicate with the CSCGI plugin via the stdin and stdout pipes.

The plugin writes the input data string to the program, and then waits and reads the program's stdout until end of data is received. The resulting string is formated as a Java String, and is returned to the Javascript function in the calling HTML page. In the HTML page, the result from the CSCGI call can be displayed as another HTML page, or used as binary data passed to another Javascript or Java applet function.

2. What can CSCGI be used for?

As a test case example of a CSCGI component application, it was decided to build a Planetary Data System (PDS) CD-ROM image search and display browser. The PDS series contains, among other images, all of the Voyager 1 and 2 images of Saturn on 2 600 Mb CD-ROMs. This catalog consists of just over 4000 images and a text file database of common image information. The PDS image format is unusual (for example, it cannot be opened by Photoshop without a special plugin), and is compressed using variable-length records which require the use of special software to convert an image to binary format.

It was our goal to build a turnkey system which would search the text-file PDS database included on the CDs and return images and file information based on search crieteria entered from an HTML page. It was envisioned that this system would work much like a search program on the web such as AltaVista, with the addition of a built-in image and file information viewing capability.

This system was built using the CSCGI technology described above. The search and display engines are stand-alone C programs which do not require an Internet connection, a remote computer, nor a web server. The overall system consists of a small number of HTML pages, Java applets, and C programs, rather than a large monolithic application. The main components of the system are an HTML search form, one or more search result pages created by the search engine, and a Java-based PDS image and file viewer:

The web pages consist of:

  1. cdsearch3.htm: The search form page.
  2. sresults#.htm: One or more pages of search results.
  3. iread.htm: The PDS image display page.
  4. finfo.htm: The PDS file information page.
The Java applets of the system consist of:

  1. imqview1.class: A Java applet to display a full-size PDS image.
  2. imqview2.class: A Java applet to display a thumbnail PDS image.
Finally, the CSCGI programs consist of:

  1. cdsearch3.exe: The search engine which produces HTML result pages.
  2. imqread3.exe: The program which produces the image display page.
  3. imqread1.exe: The program which decompresses the PDS image.
  4. cdfinfo.exe: The program which writes the HTML file info page.
Here is an example of a simple search criteria form page:

Below is an example of one of the search result pages created by the search engine. These pages are written "on the fly" by the stand-alone search program called by the search form. They contain a summary of the search query, and an ordered list of those PDS records which best match the search criteria. Search results are saved to disk (e.g. "sresults1.htm") so that the most recent search can be revisited by simply loading an HTML page. Thumbnail images of each PDS image are also decompressed from the CD-ROM and displayed "on the fly" by a set of Java applets embedded in the page. One or more such pages can be returned:

Selecting one of the thumbnail images or links in a search result page launches a PDS image viewer and file information page (those images which are in the database, but are not on the currently mounted CD-ROM, do not create links or thumbnails). Again, the PDS image is decompressed from CD-ROM and displayed by a Java applet, at one of several magnifications. Another CSCGI program reads the text database, and creates an HTML page containing a listing of the record which corresponds to the displayed image:

Here is a graphical summary of how this system fits together:

This PDS example was meant only as a test case for CSCGI, although it may become an independent product or a prototype for a more generic on-line image database search and display system. Other examples of using a component CSCGI system might also include adding file reading and writing capabilities to normal Java applets, or maintaining and interacting with a large, live, database using "continuous connect" CSCGI mode.

3. How do you get CSCGI?

All CSCGI plugins, programs, web pages, and applets described above, plus many additional resources, will be available for Windows95 on May 15, 1998, at the website listed below. A Macintosh version of the system is expected by June 15, 1998. Please note that this system only works under Netscape 3-4, and that version 4 is strongly recommended. Please also note that this system uses Java 1.0.2. Additional examples and documentation will be available in Summer of 1998. The website location for all code, documentation, and other information is:

For more information about CGI programming, consult the following website:

The authors expect to have an August 1998 workshop and tutorial in CSCGI programming for Windows95 and Macintosh in the Boulder CO area. Please contact them if you are interested in attending this workshop. You may write to the authors at: