Gethdrs

gethdrs is a program that displays HTTP headers, and optionally content provided the content is textual. It is useful primarily for debugging or testing, and handles both direct requests and HTTP redirects. When HTTP redirects are used by a web server, gethdrs will show headers for each redirect individually. gethdrs can use proxies for HTTP, HTTPS. Proxies can be turned on and off as desired, to facilitate comparing HTTP responses with and without a proxy.

gethdrs can be run using a GUI or as a command-line program. The GUI is generally more convenient.

Running Gethdrs with a GUI

To start the GUI, one simply runs gethdrs with no arguments. The application will display a window with a series of text fields, check boxes, and buttons near the top of the screen, just above a large text area which will display the output. In addition to menus, gethdrs provides support keyboard navigation.

The initial view displayed shows the data needed to formulate an HTTP request. For the simplest use, one can simply enter a URL into the text box and type RETURN. Otherwise one can use the TAB key to move forwards or SHIFT-TAB to move backwards to cycle through buttons and text fields. When at a button or checkbox, the SPACE key selects it (this is the normal Java convention for Java's default look and feel.) Output will appear in the Output view (see the View menu.) After typing RETURN in the Request Data view, the GUI will automatically shift to the Output view. If you cancel a request, however, one will switch back to the view that existed when the request was started.

gethdrs is controlled by the following controls in all views:

The URL text field has the input focus when gethdrs is started. Once a URL is entered into this box, one can simply push the RETURN key to send a request.
Selecting the use proxies checkbox will force all configured proxies to be used (see the description of gethdrs preferences for details.)
The maximum Content Characters text field allows you to specify how many characters of content to display. If blank, the value is taken to be zero. If the value "*" is used, the content length is unbounded. Otherwise a non-negative integer should be entered.
The Get Headers button simply causes the URL to be loaded. While it is loading, the Cancel button will be active and can be used to cancel the operation. One can either click on the button or simply hit the SPACE key while the contents are being loaded.

Output View

Once a request has been sent, the Output view will be displayed (you can send simple requests from this view, but to change other parameters (i.e, for POST or PUT methods), you must first use the Request Data view. For the Output view, the Page-Up and Page-Down keys will scroll the content window up and down by a full page. These keys cause text to scroll the start and end of respectively if pushed when the control key is down. The Home and End keys will move the cursor to the start and end of the URL text field when this field has the keyboard focus, and for convenience, this is usually the case. To move to the start and end of the output, use Control-PageUp and Control-PageDown respectively.

Request Data View

The Request Data view can be traversed using keyboard navigation. While the platform defaults are used, typically a TAB key will traverse in one direction, SHIFT-TAB in the reserve direction, and SPACE will select the current button. There are two exceptions. In the header table and input text frame (the last frame in the view), you need to use CONTROL-TAB and CONTROL-SHIFT-TAB. Otherwise you will either stay in that particular view or will enter a character. The controls are as follows (many contain a button to provide dialog boxes, followed by a text box containing the value, and either may be used as desired):

There are several controls:

The connection timeout control allows you to specify a timeout for establishing a connection to a server. The units are seconds and a value of 0 indicates an unlimited timeout. Setting a timeout is useful when gethdrs is used in a script as it will allow the script to run to completion if a server or connection hangs.
The read timeout control allows you to specify a timeout for reading data from a server. The units are seconds and a value of 0 indicates an unlimited timeout. Setting a timeout is useful when gethdrs is used in a script as it will allow the script to run to completion if a server or connection hangs.
The method control allows you to select the request method (GET, HEAD, POST, or PUT).
The URL Encode Request Data checkbox, when selected, causes the text displayed in the lower text area to be URL encoded and formatted properly. For this case, form-control names should be on a line with no leading spaces, and the corresponding value should appear on following lines, indicated by a leading TAB. This leading TAB will be removed. Line breaks are otherwise preserved. This syntax differs from the one specified for the application/x-www-form-urlencoded media type but is much more readable.
The Input Charset controls set the charset for any text files being read.
The Mime Type controls set the media type used in a PUT or POST request. This should be the full media type, including parameters. In particular, a CHARSET parameter should be provided (and this may be different from the one provided by the Input Charset control, with charset conversion handled automatically for text files.
The Header File controls set the file name in which additional headers can be found. These must consist of names, followed by a colon, in turn followed by a value, all on one line per header. Once loaded, the headers specified the file will appear in a table, with previous entries removed.
The Data File controls set the file name in which the input can be found. Once loaded, for recognized text formats, the contents of the file will be displayed. Otherwise the file name will be shown.

The headers and values that will be used will appear in a header table and text area (separated by a movable bar.) These are controlled by five buttons:

The Append Row button adds an empty row to the header table.
The Insert Row button inserts an empty row into the header table. A single row must be selected to indicate the insertion point.
The Delete Rows button deletes the rows that are currently selected.
The Load button loads the contents of the header and data files.
The Clear Data & Hdrs button clears the contents of the header table and input data text box.

While normally one will simply read in data from a file, the user may also choose to edit the text or header values. The edited values will be used when possible. For binary data or any text format not recognized by gethdrs, an indication of the file name will appear in the text box, and cannot be meaningfully edited. In most cases there will be no attempt at verifying that the data provided has the format specified by the corresponding media type - as am example, for testing web severs it can be useful to be able to send incorrectly formatted data.

Preferences View

Preferences can be displayed and modified by selecting the Preferences item in the View menu, which will show a series of check boxes and text fields. The check boxes enable various proxies (HTTP, HTTPS, FTP, and SOCKS.) An additional checkbox sets the default value for the "use proxy" checkbox (the value used when gethdrs is started) in the top portion of the frame. A final checkbox allows the user-agent header to be changed from the default (e.g., to mimic a specific browser for cases where a web page may provide different content for one browser than another.)

For each proxy that is going to be used, the proxy text fields should give a host (or internet address in "dot" notation) and a port number. The "Exclude from Proxy" text fields may contain a hosts for which the proxy should not be used, with each host separated from the others by a vertical bar "|". Each host name can contain a "*", which serves as a wildcard.

The "Save Preferences" button should be clicked to save a change in preferences so that these values will be loaded when gethdrs is restarted. The "Cancel Changes" button causes preferences to revert the last saved values or the initial value when gethdrs was started. Regardless of whether the current preferences have been saved, the values displayed will be in effect whenever the application gets HTTP headers from a web site. The Export Prefs menu option will store the current preferences in a file so these can be used with the -P option (when used with a GUI, this options suppresses the preferences panel).

Menus

The gethdrs application has 4 menus.

A File menu provides four options:

Save
Copy the currently displayed headers and content to a file.
Print to Stdout
Copy the currently displayed headers to the programs standard output. This is useful primarily if the output is redirected to a file or piped to another program.
Print
Print the currently displayed headers.
Export Prefs
Copy the current preference settings to a Java property file so these can be reloaded via a command-line option as described below. The preferences exported will be ones in text fields for which the corresponding checkbox has been selected (i.e., checked.)
The View menu contains there options:

Request Data
Display options for formulating a request.
Output
Display the panel that will show HTTP headers and content.
Preferences
Display the current set of preferences for editing.
The Go menu provides four options:

Page Up
Move the Output display up by the current window size.
Page Down
Move the Output display down by the current window size.
Top
Move the Output display to the beginning of the output.
Bottom
Move the Output display to the end of the output.
These options have keyboard equivalents: PageUp, PageDown, Control-PageUp, and Control-PageDown respectively. The Home and End keys are not used to move to the top and bottom of the output due to these being used by a text field using the standard Java look and feel.
The Help menu displays this manual. It has one item named Manual.

Running from a Terminal

When started with no arguments, the GUI is displayed. Otherwise the syntax is

gethdrs [-p ] [-P PROPFILE ] \
     [-u] [-m get|head|post|put] [-n N|all] \
     [-c CHARSET] [-t MEDIATYPE] \
     [-h HEADERFILE] [-i INPUTFILE] URL
gethdrs -h

The URL argument must be a URL for a supported protocol. Most of the options (there are two exceptions) take a mandatory argument. This argument must be a single token as seen by a shell, and will in some cases have to be quoted (the main case is the -t option, as a media type may include characters interpreted by a shell.) The options are as follows:

The -c option specifies the charset used by a text input file (this may be different than the one given in the request's media type.)
The -h option provides an input file containing headers. the name of the header should appear first on each line, followed by a colon (:), and then the value. Continuation lines are not supported. Note that certain headers (Content-Length) are provided automatically by the underlying java implementation. Other headers can be overridden (Accept) and some should not be provided at all (Transfer-Encoding). If the only command-line argument is -h with no value, a short help message is printed.
The -i option provides an input file for use with the POST or PUT methods. Character conversion is provided for text files (as determined by the media type). Other files are used unmodified.
The -m option provides the method used to send a request. Supported values are GET, HEAD, POST, and PUT.
The -n option provides the number of characters of the content to display, assuming the content is printable as text. The default is 0. The keyword all is equivalent to the maximum integer that Java will support.
The -r option indicates that headers should not be printed with the exception of lines indicating an HTTP redirect, and is not applicable when the graphical user interface is used.
The -t option provides a media type for use with the GET and PUT methods. A text media type should contain a charset parameter (e.g., text/plain; charset=US-ASCII) so that the request will be sent using the correct charset.
The -P option allows a file PROPFILE to be read. This file is assumed to contain a collection of Java properties that will be used to configure the application, and will turn on proxy support when this has been configured. The -P option is provided so that one may easily switch between choices of proxies, and can also set other options as well, such as those used to provide a keystore or truststore for HTTPS. Both the -p and the -P option cannot be used as the same time. Unlike other command-line options, the -P can be used when starting the GUI. In that case, the preferences panels and menu items will be suppressed.
The -p option causes the default preferences to be used. Both the -p and the -P option cannot be used as the same time.
The -u option indicates that any text file provided should be URL encoded. The names for a control appear at the start of a line, followed by one or more lines containing values. The lines containing values must start with a tab, which will be ignored.

When run with command-line arguments, output appears on standard output. The main use of the command-line options is for use of the program in a script.

Exported Preference Syntax

An exported preference file is a Java property file that contains lines with keyword-value pairs, separated by either a colon or an equal sign. blank lines and lines starting with "#" or "!" are ignored. Long lines may be continued by placing a backslash "\" at the end. The following properties are recognized:

http.ProxyHost - the host name or IP address (in "dot" notation of an HTTP proxy.
http.ProxyPort - the TCP port number for an HTTP proxy
http.nonProxyHosts - a "|" separated list of hosts or IP addresses of hosts that should not go through an HTTP or an HTTPS proxy. A "*" can be used as a wildcard in qualified domain names.
https.ProxyHost - the host name or IP address (in "dot" notation of an HTTPS (HTTP over SSL) proxy.
https.ProxyPort - the TCP port number for an HTTPS proxy.
ftp.ProxyHost - the host name or IP address (in "dot" notation of an FTP proxy.
ftp.ProxyPort - the TCP port number for an FTP proxy.
ftp.nonProxyHosts - a "|" separated list of hosts or IP addresses of hosts that should not go through an FTP proxy. A "*" can be used as a wildcard in qualified domain names.
socksProxyHost - the host name or IP address of a SOCKS proxy.
socksProxyPort - the port used by a SOCKS proxy.
http.agent The user agent that gethdrs advertises in HTTP requests. Some servers may change their responses based on this field, although many do not. The user agent advertised in HTTP requests may include some additional text indicating which version of Java is being used. To precisely mimic the behavior of some other software, one should explicitly set a User-Agent header as described in the Request section.

These are the same system properties as those defined in the Java class libraries for networking. Additional java options can be used as well and particularly useful ones are:

java.net.socks.username. This property provides a username for a SOCkSv5 server that requires user authentication
java.net.socks.password. This property provides the password to use for a SOCKS server that provides authentication
java.net.useSystemProxies. This indicates whether system proxies should be used. The value must be either "true" or "false". The default is "false".
javax.net.ssl.keyStore. This property provides the file name for the keystore
javax.net.ssl.keyStoreType. This property provides the type of the keystore. The default normally suffices (keytool should automatically provide the default type).
javax.net.ssl.keyStorePassword. This property provides the password for the keystore. A requirement when using this property is that the 'store' password and 'key' password are identical.
javax.net.ssl.trustStore. This property provides the file name for the truststore.
javax.net.ssl.keyStoreType. This property provides the type of the truststore. The default normally suffices (keytool should automatically provide the default type).
javax.net.ssl.trustStorePassword. This property provides the password for the truststore.

For example, the following program implements a trivial web server using HTTPS, coded using the libbzdev-java package:


     import org.bzdev.ejws.*;
     import org.bzdev.ejws.maps.*;

     public class STest {
         public static void main(String argv[]) throws Exception {
            EmbeddedWebServer ews =
              new EmbeddedWebServer(8080, 48, 10,
                                    new
                                    EmbeddedWebServer.SSLSetup());
            ews.add("/", DirWebMap.class, new File("./", null, 
                    true, true, true);
            ews.start();
         }
     }

The corresponding gethdrs property files (assume it is named PropFile) is


    javax.net.ssl.trustStore = /usr/share/bzdev/ejwsCerts.jks
    javax.net.ssl.trustStorePassword = changeit

which can be used with the command


     % gethdrs -P PropFile

Then gethdrs will have the certificate needed to connect to the server at <https://localhost:8080/>.