HAL, the High-performance Algorithm Laboratory, is a versatile and extensible, free environment for empirical algorithmics.

HAL supports the empirical analysis and design tasks encountered during the development, evaluation and application of high-performance algorithms for challenging computational problems. HAL has been designed to facilitate the easy and correct use of a broad range of standardised, advanced empirical analysis and design methods; to design and perform computational experiments, including large-scale analysis and design tasks involving substantial amounts of computation on potentially large clusters of machines; and to support the development and critical assessment of novel empirical analysis and design techniques.

HAL ...

• supports working with instances of and algorithms for arbitrary problems
• can run experiments on local or remote machines
• allows easy distributed execution of experiments on compute clusters
• includes plugins for commonly encountered performance analysis tasks and several state-of-the-art automated algorithm congurators
• offers detailed information about and control over computational experiments before, during and after execution
• archives and organizes algorithms, benchmark instance sets and experimental results in a centralized database
• produces statistical data and plots for inclusion in publications
• is controlled from any web browser through an intuitive, task-based interface
• provides a versatile API and powerful infrastructure for developping and deploying new meta-algorithmic analysis and design procedures.

For more details, refer to our publications, or look at some screenshots of HAL in action.

The initial release of HAL 1.0 is now available, and we encourage you to try it out. As we are actively working to extend and imporve HAL, we appreciate any and all feedback you can provide.

Readme: Download a very brief readme/quick-start guide here.

License: HAL 1.0 and all plugins linked here are released under the GPL 3.0 license. Source codes are available upon request; we are planning to make a public Git repository available soon. Other licensing options are available on request.

Requirements: HAL requires version 6.0 or higher of the Java Runtime Environment (JRE). In this version, rendering of plots additionally requires gnuplot be installed and accessible from your environment path variable. If you plan to run more than one experiment simultaneously, we highly recommend configuring using the MySQL database plugin to replace the default SQLite database; since SQLite uses file-based locking, it is prone to deadlocks when multiple processes access the same database file. To do this, run HAL, then edit the hal.json file and replace the "database" entry with a line like "jdbc:mysql://[username]@[mysql.server]:[port]/[database_name]". On restart, HAL will set up the new database schema automatically, so long as the provided URI points to a working MySQL server with appropriate access permissions.

HAL 1.0 package:

Current version: (1.0.0, 14MB, 2011/01/19). MD5: 75eefa0a9dd83a06755f0f7c3c03967f. Download here
Includes HAL 1.0.0 core, plus all plugins listed below.

To use HAL 1.0, unzip to a working directory, and double-click the hal_1.0.0.jar to start the webserver (or type "java -jar hal_1.0.0.jar"). Then, browse to the UI.

Plugins (put in "plugins" subfolder to use)

SCD-Based Analysis - analysis procedure (single-algorithm analysis)
Current version: (1.0.0, 2011/01/19). MD5: c78efbf52705b0cf8804938dd47771a0. Download here.

Comprehensive Pairwise Comparison - analysis procedure (pairwise comparison)
Current version: (1.0.0, 2011/01/19). MD5: 501bdab4baea97500f0845753c4df058. Download here.

ParamILS - design procedure (algorithm configuration)
Current version: (1.0.0, 2011/01/19). MD5: 63c9866fafbf51e0067d9115799a5515. Download here.
Embedded ParamILS 2.3.5 requires a working Ruby installation, with the Ruby interpreter in your environment's path variable.

ROAR - design procedure (algorithm configuration)
Current version: (1.0.0, 2011/01/19). MD5: e0276a319393ddf8600164c9512eebae. Download here.

GGA - design procedure (algorithm configuration)
Current version: (1.0.0, 2011/01/19). MD5: cd293971acaf6493cddc068096083a3f.
A binary of GGA is embedded which requires Linux. Also requires a working Ruby installation, with the Ruby interpreter in your environment's path variable.
As we are not licensed to distribute GGA, please contact its authors if you are interested in obtaining this plugin.

MySQL - data manager
Adds support for MySQL database servers.
Current version: (1.0.0, 2011/01/19). MD5: f9ac402dd936a1ff22b7f0b9a37619e0. Download here.

R - statistics package
Enables use of an existing R installation to perform statistical tests.
Current version: (1.0.0, 2011/01/19). MD5: 0fa12b880400ddb0321896bb7512162b. Download here.

Experiment packages simple examples:
To use, don't unzip, just use the HAL 1.0 import interface.

Sample configuration experiment
MD5 checksum: 72504ac6e15f29a6290f348ea92e2a9e (15MB, 2011/01/19). Download here.

This sample configuration experiment is based on the ones from Nell et al. (2011) below, but modified to be smaller/faster for use as an example. It contains a Spear 1.2.1 binary for Linux with a subset of (6) parameters exposed, and a subset of the SWV training set including only instances corresponding to verification of gzip (37 total). The experiment runs ROAR for 30 minutes, using PAR1 as metric with a captime of 30s per target run. We suggest running it, and then using the SCD-Based Analysis to analyze the results. Then, try setting up an analogous configuration experiment using ParamILS, and compare its design with the one from ROAR using Comprehensive Pairwise Analysis. For reference, the Spear parameters exposed in the sample are:

• sp-var-dec-heur - Variable decision heuristic.
  
• sp-learned-clause-sort-heur - Heuristic for sorting learned clauses.
  
• sp-orig-clause-sort-heur - Heuristic for sorting original clauses.
  All have categorical domains:
  0. Prefer more active variables.
  1. Prefer less active variables.
  2. If equally active, pick more frequent.
  3. If equally active, pick less frequent.
  4. If equally active, pick more asymmetric variable.
  5. If equally active, pick less asymmetric variable.
  6. If equally active, pick the one with larger product of occurrences.
  7. If equally active, pick the one with smaller product of occurrences.
  8. Pick variables with maximal number of occurrences.
  9. Pick variables with minimal number of occurrences.
  10. Pick variables with maximal product of occurrences.
  11. Pick variables with minimal product of occurrences.
  12. Pick more asymmetric variables.
  13. Pick less asymmetric variables.
  14. Prefer variables assigned at larger decision level.
  15. Prefer variables assigned at smaller decision level.
  16. Prefer smaller variables (numerical comparison).
  17. Prefer larger variables (numerical comparison).
  18. Prefer implied literals with shorter reasons.
  19. Take variables as they were created (fixed order).
• sp-phase-dec-heur: Phase decision heuristic. Categorical domain:
  
  0. Always pick false phase.
  1. Always pick true phase.
  2. Pick phase at random.
  3. Pick more frequent phase.
  4. Pick less frequent phase.
  5. Pick phase with more watched literals.
  6. Pick phase with less watched literals.
• sp-rand-var-dec-freq - Random variable decision frequency. Real domain, range [0.000001, 0.05], log-scaled.
• sp-first-restart - Number of conflicts to the first restart. Integer domain, range [25, 3200], log-scaled.

Experiment packages from Nell et al. (2011):
To use, don't unzip, just use the HAL 1.0 import interface.

Spear: SWV training set experiments
MD5 checksum: 2580b3b6664508e47046eb49f64d668c (259MB, 2011/01/19). Download here.
Requires HAL 1.0 core revision 1.0.0 or greater. Configuration experiments require respective plugins. Validation experiments require SCD-Based Analysis plugin. SWV training instance set is included. Spear binary for Linux included.
Note that the validation experiments correspond to the designs obtained on our compute cluster; the designs produced by running the packaged configuration experiments on other machines will differ. Also note that GGA configuration experiments are not included as we cannot distribute the required plugin.

Spear: SWV test set experiments
MD5 checksum: f7410229b50e45a1d6f183c8525db39c (238MB, 2011/01/19). Download here.
Requires HAL 1.0 core revision 1.0.0 or greater. Analysis experiment requires SCD-Based Analysis plugin. Comparison experiments require Comprehensive Pairwise Comparison plugin. SWV test instance set is included. Spear binary for Linux included.

CPLEX vs Gurobi experiments
Unable to distribute due to licensing restrictions for CPLEX 12.1 and Gurobi 3.0.1; please contact us for assistance in replicating these results.

HAL has been designed to be easy to use. To help with cases where it's not, we're currently working on a complete user manual for HAL 1.0; keep an eye on this spot! Until then, we suggest you read the paper to help you get started.

HAL 1.0 is under active development and testing. Chances are, if you've found a bug, we're working on fixing it. That said, please let us know about any problems, ideas, or feature requests you might have, and thanks for helping us to build a better HAL!