Examples¶

The commands below use the files in the repository's example/ directory. Run them from the repository root after installing the package.

The complete example inputs are also available directly in the documentation:

Basic selection¶

Select five instances using the CSV files:

setselector --runtimes example/times.csv --features example/features.csv --n 5 --cutoff 300

For progress and filtering information, increase the log level:

setselector --runtimes example/times.csv --features example/features.csv --n 5 --cutoff 300 --log info

Keep filtered instances¶

The default filters remove instances where all solvers time out and instances that are easy for every solver. Keep either category when it is important to your benchmark:

setselector --runtimes example/times.csv --features example/features.csv --n 5 --cutoff 300 --keep-too-hard --keep-too-easy

Change the sampling strategy¶

Use the minimum solver runtime as hardness and sample from a uniform distribution:

setselector --runtimes example/times.csv --features example/features.csv --n 5 --cutoff 300 --aggregate min --dist uni

The other supported distributions are exp (exponential) and log (log-normal). The ind aggregation treats each solver runtime separately.

Training and test split¶

Remove a deterministic random half of the input instances before selection:

setselector --runtimes example/times.csv --features example/features.csv --n 5 --cutoff 300 --split

Evaluation XML¶

Evaluation files generated by potassco-benchmark-tool can be read directly:

setselector --eval eval.xml --n 5 --cutoff 300

Evaluation features can be selected with --eval-features by providing a comma-separated list. The default features are rules_s,bodies_s.