Skip to content

Instantly share code, notes, and snippets.

@d6y

d6y/instance-list.txt

Last active November 30, 2018 12:29
Show Gist options
  • Save d6y/850639b0aa0990df374e80b4c0170048 to your computer and use it in GitHub Desktop.
Save d6y/850639b0aa0990df374e80b4c0170048 to your computer and use it in GitHub Desktop.
Download ZIP
CGP irace tuning
Raw
instance-list.txt
## This is an example of specifying instances with a file.
# Each line is an instance relative to trainInstancesDir
# (see scenario.txt.tmpl) and an optional sequence of instance-specific
# parameters that will be passed to target-runnerx when invoked on that
# instance.
# Empty lines and comments are ignored.
experiments/atari.jl --id frostbite
experiments/atari.jl --id ms_pacman
experiments/atari.jl --id space_invaders
experiments/atari.jl --id pong
Raw
irace.log
*******************************************************************************
* irace: An implementation in R of Iterated Race
* Version: 3.1.2112M
* Copyright (C) 2010-2018
* Manuel Lopez-Ibanez <[email protected]>
* Jeremie Dubois-Lacoste
* Leslie Perez Caceres <[email protected]>
*
* This is free software, and you are welcome to redistribute it under certain
* conditions. See the GNU General Public License for details. There is NO
* WARRANTY; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
*
* irace builds upon previous code from the race package:
*
* race: Racing methods for the selection of the best
* Copyright (C) 2003 Mauro Birattari
*******************************************************************************
# installed at: /home/richard/R/x86_64-pc-linux-gnu-library/3.3/irace
# called with: --parallel 16
Warning: A default scenario file './scenario.txt' has been found and will be read
# 2018-11-20 08:52:04 UTC: Initialization
# Elitist race
# Elitist new instances: 1
# Elitist limit: 2
# nbIterations: 4
# minNbSurvival: 4
# nbParameters: 4
# seed: 1198575126
# confidence level: 0.95
# budget: 1000
# mu: 10
# deterministic: FALSE
# 2018-11-20 08:52:04 UTC: Iteration 1 of 4
# experimentsUsedSoFar: 0
# remainingBudget: 1000
# currentBudget: 250
# nbConfigurations: 22
Markers:
x No test is performed.
- The test is performed and some configurations are discarded.
= The test is performed but no configuration is discarded.
! The test is performed and configurations could be discarded but elite configurations are preserved.
. All alive configurations are elite and nothing is discarded
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| | Instance| Alive| Best| Mean best| Exp so far| W time| rho|KenW| Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x| 1| 22| 17| -225.0000000| 22|04:43:19| NA| NA| NA|
|x| 2| 22| 17| -917.5000000| 44|05:04:24|+0.34|0.67|0.6305|
|x| 3| 22| 17| -651.6666667| 66|05:19:50|+0.19|0.46|0.7706|
|x| 4| 22| 17| -488.0000000| 88|05:05:33|+0.11|0.33|0.8494|
|=| 5| 22| 17| -427.4000000| 110|04:47:41|+0.07|0.26|0.9024|
|=| 6| 22| 19| -227.5000000| 132|05:11:57|+0.04|0.20|0.9395|
|=| 7| 22| 12| -426.8571429| 154|04:57:32|+0.05|0.18|0.9333|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration: 12 mean value: -426.8571429
Description of the best-so-far configuration:
.ID. ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda .PARENT.
12 12 oneplus CGPChromo :gene_mutate true true 1 0.4 0.1 5 NA
# 2018-11-21 20:02:24 UTC: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
19 oneplus CGPChromo :gene_mutate true true 0.8 0.3 0.5 6
7 oneplus CGPChromo :gene_mutate true true 0.2 0.8 0.1 9
4 oneplus CGPChromo :gene_mutate true true 0.2 0.9 0.7 2
# 2018-11-21 20:02:24 UTC: Iteration 2 of 4
# experimentsUsedSoFar: 154
# remainingBudget: 846
# currentBudget: 282
# nbConfigurations: 25
Markers:
x No test is performed.
- The test is performed and some configurations are discarded.
= The test is performed but no configuration is discarded.
! The test is performed and configurations could be discarded but elite configurations are preserved.
. All alive configurations are elite and nothing is discarded
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| | Instance| Alive| Best| Mean best| Exp so far| W time| rho|KenW| Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x| 8| 25| 40| -160.0000000| 25|06:18:00| NA| NA| NA|
|x| 2| 25| 40| -885.0000000| 46|04:55:35|+0.10|0.55|0.8505|
|x| 7| 25| 7| -1116.666667| 67|05:05:03|+0.07|0.38|0.8769|
|x| 1| 25| 24| -717.5000000| 88|04:36:59|+0.06|0.30|0.8931|
|=| 6| 25| 24| -573.2000000| 109|05:10:36|+0.07|0.26|0.8828|
|!| 5| 17| 26| -510.1666667| 130|05:07:37|-0.04|0.13|0.9753|
|=| 4| 17| 26| -436.7142857| 143|03:05:10|-0.02|0.12|0.9615|
|=| 3| 17| 24| -407.7500000| 156|03:17:32|-0.01|0.12|0.9431|
|=| 9| 17| 24| -386.3333333| 173|03:54:33|-0.01|0.10|0.9518|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration: 24 mean value: -386.3333333
Description of the best-so-far configuration:
.ID. ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda .PARENT.
24 24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2 12
# 2018-11-23 13:33:33 UTC: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2
29 oneplus CGPChromo :gene_mutate true true 1.0 0.1 0.3 9
26 oneplus CGPChromo :gene_mutate true true 1.0 0.5 0.4 6
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
# 2018-11-23 13:33:33 UTC: Iteration 3 of 4
# experimentsUsedSoFar: 327
# remainingBudget: 673
# currentBudget: 336
# nbConfigurations: 28
Markers:
x No test is performed.
- The test is performed and some configurations are discarded.
= The test is performed but no configuration is discarded.
! The test is performed and configurations could be discarded but elite configurations are preserved.
. All alive configurations are elite and nothing is discarded
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| | Instance| Alive| Best| Mean best| Exp so far| W time| rho|KenW| Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x| 10| 28| 12| -1650.000000| 28|06:43:14| NA| NA| NA|
|x| 7| 28| 12| -1245.000000| 52|05:48:43|+0.20|0.60|0.9076|
|x| 9| 28| 12| -918.3333333| 76|05:29:09|+0.16|0.44|0.9069|
|x| 3| 28| 12| -718.7500000| 100|05:27:30|-0.04|0.22|0.9785|
|=| 4| 28| 67| -330.8000000| 124|05:23:13|-0.05|0.16|1.0381|
|=| 8| 28| 67| -299.0000000| 148|05:42:06|-0.02|0.15|0.9860|
|=| 5| 28| 67| -285.5714286| 172|05:04:27|+0.01|0.15|0.9765|
|=| 6| 28| 67| -250.0000000| 196|05:38:57|+0.04|0.16|0.9425|
|=| 2| 28| 67| -271.1111111| 220|05:29:37|+0.01|0.12|0.9680|
|=| 1| 28| 24| -397.7000000| 244|05:17:20|+0.04|0.13|0.9528|
|=| 11| 28| 24| -361.3636364| 272|06:15:25|+0.04|0.13|0.9425|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration: 24 mean value: -361.3636364
Description of the best-so-far configuration:
.ID. ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda .PARENT.
24 24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2 12
# 2018-11-26 03:53:21 UTC: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
29 oneplus CGPChromo :gene_mutate true true 1.0 0.1 0.3 9
67 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 3
# 2018-11-26 03:53:21 UTC: Iteration 4 of 4
# experimentsUsedSoFar: 599
# remainingBudget: 401
# currentBudget: 401
# nbConfigurations: 31
Markers:
x No test is performed.
- The test is performed and some configurations are discarded.
= The test is performed but no configuration is discarded.
! The test is performed and configurations could be discarded but elite configurations are preserved.
. All alive configurations are elite and nothing is discarded
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| | Instance| Alive| Best| Mean best| Exp so far| W time| rho|KenW| Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x| 12| 31| 76| -160.0000000| 31|07:22:58| NA| NA| NA|
|x| 11| 31| 94| -74.00000000| 58|06:11:17|-0.21|0.39|1.1764|
|x| 6| 31| 91| -40.66666667| 85|06:42:18|-0.17|0.22|1.0816|
|x| 2| 31| 91| -433.0000000| 112|06:02:31|-0.04|0.22|0.9936|
|=| 1| 31| 88| -384.6000000| 139|05:27:29|-0.09|0.13|1.0245|
|=| 9| 31| 94| -194.0000000| 166|05:25:46|-0.03|0.14|0.9824|
|=| 5| 31| 94| -197.0000000| 193|05:27:17|-0.00|0.14|0.9681|
|=| 7| 31| 29| -495.1250000| 220|05:39:44|-0.00|0.12|0.9662|
|=| 8| 31| 94| -229.8888889| 247|06:42:14|+0.01|0.12|0.9635|
|=| 3| 31| 24| -355.9000000| 274|06:04:54|+0.01|0.11|0.9635|
|=| 4| 31| 24| -323.1818182| 301|06:09:09|+0.01|0.10|0.9617|
|=| 10| 31| 92| -489.9166667| 328|05:51:04|+0.02|0.11|0.9511|
|=| 13| 31| 90| -385.6153846| 359|06:23:55|+0.03|0.10|0.9484|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration: 90 mean value: -385.6153846
Description of the best-so-far configuration:
.ID. ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda .PARENT.
90 90 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 1 24
# 2018-11-29 11:24:04 UTC: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
90 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 1
92 oneplus CGPChromo :gene_mutate true true 0.6 0.3 0.3 8
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2
# 2018-11-29 11:24:04 UTC: Iteration 5 of 5
# experimentsUsedSoFar: 958
# remainingBudget: 42
# currentBudget: 42
# nbConfigurations: 6
Markers:
x No test is performed.
- The test is performed and some configurations are discarded.
= The test is performed but no configuration is discarded.
! The test is performed and configurations could be discarded but elite configurations are preserved.
. All alive configurations are elite and nothing is discarded
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| | Instance| Alive| Best| Mean best| Exp so far| W time| rho|KenW| Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x| 14| 6| 24| -850.0000000| 6|01:58:00| NA| NA| NA|
|x| 10| 6| 90| -1055.000000| 8|01:31:06|-0.43|0.28|1.3540|
|x| 13| 6| 12| -790.0000000| 10|01:24:41|+0.11|0.41|0.9101|
|x| 6| 6| 12| -592.7500000| 12|01:15:18|+0.06|0.29|0.8506|
|=| 3| 6| 12| -498.2000000| 14|02:13:20|+0.03|0.23|0.8586|
|=| 9| 6| 12| -459.3333333| 16|01:17:06|+0.17|0.31|0.7400|
|=| 4| 6| 90| -385.2857143| 18|01:27:51|+0.12|0.24|0.7871|
|=| 2| 6| 12| -544.7500000| 20|01:33:28|+0.06|0.18|0.7736|
|=| 1| 6| 24| -432.4444444| 22|01:25:39|+0.06|0.17|0.7957|
|=| 5| 6| 90| -365.7000000| 24|01:22:38|+0.09|0.18|0.7630|
|=| 11| 6| 24| -371.8181818| 26|01:27:31|+0.10|0.18|0.7351|
|=| 12| 6| 90| -316.0833333| 28|01:23:52|+0.11|0.18|0.7325|
|=| 8| 6| 24| -331.5384615| 30|01:34:41|+0.09|0.16|0.7527|
|=| 7| 6| 90| -391.6428571| 32|01:36:03|+0.07|0.14|0.7814|
|=| 15| 6| 90| -365.4000000| 38|02:01:57|+0.05|0.12|0.8017|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration: 90 mean value: -365.4000000
Description of the best-so-far configuration:
.ID. ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda .PARENT.
90 90 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 1 24
# 2018-11-30 10:57:22 UTC: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
90 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 1
24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2
92 oneplus CGPChromo :gene_mutate true true 0.6 0.3 0.3 8
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
# 2018-11-30 10:57:22 UTC: Stopped because there is not enough budget left to race more than the minimum (4)
# You may either increase the budget or set 'minNbSurvival' to a lower value
# Iteration: 6
# nbIterations: 6
# experimentsUsedSoFar: 996
# timeUsed: 0
# remainingBudget: 4
# currentBudget: 4
# number of elites: 4
# nbConfigurations: 4
# Best configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
ea chromosome mutate_method active_mutate weights recurrency output_mutation_rate node_mutation_rate lambda
90 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 1
24 oneplus CGPChromo :gene_mutate true true 0.7 0.3 0.2 2
92 oneplus CGPChromo :gene_mutate true true 0.6 0.3 0.3 8
12 oneplus CGPChromo :gene_mutate true true 1.0 0.4 0.1 5
# Best configurations as commandlines (first number is the configuration ID; same order as above):
90 --ea oneplus --chromosome CGPChromo --mutate_method :gene_mutate --active_mutate true --weights true --recurrency 0.7 --output_mutation_rate 0.3 --node_mutation_rate 0.2 --lambda 1
24 --ea oneplus --chromosome CGPChromo --mutate_method :gene_mutate --active_mutate true --weights true --recurrency 0.7 --output_mutation_rate 0.3 --node_mutation_rate 0.2 --lambda 2
92 --ea oneplus --chromosome CGPChromo --mutate_method :gene_mutate --active_mutate true --weights true --recurrency 0.6 --output_mutation_rate 0.3 --node_mutation_rate 0.3 --lambda 8
12 --ea oneplus --chromosome CGPChromo --mutate_method :gene_mutate --active_mutate true --weights true --recurrency 1 --output_mutation_rate 0.4 --node_mutation_rate 0.1 --lambda 5
Raw
parameters.txt
## Template for parameter description file for Iterated Race.
##
## The format is one parameter per line. Each line contains:
##
## 1: Name of the parameter. An unquoted alphanumeric string,
## example: ants
## 2: Switch to pass the parameter. A quoted (possibly empty) string,
## if the value and the switch must be separated, add a space at
## the end of the string. Example : "--version1 --ants "
## 3: Type. An unquoted single letter, among
## i: Integer, c: categorical, o: ordinal, r: real.
## 4: For c and o: All possible values, that is, a variable number of
## quoted or unquoted strings separated by commas within
## parenthesis. Empty strings and strings containing commas or
## spaces must be quoted.
## For i,r: a pair of unquoted numbers representing minimum and
## maximum values.
## 5: A conditional parameter can be defined according to the values of
## one or several other parameters. This is done by adding a
## character '|' followed by an R expression involving the names of
## other parameters. This expression must return TRUE if the
## condition is satisfied, FALSE otherwise.
# 1: 2: 3: 4: 5:
#param1 "--param1 " i (1, 10) | mode %in% c("x1", "x2")
#param2 "--param2 " i (1, 10) | mode %in% c("x1", "x3") && real > 2.5 && real <= 3.5
#mode "--" c ("x1" ,"x2", "x3")
#real "--paramreal=" r (1.5, 4.5)
#mutation "--mutation=" o ("none", "very low", "low", "medium", "high", "very high", "all")
##unused "-u " c (1, 2, 10, 20)
ea "--ea " c (oneplus)
chromosome "--chromosome " c (CGPChromo)
mutate_method "--mutate_method " c (:gene_mutate)
########## global parameters ##########
active_mutate "--active_mutate " c (true)
weights "--weights " c (true)
recurrency "--recurrency " r (0.0, 1.0)
output_mutation_rate "--output_mutation_rate " r (0.1, 1.0)
node_mutation_rate "--node_mutation_rate " r (0.1, 1.0)
########## oneplus parameters ##########
lambda "--lambda " i (1, 10)
Raw
scenario.txt
###################################################### -*- mode: r -*- #####
## Scenario setup for Iterated Race (irace).
############################################################################
## To use the default value of a parameter of iRace, simply do not set
## the parameter (comment it out in this file, and do not give any
## value on the command line).
## File that contains the description of the parameters of the target
## algorithm.
parameterFile = "./parameters.txt"
## Directory where the programs will be run.
execDir = "~/CGP.jl"
## File to save tuning results as an R dataset, either absolute path or
## relative to execDir.
logFile = "./irace.Rdata"
## Previously saved log file to recover the execution of irace, either
## absolute path or relative to the current directory. If empty or NULL,
## recovery is not performed.
# recoveryFile = ""
## Directory where training instances are located; either absolute path or
## relative to current directory. If no trainInstancesFiles is provided,
## all the files in trainInstancesDir will be listed as instances.
trainInstancesDir = ""
## File that contains a list of training instances and optionally
## additional parameters for them. If trainInstancesDir is provided, irace
## will search for the files in this folder.
trainInstancesFile = "instances-list.txt"
## File that contains a set of initial configurations. If empty or NULL,
## all initial configurations are randomly generated.
# configurationsFile = ""
## File that contains a list of logical expressions that cannot be TRUE
## for any evaluated configuration. If empty or NULL, do not use forbidden
## expressions.
# forbiddenFile = ""
## Script called for each configuration that executes the target algorithm
## to be tuned. See templates.
targetRunner = "./target-runner"
## Number of times to retry a call to targetRunner if the call failed.
# targetRunnerRetries = 0
## Optional data passed to targetRunner. This is ignored by the default
## targetRunner function, but it may be used by custom targetRunner
## functions to pass persistent data around.
# targetRunnerData = ""
## Optional R function to provide custom parallelization of targetRunner.
# targetRunnerParallel = ""
## Optional script or R function that provides a numeric value for each
## configuration. See templates/target-evaluator.tmpl
# targetEvaluator = ""
## Maximum number of runs (invocations of targetRunner) that will be
## performed. It determines the maximum budget of experiments for the
## tuning.
maxExperiments = 1000
## Maximum total execution time in seconds for the executions of
## targetRunner. targetRunner must return two values: cost and time.
# maxTime = 0
## Fraction (smaller than 1) of the budget used to estimate the mean
## computation time of a configuration. Only used when maxTime > 0
# budgetEstimation = 0.02
## Maximum number of decimal places that are significant for numerical
## (real) parameters.
digits = 1
## Debug level of the output of irace. Set this to 0 to silence all debug
## messages. Higher values provide more verbose debug messages.
# debugLevel = 0
## Number of iterations.
# nbIterations = 0
## Number of runs of the target algorithm per iteration.
# nbExperimentsPerIteration = 0
## Randomly sample the training instances or use them in the order given.
# sampleInstances = 1
## Statistical test used for elimination. Default test is always F-test
## unless capping is enabled, in which case the default test is t-test.
## Valid values are: F-test (Friedman test), t-test (pairwise t-tests with
## no correction), t-test-bonferroni (t-test with Bonferroni's correction
## for multiple comparisons), t-test-holm (t-test with Holm's correction
## for multiple comparisons).
# testType = "F-test"
## Number of instances evaluated before the first elimination test. It
## must be a multiple of eachTest.
# firstTest = 5
## Number of instances evaluated between elimination tests.
# eachTest = 1
## Minimum number of configurations needed to continue the execution of
## each race (iteration).
# minNbSurvival = 0
## Number of configurations to be sampled and evaluated at each iteration.
# nbConfigurations = 0
## Parameter used to define the number of configurations sampled and
## evaluated at each iteration.
mu = 10
## Confidence level for the elimination test.
# confidence = 0.95
## If the target algorithm is deterministic, configurations will be
## evaluated only once per instance.
# deterministic = 0
## Seed of the random number generator (by default, generate a random
## seed).
# seed = NA
## Number of calls to targetRunner to execute in parallel. Values 0 or 1
## mean no parallelization.
# parallel = 0
## Enable/disable load-balancing when executing experiments in parallel.
## Load-balancing makes better use of computing resources, but increases
## communication overhead. If this overhead is large, disabling
## load-balancing may be faster.
# loadBalancing = 1
## Enable/disable MPI. Use Rmpi to execute targetRunner in parallel
## (parameter parallel is the number of slaves).
# mpi = 0
## Specify how irace waits for jobs to finish when targetRunner submits
## jobs to a batch cluster: sge, pbs, torque or slurm. targetRunner must
## submit jobs to the cluster using, for example, qsub.
# batchmode = 0
## Enable/disable the soft restart strategy that avoids premature
## convergence of the probabilistic model.
# softRestart = 1
## Soft restart threshold value for numerical parameters. If NA, NULL or
## "", it is computed as 10^-digits.
# softRestartThreshold = ""
## Directory where testing instances are located, either absolute or
## relative to current directory.
# testInstancesDir = ""
## File containing a list of test instances and optionally additional
## parameters for them.
# testInstancesFile = ""
## Number of elite configurations returned by irace that will be tested if
## test instances are provided.
# testNbElites = 1
## Enable/disable testing the elite configurations found at each
## iteration.
# testIterationElites = 0
## Enable/disable elitist irace.
# elitist = 1
## Number of instances added to the execution list before previous
## instances in elitist irace.
# elitistNewInstances = 1
## In elitist irace, maximum number per race of elimination tests that do
## not eliminate a configuration. Use 0 for no limit.
# elitistLimit = 2
## User-defined R function that takes a configuration generated by irace
## and repairs it.
# repairConfiguration = ""
## Enable the use of adaptive capping, a technique designed for minimizing
## the computation time of configurations. This is only available when
## elitist is active.
# capping = 0
## Measure used to obtain the execution bound from the performance of the
## elite configurations: median, mean, worst, best.
# cappingType = "median"
## Method to calculate the mean performance of elite configurations:
## candidate or instance.
# boundType = "candidate"
## Maximum execution bound for targetRunner. It must be specified when
## capping is enabled.
# boundMax = 0
## Precision used for calculating the execution time. It must be specified
## when capping is enabled.
# boundDigits = 0
## Penalization constant for timed out executions (executions that reach
## boundMax execution time).
# boundPar = 1
## Replace the configuration cost of bounded executions with boundMax.
# boundAsTimeout = 1
## Percentage of the configuration budget used to perform a postselection
## race of the best configurations of each iteration after the execution
## of irace.
# postselection = 0
## END of scenario file
############################################################################
Raw
target-runner
#!/bin/bash
###############################################################################
# irace tuning script for CGP.jl
#
# PARAMETERS:
# $1 is the ID of the candidate to be evaluated
# $2 is the instance ID
# $3 is the seed
# $4 is the instance name
# The rest ($* after `shift 4') are parameters for running ACOTSP
#
# RETURN VALUE:
# This script should print a single numerical value (the value to be minimized).
###############################################################################
EXE=julia
export JULIA_LOAD_PATH=src
CONFIG_ID="$1"
INSTANCE_ID="$2"
SEED="$3"
INSTANCE="$4"
# All other parameters are the candidate parameters to be passed to program
shift 4 || error "Not enough parameters to $0"
CONFIG_PARAMS=$*
LOG_DIR=irace_logs
mkdir -p ${LOG_DIR}
STDOUT=${LOG_DIR}/c${CONFIG_ID}-${INSTANCE_ID}.stdout
STDERR=${LOG_DIR}/c${CONFIG_ID}-${INSTANCE_ID}.stderr
error() {
echo "`TZ=UTC date`: error: $@" >&2
exit 1
}
# Control number of frames in each game. Set low (200?) for irace --check
MAX_FRAMES=1000
# Number of evaluations to perform on a given tournament. Set low (500?) for irace --check
TOTAL_EVALS=900
LOG_FILE=${LOG_DIR}/${INSTANCE_ID}_${CONFIG_ID}.log
$EXE $INSTANCE --frames ${MAX_FRAMES} --total_evals ${TOTAL_EVALS} --seed $SEED --log ${LOG_FILE} ${CONFIG_PARAMS} 1> $STDOUT 2> $STDERR
COST=$(tail -n 1 ${LOG_FILE} | cut -d 'E' -f 2)
if ! [[ "$COST" =~ ^[-+0-9.e]+$ ]] ; then
error "${STDOUT}: Output is not a number"
fi
echo "$COST"
# We are done with our duty. Clean files and exit with 0 (no error).
#rm -f "${STDOUT}" "${STDERR}"
#rm -f best.* stat.* cmp.*
exit 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment