All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id

java.lang.Object
   |
   +----EDU.gatech.cc.is.learning.i_ReinforcementLearner_id

public abstract class i_ReinforcementLearner_id
extends Object
implements Cloneable, Serializable
Reinforcement learning class with discrete inputs and outputs. The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value). No committment is made to model-free or model-based learning, the class may be extended to either.

Copyright (c)1997 Georgia Tech Research Corporation

Version:
$Revision: 1.2 $
Author:
Tucker Balch (tucker@cc.gatech.edu)

Variable Index

 o logging
Indicates whether logging is turned on or not.
 o numactions
The number of actions to select from.
 o numstates
The number of situations or states the object may be in.
 o policyfilename
The name of the file to save or read the policy in/from.

Constructor Index

 o i_ReinforcementLearner_id(int, int)
Instantiates a reinforcement learner using default parameters.

Method Index

 o endTrial(double, double)
Called when the current trial ends.
 o getAvgReward()
return statistical info about the learner.
 o getPolicyChanges()
return statistical info about the learner.
 o getQueries()
return statistical info about the learner.
 o initTrial(int)
Called to initialize for a new trial and get the first action.
 o log(String)
Print something to the logfile.
 o loggingOff()
Turn logging off.
 o loggingOn()
Turn logging on, but without opening a file.
 o loggingOn(String)
Turn on logging for data gathering.
 o query(int, double)
Select an output based on the state and reward.
 o readPolicy()
Read the policy from a file.
 o savePolicy()
Write the policy to a file.
 o setPolicyFileName(String)
Set the filename for policy reading and writing.
 o toString()
Generate a String that describes the learner.

Variables

 o logging
 protected boolean logging
Indicates whether logging is turned on or not.

 o numstates
 protected int numstates
The number of situations or states the object may be in.

 o numactions
 protected int numactions
The number of actions to select from.

 o policyfilename
 protected String policyfilename
The name of the file to save or read the policy in/from.

Constructors

 o i_ReinforcementLearner_id
 public i_ReinforcementLearner_id(int numstatesin,
                                  int numactionsin)
Instantiates a reinforcement learner using default parameters. Individual implementations may allow additional parameters to be adjusted using accessor methods.

Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.

Methods

 o toString
 public String toString()
Generate a String that describes the learner.

Returns:
a String describing the learner.
Overrides:
toString in class Object
 o query
 public abstract int query(int statein,
                           double rewardin)
Select an output based on the state and reward.

Parameters:
statein - int, the current state.
rewardin - double, reward for the last output, positive numbers are "good."
 o endTrial
 public abstract void endTrial(double V,
                               double reward)
Called when the current trial ends. This is used for tasks that have "absorbing states."

Parameters:
V - double, the value of the absorbing state. Usually a large positive number if the final state is desirable, or negative otherwise.
reward - double, the single-step reward for the last state/action.
 o initTrial
 public abstract int initTrial(int statein)
Called to initialize for a new trial and get the first action.

Parameters:
statein - int, the current state.
 o getAvgReward
 public double getAvgReward()
return statistical info about the learner. Logging must be on for this to work.

Returns:
the average reward per query for the trial.
 o getPolicyChanges
 public int getPolicyChanges()
return statistical info about the learner. Logging must be on for this to work.

Returns:
the number of policy changes for the trial.
 o getQueries
 public int getQueries()
return statistical info about the learner. Logging must be on for this to work.

Returns:
the number of queries for the trial.
 o setPolicyFileName
 public void setPolicyFileName(String filename)
Set the filename for policy reading and writing.

Parameters:
filename - String, the directory and filename to use.
 o loggingOn
 public void loggingOn(String filename)
Turn on logging for data gathering. It is up to the extended classes to actually do the writing to the file.

Parameters:
filename - String, the directory and filename to use.
 o log
 public void log(String out)
Print something to the logfile. Called by the classes that extend this one.

Parameters:
out - String, the string to print.
 o loggingOff
 public void loggingOff()
Turn logging off.

 o loggingOn
 public void loggingOn()
Turn logging on, but without opening a file. This indicates we should keep track of some items, but not save them to a file.

 o readPolicy
 public abstract void readPolicy() throws IOException
Read the policy from a file. Use setPolicyFileName() to determine where the policy is saved.

Parameters:
filename - String, the name of the file to read from.
 o savePolicy
 public abstract void savePolicy() throws IOException
Write the policy to a file. Use setPolicyFileName() to determine where the policy is read from.

Parameters:
filename - String, the name of the file to write to.

All Packages  Class Hierarchy  This Package  Previous  Next  Index