All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class EDU.gatech.cc.is.learning.i_SLearner_id

java.lang.Object
   |
   +----EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
           |
           +----EDU.gatech.cc.is.learning.i_SLearner_id

public class i_SLearner_id
extends i_ReinforcementLearner_id
implements Cloneable, Serializable
An object that learns to select from several actions based on a reward. Uses the S-learning method as defined by Mataric. "S" is for "single-step."

The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value).

Copyright (c)1997 Georgia Tech Research Corporation

Version:
$Revision: 1.1 $
Author:
Tucker Balch (tucker@cc.gatech.edu)

Constructor Index

 o i_SLearner_id(int, int)
Instantiate an S learner using default parameters.
 o i_SLearner_id(int, int, int)
Instantiate an S learner.

Method Index

 o endTrial(double, double)
Called when the current trial ends.
 o getAvgReward()
Report the average reward per step in the trial.
 o getPolicyChanges()
Report the number of policy changes in the trial.
 o getQueries()
Report the number of queries in the trial.
 o initTrial(int)
Called to initialize for a new trial.
 o query(int, double)
Select an output based on the state and reward.
 o readPolicy()
Read the policy from a file.
 o savePolicy()
Write the policy to a file.
 o saveProfile(String)
Write the policy profile to a file.

Constructors

 o i_SLearner_id
 public i_SLearner_id(int numstatesin,
                      int numactionsin,
                      int criteriain)
Instantiate an S learner.

Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.
criteria - int, IGNORED, but retained for compatibility.
 o i_SLearner_id
 public i_SLearner_id(int numstatesin,
                      int numactionsin)
Instantiate an S learner using default parameters.

Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.

Methods

 o query
 public int query(int yn,
                  double rn)
Select an output based on the state and reward.

Parameters:
statein - int, the current state.
rewardin - double, reward for the last output, positive numbers are "good."
Overrides:
query in class i_ReinforcementLearner_id
 o endTrial
 public void endTrial(double Vn,
                      double rn)
Called when the current trial ends.

Parameters:
Vn - double, the value of the absorbing state.
reward - double, the reward for the last output.
Overrides:
endTrial in class i_ReinforcementLearner_id
 o initTrial
 public int initTrial(int s)
Called to initialize for a new trial.

Overrides:
initTrial in class i_ReinforcementLearner_id
 o getAvgReward
 public double getAvgReward()
Report the average reward per step in the trial.

Returns:
the average.
Overrides:
getAvgReward in class i_ReinforcementLearner_id
 o getQueries
 public int getQueries()
Report the number of queries in the trial.

Returns:
the total.
Overrides:
getQueries in class i_ReinforcementLearner_id
 o getPolicyChanges
 public int getPolicyChanges()
Report the number of policy changes in the trial.

Returns:
the total.
Overrides:
getPolicyChanges in class i_ReinforcementLearner_id
 o saveProfile
 public void saveProfile(String profile_filename) throws IOException
Write the policy profile to a file.

Parameters:
filename - String, the name of the file to write to.
 o readPolicy
 public void readPolicy()
Read the policy from a file.

Overrides:
readPolicy in class i_ReinforcementLearner_id
 o savePolicy
 public void savePolicy()
Write the policy to a file.

Overrides:
savePolicy in class i_ReinforcementLearner_id

All Packages  Class Hierarchy  This Package  Previous  Next  Index