FileLogger
Description
Use a FileLogger
object to log data to MAT-files, within the
train function or inside a custom training loop. To log data when using the train function,
specify appropriate callback functions in FileLogger
, as shown in the
examples. These callbacks are executed at different stages of training, for example,
EpisodeFinishedFcn
is executed after the completion of an episode. The
output of a callback function is a structure containing the data to log at that stage of
training.
Note
Using a FileLogger
object to log data when using the train
function does
not affect (and is not affected by) any option to save agents during training specified
within a training option object.
Note
FileLogger
is a handle object. If you assign an existing
FileLogger
object to a new FileLogger
object, both the
new object and the original one refer to the same underlying object in memory. To preserve
the original object parameters for later use, save the object to a MAT-file. For more
information about handle objects, see Handle Object Behavior.
Creation
Create a FileLogger
object using rlDataLogger
without any input arguments.
Properties
LoggingOptions
— Object containing a set of logging options
MATFileLoggingOptions
object (default)
Object containing a set of logging options, returned as a
MATFileLoggingOptions
object. A
MATFileLoggingOptions
object has the following properties, which you
can access using dot notation after creating the FileLogger
object.
LoggingDirectory
— Name of the logging directory
"logs"
subdirectory of the current
directory (default) | string or character vector
Name or fully qualified path of the logging directory, specified as a string
or a character vector. This is the name of the directory where the MAT-files
containing the logged data are saved. As a default, a subdirectory called
logs
is created in the current folder during setup and files
are saved there during training.
Example: LoggingDirectory=mylogs
FileNameRule
— Rule to name the MAT-files
"loggedData<id>"
(default) | string
| char
vector
Rule to name the MAT-files, specified as a string or a character vector. For
example, the naming rule "episode<id>"
results in the file
names episode001.mat
, episode002.mat
and so
on.
Example: FileNameRule="ThirdRun<id>"
Version
— MAT-files version
"-v7"
(default) | string
| char
vector
MAT-file version, specified as a string or character vector. The default is "-v7". For more information, see MAT-File Versions.
Example: Version="-v7.3"
UseCompression
— Option to use compression
true
(default) | false
Option to use compression when saving data to a MAT-file, specified as a
logical variable. The default is true
. For more information,
see MAT-File Versions.
Example: UseCompression=false
DataWriteFrequency
— Disk data write period
1
(default) | positive integer
Disk data write period, specified as a positive integer. It is the number of
episodes after which data is saved to disk. For example, if
DataWriteFrequency
is 5
then data from
episodes 1 to 5 will be cached in memory and be written to disk at the end of the
5-th episode. This improves performance in some cases. The default is
1
.
Example: DataWriteFrequency=10
MaxEpisodes
— Maximum number of episodes
500
(default) | positive integer
This property is read-only.
Maximum number of episodes, specified as a positive integer. When using train
, the
value is automatically initialized. Set this value when using the logger object in a
custom training loop. The default is 500
.
Example: MaxEpisodes=1000
EpisodeFinishedFcn
— Callback to log data after episode completion
[]
(default) | function handle
This property is read-only.
Callback to log data after episode completion, specified as a function handle object. The specified function is automatically called by the training loop at the end of each episode, and must return a structure containing the data to log, such as experiences, simulation information, or initial observations.
Your function must have the following signature.
function dataToLog = myEpisodeFinishedFcn(data)
Here, data
is a structure that contains the following fields:
EpisodeCount
— current episode numberEnvironment
— environment objectAgent
— agent objectExperience
— structure array containing the experiences. Each element of this array corresponds to a step and is a structure containing the fieldsNextObservation
,Observation
,Action
,Reward
andIsDone
.EpisodeInfo
— structure containing the fieldsCumulativeReward
,StepsTaken
andInitialObservation
.SimulationInfo
— contains simulation information from the episode. For MATLAB environments this is a structure with the fieldSimulationError
, and for Simulink® environments it is aSimulink.SimulationOutput
object.
The function output dataToLog
is the structure containing the data
to be logged to disk.
Example: EpisodeFinishedFcn=@myEpLoggingFcn
AgentStepFinishedFcn
— Callback to log data after training step completion
[]
(default) | function handle | cell array of function handles
This property is read-only.
Callback to log data after training step completion within an episode, specified as a function handle object. The specified function is automatically called by the training loop at the end of each training step, and must return a structure containing the data to log, such as for example the state of the agent's exploration policy.
Your function must have the following signature.
function dataToLog = myAgentStepFinishedFcn(data)
Here, data
is a structure that contains the following fields:
EpisodeCount
— current episode numberAgentStepCount
— cumulative number of steps taken by the agentSimulationTime
— current simulation time in the environmentAgent
— agent object
The function output dataToLog
is the structure containing the data
to be logged to disk.
For multi agent training, AgentStepFinishedFcn
can be a cell
array of function handles with as many elements as number of agent groups.
Note
Logging data using the AgentStepFinishedFcn
callback is not
supported when training agents in parallel with the train function.
Example: AgentStepFinishedFcn=@myAgtStepLoggingFcn
AgentLearnFinishedFcn
— Callback to log data after completion of the learn subroutine
[]
(default) | function handle | cell array of function handles
This property is read-only.
Callback to log data after completion of the learn subroutine, specified as a function handle object. The specified function is automatically called by the training loop at the end of each learning subroutine, and must return a structure containing the data to log, such as the training losses of the actor and critic networks, or, for a model-based agent, the environment model training losses.
Your function must have the following signature.
function dataToLog = myAgentLearnFinishedFcn(data)
Here, data
is a structure that contains the following fields:
EpisodeCount
— current episode numberAgentStepCount
— cumulative number of steps taken by the agentAgentLearnCount
— cumulative number of learning steps taken by the agentEnvModelTrainingInfo
— structure containing model-based agents related fieldsTransitionFcnLoss
,RewardFcnLoss
andIsDoneFcnLoss
.Agent
— agent objectActorLoss
— value of the actor lossCriticLoss
— value of the critic lossActorGradientStepCount
— cumulative number of gradient calculations for the actor (if the agent has an actor)CriticGradientStepCount
— cumulative number of gradient calculations for the critic (if the agent has a critic)
Depending on the agent, the data
structure also contains the
following fields:
TDTarget
— temporal-difference target value (for DQN, DDPG, TD3, SAC, PPO and TRPO agents)TDError
— temporal-difference target error (for DQN, DDPG, TD3, SAC, PPO and TRPO agents)SampleIndex
— indices of the minibatch experiences sampled for the current gradient step (for DQN, DDPG, TD3, and SAC agents)MaskIndex
— sequence padding mask (for DQN, DDPG, TD3, and SAC agents that use RNNs)Advantage
— advantage value (for PPO and TRPO agents)PolicyRatio
— policy ratio value (for PPO agents)AdvantageLoss
— advantage loss value (for PPO agents)EntropyLoss
— entropy loss value (for PPO agents)
The function output dataToLog
is the structure containing the data
to be logged to disk.
For multi agent training, AgentLearnFinishedFcn
can be a cell
array of function handles with as many elements as number of agent groups.
Example: AgentLearnFinishedFcn=@myAgtLearnLoggingFcn
Object Functions
Examples
Log Data to Disk During Built-in Training
This example shows how to log data to disk when using train
.
Create a FileLogger
object using rlDataLogger
.
logger = rlDataLogger();
Specify a directory to save logged data.
logger.LoggingOptions.LoggingDirectory = "myDataLog";
Create callback functions to log the data (for this example, see the helper function section), and specify the appropriate callback functions in the logger object.
logger.EpisodeFinishedFcn = @myEpisodeFinishedFcn; logger.AgentStepFinishedFcn = @myAgentStepFinishedFcn; logger.AgentLearnFinishedFcn = @myAgentLearnFinishedFcn;
To train the agent, you can now call train
, passing logger
as an argument such as in the following command.
trainResult = train(agent, env, trainOpts, Logger=logger);
While the training progresses, data will be logged to the specified directory, according to the rule specified in the FileNameRule
property of logger.LoggingOptions
.
logger.LoggingOptions.FileNameRule
ans = "loggedData<id>"
Example Logging Functions
Episode completion logging function. This function is automatically called by the training loop at the end of each episode, and must return a structure containing the episode-related data to log, such as experiences, simulation information, or initial observations.
function dataToLog = myEpisodeFinishedFcn(data) dataToLog.Experience = data.Experience; end
Agent step completion logging function. This function is automatically called by the training loop at the end of each training step, and must return a structure containing the step-related data to log, such as for example the state of the agent exploration policy.
function dataToLog = myAgentStepFinishedFcn(data) dataToLog.State = getState(getExplorationPolicy(data.Agent)); end
Learn subroutine completion logging function. This function is automatically called by the training loop at the end of each learning subroutine, and must return a structure containing the learning-related data to log, such as the training losses of the actor and critic networks, or, for a model-based agent, the environment model training losses.
function dataToLog = myAgentLearnFinishedFcn(data) dataToLog.ActorLoss = data.ActorLoss; dataToLog.CriticLoss = data.CriticLoss; end
For the specific function signatures and more information on the function input structure, see the corresponding properties of FileLogger
. For a related example, see Log Training Data to Disk.
Log Data to Disk in a Custom Training Loop
This example shows how to log data to disk when training an agent using a custom training loop.
Create a FileLogger
object using rlDataLogger
.
flgr = rlDataLogger();
Set up the logger object. This operation initializes the object performing setup tasks such as, for example, creating the directory to save the data files.
setup(flgr);
Within a custom training loop, you can now store data to the logger object memory and write data to file.
For this example, store random numbers to the file logger object, grouping them in the variables Context1
and Context2
. When you issue a write command, a MAT-file corresponding to an iteration and containing both variables is saved with the name specified in flgr.LoggingOptions.FileNameRule
, in the folder specified by flgr.LoggingOptions.LoggingDirectory
.
for iter = 1:10 % Store three random numbers in memory % as elements of the variable "Context1" for ct = 1:3 store(flgr, "Context1", rand, iter); end % Store a random number in memory % as the variable "Context2" store(flgr, "Context2", rand, iter); % Write data to file every 4 iterations if mod(iter,4)==0 write(flgr); end end
Clean up the logger object. This operation performs clean up tasks like for example writing to file any data still in memory.
cleanup(flgr);
Limitations
Logging data using the
AgentStepFinishedFcn
callback is not supported when training agents in parallel with the train function.
Version History
Introduced in R2022b
MATLAB コマンド
次の MATLAB コマンドに対応するリンクがクリックされました。
コマンドを MATLAB コマンド ウィンドウに入力して実行してください。Web ブラウザーは MATLAB コマンドをサポートしていません。
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)