Training is getting stuck halfway.

Hi Community,

I have a Battery system where agent should do the trading based on the price of electricty at an hour. The profit is to be maximised. the price data for 8760 hours is loaded as a .mat file. The environment is created successfully but while training using MATLAB RL designer app, using PPO algorithm , the trainig is getting stuck after few episodes throwing the following error.

Error in executing callback registered with ViewModel: Error using rl.internal.app.tool.TrainingSession/qeStopTimer Operands to the logical AND (&&) and OR (||) operators must be convertible to logical scalar values. Use the ANY or ALL functions to reduce operands to logical scalar values.

Please guide on the same.

The environment is successfully being created with the following details

ans =

bessim with properties:

BatteryLevel: 4000

power: 0

EnvName: 'EnergyTrading'

EnergyPricedata: [8760×1 double]

Capacity: 8000

Pmax: 2000

hoursperyear: 8760

current_hour: 1

Ts: 1

State: [3×1 double]

The code is as follows;

classdef bessim < rl.env.MATLABEnvironment
    %BESSIM: Template for defining custom environment in MATLAB.    
    
    %% Properties (set properties' attributes accordingly)
   
    properties
        % Specify and initialize environment's necessary properties    
        
        BatteryLevel     % Battery level (kWh)
        power        % Grid power (kW)
        EnvName= 'EnergyTrading';
        EnergyPricedata
        Capacity = 8000; 
        Pmax = 2000 ; 
        hoursperyear = 8760 ;
        current_hour=1;
        Ts=1;
    end
    
    properties
        State = zeros(3,1)     ;   
    end
    
    properties(Access = protected)
        IsDone = false    ;    
    end
     
    %% Necessary Methods
    methods              
        % Constructor method creates an instance of the environment
        function this = bessim(~)
    % Initialize Observation settings
    ObservationInfo = rlNumericSpec([3 1]);
    ObservationInfo.Name = 'BatteryLevel';
    ObservationInfo.LowerLimit = [0; -2000; 0];
    ObservationInfo.UpperLimit = [8000; 2000; inf];

   
    % Initialize Action settings   
    ActionInfo = rlNumericSpec([1 1]);
    ActionInfo.Name = 'Charging Power';
    ActionInfo.LowerLimit = 0;
    ActionInfo.UpperLimit = 2000;

    % Call the parent constructor to create the environment
    this = [e-mail](ObservationInfo, ActionInfo);
    
    % Initialize property values and load energy price data
    this.BatteryLevel = this.Capacity/2;
    this.power = 0;
     
    data1 = load('Energypriceprofile.mat');
    this.EnergyPricedata = data1.Energypriceprofile; % Assuming the variable in the mat file is 'energypriceprofile'
    this.current_hour = 1;
end

        % Apply system dynamics and simulates the environment with the 
        % given action for one step.
        function [Observation,Reward,IsDone,LoggedSignals] = step(this,Action)
            LoggedSignals = [];
            
            % Get action
            ChargingPower = Action;          
            DischargingPower = -Action;
            
            % Get current cost data
            CurrentPrice=this.EnergyPricedata(this.current_hour);
      
            NetProfit = DischargingPower * CurrentPrice - ChargingPower * CurrentPrice;
            % Calculate power balance
            
            
            % Update battery level
             if NetProfit >= 0
                   % Charge the battery
                   ChargingTime = min((this.Capacity - this.BatteryLevel) / ChargingPower, this.Ts);
                    this.BatteryLevel = this.BatteryLevel + ChargingPower * ChargingTime;
            
             else
                      % Discharge the battery
                       DischargingTime = min(this.BatteryLevel / abs(DischargingPower), this.Ts);
                       this.BatteryLevel = this.BatteryLevel + DischargingPower * DischargingTime;
             end
       
            
            % Calculate cost and revenue
           
            % Update grid power
            this.power = ChargingPower - DischargingPower;
            
            % Set observation
            Observation = [this.BatteryLevel; this.power; CurrentPrice];
            
            % Check terminal condition (none in this example)
            IsDone = false;
             % Increment current hour
             this.current_hour = this.current_hour + 1;
            % Set reward
            Reward = NetProfit;
        end

        function InitialObservation = reset(this)
             this.BatteryLevel = this.Capacity/2;
            
             this.State = [this.BatteryLevel] ;
             InitialObservation = [this.BatteryLevel; this.power; this.EnergyPricedata(this.current_hour)];
            
            % (optional) use notifyEnvUpdated to signal that the 
            % environment has been updated (e.g. to update visualization)
            notifyEnvUpdated(this);
        end
    end
   

Answer 1

[Emmanouil Tzorakoleftherakis]

Hi,

The error message seems to be longer than what you pasted. It appears there is an indexing error in the step method. Did not check in details, but the following line seems like a good place to start

CurrentPrice=this.EnergyPricedata(this.current_hour);