Loading

NeurIPS 2022: CityLearn Challenge

CityLearn-2-EvolveCity: A Swarm and Evolutionary Approach

Getting Started with Genetic Algorithm and Particle Swarm Optimization for CityLearn

Chemago

This Notebook shows how to use evolutionary algorithms in the CityLearn reinforcement learning environment to optimize a rulebased agent for Energy Management in Buildings.

NatureCity: Swarm and Evolutionary Algorithms for CityLearn (Action24)

Author: Chia E Tungom
Email: bamtungom@protonmail.com
Date: Aug-02-2022

This Notebook demostrates how we can use a metaheuristic particularly a Genetic Algorithm or Particle Swarm Optimization In the CityLearn Environment. To tackle the CityLearn Problem, it Lays a foundation for those aiming to explore an Evolutionary or Swarm optimization approach for Energy Management.

To achieve this I do the following:

  1. Formulate an objective function (single objective problem)
  2. Utilize Opensource implementations of GA and PSO (these Libraries contain other algorithms)

Note: The objective function is very very expensive and will take time to run one evaluation (e.g 10 particle running in 10 generations take approximately 10 minutes). This is because Swarm and evolutionary algorithms are population based approaches and evaluate the performaance of a function (one run of the environment) at evvery generation. More so, in one generation, there are given number of candidate solution corresponding to the warm population. Our implementation evaluate the environment after an entire run. A simpler approach could be to evolve the solution after every step (we dont implement this approach)

If you are still starting out with the CityLearn environment or RL, check this Notebook for an understanding of the environment.

Some parts of the Notebook are taken from here to define our environment.

To run this notebook successfully, you will need to download or clone the citylearn starter kit and place this notebook in the the root directory

Note that the objective function can be formulated as biobjective function The libraries used for implementation are pymoo and jmetalpy

Leave a question if you need clarification and/or give a like or bitcoin me :( if you find this useful

Lets Goooooo!!!

1. Build Environment

In [1]:
# import dependencies 

import numpy as np
import time

from agents.orderenforcingwrapper import OrderEnforcingAgent
from citylearn.citylearn import CityLearnEnv

from citylearn.agents.sac import SAC
from agents import rbc_agent
/Users/chemago/opt/anaconda3/envs/CityLearn2022/lib/python3.7/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
In [2]:
# Custom configure enviroment 
class Constants:
    episodes = 3
    schema_path = './data/citylearn_challenge_2022_phase_1/schema.json'

def action_space_to_dict(aspace):
    """ Only for box space """
    return { "high": aspace.high,
             "low": aspace.low,
             "shape": aspace.shape,
             "dtype": str(aspace.dtype)
    }

def env_reset(env):
    observations = env.reset()
    action_space = env.action_space
    observation_space = env.observation_space
    building_info = env.get_building_information()
    building_info = list(building_info.values())
    action_space_dicts = [action_space_to_dict(asp) for asp in action_space]
    observation_space_dicts = [action_space_to_dict(osp) for osp in observation_space]
    obs_dict = {"action_space": action_space_dicts,
                "observation_space": observation_space_dicts,
                "building_info": building_info,
                "observation": observations }
    return obs_dict
In [3]:
# CityLearn Environment

env = CityLearnEnv(schema=Constants.schema_path)

2. Define Objective Function [24 Hours Energy Control]

We model the problem as a 24 hour charge or discharge of electricity from the solar PV (time of day like night, morning, afternoon can also be used). This means that at every hour of the day, we want to determine how much charge or discharge of electric energy we want from the solar PV. Therefore our decision variable is a 24 length 1D array with upper and lower dimension of 1 and -1 respectively (see CityLearn action space).

There are 24 hours in a day so our intuition is that at every hour, we can decide to charge or discharge our battery which correspond to an action in the environment.

Our decison variables are [a, b, ... , x] where a is the action we take at hour 1 and x the action we take at hour 24.

In the Objective function (onbFunction defined in function) you can define

  • Episodes: 1 is recommended for this case
  • Days: Number of Days to run the simulation. We set 30 default corresponding to a months
    • The number of hours in a day is 24 and our current setting searches for the same control for all days therefore for faster running, we can set the Days
      • to 1 or
      • 7 to generalize for a week
  • RandnDay: True/False, We can also randomly use any day to e.g set day to one pick a random day in a year for trainiing or 7 days are randomly pick 7 days

Positive values stand for Charge while Neagive values stand for discharge.

A = [0.1 for _ in range(20)] TakeAction24(env.observation_space[0].sample() , A) env.observation_space[0].sample() #[0.1 for _ in range(20)]
In [4]:
def TakeAction24(Observation, Actions24):
    """ takes an observation and Actions for 24 hours and returns 
    an action for the given hour
    ======================
    Parameters
        Observation: observation for one building
        Actions: 1d numpy array of size 24
        
    """
    hour = Observation[2] # Hour index is 2 for all observations

    action = 0.0 # Default value
    # for time, act in enumerate(range(actions)):

    #     # time plus one because hour starts from 0
    #     if hour == time + 1:
    action = Actions24[hour-1]
    
    action = np.array([action], dtype=float)

    return action

def objFunction(Actions24, Env, Episodes = 1, Days  =  30, RandomDay = True):
    """ takes a 24 hour action array and returns it's evaluation performance 
    ======================
    Parameters
        Action24: 1d numpy array of size 24
        Env:  CitiLearn Environment 
        RandomDay: bool
            when training we to randomly pick a day (This should be convient for training evolutionary algorithms with Days = 1 in our current setting )
        """

    episodes = Episodes
    # Skipping to shorten training time
    training_steps = 24*Days

    for episode in range(1, episodes+1):
        states = Env.reset()
        done = False
        score = 0
        moves = 0
    
        skipping = True
        print(" STARTING EPISODE ", episode)

        while (not done) and skipping:
            #env.render()

            try:
                if RandomDay == True:
                    # Start Observation on some random day (NOT IMPLEMENTED)
                    action = [TakeAction24(Env.observation_space[building].sample(), Actions24) for building in range(len(states))]
                    states, reward, done, info = Env.step(action)
                    score+=reward
                    moves += 1 
                else:
                    # Take an action based on Observation
                    action = [TakeAction24(Observation, Actions24) for Observation in states]
                    states, reward, done, info = Env.step(action)
                    score+=reward
                    moves += 1
            except:
                print("UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON")
                states = Env.reset()
                continue

            # print(done, skipping)
            if moves >= training_steps:
                # print(" TERMINATING AT SET STEP ", episode)
                skipping = False

        # print('Episode:{} Score:{} Out of:{}'.format(episode, score, moves))
        print('Episode ::::: {} Evaluation :::: {}'.format(episode, Env.evaluate()))
        Env.close()

    return sum(env.evaluate())/2


# Acts24 = [random.uniform(-1,1) for _ in range(24)]
# About 6seconds for one evaluation
# objFunction(Acts24, env, Episodes=1, Days=5)

3. Optimization Using pyoo (A library for Multi-objective Optimization in Python)

To optimize using this library, we need to define our objective function as recommended by the developers. We start by adpating our objective function.

The documentation was a not very clear to me and I resorted to looking into the source code on github. This was especially useful for looing into the final results (the documentation and examples don't show muc)

In [5]:
from pymoo.core.problem import Problem

class ProBlem(Problem):

    def __init__(self, Env, Episodes = 1, Days = 10, RandomDay = False):
        # self.Actions24 = Actions24
        self.Env = Env
        self.Episodes = Episodes
        self.Days = Days
        self.RandomDay = RandomDay

        super().__init__(n_var=24, n_obj=1, n_ieq_constr=0, xl=-1.0, xu=1.0)

    def _evaluate(self, x, out, *args, **kwargs):
        # print(x[0])
        # print(len(x))
        out["F"] = [objFunction(x[i], self.Env, self.Episodes, self.Days, self.RandomDay) for i in range(len(x))]
        # out["G"] = 0.1 - out["F"]

problem  = ProBlem(env, Days = 30, Episodes=1, RandomDay=False)

# Problem
# problem??

4. Genetic Algorithm

We expeimented on two variants of GA but NSGA2 was more promissing. Its also worth noting that NSGA2 was designed for Multi-Objective problems but works well for us here. If you run the algorithm Long enough, you will get results close to the current top of 0.67ish. An overnight run will suffice

Parameters for NSGA2 are

  • population size: 100
  • number of generation: 200 (the longer the better. u can also add a stopping criteria)

For details See: Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. A. M. T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2), 182-197.

Note: I ran into crashes coming from the CityLearn environment indicating that storage capacity cannot be less than zero (couldnt get why). I added a try/except line in my objective function to do away with this error by resetting the environmet. I don't fully understand the consequences (hit me up if you find out)

If your results we run in model( name for your optimization model), get the final results after running the algorithm you can run

  • model.pop[index].X : index is one member in the final population and X is it's genetic composition AKA solution
  • model.pop[index].F : F give the function value
In [6]:
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.problems import get_problem
from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter
from pymoo.algorithms.soo.nonconvex.ga import GA
from pymoo.optimize import minimize
In [7]:
# Using GA
# res = minimize(problem, GA(pop_size=100, verbose=True), termination=("n_gen", 10), seed=1)
# print('Threads:', res.exec_time)

# pop 10 and 10 gnerations completes in 9 mins
In [8]:
# Using NSGA2

algorithm = NSGA2(pop_size=100, verbose=True)
res = minimize(problem, algorithm, termination = ('n_gen', 200), seed=1, verbose=False)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.524832745891357, 1.7068377757963087)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.1319795592090927, 2.179550811583186)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5460837532771707, 1.545279737156711)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4903653672787063, 1.6285465200902383)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.158467136002888, 2.258112984352848)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7339243366102517, 1.6820512302489286)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9021745130920356, 1.8156255040959057)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0064084510512896, 1.9357315403364612)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3901385182919526, 1.4418648153683211)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5262900198422966, 1.688492384736818)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5778995134531437, 1.8313250730391726)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8329868458772032, 1.6896862170201121)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2594911926802281, 1.5336222570991007)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6897194150960384, 1.7036044259630547)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2640878212356255, 1.308488814190818)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.018615125527451, 1.8227697092779285)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3099363112707212, 1.5173021403526525)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4596983051161159, 1.5619199767219003)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3929757695296643, 1.736499375802727)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.3445547658220423, 2.2651948981942622)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9103304767314204, 1.7153375477712545)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8008483298461175, 1.5936718052540892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6533817007541292, 1.6925532233666438)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7357282712154156, 1.9211334393568247)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8983665992169583, 1.956545592767675)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.588234271980603, 1.6924599264351752)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8592622251379864, 1.8551220721699861)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.250074428584765, 2.2074995527934167)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4767608314656826, 1.660367340435604)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9241888746496987, 1.7938420713726284)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8200068506400195, 1.819154171648038)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7655910731574809, 1.7638972750086765)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.741489016583519, 1.6792825740362671)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7201868322670342, 1.9009294413213358)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5226583562559548, 1.6405725711352472)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9238126644203841, 1.9081769544419038)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.774403037813021, 1.654345685802231)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9144773649943758, 1.8796180246507093)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3936049671761503, 1.416769667589732)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4943891306300878, 1.5689104439278225)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.487049459207883, 1.6269103874649877)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7358093098813903, 1.6827593691045435)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8674158540378494, 1.7140232710627008)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7343836791021399, 2.045277010825979)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.2289407077532344, 2.0281751942328095)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8682283033417055, 1.7962307449846253)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.932775071554423, 1.7213082049151645)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4604447041123538, 1.483371003548367)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.507848776949875, 1.6198043027693443)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0619112713599486, 1.8903581713845947)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9413804395955363, 1.7330953382875849)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9007180601774247, 2.1985365408407227)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0113271514822437, 1.9996493519593148)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7294723800354237, 1.6206444118194876)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5644574611621616, 1.7846263342034907)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5112688742316667, 1.6611407442960733)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9240068054075492, 1.9020094666099745)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0457673502291467, 2.0489473416943746)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3551877969626682, 1.527182052761619)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8744314828350175, 1.715323479297415)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.1822048122147337, 1.985096740226956)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6894725137082973, 1.6205170880733284)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4228393021569055, 1.5379412811892519)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8470738768836774, 1.9065097508748063)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7326411710825456, 1.7961080951210047)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9476009418734028, 1.874213227954133)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9133026258270864, 1.9376513890526896)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4442893093104265, 1.673753398197652)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7142945800489, 1.8089875182771622)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9815757465840083, 1.8881391672657635)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5975224460440494, 1.6485096974992293)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7484088363699206, 1.7451180637553423)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.617391976360167, 1.6219627366427718)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8021307125908614, 1.7738077101573002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1105821135950045, 1.1450551366427348)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8265506998080232, 1.8474575559386959)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0177732217696844, 1.8603917262710399)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.2223358272766354, 2.1365824845930805)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.535741887985529, 1.4345109205259512)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8574970927082388, 1.7755632584084688)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.161433446487556, 2.0453326963444645)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9731502149156175, 1.8378337174289499)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9037731127827875, 1.746764526208836)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8660239161070271, 1.780317648695207)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0574489171103814, 1.8416240895002294)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.957112879638004, 1.8851397100224307)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.401465710213974, 1.45277916919727)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0952948391009074, 1.9251921171512931)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8996542472905065, 1.7834138471042842)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5012799678049151, 1.4389892021271988)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5416649847058304, 1.6874538579125662)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7340635118746235, 1.6150703419416381)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5638063689623674, 1.6096257074982285)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8537211567476064, 1.6009551343686377)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9447036725685742, 1.8739701370813677)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0757026894465818, 1.8117353406140566)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7528438384597125, 1.6900858259405651)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6389929854522258, 1.7460966350292781)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4619064200217091, 1.574842185519417)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7493306670481281, 1.7410451788352852)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6665780676181625, 1.7236808342575258)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5166424114545516, 1.638868060889415)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8793157688533106, 2.0764954824976067)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.311912789767544, 1.360555854644866)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3919328986397008, 1.4762288167633175)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7397849195112878, 1.7129771216879361)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2100525256357535, 1.3480124506299747)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4645586143149925, 1.5939941444425259)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6606210527209666, 1.7279680851521848)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5162386975011508, 1.6840727340845791)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4995428242800048, 1.6114193229516809)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5655505388981692, 1.616695178836432)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4811251228259743, 1.5011428551544908)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.940187897867869, 1.9064949244220504)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5994412235429345, 1.685430809113763)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0745753356515753, 1.9171932790687114)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7955433712885331, 1.5793258588234667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8902050648519, 1.8707228000129468)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6966433885681864, 1.8084342729195382)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8822731647264723, 1.8275334229756208)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7309559668905319, 1.6847060078988703)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.551423502552752, 1.634234253949685)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.2031671682601304, 2.1110073452422022)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5112549218956683, 1.6494416311996098)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4296505333716272, 1.5177631737706752)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8947531529541557, 1.7991619494388433)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9017648563308123, 1.8181788847695481)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4309531823165655, 1.5425411795871722)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9965157368642628, 1.6497399567949769)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7406509727785509, 1.5768017945006945)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9069595373284607, 1.9434431890294612)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4921124457447215, 1.6322267397892836)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8117610191482552, 1.7578440593619242)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2986117375736337, 1.4012056219125544)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5679745987544413, 1.6923524443545102)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.139368913173612, 1.9804041164805297)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9439872969345362, 1.8747438266679246)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4549276477530502, 1.4592488968469282)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3251197733789382, 1.6464440701958833)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5031516830467135, 1.560067191507632)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.289476187138525, 1.3652748916469537)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5739937495400005, 1.6943956934744753)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3198327535670356, 1.3647161789330933)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7181136847301637, 1.6868225367372691)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.472805290581467, 1.5498019460234966)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4715928461818455, 1.5730847257993807)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3585961951823735, 1.6324466033326754)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1098840112628257, 1.0747044756233686)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.37713250688384, 1.3556845524823096)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.415232800431626, 1.5867517731383465)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.806801601880402, 1.8584948679772348)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4575822734803872, 1.602542171998173)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.758995616329487, 1.815521385230467)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.899654156579755, 1.7834607674618705)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0179126698975245, 2.0137592106146145)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.927350950487531, 1.7987297942452525)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6215161761980317, 1.6256488920464234)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.343999825048105, 1.4325361048215)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9528577448757862, 1.8855439382516)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2295726846385404, 1.2967022592826232)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6212562619462858, 1.873340728770982)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4228393021569055, 1.5379412811892519)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7197498895777381, 1.7721010404158088)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5221813694687512, 1.6073758352783751)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.53559071226137, 1.6455428707819426)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7375723498828255, 1.6640260246557392)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8848498021179922, 1.8669277676640992)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8827509950942445, 1.7700370499144398)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7853009915165623, 1.714809334025446)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7124554869441968, 1.8409783839334557)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6258083087052981, 1.657993991499398)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6167458601955533, 1.621602939398673)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8056521632052895, 1.6678244066502401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3811628113168035, 1.3353109480365573)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0608388344535191, 1.1418201060822983)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5789857027005796, 1.8317980201275796)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6117778606939255, 1.7089641234839321)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7700356807973519, 1.816437145464378)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.426260656917837, 1.6326403066833544)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7255667333703857, 1.681995282801517)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7540675321658143, 1.6077655586193633)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5995089820622368, 1.826828720164778)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3620883211344859, 1.5385212634184804)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8269645013855658, 1.8270242423594576)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4914590805429886, 1.6369320845084652)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3811980854247419, 1.301602780643796)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6970746662549727, 1.9031675154504675)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3061513268986435, 1.4194881092028087)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9711950114638837, 1.8969405544368108)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9014833087037915, 1.667356766621863)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9047012772758025, 1.716070994635603)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6452395295633175, 1.5328975559351206)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1455825874689094, 1.2235801530496815)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.352040732886632, 1.4721720065264463)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7740202614503706, 1.8201694271899704)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5811803613069213, 1.825804292676262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8138775839914314, 2.043615206907926)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6056997699507218, 1.7076390351140094)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.259371834396783, 1.533540419608704)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.203102505045077, 1.5066134004272875)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7123431633853068, 1.8497964617671037)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7526187752996942, 1.6681469559522564)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.288851682466202, 1.5424500228240137)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5162386975011508, 1.6840727340845791)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.892149505491238, 1.6464009049385735)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3450495634391197, 1.3942444121480975)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4638472899256127, 1.5971028324941223)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1581385739786294, 1.2297344896658569)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3996803439168422, 1.5599150427623565)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2828211891366759, 1.4509593704776982)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5008502889472346, 1.7325251092028302)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.210031914969043, 1.34798998149455)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1027707374330353, 1.126168123679893)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4393808497247755, 1.526854581870447)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.402286700019241, 1.3982427716779087)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.839620906243752, 1.7629575188935511)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7923252367011542, 1.8871257658342855)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.807437857299448, 1.9457943226039593)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6665119075546517, 1.6335972805375012)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2486892629206372, 1.294989020686053)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.519427277445317, 1.6497111758734195)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4041713517583505, 1.5535635081188885)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4282907847201292, 1.482285552977186)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4690907798688415, 1.5658408144793023)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5907291780752466, 1.8272209071056102)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4050554964065631, 1.6736395564176538)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4272581464160305, 1.7100551201060885)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6213918899533724, 1.7185182539612769)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7736607057503835, 1.9019870268980055)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3571203431446452, 1.5457851181190776)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5173002712083974, 1.6719423450262845)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3199266745593463, 1.4759143796058278)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7772533346450736, 1.8739882745792837)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5278944117086906, 1.6289860235033584)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2702673983530275, 1.5123996299414058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9049121529303392, 1.8774810363444456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3163921330094044, 1.4679723527807866)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5144191635231192, 1.6248628954251128)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4402893076367094, 1.6736192341439047)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4314761120298103, 1.5031702747675648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1937266086116183, 1.1909509931227862)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6778567800628144, 1.5215511826323753)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5202144987191262, 1.6191051483868113)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.651496462403018, 1.621722284810489)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.745615157468271, 1.7393946802756335)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2905483747220812, 1.3257313301935083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1140437092527848, 1.0785551811340668)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4304948453111033, 1.6271773969056629)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3969400003937693, 1.4971949577078636)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.838356147331745, 1.8930967438886923)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3953999566168747, 1.5687527433090198)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1376934045087224, 1.1180116926295185)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.1005284080617597, 1.997181857624094)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.29639856233792, 1.373124301585291)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.247520456484481, 1.3378088031585595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5561791478224385, 1.6269944657131257)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.482812394287682, 1.5857744668217766)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7739084520959607, 1.8412974893086556)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5221760533053819, 1.6611069473667013)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4099608924575429, 1.5177372199479777)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.560243198293548, 1.7556446153167855)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3390887265576643, 1.431534016758071)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.21785250546121, 1.2884932887452567)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.648814708282664, 1.7214806564609109)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4716987046932082, 1.558684885364144)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4575142035261086, 1.5963178233949329)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4111535350603162, 1.5729077849547646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2863118951973156, 1.4637145974940977)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1084542856327482, 1.2148038625582005)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.010991217507171, 1.095942164484487)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4542917389012326, 1.535415413544523)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5337923935611413, 1.6395666203031325)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4158292740659582, 1.7030637588250332)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6741119885971794, 1.523570535814482)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.237519013880535, 1.2938658586167207)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4696105289591197, 1.6102123230083756)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4221358588481687, 1.5075583763555271)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6104681820731155, 1.7177668895509353)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2810105451429452, 1.4991266316147194)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6709712106864543, 1.954144655353131)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3326419237205058, 1.4294772457498566)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5011047153766857, 1.6368204622886653)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4495290980124462, 1.6572947143032766)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5329515707265249, 1.4702169746115648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7970194134466144, 1.6308760670834144)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4379138123650799, 1.548849642784296)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3811980921590696, 1.3016027845144258)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.052304493514755, 1.91540049385631)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3838736683710589, 1.5128704994361357)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5089921008203446, 1.5614864280681466)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0954140969968673, 1.2043401944353604)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7668408536736806, 1.8112298555928361)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2271803763598492, 1.3547188808859907)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.517169981191882, 1.4105735790089293)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6098251114451367, 1.6767022857975171)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1884738613511632, 1.3610085792378488)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7089452395031761, 1.647862760004308)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1252069259440995, 1.1894403971719456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5622677697388443, 1.3947389792462568)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8980176385621086, 1.6802011623861495)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.50665662887682, 1.6388171343280493)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4063899175106607, 1.535546563876065)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5081266654731056, 1.533930221033411)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5228590310180918, 1.6347123318790062)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.761007568659238, 1.8873079461287678)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5175998589450976, 1.6831208360043355)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3811980916710835, 1.3016027842339446)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2847755191925716, 1.3201138275878535)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3136734325524193, 1.630828155204197)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1730704030868868, 1.27075368069757)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4974845443163587, 1.6725455314321744)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4089855079943725, 1.3130992195605324)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3920993673859303, 1.4465147094145707)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.245321726140645, 1.2891919806275225)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4434746407275998, 1.5891500446902627)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.354728082431312, 1.4172726792244565)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4498107348220144, 1.3990223887653386)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.164268211578435, 1.2247083012507998)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.437985663627205, 1.549181514275991)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2145326342189222, 1.2616729458383646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.139957324431731, 1.2183027765036327)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1662096031798521, 1.2852946910753318)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4464513893443738, 1.638795507436447)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4240601844262268, 1.5329517705331468)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5386505241427726, 1.6545377493149114)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3028729708166473, 1.4282989769844143)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3077586502736258, 1.391535535090644)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1085922397365109, 1.1753057784296916)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7321838833083787, 1.7710098351811077)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3391726803681505, 1.3711989882408735)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.089342318909889, 1.1544121755420806)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3373966829260653, 1.4510350229922275)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3907465300572257, 1.5048733538813557)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.265194192838656, 1.3624077903277996)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.39817723079563, 1.497938036201917)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0364455697026673, 1.9726160845887535)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4542251699579876, 1.4077632704744354)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.341081041647233, 1.4459682073740783)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2686406098369702, 1.3681049051208338)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4903861439835528, 1.4260013366478834)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2768312571371043, 1.3113538470210848)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1885144594622725, 1.4972544582600587)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0285562250067335, 1.1911068631621513)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3909155354176082, 1.449945196740468)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1983458886403635, 1.443550398035987)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4933262364308884, 1.6141442455110075)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9659172522877604, 1.2534801514954594)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1841905225068163, 1.262059195468517)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9985581175735244, 1.086014840148805)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4195791627453807, 1.6613979117848736)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5411866437212405, 1.6155837622457507)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3217175898624591, 1.4464694875874982)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7262541630085668, 1.5327581489869444)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1590770356371587, 1.2962726625064023)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3781332122460612, 1.2984896563294668)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.138077429377482, 1.1179930092605115)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5716550808952383, 1.7263685910425137)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3414035620649634, 1.4187141570887336)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.043152257226418, 1.0427555239137352)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2778799067216697, 1.3549263647467484)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3675166990698149, 1.649511978777963)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.284225566663942, 1.3481118314661917)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5835346679111917, 1.6748360054094409)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4196584040351354, 1.5700614109933764)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3370360026413934, 1.4175983829953507)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.359740994294722, 1.470150402901016)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1084542856327482, 1.2148038625582005)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3377354220802986, 1.4595589126810051)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2132630731744034, 1.208849698318002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.230208806627237, 1.375282897932246)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4371448038190944, 1.5808682771472375)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9927064501858469, 1.0862463557691435)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4464190130772527, 1.5006391197074826)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2051144202278903, 1.3842887636724441)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2340799201626143, 1.2564665655411178)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1457596905363314, 1.3751667562766416)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9856235334693674, 1.061888749951944)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2280358908717415, 1.3867205508672917)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.176935148302788, 1.140522065013941)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.420071138817855, 1.4701573140879494)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.193726607010204, 1.1909509916280543)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2596233626327975, 1.426367362999452)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4299724354726833, 1.4857996594455722)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8370500580299831, 1.8810487861578638)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2773162187866918, 1.3775038630776781)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4491410806294496, 1.522649104917256)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.61461128847435, 1.5087090799360463)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2756661365667676, 1.4417741460046019)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2358722047369683, 1.3165272727909527)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.079450546791489, 1.165071739522106)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3296205227005276, 1.3999619058985056)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3700963754567117, 1.4154198330947758)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7631828899466653, 1.6025472655357336)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5376721420060455, 1.6907457659417526)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4145407590920633, 1.5065058092162444)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4987533047613162, 1.6353425011083338)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7967497407157869, 1.7888339907902997)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3182472709918847, 1.406702162730272)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3661241216064535, 1.4276759804832857)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.172498745146777, 1.2325044287715445)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.948852417006679, 1.1540761181931842)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9649828591662741, 1.9513771914929008)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.24404688012507, 1.3260810294659322)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0713686392682165, 1.0773508560941265)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.628032852586953, 1.5780377553620155)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.216048076752868, 1.2636418918122716)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3364885047706332, 1.36588332346474)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.180266716883524, 1.3272171628890166)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1230399713561914, 1.1066000820909987)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000203084920114, 1.0000872428014385)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.186710072186473, 1.2427411032272662)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3332442682997223, 1.4954683782079874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7627882692085703, 2.0595284770311038)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3073040298009375, 1.4146487856136456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0680686076924863, 1.114245738681797)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4807334472255713, 1.5355080288197502)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.580581983110714, 1.4677299367258658)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1589763124970518, 1.3857450215448903)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1522072796589837, 1.1459030330007136)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6404015771481228, 1.687145171550228)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2374939234341749, 1.2938366715973413)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0583943623773677, 1.088023350908204)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.213364633227401, 1.2172629217388649)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4538798417097947, 1.7015048121109997)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4186194297180086, 1.5858266642456187)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6690658071237818, 1.71649283605559)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.178999150645325, 1.254948737903733)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.113250248447606, 1.0785719503477607)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2720911871385907, 1.3247335363372843)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3227091188887232, 1.306977300963902)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1017658894634688, 1.1254079404685402)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1674028471358269, 1.2633999182754088)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.399454372376918, 1.31628956989862)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5285003525131426, 1.6465273151559332)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9738074670890562, 1.0556216701591667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4012381522480792, 1.5197992929996713)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3896081010506192, 1.528751787072085)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2056865246699424, 1.277087612710514)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3289834132732052, 1.4323060220536843)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3887055638515182, 1.5075910454015173)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0577659977308231, 1.1255732166413028)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3723410872511135, 1.3865911208019703)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4425899887098166, 1.5282547757602283)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1627651017303005, 1.2168831557636353)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2099242509045198, 1.348438718349728)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5871877372927057, 1.4373274049551543)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2091021856543323, 1.2832633516848553)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0491884529157947, 1.0622972532992623)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.031610066577527, 1.0559581925252315)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4756643587828904, 1.749705749288251)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4821330963398376, 1.3996350350740767)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5728950716425238, 1.7237603824858312)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1006590724150527, 1.0594654279566802)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5517812874265509, 1.4115801820078087)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.149688123638645, 1.167327540848455)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.425783766806004, 1.3001257661649177)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9852198519912665, 1.061494892506612)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0548455332712352, 1.1351610926059148)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0130879321185886, 1.0979320338416476)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4891201383074868, 1.4510889195712324)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.087436575789147, 1.1102876054750814)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1396020372136297, 1.1643268716918327)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.13856569229442, 1.9998983042407734)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4636704629321051, 1.518740654509266)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.081702109498039, 1.212639471931836)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.037939467134611, 1.034440670582595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.985620939916065, 1.0618983409187308)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2979055258091532, 1.371785022799512)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0958344397561484, 1.0532975644060862)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0220919469089296, 1.039910921133406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1120354314964804, 1.137114499124826)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5399734920446835, 1.6401624101161603)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1378425868272688, 1.2561617653101627)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3938665469048928, 1.4692981165286911)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1360642869692357, 1.1817539901118372)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.599594502374825, 1.5655151923618216)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2842269659005363, 1.3768060743316017)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0613659108757638, 1.1422450146921632)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3986099384527149, 1.419674354166456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.08106973051699, 1.1241324065903355)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1953809818755137, 1.2594613620688602)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2710450926098082, 1.3647601644269525)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.251261899763205, 1.3082476744145388)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.321608359112897, 1.3369627112763294)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.369909989747395, 1.4604814516987168)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1746242173629335, 1.220821122148978)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.008600815109987, 1.0309102624018043)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1433287787024597, 1.2825068992436568)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.143083403109134, 1.1802285963750445)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7050950698569145, 1.9069430866110098)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1314884341936935, 1.293943053003809)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6342317899815784, 1.7559036835319526)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5191960914699518, 1.7244759594486152)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0189770771985924, 1.0419260045291616)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8645751043691636, 1.7599548802039195)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2634182781230785, 1.2921966221062404)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3623672263433897, 1.4477424814117)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0036449691654363, 1.0184174718673653)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1517838489988377, 1.4207443004073974)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0235931483469283, 1.099927889397832)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3227567734775738, 1.300243465181785)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0579412404487987, 1.0877818695819284)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.141702914104643, 1.1780335270300337)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0866935860513347, 1.1108823120380629)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9488524171396376, 1.1540761195942597)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1778686132115306, 1.1439987315136537)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1945287422127802, 1.2595417715135315)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0869569925814837, 1.1378265787425357)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.025551088649847, 1.0924723067123343)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9651795037027867, 1.0372700442319314)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.070973025544449, 1.124759336583048)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4171013710237508, 1.315146786853423)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3442879264042173, 1.4186113098743838)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8454045135957249, 2.048726908192926)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2597170732562741, 1.3480081148096839)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.056854142504352, 1.080458495663126)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0225187036321028, 1.0403827292567938)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000016632372664, 0.9999967408194587)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.249139833457908, 1.269613819976193)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941984040928144, 1.0622879314413631)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.057170310034955, 1.0838660441448804)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6879638012847726, 1.691601913446708)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6892894733558235, 1.7516391295647022)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2657302065845129, 1.3291307844797349)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0076221857696963, 1.010887529342634)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0110369916298863, 1.0069196758820942)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2473441291515839, 1.30002596500534)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.210603525082741, 1.3095124565368985)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2516717035785165, 1.2352629836673672)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0716598215416333, 1.1092074182917002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4140150221836936, 1.4795406150743378)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1383292453775182, 1.267502504561406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.245820976800964, 1.3350439215425773)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1488865338677403, 1.1439115563605924)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1000152911408843, 1.1881663583135051)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6928257147192012, 1.7237175983781792)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.491665065362328, 1.7194617819843758)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7250614703647011, 1.8773904132239871)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9738074670890562, 1.0556216701591667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1001103476112213, 1.058515869873802)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2382728518043857, 1.3478507233849057)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0230665790083904, 1.0409901582094545)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0913122886525288, 1.1150619603946401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6557620931336443, 1.8167740824918492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.256172679169253, 1.2249622128044328)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.987546331831983, 1.9365299159860083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009190613248756, 1.024548780731301)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4145607948927805, 1.3260156620457264)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1416670973827394, 1.2295336198763587)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.22857882769453, 1.319886974863151)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3087379156367287, 1.33864821963651)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.193430887257921, 1.1906712069583714)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0279205022176732, 1.0346821508794564)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1085922397365109, 1.1753057784296916)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0606348646365347, 1.0966087325272353)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0857048954595128, 1.10798758540514)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086884562049796, 1.0240627900805013)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1226367036116651, 1.0930512266684942)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0677945891200036, 1.0923838676403874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0700847088650962, 1.153953905044766)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0426999219085318, 1.1001192173042595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2174892275830156, 1.2681189078893371)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.039431301166405, 1.0768339646433518)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.3633888476663105, 2.0651857831144986)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0387199139880843, 1.0330557460869298)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1984925707981613, 1.342681136705142)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0819059530347916, 1.2130468029383499)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5105324148522856, 1.5064272748381218)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0637979994271536, 1.2334191609637015)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3804632530776413, 1.282225760122347)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7891689191436735, 1.8672681820747605)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9860594886782702, 1.0731416732663317)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1106892044113257, 1.1110776181236728)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.133555471990277, 1.1789444222991796)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0609413116904702, 1.1432974892627557)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0716494305127886, 1.2412763841457801)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0093005221465805, 1.015108770973406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0276624990816674, 1.0620643378830223)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6056317564653693, 1.6543814344764252)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3091810534828912, 1.377575625370183)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0634351293192081, 1.0769623439574851)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3621225146049591, 1.545872721203336)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.958846805139947, 1.0230049020289091)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.901165028084583, 1.8163028848824083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.551345182841608, 1.6885986403641053)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.167906857421411, 1.320447068603874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0957733646572474, 1.0542870121252816)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.071036683838371, 1.1045290039100588)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1128674362112978, 1.0779101894490986)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2540558249639593, 1.3906515021515626)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1401782238603204, 1.1633489539189938)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4648903109052682, 1.5017598129200391)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086854553918623, 1.0240902358515251)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.248948606229635, 1.2075460792652186)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0638711306186068, 1.077485929574595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.302366621589939, 2.14537014754687)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6800782204436746, 1.601782563108804)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1324207762644822, 1.2374258969227758)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0106239647709434, 1.0317680864118683)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9398006103566571, 1.150477172942554)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.003711483109492, 1.0184673907891648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0150423857205677, 1.0103067669238535)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9899286762176144, 1.0831523331781618)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0772381946711351, 1.0684834189593408)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5788816509521195, 1.7085125524272937)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086490183228989, 1.100993076618948)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1098840112628257, 1.0747044756233686)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0269037878637026, 1.040319696531097)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2989810246822493, 1.516018871935026)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.09579761936854, 1.0542768476270743)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7312283105258133, 1.6029147242376325)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0041341968737916, 1.0942381626961162)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9652025531199022, 1.0372713852031648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.028971096588657, 1.0496419169018198)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001600167111078, 1.0006364886795711)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5901000545161017, 1.6520210340818942)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0608388344607216, 1.1418201060888773)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6294095853926036, 1.6163785825672696)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9239194629816694, 1.0620409046996342)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9865180053344625, 1.0631700785996852)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0111045677411543, 1.0068501866530697)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0300263800398297, 1.0460945887316897)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.055656008159491, 1.0793102682927262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0592993873762195, 1.1000761948454916)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0763465311021605, 1.150805770834621)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.027312208814914, 1.0466914317467562)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0075629889992679, 1.0117655642478243)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1032720384376888, 1.1258049207286795)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9721553832545388, 1.1562210739966874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.09677874206375, 1.112563844095475)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.144192901363899, 1.2383234015846594)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0491884529157947, 1.0622972532992623)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1201481309769188, 1.23198314524209)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0321506842244292, 1.1028910037854294)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940822219328275, 1.0622150536023378)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0089889001401338, 1.0289256214349285)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.2886972882148124, 2.1481992195754844)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0221349171549556, 1.039958039867031)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.00889678874725, 1.0100369440115566)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0316100774920902, 1.0559582052343706)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7255033424144446, 1.7393040874114574)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4041550187895262, 1.4979801563587698)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0300152256075086, 1.0418258516807768)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0677222492216774, 1.1547546627711922)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9738074676474453, 1.055621670863076)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9554967094702291, 1.2407425382454345)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9852198519912659, 1.0614948925066114)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0225651148434265, 1.0404811287603104)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.023945530468186, 1.1002953897407484)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0366718740572467, 1.0219022700185652)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.89145582510874, 2.0073119818172453)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0256218506525012, 1.092572813082167)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2543424804576606, 1.1584906153804997)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2303746771822694, 1.268150140148696)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3436037993659378, 1.544638789573766)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.054195786747068, 1.1270852408171206)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1132501376936867, 1.0785984664464343)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0191178952071054, 1.0277573500979316)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0040125812148393, 1.0185725847316023)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5090777892285159, 1.6837647168160437)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0548455332712352, 1.135161092605915)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5489217678977465, 1.703872187792144)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0267775706948479, 1.0391766345286284)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086440515245, 1.0133289241223302)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0003622527749174, 1.0006421743934348)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0314807279277802, 1.0687991423520553)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0087316658505554, 1.0156950719736757)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.064672009863228, 1.0744669379189278)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0656820405033327, 1.1329682694466814)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9627709511752979, 1.159799109152461)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.03791917484076, 1.034419506409441)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.056679464796781, 1.080091081084917)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.968253524679665, 1.0560952996256234)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0279655698819599, 1.0414197049581009)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0756470202712114, 1.073197941794778)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0225187036321035, 1.0403827292567942)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0091356279595576, 1.010301773199098)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0089244991442203, 1.0244131377391028)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.958907810383543, 1.0229990123743389)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0101261847400955, 1.015592302490348)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.856487759188739, 1.987802491829756)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0957733646572474, 1.0542870121252816)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4323900975722248, 1.38764558181673)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0002030869213232, 1.0000872458552943)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001817460282, 0.9999968348942765)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6761448836707815, 1.8000159191048122)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0110369777520172, 1.0069196052774705)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0076420270566468, 1.0109130650092806)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3143444888358118, 1.4364660301512269)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6269458399195869, 1.7588137833190511)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0150423857205677, 1.0103067669238535)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9503313698708773, 1.0210101877141475)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0221103411565742, 1.0399310912541044)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.022161740185914, 1.0440362161741663)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0212291948123313, 1.037812238414688)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0230473136805212, 1.0409687987569014)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000574743026291, 1.0001944692240694)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9790271258063736, 1.0621447449651071)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0045672201764262, 1.0206986161366172)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.981552788172939, 1.0654062102517186)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5786147994618658, 1.5796723157571968)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9505085876017155, 1.0266160225205738)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8992468483830982, 1.8381705583042731)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.007640550260995, 1.0109111691283788)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1023798800220563, 1.1027946169389353)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0378383854026931, 1.0916541944229017)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.662171547715608, 1.4809388858784158)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3884009154247585, 1.4693833732295243)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.00820081147347, 1.0048237589520899)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9891217013593645, 1.0452298135042355)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2967977505086272, 1.4205821265807699)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0267734505196762, 1.189527234470338)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0095041942875886, 1.0304585780094506)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0984425260911028, 1.1801463996773247)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.008949565268555, 1.0100975605869684)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0555547511592458, 1.2005282123293055)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2019556929161193, 1.4631989134731263)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.85401212533698, 1.697045646823639)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0465953184091767, 1.078970690899583)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6315200339640263, 1.7412508264771518)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.475741136999491, 1.6184227747004782)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9813546998577536, 2.0012193652412913)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.1175332031727656, 2.0604185415667806)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.908707020100311, 1.062610305602706)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0092028625440435, 1.0245464529546735)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5536231422112985, 1.6445809489282637)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.504961884357026, 1.4765656818367083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0040416763463311, 1.0188538547942232)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0229109649916208, 1.040825047374595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9243708792845878, 1.0618213332611868)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0100659861217105, 1.0346741609410086)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001738283730812, 1.0007021852863212)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.765619624354735, 1.671608274339667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000256539704562, 1.00011472593805)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1749660006132376, 1.3348151157119925)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5196245050708794, 1.5016510294807845)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4372234986294643, 1.556964093531671)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0495427967759612, 1.0637613514438269)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8942162937965823, 1.0623074342290058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9566233368769345, 1.022180800314833)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.010297978789749, 1.015862871232645)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.022537171977591, 1.0404032050948724)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0089244991024624, 1.0244131377024719)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0002005714269862, 1.0000861425492609)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0225855613154375, 1.0474590050600343)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.008282093745896, 1.015020385021123)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3292441637907877, 1.3420852968778196)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0355842499392884, 1.0403347265719545)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5598901570117432, 1.754258521302419)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.00756367750926, 1.0117730012140096)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.589460202631202, 1.6310416833075057)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009182414588414, 1.0245368362440739)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3500679823016972, 1.5849241373278118)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6300716452223947, 1.6608549895984739)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8934598111960491, 1.0613851373263503)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0172232674741366, 1.0309227835834553)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4026965025537432, 1.3017255685461213)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0145845858779112, 1.126713426855627)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086854557919522, 1.0240882973827123)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0332645848178301, 1.0452853283629715)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9591073336357664, 1.0230972090860468)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0090239324043384, 1.010197809009051)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0438961669928477, 1.0729439648575128)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.181806091706719, 1.1839318741128002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8563512541739648, 1.9976868998238977)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9501169477878597, 1.0205649913958266)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.836137114254394, 1.7153939255824424)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3134774434799432, 1.342577163042404)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8197120532555924, 2.0141723235965414)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9207919331237537, 2.050377207587237)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9639522648751613, 1.036055644122443)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0086948638130546, 1.0241059801816847)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2725033245342992, 1.3217851085178995)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1689277456189513, 1.1614566237272046)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164099, 1.0210066614435458)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0091441437902997, 1.010337914035154)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0092865647406306, 1.0150552663051189)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0244815605193682, 1.041186509326272)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0268617336683088, 1.0402377936518334)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.0203691919367213, 2.0355428292104714)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8893296347449184, 1.8067703531209003)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000285414250728, 1.000127735592676)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.014727668091586, 1.024994903364542)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.374735755938457, 1.3683210931928274)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9591167474096071, 1.023255555647784)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.014727668091586, 1.024994903364542)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.916975185408699, 1.0965767497411336)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009283349316657, 1.0166808687633724)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0078702046233545, 1.0262933124211526)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000256539704562, 1.00011472593805)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8485257789765346, 2.121358072841311)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9601905509580709, 1.0334768742054787)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.5255282929440686, 2.452911024745238)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001813631912162, 1.0000782589566193)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8983651890409553, 1.0766186783966203)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.958846805139947, 1.0230049020289091)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9688327513277963, 1.044563674185145)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000285414250728, 1.000127735592676)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.888421415046983, 2.0389624328064775)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.897469495829563, 1.0334486669474874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0076257075690669, 1.0108914598331338)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8447784384874368, 1.8537593361734914)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8348029147999825, 1.7705575036765941)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.011180184215307, 1.0069192384807035)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9443577252528829, 1.09761943095273)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.962539015690692, 1.0367422623126292)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.003763196359698, 1.01869747451309)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000442574487853, 1.0000290664078346)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9713441551507028, 1.0418621810901223)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1649562462636027, 1.1543387918993817)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9576015593182938, 1.029787566065412)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.956985634947714, 1.017789008078671)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4398772285772838, 1.4578505988350332)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1887359576087102, 1.4116640096821176)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940847901867173, 1.0622175913578324)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6004502807844898, 1.8319691222586834)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6641820329897186, 1.780828152694232)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.955791832116544, 1.0181799290622577)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0002016077843499, 1.0000906155009845)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6708890920970974, 1.8389841661831505)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.731113267044244, 1.880056161733783)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.227388537978634, 2.1583684897174633)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869128255280491, 1.0393251197518067)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0091441437902997, 1.010337914035154)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5831518940461633, 1.6998967959198485)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6603639268468309, 1.6072962270464162)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950108053305285, 1.0207871078265653)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9274518154699893, 2.095257601452129)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.292504788859244, 1.4883772934654753)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557918317153393, 1.0181799286637723)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941983994502497, 1.0622879255884905)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0090071784623644, 1.0099373959556541)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164096, 1.0210066614435453)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0090127923955599, 1.010150670921568)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0002030869213232, 1.0000872458552943)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.278879179351104, 2.0825058086425807)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0112492158592294, 1.007657265605051)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8913331541256028, 1.0569426743997046)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (2.4361590056863096, 2.374190931569668)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939289404560917, 1.062022404926285)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0090991943468435, 1.0102877550462999)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9243708792845878, 1.0618213332611868)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4331094728613554, 1.5112456252710835)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0192733415750903, 1.0275285681001012)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0089160640918784, 1.0100617539516896)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6042447677558862, 1.7629027148367047)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6634026124917787, 1.5925843101227983)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914921705024802, 1.0585524353231401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9097744112621657, 1.0500497370526887)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9505136962575907, 1.0213377649219801)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0090239324043373, 1.0101978090090498)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8623825201586335, 1.841775191472733)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9026042075368054, 1.0690367396043892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.009486289062839, 1.0166388049448345)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.321108065322959, 1.3022205430125817)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4627883481240596, 1.3156113591698202)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950278454731098, 1.0210040021918243)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3730902106247738, 1.4271836797815871)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.00772034451788, 1.011000459386809)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1914682869757884, 1.2941662133722462)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9566051304967601, 1.0221569090939595)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001797235145369, 1.0005184894104755)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2927623396592858, 1.3793261908995886)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4512984572338474, 1.5343143802245203)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7618646009565007, 1.8747397733176263)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0093810934189602, 1.016564710112363)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9706302205069914, 1.0458661138935021)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4013784259898834, 1.495704806512474)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6000330454581484, 1.6493104962012994)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9866790255995396, 1.0615916381646795)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.840027281222045, 1.7316649697960953)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8945271508179691, 1.062425004847966)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842479077887365, 1.045410572513111)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000031083525063, 0.999997622328312)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0002203665049088, 1.0007179948702902)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5738124503212934, 1.5372375552568045)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.520338878533676, 1.5683495339813136)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9595739055422712, 1.0361899292515682)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941085492117471, 1.0621058219322854)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000011370020554, 1.000003900558499)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000574743026291, 1.0001944692240694)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6659089512776009, 1.6411694441435993)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9505085876017155, 1.0266160225205738)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941940722453688, 1.062285730482262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9026042075368054, 1.0690367396043892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9699595875812708, 1.047341440198091)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5299505481929392, 1.7556035094271705)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8925953072126493, 1.7360014096095309)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8257252605429963, 1.987568762431884)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9087070204729161, 1.0626103059155712)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6183932239248628, 1.5333785020280855)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557917578271535, 1.0181798461719413)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9624879491822841, 1.036692099157249)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9814308983958775, 1.0504867838097345)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9091874092458865, 1.0502540809236933)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001583606407198, 1.0006397026853333)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4560704394660193, 1.5245406763062734)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0082320736540555, 1.0047539818872901)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0000561676542408, 1.0001900480586785)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9501120740489738, 1.0207923943122854)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3302168684292093, 1.3910388870841244)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9564842773957657, 1.0179328280678963)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5370607748602756, 1.4814698259141177)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9566636828156103, 1.0222389933366538)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.2149201540394599, 1.2410130607210792)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1870061638011018, 1.1882924568036257)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.894502858306569, 1.0623901471374817)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557918309581602, 1.0181799277089096)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9791820258980188, 1.1297320818407426)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9556375843670228, 1.0179063885260198)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9669608449343777, 1.0389959071458281)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9505820232553542, 1.0214431078598631)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9553444300188888, 1.0133820227730423)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9569465632576678, 1.017535783432386)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.5983409781557905, 1.6311671623174848)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.000003304326776, 0.999997741871058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.9067261609115833, 1.7311009781764974)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9970148176562125, 1.138200910026389)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.956561128439479, 1.01928379408489)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843121783984705, 1.045122452388375)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914949601852116, 1.0571491736459837)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0581542682436056, 1.1298324564688413)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939624888794047, 1.0620575287811318)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9261894013752994, 1.0614450422166941)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.311894127129303, 1.596997589078736)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.398730083603971, 1.615434483494464)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9571174216446804, 1.0135612308021646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9556477291833532, 1.0182447555539234)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9595912987209998, 1.0264966554531383)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869123216684076, 1.0393239217323598)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.7993807443116585, 2.0052457968459505)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164085, 1.021006661443544)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8983879074194269, 1.076641128801191)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914921705024802, 1.0585524353231401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845384562069515, 1.0436712889104103)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8518724464847682, 2.092757728915635)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1776382188472128, 1.4300494802833006)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8011933368830362, 2.0928842843942275)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974695085832005, 1.0334885872816622)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0041697272247563, 1.0872653555042004)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.957310268432746, 1.0193575402556054)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.955794419721498, 1.018182816515744)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950700372825176, 1.0215944566750284)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9572241068979425, 1.022062159031538)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0001754844408781, 1.0006989712784398)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.6583159448476927, 1.7924984382408327)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164058, 1.0210066614435407)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8947970054494894, 1.0604894032113408)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3537714409095722, 1.3894619321159891)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941938296181465, 1.062472353424943)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.8853649310472507, 2.1105484702237076)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9591173347273355, 1.0232174044910312)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3454490354419428, 1.4105764754211916)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1702023921991176, 1.3658928807594075)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940822237354685, 1.0622150558748473)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941986977444691, 1.0622882503047892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8975475595273333, 1.0753400811218452)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9590699161253581, 1.0232043634269357)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9503125159091969, 1.0210420120912211)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.003889578630017, 1.1606345666182971)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950113403784784, 1.0205619035430495)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.3430936954250827, 1.5506968430785084)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8972369216559433, 1.0335345934747369)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.4194257987799546, 1.5864786233336352)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0152618855570221, 1.165209463789838)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502156200769863, 1.0209100452926638)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9601913299052982, 1.0382918044546101)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9704251472344442, 1.042113399407608)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8975476727079061, 1.075340222537085)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8975735902692433, 1.0335841108005055)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8986678384036879, 1.0320941678923272)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502807565864843, 1.0210065707247646)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9548962356323156, 1.0143130983249151)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868394865215237, 1.039244521458028)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8946529452885819, 1.063178560620024)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9709853608924994, 1.0336262984865476)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8938362321354126, 1.0619225875032998)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8945813186778627, 1.0625432662957484)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0537649697138565, 1.1253204289809702)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8834031453562833, 1.0252116825829065)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9002114935261128, 1.052843316232599)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939868854382584, 1.0626553015777687)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846995776717301, 1.0438764624438224)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9601866562524861, 1.0382872372565464)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.918361006993246, 1.072692087375352)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9097744112621657, 1.0500497370526887)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9599373167647364, 1.0380274564539875)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941986985350194, 1.0622882509983287)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940857801832358, 1.0621659191013397)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.0, 1.0)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8956671061768354, 1.0618203772255705)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9551180174642577, 1.0184940119441308)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941041899465134, 1.062238724206648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974693167493978, 1.0334484671193227)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9084983756494442, 1.0630820590596464)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.1177881200122328, 1.3080483398718796)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9090788392903091, 1.050292387682496)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939462854196408, 1.0621300567461451)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9539848444971053, 1.0190333946692782)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502693469209685, 1.02099383920014)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8951580173308704, 1.0632771055645738)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.896816318101885, 1.0570175214717819)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9582164623913728, 1.021791310090982)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.894502858306569, 1.0623901471374817)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502057766536014, 1.0208950929329204)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9501940966090602, 1.026164744959707)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8983651890409508, 1.0766186783966145)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9601823415880952, 1.0382830220396266)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9564363832233063, 1.0257595124077998)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9507955453045585, 1.0240585241267968)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.895107307874063, 1.069015804437798)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9557997933043807, 1.0181888126403205)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9032981133494074, 1.0681160304151667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.894198716109187, 1.0622882715933644)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886904674726375, 1.039465005504385)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9538358391697054, 1.0188676631255702)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9560997016241446, 1.0180441011597698)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843144776016898, 1.0451250179827514)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8942127496510244, 1.062219119981917)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8871011419850323, 1.0390668424158374)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.880905825285222, 1.0428735791152401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502037881623938, 1.0206625892287897)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950280837822858, 1.0210066619559381)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.898142949053932, 1.0273557070345178)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9126549682316623, 1.0932099898540915)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9554341646703016, 1.0325367939663324)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9563788644556133, 1.0179604873085566)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9556471819636251, 1.0182441449590736)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.899299100178376, 1.032994101336617)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9029986408836063, 1.0677838934625534)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8907454142599518, 1.0563427426943064)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9644780586275306, 1.1558413123215767)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9556375843670228, 1.0179063885260198)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9501169477878321, 1.0205649913957933)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8947970054495682, 1.0604894032114363)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9025628566009269, 1.0704267990929364)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842255304503367, 1.0454908877096365)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8944549755591761, 1.0610267983402195)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9503469916101253, 1.0167045624163822)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8916160225362895, 1.058634376177002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.957310469764952, 1.0193577649242986)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502674684905084, 1.020991742812167)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9221678165985119, 1.076276161902389)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164081, 1.0210066614435436)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8991665529711603, 1.0328512775486034)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870520603304433, 1.0396862636042665)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940847901867173, 1.0622175913578324)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9000456921822038, 1.0324194710975032)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941428478931885, 1.062232943148117)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9505136962575907, 1.0213377649219801)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808374164083, 1.0210066614435438)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8930905379743387, 1.059912229716592)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974695085832032, 1.0334885872816653)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842370070455763, 1.0451281007031168)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8854949161055692, 1.039102962113158)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9538215046161871, 1.0203195545601644)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845228466670344, 1.0436566417808824)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.890616300144348, 1.0549532598741442)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841505763172492, 1.0448847099839396)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8942204821106046, 1.0624040475435306)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9114526452581333, 1.0492939459825077)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9014796619112142, 1.0761924353252783)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9558429193258363, 1.0182369352211798)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502808303033987, 1.0210066534118285)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9502850609903386, 1.021011374475186)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845514649331024, 1.043529453162475)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8847000813038893, 1.0436279506451003)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9570691982054613, 1.1103112911695283)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9013204863392861, 1.0832920031898172)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868378432623664, 1.0392416657718972)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940702658270657, 1.0622413499918526)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8946607951806524, 1.0609528659393674)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8915267267745466, 1.0570885903196658)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940069920219259, 1.0626010499641199)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8988966338847353, 1.0290562752880255)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940857789804284, 1.062165917584331)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8940838379347354, 1.0623559996721639)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863315674186475, 1.038442243920968)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8944574385417562, 1.0189216305173685)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8804519575823039, 1.0425383212352792)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941940722453688, 1.062285730482262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.894107162630076, 1.0621040894177067)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.893621727962765, 1.061556070617722)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870245342920887, 1.0392143832100182)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9501169478514208, 1.0205649914759567)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941986977444691, 1.0622882503047892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883080506751018, 1.0260443788445517)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843144776016898, 1.0451250179827514)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839594384364455, 1.0425774520522129)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9116216355475159, 1.085318412600921)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841505763172515, 1.0448847099839418)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8942457346827394, 1.0623341252258116)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.897469495829563, 1.0334486669474874)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9102393965122879, 1.0507837133831952)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843170066088211, 1.0450352466927582)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846995967702157, 1.0438764837780505)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9002084297744006, 1.0528389199769097)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.950116949035539, 1.020564992968726)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844868425842727, 1.028058842244608)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8942348032083767, 1.062327611527961)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845226107850152, 1.043656742267697)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8965515811994822, 1.0622047763435993)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8934383742359895, 1.0612902851389145)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8931188494060625, 1.0599398762864045)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841602105652381, 1.0450304519593647)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870333881273703, 1.0398664751039377)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9124698816652121, 1.0489539015522962)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838724877749433, 1.0416416165207394)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8943893414750063, 1.062149554765627)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848499124824625, 1.0363897077555162)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842257259764116, 1.0454911050238098)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8996622175873834, 1.0329178942483495)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8992991001299203, 1.0329941012943542)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8911133713921698, 1.0583498777492444)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884536172266802, 1.0436687403990905)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843008045554382, 1.0451077908452195)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8962467859000214, 1.055858900483028)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8840876876524356, 1.0452930898134833)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8872712482438897, 1.039406924402545)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8847781467586744, 1.0431543303867161)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9000456918017351, 1.0324194707637366)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.895933381136212, 1.0521004817597785)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8986696773519718, 1.0320968067070329)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9558301752152614, 1.0182183499901714)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8975916782296633, 1.033604294409082)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842370072956964, 1.045128100921271)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8936448942088875, 1.0616736388480328)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914921701608283, 1.0585524350234214)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841408209618895, 1.0448738243521225)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.897469332334775, 1.0334486166146322)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8895902889473792, 1.0373244996700266)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844993612079805, 1.0436305177351837)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9813550906160847, 1.1098582569016473)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868663061843037, 1.039365606911665)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939647127190081, 1.0620599531532042)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868520564431657, 1.0394490132952843)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846554700874092, 1.0437883950477789)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869123212073634, 1.039323921327908)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974496490815799, 1.0334265202245176)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9000456919962718, 1.0324194709343946)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8837253799288081, 1.0338634703834906)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941069393649306, 1.0626509324205353)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8941572672994204, 1.0624325362536797)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842255302163166, 1.045490887414613)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863376820217886, 1.0402088785033003)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8937862504646655, 1.0600292041076598)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8975735899758811, 1.0335841105446382)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9534115813263999, 1.123342048390804)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841392031911509, 1.0449026537509514)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845246117574453, 1.0436142307738507)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914671050687435, 1.0585257440058378)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9178373185632005, 1.0473066087363918)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8973686286109105, 1.0337008490627486)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8939765147277854, 1.062111734933252)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8814599105397722, 1.0447945675036054)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9334423471315889, 1.126911387094484)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868739516404095, 1.039508318666753)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8979534219825587, 1.0651009851999302)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8916160225362895, 1.058634376177002)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974695087150142, 1.0334885874047053)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8964636943182209, 1.0559571274534332)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842370070455763, 1.0451281007031168)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841216471812222, 1.0447895367657338)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845425775133958, 1.0436085598150364)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870245172520231, 1.0389825381508841)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870401480436075, 1.039581714734841)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868841696245217, 1.0392331844300793)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841403881450398, 1.0448733421219132)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844414873055246, 1.0280764331526626)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844737621629787, 1.043688377371377)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8814614148480985, 1.044796246153705)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845267858637098, 1.0436601895367428)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884219607191592, 1.0430138413858436)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844684952850032, 1.0383096969404442)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841576335763275, 1.0448863695284558)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8840391623696698, 1.0452482758546946)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846640211169294, 1.0438367863901072)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868603382669491, 1.0393866012891804)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884180908631619, 1.0448922283064215)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8997090583650919, 1.032565374034569)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870362565866272, 1.0389980281267344)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8840291791149988, 1.0453048693540814)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8835800163582026, 1.0424493278057945)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886856063760331, 1.0394879414307447)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838121971270286, 1.0355157748833328)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9255949844662335, 1.0626131441546731)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974640427525638, 1.0334422572355222)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8847000689814379, 1.0438775736386503)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8891288961363409, 1.0465636640176579)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862880933071776, 1.0384446793191695)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868375533618861, 1.0394525283465947)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848499124824625, 1.0363897077555162)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845398842016197, 1.0436731717913503)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843273167749219, 1.045081535540481)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8840370007111185, 1.022493919106171)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869046747264594, 1.0394650055044872)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869159906292576, 1.0393196961342464)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842699791157562, 1.0449803877080743)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8816319246464079, 1.0306295586308563)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841408207651842, 1.0448738241805566)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886878371740889, 1.0393790725927041)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868653103474409, 1.0394271257654797)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868718130327133, 1.0393110992361045)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833349795200912, 1.0255822626728397)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884523802723792, 1.0436574088524633)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868075025140963, 1.0397921548266529)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8829825865699247, 1.0259113676917708)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841477506941997, 1.044913196789047)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8814599105397722, 1.0447945675036054)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883204348915901, 1.0330085174431654)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8806266253237603, 1.0427645810528077)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8894008769442711, 1.036979775680204)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863541760344726, 1.0385258663896895)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.897200256513114, 1.0429276763929074)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870649887714022, 1.0396830350670634)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842594950042909, 1.045027289953092)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9521455524184239, 1.0221118564220961)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868640699725098, 1.0393631116044628)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8957989239335423, 1.0206426422460786)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870457960656776, 1.039593424406626)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8849364703023895, 1.0433708685692302)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833601863438498, 1.03384497461026)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850245273467933, 1.03677481327519)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.880905825285222, 1.0428735791152401)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869194542872769, 1.0394345756679813)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8971691847604705, 1.032803812547496)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838280223480993, 1.0418895004546864)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842120490389063, 1.0320071018388561)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846269060691648, 1.0435230472671055)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846533779171868, 1.0440166564909756)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974403111587477, 1.0334644304168354)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8990810668573138, 1.0328536932385453)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845828630732562, 1.0441773864300856)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841654847166786, 1.0450366620346174)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869953644591441, 1.0395498153719949)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9005845375444042, 1.0325576012142559)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848228082558551, 1.0431517208845484)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845653032998577, 1.0437163485198022)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841491100237233, 1.0448827852614155)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845185371732659, 1.0436789061037213)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8969811692111858, 1.0336098810862542)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.887271247510772, 1.039406923729551)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8867595941338793, 1.0393579671691955)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (1.07966712644957, 1.1843918968521627)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869128255280486, 1.0393251197518063)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8972261716669117, 1.0335225960977896)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8842255304503492, 1.0454908877096516)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8865250553294518, 1.0387807151483246)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884125191128995, 1.0449786677872521)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844987494658294, 1.0436298351180213)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8872690757153521, 1.0393034580679124)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862570942491317, 1.0382259536884817)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8847572810192503, 1.0439344255601406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838576766964145, 1.0424314351634083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844969881782313, 1.0435568806963669)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841304694340802, 1.045014907688456)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845787593296526, 1.0442464914510472)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8871323467216152, 1.0391001318009236)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8986678385767901, 1.0320941681105362)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869975134809187, 1.0394765833673683)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874039512068537, 1.0240345831162683)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8877746284151872, 1.034815231026067)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844868497093711, 1.0444417607289582)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863007010272281, 1.0384685724471603)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836677687159668, 1.035450191692939)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843071999870654, 1.042835218919069)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843453665780665, 1.0279620697397762)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874039512068537, 1.0240345831162683)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850929541779191, 1.0368702028546122)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8810555402701352, 1.0312717098325557)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870092677324964, 1.039369527965617)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883080506751018, 1.0260443788445517)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8837810159879984, 1.040093988399115)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843523838432643, 1.0440564288299463)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8894228840304867, 1.0386181414715459)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869281604699416, 1.039486235999586)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883847903936893, 1.0338918073297667)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836296048035176, 1.0422811255069206)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8871542477979518, 1.039392575545263)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845109858961587, 1.0444223078288253)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8809018109728631, 1.0428690996408878)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8866100258201463, 1.038915554047133)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8956119528474671, 1.027772133820594)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.881197751894625, 1.033411874365739)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8821904119990687, 1.030302992232298)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864799041678686, 1.022881174465728)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863575115401833, 1.0385685444810406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8867596063620274, 1.0393483027233914)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886522936926914, 1.038777344163882)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8822213539645855, 1.0354804612831485)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883959438436387, 1.0425774520521596)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9061654464880206, 1.0643668887768918)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839172241779049, 1.031880811560815)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8851878533275718, 1.0246708439080188)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9376118441015455, 1.100316473057421)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883955427075993, 1.04253763230495)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8858732043733644, 1.0309519001029928)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850079784299527, 1.0367547567032593)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886975439950094, 1.0395185619913496)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870333867940885, 1.0398664738575873)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8867269691727351, 1.0284340908329648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8875166384729484, 1.0238354196096289)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886872960962994, 1.03940035569736)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8871503471009436, 1.0394325136590077)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884849912476245, 1.0363897077500617)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886947292523965, 1.039416692967769)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8797496336133791, 1.0431888817709927)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845911488693399, 1.0436351874752188)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870688132957278, 1.0391240198382945)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883839313387906, 1.0479413912177828)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868701799941592, 1.039394672779692)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8957328659463455, 1.0205170539991448)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8914533122296399, 1.0358892108822018)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883876416554687, 1.0416460004771249)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869965339812815, 1.0394779746485405)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88178641974889, 1.039808613138718)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8804519575823039, 1.0425383212352792)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864711694553102, 1.038721607509162)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8856840334787076, 1.0243880293521692)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88641705824979, 1.0308786893057682)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868922170037916, 1.0393543847703948)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848566649778343, 1.0364014420266998)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874904954193316, 1.0387054482888916)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862261016074253, 1.0384758997780277)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8964186875550347, 1.0245044441654427)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8984481156794475, 1.0487561262726952)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868779755442596, 1.0393791202694795)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8865144809298917, 1.0260979992964847)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8806660436095406, 1.0377276178759598)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838843017505087, 1.0416774481848603)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8812344045626206, 1.0395617874388396)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868390410094851, 1.0392437467119648)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863206431412547, 1.0384898413449337)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833331126168856, 1.025580013388065)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8857130546744113, 1.037007440454426)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844323237453654, 1.0280605092116128)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870170185671551, 1.0396590361267295)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869064405070886, 1.0396398923746468)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974072033333113, 1.0334374426348913)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868387736829476, 1.039452491910502)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870252456816653, 1.038984102940454)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8807213543533097, 1.0284419140613277)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8869219682423782, 1.0393351573721359)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870116799101543, 1.0395384103751195)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8818269591881492, 1.029879539322214)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8849275119615388, 1.0378894800650615)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870639338624404, 1.0393195049475927)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9150098617563833, 1.0663411351436822)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8888758093935458, 1.045108720455998)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868378432534189, 1.039241665764048)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864736395418353, 1.0387214001295224)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8806256403216461, 1.0425901841352745)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8870300188998725, 1.0390844019092682)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838688630209862, 1.0223833011525654)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841254204534181, 1.03547317542759)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8806266883183438, 1.042590630097335)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862669608609823, 1.0382102879814323)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8849685459853902, 1.0310377215935262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883431367710808, 1.0254038995439723)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8868343684517165, 1.0397701443822638)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862924489136382, 1.021852756957845)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8802325023131706, 1.0386051982927567)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833208932288326, 1.0254572688985601)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841254204534181, 1.03547317542759)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833960205879398, 1.0347818412495706)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848444724311246, 1.0363802616395608)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843736809349437, 1.0229440078983232)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8810940126070648, 1.0313411667690613)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8857918743670979, 1.0244745697814246)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8829008720070219, 1.0417916099260798)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850257855680215, 1.0260286236360203)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8797285504068043, 1.0281378487750228)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8805192412868336, 1.0375381519739815)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8857818062003725, 1.0236209475482687)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839117957055439, 1.0319567783163264)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8974278103063263, 1.0448933786341863)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8808565309785706, 1.0262295212048622)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8812344058977594, 1.0395617891219944)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8857566870327165, 1.030724720963988)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9008517992043875, 1.14016885457395)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8872739896382066, 1.0249781748987283)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844905018904644, 1.028173686831477)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850186672415757, 1.0247154310037414)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8813156360864791, 1.0441035949812352)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.881817639512068, 1.0397460888921104)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8831206002857801, 1.0260905487198027)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830808413786386, 1.0263544597562932)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8951563612849178, 1.0363503061889159)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8801187060936172, 1.0285424794593505)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836255784767675, 1.0237856240668897)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8853234937153627, 1.0245515992460166)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8976546290202823, 1.0204612054677062)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8803767022550749, 1.0381703079575026)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8848243975146199, 1.03080461985632)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8829268887453925, 1.0515352194687886)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8823400039474092, 1.0405499577033483)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8810122706089247, 1.0311773514731)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.899878790506009, 1.0308252556631272)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8816745096558878, 1.0334645099652346)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836996715325873, 1.0264873743345018)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8835800681549406, 1.0353143043938255)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874038675332423, 1.0240365056039387)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8970691058248013, 1.0312143684583175)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830662966255733, 1.0305070738490316)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8798452404477423, 1.0433138420261467)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8845124595596596, 1.0282902418007491)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8818176040676019, 1.029642282460642)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8821904119990687, 1.030302992232298)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8856840334787076, 1.0243880293521692)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8850779197016599, 1.0368513688619263)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863734003864407, 1.0218887540326498)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8861525229019205, 1.0305574579489853)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.880556612786355, 1.028392803469554)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884849914183206, 1.0363814077308688)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8865775112934304, 1.0386754948661256)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864108577712083, 1.0386877209285064)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8860406224749614, 1.0305339897456605)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844544647362024, 1.0314040257782406)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8809935367298692, 1.040652155530783)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8817447532382219, 1.0297684244147511)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8824517944806263, 1.0403512991684685)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8944905409329525, 1.018879348889758)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8840222544473272, 1.0430973865360902)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8872980653077914, 1.0263440168459188)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8993383789195616, 1.0269598602933792)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8815482210365153, 1.0277867340491118)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8859471489258909, 1.0378214518678608)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.896398511913371, 1.0244442126859563)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8957995474555421, 1.0218821315836872)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.882806632369967, 1.022386793794319)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838157165947296, 1.0224218078448892)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8822020516151738, 1.0354706079804974)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874819399889983, 1.0240958770319044)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874313615404985, 1.0238839090286322)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.887460038958368, 1.0241167669701143)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884945758022629, 1.0270397798007884)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836438978623783, 1.0269351622622565)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8810180524731769, 1.026313627857682)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8805360970910437, 1.0430020832501985)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844653217648661, 1.0281782870143084)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8804990836485151, 1.0281252230598963)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8880670898084847, 1.035773640787236)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8832755272877548, 1.025498587926587)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.881300428261898, 1.033487494999639)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836632794565139, 1.0298609852700185)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.881786120269805, 1.0398076082948509)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830929691185837, 1.0261067861694473)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836034388250953, 1.0354481065629202)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8813524425533105, 1.027366499952146)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8873653126844468, 1.0300286630187048)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8835975064531558, 1.0337026064186619)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88424041955985, 1.0316555341025693)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.881297494298276, 1.0397183183428083)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88641705824979, 1.0308786893057682)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.884849912476245, 1.0363897077500617)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8837703139426802, 1.0425447356451951)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8833806056421495, 1.0338076423762204)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8816319246464079, 1.0306295586308563)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8861403777858053, 1.0220759835815074)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862964538822913, 1.0384774895744242)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8800703464054904, 1.0281259072666897)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8852337208224543, 1.0247310330229955)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8801235724999943, 1.0284722273655693)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8829219277756155, 1.0224791679638434)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8837279920038196, 1.025384896633747)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846274750806976, 1.0282193663856836)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838157504015557, 1.0227899609595161)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.879860611645739, 1.0289916977068714)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8805243097451915, 1.028464607854174)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8922305414931471, 1.051981252311652)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8847290850952502, 1.0235397042957322)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8821348667493571, 1.035722153039107)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8853234937153627, 1.0245515992460166)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836186176457813, 1.0238029635977772)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8831954933480487, 1.0257160769897635)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8852451661081575, 1.0256420580313257)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844932282195671, 1.0261857442154578)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843149129848474, 1.028085513300432)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8810122705383889, 1.031177351411579)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839764324631005, 1.022736328929449)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838901087873526, 1.022636944668754)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8804573688391676, 1.0285451169756699)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839858733436812, 1.0224040760519006)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8806812328446679, 1.0295270731999482)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8818382403982071, 1.029567125282607)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8818269591881492, 1.029879539322214)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8829192618439978, 1.0249345367926537)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8797889610314171, 1.0279876730827446)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8863728183837453, 1.021885656606352)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.880869539816131, 1.0274482980290522)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8832341100915341, 1.0233391597455004)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88590272868021, 1.031598193147425)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8805557462311357, 1.0283858637713112)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8827555255510684, 1.0283964315347045)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830171754299031, 1.026179400751388)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.885011665638233, 1.0267080264845487)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844987802167745, 1.0279938323274105)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.882423929639966, 1.0252425881921148)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862434245003847, 1.021711781404769)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8838837586333645, 1.022385604786707)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8813764925661375, 1.0326583967518554)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846162298357588, 1.0278933288901146)
 STARTING EPISODE  1
UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON
Episode ::::: 1 Evaluation :::: (0.8847623922108264, 1.034287372771509)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8832772777153879, 1.0265434470611374)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844412982539935, 1.028223715430959)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830096229351724, 1.026318906113262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836458246290728, 1.0253592434397152)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8800736204756704, 1.0281220496354941)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.9050572821287955, 1.0462252697249053)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8821934977306434, 1.0303053964559374)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8857886837810696, 1.0245434029538567)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844025749163604, 1.0283811976500434)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8834891825209601, 1.0260152056110132)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8880165171146122, 1.022610014316876)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8830415218752963, 1.0263067946696354)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8836217773002284, 1.025174005173988)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864062168366843, 1.0217961913997526)
 STARTING EPISODE  1
UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON
Episode ::::: 1 Evaluation :::: (0.9150802925369884, 1.0886651105217835)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8872181106590594, 1.0239515503145886)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8874041851473254, 1.024029325084619)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8821441546583499, 1.0313017415756547)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8849685459853902, 1.0310377215935262)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8861792336737766, 1.0243937510979058)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843863387519391, 1.0224604061040612)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8894877622046623, 1.0252789895730734)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8817447509952839, 1.0297684216940026)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8892447930091479, 1.0236897671204293)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8996754818072406, 1.0223861778014818)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8846318331268077, 1.0280168799426017)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8858289504395894, 1.0226964230422662)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8864193132174071, 1.0230073171349423)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8834519611392884, 1.0251250377015788)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8849803114182522, 1.0260161489673698)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862907025238627, 1.0218527989674056)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.885009188142692, 1.0255843879829065)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8843311626945676, 1.0258640881181795)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841930462872625, 1.0226852946445635)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8841913429002656, 1.0274420696289301)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8862386698713425, 1.0222489647645912)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8860418282010637, 1.0228487286090473)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.88347428501444, 1.0254724556534198)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8800570037814501, 1.028613364171055)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8839044510378904, 1.0236153848039866)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8858590566469442, 1.0250747259801765)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8844677926831401, 1.0230651634252528)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.8825725351108615, 1.0180308004099439)
 STARTING EPISODE  1
Episode ::::: 1 Evaluation :::: (0.883603269550208, 1.0256726144720323)
 STARTING EPISODE  1
In [ ]:
import plotly.express as px

fig = px.line(x= [Time for Time in range(1,25)], y=res.pop[0].X, line_shape="spline", render_mode="svg")
fig.show()

5. Particle Swarm Optimization (PSO) (my favorite algorithm)

PSO is a popular metaheuristic with success in a handful of applications. We implement the single objective version provided by pymoo.

We set the following paramers for PSO

  • population size: 100
  • inertia wieght w: 0.4 (helps with exploration)
  • cognitive coefficient c1 and social coefficient c2: 0.9 and 1.2 (higher value for social coefficients indicates social learning outweighs cognitive lelarning)
In [ ]:
from pymoo.algorithms.soo.nonconvex.pso import PSO
from pymoo.optimize import minimize

algorithm = PSO(pop_size=50, w = 0.4, c1 = 0.6, c2 = 1.2, verbose=True)
res = minimize(problem, algorithm, termination=("n_gen", 200), seed=1)

print('Threads:', res.exec_time)
In [ ]:
for member in res.pop:
    print(f' Solution Variables ::: {member.X} Soliton Objecitve ::: {member.F}')

print(f'Optimal Solution ::: {res.opt[0].F}')

6. Miscellaneous [Other Libraries for Evolutionary Computation]

  • I tried JmetalPy and it's implementation is below (libraries have different set of algorithms)

  • pagmoo is nice (written in C++). I haven't explored it but it has a handful of algorithms

In [ ]:
from jmetal.algorithm.multiobjective import NSGAII
from jmetal.operator import SBXCrossover, PolynomialMutation
from jmetal.problem import ZDT1
from jmetal.util.termination_criterion import StoppingByEvaluations

problem = ZDT1()

algorithm = NSGAII(
    problem=problem,
    population_size=100,
    offspring_population_size=100,
    mutation=PolynomialMutation(probability=1.0 / problem.number_of_variables, distribution_index=20),
    crossover=SBXCrossover(probability=1.0, distribution_index=20),
    termination_criterion=StoppingByEvaluations(max_evaluations=25000)
)

algorithm.run()
In [ ]:
from jmetal.util.solution import get_non_dominated_solutions, print_function_values_to_file, print_variables_to_file
front = get_non_dominated_solutions(algorithm.get_result())
from jmetal.lab.visualization import Plot

plot_front = Plot(title='Pareto front approximation', axis_labels=['x', 'y'])
plot_front.plot(front, label='NSGAII-ZDT1')
# save to files
# print_function_values_to_file(front, 'FUN.NSGAII.ZDT1')
# print_variables_to_file(front, 'VAR.NSGAII.ZDT1')
In [ ]:
solutions = algorithm.get_result()

print(f' VARIABLES {solutions[1].variables} \n OBJECTIVES {solutions[1].objectives}  \n CONSTRAINTS {solutions[1].constraints}')

7. Naive Sample Model submission

  • You can define a new agent using the control parameters from your optimization algorithm
  • In the agent folder, name ur new agent e.g RuleOptimizedPSO
  • Copy the best control variables found by ur algorithm and paste them in them as a variable in the start inside your rbPSO_policy
  • An example is shown below
  • You can then define this agent as ur agent in the "user_agent.py" file
  • Go ahead and submit

Note: You can aslo use ur algorithm to find better parameters for rbc_policy (Try it )

```import numpy as np

def rbPSO_policy(observation, action_space): """ rule based 24 hour PSO policy """

Actions24 = [-0.13273342 -0.43600161 -0.91979483 -0.94664411 -0.86072348 -0.77316203
            -0.17414943 -0.11285137 -0.86327844  0.00431361  0.02807022 -0.21779452
            -0.96210762  0.19775465  0.12472641  0.06825766 -0.05606408 -0.09025377
            -0.86401095 -0.61281611 -0.51690614 -0.65854083 -0.08919786 -0.76346844]
hour = Observation[2] # Hour index is 2 for all observations

action = 0.0 # Default value
# for time, act in enumerate(range(actions)):

#     # time plus one because hour starts from 0
#     if hour == time + 1:
action = Actions24[hour-1]

action = np.array([action], dtype=float)

action = np.array([action], dtype=action_space.dtype)
assert action_space.contains(action)
return action

class RuleOptimizedPSO:

def __init__(self):
    self.action_space = {}

def set_action_space(self, agent_id, action_space):
    self.action_space[agent_id] = action_space

def compute_action(self, observation, agent_id):
    """Get observation return action"""
    return rbPSO_policy(observation, self.action_space[agent_id])

```

Comments

You must login before you can post a comment.

Execute