[ANNOUNCEMENT] Attend NeurIPS Competition Track: The CityLearn Challenge 2023 Workshop

Join us when we celebrate the winners of the CityLearn Challenge 2023. How can Buildings & AI provide resilience and support to the grid?

We received 2500+ submission from 600+ participants (100+ teams) competing in an energy forecasting challenge (organized by University of Cambridge) and a resilient demand response challenge (organized by Intelligent Environments Laboratory). More about the challenge is here.

Join us virtually at our #NeurIPS23 Workshop on Dec 15, 2023 at 10am US Central Time to hear from the top 3 solutions in each track.

Big thanks to our sponsors Mitsubishi Electric Research Laboratories and Climate Change AI

View post 0

My submission failed, how do I debug?

@dipam please, can you help out with this?

View post 0

Discussion about how to find data about Neurips 2023 citylearn

@md_abu_bakar_chowdhu please, refer to the further reading list on the competition page for information about data and related papers.

View post 0

[BUG FIX] Update your environment to CityLearn==2.1b12

TL;DR

• To apply the bug fixes, update your local evaluation requirements.txt file to use CityLearn==2.1b12 and also pip install CityLearn==2.1b12 in your local evaluation environment.
• The online evaluator has been updated to use CityLearn==2.1b12 .
• All submissions to Phase 2 will be rerun with CityLearn==2.1b12 to update the leaderboard by 2023-10-23.
• The bug fixes will affect results for control track but not forecast track. However, both tracks participants are advised to make the update.

Summary of bug fixes

We recently made some bug fixes that will affect the KPI values of any Phase 2 local or online evaluation using CityLearn<=2.1b11 in the control track only. These bug fixes are:

1. A bug introduced when transitioning from CityLearn==2.0.0 to CityLearn==2.1b1 affected how the dhw_storage and electrical_storage were charged. The round_trip_efficiency was being multiplied by both energy_init and energy instead of the latter alone. This caused the charged/discharged energy to be smaller than required which will affect net_electricity_consumption in the buildings. This was the original change that caused the bug and this was the change in CityLearn==2.1b10 that fixed the bug. Thanks to @rl11 for spotting this bug.

2. A bug introduced when transitioning from CityLearn==2.1b9 to CityLearn==2.1b10 affected how POWER_OUTAGE_NORMALIZED_UNSERVED_ENERGY_TOTAL was calculated. In an attempt to avoid a zero division error when calculating this KPI for simulations without a power outage, a bug was introduced that considered unserved energy during both outage and non-outage time steps instead of the former alone. This caused very low KPI values thus, affecting the average score of all KPIs. This was the original change in CityLearn==2.1b10 that caused the bug and this was the change in CityLearn==2.1b12 that fixed the bug. Thanks to @callum for spotting both the zero division and low value bugs.

3. An old bug introduced when transitioning from CityLearn==1.1.0 to CityLearn==1.2.0 where cooling_storage, heating_storage and dhw_storage were causing slightly overestimated electricity consumption when the energy to be added to any of these storage systems as a result of a charge action exceeded the capacity. Internally, the storage system models accounted for the excess energy however, the cooling_device, heating_device and dhw_device used to charge these storage systems used the energy before adjustments for capacity limits to calculate their own electricity consumption. Note that this bug did not affect electrical_storage. This was the pull request that fixed the bug. Thanks to @skywuuuu for leading me to back calculate net_electricity_consumption that led to finding this old issue.

What you should do

• To apply the bug fixes, update your local evaluation requirements.txt file to use CityLearn==2.1b12 and also pip install CityLearn==2.1b12 in your local evaluation environment. The online evaluator has been updated to use this new version.
• All submissions to Phase 2 will be rerun with CityLearn==2.1b12 to update the leaderboard by 2023-10-23.

View post 1

Observations data structure and indexing

@Chemago the reason all building observations are in one list is because the control is set to use a central agent i.e. one controller manages all the heat pumps, dhw storage systems and batteries in all buildings. If you want to get serve the observations in a list of lists, you will need to set central_agent=False (you can do this in the schema.json for local evaluation and see this post for online evaluation). Alternatively, you can obtain building level observations by calling env.buildings[0].observations().

View post 1

Getting building level features

@Chemago you can get carbon intensity using env.buildings[0].carbon_intensity.carbon_intensity. This time series is the same for all buildings and is in unit of kgco2e/kwh. Alternatively, you can use env.buildings[0].net_electricity_consumption_emission which differs across buildings and is in units of kgco2e.

Vintage and Area are not stored anywhere in the environment. For the others:

• Heat pump (kW): env.buildings[0].cooling_device.nominal_power
• Heater (kW): env.buildings[0].dhw_device.nominal_power
• DHW storage (kWh): env.buildings[0].dhw_storage.capacity
• Battery (kWh): env.buildings[0].electrical_storage.capacity
• PV (kW): env.buildings[0].pv.nominal_power

View post 0

JSON aicrowd file central agent

duplicate discussion here How to setup central_agent to false in the evaluation phase?

View post 0

Control track submissions failing

This should be fixed now @francescomorri

View post 0

Stochastic definition (re: Outages during Phase II)

Thanks for the questions @donnavakalis and sorry for the late response. The stochastic outages will be the latter i.e., hidden but fixed and the same for everyone throughout Phase II. However, we are introducing a private leaderboard at the end of Phase II that uses another set of random seed. We will provide more information about this in the next couple of days when Phase II commences.

View post 0

CityLearn Forecasting Track - Error Building Environment

CityLearn itself is best used with Python 3.7. It might install with later versions but not without some dependency issues from experience. I also think the AICrowd online evaluator is run in a 3.7 environment as well. So yeah, your packages should be compatible with 3.7. Also, apologies if it wasn’t properly documented!

View post 0

CityLearn Forecasting Track - Error Building Environment

@phil_aupke did you specify a version number for lightgbm? If you did, make sure it is distributed for Python 3.7 since the online evaluator uses this specific Python version.

View post 0

Clarifications needed, also potential bug?

It has been resolved. See this post. The competition pages have also been updated with current information about weights and KPIs.

View post 0

Clarifications needed, also potential bug?

@rl11 thanks for raising all these important points. Here are my responses:

1. The competition page is outdated and needs to be updated with the correct information especially information about the weights control KPIs. The correct weights are what is in the code. We will work on updating the competition page (@snehananavati please, can you update the pages)?

2. Weather will be the same in final evaluation but 3 other buildings are added. This ‘final’ evaluation is already taking place. It is the ‘CityLearn Private’ you see in your submission which you cannot see the scores for. LSTMs don’t change at any point. Most of these differences are highlighted in this post.

3. There is no heating at any point in the competition.

4. This is a very good catch, thank you! I will check it out ASAP.

View post 0

Accessing carbon intensity at a given point in time

answered here.

View post 0

About control track submission

@dipam please, can you help out with this?

View post 0

Request for Extended Runtime in Phase 2

Yeah no worries I see what you mean. That’s one way to look at it. just remember you are double counting the public 3 that way since the same public 3 buildings are used in the public and private evaluations.

View post 0

Request for Extended Runtime in Phase 2

Hi @reggie, it is 6 buildings not 9 in Phase private evaluation :). Maybe if runtime is an issue we can extend a bit. Please, any thoughts on this @dipam?

View post 0

Role of indoor temp and rh and setpoint variables in csv files

Hi @donnavakalis your understanding is correct about (1),

Historically, CityLearn could not control the heat pump and there were no temperature dynamics. Instead, there was the assumption that the ideal cooling/heating loads were always met while using the storage systems to provide load shifting flexibility. In this previous setup, the indoor temperature in the csv files were useful as they were static and didn’t change during runtime. RH values have always been included in the csv files in case the modeler found it useful as an observation but it doesn’t actually get used during simulation so it can be ignored all together.

The updated env we are using this year allows the heat pump to be controlled and can predict the temperature change based on how much energy the heat pump delivers. So the indoor temp at runtime changes from what is in the csv files. However, one might choose to use the dataset in the old set up or the new setup which is why the values are still populated in the csv. Also, the LSTM that does the temperature prediction has a lookback of 12 time steps so the very first 12 time steps of an episodes will use the data in the csv to infer indoor temp.

Regarding your second question, the setpoint in the csv is actually useful in two ways. First, it can be used as an observation to the agent. Secondly, it actually is considered when calculating delta for comfort. See here where it is used to calculate the temperature delta observation, here where it is used to calculate the comfort bands in reward function and here where it is used to calculate delta in the discomfort evaluation metric.

View post 1

How to setup central_agent to false in the evaluation phase?

@dipam please can you respond to this? I think there are a couple other people asking to be able to set central_agent to True/False.

View post 0

How to load a trained model during AIcrowd evaluation?

@dipam please, can you respond to this?

View post 0