Note

You can download this example as a Jupyter notebook or try it out directly in Google Colab.

5. Altering market configuration: Effects of a Long Term Market

This tutorial describes a showcase from the scientific paper “Market Abstraction of Energy Markets and Policies - Application in an Agent-Based Modeling Toolbox”.

In this publication the market configuration is presented which allows to configure different markets and create comparative simulations as the following.

We will describe how one can add a new market to the scenario and create a small comparison study of the results using matplotlib.

As a whole, this tutorial covers the following

1. run a small example with a single market

2. look at a market config and adjust it to include a second market as well

3. run the example with the added long term market

4. extract graphs from the simulation run and interpret results

1. Basics

In general, most simulation studies only focus on single market scenarios. Yet, the reality includes a lot more than that.

To showcase that ASSUME can handle scenarios with multiple markets as well, a scenario is created where a long term bidding of 7 days takes place before the day-ahead market. The pricing mechanism for this long-term market is pay-as-bid while the day ahead market uses uniform pricing/pay-as-clear.

To keep things simple, the agents always bid their marginal cost. For the Long-Term market, this leads to unrealistic economics, as none of the agents ever receives more than their marginal cost. But on the other hand this makes the given scenario much easier to understand.

2. Get ASSUME running

Here we just install the ASSUME core package via pip - just as we did in the other tutorials. In general the instructions for an installation can be found here: https://assume.readthedocs.io/en/latest/installation.html. All the required steps are executed here and since we are working in colab the generation of a venv is not necessary.

!pip install assume-framework

If we run in Google Colab, we need to first clone the ASSUME repository there to access the tutorial data

!git clone https://github.com/assume-framework/assume.git

And easy like this we have ASSUME installed. Now we can let it run. Please note though that we cannot use the functionalities tied to docker and, hence, cannot access the predefined dashboards in colab. For this please install docker and ASSUME on your personal machine.

To run the examples, we still need some packages imports and configure a database server URI - you can adjust this if needed

from functools import partial
import matplotlib.pyplot as plt
import pandas as pd
from sqlalchemy import create_engine

from assume import World
from assume.scenario.loader_csv import load_scenario_folder

# make sure that you have a database server up and running - preferabely in docker
# DB_URI = "postgresql://assume:assume@localhost:5432/assume"
# but you can use a file-based sqlite database too:
DB_URI = "sqlite:///./examples/local_db/assume_db_example_02d.db"

import yaml

# lets look at the scenarios we have:
config_file = "assume/examples/inputs/example_02d/config.yaml"
config_file = "examples/inputs/example_02d/config.yaml"
with open(str(config_file), "r") as f:
    config = yaml.safe_load(f)

# we have two scenarios available
list(config.keys())

While the market configurations are given in the config.yaml, the agent configurations and their bidding strategies are defined in the csv files. We do not want to go too deep into the details in this tutorial, so we selected a rather simple market participant strategy, which is suitable for both markets:

The actual behavior of the agents in this scenario is the following:

1. on opening of a new market, find the maximum power which can be provided throughout the whole product period (1h for EOM, 7 days for LTM)

2. calculate the marginal cost in €/MWh to provide this amount of energy

3. bid the maximum power and the marginal cost to the market

This is a behavior which works similarily well for both markets, though the results can not be taken to reality for various reasons.

But lets look at the scenarios themselves:

3. First Market

Now we can run the first scenario which does only include the energy-only market, we call this “eom_case”

# let us take a look at the configuration we are about to run:
config["eom_case"]

We do have a single market called EOM which is configured with pay_as_clear clearing and has a hourly clearing schedule. So lets look at the products which can be traded on this market:

config["eom_case"]["markets_config"]["EOM"]["products"]

We only have a single market product which has a duration of one hour. And every time the market opens, only the next 1 hour can be traded (see count). The first delivery of the market is one hour after the opening of the market (to have some spare time before delivery).

For more information on the market configuration and an example gantt chart, look here: https://assume.readthedocs.io/en/latest/market_config.html

So lets run this:

world = World(database_uri=DB_URI)
load_scenario_folder(
    world,
    inputs_path="assume/examples/inputs",
    scenario="example_02d",
    study_case="eom_case",
)
world.run()

This is the same as using the following CLI tool call in colab:

!cd assume && assume -s example_02d -c eom_case -db "sqlite:///./examples/local_db/assume_db_example_02d.db"

4. Long-Term Market

Now we can run the second case which also includes the long term market as can be seen in the config.yaml of it:

config["ltm_case"]

In this scenario we added a second market called LTM with a pay_as_bid clearing. Take note of the second entry in the markets_config.

It has a start time which is one hour earlier than the EOM, so that bidding on the LTM is before selling the load on the EOM - just as it is in reality.

Taking a look at the products of this market shows the current configuration:

config["ltm_case"]["markets_config"]["LTM_OTC"]["products"]

We can see, that a product is available for a whole 7 days and also only trades the next 7d. To have some more spare time before delivery, we give a first_delivery of 2h.

Now lets run this example

world = World(database_uri=DB_URI)
load_scenario_folder(
    world,
    inputs_path="examples/inputs",
    scenario="example_02d",
    study_case="ltm_case",
)
world.run()

Or use the CLI as shown:

!cd assume && assume -s example_02d -c ltm_case -db "sqlite:///./examples/local_db/assume_db_example_02d.db"

3. Visualize the results

We can visualize the results using the following functions

engine = create_engine(DB_URI)

sql = """
SELECT ident, simulation,
sum(round(CAST(value AS numeric), 2))  FILTER (WHERE variable = 'total_cost') as total_cost,
sum(round(CAST(value AS numeric), 2)*1000)  FILTER (WHERE variable = 'total_volume') as total_volume,
sum(round(CAST(value AS numeric), 2))  FILTER (WHERE variable = 'avg_price') as average_cost
FROM kpis
where variable in ('total_cost', 'total_volume', 'avg_price')
and simulation in ('example_02d_eom_case', 'example_02d_ltm_case')
group by simulation, ident ORDER BY simulation
"""
kpis = pd.read_sql(sql, engine)
kpis["total_volume"] /= 1e9
kpis["total_cost"] /= 1e6
savefig = partial(plt.savefig, transparent=False, bbox_inches="tight")

## Data preparation
eom = kpis[kpis["ident"] == "EOM"]
ltm = kpis[kpis["ident"] == "LTM_OTC"].reset_index()
# ltm.loc[0, "average_cost"] = None
xticks = list(eom["simulation"])
# xlabels = [f"{i}%" for i in range(0, 101, 10)]
xlabels = ["EOM", "EOM + LTM"]
plt.style.use("seaborn-v0_8")

fig, (ax1, ax2) = plt.subplots(2, 1)
# Total Dispatch cost
ax1.bar(eom["simulation"], eom["total_cost"], label="EOM")
eom_ltm = eom[eom.simulation == "ltm_case10"]
ax1.bar(
    ltm["simulation"],
    ltm["total_cost"],
    bottom=eom_ltm["total_cost"],
    label="LTM",
)
ax1.set_ylabel("Total dispatch cost \n per market [mill. $€$]")
ax1.set_xticks(xticks, xlabels)
ax1.legend()
# Total Average Cost
ax2.scatter(eom["simulation"], eom["average_cost"], label="EOM")
ax2.scatter(ltm["simulation"], ltm["average_cost"], label="LTM")
ax2.bar(eom["simulation"], eom["total_cost"] * 0)
ax2.set_ylabel("Average cost \n for each scenario [$€/MWh$]")
# ax2.set_xlabel("Fraction of base load traded on LTM in percent")
ax2.set_xlabel("Selected electricity market design")
ax2.set_xticks(xticks, xlabels)
ax2.legend()
savefig("overview-cost.png")
plt.show()

In the first plot, we can see, that the cost of the Energy only market is much higher, due to the used pay_as_clear market clearing. While the LTM scenario only bids a fraction of the energy on the LTM - and sells on the LTM for much less, due to the pay_as_bid clearing.

This is not very smart of those agents, but lets you see how agents can interact on multiple markets at once.

Now we create the second plot, showing the actual dispatch per market for the LTM scenario:

# second plot
sql = """
SELECT
"datetime" as "time",
sum(power) AS "market_dispatch",
market_id,
um.technology
FROM market_dispatch md
join power_plant_meta um on um."index" = md.unit_id and um.simulation = md.simulation
WHERE
md.simulation = 'example_02d_ltm_case'
GROUP BY 1, market_id, technology
ORDER BY technology, market_id desc, 1
"""

df = pd.read_sql(sql, engine, index_col="time")
# fig, ax = plt.subplots(figsize=(8,6))
series = []
for label, sub_df in df.groupby(["market_id", "technology"]):
    lab = "-".join(label)
    lab = lab.replace("LTM_OTC", "LTM")

    if "lignite" not in lab and "nuclear" not in lab:
        continue
    group_sum = sub_df.market_dispatch.groupby("time").sum()
    group_sum.name = lab
    series.append(group_sum.resample("1h").ffill())

ddf = pd.DataFrame(series)
ddf = ddf.T.fillna(method="ffill")

ddf = ddf[sorted(ddf.columns, reverse=True)]
ddf = ddf.fillna(0)
ddf /= 1e3
base = ddf[ddf.columns[0]] * 0
for col in ddf.columns:
    line = base + ddf[col]
    c = (0.3, 0.2, 0.6, 0.8) if "nuclear" in col else "g"
    alpha = 0.8 if "LTM" in col else 0.6
    plt.fill_between(line.index, line, base, alpha=alpha, label=col, color=c)
    base += ddf[col]
plt.ylabel("Hourly dispatch power [$GW$]")
plt.xlabel("Datetime")
plt.xticks(rotation=25)
plt.legend()
savefig("overview-dispatch.png")
plt.show()

Here one can see that the dispatch stays about the same, but a large amount has been traded as a base band on the LTM while the daily variances are traded on the EOM.

This also brings us to the end of this short tutorial on multiple markets.