Note
You can download this example as a Jupyter notebook or try it out directly in Google Colab.
6. Using advanced orders: Effects of regular block orders and linked orders#
This tutorial showcases the impact of regular block and linked orders in electricity market simulation outcomes.
In this example the advanced strategies using regular block orders and linked orders are presented and their impact on the market outcome both on system and unit level compared to strategies using only single hourly orders.
With the integration of block orders, minimum acceptance ratios are added to the orders as an additional field. To account for those in the clearing, a new market clearing algorithm becomes necessary.
In this tutorial, we will show how to create and integrate this advanced market clearing and adjust bidding strategies to allow for the use of regular block orders and linked orders. Finally, we will create a small comparison study of the results using matplotlib.
As a whole, this tutorial covers the following
Explain the basic rules of block and linked orders.
Run a small example with single hourly orders.
Create the new market clearing algorithm.
Adjust a given strategy to integrate block orders.
Adjust a given strategy to integrate linked orders.
Extract graphs from the simulation run and interpret results.
1. Basics#
In general, most simulation studies only focus on single hourly orders. Yet, the reality includes a lot more than that, especially different types of block orders. The question is, how much do the advanced order types deviate the market outcome.
To showcase that ASSUME can handle different order types an advanced market clearing algorithm is created and tested with three different strategies. The testing scenarios are defined as 12 units (one for each technology) and an inflexible demand, cleared on a day-ahead market.
What are single hourly orders?
These are the simplest order structures implemented, also called simple orders. They can take the form of linear piece-wise curves containing interpolated orders only, step-wise curves containing step orders only, or a hybrid form of both. The general clearing of these orders strictly follows the merit order principle: any order with a bidding price below the MCP must be fully accepted, any order with a price above the MCP must be rejected, and orders where the price equals the MCP can be either accepted (fully/partially) or rejected.
What are block orders?
The characteristic of block orders is a time horizon that spans multiple minimum time units (MTUs). They are defined by a price, a number of time periods, some volumes that can be different for each period, and a constant minimum acceptance ratio (MAR). The MAR defines a restriction on how much a bid can be curtailed before it must be rejected. They can be divided into four subcategories:
Independent block orders (BOs): The elemental form is the regular block order, with constant volume and a MAR equal to 1, also referred to as “fill-or-kill”-condition. This determines that the volume must be entirely accepted or rejected. This BO-type is used most frequently and applied in all European power exchanges Devnath et al. 2020 and implemented in most models Tanrisever et al. 2020. Slight deviations are given with the curtailable BO, where the MAR can be below 1, and the profile BO, which includes volume changes over the block order periods.
Linked groups or linked orders (LOs): These groups of orders include parent and dependent child orders with the additional condition that the acceptance ratio of a parent order must be greater or equal to those of all child orders. During market clearing, children are considered, if they increase the profit. Hence, they can “save” the parent order, but not vice versa. There can be several levels of LOs in one group, but the number of children and levels is constrained by the power exchanges.
Exclusive groups: For this order type, the sum of acceptance ratios of a set of block orders must be below 1. If the MAR is set to 1, this results in an exclusive-or-relation of all orders in that group.
Flexible orders: These order structures are created through the formation of an exclusive group where the MAR is set to 1 and each order is shifted by one hour, introducing flexibility through the dispatch time determined by the algorithm, not predefined by the participant.
In this tutorial, we will compare the simple hourly orders with regular block orders and linked orders.
Bid formulation in this example
According to flexABLE, the inflexible and flexible power of a unit is bid separately, compare Qussous et al. 2022.
The inflexible power \(P^{\mathrm{inflex}}_{t}\) at time \(t\) is the minimum volume that can be dispatched. It is defined by the current operation status of the unit, ramp-down limitations and the must-run time. The inflexible bid price depends on the marginal cost \(C^{\mathrm{marginal}}_t\) at time \(t\) and the power dispatch of the previous time step \(P^{\mathrm{dispatch}}_{t-1}\) and adds a markup, if the unit has to be started newly and a reduction, to prevent a shut-down, including the start-up costs \(C^{\mathrm{su}}_t\). Here, the average time of continuous operation is given by \(T^{\mathrm{op, avg}}\) and the average time of continuous shut down is given by \(T^{\mathrm{down, avg}}\): \begin{align} C^{\mathrm{inflex}}_t=C^{\mathrm{marginal}}_t + \frac{C^{\mathrm{su}}_t}{P^{\mathrm{inflex}}_{t} T^{\mathrm{op/down}}} \quad \mathrm{with} \: T^{\mathrm{op/down}} = \begin{cases} -T^{\mathrm{down, avg}}, & \mathrm{if} \: P^{\mathrm{dispatch}}_{t-1} > 0 \\ T^{\mathrm{op, avg}}, & \mathrm{otherwise} \end{cases} \end{align}
The flexible power \(P^{\mathrm{flex}}_{t}\) at time \(t\) is then the difference between maximum dispatchable volume and the inflexible power. It is defined by current operation status of the unit, ramp-up limitations and the must-operation time. The flexible bidding price is given by the marginal costs only:
\(C^{\mathrm{flex}}_t=C^{\mathrm{marginal}}_t\)
When transforming those bids into block orders, the volumes of the inflexible bids build the profile of one block bid over 24 hours. Because block orders are cleared according to the average market clearing price over the order period \(\mathcal{T}\), the price of the block order is given by the weighted average of the inflexible bid price:
\(C^{\mathrm{block}} = \frac{\sum_{t \in \mathcal{T}} C^{\mathrm{inflex}}_t \: P^{\mathrm{inflex}}_t}{\sum_{t \in \mathcal{T}} P^{\mathrm{inflex}}_t}\)
The linked orders are built by the flexible power for each hour linked as children to the inflexible block bid. They then use directly the flexible bid price \(C^{\mathrm{flex}}_t\).
2. Get ASSUME running#
Here we just install the ASSUME core package via pip - just as we did in the other tutorials. In general the instructions for an installation can be found here: https://assume.readthedocs.io/en/latest/installation.html. All the required steps are executed here and since we are working in colab the generation of a venv is not necessary.
In this case we will need the optional dependencies for our optimization to run, so we will install the assume[optimization] distribution.
[ ]:
import importlib.util
# Check if 'google.colab' is available
IN_COLAB = importlib.util.find_spec("google.colab") is not None
if IN_COLAB:
!pip install 'assume-framework'
If we run in Google Colab, we need to first clone the ASSUME repository there to access the tutorial data
[ ]:
if IN_COLAB:
!git clone --depth=1 https://github.com/assume-framework/assume.git assume-repo
Select input files path:
We also need to differentiate between the input file paths when using this tutorial in Google Colab and a local environment. The code snippets will include both options for your convenience.
[ ]:
colab_inputs_path = "assume-repo/examples/inputs"
local_inputs_path = "../inputs"
inputs_path = colab_inputs_path if IN_COLAB else local_inputs_path
And easy like this we have ASSUME installed. Now it is ready to run. Please note though that we cannot use the functionalities tied to docker and, hence, cannot access the predefined dashboards in colab. For this please install docker and ASSUME on your personal machine.
To run the examples, we still need some packages imports and configure a database server URI - you can adjust this if needed
[ ]:
import pandas as pd
from assume import World
from assume.scenario.loader_csv import load_scenario_folder
# make sure that you have a database server up and running - preferabely in docker
# DB_URI = "postgresql://assume:assume@localhost:5432/assume"
# but you can use a file-based sqlite database too:
DB_URI = "sqlite:///local_db/assume_db.db"
3. Market clearing algorithm#
To integrate block and linked orders, the market clearing becomes a mixed-integer linear problem (MILP). In addition to the volumes and prices, we now need to know the bid type and minimum acceptance ratio for all orders and the parent bid id in case it is a linked bid.
Those additional fields then have to be added in the market clearing: * “bid_type” defines the order structure and can be “SB” for single hourly orders (Simple Bid), “BB” for block orders (Block Bid) or “LB” for linked orders (Linked Bid). * “min_acceptance_ratio” defines how much a bid can be curtailed before it is rejected. If it is set to 1, the bid is either accepted or rejected with it’s full volume. * “parent_bid_id” is needed to include linked bids. Here the id of the parent order is defined, where the child order is linked to. The market clearing algorithm then ensures that the minimum acceptance ratio of the child order is less or equal to the one of its parent order.
First a few imports to use existing functions we do not change:
[ ]:
import logging
from operator import itemgetter
import pyomo.environ as pyo
from pyomo.opt import SolverFactory, TerminationCondition
from assume.common.market_objects import MarketConfig, Orderbook
from assume.markets.clearing_algorithms.complex_clearing import (
ComplexClearingRole,
calculate_order_surplus,
extract_results,
)
from assume.strategies.flexable import flexableEOM
log = logging.getLogger(__name__)
First, we specify the optimization problem as an MILP.
Read the comments in the following function and understand what is happening in the code. Refer to the complex clearing mechanism in the documentation for a detailed description of the optimization problem.
[ ]:
EPS = 1e-4
def market_clearing_opt(
orders: Orderbook,
market_products,
mode,
with_linked_bids,
incidence_matrix: pd.DataFrame = None,
lines: pd.DataFrame = None,
solver: str = "appsi_highs",
solver_options: dict = {},
):
"""
Sets up and solves the market clearing optimization problem.
"""
# Set nodes and lines based on the incidence matrix and lines DataFrame
if incidence_matrix is not None:
nodes = list(incidence_matrix.index)
line_ids = list(incidence_matrix.columns)
else:
nodes = ["node0"]
line_ids = ["line0"]
model = pyo.ConcreteModel()
# add dual suffix to the model (we need this to extract the market clearing prices later)
# if mode is not 'with_min_acceptance_ratio', otherwise the dual suffix is added later
if mode != "with_min_acceptance_ratio":
model.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)
model.T = pyo.Set(
initialize=[market_product[0] for market_product in market_products],
doc="timesteps",
)
model.sBids = pyo.Set(
initialize=[order["bid_id"] for order in orders if order["bid_type"] == "SB"],
doc="simple_bids",
)
model.bBids = pyo.Set(
initialize=[
order["bid_id"] for order in orders if order["bid_type"] in ["BB", "LB"]
],
doc="block_bids",
)
model.nodes = pyo.Set(initialize=nodes, doc="nodes")
model.lines = pyo.Set(initialize=line_ids, doc="lines")
# decision variables for the acceptance ratio of simple and block bids (including linked bids)
model.xs = pyo.Var(
model.sBids,
domain=pyo.NonNegativeReals,
bounds=(0, 1),
doc="simple_bid_acceptance",
)
model.xb = pyo.Var(
model.bBids,
domain=pyo.NonNegativeReals,
bounds=(0, 1),
doc="block_bid_acceptance",
)
# decision variables that define flows between nodes
# assuming the orders contain the node and are collected in nodes
if incidence_matrix is not None:
# Decision variables for flows on each line at each timestep
model.flows = pyo.Var(model.T, model.lines, domain=pyo.Reals, doc="power_flows")
if mode == "with_min_acceptance_ratio":
model.Bids = pyo.Set(
initialize=[order["bid_id"] for order in orders], doc="all_bids"
)
# decision variables for the acceptance as binary variable
model.x = pyo.Var(
model.Bids,
domain=pyo.Binary,
doc="bid_accepted",
)
# add minimum acceptance ratio constraints
model.mar_constr = pyo.ConstraintList()
for order in orders:
if order["min_acceptance_ratio"] is None:
continue
elif order["bid_type"] == "SB":
model.mar_constr.add(
model.xs[order["bid_id"]]
>= order["min_acceptance_ratio"] * model.x[order["bid_id"]]
)
model.mar_constr.add(
model.xs[order["bid_id"]] <= model.x[order["bid_id"]]
)
elif order["bid_type"] in ["BB", "LB"]:
model.mar_constr.add(
model.xb[order["bid_id"]]
>= order["min_acceptance_ratio"] * model.x[order["bid_id"]]
)
model.mar_constr.add(
model.xb[order["bid_id"]] <= model.x[order["bid_id"]]
)
# limit the acceptance of child bids by the acceptance of their parent bid
if with_linked_bids:
model.linked_bid_constr = pyo.ConstraintList()
for order in orders:
if "parent_bid_id" in order.keys() and order["parent_bid_id"] is not None:
parent_bid_id = order["parent_bid_id"]
model.linked_bid_constr.add(
model.xb[order["bid_id"]] <= model.xb[parent_bid_id]
)
# Function to calculate the balance for each node and time
def energy_balance_rule(model, node, t):
"""
Calculate the energy balance for a given node and time.
This function calculates the energy balance for a specific node and time in a complex clearing algorithm. It iterates over the orders and adjusts the balance expression based on the bid type. It also adjusts the flow subtraction to account for actual connections if an incidence matrix is provided.
Args:
model: The complex clearing model.
node: The node for which to calculate the energy balance.
t: The time for which to calculate the energy balance.
Returns:
bool: True if the energy balance is zero, False otherwise.
"""
balance_expr = 0.0 # Initialize the balance expression
# Iterate over orders to adjust the balance expression based on bid type
for order in orders:
if (
order["bid_type"] == "SB"
and order["node"] == node
and order["start_time"] == t
):
balance_expr += order["volume"] * model.xs[order["bid_id"]]
elif order["bid_type"] in ["BB", "LB"] and order["node"] == node:
for start_time, volume in order["volume"].items():
if start_time == t:
balance_expr += volume * model.xb[order["bid_id"]]
# Add contributions from line flows based on the incidence matrix
if incidence_matrix is not None:
for line in model.lines:
incidence_value = incidence_matrix.loc[node, line]
if incidence_value != 0:
balance_expr += incidence_value * model.flows[t, line]
return balance_expr == 0
# Add the energy balance constraints for each node and time period using the rule
# Define the energy balance constraint using two indices (node and time)
model.energy_balance = pyo.Constraint(
model.nodes, model.T, rule=energy_balance_rule
)
if incidence_matrix is not None:
model.transmission_constr = pyo.ConstraintList()
for t in model.T:
for line in model.lines:
capacity = lines.at[line, "s_nom"]
# Limit the flow on each line
model.transmission_constr.add(model.flows[t, line] <= capacity)
model.transmission_constr.add(model.flows[t, line] >= -capacity)
# define the objective function as cost minimization
obj_expr = 0
for order in orders:
if order["bid_type"] == "SB":
obj_expr += order["price"] * order["volume"] * model.xs[order["bid_id"]]
elif order["bid_type"] in ["BB", "LB"]:
for start_time, volume in order["volume"].items():
obj_expr += order["price"] * volume * model.xb[order["bid_id"]]
model.objective = pyo.Objective(expr=obj_expr, sense=pyo.minimize)
solver = SolverFactory(solver)
# Solve the model
instance = model.create_instance()
results = solver.solve(instance, options=solver_options)
# Fix all model.x to the values in the solution
if mode == "with_min_acceptance_ratio":
# Add dual suffix to the model (needed to extract duals later)
instance.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)
for bid_id in instance.Bids:
# Fix the binary variable to its value
instance.x[bid_id].fix(instance.x[bid_id].value)
# Change the domain to Reals (or appropriate continuous domain)
instance.x[bid_id].domain = pyo.Reals
# Resolve the model
results = solver.solve(instance, options=solver_options)
return instance, results
So this function defines how the objective is solved. Let’s create the market clearing algorithm as a MarketRole inheriting from the ComplexClearingRole in the ASSUME framework.
First, we define the class ComplexClearingRole and initiate it. Then, we specify the main function to clear the market using the function market_clearing_opt() to calculate the market outcome as optimization:
[ ]:
class AdvancedClearingRole(ComplexClearingRole):
# here we need to define the additionally required fields
# but because the minimum acceptance ratio and the parent id has to be specified only for some orders,
# we only use bid_type as required field
required_fields = ["bid_type"]
def __init__(self, marketconfig: MarketConfig):
super().__init__(marketconfig)
def validate_orderbook(self, orderbook: Orderbook, agent_addr) -> None:
"""
Checks whether the bid types are valid and whether the volumes are within the maximum bid volume.
Args:
orderbook (Orderbook): The orderbook to be validated.
agent_addr (AgentAddress): The agent address of the market.
Raises:
ValueError: If the bid type is invalid.
"""
market_id = self.marketconfig.market_id
max_volume = self.marketconfig.maximum_bid_volume
for order in orderbook:
# if bid_type is None, set to default bid_type
if order["bid_type"] is None:
order["bid_type"] = "SB"
# Validate bid_type
elif order["bid_type"] not in ["SB", "BB", "LB"]:
order["bid_type"] = "SB" # Set to default bid_type
super().validate_orderbook(orderbook, agent_addr)
for order in orderbook:
# Validate volumes
if order["bid_type"] in ["BB", "LB"]:
for key, volume in order.get("volume", {}).items():
if abs(volume) > max_volume:
order["volume"][key] = max_volume if volume > 0 else -max_volume
# Node validation
node = order.get("node")
if node:
if self.zones_id:
node = self.node_to_zone.get(node, self.nodes[0])
order["node"] = node
if node not in self.nodes:
order["node"] = self.nodes[0]
else:
if self.incidence_matrix is not None:
order["node"] = self.nodes[0]
else:
order["node"] = "node0"
def clear(
self, orderbook: Orderbook, market_products
) -> tuple[Orderbook, Orderbook, list[dict]]:
"""
Implements pay-as-clear with more complex bid structures, including acceptance ratios, bid types, and profiled volumes.
Args:
orderbook (Orderbook): The orderbook to be cleared.
market_products (list[MarketProduct]): The products to be traded.
Raises:
Exception: If the problem is infeasible.
Returns:
accepted_orders (Orderbook): The accepted orders.
rejected_orders (Orderbook): The rejected orders.
meta (list[dict]): The market clearing results.
Notes:
First the market clearing is solved using the cost minimization with the pyomo model market_clearing_opt.
Then the market clearing prices are extracted from the solved model as dual variables of the energy balance constraint.
Next the surplus of each order and its children is calculated and orders with negative surplus are removed from the orderbook.
This is repeated until all orders remaining in the orderbook have positive surplus.
Optional additional fields are: min_acceptance_ratio, parent_bid_id, node
"""
if len(orderbook) == 0:
return [], [], []
orderbook.sort(key=itemgetter("start_time", "end_time", "only_hours"))
# create a list of all orders linked as child to a bid
child_orders = []
for order in orderbook:
order["accepted_price"] = {}
order["accepted_volume"] = {}
# get child linked bids
if "parent_bid_id" in order.keys() and order["parent_bid_id"] is not None:
# check whether the parent bid is in the orderbook
parent_bid_id = order["parent_bid_id"]
parent_bid = next(
(bid for bid in orderbook if bid["bid_id"] == parent_bid_id), None
)
if parent_bid is None:
order["parent_bid_id"] = None
else:
child_orders.append(order)
with_linked_bids = bool(child_orders)
rejected_orders: Orderbook = []
mode = "default"
if "min_acceptance_ratio" in self.marketconfig.additional_fields:
mode = "with_min_acceptance_ratio"
# solve the market clearing problem
while True:
# solve the optimization with the current orderbook
instance, results = market_clearing_opt(
orders=orderbook,
market_products=market_products,
mode=mode,
with_linked_bids=with_linked_bids,
incidence_matrix=self.incidence_matrix,
lines=self.lines,
solver=self.solver,
solver_options=self.solver_options,
)
if results.solver.termination_condition == TerminationCondition.infeasible:
raise Exception("infeasible")
# extract dual from model.energy_balance
market_clearing_prices = {}
for node in self.nodes:
market_clearing_prices[node] = {
t: instance.dual[instance.energy_balance[node, t]]
for t in instance.T
}
# check the surplus of each order and remove those with negative surplus
orders_surplus = []
for order in orderbook:
children = []
if with_linked_bids:
children = [
child
for child in child_orders
if child["parent_bid_id"] == order["bid_id"]
]
order_surplus = calculate_order_surplus(
order, market_clearing_prices, instance, children
)
# correct rounding
if order_surplus != 0 and abs(order_surplus) < EPS:
order_surplus = 0
orders_surplus.append(order_surplus)
# remove orders with negative profit
if order_surplus < 0:
rejected_orders.append(order)
orderbook.remove(order)
rejected_orders.extend(children)
for child in children:
orderbook.remove(child)
# check if all orders have positive surplus
if all(order_surplus >= 0 for order_surplus in orders_surplus):
break
log_flows = True
accepted_orders, rejected_orders, meta, flows = extract_results(
model=instance,
orders=orderbook,
rejected_orders=rejected_orders,
market_products=market_products,
market_clearing_prices=market_clearing_prices,
log_flows=log_flows,
)
self.all_orders = []
return accepted_orders, rejected_orders, meta, flows
So let’s add the advanced clearing algorithm to the possible clearing mechanisms in world and load the example.
[ ]:
world = World(database_uri=DB_URI)
# add the new clearing mechanism to the world
world.clearing_mechanisms["pay_as_clear_advanced"] = AdvancedClearingRole
# overwrite used the bidding strategy with the predefined strategy flexableEOM to use single hourly orders only
world.bidding_strategies["new_advanced_strategy"] = flexableEOM
load_scenario_folder(
world,
inputs_path=inputs_path,
scenario="example_01g",
study_case="sho_case",
)
world.markets["EOM"]
In the market config we have 24 single market products which have a duration of one hour. And every time the market opens, the next 24 hours can be traded (see count). The first delivery of the market is 24 hours after the opening of the market (to have some spare time before delivery).
The market configuration does not change over the different scenarios, because we use the same market setting as day-ahead market (dam) with all additional fields required for block and linked orders. We see that the market mechanism is our newly integrated class, named “pay_as_clear_advanced”.
To check the used bidding strategy, we can access the unit over the unit operator in world, for example the hard coal unit:
[ ]:
world.unit_operators["coal_unit_operator"].units["coal_unit"].bidding_strategies["EOM"]
Now we run the actual simulation with:
[ ]:
world.run()
This is our baseline without any advanced order types, but using the newly created clearing mechanism. Next, we take a look at the block orders.
4. Block orders#
Now we can create a new strategy, which transforms the inflexible bids into one block order over the whole 24 hour-clearing horizon. For this we copy the flexable strategy and modify it where necessary (marked with # ====== new).
Again, first we import functions, which we do not change:
[ ]:
from assume.common.base import SupportsMinMax
from assume.common.market_objects import Orderbook, Product
from assume.strategies.flexable import (
calculate_EOM_price_if_off,
calculate_EOM_price_if_on,
)
[ ]:
class blockStrategy(flexableEOM):
"""
A strategy that bids on the EOM-market with block bids.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# check if kwargs contains eom_foresight argument
self.foresight = pd.Timedelta(kwargs.get("eom_foresight", "12h"))
def calculate_bids(
self,
unit: SupportsMinMax,
market_config: MarketConfig,
product_tuples: list[Product],
**kwargs,
) -> Orderbook:
"""
Calculates block bids for the EOM-market and returns a list of bids consisting of the start time, end time, only hours, price, volume and bid type.
The bids take the following form:
One block bid with the minimum acceptance ratio set to 1 spanning the total clearing period.
It uses the inflexible power and the weighted average price of the inflexible power as the price.
This price is based on the marginal cost of the inflexible power and the starting costs.
The starting costs are split across inflexible power and the average operation or down time of the unit depending on the operation status before.
Additionally, for every hour where the unit is on, a separate flexible bid is created using the flexible power and marginal costs as bidding price.
Args:
unit (SupportsMinMax): A unit that the unit operator manages.
market_config (MarketConfig): A market configuration.
product_tuples (list[Product]): A list of tuples containing the start and end time of each product.
kwargs (dict): Additional arguments.
Returns:
Orderbook: A list of bids.
"""
start = product_tuples[0][0]
end = product_tuples[-1][1]
previous_power = unit.get_output_before(start)
min_power_values, max_power_values = unit.calculate_min_max_power(start, end)
bids = []
op_time = unit.get_operation_time(start)
# we need to store the bid quantity for each hour
# and the bid price to calculate the weighted average
bid_quantity_block = {} # ====== new
bid_price_block = [] # ====== new
for product, min_power, max_power in zip(
product_tuples, min_power_values, max_power_values
):
bid_quantity_flex, bid_price_flex = 0, 0
bid_price_inflex, bid_quantity_inflex = 0, 0
start = product[0]
end = product[1]
# =============================================================================
# Powerplant is either on, or is able to turn on
# Calculating possible bid amount and cost
# =============================================================================
current_power = unit.outputs["energy"].at[start]
# adjust max_power for ramp speed
max_power = unit.calculate_ramp(
op_time, previous_power, max_power, current_power
)
# adjust min_power for ramp speed
min_power = unit.calculate_ramp(
op_time, previous_power, min_power, current_power
)
bid_quantity_inflex = min_power
# =============================================================================
# Calculating marginal cost
# =============================================================================
marginal_cost_inflex = unit.calculate_marginal_cost(
start, current_power + bid_quantity_inflex
)
marginal_cost_flex = unit.calculate_marginal_cost(
start, current_power + max_power
)
# =============================================================================
# Calculating possible price
# =============================================================================
if op_time > 0:
bid_price_inflex = calculate_EOM_price_if_on(
unit=unit,
market_id=market_config.market_id,
start=start,
marginal_cost_flex=marginal_cost_flex,
bid_quantity_inflex=bid_quantity_inflex,
foresight=self.foresight,
)
else:
bid_price_inflex = calculate_EOM_price_if_off(
unit=unit,
marginal_cost_inflex=marginal_cost_inflex,
bid_quantity_inflex=bid_quantity_inflex,
op_time=op_time,
)
if unit.outputs["heat"].at[start] > 0:
power_loss_ratio = (
unit.outputs["power_loss"].at[start]
/ unit.outputs["heat"].at[start]
)
else:
power_loss_ratio = 0.0
# Flex-bid price formulation
if op_time <= -unit.min_down_time or op_time > 0:
bid_quantity_flex = max_power - bid_quantity_inflex
bid_price_flex = (1 - power_loss_ratio) * marginal_cost_flex
# add volume and price to block bid
bid_quantity_block[product[0]] = bid_quantity_inflex # ====== new
if bid_quantity_inflex > 0: # ====== new
bid_price_block.append(bid_price_inflex) # ====== new
# add the flexible bid
bids.append( # ====== new
{
"start_time": start, # ====== new
"end_time": end, # ====== new
"only_hours": None, # ====== new
"price": bid_price_flex, # ====== new
"volume": bid_quantity_flex, # ====== new
"bid_type": "SB", # ====== new
"node": unit.node,
},
)
# calculate previous power with planned dispatch (bid_quantity)
previous_power = bid_quantity_inflex + bid_quantity_flex + current_power
op_time = max(op_time, 0) + 1 if previous_power > 0 else min(op_time, 0) - 1
# calculate weighted average of prices
volume = 0 # ====== new
price = 0 # ====== new
for i in range(len(bid_price_block)): # ====== new
price += (
bid_price_block[i] * list(bid_quantity_block.values())[i]
) # ====== new
volume += list(bid_quantity_block.values())[i] # ====== new
if volume != 0: # ====== new
mean_price = price / volume # ====== new
# add block bid
bids.append( # ====== new
{
"start_time": product_tuples[0][0], # ====== new
"end_time": product_tuples[-1][1], # ====== new
"only_hours": product_tuples[0][2], # ====== new
"price": mean_price, # ====== new
"volume": bid_quantity_block, # ====== new
"bid_type": "BB", # ====== new
"min_acceptance_ratio": 1, # ====== new
"accepted_volume": {
product[0]: 0 for product in product_tuples
}, # ====== new
"node": unit.node,
}
)
# delete bids with zero volume
bids = self.remove_empty_bids(bids)
return bids
With this the strategy is ready to test. As before, we add the new class to our world and load the scenario. Additionally, we now have to change the set bidding strategy for one example unit. Here, we choose the combined cycle gas turbine and set its strategy to our modified class ‘blockStrategy’.
Don’t forget to also add the defined advanced clearing mechanism to the newly generated world.
[ ]:
world = World(database_uri=DB_URI)
# add the new clearing mechanism to the world
world.clearing_mechanisms["pay_as_clear_advanced"] = AdvancedClearingRole
# overwrite the bidding strategy for all units
world.bidding_strategies["new_advanced_strategy"] = blockStrategy
load_scenario_folder(
world,
inputs_path=inputs_path,
scenario="example_01g",
study_case="bo_case",
)
# check the bidding strategy of the coal unit
world.unit_operators["coal_unit_operator"].units["coal_unit"].bidding_strategies["EOM"]
Now let’s run this example
[ ]:
world.run()
5. Linked orders#
In the same way, we can further adjust the block bid strategy to integrate the flexible bids as linked bids. Deviations to blockStrategy are marked with # ====== new:
[ ]:
class linkedStrategy(flexableEOM):
"""
A strategy that bids on the EOM-market with block and linked bids.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# check if kwargs contains eom_foresight argument
self.foresight = pd.Timedelta(kwargs.get("eom_foresight", "12h"))
def calculate_bids(
self,
unit: SupportsMinMax,
market_config: MarketConfig,
product_tuples: list[Product],
**kwargs,
) -> Orderbook:
"""
Calculates block and linked bids for the EOM-market and returns a list of bids consisting of the start time, end time, only hours, price, volume and bid type.
The bids take the following form:
One block bid with the minimum acceptance ratio set to 1 spanning the total clearing period.
It uses the inflexible power and the weighted average price of the inflexible power as the price.
This price is based on the marginal cost of the inflexible power and the starting costs.
The starting costs are split across inflexible power and the average operation or down time of the unit depending on the operation status before.
Additionally, for every hour where the unit is on, a separate flexible bid is created using the flexible power and marginal costs as bidding price.
This bids are linked as children to the block bid.
Args:
unit (SupportsMinMax): A unit that the unit operator manages.
market_config (MarketConfig): A market configuration.
product_tuples (list[Product]): A list of tuples containing the start and end time of each product.
kwargs (dict): Additional arguments.
Returns:
Orderbook: A list of bids.
"""
start = product_tuples[0][0]
end = product_tuples[-1][1]
previous_power = unit.get_output_before(start)
min_power_values, max_power_values = unit.calculate_min_max_power(start, end)
bids = []
op_time = unit.get_operation_time(start)
# we need to store the bid quantity for each hour
# and the bid price to calculate the weighted average
bid_quantity_block = {}
bid_price_block = []
# create a unique id for the block bid to link the flexible bids
block_id = unit.id + "_block" # ====== new
for product, min_power, max_power in zip(
product_tuples, min_power_values, max_power_values
):
bid_quantity_flex, bid_price_flex = 0, 0
bid_price_inflex, bid_quantity_inflex = 0, 0
start = product[0]
end = product[1]
# =============================================================================
# Powerplant is either on, or is able to turn on
# Calculating possible bid amount and cost
# =============================================================================
current_power = unit.outputs["energy"].at[start]
# adjust max_power for ramp speed
max_power = unit.calculate_ramp(
op_time, previous_power, max_power, current_power
)
# adjust min_power for ramp speed
min_power = unit.calculate_ramp(
op_time, previous_power, min_power, current_power
)
bid_quantity_inflex = min_power
# =============================================================================
# Calculating marginal cost
# =============================================================================
marginal_cost_inflex = unit.calculate_marginal_cost(
start, current_power + bid_quantity_inflex
)
marginal_cost_flex = unit.calculate_marginal_cost(
start, current_power + max_power
)
# =============================================================================
# Calculating possible price
# =============================================================================
if op_time > 0:
bid_price_inflex = calculate_EOM_price_if_on(
unit,
market_config.market_id,
start,
marginal_cost_flex,
bid_quantity_inflex,
self.foresight,
)
else:
bid_price_inflex = calculate_EOM_price_if_off(
unit,
marginal_cost_inflex,
bid_quantity_inflex,
op_time,
)
if unit.outputs["heat"].at[start] > 0:
power_loss_ratio = (
unit.outputs["power_loss"].at[start]
/ unit.outputs["heat"].at[start]
)
else:
power_loss_ratio = 0.0
# Flex-bid price formulation
if op_time <= -unit.min_down_time or op_time > 0:
bid_quantity_flex = max_power - bid_quantity_inflex
bid_price_flex = (1 - power_loss_ratio) * marginal_cost_flex
bid_quantity_block[product[0]] = bid_quantity_inflex
if bid_quantity_inflex > 0:
bid_price_block.append(bid_price_inflex)
# use block id as parent id for flexible bids
parent_id = block_id # ====== new
else:
# if the bid quantity is 0, the bid is not linked to the block bid
parent_id = None # ====== new
# add the flexible bid as linked bid
bids.append(
{
"start_time": start,
"end_time": end,
"only_hours": None,
"price": bid_price_flex,
"volume": {start: bid_quantity_flex}, # ====== new
"bid_type": "LB", # ====== new
"parent_bid_id": parent_id, # ====== new
"node": unit.node,
},
)
# calculate previous power with planned dispatch (bid_quantity)
previous_power = bid_quantity_inflex + bid_quantity_flex + current_power
op_time = max(op_time, 0) + 1 if previous_power > 0 else min(op_time, 0) - 1
# calculate weighted average of prices
volume = 0
price = 0
for i in range(len(bid_price_block)):
price += bid_price_block[i] * list(bid_quantity_block.values())[i]
volume += list(bid_quantity_block.values())[i]
if volume != 0:
mean_price = price / volume
# add block bid
bids.append(
{
"start_time": product_tuples[0][0],
"end_time": product_tuples[-1][1],
"only_hours": product_tuples[0][2],
"price": mean_price,
"volume": bid_quantity_block,
"bid_type": "BB",
"min_acceptance_ratio": 1,
"accepted_volume": {product[0]: 0 for product in product_tuples},
"bid_id": block_id,
"node": unit.node,
}
)
# delete bids with zero volume
bids = self.remove_empty_bids(bids)
return bids
Here, we now add the new class linkedStrategy to our available bidding_strategies, overwrite the bidding strategies to linkedStrategy and then load our scenario.
[ ]:
world = World(database_uri=DB_URI)
# add the new clearing mechanism to the world
world.clearing_mechanisms["pay_as_clear_advanced"] = AdvancedClearingRole
# overwrite the bidding strategy for all units
world.bidding_strategies["new_advanced_strategy"] = linkedStrategy
load_scenario_folder(
world,
inputs_path=inputs_path,
scenario="example_01g",
study_case="lo_case",
)
# check the bidding strategy of the coal unit
world.unit_operators["coal_unit_operator"].units["coal_unit"].bidding_strategies["EOM"]
Now we run this version:
[ ]:
world.run()
6. Visualize the results#
We can visualize the results using the following functions
[ ]:
import os
from functools import partial
import matplotlib.pyplot as plt
from sqlalchemy import create_engine
engine = create_engine(DB_URI)
os.makedirs("outputs", exist_ok=True)
[ ]:
sql = """
SELECT ident, simulation,
sum(round(CAST(value AS numeric), 2)) FILTER (WHERE variable = 'total_cost') as total_cost,
sum(round(CAST(value AS numeric), 2)*1000) FILTER (WHERE variable = 'total_volume') as total_volume,
sum(round(CAST(value AS numeric), 2)) FILTER (WHERE variable = 'avg_price') as average_cost
FROM kpis
where variable in ('total_cost', 'total_volume', 'avg_price')
and simulation in ('example_01g_sho_case', 'example_01g_bo_case', 'example_01g_lo_case')
group by simulation, ident ORDER BY simulation
"""
kpis = pd.read_sql(sql, engine)
# sort the dataframe to have sho, bo and lo case in the right order
# sort kpis in the order sho, bo, lo
kpis = kpis.sort_values(
by="simulation",
key=lambda x: x.map(
{"example_01g_sho_case": 1, "example_01g_bo_case": 2, "example_01g_lo_case": 3}
),
)
kpis["total_volume"] /= 1e9
kpis["total_cost"] /= 1e6
savefig = partial(plt.savefig, transparent=False, bbox_inches="tight")
xticks = kpis["simulation"].unique()
plt.style.use("seaborn-v0_8")
fig, ax = plt.subplots(1, 1, figsize=(10, 6))
ax2 = ax.twinx() # Create another axes that shares the same x-axis as ax.
width = 0.4
kpis.total_volume.plot(kind="bar", ax=ax, width=width, position=1, color="royalblue")
kpis.total_cost.plot(kind="bar", ax=ax2, width=width, position=0, color="green")
# set x-achxis limits
ax.set_xlim(-0.6, len(kpis["simulation"]) - 0.4)
# set y-achxis limits
ax.set_ylim(0, max(kpis.total_volume) * 1.1 + 0.1)
ax2.set_ylim(0, max(kpis.total_cost) * 1.1 + 0.1)
ax.set_ylabel("Total Volume (GWh)")
ax2.set_ylabel("Total Cost (M€)")
ax.set_xticklabels(xticks, rotation=45)
ax.set_xlabel("Simulation")
ax.legend(["Total Volume"], loc="upper left")
ax2.legend(["Total Cost"], loc="upper right")
plt.title("Total Volume and Total Cost for each Simulation")
savefig("outputs/tutorial_6_overview.png")
plt.show()
In the first plot, the total traded volume and the total costs for generation are displayed. We can see, that the total volume does not change, because we have an inflexible demand, which is always met. The total costs show significant differences only when linked orders are introduced.
Because the bidding prices are formed using the same rules, it was expected, that the introduction of new order types with additional restrictions increase prices and costs, as shown in this plot for linked orders.
Now we create the second plot, showing the accepted volumes for the different simulations for the hard coal unit.
[ ]:
# second plot for the accepted volume of the coal unit
sql = """
SELECT
start_time as "time",
sum(accepted_volume) AS "accepted_volume",
unit_id,
simulation,
bid_type
FROM market_orders
WHERE
start_time between '2020-01-22' And '2020-01-25' AND
unit_id = 'coal_unit' AND
simulation in ('example_01g_sho_case', 'example_01g_bo_case', 'example_01g_lo_case')
GROUP BY 1, unit_id, simulation, bid_type
ORDER BY 1
"""
df = pd.read_sql(sql, engine, index_col="time")
df = df.sort_values(
by="simulation",
key=lambda x: x.map(
{"example_01g_sho_case": 1, "example_01g_bo_case": 2, "example_01g_lo_case": 3}
),
)
fig, ax = plt.subplots(3, 1, figsize=(10, 10))
# plot the sum of accepted volume for each simulation over time
for i, sim in enumerate(df.simulation.unique()):
data = df[df.simulation == sim].sort_index()
# set index to datetime
data.index = pd.to_datetime(data.index)
# sum over bid_types
data = data["accepted_volume"].groupby([data.index]).sum()
# reset index and fill empty data points with zero
time_index = pd.date_range(start=min(df.index), end=max(df.index), freq="1h")
data = data.reindex(time_index, fill_value=0.0)
data.plot(ax=ax[i], label=sim, color="royalblue", linestyle="-")
ax[i].set_title(sim)
ax[i].set_ylabel("Accepted Power (MW)")
if i < 2:
ax[i].set_xlabel("")
ax[i].set_xticklabels([])
ax[i].legend(["hard coal unit"], loc="upper right")
# add horizontal corridor for min and max power
ax[i].fill_between(data.index, 385.5, 950, color="lightgreen", alpha=0.5)
ax[i].fill_between(
data.index,
0,
385.5,
where=(data > 0) & (data < 385.5),
color="lightcoral",
alpha=0.5,
)
# set the xticks for the last subplot
ax[2].set_xlabel("Time")
savefig("outputs/tutorial_6_accepted-power.png")
plt.show()
Here, you can see that the accepted power of the hard coal unit depends on the order type. The unit has a minimum power output of 385.8 MW and a maximum of 950 MW. The plot shows that in the SHO-case the minimum power restriction is not always respected. But also in the BO-case the accepted volume can be between zero and the minimum power output, because it is possible to accept the flexible single hourly bid without accepting the block bid. With the linked orders all technical restrictions can be represented in the strategy. Therefore, the dispatch of the unit can precisely follow the market outcome, but the sum of the accepted volume is slightly lower.
This brings us to the end of this short tutorial on advanced orders.