etrago.cluster package

etrago.cluster.electrical module

electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.

etrago.cluster.electrical.adjust_no_electric_network(etrago, busmap, cluster_met, apply_on='grid_model')

Adjusts the non-electric network based on the electrical network (esp. eHV network), adds the gas buses to the busmap, and creates the new buses for the non-electric network.

Parameters:

• etrago (Etrago) – An instance of the Etrago class.

• busmap (dict) – A dictionary that maps old bus_ids to new bus_ids.

• cluster_met (str) – A string indicating the clustering method to be used.

Returns:

• network (pypsa.Network) – Container for all network components of the clustered network.

• busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.cluster_on_extra_high_voltage(etrago, busmap, with_time=True)

Main function of the EHV-Clustering approach. Creates a new clustered pypsa.Network given a busmap mapping all bus_ids to other bus_ids of the same network.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap (dict) – Maps old bus_ids to new bus_ids.

• with_time (bool) – If true time-varying data will also be aggregated.

Returns:

• network (pypsa.Network) – Container for all network components of the clustered network.

• busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.delete_ehv_buses_no_lines(network)

When there are AC buses totally isolated, this function deletes them in order to make possible the creation of busmaps based on electrical connections and other purposes. Additionally, it throws a warning to inform the user in case that any correction should be done.

Parameters:

network (pypsa.network)

Return type:

None

etrago.cluster.electrical.ehv_clustering(self)

Cluster the network based on Extra High Voltage (EHV) grid.

If ‘active’ in the network_clustering_ehv argument is True, the function clusters the network based on the EHV grid.

Parameters:

self (Etrago object pointer) – The object pointer for an Etrago object.

Return type:

None

etrago.cluster.electrical.select_elec_network(etrago, apply_on='grid_model')

Selects the electric network based on the clustering settings specified in the Etrago object.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• apply_on (str) – gives information about the objective of the output network. If “grid_model” is provided, the value assigned in the args for [“network_clustering”][“method”][“per_country””] will define if the foreign buses will be included in the network. If “market_model” is provided, foreign buses will be always included.

Returns:

elec_networkpypsa.Network

Contains the electric network

n_clustersint

number of clusters used in the clustering process.

Return type:

Tuple containing

etrago.cluster.electrical.unify_foreign_buses(etrago)

Unifies foreign AC buses into clusters using the k-medoids algorithm with Dijkstra distance as a similarity measure.

Parameters:

etrago (Etrago) – An instance of the Etrago class

Returns:

busmap_foreign – A pandas series that maps the foreign buses to their respective clusters. The series index is the bus ID and the values are the corresponding cluster medoid IDs.

Return type:

pd.Series

etrago.cluster.electrical.preprocessing(etrago, apply_on='grid_model')

Preprocesses an Etrago object to prepare it for network clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• apply_on (string) – provide information about the objective of the preprocessing. Which process is going to use the result. e.g. “grid_model”, “market_model”.

Returns:

• network_elec (pypsa.Network) – Container for all network components of the electrical network.

• weight (pandas.Series) – A pandas.Series with the bus weighting data.

• n_clusters (int) – The number of clusters to use for network clustering.

• busmap_foreign (pandas.Series) – The Series object with the foreign bus mapping data.

etrago.cluster.electrical.postprocessing(etrago, busmap, busmap_foreign, medoid_idx=None, aggregate_generators_carriers=None, aggregate_links=True, apply_on='grid_model')

Postprocessing function for network clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap (pandas.Series) – mapping between buses and clusters

• busmap_foreign (pandas.DataFrame) – mapping between foreign buses and clusters

• medoid_idx (pandas.DataFrame) – mapping between cluster indices and medoids

Returns:

clusteringpypsa.network

Network object containing the clustered network

busmappandas.Series

Updated mapping between buses and clusters

Return type:

Tuple containing

etrago.cluster.electrical.weighting_for_scenario(network, save=None)

define bus weighting based on generation, load and storage

Parameters:

• network (pypsa.network) – Each bus in this network will receive a weight based on the generator, load and storages also available in the network object.

• save (str or bool, optional) – If defined, the result of the weighting will be saved in the path supplied here. The default is None.

Returns:

weight – Serie with the weight assigned to each bus to perform a k-mean clustering.

Return type:

pandas.series

etrago.cluster.electrical.run_spatial_clustering(self)

Main method for running spatial clustering on the electrical network. Allows for clustering based on k-means and k-medoids dijkstra.

Parameters:

self – The object pointer for an Etrago object containing all relevant parameters and data

Return type:

None

etrago.cluster.gas module

gas.py defines the methods to cluster gas grid networks spatially for applications within the tool eTraGo.

etrago.cluster.gas.preprocessing(etrago, carrier, apply_on='grid_model')

Preprocesses the gas network data from the given Etrago object for the spatial clustering process of the CH4 o H2 grid.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• carrier (str) – Name of the bus carrier that will be clustered

Return type:

None

Raises:

ValueError – If “n_clusters_ch4” or “n_clusters_h2” is less than or equal to the number of neighboring country gas buses.

etrago.cluster.gas.kmean_clustering_gas(etrago, network_ch4, weight, n_clusters)

Performs K-means clustering on the gas network data in the given network_ch4 pypsa.Network object.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• network_ch4 (pypsa.Network) – A Network object containing the gas network data.

• weight (str or None) – The name of the bus weighting column to use for clustering. If None, unweighted clustering is performed.

• n_clusters (int) – The number of clusters to create.

Returns:

• busmap (pandas.Series) – A pandas.Series object mapping each bus in the CH4 network to its corresponding cluster ID

• None – None is returned because k-means clustering makes no use of medoids

etrago.cluster.gas.get_h2_clusters(etrago, busmap_ch4)

Maps H2 buses to CH4 cluster IDds and creates unique H2 cluster IDs.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap_ch4 (pd.Series) – A Pandas Series mapping each bus in the CH4 network to its corresponding cluster ID.

Returns:

busmap – A Pandas Series mapping each bus in the combined CH4 and H2 network to its corresponding cluster ID.

Return type:

pd.Series

etrago.cluster.gas.sector_coupled_clustering_strategy(etrago)

Defines clustering strategies for sectors without a grid per scenario.

Parameters:

etrago (Etrago) – An instance of the Etrago class

Returns:

strategy – Dictionary containing cluster strategies for each sector.

Return type:

dict

etrago.cluster.gas.gas_postprocessing(etrago, busmap, medoid_idx=None, apply_on='grid_model')

Performs the postprocessing for the gas grid clustering based on the provided busmap and returns the clustered network.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

• medoid_idx (pd.Series) – A pandas.Series object containing the medoid indices for the gas network.

Returns:

• network_gasgrid_c (pypsa.Network) – A pypsa.Network containing the clustered network.

• busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

etrago.cluster.gas.highestInteger(potentially_numbers)

Fetch the highest number of a series with mixed types

Parameters:

potentially_numbers (pandas.Series) – Series with mixed dtypes, potentially containing numbers.

Returns:

highest – Highest integer found in series.

Return type:

int

etrago.cluster.gas.simultaneous_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)

Cluster sector coupling technology based on multiple connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters:

• network (pypsa.Network) – PyPSA network instance.

• busmap (pandas.Series) – Series with lookup table for clustered buses.

• carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

• carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns:

Busmap for the sector coupling cluster.

Return type:

dict

etrago.cluster.gas.consecutive_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)

Cluster sector coupling technology based on single connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters:

• network (pypsa.Network) – PyPSA network instance.

• busmap (pandas.Series) – Series with lookup table for clustered buses.

• carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

• carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns:

busmap_sc – Busmap for the sector coupled cluster.

Return type:

dict

etrago.cluster.gas.sc_multi_carrier_based(buses_to_cluster, connected_links)

Create busmap for sector coupled carrier based on multiple other carriers.

Parameters:

• buses_to_cluster (pandas.Series) – Series containing the buses of the sector coupled carrier which are to be clustered.

• connected_links (pandas.DataFrame) – Links that connect from the buses with other carriers to the buses of the sector coupled carrier.

Returns:

busmap – Busmap for the sector coupled carrier.

Return type:

dict

etrago.cluster.gas.sc_single_carrier_based(connected_links)

Create busmap for sector coupled carrier based on single other carrier.

Parameters:

connected_links (pandas.DataFrame) – Links that connect from the buses with other carrier to the buses of the sector coupled carrier.

Returns:

busmap – Busmap for the sector coupled carrier.

Return type:

dict

etrago.cluster.gas.get_clustering_from_busmap(network, busmap, aggregate_generators_carriers=None, line_length_factor=1.0, with_time=True, bus_strategies={}, one_port_strategies={}, generator_strategies={})

Aggregates components of the given network based on a bus mapping and returns a clustered gas grid pypsa.Network.

Parameters:

• network (pypsa.Network) – The input pypsa.Network object

• busmap (pandas.Sereies :) – A mapping of buses to clusters

• line_length_factor (float) – A factor used to adjust the length of new links created during aggregation. Default is 1.0.

• with_time (bool) – Determines whether to copy the time-dependent properties of the input network to the output network. Default is True.

• bus_strategies (dict) – A dictionary of custom strategies to use during the aggregation step. Default is an empty dictionary.

• one_port_strategies (dict) – A dictionary of custom strategies to use during the one-port component aggregation step. Default is an empty dictionary.

Returns:

network_gasgrid_c – A new gas grid pypsa.Network object with aggregated components based on the bus mapping.

Return type:

pypsa.Network

etrago.cluster.gas.join_busmap_medoids(busmap1: Series, busmap2: Series, medoid_idx1: Series, medoid_idx2: Series)

Parameters:

• busmap1 (pd.Series)

• busmap2 (pd.Series)

• medoid_idx1 (pd.Series)

• medoid_idx2 (pd.Series)

Returns:

• busmap (pd.Series) – Pandas series joining busmap1 and busmap2

• medoid_idx (pd.Series) – Pandas series joining medoid_idx1 and medoid_idx2

etrago.cluster.gas.run_spatial_clustering_gas(self)

Performs spatial clustering on the gas network using either K-means or K-medoids-Dijkstra algorithm. Updates the network topology by aggregating buses and links, and then performs postprocessing to finalize the changes.

Return type:

None

Raises:

ValueError – If the selected method is not “kmeans” or “kmedoids-dijkstra”.:

etrago.cluster.temporal module

This module contains functions for reducing the complexity of a PyPSA

network in temporal dimension by

1. downsampling to every n-th snapshot

2. clustering to typical periods (eg days, weeks)

c) clustering to segments of variable length Essentially used is the tsam package ( https://github.com/FZJ-IEK3-VSA/tsam ) developed by Leander Kotzur et al.

etrago.cluster.temporal.snapshot_clustering(self)

Function to call the snapshot clustering function with the respecting method and settings.

Raises:

ValueError – When calling a non-available function.

Return type:

None.

etrago.cluster.temporal.tsam_cluster(timeseries_df, typical_periods=10, how='daily', extremePeriodMethod='None', segmentation=False, segment_no=10, segm_hoursperperiod=24)

Conducts the clustering of the snapshots for temporal aggregation with the respecting method.

Parameters:

• timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

• typical_periods (int, optional) – Number of clusters for typical_periods. The default is 10.

• how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

• segmentation (boolean, optional) – Argument to activate segmenation method. The default is False.

• segment_no (int, optional) – Number of segments for segmentation. The default is 10.

• segm_hoursperperiod (int, optional) – Only for segmentation, ensures to cluster to segments considering all snapshots. The default is 24.

Returns:

• df_cluster (pd.DataFrame) – Information on cluster after clustering to typical periods.

• cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

• dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

• hours (int) – Hours per typical period.

• df_i_h (pd.DataFrame) – Information on cluster after clustering to typical periods.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

etrago.cluster.temporal.segmentation_extreme_periods(timeseries_df, timeseries, extremePeriodMethod)

Function to consider extreme snapshots while using segmentation.

Parameters:

• timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Raises:

ValueError – When calling wrong method to consider extreme values.

Returns:

timeseries – Information on segments including extreme snapshots after segmentation.

Return type:

pd.DataFrame

etrago.cluster.temporal.run(self, n_clusters=None, how='daily', segmented_to=False, extreme_periods='None')

Function to call the respecting snapshot clustering function and export the result to a csv-file.

Parameters:

• network (pypsa.Network object) – Container for all network components.

• n_clusters (int, optional) – Number of clusters for typical_periods. The default is None.

• how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

• segmented_to (int, optional) – Number of segments for segmentation. The default is False.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Returns:

network – Container for all network components.

Return type:

pypsa.Network object

etrago.cluster.temporal.prepare_pypsa_timeseries(network)

Prepares timeseries and residual load timeseries for clustering.

Parameters:

network (pypsa.Network object) – Container for all network components.

Returns:

df – Timeseries to be considered when clustering.

Return type:

pd.DataFrame

etrago.cluster.temporal.update_data_frames(network, cluster_weights, dates, hours, timeseries, segmentation)

Updates the snapshots, snapshot weightings and the dataframes based on the original data in the network and the medoids created by clustering these original data.

Parameters:

• network (pypsa.Network object) – Container for all network components.

• cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

• dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

• hours (int) – Hours per typical period.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

• segmentation (boolean) – Checks if segmentation of clustering to typical periods has been used.

Returns:

network – Container for all network components.

Return type:

pypsa.Network object

etrago.cluster.temporal.skip_snapshots(self)

Conducts the downsapling to every n-th snapshot.

Return type:

None.

etrago.cluster.spatial module

spatial.py defines the methods to run spatial clustering on networks.

etrago.cluster.spatial.nan_links(x)

etrago.cluster.spatial.ext_storage(x)

etrago.cluster.spatial.sum_with_inf(x)

etrago.cluster.spatial.strategies_buses()

etrago.cluster.spatial.strategies_lines()

etrago.cluster.spatial.strategies_one_ports()

etrago.cluster.spatial.strategies_generators()

etrago.cluster.spatial.strategies_links()

etrago.cluster.spatial.group_links(network, with_time=True, carriers=None, cus_strateg={})

Aggregate network.links and network.links_t after any kind of clustering

Parameters:

• network (pypsa.Network object) – Container for all network components.

• with_time (bool) – says if the network object contains timedependent series.

• carriers (list of strings) – Describe which type of carriers should be aggregated. The default is None.

• strategies (dictionary) – custom strategies to perform the aggregation

Returns:

• new_df – links aggregated based on bus0, bus1 and carrier

• new_pnl – links time series aggregated

etrago.cluster.spatial.graph_from_edges(edges)

Constructs an undirected multigraph from a list containing data on weighted edges.

Parameters:

edges (list) – List of tuples each containing first node, second node, weight, key.

Returns:

M

Return type:

networkx.classes.multigraph.MultiGraph

etrago.cluster.spatial.gen(nodes, n, graph)

Generator for applying multiprocessing.

Parameters:

• nodes (list) – List of nodes in the system.

• n (int) – Number of desired multiprocessing units.

• graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Return type:

None

etrago.cluster.spatial.shortest_path(paths, graph)

Finds the minimum path lengths between node pairs defined in paths.

Parameters:

• paths (list) – List of pairs containing a source and a target node

• graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Returns:

df – DataFrame holding source and target node and the minimum path length.

Return type:

pd.DataFrame

etrago.cluster.spatial.busmap_by_shortest_path(etrago, fromlvl, tolvl, cpu_cores=4)

Creates a busmap for the EHV-Clustering between voltage levels based on dijkstra shortest path. The result is automatically written to the model_draft on the <OpenEnergyPlatform>[www.openenergy-platform.org] database with the name ego_grid_pf_hv_busmap and the attributes scn_name (scenario name), bus0 (node before clustering), bus1 (node after clustering) and path_length (path length). An AssertionError occurs if buses with a voltage level are not covered by the input lists ‘fromlvl’ or ‘tolvl’.

Parameters:

• network (pypsa.Network) – Container for all network components.

• session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

• fromlvl (list) – List of voltage-levels to cluster.

• tolvl (list) – List of voltage-levels to remain.

• cpu_cores (int) – Number of CPU-cores.

Return type:

None

etrago.cluster.spatial.busmap_ehv_clustering(etrago)

Generates a busmap that can be used to cluster an electrical network to only extra high voltage buses. If a path to a busmap in a csv file is passed in the arguments, it loads the csv file and returns it.

Parameters:

etrago (Etrago) – An instance of the Etrago class

Returns:

busmap – Maps old bus_ids to new bus_ids.

Return type:

dict

etrago.cluster.spatial.kmean_clustering(etrago, selected_network, weight, n_clusters)

Main function of the k-mean clustering approach. Maps an original network to a new one with adjustable number of nodes and new coordinates.

Parameters:

• network (pypsa.Network) – Container for all network components.

• n_clusters (int) – Desired number of clusters.

• load_cluster (boolean) – Loads cluster coordinates from a former calculation.

• line_length_factor (float) – Factor to multiply the crow-flies distance between new buses in order to get new line lengths.

• remove_stubs (boolean) – Removes stubs and stubby trees (i.e. sequentially reducing dead-ends).

• use_reduced_coordinates (boolean) – If True, do not average cluster coordinates, but take from busmap.

Returns:

network – Container for all network components.

Return type:

pypsa.Network

etrago.cluster.spatial.dijkstras_algorithm(buses, connections, medoid_idx, cpu_cores)

Function for combination of k-medoids Clustering and Dijkstra’s algorithm. Creates a busmap assigning the nodes of a original network to the nodes of a clustered network considering the electrical distances based on Dijkstra’s shortest path.

Parameters:

• network (pypsa.Network) – Container for all network components.

• medoid_idx (pandas.Series) – Indices of k-medoids

• busmap_kmedoid (pandas.Series) – Busmap based on k-medoids clustering

• cpu_cores (string) – numbers of cores used during multiprocessing

Returns:

busmap – Mapping from bus ids to medoids ids

Return type:

pandas.Series

etrago.cluster.spatial.kmedoids_dijkstra_clustering(etrago, buses, connections, weight, n_clusters)

Applies a k-medoids clustering on the given network and calls the function to conduct a Dijkstra’s algorithm afterwards for the consideration of the network’s topology in the spatial clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• buses (pandas.DataFrame) – DataFrame with information about the buses of the network.

• connections (pandas.DataFrame) – DataFrame with information about the connections of the network (links or lines).

• weight (pandas.Series) – Series with the weight for each bus.

• n_clusters (int) – The number of clusters to create.

Returns:

• Tuple containing

• busmap (pandas.Series) – Series containing the mapping of buses to their resp. medoids

• medoid_idx (pandas.Series) – Series containing the medoid indeces

etrago.cluster.spatial.focus_weighting(etrago, network, weight, focus_region, cluster_within, per_country, cpu_cores, func='sigmoid-50', save=None)

Apply distance-based spatial weighting relative to a defined focus region.

This function modifies an existing bus weighting by applying a decay function based on the shortest-path distance (along network topology) between each bus and a specified focus region. Buses inside the focus region receive zero distance (handled as 1 m internally to avoid division by zero). Distances are computed using Dijkstra’s algorithm and multiprocessing.

The decay factor is applied multiplicatively to the input weight.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• network (pypsa.Network) – Network containing buses and either lines (AC) or carrier-specific links (e.g. CH4, H2). Must include: - network.buses with columns [“x”, “y”, “carrier”] - network.lines with [“bus0”, “bus1”, “length”] (for AC) - network.links with [“bus0”, “bus1”, “length”, “carrier”] (CH4/H2)

• weight (pandas.Series) – Initial bus weighting (indexed by bus names). Typically derived from generation capacities and/or loads. Will be modified and returned.

• focus_region (list[str] or str) – Either: - List of region identifiers (gen column in vg250_krs), or - Path to a geospatial file readable by GeoPandas. The geometry must have a valid CRS.

• cluster_within (bool) – If False, buses located inside the focus region are assigned a very large weight (100000) to prevent clustering within the region.

• cpu_cores (int or str) – Number of CPU cores for multiprocessing. If “max”, all available cores are used.

• func ({"1-dist", "gauss-20", "gauss-100",) –

  “sigmoid-20”, “sigmoid-50”, “sigmoid-100”}, optional Distance decay function applied to shortest-path distances (in meters):

  • ”1-dist” : Inverse distance weighting (1 / d)

  • ”gauss-20” : Gaussian decay (σ = 20 km)

  • ”gauss-100” : Gaussian decay (σ = 100 km)

  • ”sigmoid-20” : Sigmoid decay (σ = 20 km)

  • ”sigmoid-50” : Sigmoid decay (σ = 50 km)

  • ”sigmoid-100” : Sigmoid decay (σ = 100 km)

  Default is “sigmoid-50”.

• save (str or None, optional) – If provided, the resulting weight Series is written to this CSV path.

Returns:

Modified bus weighting indexed by bus name. The values are rounded up using np.ceil.

Return type:

pandas.Series

etrago.cluster.spatial.drop_nan_values(network)

Drops nan values after clustering an replaces output data time series with empty dataframes

Parameters:

network (pypsa.Network) – Container for all network components.

Return type:

None.