etrago.cluster package

etrago.cluster.disaggregation module

etrago.cluster.electrical module

electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.

etrago.cluster.electrical.adjust_no_electric_network(etrago, busmap, cluster_met)[source]

Adjusts the non-electric network based on the electrical network (esp. eHV network), adds the gas buses to the busmap, and creates the new buses for the non-electric network.

Parameters

  • etrago (Etrago) – An instance of the Etrago class.

  • busmap (dict) – A dictionary that maps old bus_ids to new bus_ids.

  • cluster_met (str) – A string indicating the clustering method to be used.

Returns

  • network (pypsa.Network) – Container for all network components of the clustered network.

  • busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.cluster_on_extra_high_voltage(etrago, busmap, with_time=True)[source]

Main function of the EHV-Clustering approach. Creates a new clustered pypsa.Network given a busmap mapping all bus_ids to other bus_ids of the same network.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • busmap (dict) – Maps old bus_ids to new bus_ids.

  • with_time (bool) – If true time-varying data will also be aggregated.

Returns

  • network (pypsa.Network) – Container for all network components of the clustered network.

  • busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.delete_ehv_buses_no_lines(network)[source]

When there are AC buses totally isolated, this function deletes them in order to make possible the creation of busmaps based on electrical connections and other purposes. Additionally, it throws a warning to inform the user in case that any correction should be done.

Parameters

network (pypsa.network) –

Return type

None

etrago.cluster.electrical.ehv_clustering(self)[source]

Cluster the network based on Extra High Voltage (EHV) grid.

If network_clustering_ehv argument is True, the function clusters the network based on the EHV grid.

Parameters

self (Etrago object pointer) – The object pointer for an Etrago object.

Return type

None

etrago.cluster.electrical.select_elec_network(etrago)[source]

Selects the electric network based on the clustering settings specified in the Etrago object.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

elec_networkpypsa.Network

Contains the electric network

n_clustersint

number of clusters used in the clustering process.

Return type

Tuple containing

etrago.cluster.electrical.unify_foreign_buses(etrago)[source]

Unifies foreign AC buses into clusters using the k-medoids algorithm with Dijkstra distance as a similarity measure.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

busmap_foreign – A pandas series that maps the foreign buses to their respective clusters. The series index is the bus ID and the values are the corresponding cluster medoid IDs.

Return type

pd.Series

etrago.cluster.electrical.preprocessing(etrago)[source]

Preprocesses an Etrago object to prepare it for network clustering.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

  • network_elec (pypsa.Network) – Container for all network components of the electrical network.

  • weight (pandas.Series) – A pandas.Series with the bus weighting data.

  • n_clusters (int) – The number of clusters to use for network clustering.

  • busmap_foreign (pandas.Series) – The Series object with the foreign bus mapping data.

etrago.cluster.electrical.postprocessing(etrago, busmap, busmap_foreign, medoid_idx=None)[source]

Postprocessing function for network clustering.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • busmap (pandas.Series) – mapping between buses and clusters

  • busmap_foreign (pandas.DataFrame) – mapping between foreign buses and clusters

  • medoid_idx (pandas.DataFrame) – mapping between cluster indices and medoids

Returns

clusteringpypsa.network

Network object containing the clustered network

busmappandas.Series

Updated mapping between buses and clusters

Return type

Tuple containing

etrago.cluster.electrical.weighting_for_scenario(network, save=None)[source]

define bus weighting based on generation, load and storage

Parameters

  • network (pypsa.network) – Each bus in this network will receive a weight based on the generator, load and storages also available in the network object.

  • save (str or bool, optional) – If defined, the result of the weighting will be saved in the path supplied here. The default is None.

Returns

weight – Serie with the weight assigned to each bus to perform a k-mean clustering.

Return type

pandas.series

etrago.cluster.electrical.run_spatial_clustering(self)[source]

Main method for running spatial clustering on the electrical network. Allows for clustering based on k-means and k-medoids dijkstra.

Parameters

self – The object pointer for an Etrago object containing all relevant parameters and data

Return type

None

etrago.cluster.gas module

gas.py defines the methods to cluster gas grid networks spatially for applications within the tool eTraGo.

etrago.cluster.gas.preprocessing(etrago)[source]

Preprocesses the gas network data from the given Etrago object for the spatial clustering process of the CH4 grid.

Parameters

etrago (Etrago) – An instance of the Etrago class

Return type

None

Raises

ValueError – If settings[“n_clusters_gas”] is less than or equal to the number of neighboring country gas buses.

etrago.cluster.gas.kmean_clustering_gas(etrago, network_ch4, weight, n_clusters)[source]

Performs K-means clustering on the gas network data in the given network_ch4 pypsa.Network object.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • network_ch4 (pypsa.Network) – A Network object containing the gas network data.

  • weight (str or None) – The name of the bus weighting column to use for clustering. If None, unweighted clustering is performed.

  • n_clusters (int) – The number of clusters to create.

Returns

  • busmap (pandas.Series) – A pandas.Series object mapping each bus in the CH4 network to its corresponding cluster ID

  • None – None is returned because k-means clustering makes no use of medoids

etrago.cluster.gas.get_h2_clusters(etrago, busmap_ch4)[source]

Maps H2 buses to CH4 cluster IDds and creates unique H2 cluster IDs.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • busmap_ch4 (pd.Series) – A Pandas Series mapping each bus in the CH4 network to its corresponding cluster ID.

Returns

busmap – A Pandas Series mapping each bus in the combined CH4 and H2 network to its corresponding cluster ID.

Return type

pd.Series

etrago.cluster.gas.gas_postprocessing(etrago, busmap, medoid_idx=None)[source]

Performs the postprocessing for the gas grid clustering based on the provided busmap and returns the clustered network.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

  • medoid_idx (pd.Series) – A pandas.Series object containing the medoid indices for the gas network.

Returns

  • network_gasgrid_c (pypsa.Network) – A pypsa.Network containing the clustered network.

  • busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

etrago.cluster.gas.highestInteger(potentially_numbers)[source]

Fetch the highest number of a series with mixed types

Parameters

potentially_numbers (pandas.Series) – Series with mixed dtypes, potentially containing numbers.

Returns

highest – Highest integer found in series.

Return type

int

etrago.cluster.gas.simultaneous_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]

Cluster sector coupling technology based on multiple connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters

  • network (pypsa.Network) – PyPSA network instance.

  • busmap (pandas.Series) – Series with lookup table for clustered buses.

  • carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

  • carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns

Busmap for the sector coupling cluster.

Return type

dict

etrago.cluster.gas.consecutive_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]

Cluster sector coupling technology based on single connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters

  • network (pypsa.Network) – PyPSA network instance.

  • busmap (pandas.Series) – Series with lookup table for clustered buses.

  • carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

  • carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns

busmap_sc – Busmap for the sector coupled cluster.

Return type

dict

etrago.cluster.gas.sc_multi_carrier_based(buses_to_cluster, connected_links)[source]

Create busmap for sector coupled carrier based on multiple other carriers.

Parameters

  • buses_to_cluster (pandas.Series) – Series containing the buses of the sector coupled carrier which are to be clustered.

  • connected_links (pandas.DataFrame) – Links that connect from the buses with other carriers to the buses of the sector coupled carrier.

Returns

busmap – Busmap for the sector coupled carrier.

Return type

dict

etrago.cluster.gas.sc_single_carrier_based(connected_links)[source]

Create busmap for sector coupled carrier based on single other carrier.

Parameters

connected_links (pandas.DataFrame) – Links that connect from the buses with other carrier to the buses of the sector coupled carrier.

Returns

busmap – Busmap for the sector coupled carrier.

Return type

dict

etrago.cluster.gas.get_clustering_from_busmap(network, busmap, line_length_factor=1.0, with_time=True, bus_strategies={}, one_port_strategies={})[source]

Aggregates components of the given network based on a bus mapping and returns a clustered gas grid pypsa.Network.

Parameters

  • network (pypsa.Network) – The input pypsa.Network object

  • busmap (pandas.Sereies :) – A mapping of buses to clusters

  • line_length_factor (float) – A factor used to adjust the length of new links created during aggregation. Default is 1.0.

  • with_time (bool) – Determines whether to copy the time-dependent properties of the input network to the output network. Default is True.

  • bus_strategies (dict) – A dictionary of custom strategies to use during the aggregation step. Default is an empty dictionary.

  • one_port_strategies (dict) – A dictionary of custom strategies to use during the one-port component aggregation step. Default is an empty dictionary.

Returns

network_gasgrid_c – A new gas grid pypsa.Network object with aggregated components based on the bus mapping.

Return type

pypsa.Network

etrago.cluster.gas.run_spatial_clustering_gas(self)[source]

Performs spatial clustering on the gas network using either K-means or K-medoids-Dijkstra algorithm. Updates the network topology by aggregating buses and links, and then performs postprocessing to finalize the changes.

Return type

None

Raises

ValueError – If the selected method is not “kmeans” or “kmedoids-dijkstra”.:

etrago.cluster.snapshot module

This module contains functions for reducing the complexity of a PyPSA network in temporal dimension by a) downsampling to every n-th snapshot b) clustering to typical periods (eg days, weeks) c) clustering to segments of variable length Essentially used is the tsam package ( https://github.com/FZJ-IEK3-VSA/tsam ) developed by Leander Kotzur et al.

etrago.cluster.snapshot.snapshot_clustering(self)[source]

Function to call the snapshot clustering function with the respecting method and settings.

Raises

ValueError – When calling a non-available function.

Return type

None.

etrago.cluster.snapshot.tsam_cluster(timeseries_df, typical_periods=10, how='daily', extremePeriodMethod='None', segmentation=False, segment_no=10, segm_hoursperperiod=24)[source]

Conducts the clustering of the snapshots for temporal aggregation with the respecting method.

Parameters

  • timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

  • typical_periods (int, optional) – Number of clusters for typical_periods. The default is 10.

  • how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

  • segmentation (boolean, optional) – Argument to activate segmenation method. The default is False.

  • segment_no (int, optional) – Number of segments for segmentation. The default is 10.

  • segm_hoursperperiod (int, optional) – Only for segmentation, ensures to cluster to segments considering all snapshots. The default is 24.

Returns

  • df_cluster (pd.DataFrame) – Information on cluster after clustering to typical periods.

  • cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

  • dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

  • hours (int) – Hours per typical period.

  • df_i_h (pd.DataFrame) – Information on cluster after clustering to typical periods.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

etrago.cluster.snapshot.segmentation_extreme_periods(timeseries_df, timeseries, extremePeriodMethod)[source]

Function to consider extreme snapshots while using segmentation.

Parameters

  • timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Raises

ValueError – When calling wrong method to consider extreme values.

Returns

timeseries – Information on segments including extreme snapshots after segmentation.

Return type

pd.DataFrame

etrago.cluster.snapshot.run(network, n_clusters=None, how='daily', segmented_to=False, extreme_periods='None')[source]

Function to call the respecting snapshot clustering function and export the result to a csv-file.

Parameters

  • network (pypsa.Network object) – Container for all network components.

  • n_clusters (int, optional) – Number of clusters for typical_periods. The default is None.

  • how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

  • segmented_to (int, optional) – Number of segments for segmentation. The default is False.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.cluster.snapshot.prepare_pypsa_timeseries(network)[source]

Prepares timeseries and residual load timeseries for clustering.

Parameters

network (pypsa.Network object) – Container for all network components.

Returns

df – Timeseries to be considered when clustering.

Return type

pd.DataFrame

etrago.cluster.snapshot.update_data_frames(network, cluster_weights, dates, hours, timeseries, segmentation)[source]

Updates the snapshots, snapshot weightings and the dataframes based on the original data in the network and the medoids created by clustering these original data.

Parameters

  • network (pypsa.Network object) – Container for all network components.

  • cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

  • dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

  • hours (int) – Hours per typical period.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

  • segmentation (boolean) – Checks if segmentation of clustering to typical periods has been used.

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.cluster.snapshot.skip_snapshots(self)[source]

Conducts the downsapling to every n-th snapshot.

Return type

None.

etrago.cluster.spatial module

spatial.py defines the methods to run spatial clustering on networks.

etrago.cluster.spatial.ext_storage(x)[source]
etrago.cluster.spatial.sum_with_inf(x)[source]
etrago.cluster.spatial.strategies_one_ports()[source]
etrago.cluster.spatial.strategies_generators()[source]

Aggregate network.links and network.links_t after any kind of clustering

Parameters

  • network (pypsa.Network object) – Container for all network components.

  • with_time (bool) – says if the network object contains timedependent series.

  • carriers (list of strings) – Describe which type of carriers should be aggregated. The default is None.

  • strategies (dictionary) – custom strategies to perform the aggregation

Returns

  • new_df – links aggregated based on bus0, bus1 and carrier

  • new_pnl – links time series aggregated

etrago.cluster.spatial.graph_from_edges(edges)[source]

Constructs an undirected multigraph from a list containing data on weighted edges.

Parameters

edges (list) – List of tuples each containing first node, second node, weight, key.

Returns

M

Return type

networkx.classes.multigraph.MultiGraph

etrago.cluster.spatial.gen(nodes, n, graph)[source]

Generator for applying multiprocessing.

Parameters

  • nodes (list) – List of nodes in the system.

  • n (int) – Number of desired multiprocessing units.

  • graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Return type

None

etrago.cluster.spatial.shortest_path(paths, graph)[source]

Finds the minimum path lengths between node pairs defined in paths.

Parameters

  • paths (list) – List of pairs containing a source and a target node

  • graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Returns

df – DataFrame holding source and target node and the minimum path length.

Return type

pd.DataFrame

etrago.cluster.spatial.busmap_by_shortest_path(etrago, scn_name, fromlvl, tolvl, cpu_cores=4)[source]

Creates a busmap for the EHV-Clustering between voltage levels based on dijkstra shortest path. The result is automatically written to the model_draft on the <OpenEnergyPlatform>[www.openenergy-platform.org] database with the name ego_grid_pf_hv_busmap and the attributes scn_name (scenario name), bus0 (node before clustering), bus1 (node after clustering) and path_length (path length). An AssertionError occurs if buses with a voltage level are not covered by the input lists ‘fromlvl’ or ‘tolvl’.

Parameters

  • network (pypsa.Network) – Container for all network components.

  • session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

  • scn_name (str) – Name of the scenario.

  • fromlvl (list) – List of voltage-levels to cluster.

  • tolvl (list) – List of voltage-levels to remain.

  • cpu_cores (int) – Number of CPU-cores.

Return type

None

etrago.cluster.spatial.busmap_from_psql(etrago)[source]

Retrieves busmap from model_draft.ego_grid_pf_hv_busmap on the <OpenEnergyPlatform>[www.openenergy-platform.org] by a given scenario name. If this busmap does not exist, it is created with default values.

Parameters

  • network (pypsa.Network) – Container for all network components.

  • session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

  • scn_name (str) – Name of the scenario.

Returns

busmap – Maps old bus_ids to new bus_ids.

Return type

dict

etrago.cluster.spatial.kmean_clustering(etrago, selected_network, weight, n_clusters)[source]

Main function of the k-mean clustering approach. Maps an original network to a new one with adjustable number of nodes and new coordinates.

Parameters

  • network (pypsa.Network) – Container for all network components.

  • n_clusters (int) – Desired number of clusters.

  • load_cluster (boolean) – Loads cluster coordinates from a former calculation.

  • line_length_factor (float) – Factor to multiply the crow-flies distance between new buses in order to get new line lengths.

  • remove_stubs (boolean) – Removes stubs and stubby trees (i.e. sequentially reducing dead-ends).

  • use_reduced_coordinates (boolean) – If True, do not average cluster coordinates, but take from busmap.

  • bus_weight_tocsv (str) – Creates a bus weighting based on conventional generation and load and save it to a csv file.

  • bus_weight_fromcsv (str) – Loads a bus weighting from a csv file to apply it to the clustering algorithm.

Returns

network – Container for all network components.

Return type

pypsa.Network

etrago.cluster.spatial.dijkstras_algorithm(buses, connections, medoid_idx, cpu_cores)[source]

Function for combination of k-medoids Clustering and Dijkstra’s algorithm. Creates a busmap assigning the nodes of a original network to the nodes of a clustered network considering the electrical distances based on Dijkstra’s shortest path.

Parameters

  • network (pypsa.Network) – Container for all network components.

  • medoid_idx (pandas.Series) – Indices of k-medoids

  • busmap_kmedoid (pandas.Series) – Busmap based on k-medoids clustering

  • cpu_cores (string) – numbers of cores used during multiprocessing

Returns

busmap – Mapping from bus ids to medoids ids

Return type

pandas.Series

etrago.cluster.spatial.kmedoids_dijkstra_clustering(etrago, buses, connections, weight, n_clusters)[source]

Applies a k-medoids clustering on the given network and calls the function to conduct a Dijkstra’s algorithm afterwards for the consideration of the network’s topology in the spatial clustering.

Parameters

  • etrago (Etrago) – An instance of the Etrago class

  • buses (pandas.DataFrame) – DataFrame with information about the buses of the network.

  • connections (pandas.DataFrame) – DataFrame with information about the connections of the network (links or lines).

  • weight (pandas.Series) – Series with the weight for each bus.

  • n_clusters (int) – The number of clusters to create.

Returns

  • Tuple containing

  • busmap (pandas.Series) – Series containing the mapping of buses to their resp. medoids

  • medoid_idx (pandas.Series) – Series containing the medoid indeces