etrago.cluster package

etrago.cluster.disaggregation module

etrago.cluster.electrical module

electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.

etrago.cluster.electrical.adjust_no_electric_network(etrago, busmap, cluster_met, apply_on='grid_model')

Adjusts the non-electric network based on the electrical network (esp. eHV network), adds the gas buses to the busmap, and creates the new buses for the non-electric network.

Parameters:

• etrago (Etrago) – An instance of the Etrago class.

• busmap (dict) – A dictionary that maps old bus_ids to new bus_ids.

• cluster_met (str) – A string indicating the clustering method to be used.

Returns:

• network (pypsa.Network) – Container for all network components of the clustered network.

• busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.cluster_on_extra_high_voltage(etrago, busmap, with_time=True)

Main function of the EHV-Clustering approach. Creates a new clustered pypsa.Network given a busmap mapping all bus_ids to other bus_ids of the same network.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap (dict) – Maps old bus_ids to new bus_ids.

• with_time (bool) – If true time-varying data will also be aggregated.

Returns:

• network (pypsa.Network) – Container for all network components of the clustered network.

• busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.delete_ehv_buses_no_lines(network)

When there are AC buses totally isolated, this function deletes them in order to make possible the creation of busmaps based on electrical connections and other purposes. Additionally, it throws a warning to inform the user in case that any correction should be done.

Parameters:

network (pypsa.network) –

Return type:

None

etrago.cluster.electrical.ehv_clustering(self)

Cluster the network based on Extra High Voltage (EHV) grid.

If ‘active’ in the network_clustering_ehv argument is True, the function clusters the network based on the EHV grid.

Parameters:

self (Etrago object pointer) – The object pointer for an Etrago object.

Return type:

None

etrago.cluster.electrical.select_elec_network(etrago, apply_on='grid_model')

Selects the electric network based on the clustering settings specified in the Etrago object.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• apply_on (str) – gives information about the objective of the output network. If “grid_model” is provided, the value assigned in the args for [“network_clustering”][“cluster_foreign_AC””] will define if the foreign buses will be included in the network. if “market_model” is provided, foreign buses will be always included.

Returns:

elec_networkpypsa.Network

Contains the electric network

n_clustersint

number of clusters used in the clustering process.

Return type:

Tuple containing

etrago.cluster.electrical.unify_foreign_buses(etrago)

Unifies foreign AC buses into clusters using the k-medoids algorithm with Dijkstra distance as a similarity measure.

Parameters:

etrago (Etrago) – An instance of the Etrago class

Returns:

busmap_foreign – A pandas series that maps the foreign buses to their respective clusters. The series index is the bus ID and the values are the corresponding cluster medoid IDs.

Return type:

pd.Series

etrago.cluster.electrical.preprocessing(etrago, apply_on='grid_model')

Preprocesses an Etrago object to prepare it for network clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• apply_on (string) – provide information about the objective of the preprocessing. Which process is going to use the result. e.g. “grid_model”, “market_model”.

Returns:

• network_elec (pypsa.Network) – Container for all network components of the electrical network.

• weight (pandas.Series) – A pandas.Series with the bus weighting data.

• n_clusters (int) – The number of clusters to use for network clustering.

• busmap_foreign (pandas.Series) – The Series object with the foreign bus mapping data.

etrago.cluster.electrical.postprocessing(etrago, busmap, busmap_foreign, medoid_idx=None, aggregate_generators_carriers=None, aggregate_links=True, apply_on='grid_model')

Postprocessing function for network clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• busmap (pandas.Series) – mapping between buses and clusters

• busmap_foreign (pandas.DataFrame) – mapping between foreign buses and clusters

• medoid_idx (pandas.DataFrame) – mapping between cluster indices and medoids

Returns:

clusteringpypsa.network

Network object containing the clustered network

busmappandas.Series

Updated mapping between buses and clusters

Return type:

Tuple containing

etrago.cluster.electrical.weighting_for_scenario(network, save=None)

define bus weighting based on generation, load and storage

Parameters:

• network (pypsa.network) – Each bus in this network will receive a weight based on the generator, load and storages also available in the network object.

• save (str or bool, optional) – If defined, the result of the weighting will be saved in the path supplied here. The default is None.

Returns:

weight – Serie with the weight assigned to each bus to perform a k-mean clustering.

Return type:

pandas.series

etrago.cluster.electrical.run_spatial_clustering(self)

Main method for running spatial clustering on the electrical network. Allows for clustering based on k-means and k-medoids dijkstra.

Parameters:

self – The object pointer for an Etrago object containing all relevant parameters and data

Return type:

None

etrago.cluster.gas module

etrago.cluster.snapshot module

This module contains functions for reducing the complexity of a PyPSA

network in temporal dimension by

1. downsampling to every n-th snapshot

2. clustering to typical periods (eg days, weeks)

c) clustering to segments of variable length Essentially used is the tsam package ( https://github.com/FZJ-IEK3-VSA/tsam ) developed by Leander Kotzur et al.

etrago.cluster.snapshot.snapshot_clustering(self)

Function to call the snapshot clustering function with the respecting method and settings.

Raises:

ValueError – When calling a non-available function.

Return type:

None.

etrago.cluster.snapshot.tsam_cluster(timeseries_df, typical_periods=10, how='daily', extremePeriodMethod='None', segmentation=False, segment_no=10, segm_hoursperperiod=24)

Conducts the clustering of the snapshots for temporal aggregation with the respecting method.

Parameters:

• timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

• typical_periods (int, optional) – Number of clusters for typical_periods. The default is 10.

• how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

• segmentation (boolean, optional) – Argument to activate segmenation method. The default is False.

• segment_no (int, optional) – Number of segments for segmentation. The default is 10.

• segm_hoursperperiod (int, optional) – Only for segmentation, ensures to cluster to segments considering all snapshots. The default is 24.

Returns:

• df_cluster (pd.DataFrame) – Information on cluster after clustering to typical periods.

• cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

• dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

• hours (int) – Hours per typical period.

• df_i_h (pd.DataFrame) – Information on cluster after clustering to typical periods.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

etrago.cluster.snapshot.segmentation_extreme_periods(timeseries_df, timeseries, extremePeriodMethod)

Function to consider extreme snapshots while using segmentation.

Parameters:

• timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Raises:

ValueError – When calling wrong method to consider extreme values.

Returns:

timeseries – Information on segments including extreme snapshots after segmentation.

Return type:

pd.DataFrame

etrago.cluster.snapshot.run(network, n_clusters=None, how='daily', segmented_to=False, extreme_periods='None')

Function to call the respecting snapshot clustering function and export the result to a csv-file.

Parameters:

• network (pypsa.Network object) – Container for all network components.

• n_clusters (int, optional) – Number of clusters for typical_periods. The default is None.

• how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

• segmented_to (int, optional) – Number of segments for segmentation. The default is False.

• extremePeriodMethod ({'None','append','new_cluster_center',) – ‘replace_cluster_center’}, optional Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Returns:

network – Container for all network components.

Return type:

pypsa.Network object

etrago.cluster.snapshot.prepare_pypsa_timeseries(network)

Prepares timeseries and residual load timeseries for clustering.

Parameters:

network (pypsa.Network object) – Container for all network components.

Returns:

df – Timeseries to be considered when clustering.

Return type:

pd.DataFrame

etrago.cluster.snapshot.update_data_frames(network, cluster_weights, dates, hours, timeseries, segmentation)

Updates the snapshots, snapshot weightings and the dataframes based on the original data in the network and the medoids created by clustering these original data.

Parameters:

• network (pypsa.Network object) – Container for all network components.

• cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

• dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

• hours (int) – Hours per typical period.

• timeseries (pd.DataFrame) – Information on segments after segmentation.

• segmentation (boolean) – Checks if segmentation of clustering to typical periods has been used.

Returns:

network – Container for all network components.

Return type:

pypsa.Network object

etrago.cluster.snapshot.skip_snapshots(self)

Conducts the downsapling to every n-th snapshot.

Return type:

None.

etrago.cluster.spatial module

spatial.py defines the methods to run spatial clustering on networks.

etrago.cluster.spatial.nan_links(x)

etrago.cluster.spatial.ext_storage(x)

etrago.cluster.spatial.sum_with_inf(x)

etrago.cluster.spatial.strategies_buses()

etrago.cluster.spatial.strategies_lines()

etrago.cluster.spatial.strategies_one_ports()

etrago.cluster.spatial.strategies_generators()

etrago.cluster.spatial.strategies_links()

etrago.cluster.spatial.group_links(network, with_time=True, carriers=None, cus_strateg={})

Aggregate network.links and network.links_t after any kind of clustering

Parameters:

• network (pypsa.Network object) – Container for all network components.

• with_time (bool) – says if the network object contains timedependent series.

• carriers (list of strings) – Describe which type of carriers should be aggregated. The default is None.

• strategies (dictionary) – custom strategies to perform the aggregation

Returns:

• new_df – links aggregated based on bus0, bus1 and carrier

• new_pnl – links time series aggregated

etrago.cluster.spatial.graph_from_edges(edges)

Constructs an undirected multigraph from a list containing data on weighted edges.

Parameters:

edges (list) – List of tuples each containing first node, second node, weight, key.

Returns:

M

Return type:

networkx.classes.multigraph.MultiGraph

etrago.cluster.spatial.gen(nodes, n, graph)

Generator for applying multiprocessing.

Parameters:

• nodes (list) – List of nodes in the system.

• n (int) – Number of desired multiprocessing units.

• graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Return type:

None

etrago.cluster.spatial.shortest_path(paths, graph)

Finds the minimum path lengths between node pairs defined in paths.

Parameters:

• paths (list) – List of pairs containing a source and a target node

• graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Returns:

df – DataFrame holding source and target node and the minimum path length.

Return type:

pd.DataFrame

etrago.cluster.spatial.busmap_by_shortest_path(etrago, fromlvl, tolvl, cpu_cores=4)

Creates a busmap for the EHV-Clustering between voltage levels based on dijkstra shortest path. The result is automatically written to the model_draft on the <OpenEnergyPlatform>[www.openenergy-platform.org] database with the name ego_grid_pf_hv_busmap and the attributes scn_name (scenario name), bus0 (node before clustering), bus1 (node after clustering) and path_length (path length). An AssertionError occurs if buses with a voltage level are not covered by the input lists ‘fromlvl’ or ‘tolvl’.

Parameters:

• network (pypsa.Network) – Container for all network components.

• session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

• fromlvl (list) – List of voltage-levels to cluster.

• tolvl (list) – List of voltage-levels to remain.

• cpu_cores (int) – Number of CPU-cores.

Return type:

None

etrago.cluster.spatial.busmap_ehv_clustering(etrago)

Generates a busmap that can be used to cluster an electrical network to only extra high voltage buses. If a path to a busmap in a csv file is passed in the arguments, it loads the csv file and returns it.

Parameters:

etrago (Etrago) – An instance of the Etrago class

Returns:

busmap – Maps old bus_ids to new bus_ids.

Return type:

dict

etrago.cluster.spatial.kmean_clustering(etrago, selected_network, weight, n_clusters)

Main function of the k-mean clustering approach. Maps an original network to a new one with adjustable number of nodes and new coordinates.

Parameters:

• network (pypsa.Network) – Container for all network components.

• n_clusters (int) – Desired number of clusters.

• load_cluster (boolean) – Loads cluster coordinates from a former calculation.

• line_length_factor (float) – Factor to multiply the crow-flies distance between new buses in order to get new line lengths.

• remove_stubs (boolean) – Removes stubs and stubby trees (i.e. sequentially reducing dead-ends).

• use_reduced_coordinates (boolean) – If True, do not average cluster coordinates, but take from busmap.

• bus_weight_tocsv (str) – Creates a bus weighting based on conventional generation and load and save it to a csv file.

• bus_weight_fromcsv (str) – Loads a bus weighting from a csv file to apply it to the clustering algorithm.

Returns:

network – Container for all network components.

Return type:

pypsa.Network

etrago.cluster.spatial.dijkstras_algorithm(buses, connections, medoid_idx, cpu_cores)

Function for combination of k-medoids Clustering and Dijkstra’s algorithm. Creates a busmap assigning the nodes of a original network to the nodes of a clustered network considering the electrical distances based on Dijkstra’s shortest path.

Parameters:

• network (pypsa.Network) – Container for all network components.

• medoid_idx (pandas.Series) – Indices of k-medoids

• busmap_kmedoid (pandas.Series) – Busmap based on k-medoids clustering

• cpu_cores (string) – numbers of cores used during multiprocessing

Returns:

busmap – Mapping from bus ids to medoids ids

Return type:

pandas.Series

etrago.cluster.spatial.kmedoids_dijkstra_clustering(etrago, buses, connections, weight, n_clusters)

Applies a k-medoids clustering on the given network and calls the function to conduct a Dijkstra’s algorithm afterwards for the consideration of the network’s topology in the spatial clustering.

Parameters:

• etrago (Etrago) – An instance of the Etrago class

• buses (pandas.DataFrame) – DataFrame with information about the buses of the network.

• connections (pandas.DataFrame) – DataFrame with information about the connections of the network (links or lines).

• weight (pandas.Series) – Series with the weight for each bus.

• n_clusters (int) – The number of clusters to create.

Returns:

• Tuple containing

• busmap (pandas.Series) – Series containing the mapping of buses to their resp. medoids

• medoid_idx (pandas.Series) – Series containing the medoid indeces

etrago.cluster.spatial.drop_nan_values(network)

Drops nan values after clustering an replaces output data time series with empty dataframes

Parameters:

network (pypsa.Network) – Container for all network components.

Return type:

None.