Welcome to eTraGo’s documentation!

_images/etrago_logo.png

Warning

Note, eTraGo and its documentation is in continuous development.

About eTraGo

eTraGo stands for electric Transmission Grid optimization.

The python package eTraGo provides optimization strategies of flexibility options for transmission grids based on PyPSA. A peculiarity in this context is that the German transmission grid is described by the 380, 220 and 110 kV voltage levels. Conventionally the 110kV grid is part of the distribution grid. The integration of the transmission and ‘upper’ distribution grid is part of eTraGo.

The focus of optimization are flexibility options with a special focus on energy storage and grid expansion measures.

Research projects

This software project was initially developed in the research project open_eGo. It is constantly further developed in different reserach projects, e.g. eGon and PoWerD.

The OpenEnergy Platform

Within the open_eGo project we developed the OpenEnergy Platform which this software is using in order to get and store the in- and output data. Before you start to calculate a registration on the platform is needed. For more information see openenergy-platform and login.

The OpenEnergy platform mainly addresses students, researchers and scientists in the field of energy modelling and analytics as well as interested persons in those fields. The platform provides great tools to make your energy system modelling process transparent. All data of the open_eGo project are stored at this platform. Learn more about the database access.

Tool overview

_images/ego_tools.svg

eDisGo

The python package eDisGo provides a toolbox for analysis and optimization of distribution grids. It is closely related to the python project Ding0 as this project is currently the single data source for eDisGo providing synthetic grid data for whole Germany. Learn more here.

eGo

The python package eGo is a toolbox and application which connects the tool eTraGo (optimization of flexibility options at transmission grid level) and eDisGo (optimization of distribution grids). All those python packages were initially developed in the research project open_eGo. Learn more here.

Data model creation

For the eGon project the python-tool eGon-data was implemented, which creates input data for the optimization tools eTraGo, ding0 and eDisGo and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses.

eGon-data is a further development of the Data processing developed in the former research project open_eGo. It aims for an extensions of the data models as well as for a better replicability and manageability of the data preparation and processing. The resulting data set serves as an input for the optimization tools eTraGo, ding0 and eDisGo and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses.

ego.io

The ego.io serves as a SQLAlchemy Interface to the OpenEnergy database (oedb). The oedb table ORM objects are defined here and small helpers for io tasks are contained. Learn more here.

Dingo

The DIstribution Network GeneratOr (Ding0) is a tool to generate synthetic medium and low voltage power distribution grids based on open (or at least accessible) data. Learn more here.

LICENSE

© Copyright 2015-2023 Flensburg University of Applied Sciences, Europa-Universität Flensburg, Centre for Sustainable Energy Systems and DLR-Institute for Networked Energy Systems

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see www.gnu.org/licenses.

Installation

eTraGo is designed as a Python package therefore it is mandatory to have Python 3 installed. If you have a working Python3 environment, use pypi to install the latest eTraGo version. We highly recommend you to use a virtual environment. Use following pip command in order to install eTraGo.

$ pip3 install eTraGo

Using a virtual environment

Before installing eTraGo, you create a virtual environment (where you like it) and activate it:

$ virtualenv venv --clear -p python3.8
$ source venv/bin/activate
$ cd venv

Inside your activated virtual environment you can install eTraGo with the pip command, as previously explained.

Linux and Ubuntu

The Package eTraGo is tested with Ubuntu 16.04, 18.04, 20.04 and 22.04 inside the virtual environments of virtualenv. The installation is shown above.

Windows or Mac OSX users

For Windows and/or Mac OSX user we highly recommend to install and use Anaconda for your Python3 installation. First install Conda including python 3.8 or higher version from https://www.anaconda.com/download/ and open an anaconda prompt as administrator and run:

$ conda config --add channels conda-forge
$ conda create -n etrago_env python=3.8
$ conda activate etrago_env
$ pip install eTraGo

The full Documentation can be found on this page . We use Anaconda with an own environment in order to reduze problems with Packages and different versions on our system. Learn more about Anacona environments.

Setup database connection

The package ego.io will be installed automatically when eTraGo is installed. The egoio gives you python SQL-Alchemy representations of the OpenEnergy-Database(oedb) and access to it by using the oedialect, which is a SQL-Alchemy binding Python package for the REST-API used by the OpenEnergy Platform (OEP).

In order to connect eTraGo via the oedialect with the oedb you have to create an account at openenergy-platform.org/login. You can name the ‘db’ argument of the ‘args’ of the etrago.appl.etrago() as you wish. Once the etrago.appl.etrago() is executed you will be asked to enter how you want to connect to which database. If you want to use the oedialect enter the following connection parameter. For <username> and <token> you have to take your credentials which you obtained by registering at openenergy-platform.org/login.

Your API access / login data will be saved in the folder .egoio in the file config.ini. Consequently, in the config.ini you can also change your connection parameters or add new ones. In the following you can see how the config.ini looks like when you use the oedialect, a local postgresql database or the old psycopg2 developer connection.

Once you have created a connection (which is saved in the config.ini) you do not have to enter the connection parameter again. The software will take the connection parameter which corresponds to the entry at the ‘db’ argument.

oedialect connection

[oedb]
dialect  = oedialect
username = <username>
database = oedb
host     = openenergy-platform.org
port     = 80
password = <token>

Local database connection

[local]
username = YourOEDBUserName
database = YourLocalDatabaseName
host = localhost or 127.0.0.1
port = 5433
pw = YourLocalPassword

Old developer connection

[oedb]
username = YourOEDBUserName
database = oedb
host = oe2.iws.cs.ovgu.de
port = 5432
pw = YourOEDBPassword

How to use eTraGo?

After you installed eTraGo you would typically start optimization runs by executing the ‘appl.py’ which is situated in ./eTrago/etrago/ (e.g by python3 appl.py from the terminal).

eTraGo doesn’t have a graphical user interface, the ‘appl.py’ is used as a simple user interface which can be edited with the preferred python-editor. Here parameters, calculation methods and scenario settings are set in a python dictionary called ‘args’. To run the desired calculation, it is crucial to understand these parameters. In addition, some of them contradict the usage of others. You find the documentation of all defined parameters from the ‘args’ here: etrago.appl.run_etrago().

Alternatively, the ‘args’ dictionary can be edited in a json-file. Then the path to the json-file has to be set in the initilization of the Etrago-object (etrago.tools.network.Etrago). Once a path is given the ‘args’ dictionary within the ‘appl.py’ is ignored and replaced by the ‘args’ of the json-file.

The appl.py contains the etrago.appl.run_etrago() function which uses the defined ‘args’ dictionary to start the desired calculation.

To improve the performance of the optimization of the selected solver, you might want to use solver options (part of ‘args’). For gurobi the most used ones are described here.

For more specific or extensive changes you are highly invited to write code and add new functionalities.

Once the calculation has finished the PyPSA network of the Etrago-object will contain all results. Some main results (e.g. anuual system costs) are calculated by etrago.calc_results() and can be accesed via ‘etrago.results’. You can use several plotting functions from the etrago.tools.plot() in order to visualize the results. For example the etrago.tools.plot.plot_grid() can be used to plot relative line loading in % or the optimized expansion of all AC lines and DC links of the network.

To save the results you can write them to csv files. These functionalites can be specified also in the ‘args’ dictionary.

Examples and tutorial notebooks

eTraGo version 0.5.1: etrago_OpenMod_Zuerich18.

eTraGo version 0.9: Tutorial notebook will be available in the next release.

Theoretical Background

Definitions and Units

eTraGo is based on the open source tool PyPSA and uses its definitions and units.

Assumptions on Data

eTraGo fetches the input data from the OpenEnergy Platform. The data includes electricity and gas grid topology as well as data on energy supply and load for the considered sectors (electricity, gas, heat and e-mobility) plus data on flexibility potential deriving from those sectors e.g. Dynamic Line Rating, Demand Side Management and flexibility potentials arising from e-mobility. More details on the data model can be found in the documentaton of eGon-data.

At the moment, there are two scenarios available basing on scenario C2035 of the network expansion plan ([NEP]), version 2021. The base one is called eGon2035. To analyse the effect of flexibility options, there is an eGon2035_lowflex scenario available which depicts a lower penetration of flexibilities. More scenarios are being developed. The eGon100RE scenario is being implemented which is characterised by a 100% renewable generation. Analog to the scenario above, a eGon100RE_lowflex scenario will be available.

You can see the modeling concepts of the scenarios in the figure below. The components marked green have exogenous capacity and endogenous dispatch whereas the components marked in red are optimised endogenously in capacity and dispatch.

_images/modelling_concept.png

Methods

Optimisation with PyPSA

Within eTraGo, the fetched data model is translated into a PyPSA-network. The optimisation is performed with a linear approximation assuming eTraGo to fulfill the assumptions to perfom a LOPF (as those are small voltage angle differences, branch resistances negligible to their reactances, voltage magnitudes can be kept at nominal values) since it focuses on the extra-high and high voltage levels. As objective value of the optimisation, the overall system costs are considered.

With the argument ‘pf_post_lopf’, after the LOPF a non-linear power flow simulation can be conducted.

Complexity Reduction

The data model is characterised by a high spatial (abou 8,000 electrical and 600 gas nodes) and temporal resolution (8,760 timesteps). To reduce the complexity of the resulting optimisation problem, several methods can be applied.

Reduction in spatial dimension:

The ehv clustering maps all electrical nodes with a voltage level below the extra-high voltage level to their nearest neighboring node in the extra-high voltage level with the Dijkstra’s algorithm (110 kV —> 220 / 380 kV).

The k-means Clustering reduces the electrical or gas network to an adjustable number of nodes by considering the geographical position of the respective nodes. This method has been implemented within PyPSA by [Hoersch].

The k-medoids Dijkstra Clustering aggregates nodes considering the network topology. First, a k-medoids Clustering is used dividing the original nodes of the network into groups by their geographical positions while identifiying the geographical medoid nodes per cluster. Afterwards, the original nodes in the original network are assigned to the former identified medoids considering the original network’s topology applying a Dijkstra’s algorithm considering the line lengths. Afterall, the original nodes are represented by one aggregated node per cluster at the position of the former identified medoid node.

In general, the clustering of the sector-coupled system is divided into two steps: First, the electrical and gas grid are clustered independently using one of the methods described above. Afterwards, nodes of the other sectors (hydrogen, heat, e-mobility and DSM nodes) are mapped according to their connection to electricity or gas buses and aggregated to one node per carrier.

After optimising the spatially reduced network, a spatial disaggregation can be conducted.

Reduction in temporal dimension:

The method Skip Snapshots implies a downsampling to every nth time step. The considered snapshots are weighted respectively to account for the analysis of one whole year.

By using the method called Segmentation, a hierarchical clustering of consecutive timesteps to segments with variable lengths is applied [Pineda].

The Snapshot Clustering on Typical Periods implies a hierarchical clustering of time periods with a predefined length (e.g. days or weeks) to typical periods. Those typical periods are weighted according to the number of periods in their cluster. This method optionally includes the linkage of the typical periods in a second time layer to account for the intertemporal dependencies following [Kotzur].

By applying a 2-level-approach, a temporal disaggregation can be conducted. This means optimising dispatch using the fullcomplex time series in the second step after having optimised grid and storage expansion using the complexity-reduced time series in the first step.

Grid and Storage / Store expansion

The grid expansion is realized by extending the capacities of existing lines and substations. These capacities are considered as part of the optimisation problem whereby the possible extension is unlimited. With respect to the different voltage levels and lengths, MVA-specific costs are considered in the optimisation.

As shown in the figure above, several options to store energy are part of the modeling concept. Extendable batteries (modeled as storage units) are assigned to every node in the electrical grid. A minimum installed capacity is being considered to account for home batteries ([NEP]). The expansion and operation is part of the optimisation. Furthermore, two types of hydrogen stores (modeled as stores) are available. Overground stores are optimised in operation and dispatch without limitations whereas underground stores depicting saltcaverns are limited by geographical conditions ([BGR]). Additionally, heat stores part of the optimisation in terms of power and energy without upper limits.

Miscellaneous Features

Several features were developed to enhance the functionality of eTraGo.

To customize computation settings, ‘solver_options’ and ‘generator_noise’ should be adapted. The latter adds a reproducible small random noise to the marginal costs of each generator in order to prevent an optima plateau. The specific solver options depend on the applied solver (e.g. Gurobi, CPLEX or GLPK).

In ‚extendable‘ you can adapt the type of components you want to be optimised in capacity and set upper limits for gird expansion inside Germany and of lines to foreign countries.

The ‚extra_functionality‘-argument allows to consider extra constraints like limits for energy imort and export or minimal renewable shares in generation.

‘branch_capacity_factor’ adds a factor to adapt all line capacities in order to consider (n-1) security. Because the average number of HV systems is much smaller than the one of eHV lines, you can choose factors for ‘HV’ and ‘eHV’ separately.

The ‘load_shedding’-argument is used for debugging complex grids in order to avoid infeasibilities. It introduces a very expensive generator at each bus to meet the demand. When optimising storage units and grid expansion without limiting constraints, the need for load shedding should not be existent.

With ‘foreign_lines‘ you can adapt the foreign lines to be modeled as DC-links (e.g. to avoid loop flows).

References

NEP(1,2)

Übertragungsnetzbetreiber Deutschland (2021): Netzentwicklungsplan Strom 2035, Version 2021, 1. Entwurf. 2021.

Hoersch

Jonas Hoersch et al. (2017): The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios. 2017. https://arxiv.org/pdf/1705.07617.pdf

Pineda

Salvador Pineda et al. (2018): Chronological Time-Period Clustering for Optimal Capacity Expansion Planning With Storage. 2018. https://ieeexplore.ieee.org/document/8369128

Kotzur

Leander Kotzur et al. (2018): Time series aggregation for energy system design: Modeling seasonal storage. 2018. https://arxiv.org/pdf/1710.07593.pdf

BGR

Bundesanstalt fuer Geowissenschaften und Rohstoffe et al. (2020): nSpEE-DS - Teilprojekt Bewertungskriterien und Potenzialabschätzung. 2020. https://www.bgr.bund.de/DE/Themen/Nutzung_tieferer_Untergrund_CO2Speicherung/Downloads/InSpeeDS_TP_Bewertungskriterien.pdf?__blob=publicationFile&v=3

Developer notes

Installation for Developers

Note

Installation is primarly tested on (Ubuntu like) linux OS.

  1. If you like, create a virtual environment (where you like it) and activate it (if you do not use venv start with 2.):

$ virtualenv --clear -p python3.8  etrago``
$ cd etrago/
$ source bin/activate
  1. Clone the source code from github:

$ git clone https://github.com/openego/eTraGo

You can checkout to the dev branch and create new feature branches. For the correct work-flow, please mind the Dreissen Branching Model

  1. Use the pip -e to install eTraGo directly from the cloned repository:

$ pip3 install -e /path/to/eTraGo/

What’s New

These are new features and improvements of note in each release.

Release 0.9.0 (XXXXX, 2023)

Added features

  • eTraGo is now compatible with Python 3.8

  • eTraGo can now import and optimize networks that include other energy sectors such as gas, heating and mobility

  • Various flexibility options from different energy sectors can be considered in the optimization:

  • Weather dependent capacity of transmission lines (Dynamic Line Rating)

  • Demand Side Management

  • Flexible charging of electric vehicles

  • Heat and hydrogen stores

  • Power2Hydrogen, Hydrogen2Power

  • Methanation and Steam Methane Reforming

  • eTraGo arguments can now be partially provided and updated

  • eTraGo can now import datamodels from databases without using the ego.io

  • Existing clustering methods were adapted to be able to reduce the complexity of not electrical sectors

  • Improvement of the ehv clustering (much faster now)

  • A new clustering method named “k-medoids Dijkstra Clustering” (can be called by “kmedoids-dijkstra”) was implemented. This method considers the electrical distance between the buses in the network. It is also available for the methane grid.

  • It is possible to select if foreign buses are considered or not during the clustering process.

  • The number of CPUs used to perform the clustering can be provided by the user.

  • Some more options are available to conduct a reduction in temporal dimension:

  • segmentation: clustering of adjacent hours to segments of variable length

  • clustering to typical periods extended to cluster on weeks and months

  • A temporal disaggregation is available through a 2-level-approach including a dispatch optimization on the temporally fullcomplex model. To limit the RAM usage, you can optionally divide the optimisation problem into a chosen number of slices.

  • New plotting functions to visualize the optimization results from all the included energy sectors were implemented

  • Functions to analyze results were updated to consider new sectors

Release 0.8.0 (April 8, 2021)

eTraGo has now a more object-oriented programming design.

Added features

  • eTraGo uses PyPSA version 0.17.1 directly, the fork is not needed anymore. The updated pypsa version includes various features, e.g. running a lopf without using pyomo which is faster and needs less memory.

  • (n-1)-security factors are set as line/transformer parameters s_max_pu instead of adding the additional argument s_nom_original

  • There is now one central plotting function for all grid topology plots which also allows to combine different results (e.g. plot storage expansion and line expansion at once)

  • eTraGo is now compatible to Python3.7

  • A bug in setting the line_length_factor in kmeans clustering is fixed.

Release 0.7.2 (Juni 15, 2020)

A minor release adding the following features.

Added features

  • for single use of eTraGo (not as a sub-module of eGo), we recommend to use the newest minor data release ‘gridversion’: ‘v0.4.6’. This data release includes some minor bug fixes but it is not consistent with the data on the MV and LV levels. Hence, the modelling results are only adequate for the HV and EHV level applying solely the tool eTraGo.

  • snapshot clustering includes now an approach to model seasonal storage as in Kotzur et al, 2018 ( https://www.sciencedirect.com/science/article/pii/S0306261918300242 ). Moreover the method may include extreme periods using an option of the tsam package.

  • osm maps can now be used for background plotting

  • the iterate_lopf function enables to adequately model the reactances when expanding the grid

  • important bug fix for the adjustment of reactances when harmonizing the voltage level when applying the k-means network clustering

  • multiple extra_functionalities can be called easily called now at once

  • various minor changes such as specifying installation requires for flawless performance

Release 0.7.1 (October 25, 2018)

A minor release adding new options for additional constraints, modelling assumptions and plotting.

Added features

  • Two extra functionalities were introduced in order to apply constraints concerning a minimal share of renewable energy and a global upper bound for grid expansion. You can activate these functions in the ‘args’ of the etrago() function.

  • The branch_capacity_factor can now be defined separately for the high and extra high voltage level in order to address the (n-1) criteria more accurately.

  • There are some more plotting functions e.g. plotting the state-of-charge and dispatch of storage units.

  • Storage capacities in foreign countries can easily be be optimized.

  • By default the maximum expansion of each line and transformer is set to four times its original capacity. Being an argument of the extendable() function it can be easily adjusted.

  • k-means clustered results can now also be exported to the oedb.

Release 0.7.0 (September 6, 2018)

eTraGo is able to produce feasible non-linear power flows based on optimization results and allows the disaggregation of clustered results to original spatial complexities.

Added features

  • The pf_post_lopf function was improved. Due to changes in the data set now the non-linear power flow (pf) creates feasible solutions. If network optimization is turned on, a second lopf which regards the updated reactances and optimizes only dispatch is performed before the pf is executed.

  • The disaggregation method was included. When using a network clustering method to reduce the spatial complexity of the given network, a disaggregation method can be used afterwards to distribute the nodal results (generation and storage timeseries) to the original complexity. The method ‘disaggregation’: ‘uniform’ can be used as an interface functionality for distribution grid planning tools like eDisGo.

  • For the network expansion it is now additionally possible to only optimize the German power lines or only the crossborder lines. Moreover one can choose to optimize only a predefined set of power lines which are identified by a worst-case analysis beforehand.

  • Intertemporal constraints can be applied to certain power plants. For different technologies certain parameters i.e. ‘start_up_cost’, ‘start_up_fuel’, ‘min_up_time’ and ‘min_down_time’ are defined in the ramp_limits function.

  • Crossborder lines can now easily be modelled as ‘DC’ links. Moreover the capacities of these lines can be adjusted with respect to a ACER report on thermal as well as net transfer capacities.

  • Thanks to @jankaeh manually the grid topology within the cities Stuttgart, Munich and Hannover was improved. Perspectively this function should be obsolete when openstreetmap and/or osmTGmod get better data coverage.

  • As an alternative to the normal editing of the calcualtion settings (args) within the appl.py it is now possible to load an args.json file.

Release 0.6.1 (Juli 18, 2018)

eTraGo works with pypi and is suitable for eGo 0.2.0

Added features

  • An installation issue when installing from pypi was fixed.

  • The random noise function was improved. Now you set a (reproducible) random seed.

  • snapshot.weightings are used within the plotting functions

  • bug fix for k-means clustering with respect to the aggregation of p_max_pu values of variable generators. They are weighted by their p_nom now.

Release 0.6 (June 27, 2018)

eTraGo now enables combined grid and storage expansion, snapshot clustering and the consideration of exogenous grid expansion.

Added features

  • A part from optimizing the investment of storages it is now also possible to optimize grid expansion investments. In this context we added an argument ‘extendable’ which expects an array of the different component types you want to optimize. This argument corresponds to functions in the new extendable.py file. It is possible to choose from expansion strategies which are defined within that sub-package. Capital costs for new grid components can be defined and are annualized by means of interest rate, component lifetime and operation period.

  • The k-means network clustering (‘network_clustering_kmeans’) has been refactored. It is now possible to reproduce busmaps by csv-importing already defined busmaps. Consequently it is possible to write busmaps. Here the argument ‘load_cluster’ was introduced. Moreover it is possible read and write bus_weightings. This helps to e.g. run a future scenario but using a bus weighting of the status quo. Moreover, the remove_stubs function from PyPSA is now easily usable in eTraGo.

  • The snapshot_clustering can now be used in order to reduce the temporal complexity. Typical days are taken in order to represent the entire year. Here the package tsam (developed by Leander Kotzur) is used. Standardly a hierarchical clustering method is used which is e.g. described by Nahmacher et al. ( see: https://www.sciencedirect.com/science/article/pii/S0360544216308556 ).

  • Scenario variations from the oedb can be introduced. The argument ‘scn_extension’ will activate an extension scenario which adds components such as lines or generator to the base scenario. The ‘scn_decommissioning’ argument states whether you want to remove existing components. Right now, in the oedb two scenarios are accessible which represent the grid expansion (and the corresponding removal of existing equipment) planned by the German network development plan.

  • Our data model at the oedb represents Germany and the electrical neighbors. If you consider planned grid expansion to Norway and Belgium you would most probably want to include also these countries as electrical neighbors including their aggregated generation and demand characteristics. The argument ‘add_Belgium_Norway’ therefore was introduced. Once activated it will add the countries to the model.

  • DC links are now also modelled as PyPSA DC links. Consequently Sweden is now connected by a DC link.

Other changes

  • The plotting sub-package was amplified and enhanced by new plotting functions and improvements of existing ones (e. g. introduced a legend for the plotting function storage_expansion())

  • The code complies now mostly with the pep8 standard

  • Documentation was improved (but ongoing work in progress) considering doc strings for functions and the rtd-documentation web site

  • The io was slightly reformatted and restructured.

Release 0.5.1 (February 01, 2018)

eTraGo works with ego.io 0.3.0

Added features

  • Result export to oedb functionality was improved. Now, a safe tag can be set in order to state that the result set shall be versioned and moved to the schema ‘grid’.

  • the new database sessionmaker of ego.io 0.3.0 is enabled which gets rid of dependency to oemof.db.

  • it is possible to skip snapshots in order to ingenuously simplify the problem.

Other news

  • eTraGo will be available on PyPI

  • eTraGo 0.5.1 will be used for eGo 0.0.1

Release 0.5 (December 08, 2017)

eTraGo works with PyPSA 0.11.0.

Added features

  • Readthedocs documentation

  • Result export to postgresql database in particular to the open energy data base (oedb)

  • parallelisation function hands over SOC of storages to following problem.

  • New plot for displaying voltage deviations at network buses.

  • Line loading plot displays the direction of power flows.

Bug fixes

  • k-means clustering got a more suitable scaling factor concerning calculations on the 110kV grid.

  • K-means weighting of the buses is a bit more robust to changes in the data structure.

  • the list of carriers is imported again to the pypsa network from the oedb data model.

Other changes

  • update PyPSA Version from 0.8.0 to PyPSA 0.11.0

Release 0.4 (October 12, 2017)

eTraGo integrates ego.powerflow functionalities

Release 0.4 is mainly the merging of ego.powerflow into eTraGo. Additionally, some restructuring has been carried out, plotting functions have been updated and the first approach for a documentation was set up.

Other changes

  • merging of ego.powerflow into eTraGo

Release 0.3 (September 8, 2017)

Release introducing k-means clustering and several additional functionalities

Added features

  • k-means clustering

Release 0.2 (July 20, 2017)

This is the version 0.2 of eTraGo.

This new version shall be install via pip and a setup.py. Moreover there has been minor adjustments such as the creation of a callable etrago function.

Added features

  • install via pip and a setup.py

  • callable etrago function

Release 0.1 (June 30, 2017)

First release of eTraGo

API

etrago.cluster package

etrago.cluster.disaggregation module

etrago.cluster.electrical module

electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.

etrago.cluster.electrical.adjust_no_electric_network(etrago, busmap, cluster_met)[source]

Adjusts the non-electric network based on the electrical network (esp. eHV network), adds the gas buses to the busmap, and creates the new buses for the non-electric network.

Parameters
  • etrago (Etrago) – An instance of the Etrago class.

  • busmap (dict) – A dictionary that maps old bus_ids to new bus_ids.

  • cluster_met (str) – A string indicating the clustering method to be used.

Returns

  • network (pypsa.Network) – Container for all network components of the clustered network.

  • busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.cluster_on_extra_high_voltage(etrago, busmap, with_time=True)[source]

Main function of the EHV-Clustering approach. Creates a new clustered pypsa.Network given a busmap mapping all bus_ids to other bus_ids of the same network.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • busmap (dict) – Maps old bus_ids to new bus_ids.

  • with_time (bool) – If true time-varying data will also be aggregated.

Returns

  • network (pypsa.Network) – Container for all network components of the clustered network.

  • busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.

etrago.cluster.electrical.delete_ehv_buses_no_lines(network)[source]

When there are AC buses totally isolated, this function deletes them in order to make possible the creation of busmaps based on electrical connections and other purposes. Additionally, it throws a warning to inform the user in case that any correction should be done.

Parameters

network (pypsa.network) –

Return type

None

etrago.cluster.electrical.ehv_clustering(self)[source]

Cluster the network based on Extra High Voltage (EHV) grid.

If network_clustering_ehv argument is True, the function clusters the network based on the EHV grid.

Parameters

self (Etrago object pointer) – The object pointer for an Etrago object.

Return type

None

etrago.cluster.electrical.select_elec_network(etrago)[source]

Selects the electric network based on the clustering settings specified in the Etrago object.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

elec_networkpypsa.Network

Contains the electric network

n_clustersint

number of clusters used in the clustering process.

Return type

Tuple containing

etrago.cluster.electrical.unify_foreign_buses(etrago)[source]

Unifies foreign AC buses into clusters using the k-medoids algorithm with Dijkstra distance as a similarity measure.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

busmap_foreign – A pandas series that maps the foreign buses to their respective clusters. The series index is the bus ID and the values are the corresponding cluster medoid IDs.

Return type

pd.Series

etrago.cluster.electrical.preprocessing(etrago)[source]

Preprocesses an Etrago object to prepare it for network clustering.

Parameters

etrago (Etrago) – An instance of the Etrago class

Returns

  • network_elec (pypsa.Network) – Container for all network components of the electrical network.

  • weight (pandas.Series) – A pandas.Series with the bus weighting data.

  • n_clusters (int) – The number of clusters to use for network clustering.

  • busmap_foreign (pandas.Series) – The Series object with the foreign bus mapping data.

etrago.cluster.electrical.postprocessing(etrago, busmap, busmap_foreign, medoid_idx=None)[source]

Postprocessing function for network clustering.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • busmap (pandas.Series) – mapping between buses and clusters

  • busmap_foreign (pandas.DataFrame) – mapping between foreign buses and clusters

  • medoid_idx (pandas.DataFrame) – mapping between cluster indices and medoids

Returns

clusteringpypsa.network

Network object containing the clustered network

busmappandas.Series

Updated mapping between buses and clusters

Return type

Tuple containing

etrago.cluster.electrical.weighting_for_scenario(network, save=None)[source]

define bus weighting based on generation, load and storage

Parameters
  • network (pypsa.network) – Each bus in this network will receive a weight based on the generator, load and storages also available in the network object.

  • save (str or bool, optional) – If defined, the result of the weighting will be saved in the path supplied here. The default is None.

Returns

weight – Serie with the weight assigned to each bus to perform a k-mean clustering.

Return type

pandas.series

etrago.cluster.electrical.run_spatial_clustering(self)[source]

Main method for running spatial clustering on the electrical network. Allows for clustering based on k-means and k-medoids dijkstra.

Parameters

self – The object pointer for an Etrago object containing all relevant parameters and data

Return type

None

etrago.cluster.gas module

gas.py defines the methods to cluster gas grid networks spatially for applications within the tool eTraGo.

etrago.cluster.gas.preprocessing(etrago)[source]

Preprocesses the gas network data from the given Etrago object for the spatial clustering process of the CH4 grid.

Parameters

etrago (Etrago) – An instance of the Etrago class

Return type

None

Raises

ValueError – If settings[“n_clusters_gas”] is less than or equal to the number of neighboring country gas buses.

etrago.cluster.gas.kmean_clustering_gas(etrago, network_ch4, weight, n_clusters)[source]

Performs K-means clustering on the gas network data in the given network_ch4 pypsa.Network object.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • network_ch4 (pypsa.Network) – A Network object containing the gas network data.

  • weight (str or None) – The name of the bus weighting column to use for clustering. If None, unweighted clustering is performed.

  • n_clusters (int) – The number of clusters to create.

Returns

  • busmap (pandas.Series) – A pandas.Series object mapping each bus in the CH4 network to its corresponding cluster ID

  • None – None is returned because k-means clustering makes no use of medoids

etrago.cluster.gas.get_h2_clusters(etrago, busmap_ch4)[source]

Maps H2 buses to CH4 cluster IDds and creates unique H2 cluster IDs.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • busmap_ch4 (pd.Series) – A Pandas Series mapping each bus in the CH4 network to its corresponding cluster ID.

Returns

busmap – A Pandas Series mapping each bus in the combined CH4 and H2 network to its corresponding cluster ID.

Return type

pd.Series

etrago.cluster.gas.gas_postprocessing(etrago, busmap, medoid_idx=None)[source]

Performs the postprocessing for the gas grid clustering based on the provided busmap and returns the clustered network.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

  • medoid_idx (pd.Series) – A pandas.Series object containing the medoid indices for the gas network.

Returns

  • network_gasgrid_c (pypsa.Network) – A pypsa.Network containing the clustered network.

  • busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.

etrago.cluster.gas.highestInteger(potentially_numbers)[source]

Fetch the highest number of a series with mixed types

Parameters

potentially_numbers (pandas.Series) – Series with mixed dtypes, potentially containing numbers.

Returns

highest – Highest integer found in series.

Return type

int

etrago.cluster.gas.simultaneous_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]

Cluster sector coupling technology based on multiple connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters
  • network (pypsa.Network) – PyPSA network instance.

  • busmap (pandas.Series) – Series with lookup table for clustered buses.

  • carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

  • carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns

Busmap for the sector coupling cluster.

Return type

dict

etrago.cluster.gas.consecutive_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]

Cluster sector coupling technology based on single connected carriers.

The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.

Parameters
  • network (pypsa.Network) – PyPSA network instance.

  • busmap (pandas.Series) – Series with lookup table for clustered buses.

  • carrier_based (list) – Carriers on which the clustering of the sector coupling is based.

  • carrier_to_cluster (str) – Name of the carrier which should be clustered

Returns

busmap_sc – Busmap for the sector coupled cluster.

Return type

dict

etrago.cluster.gas.sc_multi_carrier_based(buses_to_cluster, connected_links)[source]

Create busmap for sector coupled carrier based on multiple other carriers.

Parameters
  • buses_to_cluster (pandas.Series) – Series containing the buses of the sector coupled carrier which are to be clustered.

  • connected_links (pandas.DataFrame) – Links that connect from the buses with other carriers to the buses of the sector coupled carrier.

Returns

busmap – Busmap for the sector coupled carrier.

Return type

dict

etrago.cluster.gas.sc_single_carrier_based(connected_links)[source]

Create busmap for sector coupled carrier based on single other carrier.

Parameters

connected_links (pandas.DataFrame) – Links that connect from the buses with other carrier to the buses of the sector coupled carrier.

Returns

busmap – Busmap for the sector coupled carrier.

Return type

dict

etrago.cluster.gas.get_clustering_from_busmap(network, busmap, line_length_factor=1.0, with_time=True, bus_strategies={}, one_port_strategies={})[source]

Aggregates components of the given network based on a bus mapping and returns a clustered gas grid pypsa.Network.

Parameters
  • network (pypsa.Network) – The input pypsa.Network object

  • busmap (pandas.Sereies :) – A mapping of buses to clusters

  • line_length_factor (float) – A factor used to adjust the length of new links created during aggregation. Default is 1.0.

  • with_time (bool) – Determines whether to copy the time-dependent properties of the input network to the output network. Default is True.

  • bus_strategies (dict) – A dictionary of custom strategies to use during the aggregation step. Default is an empty dictionary.

  • one_port_strategies (dict) – A dictionary of custom strategies to use during the one-port component aggregation step. Default is an empty dictionary.

Returns

network_gasgrid_c – A new gas grid pypsa.Network object with aggregated components based on the bus mapping.

Return type

pypsa.Network

etrago.cluster.gas.run_spatial_clustering_gas(self)[source]

Performs spatial clustering on the gas network using either K-means or K-medoids-Dijkstra algorithm. Updates the network topology by aggregating buses and links, and then performs postprocessing to finalize the changes.

Return type

None

Raises

ValueError – If the selected method is not “kmeans” or “kmedoids-dijkstra”.:

etrago.cluster.snapshot module

This module contains functions for reducing the complexity of a PyPSA network in temporal dimension by a) downsampling to every n-th snapshot b) clustering to typical periods (eg days, weeks) c) clustering to segments of variable length Essentially used is the tsam package ( https://github.com/FZJ-IEK3-VSA/tsam ) developed by Leander Kotzur et al.

etrago.cluster.snapshot.snapshot_clustering(self)[source]

Function to call the snapshot clustering function with the respecting method and settings.

Raises

ValueError – When calling a non-available function.

Return type

None.

etrago.cluster.snapshot.tsam_cluster(timeseries_df, typical_periods=10, how='daily', extremePeriodMethod='None', segmentation=False, segment_no=10, segm_hoursperperiod=24)[source]

Conducts the clustering of the snapshots for temporal aggregation with the respecting method.

Parameters
  • timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

  • typical_periods (int, optional) – Number of clusters for typical_periods. The default is 10.

  • how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

  • segmentation (boolean, optional) – Argument to activate segmenation method. The default is False.

  • segment_no (int, optional) – Number of segments for segmentation. The default is 10.

  • segm_hoursperperiod (int, optional) – Only for segmentation, ensures to cluster to segments considering all snapshots. The default is 24.

Returns

  • df_cluster (pd.DataFrame) – Information on cluster after clustering to typical periods.

  • cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

  • dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

  • hours (int) – Hours per typical period.

  • df_i_h (pd.DataFrame) – Information on cluster after clustering to typical periods.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

etrago.cluster.snapshot.segmentation_extreme_periods(timeseries_df, timeseries, extremePeriodMethod)[source]

Function to consider extreme snapshots while using segmentation.

Parameters
  • timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Raises

ValueError – When calling wrong method to consider extreme values.

Returns

timeseries – Information on segments including extreme snapshots after segmentation.

Return type

pd.DataFrame

etrago.cluster.snapshot.run(network, n_clusters=None, how='daily', segmented_to=False, extreme_periods='None')[source]

Function to call the respecting snapshot clustering function and export the result to a csv-file.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • n_clusters (int, optional) – Number of clusters for typical_periods. The default is None.

  • how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.

  • segmented_to (int, optional) – Number of segments for segmentation. The default is False.

  • extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.cluster.snapshot.prepare_pypsa_timeseries(network)[source]

Prepares timeseries and residual load timeseries for clustering.

Parameters

network (pypsa.Network object) – Container for all network components.

Returns

df – Timeseries to be considered when clustering.

Return type

pd.DataFrame

etrago.cluster.snapshot.update_data_frames(network, cluster_weights, dates, hours, timeseries, segmentation)[source]

Updates the snapshots, snapshot weightings and the dataframes based on the original data in the network and the medoids created by clustering these original data.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • cluster_weights (dict) – Weightings per cluster after clustering to typical periods.

  • dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.

  • hours (int) – Hours per typical period.

  • timeseries (pd.DataFrame) – Information on segments after segmentation.

  • segmentation (boolean) – Checks if segmentation of clustering to typical periods has been used.

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.cluster.snapshot.skip_snapshots(self)[source]

Conducts the downsapling to every n-th snapshot.

Return type

None.

etrago.cluster.spatial module

spatial.py defines the methods to run spatial clustering on networks.

etrago.cluster.spatial.ext_storage(x)[source]
etrago.cluster.spatial.sum_with_inf(x)[source]
etrago.cluster.spatial.strategies_one_ports()[source]
etrago.cluster.spatial.strategies_generators()[source]

Aggregate network.links and network.links_t after any kind of clustering

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • with_time (bool) – says if the network object contains timedependent series.

  • carriers (list of strings) – Describe which type of carriers should be aggregated. The default is None.

  • strategies (dictionary) – custom strategies to perform the aggregation

Returns

  • new_df – links aggregated based on bus0, bus1 and carrier

  • new_pnl – links time series aggregated

etrago.cluster.spatial.graph_from_edges(edges)[source]

Constructs an undirected multigraph from a list containing data on weighted edges.

Parameters

edges (list) – List of tuples each containing first node, second node, weight, key.

Returns

M

Return type

networkx.classes.multigraph.MultiGraph

etrago.cluster.spatial.gen(nodes, n, graph)[source]

Generator for applying multiprocessing.

Parameters
  • nodes (list) – List of nodes in the system.

  • n (int) – Number of desired multiprocessing units.

  • graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Return type

None

etrago.cluster.spatial.shortest_path(paths, graph)[source]

Finds the minimum path lengths between node pairs defined in paths.

Parameters
  • paths (list) – List of pairs containing a source and a target node

  • graph (networkx.classes.multigraph.MultiGraph) – Graph representation of an electrical grid.

Returns

df – DataFrame holding source and target node and the minimum path length.

Return type

pd.DataFrame

etrago.cluster.spatial.busmap_by_shortest_path(etrago, scn_name, fromlvl, tolvl, cpu_cores=4)[source]

Creates a busmap for the EHV-Clustering between voltage levels based on dijkstra shortest path. The result is automatically written to the model_draft on the <OpenEnergyPlatform>[www.openenergy-platform.org] database with the name ego_grid_pf_hv_busmap and the attributes scn_name (scenario name), bus0 (node before clustering), bus1 (node after clustering) and path_length (path length). An AssertionError occurs if buses with a voltage level are not covered by the input lists ‘fromlvl’ or ‘tolvl’.

Parameters
  • network (pypsa.Network) – Container for all network components.

  • session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

  • scn_name (str) – Name of the scenario.

  • fromlvl (list) – List of voltage-levels to cluster.

  • tolvl (list) – List of voltage-levels to remain.

  • cpu_cores (int) – Number of CPU-cores.

Return type

None

etrago.cluster.spatial.busmap_from_psql(etrago)[source]

Retrieves busmap from model_draft.ego_grid_pf_hv_busmap on the <OpenEnergyPlatform>[www.openenergy-platform.org] by a given scenario name. If this busmap does not exist, it is created with default values.

Parameters
  • network (pypsa.Network) – Container for all network components.

  • session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.

  • scn_name (str) – Name of the scenario.

Returns

busmap – Maps old bus_ids to new bus_ids.

Return type

dict

etrago.cluster.spatial.kmean_clustering(etrago, selected_network, weight, n_clusters)[source]

Main function of the k-mean clustering approach. Maps an original network to a new one with adjustable number of nodes and new coordinates.

Parameters
  • network (pypsa.Network) – Container for all network components.

  • n_clusters (int) – Desired number of clusters.

  • load_cluster (boolean) – Loads cluster coordinates from a former calculation.

  • line_length_factor (float) – Factor to multiply the crow-flies distance between new buses in order to get new line lengths.

  • remove_stubs (boolean) – Removes stubs and stubby trees (i.e. sequentially reducing dead-ends).

  • use_reduced_coordinates (boolean) – If True, do not average cluster coordinates, but take from busmap.

  • bus_weight_tocsv (str) – Creates a bus weighting based on conventional generation and load and save it to a csv file.

  • bus_weight_fromcsv (str) – Loads a bus weighting from a csv file to apply it to the clustering algorithm.

Returns

network – Container for all network components.

Return type

pypsa.Network

etrago.cluster.spatial.dijkstras_algorithm(buses, connections, medoid_idx, cpu_cores)[source]

Function for combination of k-medoids Clustering and Dijkstra’s algorithm. Creates a busmap assigning the nodes of a original network to the nodes of a clustered network considering the electrical distances based on Dijkstra’s shortest path.

Parameters
  • network (pypsa.Network) – Container for all network components.

  • medoid_idx (pandas.Series) – Indices of k-medoids

  • busmap_kmedoid (pandas.Series) – Busmap based on k-medoids clustering

  • cpu_cores (string) – numbers of cores used during multiprocessing

Returns

busmap – Mapping from bus ids to medoids ids

Return type

pandas.Series

etrago.cluster.spatial.kmedoids_dijkstra_clustering(etrago, buses, connections, weight, n_clusters)[source]

Applies a k-medoids clustering on the given network and calls the function to conduct a Dijkstra’s algorithm afterwards for the consideration of the network’s topology in the spatial clustering.

Parameters
  • etrago (Etrago) – An instance of the Etrago class

  • buses (pandas.DataFrame) – DataFrame with information about the buses of the network.

  • connections (pandas.DataFrame) – DataFrame with information about the connections of the network (links or lines).

  • weight (pandas.Series) – Series with the weight for each bus.

  • n_clusters (int) – The number of clusters to create.

Returns

  • Tuple containing

  • busmap (pandas.Series) – Series containing the mapping of buses to their resp. medoids

  • medoid_idx (pandas.Series) – Series containing the medoid indeces

etrago.tools package

etrago.tools.calc_results module

calc_results.py defines methods to calculate results of eTraGo

etrago.tools.calc_results.calc_investment_cost(self)[source]

Function that calulates overall annualized investment costs.

Returns

  • network_costs (float) – Investments in line expansion (AC+DC)

  • link_costs (float) – Investments in sectorcoupling link expansion

  • stor_costs (float) – Investments in storage and store expansion

etrago.tools.calc_results.calc_marginal_cost(self)[source]

Function that caluclates and returns marginal costs, considering generation and link and storage dispatch costs

Returns

marginal_cost – Annual marginal cost in EUR

Return type

float

etrago.tools.calc_results.calc_etrago_results(self)[source]

Function that calculates main results of grid optimization and adds them to Etrago object.

Return type

None.

etrago.tools.constraints module

Constraints.py includes additional constraints for eTraGo-optimizations

etrago.tools.constraints.read_max_gas_generation(self)[source]

Return the values limiting the gas production in Germany

Read max_gas_generation_overtheyear from scenario.egon_scenario_parameters if the table is available in the database and return the dictionnary containing the values needed for the constraints to limit the gas production in Germany, depending of the scenario.

Returns

arg

Return type

dict

etrago.tools.constraints.add_ch4_constraints(self, network, snapshots)[source]

Add CH4 constraints for optimization with pyomo

Functionality that limits the dispatch of CH4 generators. In Germany, there is one limitation specific for biogas and one limitation specific for natural gas (natural gas only in eGon2035). Abroad, each generator has its own limitation contains in the column e_nom_max.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization

Return type

None.

etrago.tools.constraints.add_ch4_constraints_nmp(self, network, snapshots)[source]

Add CH4 constraints for optimization without pyomo

Functionality that limits the dispatch of CH4 generators. In Germany, there is one limitation specific for biogas and one limitation specific for natural gas (natural gas only in eGon2035). Abroad, each generator has its own limitation contains in the column e_nom_max.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization

Return type

None.

etrago.tools.constraints.snapshot_clustering_daily_bounds(self, network, snapshots)[source]

Bound the storage level to 0.5 max_level every 24th hour.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps that will be constrained

Return type

None

etrago.tools.constraints.snapshot_clustering_daily_bounds_nmp(self, network, snapshots)[source]

Bound the storage level to 0.5 max_level every 24th hour.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps that will be constrained

Return type

None

etrago.tools.constraints.snapshot_clustering_seasonal_storage(self, network, snapshots, simplified=False)[source]

Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.

According to:

L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.

  • simplified (bool, optional) – A flag indicating whether to use a simplified version of the model that does not include intra-temporal constraints and variables.

Return type

None

etrago.tools.constraints.snapshot_clustering_seasonal_storage_hourly(self, network, snapshots)[source]

Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.

According to:

L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.

Return type

None

etrago.tools.constraints.snapshot_clustering_seasonal_storage_nmp(self, n, sns, simplified=False)[source]

Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.

According to:

L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018

Parameters
  • n (pypsa.Network) – Overall container of PyPSA

  • sns (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.

  • simplified (bool, optional) – A flag indicating whether to use a simplified version of the model that does not include intra-temporal constraints and variables.

Return type

None

etrago.tools.constraints.snapshot_clustering_seasonal_storage_hourly_nmp(self, n, sns)[source]

Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.

According to:

L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018

Parameters
  • n (pypsa.Network) – Overall container of PyPSA

  • sns (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.

Return type

None

etrago.tools.constraints.split_dispatch_disaggregation_constraints(self, n, sns)[source]

Add constraints for state of charge of storage units and stores when separating the optimization into smaller subproblems while conducting thedispatch_disaggregation in temporally fully resolved network

The state of charge at the end of each slice is set to the value calculated in the optimization with the temporally reduced network to account to ensure compatibility and to reproduce saisonality

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization

Return type

None.

etrago.tools.constraints.split_dispatch_disaggregation_constraints_nmp(self, n, sns)[source]
class etrago.tools.constraints.Constraints(args, conduct_dispatch_disaggregation)[source]

Bases: object

Methods

functionality(network, snapshots)

Add constraints to pypsa-model using extra-functionality.

functionality(network, snapshots)[source]

Add constraints to pypsa-model using extra-functionality. Serveral constraints can be choosen at once. Possible constraints are set and described in the above functions.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • snapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization

etrago.tools.constraints.add_chp_constraints_nmp(n)[source]

Limits the dispatch of combined heat and power links based on T.Brown et. al : Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable European energy system, 2018

Parameters

n (pypsa.Network) – Network container

Return type

None.

etrago.tools.constraints.add_chp_constraints(network, snapshots)[source]

Limits the dispatch of combined heat and power links based on T.Brown et. al : Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable European energy system, 2018

Parameters
  • network (pypsa.Network) – Network container

  • snapshots (pandas.DataFrame) – Timesteps to optimize

Return type

None.

etrago.tools.execute module

execute.py defines optimization and simulation methods for the etrago object.

etrago.tools.execute.update_electrical_parameters(network, l_snom_pre, t_snom_pre)[source]

Update electrical parameters of active branch components considering s_nom of previous iteration.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • l_snom_pre (pandas.Series) – s_nom of ac-lines in previous iteration.

  • t_snom_pre (pandas.Series) – s_nom of transformers in previous iteration.

Return type

None.

etrago.tools.execute.run_lopf(etrago, extra_functionality, method)[source]

Function that performs lopf with or without pyomo

Parameters
  • etrago (etrago object) – eTraGo containing all network information and a PyPSA network.

  • extra_functionality (dict) – Define extra constranits.

  • method (dict) – Choose ‘n_iter’ and integer for fixed number of iterations or ‘threshold’ and derivation of objective in percent for variable number of iteration until the threshold of the objective function is reached.

Return type

None.

etrago.tools.execute.iterate_lopf(etrago, extra_functionality, method={'n_iter': 4, 'pyomo': True})[source]

Run optimization of lopf. If network extension is included, the specified number of iterations is calculated to consider reactance changes.

Parameters
  • etrago (etrago object) – eTraGo containing all network information and a PyPSA network.

  • extra_functionality (dict) – Define extra constranits.

  • method (dict) – Choose ‘n_iter’ and integer for fixed number of iterations or ‘threshold’ and derivation of objective in percent for variable number of iteration until the threshold of the objective function is reached.

etrago.tools.execute.lopf(self)[source]

Functions that runs lopf according to arguments.

Return type

None.

etrago.tools.execute.dispatch_disaggregation(self)[source]

Function running the tempral disaggregation meaning the optimization of dispatch in the temporally fully resolved network; therfore, the problem is reduced to smaller subproblems by slicing the whole considered time span while keeping inforation on the state of charge of storage units and stores to ensure compatibility and to reproduce saisonality.

Return type

None.

etrago.tools.execute.run_pf_post_lopf(self)[source]

Function that runs pf_post_lopf according to arguments.

Return type

None.

etrago.tools.execute.pf_post_lopf(etrago, calc_losses=False)[source]

Function that prepares and runs non-linar load flow using PyPSA pf. If crossborder lines are DC-links, pf is only applied on german network. Crossborder flows are still considerd due to the active behavior of links. To return a network containing the whole grid, the optimised solution of the foreign components can be added afterwards.

Parameters
  • etrago (etrago object) – eTraGo containing all network information and a PyPSA network.

  • add_foreign_lopf (boolean) – Choose if foreign results of lopf should be added to the network when foreign lines are DC.

  • q_allocation (str) – Choose allocation of reactive power. Possible settings are listed in distribute_q function.

  • calc_losses (bolean) – Choose if line losses will be calculated.

etrago.tools.execute.distribute_q(network, allocation='p_nom')[source]

Function that distributes reactive power at bus to all installed generators and storages.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • allocation (str) – Choose key to distribute reactive power: ‘p_nom’ to dirstribute via p_nom ‘p’ to distribute via p_set.

Return type

None.

etrago.tools.execute.calc_line_losses(network, converged)[source]

Calculate losses per line with PF result data.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • converged (pd.Series) – List of snapshots with their status (converged or not).

Return type

None.

etrago.tools.execute.set_slack(network)[source]

Function that chosses the bus with the maximum installed power as slack.

Parameters

network (pypsa.Network object) – Container for all network components.

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.tools.extendable module

Extendable.py defines function to set PyPSA components extendable.

etrago.tools.extendable.extendable(self, grid_max_D=None, grid_max_abs_D={'110': {'circuits': 2, 'i': 1020, 'wires': 4}, '220': {'circuits': 4, 'i': 1020, 'wires': 4}, '380': {'circuits': 4, 'i': 1020, 'wires': 4}, 'dc': 0}, grid_max_foreign=4, grid_max_abs_foreign=None)[source]

Function that sets selected components extendable.

Parameters
  • grid_max_D (int, optional) – Upper bounds for electrical grid expansion relative to existing capacity. The default is None.

  • grid_max_abs_D (dict, optional) – Absolute upper bounds for electrical grid expansion in Germany.

  • grid_max_foreign (int, optional) – Upper bounds for expansion of electrical foreign lines relative to the existing capacity. The default is 4.

  • grid_max_abs_foreign (dict, optional) – Absolute upper bounds for expansion of foreign electrical grid. The default is None.

Return type

None.

etrago.tools.extendable.snommax(i=1020, u=380, wires=4, circuits=4)[source]

Function to calculate limitation for capacity expansion.

Parameters
  • i (int, optional) – Current. The default is 1020.

  • u (int, optional) – Voltage level. The default is 380.

  • wires (int, optional) – Number of wires per line. The default is 4.

  • circuits (int, optional) – Number of circuits. The default is 4.

Returns

s_nom_max – Limitation for capacity expansion.

Return type

float

etrago.tools.extendable.line_max_abs(network, buses, line_max_abs={'110': {'circuits': 2, 'i': 1020, 'wires': 4}, '220': {'circuits': 4, 'i': 1020, 'wires': 4}, '380': {'circuits': 4, 'i': 1020, 'wires': 4}, 'dc': 0})[source]

Function to calculate limitation for capacity expansion of lines in network.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • buses (pypsa.Network buses) – Considered buses in network.

  • line_max_abs (dict, optional) – Line parameters considered to calculate maximum capacity.

Return type

None.

etrago.tools.extendable.transformer_max_abs(network, buses)[source]

Function to calculate limitation for capacity expansion of transformers in network.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • buses (pypsa.Network buses) – Considered buses in network.

Return type

None.

etrago.tools.extendable.extension_preselection(etrago, method, days=3)[source]

Function that preselects lines which are extendend in snapshots leading to overloading to reduce nubmer of extension variables.

Parameters
  • network (pypsa.Network object) – Container for all network components.

  • args (dict) – Arguments set in appl.py.

  • method (str) – Choose method of selection: ‘extreme_situations’ for remarkable timsteps (e.g. minimal resiudual load) ‘snapshot_clustering’ for snapshot clustering with number of days

  • days (int) – Number of clustered days, only used when method = ‘snapshot_clustering’

Returns

network – Container for all network components.

Return type

pypsa.Network object

etrago.tools.extendable.print_expansion_costs(network)[source]

Function that prints network and storage investment costs.

Parameters

network (pypsa.Network object) – Container for all network components.

Return type

None.

etrago.tools.io module

io.py

Input/output operations between powerflow schema in the oedb and PyPSA. Additionally oedb wrapper classes to instantiate PyPSA network objects.

etrago.tools.io.packagename

Package containing orm class definitions

Type

str

etrago.tools.io.temp_ormclass

Orm class name of table with temporal resolution

Type

str

etrago.tools.io.carr_ormclass

Orm class name of table with carrier id to carrier name datasets

Type

str

Notes

A configuration file connecting the chosen optimization method with components to be queried is needed for NetworkScenario class.

class etrago.tools.io.ScenarioBase(engine, session, version=None)[source]

Bases: object

Base class to address the dynamic provision of orm classes representing powerflow components from egoio

Parameters
  • session (sqla.orm.session.Session) – Handles conversations with the database.

  • version (str) – Version number of data version control in grid schema of the oedb.

class etrago.tools.io.NetworkScenario(engine, session, scn_name='Status Quo', start_snapshot=1, end_snapshot=20, temp_id=1, **kwargs)[source]

Bases: ScenarioBase

Adapter class between oedb powerflow data and PyPSA. Provides the method build_network to generate a pypsa.Network.

Parameters
  • scn_name (str) – Scenario name.

  • method (str) – Objective function.

  • start_snapshot (int) – First snapshot or timestep.

  • end_snapshot (int) – Last timestep.

  • temp_id (int) – Nummer of temporal resolution.

Methods

build_network([network])

Core method to construct PyPSA Network object.

configure_timeindex()

Construct a DateTimeIndex with the queried temporal resolution, start- and end_snapshot.

fetch_by_relname(name)

Construct DataFrame with component data from filtered table data.

series_fetch_by_relname(network, name, ...)

Construct DataFrame with component timeseries data from filtered table data.

id_to_source

configure_timeindex()[source]

Construct a DateTimeIndex with the queried temporal resolution, start- and end_snapshot.

id_to_source()[source]
fetch_by_relname(name)[source]

Construct DataFrame with component data from filtered table data.

Parameters

name (str) – Component name.

Returns

Component data.

Return type

pd.DataFrame

series_fetch_by_relname(network, name, pypsa_name)[source]

Construct DataFrame with component timeseries data from filtered table data.

Parameters
  • name (str) – Component name.

  • column (str) – Component field with timevarying data.

Returns

Component data.

Return type

pd.DataFrame

build_network(network=None, *args, **kwargs)[source]

Core method to construct PyPSA Network object.

etrago.tools.io.clear_results_db(session)[source]

Used to clear the result tables in the OEDB. Caution! This deletes EVERY RESULT SET!

etrago.tools.io.results_to_oedb(session, network, args, grid='hv', safe_results=False)[source]

Return results obtained from PyPSA to oedb

Parameters
  • session

  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • args (dict) – Settings from appl.py

  • grid (str) – Choose voltage-level, currently only ‘hv’ implemented

  • safe_results (boolean) – If it is set to ‘True’ the result set will be saved to the versioned grid schema eventually apart from being saved to the model_draft by a SQL-script. ONLY set to True if you know what you are doing.

etrago.tools.io.run_sql_script(conn, scriptname='results_md2grid.sql')[source]

This function runs .sql scripts in the folder ‘sql_scripts’

etrago.tools.io.extension(self, **kwargs)[source]

Function that adds an additional network to the existing network container. The new network can include every PyPSA-component (e.g. buses, lines, links). To connect it to the existing network, transformers are needed.

All components and its timeseries of the additional scenario need to be inserted in the fitting ‘model_draft.ego_grid_pf_hv_extension_’ table. The scn_name in the tables have to be labled with ‘extension_’ + scn_name (e.g. ‘extension_nep2035’).

Until now, the tables include three additional scenarios: ‘nep2035_confirmed’: all new lines and needed transformers planed in the ‘Netzentwicklungsplan 2035’ (NEP2035) that have been confirmed by the Bundesnetzagentur (BNetzA)

‘nep2035_b2’: all new lines and needed transformers planned in the NEP 2035 in the scenario 2035 B2

‘BE_NO_NEP 2035’: DC-lines and transformers to connect the upcomming electrical-neighbours Belgium and Norway Generation, loads and its timeseries in Belgium and Norway for scenario ‘NEP 2035’

Parameters
  • network – The existing network container (e.g. scenario ‘NEP 2035’)

  • session – session-data

  • overlay_scn_name – Name of the additional scenario (WITHOUT ‘extension_’)

  • start_snapshot

  • end_snapshot – Simulation time

Return type

Network container including existing and additional network

etrago.tools.io.decommissioning(self, **kwargs)[source]

Function that removes components in a decommissioning-scenario from the existing network container. Currently, only lines can be decommissioned.

All components of the decommissioning scenario need to be inserted in the fitting ‘model_draft.ego_grid_pf_hv_extension_’ table. The scn_name in the tables have to be labled with ‘decommissioning_’ + scn_name (e.g. ‘decommissioning_nep2035’).

Parameters
  • network – The existing network container (e.g. scenario ‘NEP 2035’)

  • session – session-data

  • overlay_scn_name – Name of the decommissioning scenario

Return type

Network container including decommissioning

etrago.tools.io.distance(x0, x1, y0, y1)[source]

Function that calculates the square of the distance between two points.

Parameters
  • x0 – x - coordinate of point 0

  • x1 – x - coordinate of point 1

  • y0 – y - coordinate of point 0

  • y1 – y - coordinate of point 1

Returns

distance – square of distance

Return type

float

etrago.tools.io.calc_nearest_point(bus1, network)[source]

Function that finds the geographical nearest point in a network from a given bus.

Parameters
  • bus1 (float) – id of bus

  • network (Pypsa network container) – network including the comparable buses

Returns

bus0 – bus_id of nearest point

Return type

float

etrago.tools.io.add_ch4_h2_correspondence(self)[source]

Method adding the database table grid.egon_etrago_ch4_h2 to self. It contains the mapping from H2 buses to their corresponding CH4 buses.

etrago.tools.network module

etrago.tools.plot module

Plot.py defines functions necessary to plot results of eTraGo.

etrago.tools.plot.set_epsg_network(network)[source]

Change EPSG from 4326 to 3857. Needed when using osm-background.

Parameters

network (PyPSA network container) –

etrago.tools.plot.plot_osm(x, y, zoom, alpha=0.4)[source]

Plots openstreetmap as background of network-plots

Parameters
  • x (array of two floats) – Define x-axis boundaries (lat) of osm plot

  • y (array of two floats) – Define y-axis boundaries (long) of osm plot

  • zoom (int) – Define zoom of osm, higher values for higher resolution

  • alpha (float) – Sets osm-visibility, increase value if osm covers network-plot

etrago.tools.plot.coloring()[source]

Return a dictionary with a color assign to each kind of carrier used in etrago.network. This is used for plotting porpuses.

Returns

colors – Color for each kind of carrier.

Return type

dict

etrago.tools.plot.plot_line_loading_diff(networkA, networkB, timestep=0, osm=False)[source]

Plot difference in line loading between two networks (with and without switches) as color on lines

Positive values mean that line loading with switches is bigger than without Plot switches as small dots

Parameters
  • networkA (PyPSA network container) – Holds topology of grid with switches including results from powerflow analysis

  • networkB (PyPSA network container) – Holds topology of grid without switches including results from powerflow analysis

  • filename (str) – Specify filename If not given, figure will be show directly

  • timestep (int) – timestep to show, default is 0

  • osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –

    If not False, osm is set as background with the following settings as dict:

    • ’x’: array of two floats, x axis boundaries (lat)

    • ’y’: array of two floats, y axis boundaries (long)

    • ’zoom’ : resolution of osm

etrago.tools.plot.network_expansion_diff(networkA, networkB, filename=None, boundaries=[], osm=False)[source]

Plot relative network expansion derivation of AC- and DC-lines.

Parameters
  • networkA (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • networkB (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • filename (str or None) – Save figure in this direction

  • boundaries (array) – Set boundaries of heatmap axis

  • osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –

    If not False, osm is set as background with the following settings as dict:

    • ’x’: array of two floats, x axis boundaries (lat)

    • ’y’: array of two floats, y axis boundaries (long)

    • ’zoom’ : resolution of osm

etrago.tools.plot.plot_residual_load(network)[source]

Plots residual load summed of all exisiting buses.

Parameters

network (PyPSA network containter) –

Return type

Plot

etrago.tools.plot.plot_stacked_gen(network, bus=None, resolution='GW', filename=None)[source]

Plot stacked sum of generation grouped by carrier type

Parameters
  • network (PyPSA network container) –

  • bus (string) – Plot all generators at one specific bus. If none, sum is calulated for all buses

  • resolution (string) – Unit for y-axis. Can be either GW/MW/KW

Return type

Plot

etrago.tools.plot.plot_gen_diff(networkA, networkB, leave_out_carriers=['geothermal', 'oil', 'other_non_renewable', 'reservoir', 'waste'])[source]

Plot difference in generation between two networks grouped by carrier type

Parameters
  • networkA (PyPSA network container with switches) –

  • networkB (PyPSA network container without switches) –

  • leave_out_carriers – list of carriers to leave out (default to all small carriers)

Return type

Plot

etrago.tools.plot.plot_voltage(network, boundaries=[], osm=False)[source]

Plot voltage at buses as hexbin

Parameters
  • network (PyPSA network container) –

  • boundaries (list of 2 values, setting the lower and upper bound of colorbar) –

  • osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –

    If not False, osm is set as background with the following settings as dict:

    • ’x’: array of two floats, x axis boundaries (lat)

    • ’y’: array of two floats, y axis boundaries (long)

    • ’zoom’ : resolution of osm

Return type

Plot

etrago.tools.plot.curtailment(network, carrier='solar', filename=None)[source]

Plot curtailment of selected carrier

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • carrier (str) – Plot curtailemt of this carrier

  • filename (str or None) – Save figure in this direction

Return type

Plot

etrago.tools.plot.calc_dispatch_per_carrier(network, timesteps)[source]

Function that calculates dispatch per carrier in given timesteps

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • timesteps (array) – Timesteps considered in dispatch calculation

Returns

dist – dispatch per carrier

Return type

pandas.Series

etrago.tools.plot.calc_storage_expansion_per_bus(network)[source]

Function that calculates storage expansion per bus and technology

Parameters

network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

Returns

dist – storage expansion per bus and technology

Return type

pandas.Series

etrago.tools.plot.gen_dist_diff(networkA, networkB, techs=None, snapshot=0, n_cols=3, gen_size=0.2, filename=None, buscmap=<matplotlib.colors.LinearSegmentedColormap object>)[source]

Difference in generation distribution Green/Yellow/Red colors mean that the generation at a location is bigger with switches than without Blue colors mean that the generation at a location is smaller with switches than without

Parameters
  • networkA (PyPSA network container) – Holds topology of grid with switches including results from powerflow analysis

  • networkB (PyPSA network container) – Holds topology of grid without switches including results from powerflow analysis

  • techs (dict) – type of technologies which shall be plotted

  • snapshot (int) – snapshot

  • n_cols (int) – number of columns of the plot

  • gen_size (num) – size of generation bubbles at the buses

  • filename (str) – Specify filename If not given, figure will be show directly

Return type

None.

etrago.tools.plot.nodal_gen_dispatch(network, networkB=None, techs=['wind_onshore', 'solar'], item='energy', direction=None, scaling=1, filename=None, osm=False)[source]

Plot nodal dispatch or capacity. If networkB is given, difference in dispatch is plotted.

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • networkB (PyPSA network container) – If given and item is ‘energy’, difference in dispatch between network and networkB is plotted. If item is ‘capacity’, networkB is ignored. default None

  • techs (None or list,) – Techs to plot. If None, all techs are plotted. default [‘wind_onshore’, ‘solar’]

  • item (str) – Specifies the plotted item. Options are ‘energy’ and ‘capacity’. default ‘energy’

  • direction (str) – Only considered if networkB is given and item is ‘energy’. Specifies the direction of change in dispatch between network and networkB. If ‘positive’, generation per tech which is higher in network than in networkB is plotted. If ‘negative’, generation per tech whcih is lower in network than in networkB is plotted. If ‘absolute’, total change per node is plotted. Green nodes have higher dispatch in network than in networkB. Red nodes have lower dispatch in network than in networkB. default None

  • scaling (int) – Scaling to change plot sizes. default 1

  • filename (path to folder) –

  • osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –

    If not False, osm is set as background with the following settings as dict:

    • ’x’: array of two floats, x axis boundaries (lat)

    • ’y’: array of two floats, y axis boundaries (long)

    • ’zoom’ : resolution of osm

Return type

None.

etrago.tools.plot.nodal_production_balance(network, timesteps, scaling=1e-05)[source]

Function that calculates residual load per node in given timesteps

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • timesteps (array) – timesteps considered in calculation

  • scaling (float, optional) – Scaling factor for bus size. The default is 0.00001.

Returns

  • bus_sizes (pandas.Series) – scaled residual load per node

  • bus_colors (pandas.Series) – ‘green’ for producer and ‘red’ for consumer

etrago.tools.plot.storage_p_soc(network, mean='1H', filename=None)[source]

Plots the dispatch and state of charge (SOC) of extendable storages.

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • mean (str) – Defines over how many snapshots the p and soc values will averaged.

  • filename (path to folder) –

Return type

None.

etrago.tools.plot.storage_soc_sorted(network, filename=None)[source]

Plots the soc (state-pf-charge) of extendable storages

Parameters
  • network (PyPSA network container) – Holds topology of grid including results from powerflow analysis

  • filename (path to folder) –

Return type

None.

etrago.tools.plot.mul_weighting(network, timeseries)[source]

Returns timeseries considering snapshot_weighting

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • timeseries (pd.Series) – timeseries not considering snapshot_weighting

Returns

timeseries considering snapshot_weightings

Return type

pd.Series

etrago.tools.plot.calc_ac_loading(network, timesteps)[source]

Calculates loading of AC-lines

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • timesteps (range) – Defines which timesteps are considered. If more than one, an average line loading is calculated.

Returns

ACC line loading in MVA

Return type

pandas.Series

etrago.tools.plot.calc_dc_loading(network, timesteps)[source]

Calculates loading of DC-lines

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • timesteps (range) – Defines which timesteps are considered. If more than one, an average line loading is calculated.

Returns

DC line loading in MW

Return type

pandas.Series

etrago.tools.plot.plotting_colors(network)[source]

Add color values to network.carriers

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Return type

None.

etrago.tools.plot.calc_network_expansion(network, method='abs', ext_min=0.1)[source]

Calculates absolute or relative expansion per AC- and DC-line

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • method (str, optional) – Choose ‘rel’ or ‘abs’. The default is ‘abs’.

  • ext_min (float, optional) – Remove lines extended less than this value. The default is 0.1.

Returns

  • all_network (:class:`pypsa.Network) – Whole network including not extended lines

  • extension_lines (pandas.Series) – AC-line expansion

  • extension_links (pandas.Series) – DC-line expansion

etrago.tools.plot.plot_background_grid(network, ax)[source]

Plots grid topology in background of other network.plot

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • ax (matplotlib.axes._subplots.AxesSubplot) – axes of plot

Return type

None.

etrago.tools.plot.demand_side_management(self, buses, snapshots, agg='5h', used=False)[source]

Calculate shifting potential of demand side management

Parameters
  • buses (array) – List of electricity buses.

  • snapshots (array) – List of snapshots.

  • agg (str, optional) – Temporal resolution. The default is ‘5h’.

  • used (boolean, optional) – State if usage should be included in the results. The default is False.

Returns

df – Shifting potential (and usage) of power (MW) and energy (MWh)

Return type

pandas.DataFrame

etrago.tools.plot.bev_flexibility_potential(self, buses, snapshots, agg='5h', used=False)[source]

Calculate shifting potential of electric vehicles

Parameters
  • buses (array) – List of electricity buses.

  • snapshots (array) – List of snapshots.

  • agg (str, optional) – Temporal resolution. The default is ‘5h’.

  • used (boolean, optional) – State if usage should be included in the results. The default is False.

Returns

df – Shifting potential (and usage) of power (MW) and energy (MWh)

Return type

pandas.DataFrame

etrago.tools.plot.heat_stores(self, buses, snapshots, agg='5h', used=False)[source]

Calculate shifting potential (and usage) of heat stores

Parameters
  • buses (array) – List of electricity buses.

  • snapshots (array) – List of snapshots.

  • agg (str, optional) – Temporal resolution. The default is ‘5h’.

  • used (boolean, optional) – State if usage should be included in the results. The default is False.

Returns

df – Shifting potential (and usage) of power (MW) and energy (MWh)

Return type

pandas.DataFrame

etrago.tools.plot.hydrogen_stores(self, buses, snapshots, agg='5h', used=False)[source]

Calculate shifting potential (and usage) of heat stores

Parameters
  • buses (array) – List of electricity buses.

  • snapshots (array) – List of snapshots.

  • agg (str, optional) – Temporal resolution. The default is ‘5h’.

  • used (boolean, optional) – State if usage should be included in the results. The default is False.

Returns

df – Shifting potential (and usage) of power (MW) and energy (MWh)

Return type

pandas.DataFrame

etrago.tools.plot.flexibility_usage(self, flexibility, agg='5h', snapshots=[], buses=[], pre_path=None)[source]

Plots temporal distribution of potential and usage for flexibilities

Parameters
  • flexibility (str) – Name of flexibility option.

  • agg (str, optional) – Temporal resolution. The default is “5h”.

  • snapshots (list, optional) – Considered snapshots, if empty all are considered. The default is [].

  • buses (list, optional) – Considered components at AC buses, if empty all are considered. The default is [].

  • pre_path (str, optional) – State of and where you want to store the figure. The default is None.

Return type

None.

etrago.tools.plot.plot_carrier(network, carrier_links=['AC'], carrier_buses=['AC'])[source]
Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • carrier_links (list) – List of links to be plotted. The default is [“AC”].

  • carrier_buses (list) – List of buses to be plotted. The default is [“AC”].

  • cartopy (bool, optional) – Provide data about the availability of Cartopy. The default is True.

Return type

None.

etrago.tools.plot.plot_grid(self, line_colors, bus_sizes=0.001, bus_colors='grey', timesteps=range(0, 2), osm=False, boundaries=None, filename=None, disaggregated=False, ext_min=0.1, ext_width=False)[source]

Function that plots etrago.network and results for lines and buses

Parameters
  • line_colors (str) –

    Set static line color or attribute to plot e.g. ‘expansion_abs’ Current options:

    • ’line_loading’: mean line loading in p.u. in selected timesteps

    • ’v_nom’: nominal voltage of lines

    • ’expansion_abs’: absolute network expansion in MVA

    • ’expansion_rel’: network expansion in p.u. of existing capacity

    • ’q_flow_max’: maximal reactive flows

  • bus_sizes (float, optional) – Size of buses. The default is 0.001.

  • bus_colors (str, optional) –

    Set static bus color or attribute to plot. The default is ‘grey’. Current options:

    • ’nodal_production_balance’: net producer/consumer in selected timeteps

    • ’storage_expansion’: storage expansion per bus and technology

    • ’storage_distribution’: installed storage units per bus

    • ’gen_dist’: dispatch per carrier in selected timesteps

  • timesteps (array, optional) – Timesteps consideredd in time depended plots. The default is range(2).

  • osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –

    If not False, osm is set as background with the following settings as dict:

    • ’x’: array of two floats, x axis boundaries (lat)

    • ’y’: array of two floats, y axis boundaries (long)

    • ’zoom’ : resolution of osm. The default is False.

  • boundaries (array) – Set fixed boundaries of heatmap axis. The default is None.

  • filename (str or None) – Save figure in this direction. The default is None.

  • disaggregated (bool, optional) – Choose if disaggregated network is shown. The default is False.

  • ext_min (float) – Choose minimum relative line extension shown in plot in p.u..

  • ext_width (float or bool) – Choose if line_width respects line extension. Turn off with ‘False’ or set linear factor to decremise extension line_width. The default is False.

Return type

None.

etrago.tools.plot.make_handler_map_to_scale_circles_as_in(ax, dont_resize_actively=False)[source]
etrago.tools.plot.make_legend_circles_for(sizes, scale=1.0, **kw)[source]
etrago.tools.plot.plot_clusters(self, carrier='AC', save_path=False, transmission_lines=False, gas_pipelines=False)[source]
Parameters
  • carrier (str, optional) – This variable set the carrier of the buses that will be plotted. The default is “AC”.

  • cartopy (bool, optional) – Set it to True when cartopy is installed and the map is supposed to include country’s boundaries and bodies of water

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

  • transmission_lines (bool, optional) – The default is False. Define if the original transmission lines are plotted or not.

  • gas_pipelines (bool, optional) – The default is False. Define if the original gas pipelines are plotted or not.

Return type

None.

etrago.tools.plot.plot_gas_generation(self, t_resolution='20H', save_path=False)[source]

Plots timeseries data for gas generation

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.plot.plot_gas_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]

Plots timeseries data for gas loads (and generation)

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total gas generation will then also be plotted to check for matching demand and generation.

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.plot.plot_h2_generation(self, t_resolution='20H', save_path=False)[source]

Plots timeseries data for H2 generation

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.plot.plot_h2_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]

Plots timeseries data for H2 loads (and generation)

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total H2 generation will then also be plotted to check for matching demand and generation.

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.plot.plot_heat_loads(self, t_resolution='20H', save_path=False)[source]

Plots timeseries data for heat loads

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.plot.plot_heat_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]

Plots timeseries data for heat generation (and demand)

Parameters
  • self (:class:`Etrago) – Overall container of Etrago

  • t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots

  • stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total heat demand will then also be plotted to check for matching generation and demand.

  • save_path (bool, optional) – Path to save the generated plot. The default is False.

Return type

None.

etrago.tools.utilities module

Utilities.py includes a wide range of useful functions.

Parameters
  • carrier (list or str) – name of the carriers of interest. Can be a list of carriers or single sting.

  • like (bool, optional) – When like set to True, the links with carrier names that includes the carrier(s) supplied are returned, Not just exact matches. The default is True.

Returns

df – Dataframe that contains just links with carriers of the types given in the argument carrier.

Return type

pandas.DataFrame object

etrago.tools.utilities.buses_of_vlvl(network, voltage_level)[source]

Get bus-ids of given voltage level(s).

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • voltage_level (list) –

Returns

List containing bus-ids.

Return type

list

etrago.tools.utilities.buses_grid_linked(network, voltage_level)[source]

Get bus-ids of a given voltage level connected to the grid.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • voltage_level (list) –

Returns

List containing bus-ids.

Return type

list

etrago.tools.utilities.geolocation_buses(self)[source]

If geopandas is installed: Use geometries of buses x/y(lon/lat) and polygons of countries from RenpassGisParameterRegion in order to locate the buses

Else: Use coordinats of buses to locate foreign buses, which is less accurate.

TODO: Why not alway use geopandas??

Parameters

etrago (etrago.Etrago) – Transmission grid object

etrago.tools.utilities.buses_by_country(self)[source]

Find buses of foreign countries using coordinates and return them as Pandas Series

Parameters

self (Etrago object) – Overall container of PyPSA

Return type

None

etrago.tools.utilities.clip_foreign(network)[source]

Delete all components and timelines located outside of Germany. If applied after optimization, transborder flows divided by country of origin are added as network.foreign_trade.

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Returns

network – Overall container of PyPSA

Return type

:class:`pypsa.Network

Change transmission technology of foreign lines from AC to DC (links).

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Returns

network – Overall container of PyPSA

Return type

:class:`pypsa.Network

etrago.tools.utilities.set_q_national_loads(self, cos_phi)[source]

Set q component of national loads based on the p component and cos_phi

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • cos_phi (float) – Choose ration of active and reactive power of foreign loads

Returns

network – Overall container of PyPSA

Return type

pypsa.Network

etrago.tools.utilities.set_q_foreign_loads(self, cos_phi)[source]

Set reative power timeseries of loads in neighbouring countries

Parameters
  • etrago (:class:`etrago.Etrago) – Transmission grid object

  • cos_phi (float) – Choose ration of active and reactive power of foreign loads

Return type

None

etrago.tools.utilities.connected_grid_lines(network, busids)[source]

Get grid lines connected to given buses.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • busids (list) – List containing bus-ids.

Returns

PyPSA lines.

Return type

class:`pandas.DataFrame

etrago.tools.utilities.connected_transformer(network, busids)[source]

Get transformer connected to given buses.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • busids (list) – List containing bus-ids.

Returns

PyPSA transformer.

Return type

class:`pandas.DataFrame

etrago.tools.utilities.load_shedding(self, temporal_disaggregation=False, **kwargs)[source]

Implement load shedding in existing network to identify feasibility problems

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • marginal_cost (int) – Marginal costs for load shedding

  • p_nom (int) – Installed capacity of load shedding generator

etrago.tools.utilities.data_manipulation_sh(network)[source]

Adds missing components to run calculations with SH scenarios.

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Return type

None

etrago.tools.utilities.export_to_csv(self, path)[source]

Write calculation results to csv-files in path.

Parameters
  • network (pypsa.Network) – Overall container of PyPSA

  • args (dict) – Contains calculation settings of appl.py

  • path (str or False or None) – Choose path for csv-files. Specify “”, False or None to not do anything.

Return type

None

etrago.tools.utilities.loading_minimization(network, snapshots)[source]

Minimizes the sum of the products of each element in the passive_branches of the model.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • snapshots ('pandas.core.indexes.datetimes.DatetimeIndex') – snapshots to perform the minimization

Return type

None

etrago.tools.utilities.agg_series_lines(l0, network)[source]

Given a pandas DataFrame l0 containing information about lines in a network and a network object, aggregates the data in l0 for all its attributes. Returns a pandas Series containing the aggregated data.

Parameters
  • l0 (pandas.DataFrame) – contain information about lines in a network.

  • network (:class:`pypsa.Network) – Overall container of PyPSA

Returns

A pandas Series containing aggregated data for the lines in the network.

Return type

pandas.Series

etrago.tools.utilities.group_parallel_lines(network)[source]

Function that groups parallel lines of the same voltage level to one line component representing all parallel lines

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Return type

None.

etrago.tools.utilities.delete_dispensable_ac_buses(etrago)[source]

Function that identifies and delete AC buses without links, transformers, generators, loads, stores or storage_units, which also are connected to just one or two other buses

Parameters

etrago (etrago object) –

Return type

None.

etrago.tools.utilities.set_line_costs(self, cost110=230, cost220=290, cost380=85, costDC=375)[source]

Set capital costs for extendable lines in respect to PyPSA [€/MVA]

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • args (dict) – containing settings from appl.py

  • cost110 – capital costs per km for 110kV lines and cables default: 230€/MVA/km, source: costs for extra circuit in dena Verteilnetzstudie, p. 146)

  • cost220 – capital costs per km for 220kV lines and cables default: 280€/MVA/km, source: costs for extra circuit in NEP 2025, capactity from most used 220 kV lines in model

  • cost380 – capital costs per km for 380kV lines and cables default: 85€/MVA/km, source: costs for extra circuit in NEP 2025, capactity from most used 380 kV lines in NEP

  • costDC – capital costs per km for DC-lines default: 375€/MVA/km, source: costs for DC transmission line in NEP 2035

etrago.tools.utilities.set_trafo_costs(self, cost110_220=7500, cost110_380=17333, cost220_380=14166)[source]

Set capital costs for extendable transformers in respect to PyPSA [€/MVA]

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • cost110_220 – capital costs for 110/220kV transformer default: 7500€/MVA, source: costs for extra trafo in dena Verteilnetzstudie, p. 146; S of trafo used in osmTGmod

  • cost110_380 – capital costs for 110/380kV transformer default: 17333€/MVA, source: NEP 2025

  • cost220_380 – capital costs for 220/380kV transformer default: 14166€/MVA, source: NEP 2025

etrago.tools.utilities.add_missing_components(self)[source]

Add a missing transformer at Heizkraftwerk Nord in Munich and a missing transformer in Stuttgart.

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

Returns

network – Overall container of PyPSA

Return type

:class:`pypsa.Network

etrago.tools.utilities.convert_capital_costs(self)[source]

Convert capital_costs to fit to considered timesteps

Parameters

etrago (:class:`etrago.Etrago) – Transmission grid object

etrago.tools.utilities.find_snapshots(network, carrier, maximum=True, minimum=True, n=3)[source]

Function that returns snapshots with maximum and/or minimum feed-in of selected carrier.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • carrier (str) – Selected carrier of generators

  • maximum (bool) – Choose if timestep of maximal feed-in is returned.

  • minimum (bool) – Choose if timestep of minimal feed-in is returned.

  • n (int) – Number of maximal/minimal snapshots

Returns

calc_snapshots – List containing snapshots

Return type

‘pandas.core.indexes.datetimes.DatetimeIndex’

etrago.tools.utilities.ramp_limits(network)[source]

Add ramping constraints to thermal power plants.

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

etrago.tools.utilities.get_args_setting(self, jsonpath='scenario_setting.json')[source]

Get and open json file with scenaio settings of eTraGo args. The settings incluedes all eTraGo specific settings of arguments and parameters for a reproducible calculation.

Parameters

json_file (str) – Default: scenario_setting.json Name of scenario setting json file

Returns

args – Dictionary of json file

Return type

dict

etrago.tools.utilities.merge_dicts(dict1, dict2)[source]

Return a new dictionary by merging two dictionaries recursively.

Parameters
  • dict1 (dict) – dictionary 1.

  • dict2 (dict) – dictionary 2.

Returns

result – Union of dict1 and dict2

Return type

dict

etrago.tools.utilities.get_clustering_data(self, path)[source]

Import the final busmap and the initial buses, lines and links

Parameters

path (str) – Name of folder from which to import CSVs of network data.

Return type

None

etrago.tools.utilities.set_random_noise(self, sigma=0.01)[source]

Sets random noise to marginal cost of each generator.

Parameters
  • etrago (:class:`etrago.Etrago) – Transmission grid object

  • seed (int) – seed number, needed to reproduce results

  • sigma (float) – Default: 0.01 standard deviation, small values reduce impact on dispatch but might lead to numerical instability

etrago.tools.utilities.set_line_country_tags(network)[source]

Set country tag for AC- and DC-lines.

Parameters

network (:class:`pypsa.Network) – Overall container of PyPSA

etrago.tools.utilities.crossborder_capacity_tyndp2020()[source]

This function downloads and extracts a scenario datafile for the TYNDP 2020 (Ten-Year Network Development Plan), reads a specific sheet from the file, filters it based on certain criteria, and then calculates the minimum cross-border capacities for a list of European countries. The minimum cross-border capacity is the minimum of the export and import capacities between two countries.

Returns

Dictionary with cossborder capacities.

Return type

dict

etrago.tools.utilities.crossborder_capacity(self)[source]

Adjust interconnector capacties.

Parameters
  • network (:class:`pypsa.Network) – Overall container of PyPSA

  • method (string) – Method of correction. Options are ‘ntc_acer’ and ‘thermal_acer’. ‘ntc_acer’ corrects all capacities according to values published by the ACER in 2016. ‘thermal_acer’ corrects certain capacities where our dataset most likely overestimates the thermal capacity.

etrago.tools.utilities.set_branch_capacity(etrago)[source]

Set branch capacity factor of lines and transformers, different factors for HV (110kV) and eHV (220kV, 380kV).

Parameters

etrago (:class:`etrago.Etrago) – Transmission grid object

etrago.tools.utilities.check_args(etrago)[source]

Function that checks the consistency of etragos input parameters.

Parameters

etrago (:class:`etrago.Etrago) – Overall container of eTraGo

Return type

None.

etrago.tools.utilities.drop_sectors(self, drop_carriers)[source]

Manually drop secors from network. Makes sure the network can be calculated without the dropped sectors.

Parameters

drop_carriers (array) – List of sectors that will be dropped. e.g. [‘dsm’, ‘CH4’, ‘H2_saltcavern’, ‘H2_grid’, ‘central_heat’, ‘rural_heat’, ‘central_heat_store’, ‘rural_heat_store’, ‘Li ion’] means everything but AC

Return type

None.

etrago.tools.utilities.update_busmap(self, new_busmap)[source]

Update busmap after any clustering process

Parameters

new_busmap (dictionary) – busmap used to clusted the network.

Return type

None.

etrago.tools.utilities.adjust_CH4_gen_carriers(self)[source]

Precise the carrier for the generators with CH4 carrier

For the eGon2035 scenario, the generators with carrier CH4 represent the prodution od biogas and methan. In the data model, these two differents types are differenciated only by the marginal cost of the generator. This function introduces a carrier distion (CH4_biogas and CH4_NG) in order to avoid the clustering of these two types of generator together and facilitate the contraint applying differently to each of them.

etrago.tools.utilities.residual_load(network, sector='electricity')[source]

Calculate the residual load for the specified sector.

In case of the electricity sector residual load is calculated using all AC loads and all renewable generators with carriers ‘wind_onshore’, ‘wind_offshore’, ‘solar’, ‘solar_rooftop’, ‘biomass’, ‘run_of_river’, and ‘reservoir’.

In case of the central heat sector residual load is calculated using all central heat loads and all renewable generators with carriers ‘solar_thermal_collector’ and ‘geo_thermal’.

Parameters
  • network (PyPSA network) – Network to retrieve load and generation time series from, needed to determine residual load.

  • sector (str) – Sector to determine residual load for. Possible options are ‘electricity’ and ‘central_heat’. Default: ‘electricity’.

Returns

Dataframe with residual load for each bus in the network. Columns of the dataframe contain the corresponding bus name and index of the dataframe is a datetime index with the corresponding time step.

Return type

pd.DataFrame

etrago.tools.utilities.manual_fixes_datamodel(etrago)[source]

Apply temporal fixes to the data model until a new egon-data run is there

Parameters

etrago (:class:`Etrago) – Overall container of Etrago

Return type

None.

etrago.appl module

Indices and tables