Welcome to eTraGo’s documentation!¶

Warning
Note, eTraGo and its documentation is in continuous development.
About eTraGo¶
eTraGo stands for electric Transmission Grid optimization.
The python package eTraGo provides optimization strategies of flexibility options for transmission grids based on PyPSA. A peculiarity in this context is that the German transmission grid is described by the 380, 220 and 110 kV voltage levels. Conventionally the 110kV grid is part of the distribution grid. The integration of the transmission and ‘upper’ distribution grid is part of eTraGo.
The focus of optimization are flexibility options with a special focus on energy storage and grid expansion measures.
Research projects¶
This software project was initially developed in the research project open_eGo. It is constantly further developed in different reserach projects, e.g. eGon and PoWerD.
The OpenEnergy Platform¶
Within the open_eGo project we developed the OpenEnergy Platform which this software is using in order to get and store the in- and output data. Before you start to calculate a registration on the platform is needed. For more information see openenergy-platform and login.
The OpenEnergy platform mainly addresses students, researchers and scientists in the field of energy modelling and analytics as well as interested persons in those fields. The platform provides great tools to make your energy system modelling process transparent. All data of the open_eGo project are stored at this platform. Learn more about the database access.
Tool overview¶
eDisGo¶
The python package eDisGo provides a toolbox for analysis and optimization of distribution grids. It is closely related to the python project Ding0 as this project is currently the single data source for eDisGo providing synthetic grid data for whole Germany. Learn more here.
eGo¶
The python package eGo is a toolbox and application which connects the tool eTraGo (optimization of flexibility options at transmission grid level) and eDisGo (optimization of distribution grids). All those python packages were initially developed in the research project open_eGo. Learn more here.
Data model creation¶
For the eGon project the python-tool eGon-data was implemented, which creates input data for the optimization tools eTraGo, ding0 and eDisGo and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses.
eGon-data is a further development of the Data processing developed in the former research project open_eGo. It aims for an extensions of the data models as well as for a better replicability and manageability of the data preparation and processing. The resulting data set serves as an input for the optimization tools eTraGo, ding0 and eDisGo and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses.
ego.io¶
The ego.io serves as a SQLAlchemy Interface to the OpenEnergy database (oedb). The oedb table ORM objects are defined here and small helpers for io tasks are contained. Learn more here.
Dingo¶
The DIstribution Network GeneratOr (Ding0) is a tool to generate synthetic medium and low voltage power distribution grids based on open (or at least accessible) data. Learn more here.
LICENSE¶
© Copyright 2015-2023 Flensburg University of Applied Sciences, Europa-Universität Flensburg, Centre for Sustainable Energy Systems and DLR-Institute for Networked Energy Systems
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see www.gnu.org/licenses.
Installation¶
eTraGo is designed as a Python package therefore it is mandatory to have Python 3 installed. If you have a working Python3 environment, use pypi to install the latest eTraGo version. We highly recommend you to use a virtual environment. Use following pip command in order to install eTraGo.
$ pip3 install eTraGo
Using a virtual environment¶
Before installing eTraGo, you create a virtual environment (where you like it) and activate it:
$ virtualenv venv --clear -p python3.8
$ source venv/bin/activate
$ cd venv
Inside your activated virtual environment you can install eTraGo with the pip command, as previously explained.
Linux and Ubuntu¶
The Package eTraGo is tested with Ubuntu 16.04, 18.04, 20.04 and 22.04 inside the virtual environments of virtualenv. The installation is shown above.
Windows or Mac OSX users¶
For Windows and/or Mac OSX user we highly recommend to install and use Anaconda for your Python3 installation. First install Conda including python 3.8 or higher version from https://www.anaconda.com/download/ and open an anaconda prompt as administrator and run:
$ conda config --add channels conda-forge
$ conda create -n etrago_env python=3.8
$ conda activate etrago_env
$ pip install eTraGo
The full Documentation can be found on this page . We use Anaconda with an own environment in order to reduze problems with Packages and different versions on our system. Learn more about Anacona environments.
Setup database connection¶
The package ego.io will be installed
automatically when eTraGo is installed. The egoio
gives you python SQL-Alchemy representations of
the OpenEnergy-Database(oedb)
and access to it by using the
oedialect, which is a SQL-Alchemy binding
Python package for the REST-API used by the OpenEnergy Platform (OEP).
In order to connect eTraGo via the oedialect with the oedb you
have to create an account at
openenergy-platform.org/login.
You can name the ‘db’
argument of the ‘args’ of the etrago.appl.etrago()
as you wish. Once the etrago.appl.etrago()
is executed you will be asked
to enter how you want to connect to which database. If you want to use
the oedialect enter the following connection parameter. For <username> and
<token> you have to take your credentials which you obtained by registering
at openenergy-platform.org/login.
Your API access / login data will be saved in the folder .egoio
in the file
config.ini
. Consequently, in the config.ini you can also change
your connection parameters or add new ones.
In the following you can see how the config.ini looks like when you use the
oedialect, a local postgresql database or the old psycopg2 developer connection.
Once you have created a connection (which is saved in the config.ini) you do not have to enter the connection parameter again. The software will take the connection parameter which corresponds to the entry at the ‘db’ argument.
oedialect connection¶
[oedb]
dialect = oedialect
username = <username>
database = oedb
host = openenergy-platform.org
port = 80
password = <token>
Local database connection¶
[local]
username = YourOEDBUserName
database = YourLocalDatabaseName
host = localhost or 127.0.0.1
port = 5433
pw = YourLocalPassword
Old developer connection¶
[oedb]
username = YourOEDBUserName
database = oedb
host = oe2.iws.cs.ovgu.de
port = 5432
pw = YourOEDBPassword
How to use eTraGo?¶
After you installed eTraGo you would typically start optimization runs by
executing the ‘appl.py’ which is situated in
./eTrago/etrago/
(e.g by python3 appl.py
from the terminal).
eTraGo doesn’t have a graphical user interface,
the ‘appl.py’ is used as a simple user interface which can be edited with
the preferred python-editor.
Here parameters, calculation methods and scenario settings are set in a python
dictionary called ‘args’.
To run the desired calculation, it is crucial to understand these parameters.
In addition, some of them contradict the usage of others.
You find the documentation of all defined parameters from the ‘args’ here:
etrago.appl.run_etrago()
.
Alternatively, the ‘args’ dictionary can be edited in a json-file.
Then the path to the json-file has to be set in the initilization of the
Etrago-object (etrago.tools.network.Etrago
). Once a path is given
the ‘args’ dictionary within the ‘appl.py’ is ignored
and replaced by the ‘args’ of the json-file.
The appl.py contains the etrago.appl.run_etrago()
function which uses the
defined ‘args’ dictionary to start the desired calculation.
To improve the performance of the optimization of the selected solver, you might want to use solver options (part of ‘args’). For gurobi the most used ones are described here.
For more specific or extensive changes you are highly invited to write code and add new functionalities.
Once the calculation has finished the PyPSA network of the Etrago-object will
contain all results. Some main results (e.g. anuual system costs) are calculated
by etrago.calc_results()
and can be accesed via ‘etrago.results’.
You can use several plotting functions from the etrago.tools.plot()
in order
to visualize the results. For example
the etrago.tools.plot.plot_grid()
can be used to plot relative line loading
in % or the optimized expansion of all AC lines and DC links of the network.
To save the results you can write them to csv files. These functionalites can be specified also in the ‘args’ dictionary.
Examples and tutorial notebooks¶
eTraGo version 0.5.1: etrago_OpenMod_Zuerich18.
eTraGo version 0.9: Tutorial notebook will be available in the next release.
Theoretical Background¶
Definitions and Units¶
eTraGo is based on the open source tool PyPSA and uses its definitions and units.
Assumptions on Data¶
eTraGo fetches the input data from the OpenEnergy Platform. The data includes electricity and gas grid topology as well as data on energy supply and load for the considered sectors (electricity, gas, heat and e-mobility) plus data on flexibility potential deriving from those sectors e.g. Dynamic Line Rating, Demand Side Management and flexibility potentials arising from e-mobility. More details on the data model can be found in the documentaton of eGon-data.
At the moment, there are two scenarios available basing on scenario C2035 of the network expansion plan ([NEP]), version 2021. The base one is called eGon2035. To analyse the effect of flexibility options, there is an eGon2035_lowflex scenario available which depicts a lower penetration of flexibilities. More scenarios are being developed. The eGon100RE scenario is being implemented which is characterised by a 100% renewable generation. Analog to the scenario above, a eGon100RE_lowflex scenario will be available.
You can see the modeling concepts of the scenarios in the figure below. The components marked green have exogenous capacity and endogenous dispatch whereas the components marked in red are optimised endogenously in capacity and dispatch.

Methods¶
Optimisation with PyPSA¶
Within eTraGo, the fetched data model is translated into a PyPSA-network. The optimisation is performed with a linear approximation assuming eTraGo to fulfill the assumptions to perfom a LOPF (as those are small voltage angle differences, branch resistances negligible to their reactances, voltage magnitudes can be kept at nominal values) since it focuses on the extra-high and high voltage levels. As objective value of the optimisation, the overall system costs are considered.
With the argument ‘pf_post_lopf’, after the LOPF a non-linear power flow simulation can be conducted.
Complexity Reduction¶
The data model is characterised by a high spatial (abou 8,000 electrical and 600 gas nodes) and temporal resolution (8,760 timesteps). To reduce the complexity of the resulting optimisation problem, several methods can be applied.
Reduction in spatial dimension:¶
The ehv clustering maps all electrical nodes with a voltage level below the extra-high voltage level to their nearest neighboring node in the extra-high voltage level with the Dijkstra’s algorithm (110 kV —> 220 / 380 kV).
The k-means Clustering reduces the electrical or gas network to an adjustable number of nodes by considering the geographical position of the respective nodes. This method has been implemented within PyPSA by [Hoersch].
The k-medoids Dijkstra Clustering aggregates nodes considering the network topology. First, a k-medoids Clustering is used dividing the original nodes of the network into groups by their geographical positions while identifiying the geographical medoid nodes per cluster. Afterwards, the original nodes in the original network are assigned to the former identified medoids considering the original network’s topology applying a Dijkstra’s algorithm considering the line lengths. Afterall, the original nodes are represented by one aggregated node per cluster at the position of the former identified medoid node.
In general, the clustering of the sector-coupled system is divided into two steps: First, the electrical and gas grid are clustered independently using one of the methods described above. Afterwards, nodes of the other sectors (hydrogen, heat, e-mobility and DSM nodes) are mapped according to their connection to electricity or gas buses and aggregated to one node per carrier.
After optimising the spatially reduced network, a spatial disaggregation can be conducted.
Reduction in temporal dimension:¶
The method Skip Snapshots implies a downsampling to every nth time step. The considered snapshots are weighted respectively to account for the analysis of one whole year.
By using the method called Segmentation, a hierarchical clustering of consecutive timesteps to segments with variable lengths is applied [Pineda].
The Snapshot Clustering on Typical Periods implies a hierarchical clustering of time periods with a predefined length (e.g. days or weeks) to typical periods. Those typical periods are weighted according to the number of periods in their cluster. This method optionally includes the linkage of the typical periods in a second time layer to account for the intertemporal dependencies following [Kotzur].
By applying a 2-level-approach, a temporal disaggregation can be conducted. This means optimising dispatch using the fullcomplex time series in the second step after having optimised grid and storage expansion using the complexity-reduced time series in the first step.
Grid and Storage / Store expansion¶
The grid expansion is realized by extending the capacities of existing lines and substations. These capacities are considered as part of the optimisation problem whereby the possible extension is unlimited. With respect to the different voltage levels and lengths, MVA-specific costs are considered in the optimisation.
As shown in the figure above, several options to store energy are part of the modeling concept. Extendable batteries (modeled as storage units) are assigned to every node in the electrical grid. A minimum installed capacity is being considered to account for home batteries ([NEP]). The expansion and operation is part of the optimisation. Furthermore, two types of hydrogen stores (modeled as stores) are available. Overground stores are optimised in operation and dispatch without limitations whereas underground stores depicting saltcaverns are limited by geographical conditions ([BGR]). Additionally, heat stores part of the optimisation in terms of power and energy without upper limits.
Miscellaneous Features¶
Several features were developed to enhance the functionality of eTraGo.
To customize computation settings, ‘solver_options’ and ‘generator_noise’ should be adapted. The latter adds a reproducible small random noise to the marginal costs of each generator in order to prevent an optima plateau. The specific solver options depend on the applied solver (e.g. Gurobi, CPLEX or GLPK).
In ‚extendable‘ you can adapt the type of components you want to be optimised in capacity and set upper limits for gird expansion inside Germany and of lines to foreign countries.
The ‚extra_functionality‘-argument allows to consider extra constraints like limits for energy imort and export or minimal renewable shares in generation.
‘branch_capacity_factor’ adds a factor to adapt all line capacities in order to consider (n-1) security. Because the average number of HV systems is much smaller than the one of eHV lines, you can choose factors for ‘HV’ and ‘eHV’ separately.
The ‘load_shedding’-argument is used for debugging complex grids in order to avoid infeasibilities. It introduces a very expensive generator at each bus to meet the demand. When optimising storage units and grid expansion without limiting constraints, the need for load shedding should not be existent.
With ‘foreign_lines‘ you can adapt the foreign lines to be modeled as DC-links (e.g. to avoid loop flows).
References¶
- NEP(1,2)
Übertragungsnetzbetreiber Deutschland (2021): Netzentwicklungsplan Strom 2035, Version 2021, 1. Entwurf. 2021.
- Hoersch
Jonas Hoersch et al. (2017): The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios. 2017. https://arxiv.org/pdf/1705.07617.pdf
- Pineda
Salvador Pineda et al. (2018): Chronological Time-Period Clustering for Optimal Capacity Expansion Planning With Storage. 2018. https://ieeexplore.ieee.org/document/8369128
- Kotzur
Leander Kotzur et al. (2018): Time series aggregation for energy system design: Modeling seasonal storage. 2018. https://arxiv.org/pdf/1710.07593.pdf
- BGR
Bundesanstalt fuer Geowissenschaften und Rohstoffe et al. (2020): nSpEE-DS - Teilprojekt Bewertungskriterien und Potenzialabschätzung. 2020. https://www.bgr.bund.de/DE/Themen/Nutzung_tieferer_Untergrund_CO2Speicherung/Downloads/InSpeeDS_TP_Bewertungskriterien.pdf?__blob=publicationFile&v=3
Developer notes¶
Installation for Developers¶
Note
Installation is primarly tested on (Ubuntu like) linux OS.
If you like, create a virtual environment (where you like it) and activate it (if you do not use venv start with 2.):
$ virtualenv --clear -p python3.8 etrago``
$ cd etrago/
$ source bin/activate
Clone the source code from github:
$ git clone https://github.com/openego/eTraGo
You can checkout to the dev branch and create new feature branches. For the correct work-flow, please mind the Dreissen Branching Model
Use the pip -e to install eTraGo directly from the cloned repository:
$ pip3 install -e /path/to/eTraGo/
What’s New¶
These are new features and improvements of note in each release.
Releases
Release 0.9.0 (XXXXX, 2023)¶
Added features¶
eTraGo is now compatible with Python 3.8
eTraGo can now import and optimize networks that include other energy sectors such as gas, heating and mobility
Various flexibility options from different energy sectors can be considered in the optimization:
Weather dependent capacity of transmission lines (Dynamic Line Rating)
Demand Side Management
Flexible charging of electric vehicles
Heat and hydrogen stores
Power2Hydrogen, Hydrogen2Power
Methanation and Steam Methane Reforming
eTraGo arguments can now be partially provided and updated
eTraGo can now import datamodels from databases without using the ego.io
Existing clustering methods were adapted to be able to reduce the complexity of not electrical sectors
Improvement of the ehv clustering (much faster now)
A new clustering method named “k-medoids Dijkstra Clustering” (can be called by “kmedoids-dijkstra”) was implemented. This method considers the electrical distance between the buses in the network. It is also available for the methane grid.
It is possible to select if foreign buses are considered or not during the clustering process.
The number of CPUs used to perform the clustering can be provided by the user.
Some more options are available to conduct a reduction in temporal dimension:
segmentation: clustering of adjacent hours to segments of variable length
clustering to typical periods extended to cluster on weeks and months
A temporal disaggregation is available through a 2-level-approach including a dispatch optimization on the temporally fullcomplex model. To limit the RAM usage, you can optionally divide the optimisation problem into a chosen number of slices.
New plotting functions to visualize the optimization results from all the included energy sectors were implemented
Functions to analyze results were updated to consider new sectors
Release 0.8.0 (April 8, 2021)¶
eTraGo has now a more object-oriented programming design.
Added features¶
eTraGo uses PyPSA version 0.17.1 directly, the fork is not needed anymore. The updated pypsa version includes various features, e.g. running a lopf without using pyomo which is faster and needs less memory.
(n-1)-security factors are set as line/transformer parameters s_max_pu instead of adding the additional argument s_nom_original
There is now one central plotting function for all grid topology plots which also allows to combine different results (e.g. plot storage expansion and line expansion at once)
eTraGo is now compatible to Python3.7
A bug in setting the line_length_factor in kmeans clustering is fixed.
Release 0.7.2 (Juni 15, 2020)¶
A minor release adding the following features.
Added features¶
for single use of eTraGo (not as a sub-module of eGo), we recommend to use the newest minor data release ‘gridversion’: ‘v0.4.6’. This data release includes some minor bug fixes but it is not consistent with the data on the MV and LV levels. Hence, the modelling results are only adequate for the HV and EHV level applying solely the tool eTraGo.
snapshot clustering includes now an approach to model seasonal storage as in Kotzur et al, 2018 ( https://www.sciencedirect.com/science/article/pii/S0306261918300242 ). Moreover the method may include extreme periods using an option of the tsam package.
osm maps can now be used for background plotting
the iterate_lopf function enables to adequately model the reactances when expanding the grid
important bug fix for the adjustment of reactances when harmonizing the voltage level when applying the k-means network clustering
multiple extra_functionalities can be called easily called now at once
various minor changes such as specifying installation requires for flawless performance
Release 0.7.1 (October 25, 2018)¶
A minor release adding new options for additional constraints, modelling assumptions and plotting.
Added features¶
Two extra functionalities were introduced in order to apply constraints concerning a minimal share of renewable energy and a global upper bound for grid expansion. You can activate these functions in the ‘args’ of the etrago() function.
The branch_capacity_factor can now be defined separately for the high and extra high voltage level in order to address the (n-1) criteria more accurately.
There are some more plotting functions e.g. plotting the state-of-charge and dispatch of storage units.
Storage capacities in foreign countries can easily be be optimized.
By default the maximum expansion of each line and transformer is set to four times its original capacity. Being an argument of the extendable() function it can be easily adjusted.
k-means clustered results can now also be exported to the oedb.
Release 0.7.0 (September 6, 2018)¶
eTraGo is able to produce feasible non-linear power flows based on optimization results and allows the disaggregation of clustered results to original spatial complexities.
Added features¶
The pf_post_lopf function was improved. Due to changes in the data set now the non-linear power flow (pf) creates feasible solutions. If network optimization is turned on, a second lopf which regards the updated reactances and optimizes only dispatch is performed before the pf is executed.
The disaggregation method was included. When using a network clustering method to reduce the spatial complexity of the given network, a disaggregation method can be used afterwards to distribute the nodal results (generation and storage timeseries) to the original complexity. The method ‘disaggregation’: ‘uniform’ can be used as an interface functionality for distribution grid planning tools like eDisGo.
For the network expansion it is now additionally possible to only optimize the German power lines or only the crossborder lines. Moreover one can choose to optimize only a predefined set of power lines which are identified by a worst-case analysis beforehand.
Intertemporal constraints can be applied to certain power plants. For different technologies certain parameters i.e. ‘start_up_cost’, ‘start_up_fuel’, ‘min_up_time’ and ‘min_down_time’ are defined in the ramp_limits function.
Crossborder lines can now easily be modelled as ‘DC’ links. Moreover the capacities of these lines can be adjusted with respect to a ACER report on thermal as well as net transfer capacities.
Thanks to @jankaeh manually the grid topology within the cities Stuttgart, Munich and Hannover was improved. Perspectively this function should be obsolete when openstreetmap and/or osmTGmod get better data coverage.
As an alternative to the normal editing of the calcualtion settings (args) within the appl.py it is now possible to load an args.json file.
Release 0.6.1 (Juli 18, 2018)¶
eTraGo works with pypi and is suitable for eGo 0.2.0
Added features¶
An installation issue when installing from pypi was fixed.
The random noise function was improved. Now you set a (reproducible) random seed.
snapshot.weightings are used within the plotting functions
bug fix for k-means clustering with respect to the aggregation of p_max_pu values of variable generators. They are weighted by their p_nom now.
Release 0.6 (June 27, 2018)¶
eTraGo now enables combined grid and storage expansion, snapshot clustering and the consideration of exogenous grid expansion.
Added features¶
A part from optimizing the investment of storages it is now also possible to optimize grid expansion investments. In this context we added an argument ‘extendable’ which expects an array of the different component types you want to optimize. This argument corresponds to functions in the new extendable.py file. It is possible to choose from expansion strategies which are defined within that sub-package. Capital costs for new grid components can be defined and are annualized by means of interest rate, component lifetime and operation period.
The k-means network clustering (‘network_clustering_kmeans’) has been refactored. It is now possible to reproduce busmaps by csv-importing already defined busmaps. Consequently it is possible to write busmaps. Here the argument ‘load_cluster’ was introduced. Moreover it is possible read and write bus_weightings. This helps to e.g. run a future scenario but using a bus weighting of the status quo. Moreover, the remove_stubs function from PyPSA is now easily usable in eTraGo.
The snapshot_clustering can now be used in order to reduce the temporal complexity. Typical days are taken in order to represent the entire year. Here the package tsam (developed by Leander Kotzur) is used. Standardly a hierarchical clustering method is used which is e.g. described by Nahmacher et al. ( see: https://www.sciencedirect.com/science/article/pii/S0360544216308556 ).
Scenario variations from the oedb can be introduced. The argument ‘scn_extension’ will activate an extension scenario which adds components such as lines or generator to the base scenario. The ‘scn_decommissioning’ argument states whether you want to remove existing components. Right now, in the oedb two scenarios are accessible which represent the grid expansion (and the corresponding removal of existing equipment) planned by the German network development plan.
Our data model at the oedb represents Germany and the electrical neighbors. If you consider planned grid expansion to Norway and Belgium you would most probably want to include also these countries as electrical neighbors including their aggregated generation and demand characteristics. The argument ‘add_Belgium_Norway’ therefore was introduced. Once activated it will add the countries to the model.
DC links are now also modelled as PyPSA DC links. Consequently Sweden is now connected by a DC link.
Other changes¶
The plotting sub-package was amplified and enhanced by new plotting functions and improvements of existing ones (e. g. introduced a legend for the plotting function storage_expansion())
The code complies now mostly with the pep8 standard
Documentation was improved (but ongoing work in progress) considering doc strings for functions and the rtd-documentation web site
The io was slightly reformatted and restructured.
Release 0.5.1 (February 01, 2018)¶
eTraGo works with ego.io 0.3.0
Added features¶
Result export to oedb functionality was improved. Now, a safe tag can be set in order to state that the result set shall be versioned and moved to the schema ‘grid’.
the new database sessionmaker of ego.io 0.3.0 is enabled which gets rid of dependency to oemof.db.
it is possible to skip snapshots in order to ingenuously simplify the problem.
Other news¶
eTraGo will be available on PyPI
eTraGo 0.5.1 will be used for eGo 0.0.1
Release 0.5 (December 08, 2017)¶
eTraGo works with PyPSA 0.11.0.
Added features¶
Readthedocs documentation
Result export to postgresql database in particular to the open energy data base (oedb)
parallelisation function hands over SOC of storages to following problem.
New plot for displaying voltage deviations at network buses.
Line loading plot displays the direction of power flows.
Bug fixes¶
k-means clustering got a more suitable scaling factor concerning calculations on the 110kV grid.
K-means weighting of the buses is a bit more robust to changes in the data structure.
the list of carriers is imported again to the pypsa network from the oedb data model.
Other changes¶
update PyPSA Version from 0.8.0 to PyPSA 0.11.0
Release 0.4 (October 12, 2017)¶
eTraGo integrates ego.powerflow functionalities
Release 0.4 is mainly the merging of ego.powerflow into eTraGo. Additionally, some restructuring has been carried out, plotting functions have been updated and the first approach for a documentation was set up.
Other changes¶
merging of ego.powerflow into eTraGo
Release 0.3 (September 8, 2017)¶
Release introducing k-means clustering and several additional functionalities
Added features¶
k-means clustering
Release 0.2 (July 20, 2017)¶
This is the version 0.2 of eTraGo.
This new version shall be install via pip and a setup.py. Moreover there has been minor adjustments such as the creation of a callable etrago function.
Added features¶
install via pip and a setup.py
callable etrago function
Release 0.1 (June 30, 2017)¶
First release of eTraGo
API¶
etrago.cluster package¶
etrago.cluster.disaggregation module¶
etrago.cluster.electrical module¶
electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.
- etrago.cluster.electrical.adjust_no_electric_network(etrago, busmap, cluster_met)[source]¶
Adjusts the non-electric network based on the electrical network (esp. eHV network), adds the gas buses to the busmap, and creates the new buses for the non-electric network.
- Parameters
- Returns
network (pypsa.Network) – Container for all network components of the clustered network.
busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.
- etrago.cluster.electrical.cluster_on_extra_high_voltage(etrago, busmap, with_time=True)[source]¶
Main function of the EHV-Clustering approach. Creates a new clustered pypsa.Network given a busmap mapping all bus_ids to other bus_ids of the same network.
- Parameters
- Returns
network (pypsa.Network) – Container for all network components of the clustered network.
busmap (dict) – Maps old bus_ids to new bus_ids including all sectors.
- etrago.cluster.electrical.delete_ehv_buses_no_lines(network)[source]¶
When there are AC buses totally isolated, this function deletes them in order to make possible the creation of busmaps based on electrical connections and other purposes. Additionally, it throws a warning to inform the user in case that any correction should be done.
- Parameters
network (pypsa.network) –
- Return type
None
- etrago.cluster.electrical.ehv_clustering(self)[source]¶
Cluster the network based on Extra High Voltage (EHV) grid.
If network_clustering_ehv argument is True, the function clusters the network based on the EHV grid.
- Parameters
self (Etrago object pointer) – The object pointer for an Etrago object.
- Return type
None
- etrago.cluster.electrical.select_elec_network(etrago)[source]¶
Selects the electric network based on the clustering settings specified in the Etrago object.
- Parameters
etrago (Etrago) – An instance of the Etrago class
- Returns
- elec_networkpypsa.Network
Contains the electric network
- n_clustersint
number of clusters used in the clustering process.
- Return type
Tuple containing
- etrago.cluster.electrical.unify_foreign_buses(etrago)[source]¶
Unifies foreign AC buses into clusters using the k-medoids algorithm with Dijkstra distance as a similarity measure.
- Parameters
etrago (Etrago) – An instance of the Etrago class
- Returns
busmap_foreign – A pandas series that maps the foreign buses to their respective clusters. The series index is the bus ID and the values are the corresponding cluster medoid IDs.
- Return type
pd.Series
- etrago.cluster.electrical.preprocessing(etrago)[source]¶
Preprocesses an Etrago object to prepare it for network clustering.
- Parameters
etrago (Etrago) – An instance of the Etrago class
- Returns
network_elec (pypsa.Network) – Container for all network components of the electrical network.
weight (pandas.Series) – A pandas.Series with the bus weighting data.
n_clusters (int) – The number of clusters to use for network clustering.
busmap_foreign (pandas.Series) – The Series object with the foreign bus mapping data.
- etrago.cluster.electrical.postprocessing(etrago, busmap, busmap_foreign, medoid_idx=None)[source]¶
Postprocessing function for network clustering.
- Parameters
etrago (Etrago) – An instance of the Etrago class
busmap (pandas.Series) – mapping between buses and clusters
busmap_foreign (pandas.DataFrame) – mapping between foreign buses and clusters
medoid_idx (pandas.DataFrame) – mapping between cluster indices and medoids
- Returns
- clusteringpypsa.network
Network object containing the clustered network
- busmappandas.Series
Updated mapping between buses and clusters
- Return type
Tuple containing
- etrago.cluster.electrical.weighting_for_scenario(network, save=None)[source]¶
define bus weighting based on generation, load and storage
- Parameters
- Returns
weight – Serie with the weight assigned to each bus to perform a k-mean clustering.
- Return type
pandas.series
- etrago.cluster.electrical.run_spatial_clustering(self)[source]¶
Main method for running spatial clustering on the electrical network. Allows for clustering based on k-means and k-medoids dijkstra.
- Parameters
self – The object pointer for an Etrago object containing all relevant parameters and data
- Return type
None
etrago.cluster.gas module¶
gas.py defines the methods to cluster gas grid networks spatially for applications within the tool eTraGo.
- etrago.cluster.gas.preprocessing(etrago)[source]¶
Preprocesses the gas network data from the given Etrago object for the spatial clustering process of the CH4 grid.
- Parameters
etrago (Etrago) – An instance of the Etrago class
- Return type
None
- Raises
ValueError – If settings[“n_clusters_gas”] is less than or equal to the number of neighboring country gas buses.
- etrago.cluster.gas.kmean_clustering_gas(etrago, network_ch4, weight, n_clusters)[source]¶
Performs K-means clustering on the gas network data in the given network_ch4 pypsa.Network object.
- Parameters
etrago (Etrago) – An instance of the Etrago class
network_ch4 (pypsa.Network) – A Network object containing the gas network data.
weight (str or None) – The name of the bus weighting column to use for clustering. If None, unweighted clustering is performed.
n_clusters (int) – The number of clusters to create.
- Returns
busmap (pandas.Series) – A pandas.Series object mapping each bus in the CH4 network to its corresponding cluster ID
None – None is returned because k-means clustering makes no use of medoids
- etrago.cluster.gas.get_h2_clusters(etrago, busmap_ch4)[source]¶
Maps H2 buses to CH4 cluster IDds and creates unique H2 cluster IDs.
- Parameters
etrago (Etrago) – An instance of the Etrago class
busmap_ch4 (pd.Series) – A Pandas Series mapping each bus in the CH4 network to its corresponding cluster ID.
- Returns
busmap – A Pandas Series mapping each bus in the combined CH4 and H2 network to its corresponding cluster ID.
- Return type
pd.Series
- etrago.cluster.gas.gas_postprocessing(etrago, busmap, medoid_idx=None)[source]¶
Performs the postprocessing for the gas grid clustering based on the provided busmap and returns the clustered network.
- Parameters
etrago (Etrago) – An instance of the Etrago class
busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.
medoid_idx (pd.Series) – A pandas.Series object containing the medoid indices for the gas network.
- Returns
network_gasgrid_c (pypsa.Network) – A pypsa.Network containing the clustered network.
busmap (pd.Series) – A Pandas Series mapping each bus to its corresponding cluster ID.
- etrago.cluster.gas.highestInteger(potentially_numbers)[source]¶
Fetch the highest number of a series with mixed types
- Parameters
potentially_numbers (pandas.Series) – Series with mixed dtypes, potentially containing numbers.
- Returns
highest – Highest integer found in series.
- Return type
- etrago.cluster.gas.simultaneous_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]¶
Cluster sector coupling technology based on multiple connected carriers.
The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.
- Parameters
- Returns
Busmap for the sector coupling cluster.
- Return type
- etrago.cluster.gas.consecutive_sector_coupling(network, busmap, carrier_based, carrier_to_cluster)[source]¶
Cluster sector coupling technology based on single connected carriers.
The topology of the sector coupling technology must be in a way, that the links connected to other sectors do only point inwards. E.g. for the heat sector, heat generating technologies from electricity or gas only point to the heat sector and not vice-versa.
- Parameters
- Returns
busmap_sc – Busmap for the sector coupled cluster.
- Return type
- etrago.cluster.gas.sc_multi_carrier_based(buses_to_cluster, connected_links)[source]¶
Create busmap for sector coupled carrier based on multiple other carriers.
- Parameters
buses_to_cluster (pandas.Series) – Series containing the buses of the sector coupled carrier which are to be clustered.
connected_links (pandas.DataFrame) – Links that connect from the buses with other carriers to the buses of the sector coupled carrier.
- Returns
busmap – Busmap for the sector coupled carrier.
- Return type
- etrago.cluster.gas.sc_single_carrier_based(connected_links)[source]¶
Create busmap for sector coupled carrier based on single other carrier.
- Parameters
connected_links (pandas.DataFrame) – Links that connect from the buses with other carrier to the buses of the sector coupled carrier.
- Returns
busmap – Busmap for the sector coupled carrier.
- Return type
- etrago.cluster.gas.get_clustering_from_busmap(network, busmap, line_length_factor=1.0, with_time=True, bus_strategies={}, one_port_strategies={})[source]¶
Aggregates components of the given network based on a bus mapping and returns a clustered gas grid pypsa.Network.
- Parameters
network (pypsa.Network) – The input pypsa.Network object
busmap (pandas.Sereies :) – A mapping of buses to clusters
line_length_factor (float) – A factor used to adjust the length of new links created during aggregation. Default is 1.0.
with_time (bool) – Determines whether to copy the time-dependent properties of the input network to the output network. Default is True.
bus_strategies (dict) – A dictionary of custom strategies to use during the aggregation step. Default is an empty dictionary.
one_port_strategies (dict) – A dictionary of custom strategies to use during the one-port component aggregation step. Default is an empty dictionary.
- Returns
network_gasgrid_c – A new gas grid pypsa.Network object with aggregated components based on the bus mapping.
- Return type
pypsa.Network
- etrago.cluster.gas.run_spatial_clustering_gas(self)[source]¶
Performs spatial clustering on the gas network using either K-means or K-medoids-Dijkstra algorithm. Updates the network topology by aggregating buses and links, and then performs postprocessing to finalize the changes.
- Return type
None
- Raises
ValueError – If the selected method is not “kmeans” or “kmedoids-dijkstra”.:
etrago.cluster.snapshot module¶
This module contains functions for reducing the complexity of a PyPSA network in temporal dimension by a) downsampling to every n-th snapshot b) clustering to typical periods (eg days, weeks) c) clustering to segments of variable length Essentially used is the tsam package ( https://github.com/FZJ-IEK3-VSA/tsam ) developed by Leander Kotzur et al.
- etrago.cluster.snapshot.snapshot_clustering(self)[source]¶
Function to call the snapshot clustering function with the respecting method and settings.
- Raises
ValueError – When calling a non-available function.
- Return type
None.
- etrago.cluster.snapshot.tsam_cluster(timeseries_df, typical_periods=10, how='daily', extremePeriodMethod='None', segmentation=False, segment_no=10, segm_hoursperperiod=24)[source]¶
Conducts the clustering of the snapshots for temporal aggregation with the respecting method.
- Parameters
timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.
typical_periods (int, optional) – Number of clusters for typical_periods. The default is 10.
how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.
extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.
segmentation (boolean, optional) – Argument to activate segmenation method. The default is False.
segment_no (int, optional) – Number of segments for segmentation. The default is 10.
segm_hoursperperiod (int, optional) – Only for segmentation, ensures to cluster to segments considering all snapshots. The default is 24.
- Returns
df_cluster (pd.DataFrame) – Information on cluster after clustering to typical periods.
cluster_weights (dict) – Weightings per cluster after clustering to typical periods.
dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.
hours (int) – Hours per typical period.
df_i_h (pd.DataFrame) – Information on cluster after clustering to typical periods.
timeseries (pd.DataFrame) – Information on segments after segmentation.
- etrago.cluster.snapshot.segmentation_extreme_periods(timeseries_df, timeseries, extremePeriodMethod)[source]¶
Function to consider extreme snapshots while using segmentation.
- Parameters
timeseries_df (pd.DataFrame) – Dataframe wit timeseries to cluster.
timeseries (pd.DataFrame) – Information on segments after segmentation.
extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.
- Raises
ValueError – When calling wrong method to consider extreme values.
- Returns
timeseries – Information on segments including extreme snapshots after segmentation.
- Return type
pd.DataFrame
- etrago.cluster.snapshot.run(network, n_clusters=None, how='daily', segmented_to=False, extreme_periods='None')[source]¶
Function to call the respecting snapshot clustering function and export the result to a csv-file.
- Parameters
network (pypsa.Network object) – Container for all network components.
n_clusters (int, optional) – Number of clusters for typical_periods. The default is None.
how ({'daily', 'weekly', 'monthly'}, optional) – Definition of period for typical_periods. The default is ‘daily’.
segmented_to (int, optional) – Number of segments for segmentation. The default is False.
extremePeriodMethod ({'None','append','new_cluster_center', 'replace_cluster_center'}, optional) – Method to consider extreme snapshots in reduced timeseries. The default is ‘None’.
- Returns
network – Container for all network components.
- Return type
pypsa.Network object
- etrago.cluster.snapshot.prepare_pypsa_timeseries(network)[source]¶
Prepares timeseries and residual load timeseries for clustering.
- Parameters
network (pypsa.Network object) – Container for all network components.
- Returns
df – Timeseries to be considered when clustering.
- Return type
pd.DataFrame
- etrago.cluster.snapshot.update_data_frames(network, cluster_weights, dates, hours, timeseries, segmentation)[source]¶
Updates the snapshots, snapshot weightings and the dataframes based on the original data in the network and the medoids created by clustering these original data.
- Parameters
network (pypsa.Network object) – Container for all network components.
cluster_weights (dict) – Weightings per cluster after clustering to typical periods.
dates (DatetimeIndex) – Dates of clusters after clustering to typical periods.
hours (int) – Hours per typical period.
timeseries (pd.DataFrame) – Information on segments after segmentation.
segmentation (boolean) – Checks if segmentation of clustering to typical periods has been used.
- Returns
network – Container for all network components.
- Return type
pypsa.Network object
etrago.cluster.spatial module¶
spatial.py defines the methods to run spatial clustering on networks.
- etrago.cluster.spatial.group_links(network, with_time=True, carriers=None, cus_strateg={})[source]¶
Aggregate network.links and network.links_t after any kind of clustering
- Parameters
network (pypsa.Network object) – Container for all network components.
with_time (bool) – says if the network object contains timedependent series.
carriers (list of strings) – Describe which type of carriers should be aggregated. The default is None.
strategies (dictionary) – custom strategies to perform the aggregation
- Returns
new_df – links aggregated based on bus0, bus1 and carrier
new_pnl – links time series aggregated
- etrago.cluster.spatial.graph_from_edges(edges)[source]¶
Constructs an undirected multigraph from a list containing data on weighted edges.
- Parameters
edges (list) – List of tuples each containing first node, second node, weight, key.
- Returns
M
- Return type
networkx.classes.multigraph.MultiGraph
- etrago.cluster.spatial.shortest_path(paths, graph)[source]¶
Finds the minimum path lengths between node pairs defined in paths.
- Parameters
paths (list) – List of pairs containing a source and a target node
graph (
networkx.classes.multigraph.MultiGraph
) – Graph representation of an electrical grid.
- Returns
df – DataFrame holding source and target node and the minimum path length.
- Return type
pd.DataFrame
- etrago.cluster.spatial.busmap_by_shortest_path(etrago, scn_name, fromlvl, tolvl, cpu_cores=4)[source]¶
Creates a busmap for the EHV-Clustering between voltage levels based on dijkstra shortest path. The result is automatically written to the model_draft on the <OpenEnergyPlatform>[www.openenergy-platform.org] database with the name ego_grid_pf_hv_busmap and the attributes scn_name (scenario name), bus0 (node before clustering), bus1 (node after clustering) and path_length (path length). An AssertionError occurs if buses with a voltage level are not covered by the input lists ‘fromlvl’ or ‘tolvl’.
- Parameters
network (pypsa.Network) – Container for all network components.
session (sqlalchemy.orm.session.Session object) – Establishes interactions with the database.
scn_name (str) – Name of the scenario.
fromlvl (list) – List of voltage-levels to cluster.
tolvl (list) – List of voltage-levels to remain.
cpu_cores (int) – Number of CPU-cores.
- Return type
None
- etrago.cluster.spatial.busmap_from_psql(etrago)[source]¶
Retrieves busmap from model_draft.ego_grid_pf_hv_busmap on the <OpenEnergyPlatform>[www.openenergy-platform.org] by a given scenario name. If this busmap does not exist, it is created with default values.
- etrago.cluster.spatial.kmean_clustering(etrago, selected_network, weight, n_clusters)[source]¶
Main function of the k-mean clustering approach. Maps an original network to a new one with adjustable number of nodes and new coordinates.
- Parameters
network (pypsa.Network) – Container for all network components.
n_clusters (int) – Desired number of clusters.
load_cluster (boolean) – Loads cluster coordinates from a former calculation.
line_length_factor (float) – Factor to multiply the crow-flies distance between new buses in order to get new line lengths.
remove_stubs (boolean) – Removes stubs and stubby trees (i.e. sequentially reducing dead-ends).
use_reduced_coordinates (boolean) – If True, do not average cluster coordinates, but take from busmap.
bus_weight_tocsv (str) – Creates a bus weighting based on conventional generation and load and save it to a csv file.
bus_weight_fromcsv (str) – Loads a bus weighting from a csv file to apply it to the clustering algorithm.
- Returns
network – Container for all network components.
- Return type
pypsa.Network
- etrago.cluster.spatial.dijkstras_algorithm(buses, connections, medoid_idx, cpu_cores)[source]¶
Function for combination of k-medoids Clustering and Dijkstra’s algorithm. Creates a busmap assigning the nodes of a original network to the nodes of a clustered network considering the electrical distances based on Dijkstra’s shortest path.
- Parameters
network (pypsa.Network) – Container for all network components.
medoid_idx (pandas.Series) – Indices of k-medoids
busmap_kmedoid (pandas.Series) – Busmap based on k-medoids clustering
cpu_cores (string) – numbers of cores used during multiprocessing
- Returns
busmap – Mapping from bus ids to medoids ids
- Return type
pandas.Series
- etrago.cluster.spatial.kmedoids_dijkstra_clustering(etrago, buses, connections, weight, n_clusters)[source]¶
Applies a k-medoids clustering on the given network and calls the function to conduct a Dijkstra’s algorithm afterwards for the consideration of the network’s topology in the spatial clustering.
- Parameters
etrago (Etrago) – An instance of the Etrago class
buses (pandas.DataFrame) – DataFrame with information about the buses of the network.
connections (pandas.DataFrame) – DataFrame with information about the connections of the network (links or lines).
weight (pandas.Series) – Series with the weight for each bus.
n_clusters (int) – The number of clusters to create.
- Returns
Tuple containing
busmap (pandas.Series) – Series containing the mapping of buses to their resp. medoids
medoid_idx (pandas.Series) – Series containing the medoid indeces
etrago.tools package¶
etrago.tools.calc_results module¶
calc_results.py defines methods to calculate results of eTraGo
- etrago.tools.calc_results.calc_investment_cost(self)[source]¶
Function that calulates overall annualized investment costs.
- Returns
network_costs (float) – Investments in line expansion (AC+DC)
link_costs (float) – Investments in sectorcoupling link expansion
stor_costs (float) – Investments in storage and store expansion
etrago.tools.constraints module¶
Constraints.py includes additional constraints for eTraGo-optimizations
- etrago.tools.constraints.read_max_gas_generation(self)[source]¶
Return the values limiting the gas production in Germany
Read max_gas_generation_overtheyear from scenario.egon_scenario_parameters if the table is available in the database and return the dictionnary containing the values needed for the constraints to limit the gas production in Germany, depending of the scenario.
- Returns
arg
- Return type
- etrago.tools.constraints.add_ch4_constraints(self, network, snapshots)[source]¶
Add CH4 constraints for optimization with pyomo
Functionality that limits the dispatch of CH4 generators. In Germany, there is one limitation specific for biogas and one limitation specific for natural gas (natural gas only in eGon2035). Abroad, each generator has its own limitation contains in the column e_nom_max.
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization
- Return type
None.
- etrago.tools.constraints.add_ch4_constraints_nmp(self, network, snapshots)[source]¶
Add CH4 constraints for optimization without pyomo
Functionality that limits the dispatch of CH4 generators. In Germany, there is one limitation specific for biogas and one limitation specific for natural gas (natural gas only in eGon2035). Abroad, each generator has its own limitation contains in the column e_nom_max.
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization
- Return type
None.
- etrago.tools.constraints.snapshot_clustering_daily_bounds(self, network, snapshots)[source]¶
Bound the storage level to 0.5 max_level every 24th hour.
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps that will be constrained
- Return type
None
- etrago.tools.constraints.snapshot_clustering_daily_bounds_nmp(self, network, snapshots)[source]¶
Bound the storage level to 0.5 max_level every 24th hour.
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps that will be constrained
- Return type
None
- etrago.tools.constraints.snapshot_clustering_seasonal_storage(self, network, snapshots, simplified=False)[source]¶
Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.
- According to:
L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.
simplified (bool, optional) – A flag indicating whether to use a simplified version of the model that does not include intra-temporal constraints and variables.
- Return type
None
- etrago.tools.constraints.snapshot_clustering_seasonal_storage_hourly(self, network, snapshots)[source]¶
Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.
- According to:
L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.
- Return type
None
- etrago.tools.constraints.snapshot_clustering_seasonal_storage_nmp(self, n, sns, simplified=False)[source]¶
Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.
- According to:
L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018
- Parameters
n (
pypsa.Network
) – Overall container of PyPSAsns (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.
simplified (bool, optional) – A flag indicating whether to use a simplified version of the model that does not include intra-temporal constraints and variables.
- Return type
None
- etrago.tools.constraints.snapshot_clustering_seasonal_storage_hourly_nmp(self, n, sns)[source]¶
Depicts intertemporal dependencies of storage units and stores when using snapshot clustering to typical periods for temporal complexity reduction.
- According to:
L. Kotzur et al: ‘Time series aggregation for energy system design: Modeling seasonal storage’, 2018
- Parameters
n (
pypsa.Network
) – Overall container of PyPSAsns (list) – A list of datetime objects representing the timestamps of the snapshots to be clustered.
- Return type
None
- etrago.tools.constraints.split_dispatch_disaggregation_constraints(self, n, sns)[source]¶
Add constraints for state of charge of storage units and stores when separating the optimization into smaller subproblems while conducting thedispatch_disaggregation in temporally fully resolved network
The state of charge at the end of each slice is set to the value calculated in the optimization with the temporally reduced network to account to ensure compatibility and to reproduce saisonality
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization
- Return type
None.
- class etrago.tools.constraints.Constraints(args, conduct_dispatch_disaggregation)[source]¶
Bases:
object
Methods
functionality
(network, snapshots)Add constraints to pypsa-model using extra-functionality.
- functionality(network, snapshots)[source]¶
Add constraints to pypsa-model using extra-functionality. Serveral constraints can be choosen at once. Possible constraints are set and described in the above functions.
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAsnapshots (pandas.DatetimeIndex) – List of timesteps considered in the optimization
- etrago.tools.constraints.add_chp_constraints_nmp(n)[source]¶
Limits the dispatch of combined heat and power links based on T.Brown et. al : Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable European energy system, 2018
- Parameters
n (pypsa.Network) – Network container
- Return type
None.
- etrago.tools.constraints.add_chp_constraints(network, snapshots)[source]¶
Limits the dispatch of combined heat and power links based on T.Brown et. al : Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable European energy system, 2018
- Parameters
network (pypsa.Network) – Network container
snapshots (pandas.DataFrame) – Timesteps to optimize
- Return type
None.
etrago.tools.execute module¶
execute.py defines optimization and simulation methods for the etrago object.
- etrago.tools.execute.update_electrical_parameters(network, l_snom_pre, t_snom_pre)[source]¶
Update electrical parameters of active branch components considering s_nom of previous iteration.
- Parameters
network (pypsa.Network object) – Container for all network components.
l_snom_pre (pandas.Series) – s_nom of ac-lines in previous iteration.
t_snom_pre (pandas.Series) – s_nom of transformers in previous iteration.
- Return type
None.
- etrago.tools.execute.run_lopf(etrago, extra_functionality, method)[source]¶
Function that performs lopf with or without pyomo
- Parameters
etrago (etrago object) – eTraGo containing all network information and a PyPSA network.
extra_functionality (dict) – Define extra constranits.
method (dict) – Choose ‘n_iter’ and integer for fixed number of iterations or ‘threshold’ and derivation of objective in percent for variable number of iteration until the threshold of the objective function is reached.
- Return type
None.
- etrago.tools.execute.iterate_lopf(etrago, extra_functionality, method={'n_iter': 4, 'pyomo': True})[source]¶
Run optimization of lopf. If network extension is included, the specified number of iterations is calculated to consider reactance changes.
- Parameters
etrago (etrago object) – eTraGo containing all network information and a PyPSA network.
extra_functionality (dict) – Define extra constranits.
method (dict) – Choose ‘n_iter’ and integer for fixed number of iterations or ‘threshold’ and derivation of objective in percent for variable number of iteration until the threshold of the objective function is reached.
- etrago.tools.execute.lopf(self)[source]¶
Functions that runs lopf according to arguments.
- Return type
None.
- etrago.tools.execute.dispatch_disaggregation(self)[source]¶
Function running the tempral disaggregation meaning the optimization of dispatch in the temporally fully resolved network; therfore, the problem is reduced to smaller subproblems by slicing the whole considered time span while keeping inforation on the state of charge of storage units and stores to ensure compatibility and to reproduce saisonality.
- Return type
None.
- etrago.tools.execute.run_pf_post_lopf(self)[source]¶
Function that runs pf_post_lopf according to arguments.
- Return type
None.
- etrago.tools.execute.pf_post_lopf(etrago, calc_losses=False)[source]¶
Function that prepares and runs non-linar load flow using PyPSA pf. If crossborder lines are DC-links, pf is only applied on german network. Crossborder flows are still considerd due to the active behavior of links. To return a network containing the whole grid, the optimised solution of the foreign components can be added afterwards.
- Parameters
etrago (etrago object) – eTraGo containing all network information and a PyPSA network.
add_foreign_lopf (boolean) – Choose if foreign results of lopf should be added to the network when foreign lines are DC.
q_allocation (str) – Choose allocation of reactive power. Possible settings are listed in distribute_q function.
calc_losses (bolean) – Choose if line losses will be calculated.
- etrago.tools.execute.distribute_q(network, allocation='p_nom')[source]¶
Function that distributes reactive power at bus to all installed generators and storages.
- Parameters
network (pypsa.Network object) – Container for all network components.
allocation (str) – Choose key to distribute reactive power: ‘p_nom’ to dirstribute via p_nom ‘p’ to distribute via p_set.
- Return type
None.
etrago.tools.extendable module¶
Extendable.py defines function to set PyPSA components extendable.
- etrago.tools.extendable.extendable(self, grid_max_D=None, grid_max_abs_D={'110': {'circuits': 2, 'i': 1020, 'wires': 4}, '220': {'circuits': 4, 'i': 1020, 'wires': 4}, '380': {'circuits': 4, 'i': 1020, 'wires': 4}, 'dc': 0}, grid_max_foreign=4, grid_max_abs_foreign=None)[source]¶
Function that sets selected components extendable.
- Parameters
grid_max_D (int, optional) – Upper bounds for electrical grid expansion relative to existing capacity. The default is None.
grid_max_abs_D (dict, optional) – Absolute upper bounds for electrical grid expansion in Germany.
grid_max_foreign (int, optional) – Upper bounds for expansion of electrical foreign lines relative to the existing capacity. The default is 4.
grid_max_abs_foreign (dict, optional) – Absolute upper bounds for expansion of foreign electrical grid. The default is None.
- Return type
None.
- etrago.tools.extendable.snommax(i=1020, u=380, wires=4, circuits=4)[source]¶
Function to calculate limitation for capacity expansion.
- Parameters
- Returns
s_nom_max – Limitation for capacity expansion.
- Return type
- etrago.tools.extendable.line_max_abs(network, buses, line_max_abs={'110': {'circuits': 2, 'i': 1020, 'wires': 4}, '220': {'circuits': 4, 'i': 1020, 'wires': 4}, '380': {'circuits': 4, 'i': 1020, 'wires': 4}, 'dc': 0})[source]¶
Function to calculate limitation for capacity expansion of lines in network.
- Parameters
network (pypsa.Network object) – Container for all network components.
buses (pypsa.Network buses) – Considered buses in network.
line_max_abs (dict, optional) – Line parameters considered to calculate maximum capacity.
- Return type
None.
- etrago.tools.extendable.transformer_max_abs(network, buses)[source]¶
Function to calculate limitation for capacity expansion of transformers in network.
- Parameters
network (pypsa.Network object) – Container for all network components.
buses (pypsa.Network buses) – Considered buses in network.
- Return type
None.
- etrago.tools.extendable.extension_preselection(etrago, method, days=3)[source]¶
Function that preselects lines which are extendend in snapshots leading to overloading to reduce nubmer of extension variables.
- Parameters
network (pypsa.Network object) – Container for all network components.
args (dict) – Arguments set in appl.py.
method (str) – Choose method of selection: ‘extreme_situations’ for remarkable timsteps (e.g. minimal resiudual load) ‘snapshot_clustering’ for snapshot clustering with number of days
days (int) – Number of clustered days, only used when method = ‘snapshot_clustering’
- Returns
network – Container for all network components.
- Return type
pypsa.Network object
etrago.tools.io module¶
io.py
Input/output operations between powerflow schema in the oedb and PyPSA. Additionally oedb wrapper classes to instantiate PyPSA network objects.
- etrago.tools.io.carr_ormclass¶
Orm class name of table with carrier id to carrier name datasets
- Type
Notes
A configuration file connecting the chosen optimization method with components to be queried is needed for NetworkScenario class.
- class etrago.tools.io.ScenarioBase(engine, session, version=None)[source]¶
Bases:
object
Base class to address the dynamic provision of orm classes representing powerflow components from egoio
- Parameters
session (sqla.orm.session.Session) – Handles conversations with the database.
version (str) – Version number of data version control in grid schema of the oedb.
- class etrago.tools.io.NetworkScenario(engine, session, scn_name='Status Quo', start_snapshot=1, end_snapshot=20, temp_id=1, **kwargs)[source]¶
Bases:
ScenarioBase
Adapter class between oedb powerflow data and PyPSA. Provides the method build_network to generate a pypsa.Network.
- Parameters
Methods
build_network
([network])Core method to construct PyPSA Network object.
Construct a DateTimeIndex with the queried temporal resolution, start- and end_snapshot.
fetch_by_relname
(name)Construct DataFrame with component data from filtered table data.
series_fetch_by_relname
(network, name, ...)Construct DataFrame with component timeseries data from filtered table data.
id_to_source
- configure_timeindex()[source]¶
Construct a DateTimeIndex with the queried temporal resolution, start- and end_snapshot.
- fetch_by_relname(name)[source]¶
Construct DataFrame with component data from filtered table data.
- Parameters
name (str) – Component name.
- Returns
Component data.
- Return type
pd.DataFrame
- etrago.tools.io.clear_results_db(session)[source]¶
Used to clear the result tables in the OEDB. Caution! This deletes EVERY RESULT SET!
- etrago.tools.io.results_to_oedb(session, network, args, grid='hv', safe_results=False)[source]¶
Return results obtained from PyPSA to oedb
- Parameters
session –
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
args (dict) – Settings from appl.py
grid (str) – Choose voltage-level, currently only ‘hv’ implemented
safe_results (boolean) – If it is set to ‘True’ the result set will be saved to the versioned grid schema eventually apart from being saved to the model_draft by a SQL-script. ONLY set to True if you know what you are doing.
- etrago.tools.io.run_sql_script(conn, scriptname='results_md2grid.sql')[source]¶
This function runs .sql scripts in the folder ‘sql_scripts’
- etrago.tools.io.extension(self, **kwargs)[source]¶
Function that adds an additional network to the existing network container. The new network can include every PyPSA-component (e.g. buses, lines, links). To connect it to the existing network, transformers are needed.
All components and its timeseries of the additional scenario need to be inserted in the fitting ‘model_draft.ego_grid_pf_hv_extension_’ table. The scn_name in the tables have to be labled with ‘extension_’ + scn_name (e.g. ‘extension_nep2035’).
Until now, the tables include three additional scenarios: ‘nep2035_confirmed’: all new lines and needed transformers planed in the ‘Netzentwicklungsplan 2035’ (NEP2035) that have been confirmed by the Bundesnetzagentur (BNetzA)
‘nep2035_b2’: all new lines and needed transformers planned in the NEP 2035 in the scenario 2035 B2
‘BE_NO_NEP 2035’: DC-lines and transformers to connect the upcomming electrical-neighbours Belgium and Norway Generation, loads and its timeseries in Belgium and Norway for scenario ‘NEP 2035’
- Parameters
network – The existing network container (e.g. scenario ‘NEP 2035’)
session – session-data
overlay_scn_name – Name of the additional scenario (WITHOUT ‘extension_’)
start_snapshot –
end_snapshot – Simulation time
- Return type
Network container including existing and additional network
- etrago.tools.io.decommissioning(self, **kwargs)[source]¶
Function that removes components in a decommissioning-scenario from the existing network container. Currently, only lines can be decommissioned.
All components of the decommissioning scenario need to be inserted in the fitting ‘model_draft.ego_grid_pf_hv_extension_’ table. The scn_name in the tables have to be labled with ‘decommissioning_’ + scn_name (e.g. ‘decommissioning_nep2035’).
- Parameters
network – The existing network container (e.g. scenario ‘NEP 2035’)
session – session-data
overlay_scn_name – Name of the decommissioning scenario
- Return type
Network container including decommissioning
- etrago.tools.io.distance(x0, x1, y0, y1)[source]¶
Function that calculates the square of the distance between two points.
- Parameters
x0 – x - coordinate of point 0
x1 – x - coordinate of point 1
y0 – y - coordinate of point 0
y1 – y - coordinate of point 1
- Returns
distance – square of distance
- Return type
etrago.tools.network module¶
etrago.tools.plot module¶
Plot.py defines functions necessary to plot results of eTraGo.
- etrago.tools.plot.set_epsg_network(network)[source]¶
Change EPSG from 4326 to 3857. Needed when using osm-background.
- Parameters
network (PyPSA network container) –
- etrago.tools.plot.plot_osm(x, y, zoom, alpha=0.4)[source]¶
Plots openstreetmap as background of network-plots
- etrago.tools.plot.coloring()[source]¶
Return a dictionary with a color assign to each kind of carrier used in etrago.network. This is used for plotting porpuses.
- Returns
colors – Color for each kind of carrier.
- Return type
- etrago.tools.plot.plot_line_loading_diff(networkA, networkB, timestep=0, osm=False)[source]¶
Plot difference in line loading between two networks (with and without switches) as color on lines
Positive values mean that line loading with switches is bigger than without Plot switches as small dots
- Parameters
networkA (PyPSA network container) – Holds topology of grid with switches including results from powerflow analysis
networkB (PyPSA network container) – Holds topology of grid without switches including results from powerflow analysis
filename (str) – Specify filename If not given, figure will be show directly
timestep (int) – timestep to show, default is 0
osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –
If not False, osm is set as background with the following settings as dict:
’x’: array of two floats, x axis boundaries (lat)
’y’: array of two floats, y axis boundaries (long)
’zoom’ : resolution of osm
- etrago.tools.plot.network_expansion_diff(networkA, networkB, filename=None, boundaries=[], osm=False)[source]¶
Plot relative network expansion derivation of AC- and DC-lines.
- Parameters
networkA (PyPSA network container) – Holds topology of grid including results from powerflow analysis
networkB (PyPSA network container) – Holds topology of grid including results from powerflow analysis
filename (str or None) – Save figure in this direction
boundaries (array) – Set boundaries of heatmap axis
osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –
If not False, osm is set as background with the following settings as dict:
’x’: array of two floats, x axis boundaries (lat)
’y’: array of two floats, y axis boundaries (long)
’zoom’ : resolution of osm
- etrago.tools.plot.plot_residual_load(network)[source]¶
Plots residual load summed of all exisiting buses.
- Parameters
network (PyPSA network containter) –
- Return type
Plot
- etrago.tools.plot.plot_stacked_gen(network, bus=None, resolution='GW', filename=None)[source]¶
Plot stacked sum of generation grouped by carrier type
- Parameters
network (PyPSA network container) –
bus (string) – Plot all generators at one specific bus. If none, sum is calulated for all buses
resolution (string) – Unit for y-axis. Can be either GW/MW/KW
- Return type
Plot
- etrago.tools.plot.plot_gen_diff(networkA, networkB, leave_out_carriers=['geothermal', 'oil', 'other_non_renewable', 'reservoir', 'waste'])[source]¶
Plot difference in generation between two networks grouped by carrier type
- Parameters
networkA (PyPSA network container with switches) –
networkB (PyPSA network container without switches) –
leave_out_carriers – list of carriers to leave out (default to all small carriers)
- Return type
Plot
- etrago.tools.plot.plot_voltage(network, boundaries=[], osm=False)[source]¶
Plot voltage at buses as hexbin
- Parameters
network (PyPSA network container) –
boundaries (list of 2 values, setting the lower and upper bound of colorbar) –
osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –
If not False, osm is set as background with the following settings as dict:
’x’: array of two floats, x axis boundaries (lat)
’y’: array of two floats, y axis boundaries (long)
’zoom’ : resolution of osm
- Return type
Plot
- etrago.tools.plot.curtailment(network, carrier='solar', filename=None)[source]¶
Plot curtailment of selected carrier
- etrago.tools.plot.calc_dispatch_per_carrier(network, timesteps)[source]¶
Function that calculates dispatch per carrier in given timesteps
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
timesteps (array) – Timesteps considered in dispatch calculation
- Returns
dist – dispatch per carrier
- Return type
pandas.Series
- etrago.tools.plot.calc_storage_expansion_per_bus(network)[source]¶
Function that calculates storage expansion per bus and technology
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
- Returns
dist – storage expansion per bus and technology
- Return type
pandas.Series
- etrago.tools.plot.gen_dist_diff(networkA, networkB, techs=None, snapshot=0, n_cols=3, gen_size=0.2, filename=None, buscmap=<matplotlib.colors.LinearSegmentedColormap object>)[source]¶
Difference in generation distribution Green/Yellow/Red colors mean that the generation at a location is bigger with switches than without Blue colors mean that the generation at a location is smaller with switches than without
- Parameters
networkA (PyPSA network container) – Holds topology of grid with switches including results from powerflow analysis
networkB (PyPSA network container) – Holds topology of grid without switches including results from powerflow analysis
techs (dict) – type of technologies which shall be plotted
snapshot (int) – snapshot
n_cols (int) – number of columns of the plot
gen_size (num) – size of generation bubbles at the buses
filename (str) – Specify filename If not given, figure will be show directly
- Return type
None.
- etrago.tools.plot.nodal_gen_dispatch(network, networkB=None, techs=['wind_onshore', 'solar'], item='energy', direction=None, scaling=1, filename=None, osm=False)[source]¶
Plot nodal dispatch or capacity. If networkB is given, difference in dispatch is plotted.
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
networkB (PyPSA network container) – If given and item is ‘energy’, difference in dispatch between network and networkB is plotted. If item is ‘capacity’, networkB is ignored. default None
techs (None or list,) – Techs to plot. If None, all techs are plotted. default [‘wind_onshore’, ‘solar’]
item (str) – Specifies the plotted item. Options are ‘energy’ and ‘capacity’. default ‘energy’
direction (str) – Only considered if networkB is given and item is ‘energy’. Specifies the direction of change in dispatch between network and networkB. If ‘positive’, generation per tech which is higher in network than in networkB is plotted. If ‘negative’, generation per tech whcih is lower in network than in networkB is plotted. If ‘absolute’, total change per node is plotted. Green nodes have higher dispatch in network than in networkB. Red nodes have lower dispatch in network than in networkB. default None
scaling (int) – Scaling to change plot sizes. default 1
filename (path to folder) –
osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –
If not False, osm is set as background with the following settings as dict:
’x’: array of two floats, x axis boundaries (lat)
’y’: array of two floats, y axis boundaries (long)
’zoom’ : resolution of osm
- Return type
None.
- etrago.tools.plot.nodal_production_balance(network, timesteps, scaling=1e-05)[source]¶
Function that calculates residual load per node in given timesteps
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
timesteps (array) – timesteps considered in calculation
scaling (float, optional) – Scaling factor for bus size. The default is 0.00001.
- Returns
bus_sizes (pandas.Series) – scaled residual load per node
bus_colors (pandas.Series) – ‘green’ for producer and ‘red’ for consumer
- etrago.tools.plot.storage_p_soc(network, mean='1H', filename=None)[source]¶
Plots the dispatch and state of charge (SOC) of extendable storages.
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
mean (str) – Defines over how many snapshots the p and soc values will averaged.
filename (path to folder) –
- Return type
None.
- etrago.tools.plot.storage_soc_sorted(network, filename=None)[source]¶
Plots the soc (state-pf-charge) of extendable storages
- Parameters
network (PyPSA network container) – Holds topology of grid including results from powerflow analysis
filename (path to folder) –
- Return type
None.
- etrago.tools.plot.mul_weighting(network, timeseries)[source]¶
Returns timeseries considering snapshot_weighting
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
timeseries (pd.Series) – timeseries not considering snapshot_weighting
- Returns
timeseries considering snapshot_weightings
- Return type
pd.Series
- etrago.tools.plot.plotting_colors(network)[source]¶
Add color values to network.carriers
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
- Return type
None.
- etrago.tools.plot.calc_network_expansion(network, method='abs', ext_min=0.1)[source]¶
Calculates absolute or relative expansion per AC- and DC-line
- Parameters
- Returns
all_network (:class:`pypsa.Network) – Whole network including not extended lines
extension_lines (pandas.Series) – AC-line expansion
extension_links (pandas.Series) – DC-line expansion
- etrago.tools.plot.plot_background_grid(network, ax)[source]¶
Plots grid topology in background of other network.plot
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
ax (matplotlib.axes._subplots.AxesSubplot) – axes of plot
- Return type
None.
- etrago.tools.plot.demand_side_management(self, buses, snapshots, agg='5h', used=False)[source]¶
Calculate shifting potential of demand side management
- Parameters
buses (array) – List of electricity buses.
snapshots (array) – List of snapshots.
agg (str, optional) – Temporal resolution. The default is ‘5h’.
used (boolean, optional) – State if usage should be included in the results. The default is False.
- Returns
df – Shifting potential (and usage) of power (MW) and energy (MWh)
- Return type
pandas.DataFrame
- etrago.tools.plot.bev_flexibility_potential(self, buses, snapshots, agg='5h', used=False)[source]¶
Calculate shifting potential of electric vehicles
- Parameters
buses (array) – List of electricity buses.
snapshots (array) – List of snapshots.
agg (str, optional) – Temporal resolution. The default is ‘5h’.
used (boolean, optional) – State if usage should be included in the results. The default is False.
- Returns
df – Shifting potential (and usage) of power (MW) and energy (MWh)
- Return type
pandas.DataFrame
- etrago.tools.plot.heat_stores(self, buses, snapshots, agg='5h', used=False)[source]¶
Calculate shifting potential (and usage) of heat stores
- Parameters
buses (array) – List of electricity buses.
snapshots (array) – List of snapshots.
agg (str, optional) – Temporal resolution. The default is ‘5h’.
used (boolean, optional) – State if usage should be included in the results. The default is False.
- Returns
df – Shifting potential (and usage) of power (MW) and energy (MWh)
- Return type
pandas.DataFrame
- etrago.tools.plot.hydrogen_stores(self, buses, snapshots, agg='5h', used=False)[source]¶
Calculate shifting potential (and usage) of heat stores
- Parameters
buses (array) – List of electricity buses.
snapshots (array) – List of snapshots.
agg (str, optional) – Temporal resolution. The default is ‘5h’.
used (boolean, optional) – State if usage should be included in the results. The default is False.
- Returns
df – Shifting potential (and usage) of power (MW) and energy (MWh)
- Return type
pandas.DataFrame
- etrago.tools.plot.flexibility_usage(self, flexibility, agg='5h', snapshots=[], buses=[], pre_path=None)[source]¶
Plots temporal distribution of potential and usage for flexibilities
- Parameters
flexibility (str) – Name of flexibility option.
agg (str, optional) – Temporal resolution. The default is “5h”.
snapshots (list, optional) – Considered snapshots, if empty all are considered. The default is [].
buses (list, optional) – Considered components at AC buses, if empty all are considered. The default is [].
pre_path (str, optional) – State of and where you want to store the figure. The default is None.
- Return type
None.
- etrago.tools.plot.plot_carrier(network, carrier_links=['AC'], carrier_buses=['AC'])[source]¶
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
carrier_links (list) – List of links to be plotted. The default is [“AC”].
carrier_buses (list) – List of buses to be plotted. The default is [“AC”].
cartopy (bool, optional) – Provide data about the availability of Cartopy. The default is True.
- Return type
None.
- etrago.tools.plot.plot_grid(self, line_colors, bus_sizes=0.001, bus_colors='grey', timesteps=range(0, 2), osm=False, boundaries=None, filename=None, disaggregated=False, ext_min=0.1, ext_width=False)[source]¶
Function that plots etrago.network and results for lines and buses
- Parameters
line_colors (str) –
Set static line color or attribute to plot e.g. ‘expansion_abs’ Current options:
’line_loading’: mean line loading in p.u. in selected timesteps
’v_nom’: nominal voltage of lines
’expansion_abs’: absolute network expansion in MVA
’expansion_rel’: network expansion in p.u. of existing capacity
’q_flow_max’: maximal reactive flows
bus_sizes (float, optional) – Size of buses. The default is 0.001.
bus_colors (str, optional) –
Set static bus color or attribute to plot. The default is ‘grey’. Current options:
’nodal_production_balance’: net producer/consumer in selected timeteps
’storage_expansion’: storage expansion per bus and technology
’storage_distribution’: installed storage units per bus
’gen_dist’: dispatch per carrier in selected timesteps
timesteps (array, optional) – Timesteps consideredd in time depended plots. The default is range(2).
osm (bool or dict, e.g. {'x': [1,20], 'y': [47, 56], 'zoom' : 6}) –
If not False, osm is set as background with the following settings as dict:
’x’: array of two floats, x axis boundaries (lat)
’y’: array of two floats, y axis boundaries (long)
’zoom’ : resolution of osm. The default is False.
boundaries (array) – Set fixed boundaries of heatmap axis. The default is None.
filename (str or None) – Save figure in this direction. The default is None.
disaggregated (bool, optional) – Choose if disaggregated network is shown. The default is False.
ext_min (float) – Choose minimum relative line extension shown in plot in p.u..
ext_width (float or bool) – Choose if line_width respects line extension. Turn off with ‘False’ or set linear factor to decremise extension line_width. The default is False.
- Return type
None.
- etrago.tools.plot.plot_clusters(self, carrier='AC', save_path=False, transmission_lines=False, gas_pipelines=False)[source]¶
- Parameters
carrier (str, optional) – This variable set the carrier of the buses that will be plotted. The default is “AC”.
cartopy (bool, optional) – Set it to True when cartopy is installed and the map is supposed to include country’s boundaries and bodies of water
save_path (bool, optional) – Path to save the generated plot. The default is False.
transmission_lines (bool, optional) – The default is False. Define if the original transmission lines are plotted or not.
gas_pipelines (bool, optional) – The default is False. Define if the original gas pipelines are plotted or not.
- Return type
None.
- etrago.tools.plot.plot_gas_generation(self, t_resolution='20H', save_path=False)[source]¶
Plots timeseries data for gas generation
- etrago.tools.plot.plot_gas_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]¶
Plots timeseries data for gas loads (and generation)
- Parameters
self (:class:`Etrago) – Overall container of Etrago
t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots
stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total gas generation will then also be plotted to check for matching demand and generation.
save_path (bool, optional) – Path to save the generated plot. The default is False.
- Return type
None.
- etrago.tools.plot.plot_h2_generation(self, t_resolution='20H', save_path=False)[source]¶
Plots timeseries data for H2 generation
- etrago.tools.plot.plot_h2_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]¶
Plots timeseries data for H2 loads (and generation)
- Parameters
self (:class:`Etrago) – Overall container of Etrago
t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots
stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total H2 generation will then also be plotted to check for matching demand and generation.
save_path (bool, optional) – Path to save the generated plot. The default is False.
- Return type
None.
- etrago.tools.plot.plot_heat_loads(self, t_resolution='20H', save_path=False)[source]¶
Plots timeseries data for heat loads
- etrago.tools.plot.plot_heat_summary(self, t_resolution='20H', stacked=True, save_path=False)[source]¶
Plots timeseries data for heat generation (and demand)
- Parameters
self (:class:`Etrago) – Overall container of Etrago
t_resolution (str, optional) – sets the resampling rate of timeseries data to allow for smoother line plots
stacked (bool, optional) – If True all TS data will be shown as stacked area plot. Total heat demand will then also be plotted to check for matching generation and demand.
save_path (bool, optional) – Path to save the generated plot. The default is False.
- Return type
None.
etrago.tools.utilities module¶
Utilities.py includes a wide range of useful functions.
- etrago.tools.utilities.filter_links_by_carrier(self, carrier, like=True)[source]¶
- Parameters
- Returns
df – Dataframe that contains just links with carriers of the types given in the argument carrier.
- Return type
pandas.DataFrame object
- etrago.tools.utilities.buses_of_vlvl(network, voltage_level)[source]¶
Get bus-ids of given voltage level(s).
- etrago.tools.utilities.buses_grid_linked(network, voltage_level)[source]¶
Get bus-ids of a given voltage level connected to the grid.
- etrago.tools.utilities.geolocation_buses(self)[source]¶
If geopandas is installed: Use geometries of buses x/y(lon/lat) and polygons of countries from RenpassGisParameterRegion in order to locate the buses
Else: Use coordinats of buses to locate foreign buses, which is less accurate.
TODO: Why not alway use geopandas??
- Parameters
etrago (
etrago.Etrago
) – Transmission grid object
- etrago.tools.utilities.buses_by_country(self)[source]¶
Find buses of foreign countries using coordinates and return them as Pandas Series
- Parameters
self (Etrago object) – Overall container of PyPSA
- Return type
None
- etrago.tools.utilities.clip_foreign(network)[source]¶
Delete all components and timelines located outside of Germany. If applied after optimization, transborder flows divided by country of origin are added as network.foreign_trade.
- etrago.tools.utilities.foreign_links(self)[source]¶
Change transmission technology of foreign lines from AC to DC (links).
- etrago.tools.utilities.set_q_national_loads(self, cos_phi)[source]¶
Set q component of national loads based on the p component and cos_phi
- Parameters
network (
pypsa.Network
) – Overall container of PyPSAcos_phi (float) – Choose ration of active and reactive power of foreign loads
- Returns
network – Overall container of PyPSA
- Return type
pypsa.Network
- etrago.tools.utilities.set_q_foreign_loads(self, cos_phi)[source]¶
Set reative power timeseries of loads in neighbouring countries
- etrago.tools.utilities.connected_grid_lines(network, busids)[source]¶
Get grid lines connected to given buses.
- etrago.tools.utilities.connected_transformer(network, busids)[source]¶
Get transformer connected to given buses.
- etrago.tools.utilities.load_shedding(self, temporal_disaggregation=False, **kwargs)[source]¶
Implement load shedding in existing network to identify feasibility problems
- etrago.tools.utilities.data_manipulation_sh(network)[source]¶
Adds missing components to run calculations with SH scenarios.
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
- Return type
None
- etrago.tools.utilities.export_to_csv(self, path)[source]¶
Write calculation results to csv-files in path.
- etrago.tools.utilities.loading_minimization(network, snapshots)[source]¶
Minimizes the sum of the products of each element in the passive_branches of the model.
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
snapshots ('pandas.core.indexes.datetimes.DatetimeIndex') – snapshots to perform the minimization
- Return type
None
- etrago.tools.utilities.agg_series_lines(l0, network)[source]¶
Given a pandas DataFrame l0 containing information about lines in a network and a network object, aggregates the data in l0 for all its attributes. Returns a pandas Series containing the aggregated data.
- Parameters
l0 (pandas.DataFrame) – contain information about lines in a network.
network (:class:`pypsa.Network) – Overall container of PyPSA
- Returns
A pandas Series containing aggregated data for the lines in the network.
- Return type
pandas.Series
- etrago.tools.utilities.group_parallel_lines(network)[source]¶
Function that groups parallel lines of the same voltage level to one line component representing all parallel lines
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
- Return type
None.
- etrago.tools.utilities.delete_dispensable_ac_buses(etrago)[source]¶
Function that identifies and delete AC buses without links, transformers, generators, loads, stores or storage_units, which also are connected to just one or two other buses
- Parameters
etrago (etrago object) –
- Return type
None.
- etrago.tools.utilities.set_line_costs(self, cost110=230, cost220=290, cost380=85, costDC=375)[source]¶
Set capital costs for extendable lines in respect to PyPSA [€/MVA]
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
args (dict) – containing settings from appl.py
cost110 – capital costs per km for 110kV lines and cables default: 230€/MVA/km, source: costs for extra circuit in dena Verteilnetzstudie, p. 146)
cost220 – capital costs per km for 220kV lines and cables default: 280€/MVA/km, source: costs for extra circuit in NEP 2025, capactity from most used 220 kV lines in model
cost380 – capital costs per km for 380kV lines and cables default: 85€/MVA/km, source: costs for extra circuit in NEP 2025, capactity from most used 380 kV lines in NEP
costDC – capital costs per km for DC-lines default: 375€/MVA/km, source: costs for DC transmission line in NEP 2035
- etrago.tools.utilities.set_trafo_costs(self, cost110_220=7500, cost110_380=17333, cost220_380=14166)[source]¶
Set capital costs for extendable transformers in respect to PyPSA [€/MVA]
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
cost110_220 – capital costs for 110/220kV transformer default: 7500€/MVA, source: costs for extra trafo in dena Verteilnetzstudie, p. 146; S of trafo used in osmTGmod
cost110_380 – capital costs for 110/380kV transformer default: 17333€/MVA, source: NEP 2025
cost220_380 – capital costs for 220/380kV transformer default: 14166€/MVA, source: NEP 2025
- etrago.tools.utilities.add_missing_components(self)[source]¶
Add a missing transformer at Heizkraftwerk Nord in Munich and a missing transformer in Stuttgart.
- etrago.tools.utilities.convert_capital_costs(self)[source]¶
Convert capital_costs to fit to considered timesteps
- Parameters
etrago (:class:`etrago.Etrago) – Transmission grid object
- etrago.tools.utilities.find_snapshots(network, carrier, maximum=True, minimum=True, n=3)[source]¶
Function that returns snapshots with maximum and/or minimum feed-in of selected carrier.
- Parameters
- Returns
calc_snapshots – List containing snapshots
- Return type
‘pandas.core.indexes.datetimes.DatetimeIndex’
- etrago.tools.utilities.ramp_limits(network)[source]¶
Add ramping constraints to thermal power plants.
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
- etrago.tools.utilities.get_args_setting(self, jsonpath='scenario_setting.json')[source]¶
Get and open json file with scenaio settings of eTraGo
args
. The settings incluedes all eTraGo specific settings of arguments and parameters for a reproducible calculation.
- etrago.tools.utilities.merge_dicts(dict1, dict2)[source]¶
Return a new dictionary by merging two dictionaries recursively.
- etrago.tools.utilities.get_clustering_data(self, path)[source]¶
Import the final busmap and the initial buses, lines and links
- Parameters
path (str) – Name of folder from which to import CSVs of network data.
- Return type
None
- etrago.tools.utilities.set_random_noise(self, sigma=0.01)[source]¶
Sets random noise to marginal cost of each generator.
- etrago.tools.utilities.set_line_country_tags(network)[source]¶
Set country tag for AC- and DC-lines.
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
- etrago.tools.utilities.crossborder_capacity_tyndp2020()[source]¶
This function downloads and extracts a scenario datafile for the TYNDP 2020 (Ten-Year Network Development Plan), reads a specific sheet from the file, filters it based on certain criteria, and then calculates the minimum cross-border capacities for a list of European countries. The minimum cross-border capacity is the minimum of the export and import capacities between two countries.
- Returns
Dictionary with cossborder capacities.
- Return type
- etrago.tools.utilities.crossborder_capacity(self)[source]¶
Adjust interconnector capacties.
- Parameters
network (:class:`pypsa.Network) – Overall container of PyPSA
method (string) – Method of correction. Options are ‘ntc_acer’ and ‘thermal_acer’. ‘ntc_acer’ corrects all capacities according to values published by the ACER in 2016. ‘thermal_acer’ corrects certain capacities where our dataset most likely overestimates the thermal capacity.
- etrago.tools.utilities.set_branch_capacity(etrago)[source]¶
Set branch capacity factor of lines and transformers, different factors for HV (110kV) and eHV (220kV, 380kV).
- Parameters
etrago (:class:`etrago.Etrago) – Transmission grid object
- etrago.tools.utilities.check_args(etrago)[source]¶
Function that checks the consistency of etragos input parameters.
- Parameters
etrago (:class:`etrago.Etrago) – Overall container of eTraGo
- Return type
None.
- etrago.tools.utilities.drop_sectors(self, drop_carriers)[source]¶
Manually drop secors from network. Makes sure the network can be calculated without the dropped sectors.
- Parameters
drop_carriers (array) – List of sectors that will be dropped. e.g. [‘dsm’, ‘CH4’, ‘H2_saltcavern’, ‘H2_grid’, ‘central_heat’, ‘rural_heat’, ‘central_heat_store’, ‘rural_heat_store’, ‘Li ion’] means everything but AC
- Return type
None.
- etrago.tools.utilities.update_busmap(self, new_busmap)[source]¶
Update busmap after any clustering process
- Parameters
new_busmap (dictionary) – busmap used to clusted the network.
- Return type
None.
- etrago.tools.utilities.adjust_CH4_gen_carriers(self)[source]¶
Precise the carrier for the generators with CH4 carrier
For the eGon2035 scenario, the generators with carrier CH4 represent the prodution od biogas and methan. In the data model, these two differents types are differenciated only by the marginal cost of the generator. This function introduces a carrier distion (CH4_biogas and CH4_NG) in order to avoid the clustering of these two types of generator together and facilitate the contraint applying differently to each of them.
- etrago.tools.utilities.residual_load(network, sector='electricity')[source]¶
Calculate the residual load for the specified sector.
In case of the electricity sector residual load is calculated using all AC loads and all renewable generators with carriers ‘wind_onshore’, ‘wind_offshore’, ‘solar’, ‘solar_rooftop’, ‘biomass’, ‘run_of_river’, and ‘reservoir’.
In case of the central heat sector residual load is calculated using all central heat loads and all renewable generators with carriers ‘solar_thermal_collector’ and ‘geo_thermal’.
- Parameters
network (PyPSA network) – Network to retrieve load and generation time series from, needed to determine residual load.
sector (str) – Sector to determine residual load for. Possible options are ‘electricity’ and ‘central_heat’. Default: ‘electricity’.
- Returns
Dataframe with residual load for each bus in the network. Columns of the dataframe contain the corresponding bus name and index of the dataframe is a datetime index with the corresponding time step.
- Return type
pd.DataFrame