Re: Importing hourly resolved data #question

Daniel Huppmann

Hi Patrick,

Thanks for reaching out!

My hunch is that the filtering does not work as expected because of a mismatch between integer and strings - usually, pyam assumes that all index columns are strings. Try filtering by the following:
subannual=list(map(str, range(1, 186+1)))

One comment, though: pyam will sort the index and you will therefore see an order like [1, 10, 100, 11, …] in plots. A possible solution is to rename the subannual timesteps to `h   1` or `h0001`.

The filtering could then be made using
subannual=[f'h{i:4}' for i in range(1, 186+1)]

Another alternative is to use a derivative the datetime-convention but without the year info, so use a format 01-01 00:00+01:00. See this description of the openENTRANCE nomenclature for details. The nomenclature package has a feature swap_time_for_subannual to translate between datetime format and two separate year/subannual columns.


Am 23.10.2020 um 10:37 schrieb patrick.juergens@...:

Dear pyam-Group,

I'm having a question about how to best import hourly resolved timeseries, e.g. for energy consumption of different sectors. My goal is to read the data, be able to aggregate each timeseries for one year and be able to filter data for specific hours, e.g. for one week (hours 1 ... 168).

By now I imported the data following the style in the Tutorial "Aggregating subannual timeseries data", i.e. I used a column "subannual" with values 1 ... 8760 for each hour in one year. Aggregating the timeseries to a year value works perfectly as expected with aggregate_time(). However, the filter()-method doesn't support filtering 'subannual'-data like df.filter('subannual'=range(1, 168+1)). In the documentary however I saw that filtering by datetime is possible.

How would I import the timeseries in a way that filtering is possible? Would I need to put the hour-information in a time-column instead of a year and subannual-column? Does aggregate_time then still work?
Otherwise, could you add the possibility to filter by the column 'subannual'?

I hope to hear from you and I would be very pleased if you add a small tutorial about importing hourly resolved timeseries on your Website.

Patrick Jürgens

Patrick Jürgens, M.Sc.
Energy Systems and Energy Economics
Energy System Analysis
Fraunhofer Institute for Solar Energy Systems ISE
Heidenhofstraße 2, 79110 Freiburg, Germany
Phone +49 761 4588-2292

Social Media:
Facebook | Twitter | LinkedIn | YouTube

If you would like to remain informed and up-to-date on the latest news,
subscribe to our newsletter and receive all the relevant information about our latest research activities. -> Register here

Join to automatically receive all group messages.