Guide for updating to Pastas 1.0 (Arrabiata) and dealing with older pastastores.

[dbrakenhoff]

Pastas 1.0 (Arrabiata) is now available (Feb 6th, 2023), and this version introduces some breaking changes to loading models created with older pastas versions. This guide is for people who want to upgrade pastas to the latest version but who have databases created with older versions of pastas.

Updating pastastore

First of all, update pastastore to version 1.0. This newest version of Pastastore supports all recent versions of pastas (and the last three versions, 0.22.0, 0.23.1 and 1.0.0 are actively tested), so you can do this without having to worry about anything breaking.

pip install pastastore --upgrade

Check your pastas and pas-file versions

1. Check your current pastas version.

import pastas as ps
ps.show_versions()

2. Check the pastas version of a stored time series model.

import pastastore as pst

conn = pst.PasConnector("my_db", path="./pastas_database")  # initialize connector
pstore = pst.PastaStore("my_db", conn) 

ml = pstore.models.random()  # get a random model
print(ml["file_info"])

My pas-file version is 0.22 or lower.

There are two options to update to pastas 1.0. The first option is the easiest and is recommended, but does require you to rerun the scripts that built the models. If this is not an option, then go to option 2.

Option 1: re-run scripts to build models (recommended)

1. Update pastas to version 1.0

pip install pastas --upgrade

2. Re-run the script to build the models. Prior to that, is probably advisable to re-run the scripts to load time series into the pastastore as well. You will probably run into some errors related to pastas 1.0 that will need to be solved. The following list will give you some ideas of the things that can go wrong, but the error messages you get should also be helpful in solving any issues you run into.

   • Time series are checked more rigorously, both by pastas and pastastore. Time series must have DateTime indexes, that must be monotonically increasing and cannot contain duplicates. Stresses time series are not allowed to contain NaNs. The functions pastas.validate_oseries(oseries) and pastas.validate_stresses(stress) should provide helpful info on how to pre-process your time series so that they can be used to create pastas models. These checks are also used by pastastore, so adding time series to your database will tell you if your time series need to be pre-processed.
   • Response functions and solvers are now passed as instances (and not classes), e.g. rfunc=ps.Exponential(), and solver=ps.LmfitSolve() (note the brackets).
   • The pastas.TimeSeries object is now an internal pastas object and should not be used. Pass pandas.Series to stressmodels instead.
   • The pastas.read module is deprecated, use hydropandas instead.

3. Once you've added the time series to the pastastore and re-run the model building script, you're good to go!

Option 2: let pastas update the pas-files

1. Update pastas to version 0.23.1 (if you're not already on that version)

pip install pastas==0.23.1 --upgrade

2. Update stored pas files. This will update the stored pas files so that they can be loaded by pastas 1.0

# given a pastastore 'pstore' 
for ml in pstore.models:
    pstore.add_model(ml, overwrite=True)

3. Update pastas to version 1.0

pip install pastas --upgrade

4. Update stored time series (if necessary). Please note that in pastas version 1.0 there is more rigorous checking of time series. If the stored time series in the pastastore do need meet the criteria for time series, the models cannot be loaded. In this case, the stored time series will have to be processed and updated first.
   • Time series are checked more rigorously, both by pastas and pastastore. Time series must have DateTime indexes, that must be monotonically increasing and cannot contain duplicates. Stresses time series are not allowed to contain NaNs. The functions pastas.validate_oseries(oseries) and pastas.validate_stresses(stress) should provide helpful info on how to pre-process your time series so that they can be used to create pastas models. These checks are also used by pastastore, so adding time series to your database will tell you if your time series need to be pre-processed.

# given some pastastore 'pstore'

o = pstore.get_oseries("my_oseries")  # load series
pstore.add_oseries(o, "my_oseries", overwrite=True)  # try to overwrite stored oseries

stress = pstore.get_stresses("my_stress")
pstore.add_stress(stress, "my_stress", overwrite=True)  # try to overwrite stored stress

# -> Deal with the resulting errors until you are able to overwrite the stored time series. 
#    The errors should be informative and help you figure out how to process the time series.

5. Update stored pas files. This will update the stored pas files to version 1.0

# given a pastastore 'pstore' 
for ml in pstore.models:
    pstore.add_model(ml, overwrite=True)

My pas-file version is 0.23.X

These files can be loaded by pastas 1.0, so the steps are simple.

1. Update pastas to version 1.0

pip install pastas --upgrade

2. Update stored pas files.

# given a pastastore 'pstore' 
for ml in pstore.models:
    pstore.add_model(ml, overwrite=True)