Migrate an instance

Note

Currently only recommended on the enterprise plan. Reach out if you are interested!

lamin’s migrations are a convenient wrapper around alembic. All scripts generated by lamin are fully compatible with alembic. The main value added by lamin is a CI-tested process for a migration.

This page documents how to create a tested migration using the CI-guided workflow for postgres-based instances.

Ensure that you have installed the schema module in editable mode for example through

pip install -e .

Further, ensure that you have loaded a Lamin instance.

1. Create a new branch (e.g. migr) in your repository: Update an ORM, e.g., by renaming a column.

   Example: Rename a column.

   Let’s try to rename the version column to v in the Transform ORM of lnschema_core.

   In the lnschema_core/_core.py Transform ORM, modify line:

   version: str = Field(default="1", primary_key=True)
   to:
   v: str = Field(default="1", primary_key=True)
   

2. On the command line (at the root of the repository), run lamin migrate generate to generate a migration script under {package_name}/migrations/versions/.

   Example

   Migration script location: The script will be named {date}-{revision}-vx_x_x.py.

   -- lnschema_core
   |-- dev
   |-- migrations
   |   |-- versions
   |       |-- 2023-02-16-dd2b4a9499f2-vx_x_x.py
   |-- __init__.py
   |-- _core.py
   

   Example of an empty migration script.

   """vX.X.X."""
   from alembic import op
   import sqlalchemy as sa  # noqa
   import sqlmodel as sqm # noqa
   
   revision = 'dd2b4a9499f2'
   down_revision = '8280855a5064'
   
   
   def upgrade() -> None:
       pass
   
   
   def downgrade() -> None:
       pass
   

   Updated _migration id in {package_name}/__init__.py.

   _migration = "dd2b4a9499f2"
   

3. Commit all changes, create a pull request from the migr branch, and inspect the CI step Build.

   Example: Bottom of failed CI output.

   op.alter_column("notebook", column_name="v", new_column_name="version", schema="core")
   

4. In case CI failed, copy the suggested changes into the update() function in the {date}-{revision}-vx_x_x.py file. Commit & push changes: Now CI should pass! 🎉

   Example: Modified migration script.

   def upgrade() -> None:
       op.alter_column("notebook", column_name="v", new_column_name="version", schema="core")
   

5. Merge PR and make a release with a new __version__.

   Example: Make a new release, e.g., “1.0.0”.

   1. Modify __version__ in the __init__.py file: __version__ = "1.0.0"

   2. Modify the migration script file name to: 2023-02-16-dd2b4a9499f2-v1_0_0.py

   3. Modify the first line of the migration script file from """vX.X.X.""" to """v1.0.0."""

The next time you load your instance, you will be asked to deploy the migration!