=======================

Database schema changes

=======================

Kallithea uses Alembic for :ref:`database migrations <upgrade_db>`

(upgrades and downgrades).

If you are developing a Kallithea feature that requires database schema

changes, you should make a matching Alembic database migration script:

1. :ref:`Create a Kallithea configuration and database <setup>` for testing

   the migration script, or use existing ``development.ini`` setup.

   Ensure that this database is up to date with the latest database

   schema *before* the changes you're currently developing. (Do not

   create the database while your new schema changes are applied.)

2. Create a separate throwaway configuration for iterating on the actual

   database changes::

    paster make-config Kallithea temp.ini

   Edit the file to change database settings. SQLite is typically fine,

   but make sure to change the path to e.g. ``temp.db``, to avoid

   clobbering any existing database file.

3. Make your code changes (including database schema changes in ``db.py``).

4. After every database schema change, recreate the throwaway database

   to test the changes::

    rm temp.db

    paster setup-db temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes

    paster repo-scan temp.ini

5. Once satisfied with the schema changes, auto-generate a draft Alembic

   script using the development database that has *not* been upgraded.

   (The generated script will upgrade the database to match the code.)

   ::

    alembic -c development.ini revision -m "area: add cool feature" --autogenerate

6. Edit the script to clean it up and fix any problems.

   Note that for changes that simply add columns, it may be appropriate

   to not remove them in the downgrade script (and instead do nothing),

   to avoid the loss of data. Unknown columns will simply be ignored by

   Kallithea versions predating your changes.

7. Run ``alembic -c development.ini upgrade head`` to apply changes to

   the (non-throwaway) database, and test the upgrade script. Also test

   downgrades.

   The included ``development.ini`` has full SQL logging enabled. If

   you're using another configuration file, you may want to enable it

   by setting ``level = DEBUG`` in section ``[handler_console_sql]``.

The Alembic migration script should be committed in the same revision as

the database schema (``db.py``) changes.

See the `Alembic documentation`__ for more information, in particular

the tutorial and the section about auto-generating migration scripts.

.. __: http://alembic.zzzcomputing.com/en/latest/

Troubleshooting

---------------

* If ``alembic --autogenerate`` responds "Target database is not up to

  date", you need to either first use Alembic to upgrade the database

  to the most recent version (before your changes), or recreate the

  database from scratch (without your schema changes applied).

   dev/dbmigrations

.. _upgrade_db:

.. note::

    If you are *downgrading* Kallithea, you should perform the database

    migration step *before* installing the older version. (That is,

    always perform migrations using the most recent of the two versions

    you're migrating between.)

First, run the following command to see your current database version::

    alembic -c my.ini current

Typical output will be something like "9358dc3d6828 (head)", which is

the current Alembic database "revision ID". Write down the entire output

for troubleshooting purposes.

The output will be empty if you're upgrading from Kallithea 0.3.x or

older. That's expected. If you get an error that the config file was not

found or has no ``[alembic]`` section, see the next section.

Next, if you are performing an *upgrade*: Run the following command to

upgrade your database to the current Kallithea version::

    alembic -c my.ini upgrade head

If you are performing a *downgrade*: Run the following command to

downgrade your database to the given version::

    alembic -c my.ini downgrade 0.4

Alembic will show the necessary migrations (if any) as it executes them.

If no "ERROR" is displayed, the command was successful.

Should an error occur, the database may be "stranded" half-way

through the migration, and you should restore it from backup.

Enabling old Kallithea config files for Alembic use

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Kallithea configuration files created before the introduction of Alembic

(i.e. predating Kallithea 0.4) need to be updated for use with Alembic.

Without this, Alembic will fail with an error like this::

    FAILED: No config file 'my.ini' found, or file has no '[alembic]' section

If Alembic complains specifically about a missing ``alembic.ini``, it is

likely because you did not specify a config file using the ``-c`` option.

On the other hand, if the mentioned config file actually exists, you

need to append the following lines to it::

    [alembic]

    script_location = kallithea:alembic

Your config file should now work with Alembic.

        raise SystemExit(

            'The "paster upgrade-db" command has been removed; please see the docs:\n'

            '    https://kallithea.readthedocs.io/en/default/upgrade.html'

        )