Changeset - 3158cf0dafb7
[Not reviewed]
default
1 11 1
Thomas De Schampheleire - 7 years ago 2018-11-18 20:02:17
thomas.de_schampheleire@nokia.com
cli: convert 'gearbox setup-db' into 'kallithea-cli db-create'
12 files changed with 70 insertions and 106 deletions:
0 comments (0 inline, 0 general)
docs/contributing.rst
Show inline comments
 
@@ -31,25 +31,25 @@ Getting started
 
To get started with Kallithea development::
 

	
 
        hg clone https://kallithea-scm.org/repos/kallithea
 
        cd kallithea
 
        virtualenv ../kallithea-venv
 
        source ../kallithea-venv/bin/activate
 
        pip install --upgrade pip setuptools
 
        pip install --upgrade -e .
 
        pip install --upgrade -r dev_requirements.txt
 
        npm install     # install dependencies - both tools and data
 
        npm run less    # for generating css from less
 
        kallithea-cli config-create my.ini
 
        gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
 
        kallithea-cli db-create -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
 
        gearbox serve -c my.ini --reload &
 
        firefox http://127.0.0.1:5000/
 

	
 
If you plan to use Bitbucket_ for sending contributions, you can also fork
 
Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and
 
then replace the clone step above by a clone of your fork. In this case, please
 
see :ref:`contributing-guidelines` below for configuring your fork correctly.
 

	
 

	
 
Contribution flow
 
-----------------
 

	
docs/dev/dbmigrations.rst
Show inline comments
 
@@ -21,25 +21,25 @@ changes, you should make a matching Alem
 
    kallithea-cli config-create temp.ini
 

	
 
   Edit the file to change database settings. SQLite is typically fine,
 
   but make sure to change the path to e.g. ``temp.db``, to avoid
 
   clobbering any existing database file.
 

	
 
3. Make your code changes (including database schema changes in ``db.py``).
 

	
 
4. After every database schema change, recreate the throwaway database
 
   to test the changes::
 

	
 
    rm temp.db
 
    gearbox setup-db -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
 
    kallithea-cli db-create -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
 
    kallithea-cli repo-scan -c temp.ini
 

	
 
5. Once satisfied with the schema changes, auto-generate a draft Alembic
 
   script using the development database that has *not* been upgraded.
 
   (The generated script will upgrade the database to match the code.)
 

	
 
   ::
 

	
 
    alembic -c development.ini revision -m "area: add cool feature" --autogenerate
 

	
 
6. Edit the script to clean it up and fix any problems.
 

	
docs/installation_win.rst
Show inline comments
 
@@ -168,25 +168,25 @@ one). When ready, type::
 
  cd C:\Kallithea\Bin
 
  kallithea-cli config-create my.ini
 

	
 
Then you must edit my.ini to fit your needs (IP address, IP
 
port, mail settings, database, etc.). `NotePad++`__ or a similar text
 
editor is recommended to properly handle the newline character
 
differences between Unix and Windows.
 

	
 
__ http://notepad-plus-plus.org/
 

	
 
For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type::
 

	
 
  gearbox setup-db -c my.ini
 
  kallithea-cli db-create -c my.ini
 

	
 
.. warning:: This time a *new* database will be installed. You must
 
             follow a different process to later :ref:`upgrade <upgrade>`
 
             to a newer Kallithea version.
 

	
 
The script will ask you for confirmation about creating a new database, answer yes (y)
 

	
 
The script will ask you for the repository path, answer C:\\Kallithea\\Repos (or similar).
 

	
 
The script will ask you for the admin username and password, answer "admin" + "123456" (or whatever you want)
 

	
 
The script will ask you for admin mail, answer "admin@xxxx.com" (or whatever you want).
docs/installation_win_old.rst
Show inline comments
 
@@ -206,25 +206,25 @@ if you closed it reopen it following the
 
  cd C:\Kallithea\Bin
 
  kallithea-cli config-create my.ini
 

	
 
Then, you must edit my.ini to fit your needs (network address and
 
port, mail settings, database, whatever). I recommend using NotePad++
 
(free) or similar text editor, as it handles well the EndOfLine
 
character differences between Unix and Windows
 
(http://notepad-plus-plus.org/)
 

	
 
For the sake of simplicity lets run it with the default settings. After
 
your edits (if any), in the previous Command Prompt, type::
 

	
 
 gearbox setup-db -c my.ini
 
  kallithea-cli db-create -c my.ini
 

	
 
.. warning:: This time a *new* database will be installed. You must
 
             follow a different process to later :ref:`upgrade <upgrade>`
 
             to a newer Kallithea version.
 

	
 
The script will ask you for confirmation about creating a NEW database,
 
answer yes (y)
 
The script will ask you for repository path, answer C:\\Kallithea\\Repos
 
(or similar)
 
The script will ask you for admin username and password, answer "admin"
 
+ "123456" (or whatever you want)
 
The script will ask you for admin mail, answer "admin@xxxx.com" (or
docs/setup.rst
Show inline comments
 
@@ -34,59 +34,59 @@ configuration file contains the various 
 
proxy port, email settings, usage of static files, cache, Celery
 
settings, and logging. Extra settings can be specified like::
 

	
 
    kallithea-cli config-create my.ini host=8.8.8.8 "[handler_console]" formatter=color_formatter
 

	
 
Next, you need to create the databases used by Kallithea. It is recommended to
 
use PostgreSQL or SQLite (default). If you choose a database other than the
 
default, ensure you properly adjust the database URL in your ``my.ini``
 
configuration file to use this other database. Kallithea currently supports
 
PostgreSQL, SQLite and MySQL databases. Create the database by running
 
the following command::
 

	
 
    gearbox setup-db -c my.ini
 
    kallithea-cli db-create -c my.ini
 

	
 
This will prompt you for a "root" path. This "root" path is the location where
 
Kallithea will store all of its repositories on the current machine. After
 
entering this "root" path ``setup-db`` will also prompt you for a username
 
and password for the initial admin account which ``setup-db`` sets
 
entering this "root" path ``db-create`` will also prompt you for a username
 
and password for the initial admin account which ``db-create`` sets
 
up for you.
 

	
 
The ``setup-db`` values can also be given on the command line.
 
The ``db-create`` values can also be given on the command line.
 
Example::
 

	
 
    gearbox setup-db -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos
 
    kallithea-cli db-create -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos
 

	
 
The ``setup-db`` command will create all needed tables and an
 
The ``db-create`` command will create all needed tables and an
 
admin account. When choosing a root path you can either use a new
 
empty location, or a location which already contains existing
 
repositories. If you choose a location which contains existing
 
repositories Kallithea will add all of the repositories at the chosen
 
location to its database.  (Note: make sure you specify the correct
 
path to the root).
 

	
 
.. note:: the given path for Mercurial_ repositories **must** be write
 
          accessible for the application. It's very important since
 
          the Kallithea web interface will work without write access,
 
          but when trying to do a push it will fail with permission
 
          denied errors unless it has write access.
 

	
 
You are now ready to use Kallithea. To run it simply execute::
 

	
 
    gearbox serve -c my.ini
 

	
 
- This command runs the Kallithea server. The web app should be available at
 
  http://127.0.0.1:5000. The IP address and port is configurable via the
 
  configuration file created in the previous step.
 
- Log in to Kallithea using the admin account created when running ``setup-db``.
 
- Log in to Kallithea using the admin account created when running ``db-create``.
 
- The default permissions on each repository is read, and the owner is admin.
 
  Remember to update these if needed.
 
- In the admin panel you can toggle LDAP, anonymous, and permissions
 
  settings, as well as edit more advanced options on users and
 
  repositories.
 

	
 

	
 
Internationalization (i18n support)
 
-----------------------------------
 

	
 
The Kallithea web interface is automatically displayed in the user's preferred
 
language, as indicated by the browser. Thus, different users may see the
docs/usage/vcs_notes.rst
Show inline comments
 
@@ -35,25 +35,25 @@ of the new repository's summary page.
 
A benefit of this method particular for Git repositories, is that the
 
Kallithea-specific Git hooks are installed automatically.  For Mercurial, no
 
hooks are required anyway.
 

	
 
Importing via the filesystem
 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

	
 
The alternative method of importing repositories consists of creating the
 
repositories in the desired hierarchy on the filesystem and letting Kallithea
 
scan that location.
 

	
 
All repositories are stored in a central location on the filesystem. This
 
location is specified during installation (via ``setup-db``) and can be reviewed
 
location is specified during installation (via ``db-create``) and can be reviewed
 
at *Admin > Settings > VCS > Location of repositories*. Repository groups
 
(defined in *Admin > Repository Groups*) are represented by a directory in that
 
repository location. Repositories of the repository group are nested under that
 
directory.
 

	
 
To import a set of repositories and organize them in a certain repository group
 
structure, first place clones in the desired hierarchy at the configured
 
repository location.
 
These clones should be created without working directory. For Mercurial, this is
 
done with ``hg clone -U``, for Git with ``git clone --bare``.
 

	
 
When the repositories are added correctly on the filesystem:
kallithea/alembic/env.py
Show inline comments
 
@@ -31,25 +31,25 @@ config = context.config
 
# [alembic] sqlalchemy.url.
 
database_url = (
 
    config.get_main_option('sqlalchemy.url') or
 
    config.get_section_option('app:main', 'sqlalchemy.url')
 
)
 

	
 
# Configure default logging for Alembic. (This can be overriden by the
 
# config file, but usually isn't.)
 
logging.getLogger('alembic').setLevel(logging.INFO)
 

	
 
# Setup Python loggers based on the config file provided to the alembic
 
# command. If we're being invoked via the Alembic API (presumably for
 
# stamping during "gearbox setup-db"), config_file_name is not available,
 
# stamping during "kallithea-cli db-create"), config_file_name is not available,
 
# and loggers are assumed to already have been configured.
 
if config.config_file_name:
 
    fileConfig(config.config_file_name, disable_existing_loggers=False)
 

	
 

	
 
def include_in_autogeneration(object, name, type, reflected, compare_to):
 
    """Filter changes subject to autogeneration of migrations. """
 

	
 
    # Don't include changes to sqlite_sequence.
 
    if type == 'table' and name == 'sqlite_sequence':
 
        return False
 

	
kallithea/bin/kallithea_cli.py
Show inline comments
 
@@ -8,15 +8,16 @@
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
# 'cli' is the main entry point for 'kallithea-cli', specified in setup.py as entry_points console_scripts
 
from kallithea.bin.kallithea_cli_base import cli
 

	
 
# import commands (they will add themselves to the 'cli' object)
 
import kallithea.bin.kallithea_cli_config
 
import kallithea.bin.kallithea_cli_db
 
import kallithea.bin.kallithea_cli_iis
 
import kallithea.bin.kallithea_cli_ishell
 
import kallithea.bin.kallithea_cli_repo
kallithea/bin/kallithea_cli_db.py
Show inline comments
 
file renamed from kallithea/lib/paster_commands/setup_db.py to kallithea/bin/kallithea_cli_db.py
 
@@ -2,113 +2,77 @@
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
"""
 
kallithea.lib.paster_commands.setup_db
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Databaset setup gearbox command for Kallithea
 
"""
 

	
 

	
 
import click
 
import kallithea.bin.kallithea_cli_base as cli_base
 

	
 
import kallithea
 
from kallithea.lib.db_manage import DbManage
 
from kallithea.lib.paster_commands.common import BasePasterCommand
 
from kallithea.model.meta import Session
 

	
 

	
 
# This is almost like SetupAppCommand ... but we have to pass options and it is
 
# thus simpler to drop websetup and reimplement everything
 
class Command(BasePasterCommand):
 
    """Kallithea: Configure the database specified in the .ini file
 
@cli_base.register_command(config_file=True)
 
@click.option('--user', help='Username of administrator account.')
 
@click.option('--password', help='Password for administrator account.')
 
@click.option('--email', help='Email address of administrator account.')
 
@click.option('--repos', help='Absolute path to repositories location.')
 
@click.option('--force-yes', is_flag=True, help='Answer yes to every question.')
 
@click.option('--force-no', is_flag=True, help='Answer no to every question.')
 
@click.option('--public-access/--no-public-access', default=True,
 
        help='Enable/disable public access on this installation (default: enable)')
 
def db_create(user, password, email, repos, force_yes, force_no, public_access):
 
    """Initialize the database.
 

	
 
    Setup Kallithea according to its configuration file.  This is
 
    the second part of a two-phase web application installation
 
    process (the first phase is prepare-app). The setup process
 
    consist of things like setting up databases and creating the admin user
 
    Create all required tables in the database specified in the configuration
 
    file. Create the administrator account. Set certain settings based on
 
    values you provide.
 

	
 
    You can pass the answers to all questions as options to this command.
 
    """
 
    dbconf = kallithea.CONFIG['sqlalchemy.url']
 

	
 
    def get_description(self):
 
        return self.__doc__.splitlines()[0]
 

	
 
    requires_db_session = False # only available after this command has been run
 

	
 
    def get_parser(self, prog_name):
 
        parser = super(Command, self).get_parser(prog_name)
 
    # force_ask should be True (yes), False (no), or None (ask)
 
    if force_yes:
 
        force_ask = True
 
    elif force_no:
 
        force_ask = False
 
    else:
 
        force_ask = None
 

	
 
        parser.add_argument('--user',
 
                          action='store',
 
                          dest='username',
 
                          default=None,
 
                          help='Admin Username')
 
        parser.add_argument('--email',
 
                          action='store',
 
                          dest='email',
 
                          default=None,
 
                          help='Admin Email')
 
        parser.add_argument('--password',
 
                          action='store',
 
                          dest='password',
 
                          default=None,
 
                          help='Admin password min 6 chars')
 
        parser.add_argument('--repos',
 
                          action='store',
 
                          dest='repos_location',
 
                          default=None,
 
                          help='Absolute path to repositories location')
 
        parser.add_argument('--force-yes',
 
                           action='store_true',
 
                           dest='force_ask',
 
                           default=None,
 
                           help='Force yes to every question')
 
        parser.add_argument('--force-no',
 
                           action='store_false',
 
                           dest='force_ask',
 
                           default=None,
 
                           help='Force no to every question')
 
        parser.add_argument('--public-access',
 
                           action='store_true',
 
                           dest='public_access',
 
                           default=None,
 
                           help='Enable public access on this installation (default)')
 
        parser.add_argument('--no-public-access',
 
                           action='store_false',
 
                           dest='public_access',
 
                           default=None,
 
                           help='Disable public access on this installation ')
 
    cli_args = dict(
 
            username=user,
 
            password=password,
 
            email=email,
 
            repos_location=repos,
 
            force_ask=force_ask,
 
            public_access=public_access,
 
    )
 
    dbmanage = DbManage(dbconf=dbconf, root=kallithea.CONFIG['here'],
 
                        tests=False, cli_args=cli_args)
 
    dbmanage.create_tables(override=True)
 
    opts = dbmanage.config_prompt(None)
 
    dbmanage.create_settings(opts)
 
    dbmanage.create_default_user()
 
    dbmanage.admin_prompt()
 
    dbmanage.create_permissions()
 
    dbmanage.populate_default_permissions()
 
    Session().commit()
 

	
 
        return parser
 
    # initial repository scan
 
    kallithea.config.middleware.make_app_without_logging(
 
            kallithea.CONFIG.global_conf, **kallithea.CONFIG.local_conf)
 
    added, _ = kallithea.lib.utils.repo2db_mapper(kallithea.model.scm.ScmModel().repo_scan())
 
    if added:
 
        click.echo('Initial repository scan: added following repositories:')
 
        click.echo('\t%s' % '\n\t'.join(added))
 
    else:
 
        click.echo('Initial repository scan: no repositories found.')
 

	
 
    def take_action(self, opts):
 
        dbconf = self.config['sqlalchemy.url']
 
        dbmanage = DbManage(dbconf=dbconf, root=self.config['here'],
 
                            tests=False, cli_args=vars(opts))
 
        dbmanage.create_tables(override=True)
 
        opts = dbmanage.config_prompt(None)
 
        dbmanage.create_settings(opts)
 
        dbmanage.create_default_user()
 
        dbmanage.admin_prompt()
 
        dbmanage.create_permissions()
 
        dbmanage.populate_default_permissions()
 
        Session().commit()
 

	
 
        # initial repository scan
 
        kallithea.config.middleware.make_app_without_logging(
 
                self.config.global_conf, **self.config.local_conf)
 
        added, _ = kallithea.lib.utils.repo2db_mapper(kallithea.model.scm.ScmModel().repo_scan())
 
        if added:
 
            print 'Initial repository scan: added following repositories:'
 
            print '\t','\n\t'.join(added)
 
        else:
 
            print 'Initial repository scan: no repositories found.'
 

	
 
        print 'Database set up successfully.'
 
    click.echo('Database set up successfully.')
kallithea/tests/scripts/create_rc.sh
Show inline comments
 
#!/bin/sh
 
psql -U postgres -h localhost -c 'drop database if exists kallithea;'
 
psql -U postgres -h localhost -c 'create database kallithea;'
 
gearbox setup-db -c server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access
 
kallithea-cli db-create -c server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access
 
API_KEY=`psql -R " " -A -U postgres -h localhost -c "select api_key from users where admin=TRUE" -d kallithea | awk '{print $2}'`
 
echo "run those after running server"
 
gearbox serve -c server.ini --pid-file=server.pid --daemon
 
sleep 3
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo1 password:qweqwe email:demo1@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo2 password:qweqwe email:demo2@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo3 password:qweqwe email:demo3@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user_group group_name:demo12
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo1
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo2
 
echo "killing server"
 
kill `cat server.pid`
scripts/dbmigrate-test
Show inline comments
 
@@ -95,25 +95,25 @@ install_kallithea() {
 
        cd "$prefix"
 
        . "$prefix-env/bin/activate"
 
        pip install --quiet --upgrade pip setuptools mercurial $EXTRA
 
        pip install --quiet -e .
 
    )
 
}
 

	
 
install_kallithea "$temp/from" "$from_rev_hash"
 
(
 
    cd "$temp/from"
 
    . "$temp/from-env/bin/activate"
 
    announce "Initializing database..."
 
    quiet_if_ok gearbox setup-db -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes
 
    quiet_if_ok kallithea-cli db-create -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes
 
    alembic -c "$config_file" current -v
 
)
 

	
 
install_kallithea "$temp/to" "$to_rev_hash"
 
(
 
    cd "$temp/to"
 
    . "$temp/to-env/bin/activate"
 

	
 
    announce "Commencing database upgrade from shown Alembic revision to head..."
 
    alembic -c "$config_file" current -v
 
    alembic -c "$config_file" upgrade head
 
    announce "Upgrade complete, now at the shown Alembic revision:"
setup.py
Show inline comments
 
@@ -153,16 +153,15 @@ setuptools.setup(
 
    kallithea-api =    kallithea.bin.kallithea_api:main
 
    kallithea-gist =   kallithea.bin.kallithea_gist:main
 
    kallithea-config = kallithea.bin.kallithea_config:main
 
    kallithea-cli =    kallithea.bin.kallithea_cli:cli
 

	
 
    [paste.app_factory]
 
    main = kallithea.config.middleware:make_app
 

	
 
    [gearbox.commands]
 
    celeryd=kallithea.lib.paster_commands.celeryd:Command
 
    make-index=kallithea.lib.paster_commands.make_index:Command
 
    make-rcext=kallithea.lib.paster_commands.make_rcextensions:Command
 
    setup-db=kallithea.lib.paster_commands.setup_db:Command
 
    upgrade-db=kallithea.lib.dbmigrate:UpgradeDb
 
    """,
 
)
0 comments (0 inline, 0 general)