Changeset - 3158cf0dafb7
[Not reviewed]
default
1 11 1
Thomas De Schampheleire - 7 years ago 2018-11-18 20:02:17
thomas.de_schampheleire@nokia.com
cli: convert 'gearbox setup-db' into 'kallithea-cli db-create'
12 files changed with 57 insertions and 93 deletions:
0 comments (0 inline, 0 general)
docs/contributing.rst
Show inline comments
 
@@ -31,25 +31,25 @@ Getting started
 
To get started with Kallithea development::
 

	
 
        hg clone https://kallithea-scm.org/repos/kallithea
 
        cd kallithea
 
        virtualenv ../kallithea-venv
 
        source ../kallithea-venv/bin/activate
 
        pip install --upgrade pip setuptools
 
        pip install --upgrade -e .
 
        pip install --upgrade -r dev_requirements.txt
 
        npm install     # install dependencies - both tools and data
 
        npm run less    # for generating css from less
 
        kallithea-cli config-create my.ini
 
        gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
 
        kallithea-cli db-create -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
 
        gearbox serve -c my.ini --reload &
 
        firefox http://127.0.0.1:5000/
 

	
 
If you plan to use Bitbucket_ for sending contributions, you can also fork
 
Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and
 
then replace the clone step above by a clone of your fork. In this case, please
 
see :ref:`contributing-guidelines` below for configuring your fork correctly.
 

	
 

	
 
Contribution flow
 
-----------------
 

	
docs/dev/dbmigrations.rst
Show inline comments
 
@@ -21,25 +21,25 @@ changes, you should make a matching Alem
 
    kallithea-cli config-create temp.ini
 

	
 
   Edit the file to change database settings. SQLite is typically fine,
 
   but make sure to change the path to e.g. ``temp.db``, to avoid
 
   clobbering any existing database file.
 

	
 
3. Make your code changes (including database schema changes in ``db.py``).
 

	
 
4. After every database schema change, recreate the throwaway database
 
   to test the changes::
 

	
 
    rm temp.db
 
    gearbox setup-db -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
 
    kallithea-cli db-create -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
 
    kallithea-cli repo-scan -c temp.ini
 

	
 
5. Once satisfied with the schema changes, auto-generate a draft Alembic
 
   script using the development database that has *not* been upgraded.
 
   (The generated script will upgrade the database to match the code.)
 

	
 
   ::
 

	
 
    alembic -c development.ini revision -m "area: add cool feature" --autogenerate
 

	
 
6. Edit the script to clean it up and fix any problems.
 

	
docs/installation_win.rst
Show inline comments
 
@@ -168,25 +168,25 @@ one). When ready, type::
 
  cd C:\Kallithea\Bin
 
  kallithea-cli config-create my.ini
 

	
 
Then you must edit my.ini to fit your needs (IP address, IP
 
port, mail settings, database, etc.). `NotePad++`__ or a similar text
 
editor is recommended to properly handle the newline character
 
differences between Unix and Windows.
 

	
 
__ http://notepad-plus-plus.org/
 

	
 
For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type::
 

	
 
  gearbox setup-db -c my.ini
 
  kallithea-cli db-create -c my.ini
 

	
 
.. warning:: This time a *new* database will be installed. You must
 
             follow a different process to later :ref:`upgrade <upgrade>`
 
             to a newer Kallithea version.
 

	
 
The script will ask you for confirmation about creating a new database, answer yes (y)
 

	
 
The script will ask you for the repository path, answer C:\\Kallithea\\Repos (or similar).
 

	
 
The script will ask you for the admin username and password, answer "admin" + "123456" (or whatever you want)
 

	
 
The script will ask you for admin mail, answer "admin@xxxx.com" (or whatever you want).
docs/installation_win_old.rst
Show inline comments
 
@@ -206,25 +206,25 @@ if you closed it reopen it following the
 
  cd C:\Kallithea\Bin
 
  kallithea-cli config-create my.ini
 

	
 
Then, you must edit my.ini to fit your needs (network address and
 
port, mail settings, database, whatever). I recommend using NotePad++
 
(free) or similar text editor, as it handles well the EndOfLine
 
character differences between Unix and Windows
 
(http://notepad-plus-plus.org/)
 

	
 
For the sake of simplicity lets run it with the default settings. After
 
your edits (if any), in the previous Command Prompt, type::
 

	
 
 gearbox setup-db -c my.ini
 
  kallithea-cli db-create -c my.ini
 

	
 
.. warning:: This time a *new* database will be installed. You must
 
             follow a different process to later :ref:`upgrade <upgrade>`
 
             to a newer Kallithea version.
 

	
 
The script will ask you for confirmation about creating a NEW database,
 
answer yes (y)
 
The script will ask you for repository path, answer C:\\Kallithea\\Repos
 
(or similar)
 
The script will ask you for admin username and password, answer "admin"
 
+ "123456" (or whatever you want)
 
The script will ask you for admin mail, answer "admin@xxxx.com" (or
docs/setup.rst
Show inline comments
 
@@ -34,59 +34,59 @@ configuration file contains the various 
 
proxy port, email settings, usage of static files, cache, Celery
 
settings, and logging. Extra settings can be specified like::
 

	
 
    kallithea-cli config-create my.ini host=8.8.8.8 "[handler_console]" formatter=color_formatter
 

	
 
Next, you need to create the databases used by Kallithea. It is recommended to
 
use PostgreSQL or SQLite (default). If you choose a database other than the
 
default, ensure you properly adjust the database URL in your ``my.ini``
 
configuration file to use this other database. Kallithea currently supports
 
PostgreSQL, SQLite and MySQL databases. Create the database by running
 
the following command::
 

	
 
    gearbox setup-db -c my.ini
 
    kallithea-cli db-create -c my.ini
 

	
 
This will prompt you for a "root" path. This "root" path is the location where
 
Kallithea will store all of its repositories on the current machine. After
 
entering this "root" path ``setup-db`` will also prompt you for a username
 
and password for the initial admin account which ``setup-db`` sets
 
entering this "root" path ``db-create`` will also prompt you for a username
 
and password for the initial admin account which ``db-create`` sets
 
up for you.
 

	
 
The ``setup-db`` values can also be given on the command line.
 
The ``db-create`` values can also be given on the command line.
 
Example::
 

	
 
    gearbox setup-db -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos
 
    kallithea-cli db-create -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos
 

	
 
The ``setup-db`` command will create all needed tables and an
 
The ``db-create`` command will create all needed tables and an
 
admin account. When choosing a root path you can either use a new
 
empty location, or a location which already contains existing
 
repositories. If you choose a location which contains existing
 
repositories Kallithea will add all of the repositories at the chosen
 
location to its database.  (Note: make sure you specify the correct
 
path to the root).
 

	
 
.. note:: the given path for Mercurial_ repositories **must** be write
 
          accessible for the application. It's very important since
 
          the Kallithea web interface will work without write access,
 
          but when trying to do a push it will fail with permission
 
          denied errors unless it has write access.
 

	
 
You are now ready to use Kallithea. To run it simply execute::
 

	
 
    gearbox serve -c my.ini
 

	
 
- This command runs the Kallithea server. The web app should be available at
 
  http://127.0.0.1:5000. The IP address and port is configurable via the
 
  configuration file created in the previous step.
 
- Log in to Kallithea using the admin account created when running ``setup-db``.
 
- Log in to Kallithea using the admin account created when running ``db-create``.
 
- The default permissions on each repository is read, and the owner is admin.
 
  Remember to update these if needed.
 
- In the admin panel you can toggle LDAP, anonymous, and permissions
 
  settings, as well as edit more advanced options on users and
 
  repositories.
 

	
 

	
 
Internationalization (i18n support)
 
-----------------------------------
 

	
 
The Kallithea web interface is automatically displayed in the user's preferred
 
language, as indicated by the browser. Thus, different users may see the
docs/usage/vcs_notes.rst
Show inline comments
 
@@ -35,25 +35,25 @@ of the new repository's summary page.
 
A benefit of this method particular for Git repositories, is that the
 
Kallithea-specific Git hooks are installed automatically.  For Mercurial, no
 
hooks are required anyway.
 

	
 
Importing via the filesystem
 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

	
 
The alternative method of importing repositories consists of creating the
 
repositories in the desired hierarchy on the filesystem and letting Kallithea
 
scan that location.
 

	
 
All repositories are stored in a central location on the filesystem. This
 
location is specified during installation (via ``setup-db``) and can be reviewed
 
location is specified during installation (via ``db-create``) and can be reviewed
 
at *Admin > Settings > VCS > Location of repositories*. Repository groups
 
(defined in *Admin > Repository Groups*) are represented by a directory in that
 
repository location. Repositories of the repository group are nested under that
 
directory.
 

	
 
To import a set of repositories and organize them in a certain repository group
 
structure, first place clones in the desired hierarchy at the configured
 
repository location.
 
These clones should be created without working directory. For Mercurial, this is
 
done with ``hg clone -U``, for Git with ``git clone --bare``.
 

	
 
When the repositories are added correctly on the filesystem:
kallithea/alembic/env.py
Show inline comments
 
@@ -31,25 +31,25 @@ config = context.config
 
# [alembic] sqlalchemy.url.
 
database_url = (
 
    config.get_main_option('sqlalchemy.url') or
 
    config.get_section_option('app:main', 'sqlalchemy.url')
 
)
 

	
 
# Configure default logging for Alembic. (This can be overriden by the
 
# config file, but usually isn't.)
 
logging.getLogger('alembic').setLevel(logging.INFO)
 

	
 
# Setup Python loggers based on the config file provided to the alembic
 
# command. If we're being invoked via the Alembic API (presumably for
 
# stamping during "gearbox setup-db"), config_file_name is not available,
 
# stamping during "kallithea-cli db-create"), config_file_name is not available,
 
# and loggers are assumed to already have been configured.
 
if config.config_file_name:
 
    fileConfig(config.config_file_name, disable_existing_loggers=False)
 

	
 

	
 
def include_in_autogeneration(object, name, type, reflected, compare_to):
 
    """Filter changes subject to autogeneration of migrations. """
 

	
 
    # Don't include changes to sqlite_sequence.
 
    if type == 'table' and name == 'sqlite_sequence':
 
        return False
 

	
kallithea/bin/kallithea_cli.py
Show inline comments
 
@@ -8,15 +8,16 @@
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
# 'cli' is the main entry point for 'kallithea-cli', specified in setup.py as entry_points console_scripts
 
from kallithea.bin.kallithea_cli_base import cli
 

	
 
# import commands (they will add themselves to the 'cli' object)
 
import kallithea.bin.kallithea_cli_config
 
import kallithea.bin.kallithea_cli_db
 
import kallithea.bin.kallithea_cli_iis
 
import kallithea.bin.kallithea_cli_ishell
 
import kallithea.bin.kallithea_cli_repo
kallithea/bin/kallithea_cli_db.py
Show inline comments
 
file renamed from kallithea/lib/paster_commands/setup_db.py to kallithea/bin/kallithea_cli_db.py
 
@@ -2,113 +2,77 @@
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
"""
 
kallithea.lib.paster_commands.setup_db
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
Databaset setup gearbox command for Kallithea
 
"""
 

	
 

	
 
import click
 
import kallithea.bin.kallithea_cli_base as cli_base
 

	
 
import kallithea
 
from kallithea.lib.db_manage import DbManage
 
from kallithea.lib.paster_commands.common import BasePasterCommand
 
from kallithea.model.meta import Session
 

	
 

	
 
# This is almost like SetupAppCommand ... but we have to pass options and it is
 
# thus simpler to drop websetup and reimplement everything
 
class Command(BasePasterCommand):
 
    """Kallithea: Configure the database specified in the .ini file
 
@cli_base.register_command(config_file=True)
 
@click.option('--user', help='Username of administrator account.')
 
@click.option('--password', help='Password for administrator account.')
 
@click.option('--email', help='Email address of administrator account.')
 
@click.option('--repos', help='Absolute path to repositories location.')
 
@click.option('--force-yes', is_flag=True, help='Answer yes to every question.')
 
@click.option('--force-no', is_flag=True, help='Answer no to every question.')
 
@click.option('--public-access/--no-public-access', default=True,
 
        help='Enable/disable public access on this installation (default: enable)')
 
def db_create(user, password, email, repos, force_yes, force_no, public_access):
 
    """Initialize the database.
 

	
 
    Setup Kallithea according to its configuration file.  This is
 
    the second part of a two-phase web application installation
 
    process (the first phase is prepare-app). The setup process
 
    consist of things like setting up databases and creating the admin user
 
    """
 

	
 
    def get_description(self):
 
        return self.__doc__.splitlines()[0]
 

	
 
    requires_db_session = False # only available after this command has been run
 

	
 
    def get_parser(self, prog_name):
 
        parser = super(Command, self).get_parser(prog_name)
 
    Create all required tables in the database specified in the configuration
 
    file. Create the administrator account. Set certain settings based on
 
    values you provide.
 

	
 
        parser.add_argument('--user',
 
                          action='store',
 
                          dest='username',
 
                          default=None,
 
                          help='Admin Username')
 
        parser.add_argument('--email',
 
                          action='store',
 
                          dest='email',
 
                          default=None,
 
                          help='Admin Email')
 
        parser.add_argument('--password',
 
                          action='store',
 
                          dest='password',
 
                          default=None,
 
                          help='Admin password min 6 chars')
 
        parser.add_argument('--repos',
 
                          action='store',
 
                          dest='repos_location',
 
                          default=None,
 
                          help='Absolute path to repositories location')
 
        parser.add_argument('--force-yes',
 
                           action='store_true',
 
                           dest='force_ask',
 
                           default=None,
 
                           help='Force yes to every question')
 
        parser.add_argument('--force-no',
 
                           action='store_false',
 
                           dest='force_ask',
 
                           default=None,
 
                           help='Force no to every question')
 
        parser.add_argument('--public-access',
 
                           action='store_true',
 
                           dest='public_access',
 
                           default=None,
 
                           help='Enable public access on this installation (default)')
 
        parser.add_argument('--no-public-access',
 
                           action='store_false',
 
                           dest='public_access',
 
                           default=None,
 
                           help='Disable public access on this installation ')
 
    You can pass the answers to all questions as options to this command.
 
    """
 
    dbconf = kallithea.CONFIG['sqlalchemy.url']
 

	
 
        return parser
 
    # force_ask should be True (yes), False (no), or None (ask)
 
    if force_yes:
 
        force_ask = True
 
    elif force_no:
 
        force_ask = False
 
    else:
 
        force_ask = None
 

	
 
    def take_action(self, opts):
 
        dbconf = self.config['sqlalchemy.url']
 
        dbmanage = DbManage(dbconf=dbconf, root=self.config['here'],
 
                            tests=False, cli_args=vars(opts))
 
    cli_args = dict(
 
            username=user,
 
            password=password,
 
            email=email,
 
            repos_location=repos,
 
            force_ask=force_ask,
 
            public_access=public_access,
 
    )
 
    dbmanage = DbManage(dbconf=dbconf, root=kallithea.CONFIG['here'],
 
                        tests=False, cli_args=cli_args)
 
        dbmanage.create_tables(override=True)
 
        opts = dbmanage.config_prompt(None)
 
        dbmanage.create_settings(opts)
 
        dbmanage.create_default_user()
 
        dbmanage.admin_prompt()
 
        dbmanage.create_permissions()
 
        dbmanage.populate_default_permissions()
 
        Session().commit()
 

	
 
        # initial repository scan
 
        kallithea.config.middleware.make_app_without_logging(
 
                self.config.global_conf, **self.config.local_conf)
 
            kallithea.CONFIG.global_conf, **kallithea.CONFIG.local_conf)
 
        added, _ = kallithea.lib.utils.repo2db_mapper(kallithea.model.scm.ScmModel().repo_scan())
 
        if added:
 
            print 'Initial repository scan: added following repositories:'
 
            print '\t','\n\t'.join(added)
 
        click.echo('Initial repository scan: added following repositories:')
 
        click.echo('\t%s' % '\n\t'.join(added))
 
        else:
 
            print 'Initial repository scan: no repositories found.'
 
        click.echo('Initial repository scan: no repositories found.')
 

	
 
        print 'Database set up successfully.'
 
    click.echo('Database set up successfully.')
kallithea/tests/scripts/create_rc.sh
Show inline comments
 
#!/bin/sh
 
psql -U postgres -h localhost -c 'drop database if exists kallithea;'
 
psql -U postgres -h localhost -c 'create database kallithea;'
 
gearbox setup-db -c server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access
 
kallithea-cli db-create -c server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access
 
API_KEY=`psql -R " " -A -U postgres -h localhost -c "select api_key from users where admin=TRUE" -d kallithea | awk '{print $2}'`
 
echo "run those after running server"
 
gearbox serve -c server.ini --pid-file=server.pid --daemon
 
sleep 3
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo1 password:qweqwe email:demo1@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo2 password:qweqwe email:demo2@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo3 password:qweqwe email:demo3@example.com
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user_group group_name:demo12
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo1
 
kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo2
 
echo "killing server"
 
kill `cat server.pid`
scripts/dbmigrate-test
Show inline comments
 
@@ -95,25 +95,25 @@ install_kallithea() {
 
        cd "$prefix"
 
        . "$prefix-env/bin/activate"
 
        pip install --quiet --upgrade pip setuptools mercurial $EXTRA
 
        pip install --quiet -e .
 
    )
 
}
 

	
 
install_kallithea "$temp/from" "$from_rev_hash"
 
(
 
    cd "$temp/from"
 
    . "$temp/from-env/bin/activate"
 
    announce "Initializing database..."
 
    quiet_if_ok gearbox setup-db -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes
 
    quiet_if_ok kallithea-cli db-create -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes
 
    alembic -c "$config_file" current -v
 
)
 

	
 
install_kallithea "$temp/to" "$to_rev_hash"
 
(
 
    cd "$temp/to"
 
    . "$temp/to-env/bin/activate"
 

	
 
    announce "Commencing database upgrade from shown Alembic revision to head..."
 
    alembic -c "$config_file" current -v
 
    alembic -c "$config_file" upgrade head
 
    announce "Upgrade complete, now at the shown Alembic revision:"
setup.py
Show inline comments
 
@@ -153,16 +153,15 @@ setuptools.setup(
 
    kallithea-api =    kallithea.bin.kallithea_api:main
 
    kallithea-gist =   kallithea.bin.kallithea_gist:main
 
    kallithea-config = kallithea.bin.kallithea_config:main
 
    kallithea-cli =    kallithea.bin.kallithea_cli:cli
 

	
 
    [paste.app_factory]
 
    main = kallithea.config.middleware:make_app
 

	
 
    [gearbox.commands]
 
    celeryd=kallithea.lib.paster_commands.celeryd:Command
 
    make-index=kallithea.lib.paster_commands.make_index:Command
 
    make-rcext=kallithea.lib.paster_commands.make_rcextensions:Command
 
    setup-db=kallithea.lib.paster_commands.setup_db:Command
 
    upgrade-db=kallithea.lib.dbmigrate:UpgradeDb
 
    """,
 
)
0 comments (0 inline, 0 general)