config: clarify that we only recommend and support single threaded operation

Sad, but true. Especially because we reuse Repository instances between
threads.

gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack

This is a step towards moving away from the Pylons stack to TurboGears2, but
still independent of it.


Some notes from the porting - it could perhaps be the missing(?) documentation
for migrating from paster to gearbox:

Note: 'gearbox' without parameters will crash - specify '-h' to get started
testing.

Replace paster
summary = 'yada yada'
with the first line of the docstring of the Command class ... or override
get_description.

Note: All newlines in the docstring will be collapsed and mangle the long help
text.

Grouping of commands is not possible. Standard commands (for development) can't
be customized under the same name or hidden. (Like for paster, the conceptual
model also assumes that the sub-command naming is namespaced so commands from
other packages won't conflict.)

The usage help is fully automated from the declared options.

For all deprecated Commands, replace paster
hidden = True
with gearbox
deprecated = True

Note: config_file, takes_config_file, min_args and max_args are not available /
relevant.

The gearbox parser is customized by overriding get_parser - there is nothing
like paster update_parser.

Gearbox is using argparse instead of optparse ... but argparse add_argument is
mostly backwards compatible with optparse add_option.

Instead of overriding command or run as in paster, override take_action in
gearbox. The parsed arguments are passed to take_action, not available on the
command instance.

Paster BadCommand is not available and must be handled manually, terminating
with sys.exit(1).

There is no standard make-config command in gearbox.

Paster appinstall has been replaced by the somewhat different setup_app module
in gearbox. There is still no clean way to pass parameters to SetupAppCommand
and it relies on websetup and other apparently unnecessary complexity. Instead,
implement setup-db from scratch.


Minor change by Thomas De Schampheleire: add gearbox logging configuration.
Because we use logging.config.fileConfig(.inifile) during gearbox command
execution, the logging settings need to be correct and contain a block for
gearbox logging itself. Otherwise, errors in command processing are not even
visible and the command exits silently.

middleware: replace references to Errormator with AppEnlight.

Errormator has been rebranded as AppEnlight a while back.

Errormator is no longer available. This is just a trivial rename that can't
make things worse.

Turbogears2 migration: use sqlalchemy.url iso sqlalchemy.db1.url

In Turbogears2, much of the application initialization is handled by the
framework, whereas in Pylons the application was responsible for it.
Initializing SQLAlchemy is one such part of initialization which is handled
by Turbogears2.

Turbogears2 expects the configuration file to refer to the database using
'sqlalchemy.url' rather than the current 'sqlalchemy.db1.url'. While the
exact name is not really important, not following this approach means we'll
need to override the sqlalchemy initialization method.

Therefore, as a preparation to the Turbogears2 migration, already change the
database reference string under Pylons.
When upgrading to a version of Kallithea containing this commit, the .ini
file will manually need to be adapted to remove the .db1 strings.

celery: use Celery 3 config settings instead of deprecated

As warned by:
The 'CELERYD_LOG_LEVEL' setting is scheduled for deprecation in version 2.4 and removal in version v4.0. Use the --loglevel argument instead
remove celeryd.log.file and celeryd.log.level from the ini file.
Instead, use:
paster celeryd my.ini --loglevel=DEBUG --logfile=my.log
or, in the future:
gearbox celeryd -c my.ini -- --loglevel=DEBUG --logfile=my.log

As warned by:
The 'BROKER_VHOST' setting is scheduled for deprecation in version 2.5 and removal in version v4.0. Use the BROKER_URL setting instead
The 'BROKER_HOST' setting is scheduled for deprecation in version 2.5 and removal in version v4.0. Use the BROKER_URL setting instead
The 'BROKER_USER' setting is scheduled for deprecation in version 2.5 and removal in version v4.0. Use the BROKER_URL setting instead
The 'BROKER_PASSWORD' setting is scheduled for deprecation in version 2.5 and removal in version v4.0. Use the BROKER_URL setting instead
The 'BROKER_PORT' setting is scheduled for deprecation in version 2.5 and removal in version v4.0. Use the BROKER_URL setting instead
change the .ini template to use:
broker.url = amqp://rabbitmq:qewqew@localhost:5672/rabbitmqhost

As warned by:
Starting from version 3.2 Celery will refuse to accept pickle by default.

The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.

If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::

CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

You must only enable the serializers that you will actually use.
change the .ini template to use:
celery.accept.content = pickle

(Note: The warning is there for a reason. It would probably be nice to change
from pickle to something like json. That is left as an exercise.)

config: clarify that we only recommend and support single threaded operation

Sad, but true. Especially because we reuse Repository instances between
threads.

db: enable use of main Kallithea config as Alembic config

Newly generated Kallithea config .ini files will be valid Alembic
config files, eliminating the need for a separate alembic.ini config
redundantly specifying the database connection string.

We reference the Alembic migration environment using kallithea:alembic,
which should work independently of how Kallithea is installed.

We also configure a default 'alembic' log level of WARNING, to reduce
the amount of clutter in the config file, reduce the changes needed
to upgrade existing config files for use with Alembic, and allowing us
to change the default Alembic log level for all users down the road.

(It makes sense to define Alembic logging in code, while all other
loggers are configured in the configuration file, because Alembic
is special: it runs on the command line, not as part of the web app.)

docs: tweak email documentation - add gmail example

smtp_auth is too obscure to have in the .ini template.

db: get rid of vcs_full_cache - it should always be used

It might make tests less deterministic, but it makes sure that we test what we actually use.

db: stop messing with convert_unicode in .ini and db scheme

convert_unicode=False is the default for String fields ... except when the .ini file contains sets it to True.

String fields are however no longer used for unicode and we can drop convert_unicode everywhere.

See
http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.String.params.convert_unicode
for the advice of never setting convert_unicode.

ini: specify utf8 for sample MySQL connection strings

By default, the MySQL stack will store unicode as UTF-8 encoded data in string
fields, thus without using any unicode capabilities in the database.

As described in
http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#mysql-unicode , set
charset=utf8 to actually put unicode in the database.

Existing databases that already store utf8 in the database should keep using
the old url.

This will only support 16 bit code points, but utf8mb4 will double the key size
and make them too big for MySQL.

cache: make instance_id = * the default and deprecate it

Auto assigning instance_id's works perfectly fine now when the clean-up issue
has been resolved.

cleanup: use example.com for tests and examples

example.com is explicitly reserved for this purpose. Using that means
we won't accidentally hammer a real server or real email address if an
example value escapes into the wild, e.g. in an automated test.

The domain "kallithea.example.com" has been used throughout to refer to the
example Kallithea server.

spelling: use "email" consistently

The common English spelling is "email", not "e-mail" (and was indeed
also the most common, but not only, variant in the Kallithea UI).

http://grammarist.com/style/e-mail-email/

docs: improve documentation of beaker session configuration

beaker.session.auto is dropped; it defaults to false and there is no reason to
ever set it true for Kallithea.

beaker.session.cookie_path and secure are dropped; like cookie_domain, they
should automatically be set to the right value.
* * *
beaker.session.cookie_expires MUST have the default value of True to provide the default value of 'browser session lifetime' when not enabling 'remember' in the login box. The cookie life is hardcoded to 365 days when remember is selected.

e-mail: add documentation about configuration settings

Document e-mail related configuration settings in the documentation
(overview) and the .ini file itself (technical details and example
settings).

e-mail: add some default values and some examples to .ini file as comments

This includes a replacement of @localhost examples to @example.com, to
indicate that real addresses should normally be used.

e-mail: clarify that smtp_use_ssl defaults to false

Commented settings in configuration files conventionally indicate the
default value.

e-mail: remove unused setting error_message from ini files

The ini files and templates contain a commented setting of 'error_message'
which does not seem to be used. It is referring to the error_message
variable in Paste, which has as description (Paste:ErrorMiddleware):

When debug mode is off, the error message to show to users.

However, setting this value apparently made no effect at all in Kallithea.

i18n: drop redundant english translation

lang should thus be set empty instead of 'en', but we keep the en .mo around
to avoid upgrade issues.

ini file: clarify that beaker.session.key should be unique

When several instances of Kallithea are running on the same machine, the
same browser cannot be logged into both instances at the same time without
conflicts. The login session are saved into the same cookie; logging into
one instance closes the session on the second instance and vice-versa.

This is caused because the cookie name is simply 'kallithea', combined with
the fact that the cookie specification (RFC6265) states that there is no
isolation of cookies based on port. This means that the browser sends all
cookies from a given domain with all services (Kallithea instances) running
on that domain, irrespective of port.

The services thus need to handle any such issue themselves, for example by
using unique cookie names and only interacting with one's own cookie.

Making the key unique when creating the configuration file proved difficult:
- it does not seem possible to hook into 'paster make-config'
- since Beaker directly interprets the beaker.session.key, changing it on
the fly from SessionMiddleware will not work correctly.

There is a kallithea-config script that is an alternative to 'paster
make-config' which would be the ideal place to make such changes. However,
it seems this method is not advocated over 'paster make-config' (yet?).

Instead, simply add a comment in the config file and let the user take care
of it.

config: the default for show_revision_number should be false

Revision numbers are not really that useful and when shared publicly they can
lead to misunderstandings

Align and update the outdated ini files and templates

Change default database for development.ini to sqlite

Second step in two-part process to rename directories.
This is the actual directory rename.