hooks: put repo_size and update hooks in kallithea namespace

Keep things separate.

Include missing migration steps for 642847355a10.

celery: move send_email task to a better home in notification model

Avoid bundling everything from many different layers in one big task library.

This is more feasible now when we don't need kallithea.CELERY_APP set at import
time.

celery: move Whoosh indexing task to whoosh library where it belongs

Avoid bundling everything from many different layers in one big task library.

This is more feasible now when we don't need kallithea.CELERY_APP set at import
time.

controllers: move controllers base class from lib/base to controllers

TG quickstart put it in lib/base.py , but it fits better on the controllers
layer as a base there.

The contributing docs were a bit ahead of time ... but with a typo.

imports: try to use global imports unless it is a layering violation

To minimize the impact, do imports that violate the layering at runtime instead
of at import time.

lib: move webhelpers2 and friends to webutils

Gives less of the unfortunate use of helpers - especially in low level libs.

lib: consistently use webutils.url instead of h.url

Templates are still the valid use case for accessing through h.

Simplify mocking of url() function.

routing: separate url handling from routing - move it to webutils

This is a helper method relying on the thread local tg.request. We didn't have
a good place to put it. Now we do.

This (re)moves unfortunate dependencies to the routing module (which almost is
a controller).

py3: trivial renaming of .iteritems() to .items()

A bit like "2to3 -f dict", but we don't want list().

admin: drop dead code for system update notifications

If we want it, we probably want to do it in a different way.

ssh: show ssh URL on summary page

Original change has been heavily modified by Mads Kiilerich.

hooks: make sure push and pull hooks always are enabled

Don't put things in the database when we pretty much assume they always have
exact content, without any reasonable use case for customization.

templates, controllers: replace webhelpers.html.literal() with webhelpers.html.HTML() where possible

Usage of webhelpers.literal (h.literal) can be a problem when variables are
not correctly escaped. Luckily, this function can be avoided in several
cases.

Several users of the construct:
h.literal(_('..A..') % (..B..))

can be simplified if (..B..) just contains a call to h.link_to. In this
case, there is actually no need to use h.literal, because the object
returned by link_to is already a literal. It is sufficient to use
webhelpers.html.HTML() like so:
h.HTML(_('..A..')) % (..B..)

which is better because it will escape the '..A..' part instead of passing
it literally.

The need to wrap the '..A..' part in HTML() is to make sure the (escaped)
end result is not a plain string but a 'literal' to avoid double escaping
later.

See also the documentation:
https://docs.pylonsproject.org/projects/webhelpers/en/latest/modules/html/builder.html
"
When literal is used in a mixed expression containing both literals and
ordinary strings, it tries hard to escape the strings and return a
literal. However, this depends on which value has “control” of the
expression. literal seems to be able to take control with all
combinations of the + operator, but with % and join it must be on the
left side of the expression. So these all work:

"A" + literal("B")
literal(", ").join(["A", literal("B")])
literal("%s %s") % (16, literal("kg"))

But these return an ordinary string which is prone to double-escaping later:

"\n".join([literal('<span class="foo">Foo!</span>'), literal('Bar!')])
"%s %s" % (literal("16"), literal("&lt;em&gt;kg&lt;/em&gt;"))
"

This same escaping with 'HTML()' was already done by default in mako
templates for constructs like ${_("something")} that do not contain format
specifiers. When the translated string _does_ contain format specifiers, we
want to use the same escaping, but we have to do it explicit and earlier so
the escaping happens already when strings are inserted into the template
string.

settings: rework logic for flash message after repository scan

Make the code more readable by extracting added_msg and removed_msg away
from the h.flash() call and reindenting the logic.
There are no functional changes here.

These changes serve as preparatory work for a subsequent commit that will
change the logic.

admin: hooks: only flash 'Updated hooks' if there are changes

Clicking 'Save' on the hook administration page currently always renders the
flash message 'Updated hooks' even if nothing was changed.
This may be particularly confusing when the action you intended to do got an
error, e.g. adding a hook that already exists, adding a builtin hook, ...

Instead, compare the old and new value when editing a hook, and only save
and create the flash if they are different.

For this to be work correctly in test, the old value needs to be passed as
well like in the real situation, otherwise the 'zip' operation will return
an empty list.

admin: hooks: prevent editing of builtin hooks (issue #226)

Builtin hooks are supposed to be read-only, but it was still possible to
'add' a new hook with the same name as an existing built-in one, changing
its value.

admin: hooks: prevent creation of existing custom hook

Trying to add a hook that already exists does not currently give an error
but does not work.

Detect the situation and report via a flash.

hooks: rename hooks to reflect what they are doing

The names of our hooks should reflect what they are doing.
So, since pre_push and pre_pull only handle the locking, rename
pre_push->push_lock_handling and pre_pull->pull_lock_handling.

This imply a database migration step which must be run as described in the
upgrade documentation.

Also rename db.Ui class variables.

less: use .label and .label-* style instead of custom .*tag

This reduces the code and makes it more consistent.

Visualy this results in bold texts (which in case of small fonts is easier to read)
and a bit more padding left and right.

For a proper spacing, this requires to remove whitespaces before and after
the texts inside the labels.

Since the name changed from tag to label, also rename the variables and functions.

auth: refactor to introduce @LoginRequired(allow_default_user=True) and deprecate @NotAnonymous()

It was error prone that @LoginRequired defaulted to allow anonymous users (if
'default' user is enabled). See also 245b4e3abf39.

Refactor code to make it more explicit and safe by default: Deprecate
@NotAnonymous by making it the default of @LoginRequired. That will make it
safe by default.

To preserve same functionality, set allow_default_user=True in all the cases
where @LoginRequired was *not* followed by @NotAnonymous or other permission
checks - that was done with some script hacks:
sed -i 's/@LoginRequired(\(..*\))/@LoginRequired(\1, allow_default_user=True)/g' `hg mani`
sed -i 's/@LoginRequired()/@LoginRequired(allow_default_user=True)/g' `hg mani`
perl -0pi -e 's/\@LoginRequired\(allow_default_user=True\)\n\s*\@NotAnonymous\(\)/\@LoginRequired()/g' `hg mani`
perl -0pi -e 's/\@LoginRequired\(allow_default_user=True\)(\n\s*\@Has(Repo)?Permission)/\@LoginRequired()\1/g' `hg mani`

It has been reviewed that all uses of allow_default_user=True are in places
where the there indeed wasn't any checking for default user before. These may
or may not be correct, but now they are explicit and can be spotted and fixed.

The few remaining uses of @NotAnonymous should probably be removed somehow.

codingstyle: trivial whitespace fixes

Reported by flake8.

admin: better handling of rescan/delete and invalidate at the same time (Issue #252)

It would try to invalidate deleted repositories before rescan would delete them.

Instead, scan first, then invalidate.

Also, handle VCS error from invalidation and report success as a flash message.

controller: add missing import for VCSError

Found by pylint. Fix crash in error handling.

controllers: rename __before__ to _before in preparation of TurboGears2

__before__ in Pylons is called _before in TurboGears2. We can prepare this
rename already in Pylons-based Kallithea, so that the real TG2 migration
commit just changes the BaseController.

Since TurboGears2 _before can pass extra arguments, we add *args and
**kwargs parameters as well.

tg: minimize future diff by some mocking and replacing some pylons imports with tg

No actual tg dependency yet, just a temporary hack faking tg as an alias for
pylons.

Based on work by Alessandro Molina.

controllers: avoid setting request state in controller instances - set it in the thread global request variable

In TurboGears, controllers are singletons and we should avoid using instance
variables for any volatile data. Instead, use the "global thread local" request
context.

With everything in request, some use of c is dropped.

Note: kallithea/controllers/api/__init__.py still use instance variables that
will cause problems with TurboGears.

admin: better handling of rescan/delete and invalidate at the same time (Issue #252)

It would try to invalidate deleted repositories before rescan would delete them.

Instead, scan first, then invalidate.

Also, handle VCS error from invalidation and report success as a flash message.

settings: fix saving of visual settings of page size

After 19f15cde2cdb it was no longer to save page_size because formfill didn't
match. Back out a part of that changeset.

style: use Bootstrap compatible markup for alert

Based on work by Dominik Ruf.

db: always add to session in Setting.create_or_update

There's no use case for not adding the newly created Setting to the
SQLAlchemy session (and thus, once we commit, the database).

With this change, all the various "create" model methods add their
result to the session.

db: inline calls to get_all

This method saves basically no typing, compared to "query().all()".
Additionally, "all()" returns a list, forcing all records to be loaded
into a memory at the same time, but some callers just need to iterate
over the objects one at a time, in which case "query()" alone is more
efficient. In one case, the caller can even use "count()" and avoid
loading any objects from the database at all.

Turbogears2 migration: replace pylons.url by kallithea.config.routing.url

In preparation for the migration to Turbogears2, introduce a
kallithea.config.routing.url to replace pylons.url. The implementation is
basically the same: wrap around routes.url().

This change involves:
- a number of import statement changes
- fixing some tests in test_libs.py; to avoid duplication, the different
implementations of fake_url were grouped in one place.

This change was first proposed by Alessandro Molina in his initial
port. Following changes were made afterwards:
- move UrlGenerator from kallithea.lib.utils to kallithea.config.routing
- add documentation to UrlGenerator
- kallithea/lib/auth.py used url_for instead of url, for no apparent reason
so this was changed.
- fix libs tests
- rebase onto Pylons-based Kallithea first

celeryd: annotate tasks so they can be run directly without run_task

This also makes the system less forgiving about celery configuration problems
and thus easier to debug. I like that.

hg: drop pointless push_ssl configuration setting - if there is a risk push can be compromised, credentials can also easily be stolen for pull

Everybody should have a ssl-only setup now. Alternatively, there is a use case
for 'only anonymous traffic on ssl - all authentication and authenticated
traffic must be on ssl'. That can be done with proper web server configuration.

controllers: remove pointless comments

These comments are mostly trivial and sometimes wrong. We don't want to
maintain or pretend we do.

auth: remove HasPermissionAll and variants

First, find all calls to HasPermissionAll with only a single permission
given, and convert to equivalent calls to HasPermissionAny.

Next, observe that it's hard to envision situations requiring multiple
permissions (of the same scope: global/repo/repo group) to be satisfied.
Sufficiently hard that there are actually no such examples in the code.

Finally, considering that (should it ever be needed) HasPermissionAll
can be trivially built as a conjunction of HasPermissionAny calls (the
decorators, too) with only a small performance impact, simply remove
HasPermissionAll and related classes and functions.

admin: remove check for empty Ui table

The table is populated on database creation, and assumed to be populated
so many places in the code, we're unlikely to even reach this point if
it's empty. (E.g. web.push_ssl must be defined to push/pull/fetch both
Mercurial and Git repositories.)

db: fix Ui.get_by_key to also filter on section

Key alone is not enough to identify a Ui row.

db: remove superfluous Ui table session adds

Only newly created objects (and objects explicitly expunged) need to be
added to the SQLAlchemy session; any object returned from a database
query is already in the session.

cleanup: replace redirect with WebOb exceptions

All redirect does is to log "Generating 302 redirect" with logging the actual
location and raise a WebOb HTTPFound exception, and the logging is redundant,
as WebOb exceptions and their status codes are already logged.

Instead, just raise the exception directly, which is both explicit and simpler
(and finally, gets rid of "return redirect" which never really returns).

cache: when invalidating a cache, always just delete all 'live cache' records instead of marking them inactive

Keep it simple. Adding the record again might be slightly more expensive than
just updating the active flag but instead we get get a simpler model and
automatic cleanup without using the cache-keys paster command.

git: add option for forcing overwrite of Git hooks when remapping and rescanning the repositories. (Issue #153)

README file was updated to include some information on updating Git hooks after migrating to Kallithea.

cleanup: consistently use 'except ... as ...:'

Use the Python 2.6+ syntax instead of the old confusing 'except ..., ...'
syntax.

controllers: consistently use formfill.render with force_defaults=False

The inconsistency could cause confusion for developers. It seems to me like
force_defaults=False should be the default ... and apparently it was that in
older versions of formfill.

It could perhaps make sense for us to have a wrapper that added the defualt
values once, instead of repeating it all over ;-)

email templates: send text/plain part as well

This change adds text parts to the email templates, as
HTML and text templates may be way to different to be
handled automatically. Also, use proper dash-dash-space
signature separator, so the email clients recognise it
for sure.

Correct licensing information in individual files.

The top-level license file is now LICENSE.md.

Also, in various places where there should have been joint copyright holders
listed, a single copyright holder was listed. It does not appear easy to add
a link to a large list of copyright holders in these places, so it simply
refers to the fact that various authors hold copyright.

In future, if an easy method is discovered to link to a list from those
places, we should do so.

Finally, text is added to LICENSE.md to point to where the full list of
copyright holders is, and that Kallithea as a whole is GPLv3'd.

Second step in two-part process to rename directories.
This is the actual directory rename.