lib: move some template filter functions from utils2 to webutils

While quite Kallithea specific, we prefer to have these functions in webutils
where they soon can be exposed when templates don't need the whole helpers
module.

model/comment: drop unused fields in email_kwargs

Keep things as simple as possible.

model: remove unused 'subject' parameter of NotificationModel.create()

The subject of emails is determined with EmailNotificationModel._subj_map,
based on the notification type. The 'subject' parameter passed to
NotificationModel.create is completely unused.

Remove this parameter and update its callers, removing code that is now no
longer used.

model/comment: inline _get_notification_data

This method was only used in one place, from code which was very related.
Inline it to group things together and reduce complexity.

model/comment: extract notification-related code into a separate method

Preparation for grouping with _get_notification_data next, and keeping clear
separation between creating the comment itself, and creating the
notification.

model: changes toward import whole modules

*If* there should be circular dependencies, importing 'from' another module
could fail because the module at that time only was partially imported. That
had to be worked around by importing at runtime instead of globally.

Instead, try to always import whole modules. (But we should still try to avoid
cycles.)

lib: move webhelpers2 and friends to webutils

Gives less of the unfortunate use of helpers - especially in low level libs.

lib: move canonical_url & co to webutils

This gives less of the unfortunate use of helpers - especially in model.

lib: move extract_mentioned_users out of utils2

Avoid model.db reference to keep utils2.py independent of Kallithea classes.

comments: add changeset author in mail notification subject

In analogy to the handling of comments on pull requests, as changed in
commit d4061c6cc0e2.

py3: remove safe_unicode in places where it no longer is needed because all strings (except bytes) already *are* unicode strings

(The remaining safe_unicode calls are still needed and can't just be removed,
generally because we in these cases still have to convert from bytes to unicode
strings.)

tests: fix ordering of get_inline_comments output

Py3 would give different ordering.

model: comments: allow selective retrieval of inline comments

Allow retrieving inline comments of only a specified file and possibly line
number, by filtering at database level.

This will be useful when adding more code context in comment notification
emails.

model: move notification types from Notification to NotificationModel

This commit is part of the removal of the UI notification feature from
Kallithea, which is not deemed useful in its current form. Only email
notifications are preserved.

As there is no database storage of notifications anymore, the Notification
class will be removed. However, the notification type definitions are still
used for email notifications, and need to live somewhere. As creating
notifications is always passing via NotificationModel, it makes sense to
move the types there.

codingstyle: trivial whitespace fixes

Reported by flake8.

tg: minimize future diff by some mocking and replacing some pylons imports with tg

No actual tg dependency yet, just a temporary hack faking tg as an alias for
pylons.

Based on work by Alessandro Molina.

model: remove BaseModel class

The only remaining purpose of this class was to provide the "sa" field,
allowing a custom SQLAlchemy session to be used for model operations.
However, this never actually worked, nor was it used anywhere.

There's always a global SQLAlchemy session associated with the current
thread; using another session for a single function call does not make
any sense (as sessions cannot be mixed), unless the code works carefully
to ensure the two sessions (and all objects loaded from them) are kept
completely separate. Suffice to say that Kallithea does no such thing,
thus there's no need to pretend to support multiple concurrent sessions.

model: refactor listing of commenters

The ChangesetComment.get_users method was only called in two places,
with each call triggering independent codepaths inside the method.

model: move code from __init__.py to base.py

Having too much code, in particular too much imports, inside a package's
__init__.py is a recipe for circular imports, and considered bad practice in
Python [1]

Move out everything from kallithea/model/__init__.py to a new file
kallithea/model/base.py and adapt the existing imports.

[1] http://docs.python-guide.org/en/latest/writing/structure/#packages

model: change ChangesetComment 'user' to 'author'

Rename the 'user_id' field to 'author_id' and replace other references
to the comment 'user' throughout the model. The database column name
'user_id' remain unchanged for now; a later Alembic script can fix the
name of these and other columns to match their Python name.

db: PullRequest/Repository/RepoGroup/UserGroup: change 'user' to 'owner'

Rename the 'user' and 'user_id' fields on the four classes to something
more informative. The database column names remain unchanged for now;
a later Alembic script can fix the name of these and other columns to
match their Python name.

This might break rcextensions, though, and external scripts that use
the HTML form interface.

mails: restructure notification mail content

* Mention "The Thing" in the header, link to online version
* Show The Thing
* A dense summary of the essentials of the context below it

The html indentation is odd in order to make the next diff smaller.

The text version is based on:

for a in kallithea/templates/email_templates/*.html; do sed -e 's,<\([^%/>]\|/[^%>]\)*>,,g' -e 's,\.html",\.txt",g' -e 's,^ *,,g' -e 's/}/|n,unicode}/g' $a > ${a%%.html}.txt ; done

model: refactor and simplify _get_instance

_get_instance is a BaseModel method, but never uses "self". Instead
it takes a class argument, which indicates that it's better suited as
a classmethod on said classes.

Also rename to something more descriptive, remove leading underscore
since it's not a private API, and refactor for readability.

model: drop BaseModel.cls

It no longer serves a purpose.

model: inline superfluous "helper" methods

__get_changeset_status wasn't even used and has simply been removed.

Notification.__get_notification reimplemented BaseModel._get_instance,
warts and all, and has been replaced by calls to that method.

The rest were just needless indirection, and have been inlined.

notifications: tweak PR mail subject lines

The subject line is used for mail threading in gmail and can thus not be
changed without impacting users ... but now we do it.

* The tag '[Review]' is more spot-on than '[Added]'.
* The subject should be short so it fits on one line, so abbreviate "pull
request" to PR.
* Add the PR owner - convenient for filtering comments on own PRs from comments
on other PRs.

email: add relevant title to subject of emails

For comment and pullrequest mails, add changeset/pullrequest title (shortened
to first 50 characters of first line)

notifications: make more template strings available for mails

To be used soon ...

utils: improve extract_mentioned_users usefulness

Previously, extract_mentioned_users performed a regex search, converted
to a set to remove duplicates, converted that set back into a list, and
sorted the list on username, ignoring case. Every single caller of the
function then promptly took the carefully pruned and sorted result and
put it back into a set, rendering the entire exercise pointless.

In addition, every caller also resolved the usernames to database User
objects.

In this changeset, extract_mentioned_users is changed to return a set
of database User objects directly. A new extract_mentioned_usernames
function, which does only username extraction, is kept for the sole
purpose of providing a more testable interface (no database needed).

Bonus feature: The new extract_mentioned_users will prune non-existent
users, as well as the DEFAULT user. This means it is no longer possible
to @mention (and send notifications to) the DEFAULT user.

pullrequests: optimize iteration over reviewers - avoid fetching users one by one

.reviewers was mainly used for iteration and then dereferencing .user one by
one. That gave lots of queries and round trips to the database and was slow.

Instead, do something else. Either query directly or use a new method for
getting the list of reviewer users.

Reviewers will explicitly be shown in the order they have been added (assuming
database id's are monotonic).

pull requests: blame the right user when an admin adds reviewers to other peoples PRs

Don't send notifications from the PR owner.

pull requests: don't filter on repo when finding comments for a PR

Filtering on repo while finding comments for a PR would miss some. Fix that.

The existing data model is inconsistent; PRs already have two repos and
programming errors could easily make the one on a comment wrong. (Revision
comments do however need the repo reference.)

pull requests: call it 'owner' instead of 'author'

Prepare for making it possible to transfer ownership of PRs.

cleanup: check for None object identity in cases where that is what the 'contract' says

Avoid applying bool() on complex objects - it might return something unexpected
such as the key (which might be 0 and thus be false). Checking for None is more
safe and faster.

user: rename property username_and_name into full_name_and_username

In analogy of existing properties full_name and full_name_or_username,
rename username_and_name into full_name_and_username.

pullrequsts: really create a comment when creating a PR and setting status to 'under review'

14d75d4b03cd changed the pullrequest creation status change comment text from
redundant blurb to just an empty string. Empty comments are however only
created if changing the status ... and in this case we didn't tell the comment
creator that it actually was a status change.

As a result of this, we ended up breaking the implicit invariant that all
status updates have a comment. That showed up as errors dereferencing None when
displaying changesets.

pullrequests: add PullRequest.nice_id method

Currently, a pull request id is referenced as #5, and the '#' symbol is
fixed and repeated in several places. This commit adds a class method
make_nice_id, that returns a string reference to a pull request,
currently in the form '#5'.

This function could be overridden by an organization if they need references
in another form, for example PR-5.

The method nice_id uses this for the pullrequest's own id.

ChangesetCommentsModel: refactor duplicate code

Comments are now always ordered by creation date ... but that should not make
any difference.

ChangesetCommentsModel: cleanup of doc strings

Docstrings that are out of date is a bug. Docstrings that doesn't add any value
is thus a liability we would like to get rid of.

[Changeset extracted and modified by Mads Kiilerich.]

comments: avoid storing 'No comments' text when changing status

When a general comment (with or without status change) is added to a
changeset or pull request, and no text was added, Kallithea automatically
used 'No comments' as text. The stub text is added to the database as if it has
been entered by the user and it can thus not easily be identified as an
automatic comment.

This commit makes following changes:
- allow adding an empty comment to the database when there is a status
change. An empty comment without status change is ignored.
- do not add a stub text to the database, but generate it on demand
- the stub text is shown in italic font to differentiate it from user-entered
text

Currently there is a large amount of duplication between
controllers/changeset.py and controllers/pullrequests.py, which is to be
cleaned up in a later commit.

notifications: fix threading - use line number

TODO: Use a more stable link, something that includes the filename.

urls: introduce canonical_url config setting

All URLs that are shown or persisted or emailed will use this instead of the
current url.

This is convenient when the server has multiple names - for instance when
transitioning from one protocol or domain or hostname to another.

notifications: insert 'References' mail headers to help MUA threading

gmail is however fubar.

helpers: user email can be unset

Check for not being empty, not for ''.
Also, don't call person() with email, as it can't be caught properly.

Correct licensing information in individual files.

The top-level license file is now LICENSE.md.

Also, in various places where there should have been joint copyright holders
listed, a single copyright holder was listed. It does not appear easy to add
a link to a large list of copyright holders in these places, so it simply
refers to the fact that various authors hold copyright.

In future, if an easy method is discovered to link to a list from those
places, we should do so.

Finally, text is added to LICENSE.md to point to where the full list of
copyright holders is, and that Kallithea as a whole is GPLv3'd.

Second step in two-part process to rename directories.
This is the actual directory rename.