db: remove redundant unique constraint for repository groups
There's already a unique constraint on 'group_name' alone, no need for one on the combination of 'group_name' and 'group_parent_id'. (The extra constraint likely stems from confusion over what exactly goes into group_name; add comment to clarify that it is the full group path.)
db: remove redundant unique constraints from primary keys
Primary keys are always unique, both in the SQL standard and in SQLite, MySQL and PostgreSQL.
Setting unique=True cases SQLAlchemy to explicitly add additional, redundant UNIQUE indexes to the columns, which is at best needless metadata overhead, and at worst causes the database engine to waste time maintaining an extra index that serves no purpose.
As of the upgrade to version 1.0 in 3c4b6ddf6735, SQLAlchemy began to pass TEXT length limits to the database during table creation. Such limits are however not supported by SQLite, MySQL nor PostgreSQL, and while SQLite simply ignores it, it is a syntax error in PostgreSQL, breaking the creation of new PostgreSQL databases. With the lengths being unused and quite arbitrary, just drop them.
Hooks receive a line of the following format on standard input:
<old-value> SP <new-value> SP <ref-name> LF
where <old-value> is the old object name stored in the ref, <new-value> is the new object name to be stored in the ref and <ref-name> is the full name of the ref.
This means, we have to strip at least the LF in order to have a correct version of the ref name after the split. Also, when parsing the ref name itself, use all components but first instead of just second, as a ref name may have slashes in it.
Previously, failure to parse ref name correctly would lead to the following behaviour. A newly created repository with no commits pushed has HEAD set to refs/heads/master by default, even though there's no such ref in the repository yet. Upon first push, Kallithea rewrites this symbolic reference with a reference to a real branch.
However, due to a bug in ref name parsing, if a ref name had a slash, Kallithea would update HEAD to an invalid reference:
git push origin feature/branch
would rewrite HEAD to refs/heads/feature. All future attempts to work with this repository would fail because dulwich would complain it can't read HEAD as it is a directory.
db: ensure git hooks work when the repositories base path is a symlink
When a Git hook starts, it thinks its repo_path is its cwd. However, if the repositories base path is a symlink to a different location, base path won't match the location of the repository where the symlink will be resolved.
git: include an LF at the end of the service advertisement (Fixes #230)
This fixes hg-git/Dulwich and possibly other conservative Git clients, which do not ignore the absence of the LF.
The original comment was a guess based on reverse engineering the protocol not specified in the documentation yet at that moment. Now that the documentation exists and states this explicitly, just do as it says.
This function strips the repository name and any slashes from the URL path, leaving the protocol command, like info/refs or git-{receive,upload}-pack.
Using .split() and .strip() for this purpose isn't entirely reliable and is entirely unreadable, so it's better to write it out, even if it's a bit verbose.
Also, error out in the unlikely case the path doesn't start with a slash and the repository name; that shouldn't happen normally.
docs: use ., not source, as the user isn't guaranteed to have it
Even though bash is the default shell nearly everywhere, a restricted user running Kallithea doesn't necessarily have it as the default, so better use POSIX-compatible . command. Besided, it's also shorter.
setup: use modern bcrypt implementation instead of unsupported old one
py-bcrypt has been deprecated by bcrypt, and is no longer developed or supported.
bcrypt requires bytestrings instead of strings, use safe_str to ensure they're encoded before they're passed to bcrypt. Also, use check_pw to minimise the number of manual conversions and comparisons.
Installation of bcrypt will probably compile a C extension and require libffi-dev.
cleanup: get rid of dn as shortcut for os.path.dirname
We keep 'dirname' as shortcut despite having removed the 'join' shortcut: * the name is less ambiguous than 'join'. * dirname is often applied multiple times - spelling it out would be too verbose.
tests: introduce tests and reference dump for notification mails
The mails are dumped to a tracked html file: * changes shows up as diffs and are easy to spot and review * all mails can easily can be investigated in a browser and checked for content and consistency
The tests are mocking canonical_url because it has deep dependencies to pylons.url which requires (thread local?) environment setup that the tests doesn't have.
lock: fix for Mercurial 3.6+ - wrap hgweb to catch Locked exceptions from hooks
With Mercurial 3.6, the handling of WSGI responses changed. The hook exceptions are no longer raised directly when app(environ, start_response) is called so the 'except HTTPLockedRC as e' block in _handle_request (a few lines above ) does not work anymore because the exception happens later.
Therefore I created a wrapper class that can catch the exceptions.
This makes locking work again and fixes lock related tests like TestVCSOperations.test_clone_after_repo_was_locked_hg which expect certain output of the hg client in case of an HTTPLockedRC exception.
tests: set EMAIL for Git commit test_push_on_locked_repo_by_other_user_git - it _is_ necessary on some machines
7db1bcf1d95b too aggressively removed setting EMAIL (which was set in a way that didn't work on Windows).
On some machines the git commit in _add_files_and_push would fail with 'Please tell me who you are' and the actual test check of "Repository %s locked by user" would fail.
On other machines - also without any local git configuration - it works fine.
hooks: don't catch ImportError Exceptions in git hooks
These hooks are important. Being friendly and hiding errors is not the good kind of friendliness. It is better if they just fail if the Kallithea module can't be imported.
tests: Mercurial hooks must use ui.status for messages sent to the client
Mercurial changed so sys.stdout and sys.stderr now are intercepted instead of being sent to the client. That caused a regression that manual_test_vcs_operations.py test_push_new_file_hg caught - we no longer reported 'Repository size' to the user.
The issue was introduced by a Mercurial change, but the fix is backwards compatible with Mercurial 2.9.
The fix is to use ui.status everywhere. ui is a Mercurial thing, but handle_git_receive also creates a ui object for Git hooks so ui.status can be used there too.
hooks: if available, use sys.executable as executable for git hooks
Windows doesn't necessarily have "python2" available in $PATH, but we still want to make sure we don't end up invoking a python3. Using the absolute path seems more safe.
notifications: improve response time when number of notifications is large
NotificationsController always retrieved materialized list of all notifications in database, even to display only 10 of them. This is improved by feeding SQLAlchemy Query object directly to webhelpers.paginate.Page, avoiding eager load of all notifications.
(4.x seems like a more major upgrade - we stick to the 3 series for now.)
Note that select2-bootstrap.css *not* is upgraded. It is a separate thing (see LICENSE.md) and currently unused. (Select2 now has similar functionality built in - that should perhaps be used with Bootstrap instead.)
This creates a separate "Upgrading Kallithea" section, instead of trying (and failing) to repeat the same information for each installation page. Trying to maintain upgrade instructions for each installation method was perhaps overly ambitious. In practice, only the Linux instructions were up-to-date.
(Incidentally, this is kind of a throwback to Kallithea 0.1, which also had a separate upgrade section.)
templates: use a better class name for an enabled toggle button
"btn-success" is rather a poor choice for a class applied when the button is active, "active" is better for that. A side effect of the change is that "Enabled" is now shown in bold.
Use markup from Web Accessibility Initiative - it is default in Bootstrap examples and "comes for free" but is currently not used, complete, or tested.
Use Bootstrap classes for dropdowns. Use explicit Bootstrap-style carets instead of :after preudoclass. Use Bootstrap class for badges. Introduce Bootstrap data-toggle properties - currently unused.
cleanup: remove dead code and templates related to 'Switch To'
a33448d81f70 introduced a new 'Switch To' functionality and removed the last reference to branch_tag_switcher and thus to switch_to_list.html and the tags, bookmarks and branches pages.
We don't need these pages ... and if we do, we will implement them differently. They are mostly YUI datatables with compare links that only can compare within the same kind of revision names.
pytest migration: rename TestControllerPytest back to TestController
The name TestControllerPytest was introduced to allow a temporary situation where nose/unittest and pytest-based tests could coexist. This situation is now over, so the base test class can be renamed again.
pytest migration: move init_stack under TestControllerPytest
Avoid top-level methods if they could be placed under classes. The init_stack method is only used from TestControllerPytest so it makes sense to scope it under that class.
pytest migration: backout declassification of remove_all_notifications
In order to accomodate both nose/unittest and pytest at the same time, method remove_all_notifications has been extracted from BaseTestCase in commit 37d713674f63. Now that nose/unittest is no longer used, we can backout this change again.
pytest migration: __init__: switch to standard assert statements
Use unittest2pytest to replace unittest-style assert statements (e.g. assertEqual) with standard Python assert statements to benefit from pytest's improved reporting on assert failures.
pytest migration: functional: switch to standard assert statements
Use unittest2pytest to replace unittest-style assert statements (e.g. assertEqual) with standard Python assert statements to benefit from pytest's improved reporting on assert failures.
The conversion by unittest2pytest was correct, except for line wrapping problems.