changelog: fix crash on display of empty commit messages

Also handle commit messages with leading whitespace.

.splitlines() is different from .split('\n') and will give an empty list when
given an empty string.

Problem reported and initial patch proposed by Brett Smith.

lib: drop is_hg and is_git

It is just as simple to be explicit.

html: put 'use strict' on separate lines

use.py:
import re
import sys
for fn in sys.argv[1:]:
with open(fn) as f:
s = f.read()
s = re.sub(r'''(<script>)('use strict';)\n( *)''', r'''\1\n\3\2\n\3''', s)
with open(fn, 'w') as f:
f.write(s)

python use.py $(hg loc 'kallithea/templates/**.html')

js: fix missing var declaration of AJAX_COMMENT_URL and AJAX_COMMENT_DELETE_URL (Issue #362)

Tech debt stroke back when fb9550946c26 cleaned things up.

Eslint wearnings for these variables had incorrectly been manually ignored.

js: use strict ... and fix the problems it points out

"use strict" gives stricter checks, both statically and at runtime. The
strictness tightens up the code and prevents some kinds of problems.

The <script> tag addition might not be pretty, but has consistently been added
with:

sed -i 's,<script>$,&'"'"'use strict'"'"';,g' `hg loc '*.html'`

js: omit redundant type="text/javascript"

We already used plain '<script>' in several places.

https://www.w3.org/TR/html52/semantics-scripting.html#element-attrdef-script-type :
"Authors should omit the attribute, instead of redundantly giving a JavaScript
MIME type."

py3: remove safe_unicode in places where it no longer is needed because all strings (except bytes) already *are* unicode strings

(The remaining safe_unicode calls are still needed and can't just be removed,
generally because we in these cases still have to convert from bytes to unicode
strings.)

clone_url: always pass a clone_uri_tmpl, with Repository.DEFAULT_CLONE_URI as last resort

clone_url() had a layering violation of using c.clone_uri_tmpl . This
refactoring now makes it clear that this only was used from
PullRequest.__json__(), so move the hack there and simplify it.

pullrequests: in PR update list, only show first line of commit messages

Make it less overwhelming, and more like other changeset lists.

pullrequests: prevent XSS in 'Potential Reviewers' list when first and last names cannot be trusted

If a user first or last name contains javascript, these fields need proper
escaping to avoid XSS attacks.

An example scenario is:
- the malicious user creates a repository. This will cause this user to be
listed automatically under 'Potential Reviewers' in pull requests.
- another user creates a pull request on that repository and selects the
suggested reviewer from the 'Potential Reviewers' list.

Reported by Bob Hogg <wombat@rwhogg.site> (thanks!).


Technical note: the other caller of addReviewMember in base.js itself does
_not_ need to be adapted to escape the input values, because the input
values (oData) are _already_ escaped (by the YUI framework).

pull requests: urlify title to make issue references clickable

Issue references and commit hashes are already clickable in:
- pull request descriptions,
- commit messages
- changelog

but not yet in:
- pull request titles
- pull request listings

This commit handles the former case.

pullrequests: don't show empty "additional changesets" (issue #280)

When opening a pullrequest on a revision range including the tipmost
revision, and then pushing a new revision on top of that, the PR page shows:
This pullrequest can be updated with changes on ...:
and then nothing.

This is because the available revisions looped over are in 'avail_cs' but
the guard around the loop is checking on 'avail_revs'. The former, while
derived from avail_revs, can become empty under circumstances like this one.

Fix the problem by changing the guard checking avail_cs rather than
avail_revs, and making sure the printed message is aligned to 'No additional
changes found'.

pullrequest: resize graph in list of available changesets when page is resized

02aef1484695 introduced dynamic width of the graph of available changesets. We
thus also need resizing of the graph when the page is resized.

templates: remove references to "breadcrumbs" class after special styling was dropped in 23a8484

panel-title and breadcrumbs() could probably also be removed.

pullrequests: fix position of edit button

The edit button belongs to the label, so lets put it there.
We then do't need the #pr-edit-btn style any more.

pullrequest: avoid unnecessary text

Less text is easier/fast to read.

pullrequest: change 'add repo owner' link to a button

Things that cause actions should be formatted as buttons, instead of simple links.

less: use .label and .label-* style instead of custom .*tag

This reduces the code and makes it more consistent.

Visualy this results in bold texts (which in case of small fonts is easier to read)
and a bit more padding left and right.

For a proper spacing, this requires to remove whitespaces before and after
the texts inside the labels.

Since the name changed from tag to label, also rename the variables and functions.

pullrequest: remove 'Still not reviewed by'

This is redundant. You can see the same information on the right, in the list
of reviewers.

pullrequest: make the explaining message for 'Next iteration' a alert-info

...for better visibility.

Also remove obsolete 'msg-div' class.

pullrequest: avoid empty lines in reviewer list after long lines

Whitespace between the reviewer name and a hidden input element could cause
line wrapping and leave an apparently empty line.

Work around that by putting the empty element at the beginning of the line.

templates: change .node elements to <span>

This way we don't need to make them 'float: left'

templates: remove redundant pull-left and pull-right for changeset progress bars

"float: left" and "float: right" for these elements are already set in
kallithea-diff.less.

Aligns it with compare_diff.html which already didn't use "pull" on "node" and
"changes".

users: remove code that is unused after most autocomplete has been switched to ajax

@mention support still require _USERS_AC_DATA as a global variable.

autocomplete: use select2 when selecting users to add as reviewers to PRs

This is a minimal change to PullRequestAutoComplete, inspired by Dominik Ruf.

The UX is slightly different than before, with select2 putting up an extra
input field, already before typing anything.

We use minimumInputLength 1 to get the same behaviour as before and avoid
displaying all users in a list.

The select2 widget is reused for adding more users, so when a user is selected,
we process it, close the widget, and abort the selection, waiting for the user
to enter another user name.

autocomplete: use select2 when selecting owner of repos and PRs

This is a minimal change to SimpleUserAutoComplete, inspired by Dominik Ruf.

The UX is slightly different than before, with select2 putting up an extra
input field, already before typing anything.

We use minimumInputLength 1 to get the same behaviour as before and avoid
displaying all users in a list. This field will always have an old value and
the placeholder is thus never used, but we show it instead of the default
"Please enter 1 or more character" message.

The initSelection iteration is not pretty, but no more complex than what
happens when filtering the user list, and it has the advantage of giving a nice
name/gravatar display of the current user.

autocompletion: drop explicit container elements - just create them when necessary

The previous wide styling of the @mention selection list didn't look pretty -
just use the default width.

Inspired by select2 port by Dominik Ruf.

style: fix display of the PR graph of changesets available for update

88ce09daea37 dropped explicit html styling - fix it by adding css styling.

style: drop form-horizontal - our style is much closer to plain Bootstrap forms

form-horizontal is made for grid markup. It give form-groups a negative margin
to break out of the grid ... but the way we use it for settings, we have to do
weird things to undo that. The default styling for forms is much closer to what
we want. It looks ok without our custom styling and is easier to style to our
style.

If we want grid markup with form-horizontal, it would be correct to
re-introduce both at once.

style: rework review status icon handling

The styling had a reference to a '.container .changeset-status-ico' which might
collide with use of the Bootstrap 'container' class.

Investigation of that lead to complete removal of all changeset-status-ico and
changeset-status-container, plus related simplification of templates.

pullrequests: fix "additional changes" js error - make sure all cs tables have a first column to align with (Issue #274)

A slight simplification of 16234f629cfb and fixing a JavaScript failure for
unauthenticated visits to PRs with pending available changesets.

nextFirstincell would be undefined becuase the "Current revision - no change"
line didn't have any elements in td, and nextFirstincell.offsetTop would thus
fail.

Fixed by adding a span around the text.

Also clarify that any element is fine - there is no need to check for
visibility.

templates: properly escape inline JavaScript values

TLDR: Kallithea has issues with escaping values for use in inline JS.
Despite judicious poking of the code, no actual security vulnerabilities
have been found, just lots of corner-case bugs. This patch fixes those,
and hardens the code against actual security issues.

The long version:

To embed a Python value (typically a 'unicode' plain-text value) in a
larger file, it must be escaped in a context specific manner. Example:

>>> s = u'<script>alert("It\'s a trap!");</script>'

1) Escaped for insertion into HTML element context

>>> print cgi.escape(s)
&lt;script&gt;alert("It's a trap!");&lt;/script&gt;

2) Escaped for insertion into HTML element or attribute context

>>> print h.escape(s)
&lt;script&gt;alert(&#34;It&#39;s a trap!&#34;);&lt;/script&gt;

This is the default Mako escaping, as usually used by Kallithea.

3) Encoded as JSON

>>> print json.dumps(s)
"<script>alert(\"It's a trap!\");</script>"

4) Escaped for insertion into a JavaScript file

>>> print '(' + json.dumps(s) + ')'
("<script>alert(\"It's a trap!\");</script>")

The parentheses are not actually required for strings, but may be needed
to avoid syntax errors if the value is a number or dict (object).

5) Escaped for insertion into a HTML inline <script> element

>>> print h.js(s)
("\x3cscript\x3ealert(\"It's a trap!\");\x3c/script\x3e")

Here, we need to combine JS and HTML escaping, further complicated by
the fact that "<script>" tag contents can either be parsed in XHTML mode
(in which case '<', '>' and '&' must additionally be XML escaped) or
HTML mode (in which case '</script>' must be escaped, but not using HTML
escaping, which is not available in HTML "<script>" tags). Therefore,
the XML special characters (which can only occur in string literals) are
escaped using JavaScript string literal escape sequences.

(This, incidentally, is why modern web security best practices ban all
use of inline JavaScript...)

Unsurprisingly, Kallithea does not do (5) correctly. In most cases,
Kallithea might slap a pair of single quotes around the HTML escaped
Python value. A typical benign example:

$('#child_link').html('${_('No revisions')}');

This works in English, but if a localized version of the string contains
an apostrophe, the result will be broken JavaScript. In the more severe
cases, where the text is user controllable, it leaves the door open to
injections. In this example, the script inserts the string as HTML, so
Mako's implicit HTML escaping makes sense; but in many other cases, HTML
escaping is actually an error, because the value is not used by the
script in an HTML context.

The good news is that the HTML escaping thwarts attempts at XSS, since
it's impossible to inject syntactically valid JavaScript of any useful
complexity. It does allow JavaScript errors and gibberish to appear on
the page, though.

In these cases, the escaping has been fixed to use either the new 'h.js'
helper, which does JavaScript escaping (but not HTML escaping), OR the
new 'h.jshtml' helper (which does both), in those cases where it was
unclear if the value might be used (by the script) in an HTML context.
Some of these can probably be "relaxed" from h.jshtml to h.js later, but
for now, using h.jshtml fixes escaping and doesn't introduce new errors.

In a few places, Kallithea JSON encodes values in the controller, then
inserts the JSON (without any further escaping) into <script> tags. This
is also wrong, and carries actual risk of XSS vulnerabilities. However,
in all cases, security vulnerabilities were narrowly avoided due to other
filtering in Kallithea. (E.g. many special characters are banned from
appearing in usernames.) In these cases, the escaping has been fixed
and moved to the template, making it immediately visible that proper
escaping has been performed.

Mini-FAQ (frequently anticipated questions):

Q: Why do everything in one big, hard to review patch?
Q: Why add escaping in specific case FOO, it doesn't seem needed?

Because the goal here is to have "escape everywhere" as the default
policy, rather than identifying individual bugs and fixing them one
by one by adding escaping where needed. As such, this patch surely
introduces a lot of needless escaping. This is no different from
how Mako/Pylons HTML escape everything by default, even when not
needed: it's errs on the side of needless work, to prevent erring
on the side of skipping required (and security critical) work.

As for reviewability, the most important thing to notice is not where
escaping has been introduced, but any places where it might have been
missed (or where h.jshtml is needed, but h.js is used).

Q: The added escaping is kinda verbose/ugly.

That is not a question, but yes, I agree. Hopefully it'll encourage us
to move away from inline JavaScript altogether. That's a significantly
larger job, though; with luck this patch will keep us safe and secure
until such a time as we can implement the real fix.

Q: Why not use Mako filter syntax ("${val|h.js}")?

Because of long-standing Mako bug #140, preventing use of 'h' in
filters.

Q: Why not work around bug #140, or even use straight "${val|js}"?

Because Mako still applies the default h.escape filter before the
explicitly specified filters.

Q: Where do we go from here?

Longer term, we should stop doing variable expansions in script blocks,
and instead pass data to JS via e.g. data attributes, or asynchronously
using AJAX calls. Once we've done that, we can remove inline JavaScript
altogether in favor of separate script files, and set a strict Content
Security Policy explicitly blocking inline scripting, and thus also the
most common kind of cross-site scripting attack.

auth: simplify repository permission checks

In practice, Kallithea has the 'repository.admin' permission imply the
'repository.write' permission, which again implies 'repository.read'.

This codifies/enforces this practice by replacing HasRepoPermissionAny
"perm function" with the new HasRepositoryLevel function, reducing the
risk of errors and saving quite a lot of typing.

style: use more Bootstrap pull-left and pull-right

Based on work by Dominik Ruf.

style: introduce Bootstrap markup for list-unstyled

Based on work by Dominik Ruf.

controllers: avoid setting request state in controller instances - set it in the thread global request variable

In TurboGears, controllers are singletons and we should avoid using instance
variables for any volatile data. Instead, use the "global thread local" request
context.

With everything in request, some use of c is dropped.

Note: kallithea/controllers/api/__init__.py still use instance variables that
will cause problems with TurboGears.

style: use panel, panel-heading, panel-title, panel-body and settings

This imply lots of tweaking of header handling and panel spacing.

Not converted yet: codeblock code-header code-body.

Based on work by Dominik Ruf.

style: apply Bootstrap-compatible table.table class, now when the conflicting div.table styling is gone

Based on work by Dominik Ruf.

style: rename div.table to Bootstrap .panel-body

Based on work by Dominik Ruf.

style: introduce pull-left and pull-right in more places

Based on work by Dominik Ruf.

style: refactor panel headings - use pull-left and pull-right and introduce clearfix like Bootstrap

Based on work by Dominik Ruf.

style: drop unnecessary <ul> markup

Based on work by Dominik Ruf.

style: introduce h2 h3 h4 h5 for emphasis and drop some explicit styling

Based on work by Dominik Ruf.

templates: textarea doesn't have a size attribute - drop it!

We could use the rows attribute ... but so far it has worked without ...

style: fix pull request styling

My recent changes broke it. Add some band aid to help it. The pull request page
might not be a perfect fit for Bootstrap forms, but that is a better
approximation than nothing.

style: add missing 'form-control' markup

Based on work by Dominik Ruf.

style: drop 'textarea' class inside 'form-group'

Temporarily, just apply styling to div inside form-group. Also, 'radios' is gone.

Based on work by Dominik Ruf.

style: drop 'input' class inside 'form-group'

As long as we use the old styling, just apply styling to div inside form-group.

Based on work by Dominik Ruf.

style: drop dead 'field' class references

The last references to the class does not match anything essential, and all
css styles can thus also be removed.

Based on work by Dominik Ruf.

templates: use Bootstrap compatible 'form-control' name instead of 'medium' & co

In Bootstrap, form controls tend to be 100%.

style: use span for reviewers instead of div

Based on Bootstrap work by Dominik Ruf.

style: use Bootstrap compatible data-toggle="tooltip" markup

Based on work by Dominik Ruf and Andrew Shadura.

style: refactor form label styling to prepare for Bootstrap and get rid of wrapping with 'label' class

Based on work by Dominik Ruf.

style: use span for tag/bookmark tags ... and make it inline-block to avoid wrapping

Based on work by Dominik Ruf.

style: adjust button sizes to be more like Bootstrap and change some btn-sm to btn-xs

Based on work by Dominik Ruf.

style: don't use div.action_button for PR reviewers

This is an odd button. The remaining action buttons are more regular.

Based on work by Dominik Ruf.

style: in preparation for bootstrap, replace kallithea title class with bootstrap compatible panel-heading

This is a subset of a bigger changeset. The subset was extracted by Mads Kiilerich, mostly by:

sed -i 's,<div class="title\>,<div class="panel-heading,g' `hg mani`
sed -i 's,\<div\.title\>,div.panel-heading,g' kallithea/public/css/style.css

style: in preparation for bootstrap, replace kallithea box with bootstrap compatible panel

This is a subset of a bigger changeset. The subset was extracted by Mads Kiilerich, mostly by:

sed -i \
-e 's,<div\(.*\) class="box",<div\1 class="panel panel-primary",g' \
`hg mani`

style: in preparation for bootstrap, use bootstrap compatible button class names

Give all buttons a styling (default, success, danger, warning) and rename the sizes to sm and xs.

This is a subset of a bigger changeset. The subset was extracted by Mads Kiilerich, mostly by:

sed -i \
-e 's,btn btn-small,btn btn-default btn-sm,g' \
-e 's,btn btn-mini,btn btn-default btn-xs,g' \
-e 's,btn-default btn-\(xs\|sm\) btn-\(success\|danger\|warning\),btn-\2 btn-\1,g' \
-e 's,class_="btn",class_="btn btn-default",g' \
`hg mani`

cleanup: use obj.foo_id instead of obj.foo.foo_id

Don't use constructs like obj.user.user_id when obj.user_id works
equally well (and potentially saves a database load).

db: PullRequest/Repository/RepoGroup/UserGroup: change 'user' to 'owner'

Rename the 'user' and 'user_id' fields on the four classes to something
more informative. The database column names remain unchanged for now;
a later Alembic script can fix the name of these and other columns to
match their Python name.

This might break rcextensions, though, and external scripts that use
the HTML form interface.

changelog: show merges as semi opaque in more places

Follow up to 8c5d1e94f9ee.

PRs available for updates (those that have a radio button) are not greyed out -
that would make it look too much like they were disabled.

These merges should perhaps be shown in a different way - perhaps not greying
the whole line. But for now, this makes it more consistent.

diff: correct handling of links to old filename in renames

There were links to the file at the parent revision ... but if the file had
been renamed, it used the wrong name.

diffs: fold diff_block_simple (used by PR and compare) into diff_block

Change to using the same datamodel and enjoy the reduced amount of code
duplication.

diff: minor cleanups

More consistency and preparing for later changes.

helpers: merge urlify_commit into urlify_text

More reuse - they are already almost doing the same.

test_urlify_test is updated to use the new repo_name parameter and urlify
hashes.

pullrequests: use monospace for hashes - columns of hashes look messy when they have different lenght

17f9f921a538 was insufficient.

pullrequests: better handling of Mercurial pullrequests with missing revisions - don't crash

Trying to display a Mercurial PR with missing changesets could give a crash
when trying to compute available updates after 3f646f7bac39 did that
c.cs_ranges could be empty. That would normally not happen on Mercurial, but
could happen when restoring an old filesystem backup ... or when using strip on
the server.

pullrequests: use css for description instead of <code> and explicit styling

b60e58094fb7 introduced use of <code>, but that is semantically incorrect.

pullrequests: show tags in lists of included and available changesets

Further improvement: Also show bookmarks and other names (preferably by using
some helper function/template instead of duplicating code). It would perhaps
also be better to avoid using floating.

pullrequests: restore display of "Current revision - no change" at the bottom of the list of available updates

Helps the user to understand that something can be selected.

comments: change comment formatting to plain text instead of rst

There might be value in enabling rich markup (especially if it is markdown
instead of rst) ... or it might be a waste of time. We might revisit that
later.

But either way: Changing to plain text makes it more feasible to do markup of
hashes and issues - for now that is more important than rtf.

diff: move looping over files into diff_block_simple

Make it slightly more like regular diff_block and make the next changes
possible.