html: put 'use strict' on separate lines

use.py:
import re
import sys
for fn in sys.argv[1:]:
with open(fn) as f:
s = f.read()
s = re.sub(r'''(<script>)('use strict';)\n( *)''', r'''\1\n\3\2\n\3''', s)
with open(fn, 'w') as f:
f.write(s)

python use.py $(hg loc 'kallithea/templates/**.html')

front-end: add eslint-plugin-html as dependency as introduced in .eslintrc.js in 4d36432bf705

Usage example:
hg up -cr.
sed -i -e 's/\${[^{}]*\({[^{}]*}[^{}]*\)*}/""/g' -e 's/%\(if\|else\|endif\|for\|endfor\)\>.*//g' -e 's/##.*//g' $(hg loc 'kallithea/templates/**.html')
vim kallithea/templates/pullrequests/pullrequest.html +139 # blank out the multi line 'var url = ${}'
( cd kallithea/front-end/ && node_modules/.bin/eslint $(hg loc 'kallithea/templates/**.html')) | tee l
hg up -Cr.

( sed -n 's/^function \([^(]*\).*/\1/gp' kallithea/public/js/base.js; sed -n 's/.* var \([^ ]*\) =.*/\1/p' kallithea/templates/base/root.html; echo pyroutes; echo CodeMirror; echo Select2; echo BranchRenderer;)|while read x; do echo $x; sed -i "/error .$x. is not defined /d" l; done; cat l

js: use strict ... and fix the problems it points out

"use strict" gives stricter checks, both statically and at runtime. The
strictness tightens up the code and prevents some kinds of problems.

The <script> tag addition might not be pretty, but has consistently been added
with:

sed -i 's,<script>$,&'"'"'use strict'"'"';,g' `hg loc '*.html'`

js: omit redundant type="text/javascript"

We already used plain '<script>' in several places.

https://www.w3.org/TR/html52/semantics-scripting.html#element-attrdef-script-type :
"Authors should omit the attribute, instead of redundantly giving a JavaScript
MIME type."

files: use the web browsers built-in js history instead of native.history.js

The history API is available in all web browsers we support.

window.history.pushState is called to register a state that we can go
back/forward to. (But contrary to native.history.js, it doesn't do any
immediate processing of the state and doesn't actually navigate to it.)

When navigation occurs, we get the popstate event and invoke load_state to
actually load the state.

front-end: use At.js for MentionsAutoComplete

We want to get rid of YUI, and select2 is not well suited for this purpose.
So use At.js, which is made just for this use case.

Original implementation was modified by Mads Kiilerich.

front-end: Use DataTables from node_modules and stop bundling it

There is no minimized version in the npm package. For now, we just use the raw
one. We can minimize later.

front-end: Use select2 from node_modules and stop bundling it

select2 3.5.4 was added in 304e83e9bcde ... but the latest npm release in the 3
series is 3.5.1, so we use that one instead. We should probably upgrade to the
4 series.

The select2 images were not in the location the generated css pointed - now we
copy them from node_modules to the right location, next to the generated css.

Note: this will drop 190cb30841de "branches: fix performance of branch
selectors with many branches - only show the first 200 results" ... but it
should no longer be relevant now when we use server side filtering.

15e507047bae introduced select2-bootstrap.css - it is not clear what version
was used, but we use the latest 1.4.6 which also is very old.

less: include pygments.css in main.less

For now, keep the old css location even though it now is a source file and
shouldn't be in public.

templates: mark site as IE10+ compatible to bypass Compatibility mode

Internet Explorer 11 (and probably other versions) seems to ship with the
setting "Display intranet sites in Compatibility View" enabled by default.
This compatibility view means that modern elements of CSS and JavaScript are
not (well) support and makes Kallithea look very bad and become unusable.

This is a problem when hosting Kallithea in a corporate environment where
its URL is detected as 'the intranet'.

Solve the issue by explicitly marking Kallithea to be compatible with IE10,
so that Internet Explorer 10 and above will show Kallithea correctly in
Intranet environments.

less: use upstream's dataTables.bootstrap style instead of jquery.dataTables.css

See https://datatables.net/examples/styling/bootstrap.html .

- use official dataTables.bootstrap.css and dataTables.bootstrap.js
- add datatable less style block to use kallithea font icons
- move dataTables related style to this new block to have them in one place
- remove style that is obsolete, overrides or gets overriden by dataTables.bootstrap.css

add icons and config to satisfy http://realfavicongenerator.net/

Without this, on some paltforms, the favicon is not used correctly.

less: build custom Bootstrap from bootstrap.less instead of using plain upstream version

- add current version of bootstrap to package.json dependencies
- import bootstrap.less in main.less
- remove bootstrap.css from root.html (the file will be removed later)

bootstrap.js is still handled as before.

less: change markup and styling for the logo shown in the top left corner

Based on work by Dominik Ruf, but in this changeset, the goal is just to
improve markup and styling while keeping exactly the same look and feel. In
that process, it also fixes some browser specific issues that gave odd
spacing/positioning.

All styling (especially the logo image URL) is moved to CSS/LESS, using
background-image styling instead of an img tag.

The logo image is made a :before on the site branding text and will usually
have to align with this text, so we use inline-block to align to the baseline.

We set the right amount of negative margin-bottom so the image can go below the
baseline.

We use a "big enough" negative margin-top to allow the image to bleed outside
the line height and into the 12px top padding.

We assume the background image doesn't need cropping or panning and do thus not
set background-size, background-position, or background-repeat.

js: use ajax requests for select2 autocomplete

When you have a big user base, with thousends of users, always using the whole
dataset makes the UI slow.

This will replace kallithea/model/repo.py get_users_js and get_user_groups_js
which were used to inline the full list of users and groups in the document.
Instead, it will expose a json service for doing the completion.

When using the autocomplete, there might be multiple ajax requests, but tests
with a userbase > 9000 showed no problems.
And keep in mind, that although we now make multiple requests (one for every
character) that
- the autocomplete is not used that often
- the requests are quite cheap
- most importanly, we no longer need to calculate the user list/group list if
the user doesn't use the autocomplete

Users and groups are still passed as parameters to the javascript functions -
they will be removed later.

changeset: make parent/child navigation activation and javascript reusable

The javascript code is moved from changeset.html to base.js with minimal
changes. The markup code is moved to base.html.

The parameters ajax url and repo_name are stored as data properties on the link
element.

Use _TM['No revisions'] for translation of text.

js: don't show number of members of groups when doing autocomplete

This is done to make the database queries cheaper.

comments: display comment previews while submitting

Instead of just saying 'Submitting' and not showing any progress to
the user until the comment has been accepted by the server, show a
preview of the comment above the comment box in a way which is makes
it obvious to the user the comment is being submitted. Apart from
that, also clear the comment box so that a repeated clicking the
submit button doesn't result in a duplicate comment.

The preview doesn't highlight URLs or support @mentions or *bold*,
which is a good enough approximation in this case. When/if we (re-)add
the rST/Markdown support, we will need a client-side parser for the
syntax we choose.

When the submission fails, display a message and offer the user to
retry or cancel the submission.

autocomplete: use select2 when selecting users or groups to give permissions

This is a minimal change to MembersAutoComplete, inspired by Dominik Ruf.

The UX is slightly different than before, with select2 putting up an extra
input field, already before typing anything.

We use minimumInputLength 1 to get the same behaviour as before and avoid
displaying all users in a list.

style: use plain Bootstrap with existing CSS styling

This is an intermediate step, trying to make our markup stay as close to plain
Bootstrap as possible, minimize the amount of customizations, and accept small
visual differences.

More long term, we will try to move to generated CSS with Bootstrap and our
customizations.

templates: properly escape inline JavaScript values

TLDR: Kallithea has issues with escaping values for use in inline JS.
Despite judicious poking of the code, no actual security vulnerabilities
have been found, just lots of corner-case bugs. This patch fixes those,
and hardens the code against actual security issues.

The long version:

To embed a Python value (typically a 'unicode' plain-text value) in a
larger file, it must be escaped in a context specific manner. Example:

>>> s = u'<script>alert("It\'s a trap!");</script>'

1) Escaped for insertion into HTML element context

>>> print cgi.escape(s)
&lt;script&gt;alert("It's a trap!");&lt;/script&gt;

2) Escaped for insertion into HTML element or attribute context

>>> print h.escape(s)
&lt;script&gt;alert(&#34;It&#39;s a trap!&#34;);&lt;/script&gt;

This is the default Mako escaping, as usually used by Kallithea.

3) Encoded as JSON

>>> print json.dumps(s)
"<script>alert(\"It's a trap!\");</script>"

4) Escaped for insertion into a JavaScript file

>>> print '(' + json.dumps(s) + ')'
("<script>alert(\"It's a trap!\");</script>")

The parentheses are not actually required for strings, but may be needed
to avoid syntax errors if the value is a number or dict (object).

5) Escaped for insertion into a HTML inline <script> element

>>> print h.js(s)
("\x3cscript\x3ealert(\"It's a trap!\");\x3c/script\x3e")

Here, we need to combine JS and HTML escaping, further complicated by
the fact that "<script>" tag contents can either be parsed in XHTML mode
(in which case '<', '>' and '&' must additionally be XML escaped) or
HTML mode (in which case '</script>' must be escaped, but not using HTML
escaping, which is not available in HTML "<script>" tags). Therefore,
the XML special characters (which can only occur in string literals) are
escaped using JavaScript string literal escape sequences.

(This, incidentally, is why modern web security best practices ban all
use of inline JavaScript...)

Unsurprisingly, Kallithea does not do (5) correctly. In most cases,
Kallithea might slap a pair of single quotes around the HTML escaped
Python value. A typical benign example:

$('#child_link').html('${_('No revisions')}');

This works in English, but if a localized version of the string contains
an apostrophe, the result will be broken JavaScript. In the more severe
cases, where the text is user controllable, it leaves the door open to
injections. In this example, the script inserts the string as HTML, so
Mako's implicit HTML escaping makes sense; but in many other cases, HTML
escaping is actually an error, because the value is not used by the
script in an HTML context.

The good news is that the HTML escaping thwarts attempts at XSS, since
it's impossible to inject syntactically valid JavaScript of any useful
complexity. It does allow JavaScript errors and gibberish to appear on
the page, though.

In these cases, the escaping has been fixed to use either the new 'h.js'
helper, which does JavaScript escaping (but not HTML escaping), OR the
new 'h.jshtml' helper (which does both), in those cases where it was
unclear if the value might be used (by the script) in an HTML context.
Some of these can probably be "relaxed" from h.jshtml to h.js later, but
for now, using h.jshtml fixes escaping and doesn't introduce new errors.

In a few places, Kallithea JSON encodes values in the controller, then
inserts the JSON (without any further escaping) into <script> tags. This
is also wrong, and carries actual risk of XSS vulnerabilities. However,
in all cases, security vulnerabilities were narrowly avoided due to other
filtering in Kallithea. (E.g. many special characters are banned from
appearing in usernames.) In these cases, the escaping has been fixed
and moved to the template, making it immediately visible that proper
escaping has been performed.

Mini-FAQ (frequently anticipated questions):

Q: Why do everything in one big, hard to review patch?
Q: Why add escaping in specific case FOO, it doesn't seem needed?

Because the goal here is to have "escape everywhere" as the default
policy, rather than identifying individual bugs and fixing them one
by one by adding escaping where needed. As such, this patch surely
introduces a lot of needless escaping. This is no different from
how Mako/Pylons HTML escape everything by default, even when not
needed: it's errs on the side of needless work, to prevent erring
on the side of skipping required (and security critical) work.

As for reviewability, the most important thing to notice is not where
escaping has been introduced, but any places where it might have been
missed (or where h.jshtml is needed, but h.js is used).

Q: The added escaping is kinda verbose/ugly.

That is not a question, but yes, I agree. Hopefully it'll encourage us
to move away from inline JavaScript altogether. That's a significantly
larger job, though; with luck this patch will keep us safe and secure
until such a time as we can implement the real fix.

Q: Why not use Mako filter syntax ("${val|h.js}")?

Because of long-standing Mako bug #140, preventing use of 'h' in
filters.

Q: Why not work around bug #140, or even use straight "${val|js}"?

Because Mako still applies the default h.escape filter before the
explicitly specified filters.

Q: Where do we go from here?

Longer term, we should stop doing variable expansions in script blocks,
and instead pass data to JS via e.g. data attributes, or asynchronously
using AJAX calls. Once we've done that, we can remove inline JavaScript
altogether in favor of separate script files, and set a strict Content
Security Policy explicitly blocking inline scripting, and thus also the
most common kind of cross-site scripting attack.

template: use Bootstrap tooltips and popover instead of handmade tooltips

Based on work from Andrew Shadura <andrew@shadura.me>.

Further modified by Mads Kiilerich.

show_changeset_tooltip is merged into tooltip_activate.

template: use Bootstrap compatible navbar markup

Based on work from Andrew Shadura <andrew@shadura.me>.

Further modified by Mads Kiilerich.

templates: drop id="body" on body - it wasn't used and caused duplicate IDs

There are lots of
<div id="body" class="codeblock">
where the body id perhaps also could be dropped.

changelog: rework changeset selection and button behaviour

This is a refactoring / rework of messy code to be simpler and work correctly.

As a result, it gives slightly different behaviour in some cases and different
button labels.

It also allows creating PRs after selecting two changesets. That is currently
pointless but will make sense soon.

javascript: provide secure_form compatible_authentication_token in all AJAX POSTs

_authentication_token is introduced as a global javascript variable. That seems
less ugly than passing it through as parameter everywhere ... and the token
really _is_ a global thing.

templates: move site branding in page title to base template

Instead of repeating the same three lines in each and every template, move
it to the base template.

jquery: upgrade to 1.11.1

Downloaded with

$ wget -N http://code.jquery.com/jquery-1.11.1.min.js

The source is available at https://kallithea-scm.org/repos/mirror/jquery/ .

We stay with 1.x to remain compatible with IE8.

We no longer include the map file - it is no longer referenced from the js.

support other analytic systems then just Google Analytics

Fallback to Google Analytics code if the string starts with "UA-"

javascript: replace YUE.onDOMReady with $(document).ready

Different execution order of YUE.onDOMReady and $(document).ready makes it hard to do gradually.

Second step in two-part process to rename directories.
This is the actual directory rename.