pytype: mute errors from import of optional or platform specific modules

Mute pytype warnings:

File "kallithea/bin/kallithea_cli_iis.py", line 69, in iis_install: Can't find module 'isapi_wsgi'. [import-error]
File "kallithea/config/post_receive_tmpl.py", line 24, in <module>: No attribute 'setmode' on module 'msvcrt' [module-attr]
File "kallithea/config/pre_receive_tmpl.py", line 24, in <module>: No attribute 'setmode' on module 'msvcrt' [module-attr]
File "kallithea/lib/compat.py", line 59, in kill: No attribute 'windll' on module 'ctypes' [module-attr]
File "kallithea/lib/utils.py", line 242, in is_valid_repo_uri: Can't find module 'hgsubversion.svnrepo'. [import-error]
File "kallithea/tests/scripts/manual_test_concurrency.py", line 203, in <module>: No attribute '_RandomNameSequence' on module 'tempfile' [module-attr]

imports: try to use global imports unless it is a layering violation

To minimize the impact, do imports that violate the layering at runtime instead
of at import time.

py3: use global next() function instead of .next() method

From 2to3 -f next.

cache: drop setup_cache_regions - tg will already have done that and coerced the types correctly

The configuration and type fixing will be invoked from make_base_app, and we
will thus not have to do it:

File "kallithea/config/middleware.py", line 31, in make_app_without_logging
return make_base_app(global_conf, full_stack=full_stack, **app_conf)
File ".../python3.7/site-packages/tg/configuration/app_config.py", line 176, in make_base_app
wrap_app)
File ".../python3.7/site-packages/tg/configurator/application.py", line 112, in _make_app
app = TGApp(conf)
File ".../python3.7/site-packages/tg/wsgiapp.py", line 49, in __init__
app_wrapper = wrapper(self.wrapped_dispatch, self.config)
File ".../python3.7/site-packages/tg/appwrappers/caching.py", line 36, in __init__
self.options = parse_cache_config_options(config)
File ".../python3.7/site-packages/beaker/util.py", line 430, in parse_cache_config_options

This will fix a py3 problem where setup_cache_regions was run *after* beaker
had coerced types, thus introducing string types in the config where beaker
expected the integers it had put there.

tests: fix missing constants in manual_test_concurrency.py

Untested ... but evidently better than before.

codingstyle: trivial whitespace fixes

Reported by flake8.

tests: add missing tempfile import

There was already a broken reference to the tempfile module.

tests: put all temporary files in one top directory

Use KALLITHEA_TESTS_TMP_PATH or a create a temporary /tmp/kallithea-test-*
directory.

Some temporary files might still be placed in /tmp ... but nothing should be
left behind.

model: remove BaseModel class

The only remaining purpose of this class was to provide the "sa" field,
allowing a custom SQLAlchemy session to be used for model operations.
However, this never actually worked, nor was it used anywhere.

There's always a global SQLAlchemy session associated with the current
thread; using another session for a single function call does not make
any sense (as sessions cannot be mixed), unless the code works carefully
to ensure the two sessions (and all objects loaded from them) are kept
completely separate. Suffice to say that Kallithea does no such thing,
thus there's no need to pretend to support multiple concurrent sessions.

model: move code from __init__.py to base.py

Having too much code, in particular too much imports, inside a package's
__init__.py is a recipe for circular imports, and considered bad practice in
Python [1]

Move out everything from kallithea/model/__init__.py to a new file
kallithea/model/base.py and adapt the existing imports.

[1] http://docs.python-guide.org/en/latest/writing/structure/#packages

tests: add as little code as possible in __init__.py

kallithea/tests/__init__.py contained quite a lot of code, including the test
base class TestController. This in itself may be considered bad practice.

Specifically, this poses a problem when using pytest 3.0+, in which asserts
in some files are not automatically rewritten to give improved assert
output. That problem can be fixed by explicitly registering such files for
assertion rewriting, but that register call should be executed _before_ said
files are imported. I.e. if the register call is in
kallithea/tests/__init__.py, assert calls in __init__.py itself can not be
rewritten.

Since the TestController base class does effectively contain asserts, and we
do not want to execute the register call from somewhere outside the
kallithea/tests directory, we need to move the TestController class to
another file (kallithea/tests/base.py) so we can have a register call in
__init__.py before loading base.py.

While not strictly necessary to fix the mentioned pytest problem, we take
the opportunity to fully clean __init__.py and move everything to
the new kallithea/tests/base.py. While doing so, unnecessary imports are
removed, and imports are ordered alphabetically. Explicit imports of symbols
from modules that were already imported as a whole, are removed in favor of
fully qualifying the references (e.g. tempfile._RandomNameSequence).

Turbogears2 migration: use sqlalchemy.url iso sqlalchemy.db1.url

In Turbogears2, much of the application initialization is handled by the
framework, whereas in Pylons the application was responsible for it.
Initializing SQLAlchemy is one such part of initialization which is handled
by Turbogears2.

Turbogears2 expects the configuration file to refer to the database using
'sqlalchemy.url' rather than the current 'sqlalchemy.db1.url'. While the
exact name is not really important, not following this approach means we'll
need to override the sqlalchemy initialization method.

Therefore, as a preparation to the Turbogears2 migration, already change the
database reference string under Pylons.
When upgrading to a version of Kallithea containing this commit, the .ini
file will manually need to be adapted to remove the .db1 strings.

utils: rename add_cache to setup_cache_regions

Rename method 'add_cache' to something that better describes what it
actually does.

cleanup: get rid of dn as shortcut for os.path.dirname

We keep 'dirname' as shortcut despite having removed the 'join' shortcut:
* the name is less ambiguous than 'join'.
* dirname is often applied multiple times - spelling it out would be too
verbose.

cleanup: get rid of jn as shortcut for os.path.join

It is not used in that many places so it is better to be explicit and use os.path.join .

Discussed on https://bitbucket.org/domruf/kallithea/commits/1da05f42bca3f7042c444fda13a1ae1f6b9c300f#comment-2948124 .

Modified by Mads Kiilerich.

tests: don't use the TESTS_TMP_PATH string

it differs each time you run the test
and therefore doesn't work if you use a separat kallithea test instance
(KALLITHEA_NO_TMP_PATH=1)

instead get the 'real' path from the DB

FIXME: breaks kallithea/tests/functional/test_admin_repos.py

cleanup: use example.com for tests and examples

example.com is explicitly reserved for this purpose. Using that means
we won't accidentally hammer a real server or real email address if an
example value escapes into the wild, e.g. in an automated test.

The domain "kallithea.example.com" has been used throughout to refer to the
example Kallithea server.

cleanup: make module self-naming consistent

The self naming is quite pointless ... but at least make it right.

cleanup: consistently use 'except ... as ...:'

Use the Python 2.6+ syntax instead of the old confusing 'except ..., ...'
syntax.

tests: restrict pytest test collection to kallithea/tests

When the kallithea root directory contains a populated virtualenv, pytest
would also collect tests in python packages installed there.

Restrict the tests to be considered to any test_*.py file inside
kallithea/tests.

Additionally, by renaming unwanted test files in kallithea/tests/scripts to
_not_ match this pattern, we can completely get rid of the 'norecursedirs'
option.