Files
@ f734d107296e
Branch filter:
Location: kallithea/docs/installation.rst - annotation
f734d107296e
5.0 KiB
text/prs.fallenstein.rst
auth: for default permissions, use existing explicit query result values instead of following dot references in ORM result objects
There has been reports of spurious crashes on resolving references like
.repository from Permissions:
File ".../kallithea/lib/auth.py", line 678, in __wrapper
if self.check_permissions(user):
File ".../kallithea/lib/auth.py", line 718, in check_permissions
return user.has_repository_permission_level(repo_name, self.required_perm)
File ".../kallithea/lib/auth.py", line 450, in has_repository_permission_level
actual_perm = self.permissions['repositories'].get(repo_name)
File ".../kallithea/lib/vcs/utils/lazy.py", line 41, in __get__
value = self._func(obj)
File ".../kallithea/lib/auth.py", line 442, in permissions
return self.__get_perms(user=self, cache=False)
File ".../kallithea/lib/auth.py", line 498, in __get_perms
return compute(user_id, user_is_admin)
File ".../kallithea/lib/auth.py", line 190, in _cached_perms_data
r_k = perm.UserRepoToPerm.repository.repo_name
File ".../sqlalchemy/orm/attributes.py", line 285, in __get__
return self.impl.get(instance_state(instance), dict_)
File ".../sqlalchemy/orm/attributes.py", line 721, in get
value = self.callable_(state, passive)
File ".../sqlalchemy/orm/strategies.py", line 710, in _load_for_state
% (orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <UserRepoToPerm at ...> is not bound to a Session; lazy load operation of attribute 'repository' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
Permissions are cached between requests: SA result records are stored in in
beaker.cache.sql_cache_short and resued in following requests after the initial
session as been removed. References in Permission objects would usually give
lazy lookup ... but not outside the original session, where we would get an
error like this.
Permissions are indeed implemented/used incorrectly. That might explain a part
of the problem. Even if not fully explaining or fixing this problem, it is
still worth fixing:
Permissions are fetched from the database using Session().query with multiple
class/table names (joined together in way that happens to match the references
specified in the table definitions) - including Repository. The results are
thus "structs" with selected objects. If repositories always were retrieved
using this selected repository, everything would be fine. In some places, this
was what we did.
But in some places, the code happened to do what was more intuitive: just use
.repository and rely on "lazy" resolving. SA was not aware that this one
already was present in the result struct, and would try to fetch it again. Best
case, that could be inefficient. Worst case, it would fail as we see here.
Fix this by only querying from one table but use the "joinedload" option to
also fetch other referenced tables in the same select. (This might
inefficiently return the main record multiple times ... but that was already
the case with the previous approach.)
This change is thus doing multiple things with circular dependencies that can't
be split up in minor parts without taking detours:
The existing repository join like:
.join((Repository, UserGroupRepoToPerm.repository_id == Repository.repo_id))
is thus replaced by:
.options(joinedload(UserGroupRepoToPerm.repository))
Since we only are doing Session.query() on one table, the results will be of
that type instead of "structs" with multiple objects. If only querying for
UserRepoToPerm this means:
- perm.UserRepoToPerm.repository becomes perm.repository
- perm.Permission.permission_name looked at the explicitly queried Permission
in the result struct - instead it should look in the the dereferenced
repository as perm.permission.permission_name
There has been reports of spurious crashes on resolving references like
.repository from Permissions:
File ".../kallithea/lib/auth.py", line 678, in __wrapper
if self.check_permissions(user):
File ".../kallithea/lib/auth.py", line 718, in check_permissions
return user.has_repository_permission_level(repo_name, self.required_perm)
File ".../kallithea/lib/auth.py", line 450, in has_repository_permission_level
actual_perm = self.permissions['repositories'].get(repo_name)
File ".../kallithea/lib/vcs/utils/lazy.py", line 41, in __get__
value = self._func(obj)
File ".../kallithea/lib/auth.py", line 442, in permissions
return self.__get_perms(user=self, cache=False)
File ".../kallithea/lib/auth.py", line 498, in __get_perms
return compute(user_id, user_is_admin)
File ".../kallithea/lib/auth.py", line 190, in _cached_perms_data
r_k = perm.UserRepoToPerm.repository.repo_name
File ".../sqlalchemy/orm/attributes.py", line 285, in __get__
return self.impl.get(instance_state(instance), dict_)
File ".../sqlalchemy/orm/attributes.py", line 721, in get
value = self.callable_(state, passive)
File ".../sqlalchemy/orm/strategies.py", line 710, in _load_for_state
% (orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <UserRepoToPerm at ...> is not bound to a Session; lazy load operation of attribute 'repository' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
Permissions are cached between requests: SA result records are stored in in
beaker.cache.sql_cache_short and resued in following requests after the initial
session as been removed. References in Permission objects would usually give
lazy lookup ... but not outside the original session, where we would get an
error like this.
Permissions are indeed implemented/used incorrectly. That might explain a part
of the problem. Even if not fully explaining or fixing this problem, it is
still worth fixing:
Permissions are fetched from the database using Session().query with multiple
class/table names (joined together in way that happens to match the references
specified in the table definitions) - including Repository. The results are
thus "structs" with selected objects. If repositories always were retrieved
using this selected repository, everything would be fine. In some places, this
was what we did.
But in some places, the code happened to do what was more intuitive: just use
.repository and rely on "lazy" resolving. SA was not aware that this one
already was present in the result struct, and would try to fetch it again. Best
case, that could be inefficient. Worst case, it would fail as we see here.
Fix this by only querying from one table but use the "joinedload" option to
also fetch other referenced tables in the same select. (This might
inefficiently return the main record multiple times ... but that was already
the case with the previous approach.)
This change is thus doing multiple things with circular dependencies that can't
be split up in minor parts without taking detours:
The existing repository join like:
.join((Repository, UserGroupRepoToPerm.repository_id == Repository.repo_id))
is thus replaced by:
.options(joinedload(UserGroupRepoToPerm.repository))
Since we only are doing Session.query() on one table, the results will be of
that type instead of "structs" with multiple objects. If only querying for
UserRepoToPerm this means:
- perm.UserRepoToPerm.repository becomes perm.repository
- perm.Permission.permission_name looked at the explicitly queried Permission
in the result struct - instead it should look in the the dereferenced
repository as perm.permission.permission_name
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 | 5f481e4e888b 5f481e4e888b d95ea48af67b d95ea48af67b d95ea48af67b 5f481e4e888b 5ae8e644aa88 03bbd33bc084 154becd92f40 5ae8e644aa88 5ae8e644aa88 5ae8e644aa88 5f481e4e888b 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 5e66d3ec9880 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 4e6dfdb3fa01 7c952ea3d7b3 7c952ea3d7b3 23057179017f 23057179017f 23057179017f 23057179017f 23057179017f 23057179017f 23057179017f 23057179017f 23057179017f 01aca0a4f876 23057179017f 23057179017f 23057179017f 23057179017f 01aca0a4f876 23057179017f 7c952ea3d7b3 7c952ea3d7b3 fbbe80e3322b 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 89e9aef9b983 8927a1ac8d41 5e66d3ec9880 8927a1ac8d41 5e66d3ec9880 89e9aef9b983 855ba1f07aeb 8c234ae2c258 29e9cb56f26f 01aca0a4f876 5e66d3ec9880 5e66d3ec9880 5e66d3ec9880 7c952ea3d7b3 5f481e4e888b fbbe80e3322b 7c952ea3d7b3 7c952ea3d7b3 69df04ee1e2b 89e9aef9b983 03bbd33bc084 7c952ea3d7b3 7c952ea3d7b3 89e9aef9b983 5f481e4e888b 89e9aef9b983 89e9aef9b983 a60cd29ba7e2 89e9aef9b983 a60cd29ba7e2 89e9aef9b983 8c234ae2c258 8af52e1224ff 855ba1f07aeb 8c234ae2c258 8af52e1224ff 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 8b8edfc25856 e73a69cb98dc 61a6a7bf2cbd a60cd29ba7e2 03bbd33bc084 8b8edfc25856 4e6dfdb3fa01 a60cd29ba7e2 29e9cb56f26f 8b8edfc25856 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 3a3d96dbd445 03bbd33bc084 3a3d96dbd445 03bbd33bc084 29e9cb56f26f 03bbd33bc084 8845ece50d51 8845ece50d51 03bbd33bc084 7c952ea3d7b3 a60cd29ba7e2 7c952ea3d7b3 7c952ea3d7b3 fbbe80e3322b 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 7c952ea3d7b3 | .. _installation:
==========================
Installation on Unix/Linux
==========================
The following describes three different ways of installing Kallithea:
- :ref:`installation-source`: The simplest way to keep the installation
up-to-date and track any local customizations is to run directly from
source in a Kallithea repository clone, preferably inside a virtualenv
virtual Python environment.
- :ref:`installation-virtualenv`: If you prefer to only use released versions
of Kallithea, the recommended method is to install Kallithea in a virtual
Python environment using `virtualenv`. The advantages of this method over
direct installation is that Kallithea and its dependencies are completely
contained inside the virtualenv (which also means you can have multiple
installations side by side or remove it entirely by just removing the
virtualenv directory) and does not require root privileges.
- :ref:`installation-without-virtualenv`: The alternative method of installing
a Kallithea release is using standard pip. The package will be installed in
the same location as all other Python packages you have ever installed. As a
result, removing it is not as straightforward as with a virtualenv, as you'd
have to remove its dependencies manually and make sure that they are not
needed by other packages.
Regardless of the installation method you may need to make sure you have
appropriate development packages installed, as installation of some of the
Kallithea dependencies requires a working C compiler and libffi library
headers. Depending on your configuration, you may also need to install
Git and development packages for the database of your choice.
For Debian and Ubuntu, the following command will ensure that a reasonable
set of dependencies is installed::
sudo apt-get install build-essential git libffi-dev python3-dev
For Fedora and RHEL-derivatives, the following command will ensure that a
reasonable set of dependencies is installed::
sudo yum install gcc git libffi-devel python3-devel
.. _installation-source:
Installation from repository source
-----------------------------------
To install Kallithea in a virtualenv using the stable branch of the development
repository, follow the instructions below::
hg clone https://kallithea-scm.org/repos/kallithea -u stable
cd kallithea
python3 -m venv ../kallithea-venv
. ../kallithea-venv/bin/activate
pip install --upgrade pip setuptools
pip install --upgrade -e .
python3 setup.py compile_catalog # for translation of the UI
You can now proceed to :ref:`setup`.
.. _installation-virtualenv:
Installing a released version in a virtualenv
---------------------------------------------
It is highly recommended to use a separate virtualenv for installing Kallithea.
This way, all libraries required by Kallithea will be installed separately from your
main Python installation and other applications and things will be less
problematic when upgrading the system or Kallithea.
An additional benefit of virtualenv is that it doesn't require root privileges.
- Assuming you have installed virtualenv, create a new virtual environment
for example, in `/srv/kallithea/venv`, using the venv command::
python3 -m venv /srv/kallithea/venv
- Activate the virtualenv in your current shell session and make sure the
basic requirements are up-to-date by running::
. /srv/kallithea/venv/bin/activate
pip install --upgrade pip setuptools
.. note:: You can't use UNIX ``sudo`` to source the ``virtualenv`` script; it
will "activate" a shell that terminates immediately. It is also perfectly
acceptable (and desirable) to create a virtualenv as a normal user.
- Make a folder for Kallithea data files, and configuration somewhere on the
filesystem. For example::
mkdir /srv/kallithea
- Go into the created directory and run this command to install Kallithea::
pip install --upgrade kallithea
.. note:: Some dependencies are optional. If you need them, install them in
the virtualenv too::
pip install --upgrade kallithea python-ldap python-pam psycopg2
This might require installation of development packages using your
distribution's package manager.
Alternatively, download a .tar.gz from http://pypi.python.org/pypi/Kallithea,
extract it and install from source by running::
pip install --upgrade .
- This will install Kallithea together with all other required
Python libraries into the activated virtualenv.
You can now proceed to :ref:`setup`.
.. _installation-without-virtualenv:
Installing a released version without virtualenv
------------------------------------------------
For installation without virtualenv, 'just' use::
pip install kallithea
Note that this method requires root privileges and will install packages
globally without using the system's package manager.
To install as a regular user in ``~/.local``, you can use::
pip install --user kallithea
You can now proceed to :ref:`setup`.
|