Files
@ f734d107296e
Branch filter:
Location: kallithea/docs/usage/vcs_notes.rst
f734d107296e
3.9 KiB
text/prs.fallenstein.rst
auth: for default permissions, use existing explicit query result values instead of following dot references in ORM result objects
There has been reports of spurious crashes on resolving references like
.repository from Permissions:
File ".../kallithea/lib/auth.py", line 678, in __wrapper
if self.check_permissions(user):
File ".../kallithea/lib/auth.py", line 718, in check_permissions
return user.has_repository_permission_level(repo_name, self.required_perm)
File ".../kallithea/lib/auth.py", line 450, in has_repository_permission_level
actual_perm = self.permissions['repositories'].get(repo_name)
File ".../kallithea/lib/vcs/utils/lazy.py", line 41, in __get__
value = self._func(obj)
File ".../kallithea/lib/auth.py", line 442, in permissions
return self.__get_perms(user=self, cache=False)
File ".../kallithea/lib/auth.py", line 498, in __get_perms
return compute(user_id, user_is_admin)
File ".../kallithea/lib/auth.py", line 190, in _cached_perms_data
r_k = perm.UserRepoToPerm.repository.repo_name
File ".../sqlalchemy/orm/attributes.py", line 285, in __get__
return self.impl.get(instance_state(instance), dict_)
File ".../sqlalchemy/orm/attributes.py", line 721, in get
value = self.callable_(state, passive)
File ".../sqlalchemy/orm/strategies.py", line 710, in _load_for_state
% (orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <UserRepoToPerm at ...> is not bound to a Session; lazy load operation of attribute 'repository' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
Permissions are cached between requests: SA result records are stored in in
beaker.cache.sql_cache_short and resued in following requests after the initial
session as been removed. References in Permission objects would usually give
lazy lookup ... but not outside the original session, where we would get an
error like this.
Permissions are indeed implemented/used incorrectly. That might explain a part
of the problem. Even if not fully explaining or fixing this problem, it is
still worth fixing:
Permissions are fetched from the database using Session().query with multiple
class/table names (joined together in way that happens to match the references
specified in the table definitions) - including Repository. The results are
thus "structs" with selected objects. If repositories always were retrieved
using this selected repository, everything would be fine. In some places, this
was what we did.
But in some places, the code happened to do what was more intuitive: just use
.repository and rely on "lazy" resolving. SA was not aware that this one
already was present in the result struct, and would try to fetch it again. Best
case, that could be inefficient. Worst case, it would fail as we see here.
Fix this by only querying from one table but use the "joinedload" option to
also fetch other referenced tables in the same select. (This might
inefficiently return the main record multiple times ... but that was already
the case with the previous approach.)
This change is thus doing multiple things with circular dependencies that can't
be split up in minor parts without taking detours:
The existing repository join like:
.join((Repository, UserGroupRepoToPerm.repository_id == Repository.repo_id))
is thus replaced by:
.options(joinedload(UserGroupRepoToPerm.repository))
Since we only are doing Session.query() on one table, the results will be of
that type instead of "structs" with multiple objects. If only querying for
UserRepoToPerm this means:
- perm.UserRepoToPerm.repository becomes perm.repository
- perm.Permission.permission_name looked at the explicitly queried Permission
in the result struct - instead it should look in the the dereferenced
repository as perm.permission.permission_name
There has been reports of spurious crashes on resolving references like
.repository from Permissions:
File ".../kallithea/lib/auth.py", line 678, in __wrapper
if self.check_permissions(user):
File ".../kallithea/lib/auth.py", line 718, in check_permissions
return user.has_repository_permission_level(repo_name, self.required_perm)
File ".../kallithea/lib/auth.py", line 450, in has_repository_permission_level
actual_perm = self.permissions['repositories'].get(repo_name)
File ".../kallithea/lib/vcs/utils/lazy.py", line 41, in __get__
value = self._func(obj)
File ".../kallithea/lib/auth.py", line 442, in permissions
return self.__get_perms(user=self, cache=False)
File ".../kallithea/lib/auth.py", line 498, in __get_perms
return compute(user_id, user_is_admin)
File ".../kallithea/lib/auth.py", line 190, in _cached_perms_data
r_k = perm.UserRepoToPerm.repository.repo_name
File ".../sqlalchemy/orm/attributes.py", line 285, in __get__
return self.impl.get(instance_state(instance), dict_)
File ".../sqlalchemy/orm/attributes.py", line 721, in get
value = self.callable_(state, passive)
File ".../sqlalchemy/orm/strategies.py", line 710, in _load_for_state
% (orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <UserRepoToPerm at ...> is not bound to a Session; lazy load operation of attribute 'repository' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
Permissions are cached between requests: SA result records are stored in in
beaker.cache.sql_cache_short and resued in following requests after the initial
session as been removed. References in Permission objects would usually give
lazy lookup ... but not outside the original session, where we would get an
error like this.
Permissions are indeed implemented/used incorrectly. That might explain a part
of the problem. Even if not fully explaining or fixing this problem, it is
still worth fixing:
Permissions are fetched from the database using Session().query with multiple
class/table names (joined together in way that happens to match the references
specified in the table definitions) - including Repository. The results are
thus "structs" with selected objects. If repositories always were retrieved
using this selected repository, everything would be fine. In some places, this
was what we did.
But in some places, the code happened to do what was more intuitive: just use
.repository and rely on "lazy" resolving. SA was not aware that this one
already was present in the result struct, and would try to fetch it again. Best
case, that could be inefficient. Worst case, it would fail as we see here.
Fix this by only querying from one table but use the "joinedload" option to
also fetch other referenced tables in the same select. (This might
inefficiently return the main record multiple times ... but that was already
the case with the previous approach.)
This change is thus doing multiple things with circular dependencies that can't
be split up in minor parts without taking detours:
The existing repository join like:
.join((Repository, UserGroupRepoToPerm.repository_id == Repository.repo_id))
is thus replaced by:
.options(joinedload(UserGroupRepoToPerm.repository))
Since we only are doing Session.query() on one table, the results will be of
that type instead of "structs" with multiple objects. If only querying for
UserRepoToPerm this means:
- perm.UserRepoToPerm.repository becomes perm.repository
- perm.Permission.permission_name looked at the explicitly queried Permission
in the result struct - instead it should look in the the dereferenced
repository as perm.permission.permission_name
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | .. _vcs_notes:
===================================
Version control systems usage notes
===================================
.. _importing:
Importing existing repositories
-------------------------------
There are two main methods to import repositories in Kallithea: via the web
interface or via the filesystem. If you have a large number of repositories to
import, importing them via the filesystem is more convenient.
Importing via web interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^
For a small number of repositories, it may be easier to create the target
repositories through the Kallithea web interface, via *Admin > Repositories* or
via the *Add Repository* button on the entry page of the web interface.
Repositories can be nested in repository groups by first creating the group (via
*Admin > Repository Groups* or via the *Add Repository Group* button on the
entry page of the web interface) and then selecting the appropriate group when
adding the repository.
After creation of the (empty) repository, push the existing commits to the
*Clone URL* displayed on the repository summary page. For Git repositories,
first add the *Clone URL* as remote, then push the commits to that remote. The
specific commands to execute are shown under the *Existing repository?* section
of the new repository's summary page.
A benefit of this method particular for Git repositories, is that the
Kallithea-specific Git hooks are installed automatically. For Mercurial, no
hooks are required anyway.
Importing via the filesystem
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The alternative method of importing repositories consists of creating the
repositories in the desired hierarchy on the filesystem and letting Kallithea
scan that location.
All repositories are stored in a central location on the filesystem. This
location is specified during installation (via ``db-create``) and can be reviewed
at *Admin > Settings > VCS > Location of repositories*. Repository groups
(defined in *Admin > Repository Groups*) are represented by a directory in that
repository location. Repositories of the repository group are nested under that
directory.
To import a set of repositories and organize them in a certain repository group
structure, first place clones in the desired hierarchy at the configured
repository location.
These clones should be created without working directory. For Mercurial, this is
done with ``hg clone -U``, for Git with ``git clone --bare``.
When the repositories are added correctly on the filesystem:
* go to *Admin > Settings > Remap and Rescan* in the Kallithea web interface
* select the *Install Git hooks* checkbox when importing Git repositories
* click *Rescan Repositories*
This step will scan the filesystem and create the appropriate repository groups
and repositories in Kallithea.
*Note*: Once repository groups have been created this way, manage their access
permissions through the Kallithea web interface.
Mercurial-specific notes
------------------------
Working with subrepositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This section explains how to use Mercurial subrepositories_ in Kallithea.
Example usage::
## init a simple repo
hg init mainrepo
cd mainrepo
echo "file" > file
hg add file
hg ci --message "initial file"
# clone subrepo we want to add from Kallithea
hg clone http://kallithea.local/subrepo
## specify URL to existing repo in Kallithea as subrepository path
echo "subrepo = http://kallithea.local/subrepo" > .hgsub
hg add .hgsub
hg ci --message "added remote subrepo"
In the file list of a clone of ``mainrepo`` you will see a connected
subrepository at the revision it was cloned with. Clicking on the
subrepository link sends you to the proper repository in Kallithea.
Cloning ``mainrepo`` will also clone the attached subrepository.
Next we can edit the subrepository data, and push back to Kallithea. This will
update both repositories.
.. _subrepositories: http://mercurial.aragost.com/kick-start/en/subrepositories/
|