kallithea Files · docs/usage/statistics.rst

Files @ 07f026cdf7ac

Branch filter:

Location: kallithea/docs/usage/statistics.rst

07f026cdf7ac 1.2 KiB text/prs.fallenstein.rst Show Annotation Show as Raw Download as Raw

domruf

vcs: catch MemoryErrors when calling Git diff

Binary diffs can make the diffs VERY big and cause MemoryError exceptions.

Before giving MemoryError, the system might start swapping, any process might
fail when allocating memory, random processes might get killed, and our process
might fail in other places. The proper fix would be to avoid the problem by not
trying to process more data than we can handle - for example by not processing
more than a certain amount of Git output.

Before, memory errors were shown to the user as a 500 Internal Server Error
page.

Now, as long as we have no better/safer way get the diff, catch the MemoryError
and show the page with a flash error message and no diff.

The error handling is placed in the diffs module to avoid leaking flash
messages into the vcs lib.

.. _statistics:

=====================
Repository statistics
=====================

Kallithea has a *repository statistics* feature, disabled by default. When
enabled, the amount of commits per committer is visualized in a timeline. This
feature can be enabled using the ``Enable statistics`` checkbox on the
repository ``Settings`` page.

The statistics system makes heavy demands on the server resources, so
in order to keep a balance between usability and performance, statistics are
cached inside the database and gathered incrementally.

When Celery is disabled:

  On each first visit to the summary page a set of 250 commits are parsed and
  added to the statistics cache. This incremental gathering also happens on each
  visit to the statistics page, until all commits are fetched.

  Statistics are kept cached until additional commits are added to the
  repository. In such a case Kallithea will only fetch the new commits when
  updating its statistics cache.

When Celery is enabled:

  On the first visit to the summary page, Kallithea will create tasks that will
  execute on Celery workers. These tasks will gather all of the statistics until
  all commits are parsed. Each task parses 250 commits, then launches a new
  task.