Changeset - d02c715e2805
[Not reviewed]
default
0 1 0
Mads Kiilerich - 9 years ago 2017-01-03 02:06:41
mads@kiilerich.com
docs: tweak formatting of the performance page - replace the odd numbered list with sections
1 file changed with 48 insertions and 35 deletions:
0 comments (0 inline, 0 general)
docs/usage/performance.rst
Show inline comments
 
@@ -4,53 +4,64 @@
 
Optimizing Kallithea performance
 
================================
 

	
 
When serving a large amount of big repositories, Kallithea can start
 
performing slower than expected. Because of the demanding nature of handling large
 
amounts of data from version control systems, here are some tips on how to get
 
the best performance.
 
When serving a large amount of big repositories, Kallithea can start performing
 
slower than expected. Because of the demanding nature of handling large amounts
 
of data from version control systems, here are some tips on how to get the best
 
performance.
 

	
 

	
 
Follow these few steps to improve performance of Kallithea system.
 
Fast storage
 
------------
 

	
 
1.  Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) is
 
    usually more important than a fast CPU.
 
Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) and plenty of RAM
 
is usually more important than a fast CPU.
 

	
 

	
 
2. Increase cache
 
Caching
 
-------
 

	
 
    Tweak beaker cache settings in the ini file. The actual effect of that
 
    is questionable.
 
Tweak beaker cache settings in the ini file. The actual effect of that is
 
questionable.
 

	
 

	
 
3. Switch from SQLite to PostgreSQL or MySQL
 
Database
 
--------
 

	
 
    SQLite is a good option when having a small load on the system. But due to
 
    locking issues with SQLite, it is not recommended to use it for larger
 
    deployments. Switching to MySQL or PostgreSQL will result in an immediate
 
    performance increase. A tool like SQLAlchemyGrate_ can be used for
 
    migrating to another database platform.
 
SQLite is a good option when having a small load on the system. But due to
 
locking issues with SQLite, it is not recommended to use it for larger
 
deployments.
 

	
 
4. Scale Kallithea horizontally
 
Switching to MySQL or PostgreSQL will result in an immediate performance
 
increase. A tool like SQLAlchemyGrate_ can be used for migrating to another
 
database platform.
 

	
 

	
 
    Scaling horizontally can give huge performance benefits when dealing with
 
    large amounts of traffic (many users, CI servers, etc.). Kallithea can be
 
    scaled horizontally on one (recommended) or multiple machines.
 
Horizontal scaling
 
------------------
 

	
 
    It is generally possible to run WSGI applications multithreaded, so that
 
    several HTTP requests are served from the same Python process at once. That
 
    can in principle give better utilization of internal caches and less
 
    process overhead.
 
Scaling horizontally means running several Kallithea instances and let them
 
share the load. That can give huge performance benefits when dealing with large
 
amounts of traffic (many users, CI servers, etc.). Kallithea can be scaled
 
horizontally on one (recommended) or multiple machines.
 

	
 
    One danger of running multithreaded is that program execution becomes much
 
    more complex; programs must be written to consider all combinations of
 
    events and problems might depend on timing and be impossible to reproduce.
 
It is generally possible to run WSGI applications multithreaded, so that
 
several HTTP requests are served from the same Python process at once. That can
 
in principle give better utilization of internal caches and less process
 
overhead.
 

	
 
One danger of running multithreaded is that program execution becomes much more
 
complex; programs must be written to consider all combinations of events and
 
problems might depend on timing and be impossible to reproduce.
 

	
 
    Kallithea can't promise to be thread-safe, just like the embedded Mercurial
 
    backend doesn't make any strong promises when used as Kallithea uses it.
 
    Instead, we recommend scaling by using multiple server processes.
 
Kallithea can't promise to be thread-safe, just like the embedded Mercurial
 
backend doesn't make any strong promises when used as Kallithea uses it.
 
Instead, we recommend scaling by using multiple server processes.
 

	
 
    Web servers with multiple worker processes (such as ``mod_wsgi`` with the
 
    ``WSGIDaemonProcess`` ``processes`` parameter) will work out of the box.
 
Web servers with multiple worker processes (such as ``mod_wsgi`` with the
 
``WSGIDaemonProcess`` ``processes`` parameter) will work out of the box.
 

	
 
    In order to scale horizontally on multiple machines, you need to do the
 
    following:
 
In order to scale horizontally on multiple machines, you need to do the
 
following:
 

	
 
    - Each instance's ``data`` storage needs to be configured to be stored on a
 
      shared disk storage, preferably together with repositories. This ``data``
 
@@ -65,7 +76,9 @@ Follow these few steps to improve perfor
 
      that will separate regular user traffic from automated processes like CI
 
      servers or build bots.
 

	
 
5. Serve static files directly from the web server
 

	
 
Serve static files directly from the web server
 
-----------------------------------------------
 

	
 
With the default ``static_files`` ini setting, the Kallithea WSGI application
 
will take care of serving the static files found in ``kallithea/public`` from
0 comments (0 inline, 0 general)