Changeset - 61954577a0df
[Not reviewed]
default
0 2 0
Mads Kiilerich - 10 years ago 2016-05-04 01:18:25
madski@unity3d.com
docs: mention use of static_files, it's setup and implications
2 files changed with 31 insertions and 1 deletions:
0 comments (0 inline, 0 general)
docs/setup.rst
Show inline comments
 
@@ -611,97 +611,97 @@ Sample config for Nginx using proxy:
 
.. code-block:: nginx
 

	
 
    upstream kallithea {
 
        server 127.0.0.1:5000;
 
        # add more instances for load balancing
 
        #server 127.0.0.1:5001;
 
        #server 127.0.0.1:5002;
 
    }
 

	
 
    ## gist alias
 
    server {
 
       listen          443;
 
       server_name     gist.example.com;
 
       access_log      /var/log/nginx/gist.access.log;
 
       error_log       /var/log/nginx/gist.error.log;
 

	
 
       ssl on;
 
       ssl_certificate     gist.your.kallithea.server.crt;
 
       ssl_certificate_key gist.your.kallithea.server.key;
 

	
 
       ssl_session_timeout 5m;
 

	
 
       ssl_protocols SSLv3 TLSv1;
 
       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;
 
       ssl_prefer_server_ciphers on;
 

	
 
       rewrite ^/(.+)$ https://kallithea.example.com/_admin/gists/$1;
 
       rewrite (.*)    https://kallithea.example.com/_admin/gists;
 
    }
 

	
 
    server {
 
       listen          443;
 
       server_name     kallithea.example.com
 
       access_log      /var/log/nginx/kallithea.access.log;
 
       error_log       /var/log/nginx/kallithea.error.log;
 

	
 
       ssl on;
 
       ssl_certificate     your.kallithea.server.crt;
 
       ssl_certificate_key your.kallithea.server.key;
 

	
 
       ssl_session_timeout 5m;
 

	
 
       ssl_protocols SSLv3 TLSv1;
 
       ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;
 
       ssl_prefer_server_ciphers on;
 

	
 
       ## uncomment root directive if you want to serve static files by nginx
 
       ## requires static_files = false in .ini file
 
       #root /path/to/installation/kallithea/public;
 
       #root /srv/kallithea/kallithea/kallithea/public;
 
       include         /etc/nginx/proxy.conf;
 
       location / {
 
            try_files $uri @kallithea;
 
       }
 

	
 
       location @kallithea {
 
            proxy_pass      http://127.0.0.1:5000;
 
       }
 

	
 
    }
 

	
 
Here's the proxy.conf. It's tuned so it will not timeout on long
 
pushes or large pushes::
 

	
 
    proxy_redirect              off;
 
    proxy_set_header            Host $host;
 
    ## needed for container auth
 
    #proxy_set_header            REMOTE_USER $remote_user;
 
    #proxy_set_header            X-Forwarded-User $remote_user;
 
    proxy_set_header            X-Url-Scheme $scheme;
 
    proxy_set_header            X-Host $http_host;
 
    proxy_set_header            X-Real-IP $remote_addr;
 
    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
 
    proxy_set_header            Proxy-host $proxy_host;
 
    proxy_buffering             off;
 
    proxy_connect_timeout       7200;
 
    proxy_send_timeout          7200;
 
    proxy_read_timeout          7200;
 
    proxy_buffers               8 32k;
 
    client_max_body_size        1024m;
 
    client_body_buffer_size     128k;
 
    large_client_header_buffers 8 64k;
 

	
 

	
 
Apache virtual host reverse proxy example
 
-----------------------------------------
 

	
 
Here is a sample configuration file for Apache using proxy:
 

	
 
.. code-block:: apache
 

	
 
    <VirtualHost *:80>
 
            ServerName kallithea.example.com
 

	
 
            <Proxy *>
 
              # For Apache 2.4 and later:
 
              Require all granted
 

	
docs/usage/performance.rst
Show inline comments
 
@@ -2,50 +2,80 @@
 

	
 
================================
 
Optimizing Kallithea performance
 
================================
 

	
 
When serving a large amount of big repositories, Kallithea can start
 
performing slower than expected. Because of the demanding nature of handling large
 
amounts of data from version control systems, here are some tips on how to get
 
the best performance.
 

	
 
Follow these few steps to improve performance of Kallithea system.
 

	
 
1.  Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) is
 
    usually more important than a fast CPU.
 

	
 
2. Increase cache
 

	
 
    Tweak beaker cache settings in the ini file. The actual effect of that
 
    is questionable.
 

	
 
3. Switch from SQLite to PostgreSQL or MySQL
 

	
 
    SQLite is a good option when having a small load on the system. But due to
 
    locking issues with SQLite, it is not recommended to use it for larger
 
    deployments. Switching to MySQL or PostgreSQL will result in an immediate
 
    performance increase. A tool like SQLAlchemyGrate_ can be used for
 
    migrating to another database platform.
 

	
 
4. Scale Kallithea horizontally
 

	
 
    Scaling horizontally can give huge performance benefits when dealing with
 
    large amounts of traffic (many users, CI servers, etc.). Kallithea can be
 
    scaled horizontally on one (recommended) or multiple machines. In order
 
    to scale horizontally you need to do the following:
 

	
 
    - Each instance's ``data`` storage needs to be configured to be stored on a
 
      shared disk storage, preferably together with repositories. This ``data``
 
      dir contains template caches, sessions, whoosh index and is used for
 
      task locking (so it is safe across multiple instances). Set the
 
      ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
 
      variables in each .ini file to a shared location across Kallithea instances
 
    - If celery is used each instance should run a separate Celery instance, but
 
      the message broker should be common to all of them (e.g.,  one
 
      shared RabbitMQ server)
 
    - Load balance using round robin or IP hash, recommended is writing LB rules
 
      that will separate regular user traffic from automated processes like CI
 
      servers or build bots.
 

	
 
5. Serve static files directly from the web server
 

	
 
With the default ``static_files`` ini setting, the Kallithea WSGI application
 
will take care of serving the static files found in ``kallithea/public`` from
 
the root of the application URL. While doing that, it will currently also
 
apply buffering and compression of all the responses it is serving.
 

	
 
The actual serving of the static files is unlikely to be a problem in a
 
Kallithea setup. The buffering of responses is more likely to be a problem;
 
large responses (clones or pulls) will have to be fully processed and spooled
 
to disk or memory before the client will see any response.
 

	
 
To serve static files from the web server, use something like this Apache config
 
snippet::
 

	
 
        Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/
 
        Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/
 
        Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/
 
        Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/
 
        Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/
 

	
 
Then disable serving of static files in the ``.ini`` ``app:main`` section::
 

	
 
        static_files = false
 

	
 
If using Kallithea installed as a package, you should be able to find the files
 
under site-packages/kallithea, either in your Python installation or in your
 
virtualenv. When upgrading, make sure to update the web server configuration
 
too if necessary.
 

	
 

	
 
.. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate
0 comments (0 inline, 0 general)