.. _api:

API

===

Starting from RhodeCode version 1.2 a simple API was implemented.

There's a single schema for calling all api methods. API is implemented

with JSON protocol both ways. An url to send API request in RhodeCode is

<your_server>/_admin/api

API ACCESS FOR WEB VIEWS

++++++++++++++++++++++++

API access can also be turned on for each web view in RhodeCode that is 

decorated with `@LoginRequired` decorator. To enable API access simple change 

the standard login decorator to `@LoginRequired(api_access=True)`. 

After this change, a rhodecode view can be accessed without login by adding a 

GET parameter `?api_key=<api_key>` to url. By default this is only

enabled on RSS/ATOM feed views.

API ACCESS

++++++++++

All clients are required to send JSON-RPC spec JSON data::

    {   

        "id:<id>,

        "api_key":"<api_key>",

        "method":"<method_name>",

        "args":{"<arg_key>":"<arg_val>"}

    }

Example call for autopulling remotes repos using curl::

    curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

Simply provide

 - *id* A value of any type, which is used to match the response with the request that it is replying to.

 - *api_key* for access and permission validation.

 - *method* is name of method to call

 - *args* is an key:value list of arguments to pass to method

.. note::

    api_key can be found in your user account page

RhodeCode API will return always a JSON-RPC response::

    {   

        "id":<id>,

        "result": "<result>",

        "error": null

    }

All responses from API will be `HTTP/1.0 200 OK`, if there's an error while

calling api *error* key from response will contain failure description

and result will be null.

API METHODS

+++++++++++

pull

----

Pulls given repo from remote location. Can be used to automatically keep

remote repos up to date. This command can be executed only using api_key

belonging to user with admin rights

INPUT::

    api_key : "<api_key>"

    method :  "pull"

    args :    {

                "repo_name" : "<reponame>"

              }

OUTPUT::

    result : "Pulled from <reponame>"

    error :  null

get_user

--------

Get's an user by username or user_id, Returns empty result if user is not found.

This command can be executed only using api_key belonging to user with admin 

rights.

INPUT::

    api_key : "<api_key>"

    method :  "get_user"

    args :    { 

.. _models:

The :mod:`models` Module

========================

.. automodule:: rhodecode.model

   :members:

.. automodule:: rhodecode.model.comment

   :members:

.. automodule:: rhodecode.model.notification

   :members:   

.. automodule:: rhodecode.model.permission

   :members:

.. automodule:: rhodecode.model.repo_permission

   :members:      

.. automodule:: rhodecode.model.repo

   :members:   

.. automodule:: rhodecode.model.repos_group

   :members:

.. automodule:: rhodecode.model.scm

   :members:

.. automodule:: rhodecode.model.user

   :members:      

.. automodule:: rhodecode.model.users_group

   :members:   

.. _changelog:

Changelog

=========

1.3.4 (**2012-XX-XX**)

----------------------

:status: in-progress

:branch: beta

news

++++

fixes

+++++

1.3.3 (**2012-03-02**)

----------------------

news

++++

fixes

+++++

- fixed some python2.5 compatibility issues 

- fixed issues with removed repos was accidentally added as groups, after

  full rescan of paths

- fixes #376 Cannot edit user (using container auth)

- fixes #378 Invalid image urls on changeset screen with proxy-prefix 

  configuration

- fixed initial sorting of repos inside repo group

- fixes issue when user tried to resubmit same permission into user/user_groups

- bumped beaker version that fixes #375 leap error bug

- fixed raw_changeset for git. It was generated with hg patch headers

- fixed vcs issue with last_changeset for filenodes

- fixed missing commit after hook delete

- fixed #372 issues with git operation detection that caused a security issue 

  for git repos

1.3.2 (**2012-02-28**)

----------------------

news

++++

fixes

+++++

- fixed git protocol issues with repos-groups

- fixed git remote repos validator that prevented from cloning remote git repos

- fixes #370 ending slashes fixes for repo and groups

- fixes #368 improved git-protocol detection to handle other clients

- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 

  Moved To Root

- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 

- fixed #373 missing cascade drop on user_group_to_perm table

1.3.1 (**2012-02-27**)

----------------------

news

++++

fixes

+++++

- redirection loop occurs when remember-me wasn't checked during login

- fixes issues with git blob history generation 

- don't fetch branch for git in file history dropdown. Causes unneeded slowness

1.3.0 (**2012-02-26**)

----------------------

news

++++

- code review, inspired by github code-comments 

- #215 rst and markdown README files support

- #252 Container-based and proxy pass-through authentication support

- #44 branch browser. Filtering of changelog by branches

- mercurial bookmarks support

- new hover top menu, optimized to add maximum size for important views

- configurable clone url template with possibility to specify  protocol like 

  ssh:// or http:// and also manually alter other parts of clone_url.

- enabled largefiles extension by default

- optimized summary file pages and saved a lot of unused space in them

- #239 option to manually mark repository as fork

- #320 mapping of commit authors to RhodeCode users

- #304 hashes are displayed using monospace font    

- diff configuration, toggle white lines and context lines

- #307 configurable diffs, whitespace toggle, increasing context lines

.. _contributing:

Contributing to RhodeCode

=========================

If you would like to contribute to RhodeCode, please contact me, any help is

greatly appreciated!

Could I request that you make your source contributions by first forking the

RhodeCode repository on bitbucket_

https://bitbucket.org/marcinkuzminski/rhodecode and then make your changes to

your forked repository. Please post all fixes into **BETA** branch since your 

fix might be already fixed there and i try to merge all fixes from beta into

stable, and not the other way. Finally, when you are finished making a change, 

please send me a pull request.

To run RhodeCode in a development version you always need to install the latest

required libs from `requires.txt` file.

after downloading/pulling RhodeCode make sure you run::

    python setup.py develop

command to install/verify all required packages, and prepare development 

enviroment.

After finishing your changes make sure all tests passes ok. You can run

the testsuite running nosetest from the project root.

| Thank you for any contributions!

|  Marcin

.. _bitbucket: http://bitbucket.org/

.. _installation:

Installation

============

``RhodeCode`` is written entirely in Python. Before posting any issues make 

sure, your not missing any system libraries and using right version of 

libraries required by RhodeCode. There's also restriction in terms of mercurial

clients. Minimal version of hg client known working fine with RhodeCode is

**1.6**. If you're using older client, please upgrade.

Installing RhodeCode from Cheese Shop

-------------------------------------

Rhodecode requires python version 2.5 or higher.

The easiest way to install ``rhodecode`` is to run::

    easy_install rhodecode

Or::

    pip install rhodecode

If you prefer to install RhodeCode manually simply grab latest release from

http://pypi.python.org/pypi/rhodecode, decompress the archive and run::

    python setup.py install

Step by step installation example

---------------------------------

For installing RhodeCode i highly recommend using separate virtualenv_. This

way many required by RhodeCode libraries will remain sandboxed from your main

python and making things less problematic when doing system python updates. 

- Assuming you have installed virtualenv_ create a new virtual environment 

  using virtualenv command:: 

    virtualenv --no-site-packages /var/www/rhodecode-venv

.. note:: Using ``--no-site-packages`` when generating your

   virtualenv is **very important**. This flag provides the necessary

   isolation for running the set of packages required by

   RhodeCode.  If you do not specify ``--no-site-packages``,

   it's possible that RhodeCode will not install properly into

   the virtualenv, or, even if it does, may not run properly,

   depending on the packages you've already got installed into your

   Python's "main" site-packages dir.

- this will install new virtualenv_ into `/var/www/rhodecode-venv`. 

- Activate the virtualenv_ by running::

    source /var/www/rhodecode-venv/bin/activate

.. note:: If you're using UNIX, *do not* use ``sudo`` to run the

   ``virtualenv`` script.  It's perfectly acceptable (and desirable)

   to create a virtualenv as a normal user.

- Make a folder for rhodecode data files, and configuration somewhere on the 

  filesystem. For example::

    mkdir /var/www/rhodecode

- Go into the created directory run this command to install rhodecode::

    easy_install rhodecode

  or::

    pip install rhodecode 

- This will install rhodecode together with pylons and all other required 

  python libraries into activated virtualenv

Requirements for Celery (optional)

----------------------------------

In order to gain maximum performance

there are some third-party you must install. When RhodeCode is used 

together with celery you have to install some kind of message broker,

recommended one is rabbitmq_ to make the async tasks work.

Of course RhodeCode works in sync mode also and then you do not have to install

any third party applications. However, using Celery_ will give you a large 

speed improvement when using many big repositories. If you plan to use 

RhodeCode for say 7 to 10 repositories, RhodeCode will perform perfectly well 

without celery running.

If you make the decision to run RhodeCode with celery make sure you run 

celeryd using paster and message broker together with the application. 

.. _setup:

Setup

=====

Setting up RhodeCode

--------------------

First, you will need to create a RhodeCode configuration file. Run the 

following command to do this::

    paster make-config RhodeCode production.ini

- This will create the file `production.ini` in the current directory. This

  configuration file contains the various settings for RhodeCode, e.g proxy 

  port, email settings, usage of static files, cache, celery settings and 

  logging.

Next, you need to create the databases used by RhodeCode. I recommend that you

use sqlite (default) or postgresql. If you choose a database other than the

default ensure you properly adjust the db url in your production.ini

configuration file to use this other database. Create the databases by running

the following command::

    paster setup-app production.ini

This will prompt you for a "root" path. This "root" path is the location where

RhodeCode will store all of its repositories on the current machine. After

entering this "root" path ``setup-app`` will also prompt you for a username 

and password for the initial admin account which ``setup-app`` sets up for you.

- The ``setup-app`` command will create all of the needed tables and an admin

  account. When choosing a root path you can either use a new empty location, 

  or a location which already contains existing repositories. If you choose a 

  location which contains existing repositories RhodeCode will simply add all 

  of the repositories at the chosen location to it's database. (Note: make 

  sure you specify the correct path to the root).

- Note: the given path for mercurial_ repositories **must** be write accessible

  for the application. It's very important since the RhodeCode web interface 

  will work without write access, but when trying to do a push it will 

  eventually fail with permission denied errors unless it has write access.

You are now ready to use RhodeCode, to run it simply execute::

    paster serve production.ini

- This command runs the RhodeCode server. The web app should be available at the 

  127.0.0.1:5000. This ip and port is configurable via the production.ini 

  file created in previous step

- Use the admin account you created above when running ``setup-app`` to login 

  to the web app.

- The default permissions on each repository is read, and the owner is admin. 

  Remember to update these if needed.

- In the admin panel you can toggle ldap, anonymous, permissions settings. As

  well as edit more advanced options on users and repositories

Try copying your own mercurial repository into the "root" directory you are

using, then from within the RhodeCode web application choose Admin >

repositories. Then choose Add New Repository. Add the repository you copied 

into the root. Test that you can browse your repository from within RhodeCode 

and then try cloning your repository from RhodeCode with::

    hg clone http://127.0.0.1:5000/<repository name>

where *repository name* is replaced by the name of your repository.

Using RhodeCode with SSH

------------------------

RhodeCode currently only hosts repositories using http and https. (The addition

of ssh hosting is a planned future feature.) However you can easily use ssh in

parallel with RhodeCode. (Repository access via ssh is a standard "out of

the box" feature of mercurial_ and you can use this to access any of the

repositories that RhodeCode is hosting. See PublishingRepositories_)

RhodeCode repository structures are kept in directories with the same name 

as the project. When using repository groups, each group is a subdirectory.

This allows you to easily use ssh for accessing repositories.

In order to use ssh you need to make sure that your web-server and the users 

login accounts have the correct permissions set on the appropriate directories.

(Note that these permissions are independent of any permissions you have set up

using the RhodeCode web interface.)

If your main directory (the same as set in RhodeCode settings) is for example

set to **/home/hg** and the repository you are using is named `rhodecode`, then

to clone via ssh you should run::

    hg clone ssh://user@server.com/home/hg/rhodecode

Using other external tools such as mercurial-server_ or using ssh key based

authentication is fully supported.

Note: In an advanced setup, in order for your ssh access to use the same

permissions as set up via the RhodeCode web interface, you can create an

authentication hook to connect to the rhodecode db and runs check functions for

.. _upgrade:

Upgrade

=======

Upgrading from Cheese Shop

--------------------------

.. note::

   Firstly, it is recommended that you **always** perform a database backup 

   before doing an upgrade.

The easiest way to upgrade ``rhodecode`` is to run::

 easy_install -U rhodecode

Or::

 pip install --upgrade rhodecode

Then make sure you run the following command from the installation directory::

 paster make-config RhodeCode production.ini

This will display any changes made by the new version of RhodeCode to your

current configuration. It will try to perform an automerge. It's always better

to make a backup of your configuration file before hand and re check the 

content after the automerge.

.. note::

   Please always make sure your .ini files are up to date. Often errors are

   caused by missing params added in new versions.

It is also recommended that you rebuild the whoosh index after upgrading since 

the new whoosh version could introduce some incompatible index changes. Please

Read the changelog to see if there were any changes to whoosh.

The final step is to upgrade the database. To do this simply run::

    paster upgrade-db production.ini

This will upgrade the schema and update some of the defaults in the database,

and will always recheck the settings of the application, if there are no new 

options that need to be set.

.. _virtualenv: http://pypi.python.org/pypi/virtualenv  

.. _python: http://www.python.org/

.. _mercurial: http://mercurial.selenic.com/

.. _celery: http://celeryproject.org/

.. _rabbitmq: http://www.rabbitmq.com/

.. _backup:

Backing up RhodeCode

====================

Settings

--------

Just copy your .ini file, it contains all RhodeCode settings.

Whoosh index

------------

Whoosh index is located in **/data/index** directory where you installed

RhodeCode ie. the same place where the ini file is located

Database

--------

When using sqlite just copy rhodecode.db.

Any other database engine requires a manual backup operation.

Database backup will contain all gathered statistics

.. _general:

General RhodeCode usage

=======================

Repository deleting

-------------------

Currently when admin/owner deletes a repository, RhodeCode does not physically

delete a repository from filesystem, it renames it in a special way so it's

not possible to push,clone or access repository. It's worth a notice that,

even if someone will be given administrative access to RhodeCode and will 

delete a repository You can easy restore such action by restoring `rm__<date>`

from the repository name, and internal repository storage (.hg/.git)

Follow current branch in file view

----------------------------------

In file view when this checkbox is checked the << and >> arrows will jump

to changesets within the same branch currently viewing. So for example

if someone is viewing files at 'beta' branch and marks `follow current branch`

checkbox the << and >> buttons will only show him revisions for 'beta' branch

Compare view from changelog

---------------------------

Checkboxes in compare view allow users to view combined compare view. You can

only show the range between the first and last checkbox (no cherry pick).

Clicking more than one checkbox will activate a link in top saying

`Show selected changes <from-rev> -> <to-rev>` clicking this will bring

compare view

Compare view is also available from the journal on pushes having more than

one changeset

Non changeable repository urls

------------------------------

Due to complicated nature of repository grouping, often urls of repositories

can change.

example::

  #before

  http://server.com/repo_name

  # after insertion to test_group group the url will be

  http://server.com/test_group/repo_name

This can be an issue for build systems and any other hardcoded scripts, moving

repository to a group leads to a need for changing external systems. To 

overcome this RhodeCode introduces a non changable replacement url. It's 

simply an repository ID prefixed with `_` above urls are also accessible as::

  http://server.com/_<ID>

Since ID are always the same moving the repository will not affect such url.

the _<ID> syntax can be used anywhere in the system so urls with repo_name 

for changelogs, files and other can be exchanged with _<ID> syntax.

Mailing

-------

When administrator will fill up the mailing settings in .ini files

RhodeCode will send mails on user registration, or when RhodeCode errors occur

on errors the mails will have a detailed traceback of error.

Trending source files

---------------------

Trending source files are calculated based on pre defined dict of known

types and extensions. If You miss some extension or Would like to scan some

custom files it's possible to add new types in `LANGUAGES_EXTENSIONS_MAP` dict

located in `/rhodecode/lib/celerylib/tasks.py`

.. _git_support:

GIT support

===========

Git support in RhodeCode 1.3 was enabled by default. 

Although There are some limitations on git usage.

- No hooks are runned for git push/pull actions.

- logs in action journals don't have git operations

- large pushes needs http server with chunked encoding support.

if you plan to use git you need to run RhodeCode with some

http server that supports chunked encoding which git http protocol uses, 

i recommend using waitress_ or gunicorn_ (linux only) for `paste` wsgi app 

replacement.

To use waitress simply change change the following in the .ini file::

    use = egg:Paste#http

To::

    use = egg:waitress#main

And comment out bellow options::

    threadpool_workers = 

    threadpool_max_requests = 

    use_threadpool = 

You can simply run `paster serve` as usual.

You can always disable git/hg support by editing a 

file **rhodecode/__init__.py** and commenting out backends

.. code-block:: python

   BACKENDS = {

       'hg': 'Mercurial repository',

       #'git': 'Git repository',

   }

.. _waitress: http://pypi.python.org/pypi/waitress

.. _gunicorn: http://pypi.python.org/pypi/gunicorn

.. _statistics:

Statistics

==========

The RhodeCode statistics system makes heavy demands of the server resources, so

in order to keep a balance between usability and performance, the statistics are

cached inside db and are gathered incrementally, this is how RhodeCode does

this:

With Celery disabled

--------------------

- On each first visit to the summary page a set of 250 commits are parsed and

  updates statistics cache.

- This happens on each single visit to the statistics page until all commits are

  fetched. Statistics are kept cached until additional commits are added to the

  repository. In such a case RhodeCode will only fetch the new commits when

  updating it's cache.

With Celery enabled

-------------------

- On the first visit to the summary page RhodeCode will create tasks that will

  execute on celery workers. This task will gather all of the stats until all

  commits are parsed, each task will parse 250 commits, and run the next task to

  parse next 250 commits, until all of the commits are parsed.

.. note::

   At any time you can disable statistics on each repository via the repository

   edit form in the admin panel. To do this just uncheck the statistics checkbox.