.. _api:

===

API

===

Starting from RhodeCode version 1.2 a simple API was implemented.

There's a single schema for calling all api methods. API is implemented

with JSON protocol both ways. An url to send API request in RhodeCode is

<your_server>/_admin/api

API ACCESS FOR WEB VIEWS

++++++++++++++++++++++++

API access can also be turned on for each web view in RhodeCode that is 

decorated with `@LoginRequired` decorator. To enable API access simple change 

the standard login decorator to `@LoginRequired(api_access=True)`. 

After this change, a rhodecode view can be accessed without login by adding a 

GET parameter `?api_key=<api_key>` to url. By default this is only

enabled on RSS/ATOM feed views.

API ACCESS

++++++++++

All clients are required to send JSON-RPC spec JSON data::

    {   

        "id:"<id>",

        "api_key":"<api_key>",

        "method":"<method_name>",

        "args":{"<arg_key>":"<arg_val>"}

    }

Example call for autopulling remotes repos using curl::

    curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

Simply provide

 - *id* A value of any type, which is used to match the response with the request that it is replying to.

 - *api_key* for access and permission validation.

 - *method* is name of method to call

 - *args* is an key:value list of arguments to pass to method

.. note::

    api_key can be found in your user account page

RhodeCode API will return always a JSON-RPC response::

    {   

        "id":<id>, # matching id sent by request

        "result": "<result>"|null, # JSON formatted result, null if any errors

        "error": "null"|<error_message> # JSON formatted error (if any)

    }

All responses from API will be `HTTP/1.0 200 OK`, if there's an error while

calling api *error* key from response will contain failure description

and result will be null.

API CLIENT

++++++++++

From version 1.4 RhodeCode adds a binary script that allows to easily

communicate with API. After installing RhodeCode a `rhodecode-api` script

will be available.

To get started quickly simply run::

  rhodecode-api _create_config --apikey=<youapikey> --apihost=<rhodecode host>

This will create a file named .config in the directory you executed it storing

json config file with credentials. You can skip this step and always provide

both of the arguments to be able to communicate with server

after that simply run any api command for example get_repo::

 rhodecode-api get_repo

 calling {"api_key": "<apikey>", "id": 75, "args": {}, "method": "get_repo"} to http://127.0.0.1:5000

 rhodecode said:

 {'error': 'Missing non optional `repoid` arg in JSON DATA',

  'id': 75,

  'result': None}

Ups looks like we forgot to add an argument

Let's try again now giving the repoid as parameters::

    rhodecode-api get_repo repoid:rhodecode   

    calling {"api_key": "<apikey>", "id": 39, "args": {"repoid": "rhodecode"}, "method": "get_repo"} to http://127.0.0.1:5000

    rhodecode said:

    {'error': None,

     'id': 39,

     'result': <json data...>}

API METHODS

+++++++++++

pull

----

Pulls given repo from remote location. Can be used to automatically keep

remote repos up to date. This command can be executed only using api_key

belonging to user with admin rights

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "pull"

    args :    {

                "repo_name" : "<reponame>"

              }

OUTPUT::

    result : "Pulled from <reponame>"

    error :  null

get_user

--------

Get's an user by username or user_id, Returns empty result if user is not found.

This command can be executed only using api_key belonging to user with admin 

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_user"

    args :    { 

                "userid" : "<username or user_id>"

              }

OUTPUT::

    result: None if user does not exist or 

            {

                "id" :       "<id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

                "permissions": {

                    "global": ["hg.create.repository",

                               "repository.read",

                               "hg.register.manual_activate"],

                    "repositories": {"repo1": "repository.none"},

                    "repositories_groups": {"Group1": "group.read"}

                 },

            }

    error:  null

get_users

---------

Lists all existing users. This command can be executed only using api_key

belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_users"

    args :    { }

OUTPUT::

    result: [

              {

                "id" :       "<id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

              },

    	      …

            ]

    error:  null

create_user

-----------

Creates new user. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "create_user"

    args :    {

                "username" :  "<username>",

                "password" :  "<password>",

                "email" :     "<useremail>",

                "firstname" : "<firstname> = None",

                "lastname" :  "<lastname> = None",

                "active" :    "<bool> = True",

                "admin" :     "<bool> = False",

                "ldap_dn" :   "<ldap_dn> = None"

              }

OUTPUT::

    result: {

              "id" : "<new_user_id>",

              "msg" : "created new user <username>",

              "user": {

                "id" :       "<id>",

                "username" : "<username>",

                "firstname": "<firstname>",

                "lastname" : "<lastname>",

                "email" :    "<email>",

                "active" :   "<bool>",

                "admin" :    "<bool>",

                "ldap_dn" :  "<ldap_dn>",

                "last_login": "<last_login>",

              },

            }

    error:  null

update_user

-----------

updates given user if such user exists. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "update_user"

    args :    {

                "userid" : "<user_id or username>",

                "username" :  "<username>",

                "password" :  "<password>",

                "email" :     "<useremail>",

                "firstname" : "<firstname>",

                "lastname" :  "<lastname>",

                "active" :    "<bool>",

                "admin" :     "<bool>",

                "ldap_dn" :   "<ldap_dn>"

              }

OUTPUT::

    result: {

              "id" : "<edited_user_id>",

              "msg" : "updated user ID:<userid> <username>"

            }

    error:  null

delete_user

-----------

deletes givenuser if such user exists. This command can 

be executed only using api_key belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "delete_user"

    args :    {

                "userid" : "<user_id or username>",

              }

OUTPUT::

    result: {

              "id" : "<edited_user_id>",

              "msg" : "deleted user ID:<userid> <username>"

            }

    error:  null

get_users_group

---------------

Gets an existing users group. This command can be executed only using api_key

belonging to user with admin rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_users_group"

    args :    {

                "group_name" : "<name>"

              }

OUTPUT::

    result : None if group not exist

             {

               "id" :         "<id>",

               "group_name" : "<groupname>",

               "active":      "<bool>",

               "members" :  [

                              { "id" :       "<userid>",

                                "username" : "<username>",

                                "firstname": "<firstname>",

                                "lastname" : "<lastname>",

                                "email" :    "<email>",

                                "active" :   "<bool>",

                                "admin" :    "<bool>",

                                "ldap" :     "<ldap_dn>"

                              },

                              …

                            ]

             }

    error : null

get_users_groups

----------------

# -*- coding: utf-8 -*-

"""

    rhodecode.controllers.api

    ~~~~~~~~~~~~~~~~~~~~~~~~~

    API controller for RhodeCode

    :created_on: Aug 20, 2011

    :author: marcink

    :copyright: (C) 2011-2012 Marcin Kuzminski <marcin@python-works.com>

    :license: GPLv3, see COPYING for more details.

"""

# This program is free software; you can redistribute it and/or

# modify it under the terms of the GNU General Public License

# as published by the Free Software Foundation; version 2

# of the License or (at your opinion) any later version of the license.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program; if not, write to the Free Software

# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,

# MA  02110-1301, USA.

import traceback

import logging

from rhodecode.controllers.api import JSONRPCController, JSONRPCError

from rhodecode.lib.auth import HasPermissionAllDecorator, \

    HasPermissionAnyDecorator, PasswordGenerator, AuthUser

from rhodecode.model.meta import Session

from rhodecode.model.scm import ScmModel

from rhodecode.model.db import User, UsersGroup, Repository

from rhodecode.model.repo import RepoModel

from rhodecode.model.user import UserModel

from rhodecode.model.users_group import UsersGroupModel

from rhodecode.lib.utils import map_groups

log = logging.getLogger(__name__)

class ApiController(JSONRPCController):

    """

    API Controller

    Each method needs to have USER as argument this is then based on given

    API_KEY propagated as instance of user object

    Preferably this should be first argument also

    Each function should also **raise** JSONRPCError for any

    errors that happens

    """

    @HasPermissionAllDecorator('hg.admin')

    def pull(self, apiuser, repo_name):

        """

        Dispatch pull action on given repo

        :param user:

        :param repo_name:

        """

        if Repository.is_valid(repo_name) is False:

            raise JSONRPCError('Unknown repo "%s"' % repo_name)

        try:

            ScmModel().pull_changes(repo_name, self.rhodecode_user.username)

            return 'Pulled from %s' % repo_name

        except Exception:

            raise JSONRPCError('Unable to pull changes from "%s"' % repo_name)

    @HasPermissionAllDecorator('hg.admin')

    def get_user(self, apiuser, userid):

        """"

        Get a user by username

        :param apiuser:

        :param username:

        """

        user = UserModel().get_user(userid)

        if user is None:

            return user

        return dict(

            id=user.user_id,

            username=user.username,

            firstname=user.name,

            lastname=user.lastname,

            email=user.email,

            active=user.active,

            admin=user.admin,

            ldap_dn=user.ldap_dn,

            last_login=user.last_login,

            permissions=AuthUser(user_id=user.user_id).permissions

        )

    @HasPermissionAllDecorator('hg.admin')

    def get_users(self, apiuser):

        """"

        Get all users

        :param apiuser:

        """

        result = []

        for user in User.getAll():

            result.append(

                dict(

                    id=user.user_id,

                    username=user.username,

                    firstname=user.name,

                    lastname=user.lastname,

                    email=user.email,

                    active=user.active,

                    admin=user.admin,

                    ldap_dn=user.ldap_dn,

                    last_login=user.last_login,

                )

            )

        return result

    @HasPermissionAllDecorator('hg.admin')

    def create_user(self, apiuser, username, email, password, firstname=None,

                    lastname=None, active=True, admin=False, ldap_dn=None):

        """

        Create new user

        :param apiuser:

        :param username:

        :param password:

        :param email:

        :param name:

        :param lastname:

        :param active:

        :param admin:

        :param ldap_dn:

        """

        if User.get_by_username(username):

            raise JSONRPCError("user %s already exist" % username)

        if User.get_by_email(email, case_insensitive=True):

            raise JSONRPCError("email %s already exist" % email)

        if ldap_dn:

            # generate temporary password if ldap_dn

            password = PasswordGenerator().gen_password(length=8)

        try:

            user = UserModel().create_or_update(

                username, password, email, firstname,

                lastname, active, admin, ldap_dn

            )

            Session.commit()

            return dict(

                id=user.user_id,

                msg='created new user %s' % username,

                user=dict(

                    id=user.user_id,

                    username=user.username,

                    firstname=user.name,

                    lastname=user.lastname,

                    email=user.email,

                    active=user.active,

                    admin=user.admin,

                    ldap_dn=user.ldap_dn,

                    last_login=user.last_login,

                )

            )

        except Exception:

            log.error(traceback.format_exc())

            raise JSONRPCError('failed to create user %s' % username)

    @HasPermissionAllDecorator('hg.admin')

    def update_user(self, apiuser, userid, username, password, email,

                    firstname, lastname, active, admin, ldap_dn):

        """

        Updates given user

        :param apiuser:

        :param username:

        :param password:

        :param email:

        :param name:

        :param lastname:

        :param active:

        :param admin:

        :param ldap_dn:

        """

        usr = UserModel().get_user(userid)

        if not usr:

            raise JSONRPCError("user ID:%s does not exist" % userid)

        try:

            usr = UserModel().create_or_update(

                username, password, email, firstname,

                lastname, active, admin, ldap_dn

            )

            Session.commit()

            return dict(

                id=usr.user_id,

                msg='updated user ID:%s %s' % (usr.user_id, usr.username)

            )

        except Exception:

            log.error(traceback.format_exc())

            raise JSONRPCError('failed to update user %s' % userid)

    @HasPermissionAllDecorator('hg.admin')

    def delete_user(self, apiuser, userid):

        """"

        Deletes an user

        :param apiuser:

        """

        usr = UserModel().get_user(userid)

        if not usr:

            raise JSONRPCError("user ID:%s does not exist" % userid)

        try:

            UserModel().delete(userid)

            Session.commit()

            return dict(

                id=usr.user_id,

                msg='deleted user ID:%s %s' % (usr.user_id, usr.username)

            )

        except Exception:

            log.error(traceback.format_exc())

            raise JSONRPCError('failed to delete ID:%s %s' % (usr.user_id,

                                                              usr.username))

    @HasPermissionAllDecorator('hg.admin')

    def get_users_group(self, apiuser, group_name):

        """"

        Get users group by name

        :param apiuser:

        :param group_name:

        """

        users_group = UsersGroup.get_by_group_name(group_name)

        if not users_group:

            return None

        members = []

        for user in users_group.members:

            user = user.user

            members.append(dict(id=user.user_id,

                            username=user.username,

                            firstname=user.name,

                            lastname=user.lastname,

                            email=user.email,

                            active=user.active,

                            admin=user.admin,

                            ldap=user.ldap_dn))

        return dict(id=users_group.users_group_id,

                    group_name=users_group.users_group_name,

                    active=users_group.users_group_active,

                    members=members)

    @HasPermissionAllDecorator('hg.admin')

    def get_users_groups(self, apiuser):

        """"

        Get all users groups

        :param apiuser:

        """

        result = []

        for users_group in UsersGroup.getAll():

            members = []

            for user in users_group.members:

                user = user.user

                members.append(dict(id=user.user_id,

                                username=user.username,

                                firstname=user.name,

                                lastname=user.lastname,

                                email=user.email,

                                active=user.active,

                                admin=user.admin,

                                ldap=user.ldap_dn))

            return res

    @classmethod

    def getAll(cls):

        return cls.query().all()

    @classmethod

    def delete(cls, id_):

        obj = cls.query().get(id_)

        Session.delete(obj)

    def __repr__(self):

        if hasattr(self, '__unicode__'):

            # python repr needs to return str

            return safe_str(self.__unicode__())

        return '<DB:%s>' % (self.__class__.__name__)

class RhodeCodeSetting(Base, BaseModel):

    __tablename__ = 'rhodecode_settings'

    __table_args__ = (

        UniqueConstraint('app_settings_name'),

        {'extend_existing': True, 'mysql_engine': 'InnoDB',

         'mysql_charset': 'utf8'}

    )

    app_settings_id = Column("app_settings_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)

    app_settings_name = Column("app_settings_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    _app_settings_value = Column("app_settings_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    def __init__(self, k='', v=''):

        self.app_settings_name = k

        self.app_settings_value = v

    @validates('_app_settings_value')

    def validate_settings_value(self, key, val):

        assert type(val) == unicode

        return val

    @hybrid_property

    def app_settings_value(self):

        v = self._app_settings_value

        if self.app_settings_name == 'ldap_active':

            v = str2bool(v)

        return v

    @app_settings_value.setter

    def app_settings_value(self, val):

        """

        Setter that will always make sure we use unicode in app_settings_value

        :param val:

        """

        self._app_settings_value = safe_unicode(val)

    def __unicode__(self):

        return u"<%s('%s:%s')>" % (

            self.__class__.__name__,

            self.app_settings_name, self.app_settings_value

        )

    @classmethod

    def get_by_name(cls, ldap_key):

        return cls.query()\

            .filter(cls.app_settings_name == ldap_key).scalar()

    @classmethod

    def get_app_settings(cls, cache=False):

        ret = cls.query()

        if cache:

            ret = ret.options(FromCache("sql_cache_short", "get_hg_settings"))

        if not ret:

            raise Exception('Could not get application settings !')

        settings = {}

        for each in ret:

            settings['rhodecode_' + each.app_settings_name] = \

                each.app_settings_value

        return settings

    @classmethod

    def get_ldap_settings(cls, cache=False):

        ret = cls.query()\

                .filter(cls.app_settings_name.startswith('ldap_')).all()

        fd = {}

        for row in ret:

            fd.update({row.app_settings_name: row.app_settings_value})

        return fd

class RhodeCodeUi(Base, BaseModel):

    __tablename__ = 'rhodecode_ui'

    __table_args__ = (

        UniqueConstraint('ui_key'),

        {'extend_existing': True, 'mysql_engine': 'InnoDB',

         'mysql_charset': 'utf8'}

    )

    HOOK_UPDATE = 'changegroup.update'

    HOOK_REPO_SIZE = 'changegroup.repo_size'

    HOOK_PUSH = 'changegroup.push_logger'

    HOOK_PULL = 'preoutgoing.pull_logger'

    ui_id = Column("ui_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)

    ui_section = Column("ui_section", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    ui_key = Column("ui_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    ui_value = Column("ui_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    ui_active = Column("ui_active", Boolean(), nullable=True, unique=None, default=True)

    @classmethod

    def get_by_key(cls, key):

        return cls.query().filter(cls.ui_key == key)

    @classmethod

    def get_builtin_hooks(cls):

        q = cls.query()

        q = q.filter(cls.ui_key.in_([cls.HOOK_UPDATE,

                                    cls.HOOK_REPO_SIZE,

                                    cls.HOOK_PUSH, cls.HOOK_PULL]))

        return q.all()

    @classmethod

    def get_custom_hooks(cls):

        q = cls.query()

        q = q.filter(~cls.ui_key.in_([cls.HOOK_UPDATE,

                                    cls.HOOK_REPO_SIZE,

                                    cls.HOOK_PUSH, cls.HOOK_PULL]))

        q = q.filter(cls.ui_section == 'hooks')

        return q.all()

    @classmethod

    def get_repos_location(cls):

        return cls.get_by_key('/').one().ui_value

    @classmethod

    def create_or_update_hook(cls, key, val):

        new_ui = cls.get_by_key(key).scalar() or cls()

        new_ui.ui_section = 'hooks'

        new_ui.ui_active = True

        new_ui.ui_key = key

        new_ui.ui_value = val

        Session.add(new_ui)

class User(Base, BaseModel):

    __tablename__ = 'users'

    __table_args__ = (

        UniqueConstraint('username'), UniqueConstraint('email'),

        {'extend_existing': True, 'mysql_engine': 'InnoDB',

         'mysql_charset': 'utf8'}

    )

    user_id = Column("user_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)

    username = Column("username", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    password = Column("password", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    active = Column("active", Boolean(), nullable=True, unique=None, default=True)

    admin = Column("admin", Boolean(), nullable=True, unique=None, default=False)

    name = Column("firstname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    lastname = Column("lastname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    _email = Column("email", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    last_login = Column("last_login", DateTime(timezone=False), nullable=True, unique=None, default=None)

    ldap_dn = Column("ldap_dn", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    api_key = Column("api_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)

    user_log = relationship('UserLog', cascade='all')

    user_perms = relationship('UserToPerm', primaryjoin="User.user_id==UserToPerm.user_id", cascade='all')

    repositories = relationship('Repository')

    user_followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_user_id==User.user_id', cascade='all')

    repo_to_perm = relationship('UserRepoToPerm', primaryjoin='UserRepoToPerm.user_id==User.user_id', cascade='all')

    repo_group_to_perm = relationship('UserRepoGroupToPerm', primaryjoin='UserRepoGroupToPerm.user_id==User.user_id', cascade='all')

    group_member = relationship('UsersGroupMember', cascade='all')

    notifications = relationship('UserNotification', cascade='all')

    # notifications assigned to this user

    user_created_notifications = relationship('Notification', cascade='all')

    # comments created by this user

    user_comments = relationship('ChangesetComment', cascade='all')

    @hybrid_property

    def email(self):

        return self._email

    @email.setter

    def email(self, val):

        self._email = val.lower() if val else None

    @property

    def full_name(self):

        return '%s %s' % (self.name, self.lastname)

    @property

    def full_name_or_username(self):

        return ('%s %s' % (self.name, self.lastname)

                if (self.name and self.lastname) else self.username)

    @property

    def full_contact(self):

        return '%s %s <%s>' % (self.name, self.lastname, self.email)

    @property

    def short_contact(self):

        return '%s %s' % (self.name, self.lastname)

    @property

    def is_admin(self):

        return self.admin

    def __unicode__(self):

        return u"<%s('id:%s:%s')>" % (self.__class__.__name__,

                                     self.user_id, self.username)

    @classmethod

    def get_by_username(cls, username, case_insensitive=False, cache=False):

        if case_insensitive:

            q = cls.query().filter(cls.username.ilike(username))

        else:

            q = cls.query().filter(cls.username == username)

        if cache: