.. _api:

===

API

===

Starting from RhodeCode version 1.2 a simple API was implemented.

There's a single schema for calling all api methods. API is implemented

with JSON protocol both ways. An url to send API request to RhodeCode is

<your_server>/_admin/api

API ACCESS FOR WEB VIEWS

++++++++++++++++++++++++

API access can also be turned on for each web view in RhodeCode that is

decorated with `@LoginRequired` decorator. To enable API access simple change

the standard login decorator to `@LoginRequired(api_access=True)`.

To make this operation easier, starting from version 1.7.0 there's a white list

of views that will have API access enabled. Simply edit `api_access_controllers_whitelist`

option in your .ini file, and define views that should have API access enabled.

Following example shows how to enable API access to patch/diff raw file and archive

in RhodeCode::

    api_access_controllers_whitelist =

        ChangesetController:changeset_patch,

        ChangesetController:changeset_raw,

        FilesController:raw,

        FilesController:archivefile

After this change, a rhodecode view can be accessed without login by adding a

GET parameter `?api_key=<api_key>` to url. By default this is only

enabled on RSS/ATOM feed views. Exposing raw diffs is a good way to integrate with

3rd party services like code review, or build farms that could download archives.

API ACCESS

++++++++++

All clients are required to send JSON-RPC spec JSON data::

    {

        "id:"<id>",

        "api_key":"<api_key>",

        "method":"<method_name>",

        "args":{"<arg_key>":"<arg_val>"}

    }

Example call for autopulling remotes repos using curl::

    curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

Simply provide

 - *id* A value of any type, which is used to match the response with the request that it is replying to.

 - *api_key* for access and permission validation.

 - *method* is name of method to call

 - *args* is an key:value list of arguments to pass to method

.. note::

    api_key can be found in your user account page

RhodeCode API will return always a JSON-RPC response::

    {

        "id":<id>, # matching id sent by request

        "result": "<result>"|null, # JSON formatted result, null if any errors

        "error": "null"|<error_message> # JSON formatted error (if any)

    }

All responses from API will be `HTTP/1.0 200 OK`, if there's an error while

calling api *error* key from response will contain failure description

and result will be null.

API CLIENT

++++++++++

From version 1.4 RhodeCode adds a script that allows to easily

will be available.

To get started quickly simply run::

This will create a file named .config in the directory you executed it storing

json config file with credentials. You can skip this step and always provide

both of the arguments to be able to communicate with server

after that simply run any api command for example get_repo::

 calling {"api_key": "<apikey>", "id": 75, "args": {}, "method": "get_repo"} to http://127.0.0.1:5000

 rhodecode said:

 {'error': 'Missing non optional `repoid` arg in JSON DATA',

  'id': 75,

  'result': None}

Ups looks like we forgot to add an argument

Let's try again now giving the repoid as parameters::

    calling {"api_key": "<apikey>", "id": 39, "args": {"repoid": "rhodecode"}, "method": "get_repo"} to http://127.0.0.1:5000

    rhodecode said:

    {'error': None,

     'id': 39,

     'result': <json data...>}

API METHODS

+++++++++++

pull

----

Pulls given repo from remote location. Can be used to automatically keep

remote repos up to date. This command can be executed only using api_key

belonging to user with admin rights

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "pull"

    args :    {

                "repoid" : "<reponame or repo_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "Pulled from `<reponame>`"

    error :  null

rescan_repos

------------

Dispatch rescan repositories action. If remove_obsolete is set

RhodeCode will delete repos that are in database but not in the filesystem.

This command can be executed only using api_key belonging to user with admin

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "rescan_repos"

    args :    {

                "remove_obsolete" : "<boolean = Optional(False)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "{'added': [<list of names of added repos>],

               'removed': [<list of names of removed repos>]}"

    error :  null

invalidate_cache

----------------

Invalidate cache for repository.

This command can be executed only using api_key belonging to user with admin

rights or regular user that have write or admin or write access to repository.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "invalidate_cache"

    args :    {

                "repoid" : "<reponame or repo_id>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : "Caches of repository `<reponame>`"

    error :  null

lock

----

Set locking state on given repository by given user. If userid param is skipped

, then it is set to id of user whos calling this method. If locked param is skipped

then function shows current lock state of given repo.

This command can be executed only using api_key belonging to user with admin

rights or regular user that have admin or write access to repository.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "lock"

    args :    {

                "repoid" : "<reponame or repo_id>"

                "userid" : "<user_id or username = Optional(=apiuser)>",

                "locked" : "<bool true|false = Optional(=None)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result : {

                 "repo": "<reponame>",

                 "locked": "<bool true|false>",

                 "locked_since": "<float lock_time>",

                 "locked_by": "<username>",

                 "msg": "User `<username>` set lock state for repo `<reponame>` to `<false|true>`"

             }

    error :  null

show_ip

-------

Shows IP address as seen from RhodeCode server, together with all

defined IP addresses for given user.

This command can be executed only using api_key belonging to user with admin

rights.

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "show_ip"

    args :    {

                "userid" : "<user_id or username>",

              }

OUTPUT::

    id : <id_given_in_input>

    result : {

                 "ip_addr_server": <ip_from_clien>",

                 "user_ips": [

                                {

                                   "ip_addr": "<ip_with_mask>",

                                   "ip_range": ["<start_ip>", "<end_ip>"],

                                },

                                ...

                             ]

             }

    error :  null

get_user

--------

Gets an user by username or user_id, Returns empty result if user is not found.

If userid param is skipped it is set to id of user who is calling this method.

This command can be executed only using api_key belonging to user with admin

rights, or regular users that cannot specify different userid than theirs

INPUT::

    id : <id_for_response>

    api_key : "<api_key>"

    method :  "get_user"

    args :    {

                "userid" : "<username or user_id Optional(=apiuser)>"

              }

OUTPUT::

    id : <id_given_in_input>

    result: None if user does not exist or

            {

                "user_id" :     "<user_id>",

                "api_key" :     "<api_key>",

                "username" :    "<username>",

                "firstname":    "<firstname>",

                "lastname" :    "<lastname>",

                "email" :       "<email>",

                "emails":       "<list_of_all_additional_emails>",

                "ip_addresses": "<list_of_ip_addresses_for_user>",

                "active" :      "<bool>",

                "admin" :       "<bool>",

                "ldap_dn" :     "<ldap_dn>",

                "last_login":   "<last_login>",

                "permissions": {

                    "global": ["hg.create.repository",

                               "repository.read",

                               "hg.register.manual_activate"],

                    "repositories": {"repo1": "repository.none"},

                    "repositories_groups": {"Group1": "group.read"}

                 },

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

kallithea.bin.api

~~~~~~~~~~~~~~~~~

Api CLI client for RhodeCode

:created_on: Jun 3, 2012

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH.

:license: GPLv3, see LICENSE for more details.

"""

from __future__ import with_statement

import sys

import argparse

from kallithea.bin.base import json, api_call, RcConf, FORMAT_JSON, FORMAT_PRETTY

def argparser(argv):

    usage = (

      "[--config=CONFIG] [--save-config] "

      "METHOD <key:val> <key2:val> ...\n"

    )

    parser = argparse.ArgumentParser(description='RhodeCode API cli',

                                     usage=usage)

    ## config

    group = parser.add_argument_group('config')

    group.add_argument('--apikey', help='api access key')

    group.add_argument('--apihost', help='api host')

    group.add_argument('--config', help='config file')

    group.add_argument('--save-config', action='store_true', help='save the given config into a file')

    group = parser.add_argument_group('API')

    group.add_argument('method', metavar='METHOD', nargs='?', type=str, default=None,

            help='API method name to call followed by key:value attributes',

    )

    group.add_argument('--format', dest='format', type=str,

            help='output format default: `%s` can '

                 'be also `%s`' % (FORMAT_PRETTY, FORMAT_JSON),

            default=FORMAT_PRETTY

    )

    args, other = parser.parse_known_args()

    return parser, args, other

def main(argv=None):

    """

    Main execution function for cli

    :param argv:

    """

    if argv is None:

        argv = sys.argv

    conf = None

    parser, args, other = argparser(argv)

    api_credentials_given = (args.apikey and args.apihost)

    if args.save_config:

        if not api_credentials_given:

            raise parser.error('--save-config requires --apikey and --apihost')

        conf = RcConf(config_location=args.config,

                      autocreate=True, config={'apikey': args.apikey,

                                               'apihost': args.apihost})

        sys.exit()

    if not conf:

        conf = RcConf(config_location=args.config, autoload=True)

        if not conf:

            if not api_credentials_given:

                parser.error('Could not find config file and missing '

                             '--apikey or --apihost in params')

    apikey = args.apikey or conf['apikey']

    apihost = args.apihost or conf['apihost']

    method = args.method

    # if we don't have method here it's an error

    if not method:

        parser.error('Please specify method name')

    try:

        margs = dict(map(lambda s: s.split(':', 1), other))

    except Exception:

        sys.stderr.write('Error parsing arguments \n')

        sys.exit()

    if args.format == FORMAT_PRETTY:

        print 'Calling method %s => %s' % (method, apihost)

    json_resp = api_call(apikey, apihost, method, **margs)

    error_prefix = ''

    if json_resp['error']:

        error_prefix = 'ERROR:'

        json_data = json_resp['error']

    else:

        json_data = json_resp['result']

    if args.format == FORMAT_JSON:

        print json.dumps(json_data)

    elif args.format == FORMAT_PRETTY:

        print 'Server response \n%s%s' % (

            error_prefix, json.dumps(json_data, indent=4, sort_keys=True)

        )

    return 0

if __name__ == '__main__':

    sys.exit(main(sys.argv))

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

kallithea.bin.backup_manager

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Repositories backup manager, it allows to backups all

repositories and send it to backup server using RSA key via ssh.

:created_on: Feb 28, 2010

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH.

:license: GPLv3, see LICENSE for more details.

"""

import os

import sys

import logging

import tarfile

import datetime

import subprocess

logging.basicConfig(level=logging.DEBUG,

                    format="%(asctime)s %(levelname)-5.5s %(message)s")

class BackupManager(object):

    def __init__(self, repos_location, rsa_key, backup_server):

        today = datetime.datetime.now().weekday() + 1

        self.id_rsa_path = self.get_id_rsa(rsa_key)

        self.repos_path = self.get_repos_path(repos_location)

        self.backup_server = backup_server

        self.backup_file_path = '/tmp'

        logging.info('starting backup for %s', self.repos_path)

        logging.info('backup target %s', self.backup_file_path)

    def get_id_rsa(self, rsa_key):

        if not os.path.isfile(rsa_key):

            logging.error('Could not load id_rsa key file in %s', rsa_key)

            sys.exit()

        return rsa_key

    def get_repos_path(self, path):

        if not os.path.isdir(path):

            logging.error('Wrong location for repositories in %s', path)

            sys.exit()

        return path

    def backup_repos(self):

        bckp_file = os.path.join(self.backup_file_path, self.backup_file_name)

        tar = tarfile.open(bckp_file, "w:gz")

        for dir_name in os.listdir(self.repos_path):

            logging.info('backing up %s', dir_name)

            tar.add(os.path.join(self.repos_path, dir_name), dir_name)

        tar.close()

        logging.info('finished backup of mercurial repositories')

    def transfer_files(self):

        params = {

                  'id_rsa_key': self.id_rsa_path,

                  'backup_file': os.path.join(self.backup_file_path,

                                             self.backup_file_name),

                  'backup_server': self.backup_server

                  }

        cmd = ['scp', '-l', '40000', '-i', '%(id_rsa_key)s' % params,

               '%(backup_file)s' % params,

               '%(backup_server)s' % params]

        subprocess.call(cmd)

        logging.info('Transfered file %s to %s', self.backup_file_name, cmd[4])

    def rm_file(self):

        logging.info('Removing file %s', self.backup_file_name)

        os.remove(os.path.join(self.backup_file_path, self.backup_file_name))

if __name__ == "__main__":

    repo_location = '/home/repo_path'

    backup_server = 'root@192.168.1.100:/backups/mercurial'

    rsa_key = '/home/id_rsa'

    B_MANAGER = BackupManager(repo_location, rsa_key, backup_server)

    B_MANAGER.backup_repos()

    B_MANAGER.transfer_files()

    B_MANAGER.rm_file()

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

configuration generator for RhodeCode

:created_on: Jun 18, 2013

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH.

:license: GPLv3, see LICENSE for more details.

"""

from __future__ import with_statement

import os

import sys

import uuid

import argparse

from mako.template import Template

TMPL = 'template.ini.mako'

here = os.path.dirname(os.path.abspath(__file__))

def argparser(argv):

    usage = (

      "VARS optional specify extra template variable that will be available in "

      "template. Use comma separated key=val format eg.\n"

      "key1=val1,port=5000,host=127.0.0.1,elements='a\,b\,c'\n"

    )

    parser = argparse.ArgumentParser(

        description='RhodeCode CONFIG generator with variable replacement',

        usage=usage

    )

    ## config

    group = parser.add_argument_group('CONFIG')

    group.add_argument('--filename', help='Output ini filename.')

    group.add_argument('--template', help='Mako template file to use instead of '

                                          'the default builtin template')

    group.add_argument('--raw', help='Store given mako template as raw without '

                                     'parsing. Use this to create custom template '

                                     'initially', action='store_true')

    group.add_argument('--show-defaults', help='Show all default variables for '

                                               'builtin template', action='store_true')

    args, other = parser.parse_known_args()

    return parser, args, other

def _escape_split(text, sep):

    """

    Allows for escaping of the separator: e.g. arg='foo\, bar'

    It should be noted that the way bash et. al. do command line parsing, those

    single quotes are required. a shameless ripoff from fabric project.

    """

    escaped_sep = r'\%s' % sep

    if escaped_sep not in text:

        return text.split(sep)

    before, _, after = text.partition(escaped_sep)

    startlist = before.split(sep)  # a regular split is fine here

    unfinished = startlist[-1]

    startlist = startlist[:-1]

    # recurse because there may be more escaped separators

    endlist = _escape_split(after, sep)

    # finish building the escaped value. we use endlist[0] becaue the first

    # part of the string sent in recursion is the rest of the escaped value.

    unfinished += sep + endlist[0]

    return startlist + [unfinished] + endlist[1:]  # put together all the parts

def _run(argv):

    parser, args, other = argparser(argv)

    if not len(sys.argv) > 1:

        print parser.print_help()

        sys.exit(0)

    # defaults that can be overwritten by arguments

    tmpl_stored_args = {

        'http_server': 'waitress',

        'lang': 'en',

        'database_engine': 'sqlite',

        'host': '127.0.0.1',

        'port': 5000,

        'error_aggregation_service': None,

    }

    if other:

        # parse arguments, we assume only first is correct

        kwargs = {}

        for el in _escape_split(other[0], ','):

            kv = _escape_split(el, '=')

            if len(kv) == 2:

                k, v = kv

                kwargs[k] = v

        # update our template stored args

        tmpl_stored_args.update(kwargs)

    # use default that cannot be replaced

    tmpl_stored_args.update({

        'uuid': lambda: uuid.uuid4().hex,

        'here': os.path.abspath(os.curdir),

    })

    if args.show_defaults:

        for k,v in tmpl_stored_args.iteritems():

            print '%s=%s' % (k, v)

        sys.exit(0)

    try:

        # built in template

        tmpl_file = os.path.join(here, TMPL)

        if args.template:

            tmpl_file = args.template

        with open(tmpl_file, 'rb') as f:

            tmpl_data = f.read()

            if args.raw:

                tmpl = tmpl_data

            else:

                tmpl = Template(tmpl_data).render(**tmpl_stored_args)

        with open(args.filename, 'wb') as f:

            f.write(tmpl)

        print 'Wrote new config file in %s' % (os.path.abspath(args.filename))

    except Exception:

        from mako import exceptions

        print exceptions.text_error_template().render()

def main(argv=None):

    """

    Main execution function for cli

    :param argv:

    """

    if argv is None:

        argv = sys.argv

    try:

        return _run(argv)

    except Exception:

        raise

if __name__ == '__main__':

    sys.exit(main(sys.argv))

# -*- coding: utf-8 -*-

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.

"""

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Gist CLI client for RhodeCode

:created_on: May 9, 2013

:author: marcink

:copyright: (c) 2013 RhodeCode GmbH.

:license: GPLv3, see LICENSE for more details.

"""

from __future__ import with_statement

import os

import sys

import stat

import argparse

import fileinput

from kallithea.bin.base import json, api_call, RcConf, FORMAT_JSON, FORMAT_PRETTY

def argparser(argv):

    usage = (

      "[--config=CONFIG] [--save-config] [GIST OPTIONS] "

      "[filename or stdin use - for terminal stdin ]\n"

    )

    parser = argparse.ArgumentParser(description='RhodeCode Gist cli',

                                     usage=usage)

    ## config

    group = parser.add_argument_group('config')

    group.add_argument('--apikey', help='api access key')

    group.add_argument('--apihost', help='api host')

    group.add_argument('--config', help='config file path DEFAULT: ~/.config/kallithea')

    group.add_argument('--save-config', action='store_true',

                       help='save the given config into a file')

    group = parser.add_argument_group('GIST')

    group.add_argument('-p', '--private', action='store_true',

                       help='create private Gist')

    group.add_argument('-f', '--filename',

                       help='set uploaded gist filename, '

                            'also defines syntax highlighting')

    group.add_argument('-d', '--description', help='Gist description')

    group.add_argument('-l', '--lifetime', metavar='MINUTES',

                       help='gist lifetime in minutes, -1 (DEFAULT) is forever')

    group.add_argument('--format', dest='format', type=str,

                       help='output format DEFAULT: `%s` can '

                       'be also `%s`' % (FORMAT_PRETTY, FORMAT_JSON),

            default=FORMAT_PRETTY

    )

    args, other = parser.parse_known_args()

    return parser, args, other

def _run(argv):

    conf = None

    parser, args, other = argparser(argv)

    api_credentials_given = (args.apikey and args.apihost)

    if args.save_config:

        if not api_credentials_given:

            raise parser.error('--save-config requires --apikey and --apihost')

        conf = RcConf(config_location=args.config,

                      autocreate=True, config={'apikey': args.apikey,

                                               'apihost': args.apihost})

        sys.exit()

    if not conf:

        conf = RcConf(config_location=args.config, autoload=True)

        if not conf:

            if not api_credentials_given:

                parser.error('Could not find config file and missing '

                             '--apikey or --apihost in params')

    apikey = args.apikey or conf['apikey']

    host = args.apihost or conf['apihost']

    DEFAULT_FILENAME = 'gistfile1.txt'

    if other:

        # skip multifiles for now

        filename = other[0]

        if filename == '-':

            filename = DEFAULT_FILENAME

            gist_content = ''

            for line in fileinput.input('-'):

                gist_content += line

        else:

            with open(filename, 'rb') as f:

                gist_content = f.read()

    else:

        filename = DEFAULT_FILENAME

        gist_content = None

        # little bit hacky but cross platform check where the

        # stdin comes from we skip the terminal case it can be handled by '-'

        mode = os.fstat(0).st_mode

        if stat.S_ISFIFO(mode):

            # "stdin is piped"

            gist_content = sys.stdin.read()

        elif stat.S_ISREG(mode):

            # "stdin is redirected"

            gist_content = sys.stdin.read()

        else:

            # "stdin is terminal"

            pass

    # make sure we don't upload binary stuff

    if gist_content and '\0' in gist_content:

        raise Exception('Error: binary files upload is not possible')

    filename = os.path.basename(args.filename or filename)

    if gist_content:

        files = {

            filename: {

                'content': gist_content,

                'lexer': None

            }

        }

        margs = dict(

            lifetime=args.lifetime,

            description=args.description,

            gist_type='private' if args.private else 'public',

            files=files

        )

        json_data = api_call(apikey, host, 'create_gist', **margs)['result']

        if args.format == FORMAT_JSON:

            print json.dumps(json_data)

        elif args.format == FORMAT_PRETTY:

            print json_data

            print 'Created %s gist %s' % (json_data['gist']['type'],

                                          json_data['gist']['url'])

    return 0

def main(argv=None):

    """