Changeset - 51bd5404529c
[Not reviewed]
CONTRIBUTORS
Show inline comments
 
List of contributors to RhodeCode project:
 
    Marcin Kuźmiński <marcin@python-works.com>
 
    Lukasz Balcerzak <lukaszbalcerzak@gmail.com>
 
    Jason Harris <jason@jasonfharris.com>
 
    Thayne Harbaugh  <thayne@fusionio.com>
 
    cejones
 
    Thomas Waldmann <tw-public@gmx.de>
 
    Lorenzo M. Catucci <lorenzo@sancho.ccd.uniroma2.it>
 
    Dmitri Kuznetsov
 
    Jared Bunting <jared.bunting@peachjean.com>
 
    Steve Romanow <slestak989@gmail.com>
 
    Augosto Hermann <augusto.herrmann@planejamento.gov.br>    
 
    Ankit Solanki <ankit.solanki@gmail.com>    
 
    Liad Shani <liadff@gmail.com>
 
    
 
    Les Peabody <lpeabody@gmail.com>
docs/changelog.rst
Show inline comments
 
.. _changelog:
 

	
 
Changelog
 
=========
 

	
 

	
 
1.3.0 (**XXXX-XX-XX**)
 
======================
 

	
 
:status: in-progress
 
:branch: beta
 

	
 
news
 
----
 

	
 
 
 
fixes
 
-----
 

	
 

	
 
1.2.2 (**2011-10-17**)
 
======================
 

	
 
news
 
----
 

	
 
- #226 repo groups are available by path instead of numerical id
 
 
 
fixes
 
-----
 

	
 
- #259 Groups with the same name but with different parent group
 
- #260 Put repo in group, then move group to another group -> repo becomes unavailable
 
- #258 RhodeCode 1.2 assumes egg folder is writable (lockfiles problems)
 
- #265 ldap save fails sometimes on converting attributes to booleans, 
 
  added getter and setter into model that will prevent from this on db model level
 
- fixed problems with timestamps issues #251 and #213
 
- fixes #266 Rhodecode allows to create repo with the same name and in 
 
  the same parent as group
 
- fixes #245 Rescan of the repositories on Windows
 
- fixes #248 cannot edit repos inside a group on windows
 
- fixes #219 forking problems on windows
 

	
 
1.2.1 (**2011-10-08**)
 
======================
 

	
 
news
 
----
 

	
 

	
 
fixes
 
-----
 

	
 
- fixed problems with basic auth and push problems 
 
- gui fixes
 
- fixed logger
 

	
 
1.2.0 (**2011-10-07**)
 
======================
 

	
 
news
 
----
 

	
 
- implemented #47 repository groups
 
- implemented #89 Can setup google analytics code from settings menu
 
- implemented #91 added nicer looking archive urls with more download options
 
  like tags, branches
 
- implemented #44 into file browsing, and added follow branch option
 
- implemented #84 downloads can be enabled/disabled for each repository
 
- anonymous repository can be cloned without having to pass default:default
 
  into clone url
 
- fixed #90 whoosh indexer can index chooses repositories passed in command 
 
  line
 
- extended journal with day aggregates and paging
 
- implemented #107 source code lines highlight ranges
 
- implemented #93 customizable changelog on combined revision ranges - 
 
  equivalent of githubs compare view 
 
- implemented #108 extended and more powerful LDAP configuration
 
- implemented #56 users groups
 
- major code rewrites optimized codes for speed and memory usage
 
- raw and diff downloads are now in git format
 
- setup command checks for write access to given path
 
- fixed many issues with international characters and unicode. It uses utf8
 
  decode with replace to provide less errors even with non utf8 encoded strings
 
- #125 added API KEY access to feeds
 
- #109 Repository can be created from external Mercurial link (aka. remote 
 
  repository, and manually updated (via pull) from admin panel
 
- beta git support - push/pull server + basic view for git repos
 
- added followers page and forks page
 
- server side file creation (with binary file upload interface) 
 
  and edition with commits powered by codemirror 
 
- #111 file browser file finder, quick lookup files on whole file tree 
 
- added quick login sliding menu into main page
 
- changelog uses lazy loading of affected files details, in some scenarios 
 
  this can improve speed of changelog page dramatically especially for 
 
  larger repositories.
 
- implements #214 added support for downloading subrepos in download menu.
 
- Added basic API for direct operations on rhodecode via JSON
 
- Implemented advanced hook management
 

	
 
fixes
 
-----
 

	
 
- fixed file browser bug, when switching into given form revision the url was 
 
  not changing
 
- fixed propagation to error controller on simplehg and simplegit middlewares
 
- fixed error when trying to make a download on empty repository
 
- fixed problem with '[' chars in commit messages in journal
 
- fixed #99 Unicode errors, on file node paths with non utf-8 characters
 
- journal fork fixes
 
- removed issue with space inside renamed repository after deletion
 
- fixed strange issue on formencode imports
 
- fixed #126 Deleting repository on Windows, rename used incompatible chars. 
 
- #150 fixes for errors on repositories mapped in db but corrupted in 
 
  filesystem
 
- fixed problem with ascendant characters in realm #181
 
- fixed problem with sqlite file based database connection pool
 
- whoosh indexer and code stats share the same dynamic extensions map
 
- fixes #188 - relationship delete of repo_to_perm entry on user removal
 
- fixes issue #189 Trending source files shows "show more" when no more exist
 
- fixes issue #197 Relative paths for pidlocks
 
- fixes issue #198 password will require only 3 chars now for login form
 
- fixes issue #199 wrong redirection for non admin users after creating a repository
 
- fixes issues #202, bad db constraint made impossible to attach same group 
 
  more than one time. Affects only mysql/postgres
 
- fixes #218 os.kill patch for windows was missing sig param
 
- improved rendering of dag (they are not trimmed anymore when number of 
 
  heads exceeds 5)
 
    
 
1.1.8 (**2011-04-12**)
 
======================
 

	
 
news
 
----
 

	
 
- improved windows support
 

	
 
fixes
 
-----
 

	
 
- fixed #140 freeze of python dateutil library, since new version is python2.x
 
  incompatible
 
- setup-app will check for write permission in given path
 
- cleaned up license info issue #149
 
- fixes for issues #137,#116 and problems with unicode and accented characters.
 
- fixes crashes on gravatar, when passed in email as unicode
 
- fixed tooltip flickering problems
 
- fixed came_from redirection on windows
 
- fixed logging modules, and sql formatters
 
- windows fixes for os.kill issue #133
 
- fixes path splitting for windows issues #148
 
- fixed issue #143 wrong import on migration to 1.1.X
 
- fixed problems with displaying binary files, thanks to Thomas Waldmann
 
- removed name from archive files since it's breaking ui for long repo names
 
- fixed issue with archive headers sent to browser, thanks to Thomas Waldmann
 
- fixed compatibility for 1024px displays, and larger dpi settings, thanks to 
 
  Thomas Waldmann
 
- fixed issue #166 summary pager was skipping 10 revisions on second page
 

	
 

	
 
1.1.7 (**2011-03-23**)
 
======================
 

	
 
news
 
----
 

	
 
fixes
 
-----
 

	
 
- fixed (again) #136 installation support for FreeBSD
 

	
 

	
 
1.1.6 (**2011-03-21**)
 
======================
 

	
 
news
 
----
 

	
 
fixes
 
-----
 

	
 
- fixed #136 installation support for FreeBSD
 
- RhodeCode will check for python version during installation
 

	
 
1.1.5 (**2011-03-17**)
 
======================
 

	
 
news
 
----
 

	
 
- basic windows support, by exchanging pybcrypt into sha256 for windows only
 
  highly inspired by idea of mantis406
 

	
 
fixes
 
-----
 

	
 
- fixed sorting by author in main page
 
- fixed crashes with diffs on binary files
 
- fixed #131 problem with boolean values for LDAP
 
- fixed #122 mysql problems thanks to striker69 
 
- fixed problem with errors on calling raw/raw_files/annotate functions 
 
  with unknown revisions
 
- fixed returned rawfiles attachment names with international character
 
- cleaned out docs, big thanks to Jason Harris
 

	
 
1.1.4 (**2011-02-19**)
 
======================
 

	
 
news
 
----
 

	
 
fixes
 
-----
 

	
 
- fixed formencode import problem on settings page, that caused server crash
 
  when that page was accessed as first after server start
 
- journal fixes
 
- fixed option to access repository just by entering http://server/<repo_name> 
 

	
 
1.1.3 (**2011-02-16**)
 
======================
 

	
 
news
 
----
 

	
 
- implemented #102 allowing the '.' character in username
 
- added option to access repository just by entering http://server/<repo_name>
 
- celery task ignores result for better performance
 

	
 
fixes
 
-----
 

	
 
- fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 
 
  apollo13 and Johan Walles
 
- small fixes in journal
 
- fixed problems with getting setting for celery from .ini files
 
- registration, password reset and login boxes share the same title as main 
 
  application now
 
- fixed #113: to high permissions to fork repository
 
- fixed problem with '[' chars in commit messages in journal
 
- removed issue with space inside renamed repository after deletion
 
- db transaction fixes when filesystem repository creation failed
 
- fixed #106 relation issues on databases different than sqlite
 
- fixed static files paths links to use of url() method
 

	
 
1.1.2 (**2011-01-12**)
 
======================
 

	
 
news
 
----
 

	
 

	
 
fixes
 
-----
 

	
 
- fixes #98 protection against float division of percentage stats
 
- fixed graph bug
 
- forced webhelpers version since it was making troubles during installation 
 

	
 
1.1.1 (**2011-01-06**)
 
======================
 
 
 
news
 
----
 

	
 
- added force https option into ini files for easier https usage (no need to
 
  set server headers with this options)
 
- small css updates
 

	
 
fixes
 
-----
 

	
 
- fixed #96 redirect loop on files view on repositories without changesets
 
- fixed #97 unicode string passed into server header in special cases (mod_wsgi)
 
  and server crashed with errors
 
- fixed large tooltips problems on main page
 
- fixed #92 whoosh indexer is more error proof
 

	
 
1.1.0 (**2010-12-18**)
 
======================
 

	
 
news
 
----
 

	
 
- rewrite of internals for vcs >=0.1.10
 
- uses mercurial 1.7 with dotencode disabled for maintaining compatibility 
 
  with older clients
 
- anonymous access, authentication via ldap
 
- performance upgrade for cached repos list - each repository has it's own 
 
  cache that's invalidated when needed.
 
- performance upgrades on repositories with large amount of commits (20K+)
 
- main page quick filter for filtering repositories
 
- user dashboards with ability to follow chosen repositories actions
 
- sends email to admin on new user registration
 
- added cache/statistics reset options into repository settings
 
- more detailed action logger (based on hooks) with pushed changesets lists
 
  and options to disable those hooks from admin panel
 
- introduced new enhanced changelog for merges that shows more accurate results
 
- new improved and faster code stats (based on pygments lexers mapping tables, 
 
  showing up to 10 trending sources for each repository. Additionally stats
 
  can be disabled in repository settings.
 
- gui optimizations, fixed application width to 1024px
 
- added cut off (for large files/changesets) limit into config files
 
- whoosh, celeryd, upgrade moved to paster command
 
- other than sqlite database backends can be used
 

	
 
fixes
 
-----
 

	
 
- fixes #61 forked repo was showing only after cache expired
 
- fixes #76 no confirmation on user deletes
 
- fixes #66 Name field misspelled
 
- fixes #72 block user removal when he owns repositories
 
- fixes #69 added password confirmation fields
 
- fixes #87 RhodeCode crashes occasionally on updating repository owner
 
- fixes #82 broken annotations on files with more than 1 blank line at the end
 
- a lot of fixes and tweaks for file browser
 
- fixed detached session issues
 
- fixed when user had no repos he would see all repos listed in my account
 
- fixed ui() instance bug when global hgrc settings was loaded for server 
 
  instance and all hgrc options were merged with our db ui() object
 
- numerous small bugfixes
 
 
 
(special thanks for TkSoh for detailed feedback)
 

	
 

	
 
1.0.2 (**2010-11-12**)
 
======================
 

	
 
news
 
----
 

	
 
- tested under python2.7
 
- bumped sqlalchemy and celery versions
 

	
 
fixes
 
-----
 

	
 
- fixed #59 missing graph.js
 
- fixed repo_size crash when repository had broken symlinks
 
- fixed python2.5 crashes.
 

	
 

	
 
1.0.1 (**2010-11-10**)
 
======================
 

	
 
news
 
----
 

	
 
- small css updated
 

	
 
fixes
 
-----
 

	
 
- fixed #53 python2.5 incompatible enumerate calls
 
- fixed #52 disable mercurial extension for web
 
- fixed #51 deleting repositories don't delete it's dependent objects
 

	
 

	
 
1.0.0 (**2010-11-02**)
 
======================
 

	
 
- security bugfix simplehg wasn't checking for permissions on commands
 
  other than pull or push.
 
- fixed doubled messages after push or pull in admin journal
 
- templating and css corrections, fixed repo switcher on chrome, updated titles
 
- admin menu accessible from options menu on repository view
 
- permissions cached queries
 

	
 
1.0.0rc4  (**2010-10-12**)
 
==========================
 

	
 
- fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)
 
- removed cache_manager settings from sqlalchemy meta
 
- added sqlalchemy cache settings to ini files
 
- validated password length and added second try of failure on paster setup-app
 
- fixed setup database destroy prompt even when there was no db
 

	
 

	
 
1.0.0rc3 (**2010-10-11**)
 
=========================
 

	
 
- fixed i18n during installation.
 

	
 
1.0.0rc2 (**2010-10-11**)
 
=========================
 

	
 
- Disabled dirsize in file browser, it's causing nasty bug when dir renames 
 
  occure. After vcs is fixed it'll be put back again.
 
- templating/css rewrites, optimized css.
 
\ No newline at end of file
docs/setup.rst
Show inline comments
 
@@ -62,512 +62,544 @@ repositories. Then choose Add New Reposi
 
into the root. Test that you can browse your repository from within RhodeCode 
 
and then try cloning your repository from RhodeCode with::
 

	
 
    hg clone http://127.0.0.1:5000/<repository name>
 

	
 
where *repository name* is replaced by the name of your repository.
 

	
 
Using RhodeCode with SSH
 
------------------------
 

	
 
RhodeCode currently only hosts repositories using http and https. (The addition
 
of ssh hosting is a planned future feature.) However you can easily use ssh in
 
parallel with RhodeCode. (Repository access via ssh is a standard "out of
 
the box" feature of mercurial_ and you can use this to access any of the
 
repositories that RhodeCode is hosting. See PublishingRepositories_)
 

	
 
RhodeCode repository structures are kept in directories with the same name 
 
as the project. When using repository groups, each group is a subdirectory.
 
This allows you to easily use ssh for accessing repositories.
 

	
 
In order to use ssh you need to make sure that your web-server and the users 
 
login accounts have the correct permissions set on the appropriate directories.
 
(Note that these permissions are independent of any permissions you have set up
 
using the RhodeCode web interface.)
 

	
 
If your main directory (the same as set in RhodeCode settings) is for example
 
set to **/home/hg** and the repository you are using is named `rhodecode`, then
 
to clone via ssh you should run::
 

	
 
    hg clone ssh://user@server.com/home/hg/rhodecode
 

	
 
Using other external tools such as mercurial-server_ or using ssh key based
 
authentication is fully supported.
 

	
 
Note: In an advanced setup, in order for your ssh access to use the same
 
permissions as set up via the RhodeCode web interface, you can create an
 
authentication hook to connect to the rhodecode db and runs check functions for
 
permissions against that.
 
    
 
Setting up Whoosh full text search
 
----------------------------------
 

	
 
Starting from version 1.1 the whoosh index can be build by using the paster
 
command ``make-index``. To use ``make-index`` you must specify the configuration
 
file that stores the location of the index. You may specify the location of the 
 
repositories (`--repo-location`).  If not specified, this value is retrieved 
 
from the RhodeCode database.  This was required prior to 1.2.  Starting from 
 
version 1.2 it is also possible to specify a comma separated list of 
 
repositories (`--index-only`) to build index only on chooses repositories 
 
skipping any other found in repos location
 

	
 
You may optionally pass the option `-f` to enable a full index rebuild. Without
 
the `-f` option, indexing will run always in "incremental" mode.
 

	
 
For an incremental index build use::
 

	
 
	paster make-index production.ini 
 

	
 
For a full index rebuild use::
 

	
 
	paster make-index production.ini -f 
 

	
 

	
 
building index just for chosen repositories is possible with such command::
 
 
 
 paster make-index production.ini --index-only=vcs,rhodecode
 

	
 

	
 
In order to do periodical index builds and keep your index always up to date.
 
It's recommended to do a crontab entry for incremental indexing. 
 
An example entry might look like this::
 
 
 
    /path/to/python/bin/paster make-index /path/to/rhodecode/production.ini 
 
  
 
When using incremental mode (the default) whoosh will check the last
 
modification date of each file and add it to be reindexed if a newer file is
 
available. The indexing daemon checks for any removed files and removes them
 
from index.
 

	
 
If you want to rebuild index from scratch, you can use the `-f` flag as above,
 
or in the admin panel you can check `build from scratch` flag.
 

	
 

	
 
Setting up LDAP support
 
-----------------------
 

	
 
RhodeCode starting from version 1.1 supports ldap authentication. In order
 
to use LDAP, you have to install the python-ldap_ package. This package is 
 
available via pypi, so you can install it by running
 

	
 
using easy_install::
 

	
 
    easy_install python-ldap
 
 
 
using pip::
 

	
 
    pip install python-ldap
 

	
 
.. note::
 
   python-ldap requires some certain libs on your system, so before installing 
 
   it check that you have at least `openldap`, and `sasl` libraries.
 

	
 
LDAP settings are located in admin->ldap section,
 

	
 
Here's a typical ldap setup::
 

	
 
 Connection settings
 
 Enable LDAP          = checked
 
 Host                 = host.example.org
 
 Port                 = 389
 
 Account              = <account>
 
 Password             = <password>
 
 Connection Security  = LDAPS connection
 
 Certificate Checks   = DEMAND
 

	
 
 Search settings
 
 Base DN              = CN=users,DC=host,DC=example,DC=org
 
 LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))
 
 LDAP Search Scope    = SUBTREE
 

	
 
 Attribute mappings
 
 Login Attribute      = uid
 
 First Name Attribute = firstName
 
 Last Name Attribute  = lastName
 
 E-mail Attribute     = mail
 

	
 
.. _enable_ldap:
 

	
 
Enable LDAP : required
 
    Whether to use LDAP for authenticating users.
 

	
 
.. _ldap_host:
 

	
 
Host : required
 
    LDAP server hostname or IP address.
 

	
 
.. _Port:
 

	
 
Port : required
 
    389 for un-encrypted LDAP, 636 for SSL-encrypted LDAP.
 

	
 
.. _ldap_account:
 

	
 
Account : optional
 
    Only required if the LDAP server does not allow anonymous browsing of
 
    records.  This should be a special account for record browsing.  This
 
    will require `LDAP Password`_ below.
 

	
 
.. _LDAP Password:
 

	
 
Password : optional
 
    Only required if the LDAP server does not allow anonymous browsing of
 
    records.
 

	
 
.. _Enable LDAPS:
 

	
 
Connection Security : required
 
    Defines the connection to LDAP server
 

	
 
    No encryption
 
        Plain non encrypted connection
 
        
 
    LDAPS connection
 
        Enable ldaps connection. It will likely require `Port`_ to be set to 
 
        a different value (standard LDAPS port is 636). When LDAPS is enabled 
 
        then `Certificate Checks`_ is required.
 
        
 
    START_TLS on LDAP connection
 
        START TLS connection
 

	
 
.. _Certificate Checks:
 

	
 
Certificate Checks : optional
 
    How SSL certificates verification is handled - this is only useful when
 
    `Enable LDAPS`_ is enabled.  Only DEMAND or HARD offer full SSL security 
 
    while the other options are susceptible to man-in-the-middle attacks.  SSL
 
    certificates can be installed to /etc/openldap/cacerts so that the
 
    DEMAND or HARD options can be used with self-signed certificates or
 
    certificates that do not have traceable certificates of authority.
 

	
 
    NEVER
 
        A serve certificate will never be requested or checked.
 

	
 
    ALLOW
 
        A server certificate is requested.  Failure to provide a
 
        certificate or providing a bad certificate will not terminate the
 
        session.
 

	
 
    TRY
 
        A server certificate is requested.  Failure to provide a
 
        certificate does not halt the session; providing a bad certificate
 
        halts the session.
 

	
 
    DEMAND
 
        A server certificate is requested and must be provided and
 
        authenticated for the session to proceed.
 

	
 
    HARD
 
        The same as DEMAND.
 

	
 
.. _Base DN:
 

	
 
Base DN : required
 
    The Distinguished Name (DN) where searches for users will be performed.
 
    Searches can be controlled by `LDAP Filter`_ and `LDAP Search Scope`_.
 

	
 
.. _LDAP Filter:
 

	
 
LDAP Filter : optional
 
    A LDAP filter defined by RFC 2254.  This is more useful when `LDAP
 
    Search Scope`_ is set to SUBTREE.  The filter is useful for limiting
 
    which LDAP objects are identified as representing Users for
 
    authentication.  The filter is augmented by `Login Attribute`_ below.
 
    This can commonly be left blank.
 

	
 
.. _LDAP Search Scope:
 

	
 
LDAP Search Scope : required
 
    This limits how far LDAP will search for a matching object.
 

	
 
    BASE
 
        Only allows searching of `Base DN`_ and is usually not what you
 
        want.
 

	
 
    ONELEVEL
 
        Searches all entries under `Base DN`_, but not Base DN itself.
 

	
 
    SUBTREE
 
        Searches all entries below `Base DN`_, but not Base DN itself.
 
        When using SUBTREE `LDAP Filter`_ is useful to limit object
 
        location.
 

	
 
.. _Login Attribute:
 

	
 
Login Attribute : required        
 
    The LDAP record attribute that will be matched as the USERNAME or
 
    ACCOUNT used to connect to RhodeCode.  This will be added to `LDAP
 
    Filter`_ for locating the User object.  If `LDAP Filter`_ is specified as
 
    "LDAPFILTER", `Login Attribute`_ is specified as "uid" and the user has
 
    connected as "jsmith" then the `LDAP Filter`_ will be augmented as below
 
    ::
 

	
 
        (&(LDAPFILTER)(uid=jsmith))
 

	
 
.. _ldap_attr_firstname:
 

	
 
First Name Attribute : required
 
    The LDAP record attribute which represents the user's first name.
 

	
 
.. _ldap_attr_lastname:
 

	
 
Last Name Attribute : required
 
    The LDAP record attribute which represents the user's last name.
 

	
 
.. _ldap_attr_email:
 

	
 
Email Attribute : required
 
    The LDAP record attribute which represents the user's email address.
 

	
 
If all data are entered correctly, and python-ldap_ is properly installed
 
users should be granted access to RhodeCode with ldap accounts.  At this
 
time user information is copied from LDAP into the RhodeCode user database.
 
This means that updates of an LDAP user object may not be reflected as a
 
user update in RhodeCode.
 

	
 
If You have problems with LDAP access and believe You entered correct
 
information check out the RhodeCode logs, any error messages sent from LDAP
 
will be saved there.
 

	
 
Active Directory
 
''''''''''''''''
 

	
 
RhodeCode can use Microsoft Active Directory for user authentication.  This
 
is done through an LDAP or LDAPS connection to Active Directory.  The
 
following LDAP configuration settings are typical for using Active
 
Directory ::
 

	
 
 Base DN              = OU=SBSUsers,OU=Users,OU=MyBusiness,DC=v3sys,DC=local
 
 Login Attribute      = sAMAccountName
 
 First Name Attribute = givenName
 
 Last Name Attribute  = sn
 
 E-mail Attribute     = mail
 

	
 
All other LDAP settings will likely be site-specific and should be
 
appropriately configured.
 

	
 

	
 

	
 
Hook management
 
---------------
 

	
 
Hooks can be managed in similar way to this used in .hgrc files.
 
To access hooks setting click `advanced setup` on Hooks section of Mercurial
 
Settings in Admin. 
 

	
 
There are 4 built in hooks that cannot be changed (only enable/disable by
 
checkboxes on previos section).
 
To add another custom hook simply fill in first section with 
 
<name>.<hook_type> and the second one with hook path. Example hooks
 
can be found at *rhodecode.lib.hooks*. 
 

	
 

	
 
Setting Up Celery
 
-----------------
 

	
 
Since version 1.1 celery is configured by the rhodecode ini configuration files.
 
Simply set use_celery=true in the ini file then add / change the configuration 
 
variables inside the ini file.
 

	
 
Remember that the ini files use the format with '.' not with '_' like celery.
 
So for example setting `BROKER_HOST` in celery means setting `broker.host` in
 
the config file.
 

	
 
In order to start using celery run::
 

	
 
 paster celeryd <configfile.ini>
 

	
 

	
 
.. note::
 
   Make sure you run this command from the same virtualenv, and with the same 
 
   user that rhodecode runs.
 
   
 
HTTPS support
 
-------------
 

	
 
There are two ways to enable https:
 

	
 
- Set HTTP_X_URL_SCHEME in your http server headers, than rhodecode will
 
  recognize this headers and make proper https redirections
 
- Alternatively, change the `force_https = true` flag in the ini configuration 
 
  to force using https, no headers are needed than to enable https
 

	
 

	
 
Nginx virtual host example
 
--------------------------
 

	
 
Sample config for nginx using proxy::
 

	
 
    server {
 
       listen          80;
 
       server_name     hg.myserver.com;
 
       access_log      /var/log/nginx/rhodecode.access.log;
 
       error_log       /var/log/nginx/rhodecode.error.log;
 
       location / {
 
               root /var/www/rhodecode/rhodecode/public/;
 
               if (!-f $request_filename){
 
                   proxy_pass      http://127.0.0.1:5000;
 
               }
 
               #this is important if you want to use https !!!
 
               proxy_set_header X-Url-Scheme $scheme;
 
               include         /etc/nginx/proxy.conf;  
 
       }
 
    }  
 
  
 
Here's the proxy.conf. It's tuned so it will not timeout on long
 
pushes or large pushes::
 

	
 
    proxy_redirect              off;
 
    proxy_set_header            Host $host;
 
    proxy_set_header            X-Host $http_host;
 
    proxy_set_header            X-Real-IP $remote_addr;
 
    proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
 
    proxy_set_header            Proxy-host $proxy_host;
 
    client_max_body_size        400m;
 
    client_body_buffer_size     128k;
 
    proxy_buffering             off;
 
    proxy_connect_timeout       7200;
 
    proxy_send_timeout          7200;
 
    proxy_read_timeout          7200;
 
    proxy_buffers               8 32k;
 
 
 
Also, when using root path with nginx you might set the static files to false
 
in the production.ini file::
 

	
 
    [app:main]
 
      use = egg:rhodecode
 
      full_stack = true
 
      static_files = false
 
      lang=en
 
      cache_dir = %(here)s/data
 

	
 
In order to not have the statics served by the application. This improves speed.
 

	
 

	
 
Apache virtual host example
 
---------------------------
 
Apache virtual host reverse proxy example
 
-----------------------------------------
 

	
 
Here is a sample configuration file for apache using proxy::
 

	
 
    <VirtualHost *:80>
 
            ServerName hg.myserver.com
 
            ServerAlias hg.myserver.com
 
    
 
            <Proxy *>
 
              Order allow,deny
 
              Allow from all
 
            </Proxy>
 
    
 
            #important !
 
            #Directive to properly generate url (clone url) for pylons
 
            ProxyPreserveHost On
 
    
 
            #rhodecode instance
 
            ProxyPass / http://127.0.0.1:5000/
 
            ProxyPassReverse / http://127.0.0.1:5000/
 
            
 
            #to enable https use line below
 
            #SetEnvIf X-Url-Scheme https HTTPS=1
 
            
 
    </VirtualHost> 
 

	
 

	
 
Additional tutorial
 
http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons
 

	
 

	
 
Apache as subdirectory
 
----------------------
 

	
 
Apache subdirectory part::
 

	
 
    <Location /<someprefix> >
 
      ProxyPass http://127.0.0.1:5000/<someprefix>
 
      ProxyPassReverse http://127.0.0.1:5000/<someprefix>
 
      SetEnvIf X-Url-Scheme https HTTPS=1
 
    </Location> 
 

	
 
Besides the regular apache setup you will need to add the following line
 
into [app:main] section of your .ini file::
 

	
 
    filter-with = proxy-prefix
 

	
 
Add the following at the end of the .ini file::
 

	
 
    [filter:proxy-prefix]
 
    use = egg:PasteDeploy#prefix
 
    prefix = /<someprefix> 
 

	
 

	
 
then change <someprefix> into your choosen prefix
 

	
 
Apache's WSGI config
 
--------------------
 

	
 
Alternatively, RhodeCode can be set up with Apache under mod_wsgi. For
 
that, you'll need to:
 

	
 
- Install mod_wsgi. If using a Debian-based distro, you can install
 
  the package libapache2-mod-wsgi::
 

	
 
    aptitude install libapache2-mod-wsgi
 

	
 
- Enable mod_wsgi::
 

	
 
    a2enmod wsgi
 

	
 
- Create a wsgi dispatch script, like the one below. Make sure you
 
  check the paths correctly point to where you installed RhodeCode
 
  and its Python Virtual Environment.
 
- Enable the WSGIScriptAlias directive for the wsgi dispatch script,
 
  as in the following example. Once again, check the paths are
 
  correctly specified.
 

	
 
Here is a sample excerpt from an Apache Virtual Host configuration file::
 

	
 
    WSGIDaemonProcess pylons user=www-data group=www-data processes=1 \
 
        threads=4 \
 
        python-path=/home/web/rhodecode/pyenv/lib/python2.6/site-packages
 
    WSGIScriptAlias / /home/web/rhodecode/dispatch.wsgi
 

	
 
Example wsgi dispatch script::
 

	
 
    import os
 
    os.environ["HGENCODING"] = "UTF-8"
 
    os.environ['PYTHON_EGG_CACHE'] = '/home/web/rhodecode/.egg-cache'
 
    
 
    # sometimes it's needed to set the curent dir
 
    os.chdir('/home/web/rhodecode/') 
 

	
 
    import site
 
    site.addsitedir("/home/web/rhodecode/pyenv/lib/python2.6/site-packages")
 
    
 
    from paste.deploy import loadapp
 
    from paste.script.util.logging_config import fileConfig
 

	
 
    fileConfig('/home/web/rhodecode/production.ini')
 
    application = loadapp('config:/home/web/rhodecode/production.ini')
 

	
 
Note: when using mod_wsgi you'll need to install the same version of
 
Mercurial that's inside RhodeCode's virtualenv also on the system's Python
 
environment.
 

	
 

	
 
Other configuration files
 
-------------------------
 

	
 
Some example init.d scripts can be found here, for debian and gentoo:
 

	
 
https://rhodecode.org/rhodecode/files/tip/init.d
 

	
 

	
 
Troubleshooting
 
---------------
 

	
 
:Q: **Missing static files?**
 
:A: Make sure either to set the `static_files = true` in the .ini file or
 
   double check the root path for your http setup. It should point to 
 
   for example:
 
   /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public
 
   
 
| 
 

	
 
:Q: **Can't install celery/rabbitmq**
 
:A: Don't worry RhodeCode works without them too. No extra setup is required.
 

	
 
|
 
 
 
:Q: **Long lasting push timeouts?**
 
:A: Make sure you set a longer timeouts in your proxy/fcgi settings, timeouts
 
    are caused by https server and not RhodeCode.
 
    
 
| 
 

	
 
:Q: **Large pushes timeouts?**
 
:A: Make sure you set a proper max_body_size for the http server.
 

	
 
|
 

	
 
:Q: **Apache doesn't pass basicAuth on pull/push?**
 
:A: Make sure you added `WSGIPassAuthorization true`.
 

	
 
For further questions search the `Issues tracker`_, or post a message in the 
 
`google group rhodecode`_
 

	
 
.. _virtualenv: http://pypi.python.org/pypi/virtualenv
 
.. _python: http://www.python.org/
 
.. _mercurial: http://mercurial.selenic.com/
 
.. _celery: http://celeryproject.org/
 
.. _rabbitmq: http://www.rabbitmq.com/
 
.. _python-ldap: http://www.python-ldap.org/
 
.. _mercurial-server: http://www.lshift.net/mercurial-server.html
 
.. _PublishingRepositories: http://mercurial.selenic.com/wiki/PublishingRepositories
 
.. _Issues tracker: https://bitbucket.org/marcinkuzminski/rhodecode/issues
 
.. _google group rhodecode: http://groups.google.com/group/rhodecode
init.d/rhodecode-daemon3
Show inline comments
 
new file 100644
 
#!/bin/sh
 
########################################
 
#### THIS IS A REDHAT INIT.D SCRIPT ####
 
########################################
 

	
 
##################################################
 
#
 
# RhodeCode server startup script
 
# Recommended default-startup: 2 3 4 5
 
# Recommended default-stop: 0 1 6
 
#
 
##################################################
 

	
 

	
 
APP_NAME="rhodecode"
 
# the location of your app
 
# since this is a web app, it should go in /var/www
 
APP_PATH="/var/www/$APP_NAME"
 

	
 
CONF_NAME="production.ini"
 

	
 
# write to wherever the PID should be stored, just ensure
 
# that the user you run paster as has the appropriate permissions
 
# same goes for the log file
 
PID_PATH="/var/run/rhodecode/pid"
 
LOG_PATH="/var/log/rhodecode/rhodecode.log"
 

	
 
# replace this with the path to the virtual environment you
 
# made for RhodeCode
 
PYTHON_PATH="/opt/python_virtualenvironments/rhodecode-venv"
 

	
 
RUN_AS="rhodecode"
 

	
 
DAEMON="$PYTHON_PATH/bin/paster"
 

	
 
DAEMON_OPTS="serve --daemon \
 
    --user=$RUN_AS \
 
    --group=$RUN_AS \
 
    --pid-file=$PID_PATH \
 
    --log-file=$LOG_PATH $APP_PATH/$CONF_NAME"
 

	
 
DESC="rhodecode-server"
 
LOCK_FILE="/var/lock/subsys/$APP_NAME"
 

	
 
# source CentOS init functions
 
. /etc/init.d/functions
 

	
 
RETVAL=0
 

	
 
remove_pid () {
 
  rm -f ${PID_PATH}
 
  rmdir `dirname ${PID_PATH}`
 
}
 

	
 
ensure_pid_dir () {
 
  PID_DIR=`dirname ${PID_PATH}`
 
  if [ ! -d ${PID_DIR} ] ; then
 
    mkdir -p ${PID_DIR}
 
    chown -R ${RUN_AS}:${RUN_AS} ${PID_DIR}
 
    chmod 755 ${PID_DIR}
 
  fi
 
}
 

	
 
start_rhodecode () {
 
    ensure_pid_dir
 
    PYTHON_EGG_CACHE="/tmp" daemon --pidfile $PID_PATH \
 
        --user $RUN_AS "$DAEMON $DAEMON_OPTS"
 
    RETVAL=$?
 
    [ $RETVAL -eq 0 ] && touch $LOCK_FILE
 
    return $RETVAL
 
}
 

	
 
stop_rhodecode () {
 
    if [ -e $LOCK_FILE ]; then
 
      killproc -p $PID_PATH
 
      RETVAL=$?
 
      rm -f $LOCK_FILE
 
      rm -f $PID_PATH
 
    else
 
      RETVAL=1
 
    fi
 
    return $RETVAL
 
}
 

	
 
status_rhodecode() {
 
  if [ -e $LOCK_FILE ]; then
 
    # exit with non-zero to indicate failure
 
    RETVAL=1
 
  else
 
    RETVAL=0
 
  fi
 
  return $RETVAL
 
}
 

	
 
restart_rhodecode () {
 
    stop_rhodecode
 
    start_rhodecode
 
    RETVAL=$?
 
}
 

	
 
case "$1" in
 
  start)
 
    echo -n $"Starting $DESC: "
 
    start_rhodecode
 
    echo
 
    ;;
 
  stop)
 
    echo -n $"Stopping $DESC: "
 
    stop_rhodecode
 
    echo
 
    ;;
 
  status)
 
    status_rhodecode
 
    RETVAL=$?
 
    if [ ! $RETVAL -eq 0 ]; then
 
      echo "RhodeCode server is running..."
 
    else
 
      echo "RhodeCode server is stopped."
 
    fi
 
    ;;
 
  restart)
 
    echo -n $"Restarting $DESC: "
 
    restart_rhodecode
 
    echo
 
    ;;
 
  *)
 
    echo $"Usage: $0 {start|stop|restart|status}"
 
    RETVAL=1
 
    ;;
 
esac
 

	
 
exit $RETVAL
 
\ No newline at end of file
rhodecode/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.__init__
 
    ~~~~~~~~~~~~~~~~~~
 

	
 
    RhodeCode, a web based repository management based on pylons
 
    versioning implementation: http://semver.org/
 

	
 
    :created_on: Apr 9, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import platform
 

	
 
VERSION = (1, 3, 0, 'beta')
 
__version__ = '.'.join((str(each) for each in VERSION[:4]))
 
__dbversion__ = 4 #defines current db version for migrations
 
__platform__ = platform.system()
 
__license__ = 'GPLv3'
 

	
 
PLATFORM_WIN = ('Windows')
 
PLATFORM_OTHERS = ('Linux', 'Darwin', 'FreeBSD', 'OpenBSD', 'SunOS')
 

	
 
try:
 
    from rhodecode.lib.utils import get_current_revision
 
    from rhodecode.lib import get_current_revision
 
    _rev = get_current_revision()
 
except ImportError:
 
    #this is needed when doing some setup.py operations
 
    _rev = False
 

	
 
if len(VERSION) > 3 and _rev:
 
    __version__ += ' [rev:%s]' % _rev[0]
 

	
 

	
 
def get_version():
 
    """Returns shorter version (digit parts only) as string."""
 

	
 
    return '.'.join((str(each) for each in VERSION[:3]))
 

	
 
BACKENDS = {
 
    'hg': 'Mercurial repository',
 
    #'git': 'Git repository',
 
}
rhodecode/config/routing.py
Show inline comments
 
"""
 
Routes configuration
 

	
 
The more specific and detailed routes should be defined first so they
 
may take precedent over the more generic routes. For more information
 
refer to the routes manual at http://routes.groovie.org/docs/
 
"""
 
from __future__ import with_statement
 
from routes import Mapper
 

	
 

	
 
# prefix for non repository related links needs to be prefixed with `/`
 
ADMIN_PREFIX = '/_admin'
 

	
 

	
 
def make_map(config):
 
    """Create, configure and return the routes Mapper"""
 
    rmap = Mapper(directory=config['pylons.paths']['controllers'],
 
                 always_scan=config['debug'])
 
    rmap.minimization = False
 
    rmap.explicit = False
 
    
 

	
 
    from rhodecode.lib.utils import is_valid_repo
 
    from rhodecode.lib.utils import is_valid_repos_group
 
    
 

	
 
    def check_repo(environ, match_dict):
 
        """
 
        check for valid repository for proper 404 handling
 
        
 
        :param environ:
 
        :param match_dict:
 
        """
 
         
 

	
 
        repo_name = match_dict.get('repo_name')
 
        return is_valid_repo(repo_name, config['base_path'])
 

	
 
    def check_group(environ, match_dict):
 
        """
 
        check for valid repositories group for proper 404 handling
 
        
 
        :param environ:
 
        :param match_dict:
 
        """
 
        repos_group_name = match_dict.get('group_name')
 
        
 

	
 
        return is_valid_repos_group(repos_group_name, config['base_path'])
 

	
 

	
 
    def check_int(environ, match_dict):
 
        return match_dict.get('id').isdigit()
 

	
 
    # The ErrorController route (handles 404/500 error pages); it should
 
    # likely stay at the top, ensuring it can always be resolved
 
    rmap.connect('/error/{action}', controller='error')
 
    rmap.connect('/error/{action}/{id}', controller='error')
 

	
 
    #==========================================================================
 
    # CUSTOM ROUTES HERE
 
    #==========================================================================
 

	
 
    #MAIN PAGE
 
    rmap.connect('home', '/', controller='home', action='index')
 
    rmap.connect('repo_switcher', '/repos', controller='home',
 
                 action='repo_switcher')
 
    rmap.connect('bugtracker',
 
                 "http://bitbucket.org/marcinkuzminski/rhodecode/issues",
 
                 _static=True)
 
    rmap.connect('rhodecode_official', "http://rhodecode.org", _static=True)
 

	
 
    #ADMIN REPOSITORY REST ROUTES
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/repos') as m:
 
        m.connect("repos", "/repos",
 
             action="create", conditions=dict(method=["POST"]))
 
        m.connect("repos", "/repos",
 
             action="index", conditions=dict(method=["GET"]))
 
        m.connect("formatted_repos", "/repos.{format}",
 
             action="index",
 
            conditions=dict(method=["GET"]))
 
        m.connect("new_repo", "/repos/new",
 
             action="new", conditions=dict(method=["GET"]))
 
        m.connect("formatted_new_repo", "/repos/new.{format}",
 
             action="new", conditions=dict(method=["GET"]))
 
        m.connect("/repos/{repo_name:.*}",
 
             action="update", conditions=dict(method=["PUT"],
 
                                              function=check_repo))
 
        m.connect("/repos/{repo_name:.*}",
 
             action="delete", conditions=dict(method=["DELETE"],
 
                                              function=check_repo))
 
        m.connect("edit_repo", "/repos/{repo_name:.*}/edit",
 
             action="edit", conditions=dict(method=["GET"],
 
                                            function=check_repo))
 
        m.connect("formatted_edit_repo", "/repos/{repo_name:.*}.{format}/edit",
 
             action="edit", conditions=dict(method=["GET"],
 
                                            function=check_repo))
 
        m.connect("repo", "/repos/{repo_name:.*}",
 
             action="show", conditions=dict(method=["GET"],
 
                                            function=check_repo))
 
        m.connect("formatted_repo", "/repos/{repo_name:.*}.{format}",
 
             action="show", conditions=dict(method=["GET"],
 
                                            function=check_repo))
 
        #ajax delete repo perm user
 
        m.connect('delete_repo_user', "/repos_delete_user/{repo_name:.*}",
 
             action="delete_perm_user", conditions=dict(method=["DELETE"],
 
                                                        function=check_repo))
 
        #ajax delete repo perm users_group
 
        m.connect('delete_repo_users_group',
 
                  "/repos_delete_users_group/{repo_name:.*}",
 
                  action="delete_perm_users_group",
 
                  conditions=dict(method=["DELETE"], function=check_repo))
 

	
 
        #settings actions
 
        m.connect('repo_stats', "/repos_stats/{repo_name:.*}",
 
             action="repo_stats", conditions=dict(method=["DELETE"],
 
                                                        function=check_repo))
 
        m.connect('repo_cache', "/repos_cache/{repo_name:.*}",
 
             action="repo_cache", conditions=dict(method=["DELETE"],
 
                                                        function=check_repo))
 
        m.connect('repo_public_journal',
 
                  "/repos_public_journal/{repo_name:.*}",
 
                  action="repo_public_journal", conditions=dict(method=["PUT"],
 
                  function=check_repo))
 
        m.connect('repo_pull', "/repo_pull/{repo_name:.*}",
 
             action="repo_pull", conditions=dict(method=["PUT"],
 
                                                        function=check_repo))
 

	
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/repos_groups') as m:
 
        m.connect("repos_groups", "/repos_groups",
 
                  action="create", conditions=dict(method=["POST"]))
 
        m.connect("repos_groups", "/repos_groups",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("formatted_repos_groups", "/repos_groups.{format}",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("new_repos_group", "/repos_groups/new",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("formatted_new_repos_group", "/repos_groups/new.{format}",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("update_repos_group", "/repos_groups/{id}",
 
                  action="update", conditions=dict(method=["PUT"],
 
                                                   function=check_int))
 
        m.connect("delete_repos_group", "/repos_groups/{id}",
 
                  action="delete", conditions=dict(method=["DELETE"],
 
                                                   function=check_int))
 
        m.connect("edit_repos_group", "/repos_groups/{id}/edit",
 
                  action="edit", conditions=dict(method=["GET"],
 
                                                 function=check_int))
 
        m.connect("formatted_edit_repos_group",
 
                  "/repos_groups/{id}.{format}/edit",
 
                  action="edit", conditions=dict(method=["GET"],
 
                                                 function=check_int))
 
        m.connect("repos_group", "/repos_groups/{id}",
 
                  action="show", conditions=dict(method=["GET"],
 
                                                 function=check_int))
 
        m.connect("formatted_repos_group", "/repos_groups/{id}.{format}",
 
                  action="show", conditions=dict(method=["GET"],
 
                                                 function=check_int))
 

	
 
    #ADMIN USER REST ROUTES
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/users') as m:
 
        m.connect("users", "/users",
 
                  action="create", conditions=dict(method=["POST"]))
 
        m.connect("users", "/users",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("formatted_users", "/users.{format}",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("new_user", "/users/new",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("formatted_new_user", "/users/new.{format}",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("update_user", "/users/{id}",
 
                  action="update", conditions=dict(method=["PUT"]))
 
        m.connect("delete_user", "/users/{id}",
 
                  action="delete", conditions=dict(method=["DELETE"]))
 
        m.connect("edit_user", "/users/{id}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("formatted_edit_user",
 
                  "/users/{id}.{format}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("user", "/users/{id}",
 
                  action="show", conditions=dict(method=["GET"]))
 
        m.connect("formatted_user", "/users/{id}.{format}",
 
                  action="show", conditions=dict(method=["GET"]))
 

	
 
        #EXTRAS USER ROUTES
 
        m.connect("user_perm", "/users_perm/{id}",
 
                  action="update_perm", conditions=dict(method=["PUT"]))
 

	
 
    #ADMIN USERS REST ROUTES
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/users_groups') as m:
 
        m.connect("users_groups", "/users_groups",
 
                  action="create", conditions=dict(method=["POST"]))
 
        m.connect("users_groups", "/users_groups",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("formatted_users_groups", "/users_groups.{format}",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("new_users_group", "/users_groups/new",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("formatted_new_users_group", "/users_groups/new.{format}",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("update_users_group", "/users_groups/{id}",
 
                  action="update", conditions=dict(method=["PUT"]))
 
        m.connect("delete_users_group", "/users_groups/{id}",
 
                  action="delete", conditions=dict(method=["DELETE"]))
 
        m.connect("edit_users_group", "/users_groups/{id}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("formatted_edit_users_group",
 
                  "/users_groups/{id}.{format}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("users_group", "/users_groups/{id}",
 
                  action="show", conditions=dict(method=["GET"]))
 
        m.connect("formatted_users_group", "/users_groups/{id}.{format}",
 
                  action="show", conditions=dict(method=["GET"]))
 

	
 
        #EXTRAS USER ROUTES
 
        m.connect("users_group_perm", "/users_groups_perm/{id}",
 
                  action="update_perm", conditions=dict(method=["PUT"]))
 

	
 
    #ADMIN GROUP REST ROUTES
 
    rmap.resource('group', 'groups',
 
                  controller='admin/groups', path_prefix=ADMIN_PREFIX)
 

	
 
    #ADMIN PERMISSIONS REST ROUTES
 
    rmap.resource('permission', 'permissions',
 
                  controller='admin/permissions', path_prefix=ADMIN_PREFIX)
 

	
 
    ##ADMIN LDAP SETTINGS
 
    rmap.connect('ldap_settings', '%s/ldap' % ADMIN_PREFIX,
 
                 controller='admin/ldap_settings', action='ldap_settings',
 
                 conditions=dict(method=["POST"]))
 

	
 
    rmap.connect('ldap_home', '%s/ldap' % ADMIN_PREFIX,
 
                 controller='admin/ldap_settings')
 

	
 
    #ADMIN SETTINGS REST ROUTES
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/settings') as m:
 
        m.connect("admin_settings", "/settings",
 
                  action="create", conditions=dict(method=["POST"]))
 
        m.connect("admin_settings", "/settings",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("formatted_admin_settings", "/settings.{format}",
 
                  action="index", conditions=dict(method=["GET"]))
 
        m.connect("admin_new_setting", "/settings/new",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("formatted_admin_new_setting", "/settings/new.{format}",
 
                  action="new", conditions=dict(method=["GET"]))
 
        m.connect("/settings/{setting_id}",
 
                  action="update", conditions=dict(method=["PUT"]))
 
        m.connect("/settings/{setting_id}",
 
                  action="delete", conditions=dict(method=["DELETE"]))
 
        m.connect("admin_edit_setting", "/settings/{setting_id}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("formatted_admin_edit_setting",
 
                  "/settings/{setting_id}.{format}/edit",
 
                  action="edit", conditions=dict(method=["GET"]))
 
        m.connect("admin_setting", "/settings/{setting_id}",
 
                  action="show", conditions=dict(method=["GET"]))
 
        m.connect("formatted_admin_setting", "/settings/{setting_id}.{format}",
 
                  action="show", conditions=dict(method=["GET"]))
 
        m.connect("admin_settings_my_account", "/my_account",
 
                  action="my_account", conditions=dict(method=["GET"]))
 
        m.connect("admin_settings_my_account_update", "/my_account_update",
 
                  action="my_account_update", conditions=dict(method=["PUT"]))
 
        m.connect("admin_settings_create_repository", "/create_repository",
 
                  action="create_repository", conditions=dict(method=["GET"]))
 

	
 

	
 
    #ADMIN MAIN PAGES
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='admin/admin') as m:
 
        m.connect('admin_home', '', action='index')
 
        m.connect('admin_add_repo', '/add_repo/{new_repo:[a-z0-9\. _-]*}',
 
                  action='add_repo')
 

	
 
    #==========================================================================
 
    # API V1
 
    #==========================================================================
 
    with rmap.submapper(path_prefix=ADMIN_PREFIX,
 
                        controller='api/api') as m:
 
        m.connect('api', '/api')
 

	
 

	
 
    #USER JOURNAL
 
    rmap.connect('journal', '%s/journal' % ADMIN_PREFIX, controller='journal')
 

	
 
    rmap.connect('public_journal', '%s/public_journal' % ADMIN_PREFIX,
 
                 controller='journal', action="public_journal")
 

	
 
    rmap.connect('public_journal_rss', '%s/public_journal_rss' % ADMIN_PREFIX,
 
                 controller='journal', action="public_journal_rss")
 

	
 
    rmap.connect('public_journal_atom',
 
                 '%s/public_journal_atom' % ADMIN_PREFIX, controller='journal',
 
                 action="public_journal_atom")
 

	
 
    rmap.connect('toggle_following', '%s/toggle_following' % ADMIN_PREFIX,
 
                 controller='journal', action='toggle_following',
 
                 conditions=dict(method=["POST"]))
 

	
 
    #SEARCH
 
    rmap.connect('search', '%s/search' % ADMIN_PREFIX, controller='search',)
 
    rmap.connect('search_repo', '%s/search/{search_repo:.*}' % ADMIN_PREFIX,
 
                  controller='search')
 

	
 
    #LOGIN/LOGOUT/REGISTER/SIGN IN
 
    rmap.connect('login_home', '%s/login' % ADMIN_PREFIX, controller='login')
 
    rmap.connect('logout_home', '%s/logout' % ADMIN_PREFIX, controller='login',
 
                 action='logout')
 

	
 
    rmap.connect('register', '%s/register' % ADMIN_PREFIX, controller='login',
 
                 action='register')
 

	
 
    rmap.connect('reset_password', '%s/password_reset' % ADMIN_PREFIX,
 
                 controller='login', action='password_reset')
 

	
 
    rmap.connect('reset_password_confirmation',
 
                 '%s/password_reset_confirmation' % ADMIN_PREFIX,
 
                 controller='login', action='password_reset_confirmation')
 

	
 
    #FEEDS
 
    rmap.connect('rss_feed_home', '/{repo_name:.*}/feed/rss',
 
                controller='feed', action='rss',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('atom_feed_home', '/{repo_name:.*}/feed/atom',
 
                controller='feed', action='atom',
 
                conditions=dict(function=check_repo))
 

	
 
    #==========================================================================
 
    # REPOSITORY ROUTES
 
    #==========================================================================
 
    rmap.connect('summary_home', '/{repo_name:.*}',
 
                controller='summary', 
 
                controller='summary',
 
                conditions=dict(function=check_repo))
 
    
 
#    rmap.connect('repo_group_home', '/{group_name:.*}',
 
#                controller='admin/repos_groups',action="show_by_name", 
 
#                conditions=dict(function=check_group))
 
    
 

	
 
    rmap.connect('repos_group_home', '/{group_name:.*}',
 
                controller='admin/repos_groups', action="show_by_name",
 
                conditions=dict(function=check_group))
 

	
 
    rmap.connect('changeset_home', '/{repo_name:.*}/changeset/{revision}',
 
                controller='changeset', revision='tip',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('raw_changeset_home',
 
                 '/{repo_name:.*}/raw-changeset/{revision}',
 
                 controller='changeset', action='raw_changeset',
 
                 revision='tip', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('summary_home', '/{repo_name:.*}/summary',
 
                controller='summary', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('shortlog_home', '/{repo_name:.*}/shortlog',
 
                controller='shortlog', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('branches_home', '/{repo_name:.*}/branches',
 
                controller='branches', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('tags_home', '/{repo_name:.*}/tags',
 
                controller='tags', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('changelog_home', '/{repo_name:.*}/changelog',
 
                controller='changelog', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('changelog_details', '/{repo_name:.*}/changelog_details/{cs}',
 
                controller='changelog', action='changelog_details',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_home', '/{repo_name:.*}/files/{revision}/{f_path:.*}',
 
                controller='files', revision='tip', f_path='',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_diff_home', '/{repo_name:.*}/diff/{f_path:.*}',
 
                controller='files', action='diff', revision='tip', f_path='',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_rawfile_home',
 
                 '/{repo_name:.*}/rawfile/{revision}/{f_path:.*}',
 
                 controller='files', action='rawfile', revision='tip',
 
                 f_path='', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_raw_home',
 
                 '/{repo_name:.*}/raw/{revision}/{f_path:.*}',
 
                 controller='files', action='raw', revision='tip', f_path='',
 
                 conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_annotate_home',
 
                 '/{repo_name:.*}/annotate/{revision}/{f_path:.*}',
 
                 controller='files', action='annotate', revision='tip',
 
                 f_path='', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_edit_home',
 
                 '/{repo_name:.*}/edit/{revision}/{f_path:.*}',
 
                 controller='files', action='edit', revision='tip',
 
                 f_path='', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_add_home',
 
                 '/{repo_name:.*}/add/{revision}/{f_path:.*}',
 
                 controller='files', action='add', revision='tip',
 
                 f_path='', conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_archive_home', '/{repo_name:.*}/archive/{fname}',
 
                controller='files', action='archivefile',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('files_nodelist_home',
 
                 '/{repo_name:.*}/nodelist/{revision}/{f_path:.*}',
 
                controller='files', action='nodelist',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('repo_settings_delete', '/{repo_name:.*}/settings',
 
                controller='settings', action="delete",
 
                conditions=dict(method=["DELETE"], function=check_repo))
 

	
 
    rmap.connect('repo_settings_update', '/{repo_name:.*}/settings',
 
                controller='settings', action="update",
 
                conditions=dict(method=["PUT"], function=check_repo))
 

	
 
    rmap.connect('repo_settings_home', '/{repo_name:.*}/settings',
 
                controller='settings', action='index',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('repo_fork_create_home', '/{repo_name:.*}/fork',
 
                controller='settings', action='fork_create',
 
                conditions=dict(function=check_repo, method=["POST"]))
 

	
 
    rmap.connect('repo_fork_home', '/{repo_name:.*}/fork',
 
                controller='settings', action='fork',
 
                conditions=dict(function=check_repo))
 

	
 
    rmap.connect('repo_followers_home', '/{repo_name:.*}/followers',
 
                 controller='followers', action='followers',
 
                 conditions=dict(function=check_repo))
 

	
 
    rmap.connect('repo_forks_home', '/{repo_name:.*}/forks',
 
                 controller='forks', action='forks',
 
                 conditions=dict(function=check_repo))
 

	
 
    return rmap
rhodecode/controllers/admin/repos.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.controllers.admin.repos
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Admin controller for RhodeCode
 

	
 
    :created_on: Apr 7, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 
import traceback
 
import formencode
 
from operator import itemgetter
 
from formencode import htmlfill
 

	
 
from paste.httpexceptions import HTTPInternalServerError
 
from pylons import request, response, session, tmpl_context as c, url
 
from pylons.controllers.util import abort, redirect
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib import helpers as h
 
from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator, \
 
    HasPermissionAnyDecorator
 
from rhodecode.lib.base import BaseController, render
 
from rhodecode.lib.utils import invalidate_cache, action_logger, repo_name_slug
 
from rhodecode.lib.helpers import get_token
 
from rhodecode.model.db import User, Repository, UserFollowing, Group
 
from rhodecode.model.forms import RepoForm
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.model.repo import RepoModel
 
from sqlalchemy.exc import IntegrityError
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class ReposController(BaseController):
 
    """
 
    REST Controller styled on the Atom Publishing Protocol"""
 
    # To properly map this controller, ensure your config/routing.py
 
    # file has a resource setup:
 
    #     map.resource('repo', 'repos')
 

	
 
    @LoginRequired()
 
    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
 
    def __before__(self):
 
        c.admin_user = session.get('admin_user')
 
        c.admin_username = session.get('admin_username')
 
        super(ReposController, self).__before__()
 

	
 
    def __load_defaults(self):
 
        c.repo_groups = Group.groups_choices()
 
        c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)
 
        
 
        repo_model = RepoModel()
 

	
 
        c.repo_groups = [('', '')]
 
        parents_link = lambda k: h.literal('&raquo;'.join(
 
                                    map(lambda k: k.group_name,
 
                                        k.parents + [k])
 
                                    )
 
                                )
 

	
 
        c.repo_groups.extend([(x.group_id, parents_link(x)) for \
 
                                            x in self.sa.query(Group).all()])
 
        c.repo_groups = sorted(c.repo_groups,
 
                               key=lambda t: t[1].split('&raquo;')[0])
 
        c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)
 
        c.users_array = repo_model.get_users_js()
 
        c.users_groups_array = repo_model.get_users_groups_js()
 

	
 
    def __load_data(self, repo_name=None):
 
        """
 
        Load defaults settings for edit, and update
 

	
 
        :param repo_name:
 
        """
 
        self.__load_defaults()
 

	
 
        c.repo_info = db_repo = Repository.get_by_repo_name(repo_name)
 
        repo = scm_repo = db_repo.scm_instance
 
        repo = db_repo.scm_instance
 

	
 
        if c.repo_info is None:
 
            h.flash(_('%s repository is not mapped to db perhaps'
 
                      ' it was created or renamed from the filesystem'
 
                      ' please run the application again'
 
                      ' in order to rescan repositories') % repo_name,
 
                      category='error')
 

	
 
            return redirect(url('repos'))
 

	
 
        c.default_user_id = User.get_by_username('default').user_id
 
        c.in_public_journal = self.sa.query(UserFollowing)\
 
            .filter(UserFollowing.user_id == c.default_user_id)\
 
            .filter(UserFollowing.follows_repository == c.repo_info).scalar()
 

	
 
        if c.repo_info.stats:
 
            last_rev = c.repo_info.stats.stat_on_revision
 
        else:
 
            last_rev = 0
 
        c.stats_revision = last_rev
 

	
 
        c.repo_last_rev = repo.count() - 1 if repo.revisions else 0
 

	
 
        if last_rev == 0 or c.repo_last_rev == 0:
 
            c.stats_percentage = 0
 
        else:
 
            c.stats_percentage = '%.2f' % ((float((last_rev)) /
 
                                            c.repo_last_rev) * 100)
 

	
 
        defaults = c.repo_info.get_dict()
 
        group, repo_name = c.repo_info.groups_and_repo
 
        defaults['repo_name'] = repo_name
 
        defaults['repo_group'] = getattr(group[-1] if group else None,
 
                                         'group_id', None)
 

	
 
        #fill owner
 
        if c.repo_info.user:
 
            defaults.update({'user': c.repo_info.user.username})
 
        else:
 
            replacement_user = self.sa.query(User)\
 
            .filter(User.admin == True).first().username
 
            defaults.update({'user': replacement_user})
 

	
 
        #fill repository users
 
        for p in c.repo_info.repo_to_perm:
 
            defaults.update({'u_perm_%s' % p.user.username:
 
                             p.permission.permission_name})
 

	
 
        #fill repository groups
 
        for p in c.repo_info.users_group_to_perm:
 
            defaults.update({'g_perm_%s' % p.users_group.users_group_name:
 
                             p.permission.permission_name})
 

	
 
        return defaults
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def index(self, format='html'):
 
        """GET /repos: All items in the collection"""
 
        # url('repos')
 

	
 
        c.repos_list = ScmModel().get_repos(Repository.query()
 
                                            .order_by(Repository.repo_name)
 
                                            .all(), sort_key='name_sort')
 
        return render('admin/repos/repos.html')
 

	
 
    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
 
    def create(self):
 
        """
 
        POST /repos: Create a new item"""
 
        # url('repos')
 
        repo_model = RepoModel()
 
        self.__load_defaults()
 
        form_result = {}
 
        try:
 
            form_result = RepoForm(repo_groups=c.repo_groups_choices)()\
 
                            .to_python(dict(request.POST))
 
            repo_model.create(form_result, self.rhodecode_user)
 
            if form_result['clone_uri']:
 
                h.flash(_('created repository %s from %s') \
 
                    % (form_result['repo_name'], form_result['clone_uri']),
 
                    category='success')
 
            else:
 
                h.flash(_('created repository %s') % form_result['repo_name'],
 
                    category='success')
 

	
 
            if request.POST.get('user_created'):
 
                #created by regular non admin user
 
                action_logger(self.rhodecode_user, 'user_created_repo',
 
                              form_result['repo_name_full'], '', self.sa)
 
            else:
 
                action_logger(self.rhodecode_user, 'admin_created_repo',
 
                              form_result['repo_name_full'], '', self.sa)
 

	
 
        except formencode.Invalid, errors:
 

	
 
            c.new_repo = errors.value['repo_name']
 

	
 
            if request.POST.get('user_created'):
 
                r = render('admin/repos/repo_add_create_repository.html')
 
            else:
 
                r = render('admin/repos/repo_add.html')
 

	
 
            return htmlfill.render(
 
                r,
 
                defaults=errors.value,
 
                errors=errors.error_dict or {},
 
                prefix_error=False,
 
                encoding="UTF-8")
 

	
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            msg = _('error occurred during creation of repository %s') \
 
                    % form_result.get('repo_name')
 
            h.flash(msg, category='error')
 
        if request.POST.get('user_created'):
 
            return redirect(url('home'))
 
        return redirect(url('repos'))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def new(self, format='html'):
 
        """GET /repos/new: Form to create a new item"""
 
        new_repo = request.GET.get('repo', '')
 
        c.new_repo = repo_name_slug(new_repo)
 
        self.__load_defaults()
 
        return render('admin/repos/repo_add.html')
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def update(self, repo_name):
 
        """
 
        PUT /repos/repo_name: Update an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="PUT" />
 
        # Or using helpers:
 
        #    h.form(url('repo', repo_name=ID),
 
        #           method='put')
 
        # url('repo', repo_name=ID)
 
        self.__load_defaults()
 
        repo_model = RepoModel()
 
        changed_name = repo_name
 
        _form = RepoForm(edit=True, old_data={'repo_name': repo_name},
 
                         repo_groups=c.repo_groups_choices)()
 
        try:
 
            form_result = _form.to_python(dict(request.POST))
 
            repo_model.update(repo_name, form_result)
 
            repo = repo_model.update(repo_name, form_result)
 
            invalidate_cache('get_repo_cached_%s' % repo_name)
 
            h.flash(_('Repository %s updated successfully' % repo_name),
 
                    category='success')
 
            changed_name = form_result['repo_name_full']
 
            changed_name = repo.repo_name
 
            action_logger(self.rhodecode_user, 'admin_updated_repo',
 
                              changed_name, '', self.sa)
 

	
 
        except formencode.Invalid, errors:
 
            defaults = self.__load_data(repo_name)
 
            defaults.update(errors.value)
 
            return htmlfill.render(
 
                render('admin/repos/repo_edit.html'),
 
                defaults=defaults,
 
                errors=errors.error_dict or {},
 
                prefix_error=False,
 
                encoding="UTF-8")
 

	
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during update of repository %s') \
 
                    % repo_name, category='error')
 
        return redirect(url('edit_repo', repo_name=changed_name))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def delete(self, repo_name):
 
        """
 
        DELETE /repos/repo_name: Delete an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="DELETE" />
 
        # Or using helpers:
 
        #    h.form(url('repo', repo_name=ID),
 
        #           method='delete')
 
        # url('repo', repo_name=ID)
 

	
 
        repo_model = RepoModel()
 
        repo = repo_model.get_by_repo_name(repo_name)
 
        if not repo:
 
            h.flash(_('%s repository is not mapped to db perhaps'
 
                      ' it was moved or renamed  from the filesystem'
 
                      ' please run the application again'
 
                      ' in order to rescan repositories') % repo_name,
 
                      category='error')
 

	
 
            return redirect(url('repos'))
 
        try:
 
            action_logger(self.rhodecode_user, 'admin_deleted_repo',
 
                              repo_name, '', self.sa)
 
            repo_model.delete(repo)
 
            invalidate_cache('get_repo_cached_%s' % repo_name)
 
            h.flash(_('deleted repository %s') % repo_name, category='success')
 

	
 
        except IntegrityError, e:
 
            if e.message.find('repositories_fork_id_fkey'):
 
                log.error(traceback.format_exc())
 
                h.flash(_('Cannot delete %s it still contains attached '
 
                          'forks') % repo_name,
 
                        category='warning')
 
            else:
 
                log.error(traceback.format_exc())
 
                h.flash(_('An error occurred during '
 
                          'deletion of %s') % repo_name,
 
                        category='error')
 

	
 
        except Exception, e:
 
            log.error(traceback.format_exc())
 
            h.flash(_('An error occurred during deletion of %s') % repo_name,
 
                    category='error')
 

	
 
        return redirect(url('repos'))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def delete_perm_user(self, repo_name):
 
        """
 
        DELETE an existing repository permission user
 

	
 
        :param repo_name:
 
        """
 

	
 
        try:
 
            repo_model = RepoModel()
 
            repo_model.delete_perm_user(request.POST, repo_name)
 
        except Exception, e:
 
            h.flash(_('An error occurred during deletion of repository user'),
 
                    category='error')
 
            raise HTTPInternalServerError()
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def delete_perm_users_group(self, repo_name):
 
        """
 
        DELETE an existing repository permission users group
 

	
 
        :param repo_name:
 
        """
 
        try:
 
            repo_model = RepoModel()
 
            repo_model.delete_perm_users_group(request.POST, repo_name)
 
        except Exception, e:
 
            h.flash(_('An error occurred during deletion of repository'
 
                      ' users groups'),
 
                    category='error')
 
            raise HTTPInternalServerError()
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def repo_stats(self, repo_name):
 
        """
 
        DELETE an existing repository statistics
 

	
 
        :param repo_name:
 
        """
 

	
 
        try:
 
            repo_model = RepoModel()
 
            repo_model.delete_stats(repo_name)
 
        except Exception, e:
 
            h.flash(_('An error occurred during deletion of repository stats'),
 
                    category='error')
 
        return redirect(url('edit_repo', repo_name=repo_name))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def repo_cache(self, repo_name):
 
        """
 
        INVALIDATE existing repository cache
 

	
 
        :param repo_name:
 
        """
 

	
 
        try:
 
            ScmModel().mark_for_invalidation(repo_name)
 
        except Exception, e:
 
            h.flash(_('An error occurred during cache invalidation'),
 
                    category='error')
 
        return redirect(url('edit_repo', repo_name=repo_name))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def repo_public_journal(self, repo_name):
 
        """
 
        Set's this repository to be visible in public journal,
 
        in other words assing default user to follow this repo
 

	
 
        :param repo_name:
 
        """
 

	
 
        cur_token = request.POST.get('auth_token')
 
        token = get_token()
 
        if cur_token == token:
 
            try:
 
                repo_id = Repository.get_by_repo_name(repo_name).repo_id
 
                user_id = User.get_by_username('default').user_id
 
                self.scm_model.toggle_following_repo(repo_id, user_id)
 
                h.flash(_('Updated repository visibility in public journal'),
 
                        category='success')
 
            except:
 
                h.flash(_('An error occurred during setting this'
 
                          ' repository in public journal'),
 
                        category='error')
 

	
 
        else:
 
            h.flash(_('Token mismatch'), category='error')
 
        return redirect(url('edit_repo', repo_name=repo_name))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def repo_pull(self, repo_name):
 
        """
 
        Runs task to update given repository with remote changes,
 
        ie. make pull on remote location
 

	
 
        :param repo_name:
 
        """
 
        try:
 
            ScmModel().pull_changes(repo_name, self.rhodecode_user.username)
 
            h.flash(_('Pulled from remote location'), category='success')
 
        except Exception, e:
 
            h.flash(_('An error occurred during pull from remote location'),
 
                    category='error')
 

	
 
        return redirect(url('edit_repo', repo_name=repo_name))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def show(self, repo_name, format='html'):
 
        """GET /repos/repo_name: Show a specific item"""
 
        # url('repo', repo_name=ID)
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def edit(self, repo_name, format='html'):
 
        """GET /repos/repo_name/edit: Form to edit an existing item"""
 
        # url('edit_repo', repo_name=ID)
 
        defaults = self.__load_data(repo_name)
 

	
 
        return htmlfill.render(
 
            render('admin/repos/repo_edit.html'),
 
            defaults=defaults,
 
            encoding="UTF-8",
 
            force_defaults=False
 
        )
rhodecode/controllers/admin/repos_groups.py
Show inline comments
 
import logging
 
import traceback
 
import formencode
 

	
 
from formencode import htmlfill
 
from operator import itemgetter
 

	
 
from pylons import request, response, session, tmpl_context as c, url
 
from pylons.controllers.util import abort, redirect
 
from pylons.i18n.translation import _
 

	
 
from sqlalchemy.exc import IntegrityError
 

	
 
from rhodecode.lib import helpers as h
 
from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator, \
 
    HasPermissionAnyDecorator
 
from rhodecode.lib.auth import LoginRequired, HasPermissionAnyDecorator
 
from rhodecode.lib.base import BaseController, render
 
from rhodecode.model.db import Group
 
from rhodecode.model.repos_group import ReposGroupModel
 
from rhodecode.model.forms import ReposGroupForm
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class ReposGroupsController(BaseController):
 
    """REST Controller styled on the Atom Publishing Protocol"""
 
    # To properly map this controller, ensure your config/routing.py
 
    # file has a resource setup:
 
    #     map.resource('repos_group', 'repos_groups')
 

	
 
    @LoginRequired()
 
    def __before__(self):
 
        super(ReposGroupsController, self).__before__()
 

	
 
    def __load_defaults(self):
 

	
 
        c.repo_groups = [('', '')]
 
        parents_link = lambda k: h.literal('&raquo;'.join(
 
                                    map(lambda k: k.group_name,
 
                                        k.parents + [k])
 
                                    )
 
                                )
 

	
 
        c.repo_groups.extend([(x.group_id, parents_link(x)) for \
 
                                            x in self.sa.query(Group).all()])
 

	
 
        c.repo_groups = sorted(c.repo_groups,
 
                               key=lambda t: t[1].split('&raquo;')[0])
 
        c.repo_groups = Group.groups_choices()
 
        c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)
 

	
 
    def __load_data(self, group_id):
 
        """
 
        Load defaults settings for edit, and update
 

	
 
        :param group_id:
 
        """
 
        self.__load_defaults()
 

	
 
        repo_group = Group.get(group_id)
 

	
 
        data = repo_group.get_dict()
 

	
 
        data['group_name'] = repo_group.name
 

	
 
        return data
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def index(self, format='html'):
 
        """GET /repos_groups: All items in the collection"""
 
        # url('repos_groups')
 

	
 
        sk = lambda g:g.parents[0].group_name if g.parents else g.group_name
 
        c.groups = sorted(Group.query().all(), key=sk)
 
        return render('admin/repos_groups/repos_groups_show.html')
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def create(self):
 
        """POST /repos_groups: Create a new item"""
 
        # url('repos_groups')
 
        self.__load_defaults()
 
        repos_group_model = ReposGroupModel()
 
        repos_group_form = ReposGroupForm(available_groups=
 
                                          c.repo_groups_choices)()
 
        try:
 
            form_result = repos_group_form.to_python(dict(request.POST))
 
            repos_group_model.create(form_result)
 
            h.flash(_('created repos group %s') \
 
                    % form_result['group_name'], category='success')
 
            #TODO: in futureaction_logger(, '', '', '', self.sa)
 
        except formencode.Invalid, errors:
 

	
 
            return htmlfill.render(
 
                render('admin/repos_groups/repos_groups_add.html'),
 
                defaults=errors.value,
 
                errors=errors.error_dict or {},
 
                prefix_error=False,
 
                encoding="UTF-8")
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during creation of repos group %s') \
 
                    % request.POST.get('group_name'), category='error')
 

	
 
        return redirect(url('repos_groups'))
 

	
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def new(self, format='html'):
 
        """GET /repos_groups/new: Form to create a new item"""
 
        # url('new_repos_group')
 
        self.__load_defaults()
 
        return render('admin/repos_groups/repos_groups_add.html')
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def update(self, id):
 
        """PUT /repos_groups/id: Update an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="PUT" />
 
        # Or using helpers:
 
        #    h.form(url('repos_group', id=ID),
 
        #           method='put')
 
        # url('repos_group', id=ID)
 

	
 
        self.__load_defaults()
 
        c.repos_group = Group.get(id)
 

	
 
        repos_group_model = ReposGroupModel()
 
        repos_group_form = ReposGroupForm(edit=True,
 
                                          old_data=c.repos_group.get_dict(),
 
                                          available_groups=
 
                                            c.repo_groups_choices)()
 
        try:
 
            form_result = repos_group_form.to_python(dict(request.POST))
 
            repos_group_model.update(id, form_result)
 
            h.flash(_('updated repos group %s') \
 
                    % form_result['group_name'], category='success')
 
            #TODO: in futureaction_logger(, '', '', '', self.sa)
 
        except formencode.Invalid, errors:
 

	
 
            return htmlfill.render(
 
                render('admin/repos_groups/repos_groups_edit.html'),
 
                defaults=errors.value,
 
                errors=errors.error_dict or {},
 
                prefix_error=False,
 
                encoding="UTF-8")
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during update of repos group %s') \
 
                    % request.POST.get('group_name'), category='error')
 

	
 
        return redirect(url('repos_groups'))
 

	
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def delete(self, id):
 
        """DELETE /repos_groups/id: Delete an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="DELETE" />
 
        # Or using helpers:
 
        #    h.form(url('repos_group', id=ID),
 
        #           method='delete')
 
        # url('repos_group', id=ID)
 

	
 
        repos_group_model = ReposGroupModel()
 
        gr = Group.get(id)
 
        repos = gr.repositories.all()
 
        if repos:
 
            h.flash(_('This group contains %s repositores and cannot be '
 
                      'deleted' % len(repos)),
 
                    category='error')
 
            return redirect(url('repos_groups'))
 

	
 
        try:
 
            repos_group_model.delete(id)
 
            h.flash(_('removed repos group %s' % gr.group_name), category='success')
 
            #TODO: in future action_logger(, '', '', '', self.sa)
 
        except IntegrityError, e:
 
            if e.message.find('groups_group_parent_id_fkey'):
 
                log.error(traceback.format_exc())
 
                h.flash(_('Cannot delete this group it still contains '
 
                          'subgroups'),
 
                        category='warning')
 
            else:
 
                log.error(traceback.format_exc())
 
                h.flash(_('error occurred during deletion of repos '
 
                          'group %s' % gr.group_name), category='error')
 

	
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during deletion of repos group %s' % gr.group_name),
 
                    category='error')
 
            h.flash(_('error occurred during deletion of repos '
 
                      'group %s' % gr.group_name), category='error')
 

	
 
        return redirect(url('repos_groups'))
 

	
 
    def show_by_name(self, group_name):
 
        id_ = Group.get_by_group_name(group_name).group_id
 
        return self.show(id_)
 

	
 
    def show(self, id, format='html'):
 
        """GET /repos_groups/id: Show a specific item"""
 
        # url('repos_group', id=ID)
 

	
 
        c.group = Group.get(id)
 

	
 
        if c.group:
 
            c.group_repos = c.group.repositories.all()
 
        else:
 
            return redirect(url('home'))
 

	
 
        #overwrite our cached list with current filter
 
        gr_filter = c.group_repos
 
        c.cached_repo_list = self.scm_model.get_repos(all_repos=gr_filter)
 

	
 
        c.repos_list = c.cached_repo_list
 

	
 
        c.repo_cnt = 0
 

	
 
        c.groups = self.sa.query(Group).order_by(Group.group_name)\
 
            .filter(Group.group_parent_id == id).all()
 

	
 
        return render('admin/repos_groups/repos_groups.html')
 

	
 
    @HasPermissionAnyDecorator('hg.admin')
 
    def edit(self, id, format='html'):
 
        """GET /repos_groups/id/edit: Form to edit an existing item"""
 
        # url('edit_repos_group', id=ID)
 

	
 
        id_ = int(id)
 

	
 
        c.repos_group = Group.get(id_)
 
        defaults = self.__load_data(id_)
 

	
 
        # we need to exclude this group from the group list for editing
 
        c.repo_groups = filter(lambda x:x[0] != id_, c.repo_groups)
 

	
 
        return htmlfill.render(
 
            render('admin/repos_groups/repos_groups_edit.html'),
 
            defaults=defaults,
 
            encoding="UTF-8",
 
            force_defaults=False
 
        )
 

	
 

	
rhodecode/controllers/admin/settings.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.controllers.admin.settings
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    settings controller for rhodecode admin
 

	
 
    :created_on: Jul 14, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import logging
 
import traceback
 
import formencode
 

	
 
from sqlalchemy import func
 
from formencode import htmlfill
 
from pylons import request, session, tmpl_context as c, url, config
 
from pylons.controllers.util import abort, redirect
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib import helpers as h
 
from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator, \
 
    HasPermissionAnyDecorator, NotAnonymous
 
from rhodecode.lib.base import BaseController, render
 
from rhodecode.lib.celerylib import tasks, run_task
 
from rhodecode.lib.utils import repo2db_mapper, invalidate_cache, \
 
    set_rhodecode_config, repo_name_slug
 
from rhodecode.model.db import RhodeCodeUi, Repository, Group, \
 
    RhodeCodeSettings
 
from rhodecode.model.forms import UserForm, ApplicationSettingsForm, \
 
    ApplicationUiSettingsForm
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.model.user import UserModel
 
from rhodecode.model.db import User
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class SettingsController(BaseController):
 
    """REST Controller styled on the Atom Publishing Protocol"""
 
    # To properly map this controller, ensure your config/routing.py
 
    # file has a resource setup:
 
    #     map.resource('setting', 'settings', controller='admin/settings',
 
    #         path_prefix='/admin', name_prefix='admin_')
 

	
 
    @LoginRequired()
 
    def __before__(self):
 
        c.admin_user = session.get('admin_user')
 
        c.admin_username = session.get('admin_username')
 
        super(SettingsController, self).__before__()
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def index(self, format='html'):
 
        """GET /admin/settings: All items in the collection"""
 
        # url('admin_settings')
 

	
 
        defaults = RhodeCodeSettings.get_app_settings()
 
        defaults.update(self.get_hg_ui_settings())
 
        return htmlfill.render(
 
            render('admin/settings/settings.html'),
 
            defaults=defaults,
 
            encoding="UTF-8",
 
            force_defaults=False
 
        )
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def create(self):
 
        """POST /admin/settings: Create a new item"""
 
        # url('admin_settings')
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def new(self, format='html'):
 
        """GET /admin/settings/new: Form to create a new item"""
 
        # url('admin_new_setting')
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def update(self, setting_id):
 
        """PUT /admin/settings/setting_id: Update an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="PUT" />
 
        # Or using helpers:
 
        #    h.form(url('admin_setting', setting_id=ID),
 
        #           method='put')
 
        # url('admin_setting', setting_id=ID)
 
        if setting_id == 'mapping':
 
            rm_obsolete = request.POST.get('destroy', False)
 
            log.debug('Rescanning directories with destroy=%s', rm_obsolete)
 
            initial = ScmModel().repo_scan()
 
            log.debug('invalidating all repositories')
 
            for repo_name in initial.keys():
 
                invalidate_cache('get_repo_cached_%s' % repo_name)
 

	
 
            added, removed = repo2db_mapper(initial, rm_obsolete)
 

	
 
            h.flash(_('Repositories successfully'
 
                      ' rescanned added: %s,removed: %s') % (added, removed),
 
                    category='success')
 

	
 
        if setting_id == 'whoosh':
 
            repo_location = self.get_hg_ui_settings()['paths_root_path']
 
            full_index = request.POST.get('full_index', False)
 
            run_task(tasks.whoosh_index, repo_location, full_index)
 

	
 
            h.flash(_('Whoosh reindex task scheduled'), category='success')
 
        if setting_id == 'global':
 

	
 
            application_form = ApplicationSettingsForm()()
 
            try:
 
                form_result = application_form.to_python(dict(request.POST))
 

	
 
                try:
 
                    hgsettings1 = RhodeCodeSettings.get_by_name('title')
 
                    hgsettings1.app_settings_value = \
 
                        form_result['rhodecode_title']
 

	
 
                    hgsettings2 = RhodeCodeSettings.get_by_name('realm')
 
                    hgsettings2.app_settings_value = \
 
                        form_result['rhodecode_realm']
 

	
 
                    hgsettings3 = RhodeCodeSettings.get_by_name('ga_code')
 
                    hgsettings3.app_settings_value = \
 
                        form_result['rhodecode_ga_code']
 

	
 
                    self.sa.add(hgsettings1)
 
                    self.sa.add(hgsettings2)
 
                    self.sa.add(hgsettings3)
 
                    self.sa.commit()
 
                    set_rhodecode_config(config)
 
                    h.flash(_('Updated application settings'),
 
                            category='success')
 

	
 
                except Exception:
 
                    log.error(traceback.format_exc())
 
                    h.flash(_('error occurred during updating '
 
                              'application settings'),
 
                            category='error')
 

	
 
                    self.sa.rollback()
 

	
 
            except formencode.Invalid, errors:
 
                return htmlfill.render(
 
                     render('admin/settings/settings.html'),
 
                     defaults=errors.value,
 
                     errors=errors.error_dict or {},
 
                     prefix_error=False,
 
                     encoding="UTF-8")
 

	
 
        if setting_id == 'mercurial':
 
            application_form = ApplicationUiSettingsForm()()
 
            try:
 
                form_result = application_form.to_python(dict(request.POST))
 

	
 
                try:
 

	
 
                    hgsettings1 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key == 'push_ssl').one()
 
                    hgsettings1.ui_value = form_result['web_push_ssl']
 

	
 
                    hgsettings2 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key == '/').one()
 
                    hgsettings2.ui_value = form_result['paths_root_path']
 

	
 
                    #HOOKS
 
                    hgsettings3 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key == 'changegroup.update').one()
 
                    hgsettings3.ui_active = \
 
                        bool(form_result['hooks_changegroup_update'])
 

	
 
                    hgsettings4 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key ==
 
                            'changegroup.repo_size').one()
 
                    hgsettings4.ui_active = \
 
                        bool(form_result['hooks_changegroup_repo_size'])
 

	
 
                    hgsettings5 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key ==
 
                            'pretxnchangegroup.push_logger').one()
 
                    hgsettings5.ui_active = \
 
                        bool(form_result['hooks_pretxnchangegroup'
 
                                         '_push_logger'])
 

	
 
                    hgsettings6 = self.sa.query(RhodeCodeUi)\
 
                    .filter(RhodeCodeUi.ui_key ==
 
                            'preoutgoing.pull_logger').one()
 
                    hgsettings6.ui_active = \
 
                        bool(form_result['hooks_preoutgoing_pull_logger'])
 

	
 
                    self.sa.add(hgsettings1)
 
                    self.sa.add(hgsettings2)
 
                    self.sa.add(hgsettings3)
 
                    self.sa.add(hgsettings4)
 
                    self.sa.add(hgsettings5)
 
                    self.sa.add(hgsettings6)
 
                    self.sa.commit()
 

	
 
                    h.flash(_('Updated mercurial settings'),
 
                            category='success')
 

	
 
                except:
 
                    log.error(traceback.format_exc())
 
                    h.flash(_('error occurred during updating '
 
                              'application settings'), category='error')
 

	
 
                    self.sa.rollback()
 

	
 
            except formencode.Invalid, errors:
 
                return htmlfill.render(
 
                     render('admin/settings/settings.html'),
 
                     defaults=errors.value,
 
                     errors=errors.error_dict or {},
 
                     prefix_error=False,
 
                     encoding="UTF-8")
 

	
 

	
 
        if setting_id == 'hooks':
 
            ui_key = request.POST.get('new_hook_ui_key')
 
            ui_value = request.POST.get('new_hook_ui_value')
 
            try:
 
                
 
                if ui_value and ui_key:
 
                    RhodeCodeUi.create_or_update_hook(ui_key, ui_value)
 
                    h.flash(_('Added new hook'),
 
                            category='success')
 

	
 
                # check for edits
 
                update = False
 
                _d = request.POST.dict_of_lists()
 
                for k, v in zip(_d.get('hook_ui_key',[]), _d.get('hook_ui_value_new',[])):
 
                    RhodeCodeUi.create_or_update_hook(k, v)
 
                    update = True
 

	
 
                if update:
 
                    h.flash(_('Updated hooks'), category='success')
 

	
 
            except:
 
                log.error(traceback.format_exc())
 
                h.flash(_('error occurred during hook creation'),
 
                        category='error')
 

	
 
            return redirect(url('admin_edit_setting', setting_id='hooks'))
 

	
 
        return redirect(url('admin_settings'))
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def delete(self, setting_id):
 
        """DELETE /admin/settings/setting_id: Delete an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="DELETE" />
 
        # Or using helpers:
 
        #    h.form(url('admin_setting', setting_id=ID),
 
        #           method='delete')
 
        # url('admin_setting', setting_id=ID)
 
        if setting_id == 'hooks':
 
            hook_id = request.POST.get('hook_id')
 
            RhodeCodeUi.delete(hook_id)
 
            
 
            
 
    @HasPermissionAllDecorator('hg.admin')
 
    def show(self, setting_id, format='html'):
 
        """
 
        GET /admin/settings/setting_id: Show a specific item"""
 
        # url('admin_setting', setting_id=ID)
 

	
 
    @HasPermissionAllDecorator('hg.admin')
 
    def edit(self, setting_id, format='html'):
 
        """
 
        GET /admin/settings/setting_id/edit: Form to
 
        edit an existing item"""
 
        # url('admin_edit_setting', setting_id=ID)
 

	
 
        c.hooks = RhodeCodeUi.get_builtin_hooks()
 
        c.custom_hooks = RhodeCodeUi.get_custom_hooks()
 

	
 
        return htmlfill.render(
 
            render('admin/settings/hooks.html'),
 
            defaults={},
 
            encoding="UTF-8",
 
            force_defaults=False
 
        )
 

	
 
    @NotAnonymous()
 
    def my_account(self):
 
        """
 
        GET /_admin/my_account Displays info about my account
 
        """
 
        # url('admin_settings_my_account')
 

	
 
        c.user = User.get(self.rhodecode_user.user_id)
 
        all_repos = self.sa.query(Repository)\
 
                     .filter(Repository.user_id == c.user.user_id)\
 
                     .order_by(func.lower(Repository.repo_name)).all()
 

	
 
        c.user_repos = ScmModel().get_repos(all_repos)
 

	
 
        if c.user.username == 'default':
 
            h.flash(_("You can't edit this user since it's"
 
              " crucial for entire application"), category='warning')
 
            return redirect(url('users'))
 

	
 
        defaults = c.user.get_dict()
 
        return htmlfill.render(
 
            render('admin/users/user_edit_my_account.html'),
 
            defaults=defaults,
 
            encoding="UTF-8",
 
            force_defaults=False
 
        )
 

	
 
    def my_account_update(self):
 
        """PUT /_admin/my_account_update: Update an existing item"""
 
        # Forms posted to this method should contain a hidden field:
 
        #    <input type="hidden" name="_method" value="PUT" />
 
        # Or using helpers:
 
        #    h.form(url('admin_settings_my_account_update'),
 
        #           method='put')
 
        # url('admin_settings_my_account_update', id=ID)
 
        user_model = UserModel()
 
        uid = self.rhodecode_user.user_id
 
        _form = UserForm(edit=True,
 
                         old_data={'user_id': uid,
 
                                   'email': self.rhodecode_user.email})()
 
        form_result = {}
 
        try:
 
            form_result = _form.to_python(dict(request.POST))
 
            user_model.update_my_account(uid, form_result)
 
            h.flash(_('Your account was updated successfully'),
 
                    category='success')
 

	
 
        except formencode.Invalid, errors:
 
            c.user = User.get(self.rhodecode_user.user_id)
 
            all_repos = self.sa.query(Repository)\
 
                .filter(Repository.user_id == c.user.user_id)\
 
                .order_by(func.lower(Repository.repo_name))\
 
                .all()
 
            c.user_repos = ScmModel().get_repos(all_repos)
 

	
 
            return htmlfill.render(
 
                render('admin/users/user_edit_my_account.html'),
 
                defaults=errors.value,
 
                errors=errors.error_dict or {},
 
                prefix_error=False,
 
                encoding="UTF-8")
 
        except Exception:
 
            log.error(traceback.format_exc())
 
            h.flash(_('error occurred during update of user %s') \
 
                    % form_result.get('username'), category='error')
 

	
 
        return redirect(url('my_account'))
 

	
 
    @NotAnonymous()
 
    @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
 
    def create_repository(self):
 
        """GET /_admin/create_repository: Form to create a new item"""
 

	
 
        c.repo_groups = [('', '')]
 
        parents_link = lambda k: h.literal('&raquo;'.join(
 
                                    map(lambda k: k.group_name,
 
                                        k.parents + [k])
 
                                    )
 
                                )
 

	
 
        c.repo_groups.extend([(x.group_id, parents_link(x)) for \
 
                                            x in self.sa.query(Group).all()])
 
        c.repo_groups = sorted(c.repo_groups,
 
                               key=lambda t: t[1].split('&raquo;')[0])
 
        c.repo_groups = Group.groups_choices()
 
        c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)
 

	
 
        new_repo = request.GET.get('repo', '')
 
        c.new_repo = repo_name_slug(new_repo)
 

	
 
        return render('admin/repos/repo_add_create_repository.html')
 

	
 
    def get_hg_ui_settings(self):
 
        ret = self.sa.query(RhodeCodeUi).all()
 

	
 
        if not ret:
 
            raise Exception('Could not get application ui settings !')
 
        settings = {}
 
        for each in ret:
 
            k = each.ui_key
 
            v = each.ui_value
 
            if k == '/':
 
                k = 'root_path'
 

	
 
            if k.find('.') != -1:
 
                k = k.replace('.', '_')
 

	
 
            if each.ui_section == 'hooks':
 
                v = each.ui_active
 

	
 
            settings[each.ui_section + '_' + k] = v
 

	
 
        return settings
rhodecode/controllers/files.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.controllers.files
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Files controller for RhodeCode
 

	
 
    :created_on: Apr 21, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import logging
 
import traceback
 

	
 
from os.path import join as jn
 

	
 
from pylons import request, response, session, tmpl_context as c, url
 
from pylons.i18n.translation import _
 
from pylons.controllers.util import redirect
 
from pylons.decorators import jsonify
 

	
 
from vcs.conf import settings
 
from vcs.exceptions import RepositoryError, ChangesetDoesNotExistError, \
 
    EmptyRepositoryError, ImproperArchiveTypeError, VCSError, NodeAlreadyExistsError
 
from vcs.nodes import FileNode, NodeKind
 
from vcs.utils import diffs as differ
 

	
 
from rhodecode.lib import convert_line_endings, detect_mode, safe_str
 
from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 
from rhodecode.lib.base import BaseRepoController, render
 
from rhodecode.lib.utils import EmptyChangeset
 
import rhodecode.lib.helpers as h
 
from rhodecode.model.repo import RepoModel
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
class FilesController(BaseRepoController):
 

	
 
    @LoginRequired()
 
    def __before__(self):
 
        super(FilesController, self).__before__()
 
        c.cut_off_limit = self.cut_off_limit
 

	
 
    def __get_cs_or_redirect(self, rev, repo_name, redirect_after=True):
 
        """
 
        Safe way to get changeset if error occur it redirects to tip with
 
        proper message
 

	
 
        :param rev: revision to fetch
 
        :param repo_name: repo name to redirect after
 
        """
 

	
 
        try:
 
            return c.rhodecode_repo.get_changeset(rev)
 
        except EmptyRepositoryError, e:
 
            if not redirect_after:
 
                return None
 
            url_ = url('files_add_home',
 
                       repo_name=c.repo_name,
 
                       revision=0, f_path='')
 
            add_new = '<a href="%s">[%s]</a>' % (url_, _('add new'))
 
            h.flash(h.literal(_('There are no files yet %s' % add_new)),
 
                    category='warning')
 
            redirect(h.url('summary_home', repo_name=repo_name))
 

	
 
        except RepositoryError, e:
 
            h.flash(str(e), category='warning')
 
            redirect(h.url('files_home', repo_name=repo_name, revision='tip'))
 

	
 
    def __get_filenode_or_redirect(self, repo_name, cs, path):
 
        """
 
        Returns file_node, if error occurs or given path is directory,
 
        it'll redirect to top level path
 

	
 
        :param repo_name: repo_name
 
        :param cs: given changeset
 
        :param path: path to lookup
 
        """
 

	
 
        try:
 
            file_node = cs.get_node(path)
 
            if file_node.is_dir():
 
                raise RepositoryError('given path is a directory')
 
        except RepositoryError, e:
 
            h.flash(str(e), category='warning')
 
            redirect(h.url('files_home', repo_name=repo_name,
 
                           revision=cs.raw_id))
 

	
 
        return file_node
 

	
 

	
 
    def __get_paths(self, changeset, starting_path):
 
        """recursive walk in root dir and return a set of all path in that dir
 
        based on repository walk function
 
        """
 
        _files = list()
 
        _dirs = list()
 

	
 
        try:
 
            tip = changeset
 
            for topnode, dirs, files in tip.walk(starting_path):
 
                for f in files:
 
                    _files.append(f.path)
 
                for d in dirs:
 
                    _dirs.append(d.path)
 
        except RepositoryError, e:
 
            log.debug(traceback.format_exc())
 
            pass
 
        return _dirs, _files
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def index(self, repo_name, revision, f_path):
 
        #reditect to given revision from form if given
 
        post_revision = request.POST.get('at_rev', None)
 
        if post_revision:
 
            cs = self.__get_cs_or_redirect(post_revision, repo_name)
 
            redirect(url('files_home', repo_name=c.repo_name,
 
                         revision=cs.raw_id, f_path=f_path))
 

	
 
        c.changeset = self.__get_cs_or_redirect(revision, repo_name)
 
        c.branch = request.GET.get('branch', None)
 
        c.f_path = f_path
 

	
 
        cur_rev = c.changeset.revision
 

	
 
        #prev link
 
        try:
 
            prev_rev = c.rhodecode_repo.get_changeset(cur_rev).prev(c.branch)
 
            c.url_prev = url('files_home', repo_name=c.repo_name,
 
                         revision=prev_rev.raw_id, f_path=f_path)
 
            if c.branch:
 
                c.url_prev += '?branch=%s' % c.branch
 
        except (ChangesetDoesNotExistError, VCSError):
 
            c.url_prev = '#'
 

	
 
        #next link
 
        try:
 
            next_rev = c.rhodecode_repo.get_changeset(cur_rev).next(c.branch)
 
            c.url_next = url('files_home', repo_name=c.repo_name,
 
                     revision=next_rev.raw_id, f_path=f_path)
 
            if c.branch:
 
                c.url_next += '?branch=%s' % c.branch
 
        except (ChangesetDoesNotExistError, VCSError):
 
            c.url_next = '#'
 

	
 
        #files or dirs
 
        try:
 
            c.files_list = c.changeset.get_node(f_path)
 

	
 
            if c.files_list.is_file():
 
                c.file_history = self._get_node_history(c.changeset, f_path)
 
            else:
 
                c.file_history = []
 
        except RepositoryError, e:
 
            h.flash(str(e), category='warning')
 
            redirect(h.url('files_home', repo_name=repo_name,
 
                           revision=revision))
 

	
 
        return render('files/files.html')
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def rawfile(self, repo_name, revision, f_path):
 
        cs = self.__get_cs_or_redirect(revision, repo_name)
 
        file_node = self.__get_filenode_or_redirect(repo_name, cs, f_path)
 

	
 
        response.content_disposition = 'attachment; filename=%s' % \
 
            safe_str(f_path.split(os.sep)[-1])
 

	
 
        response.content_type = file_node.mimetype
 
        return file_node.content
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def raw(self, repo_name, revision, f_path):
 
        cs = self.__get_cs_or_redirect(revision, repo_name)
 
        file_node = self.__get_filenode_or_redirect(repo_name, cs, f_path)
 

	
 
        raw_mimetype_mapping = {
 
            # map original mimetype to a mimetype used for "show as raw"
 
            # you can also provide a content-disposition to override the
 
            # default "attachment" disposition.
 
            # orig_type: (new_type, new_dispo)
 

	
 
            # show images inline:
 
            'image/x-icon': ('image/x-icon', 'inline'),
 
            'image/png': ('image/png', 'inline'),
 
            'image/gif': ('image/gif', 'inline'),
 
            'image/jpeg': ('image/jpeg', 'inline'),
 
            'image/svg+xml': ('image/svg+xml', 'inline'),
 
        }
 

	
 
        mimetype = file_node.mimetype
 
        try:
 
            mimetype, dispo = raw_mimetype_mapping[mimetype]
 
        except KeyError:
 
            # we don't know anything special about this, handle it safely
 
            if file_node.is_binary:
 
                # do same as download raw for binary files
 
                mimetype, dispo = 'application/octet-stream', 'attachment'
 
            else:
 
                # do not just use the original mimetype, but force text/plain,
 
                # otherwise it would serve text/html and that might be unsafe.
 
                # Note: underlying vcs library fakes text/plain mimetype if the
 
                # mimetype can not be determined and it thinks it is not
 
                # binary.This might lead to erroneous text display in some
 
                # cases, but helps in other cases, like with text files
 
                # without extension.
 
                mimetype, dispo = 'text/plain', 'inline'
 

	
 
        if dispo == 'attachment':
 
            dispo = 'attachment; filename=%s' % \
 
                        safe_str(f_path.split(os.sep)[-1])
 

	
 
        response.content_disposition = dispo
 
        response.content_type = mimetype
 
        return file_node.content
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def annotate(self, repo_name, revision, f_path):
 
        c.cs = self.__get_cs_or_redirect(revision, repo_name)
 
        c.file = self.__get_filenode_or_redirect(repo_name, c.cs, f_path)
 

	
 
        c.file_history = self._get_node_history(c.cs, f_path)
 
        c.f_path = f_path
 
        return render('files/files_annotate.html')
 

	
 
    @HasRepoPermissionAnyDecorator('repository.write', 'repository.admin')
 
    def edit(self, repo_name, revision, f_path):
 
        r_post = request.POST
 

	
 
        c.cs = self.__get_cs_or_redirect(revision, repo_name)
 
        c.file = self.__get_filenode_or_redirect(repo_name, c.cs, f_path)
 

	
 
        if c.file.is_binary:
 
            return redirect(url('files_home', repo_name=c.repo_name,
 
                         revision=c.cs.raw_id, f_path=f_path))
 

	
 
        c.f_path = f_path
 

	
 
        if r_post:
 

	
 
            old_content = c.file.content
 
            sl = old_content.splitlines(1)
 
            first_line = sl[0] if sl else ''
 
            # modes:  0 - Unix, 1 - Mac, 2 - DOS
 
            mode = detect_mode(first_line, 0)
 
            content = convert_line_endings(r_post.get('content'), mode)
 

	
 
            message = r_post.get('message') or (_('Edited %s via RhodeCode')
 
                                                % (f_path))
 
            author = self.rhodecode_user.full_contact
 

	
 
            if content == old_content:
 
                h.flash(_('No changes'),
 
                    category='warning')
 
                return redirect(url('changeset_home', repo_name=c.repo_name,
 
                                    revision='tip'))
 

	
 
            try:
 
                self.scm_model.commit_change(repo=c.rhodecode_repo,
 
                                             repo_name=repo_name, cs=c.cs,
 
                                             user=self.rhodecode_user,
 
                                             author=author, message=message,
 
                                             content=content, f_path=f_path)
 
                h.flash(_('Successfully committed to %s' % f_path),
 
                        category='success')
 

	
 
            except Exception:
 
                log.error(traceback.format_exc())
 
                h.flash(_('Error occurred during commit'), category='error')
 
            return redirect(url('changeset_home',
 
                                repo_name=c.repo_name, revision='tip'))
 

	
 
        return render('files/files_edit.html')
 

	
 
    @HasRepoPermissionAnyDecorator('repository.write', 'repository.admin')
 
    def add(self, repo_name, revision, f_path):
 
        r_post = request.POST
 
        c.cs = self.__get_cs_or_redirect(revision, repo_name,
 
                                         redirect_after=False)
 
        if c.cs is None:
 
            c.cs = EmptyChangeset(alias=c.rhodecode_repo.alias)
 

	
 
        c.f_path = f_path
 

	
 
        if r_post:
 
            unix_mode = 0
 
            content = convert_line_endings(r_post.get('content'), unix_mode)
 

	
 
            message = r_post.get('message') or (_('Added %s via RhodeCode')
 
                                                % (f_path))
 
            location = r_post.get('location')
 
            filename = r_post.get('filename')
 
            file_obj = r_post.get('upload_file', None)
 

	
 
            if file_obj is not None and hasattr(file_obj, 'filename'):
 
                filename = file_obj.filename
 
                content = file_obj.file
 

	
 
            #TODO: REMOVE THIS !!
 
            ################################
 
            import ipdb;ipdb.set_trace()
 
            print 'setting ipdb debuggin for rhodecode.controllers.files.FilesController.add'
 
            ################################
 

	
 

	
 
            node_path = os.path.join(location, filename)
 
            author = self.rhodecode_user.full_contact
 

	
 
            if not content:
 
                h.flash(_('No content'), category='warning')
 
                return redirect(url('changeset_home', repo_name=c.repo_name,
 
                                    revision='tip'))
 
            if not filename:
 
                h.flash(_('No filename'), category='warning')
 
                return redirect(url('changeset_home', repo_name=c.repo_name,
 
                                    revision='tip'))
 

	
 
            try:
 
                self.scm_model.create_node(repo=c.rhodecode_repo,
 
                                             repo_name=repo_name, cs=c.cs,
 
                                             user=self.rhodecode_user,
 
                                             author=author, message=message,
 
                                             content=content, f_path=node_path)
 
                h.flash(_('Successfully committed to %s' % node_path),
 
                        category='success')
 
            except NodeAlreadyExistsError, e:
 
                h.flash(_(e), category='error')
 
            except Exception:
 
                log.error(traceback.format_exc())
 
                h.flash(_('Error occurred during commit'), category='error')
 
            return redirect(url('changeset_home',
 
                                repo_name=c.repo_name, revision='tip'))
 

	
 
        return render('files/files_add.html')
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def archivefile(self, repo_name, fname):
 

	
 
        fileformat = None
 
        revision = None
 
        ext = None
 
        subrepos = request.GET.get('subrepos') == 'true'
 

	
 
        for a_type, ext_data in settings.ARCHIVE_SPECS.items():
 
            archive_spec = fname.split(ext_data[1])
 
            if len(archive_spec) == 2 and archive_spec[1] == '':
 
                fileformat = a_type or ext_data[1]
 
                revision = archive_spec[0]
 
                ext = ext_data[1]
 

	
 
        try:
 
            dbrepo = RepoModel().get_by_repo_name(repo_name)
 
            if dbrepo.enable_downloads is False:
 
                return _('downloads disabled')
 

	
 
            cs = c.rhodecode_repo.get_changeset(revision)
 
            content_type = settings.ARCHIVE_SPECS[fileformat][0]
 
        except ChangesetDoesNotExistError:
 
            return _('Unknown revision %s') % revision
 
        except EmptyRepositoryError:
 
            return _('Empty repository')
 
        except (ImproperArchiveTypeError, KeyError):
 
            return _('Unknown archive type')
 

	
 
        response.content_type = content_type
 
        response.content_disposition = 'attachment; filename=%s-%s%s' \
 
            % (repo_name, revision, ext)
 

	
 
        import tempfile
 
        archive = tempfile.mkstemp()[1]
 
        t = open(archive, 'wb')
 
        cs.fill_archive(stream=t, kind=fileformat, subrepos=subrepos)
 

	
 
        def get_chunked_archive(archive):
 
            stream = open(archive, 'rb')
 
            while True:
 
                data = stream.read(4096)
 
                if not data:
 
                    os.remove(archive)
 
                    break
 
                yield data
 

	
 
        return get_chunked_archive(archive)
 

	
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def diff(self, repo_name, f_path):
 
        diff1 = request.GET.get('diff1')
 
        diff2 = request.GET.get('diff2')
 
        c.action = request.GET.get('diff')
 
        c.no_changes = diff1 == diff2
 
        c.f_path = f_path
 
        c.big_diff = False
 

	
 
        try:
 
            if diff1 not in ['', None, 'None', '0' * 12, '0' * 40]:
 
                c.changeset_1 = c.rhodecode_repo.get_changeset(diff1)
 
                node1 = c.changeset_1.get_node(f_path)
 
            else:
 
                c.changeset_1 = EmptyChangeset(repo=c.rhodecode_repo)
 
                node1 = FileNode('.', '', changeset=c.changeset_1)
 

	
 
            if diff2 not in ['', None, 'None', '0' * 12, '0' * 40]:
 
                c.changeset_2 = c.rhodecode_repo.get_changeset(diff2)
 
                node2 = c.changeset_2.get_node(f_path)
 
            else:
 
                c.changeset_2 = EmptyChangeset(repo=c.rhodecode_repo)
 
                node2 = FileNode('.', '', changeset=c.changeset_2)
 
        except RepositoryError:
 
            return redirect(url('files_home',
 
                                repo_name=c.repo_name, f_path=f_path))
 

	
 
        if c.action == 'download':
 
            diff = differ.DiffProcessor(differ.get_gitdiff(node1, node2),
 
                                        format='gitdiff')
 

	
 
            diff_name = '%s_vs_%s.diff' % (diff1, diff2)
 
            response.content_type = 'text/plain'
 
            response.content_disposition = 'attachment; filename=%s' \
 
                                                    % diff_name
 
            return diff.raw_diff()
 

	
 
        elif c.action == 'raw':
 
            diff = differ.DiffProcessor(differ.get_gitdiff(node1, node2),
 
                                        format='gitdiff')
 
            response.content_type = 'text/plain'
 
            return diff.raw_diff()
 

	
 
        elif c.action == 'diff':
 
            if node1.is_binary or node2.is_binary:
 
                c.cur_diff = _('Binary file')
 
            elif node1.size > self.cut_off_limit or \
 
                    node2.size > self.cut_off_limit:
 
                c.cur_diff = ''
 
                c.big_diff = True
 
            else:
 
                diff = differ.DiffProcessor(differ.get_gitdiff(node1, node2),
 
                                        format='gitdiff')
 
                c.cur_diff = diff.as_html()
 
        else:
 

	
 
            #default option
 
            if node1.is_binary or node2.is_binary:
 
                c.cur_diff = _('Binary file')
 
            elif node1.size > self.cut_off_limit or \
 
                    node2.size > self.cut_off_limit:
 
                c.cur_diff = ''
 
                c.big_diff = True
 

	
 
            else:
 
                diff = differ.DiffProcessor(differ.get_gitdiff(node1, node2),
 
                                        format='gitdiff')
 
                c.cur_diff = diff.as_html()
 

	
 
        if not c.cur_diff and not c.big_diff:
 
            c.no_changes = True
 
        return render('files/file_diff.html')
 

	
 
    def _get_node_history(self, cs, f_path):
 
        changesets = cs.get_file_history(f_path)
 
        hist_l = []
 

	
 
        changesets_group = ([], _("Changesets"))
 
        branches_group = ([], _("Branches"))
 
        tags_group = ([], _("Tags"))
 

	
 
        for chs in changesets:
 
            n_desc = 'r%s:%s' % (chs.revision, chs.short_id)
 
            changesets_group[0].append((chs.raw_id, n_desc,))
 

	
 
        hist_l.append(changesets_group)
 

	
 
        for name, chs in c.rhodecode_repo.branches.items():
 
            #chs = chs.split(':')[-1]
 
            branches_group[0].append((chs, name),)
 
        hist_l.append(branches_group)
 

	
 
        for name, chs in c.rhodecode_repo.tags.items():
 
            #chs = chs.split(':')[-1]
 
            tags_group[0].append((chs, name),)
 
        hist_l.append(tags_group)
 

	
 
        return hist_l
 

	
 
    @jsonify
 
    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
 
                                   'repository.admin')
 
    def nodelist(self, repo_name, revision, f_path):
 
        if request.environ.get('HTTP_X_PARTIAL_XHR'):
 
            cs = self.__get_cs_or_redirect(revision, repo_name)
 
            _d, _f = self.__get_paths(cs, f_path)
 
            return _d + _f
 

	
rhodecode/lib/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.__init__
 
    ~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Some simple helper functions
 

	
 
    :created_on: Jan 5, 2011
 
    :author: marcink
 
    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 

	
 
def __get_lem():
 
    from pygments import lexers
 
    from string import lower
 
    from collections import defaultdict
 

	
 
    d = defaultdict(lambda: [])
 

	
 
    def __clean(s):
 
        s = s.lstrip('*')
 
        s = s.lstrip('.')
 

	
 
        if s.find('[') != -1:
 
            exts = []
 
            start, stop = s.find('['), s.find(']')
 

	
 
            for suffix in s[start + 1:stop]:
 
                exts.append(s[:s.find('[')] + suffix)
 
            return map(lower, exts)
 
        else:
 
            return map(lower, [s])
 

	
 
    for lx, t in sorted(lexers.LEXERS.items()):
 
        m = map(__clean, t[-2])
 
        if m:
 
            m = reduce(lambda x, y: x + y, m)
 
            for ext in m:
 
                desc = lx.replace('Lexer', '')
 
                d[ext].append(desc)
 

	
 
    return dict(d)
 

	
 
# language map is also used by whoosh indexer, which for those specified
 
# extensions will index it's content
 
LANGUAGES_EXTENSIONS_MAP = __get_lem()
 

	
 
# Additional mappings that are not present in the pygments lexers
 
# NOTE: that this will overide any mappings in LANGUAGES_EXTENSIONS_MAP
 
ADDITIONAL_MAPPINGS = {'xaml': 'XAML'}
 

	
 
LANGUAGES_EXTENSIONS_MAP.update(ADDITIONAL_MAPPINGS)
 

	
 

	
 
def str2bool(_str):
 
    """
 
    returs True/False value from given string, it tries to translate the
 
    string into boolean
 

	
 
    :param _str: string value to translate into boolean
 
    :rtype: boolean
 
    :returns: boolean from given string
 
    """
 
    if _str is None:
 
        return False
 
    if _str in (True, False):
 
        return _str
 
    _str = str(_str).strip().lower()
 
    return _str in ('t', 'true', 'y', 'yes', 'on', '1')
 

	
 

	
 
def convert_line_endings(line, mode):
 
    """
 
    Converts a given line  "line end" accordingly to given mode
 
    
 
    Available modes are::
 
        0 - Unix
 
        1 - Mac
 
        2 - DOS
 
    
 
    :param line: given line to convert
 
    :param mode: mode to convert to
 
    :rtype: str
 
    :return: converted line according to mode
 
    """
 
    from string import replace
 

	
 
    if mode == 0:
 
            line = replace(line, '\r\n', '\n')
 
            line = replace(line, '\r', '\n')
 
    elif mode == 1:
 
            line = replace(line, '\r\n', '\r')
 
            line = replace(line, '\n', '\r')
 
    elif mode == 2:
 
            import re
 
            line = re.sub("\r(?!\n)|(?<!\r)\n", "\r\n", line)
 
    return line
 

	
 

	
 
def detect_mode(line, default):
 
    """
 
    Detects line break for given line, if line break couldn't be found
 
    given default value is returned
 

	
 
    :param line: str line
 
    :param default: default
 
    :rtype: int
 
    :return: value of line end on of 0 - Unix, 1 - Mac, 2 - DOS
 
    """
 
    if line.endswith('\r\n'):
 
        return 2
 
    elif line.endswith('\n'):
 
        return 0
 
    elif line.endswith('\r'):
 
        return 1
 
    else:
 
        return default
 

	
 

	
 
def generate_api_key(username, salt=None):
 
    """
 
    Generates unique API key for given username, if salt is not given
 
    it'll be generated from some random string
 

	
 
    :param username: username as string
 
    :param salt: salt to hash generate KEY
 
    :rtype: str
 
    :returns: sha1 hash from username+salt
 
    """
 
    from tempfile import _RandomNameSequence
 
    import hashlib
 

	
 
    if salt is None:
 
        salt = _RandomNameSequence().next()
 

	
 
    return hashlib.sha1(username + salt).hexdigest()
 

	
 

	
 
def safe_unicode(str_, from_encoding='utf8'):
 
    """
 
    safe unicode function. Does few trick to turn str_ into unicode
 
     
 
    In case of UnicodeDecode error we try to return it with encoding detected
 
    by chardet library if it fails fallback to unicode with errors replaced
 

	
 
    :param str_: string to decode
 
    :rtype: unicode
 
    :returns: unicode object
 
    """
 
    if isinstance(str_, unicode):
 
        return str_
 

	
 
    try:
 
        return unicode(str_)
 
    except UnicodeDecodeError:
 
        pass
 

	
 
    try:
 
        return unicode(str_, from_encoding)
 
    except UnicodeDecodeError:
 
        pass
 

	
 
    try:
 
        import chardet
 
        encoding = chardet.detect(str_)['encoding']
 
        if encoding is None:
 
            raise Exception()
 
        return str_.decode(encoding)
 
    except (ImportError, UnicodeDecodeError, Exception):
 
        return unicode(str_, from_encoding, 'replace')
 

	
 
def safe_str(unicode_, to_encoding='utf8'):
 
    """
 
    safe str function. Does few trick to turn unicode_ into string
 
     
 
    In case of UnicodeEncodeError we try to return it with encoding detected
 
    by chardet library if it fails fallback to string with errors replaced
 

	
 
    :param unicode_: unicode to encode
 
    :rtype: str
 
    :returns: str object
 
    """
 

	
 
    if isinstance(unicode_, str):
 
        return unicode_
 

	
 
    try:
 
        return unicode_.encode(to_encoding)
 
    except UnicodeEncodeError:
 
        pass
 

	
 
    try:
 
        import chardet
 
        encoding = chardet.detect(unicode_)['encoding']
 
        print encoding
 
        if encoding is None:
 
            raise UnicodeEncodeError()
 

	
 
        return unicode_.encode(encoding)
 
    except (ImportError, UnicodeEncodeError):
 
        return unicode_.encode(to_encoding, 'replace')
 

	
 
    return safe_str
 

	
 

	
 

	
 
def engine_from_config(configuration, prefix='sqlalchemy.', **kwargs):
 
    """
 
    Custom engine_from_config functions that makes sure we use NullPool for
 
    file based sqlite databases. This prevents errors on sqlite. This only 
 
    applies to sqlalchemy versions < 0.7.0
 

	
 
    """
 
    import sqlalchemy
 
    from sqlalchemy import engine_from_config as efc
 
    import logging
 

	
 
    if int(sqlalchemy.__version__.split('.')[1]) < 7:
 

	
 
        # This solution should work for sqlalchemy < 0.7.0, and should use
 
        # proxy=TimerProxy() for execution time profiling
 

	
 
        from sqlalchemy.pool import NullPool
 
        url = configuration[prefix + 'url']
 

	
 
        if url.startswith('sqlite'):
 
            kwargs.update({'poolclass': NullPool})
 
        return efc(configuration, prefix, **kwargs)
 
    else:
 
        import time
 
        from sqlalchemy import event
 
        from sqlalchemy.engine import Engine
 

	
 
        log = logging.getLogger('sqlalchemy.engine')
 
        BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = xrange(30, 38)
 
        engine = efc(configuration, prefix, **kwargs)
 

	
 
        def color_sql(sql):
 
            COLOR_SEQ = "\033[1;%dm"
 
            COLOR_SQL = YELLOW
 
            normal = '\x1b[0m'
 
            return ''.join([COLOR_SEQ % COLOR_SQL, sql, normal])
 

	
 
        if configuration['debug']:
 
            #attach events only for debug configuration
 

	
 
            def before_cursor_execute(conn, cursor, statement,
 
                                    parameters, context, executemany):
 
                context._query_start_time = time.time()
 
                log.info(color_sql(">>>>> STARTING QUERY >>>>>"))
 

	
 

	
 
            def after_cursor_execute(conn, cursor, statement,
 
                                    parameters, context, executemany):
 
                total = time.time() - context._query_start_time
 
                log.info(color_sql("<<<<< TOTAL TIME: %f <<<<<" % total))
 

	
 
            event.listen(engine, "before_cursor_execute",
 
                         before_cursor_execute)
 
            event.listen(engine, "after_cursor_execute",
 
                         after_cursor_execute)
 

	
 
    return engine
 

	
 

	
 
def age(curdate):
 
    """
 
    turns a datetime into an age string.
 
    
 
    :param curdate: datetime object
 
    :rtype: unicode
 
    :returns: unicode words describing age
 
    """
 

	
 
    from datetime import datetime
 
    from webhelpers.date import time_ago_in_words
 

	
 
    _ = lambda s:s
 

	
 
    if not curdate:
 
        return ''
 

	
 
    agescales = [(_(u"year"), 3600 * 24 * 365),
 
                 (_(u"month"), 3600 * 24 * 30),
 
                 (_(u"day"), 3600 * 24),
 
                 (_(u"hour"), 3600),
 
                 (_(u"minute"), 60),
 
                 (_(u"second"), 1), ]
 

	
 
    age = datetime.now() - curdate
 
    age_seconds = (age.days * agescales[2][1]) + age.seconds
 
    pos = 1
 
    for scale in agescales:
 
        if scale[1] <= age_seconds:
 
            if pos == 6:pos = 5
 
            return '%s %s' % (time_ago_in_words(curdate,
 
                                                agescales[pos][0]), _('ago'))
 
        pos += 1
 

	
 
    return _(u'just now')
 

	
 

	
 
def uri_filter(uri):
 
    """
 
    Removes user:password from given url string
 
    
 
    :param uri:
 
    :rtype: unicode
 
    :returns: filtered list of strings  
 
    """
 
    if not uri:
 
        return ''
 

	
 
    proto = ''
 

	
 
    for pat in ('https://', 'http://'):
 
        if uri.startswith(pat):
 
            uri = uri[len(pat):]
 
            proto = pat
 
            break
 

	
 
    # remove passwords and username
 
    uri = uri[uri.find('@') + 1:]
 

	
 
    # get the port
 
    cred_pos = uri.find(':')
 
    if cred_pos == -1:
 
        host, port = uri, None
 
    else:
 
        host, port = uri[:cred_pos], uri[cred_pos + 1:]
 

	
 
    return filter(None, [proto, host, port])
 

	
 

	
 
def credentials_filter(uri):
 
    """
 
    Returns a url with removed credentials
 
    
 
    :param uri:
 
    """
 

	
 
    uri = uri_filter(uri)
 
    #check if we have port
 
    if len(uri) > 2 and uri[2]:
 
        uri[2] = ':' + uri[2]
 

	
 
    return ''.join(uri)
 

	
 
def get_changeset_safe(repo, rev):
 
    """
 
    Safe version of get_changeset if this changeset doesn't exists for a 
 
    repo it returns a Dummy one instead
 
    
 
    :param repo:
 
    :param rev:
 
    """
 
    from vcs.backends.base import BaseRepository
 
    from vcs.exceptions import RepositoryError
 
    if not isinstance(repo, BaseRepository):
 
        raise Exception('You must pass an Repository '
 
                        'object as first argument got %s', type(repo))
 

	
 
    try:
 
        cs = repo.get_changeset(rev)
 
    except RepositoryError:
 
        from rhodecode.lib.utils import EmptyChangeset
 
        cs = EmptyChangeset(requested_revision=rev)
 
    return cs
 

	
 

	
 
def get_current_revision(quiet=False):
 
    """
 
    Returns tuple of (number, id) from repository containing this package
 
    or None if repository could not be found.
 
    
 
    :param quiet: prints error for fetching revision if True
 
    """
 

	
 
    try:
 
        from vcs import get_repo
 
        from vcs.utils.helpers import get_scm
 
        from vcs.exceptions import RepositoryError, VCSError
 
        repopath = os.path.join(os.path.dirname(__file__), '..', '..')
 
        scm = get_scm(repopath)[0]
 
        repo = get_repo(path=repopath, alias=scm)
 
        tip = repo.get_changeset()
 
        return (tip.revision, tip.short_id)
 
    except (ImportError, RepositoryError, VCSError), err:
 
        if not quiet:
 
            print ("Cannot retrieve rhodecode's revision. Original error "
 
                   "was: %s" % err)
 
        return None
 

	
rhodecode/lib/celerylib/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.celerylib.__init__
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    celery libs for RhodeCode
 

	
 
    :created_on: Nov 27, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import sys
 
import socket
 
import traceback
 
import logging
 
from os.path import dirname as dn, join as jn
 

	
 
from hashlib import md5
 
from decorator import decorator
 
from pylons import  config
 

	
 
from vcs.utils.lazy import LazyProperty
 

	
 
from rhodecode.lib import str2bool
 
from rhodecode.lib.pidlock import DaemonLock, LockHeld
 

	
 
from celery.messaging import establish_connection
 

	
 

	
 
log = logging.getLogger(__name__)
 

	
 
try:
 
    CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
 
except KeyError:
 
    CELERY_ON = False
 

	
 

	
 
class ResultWrapper(object):
 
    def __init__(self, task):
 
        self.task = task
 

	
 
    @LazyProperty
 
    def result(self):
 
        return self.task
 

	
 

	
 
def run_task(task, *args, **kwargs):
 
    if CELERY_ON:
 
        try:
 
            t = task.apply_async(args=args, kwargs=kwargs)
 
            log.info('running task %s:%s', t.task_id, task)
 
            return t
 

	
 
        except socket.error, e:
 
            if isinstance(e, IOError) and e.errno == 111:
 
                log.debug('Unable to connect to celeryd. Sync execution')
 
            else:
 
                log.error(traceback.format_exc())
 
        except KeyError, e:
 
                log.debug('Unable to connect to celeryd. Sync execution')
 
        except Exception, e:
 
            log.error(traceback.format_exc())
 

	
 
    log.debug('executing task %s in sync mode', task)
 
    return ResultWrapper(task(*args, **kwargs))
 

	
 

	
 
def __get_lockkey(func, *fargs, **fkwargs):
 
    params = list(fargs)
 
    params.extend(['%s-%s' % ar for ar in fkwargs.items()])
 

	
 
    func_name = str(func.__name__) if hasattr(func, '__name__') else str(func)
 

	
 
    lockkey = 'task_%s.lock' % \
 
        md5(func_name + '-' + '-'.join(map(str, params))).hexdigest()
 
    return lockkey
 

	
 

	
 
def locked_task(func):
 
    def __wrapper(func, *fargs, **fkwargs):
 
        lockkey = __get_lockkey(func, *fargs, **fkwargs)
 
        lockkey_path = dn(dn(dn(os.path.abspath(__file__))))
 
        lockkey_path = config['here']
 

	
 
        log.info('running task with lockkey %s', lockkey)
 
        try:
 
            l = DaemonLock(jn(lockkey_path, lockkey))
 
            l = DaemonLock(file_=jn(lockkey_path, lockkey))
 
            ret = func(*fargs, **fkwargs)
 
            l.release()
 
            return ret
 
        except LockHeld:
 
            log.info('LockHeld')
 
            return 'Task with key %s already running' % lockkey
 

	
 
    return decorator(__wrapper, func)
rhodecode/lib/celerylib/tasks.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.celerylib.tasks
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    RhodeCode task modules, containing all task that suppose to be run
 
    by celery daemon
 

	
 
    :created_on: Oct 6, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
from celery.decorators import task
 

	
 
import os
 
import traceback
 
import logging
 
from os.path import dirname as dn, join as jn
 

	
 
from time import mktime
 
from operator import itemgetter
 
from string import lower
 

	
 
from pylons import config, url
 
from pylons.i18n.translation import _
 

	
 
from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, safe_str
 
from rhodecode.lib.celerylib import run_task, locked_task, str2bool, \
 
    __get_lockkey, LockHeld, DaemonLock
 
from rhodecode.lib.helpers import person
 
from rhodecode.lib.smtp_mailer import SmtpMailer
 
from rhodecode.lib.utils import add_cache
 
from rhodecode.lib.compat import json, OrderedDict
 

	
 
from rhodecode.model import init_model
 
from rhodecode.model import meta
 
from rhodecode.model.db import RhodeCodeUi, Statistics, Repository
 

	
 
from vcs.backends import get_repo
 

	
 
from sqlalchemy import engine_from_config
 

	
 
add_cache(config)
 

	
 

	
 

	
 
__all__ = ['whoosh_index', 'get_commits_stats',
 
           'reset_user_password', 'send_email']
 

	
 
CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
 

	
 

	
 
def get_session():
 
    if CELERY_ON:
 
        engine = engine_from_config(config, 'sqlalchemy.db1.')
 
        init_model(engine)
 
    sa = meta.Session()
 
    return sa
 

	
 

	
 
def get_repos_path():
 
    sa = get_session()
 
    q = sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
 
    return q.ui_value
 

	
 

	
 
@task(ignore_result=True)
 
@locked_task
 
def whoosh_index(repo_location, full_index):
 
    #log = whoosh_index.get_logger()
 
    from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
 
    index_location = config['index_dir']
 
    WhooshIndexingDaemon(index_location=index_location,
 
                         repo_location=repo_location, sa=get_session())\
 
                         .run(full_index=full_index)
 

	
 

	
 
@task(ignore_result=True)
 
def get_commits_stats(repo_name, ts_min_y, ts_max_y):
 
    try:
 
        log = get_commits_stats.get_logger()
 
    except:
 
        log = logging.getLogger(__name__)
 

	
 
    lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
 
                            ts_max_y)
 
    lockkey_path = dn(dn(dn(dn(os.path.abspath(__file__)))))
 
    lockkey_path = config['here']
 

	
 
    log.info('running task with lockkey %s', lockkey)
 
    try:
 
        lock = l = DaemonLock(jn(lockkey_path, lockkey))
 
        lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
 

	
 
        #for js data compatibilty cleans the key for person from '
 
        akc = lambda k: person(k).replace('"', "")
 

	
 
        co_day_auth_aggr = {}
 
        commits_by_day_aggregate = {}
 
        repos_path = get_repos_path()
 
        repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
 
        repo_size = len(repo.revisions)
 
        #return if repo have no revisions
 
        if repo_size < 1:
 
            lock.release()
 
            return True
 

	
 
        skip_date_limit = True
 
        parse_limit = int(config['app_conf'].get('commit_parse_limit'))
 
        last_rev = 0
 
        last_cs = None
 
        timegetter = itemgetter('time')
 

	
 
        sa = get_session()
 

	
 
        dbrepo = sa.query(Repository)\
 
            .filter(Repository.repo_name == repo_name).scalar()
 
        cur_stats = sa.query(Statistics)\
 
            .filter(Statistics.repository == dbrepo).scalar()
 

	
 
        if cur_stats is not None:
 
            last_rev = cur_stats.stat_on_revision
 

	
 
        if last_rev == repo.get_changeset().revision and repo_size > 1:
 
            #pass silently without any work if we're not on first revision or
 
            #current state of parsing revision(from db marker) is the
 
            #last revision
 
            lock.release()
 
            return True
 

	
 
        if cur_stats:
 
            commits_by_day_aggregate = OrderedDict(json.loads(
 
                                        cur_stats.commit_activity_combined))
 
            co_day_auth_aggr = json.loads(cur_stats.commit_activity)
 

	
 
        log.debug('starting parsing %s', parse_limit)
 
        lmktime = mktime
 

	
 
        last_rev = last_rev + 1 if last_rev > 0 else last_rev
 

	
 
        for cs in repo[last_rev:last_rev + parse_limit]:
 
            last_cs = cs  # remember last parsed changeset
 
            k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
 
                          cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
 

	
 
            if akc(cs.author) in co_day_auth_aggr:
 
                try:
 
                    l = [timegetter(x) for x in
 
                         co_day_auth_aggr[akc(cs.author)]['data']]
 
                    time_pos = l.index(k)
 
                except ValueError:
 
                    time_pos = False
 

	
 
                if time_pos >= 0 and time_pos is not False:
 

	
 
                    datadict = \
 
                        co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
 

	
 
                    datadict["commits"] += 1
 
                    datadict["added"] += len(cs.added)
 
                    datadict["changed"] += len(cs.changed)
 
                    datadict["removed"] += len(cs.removed)
 

	
 
                else:
 
                    if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
 

	
 
                        datadict = {"time": k,
 
                                    "commits": 1,
 
                                    "added": len(cs.added),
 
                                    "changed": len(cs.changed),
 
                                    "removed": len(cs.removed),
 
                                   }
 
                        co_day_auth_aggr[akc(cs.author)]['data']\
 
                            .append(datadict)
 

	
 
            else:
 
                if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
 
                    co_day_auth_aggr[akc(cs.author)] = {
 
                                        "label": akc(cs.author),
 
                                        "data": [{"time":k,
 
                                                 "commits":1,
 
                                                 "added":len(cs.added),
 
                                                 "changed":len(cs.changed),
 
                                                 "removed":len(cs.removed),
 
                                                 }],
 
                                        "schema": ["commits"],
 
                                        }
 

	
 
            #gather all data by day
 
            if k in commits_by_day_aggregate:
 
                commits_by_day_aggregate[k] += 1
 
            else:
 
                commits_by_day_aggregate[k] = 1
 

	
 
        overview_data = sorted(commits_by_day_aggregate.items(),
 
                               key=itemgetter(0))
 

	
 
        if not co_day_auth_aggr:
 
            co_day_auth_aggr[akc(repo.contact)] = {
 
                "label": akc(repo.contact),
 
                "data": [0, 1],
 
                "schema": ["commits"],
 
            }
 

	
 
        stats = cur_stats if cur_stats else Statistics()
 
        stats.commit_activity = json.dumps(co_day_auth_aggr)
 
        stats.commit_activity_combined = json.dumps(overview_data)
 

	
 
        log.debug('last revison %s', last_rev)
 
        leftovers = len(repo.revisions[last_rev:])
 
        log.debug('revisions to parse %s', leftovers)
 

	
 
        if last_rev == 0 or leftovers < parse_limit:
 
            log.debug('getting code trending stats')
 
            stats.languages = json.dumps(__get_codes_stats(repo_name))
 

	
 
        try:
 
            stats.repository = dbrepo
 
            stats.stat_on_revision = last_cs.revision if last_cs else 0
 
            sa.add(stats)
 
            sa.commit()
 
        except:
 
            log.error(traceback.format_exc())
 
            sa.rollback()
 
            lock.release()
 
            return False
 

	
 
        #final release
 
        lock.release()
 

	
 
        #execute another task if celery is enabled
 
        if len(repo.revisions) > 1 and CELERY_ON:
 
            run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
 
        return True
 
    except LockHeld:
 
        log.info('LockHeld')
 
        return 'Task with key %s already running' % lockkey
 

	
 
@task(ignore_result=True)
 
def send_password_link(user_email):
 
    try:
 
        log = reset_user_password.get_logger()
 
    except:
 
        log = logging.getLogger(__name__)
 

	
 
    from rhodecode.lib import auth
 
    from rhodecode.model.db import User
 

	
 
    try:
 
        sa = get_session()
 
        user = sa.query(User).filter(User.email == user_email).scalar()
 

	
 
        if user:
 
            link = url('reset_password_confirmation', key=user.api_key,
 
                       qualified=True)
 
            tmpl = """
 
Hello %s
 

	
 
We received a request to create a new password for your account.
 

	
 
You can generate it by clicking following URL:
 

	
 
%s
 

	
 
If you didn't request new password please ignore this email.
 
            """
 
            run_task(send_email, user_email,
 
                     "RhodeCode password reset link",
 
                     tmpl % (user.short_contact, link))
 
            log.info('send new password mail to %s', user_email)
 

	
 
    except:
 
        log.error('Failed to update user password')
 
        log.error(traceback.format_exc())
 
        return False
 

	
 
    return True
 

	
 
@task(ignore_result=True)
 
def reset_user_password(user_email):
 
    try:
 
        log = reset_user_password.get_logger()
 
    except:
 
        log = logging.getLogger(__name__)
 

	
 
    from rhodecode.lib import auth
 
    from rhodecode.model.db import User
 

	
 
    try:
 
        try:
 
            sa = get_session()
 
            user = sa.query(User).filter(User.email == user_email).scalar()
 
            new_passwd = auth.PasswordGenerator().gen_password(8,
 
                             auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
 
            if user:
 
                user.password = auth.get_crypt_password(new_passwd)
 
                user.api_key = auth.generate_api_key(user.username)
 
                sa.add(user)
 
                sa.commit()
 
                log.info('change password for %s', user_email)
 
            if new_passwd is None:
 
                raise Exception('unable to generate new password')
 

	
 
        except:
 
            log.error(traceback.format_exc())
 
            sa.rollback()
 

	
 
        run_task(send_email, user_email,
 
                 "Your new RhodeCode password",
 
                 'Your new RhodeCode password:%s' % (new_passwd))
 
        log.info('send new password mail to %s', user_email)
 

	
 
    except:
 
        log.error('Failed to update user password')
 
        log.error(traceback.format_exc())
 

	
 
    return True
 

	
 

	
 
@task(ignore_result=True)
 
def send_email(recipients, subject, body):
 
    """
 
    Sends an email with defined parameters from the .ini files.
 

	
 
    :param recipients: list of recipients, it this is empty the defined email
 
        address from field 'email_to' is used instead
 
    :param subject: subject of the mail
 
    :param body: body of the mail
 
    """
 
    try:
 
        log = send_email.get_logger()
 
    except:
 
        log = logging.getLogger(__name__)
 

	
 
    email_config = config
 

	
 
    if not recipients:
 
        recipients = [email_config.get('email_to')]
 

	
 
    mail_from = email_config.get('app_email_from')
 
    user = email_config.get('smtp_username')
 
    passwd = email_config.get('smtp_password')
 
    mail_server = email_config.get('smtp_server')
 
    mail_port = email_config.get('smtp_port')
 
    tls = str2bool(email_config.get('smtp_use_tls'))
 
    ssl = str2bool(email_config.get('smtp_use_ssl'))
 
    debug = str2bool(config.get('debug'))
 

	
 
    try:
 
        m = SmtpMailer(mail_from, user, passwd, mail_server,
 
                       mail_port, ssl, tls, debug=debug)
 
        m.send(recipients, subject, body)
 
    except:
 
        log.error('Mail sending failed')
 
        log.error(traceback.format_exc())
 
        return False
 
    return True
 

	
 

	
 
@task(ignore_result=True)
 
def create_repo_fork(form_data, cur_user):
 
    from rhodecode.model.repo import RepoModel
 
    from vcs import get_backend
 

	
 
    try:
 
        log = create_repo_fork.get_logger()
 
    except:
 
        log = logging.getLogger(__name__)
 

	
 
    repo_model = RepoModel(get_session())
 
    repo_model.create(form_data, cur_user, just_db=True, fork=True)
 
    repo_name = form_data['repo_name']
 
    repos_path = get_repos_path()
 
    repo_path = os.path.join(repos_path, repo_name)
 
    repo_fork_path = os.path.join(repos_path, form_data['fork_name'])
 
    alias = form_data['repo_type']
 

	
 
    log.info('creating repo fork %s as %s', repo_name, repo_path)
 
    backend = get_backend(alias)
 
    backend(str(repo_fork_path), create=True, src_url=str(repo_path))
 

	
 

	
 
def __get_codes_stats(repo_name):
 
    repos_path = get_repos_path()
 
    repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
 
    tip = repo.get_changeset()
 
    code_stats = {}
 

	
 
    def aggregate(cs):
 
        for f in cs[2]:
 
            ext = lower(f.extension)
 
            if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
 
                if ext in code_stats:
 
                    code_stats[ext] += 1
 
                else:
 
                    code_stats[ext] = 1
 

	
 
    map(aggregate, tip.walk('/'))
 

	
 
    return code_stats or {}
rhodecode/lib/compat.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.compat
 
    ~~~~~~~~~~~~~~~~~~~~
 

	
 
    Python backward compatibility functions and common libs
 
    
 
    
 
    :created_on: Oct 7, 2011
 
    :author: marcink
 
    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>    
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
from rhodecode import __platform__, PLATFORM_WIN
 

	
 
#==============================================================================
 
# json
 
#==============================================================================
 
try:
 
    import json
 
except ImportError:
 
    import simplejson as json
 

	
 

	
 
#==============================================================================
 
# izip_longest
 
#==============================================================================
 
try:
 
    from itertools import izip_longest
 
except ImportError:
 
    import itertools
 

	
 
    def izip_longest(*args, **kwds): # noqa
 
        fillvalue = kwds.get("fillvalue")
 

	
 
        def sentinel(counter=([fillvalue] * (len(args) - 1)).pop):
 
            yield counter() # yields the fillvalue, or raises IndexError
 

	
 
        fillers = itertools.repeat(fillvalue)
 
        iters = [itertools.chain(it, sentinel(), fillers)
 
                    for it in args]
 
        try:
 
            for tup in itertools.izip(*iters):
 
                yield tup
 
        except IndexError:
 
            pass
 

	
 

	
 
#==============================================================================
 
# OrderedDict
 
#==============================================================================
 

	
 
# Python Software Foundation License
 

	
 
# XXX: it feels like using the class with "is" and "is not" instead of "==" and
 
# "!=" should be faster.
 
class _Nil(object):
 

	
 
    def __repr__(self):
 
        return "nil"
 

	
 
    def __eq__(self, other):
 
        if (isinstance(other, _Nil)):
 
            return True
 
        else:
 
            return NotImplemented
 

	
 
    def __ne__(self, other):
 
        if (isinstance(other, _Nil)):
 
            return False
 
        else:
 
            return NotImplemented
 

	
 
_nil = _Nil()
 

	
 
class _odict(object):
 
    """Ordered dict data structure, with O(1) complexity for dict operations
 
    that modify one element.
 
    
 
    Overwriting values doesn't change their original sequential order.
 
    """
 

	
 
    def _dict_impl(self):
 
        return None
 

	
 
    def __init__(self, data=(), **kwds):
 
        """This doesn't accept keyword initialization as normal dicts to avoid
 
        a trap - inside a function or method the keyword args are accessible
 
        only as a dict, without a defined order, so their original order is
 
        lost.
 
        """
 
        if kwds:
 
            raise TypeError("__init__() of ordered dict takes no keyword "
 
                            "arguments to avoid an ordering trap.")
 
        self._dict_impl().__init__(self)
 
        # If you give a normal dict, then the order of elements is undefined
 
        if hasattr(data, "iteritems"):
 
            for key, val in data.iteritems():
 
                self[key] = val
 
        else:
 
            for key, val in data:
 
                self[key] = val
 

	
 
    # Double-linked list header
 
    def _get_lh(self):
 
        dict_impl = self._dict_impl()
 
        if not hasattr(self, '_lh'):
 
            dict_impl.__setattr__(self, '_lh', _nil)
 
        return dict_impl.__getattribute__(self, '_lh')
 

	
 
    def _set_lh(self, val):
 
        self._dict_impl().__setattr__(self, '_lh', val)
 

	
 
    lh = property(_get_lh, _set_lh)
 

	
 
    # Double-linked list tail
 
    def _get_lt(self):
 
        dict_impl = self._dict_impl()
 
        if not hasattr(self, '_lt'):
 
            dict_impl.__setattr__(self, '_lt', _nil)
 
        return dict_impl.__getattribute__(self, '_lt')
 

	
 
    def _set_lt(self, val):
 
        self._dict_impl().__setattr__(self, '_lt', val)
 

	
 
    lt = property(_get_lt, _set_lt)
 

	
 
    def __getitem__(self, key):
 
        return self._dict_impl().__getitem__(self, key)[1]
 

	
 
    def __setitem__(self, key, val):
 
        dict_impl = self._dict_impl()
 
        try:
 
            dict_impl.__getitem__(self, key)[1] = val
 
        except KeyError, e:
 
            new = [dict_impl.__getattribute__(self, 'lt'), val, _nil]
 
            dict_impl.__setitem__(self, key, new)
 
            if dict_impl.__getattribute__(self, 'lt') == _nil:
 
                dict_impl.__setattr__(self, 'lh', key)
 
            else:
 
                dict_impl.__getitem__(
 
                    self, dict_impl.__getattribute__(self, 'lt'))[2] = key
 
            dict_impl.__setattr__(self, 'lt', key)
 

	
 
    def __delitem__(self, key):
 
        dict_impl = self._dict_impl()
 
        pred, _ , succ = self._dict_impl().__getitem__(self, key)
 
        if pred == _nil:
 
            dict_impl.__setattr__(self, 'lh', succ)
 
        else:
 
            dict_impl.__getitem__(self, pred)[2] = succ
 
        if succ == _nil:
 
            dict_impl.__setattr__(self, 'lt', pred)
 
        else:
 
            dict_impl.__getitem__(self, succ)[0] = pred
 
        dict_impl.__delitem__(self, key)
 

	
 
    def __contains__(self, key):
 
        return key in self.keys()
 

	
 
    def __len__(self):
 
        return len(self.keys())
 

	
 
    def __str__(self):
 
        pairs = ("%r: %r" % (k, v) for k, v in self.iteritems())
 
        return "{%s}" % ", ".join(pairs)
 

	
 
    def __repr__(self):
 
        if self:
 
            pairs = ("(%r, %r)" % (k, v) for k, v in self.iteritems())
 
            return "odict([%s])" % ", ".join(pairs)
 
        else:
 
            return "odict()"
 

	
 
    def get(self, k, x=None):
 
        if k in self:
 
            return self._dict_impl().__getitem__(self, k)[1]
 
        else:
 
            return x
 

	
 
    def __iter__(self):
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lh')
 
        while curr_key != _nil:
 
            yield curr_key
 
            curr_key = dict_impl.__getitem__(self, curr_key)[2]
 

	
 
    iterkeys = __iter__
 

	
 
    def keys(self):
 
        return list(self.iterkeys())
 

	
 
    def itervalues(self):
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lh')
 
        while curr_key != _nil:
 
            _, val, curr_key = dict_impl.__getitem__(self, curr_key)
 
            yield val
 

	
 
    def values(self):
 
        return list(self.itervalues())
 

	
 
    def iteritems(self):
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lh')
 
        while curr_key != _nil:
 
            _, val, next_key = dict_impl.__getitem__(self, curr_key)
 
            yield curr_key, val
 
            curr_key = next_key
 

	
 
    def items(self):
 
        return list(self.iteritems())
 

	
 
    def sort(self, cmp=None, key=None, reverse=False):
 
        items = [(k, v) for k, v in self.items()]
 
        if cmp is not None:
 
            items = sorted(items, cmp=cmp)
 
        elif key is not None:
 
            items = sorted(items, key=key)
 
        else:
 
            items = sorted(items, key=lambda x: x[1])
 
        if reverse:
 
            items.reverse()
 
        self.clear()
 
        self.__init__(items)
 

	
 
    def clear(self):
 
        dict_impl = self._dict_impl()
 
        dict_impl.clear(self)
 
        dict_impl.__setattr__(self, 'lh', _nil)
 
        dict_impl.__setattr__(self, 'lt', _nil)
 

	
 
    def copy(self):
 
        return self.__class__(self)
 

	
 
    def update(self, data=(), **kwds):
 
        if kwds:
 
            raise TypeError("update() of ordered dict takes no keyword "
 
                            "arguments to avoid an ordering trap.")
 
        if hasattr(data, "iteritems"):
 
            data = data.iteritems()
 
        for key, val in data:
 
            self[key] = val
 

	
 
    def setdefault(self, k, x=None):
 
        try:
 
            return self[k]
 
        except KeyError:
 
            self[k] = x
 
            return x
 

	
 
    def pop(self, k, x=_nil):
 
        try:
 
            val = self[k]
 
            del self[k]
 
            return val
 
        except KeyError:
 
            if x == _nil:
 
                raise
 
            return x
 

	
 
    def popitem(self):
 
        try:
 
            dict_impl = self._dict_impl()
 
            key = dict_impl.__getattribute__(self, 'lt')
 
            return key, self.pop(key)
 
        except KeyError:
 
            raise KeyError("'popitem(): ordered dictionary is empty'")
 

	
 
    def riterkeys(self):
 
        """To iterate on keys in reversed order.
 
        """
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lt')
 
        while curr_key != _nil:
 
            yield curr_key
 
            curr_key = dict_impl.__getitem__(self, curr_key)[0]
 

	
 
    __reversed__ = riterkeys
 

	
 
    def rkeys(self):
 
        """List of the keys in reversed order.
 
        """
 
        return list(self.riterkeys())
 

	
 
    def ritervalues(self):
 
        """To iterate on values in reversed order.
 
        """
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lt')
 
        while curr_key != _nil:
 
            curr_key, val, _ = dict_impl.__getitem__(self, curr_key)
 
            yield val
 

	
 
    def rvalues(self):
 
        """List of the values in reversed order.
 
        """
 
        return list(self.ritervalues())
 

	
 
    def riteritems(self):
 
        """To iterate on (key, value) in reversed order.
 
        """
 
        dict_impl = self._dict_impl()
 
        curr_key = dict_impl.__getattribute__(self, 'lt')
 
        while curr_key != _nil:
 
            pred_key, val, _ = dict_impl.__getitem__(self, curr_key)
 
            yield curr_key, val
 
            curr_key = pred_key
 

	
 
    def ritems(self):
 
        """List of the (key, value) in reversed order.
 
        """
 
        return list(self.riteritems())
 

	
 
    def firstkey(self):
 
        if self:
 
            return self._dict_impl().__getattribute__(self, 'lh')
 
        else:
 
            raise KeyError("'firstkey(): ordered dictionary is empty'")
 

	
 
    def lastkey(self):
 
        if self:
 
            return self._dict_impl().__getattribute__(self, 'lt')
 
        else:
 
            raise KeyError("'lastkey(): ordered dictionary is empty'")
 

	
 
    def as_dict(self):
 
        return self._dict_impl()(self.items())
 

	
 
    def _repr(self):
 
        """_repr(): low level repr of the whole data contained in the odict.
 
        Useful for debugging.
 
        """
 
        dict_impl = self._dict_impl()
 
        form = "odict low level repr lh,lt,data: %r, %r, %s"
 
        return form % (dict_impl.__getattribute__(self, 'lh'),
 
                       dict_impl.__getattribute__(self, 'lt'),
 
                       dict_impl.__repr__(self))
 

	
 
class OrderedDict(_odict, dict):
 

	
 
    def _dict_impl(self):
 
        return dict
 

	
 

	
 
#==============================================================================
 
# OrderedSet
 
#==============================================================================
 
from sqlalchemy.util import OrderedSet
 

	
 

	
 
#==============================================================================
 
# kill FUNCTIONS
 
#==============================================================================
 
if __platform__ in PLATFORM_WIN:
 
    import ctypes
 

	
 
    def kill(pid, sig):
 
        """kill function for Win32"""
 
        kernel32 = ctypes.windll.kernel32
 
        handle = kernel32.OpenProcess(1, 0, pid)
 
        return (0 != kernel32.TerminateProcess(handle, 0))
 

	
 
else:
 
    kill = os.kill
rhodecode/lib/helpers.py
Show inline comments
 
@@ -217,457 +217,456 @@ def pygmentize_annotation(repo_name, fil
 
            i = int(h * 6.0) # XXX assume int() truncates!
 
            f = (h * 6.0) - i
 
            p = v * (1.0 - s)
 
            q = v * (1.0 - s * f)
 
            t = v * (1.0 - s * (1.0 - f))
 
            i = i % 6
 
            if i == 0: return v, t, p
 
            if i == 1: return q, v, p
 
            if i == 2: return p, v, t
 
            if i == 3: return p, q, v
 
            if i == 4: return t, p, v
 
            if i == 5: return v, p, q
 

	
 
        golden_ratio = 0.618033988749895
 
        h = 0.22717784590367374
 

	
 
        for _ in xrange(n):
 
            h += golden_ratio
 
            h %= 1
 
            HSV_tuple = [h, 0.95, 0.95]
 
            RGB_tuple = hsv_to_rgb(*HSV_tuple)
 
            yield map(lambda x:str(int(x * 256)), RGB_tuple)
 

	
 
    cgenerator = gen_color()
 

	
 
    def get_color_string(cs):
 
        if color_dict.has_key(cs):
 
            col = color_dict[cs]
 
        else:
 
            col = color_dict[cs] = cgenerator.next()
 
        return "color: rgb(%s)! important;" % (', '.join(col))
 

	
 
    def url_func(repo_name):
 

	
 
        def _url_func(changeset):
 
            author = changeset.author
 
            date = changeset.date
 
            message = tooltip(changeset.message)
 

	
 
            tooltip_html = ("<div style='font-size:0.8em'><b>Author:</b>"
 
                            " %s<br/><b>Date:</b> %s</b><br/><b>Message:"
 
                            "</b> %s<br/></div>")
 

	
 
            tooltip_html = tooltip_html % (author, date, message)
 
            lnk_format = '%5s:%s' % ('r%s' % changeset.revision,
 
                                     short_id(changeset.raw_id))
 
            uri = link_to(
 
                    lnk_format,
 
                    url('changeset_home', repo_name=repo_name,
 
                        revision=changeset.raw_id),
 
                    style=get_color_string(changeset.raw_id),
 
                    class_='tooltip',
 
                    title=tooltip_html
 
                  )
 

	
 
            uri += '\n'
 
            return uri
 
        return _url_func
 

	
 
    return literal(annotate_highlight(filenode, url_func(repo_name), **kwargs))
 

	
 
def is_following_repo(repo_name, user_id):
 
    from rhodecode.model.scm import ScmModel
 
    return ScmModel().is_following_repo(repo_name, user_id)
 

	
 
flash = _Flash()
 

	
 
#==============================================================================
 
# SCM FILTERS available via h.
 
#==============================================================================
 
from vcs.utils import author_name, author_email
 
from rhodecode.lib import credentials_filter, age as _age
 

	
 
age = lambda  x:_age(x)
 
capitalize = lambda x: x.capitalize()
 
email = author_email
 
email_or_none = lambda x: email(x) if email(x) != x else None
 
person = lambda x: author_name(x)
 
short_id = lambda x: x[:12]
 
hide_credentials = lambda x: ''.join(credentials_filter(x))
 

	
 
def bool2icon(value):
 
    """Returns True/False values represented as small html image of true/false
 
    icons
 

	
 
    :param value: bool value
 
    """
 

	
 
    if value is True:
 
        return HTML.tag('img', src=url("/images/icons/accept.png"),
 
                        alt=_('True'))
 

	
 
    if value is False:
 
        return HTML.tag('img', src=url("/images/icons/cancel.png"),
 
                        alt=_('False'))
 

	
 
    return value
 

	
 

	
 
def action_parser(user_log, feed=False):
 
    """This helper will action_map the specified string action into translated
 
    fancy names with icons and links
 

	
 
    :param user_log: user log instance
 
    :param feed: use output for feeds (no html and fancy icons)
 
    """
 

	
 
    action = user_log.action
 
    action_params = ' '
 

	
 
    x = action.split(':')
 

	
 
    if len(x) > 1:
 
        action, action_params = x
 

	
 
    def get_cs_links():
 
        revs_limit = 3 #display this amount always
 
        revs_top_limit = 50 #show upto this amount of changesets hidden
 
        revs = action_params.split(',')
 
        repo_name = user_log.repository.repo_name
 

	
 
        from rhodecode.model.scm import ScmModel
 
        repo = user_log.repository.scm_instance
 

	
 
        message = lambda rev: get_changeset_safe(repo, rev).message
 
        cs_links = []
 
        cs_links.append(" " + ', '.join ([link_to(rev,
 
                url('changeset_home',
 
                repo_name=repo_name,
 
                revision=rev), title=tooltip(message(rev)),
 
                class_='tooltip') for rev in revs[:revs_limit] ]))
 

	
 
        compare_view = (' <div class="compare_view tooltip" title="%s">'
 
                        '<a href="%s">%s</a> '
 
                        '</div>' % (_('Show all combined changesets %s->%s' \
 
                                      % (revs[0], revs[-1])),
 
                                    url('changeset_home', repo_name=repo_name,
 
                                        revision='%s...%s' % (revs[0], revs[-1])
 
                                    ),
 
                                    _('compare view'))
 
                        )
 

	
 
        if len(revs) > revs_limit:
 
            uniq_id = revs[0]
 
            html_tmpl = ('<span> %s '
 
            '<a class="show_more" id="_%s" href="#more">%s</a> '
 
            '%s</span>')
 
            if not feed:
 
                cs_links.append(html_tmpl % (_('and'), uniq_id, _('%s more') \
 
                                        % (len(revs) - revs_limit),
 
                                        _('revisions')))
 

	
 
            if not feed:
 
                html_tmpl = '<span id="%s" style="display:none"> %s </span>'
 
            else:
 
                html_tmpl = '<span id="%s"> %s </span>'
 

	
 
            cs_links.append(html_tmpl % (uniq_id, ', '.join([link_to(rev,
 
                url('changeset_home',
 
                repo_name=repo_name, revision=rev),
 
                title=message(rev), class_='tooltip')
 
                for rev in revs[revs_limit:revs_top_limit]])))
 
        if len(revs) > 1:
 
            cs_links.append(compare_view)
 
        return ''.join(cs_links)
 

	
 
    def get_fork_name():
 
        repo_name = action_params
 
        return _('fork name ') + str(link_to(action_params, url('summary_home',
 
                                          repo_name=repo_name,)))
 

	
 
    action_map = {'user_deleted_repo':(_('[deleted] repository'), None),
 
           'user_created_repo':(_('[created] repository'), None),
 
           'user_forked_repo':(_('[forked] repository'), get_fork_name),
 
           'user_updated_repo':(_('[updated] repository'), None),
 
           'admin_deleted_repo':(_('[delete] repository'), None),
 
           'admin_created_repo':(_('[created] repository'), None),
 
           'admin_forked_repo':(_('[forked] repository'), None),
 
           'admin_updated_repo':(_('[updated] repository'), None),
 
           'push':(_('[pushed] into'), get_cs_links),
 
           'push_local':(_('[committed via RhodeCode] into'), get_cs_links),
 
           'push_remote':(_('[pulled from remote] into'), get_cs_links),
 
           'pull':(_('[pulled] from'), None),
 
           'started_following_repo':(_('[started following] repository'), None),
 
           'stopped_following_repo':(_('[stopped following] repository'), None),
 
            }
 

	
 
    action_str = action_map.get(action, action)
 
    if feed:
 
        action = action_str[0].replace('[', '').replace(']', '')
 
    else:
 
        action = action_str[0].replace('[', '<span class="journal_highlight">')\
 
                   .replace(']', '</span>')
 

	
 
    action_params_func = lambda :""
 

	
 
    if callable(action_str[1]):
 
        action_params_func = action_str[1]
 

	
 
    return [literal(action), action_params_func]
 

	
 
def action_parser_icon(user_log):
 
    action = user_log.action
 
    action_params = None
 
    x = action.split(':')
 

	
 
    if len(x) > 1:
 
        action, action_params = x
 

	
 
    tmpl = """<img src="%s%s" alt="%s"/>"""
 
    map = {'user_deleted_repo':'database_delete.png',
 
           'user_created_repo':'database_add.png',
 
           'user_forked_repo':'arrow_divide.png',
 
           'user_updated_repo':'database_edit.png',
 
           'admin_deleted_repo':'database_delete.png',
 
           'admin_created_repo':'database_add.png',
 
           'admin_forked_repo':'arrow_divide.png',
 
           'admin_updated_repo':'database_edit.png',
 
           'push':'script_add.png',
 
           'push_local':'script_edit.png',
 
           'push_remote':'connect.png',
 
           'pull':'down_16.png',
 
           'started_following_repo':'heart_add.png',
 
           'stopped_following_repo':'heart_delete.png',
 
            }
 
    return literal(tmpl % ((url('/images/icons/')),
 
                           map.get(action, action), action))
 

	
 

	
 
#==============================================================================
 
# PERMS
 
#==============================================================================
 
from rhodecode.lib.auth import HasPermissionAny, HasPermissionAll, \
 
HasRepoPermissionAny, HasRepoPermissionAll
 

	
 
#==============================================================================
 
# GRAVATAR URL
 
#==============================================================================
 

	
 
def gravatar_url(email_address, size=30):
 
    if not str2bool(config['app_conf'].get('use_gravatar')) or \
 
        email_address == 'anonymous@rhodecode.org':
 
        return url("/images/user%s.png" % size)
 

	
 
    ssl_enabled = 'https' == request.environ.get('wsgi.url_scheme')
 
    default = 'identicon'
 
    baseurl_nossl = "http://www.gravatar.com/avatar/"
 
    baseurl_ssl = "https://secure.gravatar.com/avatar/"
 
    baseurl = baseurl_ssl if ssl_enabled else baseurl_nossl
 

	
 
    if isinstance(email_address, unicode):
 
        #hashlib crashes on unicode items
 
        email_address = safe_str(email_address)
 
    # construct the url
 
    gravatar_url = baseurl + hashlib.md5(email_address.lower()).hexdigest() + "?"
 
    gravatar_url += urllib.urlencode({'d':default, 's':str(size)})
 

	
 
    return gravatar_url
 

	
 

	
 
#==============================================================================
 
# REPO PAGER, PAGER FOR REPOSITORY
 
#==============================================================================
 
class RepoPage(Page):
 

	
 
    def __init__(self, collection, page=1, items_per_page=20,
 
        item_count=None, url=None, branch_name=None, **kwargs):
 

	
 
        """Create a "RepoPage" instance. special pager for paging
 
        repository
 
        """
 
        self._url_generator = url
 

	
 
        # Safe the kwargs class-wide so they can be used in the pager() method
 
        self.kwargs = kwargs
 

	
 
        # Save a reference to the collection
 
        self.original_collection = collection
 

	
 
        self.collection = collection
 

	
 
        # The self.page is the number of the current page.
 
        # The first page has the number 1!
 
        try:
 
            self.page = int(page) # make it int() if we get it as a string
 
        except (ValueError, TypeError):
 
            self.page = 1
 

	
 
        self.items_per_page = items_per_page
 

	
 
        # Unless the user tells us how many items the collections has
 
        # we calculate that ourselves.
 
        if item_count is not None:
 
            self.item_count = item_count
 
        else:
 
            self.item_count = len(self.collection)
 

	
 
        # Compute the number of the first and last available page
 
        if self.item_count > 0:
 
            self.first_page = 1
 
            self.page_count = int(math.ceil(float(self.item_count) /
 
                                            self.items_per_page))
 
            self.last_page = self.first_page + self.page_count - 1
 

	
 
            # Make sure that the requested page number is the range of valid pages
 
            if self.page > self.last_page:
 
                self.page = self.last_page
 
            elif self.page < self.first_page:
 
                self.page = self.first_page
 

	
 
            # Note: the number of items on this page can be less than
 
            #       items_per_page if the last page is not full
 
            self.first_item = max(0, (self.item_count) - (self.page *
 
                                                          items_per_page))
 
            self.last_item = ((self.item_count - 1) - items_per_page *
 
                              (self.page - 1))
 

	
 
            iterator = self.collection.get_changesets(start=self.first_item,
 
                                                      end=self.last_item,
 
                                                      reverse=True,
 
                                                      branch_name=branch_name)
 
            self.items = list(iterator)
 

	
 
            # Links to previous and next page
 
            if self.page > self.first_page:
 
                self.previous_page = self.page - 1
 
            else:
 
                self.previous_page = None
 

	
 
            if self.page < self.last_page:
 
                self.next_page = self.page + 1
 
            else:
 
                self.next_page = None
 

	
 
        # No items available
 
        else:
 
            self.first_page = None
 
            self.page_count = 0
 
            self.last_page = None
 
            self.first_item = None
 
            self.last_item = None
 
            self.previous_page = None
 
            self.next_page = None
 
            self.items = []
 

	
 
        # This is a subclass of the 'list' type. Initialise the list now.
 
        list.__init__(self, self.items)
 

	
 

	
 
def changed_tooltip(nodes):
 
    """
 
    Generates a html string for changed nodes in changeset page.
 
    It limits the output to 30 entries
 
    
 
    :param nodes: LazyNodesGenerator
 
    """
 
    if nodes:
 
        pref = ': <br/> '
 
        suf = ''
 
        if len(nodes) > 30:
 
            suf = '<br/>' + _(' and %s more') % (len(nodes) - 30)
 
        return literal(pref + '<br/> '.join([safe_unicode(x.path)
 
                                             for x in nodes[:30]]) + suf)
 
    else:
 
        return ': ' + _('No Files')
 

	
 

	
 

	
 
def repo_link(groups_and_repos):
 
    """
 
    Makes a breadcrumbs link to repo within a group
 
    joins &raquo; on each group to create a fancy link
 
    
 
    ex::
 
        group >> subgroup >> repo
 
    
 
    :param groups_and_repos:
 
    """
 
    groups, repo_name = groups_and_repos
 

	
 
    if not groups:
 
        return repo_name
 
    else:
 
        def make_link(group):
 
            return link_to(group.group_name, url('repos_group',
 
                                                 id=group.group_id))
 
            return link_to(group.name, url('repos_group_home',
 
                                           group_name=group.group_name))
 
        return literal(' &raquo; '.join(map(make_link, groups)) + \
 
                       " &raquo; " + repo_name)
 

	
 

	
 
def fancy_file_stats(stats):
 
    """
 
    Displays a fancy two colored bar for number of added/deleted
 
    lines of code on file
 
    
 
    :param stats: two element list of added/deleted lines of code
 
    """
 

	
 
    a, d, t = stats[0], stats[1], stats[0] + stats[1]
 
    width = 100
 
    unit = float(width) / (t or 1)
 

	
 
    # needs > 9% of width to be visible or 0 to be hidden
 
    a_p = max(9, unit * a) if a > 0 else 0
 
    d_p = max(9, unit * d) if d > 0 else 0
 
    p_sum = a_p + d_p
 

	
 
    if p_sum > width:
 
        #adjust the percentage to be == 100% since we adjusted to 9
 
        if a_p > d_p:
 
            a_p = a_p - (p_sum - width)
 
        else:
 
            d_p = d_p - (p_sum - width)
 

	
 
    a_v = a if a > 0 else ''
 
    d_v = d if d > 0 else ''
 

	
 

	
 
    def cgen(l_type):
 
        mapping = {'tr':'top-right-rounded-corner',
 
                   'tl':'top-left-rounded-corner',
 
                   'br':'bottom-right-rounded-corner',
 
                   'bl':'bottom-left-rounded-corner'}
 
        map_getter = lambda x:mapping[x]
 

	
 
        if l_type == 'a' and d_v:
 
            #case when added and deleted are present
 
            return ' '.join(map(map_getter, ['tl', 'bl']))
 

	
 
        if l_type == 'a' and not d_v:
 
            return ' '.join(map(map_getter, ['tr', 'br', 'tl', 'bl']))
 

	
 
        if l_type == 'd' and a_v:
 
            return ' '.join(map(map_getter, ['tr', 'br']))
 

	
 
        if l_type == 'd' and not a_v:
 
            return ' '.join(map(map_getter, ['tr', 'br', 'tl', 'bl']))
 

	
 

	
 

	
 
    d_a = '<div class="added %s" style="width:%s%%">%s</div>' % (cgen('a'),
 
                                                                 a_p, a_v)
 
    d_d = '<div class="deleted %s" style="width:%s%%">%s</div>' % (cgen('d'),
 
                                                                   d_p, d_v)
 
    return literal('<div style="width:%spx">%s%s</div>' % (width, d_a, d_d))
 

	
 

	
 
def urlify_text(text):
 
    import re
 

	
 
    url_pat = re.compile(r'(http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+)')
 

	
 
    def url_func(match_obj):
 
        url_full = match_obj.groups()[0]
 
        return '<a href="%(url)s">%(url)s</a>' % ({'url':url_full})
 

	
 
    return literal(url_pat.sub(url_func, text))
rhodecode/lib/indexers/__init__.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.indexers.__init__
 
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

	
 
    Whoosh indexing module for RhodeCode
 

	
 
    :created_on: Aug 17, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
import os
 
import sys
 
import traceback
 
from os.path import dirname as dn, join as jn
 

	
 
#to get the rhodecode import
 
sys.path.append(dn(dn(dn(os.path.realpath(__file__)))))
 

	
 
from string import strip
 
from shutil import rmtree
 

	
 
from whoosh.analysis import RegexTokenizer, LowercaseFilter, StopFilter
 
from whoosh.fields import TEXT, ID, STORED, Schema, FieldType
 
from whoosh.index import create_in, open_dir
 
from whoosh.formats import Characters
 
from whoosh.highlight import highlight, SimpleFragmenter, HtmlFormatter
 

	
 
from webhelpers.html.builder import escape
 
from sqlalchemy import engine_from_config
 
from vcs.utils.lazy import LazyProperty
 

	
 
from rhodecode.model import init_model
 
from rhodecode.model.scm import ScmModel
 
from rhodecode.model.repo import RepoModel
 
from rhodecode.config.environment import load_environment
 
from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP
 
from rhodecode.lib.utils import BasePasterCommand, Command, add_cache
 

	
 
#EXTENSIONS WE WANT TO INDEX CONTENT OFF
 
INDEX_EXTENSIONS = LANGUAGES_EXTENSIONS_MAP.keys()
 

	
 
#CUSTOM ANALYZER wordsplit + lowercase filter
 
ANALYZER = RegexTokenizer(expression=r"\w+") | LowercaseFilter()
 

	
 

	
 
#INDEX SCHEMA DEFINITION
 
SCHEMA = Schema(owner=TEXT(),
 
                repository=TEXT(stored=True),
 
                path=TEXT(stored=True),
 
                content=FieldType(format=Characters(ANALYZER),
 
                             scorable=True, stored=True),
 
                modtime=STORED(), extension=TEXT(stored=True))
 

	
 

	
 
IDX_NAME = 'HG_INDEX'
 
FORMATTER = HtmlFormatter('span', between='\n<span class="break">...</span>\n')
 
FRAGMENTER = SimpleFragmenter(200)
 

	
 

	
 
class MakeIndex(BasePasterCommand):
 

	
 
    max_args = 1
 
    min_args = 1
 

	
 
    usage = "CONFIG_FILE"
 
    summary = "Creates index for full text search given configuration file"
 
    group_name = "RhodeCode"
 
    takes_config_file = -1
 
    parser = Command.standard_parser(verbose=True)
 

	
 
    def command(self):
 

	
 
        from pylons import config
 
        add_cache(config)
 
        engine = engine_from_config(config, 'sqlalchemy.db1.')
 
        init_model(engine)
 

	
 
        index_location = config['index_dir']
 
        repo_location = self.options.repo_location \
 
            if self.options.repo_location else RepoModel().repos_path
 
        repo_list = map(strip, self.options.repo_list.split(',')) \
 
            if self.options.repo_list else None
 

	
 
        #======================================================================
 
        # WHOOSH DAEMON
 
        #======================================================================
 
        from rhodecode.lib.pidlock import LockHeld, DaemonLock
 
        from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
 
        try:
 
            l = DaemonLock(file=jn(dn(dn(index_location)), 'make_index.lock'))
 
            l = DaemonLock(file_=jn(dn(dn(index_location)), 'make_index.lock'))
 
            WhooshIndexingDaemon(index_location=index_location,
 
                                 repo_location=repo_location,
 
                                 repo_list=repo_list)\
 
                .run(full_index=self.options.full_index)
 
            l.release()
 
        except LockHeld:
 
            sys.exit(1)
 

	
 
    def update_parser(self):
 
        self.parser.add_option('--repo-location',
 
                          action='store',
 
                          dest='repo_location',
 
                          help="Specifies repositories location to index OPTIONAL",
 
                          )
 
        self.parser.add_option('--index-only',
 
                          action='store',
 
                          dest='repo_list',
 
                          help="Specifies a comma separated list of repositores "
 
                                "to build index on OPTIONAL",
 
                          )
 
        self.parser.add_option('-f',
 
                          action='store_true',
 
                          dest='full_index',
 
                          help="Specifies that index should be made full i.e"
 
                                " destroy old and build from scratch",
 
                          default=False)
 

	
 
class ResultWrapper(object):
 
    def __init__(self, search_type, searcher, matcher, highlight_items):
 
        self.search_type = search_type
 
        self.searcher = searcher
 
        self.matcher = matcher
 
        self.highlight_items = highlight_items
 
        self.fragment_size = 200 / 2
 

	
 
    @LazyProperty
 
    def doc_ids(self):
 
        docs_id = []
 
        while self.matcher.is_active():
 
            docnum = self.matcher.id()
 
            chunks = [offsets for offsets in self.get_chunks()]
 
            docs_id.append([docnum, chunks])
 
            self.matcher.next()
 
        return docs_id
 

	
 
    def __str__(self):
 
        return '<%s at %s>' % (self.__class__.__name__, len(self.doc_ids))
 

	
 
    def __repr__(self):
 
        return self.__str__()
 

	
 
    def __len__(self):
 
        return len(self.doc_ids)
 

	
 
    def __iter__(self):
 
        """
 
        Allows Iteration over results,and lazy generate content
 

	
 
        *Requires* implementation of ``__getitem__`` method.
 
        """
 
        for docid in self.doc_ids:
 
            yield self.get_full_content(docid)
 

	
 
    def __getitem__(self, key):
 
        """
 
        Slicing of resultWrapper
 
        """
 
        i, j = key.start, key.stop
 

	
 
        slice = []
 
        for docid in self.doc_ids[i:j]:
 
            slice.append(self.get_full_content(docid))
 
        return slice
 

	
 

	
 
    def get_full_content(self, docid):
 
        res = self.searcher.stored_fields(docid[0])
 
        f_path = res['path'][res['path'].find(res['repository']) \
 
                             + len(res['repository']):].lstrip('/')
 

	
 
        content_short = self.get_short_content(res, docid[1])
 
        res.update({'content_short':content_short,
 
                    'content_short_hl':self.highlight(content_short),
 
                    'f_path':f_path})
 

	
 
        return res
 

	
 
    def get_short_content(self, res, chunks):
 

	
 
        return ''.join([res['content'][chunk[0]:chunk[1]] for chunk in chunks])
 

	
 
    def get_chunks(self):
 
        """
 
        Smart function that implements chunking the content
 
        but not overlap chunks so it doesn't highlight the same
 
        close occurrences twice.
 
        
 
        :param matcher:
 
        :param size:
 
        """
 
        memory = [(0, 0)]
 
        for span in self.matcher.spans():
 
            start = span.startchar or 0
 
            end = span.endchar or 0
 
            start_offseted = max(0, start - self.fragment_size)
 
            end_offseted = end + self.fragment_size
 

	
 
            if start_offseted < memory[-1][1]:
 
                start_offseted = memory[-1][1]
 
            memory.append((start_offseted, end_offseted,))
 
            yield (start_offseted, end_offseted,)
 

	
 
    def highlight(self, content, top=5):
 
        if self.search_type != 'content':
 
            return ''
 
        hl = highlight(escape(content),
 
                 self.highlight_items,
 
                 analyzer=ANALYZER,
 
                 fragmenter=FRAGMENTER,
 
                 formatter=FORMATTER,
 
                 top=top)
 
        return hl
rhodecode/lib/pidlock.py
Show inline comments
 
import os
 
import sys
 
import time
 
import errno
 

	
 
from warnings import warn
 
from multiprocessing.util import Finalize
 

	
 
from rhodecode import __platform__, PLATFORM_WIN
 

	
 
if __platform__ in PLATFORM_WIN:
 
    import ctypes
 

	
 
    def kill(pid, sig):
 
        """kill function for Win32"""
 
        kernel32 = ctypes.windll.kernel32
 
        handle = kernel32.OpenProcess(1, 0, pid)
 
        return (0 != kernel32.TerminateProcess(handle, 0))
 

	
 
else:
 
    kill = os.kill
 

	
 
from rhodecode.lib.compat import kill
 

	
 
class LockHeld(Exception):
 
    pass
 

	
 

	
 
class DaemonLock(object):
 
    """daemon locking
 
    USAGE:
 
    try:
 
        l = DaemonLock(desc='test lock')
 
        l = DaemonLock(file_='/path/tolockfile',desc='test lock')
 
        main()
 
        l.release()
 
    except LockHeld:
 
        sys.exit(1)
 
    """
 

	
 
    def __init__(self, file=None, callbackfn=None,
 
    def __init__(self, file_=None, callbackfn=None,
 
                 desc='daemon lock', debug=False):
 

	
 
        self.pidfile = file if file else os.path.join(
 
        self.pidfile = file_ if file_ else os.path.join(
 
                                                    os.path.dirname(__file__),
 
                                                    'running.lock')
 
        self.callbackfn = callbackfn
 
        self.desc = desc
 
        self.debug = debug
 
        self.held = False
 
        #run the lock automatically !
 
        self.lock()
 
        self._finalize = Finalize(self, DaemonLock._on_finalize,
 
                                    args=(self, debug), exitpriority=10)
 

	
 
    @staticmethod
 
    def _on_finalize(lock, debug):
 
        if lock.held:
 
            if debug:
 
                print 'leck held finilazing and running lock.release()'
 
            lock.release()
 

	
 
    def lock(self):
 
        """
 
        locking function, if lock is present it
 
        will raise LockHeld exception
 
        """
 
        lockname = '%s' % (os.getpid())
 
        if self.debug:
 
            print 'running lock'
 
        self.trylock()
 
        self.makelock(lockname, self.pidfile)
 
        return True
 

	
 
    def trylock(self):
 
        running_pid = False
 
        if self.debug:
 
            print 'checking for already running process'
 
        try:
 
            pidfile = open(self.pidfile, "r")
 
            pidfile.seek(0)
 
            running_pid = int(pidfile.readline())
 

	
 
            pidfile.close()
 

	
 
            if self.debug:
 
                print ('lock file present running_pid: %s, '
 
                       'checking for execution') % running_pid
 
            # Now we check the PID from lock file matches to the current
 
            # process PID
 
            if running_pid:
 
                try:
 
                    kill(running_pid, 0)
 
                except OSError, exc:
 
                    if exc.errno in (errno.ESRCH, errno.EPERM):
 
                        print ("Lock File is there but"
 
                               " the program is not running")
 
                        print "Removing lock file for the: %s" % running_pid
 
                        self.release()
 
                    else:
 
                        raise
 
                else:
 
                    print "You already have an instance of the program running"
 
                    print "It is running as process %s" % running_pid
 
                    raise LockHeld()
 

	
 
        except IOError, e:
 
            if e.errno != 2:
 
                raise
 

	
 
    def release(self):
 
        """releases the pid by removing the pidfile
 
        """
 
        if self.debug:
 
            print 'trying to release the pidlock'
 

	
 
        if self.callbackfn:
 
            #execute callback function on release
 
            if self.debug:
 
                print 'executing callback function %s' % self.callbackfn
 
            self.callbackfn()
 
        try:
 
            if self.debug:
 
                print 'removing pidfile %s' % self.pidfile
 
            os.remove(self.pidfile)
 
            self.held = False
 
        except OSError, e:
 
            if self.debug:
 
                print 'removing pidfile failed %s' % e
 
            pass
 

	
 
    def makelock(self, lockname, pidfile):
 
        """
 
        this function will make an actual lock
 

	
 
        :param lockname: acctual pid of file
 
        :param pidfile: the file to write the pid in
 
        """
 
        if self.debug:
 
            print 'creating a file %s and pid: %s' % (pidfile, lockname)
 
        pidfile = open(self.pidfile, "wb")
 
        pidfile.write(lockname)
 
        pidfile.close
 
        self.held = True
rhodecode/lib/utils.py
Show inline comments
 
# -*- coding: utf-8 -*-
 
"""
 
    rhodecode.lib.utils
 
    ~~~~~~~~~~~~~~~~~~~
 

	
 
    Utilities library for RhodeCode
 

	
 
    :created_on: Apr 18, 2010
 
    :author: marcink
 
    :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
 
    :license: GPLv3, see COPYING for more details.
 
"""
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by
 
# the Free Software Foundation, either version 3 of the License, or
 
# (at your option) any later version.
 
#
 
# This program is distributed in the hope that it will be useful,
 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 
# GNU General Public License for more details.
 
#
 
# You should have received a copy of the GNU General Public License
 
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

	
 
import os
 
import logging
 
import datetime
 
import traceback
 
import paste
 
import beaker
 
from os.path import dirname as dn, join as jn
 

	
 
from paste.script.command import Command, BadCommand
 

	
 
from mercurial import ui, config
 

	
 
from webhelpers.text import collapse, remove_formatting, strip_tags
 

	
 
from vcs import get_backend
 
from vcs.backends.base import BaseChangeset
 
from vcs.utils.lazy import LazyProperty
 
from vcs.utils.helpers import get_scm
 
from vcs.exceptions import VCSError
 

	
 
from rhodecode.model import meta
 
from rhodecode.model.caching_query import FromCache
 
from rhodecode.model.db import Repository, User, RhodeCodeUi, UserLog, Group, \
 
    RhodeCodeSettings
 
from rhodecode.model.repo import RepoModel
 

	
 
log = logging.getLogger(__name__)
 

	
 

	
 
def recursive_replace(str, replace=' '):
 
    """Recursive replace of given sign to just one instance
 

	
 
    :param str: given string
 
    :param replace: char to find and replace multiple instances
 

	
 
    Examples::
 
    >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
 
    'Mighty-Mighty-Bo-sstones'
 
    """
 

	
 
    if str.find(replace * 2) == -1:
 
        return str
 
    else:
 
        str = str.replace(replace * 2, replace)
 
        return recursive_replace(str, replace)
 

	
 

	
 
def repo_name_slug(value):
 
    """Return slug of name of repository
 
    This function is called on each creation/modification
 
    of repository to prevent bad names in repo
 
    """
 

	
 
    slug = remove_formatting(value)
 
    slug = strip_tags(slug)
 

	
 
    for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:
 
        slug = slug.replace(c, '-')
 
    slug = recursive_replace(slug, '-')
 
    slug = collapse(slug, '-')
 
    return slug
 

	
 

	
 
def get_repo_slug(request):
 
    return request.environ['pylons.routes_dict'].get('repo_name')
 

	
 

	
 
def action_logger(user, action, repo, ipaddr='', sa=None):
 
    """
 
    Action logger for various actions made by users
 

	
 
    :param user: user that made this action, can be a unique username string or
 
        object containing user_id attribute
 
    :param action: action to log, should be on of predefined unique actions for
 
        easy translations
 
    :param repo: string name of repository or object containing repo_id,
 
        that action was made on
 
    :param ipaddr: optional ip address from what the action was made
 
    :param sa: optional sqlalchemy session
 

	
 
    """
 

	
 
    if not sa:
 
        sa = meta.Session()
 

	
 
    try:
 
        if hasattr(user, 'user_id'):
 
            user_obj = user
 
        elif isinstance(user, basestring):
 
            user_obj = User.get_by_username(user)
 
        else:
 
            raise Exception('You have to provide user object or username')
 

	
 
        rm = RepoModel()
 
        if hasattr(repo, 'repo_id'):
 
            repo_obj = rm.get(repo.repo_id, cache=False)
 
            repo_name = repo_obj.repo_name
 
        elif  isinstance(repo, basestring):
 
            repo_name = repo.lstrip('/')
 
            repo_obj = rm.get_by_repo_name(repo_name, cache=False)
 
        else:
 
            raise Exception('You have to provide repository to action logger')
 

	
 
        user_log = UserLog()
 
        user_log.user_id = user_obj.user_id
 
        user_log.action = action
 

	
 
        user_log.repository_id = repo_obj.repo_id
 
        user_log.repository_name = repo_name
 

	
 
        user_log.action_date = datetime.datetime.now()
 
        user_log.user_ip = ipaddr
 
        sa.add(user_log)
 
        sa.commit()
 

	
 
        log.info('Adding user %s, action %s on %s', user_obj, action, repo)
 
    except:
 
        log.error(traceback.format_exc())
 
        sa.rollback()
 

	
 

	
 
def get_repos(path, recursive=False):
 
    """
 
    Scans given path for repos and return (name,(type,path)) tuple
 

	
 
    :param path: path to scann for repositories
 
    :param recursive: recursive search and return names with subdirs in front
 
    """
 
    from vcs.utils.helpers import get_scm
 
    from vcs.exceptions import VCSError
 

	
 
    if path.endswith(os.sep):
 
        #remove ending slash for better results
 
        path = path[:-1]
 

	
 
    def _get_repos(p):
 
        if not os.access(p, os.W_OK):
 
            return
 
        for dirpath in os.listdir(p):
 
            if os.path.isfile(os.path.join(p, dirpath)):
 
                continue
 
            cur_path = os.path.join(p, dirpath)
 
            try:
 
                scm_info = get_scm(cur_path)
 
                yield scm_info[1].split(path)[-1].lstrip(os.sep), scm_info
 
            except VCSError:
 
                if not recursive:
 
                    continue
 
                #check if this dir containts other repos for recursive scan
 
                rec_path = os.path.join(p, dirpath)
 
                if os.path.isdir(rec_path):
 
                    for inner_scm in _get_repos(rec_path):
 
                        yield inner_scm
 

	
 
    return _get_repos(path)
 

	
 

	
 
def is_valid_repo(repo_name, base_path):
 
    """
 
    Returns True if given path is a valid repository False otherwise
 
    :param repo_name:
 
    :param base_path:
 

	
 
    :return True: if given path is a valid repository
 
    """
 
    full_path = os.path.join(base_path, repo_name)
 

	
 
    try:
 
        get_scm(full_path)
 
        return True
 
    except VCSError:
 
        return False
 

	
 
def is_valid_repos_group(repos_group_name, base_path):
 
    """
 
    Returns True if given path is a repos group False otherwise
 
    
 
    :param repo_name:
 
    :param base_path:
 
    """
 
    full_path = os.path.join(base_path, repos_group_name)
 

	
 
    # check if it's not a repo
 
    if is_valid_repo(repos_group_name, base_path):
 
        return False
 

	
 
    # check if it's a valid path
 
    if os.path.isdir(full_path):
 
        return True
 

	
 
    return False
 

	
 
def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):
 
    while True:
 
        ok = raw_input(prompt)
 
        if ok in ('y', 'ye', 'yes'):
 
            return True
 
        if ok in ('n', 'no', 'nop', 'nope'):
 
            return False
 
        retries = retries - 1
 
        if retries < 0:
 
            raise IOError
 
        print complaint
 

	
 
#propagated from mercurial documentation
 
ui_sections = ['alias', 'auth',
 
                'decode/encode', 'defaults',
 
                'diff', 'email',
 
                'extensions', 'format',
 
                'merge-patterns', 'merge-tools',
 
                'hooks', 'http_proxy',
 
                'smtp', 'patch',
 
                'paths', 'profiling',
 
                'server', 'trusted',
 
                'ui', 'web', ]
 

	
 

	
 
def make_ui(read_from='file', path=None, checkpaths=True):
 
    """A function that will read python rc files or database
 
    and make an mercurial ui object from read options
 

	
 
    :param path: path to mercurial config file
 
    :param checkpaths: check the path
 
    :param read_from: read from 'file' or 'db'
 
    """
 

	
 
    baseui = ui.ui()
 

	
 
    #clean the baseui object
 
    baseui._ocfg = config.config()
 
    baseui._ucfg = config.config()
 
    baseui._tcfg = config.config()
 

	
 
    if read_from == 'file':
 
        if not os.path.isfile(path):
 
            log.warning('Unable to read config file %s' % path)
 
            return False
 
        log.debug('reading hgrc from %s', path)
 
        cfg = config.config()
 
        cfg.read(path)
 
        for section in ui_sections:
 
            for k, v in cfg.items(section):
 
                log.debug('settings ui from file[%s]%s:%s', section, k, v)
 
                baseui.setconfig(section, k, v)
 

	
 
    elif read_from == 'db':
 
        sa = meta.Session()
 
        ret = sa.query(RhodeCodeUi)\
 
            .options(FromCache("sql_cache_short",
 
                               "get_hg_ui_settings")).all()
 

	
 
        hg_ui = ret
 
        for ui_ in hg_ui:
 
            if ui_.ui_active:
 
                log.debug('settings ui from db[%s]%s:%s', ui_.ui_section,
 
                          ui_.ui_key, ui_.ui_value)
 
                baseui.setconfig(ui_.ui_section, ui_.ui_key, ui_.ui_value)
 

	
 
        meta.Session.remove()
 
    return baseui
 

	
 

	
 
def set_rhodecode_config(config):
 
    """Updates pylons config with new settings from database
 

	
 
    :param config:
 
    """
 
    hgsettings = RhodeCodeSettings.get_app_settings()
 

	
 
    for k, v in hgsettings.items():
 
        config[k] = v
 

	
 

	
 
def invalidate_cache(cache_key, *args):
 
    """Puts cache invalidation task into db for
 
    further global cache invalidation
 
    """
 

	
 
    from rhodecode.model.scm import ScmModel
 

	
 
    if cache_key.startswith('get_repo_cached_'):
 
        name = cache_key.split('get_repo_cached_')[-1]
 
        ScmModel().mark_for_invalidation(name)
 

	
 

	
 
class EmptyChangeset(BaseChangeset):
 
    """
 
    An dummy empty changeset. It's possible to pass hash when creating
 
    an EmptyChangeset
 
    """
 

	
 
    def __init__(self, cs='0' * 40, repo=None, requested_revision=None, alias=None):
 
        self._empty_cs = cs
 
        self.revision = -1
 
        self.message = ''
 
        self.author = ''
 
        self.date = ''
 
        self.repository = repo
 
        self.requested_revision = requested_revision
 
        self.alias = alias
 

	
 
    @LazyProperty
 
    def raw_id(self):
 
        """Returns raw string identifying this changeset, useful for web
 
        representation.
 
        """
 

	
 
        return self._empty_cs
 

	
 
    @LazyProperty
 
    def branch(self):
 
        return get_backend(self.alias).DEFAULT_BRANCH_NAME
 

	
 
    @LazyProperty
 
    def short_id(self):
 
        return self.raw_id[:12]
 

	
 
    def get_file_changeset(self, path):
 
        return self
 

	
 
    def get_file_content(self, path):
 
        return u''
 

	
 
    def get_file_size(self, path):
 
        return 0
 

	
 

	
 
def map_groups(groups):
 
    """Checks for groups existence, and creates groups structures.
 
    It returns last group in structure
 

	
 
    :param groups: list of groups structure
 
    """
 
    sa = meta.Session()
 

	
 
    parent = None
 
    group = None
 
    for lvl, group_name in enumerate(groups[:-1]):
 

	
 
    # last element is repo in nested groups structure
 
    groups = groups[:-1]
 

	
 
    for lvl, group_name in enumerate(groups):
 
        group_name = '/'.join(groups[:lvl] + [group_name])
 
        group = sa.query(Group).filter(Group.group_name == group_name).scalar()
 

	
 
        if group is None:
 
            group = Group(group_name, parent)
 
            sa.add(group)
 
            sa.commit()
 

	
 
        parent = group
 

	
 
    return group
 

	
 

	
 
def repo2db_mapper(initial_repo_list, remove_obsolete=False):
 
    """maps all repos given in initial_repo_list, non existing repositories
 
    are created, if remove_obsolete is True it also check for db entries
 
    that are not in initial_repo_list and removes them.
 

	
 
    :param initial_repo_list: list of repositories found by scanning methods
 
    :param remove_obsolete: check for obsolete entries in database
 
    """
 

	
 
    sa = meta.Session()
 
    rm = RepoModel()
 
    user = sa.query(User).filter(User.admin == True).first()
 
    added = []
 
    # fixup groups paths to new format on the fly
 
    # TODO: remove this in future
 
    for g in Group.query().all():
 
        g.group_name = g.get_new_name(g.name)
 
        sa.add(g)    
 
    for name, repo in initial_repo_list.items():
 
        group = map_groups(name.split(os.sep))
 
        group = map_groups(name.split(Repository.url_sep()))
 
        if not rm.get_by_repo_name(name, cache=False):
 
            log.info('repository %s not found creating default', name)
 
            added.append(name)
 
            form_data = {
 
                         'repo_name': name,
 
                         'repo_name_full': name,
 
                         'repo_type': repo.alias,
 
                         'description': repo.description \
 
                            if repo.description != 'unknown' else \
 
                                        '%s repository' % name,
 
                         'private': False,
 
                         'group_id': getattr(group, 'group_id', None)
 
                         }
 
            rm.create(form_data, user, just_db=True)
 

	
 
    removed = []
 
    if remove_obsolete:
 
        #remove from database those repositories that are not in the filesystem
 
        for repo in sa.query(Repository).all():
 
            if repo.repo_name not in initial_repo_list.keys():
 
                removed.append(repo.repo_name)
 
                sa.delete(repo)
 
                sa.commit()
 

	
 
    return added, removed
 

	
 
#set cache regions for beaker so celery can utilise it
 
def add_cache(settings):
 
    cache_settings = {'regions': None}
 
    for key in settings.keys():
 
        for prefix in ['beaker.cache.', 'cache.']:
 
            if key.startswith(prefix):
 
                name = key.split(prefix)[1].strip()
 
                cache_settings[name] = settings[key].strip()
 
    if cache_settings['regions']:
 
        for region in cache_settings['regions'].split(','):
 
            region = region.strip()
 
            region_settings = {}
 
            for key, value in cache_settings.items():
 
                if key.startswith(region):
 
                    region_settings[key.split('.')[1]] = value
 
            region_settings['expire'] = int(region_settings.get('expire',
 
                                                                60))
 
            region_settings.setdefault('lock_dir',
 
                                       cache_settings.get('lock_dir'))
 
            region_settings.setdefault('data_dir',
 
                                       cache_settings.get('data_dir'))
 

	
 
            if 'type' not in region_settings:
 
                region_settings['type'] = cache_settings.get('type',
 
                                                             'memory')
 
            beaker.cache.cache_regions[region] = region_settings
 

	
 

	
 
def get_current_revision():
 
    """Returns tuple of (number, id) from repository containing this package
 
    or None if repository could not be found.
 
    """
 

	
 
    try:
 
        from vcs import get_repo
 
        from vcs.utils.helpers import get_scm
 
        from vcs.exceptions import RepositoryError, VCSError
 
        repopath = os.path.join(os.path.dirname(__file__), '..', '..')
 
        scm = get_scm(repopath)[0]
 
        repo = get_repo(path=repopath, alias=scm)
 
        tip = repo.get_changeset()
 
        return (tip.revision, tip.short_id)
 
    except (ImportError, RepositoryError, VCSError), err:
 
        logging.debug("Cannot retrieve rhodecode's revision. Original error "
 
                      "was: %s" % err)
 
        return None
 

	
 

	
 
#==============================================================================
 
# TEST FUNCTIONS AND CREATORS
 
#==============================================================================
 
def create_test_index(repo_location, config, full_index):
 
    """
 
    Makes default test index
 
    
 
    :param config: test config
 
    :param full_index:
 
    """
 

	
 
    from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
 
    from rhodecode.lib.pidlock import DaemonLock, LockHeld
 

	
 
    repo_location = repo_location
 

	
 
    index_location = os.path.join(config['app_conf']['index_dir'])
 
    if not os.path.exists(index_location):
 
        os.makedirs(index_location)
 

	
 
    try:
 
        l = DaemonLock(file=jn(dn(index_location), 'make_index.lock'))
 
        l = DaemonLock(file_=jn(dn(index_location), 'make_index.lock'))
 
        WhooshIndexingDaemon(index_location=index_location,
 
                             repo_location=repo_location)\
 
            .run(full_index=full_index)
 
        l.release()
 
    except LockHeld:
 
        pass
 

	
 

	
 
def create_test_env(repos_test_path, config):
 
    """Makes a fresh database and
 
    install test repository into tmp dir
 
    """
 
    from rhodecode.lib.db_manage import DbManage
 
    from rhodecode.tests import HG_REPO, GIT_REPO, NEW_HG_REPO, NEW_GIT_REPO, \
 
        HG_FORK, GIT_FORK, TESTS_TMP_PATH
 
    import tarfile
 
    import shutil
 
    from os.path import abspath
 

	
 
    # PART ONE create db
 
    dbconf = config['sqlalchemy.db1.url']
 
    log.debug('making test db %s', dbconf)
 

	
 
    # create test dir if it doesn't exist
 
    if not os.path.isdir(repos_test_path):
 
        log.debug('Creating testdir %s' % repos_test_path)
 
        os.makedirs(repos_test_path)
 

	
 
    dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=config['here'],
 
                        tests=True)
 
    dbmanage.create_tables(override=True)
 
    dbmanage.create_settings(dbmanage.config_prompt(repos_test_path))
 
    dbmanage.create_default_user()
 
    dbmanage.admin_prompt()
 
    dbmanage.create_permissions()
 
    dbmanage.populate_default_permissions()
 

	
 
    # PART TWO make test repo
 
    log.debug('making test vcs repositories')
 

	
 
    idx_path = config['app_conf']['index_dir']
 
    data_path = config['app_conf']['cache_dir']
 

	
 
    #clean index and data
 
    if idx_path and os.path.exists(idx_path):
 
        log.debug('remove %s' % idx_path)
 
        shutil.rmtree(idx_path)
 

	
 
    if data_path and os.path.exists(data_path):
 
        log.debug('remove %s' % data_path)
 
        shutil.rmtree(data_path)
 

	
 
    #CREATE DEFAULT HG REPOSITORY
 
    cur_dir = dn(dn(abspath(__file__)))
 
    tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test_hg.tar.gz"))
 
    tar.extractall(jn(TESTS_TMP_PATH, HG_REPO))
 
    tar.close()
 

	
 

	
 
#==============================================================================
 
# PASTER COMMANDS
 
#==============================================================================
 
class BasePasterCommand(Command):
 
    """
 
    Abstract Base Class for paster commands.
 

	
 
    The celery commands are somewhat aggressive about loading
 
    celery.conf, and since our module sets the `CELERY_LOADER`
 
    environment variable to our loader, we have to bootstrap a bit and
 
    make sure we've had a chance to load the pylons config off of the
 
    command line, otherwise everything fails.
 
    """
 
    min_args = 1
 
    min_args_error = "Please provide a paster config file as an argument."
 
    takes_config_file = 1
 
    requires_config_file = True
 

	
 
    def notify_msg(self, msg, log=False):
 
        """Make a notification to user, additionally if logger is passed
 
        it logs this action using given logger
 

	
 
        :param msg: message that will be printed to user
 
        :param log: logging instance, to use to additionally log this message
 

	
 
        """
 
        if log and isinstance(log, logging):
 
            log(msg)
 

	
 
    def run(self, args):
 
        """
 
        Overrides Command.run
 

	
 
        Checks for a config file argument and loads it.
 
        """
 
        if len(args) < self.min_args:
 
            raise BadCommand(
 
                self.min_args_error % {'min_args': self.min_args,
 
                                       'actual_args': len(args)})
 

	
 
        # Decrement because we're going to lob off the first argument.
 
        # @@ This is hacky
 
        self.min_args -= 1
 
        self.bootstrap_config(args[0])
 
        self.update_parser()
 
        return super(BasePasterCommand, self).run(args[1:])
 

	
 
    def update_parser(self):
 
        """
 
        Abstract method.  Allows for the class's parser to be updated
 
        before the superclass's `run` method is called.  Necessary to
 
        allow options/arguments to be passed through to the underlying
 
        celery command.
 
        """
 
        raise NotImplementedError("Abstract Method.")
 

	
 
    def bootstrap_config(self, conf):
 
        """
 
        Loads the pylons configuration.
 
        """
 
        from pylons import config as pylonsconfig
 

	
 
        path_to_ini_file = os.path.realpath(conf)
 
        conf = paste.deploy.appconfig('config:' + path_to_ini_file)
 
        pylonsconfig.init_app(conf.global_conf, conf.local_conf)
 

	

Changeset was too big and was cut off... Show full diff anyway

0 comments (0 inline, 0 general)