kallithea Changeset - 193138922d56

Changeset - 193138922d56

Parent rev.

Child rev.

[Not reviewed]

default

0 2 0

Mads Kiilerich - 6 years ago 2020-01-25 20:18:59
mads@kiilerich.com

Grafted from: 614f8bed60e9

celery: introduce make_app instead of creating app at import time

It is dirty to instantiate things at import time (unless it really is basic
singletons).

In 0.5.1 (and earlier), such dirtyness made partial test execution fail when
other things had global side effects and things didn't use the usual import
order:

$ py.test kallithea/lib/
collecting ...
――― kallithea/lib/celerypylons/__init__.py ―――
kallithea/lib/celerypylons/__init__.py:58: in <module>
app.config_from_object(celery_config(tg.config))
kallithea/lib/celerypylons/__init__.py:28: in celery_config
assert config['celery.imports'] == 'kallithea.lib.celerylib.tasks', 'Kallithea Celery configuration has not been loaded'
data/env/lib/python2.7/site-packages/tg/configuration/tgconfig.py:31: in __getitem__
return self.config_proxy.current_conf()[key]
E KeyError: 'celery.imports'

Avoid that by running a "factory" function when the celery app actually is
needed.

2 files changed with 6 insertions and 3 deletions:

kallithea/bin/kallithea_cli_celery.py

kallithea/lib/celerypylons/__init__.py

0 comments (0 inline, 0 general)

kallithea/bin/kallithea_cli_celery.py

➞

Show inline comments

 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
+#
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 import click
 import kallithea
 import kallithea.bin.kallithea_cli_base as cli_base
 from kallithea.lib import celerypylons
 @cli_base.register_command(config_file_initialize_app=True)
 @click.argument('celery_args', nargs=-1)
 def celery_run(celery_args):
     """Start Celery worker(s) for asynchronous tasks.
     This commands starts the Celery daemon which will spawn workers to handle
     certain asynchronous tasks for Kallithea.
     Any extra arguments you pass to this command will be passed through to
     Celery. Use '--' before such extra arguments to avoid options to be parsed
     by this CLI command.
     """
     if not kallithea.CELERY_ON:
         raise Exception('Please set use_celery = true in .ini config '
                         'file before running this command')
     from kallithea.lib import celerypylons
     cmd = celerypylons.worker.worker(celerypylons.app)
     app = celerypylons.make_app()
     cmd = celerypylons.worker.worker(app)
     return cmd.run_from_argv(None, command='celery-run -c CONFIG_FILE --', argv=list(celery_args))

kallithea/lib/celerypylons/__init__.py

➞

Show inline comments

@@ @@ -32,27 +32,29 @@ def celery_config(config): @@
     celery_config = CeleryConfig()
     PREFIXES = """ADMINS BROKER CASSANDRA CELERYBEAT CELERYD CELERYMON CELERY EMAIL SERVER""".split()
     LIST_PARAMS = """CELERY_IMPORTS ADMINS ROUTES CELERY_ACCEPT_CONTENT""".split()
     for config_key, config_value in sorted(config.items()):
         celery_key = config_key.replace('.', '_').upper()
         if celery_key.split('_', 1)[0] not in PREFIXES:
             continue
         if not isinstance(config_value, basestring):
             continue
         if celery_key in LIST_PARAMS:
             celery_value = config_value.split()
         elif config_value.isdigit():
             celery_value = int(config_value)
         elif config_value.lower() in ['true', 'false']:
             celery_value = config_value.lower() == 'true'
         else:
             celery_value = config_value
         setattr(celery_config, celery_key, celery_value)
     return celery_config
 # Create celery app from the TurboGears configuration file
 def make_app():
     """Create celery app from the TurboGears configuration file"""
 app = celery.Celery()
 app.config_from_object(celery_config(tg.config))
     return app

0 comments (0 inline, 0 general)