kallithea Files · scripts/make-release

Files @ b66725ba01ed

Branch filter:

Location: kallithea/scripts/make-release

b66725ba01ed 2.6 KiB text/plain Show Annotation Show as Raw Download as Raw

Thomas De Schampheleire

cli: add command 'kallithea-cli front-end-build'

Kallithea is under the GPL license, and we can thus only distribute any
generated code if we also ship the corresponding source.

We are moving towards a web front-end that use npm to download and compile
various open source components. The components might not be GPL, but if we
distribute any parts of their code (compiled or converted to other
representation), then we also must distribute the corresponding source under
the GPL.

It doesn't seem feasible for us to distribute the source of everything that
npm downloads and includes when we are building. It thus also doesn't seem
feasible for us to build and ship the compiled (possibly minified) front-end
code. Instead, we have to make it as smooth as possible for our users to
get up and running.

It doesn't seem feasible for us to ship or install npm. We must assume it is
available. That requirement must be documented clearly, and we must recommend
how to install npm for the most common platforms.

We could perhaps just document what manual steps to run. Kallithea doesn't
work out of the box anyway - it has to be configured and initialized. Extra
steps might not be a big problem.

Another approach is to call out to npm while pip is installing Kallithea and
download the requirements and build the files. It can be done by customizing
setuptools commands in setup.py. But: Python packaging is fragile. Even
though we only support pip, it really isn't built for things like this.
Custom output is muted and buffered and only shown if running with -v or the
command fails. And pip and setup.py can be used to build and install in so
many ways that we probably can't make it work reliably with all ways of
installing Kallithea.

The approach implemented by this commit is to add a custom cli command
'front-end-build' to run the required commands. This single user-facing
command can internally run various steps as needed. The only current
requirement is the presence of npm and an internet connection.

For now, this will just create/update style.css ... but currently probably
without any actual changes. The files created by npm (and the node_modules
directory) must *not* be a part of the release package made with 'setup.py
sdist'.

(Commit message is mostly written by Mads Kiilerich)

#!/bin/bash
set -e
set -x

cleanup()
{
  echo "Removing venv $venv"
  rm  -rf "$venv"
}

echo "Checking that you are NOT inside a virtualenv"
[ -z "$VIRTUAL_ENV" ]

venv=$(mktemp -d --tmpdir kallithea-release-XXXXX)
trap cleanup EXIT

echo "Setting up a fresh virtualenv in $venv"
virtualenv -p python2 "$venv"
. "$venv/bin/activate"

echo "Install/verify tools needed for building and uploading stuff"
pip install --upgrade -e .
pip install --upgrade -r dev_requirements.txt Sphinx Sphinx-PyPI-upload

echo "Cleanup and update copyrights ... and clean checkout"
scripts/run-all-cleanup
scripts/update-copyrights.py
hg up -cr .

echo "Make release build from clean checkout in build/"
rm -rf build dist
hg archive build
cd build

echo "Check MANIFEST.in"
sed -e 's/[^ ]*[ ]*\([^ ]*\).*/\1/g' MANIFEST.in | grep -v '^node_modules/bootstrap\|^kallithea/public/css/style.css' | xargs ls -lad

echo "Build dist"
python2 setup.py compile_catalog
python2 setup.py sdist

echo "Verify VERSION from kallithea/__init__.py"
namerel=$(cd dist && echo Kallithea-*.tar.gz)
namerel=${namerel%.tar.gz}
version=${namerel#Kallithea-}
ls -l $(pwd)/dist/$namerel.tar.gz
echo "Releasing Kallithea $version in directory $namerel"

echo "Verify dist file content"
diff -u <((hg mani | grep -v '^\.hg') | LANG=C sort) <(tar tf dist/Kallithea-$version.tar.gz | sed "s|^$namerel/||" | grep . | grep -v '^kallithea/i18n/.*/LC_MESSAGES/kallithea.mo$\|^Kallithea.egg-info/\|^PKG-INFO$\|/$' | LANG=C sort)
! tar tf dist/Kallithea-$version.tar.gz | grep "$namerel/node_modules/bootstrap/\$"

echo "Verify docs build"
python2 setup.py build_sphinx # not used yet ... but we want to make sure it builds

cat - << EOT

Now, make sure
* all tests are passing
* release note is ready
* announcement is ready
* source has been pushed to https://kallithea-scm.org/repos/kallithea

EOT

echo "Verify current revision is tagged for $version"
hg log -r "'$version'&." | grep .

echo -n "Enter \"pypi\" to upload Kallithea $version to pypi: "
read answer
[ "$answer" = "pypi" ]

echo "Upload docs to pypi"
# See https://wiki.python.org/moin/PyPiDocumentationHosting
python2 setup.py build_sphinx upload_sphinx
xdg-open http://packages.python.org/Kallithea/installation.html

echo "Rebuild readthedocs for docs.kallithea-scm.org"
xdg-open https://readthedocs.org/projects/kallithea/
curl -X POST http://readthedocs.org/build/kallithea
xdg-open https://readthedocs.org/builds/kallithea/
xdg-open http://docs.kallithea-scm.org/en/latest/ # or whatever the branch is

extraargs=${EMAIL:+--identity=$EMAIL}
python2 setup.py sdist upload --sign $extraargs
xdg-open https://pypi.python.org/pypi/Kallithea