# HG changeset patch # User Paul Boddie # Date 1485447381 -3600 # Node ID b91f5920af7c98eda5a65eed5947fb2a416b6420 # Parent 8e9c052f4e5fe6aa7f465e79b68e19619e18748b Configure storage modules according to those present in the stores package, similar to the way that scheduling modules are configured, introducing a new tool to update the configuration when new modules are added. diff -r 8e9c052f4e5f -r b91f5920af7c docs/wiki/Administration --- a/docs/wiki/Administration Tue Dec 20 23:33:05 2016 +0100 +++ b/docs/wiki/Administration Thu Jan 26 17:16:21 2017 +0100 @@ -132,6 +132,23 @@ operating system distribution. The `tools/install.sh` script runs the above tool as part of the installation process. +== Adding Storage Modules == + +Storage modules reside in the `imiptools.stores` package. Extra modules can be +installed in this package by adding files or directories to the `stores` +directory within the software installation. + +After adding modules, a tool must be run to register the new modules: + +{{{ +tools/update_storage_modules.py +}}} + +It is envisaged that the installation of additional storage modules and the +use of this tool will be performed by the packaging system provided by an +operating system distribution. The `tools/install.sh` script runs the above +tool as part of the installation process. + == Copying Stores and Changing Store Types == A rudimentary tool is provided that can copy data between stores, even those of diff -r 8e9c052f4e5f -r b91f5920af7c imiptools/stores/__init__.py --- a/imiptools/stores/__init__.py Tue Dec 20 23:33:05 2016 +0100 +++ b/imiptools/stores/__init__.py Thu Jan 26 17:16:21 2017 +0100 @@ -3,7 +3,7 @@ """ General support for calendar data storage. -Copyright (C) 2016 Paul Boddie +Copyright (C) 2016, 2017 Paul Boddie This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -19,15 +19,7 @@ this program. If not, see . """ -from imiptools.stores import file -from imiptools.stores.database import stores as database_stores - -# Build a catalogue of store types. - -stores = { - "file" : file, - } -stores.update(database_stores) +from imiptools.stores.manifest import stores # Access functions. @@ -35,7 +27,7 @@ return stores[store_type].Store(store_dir) def get_publisher(publishing_dir): - return file.Publisher(publishing_dir) + return stores["file"].Publisher(publishing_dir) def get_journal(store_type, journal_dir): return stores[store_type].Journal(journal_dir) diff -r 8e9c052f4e5f -r b91f5920af7c imiptools/stores/database/__init__.py --- a/imiptools/stores/database/__init__.py Tue Dec 20 23:33:05 2016 +0100 +++ b/imiptools/stores/database/__init__.py Thu Jan 26 17:16:21 2017 +0100 @@ -1,28 +0,0 @@ -#!/usr/bin/env python - -""" -General support for database stores. - -Copyright (C) 2016 Paul Boddie - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; either version 3 of the License, or (at your option) any later -version. - -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more -details. - -You should have received a copy of the GNU General Public License along with -this program. If not, see . -""" - -from imiptools.stores.database import postgresql - -stores = { - "postgresql" : postgresql, - } - -# vim: tabstop=4 expandtab shiftwidth=4 diff -r 8e9c052f4e5f -r b91f5920af7c tools/install.sh --- a/tools/install.sh Tue Dec 20 23:33:05 2016 +0100 +++ b/tools/install.sh Thu Jan 26 17:16:21 2017 +0100 @@ -3,7 +3,7 @@ # This tool installs the imip-agent software and message resources. It is # configured by the contents of the config.sh script. # -# Copyright (C) 2014, 2015, 2016 Paul Boddie +# Copyright (C) 2014, 2015, 2016, 2017 Paul Boddie # # This program is free software; you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software @@ -122,7 +122,8 @@ # Tools TOOLS="copy_store.py fix.sh init.sh init_user.sh make_freebusy.py set_delegates.py "\ -"set_quota_groups.py set_quota_limits.py update_quotas.py update_scheduling_modules.py" +"set_quota_groups.py set_quota_limits.py update_quotas.py update_scheduling_modules.py "\ +"update_storage_modules.py" if [ ! -e "$INSTALL_DIR/tools" ]; then mkdir -p "$INSTALL_DIR/tools" @@ -170,3 +171,7 @@ # Run the scheduling module update tool to regenerate the manifest module. PYTHONPATH="$INSTALL_DIR" "$INSTALL_DIR/tools/update_scheduling_modules.py" + +# Run the storage module update tool to regenerate the manifest module. + +PYTHONPATH="$INSTALL_DIR" "$INSTALL_DIR/tools/update_storage_modules.py" diff -r 8e9c052f4e5f -r b91f5920af7c tools/update_storage_modules.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/update_storage_modules.py Thu Jan 26 17:16:21 2017 +0100 @@ -0,0 +1,86 @@ +#!/usr/bin/env python + +""" +Update the storage modules import manifest. + +Copyright (C) 2016, 2017 Paul Boddie + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation; either version 3 of the License, or (at your option) any later +version. + +This program is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. See the GNU General Public License for more +details. + +You should have received a copy of the GNU General Public License along with +this program. If not, see . +""" + +from glob import glob +from os import listdir +from os.path import commonprefix, isdir, join, split, splitext +import imiptools.stores + +reserved = ["__init__.py", "common.py", "manifest.py"] + +def get_extensions(dirname): + filenames = [] + for filename in glob(join(dirname, "*.py")): + leafname = split(filename)[-1] + if leafname not in reserved: + filenames.append(filename) + return filenames + +# The main program generating a new version of the manifest module. + +if __name__ == "__main__": + dirname = join(split(imiptools.stores.__file__)[0], "") + manifest = join(dirname, "manifest.py") + + # Get all Python files in the stores directory, filtering out the + # reserved files that do not provide storage functions. + + filenames = get_extensions(dirname) + + # Get all extensions from directories in the stores directory. + + for filename in listdir(dirname): + filename = join(dirname, filename) + if isdir(filename): + filenames += get_extensions(filename) + + # Open the manifest module and write code to import and combine the + # functions from each module. + + f = open(manifest, "w") + try: + print >>f, """\ +stores = {} +""" + + for filename in filenames: + relative = filename[len(commonprefix([filename, dirname])):] + + # NOTE: Converting POSIX paths to module paths. + + module = splitext(relative)[0].replace("/", ".") + module_parts = module.rsplit(".", 1) + + # Get subpackage location and module. + + module_parents = len(module_parts) > 1 and module_parts[0] + module_name = module_parts[-1] + + print >>f, """\ +from imiptools.stores%s import %s +stores[%r] = %s +""" % (module_parents and ".%s" % module_parents or "", + module_name, module_name, module_name) + + finally: + f.close() + +# vim: tabstop=4 expandtab shiftwidth=4