# HG changeset patch # User Paul Boddie # Date 1454959095 -3600 # Node ID 1fc5f34543dc6d1784e7176cf2fa4baeef59df51 # Parent a55598b6f053ab244c97a4f8336e3dec240125bf Added support for quota expiry so that past events do not consume quotas. diff -r a55598b6f053 -r 1fc5f34543dc conf/cron/cron.daily/imip-agent --- a/conf/cron/cron.daily/imip-agent Mon Feb 08 18:46:31 2016 +0100 +++ b/conf/cron/cron.daily/imip-agent Mon Feb 08 20:18:15 2016 +0100 @@ -4,3 +4,4 @@ export PYTHONPATH="$INSTALL_DIR" "$INSTALL_DIR/tools/make_freebusy.py" all -n -s +"$INSTALL_DIR/tools/update_quotas.py" all -s diff -r a55598b6f053 -r 1fc5f34543dc docs/wiki/CronIntegration --- a/docs/wiki/CronIntegration Mon Feb 08 18:46:31 2016 +0100 +++ b/docs/wiki/CronIntegration Mon Feb 08 20:18:15 2016 +0100 @@ -1,5 +1,22 @@ = Cron Task Scheduler Integration = +The `conf/cron/cron.daily/imip-agent` file contains commands that update +the following: + + * Free/busy collections for all known users + * Quota records for all known quota groups + +This file should be copied to the appropriate destination. For example: + +{{{ +cp conf/cron/cron.daily/imip-agent /etc/cron.daily/ +}}} + +Where frequency-specific directories are not supported by cron on a system, a +`crontab` entry of the appropriate format is required instead. + +== Event Recurrences == + The periods defined by recurring events are not all recorded in a user's free/busy collection if such events recur indefinitely. Instead, only the periods within a certain window of time are recorded for such events. As a @@ -7,17 +24,6 @@ time to include periods that were ignored when previously recording free/busy information for an event. -The `conf/cron/cron.daily/imip-agent` file contains commands that update -free/busy collections for all known users, and this should be copied to the -appropriate destination. For example: - -{{{ -cp conf/cron/cron.daily/imip-agent /etc/cron.daily/ -}}} - -Where frequency-specific directories are not supported by cron on a system, a -`crontab` entry of the appropriate format is required instead. - See the [[../EventRecurrences|guide to event recurrences]] for more information on how recurring events are supported. @@ -25,3 +31,18 @@ `tools/make_freebusy.py` program, which is a general tool that can also reset the free/busy records defined for a user or those made available to a user. + +== Quota Journals == + +As events are confirmed for resources, where quotas on resources have been +imposed, such quotas will be consumed until eventually exhausted, thus +preventing future reservations. By expiring records of past events, quotas +can effectively be replenished, allowing reservations to be made for future +events. + +See the [[../Resources|resources guide]] for more information in imposing +quotas on groups of resources. + +Responsibility for updating the quota records lies with the +`tools/update_quotas.py` program, which can be used manually to update quota +information for the indicated quota groups. diff -r a55598b6f053 -r 1fc5f34543dc imip_store.py --- a/imip_store.py Mon Feb 08 18:46:31 2016 +0100 +++ b/imip_store.py Mon Feb 08 20:18:15 2016 +0100 @@ -961,6 +961,24 @@ def __init__(self, store_dir=None): FileBase.__init__(self, store_dir or JOURNAL_DIR) + # Quota and user identity/group discovery. + + def get_quotas(self): + + "Return a list of quotas." + + return listdir(self.store_dir) + + def get_quota_users(self, quota): + + "Return a list of quota users." + + filename = self.get_object_in_store(quota, "journal") + if not filename or not isdir(filename): + return [] + + return listdir(filename) + # Groups of users sharing quotas. def get_groups(self, quota): diff -r a55598b6f053 -r 1fc5f34543dc tools/install.sh --- a/tools/install.sh Mon Feb 08 18:46:31 2016 +0100 +++ b/tools/install.sh Mon Feb 08 20:18:15 2016 +0100 @@ -81,7 +81,7 @@ # Tools -TOOLS="fix.sh init.sh init_user.sh make_freebusy.py update_scheduling_modules.py" +TOOLS="fix.sh init.sh init_user.sh make_freebusy.py update_quotas.py update_scheduling_modules.py" if [ ! -e "$INSTALL_DIR/tools" ]; then mkdir -p "$INSTALL_DIR/tools" diff -r a55598b6f053 -r 1fc5f34543dc tools/update_quotas.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/update_quotas.py Mon Feb 08 20:18:15 2016 +0100 @@ -0,0 +1,185 @@ +#!/usr/bin/env python + +""" +Remove expired events from quota journals. + +Copyright (C) 2014, 2015, 2016 Paul Boddie + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation; either version 3 of the License, or (at your option) any later +version. + +This program is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. See the GNU General Public License for more +details. + +You should have received a copy of the GNU General Public License along with +this program. If not, see . +""" + +from os.path import split +import sys + +# Find the modules. + +try: + import imiptools +except ImportError: + parent = split(split(__file__)[0])[0] + if split(parent)[1] == "imip-agent": + sys.path.append(parent) + +from codecs import getwriter +from imiptools.dates import get_datetime, get_default_timezone, get_time, \ + to_utc_datetime +from imip_store import FileJournal + +def remove_expired_entries(entries, expiry): + + "Remove from 'entries' events that end at or before 'expiry'." + + removed = [] + + i = 0 + while i < len(entries): + uid, recurrenceid, duration, found_expiry = entry = entries[i] + found_expiry = get_datetime(found_expiry) + + if found_expiry <= expiry: + removed.append(entry) + del entries[i] + else: + i += 1 + + return removed + +def update_entries(journal, quota, expiry, store, verbose): + + """ + Using the given 'journal' process quota records for the given 'quota', with + the given 'expiry' time used to expire events ending before or at this time, + with None meaning the current time. + + If 'store' is set, the stored details will be updated; otherwise, the + details will be written to standard output. + + If 'verbose' is set, messages will be written to standard error. + """ + + if not store: + stdout = getwriter("utf-8")(sys.stdout) + if verbose: + stderr = getwriter("utf-8")(sys.stderr) + + if not expiry: + expiry = get_time() + + journal.acquire_lock(quota) + + try: + for user in journal.get_quota_users(quota): + if verbose: + print >>stderr, user + + entries = journal.get_entries(quota, user) + removed = remove_expired_entries(entries, expiry) + + if verbose: + for entry in removed: + print >>stderr, "Removed", entry + + # Store the processed entries. + + if store: + journal.set_entries(quota, user, entries) + + # Alternatively, just write the entries to standard output. + + else: + for entry in entries: + print >>stdout, "\t".join([(s or "") for s in entry]) + finally: + journal.release_lock(quota) + +# Main program. + +if __name__ == "__main__": + + # Interpret the command line arguments. + + quotas = [] + args = [] + journal_dir = [] + expiry = [] + ignored = [] + + # Collect quota details first, switching to other arguments when encountering + # switches. + + l = quotas + + for arg in sys.argv[1:]: + if arg in ("-s", "-v"): + args.append(arg) + l = ignored + elif arg == "-j": + l = journal_dir + elif arg == "-e": + l = expiry + else: + l.append(arg) + + try: + quota = quotas[0] + except IndexError: + print >>sys.stderr, """\ +Usage: %s + +Need a quota along with the -s option if updating the journal. +Specify -v for additional messages on standard error. + +General options: + +-e indicate an expiry time for events (default is now) +-j indicate the journal directory location +""" % split(sys.argv[0])[1] + sys.exit(1) + + # Define any other options. + + store = "-s" in args + verbose = "-v" in args + + # Override defaults if indicated. + + journal_dir = journal_dir and journal_dir[0] or None + expiry = expiry and expiry[0] or None + + if expiry: + expiry = to_utc_datetime(get_datetime(expiry), get_default_timezone()) + if not expiry: + print >>sys.stderr, "Expiry time must be a valid datetime." + sys.exit(1) + + # Obtain store-related objects. + + journal = FileJournal(journal_dir) + + # Obtain a list of users for processing. + + if quota in ("*", "all"): + quotas = journal.get_quotas() + + # Process the given users. + + if verbose: + stderr = getwriter("utf-8")(sys.stderr) + + for quota in quotas: + if verbose: + print >>stderr, quota + update_entries(journal, quota, expiry, store, verbose) + +# vim: tabstop=4 expandtab shiftwidth=4