docs/tech-manual.html
<html>
<head>
<title>Technical Manual: RedBrick Registration System</title>
<link rel="stylesheet" href="doc.css" type="text/css">
</head>
<body>
<h1>Technical Manual:<br>RedBrick Registration System</h1>
<p><i>Cillian Sharkey, CASE3, 50716197</i></p>
<ol>
<li><a href="#overview">Overview of project</a>
<ol>
<li><a href="#background">Background</a>
<li><a href="#users">Users</a>
<li><a href="#features">Features</a>
<li><a href="#constraints">Constraints</a>
</ol>
<li><a href="#arch">System architecture</a>
<ol>
<li><a href="#clients">Clients</a>
<li><a href="#middle">Middle end</a>
<li><a href="#back">Back end</a>
</ol>
<li><a href="#data">User data</a>
<ol>
<li><a href="#sql-users">users</a>
<li><a href="#usertypes">usertypes</a>
<li><a href="#students">students</a>
<li><a href="#staff">staff</a>
<li><a href="#reserved">reserved</a>
</ol>
<li><a href="#boundary">Software and Hardware Boundaries</a>
<li><a href="#refs">References</a>
</ol>
<a name=overview></a><h2>Overview of project</h2>
<a name=background></a><h3>Background</h3>
<p>There are just over 1,800 accounts on the RedBrick UNIX system and so the
administration of user accounts forms a large part of the system
administrators' workload. As one of the system administrators for RedBrick
myself, I have first hand experience of the amount of work required in
administrating user accounts and dealing with account requests on an often
daily basis. In addition to this, each year at the clubs & societies day
the registration of new and renewal of existing users takes place. Hundreds
of users are processed on a small isolated network of computers. The goal of
this project is to reduce the workload of system administrators and ease the
administration and registration of users both on a daily basis and for the annual
clubs and societies day.</p>
<a name=users></a><h3>Users</h3>
<p>The users of the system will be restricted to the system administrators
for day to day user administration as it requires root (superuser) access, however
for registration on clubs & societies day it is usual for other members of the
society committee to help out.</p>
<a name=features></a><h3>Features</h3>
<ul>
<li><p>The entire system is written in Python, whose website describes
it as:</p>
<blockquote>
<p>"Python is an interpreted, interactive, object-oriented
programming language. It is often compared to Tcl, Perl, Scheme
or Java. Python combines remarkable power with very clear
syntax. It has modules, classes, exceptions, very high level
dynamic data types, and dynamic typing. The Python
implementation is portable: it runs on many brands of UNIX, on
Windows, DOS, OS/2, Mac, Amiga... "</p>
</blockquote>
<li><p>Provides a flexible, automated and consistent (UNIX command
line) interface (<code>useradm</code>) for performing both common
day-to-day and occassional "batch" user administration operations on
the actual accounts (home directories, mail spools, disk quotas etc.),
the UNIX <code>/etc/passwd</code> "database" and the user database
ensuring that all are kept in sync. The following functions are
provided:</p>
<ul>
<li>adding new accounts,
<li>deleting existing accounts,
<li>renaming existing accounts,
<li>converting existing accounts "usertype",
<li>renewing existing accounts,
<li>reseting accounts with new random password (and emailing password
to user)
<li>reseting accounts with disabled Unix shells,
<li>retrieving account information for display,
<li>checking the availability of usernames for new accounts,
</ul>
<p>Batch account operations include:</p>
<ul>
<li>emailing renewal reminders to non-renewed accounts
<li>expiring non-renewed accounts (i.e. disabling login shell)
<li>deleting non-renewed accounts after a grace period
<li>checking for inconsistencies between the user database and the
UNIX <code>/etc/passwd</code> database
<li>interactive update of the user database from the latest copy of
the public DCU student database
</ul>
<p>Other information available:</p>
<ul>
<li>searching user and DCU databases,
<li>database and account statistics
</ul>
<p>The following is the output for the command usage help from
<code>useradm</code> when given the '-h' option which lists all
the names of the functions the script performs.</p>
<pre>Usage: useradm command [options]
Single user commands:
add Add new user
delete Delete user
renew Renew user
update Update user
rename Rename user
convert Change user to a different usertype
Single account commands:
resetpw Set new random password and mail it to user
resetsh Reset user's shell
disuser Disuser a user
reuser Re-user a user
Single user information commands:
show Show user details
freename Check if a username is free
Interactive batch commands:
search Search user and dcu databases
sync Synchronise accounts with userdb (for RRS)
sync_dcu_info Interactive update of userdb using dcu database info
Batch commands:
newyear Prepare database for start of new academic year
unpaid_warn Warn (mail) all non-renewed users
unpaid_disable Disable all normal non-renewed users
unpaid_delete Delete all grace non-renewed users
Batch information commands:
newbies_list List all paid newbies
renewals_list List all paid renewals (non-newbie)
freename_list List all usernames that are taken
unpaid_list List all non-renewed users
unpaid_list_normal List all normal non-renewed users
unpaid_list_reset List all normal non-renewed users with reset shells
unpaid_list_grace List all grace non-renewed users
Miscellaneous commands:
checkdb Check database for inconsistencies
stats Show database and account statistics
'useradm command -h' for more info on a command's options & usage.</pre>
<li><p>Provides a web interface (<code>rrs</code>) to offer a similar
set of single user administration operations for use on the clubs &
societies day. While not as fully featured or as flexible as
<code>useradm</code>, the interface is designed primarily for speed and
efficiency.</p>
<p>To achieve this, user input fields are automatically focused on page
load (using Javascript) to save using the keyboard or mouse to select
the input box; input from barcode scanners or magnetic strip
readers (for use with DCU staff and student cards) is accepted which
considerably speeds up the process and eliminates the potential for
human error.</p>
<p>The setup is generally that of a small number of networked computers
that are isolated from the RedBrick servers so changes would be made to
a local (seperate) copy of the user database and at the end of the day
all of these changes (i.e. new, renewed, renamed, converted, etc.
accounts) must be detected and synchronised with the actual accounts
and UNIX <code>/etc/passwd</code> database on the system. This needs to
be done in batch as hundreds of accounts are processed on clubs &
societies day. This is implemented as the <a
href=useradm.html#-sync>sync</a> command in
<code>useradm</code>. Features which <code>rrs</code> provides:</p>
<ul>
<li>adding new users,
<li>deleting existing users,
<li>renaming existing users,
<li>converting existing users "usertype",
<li>renewing existing users,
<li>retrieving user information for display,
<li>checking the availability of usernames for new users,
<li>searching user and DCU databases,
<li>database statistics,
<li>a detailed log of all actions performed,
</ul>
<li>Prevent username conflicts with other namespaces on the system e.g.
email aliases, mailing lists, DNS entries for the redbrick.dcu.ie and
other domains the society host. Achieved through the use of the <a
href=#reserved>reserved</a> table.
</ul>
<a name=constraints></a><h3>Constraints</h3>
<ul>
<li>All accounts must have a name (or description) and an alternate
email address for contact purposes (and as such all UNIX accounts must
have a corresponding entry in the user database).
<li>Users may only have one account on the system (i.e. one student ID
per account).
<li>Member, staff and associate accounts should have a valid DCU
student/staff ID associated with them.
<li>Usernames are limited to what the underlying UNIX OS supports, i.e.
maximum length of 8 characters, small set of available characters (a-z,
0-9, '-', '_', '.'), etc.
</ul>
<a name=arch></a><h2>System architecture</h2>
<p>The system is 3 tier in nature:</p>
<a name=clients></a><h3>Clients</h3>
<p>There are two client applications:</p>
<ul>
<li><a href=rrs.html>rrs:</a> a CGI web interface (primarily for use on
the annual clubs & societies day to register new members and renew
existing members)
<li><a href=useradm.html>useradm:</a> a UNIX command line interface for
day-to-day user administration.
</ul>
<p>Additional maintenance scripts:</p>
<ul>
<li><a href=rebuild_userdb_reserved.html>rebuild_userdb_reserved</a> -
rebuilds reserved table
<li><a href=rebuild_userdb_staff.html>rebuild_userdb_staff</a> - rebuilds
staff table
<li><a href=rebuild_userdb_students.html>rebuild_userdb_students</a> -
rebuilds students table
<li>dump_userdb.sh - Unix shell script to backup entire user database
and provide local text copies for quick perusal with standard Unix
tools such as grep, etc.
</ul>
<a name=middle></a><h3>Middle end</h3>
<p>There are two main modules and a number of supporting modules which the two
client applications (and any potential future applications) use. These are:</p>
<ul>
<li><a href=rbuserdb.html>rbuserdb:</a> performs all database operations
<li><a href=rbaccount.html>rbaccount:</a> performs all local UNIX
account operations
<li><a href=rberror.html>rberror:</a> contains custom exception classes
(RBError, RBFatalError, RBWarningError) for our own types of errors
<li><a href="rbuser.html">rbuser:</a> contains RBUser class for
representing a user
<li><a href="rbopt.html">rbopt:</a> contains RBOpt class for storing
user options and sharing between the various modules
</ul>
<p>Note that the web interface does not make use of the rbaccount module however,
as it operates on the database only.</p>
<a name=back></a><h3>Back end</h3>
<p>A PostgresSQL database contains the following tables:</p>
<ul>
<li><a href=#users>users</a> - the current user table
<li>users<YEAR> - the user table(s) for previous years. These are
kept for reference and archival purposes only and are not actively used.
<li><a href=#usertypes>usertypes</a> - contains a list of accepted usertypes
<li><a href=#students>students</a> - contains basic information on all
DCU students (id number, name, email, course, year).
<li><a href=#staff>staff</a> - contains basic information on most
DCU staff (id number, name, email).
<li><a href=#reserved>reserved</a> - contains a list of additional
reserved names.
</ul>
<p>The traditional <code>/etc/passwd</code>, <code>/etc/shadow</code> and
<code>/etc/group</code> files in addition to the filesystem store Unix account
information.</p>
<a name=data></a><h2>User data</h2>
<p>The traditional UNIX <code>/etc/passwd</code> database doesn't contain
enough information for user accounts for RedBrick's needs (nor does it support
complex queries that a database can) hence the need for a user database. Following
are descriptions of the various database tables:
<a name=sql-users></a><h3>users [<a href=userdb_users.sql>schema</a>]</h3>
<dt>Username</dt>
<dd>Unique UNIX username. Primary key, must be non-null and 8 characters maximum.</dd>
<dt>Usertype</dt>
<dd>Corresponds to both UNIX group and type of user. This references
the usertype foreign key in the <a href=#usertypes>usertypes</a> table to
ensure it is valid.</dd>
<dt>Newbie</dt>
<dd>Boolean; true if user is a new user this academic year. Must be
non-null.</dd>
<dt>Name/Description</dt>
<dd>User's full name or a description for non-user accounts (e.g.
project/system accounts). Must be non-null.</dd>
<dt>Email</dt>
<dd>Alternate contact address (typically DCU email address). Must be non-null.</dd>
<dt>Student ID</dt>
<dd>DCU student (or staff) ID number, compulsory for member, staff and
associates optional for all other usertypes. Must be unique.</dd>
<dt>Years paid</dt>
<dd>Number of years paid, compulsory for member, staff and associates optional
for all other usertypes (only used by committe and guests though). Non-renewed
accounts (referred to as 'normal') will be 0 if they were renewed for the
previous year or -1 (referred to as 'grace') if they were not.</dd>
<dt>Updated by</dt>
<dd>Username of the administrator who last updated the database entry. Must be
non-null.</dd>
<dt>Updated at</dt>
<dd>Date & time of when the database entry was last updated. This is
updated automatically by a database trigger for all INSERT and UPDATE
operations on the table by setting it to the current time & date.</dd>
<dt>Created by</dt>
<dd>Username of the administrator who first created the account. This should be
set once at account creation time (INSERT). The default value for this is the
same as updated_by, so there's no need to explicitly set it.</dd>
<dt>Created at</dt>
<dd>Date & time of when the database entry was first created. This should
be set once at account creation time (INSERT). The default value for this is
the current time & date so there's no need to explicitly set it.</dd>
<dt>Birthday</dt>
<dd>Optional date, to be potentially used for birthday notifications, etc.
Only applies to real user accounts (i.e. member, staff, associat, committe,
guest).</dd>
<a name=usertypes></a><h3>usertypes [<a href=userdb_usertypes.sql>schema</a>]</h3>
<dt>Usertype</dt>
<dd>Name of usertype. As these correspond to real Unix groups, there is a limit
of maximum length 8 characters.
<p>Description of usertypes:</p>
<dt>system</dt>
<dd>System accounts, pseudo-user accounts etc. Default name: 'System Account',
default email: 'admins@redbrick.dcu.ie'.</dd>
<dt>reserved</dt>
<dd>Permanent reserved names (e.g. server hostnames, usernames that may cause
potential confusion or misuse e.g. a username of 'redbrick', 'admin' etc.). These
entries do NOT have corresponding Unix accounts (the system usertype is for that).
Default name: 'Reserved Name, default email: 'admins@redbrick.dcu.ie'.
<dt>founders</dt>
<dd>A founder of the society member. A rather closed group, admittedly.</dd>
<dt>member</dt>
<dd>Current students of DCU only. Email should be their DCU email address.</dd>
<dt>associat</dt>
<dd>Graduates, former DCU students/staff or people otherwise officially
associated with DCU (must have a DCU ID number).</dd>
<dt>staff</dt>
<dd>Current DCU staff members only. Email should be their DCU email address.</dd>
<dt>club / society</dt>
<dd>DCU Club/Society account. Name set to official name of club/society, email
preferably set to a contact address for the current secretary/chair of that
club/society.</dd>
<dt>dcu</dt>
<dd>General DCU body, organisation or role account (e.g. 'supres'). Name must
be full description of entity, email address should be a DCU one.</dd>
<dt>intersoc</dt>
<dd>Account for another networking society. Username set to abbreviation of
society name, name set to society name, email set to a contact address for the
current secretary/chair of the society.
<dt>projects</dt>
<dd>Account for a Redbrick/DCU/Academic/Course project. Username set to
abbreviation of project name, name set to full project name, email set to a
contact address for the project leader.</a>
<dt>redbrick</dt>
<dd>Account for miscellaneous Redbrick activities (e.g. 'semnet', 'tutorial').
Name set to description of account. Email set to redbrick user in charge of it
if appropriate, otherwise the redbrick committee.</dd>
<dt>guest</dt>
<dd>Guest accounts. For accounts that don't fit into any of the above types.
These must be approved by the committee on a per-user basis.</dd>
<a name=students></a><h3>students [<a href=userdb_students.sql>schema</a>]</h3>
<p>This database is updated nightly from DCU's LDAP server (host: atlas.dcu.ie,
base DN: 'ou=Students, o=DCU', search filter: objectClass=person) by <a
href=rebuild_userdb_students.html>rebuild_userdb_students</a> launched by
cron.</p>
<dt>id</dt>
<dd>DCU Student ID (one of the values of 'cn'). Non-null unique integer.</dd>
<dt>name</dt>
<dd>Name (value of 'givenName' and 'sn' concatenated). Non-null string.</dd>
<dt>email</dt>
<dd>DCU email address. Non-null string.</dd>
<dt>course</dt>
<dd>DCU course (abbreviated version). Non-null string.</dd>
<dt>year</dt>
<dd>Year in DCU or 'N/A' if not available. Non-null string (characters are also
used sometimes, e.g: 'X' for exchange student, so can't use an integer
type).</dd>
<a name=staff></a><h3>staff [<a href=userdb_staff.sql>schema</a>]</h3>
<p>This database is updated nightly from DCU's LDAP server (host: atlas.dcu.ie,
base DN: 'ou=Staff, o=DCU', search filter: objectClass=person) by <a
href=rebuild_userdb_staff.html>rebuild_userdb_staff</a> launched by
cron.</p>
<dt>id</dt>
<dd>DCU Staff ID (one of the values of 'cn', or extracted from 'gecos').
Non-null unique integer.</dd>
<dt>name</dt>
<dd>Name (value of 'givenName' and 'sn' concatenated). Non-null string.</dd>
<dt>email</dt>
<dd>DCU email address. Non-null string.</dd>
<a name=reserved></a><h3>reserved [<a href=userdb_reserved.sql>schema</a>]</h3>
<p>This database is updated nightly from the various email alias files, Unix
group names and DCU entries for the redbrick domain and any other zones the
society host. Note that the entries in this table are not strictly reserved,
they can be ignored if necessary (e.g. creating a system user in the database
which already has the Unix group setup in advance, etc.). Also of note is that
these entries are not permanent, as the entire table is purged and rebuilt.
Permanent reserved entries belong in the users table as noted previously.</p>
<dt>username</dt>
<dd>A reserved username. Primary key.</dd>
<dt>info</dt>
<dd>Information about the source/nature of this reserved username. Non-null
string.</dd>
<a name=boundary></a><h2>Software and Hardware Boundaries</h2>
<p>The project uses the OS native command line tools for performing all
UNIX <code>/etc/passwd</code> operations and the majority of account
operations. The 3rd party utility <code>setquota</code> is used for
the modification of disk quotas. DCU student and staff information is retrieved
from the publically accessible LDAP service on <code>atlas.dcu.ie</code>.</p>
<a name=refs></a><h2>References</h2>
<h3>Software</h3>
<ul>
<li><a href='http://www.python.org/'>Python</a>
<li><a href='http://www.postgresql.org/'>PostgresSQL</a>
</ul>
<h3>UNIX (Solaris) user administration tool manpages</h3>
<ul>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/useradd.1.html">useradd</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/usermod.1.html">usermod</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/userdel.1.html">userdel</a>
</ul>
<hr>
$Id: tech-manual.html,v 1.1 2003/03/28 16:37:38 cns Exp $
</body>
</html>