# Design notes for hourly.pl
#
# $Id: hourly.txt,v 1.3 2000/02/29 16:22:27 bwilson Exp $

X = [last log we processed]
Y = X + [most logs we can skip] - 1
J = NULL
for (I = Y; I > X; I--)
	if exists (I) on keymaster
		J = I
next I

If J is not NULL
	if (datepart(day, J)) > (datepart(day, X))
		run daily.pl and wait for it to finish
			(daily.pl must clear out daytables when finished)
	endif

	copy J from keymaster to tally
	unzip J (Abort with error if unzip fails)
	preprocess J (cut blockid column)
	bcp in J
	SQL to move J into holding tables
	write J as X
	goto top (is there another log waiting?)
endif
wait for next scheduled run

# Comments / Questions

- Where do we intend to store "X"?  in sql or in a work file?
  If we're placing it in sql, how should that be handled?  Is it time
  for a projectinfo table?  (could possibly assume some of the work
  being done by Stats.pm)
[bw] I envisioned it stored in a work file.  It can be SQL if that's
  a lot easier.  My OGR design includes a "Projects" table, since OGR
  projects will share a table.  Don't know Stats.pm.
=======
OK, I basically scrubbed the above.  Since the logfiles live on another machine,
on the other side of the net, it's much quicker and probably saner to just treat
the "look at the logs dir" operation as atomic.
>>>>>>> 1.2

Current code grabs a directory listing of the logs dir on the master and works
from that.  loops from "lastlog+1" to "highest log present" and pulls the lowest
qualifying log found.

code assumes existence of ~/var/

Only moderately tested at this point.  Handles obvious exceptions.

