go vendor

vendor/github.com/revel/modules/jobs/README.md

@@ -0,0 +1,220 @@
+---
+title: Jobs
+github:
+  labels:
+    - topic-jobs
+    - topic-runtime
+---
+
+The [`Jobs`](https://godoc.org/github.com/revel/modules/jobs/app/jobs) framework for performing work asynchronously, outside of the
+request flow.  This may take the form of [recurring tasks](#jobs) that updates cached data
+or [one-off tasks](#OneOff) such as sending emails.
+
+## Activation
+
+The [`Jobs`](https://godoc.org/github.com/revel/modules/jobs/app/jobs) framework is included as an optional [module](index.html), and is not enabled by default.
+To activate it, add `module.jobs` to the [app.conf](../manual/appconf.html) file:
+
+```ini
+module.jobs = github.com/revel/modules/jobs
+```
+
+Additionally, in order to access the job monitoring page, you will need to add
+this line to the `conf/routes` file, which will insert the `/@jobs` url:
+
+	module:jobs
+
+
+## Options
+
+There are some [configuration settings](../manual/appconf.html#jobs) that tell the framework what sort of limitations
+to place on the jobs that it runs. These are listed below with their default values;
+
+- [`jobs.pool = 10`](appconf.html#jobspool) - The number of jobs allowed to run simultaneously
+- [`jobs.selfconcurrent = false`](appconf.html#jobsselfconcurrent) - Allow a job to run only if previous instances are done
+- [`jobs.acceptproxyaddress = false`](appconf#jobsacceptproxyaddress) - Accept `X-Forwarded-For` header value (which is spoofable) to allow or deny status page access
+
+## Implementing Jobs
+
+To create a Job, implement the [`cron.Job`](https://github.com/robfig/cron/) interface.  The
+[`Job`](https://godoc.org/github.com/revel/modules/jobs/app/jobs#Job) interface has the following signature:
+
+{% highlight go %}
+// https://github.com/robfig/cron/blob/master/cron.go
+type Job interface {
+	Run()
+}
+{% endhighlight %}
+
+For example:
+
+{% highlight go %}
+type MyJob struct {}
+
+func (j MyJob) Run() {
+   // Do something
+}
+{% endhighlight %}
+
+## Startup jobs
+
+To run a task on application startup, use
+[`revel.OnAppStart()`](https://godoc.org/github.com/revel/revel#OnAppStart) to register a function.
+Revel runs these tasks serially, before starting the server.  Note that this
+functionality does not actually use the jobs module, but it can be used to
+submit a job for execution that doesn't block server startup.
+
+{% highlight go %}
+func init() {
+    revel.OnAppStart(func() { jobs.Now(populateCache{}) })
+}
+{% endhighlight %}
+
+<a name="RecurringJobs"></a>
+
+## Recurring Jobs
+
+Jobs may be scheduled to run on any schedule.  There are two options for expressing the schedule:
+
+1. A cron specification
+2. A fixed interval
+
+Revel uses the [`cron library`](https://godoc.org/github.com/revel/cron) to parse the
+schedule and run the jobs.  The library's
+[README](https://github.com/revel/cron/blob/master/README.md) provides a detailed
+description of the format accepted.
+
+Jobs are generally registered using the
+[`revel.OnAppStart()`](https://godoc.org/github.com/revel/revel#OnAppStart) hook, but they may be
+registered at any later time as well.
+
+Here are some examples:
+
+{% highlight go %}
+import (
+    "github.com/revel/revel"
+    "github.com/revel/modules/jobs/app/jobs"
+    "time"
+)
+
+type ReminderEmails struct {
+    // filtered
+}
+
+func (e ReminderEmails) Run() {
+    // Queries the DB
+    // Sends some email
+}
+
+func init() {
+    revel.OnAppStart(func() {
+        jobs.Schedule("0 0 0 * * ?",  ReminderEmails{})
+        jobs.Schedule("@midnight",    ReminderEmails{})
+        jobs.Schedule("@every 24h",   ReminderEmails{})
+        jobs.Every(24 * time.Hour,    ReminderEmails{})
+    })
+}
+{% endhighlight %}
+
+<a name="NamedSchedules"></a>
+
+## Named schedules
+
+You can [configure schedules ](appconf.html#jobs) in the [`app.conf`](appconf.html) file and reference them anywhere.
+This provides an easy way to reuse, and a useful description for crontab specs.
+
+Here is an example **named cron schedule**, in an [`app.conf`](appconf.html) file:
+
+    cron.workhours_15m = 0 */15 9-17 ? * MON-FRI
+
+Use the named schedule by referencing it anywhere you would have used a cron spec.
+
+{% highlight go %}
+func init() {
+    revel.OnAppStart(func() {
+        jobs.Schedule("cron.workhours_15m", ReminderEmails{})
+    })
+}
+{% endhighlight %}
+
+<div class="alert alert-warning">
+<b>IMPORTANT</b>: The cron schedule's name must begin with <b>cron</b>.
+
+</div>
+
+
+<a name="OneOff"></a>
+
+## One-off Jobs
+
+Sometimes it is necessary to do something in response to a user action.  In these
+cases, the jobs module allows you to submit a job to be run a single time.
+
+The only control offered is how long to wait until the job should be run.
+
+{% highlight go %}
+type AppController struct { *revel.Controller }
+
+func (c AppController) Action() revel.Result {
+    // Handle the request.
+    ...
+
+    // Send them email asynchronously, right now.
+    jobs.Now(SendConfirmationEmail{})
+
+    // Or, send them email asynchronously after a minute.
+    jobs.In(time.Minute, SendConfirmationEmail{})
+}
+{% endhighlight %}
+
+## Registering functions
+
+It is possible to register a `func()` as a job by wrapping it in the [`jobs.Func`](https://godoc.org/github.com/revel/modules/jobs/app/jobs#Func)
+type.  For example:
+
+{% highlight go %}
+func sendReminderEmails() {
+    // Query the DB
+    // Send some email
+}
+
+func init() {
+    revel.OnAppStart(func() {
+        jobs.Schedule("@midnight", jobs.Func(sendReminderEmails))
+    })
+}
+{% endhighlight %}
+
+
+## Job Status
+
+The jobs module provides a status page (`/@jobs` url) that shows:
+
+- a list of the scheduled jobs it knows about
+- the current status; **IDLE** or **RUNNING**
+- the  previous and next run times
+
+<div class="alert alert-info">For security purposes, the status page is restricted to requests that originate
+from 127.0.0.1.</div>
+
+![Job Status Page](../img/jobs-status.png)
+
+
+
+## Constrained pool size
+
+It is possible to configure the job module to limit the number of jobs that are
+allowed to run at the same time.  This allows the developer to restrict the
+resources that could be potentially in use by asynchronous jobs -- typically
+interactive responsiveness is valued above asynchronous processing.  When a pool
+is full of running jobs, new jobs block to wait for running jobs to complete.
+
+**Implementation Note**: The implementation blocks on a channel receive, which is
+implemented to be [FIFO](http://en.wikipedia.org/wiki/FIFO) for waiting goroutines (but not specified/required to be
+so). [See here for discussion](https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/CPwv8WlqKag).
+
+## Future areas for development
+
+* Allow access to the job status page with HTTP Basic Authentication credentials
+* Allow administrators to run scheduled jobs interactively from the status page
+* Provide more visibility into the job runner, e.g. the pool size, the job queue length, etc.

vendor/github.com/revel/modules/jobs/app/controllers/status.go

@@ -0,0 +1,35 @@
+package controllers
+
+import (
+	"strings"
+
+	"github.com/revel/cron"
+	"github.com/revel/modules/jobs/app/jobs"
+	"github.com/revel/revel"
+)
+
+type Jobs struct {
+	*revel.Controller
+}
+
+func (c Jobs) Status() revel.Result {
+	remoteAddress := c.Request.RemoteAddr
+	if revel.Config.BoolDefault("jobs.acceptproxyaddress", false) {
+		if proxiedAddress := c.Request.GetHttpHeader("X-Forwarded-For"); proxiedAddress!="" {
+			remoteAddress = proxiedAddress
+		}
+	}
+	if !strings.HasPrefix(remoteAddress, "127.0.0.1") &&
+		!strings.HasPrefix(remoteAddress, "::1") &&
+		!strings.HasPrefix(remoteAddress, "[::1]") {
+		return c.Forbidden("%s is not local", remoteAddress)
+	}
+	entries := jobs.MainCron.Entries()
+	return c.Render(entries)
+}
+
+func init() {
+	revel.TemplateFuncs["castjob"] = func(job cron.Job) *jobs.Job {
+		return job.(*jobs.Job)
+	}
+}

vendor/github.com/revel/modules/jobs/app/jobs/init.go

@@ -0,0 +1,13 @@
+package jobs
+
+import (
+	"github.com/revel/revel"
+)
+
+var jobLog = revel.AppLog
+
+func init() {
+	revel.RegisterModuleInit(func(m *revel.Module){
+		jobLog = m.Log
+	})
+}

vendor/github.com/revel/modules/jobs/app/jobs/job.go

@@ -0,0 +1,67 @@
+package jobs
+
+import (
+	"reflect"
+	"runtime/debug"
+	"sync"
+	"sync/atomic"
+
+	"github.com/revel/cron"
+	"github.com/revel/revel"
+)
+
+type Job struct {
+	Name    string
+	inner   cron.Job
+	status  uint32
+	running sync.Mutex
+}
+
+const UnNamed = "(unnamed)"
+
+func New(job cron.Job) *Job {
+	name := reflect.TypeOf(job).Name()
+	if name == "Func" {
+		name = UnNamed
+	}
+	return &Job{
+		Name:  name,
+		inner: job,
+	}
+}
+
+func (j *Job) Status() string {
+	if atomic.LoadUint32(&j.status) > 0 {
+		return "RUNNING"
+	}
+	return "IDLE"
+}
+
+func (j *Job) Run() {
+	// If the job panics, just print a stack trace.
+	// Don't let the whole process die.
+	defer func() {
+		if err := recover(); err != nil {
+			if revelError := revel.NewErrorFromPanic(err); revelError != nil {
+				jobLog.Error("Job Recovery ", "error", err, "stack", revelError.Stack)
+			} else {
+				jobLog.Error("Job Recovery ", "error", err, "stack", string(debug.Stack()))
+			}
+		}
+	}()
+
+	if !selfConcurrent {
+		j.running.Lock()
+		defer j.running.Unlock()
+	}
+
+	if workPermits != nil {
+		workPermits <- struct{}{}
+		defer func() { <-workPermits }()
+	}
+
+	atomic.StoreUint32(&j.status, 1)
+	defer atomic.StoreUint32(&j.status, 0)
+
+	j.inner.Run()
+}

vendor/github.com/revel/modules/jobs/app/jobs/jobrunner.go

@@ -0,0 +1,70 @@
+// A job runner for executing scheduled or ad-hoc tasks asynchronously from HTTP requests.
+//
+// It adds a couple of features on top of the cron package to make it play nicely with Revel:
+//
+// 1. Protection against job panics.  (They print to ERROR instead of take down the process)
+//
+// 2. (Optional) Limit on the number of jobs that may run simulatenously, to
+// limit resource consumption.
+//
+// 3. (Optional) Protection against multiple instances of a single job running
+// concurrently.  If one execution runs into the next, the next will be queued.
+//
+// 4. Cron expressions may be defined in app.conf and are reusable across jobs.
+//
+// 5. Job status reporting.
+package jobs
+
+import (
+	"strings"
+	"time"
+
+	"github.com/revel/cron"
+	"github.com/revel/revel"
+)
+
+// Callers can use jobs.Func to wrap a raw func.
+// (Copying the type to this package makes it more visible)
+//
+// For example:
+//    jobs.Schedule("cron.frequent", jobs.Func(myFunc))
+type Func func()
+
+func (r Func) Run() { r() }
+
+func Schedule(spec string, job cron.Job) error {
+	// Look to see if given spec is a key from the Config.
+	if strings.HasPrefix(spec, "cron.") {
+		confSpec, found := revel.Config.String(spec)
+		if !found {
+			jobLog.Panic("Cron spec not found: " + spec)
+		}
+		spec = confSpec
+	}
+	sched, err := cron.Parse(spec)
+	if err != nil {
+		return err
+	}
+	MainCron.Schedule(sched, New(job))
+	return nil
+}
+
+// Run the given job at a fixed interval.
+// The interval provided is the time between the job ending and the job being run again.
+// The time that the job takes to run is not included in the interval.
+func Every(duration time.Duration, job cron.Job) {
+	MainCron.Schedule(cron.Every(duration), New(job))
+}
+
+// Run the given job right now.
+func Now(job cron.Job) {
+	go New(job).Run()
+}
+
+// Run the given job once, after the given delay.
+func In(duration time.Duration, job cron.Job) {
+	go func() {
+		time.Sleep(duration)
+		New(job).Run()
+	}()
+}

vendor/github.com/revel/modules/jobs/app/jobs/plugin.go

@@ -0,0 +1,31 @@
+package jobs
+
+import (
+	"github.com/revel/cron"
+	"github.com/revel/revel"
+)
+
+const DefaultJobPoolSize = 10
+
+var (
+	// Singleton instance of the underlying job scheduler.
+	MainCron *cron.Cron
+
+	// This limits the number of jobs allowed to run concurrently.
+	workPermits chan struct{}
+
+	// Is a single job allowed to run concurrently with itself?
+	selfConcurrent bool
+)
+
+func init() {
+	MainCron = cron.New()
+	revel.OnAppStart(func() {
+		if size := revel.Config.IntDefault("jobs.pool", DefaultJobPoolSize); size > 0 {
+			workPermits = make(chan struct{}, size)
+		}
+		selfConcurrent = revel.Config.BoolDefault("jobs.selfconcurrent", false)
+		MainCron.Start()
+		jobLog.Info("Go to /@jobs to see job status.")
+	})
+}

vendor/github.com/revel/modules/jobs/app/views/Jobs/Status.html

@@ -0,0 +1,39 @@
+<html>
+	<head>
+		<style>
+body {
+  font-size: 12px;
+  font-family: sans-serif;
+}
+table {
+  border-collapse: collapse;
+  border: none;
+}
+table td, table th {
+  padding: 4 10px;
+  border: none;
+}
+table tr:nth-child(odd) {
+  background-color: #f0f0f0;
+}
+th {
+  text-align: left;
+}
+		</style>
+	</head>
+	<body>
+
+<h1>Scheduled Jobs</h1>
+
+<table>
+	<tr><th>Name</th><th>Status</th><th>Last run</th><th>Next run</th></tr>
+{{range .entries}}
+	{{$job := castjob .Job}}
+	<tr>
+		<td>{{$job.Name}}</td>
+		<td>{{$job.Status}}</td>
+		<td>{{if not .Prev.IsZero}}{{.Prev.Format "2006-01-02 15:04:05"}}{{end}}</td>
+		<td>{{if not .Next.IsZero}}{{.Next.Format "2006-01-02 15:04:05"}}{{end}}</td>
+	</tr>
+{{end}}
+</table>

vendor/github.com/revel/modules/jobs/conf/routes

@@ -0,0 +1 @@
+GET     /@jobs      Jobs.Status

vendor/github.com/revel/modules/jobs/jobs.go

@@ -0,0 +1,3 @@
+package jobs
+
+// Required for vendoring see golang.org/issue/13832