Introducing Bigtop (cue the circus music)

BACKGROUND

I'm about to tell you a story.  It's the story of invention.  As I am
the inventor I can't tell whether the invention is the best thing since
sliced bread or a candidate for the most outrageous category on the American
Inventor TV show.  You'll have to decide whether I've made a better
bread slicer or a Perfect Pet Petter.  (For those who didn't see it:
The Perfect Pet Petter has a motion sensor, so it knows when a dog is
ready for petting, and a mechanical hand to do the petting.  Very scary.)

It was Fred Brooks who told us that revealing our table structure would
make the workings of our apps obvious.  Some may think that the wide adoption
of object-oriented programming and the advent of web delivered apps obviates
Mr. Brooks assertion.  I don't.

I had been out of the data driven app development business for quite a
while when I took my current job a little more than a year ago.  After
coming up to speed on mod_perl and Postgres, I discovered that our apps
are strikingly similar to the academic records apps I worked on during
my first job, as a registrar's office programmer for a small college,
twenty years ago.

While the apps look nicer and generally run faster, their basic ideas have
not changed.  They are still firmly centered around their data models.

As I worked on the tedious bits of a new app last year I found myself
violating the first principle of coding I learned in college: don't duplicate
code.  (Since I used to teach college Math and Computer Science, I don't use
the term Don't Repeat Yourself or DRY.  Just try to imagine a college teacher
who never repeats himself.)  Many others had already discovered and named the
bit of code I was writing over and over.  It was CRUD (Create, Retrieve,
Update, and Delete).

There are many web frameworks.  My guess is most of them started
to avoid hand coding CRUD.  They must have all started on the premise
of duplication reduction.

Since we were using a framework, yet were forced to repeat ourselves,
we began to think that a major revision of the framework was in order.
This led to Gantry (http://www.usegantry.org), which kept the best bits
of our shop's original framework while eliminating yet more duplication.

At the risk of shameless self promotion -- since I wrote the relevant
parts -- I'll assert that Gantry has a pair of nice CRUD schemes.  The
first, called AutoCRUD, does what you want for the 60-80% of your tables
which are dull.  The second, called CRUD, takes care of the others, with
a bit more help from you.

Having test driven the CRUD schemes in Gantry, I wanted more.  I didn't
want to have to write an SQL schema file describing the table structure,
then go back and write object relational mapping code (for Class::DBI or
something similar), and still have to return a third time, to the same
data model, to describe its appearance in an on screen form.

I wanted a single place to say everything that needed saying about the
data.  That place is now a bigtop file.  In it you can fully describe
your data model and its controllers.  A relatively quick generation
step builds the app.  Further, the code it builds is always separated
into two pieces.  One piece is for you to revise and expand to do the
things that make your app different from others.  The other piece is
generated and safely regenerated to allow for changes to the data model.

Bigtop currently supports Gantry with the Postgres database engine.
But it allows for backend plugins to generate whatever is needed for
your app.  Backends can easily add keywords to the grammar and are
free to generate whatever you like.  For example, there are already
backends for two different ORMs.

EXAMPLE

Here I will show the smallest app I can think of (that does anything
useful) with bigtop.  I built this app with the tentmaker, a browser
delivered bigtop editor.  (See the following section for more info on
tentmaker.)

If you want to understand better what the bigtop description means,
start by reading Bigtop::Docs::Tutorial (or even Gantry::Docs::Tutorial).
I won't show details here.

    config {
        engine CGI;
        template_engine TT;
        Init     Std {  }
        CGI      Gantry { with_server 1; }
        Control  Gantry {  }
        SQL      Postgres {  }
        Model    GantryCDBI {  }
        SiteLook GantryDefault {
            gantry_wrapper `/home/pcrow/srcgantry/root/sample_wrapper.tt`;
        }
    }
    app Contact {
        config {
            dbconn `dbi:Pg:dbname=contact` => no_accessor;
            dbuser apache => no_accessor;
            dbpass `$ecret` => no_accessor;;
            template_wrapper `genwrapper.tt` => no_accessor;
            root `/home/pcrow/build/Contact/html:/home/pcrow/srcgantry/root`
                    => no_accessor;
        }
        authors `Phil Crow`;
        email `philcrow2000@yahoo.com`;
        sequence contact_seq {}
        table contact {
            field id {
                is int4, primary_key, auto;
            }
            sequence contact_seq;
            field name {
                is varchar;
                label Name;
                html_form_type text;
            }
            field phone {
                is varchar;
                label Phone;
                html_form_type text;
            }
        }
        controller Contacts is AutoCRUD {
            rel_location contacts;
            controls_table contact;
            text_description Contact;
            method do_main is main_listing {
                cols name, phone;
                header_options Add;
                row_options Edit, Delete;
                title `Contact Management`;
            }
            method form is AutoCRUD_form {
                all_fields_but id;
            }
        }
    }

Building and Running

With the bigtop file in hand, you can be running your app in a few steps:

    bigtop --create contact.bigtop all
    cd Contact
    createdb contact -U postgres
    psql contact -U regular_user < docs/schema.postgres
    app.server [ port_defaults_to_8080 ]

You are ready to use the app through a browser.

You may need to update the data model, even after you've added code
to Contact/Contacts.pm.  This is a complete safe operation, but source
code control is still a good idea.

Simply edit the bigtop file, in the docs subdirectory of the working directory,
(with your favorite editor or the tentmaker).  Save the result and rebuild:

    bigtop docs/contact.bigtop all

Note that you may want to turn off generation of things which shouldn't change.
Do this by adding no_gen statements (or checking no_gen boxes in tentmaker):

    Init Std { no_gen 1; }

Turning of Init generation is particularly common since failure to do
so invites bigtop to overwrite README, Changes, etc.

You can turn off generation for any backend in this way.  Further, you
can include no_gen statements at the table and controller levels too (as
well as for the whole app, a feature for the truly paranoid).

MAKING TENTS

Even as the bigtop language has grown, the basic structure remain simple.
There are two top level blocks: config and app.  In the config block there
are statements and backend blocks.  In the app block there are sequence,
table, config, and controller blocks (plus a set of literal statements).
Table blocks have field blocks, while controller blocks have methods.

But the simplicity of bigtop structure still leaves a little to be desired.
The problem is the proliferation of useful keywords.  While each has its
place and use, remembering them is hard.

After much thought and several abortive attempts at helping you write
bigtop files, I began the tentmaker.  It allows you to edit your bigtop
files with a browser (as long as your browser is DOM compliant, think
firefox not IE).

If you've install bigtop and the tentmaker templates that come with it,
you can start tentmaker easily enough:

    tentmaker [--port=8081]

Then point your browser to the port you chose.

Tentmaker does not do everything.  Here's a list of what it can't handle:

    * whitespace preservation

    * comment preservation or management

    * table level data statements (but you can fake these with literal SQL
      statements

    * non-standard destinations for do_add, do_edit, etc.

Unfortunately, editing a bigtop file which uses the above features with
tentmaker will likely silently delete things you care about when it saves
your file.  For whitespace, this is a minor annoyance, for the others
it could be fatal to your app's proper functioning.
