Not logged in (Log in or Sign up)

unwwwritten

Domain Specific Languages

So, the kiwi that worked with me (until last week) and I have been working on a mechanism for synchronising a legacy FileMaker database and a MySQL database.

Our reasons for doing this include the fact that we're using Ruby on Rails to run our website and the company currently runs most of its day-to-day operations in FileMaker. We're also considering moving everything over to RoR and MySQL, but the data will have to live both places for quite a while yet.

So, meanwhile, Rowan bashed together acts_as_syncable, to do as much of the heavy lifting as possible. So far, things are working pretty well (even if it's not live yet). We use ActiveRecord to interact with MySQL on the Rails side, and rubyodbc to access the FileMaker data through ODBC. Pulling data over to the MySQL side with transformations occurring for much of the legacy data (cleaning it up on the fly) we're getting about 200,000 records per hour.

Now, on to the fun part. As I move forward with this project, we've got about 3 tables synchronizing... leaving over 150 more to contend with. The trick at this point is implementing the transformations for all the miscellaneous noise in the FileMaker database.

So we started with having to add something like the following to the models...

  acts_as_syncable  :sync_table => 'staff',
                    :sync_primary_key => 'serial num',
                    :sync_pk_min_length => 5,
                    :sync_fieldset => [
                      {
                        :attr_name => :contact_id,
                        :odbc_name => 'contact_serial',
                        :transform => '%05dC'
                      },
                      {
                        :attr_name => :status,
                        :odbc_name => 'status'
                      },
                      {
                        :attr_name => :user_name,
                        :odbc_name => 'user name'
                      },
                      {
                        :attr_name => :phone_extension ,
                        :odbc_name => 'extension'
                      },
                      {
                        :attr_name => :team_id,
                        :odbc_name => 'team',
                        :transform => 'lookup',
                        :lookup_model => 'Team'
                      },
                      {
                        :attr_name => :office,
                        :odbc_name => 'office'
                      }
                    ]

This worked, but seemed a bit verbose for me when mapping attributes that had the same name in the FileMaker (odbc) database. Plus, I didn't like the way that a transformation from a "12345C" serial number to an integer primary key value (and back) required the actual printf format string as the transformation specification.

By checking the type of the entry in the fieldset array (ie. Hash, Symbol, etc.) and looking at what values were included I could provide simpler ways to specify mappings for fields needing no transformation and possibly even sharing the same name.

So, my next revision looked like this...

  acts_as_syncable  :sync_primary_key => 'serial num',
                    :sync_fieldset => [
                      {
                        :attr_name => :contact_id,
                        :odbc_name => 'contact_serial',
                        :transform => :serial,
                        :suffix => "C"
                      },
                      :status,
                      { :user_name => 'user name' },
                      { :phone_extension => 'extension' },
                      {
                        :attr_name => :team_id,
                        :odbc_name => 'team',
                        :transform => 'lookup',
                        :lookup_model => 'Team'
                      },
                      :office
                    ]

Again, this worked, but as I got into more complex transformations (such as lookup tables where I may or may not want to create the corresponding entry if the lookup fails), this does not seem the most straightforward way to deal with it.

In one of my last conversations with "the kiwi" we both arrived at the conclusion that a DSL may be called for. So, now, I'm trying to take it one step further... a DSL (domain-specific language) for specifying the synchronisation mappings.

My first attempt looks like this...

acts_as_syncable do
  sync :contact_id, :from => 'contact_serial', :as => :serial, :with => 'C'
  sync :status
  sync :user_name, :from => 'user name'
  sync :phone_extension, :from => 'extension'
  sync :team_id, :from => 'team', :as => :lookup, :in => 'Team'
  sync :office
end

I think that works. We'll have to see how it holds up with the more complex transformations. Also, we'll have to see what it's like to implement. In the meantime, here's some more reading about DSL's...

Martin Fowler on DSLs...
http://martinfowler.com/bliki/DomainSpecificLanguage.html

Martin Fowler on what is and what isn't a DSL...
http://martinfowler.com/bliki/DslBoundary.html

Paul Graham on Programming Bottom-up...
http://www.paulgraham.com/progbot.html

Jamis Buck on Writing Domain Specific Languages...
http://weblog.jamisbuck.org/2006/4/20/writing-domain-specific-languages

Jim Freeze on Creating DSLs with Ruby...
http://www.artima.com/rubycs/articles/ruby_as_dsl.html

Russ Olsen with a 2-parter on Building a DSL in Ruby...
http://jroller.com/rolsen/entry/building_a_dsl_in_ruby
http://www.jroller.com/rolsen/entry/building_a_dsl_in_ruby1

Ron Evans with a 4-part series on Ruby Domain Specific Languages...
http://deadprogrammersociety.blogspot.com/2006/11/ruby-domain-specific-languages-basics.html
http://deadprogrammersociety.blogspot.com/2006/11/ruby-domain-specific-languages-basics_08.html
http://deadprogrammersociety.blogspot.com/2006/11/ruby-domain-specific-languages-basics_19.html
http://deadprogrammersociety.blogspot.com/2006/11/ruby-domain-specific-languages-basics_27.html

A Google Talk by David Pollak entitled "Ruby Sig: How To Design a Domain Specific Language"...
http://video.google.com/videoplay?docid=-8103284744220333344

blog comments powered by Disqus