RailsConf 2007: Bradley Taylor: Virtual Clusters

Posted by Nick Sieger Sat, 19 May 2007 19:30:06 GMT

How does Rails figure into virtualization? Bradley will cover this topic with examples and case studies. Along the way, hardware items may be mentioned, but are not critical. Really, it’s about the design of the clusters, not the bits of plumbing you use to connect them up.

Virtualization is partitioning of physical servers that allow you to run multiple servers on it. Xen, Virtuozzo, VMWare, Solaris containers, KVM, etc. Bradley uses Xen. The virtual servers share the same processor (hopefully multi-core), memory, storage, network cards (but with indepenent IP addresses), etc., but run independently of each other. VPS, slice, container, accelerator, VM, it’s all the same. Memory, storage, and CPU can be guaranteed with the virtualization layer.

Why would you do this? Consolidate servers for less hardware and cost; Isolate applications -- bad apps don’t drag the server down, contain intrusions, use different software stacks; Replicate -- easily create new servers and deploy in a standardized and automated way; Utilize -- take advantage of all CPU, memory, storage, resources; Allocate resources, give a server exactly what it requires, grow/shrink up and down, and balance them. Bradley says, “Once you go to virtualization you won’t want to go back. Do the simplest thing that could possibly work.”

Virtual clusters, then, are a bunch of servers cooperating toward a common goal -- if you have many versions or copies of one thing. More than one customer, more than one version of software, etc.

For Rails, this means a lot of things: you can have many development environments and stages, take advantage of memory isolation, protect against PHP/Java, and make multiple-server scaling accessible.

Examples

  • Two servers for production and staging
  • Three for web/db/staging
  • Mixed languages -- instead of 1x1GB server use 3x300MB servers
  • High availability applications with fewer servers
  • Multiple applications -- one server per application
  • Standardized roles/appliances -- mail, ftp, dns, web, db

EastMedia

  • They can incubate customers in separate images
  • Dev/staging/production servers
  • Shared SVN/trac
  • 2 physical servers => 8 virtual servers

Boom Design

  • Again, multiple stages
  • Customer staging, with lower uptime requirements
  • Low-traffic apps on a single server, but everything else gets its own dedicated server
  • 2GB memory spread across 9 virtual servers

Tags ,  | no comments | no trackbacks

RailsConf 2007: Saturday Morning Keynotes

Posted by Nick Sieger Sat, 19 May 2007 17:22:42 GMT

Cyndi Mitchell -- ThoughtWorks Studios

Enterprise (the “e” word)

Before IT got involved, “enterprise” was a bold new venture. Toyota manufacturing, Skype disruption of telephony.

Enterprise in terms of IT has come to mean bloatware, incompetence, corruption, waste of time, no value.

So this is the battle: The enterprise (to boldly go where no man has gone before) we need to reclaim vs. the bloatware/competence/corruption/fear-based selling etc.

RubyWorks -- package stack with haproxy, mongrel, monit through an RPM repository

For JRuby support, call Ola.

Tim Bray -- Web Guy from Sun Microsystems

Change the world that are better than just using a cool web framework: http://pragmaticstudio.com/donate/

Sun loves Ruby. Ruby and Rails, that is. The impact of the Ruby language is going to be at least as big as Rails is for web development.

Sun provided servers for Ruby 2.0 development, and can provide servers for your potentially cool, worthy, open source project, just drop Tim an email.

A few more obligatory plugs for NetBeans and Sun sponsoring the conference. “Pre-alpha,” he says. Hmm, I wonder what Tor would say about that!

JRuby: when would you use JRuby vs. Ruby? If you have no pain, keep using C Ruby. But if you have management concerns, deployment concerns, etc. then by all means do try it!

Obligatory handshake/sandal connection with ThoughtWorks and Cyndi -- running Mingle (and cruisecontrol.rb) with JRuby.

Sun: “Hi, the answer is Java, what was the question?” So why would Sun want to support Ruby? Well, you guys are programmers. Programmers who deliver quality software fast. And those programmers need computers, and OSes, and web servers, and support and services, etc. Plug, plug, plug.

How do you make money on free products? Sun has open-sourcing Java, Solaris, even Sparc. Joyent is open-sourcing their stuff. Where does the money come from? 1. Adoption 2. Deployment 3. Monetization at the point of value

What if we win? Are our problems over? No, we’ll have to deal with Java. And .NET. And PHP. From the audience: And COBOL. The Network Is The Computer. The Network Is Heterogeneous. Deal with it. So how do we interoperate?

  • Just Run Java (and JRuby, of course!, and JavaScript, and PHP, etc.)
  • Use Atom/REST. Everything should have a publish button. Don’t use WS-DeathStar or WCF or WSIT.

Developer issues: Scaling, Static vs. Dynamic, Maintainability, Concurrency, Tooling, Integration, Time to Market. Which two of these matter the most?

Tim’s final assertion: Maintainability and Time to Market, and that’s why we’re all at RailsConf.

Tags ,  | 2 comments | no trackbacks

RailsConf 2007: Evan Weaver: Going Off Grid

Posted by Nick Sieger Fri, 18 May 2007 19:33:31 GMT

Evan is talking about leaving Rails as a full-stack framework and remixing bits and pieces for integration projects. He’s doing it in the context of a case study on Bio: a project at the University of Delaware working with DNA data in large SQL databases. Evan states that all of bioinformatics is an integration problem. (Me: That’s probably true of any research project where data is coming from multiple, varied sources. So where does Rails fit in this?)

So how do you cope with this? Use the Rails console as an admin interface, mapping AR onto the legacy schema.

Shadow (gem install shadow) is a REST-ful record server -- a small Mongrel handler that allows you to manipulate the database remotely. It uses dynamic ActiveRecord classes that are created and trashed for each request.

Parallelization -- uses the Sun 1 grid engine that distributes shell scripts across 128 nodes. Used for job and backend processing.

bioruby/bioperl/biopython -- bioinformatics libraries in other languages -- bioruby is not complete, but we still want to use Ruby, so he looked at ways of integrating Ruby with other languages. No RubyInline for Perl or Python, no up-to-date direct/C bindings. He ended up building a socket-level interface into python.

Admin tools to consider -- streamlined, active_scaffold, autoadmin, Django (manage.py inspectdb; manage.py syncdb; manage.py runserver). (Wow, come to RailsConf, get a Django demo. Unexpected surprise!)

Extending Rails -- has_many_polymorphs for easy creation directed graphs

Frustrating AR tidbits: has_many_through has a huge case statement, with sql strings everywhere, and tightly intertwined classes. Ugh.

Scaling big webapps: AR/SQL is not the way. Instead, go to a hyper-denormalized model, where the DB is just a big hash. This leads to things like berkeleydb, memcached, madeleine, etc. and MySQL just becomes a persistence store for memcache. One key is moving joins at write-time, so that reads don’t need to re-join associations. You’re essentially duplicating/caching the data out to each association, but this makes sharding/splitting of data easier. Example: Flickr user photos vs. photos placed in a group.

Evan doesn’t believe that SQL is a viable data store for webapps -- I think he means large-scale webapps. Not everyone who’s trying to build a web application will run into these kinds of issues, so your mileage may vary. Still, it’s refreshing to see more people rebel against the incumbent 30-year gorilla of SQL.

Tags ,  | no comments | no trackbacks

RailsConf 2007 Opening Keynote: David Heinemeier Hansson

Posted by Nick Sieger Fri, 18 May 2007 17:10:39 GMT

Rails 2.0

Where we’ve been

David is surprised and proud of the community that we already have, and wants us to be comfortable with where we are, and not always looking toward the future. We have:

  • Million gem downloads
  • Hundreds of plugins
  • 10k users on the rubyonrails-talk mailing list
  • Ruby job descriptions (asking for 3 years RoR experience, longer than David)
  • Books, books, books (and not just English books, but non-English titles as well), surpassing VBA, Perl, and Python in book sales
  • IDEs from NetBeans, Borland, Aptana, etc.

Rails 2.0 is not going to be the “Unicorn”. It’s not going to be a total rewrite, it actually has a release schedule, it will not break backwards-compatibility. Instead, it will build upon what we already have, and continue the philosophy of building on what is useful and needed. In fact, 95% of what’s in 2.0 already works today, in the edge. Example, a simple controller that handles three formats of input/output, with a person resource for accessing the data from a remote server.

class PeopleController < ApplicationController
  ...
  def create
    @person = Person.create(...)
    respond_to do |format|
      format.html { redirect_to person_url(@person) }
      format.xml  { render :status => :created, :location => person_url(@person), ... }
      format.js {
        render :update do |js|
          ...
        end
      }
    end
  end
end

class Person < ActiveResource::Base
  self.site = "http://example.com/"
end

David then goes into a live demo of the new scaffold resource, which by appearance is identical to the old scaffolding, except it comes pre-baked with a REST-ful XML interface. He then adds support for a text format with a couple of lines of code, jumps into IRB, defines an active resource, and proceeds to change the data remotely.

If you want to add search to your controller, you can do it in a DRY way, and all the format/view work you’ve done will benefit:

class PeopleController < ApplicationController
  def index
    if params[:name]
      @people = Person.find :all, :conditions => ["name like ?", "#{params[:name]}%"]
    else
      @people = Person.find :all
    end
    ...
  end
end

David points out that 37signals, Shopify, Fluxiom, et. al. are real sites, with non-trivial domains that are still well executed in Rails, so it’s not just about simple scaffolding demos.

In Rails 2.0, ActiveResource will be bundled with Rails, and ActionWebService will not.

Friends of Rails

  • AJAX!
  • REST!
  • Atom? -- Atom should be more native to Rails
  • Openid? -- Openid is not necessarily something that needs to be used by all, but still a strong ally.

9 other things I like about Rails 2

  • Breakpoints are back -- no longer depends on Binding.of_caller; instead Rails depends and builds upon ruby-debug by Kent Sibilev.
  • HTTP Performance -- streamlining .js and .css, even though it feels better to break up Javascript and CSS into many little pieces, and gzip them
<%= javascript_include_tag :all, :cache => true %>
<%= stylesheet_link_tag :all, :cache => true %>

We can also fake out the browser and configure multiple asset hosts (4) you can maximize browser connections

config.action_controller.asset_host = 'assets%d.highrisehq.com'
  • Query cache
ActiveRecord::Base.cache do
  # actions here are cached
end
  • Rendering and MIME types -- bake the MIME convention into the template, and separate from the rendering mechanism people/index.html.erb people/index.xml.builder people/index.rss.erb people/index.atom.builder
  • config/initializers replacing config/environment. Initializers are .rb files in the config/initializers directory of your app that are automatically loaded during initialization time.
  • Sexy migrations
create_table :people do |t|
  t.integer :account_id
  t.string :first_name, :last_name, :null => false
  t.text :description
  t.timestamps
end
  • HTTP authentication (authenticate_or_request_with_http_basic, authenticate_with_http_basic)
  • The MIT assumption -- the licensing question -- make it easier to understand
  • Spring cleaning -- getting rid of the cruft -- stay tuned!

Tags ,  | 4 comments | no trackbacks

RailsConf Releases

Posted by Nick Sieger Thu, 17 May 2007 17:35:00 GMT

Just a quick update. Firstly, I just released ci_reporter 1.3; it should be available in the gem index shortly. Thanks to Bret Pettichord, Jeremy Beheler, and Charlie Kunz for reporting issues and prodding me to fix a couple of bugs. The two new items in this release are:

  • RSpec 0.9/trunk-compatible. You can now describe/it all you want with ci_reporter.
  • Errors and failure stack traces now include the full error message and exception type.

Secondly, JRuby 1.0RC2 has been released. Although there is no official release announcement at the moment, it is available for download and has been propagated to the central Maven repository also. Please do check it out and let us know on the mailing lists or in JIRA if you come across any blocker issues or regressions. Just a couple more weeks of stabilization; expect a rockin’ 1.0 release in June!

Lastly, expect an ActiveRecord-JDBC 0.3.2 release Real Soon Now.

Tags , , , ,  | no comments | no trackbacks

RailsConf Wrapup

Posted by Nick Sieger Thu, 29 Jun 2006 17:02:00 GMT

RailsConf has been over for three four days, and I’m just now flushing out my wrap-up? Better late than...whatever.

Keynotes

All the keynotes were exceptionally good. I had heard Paul Graham speak at OSCON last year so while the theme of his new talk was good, the controversy wasn’t that controversial to me.

Taking notes during keynote sessions is tough! With the exception of Dave Thomas, all the keynotes were in the evening, just when you’re ready to kick back with a beer (which we did on Saturday night during David’s talk). After a day full of sessions, your brain’s done and the best you can do is osmosis, or wait for the video! That said, here are a couple of points and quotes I managed to snag.

Martin Fowler’s talk

  • Rails does things a lot differently than many of the frameworks that were trying to become fashionable in enterprise application design
  • Opinionated software -- Rails does not claim to be the right framework for everything
  • Even if Rails doesn’t succeed or become ubiquitous, it has made an impact on the way applications are designed and built.
  • If you do something quick, it has to be dirty, and if you do something well, it has to take a long time. Rails breaks this dichotomy -- quick doesn’t have to be dirty.
  • Iterations and cycle time -- the faster a feature gets into production, the more engaged the customer is in future cycles. It introduces a conversational style of software development.
  • “I’m not saying Ruby is that much better than Python, it just suited me more.”
  • “This conference is a failure, really, because if Rails had succeeded it would be so simple that there would be nothing to talk about.”

David’s talk

David’s talk was subtitled “How I Learned to Stop Worrying and Love the CRUD” (slides now online). During the talk David describes the thought process he arrived at while attempting to boil down most operations in a Rails app to the simplest possible level of Create, Read, Update, and Delete. The table below illustrates the thinking:

GETPOSTPUTDELETE
findcreateupdatedestroy
SELECTINSERTUPDATEDELETE

But! CRUD is not a goal, it’s an aspiration, a design technique (quoted from the slides). I was reminded greatly of Eric Evans’ superb book Domain Driven Design as David implorded us to model relationships, events, and closures in addition to tangible nouns, as this makes relationship-building a CRUD operation as well. Adding a User to a Group is made simpler by simply creating a Membership rather than hanging additional non-CRUD operations off of both User and Group, which makes for DRY-er code as well.

As others have remarked, we need better jargon for CRUD.

Add to the conversation the newly unveiled ActiveResource, and suddenly Rails is an exciting integration platform!

Others

  • Stu Halloway’s MetaRails talk was a well-paced progression through some of the metaprogramming techniques, and hurt the brain less that Bill Katz’s talk. Stu mentioned how he runs the presentation slides in a Rails app that pulls snippets directly from the Rails codebase. Except now it appears that it will be broken without some intervention now that the Reloadable module has gone away.
  • Duncan’s talk on deployment introduced a not necessarily new, but great metaphor of the web as pipe and why we should kick our FCGI habit.

Summary

  • There is a lot of energy to be harnessed in the Rails community. This thing is just getting started.
  • There are a ton of smart people working out the few remaining kinks in Rails. Deployment is about to get ten times easier.
  • Mongrel is a mad pup. Don’t mess with him.
  • Rails will soon be a killer service platform. There are going to be Rails apps deployed all over the place that will be well integrated through REST interfaces while all the SOA people still argue about what SOA means.

Posted in ,  | Tags  | no comments | no trackbacks

RailsConf: Mike Pence - Laszlo on Rails

Posted by Nick Sieger Wed, 28 Jun 2006 15:00:00 GMT

Mike Pence, professional web surfer, and Java free since March 15, talked about Sex, drugs, rock and roll or Laszlo on Rails.

Where are we going on the web?

  • Google Maps, Yahoo Music Engine, Google Spreadsheets -- the web is looking more and more like a desktop application.
  • “Web two oh” -- attention to design and more attractive interfaces
  • User customization a.k.a. “Pimp my site”
  • Use of rich media on the web, e.g., YouTube. It’s an expectation of the next generation of users that the web will be content-rich and an entertainment experience.
  • The Holy Grail! Applications that require no downloads, are instant/automatically updated, are distributed.

Open Laszlo

  • Pandora cited as an introductory example
  • Mike gave a 10 minute overview of Laszlo using the Open Laszlo Explorer.
  • Laszlo explorer shows you standard widgetry -- canvas, text, buttons, windows, forms
  • The power of Laszlo starts to show with data sets, with convenient data binding utilities, an extensible object model, and a declarative style. Mike showed 10 lines of code with a checkbox that controlled the visibility of a window, without having to attach an event handler to the checkbox.
  • http://www.openlaszlo.org/ has the 10 minute overview (explorer) and many other demos including LZPIX, which Mike demoed.
  • The newest version of Laszlo has DHTML support that allows a flash app to be served as DHTML instead, with little difference. Laszlo gives you the power of one runtime that rises above browser incompatibilities.

Laszlo on Rails

  • Install # install laszlo gem install ropenlaszlo rails laszlo-app && cd laszlo-app ./script/plugin install svn://rubyforge.org/var/svn/laszlo-plugin/tags/openlaszlo
  • More info at http://laszlo-plugin.rubyforge.org/

Pros & Cons

  • Pro: Rich possibilities
    • Blogbox -- cross-site window
    • Publish and subscribe for chat and collaboration, event-driven updates
  • Pro: deep API
  • Pro: in-browser development, like Seaside
  • Con: Consumes resources
  • Con: Accessibility, printability and searchability are not its strengths
  • Con: mature, yet requires experimentation
  • Con: performance can be an issue, especially on some older platforms

Store it away -- Laszlo is a promising technology, it’s free and open source it’s here today, and it appears to be getting good at serving standards-based interfaces. When combined with Rails’ increasing support for RESTian interfaces, the task of building compatible, dynamic applications should only get easier.

Posted in ,  | Tags  | no comments | no trackbacks

RailsConf: Ezra Zygmuntowicz - Rails Deployment

Posted by Nick Sieger Sat, 24 Jun 2006 19:18:00 GMT

Ezra: Programming in Ruby for 3 years, Rails for 2. Just another Ruby hacker, he says. Right!

Yakima Herald

  • http://yakimaherald.com -- Local newspaper website
  • CMS
  • Lots of non-SQL content
  • Classified and obituary workflow
  • Internal credit card transaction

Data Aggregation

  • Mysql
  • XML feeds from AP
  • RSS and Atom feeds from parent newspaper
  • Proprietary database for articles
    • Yucky Vendorscript
    • Crusty OS 9 server!
    • Wrote DSL to emit script over a socket to remote machine

Deployment

  • Apache? Seems to be the industry standard...
class ApacheWithFCGI < ActiveRecord::Base
  has_many :random_500_errors
end
  • Lighttpd
    • Better FCGI support
    • More stable
  • Switched to standalone, external FCGIs with capistrano spinner/spawner/reaper tasks, and finally got stable.

Tuning

Shared hosting

  • Apache with ProxyPass/ProxyPassReverse talking to multiple lighty instances is about as good as it gets.
  • But let’s face it...shared hosting + Rails suck.
  • Only as reliable as the worst application on the server
  • Not always good neighbors
  • Resource constraints
  • Free-for-all

Ezra recommends a Xen VPS for Rails deployment. I’ve followed his advice; this blog is hosted on a Xen VPS at http://rimuhosting.com.

Deployment models

  • Starting point, web, app and db on the same server.
  • Next, move database to a separate server. Use the more powerful hardware to the db.
  • Third, split each role to a separate server. But now you need an NFS to serve static and cached contents. Ezra states you can go a long way with this setup
class FutureOfRailsDeployment < ActiveRecord::Base
  belongs_to :mongrel
end

Mongrel

  • Speaks plain old HTTP, so you can take advantage of existing tools for load balancing, proxying or caching
  • You can use frontends: Pound, Pen, Balance, Haproxy, Lighttpd (although mod_proxy needs to be fixed), or Apache 2.2 modproxybalancer
  • Mongrel cluster gem allows easy management of multiple mongrel processes. It’s straightforward to integrate with capistrano too.

Engine Yard

Ezra describes a new hosting service he’s been working on that will give you a greatly simplified but powerful and performant hosting environment for your Rails applications, including a web admin console and deployment directly from SVN revisions and tags.

Go to Engine Yard for a beta signup email now!

BackgroundDrb

  • Distributed Ruby server that manages long runnng tasks.
  • Uses a MiddleMan proxy/stub class that allows the Rails process to connect to the background server.
  • Can use it as a worker queue, (e.g. for sending email) or as a distributed memory cache.
  • You can even query for progress on the background workers and push the result all the way to the browser with an ajax progress bar.
  • The code for BackgroundDrb looks really clean! Go to http://backgroundrb.rubyforge.org/ for the nitty gritty.

Posted in ,  | Tags  | no comments | no trackbacks

RailsConf: Bill Katz - Metaprogramming Writertopia

Posted by Nick Sieger Sat, 24 Jun 2006 17:27:00 GMT

Bill Katz took a code-focused approach to explaining metaprogramming and DSLs.

Resources

Bill applies Paul Graham’s quote on bottom-up design and lisp as justification for using Ruby to program DSLs.

Authorization plugin

In search of a better authorization scheme, Bill wanted an elegant syntax using blocks to delineate sections of code that require specific permissions, and a more intuitive way of managing the data surrounding access controls.

current_user.has_role? "moderator"
current_user.has_role? "member", workshop
workshop.accepts_role? "member", user

current_user.is_moderator_of workshop
user.is_eligible_for? campbell_award

Implementing permit

class MeetingController < ApplicationController
  permit "rubyists and wanna_be_rubyists", :except => :public_text

We need a class method. Where to put it? How does this work? (Insert pretty diagram of instance, singleton class and superclass here.) Metaclassees are “anonymous”, you don’t see them by looking at class ancestors.

ActionController::Base.send(:include, Authorization::Base)
ActionView::Base.send(:include, Authorization::Base::ControllerInstanceMethods)

Singleton methods from the point of view of the object are the same as instance methods in the metaclass. See slides on “How do we permit?” for details. Bill chose to tuck the DSL-related methods into modules and used self.included, class_eval and other goodies to set up the methods in the correct places. There is more than way to do this but I suspect Bill chose to do it for purposes of cleanly separating concerns along the lines of where in the class hierarchy the methods live.

ActiveRecord::Base.send(:include,
  Authorization::ObjectRolesTable::UserExtensions,
  Authorization::ObjectRolesTable::ModelExtensions)

This loads an acts_as_authorizable plugin which, when used in a model, inserts an accepts_role family of methods for querying and setting user roles.

Bill blazed through a whole lot of pretty-looking code here for how to set up the DSL infrastructure. You’ll have to go over to the writertopia developer link to see it. Bill has said he’ll post his slides online as well.

Questions

  • Use of method_missing -- does that clash with ActiveRecord::Base usage of method_missing? No, as long as you use super to invoke the next method in the chain, you’re ok. Ruby module inclusion semantics handle this for you.
  • How to insert the class methods? A discussion started around ActionController::Base.send(:include, ...) and There is More Than One Way to Do It.
ActionController::Base.send(:include, Authorization::Base)

class ActionController::Base; include Authorization::Base; end

ActionController::Base.class_eval { include Authorization::Base }
  • Question about the class inheritance chain. There are actually two inheritance chains, one for the classes and one for the metaclasses. Resolution at method invocation time travels first to the object’s metaclass, then to the object’s class, then to the superclass of the singleton class, then the class’s superclass, etc. Blech, that makes no sense. Instead, go read Seeing Metaclasses Early.

Posted in ,  | Tags  | no comments | no trackbacks

RailsConf: Steven Hammond - DSLs and Rails Plugins

Posted by Nick Sieger Sat, 24 Jun 2006 16:18:00 GMT

Intro

Steven talks on the whats and whys of domain specific languages.

Steven’s background is is as a java developer and more recently, a project manager, or “sith lord” as he put it.

Some early exposure to DSLs for him involved the use of something called Program Composition Notation that was used to describe a weather modeling program to be run on massively-parallel supercomputers.

Features of DSLs

  • Usually a partial programming language focused on a single problem domain.
  • Replicate syntax/structure that is common to users of that domain.
  • Tries to be as elegant, concise and expressive as possible in that domain.

Horizontal vs. Vertical

  • Horizontal -- programming problems (e.g., Rails migrations)
  • Vertical -- business domain problems (e.g., a game DSL, insurance product definition, circuit board specification)

Recipe example

Steve contrasted a procedural, C-style recipe vs. a Ruby-based DSL that used blocks to delineate ingredients from procedures. The differentiating point was “which one could your mom translate into meatloaf”? I suppose neither if you don’t like your mom’s meatloaf.

Rails DSLs

  • The framework itself (“is the whale shark a fish?”)
  • Migrations
  • RJS templates
  • Plugins
    • acts_as_state_machine
    • RESTful Rails
    • game_dsl

When to create a DSL

  • established syntax or vertical domain
  • soemthing that’s inelegant in normal imperative or object-oriented style
  • abstract implementation details so that you don’t depend on them (e.g., migrations does this for database DDL)
  • reuse

DSLs and Reuse

  • “Reuse is vastly overrated” -- DHH in 1/2006
    • Business logic is very application specific and is difficult to reuse across projects
    • Level of reuse is at a lower lever where there is an information barrier to understanding the component (e.g., LINPACK)

Designing a DSL

  • Be careful to balance forward-looking with extraction. In other words, DSLs should be extracted or driven by need just like anything else.
  • Layout a domain syntax (not necessarily in Ruby)
  • Extract and evolve over time
  • Translate business requirements/statements into succinct programming fragments. (e.g., rolling dice and assigning attributes)
# extending Integer to deal with die-rolling
3.d6
3.d6 :keep_best => 3

# Deck-dealing example
class Tile < AciveRecord::Base
  acts_as_deck
end

class Player < ActiveRecord::Base
  acts_as_player
end

class DeckController < ApplicationController
  def deal_cards
    @deck = Tile.find :all
    @deck.shuffle
    @card = @deck.draw
    @deck.deal :cards => 6, :to => @players
  end
end

There was a discussion at this point about whether it was necessary to use a DSL in this case, and Obie Fernandez piped in that if you’re aiming for reuse or as an API, DSLs give a readable, understandable representation that makes reuse and sharing easier.

Another comment was regarding namespace collisions, and Steven replied honestly that there is not a good answer. If you combine a game-playing DSL with a home-building DSL, are you randomizing your porch?

YAML provides another good medium for succinct data layout (e.g., database.yml). Steven’s example followed the same pattern to apply to selecting different sources for a random number generator.

Pitfalls

  • DSLs should not necessarily be the first solution -- the domain could be too narrow or too broad.
  • Too many DSLs (or even plugin bloat) could end up compromising the readability of the code. Namespacing comes in here. Which DSL/plugin provided the given functionality?
  • Interaction side-effects -- it’s not clear how mixing metaphors can make the solution clearer.
  • Vertical DSLs additionally suffer from the problem that it’s still programming, and there are syntax issues to overcome, even though the DSL can faciliate interactions with the customer.

A comment came from Aslak Hellesoy about the use of external DSLs (i.e., built using parser generators). Steven replied that it is appropriate in some cases, but the complexity will be higher.

It seems to me that in the case of external DSLs, the freedom (lack of constraints) can actually be a burden. You can design syntax however you want but you need to be sure that it makes sense to the people who will be using it. Unless you’re really good with language and interface design it can be easy to create a custom, niche language that nobody knows anything about and is inflexible. In my experience it’s hard to use parser generators like yacc to build flexible parsers. Ruby turns out to be really good for DSLs because you can omit details (parentheses and hash curly braces) and it still does what you want.

Posted in ,  | Tags  | no comments | no trackbacks

Older posts: 1 2 3