RubyConf 2006 Begins
Posted by Nick Sieger Fri, 20 Oct 2006 19:04:00 GMT
The RubyConf room is filling up this morning. I’ll be doing my best to live-blog the conference here so stay tuned!
Auto RSpec
Posted by Nick Sieger Wed, 13 Sep 2006 20:35:00 GMT
Update: (2 months later) If you’re reading this, you’re probably interested in my Rails plugin for this instead.
Hot off the presses, after a few hours of hacking and tweaking, may I present Auto+RSpec, otherwise known as The Mashup of RSpec on Rails and autotest. This is not an official release of any sort, but “may work for you.” It’s not a clean hack, as it exposes some areas for autotest to grow if the maintainers decide to open it up to alternatives to Test::Unit. After spending a little time looking at the autotest code, I think it would be nice to allow hooks for autotest plugins to define project conventions (i.e., @exceptions and the #tests_for_file method) as well as a result parsing API.
For now, if you’re an RSpec on Rails user, you can try this out as follows:
- Install ZenTest if you haven’t already:
sudo gem install ZenTest. - Download rspec_autotest.rb and put in your
vendor/plugins/rspec/libdirectory (you did say you’re using RSpec on Rails didn’t you?) - Download rspec_autotest.rake and put in your
lib/tasksdirectory - Start
autotestwith rake by typingrake spec:autotest - Note: if you’re using RSpec 0.6, you might have better success with the files located here.
Next steps for this will be to work out whether this code should live in RSpec on Rails or autotest, or some combination of those.
Now, spec’ers, be off in search of that Red/Green/Refactor rhythm of which sage agilists speak!
Bonus tip: add the following code to your .autotest file to run spec with rcov:
Autotest.add_hook :initialize do |at|
# run spec with rcov
if at.respond_to? :spec_command
at.spec_command = %{rcov --exclude "lib/spec/.*" -Ilib --rails "/usr/lib/ruby/gems/1.8/gems/rspec-0.6.0/bin/spec" -- --diff}
end
endJRuby and Sun
Posted by Nick Sieger Fri, 08 Sep 2006 15:11:00 GMT
The noise is deafening by now, but I’m feeling the desire to chime in publicly with my congratulations and support for Charlie and Tom. JRuby has come a long way in the past six months, and this is strong validation of that fact.
As I hinted back in May, this is getting big, and it’s been a pleasure to have been on the JRuby train! The future is bright for Ruby and Java the platform, and JRuby is leading the way.
Ruby's Exception Hierarchy
Posted by Nick Sieger Wed, 06 Sep 2006 20:16:00 GMT
Today I needed to know the class hierarchy under Exception, and maybe it’s there online but I couldn’t find it. Blecch. Hint: Pickaxe, 2nd ed., page 462.
Well, you could always use Ruby itself, too, that way you’ll always have an up-to-date list:
exceptions = []
tree = {}
ObjectSpace.each_object(Class) do |cls|
next unless cls.ancestors.include? Exception
next if exceptions.include? cls
next if cls.superclass == SystemCallError # avoid dumping Errno's
exceptions << cls
cls.ancestors.delete_if {|e| [Object, Kernel].include? e }.reverse.inject(tree) {|memo,cls| memo[cls] ||= {}}
end
indent = 0
tree_printer = Proc.new do |t|
t.keys.sort { |c1,c2| c1.name <=> c2.name }.each do |k|
space = (' ' * indent); space ||= ''
puts space + k.to_s
indent += 2; tree_printer.call t[k]; indent -= 2
end
end
tree_printer.call treeException
NoMemoryError
ScriptError
LoadError
NotImplementedError
SyntaxError
SignalException
Interrupt
StandardError
ArgumentError
IOError
EOFError
IndexError
LocalJumpError
NameError
NoMethodError
RangeError
FloatDomainError
RegexpError
RuntimeError
SecurityError
SystemCallError
SystemStackError
ThreadError
TypeError
ZeroDivisionError
SystemExit
fatal
Results also entered into cheat; sudo gem install cheat --source require.errtheblog.com; cheat exceptions
for future reference.
Security Threat Last Week
Posted by Nick Sieger Tue, 15 Aug 2006 04:12:00 GMT
What was the biggest security threat story for me last week? No, it was not the disrupted liquid bomb plot, it was the Rails security hole that caused quite a brouhaha among the Ruby community. (Guess that shows my increasing tendency to lose touch with reality. Maybe a sign of the miserable state of unrest in the world and how living in the land of the world’s only super-power makes it easy to turn the other cheek? Or...ok, ok...it’s just me.)
From my view of the Rails security issue, there are actually quite a few interesting angles that came out of this story.
Rails is Growing Up
This is the obvious one. The first major fault to be discovered in Rails shows that Rails the codebase, Rails the core team, Rails the technology stack, and Rails the community is going through growing pains. David was both praised and criticized widely for his handling of the disclosure. Many rightly complained that the initial announcement didn’t give system maintainers enough information to decide whether the risk warranted disrupting normal operations to spend time to test and roll out the patch. This was compounded by the fact that the initial announcement did not identify versions affected and instead assumed all past versions, which turned out not to be the case.
Others thanked the Rails team for their discretion and trusted the recommendation despite the fuzziness and lack of details. These folks either were able to perform the upgrade much more easily or had some inkling of just how serious the issue was.
The aftermath showed that the Rails core quickly learned from the experience. A security mailing list and google group were set up for future incidents and David promised to apply more rigor and policy to future announcements.
It seems pretty obvious that the size of the gaffe was such that to expose the details immediately would have had way too much potential to cause widespread data loss and denial of service. In fact, the nature of the bug strikes me as one of those embarrassing bugs that every software developer commits at one point in their coding life where you amaze yourself at the short-sightedness of your implementation. I think the initial message could have been dispatched with information on the severity of the threat without necessarily disclosing the exact exploit. So, essentially I agree with the approach that was taken, but the message left out details required to evaluate the threat.
Threat Analysis
Two early blog posts came out the day after claiming to know the details of the exploit. It turned out that they didn’t quite understand what was afoot. (Although Evan Weaver has since updated his post to clarify his original analysis.)
The threat turned out to be a simple remote code execution issue. The
:controller dynamic expansion aspect of routing contained a bug that
allowed arbitrary .rb files in a Rails application to be executed
undesirably. By far the most dramatic consequence would be
experienced if one’s db/schema.rb file were to be executed with a
request for /db/schema, causing your entire database contents to be
dropped and reloaded.
By examining the safe_load_paths method defined in affected
versions, it appears that the implementation tried to limit elements
of the load path that matched the expanded RAILS_ROOT of the
application. Combine this with the fact that other elements of the
routing system eagerly require‘d files with inadequate
bounds-checking spells your recipe for disaster.
Many posters and commenters quipped that a simple svn diff was
enough to give script kiddies or other black hats the information
needed to exploit the issue. Or was it? Given that the two early
analyses turned out to be off the mark, were people in the know
exercising more discretion by not disclosing more details?
Personally, I spent more than an hour staring at the affected routing code trying to untangle the various metaprogramming tricks and regular expressions that make up the Rails routing system. And I consider myself fairly adept at reading and understanding code!
The truth of the matter is that, unless you’re a member of core or have a high level of familiarity and involvement with the Rails codebase, the svn diffs provide far too little context to decode the actual problem.
Does this speak to the obfuscated nature of the Rails codebase or to the relatively advanced nature of web programming in Ruby? If I had to pick one, it would be the latter, but I’m leaning towards neither. The Rails codebase is not the most readable, comprehensible piece of code I’ve ever seen, but it does its job remarkably well. Perhaps if the routing code in question was a bit more understandable by the masses, this rather obvious security issue wouldn’t have gone undetected for so long.
Post-1.1.6 Release Triage
A group of enthusiastic Railsers jumped onto #rails-security on freenode shortly after the 1.1.6 release, where an effort had been organized to verify all the patches across various combinations of web servers and Rails versions. An IRC channel, a wiki, Ruby, Zed’s RFuzz, and a piece of code were all the tools required to get a distributed test verification process up and running. This sort of thing happens all the time in the open source world, with programmers around the globe pitching in to raise the triage tent of the MASH unit. Still, it was exciting to see and be a part of the action and to be reminded of the power of the collective whole working for a common cause.
Dynamic Routing Harmful?
Rails’s dynamic routing code came under fire too, understandably so. Maybe this is one case where the developer-friendly approach of magically recognizing URLs goes a little too far? Production-only routes that do away with the expandable path elements could easily be generated by visiting all the controllers in the codebase and generating a more static route for each -- sounds like a good idea for a plugin. Perhaps the controller is the better place to store routing metadata anyway?
class UsersController < ActionController::Base
map_default_route # could be optional
end
class PostsController < ActionController::Base
map_route_as_resource
endSounds like good fodder for future investigation!
My version of MetaRails
Posted by Nick Sieger Wed, 26 Jul 2006 19:42:00 GMT
I gave my own version of a talk on MetaRails based on Stuart Holloway’s talk at RailsConf at the Ruby Users of Minnesota meeting last night. It was originally planned to be a regurgitation/sharing session, but since Stuart’s slides broke pretty badly shortly after he gave the talk, I created my own slides, which are available for download. I did pilfer some of Stuart’s content, specifically the theme/example tables, which I especially liked for their concise summary of various patterns you can see in the Rails codebase.
The slides aren’t quite as interesting without my narration, but maybe some of the pretty pictures in it will help with understanding singleton classes.
The first part of the talk did a slow buildup of the need for singleton classes in Ruby. I tried to do this by showing first the language syntax for doing various simple operations (e.g., defining classes and methods), followed by an equivalent programmatic approach to the same.
I’m pretty happy with how the talk went, but if I were to develop it further, I’d probably try to find a way to deepen the connections between the introductory material and uses of those in the Rails codebase. As it stands, the slides show several neat examples of metaprogramming in Rails along the same lines as what Stuart presented at RailsConf.
Thanks to Stuart for a great talk, and I hope you get some use out of my variation on the subject!
Metaprogramming Pattern No. 1: Self-specialize
Posted by Nick Sieger Tue, 11 Jul 2006 03:48:00 GMT
One of the biggest aspects of Ruby that I’ve been digging are the metaprogramming facilities, many of which draw from the “code as data” philosophy that comes from Lisp. Metaprogramming has become somewhat of a buzzword in the Ruby community, about as popular as “domain specific language” in terms of its presence in the titles of conference presentations and the like.
So it seems to me that, while a good many smart people are talking and writing about metaprogramming, that we haven’t yet started cataloguing all the different techniques in a shareable way. Is it time to start writing a catalog of metaprogramming patterns? Or do I risk being taken to trial for attempting to unveil the rubyist’s magic tricks?
No, what I am really looking for is the metaprogramming ubiquitous
language. Ruby has a lot of language-level features that
facilitate metaprogramming, but without a strong jargon to describe
what’s going on. Sorry, but module_eval, instance_eval, metaclass
vs. singleton class vs. eigenclass, method_missing etc. just don’t
cut it. So here’s a first shot at re-invigorating the conversation
and taking it to the “HNL” (‘hole nuther level). So herewith begins
Metaprogramming Pattern No. 1: Self-specialize.
Why is this one number 1? No reason, pretty arbitrary. I haven’t taken the time to document any more yet. This will be an ongoing project for me.
Self-specialize
You have an algorithm or a “method object” that you wish to make flexible by parameterizing with additional code and/or data that isn’t known at the time you wrote the original class definition.
Therefore, write a method that redefines itself in the singleton class of the currently instantiated object, then re-sends the message to the customized method. In a way, this is simply memoizing the method body itself as a speed-up.
Trivial example:
class Foo
def initialize(p)
@prefix = p
end
def result(val)
specialize_result
result val # send the message again
end
private
def specialize_result
method_decl = "def result(val); "#{@prefix}: #{val}"; end"
instance_eval method_decl, "generated code (#{__FILE__}:#{__LINE__})"
end
end
f = Foo.new("foo")
f.result("hi") # => "foo: hi"
g = Foo.new("bar")
g.result("hi") # => "bar: hi"Why would you do this rather than writing the logic into the original method? It’s hard to justify use of the technique in the example above. But it could be used to DRY up tedious, redundant code that for performance reasons you would prefer to have inlined rather than invoking an additional instance method.
OK, so honestly I can’t come up with a decent reason to do this yet; I haven’t done enough metaprogramming to have had the need for it. Still, it seems like a nifty enough trick and maybe it will come in handy for you. This does show you just how dynamic Ruby’s method resolution is, that you can suddenly define and call a different method implementation inside of the method itself!
This technique was spotted in the rewritten routes implementation for
Rails 1.2 (currently on the trunk) -- see the #write_generate
and #write_recognize methods.
Update: Good timing. _why just posted much more succinct description of the same phenomenon, with better examples.
RailsConf Wrapup
Posted by Nick Sieger Thu, 29 Jun 2006 17:02:00 GMT
RailsConf has been over for three four days, and I’m just now flushing out my wrap-up? Better late than...whatever.
Keynotes
All the keynotes were exceptionally good. I had heard Paul Graham speak at OSCON last year so while the theme of his new talk was good, the controversy wasn’t that controversial to me.
Taking notes during keynote sessions is tough! With the exception of Dave Thomas, all the keynotes were in the evening, just when you’re ready to kick back with a beer (which we did on Saturday night during David’s talk). After a day full of sessions, your brain’s done and the best you can do is osmosis, or wait for the video! That said, here are a couple of points and quotes I managed to snag.
Martin Fowler’s talk
- Rails does things a lot differently than many of the frameworks that were trying to become fashionable in enterprise application design
- Opinionated software -- Rails does not claim to be the right framework for everything
- Even if Rails doesn’t succeed or become ubiquitous, it has made an impact on the way applications are designed and built.
- If you do something quick, it has to be dirty, and if you do something well, it has to take a long time. Rails breaks this dichotomy -- quick doesn’t have to be dirty.
- Iterations and cycle time -- the faster a feature gets into production, the more engaged the customer is in future cycles. It introduces a conversational style of software development.
- “I’m not saying Ruby is that much better than Python, it just suited me more.”
- “This conference is a failure, really, because if Rails had succeeded it would be so simple that there would be nothing to talk about.”
David’s talk
David’s talk was subtitled “How I Learned to Stop Worrying and Love the CRUD” (slides now online). During the talk David describes the thought process he arrived at while attempting to boil down most operations in a Rails app to the simplest possible level of Create, Read, Update, and Delete. The table below illustrates the thinking:
| GET | POST | PUT | DELETE |
| find | create | update | destroy |
| SELECT | INSERT | UPDATE | DELETE |
But! CRUD is not a goal, it’s an aspiration, a design technique (quoted from the slides). I was reminded greatly of Eric Evans’ superb book Domain Driven Design as David implorded us to model relationships, events, and closures in addition to tangible nouns, as this makes relationship-building a CRUD operation as well. Adding a User to a Group is made simpler by simply creating a Membership rather than hanging additional non-CRUD operations off of both User and Group, which makes for DRY-er code as well.
As others have remarked, we need better jargon for CRUD.
Add to the conversation the newly unveiled ActiveResource, and
suddenly Rails is an exciting integration platform!
Others
- Stu Halloway’s MetaRails talk was a well-paced
progression through some of the metaprogramming techniques, and hurt
the brain less that Bill Katz’s talk. Stu mentioned how he runs the
presentation slides in a Rails app that pulls snippets directly from
the Rails codebase. Except now it appears that it will be broken
without some intervention now that the
Reloadablemodule has gone away. - Duncan’s talk on deployment introduced a not necessarily new, but great metaphor of the web as pipe and why we should kick our FCGI habit.
Summary
- There is a lot of energy to be harnessed in the Rails community. This thing is just getting started.
- There are a ton of smart people working out the few remaining kinks in Rails. Deployment is about to get ten times easier.
- Mongrel is a mad pup. Don’t mess with him.
- Rails will soon be a killer service platform. There are going to be Rails apps deployed all over the place that will be well integrated through REST interfaces while all the SOA people still argue about what SOA means.
RailsConf: Mike Pence - Laszlo on Rails
Posted by Nick Sieger Wed, 28 Jun 2006 15:00:00 GMT
Mike Pence, professional web surfer, and Java free since March 15, talked about Sex, drugs, rock and roll or Laszlo on Rails.
Where are we going on the web?
- Google Maps, Yahoo Music Engine, Google Spreadsheets -- the web is looking more and more like a desktop application.
- “Web two oh” -- attention to design and more attractive interfaces
- User customization a.k.a. “Pimp my site”
- Use of rich media on the web, e.g., YouTube. It’s an expectation of the next generation of users that the web will be content-rich and an entertainment experience.
- The Holy Grail! Applications that require no downloads, are instant/automatically updated, are distributed.
Open Laszlo
- Pandora cited as an introductory example
- Mike gave a 10 minute overview of Laszlo using the Open Laszlo Explorer.
- Laszlo explorer shows you standard widgetry -- canvas, text, buttons, windows, forms
- The power of Laszlo starts to show with data sets, with convenient data binding utilities, an extensible object model, and a declarative style. Mike showed 10 lines of code with a checkbox that controlled the visibility of a window, without having to attach an event handler to the checkbox.
- http://www.openlaszlo.org/ has the 10 minute overview (explorer) and many other demos including LZPIX, which Mike demoed.
- The newest version of Laszlo has DHTML support that allows a flash app to be served as DHTML instead, with little difference. Laszlo gives you the power of one runtime that rises above browser incompatibilities.
Laszlo on Rails
- Install # install laszlo gem install ropenlaszlo rails laszlo-app && cd laszlo-app ./script/plugin install svn://rubyforge.org/var/svn/laszlo-plugin/tags/openlaszlo
- More info at http://laszlo-plugin.rubyforge.org/
Pros & Cons
- Pro: Rich possibilities
- Blogbox -- cross-site window
- Publish and subscribe for chat and collaboration, event-driven updates
- Pro: deep API
- Pro: in-browser development, like Seaside
- Con: Consumes resources
- Con: Accessibility, printability and searchability are not its strengths
- Con: mature, yet requires experimentation
- Con: performance can be an issue, especially on some older platforms
Store it away -- Laszlo is a promising technology, it’s free and open source it’s here today, and it appears to be getting good at serving standards-based interfaces. When combined with Rails’ increasing support for RESTian interfaces, the task of building compatible, dynamic applications should only get easier.
ActiveRecord JDBC adapter as Rails plugin
Posted by Nick Sieger Mon, 26 Jun 2006 17:53:00 GMT
I’ve got things set up in my svn repo such that you pull down my ActiveRecord JDBC adapter as a Rails plugin. Although it appears that using ‘script/plugin’ inside of JRuby may have some issues. So for now, use C Ruby. Inside your Rails app, do:
ruby script/plugin install http://svn.caldersphere.net/svn/main/activerecord-jdbc/trunk/activerecord_jdbc/This should pull down and configure the JDBC adapter for you with no additional setup. If it doesn’t, let me know and we’ll work through it. I haven’t yet tried running a Rails app inside of JRuby yet but I hope to in the next couple of days.
I am also starting to investigate testing with another database, hsqldb. I think the driver could benefit from attempting to use some additional databases, if we’re ever going to fulfill the promise of leveraging any JDBC data source, and also I think it would be cool to use an embedded database in the spirit of SQLite which has become popular with smaller Rails apps in C Ruby-based Rails land.
Update: it turns out you can’t easily install a new ActiveRecord adapter as a Rails plugin at this point with out some extra finagling, because of the way Rails::Initializer initializes the database before any of the plugins. For now, I’ve got the plugin set up to re-initialize all of the ActiveRecord infrastructure, so in order to use the JDBC adapter plugin, you’ll need to add the following to your config/environment.rb. Note that you’re not “skipping” ActiveRecord, just initializing it later.
# Skip frameworks you're not going to use
config.frameworks -= [:active_record]