Nick Sieger: Tag railsconfdo what you lovetag:blog.nicksieger.com,2005:TypoTypo2007-07-13T09:45:27+00:00Nick Siegerurn:uuid:40171862-80d2-460f-9ad6-ead9fe29fd912007-05-19T20:34:00+00:002007-07-13T09:45:27+00:00RailsConf 2007: Chris Wanstrath: Kickin' Ass with Cache-fu<p>Chris is here to talk about games, since he used to work for Gamespot. He coded PHP, which is like training wheels without the bike. He had to sit in a glass cube and help keep the site running during E3 last year. There were 100 gajillion teenage boys during their lunch break hitting refresh, and it all blew up. Couldn’t even gzip the responses, because the servers heated up to much. They served 50M pages in a day, without downtime. They did it with Memcache.</p>
<p>Memcache is a distributed hash – multiple daemons running on different servers. Developed by Livejournal for their infrastructure, you just put up the servers, and they just work.</p>
<p>Should you use Memcache? No. <a href="http://c2.com/xp/YouArentGonnaNeedIt.html">YAGNI</a>, UYRDNI (unless you really do need it).</p>
<h2>Rails and Memcache</h2>
<p>Fragments, Actions, Sessions, Objects, cache it all. You can use:</p>
<ul>
<li><code>memcache-client</code> (by Robot-coop guys/Eric Hodel). Marshal.unload is 40 times faster than Object.new/loading from the database.</li>
<li>CachedModel – integration with ActiveRecord</li>
<li>Fragment Cache Store</li>
<li>Memcache session store</li>
</ul>
<p>…or…</p>
<h2><code>cache_fu</code></h2>
<p>Or, <code>acts_as_cached</code>. It knows about all the aforementioned objects, with a single YAML config file (<code>config/memcached.yml</code>). Word to the wise: don’t use names in your server config file. Use IPs, avoid BIND and connections to the servers with every connection. Don’t let DNS outages bring down your servers.</p>
<ul>
<li><code>get_cache</code></li>
<li><code>expire_cache</code></li>
</ul>
<p>This is all you need – if you’re using <code>set_cache</code>, you probably don’t understand how the plugin works. Expire cache on the “after save” hook, which allows you to cache ID misses as well.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Presentation</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">acts_as_cached</span>
<span class="ident">after_save</span> <span class="symbol">:expire_cache</span>
<span class="keyword">end</span></code></pre></div>
<p>Example: only cache published items</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Presentation</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">acts_as_cached</span> <span class="symbol">:conditions</span> <span class="punct">=></span> <span class="punct">'</span><span class="string">published = 1</span><span class="punct">'</span>
<span class="keyword">end</span></code></pre></div>
<p>Cached-scoped-finders (if somebody thinks of a good name, let Chris know). The idea is to move custom finder logic to a method on your model, and then wrap a cache-scoping thingy around it. <code>cache_fu</code> ties this up nicely by giving you a <code>cached</code> method on AR::Base.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Topic</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="keyword">def </span><span class="method">self.weekly_popular</span>
<span class="constant">Topic</span><span class="punct">.</span><span class="ident">find</span> <span class="symbol">:all</span><span class="punct">,</span> <span class="punct">...</span>
<span class="keyword">end</span>
<span class="keyword">end</span>
<span class="constant">Topic</span><span class="punct">.</span><span class="ident">cached</span><span class="punct">(</span><span class="symbol">:weekly_popular</span><span class="punct">)</span></code></pre></div>
<p>Adding date to cache key with <code>alias_method_chain</code>:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">def </span><span class="method">self.cache_key_with_date</span><span class="punct">(</span><span class="ident">id</span><span class="punct">)</span>
<span class="punct">...</span>
<span class="keyword">end</span>
<span class="keyword">class </span><span class="punct"><<</span> <span class="constant">self</span>
<span class="ident">alias_method_chain</span> <span class="symbol">:cache_key</span><span class="punct">,</span> <span class="symbol">:date</span>
<span class="keyword">end</span></code></pre></div>
<p>Cached loads by ID: <code>Topic.find(1, 2, 3)</code> moves to <code>Topic.get_cache(1, 2, 3)</code>, which can parallelize calls to memcached and bring them back as they’re ready.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">user_ids</span> <span class="punct">=</span> <span class="attribute">@topic</span><span class="punct">.</span><span class="ident">posts</span><span class="punct">.</span><span class="ident">map</span><span class="punct">(&</span><span class="symbol">:user_id</span><span class="punct">).</span><span class="ident">uniq</span>
<span class="attribute">@users</span> <span class="punct">=</span> <span class="constant">User</span><span class="punct">.</span><span class="ident">get_cache</span><span class="punct">(</span><span class="ident">user_ids</span><span class="punct">)</span></code></pre></div>
<p>You can also cache associations, so that you’re navigating associations via Memcache.</p>
<p>Cache overrides</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">ApplicationController</span> <span class="punct"><</span> <span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">before_filter</span> <span class="symbol">:set_cache_override</span>
<span class="keyword">def </span><span class="method">set_cache_override</span>
<span class="constant">ActsAsCached</span><span class="punct">.</span><span class="ident">skip_cache_gets</span> <span class="punct">=</span> <span class="punct">!!</span><span class="ident">params</span><span class="punct">[</span><span class="symbol">:skip_cache</span><span class="punct">]</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p><code>reset_cache</code>: Slow, uncached operations can sometimes queue up and wedge a site. Instead, issue cache resets on completion of a request, rather than expiring beforehand. That way, requests that continue to pile up will still use the cached copy until the rebuild is complete.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Presentation</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">after_save</span> <span class="symbol">:reset_cache</span>
<span class="keyword">end</span></code></pre></div>
<p>Versioning: a way to expire cache on new code releases</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Presentation</span> <span class="punct"><</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span>
<span class="ident">acts_as_cached</span> <span class="symbol">:version</span> <span class="punct">=></span> <span class="number">1</span>
<span class="keyword">end</span></code></pre></div>
<p>Deployment: Chris recommends using Monit to ensure your Memcache servers are up.</p>
<p><code>libketama</code>: consistent hashing that gives you the ability to redeploy Memcache servers without invalidating all the keys.</p>
<p>Q: Page caching? A: Nginx with native Memcache page caching, but outside of Rails domains.</p>
<p>Lots of other questions, but dude, Chris talks too fast!</p>Nick Siegerurn:uuid:f0f0e50c-df6a-47dc-81fe-0845c594512c2007-05-19T19:30:06+00:002007-07-13T09:45:27+00:00RailsConf 2007: Bradley Taylor: Virtual Clusters<p>How does Rails figure into virtualization? Bradley will cover this topic with examples and case studies. Along the way, hardware items may be mentioned, but are not critical. Really, it’s about the design of the clusters, not the bits of plumbing you use to connect them up.</p>
<p>Virtualization is partitioning of physical servers that allow you to run multiple servers on it. Xen, Virtuozzo, VMWare, Solaris containers, KVM, etc. Bradley uses Xen. The virtual servers share the same processor (hopefully multi-core), memory, storage, network cards (but with indepenent IP addresses), etc., but run independently of each other. VPS, slice, container, accelerator, VM, it’s all the same. Memory, storage, and CPU can be guaranteed with the virtualization layer.</p>
<p>Why would you do this? <em>Consolidate</em> servers for less hardware and cost; <em>Isolate</em> applications – bad apps don’t drag the server down, contain intrusions, use different software stacks; <em>Replicate</em> – easily create new servers and deploy in a standardized and automated way; <em>Utilize</em> – take advantage of all CPU, memory, storage, resources; <em>Allocate</em> resources, give a server exactly what it requires, grow/shrink up and down, and balance them. Bradley says, “Once you go to virtualization you won’t want to go back. Do the simplest thing that could possibly work.”</p>
<p>Virtual clusters, then, are a bunch of servers cooperating toward a common goal – if you have many versions or copies of one thing. More than one customer, more than one version of software, etc.</p>
<p>For Rails, this means a lot of things: you can have many development environments and stages, take advantage of memory isolation, protect against PHP/Java, and make multiple-server scaling accessible.</p>
<h2>Examples</h2>
<ul>
<li>Two servers for production and staging</li>
<li>Three for web/db/staging</li>
<li>Mixed languages – instead of 1x1GB server use 3x300MB servers</li>
<li>High availability applications with fewer servers</li>
<li>Multiple applications – one server per application</li>
<li>Standardized roles/appliances – mail, ftp, dns, web, db</li>
</ul>
<h2><a href="http://www.eastmedia.com/">EastMedia</a></h2>
<ul>
<li>They can incubate customers in separate images</li>
<li>Dev/staging/production servers</li>
<li>Shared SVN/trac</li>
<li>2 physical servers => 8 virtual servers</li>
</ul>
<h2>Boom Design</h2>
<ul>
<li>Again, multiple stages</li>
<li>Customer staging, with lower uptime requirements</li>
<li>Low-traffic apps on a single server, but everything else gets its own dedicated server</li>
<li>2GB memory spread across 9 virtual servers</li>
</ul>Nick Siegerurn:uuid:c67fa477-41f0-4798-9fa2-2c27ae65e5372007-05-19T17:22:42+00:002007-07-13T09:45:27+00:00RailsConf 2007: Saturday Morning Keynotes<p><strong>Cyndi Mitchell – ThoughtWorks Studios</strong></p>
<p>Enterprise (the “e” word)</p>
<p>Before IT got involved, “enterprise” was a bold new venture. Toyota manufacturing, Skype disruption of telephony.</p>
<p>Enterprise in terms of IT has come to mean bloatware, incompetence, corruption, waste of time, no value.</p>
<p>So this is the battle: The enterprise (to boldly go where no man has gone before) we need to reclaim vs. the bloatware/competence/corruption/fear-based selling etc.</p>
<p>RubyWorks – package stack with haproxy, mongrel, monit through an RPM repository</p>
<p>For JRuby support, call <a href="http://ola-bini.blogspot.com/">Ola</a>.</p>
<p><strong>Tim Bray – Web Guy from Sun Microsystems</strong></p>
<p>Change the world that are better than just using a cool web framework: http://pragmaticstudio.com/donate/</p>
<p>Sun loves Ruby. Ruby <em>and</em> Rails, that is. The impact of the Ruby language is going to be at least as big as Rails is for web development.</p>
<p>Sun provided servers for Ruby 2.0 development, and can provide servers for your potentially cool, worthy, open source project, just drop Tim an email.</p>
<p>A few more obligatory plugs for NetBeans and Sun sponsoring the conference. “Pre-alpha,” he says. Hmm, I wonder what <a href="http://blogs.sun.com/tor/">Tor</a> would say about that!</p>
<p>JRuby: when would you use JRuby vs. Ruby? If you have no pain, keep using C Ruby. But if you have management concerns, deployment concerns, etc. then by all means do try it!</p>
<p>Obligatory handshake/sandal connection with ThoughtWorks and Cyndi – running <a href="http://studios.thoughtworks.com/2007/5/7/mingle-to-run-on-jruby">Mingle</a> (and cruisecontrol.rb) with JRuby.</p>
<p>Sun: “Hi, the answer is Java, what was the question?” So why would Sun want to support Ruby? Well, you guys are programmers. Programmers who deliver quality software fast. And those programmers need computers, and OSes, and web servers, and support and services, etc. Plug, plug, plug.</p>
<p>How do you make money on free products? Sun has open-sourcing Java, Solaris, even Sparc. Joyent is open-sourcing their stuff. Where does the money come from? 1. Adoption 2. Deployment 3. Monetization at the point of value</p>
<p>What if we win? Are our problems over? No, we’ll have to deal with Java. And .NET. And PHP. <em>From the audience: And COBOL.</em> The Network Is The Computer. The Network Is Heterogeneous. <strong>Deal with it.</strong> So how do we interoperate?</p>
<ul>
<li>Just Run Java (and JRuby, of course!, and JavaScript, and PHP, etc.)</li>
<li>Use Atom/REST. Everything should have a publish button. <em>Don’t use WS-DeathStar or WCF or WSIT.</em></li>
</ul>
<p>Developer issues: Scaling, Static vs. Dynamic, Maintainability, Concurrency, Tooling, Integration, Time to Market. Which two of these matter the most?</p>
<p>Tim’s final assertion: Maintainability and Time to Market, and that’s why we’re all at RailsConf.</p>Nick Siegerurn:uuid:f408e374-3a8c-4436-b31f-cf1d3fe7e4be2007-05-18T19:33:31+00:002007-07-13T09:45:27+00:00RailsConf 2007: Evan Weaver: Going Off Grid<p>Evan is talking about leaving Rails as a full-stack framework and remixing bits and pieces for integration projects. He’s doing it in the context of a case study on Bio: a project at the University of Delaware working with DNA data in large SQL databases. Evan states that all of bioinformatics is an integration problem. (Me: That’s probably true of any research project where data is coming from multiple, varied sources. So where does Rails fit in this?)</p>
<p>So how do you cope with this? Use the Rails console as an admin interface, mapping AR onto the legacy schema.</p>
<p>Shadow (<code>gem install shadow</code>) is a REST-ful record server – a small Mongrel handler that allows you to manipulate the database remotely. It uses dynamic ActiveRecord classes that are created and trashed for each request.</p>
<p>Parallelization – uses the Sun 1 grid engine that distributes shell scripts across 128 nodes. Used for job and backend processing.</p>
<p>bioruby/bioperl/biopython – bioinformatics libraries in other languages – bioruby is not complete, but we still want to use Ruby, so he looked at ways of integrating Ruby with other languages. No RubyInline for Perl or Python, no up-to-date direct/C bindings. He ended up building a socket-level interface into python.</p>
<p>Admin tools to consider – streamlined, active_scaffold, autoadmin, Django (<code>manage.py inspectdb; manage.py syncdb; manage.py runserver</code>). (Wow, come to RailsConf, get a Django demo. Unexpected surprise!)</p>
<p>Extending Rails – <code>has_many_polymorphs</code> for easy creation directed graphs</p>
<p>Frustrating AR tidbits: <code>has_many_through</code> has a huge case statement, with sql strings everywhere, and tightly intertwined classes. Ugh.</p>
<p>Scaling big webapps: AR/SQL is not the way. Instead, go to a hyper-denormalized model, where the DB is just a big hash. This leads to things like berkeleydb, memcached, madeleine, etc. and MySQL just becomes a persistence store for memcache. One key is moving joins at write-time, so that reads don’t need to re-join associations. You’re essentially duplicating/caching the data out to each association, but this makes sharding/splitting of data easier. Example: Flickr user photos vs. photos placed in a group.</p>
<p>Evan doesn’t believe that SQL is a viable data store for webapps – I think he means large-scale webapps. Not everyone who’s trying to build a web application will run into these kinds of issues, so your mileage may vary. Still, it’s refreshing to see more people rebel against the incumbent 30-year gorilla of SQL.</p>Nick Siegerurn:uuid:032b3183-e579-4896-a439-2f62ee8279e62006-06-23T20:40:00+00:002007-08-31T16:41:56+00:00RailsConf: Stefan Kaes - Rails Performance<p>Stefan starts by citing a factor of 4-5 improvement in performance in
Rails over the last year.</p>
<h2>Performance, broken down</h2>
<ul>
<li>Latency – how fast</li>
<li>Throughput – how many</li>
<li>Utilization – how idle is the cpu</li>
<li>Cost efficiency – performance per unit cost</li>
</ul>
<p>For completeness calculate the min, max, mean and standard deviation
of these metrics and use the deviation as your guide for how reliable
the data is.</p>
<h2>Tools</h2>
<ul>
<li>Log files (level > <code>Logger::DEBUG</code>)</li>
<li><a href="http://rails-analyzer.rubyforge.org/">Rails Analyzer Tools</a> (Eric Hodel)</li>
<li>Benchmarker (<code>script/benchmarker</code>)</li>
<li>DB vendor tools</li>
<li>Apache bench (<code>ab</code> or <code>ab2</code>)</li>
<li>httperf</li>
<li><a href="http://rubyforge.org/projects/railsbench">railsbench</a> (Stefan Kaes)</li>
</ul>
<h2>Railsbench</h2>
<p>Railsbench measuress raw performance of rails request processing.
It’s configured using <code>config/benchmarks.yml</code> and
<code>config/benchmarks.rb</code>. These files let you control which requests
get benchmarked, whether to create a new session when benchmarking
them, etc.</p>
<h2>Profiling Tools</h2>
<ul>
<li>Ruby Profiler</li>
<li>Zen Profiler</li>
<li>rubyprof</li>
<li>Rails profiler script</li>
<li><a href="http://www.softwareverify.com/rubyPerformanceValidator/index.html">Ruby Performance Validator</a></li>
</ul>
<p>At this point Stefan gave an overview of RPV, which appears to be a
nifty tool that lets you get typical hotspot tree views of where time
is spent in code. It currently only runs on Windows.</p>
<h2>Top Rails Performance Problems</h2>
<ul>
<li>slow helper methods</li>
<li>complicated routes</li>
<li>associations – navigating and eager loading vs. proxy loading</li>
<li>retrieving too much data from the DB</li>
<li>slow session storage (e.g., ActiveRecord store)</li>
</ul>
<p>Stefan says that in his experience, DB performance is generally not a
big factor or bottleneck. Instantiating ActiveRecord objects is
expensive, though.</p>
<h2>Session containers</h2>
<ul>
<li>In memory – if you server crashes…oops. Also doesn’t scale.</li>
<li>File system – easy to set up, scales with NFS, but slower than…</li>
<li>ActiveRecordStore – easy to set up since it comes with Rails, but
much slower than…</li>
<li>SQLSessionStore – which uses the same table structure as
ActiveRecordStore, but was written by Stefan to overcome performance
issues with ActiveRecordStore. Setup is more involved.</li>
<li>memcached – slightly faster than SQLSessionStore, scales best, but
setup is also more involved.</li>
<li>DrbStore – distributed ruby store</li>
</ul>
<h2>Caching</h2>
<ul>
<li>Full pages – fastest, complete pages are served on the
filesystem. Web server bypasses appserver for rendering. If you
have private pages, you can’t use it.</li>
<li>Actions – pages are cached after an action is rendered. The
user ID can be used as part of the storage key.</li>
<li>Fragments – fragments can be cached in memory, on the file system,
in a DrbStore, or in memcached. Memcached scales the best but
doesn’t support expiring fragments by regular expression.</li>
</ul>
<h2>ActionController</h2>
<ul>
<li>Stefan recommends avoiding components, and replacing them with
helpers or partials. He has not found a use for them.</li>
</ul>
<h2>ActionView</h2>
<ul>
<li>Don’t create unnecessary instance variables in the controller;
creating them in the view with <code>instance_variable_set</code> and accessing
with <code>instance_variable_get</code> is slow.</li>
</ul>
<h2>Helpers</h2>
<ul>
<li>pluralize – don’t use the inflector if you don’t need to, it’s expensive.</li>
<li>link<em>to and url</em>for are among the slowest helpers, since they need
to use routes. Instead, if you have page with lots of links, you
might consider hard-coding the links. This reduces the amount of GC
by up to 50% and the GC time down by a few percentage points (11.3%
to 8.7% of total processing time).</li>
</ul>
<h2>ActiveRecord</h2>
<ul>
<li>use the <code>:include</code> option to prefetch associations, it avoids extra
onesy-twosy SQL statements.</li>
<li>use piggy-backing plugin for <code>has_one</code> or <code>belongs_to</code> relationships
– allows you to retrieve extra attributes from additional tables in
the same fetch query.</li>
<li>Field values are retrieved from the DB mostly as strings, so type
conversion happens on each access, which can be slow.</li>
</ul>
<h2>Language-level and miscellaneous issues</h2>
<ul>
<li>Method calls are the slowest – don’t needlessly create method
abstractions</li>
<li>Short-circuit intermediate results to improve performance</li>
<li>Cache results in instance variables or class variables</li>
<li>Don’t call <code>ObjectSpace.each_object</code> on each request</li>
</ul>
<h2>Ruby Memory Management</h2>
<ul>
<li>designed for batch scripts, not long-running servers.</li>
<li>no generational garbage collection.</li>
<li>this is suboptimal for Rails because ASTs are stored on the heap
(biggest portion of non-garbage for Rails apps), and get
processed/traversed more often than they need to be</li>
<li><a href="http://rubyforge.org/projects/railsbench">Railsbench</a> includes a patch to allow one to recompile Ruby
and tweak the garbage collector.</li>
</ul>
<h2>Rails Template Optimizer</h2>
<ul>
<li>Stefan has started a project to “compile” templates.</li>
<li>The idea is to cache results of some ERb scriptlets and essentially
“compile” or replace the template with one that has more expressions
expanded or inlined.</li>
<li>Code forthcoming; I assume you can stay tuned to <a href="http://railsexpress.de/blog/">Rails
Express</a> for news.</li>
</ul>
<h2>Questions</h2>
<ul>
<li>There was a question on JRuby – Stefan replied that it would
certainly solve GC issues, but he doesn’t know if it’s in a state to
be able to benchmark Rails requests.</li>
<li>What are your recommendations for a web server. <em>I don’t have
any.</em></li>
<li>Is horizontal or vertical scaling better? <em>I don’t know, I’ve
been focused on making single requests go fast, so I don’t have
enough experience.</em></li>
</ul>Nick Siegerurn:uuid:fd4b197e-ec51-47aa-bc0c-7e90a4339f192006-06-23T16:45:00+00:002007-08-31T16:41:56+00:00RailsConf: Mike Clark -- Introduction to Capistrano<h2>Introduction to Capistrano</h2>
<p>Mike Clark has been Java-free for 15 months and 16 days. He’s here to
talk about deployment with Capistrano.</p>
<p>First off, props to Jamis! There are several hundred people in the
room and it’s a tribute to him.</p>
<p>Capistrano is about making it easy to deploy web applications. “Cap”,
the short name has quickly become jargon in the Rails community.
“Stop wasting time, just cap it!”</p>
<p>Capistrano is built to scale. From a single machine all the way up to
clusters, it is intended to make application deployment as simple as
the push of a button. That’s priceless for reducing friction as your
project nears its first production live date, as well as subsequent
maintenance releases.</p>
<h3>Configuring Capistrano - create a recipe</h3>
<h4>Set the application name</h4>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">set</span> <span class="symbol">:application</span><span class="punct">,</span> <span class="punct">"</span><span class="string">depot</span><span class="punct">"</span></code></pre></div>
<h4>Set the repository</h4>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">set</span> <span class="symbol">:repository</span><span class="punct">,</span> <span class="punct">"</span><span class="string">http://svn.yourhost.com/<span class="expr">#{application}</span>/trunk</span><span class="punct">"</span></code></pre></div>
<h4>Roles</h4>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">role</span> <span class="symbol">:app</span><span class="punct">,</span> <span class="punct">"</span><span class="string">app01.example.com</span><span class="punct">",</span> <span class="punct">"</span><span class="string">app02.example.com</span><span class="punct">"</span>
<span class="ident">role</span> <span class="symbol">:web</span><span class="punct">,</span> <span class="punct">"</span><span class="string">web01.example.com</span><span class="punct">",</span> <span class="punct">"</span><span class="string">web02.example.com</span><span class="punct">"</span>
<span class="ident">role</span> <span class="symbol">:db</span><span class="punct">,</span> <span class="punct">"</span><span class="string">db.example.com</span><span class="punct">",</span> <span class="symbol">:primary</span> <span class="punct">=></span> <span class="constant">true</span></code></pre></div>
<h4>Deployment Root</h4>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">set</span> <span class="symbol">:deploy_to</span><span class="punct">,</span> <span class="punct">"</span><span class="string">/Library/Rails/<span class="expr">#{application}</span></span><span class="punct">"</span></code></pre></div>
<h3>Setup</h3>
<p>Run <code>cap setup</code> once to setup the deployment directory structure on
all the roles you’ve configured:</p>
<pre><code>depot
`- releases
`- shared
`- log
`- system
</code></pre>
<h3>First time deploy</h3>
<p>Run <code>cap update_code symlink</code>. This checks out the code, adds a
“current” symlink to the code on the remote machine that points to the
timestamped releases directory where the code was checked out. You
can then manually start your web server(s) if you wish.</p>
<h3>New Release</h3>
<p>Run <code>cap deploy</code>. The newly committed code to your repository gets
pulled, a new release created, and it also restarts fcgi processes.</p>
<h3>Rollback release</h3>
<p>Run <code>cap rollback</code>. The “current” symlink gets updated to the previous release,
and the fcgi processes get restarted.</p>
<h3>Scheduled downtime</h3>
<p>Run <code>cap disable_web</code>. A maintenance screen is put up during the
maintenance period. This is great when you’re in firefighting mode
and you don’t want to think, just get the page up there. To enable
again, use <code>cap enable_web</code>.</p>
<h3>Customization</h3>
<p>Here are examples of several useful, real-world tasks. The fourth
task shows aggregation of tasks together.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">desc</span> <span class="punct">"</span><span class="string">locate ruby</span><span class="punct">"</span>
<span class="ident">task</span> <span class="symbol">:which_ruby</span><span class="punct">,</span> <span class="symbol">:roles</span> <span class="punct">=></span> <span class="punct">[</span><span class="symbol">:app</span><span class="punct">]</span> <span class="keyword">do</span>
<span class="ident">puts</span> <span class="punct">"</span><span class="string">You're running:</span><span class="punct">"</span>
<span class="ident">run</span> <span class="punct">"</span><span class="string">which ruby</span><span class="punct">"</span>
<span class="keyword">end</span>
<span class="ident">task</span> <span class="symbol">:current_revision</span> <span class="keyword">do</span>
<span class="ident">run</span> <span class="punct">"</span><span class="string">echo Current rev is <span class="expr">#{revision}</span></span><span class="punct">"</span>
<span class="ident">run</span> <span class="punct">"</span><span class="string">echo Current rev is at <span class="expr">#{releast_path}</span></span><span class="punct">"</span>
<span class="keyword">end</span>
<span class="ident">task</span> <span class="symbol">:uptime</span><span class="punct">,</span> <span class="symbol">:roles</span> <span class="punct">=></span> <span class="punct">[</span><span class="symbol">:app</span><span class="punct">,</span> <span class="symbol">:web</span><span class="punct">,</span> <span class="symbol">:db</span><span class="punct">]</span> <span class="keyword">do</span>
<span class="ident">run</span> <span class="punct">"</span><span class="string">uptime</span><span class="punct">"</span>
<span class="keyword">end</span>
<span class="ident">task</span> <span class="symbol">:status</span> <span class="keyword">do</span>
<span class="ident">which_ruby</span>
<span class="ident">current_revision</span>
<span class="ident">uptime</span>
<span class="keyword">end</span></code></pre></div>
<h3>Channels and streams</h3>
<p>Mike showed a log file tailing task here. I failed to capture the full task code, so you’re on your own here.</p>
<h3>Hooks</h3>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">task</span> <span class="symbol">:before_deploy</span><span class="punct">,</span> <span class="symbol">:roles</span> <span class="punct">=></span> <span class="punct">[</span><span class="symbol">:app</span><span class="punct">]</span> <span class="keyword">do</span>
<span class="ident">cleanup</span>
<span class="keyword">end</span>
<span class="ident">task</span> <span class="symbol">:after_update_code</span><span class="punct">,</span> <span class="symbol">:roles</span> <span class="punct">=></span> <span class="punct">[</span><span class="symbol">:app</span><span class="punct">]</span> <span class="keyword">do</span>
<span class="ident">production_config</span> <span class="punct">=</span> <span class="punct">"</span><span class="string">path/to/database.yml</span><span class="punct">"</span>
<span class="ident">run</span> <span class="punct">"</span><span class="string">cp <span class="expr">#{production_config}</span> <span class="expr">#{current_path}</span>/config/database.yml</span><span class="punct">"</span>
<span class="keyword">end</span></code></pre></div>
<h3>Multiple configs</h3>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">if</span> <span class="ident">where</span> <span class="punct">==</span> <span class="punct">"</span><span class="string">production</span><span class="punct">"</span>
<span class="ident">set</span> <span class="symbol">:user</span><span class="punct">,</span> <span class="punct">"</span><span class="string">ops_gal</span><span class="punct">"</span>
<span class="keyword">else</span>
<span class="ident">set</span> <span class="symbol">:user</span><span class="punct">,</span> <span class="punct">"</span><span class="string">dev_guy</span><span class="punct">"</span>
<span class="keyword">end</span></code></pre></div>
<h3>Task libraries</h3>
<p>You can creat a file full of extra, reusable tasks and simply
<code>require</code> them in. The files can be published and installed as gems
or placed anywhere on your load path.</p>
<h3>Capistrano isn’t just for Rails</h3>
<p>It’s great at shuttling files around and executing remote commands on
the server. Mike describes some automation James Duncan Davidson did
to replicate new servers into a cluster with the push of a button.</p>
<h3>Assumptions</h3>
<p>Capistrano, out of the box comes with some assumptions:</p>
<ul>
<li>Remote servers talk POSIX</li>
<li>Same deploy structure and password on each machine</li>
<li>Web application uses FastCGI with Apache or Lighty</li>
</ul>
<p>It looks like you can bend some of these assumptions if you are
willing to write your own custom deployment tasks.</p>
<h3>Take Home</h3>
<p>Capistrano provides easy, consistent, worry-free deployment of new
releases. The friction to deploying your app just got simpler, so you
can go enjoy your life!</p>
<h3>Notes and Questions</h3>
<ul>
<li>Can you use SSH public keys to authenticate? <em>Yes, this is the most
typical scenario.</em></li>
<li>You can use <code>sudo</code> to execute remote commands – this is built-in.</li>
<li>Can you/would you want to check out subsets of files on each server,
rather than pulling all the code everywhere? <em>So far most of our
stuff has been on the same server, so this hasn’t been an issue. It
seems like you would need to override deployment tasks for some
roles.</em></li>
<li>Sometimes I want to deploy a lot, but what if the app is big, lots
of media files? It can take a longer time to pull all the
code. <em>One solution I’ve seen is to check out once and rsync
everywhere else. I haven’t had to do that myself.</em></li>
<li>Who can I talk to for promoting my own extensions into Capistrano.
<em>You probably don’t. Instead, publish your extensions, blog
them, and see if it takes.</em></li>
<li><code>deploy_with_migrations</code> – use with caution as it’s not as easy to
rollback, and not all migrations can be rolled back cleanly.</li>
</ul>Nick Siegerurn:uuid:1fd545f6-99b1-46e5-b8ae-50093b28f0672006-06-23T15:12:00+00:002007-08-31T16:41:56+00:00RailsConf: Dave Thomas keynote<h3>Dave Thomas, Professional Cassandra</h3>
<h2>Intro</h2>
<p>Dave starts by dissecting a google search and pointing out how the sponsored links sidebar contains a wealth of paid ads for various topics:</p>
<ul>
<li>Hosting</li>
<li>Consulting</li>
<li>Commercial IDEs and tools</li>
<li>Alternatives to RoR!</li>
</ul>
<p>Moving on to rubyforge, Dave examines some download stats, and points out that there have been 536835 downloads of Rails through <code>gem install rails</code></p>
<p>Dave examines some Google trends mapping of ruby on rails vs the following, showing RoR catching up on some and overtaking others:</p>
<ul>
<li>websphere</li>
<li>jboss</li>
<li>tapestry</li>
<li>spring framework</li>
<li>zend</li>
</ul>
<h2>3 Problems for Rails</h2>
<p>Dave proceeds to give his own, tailored for Rails, version of David Hilbert’s famous “23 unsolved problems”…except due to time constraints, can only discuss three.</p>
<h3>PDI</h3>
<p>An acronym, PDI, has emerged on the Rails core team as a way to encourage people to contribute. <em>Please Do Investigate</em></p>
<p>The challenge is to get new features to be proved out in the community before taking it into the core, the Rails core team is only 12 and has limited bandwidth, and that is the idea behind PDI</p>
<p>The three problem areas are:</p>
<ul>
<li>Data integration</li>
<li>CRUD</li>
<li>Deployment</li>
</ul>
<h3>Data integrations</h3>
<p>Dave mentions that he and David “disagree mildly” on the point of using constraints in the database.</p>
<ul>
<li>automatic validation based on schema</li>
<li>work with foreign keys - make it easy to define in a migration</li>
<li>add a “belongs_to” relationship if a foreign key constraint is detected</li>
<li>primary key support, e.g., non-integer keys, particularly in migrations, and composite primary keys</li>
</ul>
<p>The argument here is that real-world applications, and legacy databases today require these features, and Rails needs to be able to have answers to those questions if it wants to be viable in that area.</p>
<p>Dave is asking for distributed transactions now. Should we be pushing Rails into such an enterprisey direction? If you really need XA transactions, shouldn’t you be banished to a java world?</p>
<p>Non-database models would be a fabulous addition to rails. ActiveRecord::Base currently assumes that it’s talking to a sinle relational datastore. If we could integrate multiple disparate data soures, it would make integrations, REST data sources, and mashups much easier. Microsoft is doing this with LINQ today, and if Rails wants to keep up, it will need this.</p>
<h3>Real-world CRUD</h3>
<p>Dave is arguing that we should beef up scaffolding and make it useful. Scaffolding was the star of the initial 10 minute Rails video that started everything, but yet it’s the ugliest web 1.0 interface ever! No AJAX, no automatic, DRY, in-browser validation. But should we, can we really genericize these interfaces? Maybe a scaffolding widget/component system? It can and probably will happen, but at the momet I’m inclined to think that this will complicate things. Do we want Rails view to turn into Wicket widgets or ASP.NET components? It’s a completely different way of writing web programs. We’ll have to see how it plays out.</p>
<h3>Deployment</h3>
<p>Dave argues for improved deployment. Yes we have Capistrano, and it’s arguably the best web deployment mechanism in existence, but the web of knowledge required to deploy a Rails application is too large, varied and unwieldy.</p>
<p>Capistrano is a push model, ideal for small shops where the developers manage and maintain everything, but it’s not the real world. Larger shops have separate development and admin departments. They have stricter requirements for where files go, pre
and post-deployment hooks, roles, passwords, security, etc. Capistrano currently falls down a bit in this environment.</p>
<p>Dave discusses <code>cap --deploy-on</code>: a hypothetical extension to Capistrano that would allow staged deployment to multiple servers, and the possibility to introduce workflow between developers and administrators. What if ISPs had a standardized infrastructure that would allow hobbyists to issue a <code>cap --deploy-on</code> and instantly have their apps running up on TextDrive, DreamHost, etc. To go further, what if we could deploy from rubygems? <code>gem deploy <name> --on cap://my.isp.com</code>. Typo installation would literally be a single command.</p>
<h3>Closing</h3>
<p>All developers need to be happy, so let’s make their lives easier!</p>Nick Siegerurn:uuid:824696e4-63b3-4812-b0bc-cb58ccec05742006-06-23T14:10:00+00:002007-08-31T16:41:58+00:00RailsConf begins<p>I got lucky with a last minute ticket and found a way to get to the <a href="http://railsconf.org/">First International Rails Conference</a> in Chicago.</p>
<p><img src="http://codefluency.com/files/railsconf/railsconf-attendee.png" alt="Railsconf attendee"/> <img src="http://facebook.railsconf.org/portraits/457/njs-rails-face_small.jpg?1150985751" alt="Nick on Facebook"/></p>
<p>That’s <a href="http://facebook.railsconf.org/users/234">me</a>, hope to see you this weekend!</p>Nick Siegerurn:uuid:1cf8443e-d948-41d8-81cd-166a92b98ab32006-06-28T15:00:00+00:002007-08-31T16:41:58+00:00RailsConf: Mike Pence - Laszlo on Rails<p>Mike Pence, professional web surfer, and Java free since March 15,
talked about <em>Sex, drugs, rock and roll</em> or <em>Laszlo on Rails</em>.</p>
<h2>Where are we going on the web?</h2>
<ul>
<li>Google Maps, Yahoo Music Engine, Google Spreadsheets – the web is
looking more and more like a desktop application.</li>
<li>“Web two oh” – attention to design and more attractive interfaces</li>
<li>User customization a.k.a. “Pimp my site”</li>
<li>Use of rich media on the web, e.g., YouTube. It’s an expectation of
the next generation of users that the web will be content-rich and
an entertainment experience.</li>
<li>The Holy Grail! Applications that require no downloads,
are instant/automatically updated, are distributed.</li>
</ul>
<h2>Open Laszlo</h2>
<ul>
<li><a href="http://www.pandora.com">Pandora</a> cited as an introductory example</li>
<li>Mike gave a 10 minute overview of Laszlo using the Open Laszlo
Explorer.</li>
<li>Laszlo explorer shows you standard widgetry – canvas, text,
buttons, windows, forms</li>
<li>The power of Laszlo starts to show with data sets, with convenient
data binding utilities, an extensible object model, and a
declarative style. Mike showed 10 lines of code with a checkbox
that controlled the visibility of a window, without having to attach
an event handler to the checkbox.</li>
<li><a href="http://www.openlaszlo.org/">http://www.openlaszlo.org/</a> has the 10 minute overview (explorer)
and many other demos including LZPIX, which Mike demoed.</li>
<li>The newest version of Laszlo has DHTML support that allows a flash
app to be served as DHTML instead, with little difference. Laszlo
gives you the power of one runtime that rises above browser
incompatibilities.</li>
</ul>
<h2>Laszlo on Rails</h2>
<ul>
<li>Install
# install laszlo
gem install ropenlaszlo
rails laszlo-app && cd laszlo-app
./script/plugin install svn://rubyforge.org/var/svn/laszlo-plugin/tags/openlaszlo</li>
<li>More info at <a href="http://laszlo-plugin.rubyforge.org/">http://laszlo-plugin.rubyforge.org/</a></li>
</ul>
<h2>Pros & Cons</h2>
<ul>
<li>Pro: Rich possibilities
<ul>
<li>Blogbox – cross-site window</li>
<li>Publish and subscribe for chat and collaboration, event-driven
updates</li>
</ul></li>
<li>Pro: deep API</li>
<li>Pro: in-browser development, like Seaside</li>
<li>Con: Consumes resources</li>
<li>Con: Accessibility, printability and searchability are not its
strengths</li>
<li>Con: mature, yet requires experimentation</li>
<li>Con: performance can be an issue, especially on some older platforms</li>
</ul>
<p>Store it away – Laszlo is a promising technology, it’s free and open
source it’s here today, and it appears to be getting good at serving
standards-based interfaces. When combined with Rails’ increasing
support for RESTian interfaces, the task of building compatible,
dynamic applications should only get easier.</p>Nick Siegerurn:uuid:beeab649-29ff-4113-b389-bf4514295d5f2006-06-24T17:27:00+00:002007-08-31T16:41:58+00:00RailsConf: Bill Katz - Metaprogramming Writertopia<p>Bill Katz took a code-focused approach to explaining metaprogramming
and DSLs.</p>
<h2>Resources</h2>
<ul>
<li><a href="http://www.writertopia.com/developers">http://www.writertopia.com/developers</a> has an authorization plugin used
in writertopia, which is the main focus of this talk.</li>
<li><a href="http://www.simbiome.org">http://www.simbiome.org</a> and <a href="https://wimtk.org/svn/simbiome">https://wimtk.org/svn/simbiome</a> contain
code for physics based simulation resources.</li>
</ul>
<p>Bill applies Paul Graham’s quote on bottom-up design and lisp as
justification for using Ruby to program DSLs.</p>
<h2>Authorization plugin</h2>
<p>In search of a better authorization scheme, Bill wanted an elegant
syntax using blocks to delineate sections of code that require
specific permissions, and a more intuitive way of managing the data
surrounding access controls.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">current_user</span><span class="punct">.</span><span class="ident">has_role?</span> <span class="punct">"</span><span class="string">moderator</span><span class="punct">"</span>
<span class="ident">current_user</span><span class="punct">.</span><span class="ident">has_role?</span> <span class="punct">"</span><span class="string">member</span><span class="punct">",</span> <span class="ident">workshop</span>
<span class="ident">workshop</span><span class="punct">.</span><span class="ident">accepts_role?</span> <span class="punct">"</span><span class="string">member</span><span class="punct">",</span> <span class="ident">user</span>
<span class="ident">current_user</span><span class="punct">.</span><span class="ident">is_moderator_of</span> <span class="ident">workshop</span>
<span class="ident">user</span><span class="punct">.</span><span class="ident">is_eligible_for?</span> <span class="ident">campbell_award</span></code></pre></div>
<h2>Implementing <code>permit</code></h2>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">MeetingController</span> <span class="punct"><</span> <span class="constant">ApplicationController</span>
<span class="ident">permit</span> <span class="punct">"</span><span class="string">rubyists and wanna_be_rubyists</span><span class="punct">",</span> <span class="symbol">:except</span> <span class="punct">=></span> <span class="symbol">:public_text</span></code></pre></div>
<p>We need a class method. Where to put it? How does this work? (Insert
pretty diagram of instance, singleton class and superclass here.)
Metaclassees are “anonymous”, you don’t see them by looking at class
ancestors.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">send</span><span class="punct">(</span><span class="symbol">:include</span><span class="punct">,</span> <span class="constant">Authorization</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">)</span>
<span class="constant">ActionView</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">send</span><span class="punct">(</span><span class="symbol">:include</span><span class="punct">,</span> <span class="constant">Authorization</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">::</span><span class="constant">ControllerInstanceMethods</span><span class="punct">)</span></code></pre></div>
<p>Singleton methods from the point of view of the object are the same as
instance methods in the metaclass. See slides on “How do we permit?”
for details. Bill chose to tuck the DSL-related methods into modules
and used <code>self.included</code>, <code>class_eval</code> and other goodies to set
up the methods in the correct places. There is more than way to do
this but I suspect Bill chose to do it for purposes of cleanly
separating concerns along the lines of where in the class hierarchy
the methods live.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">send</span><span class="punct">(</span><span class="symbol">:include</span><span class="punct">,</span>
<span class="constant">Authorization</span><span class="punct">::</span><span class="constant">ObjectRolesTable</span><span class="punct">::</span><span class="constant">UserExtensions</span><span class="punct">,</span>
<span class="constant">Authorization</span><span class="punct">::</span><span class="constant">ObjectRolesTable</span><span class="punct">::</span><span class="constant">ModelExtensions</span><span class="punct">)</span></code></pre></div>
<p>This loads an <code>acts_as_authorizable</code> plugin which, when used in a
model, inserts an <code>accepts_role</code> family of methods for querying and
setting user roles.</p>
<p>Bill blazed through a whole lot of pretty-looking code here for how to
set up the DSL infrastructure. You’ll have to go over to the
writertopia developer link to see it. Bill has said he’ll post his
slides online as well.</p>
<h2>Questions</h2>
<ul>
<li>Use of <code>method_missing</code> – does that clash with <code>ActiveRecord::Base</code>
usage of <code>method_missing</code>? <em>No, as long as you use <code>super</code> to
invoke the next method in the chain, you’re ok.</em> Ruby module
inclusion semantics handle this for you.</li>
<li>How to insert the class methods? A discussion started around
<code>ActionController::Base.send(:include, ...)</code> and There is More Than
One Way to Do It.</li>
</ul>
<div class="typocode"><pre><code class="typocode_ruby "><span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">send</span><span class="punct">(</span><span class="symbol">:include</span><span class="punct">,</span> <span class="constant">Authorization</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">)</span>
<span class="keyword">class </span><span class="class">ActionController::Base</span><span class="punct">;</span> <span class="ident">include</span> <span class="constant">Authorization</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">;</span> <span class="keyword">end</span>
<span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">class_eval</span> <span class="punct">{</span> <span class="ident">include</span> <span class="constant">Authorization</span><span class="punct">::</span><span class="constant">Base</span> <span class="punct">}</span></code></pre></div>
<ul>
<li>Question about the class inheritance chain. There are actually two
inheritance chains, one for the classes and one for the metaclasses.
Resolution at method invocation time travels first to the object’s
metaclass, then to the object’s class, then to the superclass of the
singleton class, then the class’s superclass, etc. Blech, that
makes no sense. Instead, go read <a href="http://whytheluckystiff.net/articles/seeingMetaclassesClearly.html">Seeing Metaclasses Early</a>.</li>
</ul>