Posted by Nick Sieger
Sun, 06 Jul 2008 03:48:51 GMT
Well, the recent Ruby 1.8.6 issues, including an apparent memory leak problem that was plaguing my blog’s mongrel caused me to evaluate what should have been obvious long ago. That I should put my money where my mouth is and upgrade to JRuby on Glassfish using JRuby-Rack and Warbler.
And so it’s done.
I’m now running the blog in Glassfish V2 with JRuby 1.1.1 and JRuby-Rack 0.9 (facilitated by Warbler 0.9.9), along with activerecord-jdbc-adapter 0.8.2. Here’s to dog-fooding!
(Now if only I could find a situation painful enough to force up to re-skin this blog beyond the ancient Typo azure theme...)
Tags jruby | 6 comments
Posted by Nick Sieger
Wed, 04 Jun 2008 21:57:00 GMT
The long-delayed and much-awaited 0.8.1 release is here. It fixes
quite a few reported bugs (but not quite all). See the full changelog
below for details.
Please help me make more frequent releases of ar-jdbc by submitting patches against the trunk of
activerecord-jdbc. Test cases appreciated also.
File bugs in JRuby’s JIRA. Use the “ActiveRecord-JDBC” component when filing them.
You can check out the source here:
svn co http://jruby-extras.rubyforge.org/svn/trunk/activerecord-jdbc
git clone git://github.com/nicksieger/activerecord-jdbc-adapter
Recently I started keeping a mirror of activerecord-jdbc-adapter on Github. Feel free to watch or fork and send me patches via git format-patch as well.
One of the cool new things in this release is a JDBC version of sqlite3 using the Zentus Sqlite JDBC driver. The crazy thing is how the driver is created. The original sqlite3 codebase is cross-compiled to MIPS and the resulting output coverted to Java bytecode using NestedVM! The adapter is still in its early stages, but basic stuff seems to be working. Try it out using jruby -S gem install activerecord-jdbcsqlite3-adapter.
Let me know how the release works for you!
Changes in 0.8.1
- Now sporting a JDBC sqlite3 adapter! Thanks Joseph Athman.
- Added support for InterSystems Cache database (Ryan Bell)
- Fix for JRUBY-2256
- JRUBY-1638, JRUBY-2404, JRUBY-2463: schema.table handling and Oracle
NUMBER fixes (Darcy Schultz & Jesse Hu)
- Add structure dump and other DDL-ish for DB2 (courtesy abedra and
stuarthalloway)
- Fix missing quotetablename function under Rails 1.2.6 and earlier
- Small tweaks to jdbc.rake to select proper config
- JRUBY-2011: Fix MSSQL string un-quoting issue (Silvio Fonseca)
- JRUBY-1977, 17427: Fix information_schema select issue with MSSQL (Matt Burke)
- 20479: Improve gettablename for MSSQL (Aslak Hellesøy)
- 20243: numerics improvements for MSSQL (Aslak Hellesøy)
- 20172: don’t quote table names for MSSQL (Thor Marius Henrichsen)
- 19729: check for primary key existence in postgres during insert
(Martin Luder)
- JRUBY-2297, 18846: retrying failing SQL statements is harmful when
not autocommitting (Craig McMillan)
- 10021: very preliminary sybase support. (Mark Atkinson) Not usable
until collision w/ sqlserver driver is resolved.
- JRUBY-2312, JRUBY-2319, JRUBY-2322: Oracle timestamping issues
(Jesse Hu & Michael König)
- JRUBY-2422: Fix MySQL referential integrity and rollback issues
- JRUBY-2382: mysql string quoting fails with ArrayIndexOutofBoundsException
Tags jruby, rails | no comments
Posted by Nick Sieger
Sun, 01 Jun 2008 22:07:33 GMT
These are the notes I took during our “panelish” Q & A session on Sunday afternoon at RailsConf.
Exceptions (behavior between ruby and java)
- Embedding: JSR-223 preferred, BSF fallback
Java integration, mocking, proxying, extension
Multiple VM support
- multiple JRuby instances can be run in the same VM
- JavaSand (google it) -- Ola’s port of _why’s (freaky freaky)
sandbox
Startup time performance gain recently
- -Xbootclasspath: VM does not verify classes in the “boot” classpath
- -Xverify:none is not recommended because of runtime-generated code
- Nailgun to keep VM running and “send commands” to it
Compiling Ruby code to JVM bytecode
- Keep # of class files low
- Walk AST and dump out high-level, abstracted operations
(local variable access, dynamic invocation) and low-level bytecode
is built for each of those
- 4096 -- maximum # of methods to be compiled
- Compiled methods are shared between runtimes
- Future:
- raise/eliminate overhead of compiled methods
- share AST, reduce memory load
Community tasks/actions
- Projects needing help (e.g., ActiveHibernate)
- Adoption, acceptance, blogging
- Documentation (http://wiki.jruby.org/ and others), screencasts
- Who’s using it?
Sun’s commitment
- Ruby vs. Groovy vs. Python vs. Scala vs. other langs
- Rails vs. Grails vs. Lift vs. JSF vs. Struts 2 vs. ???
- all of the above
- improve the JVM’s support for all dynamic languages
- JRuby is a full open source project, not owned or controlled by Sun,
with history that extends years before main devs hired by Sun
Ruby programming language, JRuby is an implementation
- community evangelism of Ruby the language vs. implementations
- despite MVM, FFI and emerging areas that are not standard yet
How do you pitch JRuby/Rails in a legacy environment?
- Consider demonstrating running application without discussing
technology specifics
- IBM and JDK 1.5 issues have been reported
- Free support as long as these issues are new and help improve
JRuby and the community!
Testing
- JRuby enables more agile testing
- Without production risks
- Ceremony vs. Essence discussion
Terracotta/DSO
Windows/Mongrel/ImageMagick issues
- ImageVoodoo is an imaging library using Java2D, comes with Java
- MiniMagick also reportedly works
Rails benchmarks
- Micro-benchmarks are problematic and usually don’t lead to
measurable application speedups
- Community needs a real-world, full application
- Antonio Cangiano working on new application-level benchmarks?
- “Richards” -- Smalltalk-originated application benchmark
JSR-292 (“invokedynamic”)
- Actually extension of “invokeinterface” bytecode
- Call site structure/method handles
- Expose the dynamic nature of the JVM
(get the Java-specific stuff out of the way)
- Language-specific calling semantics, method invocation
- “Punching a hole” through the JVM
- Comments on JSR-292 Early Draft Review currently ongoing
Tags jruby, railsconf, railsconf2008 | no comments
Posted by Nick Sieger
Sat, 31 May 2008 23:42:00 GMT
If you already saw my JavaOne slides, these aren’t too different, but I think they’re better and prettier, too.
Get them here.
Update. The slides are pretty lean on explanation. Just in case you’re confused, the narrative goes like this:
- What’s your deployment nirvana look like? (various existing options)
- With JRuby, you can deploy Rails applications in a single operating system process instead of many.
- However, there are a few configuration changes needed to accomodate JRuby. (Steps for converting existing apps, configuration code snippets.)
- Now, JRuby-Rack helps with the configuration, because all of the logging/session/public path re-jiggering is taken care of for you. It’s now bundled with Warbler as of version 0.9.9.
- Performance is good and getting better. You can scale up the number of requests you can handle really easily just by setting the number of runtimes to create inside Warbler’s
config/warble.rb file.
- Lots of new stuff is happening right now to make Rails better. JRuby will be able to leverage these changes and become an even more desirable deployment platform. Stay tuned!
Tags jruby, railsconf, railsconf2008 | 3 comments
Posted by Nick Sieger
Thu, 22 May 2008 15:40:00 GMT
Thanks to our friends at LinkedIn, Joyent and Sun, we’re having a hackfest at McMenamins/Kennedy School on Thursday evening, May 29, starting at 6:30pm, complete with food and beverages!
Do stop by and hang out with us! Leave a comment over at Charlie’s announcement if you’re interested in joining us.
Kennedy School
Local: (503) 249-3983
Elsewhere: (888) 249-3983
View Larger Map
Tags jruby, railsconf, railsconf2008 | no comments
Posted by Nick Sieger
Thu, 08 May 2008 17:31:00 GMT
Continuing in the spirit of Conference-Driven Development, I’m happy to announce the first public release of JRuby-Rack! You can use it to run Rails, Merb, or any Rack-compatible application inside a Java application server.
Also released today is Warbler 0.9.9, which has been updated to bundle JRuby-Rack.
In addition to providing as seamless a connection as possible between the servlet environment and Rack, JRuby-Rack (along with Warbler) is also bridging the gap between Ruby and Java web development. Some of the things it does are:
- Makes the Java servlet context and servlet request available to Ruby through special variables in the Rack environment
- Servlet request attributes from Java are passed through and available in the Rack environment. Request attributes can override Rack variables such as
PATH_INFO, QUERY_STRING etc.
- Configures Rails deployment options such as page caching directories and session handling automatically and optimally for the servlet environment.
I’ve also included the beginnings of some extensions that should help integrate Rails with existing Java web frameworks, servlets, JSPs, and other code. For example, you can invoke a Rails request from within a JSP with a tag:
<jruby-rack:rails path="/projects/activity" params="layout=none"/>
You can set servlet and session attributes and forward to other servlets and JSPs from your Rails controllers:
class DemoController < ApplicationController
def index
servlet_request["hello"] = "world!"
session["rails"] = "Visible to java!"
forward_to "/attributes.jsp"
end
end
and read them from within the servlet or JSP:
<dl>
<dt><tt>servlet_request["hello"] | request.getAttribute("hello")</tt></dt>
<dd><%= request.getAttribute("hello") %></dd>
<dt><tt>session["rails"] | session.getAttribute("rails")</tt></dt>
<dd><%= session.getAttribute("rails") %></dd>
</dl>
This is just the beginning of this kind of integration, and I’m interested where people take it. I think this provides a nifty way to start integrating Rails bits into existing applications or reuse existing Java web application code.
I’ve tagged the release with an 0.9 version number. I believe the bits are ready for serious use, but could use some help pounding out a few more bugs before calling it 1.0. So jruby -S gem install warbler today, try it out, and bring plenty of feedback to the JRuby user list!
Tags jruby, merb, rack, rails | 17 comments
Posted by Nick Sieger
Thu, 27 Mar 2008 21:39:23 GMT
Introducing ImageVoodoo
I just pushed out the first release of ImageVoodoo, a nifty little image manipulation library conceived as a quick hack by Tom. It’s a play-on-words of ImageScience, of course, the quick, lightweight imaging library for Ruby. To get it,
jruby -S gem install image_voodoo
What’s cool about ImageVoodoo (other than the name) is that we were able to make it API-compatible with ImageScience. In fact, ImageVoodoo’s image_science.rb simply looks like this:
require 'image_voodoo'
ImageScience = ImageVoodoo
So, you can use it pretty much anywhere you might use ImageScience, and it should just work. At work, we’re using it with attachment_fu, and it works great. ImageVoodoo even steals and uses ImageScience’s unit tests (which all run successfully, too). Speed-wise, it’s about twice as slow as ImageScience running on MatzRuby, but still plenty fast enough for most cases.
But we wouldn’t be having fun unless we embraced and extended a little bit, right? So I added a couple of extra features you might find useful.
Preview
Since ImageVoodoo is just leveraging the Java Platform’s imaging libraries, image rendering can be easily tied into a simple preview frame. This code:
ImageVoodoo.with_image("samples/checkerboard.jpg") do |img|
img.preview
end
Will pop up a little frame like this:
The code that displays the preview frame is nice and compact, and shows off how nicely you can write clean swing GUI code using JRuby’s java integration features.
class ImageVoodoo
class JImagePanel < javax.swing.JPanel
def initialize(image, x, y)
super()
@image, @x, @y = image, x, y
end
def paintComponent(graphics)
graphics.drawImage(@image, @x, @y, nil)
end
end
class WindowClosed
def initialize(block = nil)
@block = block || proc { java.lang.System.exit(0) }
end
def method_missing(meth,*args); end
def windowClosing(event); @block.call; end
end
def preview(&block)
frame = javax.swing.JFrame.new("Preview")
frame.add_window_listener WindowClosed.new(block)
frame.set_bounds 0, 0, width + 20, height + 40
frame.add JImagePanel.new(@src, 10, 10)
frame.visible = true
end
end
Command-line
As I was fixing a bug in ImageVoodoo’s file saving I whipped up a little command-line utility to aid debugging. It allows you to string along several image manipulation actions on a single command-line. For example,
jruby -S image_voodoo --push --resize 50x50 --preview --save t1.jpg \
--pop --resize 40x40 --preview --save t2.jpg \
--pop --resize 30x30 --preview --save t3.jpg image.jpg
This will resize image.jpg into three smaller images, t[1-3].jpg, but will pop up a preview frame at each step of the way. Simply close the preview frame to continue to the next action, or quit out of the application to abort.
Summary
And so, another functional area, image manipulation, becomes as easy on JRuby as it is on MatzRuby. Now that fancy social networking application you’ve been working on should have one less reason to be able to run unmodified on JRuby!
Tags image, jruby, ruby, science, voodoo | 3 comments
Posted by Nick Sieger
Thu, 21 Feb 2008 05:52:44 GMT
One of the aspects we have to work around building and improving a dynamic language implementation on the Java Virtual Machine is the way the JVM loads and executes bytecode. In order for JRuby to take advantage of the Hotspot just-in-time (JIT) compiler, JRuby needs to generate Java bytecode at runtime, during execution of Ruby code. If that bytecode gets executed often enough and meets certain other rather mysterious conditions, Hotspot will turn it into machine code.
Unfortunately the VM was originally designed to run one language well, and that’s Java. The only way to get bytecode loaded into the JVM is through the Java class loader. With Java, most if not all the code is compiled to bytecode ahead of time and the VM assumes (and optimizes) for the fact that classes will not be shuffled around in memory. Partly due to these assumptions, the JVM stores bytecode along with other class metadata in a separate heap called the “permanent generation”, or just “permgen”. (I’m guessing the name “permanent” was used because originally objects in this heap were probably not garbage collected).
However, because JRuby needs to provide the amount of dynamism that a Ruby programmer would expect (open classes; modules included; methods added/removed at any time), a Ruby class does not cleanly map one-to-one with a Java class. Instead, it’s easier to think of Ruby classes as method-bags. As a result, JRuby creates a new Java class for every Ruby method that it decides to compile down to bytecode. Additionally, since Ruby methods can come and go, in order for the method to be collectible by the garbage collector, it needs to live in its own class loader.
Of course, JRuby is not the first Java program to generate classes and load them at runtime (JSPs have been doing this for ten years). But it may create more class and class loader garbage than just about any program ever run on the JVM. For small programs, generating a class per method would be no big deal, but consider a Rails application: The Rails codebase itself has thousands of methods, but it also generates plenty of new methods at runtime.
Consider a non-trivial Rails application that makes liberal usage of the Ruby standard library, and also uses a handful of plugins, and the number of methods available for JRuby to compile can easily exceed 10,000. If the average overhead of a single JRuby method class is around 8K (varying due to method size, of course), this would occupy up to 80 megabytes of permgen space. (By contrast, the JVM’s default size of the permgen space is 64 megabytes, so we’re already over the limit). Now consider that, with Goldspike we need to use multiple JRuby runtimes in order to achieve concurrent requests due to Rails’ lack of thread-safety, and the number is multiplied further. If you were to deploy 4 Rails applications each with 4 active runtimes into a single application server, you’re looking at almost 1.2 gigabytes of permgen space necessary to run your applications! (Usually, it’s common to run multiple applications in a Java application server, but with Rails applications that may need to be reconsidered.)
Because of this multiplicative cost, shortly after JRuby 1.1RC1 was released we took the somewhat drastic measure of capping the number of methods that each runtime would JIT-compile to 2048. But after a while it became obvious even with a threshold-based approach, JRuby was still wasting a ton of permgen space with duplicate copies of compiled methods. So for 1.1RC2 we introduced a JIT cache that could be set up to be shared among multiple runtimes.
The figure below shows the effect. Under consideration is a single JRuby/Rails/Goldspike application deployed in Glassfish, with varying numbers of runtimes, right after deployment (cold) and after some warmup to load more Rails code and allow JIT to reach the method threshold. JRuby trunk revision 5545 had the 2K JIT threshold, but not the sharing. Revision 5931 is right before RC2 was released, with the method cache wired up. (We also took some measures to reduce permgen consumption in those 500 revisions, so some of that is visible as well.)
The just-released Goldspike 1.5 will use this shared method cache by default, so all you’ll have to do is upgrade to receive the benefits. An easy way to do that is to install Warbler 0.9.3, an update which bundles Goldspike 1.5 and JRuby 1.1RC2.
These kind of techniques to reduce JRuby’s permgen overhead are only going to go so far when the underlying VM still isn’t expecting to be abused in this way. That’s why we’re looking forward to a JRuby that will be able to take advantage of proposed future enhancements like anonymous classes/method handles as part of John Rose’s Da Vinci Machine project. For more information along those lines, head over to Charlie’s discussion of what comes next.
Tags jruby, jvm | 2 comments
Posted by Nick Sieger
Fri, 08 Feb 2008 15:13:16 GMT
A new screen cast is up with yours truly showing off NetBeans’ RSpec support. Additionally, I tried to make it interesting to a wider audience by really showcasing RSpec’s strengths, and trying to capture some of the red-green-refactor rhythm. NetBeans does work really well for this, but in my mind, the star of the show is RSpec.
I’m pleased with how it turned out considering I hadn’t done this sort of thing before. Special thanks to Cindy Church for putting it all together, including all the production: setup, recording, editing, even the music!
A QuickTime movie version is available as well. Check it out and let me know what you think.
Tags jruby, netbeans, rspec, ruby | 1 comment
Posted by Nick Sieger
Thu, 17 Jan 2008 23:48:00 GMT
Next up in our performance series: Builder::XChar. (Another fine Sam Ruby production!) While this piece of code in the Builder library strikes me as perfectly fine, it also tends to slow down quite a bit with larger documents or chunks of text.
Our path to the bottleneck is as follows: ActiveRecord::Base#to_xml => Builder::XMLMarkup#text! => String#to_xs => Fixnum#xchr. Consider:
require 'rubygems'
gem 'activesupport'
require 'active_support'
require 'benchmark'
module Benchmark
class << self
def report(&block)
n = 10
times = (1..10).map do
bm = measure(&block)
puts bm
bm
end
sum = times.inject(0) {|s,t| s + t.real}
mean = sum / n
sumsq = times.inject(0) {|s,t| s + t.real * t.real}
sd = Math.sqrt((sumsq - (sum * sum / n)) / (n - 1))
puts("Mean: %0.6f SDev: %0.6f" % [mean, sd])
end
end
end
page = File.open("page.xml") {|f| f.read }
Benchmark.report do
20.times { page.to_xs }
end
On Ruby and JRuby, this produces:
$ ruby to_xs.rb
21.430000 0.400000 21.830000 ( 22.022769)
21.530000 0.360000 21.890000 ( 22.005737)
21.540000 0.370000 21.910000 ( 22.065165)
21.530000 0.370000 21.900000 ( 22.028591)
21.500000 0.350000 21.850000 ( 21.990395)
21.550000 0.370000 21.920000 ( 22.033164)
21.520000 0.360000 21.880000 ( 21.984129)
21.550000 0.370000 21.920000 ( 22.116802)
21.550000 0.370000 21.920000 ( 22.051421)
21.520000 0.380000 21.900000 ( 22.084736)
Mean: 22.038291 SDev: 0.041985
$ jruby -J-server to_xs.rb
79.112000 0.000000 79.112000 ( 79.112000)
81.480000 0.000000 81.480000 ( 81.481000)
84.745000 0.000000 84.745000 ( 84.745000)
84.384000 0.000000 84.384000 ( 84.384000)
121.933000 0.000000 121.933000 (121.933000)
85.533000 0.000000 85.533000 ( 85.532000)
82.762000 0.000000 82.762000 ( 82.763000)
82.090000 0.000000 82.090000 ( 82.090000)
81.298000 0.000000 81.298000 ( 81.299000)
80.774000 0.000000 80.774000 ( 80.773000)
Mean: 86.411200 SDev: 12.635700
(Hmm, I must have accidentally swapped in some large program in the middle of that JRuby run. The perils of benchmarking on a desktop machine. I don’t claim that the numbers are scientific, just illustrative!)
Fortunately, the fix again is very simple, and has previously been acknowledged. The latest (unreleased?) Hpricot has a new native extension, fast_xs, which is an almost drop-in replacement for the pure-ruby String#to_xs. (Almost, because it creates the method String#fast_xs instead of String#to_xs. ActiveSupport 2.0.2 and later take care of aliasing it for you). Unbeknownst to me, I ported fast_xs recently as part of upgrading JRuby extensions that have Java code in them. And so it happens to come in handy at this time. The patch for that is here.
I have the latest Hpricot gems on my server, so you can install it yourself (for either Ruby or JRuby):
gem install hpricot --source http://caldersphere.net
or
jruby -S gem install hpricot --source http://caldersphere.net
With that installed, the script now produces these results:
$ ruby to_xs.rb
0.460000 0.080000 0.540000 ( 0.537793)
0.420000 0.070000 0.490000 ( 0.501965)
0.430000 0.070000 0.500000 ( 0.501359)
0.400000 0.070000 0.470000 ( 0.484495)
0.400000 0.070000 0.470000 ( 0.479995)
0.400000 0.070000 0.470000 ( 0.469118)
0.390000 0.070000 0.460000 ( 0.468864)
0.390000 0.070000 0.460000 ( 0.465009)
0.390000 0.060000 0.450000 ( 0.452902)
0.390000 0.070000 0.460000 ( 0.466881)
Mean: 0.482838 SDev: 0.024926
$ jruby -J-server to_xs.rb
0.882000 0.000000 0.882000 ( 0.883000)
0.832000 0.000000 0.832000 ( 0.832000)
0.851000 0.000000 0.851000 ( 0.850000)
0.837000 0.000000 0.837000 ( 0.837000)
0.846000 0.000000 0.846000 ( 0.846000)
0.843000 0.000000 0.843000 ( 0.843000)
0.835000 0.000000 0.835000 ( 0.835000)
0.825000 0.000000 0.825000 ( 0.826000)
0.830000 0.000000 0.830000 ( 0.830000)
0.834000 0.000000 0.834000 ( 0.833000)
Mean: 0.841500 SDev: 0.016379
Tags jruby, performance, rails, ruby | 3 comments
