cdivilly

Maven, JRuby-Complete and Rake

Posted in java, notetoself by cdivilly on January 11, 2010

In a previous post I described how to configure Maven to use JRuby to perform a build using Rake from within a Maven build process. While running that same process on a newly re-imaged windows box I encountered the following error output:

jruby: No such file, directory, or command -- rake

This confused me for some time, I’m using jruby-complete which packages rake within itself, how could it not be able to find its own rake? Eventually it occurred to me that the only difference between my new and old windows images was the location of my maven repository. My old image had the repository manually configured to be in a folder which has no spaces in its path, my new image just had the default setting, which placed the repository in c:\Documents And Settings\username\.m2\repository. I manually configured Maven to store the repository in a path with no spaces and hey presto the problem went away.

So this is just a short note to remind myself when this problem next crops up: even now in the second decade of the 21st century avoid spaces in file names!!!

Tagged with: , ,

Comments Off

links for 2009-09-01

Posted in links by cdivilly on September 2, 2009

Comments Off

Recording API Invocations for Debugging

Posted in java, notetoself by cdivilly on August 20, 2009

Today I encountered a problem when using the Oracle JDBC API that only manifested itself in a certain complicated sequence of API invocations. I wanted to understand exactly what the sequence of API invocations was and also be able to extract them, so I could spit them out into a standalone test-case that I could use to debug the problem. Below is a class I whipped up to help automate most of this task.

So now I can do something like:

Invocations t = new Invocations();
PoolDataSource ods= t.trace(PoolDataSourceFactory.getPoolDataSource(),PoolDataSource.class);

ods.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
ods.setURL("jdbc:oracle:thin://localhost:1521/orcl");
ods.setUser("scott");
ods.setPassword("tiger");
Connection conn = ods.getConnection();
CallableStatement stmt = conn.prepareCall("begin dbms_output.put_line('Hello'); end;");
stmt.execute();
stmt.close();
conn.close();
System.out.print(t);

Which produces the following output:

oracle.ucp.jdbc.PoolDataSource v1;
v1.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
v1.setURL("jdbc:oracle:thin://localhost:1521/orcl");
v1.setUser("scott");
v1.setPassword("tiger");
java.sql.Connection v2;
v2 = v1.getConnection();
java.sql.CallableStatement v3;
v3 = v2.prepareCall("begin dbms_output.put_line('Hello'); end;");
boolean v4;
v4 = v3.execute();
v3.close();
v2.close();

Which is a pretty good facsimile of the original code. Note there are obviously limitations to what this kind of approach can achieve:

  • Invocations can only handle primitive values and values created within the API itself
  • Any non primitive values (including arrays) created outside of the API can only be declared, code to initialize these values will need to be added manually.

The major benefit of the generated code is that it does not have any dependencies other than the API being traced, thus making it easier to extract a stand-alone test case that exhibits the problem. This makes it easier to focus on the problem by eradicating superfluous code. It also very useful when you need a stand-alone test-case for a bug report for a third party API.

The Source Code

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.IdentityHashMap;

public class Invocations {

	@Override
	public String toString() {
		return b.toString();
	}
	public <T> T trace(final Object target, final Class<T> type) {
		return trace(target, type, false);
	}

	private void declare(final Object object, final Class<?> type,
			final boolean force) {
		if (!void.class.equals(type)) {
			if (force
					|| (object != null && !variables.containsKey(object) && !isPrimitive(type))) {
				b.append(typeName(type));
				b.append(" ");
				b.append(register(object));
				b.append(";\n");
			}
		}
	}
	private boolean isPrimitive(final Class<?> type) {
		boolean isPrimitive = type.isPrimitive();
		if (!isPrimitive) {
			for (final Class<?> c : PRIMITIVES) {
				if (c.equals(type)) {
					isPrimitive = true;
					break;
				}
			}
		}
		return isPrimitive;
	}
	private String name(final Object object) {
		String name = "null";
		if (object != null) {
			name = variables.get(object);
			if (name == null) {
				if (isPrimitive(object.getClass())) {
					return render(object);
				} else {
					throw new RuntimeException("Could not find name for: "
							+ object);
				}
			}
		}
		return name;
	}
	private String register(final Object object) {
		String name;
		name = variables.get(object);
		if (name == null) {
			name = "v" + (++var);
			variables.put(object, name);
		}
		return name;
	}
	private String render(final Object object) {
		final Class<?> type = object.getClass();
		if (String.class.equals(type)) {
			return "\"" + object.toString() + "\"";
		} else if (short.class.equals(type) || Short.class.equals(type)) {
			return "(short)" + object.toString();
		} else if (long.class.equals(type) || Long.class.equals(type)) {
			return object.toString() + "L";
		} else if (char.class.equals(type) || Character.class.equals(type)) {
			return "'" + object.toString() + "'";
		} else {
			return object.toString();
		}
	}
	@SuppressWarnings("unchecked")
	private <T> T trace(final Object target, final Class<T> type,
			final boolean force) {
		T result = null;
		if (target != null) {
			if (target.getClass().getInterfaces().length > 0
					&& !isPrimitive(target.getClass())) {
				result = type.cast(Proxy.newProxyInstance(Thread
						.currentThread().getContextClassLoader(), target
						.getClass().getInterfaces(), new Recorder(target)));
			} else {
				result = (T) target;
			}
		}
		declare(result, type, force);
		return result;
	}
	private String typeName(final Class<?> type) {
		if (type.isArray()) {
			return typeName(type.getComponentType()) + "[]";
		} else {
			return type.getName();
		}
	}

	private class Recorder implements InvocationHandler {

		Recorder(final Object target) {
			this.target = target;
		}

		public Object invoke(final Object proxy, final Method method,
				final Object[] args) throws Throwable {
			Object result = null;
			final Class<?> type = method.getReturnType();
			try {
				result = method.invoke(target, args);
			} catch (final InvocationTargetException e) {
				final Throwable t = e.getCause();
				for (final Class<?> ex : method.getExceptionTypes()) {
					if (ex.isAssignableFrom(t.getClass())) {
						throw t;
					}
				}
				throw e;
			}
			result = trace(result, type, true);
			record(result, proxy, method, args);
			return result;
		}

		private void record(final Object result, final Object proxy,
				final Method method, final Object[] args) {
			if (args != null) {
				final Class<?>[] types = method.getParameterTypes();
				for (int i = 0; i < args.length; ++i) {
					declare(args[i], types[i], false);
				}
			}
			if (result != null) {
				b.append(name(result));
				b.append(" = ");
			}
			b.append(name(proxy));
			b.append('.');
			b.append(method.getName());
			b.append('(');
			if (args != null) {
				for (int i = 0; i < args.length; ++i) {
					b.append(name(args[i]));
					if (i < args.length - 1) {
						b.append(',');
					}
				}
			}
			b.append(");\n");
		}

		private final Object target;

	};

	private final StringBuilder b = new StringBuilder();

	private int var = 0;

	private final IdentityHashMap<Object, String> variables = new IdentityHashMap<Object, String>();
	private static final Class<?>[] PRIMITIVES = { String.class, Integer.class,
			Boolean.class, Double.class, Long.class, Short.class, Byte.class,
			Character.class, Float.class };
}

I’ve only used this code with the specific problem I was facing, I’m sure there are issues with it, I haven’t tested it comprehensively, YMMV. For example I know it doesn’t handle arrays very well, particularly those with multiple dimensions.

Maven, JRuby and Rails Migrations

Posted in notetoself by cdivilly on July 20, 2009

Here’s how I managed to get Rails Migrations working with a Maven build process, enabling me to leverage Migrations to maintain the RDBMS schema of a Java application.

Add JRuby and Database dependencies

Add the following dependencies to your pom.xml (adjust for whatever is latest version of jruby and whatever your database is).

  <dependency>
   <groupId>org.jruby</groupId>
   <artifactId>jruby-complete</artifactId>
   <version>1.3.0RC1</version>
  </dependency>
  <dependency>
   <groupId>oracle</groupId>
   <artifactId>jdbc</artifactId>
   <version>11.1.0</version>
  </dependency>

Add a goal to install required Ruby gems

To get Rails migrations running we’ll need the following gems:

  • activerecord
  • activerecord-jdbc-adapter
  • your database specific jdbc adapter

We’ll set up a profile to install these gems if they are missing (based on a approach used by Binil Thomas):

<profiles>
  <profile>
   <id>first.time</id>
   <activation>
    <file>
     <missing>${user.home}/.gem/jruby/1.8/gems/activerecord-2.3.2</missing>
    </file>
   </activation>
   <build>
    <plugins>
     <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>exec-maven-plugin</artifactId>
      <executions>
       <execution>
        <id>install-active-record-gem</id>
        <phase>initialize</phase>
        <goals>
         <goal>java</goal>
        </goals>
        <configuration>
         <mainClass>org.jruby.Main</mainClass>
         <arguments>
          <argument>-S</argument>
          <argument>gem</argument>
          <argument>install</argument>
          <argument>activerecord</argument>
          <argument>activerecord-jdbc-adapter</argument>
          <argument>activerecord-oracle_enhanced-adapter</argument>
          <argument>--no-ri</argument>
          <argument>--no-rdoc</argument>
          <argument>--no-test</argument>
         </arguments>
        </configuration>
       </execution>
      </executions>
     </plugin>
    </plugins>
   </build>
  </profile>
 </profiles>

Replace activerecord-oracle_enhanced-adapter with the name of your chosen database’s jdbc adapter.

Add a goal to run the migration

 <build>
  <plugins>
   <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
     <execution>
      <id>run-migrate</id>
      <phase>install</phase>
      <goals>
       <goal>java</goal>
      </goals>
      <configuration>
       <mainClass>org.jruby.Main</mainClass>
       <arguments>
        <argument>-S</argument>
        <argument>rake</argument>
        <argument>-f</argument>
        <argument>${basedir}/Rakefile</argument>
       </arguments>
      </configuration>
     </execution>
    </executions>
   </plugin>
  </plugins>
 </build>

Create a Rakefile

You’ll need a Rakefile to carry out the migration, in my case I wanted to avoid depending on all of Rails so I just used the following:

require 'active_record'
require 'yaml'

RAILS_ROOT = "#{File.dirname(__FILE__)}" unless defined?(RAILS_ROOT)

task :default => :migrate

task :migrate => :environment do
  ActiveRecord::Migrator.migrate('db/migrate', ENV["VERSION"] ? ENV["VERSION"].to_i : nil )
end

task :environment do
  ActiveRecord::Base.establish_connection(YAML::load(File.open("#{RAILS_ROOT}/config/database.yml")))
  ActiveRecord::Base.logger = Logger.new(File.open('database.log', 'a'))
end

Write your migrations

Create your migrations in db/migrate.

Apply database changes

Finally type:

mvn install

to deploy your database changes.

The full POM

<project>
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.example</groupId>
 <artifactId>schema</artifactId>
 <name>Example POM for doing Rails Migrations</name>
 <packaging>pom</packaging>
 <dependencies>
  <dependency>
   <groupId>org.jruby</groupId>
   <artifactId>jruby-complete</artifactId>
   <version>1.3.0RC1</version>
  </dependency>
  <dependency>
   <groupId>oracle</groupId>
   <artifactId>jdbc</artifactId>
   <version>11.2.0</version>
  </dependency>
 </dependencies>
 <build>
  <plugins>
   <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
     <execution>
      <id>run-migrate</id>
      <phase>install</phase>
      <goals>
       <goal>java</goal>
      </goals>
      <configuration>
       <mainClass>org.jruby.Main</mainClass>
       <arguments>
        <argument>-S</argument>
        <argument>rake</argument>
        <argument>-f</argument>
        <argument>${basedir}/Rakefile</argument>
       </arguments>
      </configuration>
     </execution>
    </executions>
   </plugin>
  </plugins>
 </build>
 <profiles>
  <profile>
   <id>first.time</id>
   <activation>
    <file>
     <missing>${user.home}/.gem/jruby/1.8/gems/activerecord-2.3.2</missing>
    </file>
   </activation>
   <build>
    <plugins>
     <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>exec-maven-plugin</artifactId>
      <executions>
       <execution>
        <id>install-active-record-gem</id>
        <phase>initialize</phase>
        <goals>
         <goal>java</goal>
        </goals>
        <configuration>
         <mainClass>org.jruby.Main</mainClass>
         <arguments>
          <argument>-S</argument>
          <argument>gem</argument>
          <argument>install</argument>
          <argument>activerecord</argument>
          <argument>activerecord-jdbc-adapter</argument>
          <argument>activerecord-oracle_enhanced-adapter</argument>
          <argument>--no-ri</argument>
          <argument>--no-rdoc</argument>
          <argument>--no-test</argument>
         </arguments>
        </configuration>
       </execution>
      </executions>
     </plugin>
    </plugins>
   </build>
  </profile>
 </profiles>
</project>

Comments Off

Where is osso1013?

Posted in notetoself by cdivilly on July 20, 2009

When setting up single sign on for an Oracle Application Server 10.1.3.1 instance, I need to run the osso1013 script, and I could not recall its location, so for future reference it is:

$ORACLE_HOME/Apache/Apache/bin/osso1013

where $ORACLE_HOME is the location of the Oracle Application Server installation

Comments Off

links for 2009-07-16

Posted in links by cdivilly on July 17, 2009

Comments Off

Add Oracle JDBC Jar to Maven Repository

Posted in notetoself by cdivilly on July 16, 2009

If a jar is not available from a public repository (for example, Oracle JDBC jars), then you can get Maven to manually install a copy of the jar in the local repository. To manually deploy the Oracle JDBC driver to your local repository type the following:

mvn install:install-file -Dfile={ORACLE_HOME}/jdbc/lib/ojdbc6.jar -Dpackaging=jar\
 -DgroupId=com.oracle -DartifactId=ojdbc6 -Dversion=11.1.0

where {ORACLE_HOME} is the path to the Oracle Database installation.

You can then reference the the jar via a dependency declaration like the following:

<dependency>
 <groupId>com.oracle</groupId>
 <artifactId>ojdbc6</artifactId>
 <version>11.1.0</version>
</dependency>
Tagged with: , , ,

Comments Off

Remote Debug JRuby

Posted in notetoself by cdivilly on July 16, 2009
$ jruby -J-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=4000 -J-Xdebug ...
Tagged with: ,

Comments Off

links for 2009-06-30

Posted in links by cdivilly on July 1, 2009

Comments Off

Meta-Pubs Again (Again)

Posted in AtomPub by cdivilly on June 12, 2009

I’m revisiting how to extract the meta-publishing (AtomPub Collection creation) portions of the first Atom Hierarchy Draft into a standalone spec independent of hierarchy considerations. I am contemplating changing the actual process of creating the Collection. I see publishing a new Collection as very akin to publishing an AtomPub Media Resource. An example of the interaction I think makes sense is below:

Imagine a new media type: application/atomcoll+xml, contents of the document are an app:collection element per RFC5023.

Advertising MetaPub Capability

A Collection advertises that it supports the creation of new Collections via the RFC5023 app:accepts mechanism:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app">
  <link rel="self" href="http://example.org/collection"/>
  <app:collection href="http://example.org/collection">
    <title>Collection Feed</title>
    <app:accept>application/atomcoll+xml</app:accept>
    <app:accept>application/atom+xml;type=entry</app:accept>
  </app:collection>
  ...
</feed>

Creating a New Collection

A client creates a new AtomPub Collection by sending an application/atomcoll+xml document to the above collection URI:

POST /collection HTTP/1.1
Host: example.org
Content-Type: application/atomcoll+xml
Slug: My New Collection
Content-Length: nnn

<collection xmlns="http://www.w3.org/2007/app">
  <title xmlns="http://www.w3.org/2005/Atom">My New Collection</title>
  <categories href="http://example.org/cats/forMain.cats"/>
</collection>

Server responds with:

HTTP/1.1 201 Created
Content-Length: nnn
Content-Type: application/atom+xml;type=entry;charset="utf-8"
Location: http://example.org/my-new-collection.atom

<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <title>My New Collection</title>
  <id>urn:uuid:de46e3a1-e489-41a6-88a6-21e7f0e8e2d8</id>
  <updated>2009-06-12T12:13:46Z</updated>
  <author><name>Daffy</name></author>
  <summary type="text" />
  <content type="application/atom+xml;type=feed"
     src="http://example.org/my-new-collection"/>
  <link rel="alternate" type="application/atom+xml;type=feed"
     href="http://example.org/my-new-collection"/>
  <link rel="edit-media"
     href="http://example.org/my-new-collection.atomcoll" />
  <link rel="edit"
     href="http://example.org/my-new-collection.atom" />
</entry>
  • the content@src attribute points to the feed representation of the newly created Collection. This server also happens to include a link@rel=”alternate” pointing to the same resource
  • the link@rel=”edit-media” points to the Collection’s own metadata. A client can PUT to this URI to update the Collection configuration ( e.g. to add/change app:categories element)
  • Note the document posted to create the collection lacks a collection@href attribute. This is prohibited by RFC5023, what’s people’s feelings on ignoring/relaxing this requirement?

An Alternative Approach

An alternative approach is to stick with posting an Atom Entry document to cause the Collection to be created, but use an Atom Entry Document media type with some kind of parameter qualification that clearly communicates that the purpose of the Entry is to cause the creation of a Collection, so the above example becomes:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app">
  <link rel="self" href="http://example.org/collection"/>
  <app:collection href="http://example.org/collection">
    <title>Collection Feed</title>
    <app:accept>application/atom+xml;type=entry;profile="http://purl.org/atom/metapub"</app:accept>
    <app:accept>application/atom+xml;type=entry</app:accept>
  </app:collection>
  ...
</feed>

POST /collection HTTP/1.1
Host: example.org
Content-Type: application/atom+xml;type=entry;profile="http://purl.org/atom/metapub"
Slug: My New Collection
Content-Length: nnn

<entry xmlns="http://www.w3.org/2005/Atom">
  <title>My New Collection</title>
  <id>urn:uuid:eb53f957-b7c0-4484-bf0b-c0a6948a33f5</id>
  <updated>2009-06-12T12:13:43Z</updated>
  <author><name/></author>
  <summary/>
  <app:categories href="http://example.org/cats/forMain.cats" xmlns:app="http://www.w3.org/2007/app"/>
</entry>

HTTP/1.1 201 Created
Content-Length: nnn
Content-Type: application/atom+xml;type=entry;profile="http://purl.org/atom/metapub";charset="utf-8"
Location: http://example.org/my-new-collection.atom

<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <title>My New Collection</title>
  <id>urn:uuid:de46e3a1-e489-41a6-88a6-21e7f0e8e2d8</id>
  <updated>2009-06-12T12:13:46Z</updated>
  <author><name>Daffy</name></author>
  <summary type="text" />
  <content type="application/atom+xml;type=feed"
     src="http://example.org/my-new-collection"/>
  <link rel="alternate" type="application/atom+xml;type=feed"
     href="http://example.org/my-new-collection"/>
  <link rel="edit"
     href="http://example.org/my-new-collection.atom" />
  <app:categories href="http://example.org/cats/forMain.cats" xmlns:app="http://www.w3.org/2007/app"/>
</entry>

My Two Cents

  • I have a preference for the first approach, the primary motivation of the POST in each case is to cause the creation of a new Collection. I feel posting an ‘AtomPub Collection Document’ (application/atomcoll+xml) makes that intent very clear. The second option seems somehow backwards to me. However my colleague Nikunj has expressed a preference for the second option, and others have also implemented a similar approach. I’d really like to hear others points of view on this.
  • I dislike the Atom Entry boilerplate required by the second approach.
  • A weakness of both approaches is that it is not clear how the client should specify the metadata that may be included in the feed representation of the collection( e.g subtitle, rights, category, logo etc.). Perhaps that points to the need for a third option, similar to the first option but sending a document with a new xml syntax that allows specification of the feed metadata and the collection metadata, something like:
    <collection xmlns="http://purl.org/atom/metapub" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app">
     <atom:title>My Spiffy New Collection</atom:title>
     <app:categories href="http://example.org/cats/forMain.cats"/>
     <atom:feed>
      <atom:subtitle>Its got lots of metadata</atom:subtitle>
      <atom:logo>...</atom:logo>
      ... etc ...
     </atom:feed>
    </collection>

Feedback Please

I’m very keen to get people’s feedback on the above proposals before attempting to draft another I-D revision, please leave your comments here or on the atom-protocol mailing list (http://www.imc.org/atom-protocol/)

Tagged with: , , ,

Comments Off