22 November 2006

VMware guest clock runs fast

I'm running VMWare Server on a dual core laptop with Ubuntu. I was finding the guest clock on my Windows XP guest was running fast. Since I use the Windows guest for running Outlook, I found my meeting reminders were appearing hours early.

The problem appears to be due to cpu frequency scaling. The article Host Power Management Causes Problems with Guest Timekeeping on Linux Hosts has details on solving the problem.

You have to edit a config file to specify your max cpu frequency. You can find it out in a couple of ways:

$ cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq
1833000

or

$ cpufreq-info
cpufrequtils 002: cpufreq-info (C) Dominik Brodowski 2004-2006
Report errors and bugs to linux@brodo.de, please.
analyzing CPU 0:
driver: centrino
CPUs which need to switch frequency at the same time: 0
hardware limits: 1000 MHz - 1.83 GHz
available frequency steps: 1.83 GHz, 1.33 GHz, 1000 MHz
available cpufreq governors: userspace, powersave, ondemand, conservative, performance
current policy: frequency should be within 1000 MHz and 1.83 GHz.
The governor "ondemand" may decide which speed to use
within this range.
current CPU frequency is 1000 MHz.
analyzing CPU 1:
driver: centrino
CPUs which need to switch frequency at the same time: 1
hardware limits: 1000 MHz - 1.83 GHz
available frequency steps: 1.83 GHz, 1.33 GHz, 1000 MHz
available cpufreq governors: userspace, powersave, ondemand, conservative, performance
current policy: frequency should be within 1000 MHz and 1.83 GHz.
The governor "ondemand" may decide which speed to use
within this range.
current CPU frequency is 1000 MHz.

or (verified on Mac)

$ sysctl -a | grep cpufreq
hw.cpufrequency = 2400000000
hw.cpufrequency_max: 2400000000
hw.cpufrequency_min: 2400000000
hw.cpufrequency: 2400000000


Since my max CPU is 1.83 GHz, I added these lines to /etc/vmware/config:

host.cpukHz = 1830000
host.noTSC = TRUE
ptsc.noTSC = TRUE



I restarted VMware. So far, it seems to be keeping time. I think it simply stops the clock getting ahead of itself, then the VMWare Tools time synchronization ensures if it runs slow, it periodically catches up to the correct time.

03 October 2006

Vmware won't start on Edgy Eft

After upgrading to Ubuntu 6.10 Edgy Eft (beta), vmware fails to start.

Problem outlined here: libhal1 0.5.7.1-0ubuntu8 with new dbus breaks vmware.

Comment #4 solves it.

Juniper Networks network connect on Ubuntu

Was trying to get the Java-applet / browser based network connect to work on Ubuntu.

This site Kubuntu Dapper, Juniper SSL VPN got most of the way.

My only issue were the names of the openssl and crypto libraries. Mine were 0.9.8 instead. Aside from that, everything worked.

26 September 2006

Up arrow in Java console input on Linux

[edit] Whaddaya know, this is documented here: http://wiki.python.org/jython/ReadlineSetup

So I was running Jython on Linux which has a console application for entering Jython commands. Unfortunately pressing the up-arrow did not recall the last command in the history. Instead, it printed the escape code for up-arrow (^[[A).

I've seen this problem before on Linux. Not sure if it is limited to Java console applications. I'm not even sure why it occurs. Perhaps something to do with buffered input?

Anyway, I did find a solution: JLine. You don't even have to modify a program to get it to work. You can wrap the invocation of the Java main class as follows:


java -cp jline-0.9.9.jar jline.ConsoleRunner MyMainClass


So my command line for Jython became:

java -classpath "jline-0.9.9.jar:jython.jar:$CLASSPATH" jline.ConsoleRunner org.python.util.jython


You can simply edit the Jython startup script to add the JLine jar and insert the jline.ConsoleRunner before the jython main class.

Up-arrow now recalls the last command. Left, right and down arrow work too.

18 September 2006

Enabling Oracle Managed Files

It turns out I was wrong about Oracle XE not supporting Oracle Managed Files. It is just turned off by default. Here is how to turn it on:


sqlplus system@XE
...
SQL> alter system set db_create_file_dest='/usr/lib/oracle/xe/oradata/';

System altered.


You can confirm it works with:

SQL> create tablespace my_test_tablespace datafile size 1M;

Tablespace created.

SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
/usr/lib/oracle/xe/oradata/XE/users.dbf
/usr/lib/oracle/xe/oradata/XE/sysaux.dbf
/usr/lib/oracle/xe/oradata/XE/undo.dbf
/usr/lib/oracle/xe/oradata/XE/system.dbf
/usr/lib/oracle/xe/oradata/XE/datafile/o1_mf_my_test__2jxoyjpr_.dbf

SQL> drop tablespace my_test_tablespace;

Tablespace dropped.

SQL> exit

Oracle 10g Standard Edition on Ubuntu

I've been using Oracle XE on Ubuntu 6.06 (installed from http://oss.oracle.com/debian). It works fine. However, I was trying to create a database from scripts provided to me that assumed Oracle Managed Files were supported.

That is, the scripts attempted to create a tablespace without explicitly naming the datafile.

This failed on XE. I figured I could install Oracle 10g Standard Edition. However, the installer wouldn't even run. It fails after performing a distribution check.

You can workaround this issue with the following command to run the installer:

./runInstaller -ignoreSysPrereqs

The only other issue I encountered is documented here:
http://forums.oracle.com/forums/thread.jspa?threadID=413032&tstart=0

Make sure that you either remove Oracle XE or override $ORACLE_HOME to point to your new Oracle home before executing those commands.

This also seems to be a good reference http://www.dizwell.com/prod/node/52

13 September 2006

Unnecessary transitive Maven2 dependencies considered harmful

While deploying an EAR, I kept getting this error when instantiating my Spring beans which happen to use Hibernate to talk to the database:


java.lang.ClassCastException: org.jboss.tm.TxManager
at org.hibernate.transaction.JNDITransactionManagerLookup.getTransactionManager(JNDITransactionManagerLookup.java:23)
at org.hibernate.impl.SessionFactoryImpl.(SessionFactoryImpl.java:302)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1176)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:825)


Of course, good old ClassCastException doesn't tell me what class it expected, only what class it got. From looking at the javadocs, its expecting a javax.transaction.TransactionManager. However, org.jboss.tm.TxManager does, in fact, implement that interface.

So what gives? Well, its going to be a classloader issue. Again, ClassCastException (or most likely the caller) fails to realize that a class's name isn't enough to uniquely identify a class. The classloader that loads it is very important.

Given that I'm using Maven2, I figured that I must be including a copy of javax.transaction.TxManager somewhere.

This command will list the contents of all jars:
find -name '*.jar' -exec unzip -l '{}' \;

Redirect that to a text file, then grep for javax/transaction/TransactionManager

I found I was bundling jta-1.0.1B.jar in my ear. How did that get there? Transitive dependencies, of course. I didn't have to guess too hard as to who might be including that: Hibernate. Now, I thought I was already exlcuding the jta dependency. Turns out I'd only done it in a sub-project. I decided to fix it for all hibernate dependencies in my project by adding it in the parent pom.xml:


<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate</artifactId>
<version>3.1.3</version>
<exclusions>
<!--
This is container-provided; we don't need the dependency here;
In fact, java.lang.ClassCastException: org.jboss.tm.TxManager may occur if we
accidentally include a jta jar
-->
<exclusion>
<groupId>javax.transaction</groupId>
<artifactId>jta</artifactId>
</exclusion>
</exclusions>
</dependency>


With that, the problem is solved.

Maven2 antrun plugin and xdoclet issues

I was adding a custom ant task to maven2 pom.xml. It worked just fine when I built from the sub-project. E.g:

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<configuration>
<tasks>
<echo>Hello</echo>
</tasks>
</configuration>
</plugin>


However, when I stepped back up to the parent project and ran the build, it failed with:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Internal error in the plugin manager executing goal 'org.apache.maven.plugins:maven-antrun-plugin:1.0:run': Unable to find the mojo 'org.apache.maven.plugins:maven-antrun-plugin:1.0:run' in the plugin 'org.apache.maven.plugins:maven-antrun-plugin'
Component descriptor cannot be found in the component repository: org.apache.maven.plugin.Mojoorg.apache.maven.plugins:maven-antrun-plugin:1.0:run.


It turns out that when building from the parent build, XDOclet is used by certain sub-projects. XDoclet depends on the maven antrun plugin version 1.0. With Maven 2.0.4 the default antrun plugin is 1.1. Thus, a conflict arises.

Found this reference: http://jira.codehaus.org/browse/MANTRUN-37

While there is no fix (XDoclet can't [yet?] be modified to use antrun 1.1, there is a workaround: Simply specify version 1.0 of the antrun plugin. I did this in pluginManagement in my parent pom.xml:


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.0</version>
</plugin>

Maven2 ejb-client woes

The Maven2 ejb plugin can automatically generate an ejb-client for you:


<plugin>
<artifactId>maven-ejb-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<generateClient>true</generateClient>
</configuration>
</plugin>


This can be dependend-upon from other projects:


<dependency>
<groupid>com.shopzilla.ca.bidding.impl.google</groupid>
<artifactid>ejbs</artifactid>
<type>ejb-client</type>
</dependency>


Unfortunately, it often includes many classes / resources that you don't want to provide to the client. The default excludes get rid of some stuff. In my case, I had a beanRefContext.xml Spring config file in there that broke clients (since it referred to other Spring xml files that are in a server-side jar).

Supposedly you can add <clientexcludes> elements:

<clientexcludes>
<clientexclude>beanRefContext.xml</clientexclude>
<clientexclude>META-INF/ejb-jar.xml</clientexclude>
<clientexclude>META-INF/jboss.xml</clientexclude>
<clientexclude>**/mdb/</clientexclude>
</clientexcludes>


Sadly, that doesn't work with Maven 2.0.4 due to a bug in the ejb plugin: http://jira.codehaus.org/browse/MEJB-19

My workaround was to add an Ant task that rather heavy-handedly simulated this by unzipping the client jar and re-zipping with the exlcuded files. here it is:

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- Manually strip client files since clientExcludes doesn't work on ejb plugin -->
<tasks>
<property name="ejb-client-excludes-list" value="beanRefContext.xml, META-INF/ejb-jar.xml, META-INF/jboss.xml, **/mdb/" />
<echo>Manually excluding from ejb client jar: ${ejb-client-excludes-list}</echo>
<property name="ejb-client-expanded.dir" value="${project.build.directory}/tmpClient" />
<property name="ejb-client-artifact-name"
value="${project.build.directory}/${project.artifactId}-${project.version}-client.jar" />
<delete dir="${ejb-client-expanded.dir}" />
<unzip src="${ejb-client-artifact-name}" dest="${ejb-client-expanded.dir}" />
<delete file="${ejb-client-artifact-name}" />
<zip destfile="${ejb-client-artifact-name}" basedir="${ejb-client-expanded.dir}"
excludes="${ejb-client-excludes-list}" />
</tasks>
</configuration>
</plugin>

01 September 2006

Sending emails in Spring using Freemarker

Create a mail sender

<bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
<property name="host" value="localhost">
<property name="port" value="25">
</bean>


Create the freemarker configuration

<bean id="freemarkerMailConfiguration" class="org.springframework.ui.freemarker.FreeMarkerConfigurationFactoryBean">
<property name="templateLoaderPath" value="/WEB-INF/mail">
</bean>


Create a class for sending the mail:

public class InfoSenderMail implements InfoSender {

private Configuration configuration;
private String freemarkerTemplate;
private MailSender mailSender;

public void send() throws SendException {
SimpleMailMessage message = new SimpleMailMessage();

// Add stuff to them model for inserting into the template
final Map model = new HashMap();
model.put("name", "Tim");

// Merge the model into the template
final String result;
try {
result = FreeMarkerTemplateUtils.processTemplateIntoString(configuration.getTemplate(
freemarkerTemplate), model);
} catch (IOException e) {
throw new SendException(
"Unable to read or process freemarker configuration or template", e);
} catch (TemplateException e) {
throw new SendException("Problem initializing freemarker or rendering template '"
+ freemarkerTemplate + "'", e);
}

// Construct and send the message
message.setTo("someone@somewhere.com");
message.setSubject("Here is the info you requested");
message.setText(result);

mailSender.send(message);
}

public void setFreemarkerMailConfiguration(Configuration configuration) {
this.configuration = configuration;
}

public void setFreemarkerTemplate(String freemarkerTemplate) {
this.freemarkerTemplate = freemarkerTemplate;
}

public void setMailSender(MailSender mailSender) {
this.mailSender = mailSender;
}

}


Define the bean; the template should be in the location specified by templateLoaderPath (e.g. /WEB-INF/mail)


<bean id="infoSender" class="InfoSenderMail"
<property name="mailSender" ref="mailSender">
<property name="freemarkerMailConfiguration" ref="freemarkerMailConfiguration">
<property name="freemarkerTemplate" value="mytemplate.txt">
</bean>


Create the template. E.g. mytemplate.txt


Hello, ${name}!


Then simply inject the bean in another class and invoke send();

24 August 2006

MSN Messenger doesn't like Hibernate code?

So I was in the middle of an IM conversation using MSN messenger when I pasted some java code which uses Hibernate's APIs when I got this error:

Message could not be sent because a connection error occurred


Weird... everything else I'd typed was fine. Subsequent messages were fine. Here is what I tried to send:

ScrollableResults scrollableResults = query.scroll();

Of course, silly me! MSN messenger doesn't let you send a message with the characters ".scr" in it. Thanks to Wikipedia for a full explanation.

14 March 2006

Classpath Gymnastics: JAXP, Java5 and JBoss


Started to get a problem with JBoss and Java5. Not sure what caused it to _start_ occurring. I'm using isolated EAR deployments and excluding all xml-apis and xerces jars from my EARs since all this stuff should be provided by the java runtime.



Anyway, started to get this error:


XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: No XPathFctory implementation found for the object model: http://java.sun.com/jaxp/xpath/dom



It turns out you can enable jaxp debugging with the following system property:
-Djaxp.debug=true

This resulted in some useful messages:


08:17:52,874 INFO [STDOUT] JAXP: using thread context class loader (org.jboss.util.loading.DelegatingClassLoader@11c9fcc) for search
08:17:52,875 INFO [STDOUT] JAXP: Looking up system property 'javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom'
08:17:52,875 INFO [STDOUT] JAXP: The property is undefined.
08:17:52,876 INFO [STDOUT] JAXP: found null in $java.home/jaxp.properties
08:17:52,876 INFO [STDOUT] JAXP: no META-INF/services/javax.xml.xpath.XPathFactory file was found
08:17:52,876 INFO [STDOUT] JAXP: attempting to use the platform default W3C DOM XPath lib
08:17:52,876 INFO [STDOUT] JAXP: instanciating org.apache.xpath.jaxp.XPathFactoryImpl
08:17:52,877 INFO [STDOUT] JAXP: failed to instanciate org.apache.xpath.jaxp.XPathFactoryImpl
08:17:52,880 INFO [STDOUT] java.lang.ClassNotFoundException: No ClassLoaders found for: org.apache.xpath.jaxp.XPathFactoryImpl



Looking at the JAXP sources revealed the problem. In the source for Java5 javax.xml.xpath.XPathFactoryFinder we see this:


debugPrintln("attempting to use the platform default W3C DOM XPath lib");
return createInstance("com.sun.org.apache.xpath.internal.jaxp.XPathFactoryImpl");


In the log, we see this message: instanciating org.apache.xpath.jaxp.XPathFactoryImpl

com.sun.org.apache.xpath.internal.jaxp.XPathFactoryImpl != org.apache.xpath.jaxp.XPathFactoryImpl



So why is this? Looking at the JBoss endorsed directory at /lib/endorsed we can see it contains xml-apis.jar which contains javax.xml.xpath.XPathFactoryFinder which overrides the Java5 version. That overridden version tries to lookup org.apache.xpath.jaxp.XPathFactoryImpl but that doesn't exist in Java5; it was renamed. And since I've excluded all xml jars from my EARs, I don't have that class anywhere.



Of course, I'm not the first to have this problem.


I simply renamed the lib/endorsed directory and restarted. This seemed to work, although based on the second link above, there might be more work to do later.

24 February 2006

MacBook Pro vs Dell Pentium M w/ Ubuntu Linux


Matt Raible documents his experience with unwrapping his new Duo Core MacBook Pro.

For grins, I decided to compare his performance measurements with my 10-month old Dell Inspiron 9300 laptop. Obviously this test is far from scientific. Anyway, here are my specs:

Dell Inspiron 9300
Intel Pentium M @ 2Ghz
2Gb RAM
60gb 7200 RPM HDD
Ubuntu Linux 5.10 w/ Kernel 2.6.12-10-686

This laptop cost less than $3K in April 2005. Here is the precise cpuinfo specs:
vendor_id : GenuineIntel
cpu family : 6
model : 13
model name : Intel(R) Pentium(R) M processor 2.00GHz
stepping : 8

And here are my results. Note: I ran each test 4 times.

ant clean package-web
16 seconds
14 seconds
13 seconds
13 seconds

ant setup test-all
1 minute 25 seconds
1 minute 21 seconds
1 minute 23 seconds
1 minute 23 seconds

His results on the new MacBook Pro were 12 seconds for the first test and 1 minute 28 seconds for the second test.

Conclusion? Its all about the file-system and cacheing. Running OS X or Linux greater improves file IO, 2 Gb RAM allows the OS to aggressively cache disk access and a 7200 RPM HDD offers substantial performance improvements when it does have to hit the disk.

In my opinion CPU and clock speed matters the least.

Here is how I reproduced his tests:
Prerequsite:
* Mysql installed and running;
* CATALINA_HOME environment variable pointing to Tomcat 5.5 installation

cvs -d :pserver:guest@cvs.dev.java.net:/cvs login
cvs -d :pserver:guest@cvs.dev.java.net:/cvs checkout appfuse
cd appfuse
ant clean package-web
ant clean package-web
ant clean package-web
ant clean package-web
ant setup test-all
ant setup test-all
ant setup test-all
ant setup test-all

09 February 2006

Maven2 and xdoclet and antrun

I found it very difficult to get the maven2 xdoclet plugin to generate my struts config xml file.

I first tried this

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>xdoclet-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<configuration>
<tasks>
<webdoclet destdir="target/generated-sources/xdoclet" mergedir="src/main/xdoclet" verbose="true">
<fileset dir="src/main/java"/>
<strutsconfigxml validatexml="true" version="1.1" destinationFile="struts-config.xml" />
</webdoclet>
</tasks>
</configuration>
<phase>generate-sources</phase>
<goals>
<goal>xdoclet</goal>
</goals>
</execution>
</executions>
</plugin>


which procedded to generate an empty struts-config.xml (well, it merged in my merge files but I had no actions or forms). It turns out that since my Forms and Actions inherit from a sub-class defined elsewhere, that class must be available in the classpath of the plugin. Unfortunately, there does not seem to be a way to modify a plugin's classpath.

I tried adding dependencies to the plugin, but none of that worked.

I tried the xdoclet2 maven2 plugin; it produced garbage struts-config.xml files; it seems to screw up the merge files.

I finally settled on using the antrun plugin to use the ant tasks. This allowed me to modify the webdoclet's classpath:

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<configuration>
<tasks>
<taskdef name="webdoclet" classname="xdoclet.modules.web.WebDocletTask" classpathref="maven.compile.classpath" />
<webdoclet destdir="${project.build.directory}/generated-sources/xdoclet" mergedir="src/main/xdoclet" verbose="true">
<fileset dir="src/main/java" />
<strutsconfigxml validatexml="true" version="1.1" destinationFile="struts-config.xml" />
</webdoclet>
</tasks>
</configuration>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>

What do you know, it worked!

07 February 2006

Struts Menu - Velocity Rendering


Struts Menu Tag Library - Overview

I recently started using Struts-menu to replace a home-grown menu system. It works well, although there were a couple of small speedbumps along the way, mostly with the Velocity rendering:

  1. Velocity rendered didn't invoke my custom permissions adapter for all menu items; it only checked one level deep.

  2. The Velocity renderer modifies the application-scoped menu definition when checking permissions; thus if permissions are on a per-user (i.e. per-session) basis, each user trashes the other users' menus.

  3. I also found that I couldn't really define a menu that was a Struts action in another module; the menu always appended the current module to the generated link.


To solve (1) I modified the VelocityMenuDisplayer to recurse into all levels. I submitted a JIRA issue with a patch: http://issues.appfuse.org/browse/SM-2

To solve (2) I defined a Session-listener (very similar to MenuContextListener) that loaded the menu definition into session scope. The taglib is already coded to find the menu definition at the lowest scope, so it picked it up from there.

To solve (3) I added a "module" attribute to the menu and modified the taglib to compute the URL with the module (if any). I submitted a JIRA issue with a patch: http://issues.appfuse.org/browse/SM-1

Here is my session-scoped listener. There is a lot of duplication with the MenuContextListener class.


import javax.servlet.http.HttpSessionEvent;
import javax.servlet.http.HttpSessionListener;
import net.sf.navigator.menu.MenuRepository;
import net.sf.navigator.util.LoadableResourceException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

/**
* Session listener for loading Struts-menu into session.
* <p>
* This is necessary since the Velocity displayer trashes the menu based on permissions.
* Thus we can't store it in application scope.
* </p>
* @author Tim Morrow
* @since Feb 6, 2006
*/
public class MenuSessionContextListener implements HttpSessionListener {

private static final Log log = LogFactory.getLog(MenuSessionContextListener.class);

private static final String DEFAULT_MENU_CONFIG_LOCATION = "/WEB-INF/menu-config.xml";

public void sessionCreated(HttpSessionEvent event) {

if (log.isDebugEnabled()) {
log.debug("Starting struts-menu initialization");
}

String menuConfig = DEFAULT_MENU_CONFIG_LOCATION;

// check for menuConfigLocation context-param
String override = event.getSession().getServletContext().getInitParameter("menuConfigLocation");
if (override != null) {
if (log.isDebugEnabled()) {
log.debug("using menuConfigLocation: " + override);
}
menuConfig = override;
}

MenuRepository repository = new MenuRepository();
repository.setLoadParam(menuConfig);
repository.setServletContext(event.getSession().getServletContext());

try {
repository.load();
event.getSession().setAttribute(MenuRepository.MENU_REPOSITORY_KEY, repository);

log.info("struts-menu successfully loaded into session scope from " + menuConfig);

} catch (LoadableResourceException lre) {
log.fatal("Failure initializing struts-menu: " + lre.getMessage());
}
}

public void sessionDestroyed(HttpSessionEvent event) {
event.getSession().removeAttribute(MenuRepository.MENU_REPOSITORY_KEY);
}

}


Usage is the same as the servlet context MenuContextListener:

<listener>
<listener-class>MenuSessionContextListener</listener-class>
</listener>