Memory Leak Update

I have been using Hibernate with the default C3P0 connection pooling.

I have been receiving PERM GEN errors.

After reading Frank Kievet's blog about memory leaks, I started investigating.
http://frankkieviet.blogspot.com/2006/10/classloader-leaks-dreaded-permgen-space.html
http://frankkieviet.blogspot.com/2006/10/how-to-fix-dreaded-permgen-space.html

I added his code for causing a garbage collection in the perm gen and noticed that a lot of memory was being lost on each redeploy of the web app.

I downloaded the Eclipse, stand-alone Memory Analyzer (MAT).
http://www.eclipse.org/mat/

My steps to find the leak:

1. I ran the web app and used MAT to obtain a heap dump from Tomcat. 
2. I reran the web app again and obtained another heap dump. 
3. I opened the dominator tree for each dump.
4. The second dump contained the web app loader from the first dump. This class is causing the class loader memory leak.
5. In the second dominator tree, I right-clicked the stale web app loader and selected Path to GC Roots (excluding weak references).
6. The references listed are the ones causing the memory leak.

The leaks are usually for a third-party tool. It is important to close these tools properly. The tools I am using are Hibernate, C3P0 and MySQL. Each of these has to be closed properly. The best place for the code is in a SevletContextListener. I used several sites to find this information.
https://hibernate.atlassian.net/browse/HHH-7364
http://stackoverflow.com/questions/11872316/tomcat-guice-jdbc-memory-leak
http://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-23.html

I also had to update several jar files to newer versions:
hibernate-c3p0-4.1.1.Final.jar
mysql-connecto-java-5.1.28-bin.jar

package shared;

import com.mysql.jdbc.AbandonedConnectionCleanupThread;
import java.sql.Driver;
import java.sql.DriverManager;
import java.util.Enumeration;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;

@WebListener
public class WebappListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent sce) {
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        try {
            AbandonedConnectionCleanupThread.shutdown();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        try {
            Enumeration enumer = DriverManager.getDrivers();
            while (enumer.hasMoreElements()) {
                DriverManager.deregisterDriver(enumer.nextElement());
            }
        } catch (java.sql.SQLException se) {
            se.printStackTrace();
        }
        shared.HibernateHelper.closeFactory();
    }
}

The HibernateHelper class is a helper class for using Hibernate. All the methods are static. It has a static variable for the session factory that already exists.

    static public void closeSessionFactory(SessionFactory factory) {
        if (factory != null) {
            if (factory instanceof SessionFactoryImpl) {
                SessionFactoryImpl sf = (SessionFactoryImpl) factory;
                ConnectionProvider conn = sf.getConnectionProvider();
               
                if (conn instanceof C3P0ConnectionProvider) {
                    ((C3P0ConnectionProvider) conn).close();
                }
            }
            factory.close();
        }
    }

    static public void closeFactory() {
        closeSessionFactory(sessionFactory);
    }

The relevant imports for this method are:

import org.hibernate.SessionFactory;
import org.hibernate.internal.SessionFactoryImpl;
import org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider;
import org.hibernate.service.jdbc.connections.spi.ConnectionProvider;

Starting with JSON

I am using jQuery in an application to build a user interface that creates a tree of classes. I want to use that tree as a template in another application that looks for a similar structure.

I am trying to use JSON to save the tree from the first app, read it in the second app, convert it to a Java object and then compare it to the other structure.

I took a quick look at JSON.simple, but I only looked at one post where the advice was to use either Jackson or GSON. I will try GSON first.

Download: http://code.google.com/p/google-gson/downloads/list

Tutorial: http://www.studytrails.com/java/json/java-google-json-parse-json-to-java.jsp

I have successfully completed my task.

1. Created JSON from the object structure using jQuery and javascript.
2. Added the JSON to a hidden field that is submitted to the server application.
3. In the server application using Java and GSON, I read the JSON data and created a class hierarchy from it.

These are the methods that created the JSON from my object structure. There are two structures, TextPart and Group. Group contains TextPart and other Groups. TextPart contains text and another field to indicate if the text is equivalent to another TextPart.
    
    //grab the text, replacing " with \".
    function getTextPartText(textPart) {
        return $.trim($(textPart).children('code').text()).replace(/\"/g, "\\\"");     
    }
  
    //grab the text, replacing " with \". Change to lower case, change space to underscore.
    function getTextPartEquivalent(textPart) {
        var result = $.trim($(textPart).children('var').text().toLowerCase());
        result = result.toLowerCase().replace(/\s/g,'_');
        return result.replace(/\"/g, "\\\"");     
    }
  
    function textPartToJson(textPart) {
        return '{ ' +
                '"textPart" : "' + getTextPartText(textPart) + '"' +
                ', "equivalent" : "' + getTextPartEquivalent(textPart) + '"' +
               '}';
    }
  
    function groupToJson(group) {
        var state, result, children, comma, classAttr, i, radioGroup;
      
        state = "sequential";      
        radioGroup = $(group).children('.radioGroupType');
        if (radioGroup.length > 0) {
            if (radioGroup.children(".groupTypeRandom").prop('checked')) {
                state = "random";
            }
        }
        result = '{ "groupType":"' + state + '", "children": [';
        children = $(group).children();
        comma="";
        for (i = 0; i < children.length; i++) {
            classAttr = $(children[i]).attr('class');
            if ( classAttr === 'group') {
                result += comma + groupToJson(children[i]);         
                comma = ", ";
            } else if ( classAttr === 'textPart') {
                result += comma + textPartToJson(children[i]);                
                comma = ", ";
            } else {
                console.log("warn: skipping object in groupToJson: ", classAttr);
            }
        }
        result += "]}";
        return result;
    }

In the server application, create an equivalent object structure using GSON.

Object processJsonObject(JsonObject object) {
        if (object.has("children")) {
            Group group = new Group();
            group.setGroupType(GroupType.valueOf(object.get("groupType").getAsString().toUpperCase()));
            group.setChildren(processJsonArray(object.getAsJsonArray("children")));
            System.out.print("---start group");
            System.out.println(String.format(" (%s) ---", object.get("groupType").getAsString()));     
            processJsonArray(object.getAsJsonArray("children"));
            System.out.println("---end group---");
            return group;
        } else if (object.has("textPart")) { 
            TextPart textPart = new TextPart();
            textPart.setTextPart(object.get("textPart").getAsString());
            textPart.setEquivalent(object.get("equivalent").getAsString());
            System.out.print(object.get("textPart").getAsString());       
            System.out.println(String.format(", %s", object.get("equivalent").getAsString()));
            return textPart;
        } else {
            return null;
        }
    }
   
    Object[] processJsonArray(JsonArray array) {
        ArrayList list = new ArrayList();
        for (JsonElement element : array) {
            if (element.isJsonArray()) {
                list.add(processJsonArray((JsonArray)element));
            } else if (element.isJsonObject()) {
                list.add(processJsonObject((JsonObject)element));
            } else {
                System.out.println(String.format("No object or array: %s", element));
            }
        }
        return list.toArray();
    }

I added a serialize and deserialize routine for saving the object structure. This allows the structure to be referenced by other applications.

public void serialize() {
      try
      {
         FileOutputStream fileOut =
           new FileOutputStream(context.getRealPath("/WEB-INF/templates/example.rote"));
         ObjectOutputStream out = new ObjectOutputStream(fileOut);
         out.writeObject(getRoot());
         out.close();
         fileOut.close();
         System.out.printf("Serialized data is saved in /WEB-INF/templates/example.rote");
      }catch(IOException i)
      {
         i.printStackTrace();
      }
    }
   
    public void deserialize() {
        try
      {
         FileInputStream fileIn =
           new FileInputStream(context.getRealPath("/WEB-INF/templates/example.rote"));
         ObjectInputStream in = new ObjectInputStream(fileIn);
         setRoot((Group) in.readObject());
         in.close();
         fileIn.close();
      }catch(IOException i)
      {
         i.printStackTrace();
         return;
      }catch(ClassNotFoundException c)
      {
         System.out.println("Example class not found");
         c.printStackTrace();
         return;
      }
    }

In the other application, I read the file that was created by the first application.

Using jQuery

Download jQuery form http://jquery.com/download/

Add a script tag to the HTML page for jQuery. I am using Tomcat, so I am using JSP.

    <script src="${pageContext.request.contextPath}/jquery/jquery-2.0.3.min.js">

Place some jQuery inside another script tag.

        <script>
        $( document ).ready(function() {
            console.log( "ready!" );
        });
        </script>

Found a great tool for playing with jQuery: http://jsfiddle.net/

Kiteboard Helmet Search

I am looking for a helmet for kite boarding. I have a large head. With short hair, my head is 24-1/4". My local stores do not have helmets that fit. I am looking for a helmet that fits and has good safety features.

I read this from an excellent post:

"Liner: There are three main types of insulation. Expanded Poly-Propylene (EPP) is the most common. It’s very stiff, not affected by moisture, and will take multiple impacts, according to Halstead. Expanded Poly-Styrene is "the very best" energy attenuating substance. It’s what’s used in bike helmets, but generally is only expected to be hit once, and with one exception noted below isn’t used in whitewater helmets. Finally, there are closed-cell foams. The best of these are otherwise known as Vinyl Nitrile, which is a PVC synthetic rubber, heavier in a safe density than EPS/EPP, and will absorb water, but it is usually much softer. A closed-cell foam will also absorb sweat along with water, which can make for a ‘funky’ brain bucket. A fourth insulation, Ethyl Vinyl Acetate or EVA, "isn’t a functional energy attenuator because it’s so springy," says Halstead.

Halstead says stiffer is better. "When you look at the stuff that will actually protect your head, it ought to be so stiff that you think, ‘my God, this is going to hurt.’ But you need that much stiffness to spread the impact. If you pick up a helmet with soft foam inside, the chance of it protecting you is slim." Of course, not everyone agrees with Halstead. Jay Norfleet from Grateful Heads thinks EPP is not a multiple-impact foam because it compresses. And Doug Poe of Pro-Tec believes EVA gives the same protection as closed-cell foam."

It is impossible to find a water helmet that fits a large head. My head is 62.5 cm. That is the upper limit of some helmets and above the limit for most.

Liquid Force XL 24in

ShredReady makes some helmets that are one-size-fits-all. I bought the Standard cut helmet and removed all the padding. I also cut away some of the mesh liner. It fits, barely.

Using Yii Framework

I have downloaded the Yii plugin for NetBeans, but first I had to install NetBeans 7.3.

Download the .nbm file for the plugin.
Go to Tools -> Plugins and add a plugin.
Navigate to the location of the .nbm file.

Immediately after installing the plugin, I noticed that there were some updates for it.
After update, Yii was no longer installed. Actually, it is still there, but it is part of PHP. Click 'show details' to see that Yii is installed.

Yii documentation is asking me to set the path to yiic.php. I cannot find it on my Mac. Ignored it and continued.

Create new PHP project in NetBeans and set the framework to Yii. It is taking a loooonnnngggg time to create the project. After 5 minutes, I closed the new project wizard. Lather, rinse, repeat. I am able to open the project. When I view properties, it indicates that yii is part of the project.

I am trying this tutorial: http://www.larryullman.com/series/learning-the-yii-framework/

Now I see. The plugin is not Yii. Yii must be installed separately. Download Yii from http://www.yiiframework.com/download/. Lather, rinse, repeat.

In NetBeans, set the path to yiic.php from Preferences -> PHP -> Yii.

Create new PHP project, select PHP framework. Project has been created with lots of files.

Uploaded framework and requirements folder to remote PHP server. I accessed /yii/requirements to test the PHP installation for conformance with Yii. All passed, with some warnings for databases that I am not using.

PHP does not run out of the box with NetBeans. It is necessary to install PHP, Apache and MySql. Download MAMP at http://www.mamp.info/en/index.html.

Tutorial for installing: https://netbeans.org/kb/docs/php/configure-php-environment-mac-os.html.
There is no MYSQL option in the Services -> Databases. Trying to add new connection.

I already had a MYSQL server set up. I deleted it and was able to access the server config window.

Set up admin properties for MAMP, not MySql tool. Am able to start/stop mysql from IDE.

Created PHP project. Created index.php in Source Files. Ran project. Index file appeared.

Created a Yii project. Be sure to set the path to yii.php or the require statement will not work and the page will be empty.

Configured Yii to use Gii to create models, views, controllers. When creating the model, I had to rename some columns in the database that had embedded spaces.

http://localhost/appname/index.php/gii/default/index

Gii created the model, but only in the destination folder, not in the NetBeans source folder.

Created new NetBeans project in htdocs of Apache. Now I can use Gii to generate code. I do not like using htdocs. I will add alias that points to location of PHP projects.

Alias /php "/Users/xxx/yyy/zzz/php"

I had to stop Apache before adding the alias.

I edited the URL for the configuration (config/main.php) for the PHP project to include the new alias.

When I use Gii to generate code from the URL, the NetBeans project is updated. The source for the project is in the user folder, not in the MAMP application folder.

I bought the book Yii Application Development Cookbook Second Edition by Alexander Makarov. It has more information than the above tutorial. The tutorial allowed my to get my app running, but the book is showing me the inner workings of Yii.

Be careful not to set up a general route like => website/page, as it will prevent gii from running.

 MAMP has phpMyAdmin.

Edit the protected/components/UserIdentity.php file to add authenticated users. Modify the accessRules in each controller to give access rights to each user.

Modify .htaccess

Modify .htaccess in the root of the project folder, so that index.php does not have to be added to each URL. I had to modify the Apache server httpd.conf to allow options to be overridden in .htaccess.

Options +FollowSymLinks
RewriteEngine on

RewriteBase "/alias-to-php-projects/"
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule (.*) php-project-name/index.php/$1

Place this in the .htaccess folder that hosts the yii application.

Alternate Rewrite

Instead of repeating this in all the .htaccess files, I used a Directory tag in httpd.conf for Apache. I created an Alias to a folder that contains all my Yii projects, then add the rewrite for that folder.

  Alias /php "/Users/xxx/yyy/zzz/php" 

   
   
        Options +FollowSymLinks
        RewriteEngine on

        RewriteBase "/php/"
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d  

        RewriteRule ^([^/]*)/(.*)$ $1/index.php/$2

   

The regex strips out the first part of the path and treats it as the name of the project, then appends index.php, then appends the rest of the path.

 Revisited Several Months Later

Several months after the initial controller, I wanted to create another one. Of course, I had forgotten everything!

Important files:

• components/UserIdentity for adding users
• config/main.php for defining URLs and other configurations
• controllers
• models
• views

Renaming PHP project for NetBeans

To rename the PHP project, I had to copy the project to a new folder with the new name, then use the new project in NetBeans and delete the old one.

Change the project URL in the Run Configuration of Properties.

This does not work for a yii project, since the model was created for the original app.

MIME_PDF missing from org.apache.fop.apps.MimeConstants

I am building a Java app that uses Apache FOP.

I downloaded the source from http://xmlgraphics.apache.org/fop/download.html

I tried creating an instance with
            Fop fop;
            fop = fopFactory.newFop(MimeConstants.MIME_PDF, out);

I received the error that MIME_PDF did not exist.

I neglected to include the xmlgraphics-commons jar that is in the build folder of the fop install.

These are the jar files I needed to include:

xmlgraphics-commons
avalon-framework
fop

Next, I received an error for any XSL files that used the ends-with xpath function. I needed two more jars from the fop build folder.

serializer
xalan

Screen Casts on Mac

Screen casts can be created on a Mac using QuickTime. To improve resolution, use a selection rectangle that is smaller than the screen. Be sure to maintain the correct aspect ratio, or parts of the final video will be cropped.

I use dimensions of 760x570 when creating screen casts of applications (4:3 aspect ratio).

After creating the video, I can share it to iMovie and edit it.
Share -> iMovie

In iMovie, Import -> Movies and select the screen recording. Create a new project and new event. The event will hold the movie clips. Drag clips into the project.

Share -> Export to QuickTime as MPEG-4

MPEG-4 was scaled too small. I exported it to youtube and it was full sized.

Virtual Box Additions for Fedora

When installing the vbox additions in Fedora 16, I encountered errors.

First, I had to install a kernel development kit.

The suggestion kit was not found:
yum install kernel-devel-3.6.11.4.fc16.i686

I tried command completion and was suggested to use
yum install kernel-devel.i686

It was found and installed.

I switched to the /media folder where the vbox additions .iso was mounted and switched to the vbox folder. I ran the VBoxLinuxAdditions.run. It could not find gcc.

I ran
yum install gcc

It installed.

Once again, I ran
./VBoxLinuxAdditions.run

No good.

I looked in the vbox additions /ver/log/vboxadd-install.log file and saw that my kernel needed updating.
After doing rpm -qa | grep kernel I saw that two kernels were installed. I removed the older one.

Tried guest. Failed.

Rebooted.

Tried guest. Failed.

Ran /etc/init.d/vboxadd setup

Tried guest. Failed.

Rebooted.

Tried guest. Failed.

Ran yum update -y. Lots of stuff to update. Install 29, update 409.

Tried guest. Failed.

Ran  /etc/rc.d/init.d/vboxadd setup

Tried guest. Failed.

Rebooted.

Tried guest. Failed.

Tried the recommendation from the vbox log file
make oldconfig
make prepare
I ran these in the source folder for the installed kernel. Error.
no rule to make .../syscall_32.tbl

Trying to rebuild kernel with
make xconfig

Cannot find qt
yum install qt4-devel

I still cannot install the guest additions.

I read in a bug report that getting the additions from RPM Fusion will work.

Went to this page and installed the free and nonfree versions of RPM Fusion
http://rpmfusion.org/Configuration

From the GUI add/remove program application, I searched for guest additions and then installed the Virtual Box Guest Additions. IT WORKED!!!

Automating Book Edits

Automating Book Edits

When I published my book, I used NetBeans and Tomcat to automate some processes.

I created special tags for the index and the table of contents.

The IndexController determines what resources can be accessed. All additional resources can be accessed from the IndexController: preface, references, TOC, appendix, index pages for each chapter.

The anchor tags add information to a static list when the tag is used. The TOC tag writes the list to the TOC file.

The table of contents is generated when the TOC is requested from the IndexController:
http://localhost:8084/book/Index/TOC/

The index is processed

Connecting Mac to AppleTV

I am visiting a friend who has AppleTV. I wanted to show some photos.

My friend already had set up the AppleTV. He switched the input on the TV to the AppleTV.

I found a manual for AppleTV at Setup Guide

From my computer, I did the following

• In iTunes, select File -> Home Sharing -> Turn On Home Sharing. The documentation indicated this menu was under Advanced, but it was in File -> Home Sharing.
• Open the same menu and select Choose Photos to Share with AppleTV. It is better to limit the number of photos to share, if you have a lot of photos. It takes a long time for AppleTV to load a lot of photos.
• I had to have my friend authorize AppleTV to use my computer. Only five computers can be authorized.  

Viewing a slideshow.

• From AppleTV, select Computer and then Photos.
• From the settings, select the type of the slideshow. Leaving the mouse over an option will preview the option.
• Move to the top of the slideshow list for some options on music. Select music for the current slideshow or set default music from the library of the computers connected to AppleTV.

On a separate topic, iTunes has another menu for connecting to AppleTV to play music only.

• In iTunes, find the rectangle with the arrow in it. I could not find it where the documentation indicated, but I did locate it next to the progress bar for playing a song. 
• Select AppleTV from the list. 
• When you play music in iTunes, it will play through AppleTV.
• Playing music directly from iTunes will override the soundtrack from a slideshow in AppleTV.

Servlets in Eclipse

Update Eclipse so it can use Tomcat to Run Servlets

First, add ability to run servlet in Eclipse.

1. Download Eclipse with Java EE.
2. Add extensions for JAX-WS from
   http://wiki.eclipse.org/images/b/bf/Jaxws-1.0.0-201003201732.psf
3. Add them to Eclipse with File->Import->Team->Team Project File. The username is anonymous and the password is an email address.
4. Create a Dynamic Web Application. Create a New Runtime (Server) for Tomcat. Select Create local server, to save a step later.
5. Create servlet from New -> Servlet.
6. Works perfectly.
7. After I had to restore my computer, the eclipse installation was no longer working. Tomcat was not recognized. I switched workspaces (File->Switch Workspace) and reinstalled the Tomcat plugin. It is working again, but there are still some red Xs for some packages.

JSPs in Eclipse

1. Be sure that the project is using the Tomcat runtime.
2. 
   

Backspace in nano after ssh from Mac OS X

When I used ssh from a mac to connect to a remote linux machine, the nano editor did strange things to the backspace key.

It would work for three of four deletes, but then it would jump to the end of the line and start deleting there. It was very strange.

The TERM variable on the local machine is sent to the remote computer. Both machines were using xterm-256color.

I disconnected from the remote and set the TERM variable in the Mac terminal:

export TERM=xterm-color

I reconnected to the remote machine and the backspace worked properly.

Running Different Version of Perl on Mac

Running Different Version of Perl on Mac

Upgrading the installed version of perl on a mac can create problems.

An alternative is to use Perlbrew. It allows other installations to be installed and used. New packages can be added to the active perl, without adding it to all perl installations.

Install with

curl -kL http://install.perlbrew.pl | bash

Once installed, initialize in .bash_profile with
source ~/perl5/perlbrew/etc/bashrc

Mac does not have a make command by default. It must be installed before installing any perl installations. It can be found in Xcode and then the command line tools.

I was able to get the command line tools without installing Xcode from
https://developer.apple.com/downloads/index.action?name=for%20Xcode%20-

Find the available perl installations with
perlbrew available

Install a version with
perlbrew install perl-5-16.2

Next, I tried to install cpanminus with
cpan App:cpanminus

It did not install. After switching to perl-5.16.2, it installed.

I am now able to install modules using cpanm into the current perl selected via perlbrew.

------------

I have created a bash script that allows me to switch to a different perl and create an alias named local for the new perl. I also have created a sym link in /usr/local/bin to the local alias. This allows me to change to any perl that I want and not have to change the shebang in the perl files. I use
#!/usr/local/bin/perl


#!/bin/bash

strip() {
    echo -e $1 | tr -d '\r'
}

#$1 is (out) selection, $2 is prompt, $3 is list of choices                                                         
readSelect () {
    local L_CHOICE L_LIST=`strip "$3"`

    if [ -n "$L_LIST" ]; then
        PS3="$2"
        select L_CHOICE in $L_LIST
        do
          if [ -n "$L_CHOICE" ]; then
            break;
          fi
          break;
        done
        eval "$1=\"$L_CHOICE\""
    else
        error "Nothing to select"
    fi
}

choices=`perlbrew list | grep -v "(" | sed -e 's/\* //g' -e 's/ (/_(/g' -e 's/^[[:space:]]*//'`
readSelect perl "Select perl installation: " "$choices"

echo "Setting $perl"

perlbrew switch $perl

perlbrew alias delete local
perlbrew alias create $perl local