Mobile-friendly websites

Many people these days rely more on their mobile devices - phones, tablets etc - to access the Internet than on desktop / laptop computers. As such, it is a common requirement in many websites today that we design / test / tune the usability not just for different computer browsers but also for mobile devices.

While not every website has to cater to mobile devices this article gives some ideas on how you can make new and existing websites more mobile-friendly without requiring lots of coding or duplication.

Translate your website with Google Website Translator

Oftentimes we want to support multiple languages in our websites. One "easy" way is to keep one separate resource bundle for each language we support, but this isn't as simple as it seems. First you need a tag / mapping mechanism to match labels in your pages with tags in the property files (frameworks like JSF already provide it) but then you need to maintain multiple sets of labels. Any new field / screen etc will require changes all resource bundles, which can quickly become a burden. In many cases this burden may be  acceptable in exchange for full control and performance, but in other cases we can use Google's Website Translator tool.

Follow these steps to find out how to easily include a styled language selector into your website, and to remove the Google Website Translator toolbar. I recently applied these steps in Eventfy and the final result looks like this:

Hide input type file from older iOS devices

I recently added support for user image upload in Eventfy. You can find more details and screenshots here.

Safari mobile in iOS devices like iPhone and iPad do not yet support file uploads and render input type file disabled. It may change with the upcoming update to iOS 6, but until then our web applications must take that limitation into account.

I wanted to hide the file update controls in iOS and show a note in its place. Instead of checking the browser version I am simply checking the state of the input file, which may also cover other devices with similar limitations.

The code below describes how to set it up with jQuery, but it is simple to do the same with plain Javascript.

1. HTML part:

<div id="imageUploadDiv" class="control-group">
 <label class="control-label" for="imageUpload">Image</label>
 <div class="controls">
   <input id="imageUploadInt" name="imageUploadInt" class="input-file" type="file" />
   <p class="help-block">You can optionally include a picture or image in your invite</p>
   <span id="imgPreview" class="thumbnail" style="display:none;width: 250px;"></span>
 </div>
</div>
<div id="imageUploadiOSWarning" class="alert alert-info" style="display: none;">
 <button type="button" class="close" data-dismiss="alert">×</button>
 <strong>Note</strong> Image upload isn't supported by iOS before version 6.0
</div>

2. Javascript / jQuery part:

<script>
$(document).ready(function(){
    if ($("#imageUploadInt").prop("disabled")) { 
        $('#imageUploadiOSWarning').show();
        $('#imageUploadDiv').hide();
    }
}
</script>

In this simple example DIV "imageUploadInt" is presented by default. But as soon the document loads we check if the input file was automatically disabled by the browser, and if so we hide "imageUploadInt" and show instead DIV "imageUploadiOSWarning".

Note: this DIV structure and CSS classes are defined in Twitter Bootstrap. You can find more about Bootstrap control forms here, on section 'Validation States'.

Update: I tested the image upload function in Eventfy yesterday (9/19) with iOS 6 and this code works great. Now users can upload pictures (if their device supports it), otherwise they are presented with a message explaining why the control is disabled.

Here are some screenshots to show how it works with iOS 6:

Input type file is enabled with iOS 6

Photo selection

Photo uploaded to Eventfy

Quest to find the best Linux for slow computers, Fedora 12 LXDE upgrade to 13

I replaced Kubuntu Netbook Remix (KNR) on my netbook with the LXDE spin of Fedora 12. KNR looks nice and has many nice features for devices with small screens but it was just painfully slow on my netbook. LXDE is (in theory) a more crappy-hardware-friendly Linux flavor, so I decided to give it a try.

LXDE's UI looks uglier and less flexible than KNR's, but performance is indeed better. With that in mind I put together a development environment based on a Fedora LXDE virtual machine I can use to test my own projects in Linux. Nothing terribly resource-intensive but things like Grails, Gradle, AppFuse, etc. I wanted to know if this VM would be fast enough so I could use it as my development environment.

Click on "Read more" below to check out the rest of this post on a development environment VM and instructions on how to update Fedora 12 LXDE with the unofficial release 13.

NFJS 2010 - Northern Virginia Software Symposium II

This is my last post about the No Fluff Just Stuff Software Symposium here in Reston, VA. Sad to say I didn't get any of the 3 iPads! I did see my name scrolling down for the last iPad (the most expensive model!) but the system had to pick someone else's name just 2 or 3 positions from mine! Oh well... guess I am not destined to get one, since I am not gonna buy one and my luck isn't gonna get me a free one either. This is something else I might consider buying, if it ever sees the light of day Notion Ink Adam.

Anyways, the symposium was amazing - Jay Zimmerman put together a great set of speakers and enough varied content for everybody. Thanks Jay! These are the sessions I attended, along with my comments:

Click on the "Read more" link below to check out my notes on Friday's presentations.

NFJS 2010 - Northern Virginia Software Symposium

I am attending the No Fluff Just Stuff Software Symposium here in Reston, VA. The schedule this year is really good, with names like Neil Ford, Jeff Brown, Matthew McCullough and so many other experienced speakers and book authors discussing things like REST, polyglot development with languages like Ruby, Clojure etc, cloud computing, testing, Agile development etc. Many of these guys are either book authors, code committers or both for really cutting edge technologies that are shaping up what most of us will be doing and working with years from now. Sometimes it is hard to decide which session you want to attend - at any given time during the event they have 6 parallel sessions going on!

I attended today 2 sessions with Neal Ford, "Implementing Evolutionary Architecture" and "Testing the Entire Stack". He is not just a gifted speaker but also makes it obvious he knows well what he is talking about from hands on experience. If you ever have the chance to attend one of his sessions at NFJS or some other symposium make try not to miss it, I am sure you will learn something you can use. His keynote "Smithing in the 21st Century", where he presented his views of how (and to what extent) we can predict the future and to where we are headed in the near future was great, and it was interesting not just for developers / architects but to pretty much anyone interested in technology.

Unfortunately I just found out about the symposium less than 2 weeks back so I couldn't convince my company to fund my tickets (who knows if they change their mind in the next weeks? that would be a good surprise :D). Despite of that, a good thing is that I got to do volunteer work helping the event organizers on some basic tasks (things like attendee registration, organizing the presentation printouts etc), got to meet some nice people and should even get some credit back. Also, NFJS will be giving away 2 iPads, books and other prizes at random for the attendees - who knows, if my name gets picked for one of the iPads that wouldn't be bad at all, especially because this is likely the only way I would ever get one! :D

I will let you know on Sunday if I got one of the iPads hehehe... along with my personal feedback of the sessions I attended. I know these things are subjective, but who knows, if you get the NFJS guys in your town this could help you pick your sessions.

Quick web app prototyping with AppFuse

Sometimes we just need to put together a quick web app for prototyping, discuss screen flows, design layouts, create web services etc. I found AppFuse to be a pretty good option for this purpose.

You can follow the steps in this article to setup your environment and create a new web application with AppFuse. The best thing is that once your environment has all the requirements setup it is super quick to create as many new web apps whenever you need. Even if you aren't very familiar with things like AppFuse, Maven, Spring, Hibernate etc but still want to be able to generate new web apps, just try to endure the first steps - once you see the web app up and running on your computer I am sure you will be more motivated to learn it all through AppFuse.

Click on the "Read more" link below to get started with AppFuse and build a working web application in no time.

J2EE Performance Tuning

Found a very good overview on "How to Diagnose a Performance Problem in a J2EE System" written by John Bley: http://weblogic.sys-con.com/node/43024

Even though this article was written in 2003 somehow it is still pretty up to date. Guess the ways we build our systems are not changing so fast afterall.

Virtual Machines - managing virtual drives

From time to time I have to expand the disks on my virtual machines. Or convert disks from preallocated 9+GBs into sets of 2GB files (so they fit in FAT32 USB sticks, for example). I found this tool vmware-vdiskmanager to be the best way to manage these tasks, but it does not come bundled with VMWare Player. It does come, however, in the also free-as-in-beer VMWare Server. So if you need it, download and install VMWare Server here so you can use the examples below.

# VMware Virtual Disk Manager Utility (vmware-vdiskmanager.exe):
- Documentation: http://www.vmware.com/support/ws55/doc/ws_disk_manager_running.html

# Examples:

- Convert existing virtual disk from fixed preallocated size to growable, split into 2GB files (trust me, it helps wonders to set all your virtual disks this way to avoid future headaches):

C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -r data.vmdk -t 1 data-1.vmdk

- Defragment existing virtual disk (unless you have a SSD you should do this every now and then both in the VM and in the host OS):

C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -d data.vmdk

- Shrink virtual disk:

C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -k data.vmdk

- Expand virtual disk:

C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -x 32GB data.vmdk

Note: when you expand an already partitioned virtual disk it will not show immediately the new size when you boot your virtual machine. However, now you can use tools inside the virtual machine to expand the partition so the operating system can see and use it. In Linux, for example, you can do this:

df <- run this to list all mounted partitions
Filesystem ... Mounted on
/dev/sda1 ... /
/dev/sdb1 ... /data <- this is the partition I want to expand

Note: the block below worked for me, but as always, MAKE BACKUPS OF EVERYTHING if you don't want to risk losing your precious data. Or don't backup, but don't blame me if you mess up and lose your whole filesystem...

login as root and run these commands:
umount /dev/sdb1

fdisk -l /dev/sdb - write down the start and end cylinders for the target partition

fdisk /dev/sdb

in fdisk use the [d] command to delete the partition - yeah, I know, this is scary, but if you REALLY took the time to backup your virtual machine then go ahead; if everything works your data will not be lost in fdisk

use the [n] command to add a new primary partition, using the same partition number and start cylinder, but use the default value for end cylinder for it to use all the space you made available in fdisk

use the [w] command to write the partition table to the disk and exit - if it asks you to restart your server, do that before proceeding, otherwise keep going

login as root again and run these commands:

e2fsck -f /dev/sdb1

resize2fs /dev/sdb1

mount /dev/sdb1

df -h

If everything worked as planned you should now have a bigger partition, and all your precious data should still be there. If something didn't work, well, you still have your backups, right? RIGHT? ;D

Java magic with AspectJ

Sometimes in our programming projects we can start things from scratch and implement them exactly how we want them. In these happy and creative times we so much have control of everything, if we don't like some of our own designs, APIs or mechanisms we can just go and change it, simple like that. Bad thing oftentimes we aren't so lucky, and we have to work on more constrained settings, using existing crappy code and macaroni architectures. In times like these we gotta be happy if we at least have the source code we are supposed to extend / use!

Recently I had to work on a project whose a big chunk of the code was implemented by another team. And we were not supposed to change that code. But we were still somehow supposed to change the exception handling and logging for a certain kinds of Java classes on both that codebase and our own custom code.

Since we are on a pretty bad economy, instead of spending time to update my resume I thought of using this as an opportunity to test and learn AspectJ. It was a long shot but I read about it in the past, but never had a strong need to use it until now and thought it was worth trying. The thing is, I was interested in it a while back but thought it would be just like other technologies, like Spring, Hibernate, JSF etc. Think about it, if you need to do real work and have only 2 or 3 days, can you do something with Spring + Hibernate + JSF from scratch if you never worked with it before? The learning curve for most of these things is just too high if you need quick results!

Man, I am glad I tried AspectJ. I know its inner workings use some arcane technology (bytecode manipulation isn't for the faint of hearth), but it was SO much easier than I expected to hook my custom exception handling logic in the existing code I felt it was almost too easy to be true! :P

Since I think other people may benefit from this mechanism let me share some basic but still useful information about it. In a nutshell AspectJ is an arcane technique to alter behavior of existing Java binaries. You can transparently inject logic into certain "point cuts", like when a certain method name is executed, or when classes from a certain package are loaded etc etc. Few cool examples of what you can do with that are:
- a custom global logging mechanism to existing classes from a certain package (e.g. my.company.*)
- custom exception handling mechanism for certain exception types
- method profiling, for performance monitoring and tuning
- reduce LOC and complexity on your new code using aspects to handle multiple functional concerns transparently, outside your business classes

There are 2 main ways of using AspectJ:

1. Static-weaving: You can use AspectJ's ajc compiler to alter existing class files (either .java or .class), injecting your custom aspects and pack all changed files into a new JAR file. Then you put the new JAR in your classpath instead of the old classes (in most cases it may also work to just put the JAR first in the JVM's CLASSPATH). The Eclipse team divides this method as Compile-Time Weaving, when ajc compiles *.java into *.class injecting the aspects, and Post-Compile Weaving, when ajc injects aspects into pre-compiled class files. The performance penalty here is 0, it will be the same thing if you recompiled the original classes with your aspect's logic in it.

2. Load-time weaving: This is where AspectJ's magic is more evident - your original classes and JARs are modified when they are loaded by the JVM! You do not have to replace any JARs, it will all be done dynamically. In most cases the performance overhead imposed by AspectJ to "weave" your aspects in the existing classes is negligible when loading the classes, but note that there will be no runtime overhead - once AspectJ injects your aspects it will not mess with that class again.

If I have to choose between one of the 3 methods described above I would try Load-Time Weaving (LTW) first, because I think in most cases the performance overhead during class load times (note it is "load times" and not "run time") should not be enough to overshadow the benefits of not having to bother updating JAR files every time the original class must be updated. If there is a significant performance hit due a broad scope of weaved classes then it is easy to move to one of the two static weaving methods described in item 1 above.

Ok, enough talk, how can we actually use it? Let me describe it with a very simple example that can serve as the basis for whatever else you want to do with AspectJ: (yeah, you guessed) a custom logging and performance monitoring mechanism!

# Dev env setup

1. Download and install AspectJ Development Tools (AJDT) for Eclipse. You can download it here.

2. Download my AspectJ examples project here.

3. Extract the AspectJ examples project somewhere in your development environment and import it in Eclipse.

# Project notes

1. "test.Test.java" is a simple test class. Their methods will be intercepted by AspectJ to transparently inject additional behavior without us having to change any code in Test.java.

2. Our two aspects LogAspect.aj and MethodProfilerAspect.aj are located in package "aspects".

3. Eclipse with AJDT show us what aspects affecting each method in class Test.java in the Cross References view (we can see it in the screenshot above, in the bottom right). Just select a method in the Java editor and it will list the aspects in that view. Or select an aspect method in LogAspect.aj and it will list the intercepted methods in that view. This is pretty useful to control scoping, just keep in mind that AJDT will only list methods in its own classpath, and it may be different (i.e. the aspects may catch more classes) when you deploy the aspect JAR in the app server.

Without any aspects, this is the output of test.Test.java:

Test.method1
Test.method2
Test.method3

Enabling just LogAspect.aj we then get this output:

[LogAspect] before public static void test.Test.main(java.lang.String[])
[LogAspect] param[0]='[Ljava.lang.String;@12ac982'
[LogAspect] before public void test.Test.method1()
Test.method1
[LogAspect] after public void test.Test.method1()
[LogAspect] before public java.lang.Object test.Test.method2()
Test.method2
[LogAspect] after public java.lang.Object test.Test.method2()
[LogAspect] returned 'null'
[LogAspect] before public java.lang.Object test.Test.method3(java.lang.String)
[LogAspect] param[0]='InputValue'
Test.method3
[LogAspect] after public java.lang.Object test.Test.method3(java.lang.String)
[LogAspect] returned 'ReturnValue'
[LogAspect] after public static void test.Test.main(java.lang.String[])

And finally, enabling just MethodProfilerAspect.aj we get this output:

[MethodProfilerAspect] entering public static void test.Test.main(java.lang.String[])
[MethodProfilerAspect] entering public void test.Test.method1()
Test.method1
[MethodProfilerAspect] exiting public void test.Test.method1() in 31ms
[MethodProfilerAspect] entering public java.lang.Object test.Test.method2()
Test.method2
[MethodProfilerAspect] exiting public java.lang.Object test.Test.method2() in 62ms
[MethodProfilerAspect] entering public java.lang.Object test.Test.method3(java.lang.String)
Test.method3
[MethodProfilerAspect] exiting public java.lang.Object test.Test.method3(java.lang.String) in 16ms
[MethodProfilerAspect] exiting public static void test.Test.main(java.lang.String[]) in 109ms

If you look at the aspects' source code you will see how little coding was required for that. But this is just the tip of the iceberg, there are many other cool things we can do with AspectJ, like dynamically change the return value of certain methods, catch, modify and re-throw Exceptions etc.

So this is pretty powerful... but "with power comes responsibility" - you must be careful and not depend on too many aspects, or your project could become a maintenance nightmare!


# Additional interesting articles about AspectJ

• Pointcuts and Advices: http://www.eclipse.org/aspectj/doc/next/adk15notebook/ataspectj-pcadvice.html
• Join Points and Pointcuts: http://www.eclipse.org/aspectj//doc/next/progguide/language-joinPoints.html

Setup Oracle's SGA size

I always work with Oracle databases on my laptop. I prefer to install them myself so I can tune it to run without using all my memory. Also, instead of installing Oracle directly on my laptop I always do so on virtual machines because this way it is easier to share my environments with my colleagues and to setup individual sandboxes for each project I am working on.

When I have to work with databases installed by others I found I usually have to decrease how much memory is assigned to Oracle. The scripts below describe how to do just that.

• update Oracle's SGA size:

-- show current SGA size
select sum(value)/1024/1024 from v$sga;

-- set SGA target size to 256MB
alter system set sga_target = 256M scope=both;

-- list buffer size
select current_size from v$buffer_pool;

-- list all pools
select pool, sum(bytes)/1024/1024 Mbytes from v$sgastat group by pool;

-- additional Oracle memory settings:
-- make sure you don't setup SGA to the same or higher size than MEMORY
alter system set memory_max_target = 512M scope=spfile;
alter system set memory_target = 512M scope=spfile;
alter system set sga_max_size = 256M scope=spfile;
alter system set sga_target = 256M scope=spfile;
-- you will need to restart Oracle for the changes to take effect (e.g. dbshut and dbstart)

UML diagram editors

From time to time people ask me to write UML diagrams. Sometimes it makes sense, but oftentimes I could be better off using my time on something else, like writing code, prototyping etc. Please don't get me wrong, I am not against good documentation or UML per se, but many people like to focus too much time on pretty documents with beautiful colorful diagrams while others, more pragmatic ones, prefer instead to have a system built on time, with quality and running in production. I like to think of myself as part of the "pragmatic" group.

I KNOW documentation is important, it can make the difference of people using your software or not, or your development team implementing what your client actually needs instead of something completely outlandish. That is not my point. My point for this post is that we should generally try to stick with what we are good on, but sometimes we just dont have a choice and have to work on something we KNOW isn't one of our strong skills. I thought PMs were supposed to limit such times, making sure tasks are assigned to the best resources based on their talents, but from my experience this is not exactly how it generally works. Not necessarily because of the PMs, sometimes they just don't have better resources to tackle the job.

But regardless what I think, as much I try to avoid wasting a lot of time writing documents that no one will read, sometimes I have no choice and just have to do it. I guess that is why they call it work, we can't like all that we do for a living. But what I can do is to try to finish it as quickly as possible, while still developing good quality documents. Which bring us to graphical design talent... and this is one area of talent that, I must confess, I am not especially gifted. I've worked with people born with natural talent for graphical design, they could use pretty much any tool and come up in few minutes with amazing diagrams or images or web site layouts etc. Think like someone opening Microsoft Paint and drawing a Mona Lisa in few minutes - yeah, I know, that was a stretch, but the point is, this is one of the things in life that you are either born with or you are not. Sure you can learn techniques to improve, but your true potential is more or less apparent from the early start.

Now the question is, how someone who's not a superfast gifted designer can create decent looking diagrams fast? My answer is "with especialized tools that format most things for you and you just have to enter the data". Think something a bit more especialized than Microsoft Visio. I can spend hours on Visio, moving lines and boxes, changing colors etc, and no matter what I do on it it will always end up looking like a pig. Before people say I did not try enough I can say that I tried learning from articles and books about it, but after some time I just accepted that I'm better of spending my time designing server-side frameworks, integrating enterprise applications, working with web services etc than creating striking-looking diagrams and working with web design. Which is actually quite liberating, to accept that I should leave such things for other people more gifted than myself while focusing on what, I like to think, am good on and like a lot to do. We should not try to do everything, but to focus on what we LIKE to do, right?

So what can I do when I need to write diagrams so they at least look decent and describe the concepts and ideas I am trying to document? And are finished quick so I can move on to other tasks? Well... tools that automatically generate the diagrams for me is a good start, I would think.

For example, NetBeans can automatically generate a UML class diagram from your source code. The generated diagram will be complex and unless you do some additional editing to filter out some of the noise no developer would ever waste time reading it, they would obviously prefer following the source code instead (assuming your source code is decently written - it is, right??), but it does look decent to see on a document or presentation. For those who dont know, it would appear that you put a lot of time on it... yeah, right! :P

Ok, so NetBeans is fine... but most times when I need to write UML diagrams I still do not have the classes yet, I am still discussing the solution design with a team. So I need something more flexible than that. But not too flexible that I would have to spend ages moving lines and boxes here and there! That is when I found a ridiculously small application called Quick Sequence Diagram Editor (QSDE).

Different than most similar tools, in QSDE instead of moving graphical elements around with your mouse you actually PROGRAM the sequence diagrams! I initially thought it was stupid and it would take much longer to write diagrams this way than just using the mouse... man I was wrong, and I am glad I kept trying to use this little gem. Now I can draw sequence diagrams real fast using QSDE.



Another decent UML tool that I used for other UML diagrams than sequence is StarUML. It is free and pretty decent for class diagrams, check it out.



This is just a simple (and lame) diagram made with StarUML

# References:

• StarUML - http://staruml.sourceforge.net/

• Quick Sequence Diagram Editor - http://sourceforge.net/projects/sdedit/

Setting up a Maven repository proxy with Nexus

Background

Hey, quick hint: if you need quick get-to-the-point instructions on how to setup a free Maven repository proxy you can skip this section and scroll down to the next one. But, if you have some time to spare and are curious to know why one would need to setup his own Maven repository proxy, please bear with me a little and keep reading...

It should be a bit more than 3 years since I stopped using Ant in favor of Maven to automate builds, tests etc. After years using Ant for both personal and professional projects I got into a (then) new project where the dev team was using Maven. By that time I had already read about Maven and tried it briefly on minor prototypes just to evaluate it against Ant. In those first contacts it just seemed much more complicated than Ant, everything was considerably different etc.

Like I said, in this new project other analysts had already set it up the whole development environment, and since I had other pressing issues to focus on I did not learn much about it. The guys who set it up were amazing technical analysts, they picked excellent tools (they used AntHill Pro, among some others) and had a deep understanding of best practices for build management, test automation, quality control etc, and because of that I never had to trouble with nor had to deal with Maven or the build process myself.

Then, about another 1 and a half years ago, I began as the leading developer (and pretty much THE ONLY developer, designer, architect etc, I should add) on another project I wanted to stick with Maven. I guess I just got spoiled with things like automated dependencies download and the ability to integrate with such amazing apps like Hudson and Trac. Finally, the fact that AppFuse is heavily based on Maven also helped me made my mind.

A quick note on AppFuse: it is amazing how simple AppFuse makes it for us to kick-start our solutions rather than spend time on mundane things like defining your project hierarchy, setup Maven, Spring, Hibernate etc, but it comes with the price of being another level of abstraction – it help make things work even if you don't quite know how they work. Based on that, my advice is to use it or a similar super framework to save you time on redundant tasks but do take the time to learn how it does what it does!

So, long story short, that project I was working almost by myself grew, it got more developers and made its way to a production environment. During one of our code drops I got a message from the release manager that the build server was not able to build the application, something to do with a missing dependency. Well, yeah, if you ever worked with Maven you know one of the first things you learn is that it can download dependencies for you from the Internet. But this time it did not work. I built it locally, and it worked fine, no problem. I then removed my Maven repository, and that is when I got the exact same error we got in the build server – Maven was unable to contact one of the public Maven repositories the project depended on.

It is funny that I remember speaking to the build manager couple weeks before that incident that we should setup our project's own Maven repository to avoid something like that. But, as much as an outage of a Maven public repository is to be expected and therefore should be prevented, since I had never seen it before it got a very low priority in our To-Do list.

I found a quick fix for that by simply pointing the build server to my personal Maven repository, which had all required dependencies and did not need to download anything from the internet. So, in the end, there was no big harm other than me and the build manager having to spend about one hour to figure it out, upload files to build server and finally get the build completed.

What we can learn from this story? I don't know about you, but I learned that if we blindly depend on things beyond our control, as stable as they look like or use to be, they will fail us on unpredictable times.

It is my responsibility to shield my project from any outages on systems I do not control. The same way I always build simulators and stubs for external systems I must do something similar for external Maven repositories. And that is where a local project Maven repository fits in.

A Maven repository proxy is just a simple mechanism that acts as a mediator between your Maven-based applications and the Maven public repositories in the internet. It gives you control over your application dependencies – say, if you built your application yesterday with success, and assuming then that your Maven repository proxy is up to date, the public Maven repositories can burn and you will still be able to build your application. I hope they don't actually burn though, that was just an expression to make the point! Also, since you are not downloading hundreds of JAR files over the internet but from a local server, it may even improve your build times considerably.

So, which one to use? I tried 3 or 4 different Maven repository proxies and found Nexus to be the best one of the bunch, on my humble opinion. It could not be much simpler to install and set it up, and once you do you can forget about it, it just works!

If you are one of my 2 or 3 readers who might had the patience to get to this point with me, be happy this section is over. I will describe in the section below how easy it is to install and setup Nexus as your free-as-in-beer local Maven repository proxy.


To-the-point Nexus installation and configuration

Nexus is released both as a stand-alone server and as a standard WAR file. This section will describe how to install the WAR file release only.

To install Nexus as a WAR file just follow these instructions:

1. You need a servlet container to install Nexus WAR file. While any recent application server should work just fine I only tested it with Tomcat 6.0.20. If you do not have a servlet container I recommend you to download and install the latest stable release of Tomcat 6.x into your build server. For the purposes of this article I will assume you have Tomcat installed in $TOMCAT_ROOT (wherever it is) running on 127.0.0.1, port 8080. Make sure you replace IP address, port and paths on the next steps based on your environment.

2. Download the Nexus WAR file – usually I would recommend you to download the latest stable version but I had some trouble with Nexus 1.4.0 on Tomcat. The previous version, 1.3.6, works fine though and you can get it directly from this URL: http://nexus.sonatype.org/downloads/older/nexus-webapp-1.3.6.war

3. Rename the WAR file to nexus.war – this will make your Nexus portal URL much simpler.

4. With your Tomcat running, copy or upload the file nexus.war into “$TOMCAT_ROOT/webapps” - this should deploy Nexus in just few seconds, as shown in the image below.





If everything worked you should now be able to access Nexus web console in this URL: http://127.0.0.1:8080/nexus/




Click on Repositories link in the left menu to see which repositories are available and their URLs, as shown in the figure below. Nexus comes pre-configured with the most popular repositories including Maven Central, Codehaus and Apache. If your application requires only the most popular libraries the odds are that you may not need to change Nexus' configurations at all. If that is the case, the only thing you need to do is to edit your application's pom.xml to use Nexus repository paths (e.g. “http://127.0.0.1:8080/nexus/content/repositories/central/”). If you want to confirm exactly where each one of these URLs are pointing to or if you need to include new repositories you need to log in as an administrator.




To log in as an administrator and configure Nexus click on the link “Log In” on the top-right portal. It will ask for credentials, to witch you can use admin / admin123.




Once you log in, click on Repositories and select “Maven Central”. As an ADMIN you will see new tabs, click on “Configuration”. In this screen you will find that URL “http://127.0.0.1:8080/nexus/content/repositories/central/” is associated with a remote repository URL “http://repo1.maven.org/maven2/”, the Maven Central public repository. So, in essence, when your application tries to download a library from the Nexus URL that is not in your Nexus repository, Nexus will download it from Maven Central, but the next time it will not need to, since Nexus will keep the library in its local repository.

For my particular project I had to setup 2 or 3 additional repositories in Nexus. That was also easy, I just “cloned” one of the existing entries in the Nexus web console. I then updated the repositories section on my project's pom.xml to point to the repository paths listed in Nexus' web console. I was amazed my project got built successfully the first time, it was so simple even my humble self got it right the first time!

If you tried this but found that I missed some important step this may help - there is much more detailed information on how to install Nexus here: http://www.sonatype.com/books/nexus-book/reference/install-sect-as-a-war.html


References

• Nexus: http://nexus.sonatype.org/

• AppFuse: http://appfuse.org/

• AntHill Pro: http://www.anthillpro.com/

Oracle database quick reference

Similar to the Unix/Linux quick reference post, this one lists multiple snippets about the Oracle database. Feel free to drop a comment with your own Oracle "gems".

# how to list all triggers affecting a table
select * from all_triggers where triggering_event like '%INSERT%' and table_name='PUT_TABLE_NAME_HERE';

# if you get error ORA-24170 when dropping an user see the article below
drop user myuser cascade;
ORA-24170: MYUSER.SOME_QUEUE_NAME is created by AQ, cannot be dropped directly
read: http://forums.oracle.com/forums/thread.jspa?messageID=1184946

# set password for sys/system oracle users
ALTER USER SYSTEM IDENTIFIED BY password;
ALTER USER SYS IDENTIFIED BY password;

# create a full export of an Oracle database:
exp system/password file=database.dmp full=y statistics=none

# import an Oracle database dump file:
imp USERNAME/PASSWORD@ORACLE_SID file=database.dmp full=y
imp USERNAME/PASSWORD@ORACLE_SID file=database.dmp fromuser=ORIGINAL_USER touser=NEW_USER grants=n

# run Oracle imp as a SYSDBA:
imp \'/ as sysdba\' file=database.dmp

# purge and disable Oracle 10g recyclebin tables:
-- login as sysdba
purge dba_recyclebin;
alter system set recyclebin = OFF scope=both;

# list tables with a field with a given name:
select * from all_tab_columns where column_name like '%PUT_A_COLUMN_NAME_HERE%';

# rebuild all Oracle indexes for performance tuning:
select 'ALTER INDEX ' || OWNER || '.' || OBJECT_NAME || ' REBUILD;' from dba_objects where status = 'VALID' and object_type in ('INDEX')

# improve indexes performance:
select 'analyze table ' || OWNER || '.' || object_name || ' compute statistics;' from dba_objects where object_type in ('TABLE');

# Oracle database performance optimizer - run it every time we have significant changes on a database schema
sqlplus /nolog
connect system/
begin
dbms_stats.gather_schema_stats (
ownname=>'PUT_SCHEMA_NAME_HERE',
degree=>3);
end;
/
quit;

# compile all invalid objects in Oracle:
The best way to compile all database objects that are invalid is to use a script in the $ORACLE_HOME/rdbms/admin directory named utlrp.sql. This script finds all objects in the data dictionary that are invalid and compiles them. This script is typically mentioned in patch notes but you can use it any time a schema change occurs.
I also included other alternative approaches just in case.

1. best way:
sqlplus system/password
@$ORACLE_HOME/rdbms/admin/utlrp.sql

2. by schema:
select 'ALTER ' || OBJECT_TYPE || ' ' || OWNER || '.' || OBJECT_NAME || ' COMPILE;' from dba_objects where status = 'INVALID' and object_type in ('PACKAGE','FUNCTION','PROCEDURE') and owner='PUT_SCHEMA_NAME_HERE'
union
select 'ALTER PACKAGE ' || OWNER || '.' || OBJECT_NAME || ' COMPILE BODY;' from dba_objects where status = 'INVALID' and object_type in ('PACKAGE BODY') and owner='PUT_SCHEMA_NAME_HERE';
set pages 50000
spo compile.sql
/
spo off
@compile.sql

3. using the utl_recomp APIs:
connect sys/password as sysdba;
select 'utl_recomp_begin: ' || to_char(sysdate, 'HH:MI:SS') from dual;execute utl_recomp.recomp_serial();
select 'utl_recomp_end: ' || to_char(sysdate, 'HH:MI:SS') from dual;
quit;

# how to enable table shrink in a Oracle 10g database:

alter table mytable enable row movement;
alter table mytable shrink space;

# how to shrink multiple schemas in a Oracle 10g database:

select 'ALTER TABLE ' || OWNER || '.' || OBJECT_NAME || ' ENABLE ROW MOVEMENT;' from dba_objects where object_type in ('TABLE') and owner in ('MY_SCHEMA1','MY_SCHEMA2','MY_SCHEMA3');

select 'ALTER TABLE ' || OWNER || '.' || OBJECT_NAME || ' SHRINK SPACE CASCADE;' from dba_objects where object_type in ('TABLE') and owner in ('MY_SCHEMA1','MY_SCHEMA2','MY_SCHEMA3');

# how to shrink datafiles:
select 'ALTER TABLE ''' || OWNER || '''.''' || TABLE_NAME || ''' SHRINK SPACE COMPACT' from all_tables;

SELECT VALUE FROM V$PARAMETER WHERE upper(NAME) = 'DB_BLOCK_SIZE';

SELECT 'ALTER DATABASE DATAFILE ''' || FILE_NAME || ''' RESIZE ' || CEIL( (NVL(HWM,1)*&&BLKSIZE)/1024/1024 ) || 'M;' SHRINK_DATAFILES FROM DBA_DATA_FILES DBADF,
(SELECT FILE_ID, MAX(BLOCK_ID+BLOCKS-1) HWM FROM DBA_EXTENTS GROUP BY FILE_ID ) DBAFS
WHERE DBADF.FILE_ID = DBAFS.FILE_ID(+) AND CEIL(BLOCKS*&&BLKSIZE/1024/1024)- CEIL((NVL(HWM,1)* &&BLKSIZE)/1024/1024 ) > 0 ;

# how to find which objects have extents at the end of a datafile.
Relocating these objects makes shrinking of relevant datafile possible.

select *
from (
select owner, segment_name,
segment_type, block_id
from dba_extents
where file_id =
( select file_id
from dba_data_files
where file_name = &FILE )
order by block_id desc
)
where rownum <= 5;

UNIX / LINUX quick reference

I have compiled the Unix/Linux code snippets below as a quick reference covering the "how to" on various topics. Please feel free to leave a comment with your own "Unix command line gems".

How to:

# list top memory processes on a Linux machine:
ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr

# show how many CPUs on a Linux machine:
cat /proc/cpuinfo

# create a symbolic link:
ln -s originalPath newLinkPath
e.g. ln -s /usr/local/apache/logs /tmp/apachelogs

# list largest files in Linux (good for file system cleanup):
find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2

# list largest directories in Linux:
du -h / | grep ^[0-9.]*G | sort -g
du -h / | grep ^[0-9.]*M | sort -g | more
find / -type d -size +1G

# split big text files into smaller and more manageable files
split --lines=100000 --suffix-length=3 application.log application.log.a

# redirect Linux output:
ls 1> out.log 2> err.log

# do global string replaces in the VI editor:
:%s/old_string/new_string/g

# set executable tag in shell script files recursivelly:
find . -name "*.sh" -exec chmod +x+x+x {} \;

# delete bak and log files recursivelly:
find . -name "*.bak" -exec rm {} \;
find . -name "*.log" -exec rm {} \;

# recursively determine how much space a directory is taking:
df -h : shows how much space we have on each partition
du -sh * : show how much space the current directory is taking
du -h | grep -v '/' | awk '{print $1}' : does the same without iterating thru the subdirs

# find files that contain a given text string
grep -lir "/data/platform" *

# find files that contain a given text string recursively
find . * -exec grep -li "/data" {} \;
find . * -exec grep -li "/data" {} \; > files.txt

# poll the number of files in a folder every 3 seconds
ksh
while true
do
ls *.retry.xml | wc -l
sleep 3
done

# archive a folder and sub-directories in tar/gz
tar -zcvf archive.tgz directory
e.g. tar -zcvf apacheLogs.tgz /usr/local/apache/logs

# extract a tar/gz archive
tar -xvf archive.tgz directory
e.g. tar -xvf  apacheLogs.tgz .

These are just some of the ones I use the most. You can also find many more at the Unix Toolbox here.

Automated handling of header notices in source files

I recently implemented a web-based application that grew from a proof-of-concept into a real solution for a telecom operator, and it has also been offered to other potential clients. And, since it uses only open technologies such as JSF, Spring and Hibernate, it is pretty easy for other developers to reuse it for demos and prototypes without training on proprietary technologies. It is so that other teams in my company have been asking me to share the source code so they too can use it on their own projects.

While I've worked with few developers in the past who don't quite like the idea of sharing "their" code with other teams, I'm quite proud of this app and have no problems sharing it internally if it could help my company. However, to protect my company’s IP I thought I should at least include copyright messages on every single source code file before sharing it. I know I should have included it from the beginning but when we are working around the clock and we do not have to share the code these kinds of things are easily overlooked. So then I had a project with hundreds of classes and other artifacts without any copyright message. That should be a simple and common task to solve, right? Well, I thought as much... but when looking for a good way to handle it I found that my development tools do not have any functionality I could use to easily handle copyright messages on existing projects.

There are many possible ways to automate this task, from SED-based Unix shell scripts to regular expressions in search/replace functions of more advanced text editors, but I wanted to find something easier to use and more well-integrated to my development environment.

After digging for something in those lines I found a good free solution in an Eclipse plugin called Eclipse Releng Tools plugin, or simply Releng. It can be used to automate the task of including a header notice stating the code’s ownership and/or its license. This article will guide you step by step on how to install, setup and use this Eclipse plugin.

But before we start, a note of warning – YOU should make backups before following these instructions, install and run Releng. It did not cause me any troubles but I should not be held liable in any way if it causes you any problem.


Installation

If you still do not have Eclipse installed, download it here. You should select the distribution package that best suits your specific project, or just pick "Eclipse IDE for Java EE Developers" since it works best for most projects. Note that for Eclipse to run you also need a Java Development Kit – and if you do not have it and do not know how to install it you are not the target audience for this article and should probably not follow its instructions.

Once the download is finished extract the ZIP archive and start Eclipse. It will ask for a workspace folder. You can enter the path where your existing project is, or enter something else if you wish to start a new project or check out from a source code repository system. Click on the Workbench icon to close the welcome screen.

Eclipse's welcome screen

Since the Eclipse Releng Tools plugin is not installed by default in Eclipse we must install it now. In Eclipse’s top menu select “Help” / “Install New Software”.
In the “Work with” tab select “--All Available Sites--”. This will list all the available Eclipse plugins. In the filter bar enter “Releng” to filter the results and show only the plugin we need to install. Check the “Eclipse Releng Tools” plugin and complete the installation process.

Eclipse's available plugins window

After the plugin is installed Eclipse will ask you to restart it. Do that so that it is activated when Eclipse comes back up.

Open your project and note that when you right-click the project root you see the new menu item Fix Copyrights. Do not click on it now; this is just to show that the plugin was installed correctly.

Releng's new menu item


Setup

In Eclipse’s top menu select “Window” / ”Preferences”. In the Preferences window click on Copyright Tool. This will present you with the copyright template.

Releng's default copyright template

Replace the default message with this one below, including the empty lines (one at the top and two at the bottom):

This file contains proprietary information of MyCompany. Copying or reproduction without prior written approval is prohibited. Copyright (C) ${date} MyCompany. All rights reserved.

Set the year fields to 2009 or the current year.
Optionally you can enable or disable the copyright messages on properties and XML files. You can also instruct the plugin to replace existing copyright messages with this new template.


Usage

Now that Releng is properly setup, right-click on your project’s root node and select “Fix Copyrights”. All of your source code should now include a header message with your notice, as shown in the image below.

Project processed by Releng

Super Frameworks - develop Java web apps at lightning speeds

In the early days of J2EE (that's how we used to call Java Enterprise Development back then) we had fewer alternatives of technologies, frameworks and reusable components. Web application development was then far simpler than it is today - there was no need for developers to learn a bunch of different things like Spring, Hibernate, JSTL, JSF etc - and things were much simpler to learn and understand. I am not that nostalgic, however, because some big drawbacks compared to what we can do today are that most of what we did was not really reusable (it is hard to consider Copy/Paste as reusable code), we had to spend time writing low-level logic to handle aspects like persistency and logging, code was hard to unit test... the bottomline is that we had to spend more time on infrastructural aspects of our systems than on business logic, essentially killing our productivity and making code quality suffer.

When someone pays you to write a system they want it finished on time and working without bugs. And 99% of the time they have no idea what a JDBC or some other arcane technical acronym is, so they expect you to focus on the screens and public APIs, not in infrastructural code that has nothing to do with visible functionality that they are interested in.

Then few years back we started to see many production-ready frameworks and technologies pop up as new alternatives in our Java development toolkit - far too many actually, but I still think in this case it is better to have more than less. Frameworks and tools like Spring, Hibernate, Struts, jUnit, Ant etc focused on multiple different aspects common to most Java projects like persistency, security, presentation, logging, building and testing, thus improving our productivity.

But then, while this new set of tools helped reduce the need for us to re-invent the wheel it also made the learning curve considerably more steep. Now, just to create a new project we have to install a bunch of frameworks and then integrate each one. It doesn't sound that bad, but let me tell you, it is bad. Instead of wasting time writing connection pools developers now can spend (many) days figuring out why Spring isn't working properly with Hibernate or some other framework. Again, they are not focusing on the business functionality their clients are paying them to deliver.

There are multiple alternative solutions to this problem, each with its own set of advantages and drawbacks. One is to forget Java and move on to another platform like Ruby / Ruby-On-Rails. Instead of a Frankenstein-like system requiring hundreds of independent frameworks patched together, RoR is just one single framework that tries to address most of the aspects I mentioned before. Because of this RoR is more consistent than a bunch of frameworks implemented by different teams. But while many developers love RoR with passion (or perhaps they needed a new theme to sell books?) it is still just a very small niche if we compare it with Java.

A more enterprise-friendly alternative (and therefore one that may help you land in more jobs) is the use of a super framework - which, as the name implies, is a set of frameworks packaged in a pre-configured, pre-integrated manner, with tools to automate redundant manual tasks to improve productivity and quality. There are also dozens of such super-frameworks based on Java but probably the two most known ones are JBoss Seam and AppFuse.

Seam comes packed with more functionality than AppFuse, but it is also less flexible. I don't advocate one over another, but I do believe that if you work implementing web-based Java applications you should consider using something like that. Seriously, after you learn well how to use one of these frameworks you will be amazed how much faster you can do things you did before, like web UI tests, web services, security etc. Not to mention your boss will think you are the fastest developer on Earth, if you do a good marketing of your work.

There are few books available describing both frameworks in detail but you can learn a lot from the project official pages and user forums. There are tutorials you can follow, and the forums have very responsive developers (at least that was my experience) to help answer questions and troubleshoot issues.

I spent a good time learning and working with AppFuse since its version 1.9.x and I can say it was one of the best things I did for my career since I took my first Java certification, back when these certifications were worth something.

Tools for an efficient development environment

In this post you will find a list of the most common tools and apps I use for software development with a brief description of each.

In the next posts I will describe in more detail some of these tools, especially those that I believe are not that well known. Obviously I am not saying that the tools listed below are perfect for every developer or every situation, nor I am suggesting you should try it (i.e. don't sue me if one of these tools make you sick, kill your cat or something). I am just saying that, so far, they are helping me get the job done.

Lastly, note that each tool may have different licenses (GPL, LGPL, Apache etc) but all are (or have a version that is) "free-as-in-beer" when it comes to simple use.

Development

• Eclipse IDE for Java EE Developers - Great editor (or Integrated Development Environment) for Java, JSPs, JSF etc. It used to require hundreds of 3rd-party plug-ins to be decent but since they began bundling it with parallel projects it became truly powerful and way more stable. The only plug-in I find myself installing on a clean instance of Eclipse these days is Subversive, a Subversion client.
• Maven - Build manager system. I took some time to get used to Maven after years using Ant but there are just too many cool functions here to let it pass, like built-in online dependencies download, archetypes and hundreds of great plug-ins.
• Subversion - SVN is a CVS-like source code versioning and repository system. If you are working on a team of developers you need one of these. But even if you are working alone you need one too. :D
• Jenkins (or its former incarnation Hudson) - Continuous integration web portal. Basically you take the time to set it up once and it will pay back in interest every time you have to run a build.
• Nexus - This is a maven repository proxy. I will write more about this one in the next posts, but enough to say that if you use Maven you should not rely on the public Maven repositories out there, you need a maven repository proxy. Nexus has a free version and it is powerful and freaking easy to use.
• Trac - Trac is a project management, task tracking and WIKI system. I just hoped Trac was based in Java - it is not hard to install it but I guess I'm getting spoiled with tools like Nexus and Hudson that you can install simply by dropping a WAR file in Tomcat's webapp folder. But once Trac is setup it shines - you can easily integrate it with many other tools like Hudson and Subversion.
• SoapUI - Use this when you want to run functional tests on your web services. Just do not use it for high-volume performance tests.
• JMeter - Use this for high-volume performance tests. I know, SoapUI and LoadUI look nicer and shinier (though I really hate LoadUI's funky UI) but pair JMeter with this and it will save you lots of headaches.

Note: Your development environment gets especially powerful and productive when you integrate tools like Subversion, Trac and Jenkins together to automate aspects like documentation, deployment and testing, and to streamline the communication in your development team.

General

• Notepad++ - This is what I use to replace the Notepad application that comes with Windows.
• 7-Zip - High-performance archiver that also works with almost every format out there.
• Agent Ransack - Replacement to the Search function that comes with Windows. I could not hate more that dog that shows up during searches, especially when it does not list files that I KNOW it should be listing (see here for more details on a Windows "feature" in the search function that causes it to not show files under certain conditions)!
• WinMerge - Great to compare folders with hundreds of files, like when we want to compare two slightly different local copies of the same project.
• FileZilla Client - Great SFTP client.
• CCleaner - Remove junk from your disk so you get more available space... and while at it, also remove junk from the Windows' registry. Tip: you can make it even more powerful by dropping this file into CCleaner's path. This will add lots of options to CCleaner (all disabled by default). Just be careful not to set the "enhanced CCleaner" to remove something you might need later.
• MyDefrag - Simple, powerful, configurable... best disk defragmenter out there. You can also use Defraggler for quick defrags and leave MyDefrag for big environment updates. Just make sure you don't use any defragger's in your SSD.
• Paint.NET - Best free Paint replacement out there.

You can automate the installation of many of these tools with Ninite. This is a great tool that can save you time especially if you use / manage multiple computers and want similar tools in all of them. Go to the website, select the tools you want and download the executable file. Then, anywhere you run this file it will silently install the tools you marked, no questions asked.

You can also use FileHippo Update Checker to keep most of these apps up to date. This app can quickly let you know about updates to most of your installed apps.

And lastly, if you are interested in many more great freeware tools you can find the best ones at the TechSupportAlert's website.


So, this is the initial list. In the next posts I will describe in more details few of these tools, including Hudson, Nexus and SoapUI, and concepts like Build Automation, Continuous Integration etc. If you like this list, a comment would definitely help motivate me to write more. If you find better tools that do about the same of one of the ones I listed, please drop a comment, I am always happy to learn a new trick.