Setting up a Maven repository proxy with Nexus

Background

Hey, quick hint: if you need quick get-to-the-point instructions on how to setup a free Maven repository proxy you can skip this section and scroll down to the next one. But, if you have some time to spare and are curious to know why one would need to setup his own Maven repository proxy, please bear with me a little and keep reading...

It should be a bit more than 3 years since I stopped using Ant in favor of Maven to automate builds, tests etc. After years using Ant for both personal and professional projects I got into a (then) new project where the dev team was using Maven. By that time I had already read about Maven and tried it briefly on minor prototypes just to evaluate it against Ant. In those first contacts it just seemed much more complicated than Ant, everything was considerably different etc.

Like I said, in this new project other analysts had already set it up the whole development environment, and since I had other pressing issues to focus on I did not learn much about it. The guys who set it up were amazing technical analysts, they picked excellent tools (they used AntHill Pro, among some others) and had a deep understanding of best practices for build management, test automation, quality control etc, and because of that I never had to trouble with nor had to deal with Maven or the build process myself.

Then, about another 1 and a half years ago, I began as the leading developer (and pretty much THE ONLY developer, designer, architect etc, I should add) on another project I wanted to stick with Maven. I guess I just got spoiled with things like automated dependencies download and the ability to integrate with such amazing apps like Hudson and Trac. Finally, the fact that AppFuse is heavily based on Maven also helped me made my mind.

A quick note on AppFuse: it is amazing how simple AppFuse makes it for us to kick-start our solutions rather than spend time on mundane things like defining your project hierarchy, setup Maven, Spring, Hibernate etc, but it comes with the price of being another level of abstraction – it help make things work even if you don't quite know how they work. Based on that, my advice is to use it or a similar super framework to save you time on redundant tasks but do take the time to learn how it does what it does!

So, long story short, that project I was working almost by myself grew, it got more developers and made its way to a production environment. During one of our code drops I got a message from the release manager that the build server was not able to build the application, something to do with a missing dependency. Well, yeah, if you ever worked with Maven you know one of the first things you learn is that it can download dependencies for you from the Internet. But this time it did not work. I built it locally, and it worked fine, no problem. I then removed my Maven repository, and that is when I got the exact same error we got in the build server – Maven was unable to contact one of the public Maven repositories the project depended on.

It is funny that I remember speaking to the build manager couple weeks before that incident that we should setup our project's own Maven repository to avoid something like that. But, as much as an outage of a Maven public repository is to be expected and therefore should be prevented, since I had never seen it before it got a very low priority in our To-Do list.

I found a quick fix for that by simply pointing the build server to my personal Maven repository, which had all required dependencies and did not need to download anything from the internet. So, in the end, there was no big harm other than me and the build manager having to spend about one hour to figure it out, upload files to build server and finally get the build completed.

What we can learn from this story? I don't know about you, but I learned that if we blindly depend on things beyond our control, as stable as they look like or use to be, they will fail us on unpredictable times.

It is my responsibility to shield my project from any outages on systems I do not control. The same way I always build simulators and stubs for external systems I must do something similar for external Maven repositories. And that is where a local project Maven repository fits in.

A Maven repository proxy is just a simple mechanism that acts as a mediator between your Maven-based applications and the Maven public repositories in the internet. It gives you control over your application dependencies – say, if you built your application yesterday with success, and assuming then that your Maven repository proxy is up to date, the public Maven repositories can burn and you will still be able to build your application. I hope they don't actually burn though, that was just an expression to make the point! Also, since you are not downloading hundreds of JAR files over the internet but from a local server, it may even improve your build times considerably.

So, which one to use? I tried 3 or 4 different Maven repository proxies and found Nexus to be the best one of the bunch, on my humble opinion. It could not be much simpler to install and set it up, and once you do you can forget about it, it just works!

If you are one of my 2 or 3 readers who might had the patience to get to this point with me, be happy this section is over. I will describe in the section below how easy it is to install and setup Nexus as your free-as-in-beer local Maven repository proxy.


To-the-point Nexus installation and configuration

Nexus is released both as a stand-alone server and as a standard WAR file. This section will describe how to install the WAR file release only.

To install Nexus as a WAR file just follow these instructions:

1. You need a servlet container to install Nexus WAR file. While any recent application server should work just fine I only tested it with Tomcat 6.0.20. If you do not have a servlet container I recommend you to download and install the latest stable release of Tomcat 6.x into your build server. For the purposes of this article I will assume you have Tomcat installed in $TOMCAT_ROOT (wherever it is) running on 127.0.0.1, port 8080. Make sure you replace IP address, port and paths on the next steps based on your environment.

2. Download the Nexus WAR file – usually I would recommend you to download the latest stable version but I had some trouble with Nexus 1.4.0 on Tomcat. The previous version, 1.3.6, works fine though and you can get it directly from this URL: http://nexus.sonatype.org/downloads/older/nexus-webapp-1.3.6.war

3. Rename the WAR file to nexus.war – this will make your Nexus portal URL much simpler.

4. With your Tomcat running, copy or upload the file nexus.war into “$TOMCAT_ROOT/webapps” - this should deploy Nexus in just few seconds, as shown in the image below.





If everything worked you should now be able to access Nexus web console in this URL: http://127.0.0.1:8080/nexus/




Click on Repositories link in the left menu to see which repositories are available and their URLs, as shown in the figure below. Nexus comes pre-configured with the most popular repositories including Maven Central, Codehaus and Apache. If your application requires only the most popular libraries the odds are that you may not need to change Nexus' configurations at all. If that is the case, the only thing you need to do is to edit your application's pom.xml to use Nexus repository paths (e.g. “http://127.0.0.1:8080/nexus/content/repositories/central/”). If you want to confirm exactly where each one of these URLs are pointing to or if you need to include new repositories you need to log in as an administrator.




To log in as an administrator and configure Nexus click on the link “Log In” on the top-right portal. It will ask for credentials, to witch you can use admin / admin123.




Once you log in, click on Repositories and select “Maven Central”. As an ADMIN you will see new tabs, click on “Configuration”. In this screen you will find that URL “http://127.0.0.1:8080/nexus/content/repositories/central/” is associated with a remote repository URL “http://repo1.maven.org/maven2/”, the Maven Central public repository. So, in essence, when your application tries to download a library from the Nexus URL that is not in your Nexus repository, Nexus will download it from Maven Central, but the next time it will not need to, since Nexus will keep the library in its local repository.

For my particular project I had to setup 2 or 3 additional repositories in Nexus. That was also easy, I just “cloned” one of the existing entries in the Nexus web console. I then updated the repositories section on my project's pom.xml to point to the repository paths listed in Nexus' web console. I was amazed my project got built successfully the first time, it was so simple even my humble self got it right the first time!

If you tried this but found that I missed some important step this may help - there is much more detailed information on how to install Nexus here: http://www.sonatype.com/books/nexus-book/reference/install-sect-as-a-war.html


References

• Nexus: http://nexus.sonatype.org/

• AppFuse: http://appfuse.org/

• AntHill Pro: http://www.anthillpro.com/

Oracle database quick reference

Similar to the Unix/Linux quick reference post, this one lists multiple snippets about the Oracle database. Feel free to drop a comment with your own Oracle "gems".

# how to list all triggers affecting a table
select * from all_triggers where triggering_event like '%INSERT%' and table_name='PUT_TABLE_NAME_HERE';

# if you get error ORA-24170 when dropping an user see the article below
drop user myuser cascade;
ORA-24170: MYUSER.SOME_QUEUE_NAME is created by AQ, cannot be dropped directly
read: http://forums.oracle.com/forums/thread.jspa?messageID=1184946

# set password for sys/system oracle users
ALTER USER SYSTEM IDENTIFIED BY password;
ALTER USER SYS IDENTIFIED BY password;

# create a full export of an Oracle database:
exp system/password file=database.dmp full=y statistics=none

# import an Oracle database dump file:
imp USERNAME/PASSWORD@ORACLE_SID file=database.dmp full=y
imp USERNAME/PASSWORD@ORACLE_SID file=database.dmp fromuser=ORIGINAL_USER touser=NEW_USER grants=n

# run Oracle imp as a SYSDBA:
imp \'/ as sysdba\' file=database.dmp

# purge and disable Oracle 10g recyclebin tables:
-- login as sysdba
purge dba_recyclebin;
alter system set recyclebin = OFF scope=both;

# list tables with a field with a given name:
select * from all_tab_columns where column_name like '%PUT_A_COLUMN_NAME_HERE%';

# rebuild all Oracle indexes for performance tuning:
select 'ALTER INDEX ' || OWNER || '.' || OBJECT_NAME || ' REBUILD;' from dba_objects where status = 'VALID' and object_type in ('INDEX')

# improve indexes performance:
select 'analyze table ' || OWNER || '.' || object_name || ' compute statistics;' from dba_objects where object_type in ('TABLE');

# Oracle database performance optimizer - run it every time we have significant changes on a database schema
sqlplus /nolog
connect system/
begin
dbms_stats.gather_schema_stats (
ownname=>'PUT_SCHEMA_NAME_HERE',
degree=>3);
end;
/
quit;

# compile all invalid objects in Oracle:
The best way to compile all database objects that are invalid is to use a script in the $ORACLE_HOME/rdbms/admin directory named utlrp.sql. This script finds all objects in the data dictionary that are invalid and compiles them. This script is typically mentioned in patch notes but you can use it any time a schema change occurs.
I also included other alternative approaches just in case.

1. best way:
sqlplus system/password
@$ORACLE_HOME/rdbms/admin/utlrp.sql

2. by schema:
select 'ALTER ' || OBJECT_TYPE || ' ' || OWNER || '.' || OBJECT_NAME || ' COMPILE;' from dba_objects where status = 'INVALID' and object_type in ('PACKAGE','FUNCTION','PROCEDURE') and owner='PUT_SCHEMA_NAME_HERE'
union
select 'ALTER PACKAGE ' || OWNER || '.' || OBJECT_NAME || ' COMPILE BODY;' from dba_objects where status = 'INVALID' and object_type in ('PACKAGE BODY') and owner='PUT_SCHEMA_NAME_HERE';
set pages 50000
spo compile.sql
/
spo off
@compile.sql

3. using the utl_recomp APIs:
connect sys/password as sysdba;
select 'utl_recomp_begin: ' || to_char(sysdate, 'HH:MI:SS') from dual;execute utl_recomp.recomp_serial();
select 'utl_recomp_end: ' || to_char(sysdate, 'HH:MI:SS') from dual;
quit;

# how to enable table shrink in a Oracle 10g database:

alter table mytable enable row movement;
alter table mytable shrink space;

# how to shrink multiple schemas in a Oracle 10g database:

select 'ALTER TABLE ' || OWNER || '.' || OBJECT_NAME || ' ENABLE ROW MOVEMENT;' from dba_objects where object_type in ('TABLE') and owner in ('MY_SCHEMA1','MY_SCHEMA2','MY_SCHEMA3');

select 'ALTER TABLE ' || OWNER || '.' || OBJECT_NAME || ' SHRINK SPACE CASCADE;' from dba_objects where object_type in ('TABLE') and owner in ('MY_SCHEMA1','MY_SCHEMA2','MY_SCHEMA3');

# how to shrink datafiles:
select 'ALTER TABLE ''' || OWNER || '''.''' || TABLE_NAME || ''' SHRINK SPACE COMPACT' from all_tables;

SELECT VALUE FROM V$PARAMETER WHERE upper(NAME) = 'DB_BLOCK_SIZE';

SELECT 'ALTER DATABASE DATAFILE ''' || FILE_NAME || ''' RESIZE ' || CEIL( (NVL(HWM,1)*&&BLKSIZE)/1024/1024 ) || 'M;' SHRINK_DATAFILES FROM DBA_DATA_FILES DBADF,
(SELECT FILE_ID, MAX(BLOCK_ID+BLOCKS-1) HWM FROM DBA_EXTENTS GROUP BY FILE_ID ) DBAFS
WHERE DBADF.FILE_ID = DBAFS.FILE_ID(+) AND CEIL(BLOCKS*&&BLKSIZE/1024/1024)- CEIL((NVL(HWM,1)* &&BLKSIZE)/1024/1024 ) > 0 ;

# how to find which objects have extents at the end of a datafile.
Relocating these objects makes shrinking of relevant datafile possible.

select *
from (
select owner, segment_name,
segment_type, block_id
from dba_extents
where file_id =
( select file_id
from dba_data_files
where file_name = &FILE )
order by block_id desc
)
where rownum <= 5;

UNIX / LINUX quick reference

I have compiled the Unix/Linux code snippets below as a quick reference covering the "how to" on various topics. Please feel free to leave a comment with your own "Unix command line gems".

How to:

# list top memory processes on a Linux machine:
ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr

# show how many CPUs on a Linux machine:
cat /proc/cpuinfo

# create a symbolic link:
ln -s originalPath newLinkPath
e.g. ln -s /usr/local/apache/logs /tmp/apachelogs

# list largest files in Linux (good for file system cleanup):
find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2

# list largest directories in Linux:
du -h / | grep ^[0-9.]*G | sort -g
du -h / | grep ^[0-9.]*M | sort -g | more
find / -type d -size +1G

# split big text files into smaller and more manageable files
split --lines=100000 --suffix-length=3 application.log application.log.a

# redirect Linux output:
ls 1> out.log 2> err.log

# do global string replaces in the VI editor:
:%s/old_string/new_string/g

# set executable tag in shell script files recursivelly:
find . -name "*.sh" -exec chmod +x+x+x {} \;

# delete bak and log files recursivelly:
find . -name "*.bak" -exec rm {} \;
find . -name "*.log" -exec rm {} \;

# recursively determine how much space a directory is taking:
df -h : shows how much space we have on each partition
du -sh * : show how much space the current directory is taking
du -h | grep -v '/' | awk '{print $1}' : does the same without iterating thru the subdirs

# find files that contain a given text string
grep -lir "/data/platform" *

# find files that contain a given text string recursively
find . * -exec grep -li "/data" {} \;
find . * -exec grep -li "/data" {} \; > files.txt

# poll the number of files in a folder every 3 seconds
ksh
while true
do
ls *.retry.xml | wc -l
sleep 3
done

# archive a folder and sub-directories in tar/gz
tar -zcvf archive.tgz directory
e.g. tar -zcvf apacheLogs.tgz /usr/local/apache/logs

# extract a tar/gz archive
tar -xvf archive.tgz directory
e.g. tar -xvf  apacheLogs.tgz .

These are just some of the ones I use the most. You can also find many more at the Unix Toolbox here.

Automated handling of header notices in source files

I recently implemented a web-based application that grew from a proof-of-concept into a real solution for a telecom operator, and it has also been offered to other potential clients. And, since it uses only open technologies such as JSF, Spring and Hibernate, it is pretty easy for other developers to reuse it for demos and prototypes without training on proprietary technologies. It is so that other teams in my company have been asking me to share the source code so they too can use it on their own projects.

While I've worked with few developers in the past who don't quite like the idea of sharing "their" code with other teams, I'm quite proud of this app and have no problems sharing it internally if it could help my company. However, to protect my company’s IP I thought I should at least include copyright messages on every single source code file before sharing it. I know I should have included it from the beginning but when we are working around the clock and we do not have to share the code these kinds of things are easily overlooked. So then I had a project with hundreds of classes and other artifacts without any copyright message. That should be a simple and common task to solve, right? Well, I thought as much... but when looking for a good way to handle it I found that my development tools do not have any functionality I could use to easily handle copyright messages on existing projects.

There are many possible ways to automate this task, from SED-based Unix shell scripts to regular expressions in search/replace functions of more advanced text editors, but I wanted to find something easier to use and more well-integrated to my development environment.

After digging for something in those lines I found a good free solution in an Eclipse plugin called Eclipse Releng Tools plugin, or simply Releng. It can be used to automate the task of including a header notice stating the code’s ownership and/or its license. This article will guide you step by step on how to install, setup and use this Eclipse plugin.

But before we start, a note of warning – YOU should make backups before following these instructions, install and run Releng. It did not cause me any troubles but I should not be held liable in any way if it causes you any problem.


Installation

If you still do not have Eclipse installed, download it here. You should select the distribution package that best suits your specific project, or just pick "Eclipse IDE for Java EE Developers" since it works best for most projects. Note that for Eclipse to run you also need a Java Development Kit – and if you do not have it and do not know how to install it you are not the target audience for this article and should probably not follow its instructions.

Once the download is finished extract the ZIP archive and start Eclipse. It will ask for a workspace folder. You can enter the path where your existing project is, or enter something else if you wish to start a new project or check out from a source code repository system. Click on the Workbench icon to close the welcome screen.

Eclipse's welcome screen

Since the Eclipse Releng Tools plugin is not installed by default in Eclipse we must install it now. In Eclipse’s top menu select “Help” / “Install New Software”.
In the “Work with” tab select “--All Available Sites--”. This will list all the available Eclipse plugins. In the filter bar enter “Releng” to filter the results and show only the plugin we need to install. Check the “Eclipse Releng Tools” plugin and complete the installation process.

Eclipse's available plugins window

After the plugin is installed Eclipse will ask you to restart it. Do that so that it is activated when Eclipse comes back up.

Open your project and note that when you right-click the project root you see the new menu item Fix Copyrights. Do not click on it now; this is just to show that the plugin was installed correctly.

Releng's new menu item


Setup

In Eclipse’s top menu select “Window” / ”Preferences”. In the Preferences window click on Copyright Tool. This will present you with the copyright template.

Releng's default copyright template

Replace the default message with this one below, including the empty lines (one at the top and two at the bottom):

This file contains proprietary information of MyCompany. Copying or reproduction without prior written approval is prohibited. Copyright (C) ${date} MyCompany. All rights reserved.

Set the year fields to 2009 or the current year.
Optionally you can enable or disable the copyright messages on properties and XML files. You can also instruct the plugin to replace existing copyright messages with this new template.


Usage

Now that Releng is properly setup, right-click on your project’s root node and select “Fix Copyrights”. All of your source code should now include a header message with your notice, as shown in the image below.

Project processed by Releng