Found a very good overview on "How to Diagnose a Performance Problem in a J2EE System" written by John Bley: http://weblogic.sys-con.com/node/43024
Even though this article was written in 2003 somehow it is still pretty up to date. Guess the ways we build our systems are not changing so fast afterall.
Wednesday, March 17, 2010
Wednesday, March 10, 2010
Virtual Machines - managing virtual drives
From time to time I have to expand the disks on my virtual machines. Or convert disks from preallocated 9+GBs into sets of 2GB files (so they fit in FAT32 USB sticks, for example). I found this tool vmware-vdiskmanager to be the best way to manage these tasks, but it does not come bundled with VMWare Player. It does come, however, in the also free-as-in-beer VMWare Server. So if you need it, download and install VMWare Server here so you can use the examples below.
# VMware Virtual Disk Manager Utility (vmware-vdiskmanager.exe):
- Documentation: http://www.vmware.com/support/ws55/doc/ws_disk_manager_running.html
# Examples:
- Convert existing virtual disk from fixed preallocated size to growable, split into 2GB files (trust me, it helps wonders to set all your virtual disks this way to avoid future headaches):
- Defragment existing virtual disk (unless you have a SSD you should do this every now and then both in the VM and in the host OS):
- Shrink virtual disk:
- Expand virtual disk:
Note: when you expand an already partitioned virtual disk it will not show immediately the new size when you boot your virtual machine. However, now you can use tools inside the virtual machine to expand the partition so the operating system can see and use it. In Linux, for example, you can do this:
Note: the block below worked for me, but as always, MAKE BACKUPS OF EVERYTHING if you don't want to risk losing your precious data. Or don't backup, but don't blame me if you mess up and lose your whole filesystem...
fdisk -l /dev/sdb - write down the start and end cylinders for the target partition
fdisk /dev/sdb
in fdisk use the [d] command to delete the partition - yeah, I know, this is scary, but if you REALLY took the time to backup your virtual machine then go ahead; if everything works your data will not be lost in fdisk
use the [n] command to add a new primary partition, using the same partition number and start cylinder, but use the default value for end cylinder for it to use all the space you made available in fdisk
use the [w] command to write the partition table to the disk and exit - if it asks you to restart your server, do that before proceeding, otherwise keep going
login as root again and run these commands:
e2fsck -f /dev/sdb1
resize2fs /dev/sdb1
mount /dev/sdb1
df -h
If everything worked as planned you should now have a bigger partition, and all your precious data should still be there. If something didn't work, well, you still have your backups, right? RIGHT? ;D
# VMware Virtual Disk Manager Utility (vmware-vdiskmanager.exe):
- Documentation: http://www.vmware.com/support/ws55/doc/ws_disk_manager_running.html
# Examples:
- Convert existing virtual disk from fixed preallocated size to growable, split into 2GB files (trust me, it helps wonders to set all your virtual disks this way to avoid future headaches):
C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -r data.vmdk -t 1 data-1.vmdk
- Defragment existing virtual disk (unless you have a SSD you should do this every now and then both in the VM and in the host OS):
C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -d data.vmdk
- Shrink virtual disk:
C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -k data.vmdk
- Expand virtual disk:
C:\"Program Files"\VMware\"VMware Server"\vmware-vdiskmanager.exe -x 32GB data.vmdk
Note: when you expand an already partitioned virtual disk it will not show immediately the new size when you boot your virtual machine. However, now you can use tools inside the virtual machine to expand the partition so the operating system can see and use it. In Linux, for example, you can do this:
df <- run this to list all mounted partitions Filesystem ... Mounted on /dev/sda1 ... / /dev/sdb1 ... /data <- this is the partition I want to expand
Note: the block below worked for me, but as always, MAKE BACKUPS OF EVERYTHING if you don't want to risk losing your precious data. Or don't backup, but don't blame me if you mess up and lose your whole filesystem...
login as root and run these commands:umount /dev/sdb1
fdisk -l /dev/sdb - write down the start and end cylinders for the target partition
fdisk /dev/sdb
in fdisk use the [d] command to delete the partition - yeah, I know, this is scary, but if you REALLY took the time to backup your virtual machine then go ahead; if everything works your data will not be lost in fdisk
use the [n] command to add a new primary partition, using the same partition number and start cylinder, but use the default value for end cylinder for it to use all the space you made available in fdisk
use the [w] command to write the partition table to the disk and exit - if it asks you to restart your server, do that before proceeding, otherwise keep going
login as root again and run these commands:
e2fsck -f /dev/sdb1
resize2fs /dev/sdb1
mount /dev/sdb1
df -h
If everything worked as planned you should now have a bigger partition, and all your precious data should still be there. If something didn't work, well, you still have your backups, right? RIGHT? ;D
Saturday, March 6, 2010
Solid State Drives - Part 2
On my previous post I tried to convince you that it makes sense to buy an overpriced storage device called SSD if you use computers for a living. Now, to conclude my discussion on SSDs I will go into more low-level details such as:
1. Why I bought specifically Intel's I-25M?
2. How I installed it, and why I cloned my old drive instead of starting it "fresh"?
3. What are the tweaks one needs to apply on a XP machine with SSD?
4. Good comparison of SSDs x HDDs
--------------------
1. Why I bought specifically Intel's I-25M?
- Quick answer: because it currently has the best value if you weight compatibility, performance and price among the top SSDs.
- Verbose answer:
If you are joining the SSD bandwagon you should try to pair it up with a modern operating system, such as Windows 7. Also, plan on doing a clean install of the OS instead of imaging your current hard drive into the SSD. That will simplify things a lot, since older operating systems were not designed with SSDs in mind and may cause issues ranging from sub-optimal performance to even significantly decrease the life of SSDs.
Sadly I could not follow this advice myself - I wanted to install my SSD on my work laptop, since this is what I use 99% of the time for "money-making" tasks. Ya know, unless we rob a bank or do some other illegal stuff we need to do something they call work to get money to buy cool stuff like electronics. My company currently doesn't allow me to use Windows 7 yet, and I didn't want to mess with my VPN. Not having the option to switch my work computer to Windows 7 I had to figure out a way to make it work with XP.
Some SSDs do not support XP well. More specifically, many of them only support something called TRIM in Windows 7. You can look for TRIM and SSD on Google or Wikipedia, but it's enough to say that you NEED the ability to run TRIM at least every week or month, otherwise your SSD's performance and life will sink fast. Intel's SSD works pretty well with XP - in fact, their SSDs are one of the simpler devices to use, requiring the least amount of tweaks to work in top shape with XP. Sure it also helps I got a decent deal from Newegg the last Black Friday! :D
2. How I installed it, and why I cloned my old drive instead of starting it "fresh"?
I had to find a way to clone my original HDD into the SSD so I would not have to reinstall everything, including apps, network settings and more importantly the VPN. Then, after digging through zillions of forums I found that the most important thing you need to pay attention to when cloning XP drives into an SSD is something called partition alignment. You can look it up, but basically it has to do where the XP partition starts in the disk, and for you to get the best performance it must start on a sector that is a multiple of your SSD's page size. Since XP wasn't built with SSDs in mind it always start its partitions on a position that doesn't match the start of a page in your SSD. What this means is that when XP needs to read or write something to the SSD, the SSD needs to do double the work to access two pages instead of one. Some SSDs handle it better than others, but it is a performance constraint nevertheless. If you partition your SSD using Vista or Windows 7 you should be safe, these operating systems always start a partition on sector 2048, which is compatible with most SDDs out there.
Note that partition alignment can give a good performance boost even for regular HDDs, as described by Microsoft here (http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/prork/pree_exa_oori.mspx?mfr=true). For (much) more detailed information on benefits of partition alignment for SSDs see here (http://www.ocztechnologyforum.com/forum/showthread.php?48309-Partition-alignment-importance-under-Windows-XP-%2832-bit-and-64-bit%29-why-it-helps-with-stuttering-and-increases-drive-working-life).
- SSD partitioning:
To ensure proper partition alignment I used the Disk Management tool from my Windows 7 computer to remove the old and re-create a new partition on the SSD. This tool comes bundled with Windows 7 and Vista, just go to Control Panel, Administrative Tools, Computer Management. There I removed all partitions from the SSD (it was a new drive, so I had no data to lose), created a new partition and formatted it with NTFS.
- Disk cloning:
I first used a free tool called Macrium Reflect (http://www.macrium.com/reflectfree.asp) to backup my original HDD. This tool is great for disk image backups, you can even mount the image as a virtual drive in Windows Explorer if you need to retrieve files without imaging a disk. Unfortunately, I found that every time I imported the original HDD image into the SSD Macrium was also bringing the bad alignment from the original disk. I could not find a workaround for it, but at least I was happy to backup the original disk in case something bad happened.
Looking for a (free) solution I found on some forums people recommending a trial version of a tool called Drive Snapshot. It didn't work for me for some reason, like Macrium this tool was reverting my SSD to the original bad alignment.
Then I found something that finally worked, a free tool called XXClone (http://www.xxclone.com/). I thought it wouldn't work, it seems the latest version of this tool was released in 2007. I was surprised to find that the SSD was able to boot after XXClone, and better yet, the partition was still properly aligned!
3. What are the tweaks one needs to apply on a XP machine with SSD?
Ok, so after cloning was completed I got the disk to boot on my laptop. Now it was time to tweak XP so it would work with the best performance and wouldn't decrease the life of my SSD.
Below are the changes I made to the XP registry for it to work better with the SSD. If you want you can just copy/paste this block into a new file called "SSD.reg" and double-click it to apply the changes. While it worked fine for me I WON'T BE LIABLE IF IT BREAKS YOUR COMPUTER, SO USE IT ON YOUR OWN RISK.
4. Good comparison of SSDs x HDDs
On my previous post I presented some synthetic benchmarks taken with CrystalMark. That really doesn't mean much in terms of real life experience. To get a glimpse of how SSDs perform when compared to regular HDDs take a look at videos like these on YouTube:
- http://www.youtube.com/watch?v=T_Jz7IMwBt4&feature=related
- http://www.youtube.com/watch?v=96dWOEa4Djs&feature=related
1. Why I bought specifically Intel's I-25M?
2. How I installed it, and why I cloned my old drive instead of starting it "fresh"?
3. What are the tweaks one needs to apply on a XP machine with SSD?
4. Good comparison of SSDs x HDDs
--------------------
1. Why I bought specifically Intel's I-25M?
- Quick answer: because it currently has the best value if you weight compatibility, performance and price among the top SSDs.
- Verbose answer:
If you are joining the SSD bandwagon you should try to pair it up with a modern operating system, such as Windows 7. Also, plan on doing a clean install of the OS instead of imaging your current hard drive into the SSD. That will simplify things a lot, since older operating systems were not designed with SSDs in mind and may cause issues ranging from sub-optimal performance to even significantly decrease the life of SSDs.
Sadly I could not follow this advice myself - I wanted to install my SSD on my work laptop, since this is what I use 99% of the time for "money-making" tasks. Ya know, unless we rob a bank or do some other illegal stuff we need to do something they call work to get money to buy cool stuff like electronics. My company currently doesn't allow me to use Windows 7 yet, and I didn't want to mess with my VPN. Not having the option to switch my work computer to Windows 7 I had to figure out a way to make it work with XP.
Some SSDs do not support XP well. More specifically, many of them only support something called TRIM in Windows 7. You can look for TRIM and SSD on Google or Wikipedia, but it's enough to say that you NEED the ability to run TRIM at least every week or month, otherwise your SSD's performance and life will sink fast. Intel's SSD works pretty well with XP - in fact, their SSDs are one of the simpler devices to use, requiring the least amount of tweaks to work in top shape with XP. Sure it also helps I got a decent deal from Newegg the last Black Friday! :D
2. How I installed it, and why I cloned my old drive instead of starting it "fresh"?
I had to find a way to clone my original HDD into the SSD so I would not have to reinstall everything, including apps, network settings and more importantly the VPN. Then, after digging through zillions of forums I found that the most important thing you need to pay attention to when cloning XP drives into an SSD is something called partition alignment. You can look it up, but basically it has to do where the XP partition starts in the disk, and for you to get the best performance it must start on a sector that is a multiple of your SSD's page size. Since XP wasn't built with SSDs in mind it always start its partitions on a position that doesn't match the start of a page in your SSD. What this means is that when XP needs to read or write something to the SSD, the SSD needs to do double the work to access two pages instead of one. Some SSDs handle it better than others, but it is a performance constraint nevertheless. If you partition your SSD using Vista or Windows 7 you should be safe, these operating systems always start a partition on sector 2048, which is compatible with most SDDs out there.
Note that partition alignment can give a good performance boost even for regular HDDs, as described by Microsoft here (http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/prork/pree_exa_oori.mspx?mfr=true). For (much) more detailed information on benefits of partition alignment for SSDs see here (http://www.ocztechnologyforum.com/forum/showthread.php?48309-Partition-alignment-importance-under-Windows-XP-%2832-bit-and-64-bit%29-why-it-helps-with-stuttering-and-increases-drive-working-life).
- SSD partitioning:
To ensure proper partition alignment I used the Disk Management tool from my Windows 7 computer to remove the old and re-create a new partition on the SSD. This tool comes bundled with Windows 7 and Vista, just go to Control Panel, Administrative Tools, Computer Management. There I removed all partitions from the SSD (it was a new drive, so I had no data to lose), created a new partition and formatted it with NTFS.
- Disk cloning:
I first used a free tool called Macrium Reflect (http://www.macrium.com/reflectfree.asp) to backup my original HDD. This tool is great for disk image backups, you can even mount the image as a virtual drive in Windows Explorer if you need to retrieve files without imaging a disk. Unfortunately, I found that every time I imported the original HDD image into the SSD Macrium was also bringing the bad alignment from the original disk. I could not find a workaround for it, but at least I was happy to backup the original disk in case something bad happened.
Looking for a (free) solution I found on some forums people recommending a trial version of a tool called Drive Snapshot. It didn't work for me for some reason, like Macrium this tool was reverting my SSD to the original bad alignment.
Then I found something that finally worked, a free tool called XXClone (http://www.xxclone.com/). I thought it wouldn't work, it seems the latest version of this tool was released in 2007. I was surprised to find that the SSD was able to boot after XXClone, and better yet, the partition was still properly aligned!
3. What are the tweaks one needs to apply on a XP machine with SSD?
Ok, so after cloning was completed I got the disk to boot on my laptop. Now it was time to tweak XP so it would work with the best performance and wouldn't decrease the life of my SSD.
Below are the changes I made to the XP registry for it to work better with the SSD. If you want you can just copy/paste this block into a new file called "SSD.reg" and double-click it to apply the changes. While it worked fine for me I WON'T BE LIABLE IF IT BREAKS YOUR COMPUTER, SO USE IT ON YOUR OWN RISK.
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters] "EnablePrefetcher"=dword:00000000 [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Dfrg\BootOptimizeFunction] "Enable"="N" [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\OptimalLayout] "EnableAutoLayout"=dword:00000000 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management] "LargeSystemcache"=dword:00000001 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management] "DisablePagingExecutive"=dword:00000001 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management] "ClearPageFileAtShutdown"=dword:00000000 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem] "NtfsDisable8dot3nameCreation"=dword:00000001 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem] "NtfsDisableLastAccessUpdate"=dword:00000001 [HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Explorer] "AlwaysUnloadDLL"=dword:00000001
4. Good comparison of SSDs x HDDs
On my previous post I presented some synthetic benchmarks taken with CrystalMark. That really doesn't mean much in terms of real life experience. To get a glimpse of how SSDs perform when compared to regular HDDs take a look at videos like these on YouTube:
- http://www.youtube.com/watch?v=T_Jz7IMwBt4&feature=related
- http://www.youtube.com/watch?v=96dWOEa4Djs&feature=related
Subscribe to:
Posts (Atom)