Ever since Sun Microsystems agreed to acquire MySQL back in 2008, there has been a fair bit of uncertainty and chaos surrounding the world’s most popular Open Source database. With many big names in the MySQL community pulling in different directions and the recent Oracle / Sun acquisition, the choice of which Open Source database to use is now easier than ever – PostgreSQL.

One of MySQL’s key strengths has always been its simplicity to both install, manage and code against, resulting in mass adoption over time, which in turn forced the once web bound database to evolve into a near fully fledged enterprise RDBMS.

PostgreSQL on the other hand was born an RDBMS and instead worked silently in the background improving the performance, reliability and feature list of the world’s most advanced Open Source database, and all under the lesser restrictive BSD license – nice work!.

Getting up to speed with PostgreSQL can take a little longer for MySQL veterans, mostly due to the many subtle differences, but more importantly the extensive feature set (one of the reasons it is taking longer to add PostgreSQL support to dbRecorder ) but the rewards are worth it, as companies such as Skype can attest to.

It was installing OpenNMS several years ago that convinced me to get to grips with PostgreSQL, and although there is a distinct lack of new reading material out there the manual covers everything you need to start building your next web app.

The GUI tool pgAdmin has also come a long way and provides a great interface to PostgreSQL, with excellent instructional prompts and support for all major platforms, there really is no better tool to help you work with PostgreSQL while getting to grips with the vast feature set. Check out the latest beta over at the pgAdmin site.

If you’re not yet excited about PostgreSQL, maybe you should check out the feature list planned for release 8.4 (currently at beta2), with cool features such as Column Level Permissions and SSL Connections you can feel less stressed about the security of your data, oh and the performance improvements won’t hurt either!.

While keeping with the BSD license, another notable release last month was FreeBSD 7.2, a great platform to complement the reliability of PostgreSQL and another way to gain some performance improvements.
Speaking of performance I was pleased to read that Jails are much improved in the latest release, and that it seems I am not the only one that prefers Jails / Zones over VM solutions!.

I’m off to make some progress with PostgreSQLDump.java, the worker class that will allow me to add PostgreSQL support to dbRecorder.
Watch this space…..

There was a time when Novell ruled the server…a time when CNE was a mark of technical excellence and server uptime was measured in years not weeks…a time and when Novell Directory Services (NDS) provided a single point of administration for every aspect of the enterprise environment, including the then evolving Active Directory!.

Yet just when everything was coming together for Novell the attraction of owning the desktop led them to acquire Ximian, the coolest Linux desktop on the block, before joining forces with IBM and acquiring SuSE – possibly the biggest competitor to RedHat at the time.

It was not long after the SuSE acquisition that Novell announced their intention to phase out NetWare, a move that saw many IT Managers faced with the decision of migrating their servers to the then largely untested OES (Open Enterprise Server) or crumbling under the ever increasing pressure of Active Directory and Exchange. Needless to say not many survived to log in via the Novell Client…

I personally focused my energy on FreeBSD and other OpenSource solutions, occasionally rolling out OES to companies that required enterprise support, secure file synchronisation (iFolder) or a rock solid collaboration platform (GroupWise). Novell have always had sensible license fees that were often based on the user model and not server providing massive cost savings.

It was cost savings that I was looking for recently while searching for a cross platform LDAP solution that was easy to manage, secure, reliable and had enterprise support to boot. It was exactly those requirements that led me back to Novell, back to eDirectory, which after all this time continue to offer the Full Service Directory promise that ruled the enterprise several years ago.

Installation of eDirectory on SuSE is seamless, RedHat however required a few dependencies (yum install) but still took under 10 minutes, and that included bolting on iManager the secure web interface to many Novell products (think Console One). Configuring the RedHat clients was just as easy, in fact using the authconfig tool you can configure the client to talk to eDirectory with a single command:

authconfig --enableldap --enableldapauth --enableldaptls --ldapserver=myserver --ldapbasedn=o=myorganisation --enablemkhomedir --updateall

As for adding a replica for resilience, well you only need to install eDirectory onto another server on your network (yes it runs on Windows too!), and as long as you select your existing tree during the install process you have a secure read/write replica – simple eh!.

So it seems Novell never gave up. They focused their energy into migrating all the great tools we took for granted with NetWare while combining the best of breed Linux tools into SUSE Linux Enterprise 11 – Novell’s latest enterprise server platform.

With features such as .NET support for System z mainframes, OCFS2 / cLVM2 and new fine-grained management of CPU, memory, storage and networking resources, it looks like Novell have a worthy contender to the enterprise Linux throne!.
Could Novell be ready to pick up where they left off with NetWare?, only time will tell….now if only eDirectory ran on FreeBSD.

I know it’s a little late but the usual drive of new year innovation and releases within the Open Source world have been keeping me busy recently.

New years resolutions

1. Package AuthStor
2. Learn another RDBMS – Oracle / MSSQL?
3. Upgrade my local servers to FreeBSD 7.1
4. Pick up the guitar at least once a week
5. Lay down my first audio track in Ableton Live

The year started with my attempt at some resolutions (you know the ones you never keep) of which I am already making good progress on.

First on my list was AuthStor which is working on a basic level at the moment but there is so much more that can be done to make it easier to use. Why has it taken so long you may ask?

Well apart from being the only developer last year, I made several attempts to re-write the app in something like JSP or Zend Framework, partly because there is a lot of activity in those communities but mostly due to the complications faced with packaging around 50 Perl modules that each have their own compile time dependencies – some of which change frequently often breaking the application (no strict API/Interfaces to adhere to).

So after several prototypes why did I decide to stick with Perl?
Well for starters the changes that did break the application often forced me to change the app in a way that was more logical or introduced a more structured design. I also found that JSP and Zend although cool frameworks tended to feel incomplete and introduced complexities elsewhere e.g. containers / authentication / roles. Catalyst is simple and extremely powerful providing both a development environment and production framework that takes little effort to introduce new features quickly. I can start 2009 in the knowledge that Perl and Catalyst are the right language / framework for the job. I just need to work on the packaging now!

The next item on my list is to learn yet another RDBMS. Why? Enlightenment first and foremost, plus I am a big believer in gaining as much knowledge as possible in my chosen field. You’ll never complete the jigsaw if you have several pieces missing, and seeing as I spent a good part of last year getting to know DB2 I decided this year that my focus would by Oracle and MSSQL, thus completing the picture.

With the release of FreeBSD 7.1 there are now so many cool features that I decided to upgrade some of my older servers that have been sitting in the corner running nicely on FreeBSD 6 (they never fail). DTrace and ZFS put a stop to FreeBSD slipping behind OpenSolaris and the ULE scheduler helps improve performance over your average Linux distro, and with ULE 3 due for release in FreeBSD 8.0 (June) you can expect to see further dramatic performance improvements.

Last on my list will be to pick the Les Paul at least once a week!. One of my prezzies in 2008 was a new orange amp! If that does not encourage me to pick up the guitar I don’t know what will!. You never know I might even mix something up with the copy of Live I got for Christmas

I spent the last month taking a break from Open Source, partly due to FreeBSD 7.1 failing to hit the FTP servers as scheduled (any excuse), but mostly due to Santa’s early delivery of Ableton Live. Not really Open Source I may add but extremely cool nonetheless and a much needed break from coding!.

I find that staring too much at code can hamper the creative process (hehe), and as you know Creativity and Innovation go hand in hand with Open Source Software!.

Speaking of which, while I have been busy playing with waveforms and elastic audio, the Open Source world has been all guns blazing with cool releases like MySQL 5.1 GA, Open Solaris 2008.11, VirtualBox 2.1 and Wordpress 2.7 to name a few.
You really cannot take a break these days without something big in the world of Open Source happening in your absence.

So the year is nearly at an end which got me thinking, what was the highlight of 2008 in the world of Open Source?.
I have to confess to spending more time than I would have liked this year working with proprietary software, always in combination with Open Source tools of course!. So if I had to highlight anything from 2008 it would be that Open Source is now more pervasive than ever. From your OSX laptop at home to the Linux servers at work, right the way up to the Internet’s biggest sites and services (running FreeBSD ), you’ll find Open Source Software behind every successful venture. In fact it is now so popular even Microsoft could not resist the urge to get in on the game!.

In fact there are so many Companies using Open Source now that proprietary software may actually be the minority! We all know everyone loves an underdog, so maybe propreitary software will be the hit in 2009? Then again with the state of the economy maybe not!

The biggest influence for me this year (both good and bad) had to be Sun Microsystems who played a large part in Open Source adoption in the enterprise and continue to lead the way with their open business model (GlassFish, MySQL, Netbeans, OpenJDK, OpenSolaris and VirtualBox to name just a few of their many contributions).

Although most of the time it seems like they cannot do anything right, one thing is for sure, without the corporate weight of Sun behind the Open Source ethos we may not have made it here so quickly. So while your saving big bucks with your ZFS storage and processing millions of transactions with MySQL, don’t forget to say thank you to Sun next time you are complaining about JavaFX or MySQL 5.1.
You may even want to swap those complaints for contributions (I know complaining is often easier). Thanks Sun!

So if you are one of the few companies yet to Open your code to the world, what are you waiting for? This Christmas give the gift of knowledge and share your achievements instead of fearing the competition and wrapping your code in a black box (don’t worry we won’t laugh at it). You’ll feel much better knowing you helped make the world a better place!

Until next year I wish you all a Very Open Source Christmas!! and as always Happy Coding!

If you are security conscious and nowadays who isn’t?, you may have pondered over ways to make your external SSH connection safer.
Of course there’s always One-time Passwords and Public Keys, but if like me you watch the daily barrage of brute force attacks you may be thinking it’s only a matter of time…

That’s what I was thinking at about the same time an SMS arrived on my mobile….then it hit me….I take my mobile everywhere, if only my server could call me and ask if it is really me trying to log in.

My first instinct was to try and insert a Perl or Python script into the login process. After all Clickatell have several libraries that you can use to send SMS messages. How hard could it be?
Well for starters it didn’t feel right relying on a script that runs within an interpreter that is running within a shell, but how else could it be done? Pluggable Authentication sprang to mind but that’s hard right? Wrong!.

Hitting the FreeBSD handbook I was able to gain a decent understanding of PAM functionality and even managed to find some sample code. Shame the programming section had not yet been written. O’Reilly to the rescue. After reading a few articles on PAM it wasn’t long before I had a working prototype, in fact most of my time was spent deciding upon the right library/s to use.
The end result is a pam module that can be dropped into the authentication chain for any PAM aware application/service – think SSH, FTP, POP3 etc.

Upon successfully entering your Username/Password the pam_codemein module uses libcurl to send an HTTPS request via Clickatell, my preferred SMS gateway, which then forwards the random Code to your mobile before prompting you to enter the same Code at the prompt to gain access. Of course there are issues with using SMS, such as mobile reception and timeouts, but in testing I didn’t notice any.

If you are concerned over the cost of SMS messages, Clickatell are a bulk SMS provider so rates are good, and if you prefer to be contacted via some other method the code can be adapted to any other service e.g. XMPP via TLS, you could even GPG encrypt the Code in an email if you prefer.

It was only after I had the fun of writing this module that I stumbled across this blog post. Shame as it may have saved me a lot of time, but hey where’s the fun in that? I’ll just have to get cracking on the XMPP method – either that or search deeper within Google.

Have you seen RT 3.8!

It’s been a long time coming but after 2 years in the making the enterprise-grade ticketing system that is RT has had a well earned facelift, and what a difference!!.

Upgrading from 3.6.3 would have been straightforward if it was not for the fact that I also decided to upgrade to MySQL 5.1 at the same time. The final product made the pitfalls worthwhile however and RT is now slicker than ever. Plus I got know RT better along the way – bonus!.

If your thinking of upgrading to RT 3.8 / MySQL 5.1 just remember to use the MyISAM engine for the Attachments table (It’s worked so far for me with around 84k tickets). Failing to do so will result in slow response from the UI on certain actions due to MySQL failing to select the correct index on certain queries:

ALTER TABLE Attachments ENGINE=MyISAM;

It looks like the optimiser is broken for the InnoDB plugin, however if you check out this bug it may have been fixed by now (I was using FreeBSD port: mysql-server-5.1.26).

I spend the majority of my days working with OpenSource Unix derivatives and have to say I enjoy every minute of it! Not a day goes by where I do not learn something new or discover a new way to solve a problem all thanks to Unics (Everything has a beginning).

If you are passionate about GNU / Unix / Linux / OpenSource I apologise for the generalization “Unix” but wanted to focus on the OS and tools that make moving between one Unix variant and another as easy as pie.
Putting aside all the nuances that distinguish the many flavors of Unix, once you gain a core understanding of the Community, Tools and Kernels, you find that moving from one OS flavor to another is as simple as uname and man.

Take for instance my latest branch off the Unix tree – HP-UX. Having minimal exposure to HP-UX I wanted to get up to speed quick as possible, so I grabbed a copy of the HP-UX Study Guide and booked the CSA exam knowing that if I had a goal I would not just move onto the next flavor.

The book was a great resource and I only wished I had read it sooner as it covered many tools and concepts that are commonplace in SVR4 Unix derivatives such as Solaris. I felt like I was stepping back 10 years to when I first started to discover basic utilities such as cat, sed and awk but explained in so much detail that I found myself learning some new tricks!. Even if you are not interested in HP-UX this book is so well written and easy to condense that it is worth a place on your Unix bookshelf!.

As for HP-UX, well I felt right at home and only had to spend time learning the key differences such as Software Management and HP’s Partitioning Continuum technology (Virtualization to you and me) – very cool!.

Would I run HP-UX on my own servers? Probably not, it’s not OpenSource for starters (plus I don’t have the hardware), and to be honest I have been running FreeBSD for over 10 years now and no matter how many Unix/Linux derivatives I encounter not one has been able to surpass FreeBSD’s community, stability and manageability. Plus FreeBSD is Free in every sense!.

Speaking of FreeBSD you might have noticed the image in the upper left corner of this post – yes you guessed it; FreeBSD 7.0 has been released!.
I have been running a release candidate for a while now and have to say that the one thing you will notice is how fast it runs. Don’t take my word for that either, check out the Benchmarks and read through the new features explained in detail over at the BSD Devcenter. With performance like that why would you want to run any other OS?

Download your copy from here and never look back.

I read a lot about MySQL backups using LVM Snapshots on Linux, WAFL Snapshots on NetApp and more recently ZFS Snapshots. But did you know you can do the same under FreeBSD?

FreeBSD has had snapshot capability since around 2001 allowing administrators to take a frozen image of a filesystem at a given instant in time with minimal impact on the server / filesystem. So how does this help with MySQL Backups?

If like me you have loads of space on your database volume and often find yourself making frequent risky modifications to your databases, a snapshot can save you loads of time and headaches by creating a point in time marker that you can fallback on if needed.

Snapshots provide the following benefits:

1. Efficient – Only as blocks in the active filesystem are modified and written to new locations on disk does the snapshot begin to consume extra space.
2. Fast – It takes about 30 seconds to create a snapshot of an 8Gb filesystem. Of that time 25 seconds is spent in preparation; filesystem activity is only suspended for the final 5 seconds of that period.
3. Reliable – FreeBSD ensures that no disk write activity can take place during the setup of the snapshot (typically less than a second). The result is a consistent replica.

If you are running MySQL on FreeBSD, snapshots can certainly help improve your backup strategy. So how do you get started?
Well if you head over to the Wiki you’ll find a detailed guide to creating your first snapshot, but before you click that link there is another new FreeBSD feature that you may not be aware of – Security Event Auditing.

Protecting your data requires more than just regular backups and the security auditing feature in FreeBSD 6.2+ can help assist with the logging of activity on your MySQL server.

Based on Sun’s published Basic Security Module (BSM), the de facto industry standard for auditing, FreeBSD Security Event Auditing provides reliable, fine-grained, and configurable logging of a variety of security-relevant system events, including logins, configuration changes, and file and network access. These log records can be invaluable for live system monitoring, intrusion detection, and postmortem analysis.

At the moment the stable release of FreeBSD will require a Kernel compile to enable audit capability, but future releases should have audit enabled in the generic kernel. You never know by the time FreeBSD 7.0 is available I may have figured out a way to audit MySQL logins using auditd.

If your feeling brave you can download the latest monthly snapshot of the upcoming FreeBSD 7.0 release and put auditing to the test.

Right – I’m off to get a working NetUnit jar ready for the end of the week – I hope

Whenever I decide to take a holiday I can usually guarantee that something cool will occur in the IT industry in my absence.

Now this holiday has been slightly different in many ways. I had to cancel my flight due to a sudden onset of Vertigo 1 day before I was supposed to fly. If like me you have never experienced Vertigo before, thank yourself lucky!. Having no sense of balance and intense room spin whenever you open your eyes is no fun at all, it took at least 2 days for me to stop being sick.

Having been able to actually use my laptop in the last couple of days I notice that the “cool thing” I usually miss on holiday turns out to be big trouble all over OpenSourceVille, the majority of which seems to revolve around licensing and intellectual property.

Who could miss the BSD vs GPL debate that I’m sure will rage for a long time after the ath5k driver had the BSD license removed in favour of the GPL. Stealing code was suddenly the headline for this action and the rift between the “Linux” developers and “BSD” developers increased once more.
Personally I feel the license should be as “Open” as possible and always apply the BSD license where I can. I want to be recognised as the author of any work I produce and protected against any misuse of such work (liability). With OpenSource development taking place in clear view nowadays, such visibility can only enforce your ideas and intellectual property. Why complicate things with over restrictive licenses…

While still on the topic of licensing it seems John Birrell is having to start all over again with a new port of DTrace on FreeBSD. The CDDL this time preventing kernel hooks from being added to the FreeBSD kernel. This seems such a shame as DTrace technology provides a massive benefit for software developers and will starting afresh really change the situation?, If not through licensing I am sure there must be patents on the intellectual property that Sun could enforce on any DTrace port? Just look at what is going on between NetApp and Sun at the moment.

So from licensing issues to patent infringement where it seems ZFS may not be as revolutionary as Sun claim and in fact NetApp actually came up with some of the functionality back in 1998. Dave’s Blog outlines all the details in what I can only describe as an excellent explanation as to how NetApp have had to file a lawsuit against Sun. Strangely enough I originally heard about this from the MySQL Planet! and not from any of the Solaris / OpenSolaris Planets each of which have around 45 Sun employee feeds?

We develop software to improve our way of life. Licensing and Patents although often necessary seem only to be hindering progress… Still it keeps us busy re-inventing the wheel.

On a good note, I was around to see the launch of the iPod Touch – if you can call that good…I might just wait until the drive space increases before I buy one, 16Gig seems tiny these days but then again the Touch would come in handy for on the go web browsing.
I also noticed Oracle 11g had been released (I obviously missed that one), although only for Linux at the moment?. I’m no Oracle expert but seeing as Oracle provide Java based development tools and my previous installs had a lot of Java involved surely Solaris would make a better fit?

Image combined using artistry from Project TIMEMACHINE – http://www.timemachine.ne.jp/

I have been running FreeBSD on my servers for years and have made many attempts during that time to use it on my Laptop – all attempts failed… But with the recent commit of xorg 7.2 to the Ports Tree I might just have to give it another go

The upgrade has been 6 months in the making and would not have been possible without the dedicated work of Florent Thoumie , Dejan Lesjak and many other top FreeBSD developers – cheers folks – I look forward to putting it to the test in the next few weeks.

I have spent the last few days trying to find a decent tool to help with software design. I wanted something that could do everything, UML diagrams, Use Case etc, but of all the tools I tried not one of them felt right. There are many tools out there but the majority seemed too slow to work with or too complicated to just whip up a quick diagram.

After all the searching I have yet to find a tool that works better than good old pencil and paper! One tool however does stand out from the crowd and although it focuses on a single task, it performs that task exceptionally well. The tool you ask? Freemind.

I have been using Freemind for a few months now and keep finding new ways to make use of the simple Parent / Child Node tree. Project planning, personal achievement records, website maps etc etc. The list is endless and after reading an article on Developer.com now includes Software Design.

See Wave2 Informeer for a simple example.

I hope to get the mmEvents Daemon running this week, time permitting of course. I seem to be having so many ideas at the moment and unfortunately so little time….

For quite a while I have been playing with Web Frameworks in the hope that I would find a more “Large Scale” method of writing Web Applications.

Since being hooked on Python at the time I started with Zope, reading the programmers guide from front to back I ended up believing that Zope was indeed a great Framework, It made sense why it was rebuilt from scratch, but I was no closer to being able to use it.
Then came Django and TurboGears. Once again I skimmed the manuals and even gave Django an install but they still felt alien to me and incomplete (Maybe things have changed now – that was over a year ago:)).

So I lay down my Python tools and picked up my Java book, after all Java 6 was much faster now and many Java based Web Frameworks were out there and driving some of the largest sites in the world. My only problem now was choice.
Choosing a decent Framework using Java is tough – not only do you need to choose the Framework you need to choose a container to host your application. Do you use JBoss,Geronimo (My Fave), GlassFish or any of the other App Servers out there? and do they support your chosen OS? Java was written by Sun and always runs faster on Solaris – but I use FreeBSD – something that Sun don’t even offer a download of the JDK for!. In the end I got Geronimo working fine on FreeBSD – but it still did not feel right.

That’s when I found Catalyst

Described as an Elegant MVC Framework, Catalyst felt right, looked right and worked perfectly on FreeBSD.
Written in Perl and designed for Perl developers my enthusiasm for Perl 6 drove me to give it a whirl and am I glad I did!
The documentation is excellent, the built in debugging actually assists the developer and the Framework is so flexible you can write your App pretty much any way you like.

I plan to use the Catalyst Framework for the notification application Wave2 Infomeer and will let you know how things work out, but in the meantime if like me you have tried several Web Frameworks and never felt comfortable I’d suggest you give Catalyst an hour of your time – you’ll be impressed!.