Distributed software development collaboration

I'm reading a book, "The World is Flat" by Thomas L. Friedman. It's a pretty good read, with surprisingly informed descriptions, and fairly accurate diagnoses.

The main thrust of the book is how technology is changing the world (making it flat), and it talks about offshoring and global competition. It's definitely food for thought for me, since I'm thinking a lot about that. The "one shore" idea is that you can have a local or distributed software development team, and increase or at least maintain productivity, using new technologies and tools, particularly open source tools.

I'm not trying to build an offshore consultancy, except in the sense that I want to be "offshore", literally, and still working from my sailboat. But I'm not opposed to collaborating with offshore individuals or teams.

As a matter of fact, two strategies I'm interested in pursuing are 1) collaborating with other independent developers, testers, sysadmins, designers, project managers, etc. and 2) building a team from local talent where I'm living, whether that be Seattle, Ecuador, Fiji, or wherever.

The products and services that I'm targeting are software development lifecycle and collaboration tools.

For the first form of collaboration, the idea is to enable a distributed team to work as efficiently as if they were centrally located. Not, something I believe completely possible, but a goat to strive for. A lot can be done in this less than ideal collaboration environment, and there are definite advantages, including the real productivity gains that can be achieved by those that prefer this environment, and the enabling it does by spreading development over a wider geographic pool. Not to mention the typical aims of traditional offshoring that include reduced labor and infrastructure costs.

For the second for of collaboration, the idea is to develop best practices and tools for teams that might otherwise not be able to pool the capital and experience that large scale professional teams in big budget organizations can muster.

The primary purpose of both forms of collaboration for One Shore, and for me specifically, is not, as I said, to build an outsourcing consultancy, but to develop the tools and processes for my own development aims. So the result is the same whether it's a widely distributed development team building the tools that increase our own productivity, and learning the techniques to work disperse, or whether it's building the tools that allows a team of talented, but untrained individuals to work together in a way that typically, only experienced enterprise groups can collaborate with big budget software.

Whoops

I just created one-shore.blogspot.com thinking that this blog had been deleted. Basically, I wanted to say what I'm working on (in my spare time) these days:

One Shore -- my company website for consulting (not actively working on it)
QA Site -- my project for building hosted tools for testing
CuencaTravel -- a portal site based on Joomla started in Ecuador (currently on hold)
PM Site - a spin off of qa-site.com geared towards tasks and requirements instead of bugs and tests

And my new project, code named "fluffy", researching Flex testing tools and strategies. I'm working with Nate on this. He's kind of mentoring my learning Flex. We brainstormed the
idea at a BBQ last Monday. I set up a repository and QA site with all the regular tools: bugzilla, projectpier, trac, blogs, and forum.

So far, I've:

• set up the QA Site
  
• downloaded FlexBuilder 60 day trial and installed the FlexBuilder plugin for Eclipse 3.3
  
• did a couple of tutorials and checked out a bunch of books on safari
• installed FlexUnit and did a couple tutorials with it

I'm pretty impressed with FlexBuilder. It seems well put together. It's impressive for an Eclipse plugin, and I did a lot of work with custom plugins at Varolii. FlexUnit seems to be a half-measure, but probably gets the job done, as far as testing ActionScript at the component level. ASUnit is another project (open source) in the same area.

On wordpress

I've been doing most of my posts on Wordpress lately at http://fijiaaron.wordpress.com and http://oneshore.wordpress.com. Because I like the theme there and I have built up quite an archive. I'm going to resolve to give Blogger another shot.

Like I mentioned on my last post there, At least it has a bigger text area. But adding images is a real pain on blogger. Pointing at custom URL like http://blog.cuencatravel.com is easier with blogger, but hosting my own is easier with wordpress.

Requirements as versioned documents

A requirements document is almost always under revision control. Any requirements management process worth it's salt has a change process. But individual requirements as versioned documents... Again, requirements can be tied to code releases. And it should make the change process easier, as well as querying for requirements. A requirements compatibility matrix could be generated (based on versions of each requirement.) Requirements implemented in this release is just another generated document, based on the source control logs.

Tests as versioned documents

A test under version control could also be branched with a code release. A versioned test suite could change to meet new requirements. A tag of tests could be a suite. Versions could even keep track of test completion status and actual results, but that might be going too far.

Bugs as versioned documents

A bug that is a document under version control could have the heading and description edited without worrying about affecting credibility. Previous versions of the bug would contain the original text. A bug under version control could be branched, and branches could be tied to certain code branches or tags.

Document centrism

A test is a document
A bug is a document
A requirement is a document
A task is a document

A document should have the following capabilities:
Editing,
Versioning,
Commenting,
Sharing,
Searching

A semantic document (like XML) could also be queryable, and have better searching. It could be populated from a template plus data.

Reports are documents generated from a template plus data.
Forms are applications that retrieve data, potentially by processing documents.

A collection of tests, requirements, or bugs is a report generated by querying multiple documents.

System Integrator

According to wikipedia, One Shore pretty much matches the definition of a system integrator.
We pretty much build a QA system from open source parts. A QA site is a hosting solution, a Linux server or VPS, a collection of open source tools, including version control (svn,cvs), build automation (ant, maven, luntbuild), bug tracking (bugzilla, trac, mantis), test management (qatraq, test-site, testopia), automated tests(fit, testng, junit, selenium, watir, samie)project management (dotproject, phprojekt, trac), document management (knowledgetree, alfresco), wiki (dokuwiki, dekiwiki, moinmoin, trac), forum(punbb), blogs(wordpress-mu), etc. and integrating them.

I noticed trac is in there three times. Fit and trac are popular for a reason, and I think it has to do with document-centrism.

Java EE5 setup

Installation Directory [/opt/SDK]: /opt/java/ee5/sdk
Admin User [admin] : admin
Admin Password : secret123
Admin Port: 4848
HTTP Port: 8080
HTTPS Port: 8181
The following items for the product Java Platform, Enterprise Edition 5 SDK
will be installed:

Product: Java Platform, Enterprise Edition 5 SDK
Location: /opt/java/ee5/sdk
Space Required: 215.65 MB
------------------------------------------------
Java 2 SDK, Standard Edition 5.0
Sun Java System Message Queue 4.0
Sun Java System Application Server Platform Edition 9 Update 1
Sample Applications
Java BluePrints
First Cup of Java Tutorial
Open ESB Starter Kit with JBI and BPEL

------------------------------------------------


Installation Successful.

Next Steps:

1. Access the About Application Server PE 9 welcome page at:
file:///opt/java/ee5/sdk/docs/about.html

2. Start the Application Server by executing:
/opt/java/ee5/sdk/bin/asadmin start-domain domain1

3. Start the Admin Console:
http://localhost:4848

Java 6 on Fedora, CentOS5, RHEL5 needs compat-libstdc++-33.i386

I downloaded Java 1.6 with EE 5 and tried to install it on CentOS 5. But it complained about libstdc++.

./java_ee_sdk-5_01-linux.bin: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory

I have libstdc++ installed, but have /usr/lib/libstdc++.so.6.0.8

Sun has a bug on it.

Aparently, Fedora, Redhat, and CentOS are not supported. What they needed was to statically link. The solution is to install a previous version of libstdc++ (libstdc++.s0.5). You can do that with yum:

yum install compat-libstdc++-33.i386

and then run the java installer:

./java_ee_sdk-5_01-linux.bin

--

I got Java 1.6 installed, but java_ee_sdk-5 is still hanging:

see http://docs.sun.com/app/docs/doc/820-0135/gazry?a=view

export JAVA_HOME=/usr/java/jdk1.6.0_03

Bugzilla, CPAN, CentOS, VMWare, VPSLAND

I've been fighting with VMWare, CPAN and CentOS this week.

Okay, really only fighting with VPSLAND, who say they've fixed my account and allow me to re-order. Wrongo. Account not fixed. Ability to order (even with a new account) not possible. So I guess I have to look for another hosting provider.


What I've been doing in the mean time is setting up a VMWare server instance of CentoOS 5. My first goal is getting a development version of Bugzilla running so I can start on writing some tests and dig into improving the Bugzilla Webservice module. That was three days ago.

To be fair, I've been sick, and yesterday I mostly played Tower Defense, a cool flash strategy game. Possibly my new favorite after Dicewars.

VMWare has been flaky with network connections, making it difficult to download dependencies. Part of the problem is my piece of junk Gateway MT3423 laptop, combined with Windows Vista Ultimate, which has made losing wifi a frequent occurence. The other part is with either VMWare or Vista, or the two in cohoots, dropping the connection. It seems VMWare is unable to establish a connection after Windows loses it and regains it. Not without a hard boot at least. I plugged into the router today to fix that, and have been merrily downloading, building, and testing all day.

That's right. It took all day to resolve the dependencies, using CPAN, Yum, RPMFind, and building from scratch when necessary. This is not a process I look forward to with client sites. I still don't have mod_php or java set up.

I wanted to have hand compiled perl,php,apache, apache modules, perl modules, etc. that I could copy from my CentOS VM to a production site, but that's not happening yet. I'm itching to do some development.

CPAN truly is a fragile monstrosity that makes DLL hell look like Nirvana and RPM hell like Moksha. It's the number one reason not to use Perl. But unfortunately, also the number noe reason to use Perl.

I have a working Bugzilla on Windows, using PPMs, but 001compile.t fails on install_module.pl and CPAN.pm. I talked to someone on #mozwebtools about it, but didn't get any info. I wouldn't want to support a Windows/ActivePerl install either. It looks to me like it's just coded to use 'cpan install', and not 'ppm install.' I could have got by, but want to have as clean a test environment as possible for when I ask questions.

Once I got all the bugzilla perl dependencies installed (XML::Parser has circular dependencies that can't be resolved vi CPAN, Chart::Base apparently wasn't on any of the mirrors, and GD just plain doesn't work), then I ran into what I thought was a breaker. It looked like VMWare was up to no good with the network interface, but I guessed, and discovered that CentOS has restrictive firewall rules that prevent anything but localhost from accessing all but port 22 (SSH) and ping requests. I just turned IPTables off for now, and then, after some messing with httpd.conf, I finally got bugzilla working.

Email is next. I can send email from the command line now, but bugzilla didn't send an admin email.