The view from a Document Examiner

Michael Ansell, Document & Handwriting Examiner

EWI member Michael Ansell describes his professional activity and demonstrates that, far from a quiet life in the laboratory, his work can produce some startling experiences. 

The fundamental problems

Disputes over failed software implementation projects raise interlinked technical and legal issues which are complex, costly and time-consuming to unravel. This is so whatever the financial size of the claims and counterclaims, the facts and circumstances of the contract between the parties, or the conduct of the project and the management and software development methodologies which may have been used.  Such projects are often terminated, with the software rejected, amidst a considerable range and variety of allegations expressed by both supplier and customer.  These include allegations of incomplete or inadequate delivery, software defects, database errors, faulty design, operational deficiencies, performance shortfalls, systems instability or unreliability, shifting user or business requirement specifications, poor project management, inadequate systems or acceptance testing, lack of co-operation, delays and cost over-runs. 

Increasingly, such disputes are ending up in the courts or are dealt with by other forms of dispute resolution such as mediation or arbitration. An IT expert witness is retained by each party to give an independent opinion on what went wrong technically/managerially, what and how bad are the consequences, what is the final state of the software, and who is to blame.

I have been involved as an independent expert witness in over one hundred cases over the past fifteen years, for claimants and defendants; systems/software customers and suppliers; in the High Court, or in arbitration and mediation, and in other forms of alternative dispute resolution.  I am also experienced as a (CEDR-trained) mediator and as an ICC arbitrator.  Matters of dispute have included the software development cases in the English High Court which hold the record for the longest trial (GEC Marconi v LFCDA, 1991-92), where a CASTELL consultant spent some 40 court days being examined in the witness box in a trial which went on for well over one year; and for the largest claim (£200m+, with £50m counterclaimed, heard briefly in the Technology and Construction Court of the High Court in October 2001 prior to settlement), over the failure of a high-profile outsourcing agreement.

The expertise and experience generally required for this software development/implementation contract expert witness work is that of an IT professional skilled in software engineering, implementation and development practice; project management; and information systems strategy and procurement.  The work has inter alia these important features:

• Responsibilities and duties:  an expert has responsibilities to the client and to the instructing solicitor to apply expertise to define and investigate the technical issues on foot between the parties as thoroughly, but nevertheless cost-effectively, as possible.  The expert’s primary duty is, however, to the court in establishing the ‘technical truth’ and assisting the court in determining the issues, on an objective and unbiased basis.

• Tasks:  these come down essentially to ‘investigation, investigation and investigation’: of documents, of people, and, most importantly, of the disputed software and system itself (usually the best evidence there is, to the expert).

• Main deliverable – the expert’s report:  the fundamental reason why the IT expert witness is there is to produce a thorough and well-rounded report, written in clear English, explaining the expert investigations carried out, the analysis of the evidence, the IT software engineering principles (and industry custom and practice) which have been applied.

• Providing opinions on the key technical issues which lie at the heart of the case.  The expert witness is, apart from the judge, the only person in the litigation who is entitled to give opinion evidence (rather than factual evidence).  Such opinions must be objectively justified wherever possible, unequivocally stated (‘on the one hand, on the other hand’ formulations generally have no place in experts’ reports), and compellingly expressed.  ‘If you haven’t yet reached a firm opinion, you haven’t yet done the work’ is a good principle to go by as an expert.  Mind you, you do not always get the budget to do all the work you would like; and so to…

• Fees/costs:  as an expert, or consultant, or AICS member), one always has to remember that ‘one man’s fees are another man’s costs’.  Expert work has an additional guiding principle which all involved in litigation have to follow. Arising from the Civil Procedure Rules – namely, that costs must also be ‘proportionate’ (for example - you must be careful not to embark on work likely to cost £0.5m in professional fees for a case where the claim is only for, say, £0.25m).  This is not always an easy principle to observe in software development cases. The difficulty and effort of investigating technically complex issues of software design, construction, testing, functioning, performance and so forth can in many ways be just as great for a £100,000 ‘customised off the shelf package’ incomplete implementation as it can for a failed £10m bespoke software development project.  This problem is lessened by, wherever possible, agreeing with the expert on the other side a core list of issues; and also by producing joint statements on matters on which experts can agree, thus taking areas of the dispute out of issue and reducing the need for investigation (and the associated costs) of those particular points.

It is worth saying a few words on what an IT expert is not:

• An advocate: the expert is not there to argue the client’s case – there are already enough expensive lawyers on the case to do that!  Seriously, the courts take a dim view of experts straying into advocacy.  The judge requires the expert to be an independent ‘partner’, providing clear understanding, explanation and opinion, in a dispassionate and non-partisan fashion, in order to gain determinative illumination and insight into complex technical matters.  If, for example, the expert’s true opinion is that some (or all) of the client’s claims or counter-claims, then this should be made clear, early and unequivocally.

• A narrow technical specialist: disputes over software development or implementation contracts can often call for some narrow specialist technical expertise (such as practical experience of and proficiency with a particular software development language, RDBMS, toolset), but limited technical specialisation of this type is generally not sufficient for arriving at the full-bodied opinion on the usual range of technical issues in such cases which is necessary in order for that opinion to be credible and therefore valuable to the court.  When appointed expert on some larger cases, I may make use of a team of other consultants, including technical specialists with particular skills and proficiencies to assist in technical investigations.  At the end of the day, however, I have to direct and understand the rationale for and results of such work, be personally convinced of the conclusions reached and thus be able fully to explain, support and ‘own’ (and be examined in court on) the final opinions I give.

• “I would have done it this way”:  it is not adequate to provide an opinion expressed simply in this way, however good the expert’s personal record as a software engineer or project manager in the field historically may be.  The court needs to be presented with an objective, rational basis for analysis and how conclusions and opinions have been reached, underpinned by engineering and project management principles and/or justifiable statements as to accepted industry custom and practice.

• A software fixer:  beware! – it can be a temptation for an expert team to stray into ‘trying to put the failed project right’ mode when carrying out their investigations.  To give a strong, analysed and justified opinion in the expert’s report on whether or not some particular software behaviour is a serious fault does not generally require first to find, test the fix for it and put the software right.

• An academic (‘custom and practice, not theory’):  it is true that for some types of IT litigation, background and experience as an academic (e.g as a professor of computer science or software engineering at a British university) can be useful. Instances include some intellectual property and ‘hacking’ cases.  However, in my experience, most commercial software development/implementation contract disputes are really looking for assistance from the type of opinion that comes from the ‘grey hairs and scars’ of real-world project involvements (and failures!), and for seasoned practical project management experience serving actual paying customers/clients. This is something which (only?) years of working as an information systems practitioner can give you.

Succinctly, the typical scope of work as an expert witness in such disputes is framed by consideration of the following:

What investigations should be done?  At the start of looking at a case, it can be confusing to the inexperienced to know just where to start applying your (costly) expertise.

What evidence should be considered?  Usually, but by no means always, there is already assembled a possibly large portfolio of project and technical documents by the time the expert witness is instructed.  So the first task is to read all the documents.  Equally important is it to identify and request all the documents not yet to hand but which the expert considers should be in existence for a software project of this nature.  And another first task is to interview as many of the key people, singly, in the shortest reasonable time.

The software itself: best evidence?  ‘Let the software speak for itself’.  Much of the work of an expert, when the disputed software/system is still available to be tested (not always the case), consists of identifying, planning, preparing and running appropriate expert testing (usually jointly with the expert for the other side) to demonstrate the presence or absence of the alleged faults in, or enhancements to, the system which are often at the heart of the dispute.  The results of such testing can have a dramatic effect on the continuation of the action. In my experience, it is not unknown for even the suggestion of rigorous independent expert testing of the software itself (perhaps proposed as to be carried out before the judge) to hasten a satisfactory settlement of the dispute.

What is the expert’s opinion?  To repeat – this is what the expert has been instructed to give. If the expert is not the sort of person who can reach a firm opinion, with objectively good, rational, defensible analysis to back it up, and expressed in compelling, clear English, and who is prepared to defend a position, convincingly and unshakeably, when cross-examined on it in court, then…

Settlement! or ‘Good experts don’t go to court’.  Just as the Observer Corps’ motto is ‘Time spent in reconnaissance is never wasted’, in my experience that for IT software contract disputes runs: ‘Money spent on a good expert is never wasted’.  In the great majority of all the cases in which I have been involved as an expert witness, a satisfactory settlement of the dispute was achieved between the parties shortly after serving the report, avoiding the need to proceed to trial and resulting in a huge saving in costs to both sides.

One of the most important issues on which the expert is almost always asked to give an expert opinion in these cases is:  what was the quality of the delivered software and was it fit for purpose ?  To answer this, CASTELL Consulting has over the years developed a range of rigorous analytical techniques, Forensic Systems Analysis, for assessing failed, stalled, delayed or generally troublesome software development and implementation projects.  These techniques, founded on sound software engineering principles, are objective and impartial, favouring neither customer nor supplier, software user nor software developer.  This objective and unbiased approach accords with what is required of the expert witness, whose primary duty is to the court in finding the ‘technical truth’.  It is particularly important technically where software projects involve a mixture of customised software packages and ‘bespoke’ software construction.  These techniques are becoming internationally accepted and an account of the Forensic Systems Analysis methodology was published in the October 2000 issue of Charter, the Journal of the Institute of Chartered Accountants in Australia; and in the October 2001 issue of the UK Barrister magazine.  They are to be further developed and promulgated, as a contribution to good IT professional practice, on the proposed website www.ForensicSystemsAnalysis.com.

Members of the AICS could well consider developing their careers in the expert witness direction. They should have wide experience, be technically rigorous and independent in thought, and be critically objective. Above all, they should be able to write clear, good English.  One of the best ways to get started in this field is to work with an established consultant who frequently does expert work.  I myself am always interested in hearing from independent software practitioners with particular expertise and interests who could for example assist on some of the larger projects where an expert team is necessary.  Bodies like the Expert Witness Institute and the Law Society, which publishes an annual directory of ‘checked’ expert witnesses, welcome IT consultants and contractors who have had some practical experience of expert witness work.