Home

davidhope & associates pty ltd

 

Justification For Large SCADA Systems

by David Hope, B.E. (Hons.), Sydney Water Corporation.

Presented to SCADA at the Crossroads Conference. Hosted by the Institution of Engineers, Australia.

Perth, Western Australia. November, 1998.

ABSTRACT

The Sydney Water Corporation’s IICATS Project has been the largest single water utility SCADA (monitoring, control and telemetry) project undertaken over the last decade in Australia. It was designed to cover assets such as water and wastewater pumping stations, water reservoirs, and minor geographically distributed assets.

This paper covers the major business cases and business decisions made throughout the Project, the first being in 1989 and the latest being in 1998.

The experience of Sydney Water Corporation in justifying SCADA systems is similar to that of most utilities. Although hard tangible benefits can, and should, be identified to meet business justification requirements, many unquantifiable or unanticipated benefits have subsequently been demonstrated. These less tangible benefits add considerable value to the investment and help to illustrate that the use of modern SCADA technology is a fundamental strategic tool for operations as the new century approaches.

To ensure the benefits are properly evaluated, the tactical and strategic needs of all users in an organisation should be taken into account when formulating and justifying a proposal. In stategic terms, embracing the potential benefits offered by appropriate modern technology is necessary in maintaining an organisation’s competitive position, and in meeting contemporary expectations.

The author, David Hope from Sydney Water Corporation, is an end user of SCADA, and has been responsible for obtaining approval for the 1993 and 1998 Business Cases. This paper presents the business issues from a non-technical perspective.

SUMMARY

The Sydney Water Corporation’s IICATS Project has been the largest single water utility SCADA (monitoring, control and telemetry) project undertaken over the last decade in Australia. It was designed to cover assets such as water and wastewater pumping stations, water reservoirs, and minor geographically distributed assets.

In this paper a brief history of the Project is given, and a picture of the overall vision painted. The Project has had changes in direction over its life, and the author will cover:

  • the original business decision to go ahead with the Project,
  • the 1993 business review of the Project which resulted in a reduction in the initial scope to cover the water system only at a cost of $94M,
  • the 1998 decision to proceed with much of the remainder of the original scope (mainly wastewater pumping stations) at an estimated cost of $57M.

The experience of Sydney Water Corporation in justifying SCADA systems is similar to that of most utilities. Although hard tangible benefits can, and should, be identified to meet business justification requirements, many unquantifiable or unanticipated benefits have subsequently been demonstrated. These less tangible benefits add considerable value to the investment and help to illustrate that the use of modern SCADA technology is a fundamental strategic tool for operations as the new century approaches.

To ensure the benefits are properly evaluated, the tactical and strategic needs of all users in an organisation should be taken into account when formulating and justifying a proposal. In stategic terms, embracing the potential benefits offered by appropriate modern technology is necessary in maintaining an organisation’s competitive position, and in meeting contemporary expectations.

The author, David Hope, from Sydney Water Corporation, is an end user of SCADA, and has been responsible for obtaining approval for the 1993 and 1998 Business Cases. This paper presents the business issues from a non-technical perspective.

1 INTRODUCTION

Sydney Water Corporation’s (SWC) principal business is the supply of water and sewerage services to over 1.5 million homes and businesses in the Sydney metropolitan and Illawarra areas. The nett current value of the operating assets is around $Aus 13 billion, and the annual income from these operations exceeds $Aus 1.0 billion.

Since 1960, telemetry has been used to monitor the performance of water and sewage pumping stations, and water reservoirs. This telemetry was designed, assembled and installed in-house, and eventually featured a variety of technologies, depending on the "era" of installation at each site. The technology more-than-met the needs of the Corporation until the late 1980’s.

As business needs changed, driven by higher customer and community expectations, the limitations in reliability and capability of the old technology was beginning to become increasingly apparent. This led the Corporation to consider a program of complete replacement, taking advantage of the opportunities for improvement in its operations offered by the rapid advances taking place in electronic and information technology.

Investigations began in 1987, and in 1989 an initial Business Case gained approval in principle for the replacement of all the old telemetry at an estimated cost of $Aus100 million.

The proposed new system was given the name IICATS - Integrated Instrumentation, Control, Automation and Telemetry System.

The current status of IICATS is:

  • Illawarra IICATS was completed in 1994 at a cost of $Aus12 million, including some site upgrades. It covers about 150 water and wastewater sites in the geographically separate Illawarra Region. The system was supplied by MITS-Megadata.
  • Metropolitan IICATS - Water was completed in 1997 at a cost of $Aus94million and covers 350 sites. It included considerable provision for the later connection of sewer assets. The system is a proprietary system supplied by Logica of the UK.
  • Metropolitan IICATS - Wastewater was commenced in 1998, and is due for completion in 2002. The estimated capital cost is $Aus57million, covering 700 sites. Sewer sites will be connected to the same telemetry kernel as the water sites. The kernel will be upgraded as required.

This paper will concentrate on the justification of Metropolitan IICATS, and the benefits obtained.

2 DESCRIPTION OF METROPOLITAN IICATS

The major components of the Metropolitan IICATS- Water Telemetry System are shown diagrammatically in Appendix A. The components are:

  • The IICATS Telemetry system software kernel
  • Four telemetry computers (DEC Alpha 3000/400’s)
  • 350 intelligent remote terminal units (RTUs)
  • An intelligent and resilient wide-area communications network, utilising a Telstra X.25 network with around 60 Sydney Water owned packet switching exchanges
  • 50 windows-based workstations
  • Connection to Corporate mainframe and web browser

RTU sites vary in complexity from Ryde Pumping Station, where 13 large units pump to 5 main destinations, to simple flow meter or pressure monitoring sites. A typical large site might contain an elevated reservoir, two surface reservoirs and a four unit pumping station.

Features of the system include:

  • Distributed intelligence at each RTU, allowing independent operation and storage of data in the event of a communications failure
  • Ability to down-load programs and data to RTU’s from the Telemetry Computer
  • Ability to take manual control of sites
  • Ability to supervise the operation of multiple sites from the Telemetry Computers (e.g. schedule pump operation to save energy, or move water through reservoir storages to match Water Filtration Plant production to demand)
  • Flexible alarm reporting
  • Greatly increased information about each asset to assist operations, maintenance and incident management
  • Trending of data from various sites to facilitate fault finding and system optimisation

These advanced features allow the water supply system to be operated in an integrated way, not as a series of individual "islands" of assets, as previously.

The water system is operated and monitored from a specially established Systems Operation Centre, which is open 24 hours per day. The sewerage system is also monitored using the old system field devices and communications, re-routed through the IICATS top-end.

3 VISION AND BUSINESS DRIVERS

In the late 1980’s, the revolution taking place in information technology was seen at strategic management levels within Sydney Water as providing an enormous opportunity to help break old, unproductive work, management and power paradigms(remember that word!), by empowering people and encouraging teamwork.

The vision for IICATS was that it would provide integrated information and modern capability to a wide range of users including planners, operators, contractors, business strategists and senior management, so that the customer service and asset management functions of the Corporation as a whole could be optimised.

The original objectives for the project were to:

  • Improve levels of customer service as a result of improved real-time information and capability
  • Reduce recurrent expenditure through more efficient systems operation and maintenance by availability of better information and systems diagnostics, and by system automation
  • Reduce capital expenditure through improved planning, and optimisation of existing infrastructure

In more recent times, objectives or business drivers have been expressed thus:

  • Financial performance - Return on Investment
  • Meet customer contract
  • Meet requirements of Regulators (e.g. EPA, Health, WorkCover)
  • Display competitive performance to maintain Operating Licence
  • Meet contemporary expectations of performance
  • Meet performance benchmarks compared to other organisations
  • Position the Corporation for changing demands in the future
  • Minimise risks by avoiding/managing incidents

The use of these strategic business drivers, which reflect high-level Corporate planning, allows SCADA projects to be placed in a strategic context where the benefits to the organisation as a whole may be more easily evaluated.

4 ORIGINAL BUSINESS CASE - 1989

The original Business Case included nearly 2000 sites in the Sydney and Illawarra water and wastewater systems, and had an estimated capital cost of $A100million.

The general objectives are stated above. The formal justification relied heavily on broad assumptions and global estimates of future savings in operation, maintenance and capital for the water and sewer assets, within the context of the vision. Typically, savings of 5 or 10% of total expenditure on these items were used.

On this basis, the project had a pay-back period of about six years.

5 BUSINESS REVIEW - 1993

By 1993, the estimated capital cost of Metropolitan IICATS had risen to $150 million, and an intensive financial review was ordered by the incoming Managing Director.

Items to be given special consideration included:

  • staging the project
  • reducing capital requirements
  • financial viability
  • operators needs, and
  • extending life of existing system

No benefits were to be claimed unless they could be properly quantified and unless the responsible manager was prepared to sign off that the benefits would be achieved by being reflected in future budgets.

Rigorous needs and benefits analyses was undertaken involving all users of the telemetry system, including system planners, operators and business planners.

The results were as follows:

Water System

The major quantifiable benefits were:

Capital savings - $A35million

Savings in sizing the clear water tanks at the planned privately-owned Water Filtration Plants serving over 90% of Sydney’s population. Without IICATS, much larger storages would be required at the plants to even out supply and demand.

Energy savings - $A1.8million per annum

Scheduling pumps to run in off-peak periods & limiting number of pumps running to reduce demand charges.

Efficiency gains - $A1.5million per annum

The new technology allowed more effective use of staff, and improved productivity.

All these quantifiable benefits have now been achieved. Some smaller quantifiable benefits are scheduled for later years.

The Net Present Value (NPV) analysis based on these quantifiable benefits, and on discounting the sunk costs, showed a positive return period of greater than 10 years - only marginally financially viable. However, by relying on very strict criteria for quantifiable benefits, the calculated NPV represented the most pessimistic result, unlike many business cases where the most optimistic, and usually unrealistic, benefit scenario is used to calculate the financial indicators. When the sensitivity was checked using relatively modest estimates of benefits classified as unquantifiable (e.g. maintenance, incident management, capital avoidance or deferral), the payback period is reduced to around 5 years.

Wastewater System

The wastewater operators already received a monitoring service from the existing telemetry system, which just met their current needs, although it was sometimes unreliable. The system was still capable of being maintained, despite the fact that some parts were over 30 years old. The operators were not able to find enough benefits to justify the capital expenditure at that time.

The planners gave some indication that licencing and regulatory requirements were liable to tighten considerably in a few years’ time, and that the old system was unlikely to meet minimum standards when that happened.

The wastewater system implementation was therefore staged to follow the water implementation, with allowance in the water implementation made to ensure that wastewater could be readily added to the same telemetry kernel.

6 WASTEWATER SYSTEM BUSINESS CASE - 1998.

The preparation of the Business Case was preceded by a strategic level Value Management Study (VMS) in 1997. Apart from being mandatory for large projects in Sydney Water, the VMS provides an ideal forum to:

  • Bring all the prospective users together to ensure a broad understanding of all needs and opportunities arising from the proposal for the Corporation as a whole
  • Confirm overall requirements
  • Test alternative options
  • Provide guidance on scope, justification and other aspects of the proposal in the context of the Corporation’s strategic plans

The VMS identified the major relevant areas of Corporate need as:

Regulator - Environment Protection Authority (EPA)

  • Licence compliance
  • Reporting on performance
  • Demonstrated diligence in management
  • Demonstrated excellence in extracting the best overall performance from the wastewater system

Shareholder

  • Efficiency of operation reflected in cost per property
  • Protection of Environment
  • Return on funds invested
  • Enhanced incident management capacity
  • Credibility in the community

Employees

  • Appropriate tools to enable business needs to be met effectively
  • Modern standards of information availability

The VMS concluded that:

  • The existing telemetry system did not have the minimum capability or reliability to meet these needs, in particular to avoid successful prosecutions under NSW environmental protection laws(particularly for dry weather overflows from pumping stations), or to meet new requirements for sewerage system licencing to be imposed by the NSW Environment Protection Authority (EPA).
  • IICATS is now proven technology which can meet these requirements, at the same time maximising the Corporation’s investment in the IICATS telemetry kernel
  • The project should cover all Sewage Pumping Stations(monitor and remote control), Wastewater Treatment Plants (monitor main parameters), and a selection of wastewater system pilot sites (overflows - some with remote control, sewage flow gauges, and rainfall gauges). The pilot sites will allow testing of opportunities for whole system operation, and positions the Corporation for the future.

The VMS considered a series of options ranging from indefinitely patching up the existing system, to providing a "Rolls-Royce" solution which involved the renewal of much of the existing control equipment. Options involving retaining existing equipment, or monitor only via IICATS, were rejected as not meeting minimum standards when measured against current strategic needs and objectives.

The option recommended in the Business Case involved connecting all available signals at existing sites, to ensure that IICATS remote control and monitoring reliability were secured, without chasing all standard IICATS signals by replacing starters, etc. Seven options, with varying scopes and implementation strategies, were considered in detail and subjected to financial modelling.

The recommended option featured a programmed implementation over four years, compared to ad hoc replacement, which was assessed at taking place over an eight year period. Based on the actual results of previous projects, the capital savings due to an organised, programmed approach were taken as a minimum of 10% of the total capital cost. This approach also had the advantage of delivering benefits four years earlier, and considerably reduced the risks posed by breakdowns of old equipment over an extended period.

Justification

The justification resulted from a rigorous analysis of options, along with a detailed assessment of benefits accruing to users.

Major benefit elements were:

Capital saving: 10% of the total capital cost due to programmed approach.

Operational Cost Savings:

  • 21 Benefits were quantified resulting in estimated annual savings of $2.2million per annum.

These benefits were split into:

Direct Cost Savings (reduced overtime, maintenance contract payments, etc.) - 45%

Opportunity Cost Savings (existing staff freed to do other work) - 55%

Confidence factors ranging from 20% to 80% were applied to each item in this category to ensure that the value used in the financial model gave the most pessimistic result. The net amount claimed in the Business Case using this method was $1.15 million.

  • 31 Benefits which could not be quantified, which were assessed High, Medium or Low Value.

Most of these benefits related to better access to information, better co-ordination of resources, labour savings due to future reorganisations which would not be possible without the IICATS tool, better image in the community, ability to demonstrate due diligence, assistance to other programs, reduction of risk and better incident management.

All quantifiable capital and operational benefits were fed into the Corporation’s Cost/Benefit Analysis and Economic Model, with the following results for the pessimistic case (pilot sites excluded):

Capital Cost $54M

Internal Rate of Return 8.1%

Net Present Value $1M

Benefit Cost Ratio 1.02

Sensitivity analysis shows that the Internal Rate of Return (IRR) will fall between 8.1% and 13% as the confidence factors on operational savings mentioned above rise to 100%.

The project is on the borderline of acceptance with a minimum IRR of 8.1%, but when coupled with a sensitivity analysis showing the higher possible range above, and presented within a strategic needs framework, the full value of the project can be appreciated. Implementation risks have been minimised in this case by using proven and familiar technology.

7 CASE STUDIES

7.1 Water Quality - Chlorine Residuals

Four reservoirs on the south-western outskirts of Sydney were requiring regular manual re-chlorination to keep chlorine residuals to specified levels. Investigations using IICATS revealed that, using traditional operating procedures, the reservoirs were acting mainly as balancing tanks, with limited flow in and out. Alternative operating options were formulated and assessed, and some of these options were then trialed using the IICATS automatic remote control capability. In the past, multiple site visits would have been required to trial different control settings. The pumping regime has now been changed using a few keystrokes at a workstation to allow levels to move up and down to a greater extent. With the regular cycling produced, chlorine levels are now maintained eliminating the need for rechlorination. This saving came under the "unquantified" heading in the Business Case, but has saved over $300,000 per annum, and eliminated undesirable spikes in chlorine levels at the customer’s tap.

 

7.2 Incident Management

IICATS was used extensively during the recent cryptosporidium/giardia water quality incident in Sydney. From Sydney Water’s Systems Operation Centre, estimates of travelling time for contaminated water could be made, and time stamped information from IICATS was used extensively to help verify clearance of contamination. Operationally, reservoirs were isolated to prevent or contain contamination, and adjustments to control settings were made to ensure customers continued to receive full quantities of water, even though over 30 reservoirs were off-line at times.

During the water quality incident, a valve linking the 30 km long, twin 1800mm pipelines from Warragamba Dam to Prospect Water Filtration Plant closed suddenly during a maintenance activity. This caused "water hammer" along the full length of the pipelines, causing all anti-vacuum valves to activate, as designed, to protect structural integrity. This resulted in spectacular water spouts, but also meant that the normal Spring flow to the Filtration Plant of about 1400 megalitres per day was interrupted for over 13 hours, while the anti-vacuum valves were re-set. IICATS was utilised from the Systems Operation Centre to ration water throughout the distribution network, at a time when many reservoirs were isolated. In a convincing demonstration of the capability and value of IICATS, no customers went without their normal full quantity of water.

7.3 Dry Weather Sewage Overflow

The pumps at a Sewage Pumping Station failed due to a control problem. Due to a communications failure in the old telemetry system connected to the site, the pump failure alarm was not received. Furthermore, limitations in the old telemetry meant that the alarm was not re-transmitted when communications were restored, allowing a continuous overflow of sewage for several days before the problem was discovered on site. Following implementation of the new IICATS technology, such a situation will not occur, allowing Sydney Water to meet its diligence requirements under the current tough NSW environmental laws.

8 CONCLUSIONS

  • SCADA Projects have their best chance of success when evaluated in terms of all the strategic needs of the organisation, including future positioning. Minimum standards of performance should be established, taking into account all needs.
  • SCADA systems with the advanced overall functionality of a system like IICATS should be seen for their value as a powerful, integrated information system, linked to other Corporate systems such as the customer service, geographic information, business planning and maintenance systems.
  • SCADA systems should take full advantage of the opportunities provided by modern information and electronic technology to maximise potential benefits.
  • Modern SCADA systems can be an integral part of the success of a water and wastewater organisation. These systems bring with them an advanced capability and a flexibility which allows organisations to operate to the expectations of their customers, regulators, owners, and the community at large into the future, and to remain competitive.
  • All potential users/stakeholders should have extended input to decision making through processes such as Value Management Studies, as SCADA is an expensive tool which can yield great benefits if the right proposal is formulated and the full potential is harvested. These users include planners, operators, customer service staff, contractors, business strategists, and senior management. Decisions should not be made which limit the use of systems to a single group of users in the organisation.
  • Business Cases should be based on all the needs of the Corporation, from the tactical to the strategic, if they are to succeed. They need to take a broad and innovative approach to matching the needs to the potential of the available technology. The thinking that goes into the development of the Business Case should ensure that the SCADA System reflects the Business Needs.
  • Business Cases should be put together by users.
  • Benefit analyses should make an attempt to value unanticipated and unquantifiable benefits, otherwise opportunities to gain the most out of SCADA systems may be missed. A useful technique is to calculate minimum quantifiable benefits, and then carry out sensitivity analyses by allocating dollar values to various realisation scenarios for unquantifiable benefits.
  • Based on hard, tangible benefits, SCADA is often a marginal investment. It is often the strategic value that prompts water utilities to invest in SCADA.
  • SCADA users should be prepared to change their organisations and work practices to exploit opportunities for benefits.
  • The new IICATS technology has delivered all the quantifiable Business Case benefits forecast to-date. Moreover, considerable additional benefits have been realised. Of particular value have been some of the unquantifiable benefits - those that did not count in the financial evaluations - covering normal operations, information management and incident management.

And lastly, a few thoughts from a SCADA user about the future...

SCADA buyers in the future will be looking for systems that match what’s available in other computer-based systems. They will expect RTU’s and Telemetry Computers to behave for them like their PC; their communications network to keep up with advances in telecommunications and the Internet; their workstations to give seamless access to other computer applications; and their configuration to be done without programming. They will expect to be able to buy hardware and software from a range of suppliers, and for these items to be compatible and standardised. They will need to be able to upgrade parts of their systems in stages when demands or business drivers change.

 

Email David Hope

 

Home

davidhope & associates

http://www.davidhope.net