EFFECTIVE BRIDGE MAINTENANCE USING MULTIMEDIA AND MOBILE SYSTEMS

SIXTH INTERNATIONAL CONFERENCE ON STRUCTURAL FAULTS AND REPAIRS

LONDON, ENGLAND

JULY 1995

Sanjiv Nathwani, Trilon, Inc. (New Brunswick, NJ)

Avanti Shroff, Iffland Kavanagh Waterbury, PLLC (New York, NY)

ABSTRACT:

Bridges are not a modern concept, but they have proliferated only recently with the advent of railway and automobile transportation. They were built with unbridled enthusiasm as humanity poured into every nook and cranny of the world. Unfortunately, at the time, the engineering world could only speculate as to the long-term viability of these structures. And, no one could predict the impact of relentless use and natural wear and tear.

Today we are in danger of having our highway networks disconnected by defunct, deteriorated bridges. This could effectively choke off the very circulation that modern civilization has been built on. As engineers around the world are recognizing, the time to act is now. We must repair or replace the severely damaged structures before the cost is translated into human lives. More importantly, we must anticipate and prevent further deterioration by implementing strategic periodic inspection and maintenance programs.

It is a daunting task; each bridge needs to be examined and documented in detail. The information then needs to be analyzed to devise a strategy for repair, rehabilitation, and maintenance. The implementation of the strategy will itself reveal new information that must be quickly incorporated to fine-tune the process and keep it accurate. It is an information management burden of nightmarish proportions.

The challenge for the engineering world is to create a computing environment for the bridge community that will alleviate the information management burden and leave the engineers free to perform engineering functions. In the current environment, information management is the bottleneck that hinders response.

This paper examines the role that imaging, multimedia, and other state-of-the-art technologies can play in streamlining a bridge maintenance program, the manner in which they can be implemented effectively, the potential benefits, and the experiences of The Commonwealth of Massachusetts Highway Department (MHD) and Rhode Island Department of Transportation (RIDOT).

BACKGROUND:

The bridge management community is not without automation. The current impact of computer technology on this industry can be divided into three broad categories:

Database Management Systems (DBMS): These are primarily inventory systems that maintain lists of bridges and their various elements. They may include alphanumeric rating information, if the bridges are part of an inspection program. They are also being supplemented by expert systems such as the government-mandated Bridge Management Systems (BMS) in the United States. BMSs are decision-support tools that use inspection rating information to determine how funds can be allocated to repair and rehabilitation programs for maximum impact.

Design and analysis tools: These are largely used for performing complex engineering calculations, such as load rating analysis.

Computer Graphics: Computer Aided Design (CAD) has taken the engineering world by storm, and the bridge management community is no exception.

In some cases, imaging technology is being used to scan large-format drawings and plans to raster files. This process is sometimes combined with raster-to-vector conversion to integrate with CAD functions.

Computer automation has been implemented in a piecemeal manner. Each function that is automated is treated as an independent system, and may run on different hardware platforms and operating systems. In some cases, entire departments may be created around the newly automated function, such as CAD.

It is important to note that most of the inspection information is still maintained in physical files. This information is created largely in the field and may include such diverse documents as forms, handwritten comment sheets, sketches, maps, color photographs, full-motion video, contour plots, and non-destructive testing data.

ANALYSIS:

As a fundamental, we must understand that information access is prerequisite to the information processing that will result in an efficient plan of attack from the engineering perspective. Extrapolating, information flow is prerequisite to adequate access. Thus the requirement is for comprehensive systems that will speed the flow of information from point to point in the bridge management process and enhance access at all points.

Assuming that a bridge inspection program has already been implemented, the information flow for bridge management may be modeled as follows:

• Engineers collect information during field inspections. The field inspection process is largely a matter of assessing and documenting the condition of a structure based on certain guidelines. If a hazardous condition is discovered relating to either the safety of the traveling public or the load carrying capacity of the structure, the information may need to be communicated immediately to the managing office for urgent action. Otherwise, the information is simply compiled and collated for permanent record. The information generated will consist largely of forms detailing the conditions of the various elements, as well as the overall condition. The inspector may be required to translate the assessment of a condition into an alphanumeric rating and corroborate his or her assessment with photographic or video evidence. In addition, he or she may supplement the information with measurements of key elements and instrumentation data such as fathometer readings, strip charts, or infrared thermography studies. In some cases, diving inspections, documented on videotape, may be required to assess the condition of bridge elements below water.

• The inspection information is brought back to the managing office for permanent storage. Some of numerical information, such as condition ratings, may be entered into a DBMS application for comparative study.

• If the bridge requires immediate attention, the field inspection information will be used to create a repair detail to be acted on immediately. Otherwise, the information is reviewed at a later date to schedule periodic maintenance and/or preventative rehabilitation. Expert systems, such as BMS, may be utilized to create a cost-effective schedule.

• Each repair detail, scheduled maintenance, or rehabilitation results in additional information. The permanent file is amended to reflect the changes; the ratings database may also be updated. In certain cases, the original design of the structure and/or the load carrying capacity may have been altered. This will result in the production of new contract plans, updating the CAD files, and redoing the load rating analysis.

• Accidental damage to a bridge may require unscheduled inspection of the structure and possibly emergency repair. The resulting changes must be carried over to the permanent files including updates to the computer files.

The flow of information seems simple enough at first glance. However, when the flow is overlaid with the needs for access to the information, the situation is considerably more complex:

• In order to perform inspection effectively, inspectors must have access to information from previous scheduled inspections at the bridge site. This will allow them to rapidly examine previously noted problem areas for worsening conditions and determine the rate of deterioration, an important factor in assessing the potential for structural failure. To accomplish the above, the inspector must take the entire bridge history into the field for an extended time period. This will prevent other engineers from accessing the information at the managing office. If duplicate copies are maintained for this purpose, it may result in additional problems in trying to keep all copies up to date. On the other hand, there is no mechanism for the field engineer to quickly look up information stored on computers, such as element ratings and CAD files.

• In the case of emergency situations, information will need to be communicated back to the home office for a recommendation on how to proceed. The situation will generally require getting the field information back to the office as soon as possible and having authorized engineers come out to the field. Voice communication from the field is insufficient since the damage will often require visual verification by the decision makers.

• Once the field information is added to the main files at the managing office, it may often be required by several different divisions or departments at once. Unlike environments like insurance claims processing, the work flow for a bridge engineering office is not serial; many departments will probably process the core information simultaneously for different purposes.

• It is often the case that several engineers from different departments in different locations will be required to review the same information to make a joint decision regarding a structural condition. In emergency situations, it would be expedient if all individuals were to have access to the data simultaneously.

• Frequently, engineers will need to combine information from multiple sources to arrive at a determination. They may request an original plan from the archive or microfiche, while reviewing load rating analysis output and color photographs, to make a decision on how to design or repair a new element using CAD for optimal load-bearing capacity. This scenario often reduces to a serial process requiring the engineers to move between the various geographic locations of the different document types.

Besides the problems of information flow and access, there are several other problems with this method of managing information. Some of the problems are common to other paper-based filing systems:

• Paper documents are prone to accidental loss or damage and deterioration.

• They are easily misfiled and are difficult to cross-reference

• Paper files are bulky and difficult to transport.

Some of the problems are associated specifically with managing color photographs:

• The relatively low cost of film and processing can quickly become prohibitive when volume is taken into account.

• There is a time delay associated with film development that can be disastrous in emergency situations.

• The time delay also means that photographs cannot be verified for accuracy or quality before inspectors leave the bridge site.

Similarly, there are a host of problems dealing with non-standard data such as audio, video, fathometer surveys, and non-destructive testing data. In combination these problems conspire to distract engineers from the real task of maintaining and rehabilitating bridges.

IMAGING:

Imaging technology holds many of the answers to the problems listed above. The technology facilitates the conversion of paper files to electronic media by using scanners that capture digital images of the source documents. Once converted in this way, the image files can be made available across standard local and wide area networks, for immediate and simultaneous access at any properly equipped desktop workstation.

Black-and-white document imaging is very much a mainstream technology today. The combination of compression standards such as CCITT Group IV and the availability of high volume scanners and optical storage devices have made it practical to manage large image files. The cost of specialized equipment is insignificant versus the cost of keying in data to conventional alphanumeric systems.

Color imaging is similar in concept, but far more difficult to implement due to the need for 24-bit (true) color imaging. Subtle color variations are often the only characteristic which identifies severe underlying damage in photographs of deterioration. The loss of detail at lower bit depths, due to dithering or the remapping of 16.7 million possible colors to a reduced color palette, may obscure critical information. Thus, only 24-bit color scanning and viewing is recommended for this application. Files for 24-bit images are 24 times the size of uncompressed black-and-white (1-bit) image files of equal dimensions and resolution, resulting in proportionately higher storage requirements and network response times.

The recent release of the Joint Photographic Experts Group (JPEG) color image compression standard alleviates many of the problems associated with managing color images. Using the lossy algorithm, compression ratios ranging from 10:1 to 30:1 can be achieved without significant loss of detail, depending on the image composition. The resulting file sizes are significantly easier to manage.

In combination, black-and-white and color document imaging can facilitate the conversion of all paper-based and photographic field inspection documents to digital format. The potential impact on bridge information management could include the ability to:

• Provide immediate and simultaneous access to all paper and photographic records from centralized optical storage. This would allow engineers to parallel-process and make joint decisions while all viewing the same information at their desktop computers. In addition, they would no longer be required to move from location to location to view different paper files.

• Allow as many duplicate copies as necessary without compromising the integrity of the central repository of information. Copies of the images can be easily printed using laser printers and photorealistic color printers. This would allow field engineers to create fresh copies of the latest information when they go to the field without impacting access at the managing office. Loss or damage to the copies would be of no consequence, since they could be easily replaced with fresh copies. Further, the copies could be discarded after the updated information is scanned to the system.

• Provide sophisticated cross-indexed access to the documents. Since the files will be accessed using a database query mechanism, users will have the convenience and flexibility of DBMS technology to index the documents. This should also help reduce misfiling of documents.

Despite the obvious benefits that imaging technology brings to the problem at hand, it is by no means a complete solution. There is still no way of dealing with non-standard data, and it does nothing to unite various computing environments.

MULTIMEDIA:

The term multimedia is often used to refer to the use of full-motion video and sound in computers. For the purpose of this paper, the term will be used in its broadest sense: a computer environment where all information, regardless of its source, type, and characteristics can be dealt with simply as data. Thus, the concept encompasses full-motion video and sound data, as well as image data and other computer files. This level of abstraction is necessary to define an application that is capable of managing all the information types associated with bridge management.

The ability to digitize both sound and video information is quite common today. The digitized information is stored as a file that can be manipulated just like any other computer files. Specialized software is required to play back the sound and video files, but this is no different from the need for specific software to open a word processing file or spreadsheet.

What is different under the multimedia information system concept is the ability to treat all file types as objects that can be stored to the same database, and indexed for access through the same query interface. The controlling application will decide which software is required for presenting the retrieved object and automatically open the retrieved file in the appropriate application. The simplicity of such an interface would relieve engineers from the burden of remembering the vagaries of each file type, and of understanding the complexities of storage on different media.

Bridge information management is impacted in the following ways:

• Engineers can now retrieve information in a sequence dictated by the task at hand, not by some artificial restriction based on the distribution of various types of data. If the task requires the engineer to look at a paper form followed by a CAD drawing followed by plans, he can do just that instead of reviewing the paper form with the plans first since they happened to reside in the same location.

• Taking advantage of a windowing environment, the engineer can open several different files simultaneously on the screen and move quickly between them by the simple click of a mouse. Thus, a user can immediately go from viewing a video clip to a color photograph and word processing document. No longer are engineers required to move back and forth between a room outfitted with a VCR, the filing room for photographs, and a room with a computer equipped with word processing software.

• Non-standard data such as fathometer surveys can be plotted using any one of many graphing tools to provide a visual interpretation. The resulting three-dimensional contour map can be viewed concurrently with an underwater diving inspection video clip of the same structure. This can enhance understanding both of non-standard data and the various elements of the diving video.

• Video and sound data can literally recreate the field events for management decisions at the office. This could greatly reduce the need to verify inspection information at the bridge site.

MOBILE COMPUTING:

Mobile systems are the final piece to the puzzle of total bridge information management. Their greatest impact is obviously on field inspections. The pertinent technologies are:

Notebook computers: These devices can be used to carry required information to the field electronically, instead of relying on bulky paper files. The information would simply be 'checked out' from the central database. The same multimedia application that is used at the managing office can be used for retrieving information in the field on notebook computers.

Electronic still cameras: These devices can record full color photographs directly onto electronic media, bypassing the entire step of film development and scanning. These cameras typically offer lower resolution than fine grained film, but are more than adequate for the needs of inspection photography. They are still fairly expensive compared with conventional cameras, but the prices are dropping steadily. They completely solve the three major problems with film based photography: There are no development costs, and the storage media is typically erasable and thus infinitely reusable; there is no time delay for development - the photographs can be immediately verified in the field and entered into the central database.

Personal Digital Assistants (PDAs): This class of devices has just entered the mainstream market. Their main attraction is their low cost (relative to portable computers), miniature size and ease of use. They typically support pen based input, high quality writing recognition, and are easily adapted for forms-based electronic entry. These features mean that the field engineer need only carry a light, easy-to-use device while inspecting a bridge. Since the data is already in electronic format, there is no need for compilation and collation.

Video cameras: The new class of video recorders are small, handheld items that are easy to carry and use. It is conceivable that an entire field inspection for a structure could be performed using nothing but narrated video footage. This way, the engineer could quickly capture the necessary details for analysis later at the office. The pan and zoom functions are also important, since they mean that context and visual details can be discerned from a significant distance.

Wireless networking: This technology is likely to revolutionize handling of emergency structural conditions. As greater bandwidth becomes available, it should be possible to transfer complete multimedia files between bridge sites and the managing office; mobile computer-based videoconferencing also becomes a viable, and extremely useful, option. This way, decisions to close damaged structures to traffic can be made immediately without waiting for authorized engineers to travel to the bridge site for visual verification. Similarly, field engineers could be empowered to act remotely on information from the central database to prepare an emergency repair detail while still at the bridge site, and have it instantly communicated to the repair department.

Mobile computing combined with an appropriately designed multimedia information system can completely fulfill the goal of bringing the field to the office and the office to the field, thus blurring the boundaries between the two.

CASE HISTORY - RHODE ISLAND DEPARTMENT OF TRANSPORTATION (RIDOT):

RIDOT started their search for a multimedia bridge information management system in September 1992. They issued a Request For Proposal (RFP) in March, 1993. The RFP included the following requirements:

• The system must provide an electronic central repository of all bridge related data on optical media for 1,200 bridges.

• The system must be able to provide virtually instantaneous access to all bridge information to a networked computer workstation.

• The system must be able to provide simultaneous access to the bridge data to anyone connected to the network via a computer workstation.

• The system must be capable of storing all types of bridge information including paper forms, hand written comment sheets, color photographs, plans, still video frames and full motion video clips.

• The system must be capable of capturing, displaying and outputting the various data types. It must allow paper forms to be captured using black-and-white scanners, photographs to be captured using color scanners and electronic still video cameras, videos to be captured using standard video camcorders, computer files to be captured directly from the native computer environment. It must allow display of the various data types simultaneously on the desktop of a single workstation. It must allow printing of black-and-white images to high speed bitmap printers, and color images to photorealistic color printers.

• The system must allow information to be 'checked out' from the central repository to portable computers, and to provide for rapid review of the information while in the field.

• The system must be capable of being expanded to store virtually unlimited amounts of data, and support virtually unlimited numbers of networked users.

The project was awarded in June, 1993 and the system was put into production in October 1993. Today, RIDOT has over 70% of all bridge information on-line on the system, with 100% on-line expected by the end of this year. In addition, they have adapted the system for storage of signage information, since the requirements were very similar to those for bridge information. They expect to continue expansion of the system to include all non-standard data handled by RIDOT.

Among the many benefits, RIDOT has noticed that the system has been extremely useful in lowering management and administrative overhead - people that need information on a specific structure have to do nothing more than log on to the system from a centralized workstation, regardless of what department they are in. RIDOT expects to see a complete return on investment within two (2) years of the system being in production.

CASE HISTORY - MASSACHUSETTS HIGHWAY DEPARTMENT (MHD):

MHD started their search for a multimedia bridge information management system in October 1992. Their goal was to find tools that would solve their information management problems, integrate well with their BMS (Pontis) and National Bridge Inventory (NBI) system, and provide an easy transition to automated procedures for 6,000 bridges.

MHD placed a Request For Information (RFI) advertisement in the Engineering News Review (ENR) magazine in order to find a vendor/consultant that would be able to provide a turnkey solution. The project was awarded in June 1994.

The scope of the project was to specify, implement, test, train and maintain a complete multimedia bridge information management system to cover the central office and the five (5) geographically dispersed district offices. Due to the complexity of the project, MHD opted for a phased implementation approach. Each phase was designed to provide an important piece of the overall functionality, result in manageable changes to procedures, and show a clear return on investment in terms of productivity and cost savings.

Phase 1: Started in June 1994 and completed in October 1994. The goal of this phase was to provide complete functionality to the central office in Boston and limited functionality to the district offices. The Boston office would have the ability to capture, store, retrieve and output information from a networked system. In this phase there would be no network connection with the district offices, thus archival information from the district offices would be received monthly on optical disks. The districts would also be restricted to reviewing information that was sent to them monthly. However, the inspection teams assigned to each district would begin using video cameras for still photography and full-motion video, and portable computers to enter NBI information and comments.

Phase 2: Due to begin in July 1995. The goal of this phase was to complete backfiling of all existing information to the system. This would bring the complete history of all bridges in MHD's jurisdiction on line, and provide an electronic system of record.

Phase 3: Due to begin in June, 1995. The goal of this phase was to connect the district offices to the central office via a high speed Wide Area Network (WAN) and provide complete functionality for the district offices. The district offices would have complete on-line access to the central repository. This would allow them to archive information and review information directly, without involving the Boston office. This will dramatically reduce the management burden at the Boston office. In addition, the expanded network would be extremely beneficial for response to emergency situations.

Phase 4: Due to begin in June 1995. The goal of this phase was to provide PDAs to all field inspection teams for the electronic capture of bridge condition rating data. In combination with electronic photography, this would allow MHD to perform completely electronic field inspections.

Among the many benefits observed in Phase 1, it was noted that department was saving an estimated $80,000 in film and development costs annually since the inspection teams switched over to electronic still photography. They have also found that the use of full-motion video in bridge inspections has greatly enhanced their handling of emergency conditions.

CONCLUSION:

It is fair to conclude that technology is the key to effective bridge information management, and that bridge information management is the key to an effective bridge maintenance program. We have become quite adept at collecting information on the condition of our bridges. We must now find the tools that help us make sense of all this collected information, so that we can make better decisions and engineer better strategies for bridge management. With technology, we can succeed in halting, and even reversing the accelerating deterioration of the world's bridges. Our efficiency is measured in the ultimate currency: human lives.

TRILON, INC. 528 Winfield Avenue Upper Darby, PA 19082-2122 (215) 790-6265 voice (215) 790-6231 fax Info@Trilon.com http://www.Trilon.com