Published as: Klapan Ivica, et all., Ear Nose Throat J, 2006, 85(5):318-321.
Medical Institutions:
University Polyclinic Klapan Medical Group, Zagreb and University Department of ENT, Head & Neck Surgery, Division of Plastic and Reconstructive Head & Neck Surgery and Rhinosinusology, Zagreb University School of Medicine and Zagreb University Hospital Center, Zagreb, Croatia
Reference Center for Computer Aided Surgery and Telesurgery, Ministry of Health, Republic of Croatia, Zagreb, Croatia
University Department of ENT, Head & Neck Surgery, Osijek University School of Medicine and Osijek University Hospital, Osijek, Croatia
Physics Department, University of Zagreb, Zagreb, Croatia
Makarska Medical Center, Division of ENT, Makarska, Croatia
Division of Surgery, Našice General Hospital, Croatia
Address for correspondence:
Ivica Klapan, M.D., Ph.D., University Polyclinic Klapan Medical Group, Ilica 191A, HR-10000 Zagreb, Croatia (poliklinika.klapan@vodate.net)
Key words: Computer assisted surgery, Telesurgery, Three-dimensional visualization, Endoscopy, Telemedicine
This work was in part supported by an unrestricted grant by the Ministry of Science and Technology, Republic of Croatia, No. 5-01-543 (Dr. I. Klapan)
INTRODUCTION
Extremely valuable information on anatomic relationships in particular regions while planning and performing endoscopic surgery is provided by high quality computer tomography (CT) diagnosis1 (Fig. 1), thus contributing greatly to the safety of endoscopic surgery2. However, in spite of a number of advantages, this specific visualization of head anatomy still suffers from some drawbacks. For example, it may occasionally be impossible to precisely localize the tip of the endoscope or some other instrument in the area relative to the target site shown on CT image. This requires maximal concentration from the surgeon, expecting him to rely on his own experience or even intuition to enable both real and visual progression of the procedure. Thus, the need has emerged of developing a new approach in visualizing patient's head before, during and after the operative procedure.
Fig. 1. CT of the nose and paranasal sinuses
The basic requirement resulting from the above mentioned needs refers to the use of a computer system for visualization of anatomic 3D-structures and integral operative field to be operated on. Such an approach will offer the surgeon a considerably better insight in the operative field and thus improve the safety of the procedure3.
The mode of computer visualization of anatomic structures of the human body used to date could only provide diagnostic information and possibly assist in the preoperative preparation. Intraoperative use of computer generated operative field 3D-model has not been widely adopted to date. The intraoperative use of computer in real time requires development of appropriate hardware and software to connect medical instrumentarium with the computer, and to operate the computer by thus connected instrumentarium and sophisticated multimedia interfaces.
HIGH QUALITY DIAGNOSIS (DICOM STANDARD)
High quality diagnostic image is the main prerequisite for appropriate utilization of computer systems during the preparation, performance and analysis of an operative procedure.
Development of a system for data exchange between multiple medical diagnostic devices as well as between diagnostic devices and computer networks has led to the establishment of DICOM standards describing the forms and modes of data exchange (DICOM = Digital Imaging and Communication in Medicine).
Before the introduction of DICOM standards, image recordings were stored on films, where the information obtained from the diagnostic device was in part lost. In ideal conditions, sixteen different image levels could be distinguished on films at the most. When film images were to be stored in computer systems, films had to be scanned, thus inevitably losing a part of significant data and probably introducing some unwanted artifacts. The level setting and window width to be observed on the images could not be subsequently changed. Visualization of the image on the diagnostic device monitor was of a considerably higher quality, thus it was quite naturally used for record receipt and storage in computer media. Video image allows for the receipt of 256 different levels at the most. Neither it is possible to subsequently modify the level setting and window width to be observed on the images that have already been stored in the computer system.
When stored in computer systems by use of DICOM protocol, images are stored in the form generated by the diagnostic device detector. These image recordings can then be properly explored by use of powerful computer systems. This is of special relevance when data in the form of images are to be used for complex examinations and testing, or in preoperative preparation where rapid and precise demarcation between the disease involved and intact tissue is required. It is also very important for the images to be visualized in various forms and from different aspects and then – which is most demanding indeed – to develop spatial models to aid the surgeon in preparing and performing the procedure as well as in postoperative analysis of the course of the procedure.
The entire operative procedure can be simulated and critical areas avoided during the real procedure by employing real patient images in the operation preparatory phase using complex spatial models and simulated operative field entry (Virtual Endoscopy, Virtual Surgery)4,5.
PREOPERATIVE PREPARATION
Use of the latest program systems enables development of 3D spatial models, exploration in various projections, simultaneous presentation of multiple model sections and, most important, model development according to open computer standards (Open Inventor). Such a preoperative preparation can be applied in a variety of program systems that can be transmitted to distant collaborating radiologic and surgical work sites for preoperative consultation as well as during the operative procedure in real time6 (telesurgery, tele-FESS) (Fig. 2).
Fig. 2. Our 3D models of the human head in different projections
Advanced technologies of exploring 3D spatial models allow for simulation of endoscopic surgery and planning the course of the future procedure (Virtual Endoscopy) or telesurgery (Tele-Virtual Endoscopy). By entering the models and navigating through the operable regions the surgeon becomes aware of the problems he will encounter during the real operation. In this way, preparation for the operation could be done including identification of the shortest and safest mode for the real operation to perform6,7 (Fig. 3).
Fig. 3. An example of 3D computer-assisted microsurgery of the nose and paranasal sinuses (3D-C-FESS) with simulation and planning of the course of subsequent endoscopic operation (Virtual Endoscopy)
During the course of our Three-Dimensional Computer Assisted Functional Endoscopic Sinus Surgery (3D-C-FESS) method development, a variety of program systems were employed to design an operative field model by use of spatial volume rendering techniques (www.mef.hr/3D-CFESS). Initially, the modeling was done by use of the VolVis, Volpack/Vprender, GL Ware programs on a DEC Station 3100 computer. With the advent of 3D Viewnix V1.0 software, we started using this program, and then 3D Viewnix V1.1 system, AnalyzeAVW system, T-Vox system and OmniPro 2 system on Silicon Graphics O2, Origin200 and Origin2000 computers (Fig. 4).
Fig. 4. 3D Viewnix V1.0 and AnalyzeAVW (OmniPro 2 is shown in Fig. 3)
COMPUTER ASSISTED SURGERY
The use of computer during a surgery/telesurgery requires highly reliable, stable and fast computer systems. Computer work stations with UNIX compatible operative systems are most commonly used. During the procedure, the surgeon is engaged in performing the surgery, so he cannot operate the computer. Therefore, the presence of a computer system expert is necessary in the operative theater on performing computer aided operative procedures.
During the procedure, the surgeon can operate the computer system by his voice (Voice Navigation). Model movements on the monitor, various projections and sections can be obtained by simple and short voice instructions during the surgery.
On initial computer aided operative procedures, spatial orientation within the operative field of a 3D computer model and transfer of the particular point to the real operative field of the patient were performed by arbitrary approximation of the known reference points of the operative field anatomy8. In this way, the given entities were recognized on the model and in the real operative field9.
The use of 3D spatial model of the operative field during the surgery has pointed to the need of positioning the tip of the instrument (endoscope, forceps, etc.) within the computer model. The major problem is transmission of the real patient operative field co-ordinate system to the co-ordinate system of the computer 3D spatial model of the same patient, which has been previously designed from a series of CT images during preoperative preparation10 (Fig. 5).
Fig. 5. An example of our 3D computer assisted surgery
Several modes of instrument localization within the operative field are used, i.e. electromagnetic, optic and mechanical methods. Electromagnetic method is very sensitive to environmental electromagnetic fields (electrical devices, lighting) and large amounts of metal (cabinets, table, instruments), and the basic, ideal precision of localization within the field is inadequate for surgery performance. Optic locators have proved suitable but are relatively expensive and less precise than the mechanical ones. Mechanical locators are virtually 3D digitalizers sending their shifts within six degrees of freedom to the computer, which then converts them to shifts within the co-ordinate system of the operative field 3D model.
The main problem and shortcoming of the current mechanical locators is the inability to reach deep regions within the operative field. The issue could be solved by replacing current tips by thinner and longer endings, or even better by the original surgical instrument (e.g., forceps or endoscope) (www.mef.hr/MODERNRHINOLOGY). The endoscope is mounted at the end of the 3D digitalizer instead of the existing ending or outside the existing ending axis. The depth of the reachable entity is identical to the depth attainable by the standard endoscope or pump (Fig. 6).
Fig. 6. The main problem encountered in 3D-CA-surgery is how to transmit the real patient operative field co-ordinate system to the co-ordinate system of the computer 3D spatial model of the same patient, previously developed from a series of CT images during preoperative preparation
Using a special digitalizer (endoscope simulation) model and computer model, the preoperative preparation and simulation of the entire procedure can be done on the computer model of the real patient. Employing 3D digitalizer on the real procedure, the tip of the instrument (simulated endoscope) can be precisely identified in the real operative field and visualized on the computer model5,11. The freedom of endoscope manipulation during the procedure is not reduced because the connection is realized at the sites of instrument handle and endocamera link.
COMPUTER ASSISTED TELESURGERY
Computer technologies allow for computer assisted surgery to be performed at distance. The basic form of telesurgery can be realized by using audio and video consultations during the procedure.
Sophisticated endoscopic cameras show the operative field on the monitor mounted in the operating theater, however, the image can also be transmitted to a remote location by use of video transmission. The latest computer technology enables receipt of CT images from a remote location, examination of these images, development of 3D spatial models, and transfer of thus created models back to the remote location12. All these can be done nearly within real time. These procedures also imply preoperative consultation. During the surgery, those in the operating theater and remote consultants follow on the patient computer model the procedure images, the 'live' video image generated by the endoscopic camera, and instrument movements made by the remote surgeon6. Simultaneous movement of the 3D spatial model on the computers connected to the system providing consultation is enabled6,12. It should be noted that in most cases, intraoperative consultation can be realized from two or more locations, thus utmost care should be exercised to establish proper network among them.
The extreme usage of computer networks and telesurgery implies the use of robot technologies operated by remote control. In such a way, complicated operative procedures could be carried out from distant locations. The main idea considering the use of computer networks in medicine is: IT IS PREFERABLE TO MOVE THE DATA RATHER THAN THE PATIENT (Fig. 7).
Fig. 7. An example of our Tele-3D-computer assisted surgery of
the nose and paranasal sinuses
POSTOPERATIVE ANALYSIS
The use of computer technology during preoperative preparation and surgery performance allows for all relevant patient data to store during the treatment. CT images, results of other tests and examinations, computer images, 3D spatial models, and both computer and video records of the course of operation and teleoperation are stored in the computer and in CD-R devices for subsequent analysis7 (www.mef.hr/MODERNRHINOLOGY). Also, these are highly useful in education on and practice of different approaches in surgery for surgery residents as well as for specialists in various surgical subspecialties.
In this way, the real surgery and telesurgery procedures can be subsequently analyzed and possible shortcomings defined in order to further improve operative treatment. The use of latest computer technologies enables connection between the computer 3D spatial model of the surgical field and video recording of the course of surgery to observe all critical points during the procedure, with the ultimate goal to improve future procedures and to develop such an expert system that will enable computer assisted surgery and telesurgery with due account of all the experience acquired on previous procedures. Also, using the computer recorded co-ordinate shifts of 3D digitalizer during the telesurgery procedure, an animated image of the course of surgery can be created in the form of navigation, i.e. the real patient operative field fly-through, as it was done from the very begining (from 1998) in our telesurgeries.
COMPUTER NETWORKS
Following the application of computers in surgery and connecting diagnostic devices with computer networks by use of DICOM protocol, the next step is directed toward connecting these local computer networks with broad range networks, i.e. within a clinical center, city, country, or countries. The establishment of complex computer networks of diagnostic systems across the country offers another significant application of computer networks in medicine, i.e. telemedicine (distant medical consultation in the diagnosis and treatment). Current computer networks using ATM technology allow for very fast and simultaneous communication among a number of physicians for joint diagnostic or therapeutic consultation. Textual, image, audio and video communication as well as exchange of operative field spatial models are thus enabled. Patient images and 3D spatial models can be simultaneously examined by a number of phyisicans, who then can outline and describe image segments by use of textual messages, indicator devices, sound or live image. The course and conclusions of such a consultation can be stored in computer systems and subsequently explored, used or forwarded to other users of the computer assisted diagnostic system.
The use of computer networks in medicine allows for high quality emergency interventions and consultations requested from remote and less equipped medical centers in order to achieve the best possible diagnosis and treatment (e.g., surgery). In addition to this, through consultation with a surgeon, a physician in a remote diagnostic center can perform appropriate imaging of a given anatomic region, which is of utmost importance for subsequent operation to be carried out by the consultant surgeon from the remote hospital center.
SYSTEM IMPLEMENTATION
In 1992, a scientific research rhinosurgical team was organized at the University Department of ENT, Head & Neck Surgery, Zagreb University School of Medicine and Zagreb University Hospital Center in Zagreb, who have developed the idea of a novel approach in head surgery. This computer aided functional endoscopic sinus microsurgery has been named 3D-C-FESS. The first 3D-C-FESS operation in Croatia was carried out at the Šalata University Department of ENT, Head & Neck Surgery in May 1994.
when a 12-year-old child, was inflicted a gunshot wound in the region of the left eye. Status: gunshot wound of the left orbit, injury to the lower eyelid and conjunctiva of the left eye bulb. Massive subretinal, retinal and preretinal hemorrhage. The vitreous diffusely blurred with blood. The child was blinded on the injured eye. Six years after the 3D C-FESS surgery, the status of the left eye was completely normal, as well as the vision, which was normal bilaterally.
With due understanding and support from the University Department of ENT, Head & Neck Surgery, Zagreb University Hospital Center; Merkur University Hospital; HT Company; InfoNET; and SiliconMaster, in May 1996 the scientific research rhinosurgical team from the Šalata University Department of ENT, Head & Neck Surgery organized and successfully conducted the first distant radiologic-surgical consultation (teleradiology) within the frame of the 3D-C-FESS project. The consultation was performed before the operative procedure between two distant clinical work posts in Zagreb (Šalata University Department of ENT, Head & Neck Surgery and Merkur University Hospital) (outline/network topology).
In 1998, and on several occasions thereafter, the team conducted a number of first tele-3D-computer asssisted operations (Fig. 8) as unique procedures of the type not only in Croatia but worldwide6,12 (www.mef.hr/MODERNRHINOLOGY).
Fig. 8. An example of our Tele-3D-C-FESS surgery initially performed in 1998
REFERENCES
1. Mladina R, Hat J, Klapan I, Heinzel B. An endoscopic approach to metallic foreign bodies of the nose and paranasal sinuses. Am J Otolaryngol 16(4):276-279, 1995.
2. Rišavi R, Klapan I, Handžić-Ćuk J, Barčan T. Our experience with FESS in children. Int J Pediatric Otolaryngol 43:271-275, 1998.
3. Elolf E, Tatagiba M, Samii M. 3D-computer tomographic reconstruction: planning tool for surgery of skull base pathologies. Comput Aided Surg 3:89-94, 1998.
4. Holtel MR, Burgess LP, Jones SB. Virtual reality and technologic solutions in otolaryngology. Otolaryngol Head Neck Surg 121:181, 1999.
5. Klapan I, Šimičić Lj, Rišavi R, Bešenski N, Bumber Ž, Stiglmajer N, Janjanin S. Dynamic 3D computer-assisted reconstruction of metallic retrobulbar foreign body for diagnostic and surgical purposes. Case report: orbital injury with ethmoid bone involvement. Orbit 20:35-49, 2001.
6. Klapan I, Šimičić Lj, RišaviR, Pasari K, SrukV, Schwarz D,BarišićJ. Real time transfer of live video images in parallel with three-dimensional modeling of the surgical field in computer-assisted telesurgery. J Telemed Telecare 8:125-130, 2002.
7. Klapan I, Šimičić Lj, Bešenski N Bumber Ž, Janjanin S, Rišavi R, Mladina R. Application of 3D-computer assisted techniques to sinonasal pathology. Case report: war wounds of paranasal sinuses with metallic foreign bodies. Am J Otolaryngol 23:27-34, 2002.
8. Klimek L, Mosges M, Schlondorff G, Mann W. Development of computer-aided surgery for otorhinolaryngology. Comput Aided Surg 3:194-201, 1998.
9. Mann W, Klimek L. Indications for computer-assisted surgery in otorhinolaryngology. Comput Aided Surg 3:202-204, 1998.
10. Anon J. Computer-aided endoscopic sinus surgery. Laryngoscope 108:949-961, 1998.
11. Olson G, Citardi M. Image-guided functional endoscopic sinus surgery. Otolaryngol Head Neck Surg 121:187, 1999.
12. Klapan I, Šimičić Lj,Rišavi R, Bešenski N, Pasarić K,Gortan D, Janjanin S, Pavić D, Vranješ Ž. Tele-3D computer assisted functional endoscopic sinus surgery: new dimension in the surgery of the nose and paranasal sinuses. Otolaryngol Head Neck Surg 127:549-557, 2002.
