Published as: Ivica Klapan, et. all., Otolaryngology Head Neck Surg, 2002, 127:549-557.
Medical Institutions:
University Polyclinic Klapan Medical Group, Zagreb and Department of Otorhinolaryngology and Head & Neck Surgery, Division of Plastic and Reconstructive Head & Neck Surgery and Rhinosinusology, Zagreb University School of Medicine and Zagreb Clinical Hospital Center, Zagreb, Croatia
Department of Radiology, Zagreb Clinical Hospital Center, Zagreb, Croatia
University Polyclinic Klapan Medical Group, Zagreb and Department of Otorhinolaryngology and Head & Neck Surgery, Division of Plastic and Reconstructive Head & Neck Surgery and Rhinosinusology, Zagreb University School of Medicine and Zagreb Clinical Hospital Center, Zagreb, Croatia
Department of Radiology, Zagreb Clinical Hospital Center, Zagreb, Croatia
Physics Department University of Zagreb, Croatia
Croatian Society for Telemedicine, Croatian Medical Association, Zagreb, Croatia
Reference Center for Computer Aided Surgery and Telesurgery, Croatian Ministry of Health, Zagreb, Croatia
Croatian Telecom, Zagreb, Croatia
Address for correspondence: Ivica Klapan, M.D., Ph.D., University Polyclinic Klapan Medical Group, HR-10000 Zagreb, Croatia (poliklinika.klapan@vodatel.com)
Keywords: Telesurgery, FESS, Computer Assisted Surgery, Three-dimensional visualization
Tele-3D-C FESS project supported by an unrestricted grant provided by the Croatian Ministry of Science, No. 3-01-543/96-2001 (Dr. Klapan)
Abbreviations:
FESS- Functional endoscopic sinus surgery
3D C-FESS -Three dimensional computer assisted functional endoscopic sinus surgery
OR - Operation room
Tele-FESS - Tele-functional endoscopic sinus surgery
Tele-3D CAS- Tele three dimensional computer assisted surgery
Tele-3D C-FESS - Tele-three dimensional computer assisted functional endoscopic sinus surgery
Tele VE- Tele virtual endoscopy
VE - Virtual endoscopy
VS - Virtual surgery
2D/3D - Two dimensional/Three dimensional
CT - Computed thomography
LCD/TFT - Liquid Crystal Display/Thin Film Transistor
PC - Personal computer
OMC - Ostio meatal complex
1st TS - First telesurgery
SGI O2 - Silicon Graphics O2
ATM OC-3 - Asynchronous Transfer Mode
LANE - ATM LAN Emulation (LANE)
TCP/IP - Transmission control protocol/Internet protocol
MPEG - Moving Pictures Experts Group
DVD-ROM - Digital Video Disc or Digital Versatile Disc
CD-ROM - Compact Disk
AAL-5 - ATM Adaptation Layer Type 5
T1 - A T1 line is a high-speed digital connection
ABSTRACT
One of the main objectives of our Three dimensional computer assisted functional endoscopic sinus surgery was to design a computer assisted 3D-approach in presurgical planning, intraoperative guidance and postoperative analysis of anatomical regions of the nose and paranasal sinuses. Such an extremely powerful approach should allow a better insight into the operating field including significantly greater safety of the procedure itself.
The last and very natural step of technology implementation in the operating room was to connect the computer workstations and video equipment to remote locations using a high speed & wide bandwidth computer network. During patient preparation, the surgeon in the operating room consults remote, experienced and skilful surgeons using computed thomography images and three-dimensional models on computer workstations. The surgeon and consultants use software for computed thomography image previews and three-dimensional model manipulations on top of collaboration tools to define the pathology, to produce an optimal path to the pathology and to decide how to perform the real surgical procedure. Using tele-fly-through or tele-virtual endoscopy via three-dimensional models, both surgeons can preview all the characteristics of the region (anatomy; pathology) and so predict and determine the next steps of the operation. This ensures higher safety of the operation guidance and reduces the possibility or intraoperative error. The duration of the tele-consultation is thus shortened, which may prove the greatest benefit from Tele-three-dimensional computer assisted surgery. By this way clinical institutions would spend less money for time fees for telesurgical consultation.
INTRODUCTION
Endoscopy, as a surgical technique, is based on the implementation of very complicated technology. FESS, as an endoscopic operation, is performed with the use of "live image" viaan endoscope, endomicrocamera, cold light sources and video equipment. We have also added a high performance computer, surgical field videocamera and spatial localizer with mounted surgical instrument1 (Figure 1).
Endoscopy, as a surgical technique, is based on the implementation of very complicated technology. FESS, as an endoscopic operation, is performed with the use of "live image" viaan endoscope, endomicrocamera, cold light sources and video equipment. We have also added a high performance computer, surgical field videocamera and spatial localizer with mounted surgical instrument1 (Figure 1).
Figure 1: Our 3D C-FESS equipment in operating room
At present, every physician (scientist and clinician) using computer for diagnostic (e.g., radiology) and therapeutic (e.g., surgery) purposes should know that images are processed by use of graphic and computer systems as well as by specialized program systems, in order to better present the anatomy of a particular part of the body with identified diseased areas2,3.
Beside otorhinolaryngology4 , this has also been used in other fields5. The more so, in addition to educational applications, VS has offered us the possibility of preoperative planning in sinus surgery, and has become a very important segment in surgical training and planning of each individual surgical intervention. In the activities of our 3D C-FESS team, these analyses are becoming routine procedures, as well as in other ENT centers elsewhere, for oral, maxillofacial and plastic surgery.
The basic goal of our 3D C-FESS6 (http://salata.mef.hr/3D-CFESS) and Tele-FESS (http://salata.mef.hr/Tele-FESS) with 3D-computer assisted support (Tele 3D C-FESS)7 is to achieve safer surgical procedure using new computer and medical technologies in telesurgical consultation, and provide visualization of the anatomy as well the pathology in the 2D-black and white form as well as in the form of 3D-models. So, using our own approach in C-FESS, we were able to "look inside" the patient during the real surgical procedure.
According to our original idea, the computer network, essential for computer collaboration between telesurgical sites, has to be built in parallel to the video network8. Every telesurgical site must have compliant collaboration software. On computer workstations, all sites have CT images and 3D models with movies. The consultant, an experienced surgeon, assists the less experienced surgeon to reach the pathology in the operating field.
This kind of our Tele-3D-CAS has to enable less experienced surgeons to perform critical surgeries using guidance and assistance from a remote, experienced surgeon. In telesurgery, more than two locations can be involved; thus less experienced surgeon can be assisted by one, two or more experienced surgeons, depending on the complexity of the surgical procedure. Our Tele-3D-CAS provides also the transfer of computer data (images, 3D-models) in real time during the surgery and, in parallel, of the encoded live video signals. Through this network, the two encoded live video signals from the endocamera and OR camera have to be transferred to the remote locations involved in the telesurgery/consultation procedure9.
MATHERIAL AND METHODS
1. Case report
A 38-year-old male with chronic sinusitis. Computed tomography of the sinuses in coronal and axial sections shows disease of the ethmoidal infundibulum bilaterally with homogenous opacification in the region of the anterior ethmoids. OMC bilaterally, with suspected spread of the disease to the frontal recess. A huge middle turbinate on the left. A minor retention cyst observed in the region of the alveolar recess of the right maxillary sinus, with signs of mild inflammatory changes in the region of both maxillary sinuses. Retention in the region of the anterior and posterior ethmoidal cells. Frontal and sphenoidal sinuses of normal transparency. Intact bony borders of the orbits. The patient reported headaches in the fronto-ethmoidal region bilaterally, recurrent sinusitis with postnasal dripping (Figure 2).
A 38-year-old male with chronic sinusitis. Computed tomography of the sinuses in coronal and axial sections shows disease of the ethmoidal infundibulum bilaterally with homogenous opacification in the region of the anterior ethmoids. OMC bilaterally, with suspected spread of the disease to the frontal recess. A huge middle turbinate on the left. A minor retention cyst observed in the region of the alveolar recess of the right maxillary sinus, with signs of mild inflammatory changes in the region of both maxillary sinuses. Retention in the region of the anterior and posterior ethmoidal cells. Frontal and sphenoidal sinuses of normal transparency. Intact bony borders of the orbits. The patient reported headaches in the fronto-ethmoidal region bilaterally, recurrent sinusitis with postnasal dripping (Figure 2).
Figure 2. CT-slices of the first case report
2. Video technologies
During telesurgical transmission, two video signals have to be transferred from the OR site and one video signal from every remote site involved in the telesurgery procedure. As about 24 Mb/s of bandwidth are needed for the native video signal, and there are only 155 Mb/s or multiple 2Mb/s lines of bandwidth, the video signals must be compressed using standard video compression systems10.
At each of the four locations involved in the telesurgical procedure11,12, there was a remotely controlled video switch with 8 video inputs and 8 video outputs. At the expert location, remote from the OR, there was a video processor for the acquisition of all video signals from all sites involved in the telesurgery procedure and software for the remote control of all video inputs/outputs and pan/tilt/zoom cameras of all locations. Thus, from this point in the telesurgery network, a consultant or conference moderator13 can view all the video signals or just the primary display. For all these possibilities, a bandwidth of at least 155Mb/s (ATM OC-3) is needed.
3. Network technologies
ATM switches and AAL-5 were used for video transmission and native or LANE for TCP/IP computer communications.
4. Collaboration
InPerson teleconferencing software and the native TCP/IP network was used for communication between all sites. Consultations using computer images and 3D-models (Figure 3) were performed using the video network; outputs from the computer were encoded into video stream and transmitted to the remote locations through video communication protocols. The advantage is that only standard video equipment, without any type of computer, needs to be installed at the remote location; the disadvantage is the image or 3D-model from the local computer can only be viewed at the remote location and cannot be manipulated with computer software.
Figure 3. Transmission of 3D-models as a virtual endoscopy of the human head, realized during our first Croatian Tele-3D C-FESS in November 1998.
Video records of the procedure, MPEG streams of the procedure in combination with CT images and 3D models are essential for the creation of a computer database system for education and the preparation of future surgical procedures (Broadcasting, Tele-education, DVD-ROM, CD-ROM, www). In the real procedure, 3D computer models (Open Inventor) can be texture-mapped using the live video signal from the endocamera. The live video signal can be positioned using a 3D digitizer or any other spatial localizer. The texture mapped 3D model, with live video signal during the surgery, provides the surgeons with a more realistic computer presentation of the real surgical field.
Figure 4. Operating room equipment
In our preliminary tests and the 1st telesurgery, we used the following (Figure 4):
· SGI O2 workstation,
· Newbridge ATM Switch,
· ATM switched networks,
· Video streams over AAL-5,
· Computer communications over LANE (TCP/IP),
· 3Com Inverse Multiplexing (4xT1),
· Optivision MPEG1/MPEG2 encoders,
· Newbridge M-JPEG/MPEG1/MPEG2 encoders,
Network topologies:
· point to point T1 lines,
· nonroutable/routable shared FastEthernet/Ethernet,
· nonroutable/routable switched FastEthernet/Ethernet,
· ATM switched networks with AAL-5 and LANE,
· multiple T1 lines (today).
Usage of collaboration tools (H.120, H.323):
· SGI InPerson (Video, Audio, WhiteBoard),
· SGI Meeting (Whiteboard, Application Share/Collaboration),
· Microsoft NetMeeting(Video, Audio, Whiteboard, Application Share/Collaboration),
· StarVison StarMED, StarED.
The literature and different softwares used in our projects are shown in Table 1.
|
Literature
|
Software
|
|
|
|
|
Silicon Graphics
|
3D Viewnix - http://www.mipg.upenn.edu
|
|
General Electric
|
Analyze AVW - http://www.mayo.edu/bir
|
|
Elscint
|
Adobe Photoshop - SGI - http://www.adobe.com
|
|
Storz
|
Adobe Illustrator - SGI - http://www.adobe.com
|
|
Immersion
|
Adobe Premiere - SGI - http://www.adobe.com
|
|
3Com
|
SGI Irix Media Tools - http://www.sgi.com
|
|
Optivision
|
SGI InPerson - http://www.sgi.com
|
|
Siemens
|
SGI Meeting - http://www.sgi.com
|
|
Newbridge
|
Elscint OmniPro - http://www.elscint.co.il
|
|
Starvision
|
Optivison Video Network - http://www.optivision.com
|
|
|
Starvision StarMED - http://www.starvision.com
|
Table 1. The literature and different softwares used in our projects
DISCUSSION
The complex software systems allow visualization of a CT or MRI section in its natural course (the course of examination performed), or in an imaginary, arbitrary course. Particular images can be transferred, processed and deleted, or can be presented in animated images. The series of images can be changed, or images can be generated in different projections through the volume recorded.
Before the development of 3D spatial model, each individual image or a series of images have to be segmented, in order to single out the image parts of interest.
3D-model completely presents the relative relations of borderline areas that are of utmost diagnostic importance, which is a considerable advancement compared with standard 2D stratified imaging. Comparative analysis of 3D anatomic models with intraoperative finding shows the 3D volume rendering image to be very good, actually a visualization standard that allows imaging likewise the real intraoperative anatomy14,15.
The technique of examination intended to realize in the proposed study provides, according to many authors, optimal visualization of the paranasal sinus anatomy. Moreover, by use of this approach, some substantial advancements are achieved in the diagnosis of the pathologic state of paranasal sinuses, based on CT analysis, e.g. (a) basic CT diagnostics has become an important aid in the diagnosis of chronic sinusitis, in terms of the follow-up and prognosis of the course and treatment outcome; (b) additional axial or coronal sections are avoided by the development and use of 3D model (so-called volume-rendering technique) for diagnostic purposes; (c) as indicated in (b), the dose of irradiation to which the patient is exposed is considerably reduced by the use of volume rendering technique; etc.
Therefore, we made endeavors to implement all these concepts and advantages of the new mode of 3D visualization not only in daily 3D-CAS performed in our operating theaters but also as a specific form of telesurgery with 3D computer assisted support, currently a completely new and original type of telesurgery in the world. Our new approach to telesurgery, called Tele-3D-C-FESS, for the first time realized in 1998, is a breakthrough in telesurgery, as it ensures the use of most sophisticated computer technology in any operating room.
The first kind of our public Tele 3D C-FESS, developed as a part of the Tele-3D-CAS project, took place between two locations in the city of Zagreb, 10 km apart, with interactive collaboration from a third location. A surgical team carrying out an operative procedure at the Šalata ENT Department, Zagreb University School of Medicine and Zagreb Clinical Hospital Center, received instructions, suggestions and guidance through the procedure by an expert surgeon from an expert center. The third active point was the Faculty of Electrical Engineering and Computing, where ENT specialists, students and residents took an active part in the entire surgical procedure. This Tele 3D C-FESS surgery, performed as described above, was successfully completed in 15 minutes.
Figure 5: First Tele-3D C-FESS in Croatia (November 1998).
The surgeon and consultants view four split video signals (quad split video processing) on the primary video display (Figure 5,6): one from the endocamera, one from the OR camera (Figure 7), one from the first remote site and one from the second remote site. However, during real surgery, all locations involved in the telesurgery usually view only the video signal from the endocamera procedure on their primary displays.
The new video encoders using MPEG2 standards and ATM computer networks using inverse multiplexing greatly improve the safety of surgical procedures, especially in endoscopic surgery. The best results were obtained using ATM-OC3 technologies, with the most acceptable price-performance using inverse multiplexing method across 4-8 E1 lines.
Figure 6. Primary video display.
The surgeon and consultants view four split video signals (quad split video processing) on the primary video display (Figure 5,6): one from the endocamera, one from the OR camera (Figure 7), one from the first remote site and one from the second remote site. However, during real surgery, all locations involved in the telesurgery usually view only the video signal from the endocamera procedure on their primary displays.
Figure 7. Operating room camera.In real surgery, there is one patient at one location included in the "Telesurgical Procedure", with one or more consultants at one or more remote locations (the local location is where the surgery is performed, and remote locations are the other locations included in the telesurgery procedure)16. Thus, two video streams (the endocamera and the conference camera) have to be transferred from the local location (patient location), and one video stream (the conference camera) has to be transferred from each remote location (consultants). At each location, video monitors are needed for video streams from any other location included in the telesurgery procedure (up to four video streams on one large video monitor, quad-split, can be used). An LCD/TFT video projector can be put in the OR.
Definitely, the possibility of exact preoperative, non-invasive visualization of the spatial relationships of anatomic and pathologic structures, including extremely fragile ones17,18, size and extent of pathologic process19,20, and to precisely predict the course of surgical procedure15, allows the surgeon in any 3D-CAS or Tele-CAS procedure to achieve considerable advantage in the preoperative examination of the patient, and to reduce the risk of intraoperative complications, all this by use of VS21 or diagnosis22. With the use of 3D model, the surgeon's orientation in the operative field is considerably facilitated20 (“patient location” as well as the “location of the tele-expert consultant”), and all procedures and manipulations are performed with higher certainty and safety23.
As it can be seen, one of their main applications of Tele-3D C-FESS is 3D-navigation (VE) in the study of anatomy and surgery (“computed journey through the nose and paranasal sinuses”). From this point of view this approach can be compared with similar simulator systems for the training of endoscopic sinus surgery presently available on the market, but definitely we have to be aware that it is not the study of or training in anatomy or surgery, but pure implementation of live surgical procedure with computer support in real time, the prime and foremost aim being the achievement of faster and safer procedure. So, it should be made clear that the main message of our Tele-3D-C-FESS surgery, as differentiated from the standard tele-surgeries worldwide, is the use of the 3D-model operative field, and thus of ”VS”, which in addition to higher safety allows for successful course of operation, especially in small, detached medical institutions where advanced endoscopic techniques are not available. This is of paramount importance for emergency surgical interventions which have to be performed in distant medical institutions where the service of “well known surgical experts” (e.g., skull base surgery) is not available.
Conclusions
Our telesurgical approach allows surgeons not only to see and transfer video signals but also to transfer 3D computer models and surgical instrument movements with image/3D-model manipulations in real time during the surgery.
Our telesurgical approach allows surgeons not only to see and transfer video signals but also to transfer 3D computer models and surgical instrument movements with image/3D-model manipulations in real time during the surgery.
The new video encoders using MPEG2 standards and ATM computer networks using inverse multiplexing greatly improve the safety of surgical procedures, especially in endoscopic surgery. The best results were obtained using ATM-OC3 technologies, with the most acceptable price-performance using inverse multiplexing method across 4-8 E1 lines.
Considering the specificities and basic features of Tele-3D C-FESS, we believe that this type of surgery would be acceptable to many surgeons all over the world for the following reasons:
· the technology is readily available in collaboration with any telecom worldwide;
· the improved safety and reduced cost will allow the inclusion of a greater number of patients from distant hospital institutions in such a telesurgical expert system,
· the “presence” of leading international surgical experts as tele-consultants in any OR in the world will thus be possible in the near future, which will additionally stimulate the development of surgery in all settings; and
· the results obtained in the Tele-3D C-FESS project in Croatia are encouraging and favor the further development of the method
REFERENCES
1. Klapan I, Šimičić Lj, Kubat M et al. Our three-dimensional Computer Assisted Functional Endoscopic Sinus Surgery (3D C-FESS). In: McCafferty GJ, Coman WB, eds. XVI World Congress of Otorhinolaryngology, Head and Neck Surgery, Sydney, Australia. Bologna: Monduzzi Editore, 1997: 1543-1547.
2. Ecke U, Klimek L, Muller W, Ziegler R, Mann W. Virtual reality: preparation and execution of sinus surgery. Comput Aided Surg 1998; 3:45-50.
3. Urban V, Wapler M, Neugenbauer J. Hiller A, Stallkamp J, Weisener T. Robot-assisted surgery system with kinesthetic feedback. Comp Aided Surg 1998; 3:205-209.
4. Klimek L, Mosges M, Schlondorff G, Mann W. Development of computer-aided surgery for otorhinolaryngology. Comput Aided Surg 1998; 3:194-201.
5. Hassfeld S, Muhling J. Navigation in maxillofacial and craniofacial surgery. Comput Aided Surg 1998; 3:183-187.
6. Klapan I, Barbir A, Šimičić Lj, Rišavi R, Bešenski N, Bumber Ž, Štiglmayer N, Antolić S,
Janjanin S, Bilić M. Dynamic 3D computer-assisted reconstruction of a metallic retrobulbar
foreign body for diagnostic and surgical purposes. Case report: orbital injury with ethmoid bone involvement. Orbit 2001 (in press).
7. Klapan I, Rišavi R, Šimičić Lj et al. Tele-3D-C-FESS approach with high-quality video transmission. Otolaryngol Head Neck Surg 1999; 121:187-188.
8. Klapan I, Šimičić Lj, Rišavi R et al. Tele-3D-Computer Assisted Functional Endoscopic Sinus Surgery (Tele-3D-C-FESS). In: Lemke HU, Vannier MW, Inamura K, Farman AG, eds. Proceedings of the 13th CARS'99, Elsevier, Amsterdam, NewYork, Oxford, Shannon, Singapore, Tokyo, 1999: 784-789.
9. Klapan I, Šimičić Lj, Rišavi R, Pasarić K, Schwarz D, Antolić S, Janjanin S. Real time transfer of live video image in parallel with volume 3D-models of the surgical field in Tele-3D-CAS: two different approaches. J Telemed Telecare (submitted) 2001.
10. Satava RM. Telesurgery - acceptability of compressed video for remote surgical proctoring - invited commentary. Arch Surg 1996; 131: 401.
11. Schlag PM, Engelmurke F, Zurheyde MM et al. Teleconference and telesurgery. Chirurg 1998; 69:1134-1140.
12. Go Pmnyh, Payne JH. Endoscopic surgery teleconferencing. Int Surg 1996; 81:18-20.
13. Schlag PM, Engelmurke F, Zurheyde MM et al. Teleconference and telesurgery. Chirurg 1998; 69:1134-1140.
14. Hamadeh A, Lavallee S, Cinquin P. Automated 3D CT and fluoroscopic image registration. Comput Aided Surg 1998; 3:11-19.
15. Mann W, Klimek L. Indications for computer-assisted surgery in otorhinolaryngology. Comp Aided Surg 1998; 3:202-204.
16. Sezeur A. Surgical applications of telemedicine. Ann Chir 1998; 52:403-411.
17. Hauser R, Westermann B, Probst R. Non.invasive 3D patient registration for image guided intranasal surgery. Medical Robotics Comput Assisted Surg, New York, Springer, str. 1997; 327-336.
18. Vinas FC, Zamorano L, Buciuc R, QH L, Shamsa F. Application accuracy study of a semipermanentm fiducial system for frameless stereotaxis. Comput Aided Surg 1997; 2:257-263.
19. Elolf E, Tatagiba M, Samii M. 3D-computer tomographic reconstruction: planning tool for surgery of skull base pathologies. Comput Aided Surg 1998; 3:89-94.
20. Burtscher J, Kremser C, Seiwald M, Obwegeser A, Wagner M, Aichner F, Twerdy K, Felber S. 3D-computer assisted MRI for neurosurgical planning in parasagittal and parafalcine central region tumors. Comput Aided Surg 1998; 3:27-32.
21. Holtel MR, Burgess LP, Jones SB. Virtual realiry and technologic solutions in Otolaryngology. Otolaryngol Head Neck Surg 1999; 121(2):181.
22. Vrabec JT, Johnson RF, Guinto FC. 3D-virtual endoscopy of the temporal bone. Otolaryngol Head Neck Surg 1999; 121(2):225.
23. Olson G, Citardi M. Image-guided functional endoscopic sinus surgery. Otolaryngol Head Neck Surg 1999; 121(2):187.
