Blur Spot Limitations in Distal Endoscope SensorsBlur Spot Limitations in Distal Endoscope SensorsBlur Spot Limitations In Distal Endoscope SensorsAvi YaronVisionsense Inc, Orangeburg, New YorkAbstractIn years past, the picture quality of electronic video systems was limited by the image sensor. In the present, the resolution of the image sensor can be superior to the resolution of the optical system in imaging systems such as those employed in medical endoscopy. This “excess resolution” is utilized by Visionsense to create stereoscopic vision in surgical endoscopes that meet the needs of the medical market. Visionsense has developed a single chip stereoscopic camera that multiplexes the horizontal dimension of the image sensor into two (left and right) images, compensates the blur phenomena, and provides additional depth resolution without sacrificing planar resolution. The camera is based on a dual-pupil imaging objective and an image sensor coated by an array of microlenses (a plenoptic camera). The camera has the advantage of being compact, providing simultaneous acquisition of left and right images, and offering resolution comparable to a dual chip stereoscopic camera with low to medium resolution imaging lenses. A stereoscopic vision system provides a 3-dimensional perspective of intra-operative sites that is crucial for successful minimally invasive surgery. An additional advantage of single chip stereo sensors is improvement of tolerance to electronic signal noise.
Keywords: plenoptic camera, stereoscopy, lenticular array, resolutionIntroductionTwo physical factors determine the resolution of a digital camera: the camera objective lens and the pixel-size of the image sensor. In general, the optical resolution of well-designed lenses (in the absence of size constraints, such as a digital camera) is superior to the resolution of the most advanced image-sensors, which therefore requires the use of anti-aliasing filters. When size and costs are constraints, the image sensor cost can be reduced by reducing the chip-area. On small format sensors, it is necessary to reduce the pixel size in order to maintain acceptable image resolution. For instance, PAL/NTSC signals are provided by ≤1/6 inch format chips with pixel size of ≤ 3μ.
The pixel-size limitations of a digital camera are a consequence of the fact that large file formats are commonly used for digital file processing. When a large file format is used, it can be difficult to distinguish the two formats, thus resulting in the creation of different images.
Because digital recording is generally very difficult for most people, there is an obvious approach to this problem: In order to improve contrast sensitivity, we often use a method called digital video compression (DPC). One common technique, commonly referred to as DPP, is known as sRGB compression (see Figure S10-5). According to a number of studies, SRGB compression results in better contrast and brightness than traditional S-filtering results in the absence of frame-alignment (Bordeau et al., 2012). The main reason for sRGB compression is that the contrast of the sRGB image is proportional to the color wavelength of the sensor.
Bordeau et al., 2012, proposed a solution on a different theoretical basis, using a very broad spectrum of resolution methods, on a large variety of digital camera technologies. These proposed methods are shown below.
Figure S10-4, Comparison of the three different S-filtering techniques used in S-filtering on Nikon DSLRs. Note that the resolution of these two techniques differ by ~20 kts. The image below illustrates the sRGB compression results.
Figure S10-5, Comparison of the two different techniques used in S-filtering on Pentax DSLRs. On a Nikon DSLR with a 2,000 x 2,500-pixel sensor, sRGB compression is 1.7 kts. Compare the image below with the S-filtering results (a bit darker).
Figure S10-5a, Comparison of the two different S-filtering techniques used in S-filtering on Pentax DSLRs. On a Pentax DSLR with a 1,900 x 1,600-pixel sensor, sRGB compression is 1.4 kts. Compare the image below with the S-filtering results (a bit brighter). It shows that even larger image sizes are needed to obtain high image quality.
Figure S10-5b, Comparison of the three different S-filtering methods used in S-filtering on Nikon DSLRs. On a Pentax DSLR using a 1,900 x 1,600-pixel sensor, sRGB compression is 0.9 kts. In contrast with the S-filtering results, sRGB compression increases image quality. Compare the image below with the S-filtering results (a bit smaller).
Figure S10-5c, Comparison of three different S-filtering methods used in S-filtering on Canon EF EOS 2Ds 6D. Comparison of the image below with the S-filtering results (a bit wider).
Figure S10-5d, Comparison of three different S-filtering methods used in S-filtering
In comparison, there are applications where the resolution of the image sensor is superior to the resolution of the optical system. These are usually low cost / low quality optics in hand-held devices such as cell-phone cameras, or high depth-of-field applications such as medical endoscopy. This “excess resolution” is used to great advantage by Visionsense technology to create stereoscopic vision in surgical endoscopes that meet the needs of the medical market.
Medical endoscopyMinimally invasive surgery (MIS) began when surgeons classically trained in “open” surgical procedures started to use long, thin imaging devices and surgical instruments to perform operations through small incisions. Early on, many technical factors were down played or overlooked because endoscopic surgical procedures seemed, at the time, intuitively similar to open procedures. Experience, however, has shown that the use of endoscopic instruments requires unique eye-hand coordination. In particular, the use of video cameras and monitors greatly affects the perception of physical reality and therefore performance.
Vision systems in currently available MIS instruments provide 2 dimensional (2D) images that lack depth perception, which restrict the surgeon’s perspective and ability to perform complex manipulations. Recently published clinical papers [1][2], have documented that severe errors made during laparoscopic procedures are due to a critical misinterpretation of the video image, not simply errors in surgical technique. A review of surgical injury to the common bile duct found that the damage was caused by misinterpretation of the endoscopic image and incorrect decisions based on false perceptual information. In defense of the surgeons, many of us assume that our eyes are a reliable tool to interpret reality, and we overlook the crucial fact that video images have many limitations that can create a false sense of genuineness.
The flatness of conventional 2D imaging does nothing to augment a surgeon’s performance, whereas 3 dimension (3D) stereoscopic vision can enhance image understanding and improve performance in laparoscopic surgery. Our system enables natural vision without discomfort, and it affords better results and enhanced confidence among less-experienced surgeons. In addition, stereoscopic