CMOS Sensor Selection Defines the Performance Ceiling of the Medical Endoscope Camera module

What the Sensor Must Survive Before It Can Image

Medical constraints mean sensor design is very different compared to the consumer market. The sensor must survive many cycles of sterilisation (hundreds), if it is a chip on a re-usable endoscope. If a chip on tip design then the sensor must survive in a couple of mm away from tissue, constantly exposed to light. The sensor must have a low power consumption as heat generated in the distal tip from inefficient energy will increase dark current and damage the mucosa. The sensor must turn on quickly, within seconds to start video, and have low frame delay times to fit in with the feedback times during surgery. The clock rates need to match the light.

The environment will limit what optical format can be used on the sensor. Sensors in medical endoscopes usually range from 1/18 inch micro sensors (much less than 1mm) to 4K laparoscopic sensors which have a 1/1.8 inch format. The smaller the sensor format the more difficult the lens designer has to work to get enough light onto the sensor. This means the sensor's Quantum Efficiency becomes key. This means the sensor can't be tested as an individual component but needs to be tested as part of the complete Medical Endoscope Camera module.

 

Pixel Size and the Sensitivity Trade-Off

The most important specification to look at will be pixel size. This is due to the impact it has on how much light can be captured per pixel. Due to the nature of Endoscopy (low light, non-uniform lighting and often scenes with lots of deep reds, which are highly absorbed by tissues), having a high sensitivity is critical. In medical endoscopes, pixel sizes will vary from around 0.9 microns in small chip on tip sensors up to around 2.3 microns in larger 4K. The amount of light gathered can vary significantly.

Smaller pixels provide higher resolution and compact optics but suffer from a loss in signal-to-noise ratio. However, there are some technologies that can help:

Back Side Illuminated (BSI) – Wiring can be moved under the photodiode to increase the quantum efficiency. This can provide efficiencies over 80-90% at visible wavelengths. This is significantly better than traditional front illuminated designs.

Deep Trench Isolation – This reduces crosstalk between pixels so colour accuracy is maintained even at high frequencies.

Dual Conversion Gain – This enables the pixel to operate in different modes. High capacity for bright scenes and high sensitivity for dim scenes. This increases the dynamic range of the pixel without increasing the pixel size.

Whilst looking at specifications can help compare different sensors, it can be useful to know how they perform using the EMVA 1288 standard. This provides quantum efficiency curves, temporal dark noise, full well and dynamic range. This makes sure you can directly compare different vendors and also shows what the weaknesses of the sensor are. If you have any serious Medical Endoscope Camera modules planned then it would be worth allocating some lab time to do EMVA testing on some real parts.

 

Resolution, Frame Rate, and the Bandwidth Bill

The resolution and frame rate will dictate how much data the sensor is trying to push through to the serializer inside the module. This will have an impact on how the cable is designed, how much power is required and ultimately cost. For example a sensor imaging 4K at 60fps and producing 12 bit RAW data will produce a few Gigabits/sec of data. This needs to be passed through a tough environment inside the cable of a flexible endoscope, involving small bend radii, impedance mismatches at the connectors and little shielding.

Sensor interfaces can be just as important as sensor quality. Modern sensors provide interfaces such as MIPI CSI-2 with 4+ lanes, in-sensor serializers that produce COAX-press or FPD-Link style signals and optionally compression on the sensor. If the sensor has a native interface then it will reduce the distance between the pixel and serializer within the module and remove an entire area of risk around signal integrity. Whereas if a sensor only provides wide parallel outputs then there will be more electronics required on the tip of the module which can cause heat and space issues.

Latency should also be considered. Most endoscopes use rolling shutter sensors. This is fine if global exposure can be controlled by illumination strobing or software correction. However you should check how the latency of your candidate sensors works with strobed fluorescence. The exposure window must be timed with the light pulses.

 

Spectral Response: Seeing Beyond Visible Light

Spectral imaging has become a critical factor when using fluorescence guided surgery. Indocyanine green emits fluorescence in the near infrared at around 820nm. It is therefore important that the sensors used have high quantum efficiency in the NIR region with minimal crosstalk to the visible regions.

There are three main architectures to consider. Normal CMOS sensors can be used with a modified colour filter array but will sacrifice some visible fidelity. Specialist dual mode sensors can be used but are becoming more prevalent in leading platforms. They provide separate signal paths to the white light and NIR regions. Finally, dual sensor systems can be used which split the optical signal. This provides perfect signal integrity to both regions, however they do take up more space and can be complex to align. The correct option will depend on the positioning of the product but needs to be decided when choosing the sensors as it will affect the optical design, illumination timing and contracting the interface of the module.

 

Color Filter Arrays and Downstream Image Tuning

Selecting a Sensor will also dictate the colour architecture that the ISP operates with. Whilst most Endoscopic Sensors use the Bayer colour filter array, there can be significant differences between vendors in the dyes used and cut-off wavelengths. This will affect white balancing, tissue colours and how easily you can separate the fluorescence signal from the reflectance.

There are a few factors to think about here. If you have two sensors with the same quantum efficiency curve but different dyes in the filters you may see a difference in the tissue colours captured using the same LED illumination. Some vendors also have unfiltered sensors (monochrome) giving the highest sensitivity. This is great if you are designing a fluorescence or near infrared only product but forces you to operate fully within this modality. The availability of raw data is really important in medical imaging. If your sensor can provide clean raw data using a linear response then you have complete control over the tone mapping and noise reduction in the ISP. If your sensor is highly processed then you have less flexibility in the tuning and you may lose clinically important information due to aggressive noise reduction.

Request spectral response charts for the sensor. Where possible request sample images using the illumination you will actually be using. Colour science can be tailored to your sensor but cannot be independent. It will cost significantly less if you can get good documentation on the raw performance of your sensor.

 

Packaging, Form Factor, and the Chip-on-Tip Reality

The packaging of the sensor will dictate what your mechanical design can do. By using wafer level optics and wafer level packaging we have been able to create full camera cubes less than 1.5mm in size. This has made single use bronchoscopes and ureteroscopes commercially viable. The unit cost of the sensor will dominate the economic factors of the system. Suppliers providing high yields and consistent performance between lots will win the socket.

Chip on tip designs for re-use will require the sensor to be resistant to sterilisation methods. It must be stable across thousands of thermal cycles and maintain its calibration over the lifetime of the product (years). For a Medical Endoscope Camera module for re-use, the sensor lifetime data, moisture sensitivity level and willingness of the vendor to provide medical grade process change notification are just as important as image performance. A drifting sensor after 80 autoclave cycles would not be cheap!

 

Thermal Behavior and Noise Under Real Operating Conditions

Datasheets will often state parameters for dark current and noise performance at either 25 or 60 degrees Celsius. However, in operation the internal temperatures of an endoscope tip will be higher. Dark current doubles roughly every 5-6 degrees Celsius. Sensors that look good on the bench may become the main contributor in your module. Thermal modelling of the real module should be performed. Considerations such as power from the sensor, distance from the LED driver and thermal conductivity of the housing will all play a role in the sensor temperature.

Noise performance in the power domain should also be considered. While it may be convenient to have CMOS sensors with on-chip clocking, the phase locking loops can cause spurs in any sensitive analog circuits nearby. It is important to confirm the noise performance using the real regulator topology in your module. This will prevent any nasty surprises!

 

Supply Longevity: Matching Sensor and Product Lifecycles

Medical devices can be deployed in the market for a decade or more. Consumer sensors are frequently updated or dropped from product lines every 2-3 years. This means when selecting a sensor for your camera module you need to know how long term the company will support that part. Will they have an end of life policy? Do they want to qualify new dies for medical use? Some manufacturers will qualify two pin compatible sensors to help overcome this issue. Alternatively they will have a contract where they get notified of changes during initial sourcing. Ultimately the objective is to have a silicon life cycle which exceeds the life cycle of the product your module is being used in.

 

A Practical Selection Checklist

1. If you are choosing sensors with care then typically the following steps will be taken:
2. Define your image specifications (resolution, frame rate, low light, latency and fluorescence) before opening any datasheets.
3. Narrow down which sensors can be used based on interface & package options. You wouldn't want to have a sensor that has the perfect image spec but isn't usable.
4. Measure your shortlisted sensors to EMVA 1288 specifications using endoscope lighting. This includes measuring low light SNR with realistic exposure times.
5. Model the thermal and electrical conditions of your real module. Re-measure noise and dark current in these conditions.
6. Ask about the supply. What guarantees do they give? How will you know when things change? Are they good at supporting the medical market? What is their second source strategy?
7. Once you've had a reference design go through your cable and connector chain, lock it down. This proves that your interface will survive the mechanical conditions.

 

Sensor selection may seem like a matter of component purchasing but it is an architectural choice that constrains the quality of images, thermal performance, bandwidth, fluorescence capabilities and service life of any system you build. Choosing the right sensor for your Medical Endoscope Camera means that you can innovate with new optics, signal processing and even clinical indications of use without requiring changes to your silicon. Choosing the wrong sensor means you have to redesign from scratch. Teams shipping high quality endoscopic imaging systems will invest in understanding the sensor characteristics, require long life from the silicon supplier and will think of the sensor as the core of their Medical Endoscope Camera module. If you choose the right sensor with these considerations in mind then you will have plenty of scope for success. If you don't take this seriously then no amount of downstream engineering will help lift your ceiling.