FAQ, VayTek, Deconvolution, Digital Microscopy, Digital Imaging Systems, VayTek, confocal microscopy, algorithms

Deconvolution software, like VayTek's "MicroTome" software, uses several algorithms (user-selected) to mathematically calculate and remove out-of-focus haze from microscope images. It is designed for use with conventional microscopes or to supplement pinhole-based confocal microscopes.

2. What is a confocal microscope?

Minski (1961) was the first to propose the technique of confocal microscopy used by laser scanning confocal microscopes. The principle is quite simple and is illustrated in the light paths in Figure 1.

The image seen through a microscope includes the in-focus portion and the out-of-focus portion above and below the plane of focus. The smear or blur produced by the out-of-focus planes is a natural consequence of the optics of the microscope.

Confocal microscopy removes out-of-focus haze by passing the light through one or more small apertures, leaving only a thin, highly focused plane. The light from this focused plane can be digitized and stored on a computer.

The distance between the specimen and the microscope objective is then changed producing a new focal plane. The new focal plane is digitized and stored. After a series of planes has been collected, individual slices can be examined or the whole specimen can be digitally reconstructed by a computer as a three-dimensional volume.

A confocal microscope consists of a standard microscope with a number of complex attachments to direct and process the beam of light. Most confocal microscopes use an intense laser light to scan the specimen. This intense light source is needed to compensate for the light loss which occurs as the light passes through small apertures.

3. How does VayTek's MicroTome software work?

In part, MicroTome does in software what the confocal microscope does by virtue of hardware (i.e. the pinhole). Both systems use image processing but MicroTome is more flexible and can be used to great advantage with conventional microscopy and to improve confocal microscopy.

MicroTome software, as illustrated in Figure 2, uses a standard white-light microscope and requires no special attachments. A video camera captures and digitizes the images from the microscope, which are then stored by a computer. Image enhancement algorithms are used to deconvolve the image, i.e. remove the blur or haze contributed by the out-of-focus image planes. The algorithms used by MicroTome have the same function as the apertures in the laser scanning confocal microscopes - removing the out-of-focus portion of the image.

You can transform your standard microscope and your computer by simply adding VayTek's deconvolution software package.

4. What is the value of a confocal image?

A confocal image has the out-of-focus haze removed. This can theoretically increase image resolution. The increase in resolution, by as much as 1.4 times (Brackenhoff, 1989), results in improved measurements (Yelamarty, 1990) and visualization. In addition, the optical sectioning is non-invasive and can be performed on living specimens. Also, it is possible to acquire images with multiple wavelengths of light and merge the results for greater information.

Besides increasing the resolution of the image, the deconvolved slices can be stacked to produce a three-dimensional representation of the specimen. Visualization of a three-dimensional data set can lead to new insights.

5. What are the advantages of using deconvolution software instead of, or in conjunction with, a confocal microscope?

a. It costs less than a laser scanning confocal microscope because the microscope you currently have can be used with the digital deconvolution approach. New optical equipment is not required. Prices for laser scanning confocal microscopes typically range between $75,000 and $300,000. The price of MicroTome is a fraction of this cost.

Depending upon components already in place, you may need to add a framegrabber, camera, and stepper motor in addition to MicroTome software. For a complete description of products, turnkey systems and other offers please refer to VayTek's Product Guide or call directly (515) 641-2227.

b. the high intensity laser light, required by the LSM's, can harm living specimens. The digital approach is less harmful to living material since it typically uses a small fraction of the light used by the laser scanning microscopes.

c. many fluorescence preparations bleach easily, even with standard light sources. These dyes cannot be used with laser scanning confocal microscopes. Even robust preparations can fade after many scans producing a brightness gradient along the vertical axis. Deconvolution software will result in less photobleaching.

d. MicroTome is more flexible. When using a laser scanning confocal microscope, the amount of haze removed is set by the aperture size and thus cannot be adjusted after the image is captured. HB/MT, on the other hand, allows the user to set the amount of haze to be removed as a part of the deconvolution process after the image has been captured. Thus, the HB/MT user can explore the same data set multiple times with different degrees of haze removal.

e. data acquisition for MicroTome can be faster than confocal microscopes. Video cameras, used by MicroTome, can average or integrate several slices per second. Including the time to move the stage, three images can be captured in about 3 seconds. Each image is then deconvolved, requiring no more than four seconds (MicroTome version) per 512 x 512 slice.

Confocal microscopes can be slower. Each slice is often scanned and integrated multiple times with reduced laser power in an attempt to attenuate photobleaching effects.

6. What are the limitations of deconvolution software like MicroTome?

The principal limitation of the digital deconvolution approach has been the amount of computer time required to deconvolve a single slice. Until now, a single slice could require several minutes to deconvolve on a personal computer. A large data set could take an entire day to process.

With the introduction of MicroTome, however, processing time has been reduced to no more than a few seconds per slice on a PC or Power Mac. These speeds are possible because of VayTek's unique, efficient implementations of the algorithms.

There is an additional limitation with MicroTome. Slices must be relatively close to each other to achieve the proper resolution after deconvolution. The exact distance will vary from sample to sample, but experience has shown that it should be between .1 micron and 10 microns.

7. What is a deconvolution algorithm?

The word "deconvolve" means to "untangle or unwind". A deconvolution algorithm is a systematic procedure for removing noise or haze from an image.

There are several well known deconvolution algorithms that can be applied to microscope images to remove the out-of-focus haze. The easiest to use is the nearest neighbor algorithm. This approach has the advantage of being very fast and yielding very good results. The nearest neighbor algorithm requires a minimum of three slices. Other algorithms include the inverse filter and the constrained iterative. These algorithms will yield slightly more precise results, but require many more slices and more computation time (Agard, 1989). For more information refer to the Technical Note.

8. How do you adjust the haze removal with MicroTome?

The laser scanning confocal microscope varies the amount of haze removal by altering the size of the aperture. With MicroTome you vary the amount of haze that is removed after a data set has been collected by adjusting the haze removal parameter used during deconvolution.

With MicroTome you specify the amount of haze to be removed at the time of deconvolution, giving you more flexibility while working with your data. The ability to set this parameter, however, raises the issue of what the optimal haze removal setting should be. This setting will vary from data set to data set, but experience has shown that 90% removal is optimal for most data.

9. Will the deconvolution approach replace the confocal microscope?

Most experts in the field of digital deconvolution agree that deconvolution technology and pinhole based microscopes complement each other. In fact, many believe that the two technologies should be available on the same system so the researcher can choose which to use. In fact, digital deconvolution can be used to further enhance images captured with a confocal microscope (Shaw, 1991).

The relationship between the two technologies is illustrated in Figure 3. The smaller circle in Figure 3 represents the collection of all images that can be acquired with the laser scanning confocal microscope. The larger circle represents the collection of all images that can be successfully acquired with MicroTome. The intersection of the two circles represents those images that can be successfully produced on either system.

Most experts on digital deconvolution agree that at least 90% of the images that can be created with the laser scanning confocal microscope can be produced equally well with digital deconvolution.

However, there are some images that can only be produced with a laser scanning confocal microscope. These images include those in which the distance between slices is, by necessity, quite large. Also, thick or semi-transparent non-living specimens that require powerful laser light to penetrate into the material will be best imaged by a laser scanning confocal microscope.

Conversely, there are some images that can only be produced by the digital deconvolution approach. For example, those specimens with sensitive fluorescent dyes.

10. How does the nearest neighbor algorithm work?

The nearest neighbor algorithm needs a minimum of three optical slices. There is no theoretical limit to the maximum number of slices that could be used, although, as a practical matter, a researcher would seldom acquire more than 100 slices.

Based on the data in the slices, MicroTome computes a characteristic optical point spread function (PSF) for the lens in the microscope that was used to acquire the data. The PSF is then used to deconvolve each image using the images above and below the image being processed. This process identifies the haze which is subtracted from the slice of interest, resulting in the deconvolved image. See our technical note on algorithms

11. What is a point spread function (PSF)?

A point spread function is a mathematical term for the impulse response of a system. When the term "point spread function" is used in connection with an optical system it means the impulse or point response of an optical system to a point input.

A single point of light is focused by the lens into a complex shape known as a point spread function (PSF). The shape of the PSF depends upon light wavelength, lens numerical aperture (NA), and the optical aberration of the lens. By knowing the shape of the PSF the operator can remove the excess light from any image plane thus producing a high resolution image. See our technical note on algorithms

12. How does MicroTome calculate a PSF?

The PSF is calculated using diffraction theory. The parameters needed to calculate the PSF are light wavelength, numerical aperture of the lens, the distance between pixels within a plane and the distance between the acquired image planes. The user must supply HB/MT with these parameters. See our technical note on algorithms.

13. Is a theoretical PSF accurate enough to produce high quality images?

Yes. The nearest neighbor algorithm is tolerant of the difference between a theoretically and experimentally obtained PSF. MicroTome allows the user to input an experimental PSF, if desired, however.

14. How do I know that what I see in the deconvolved image is real?

There are two ways to assess the reliability and validity of the images produced by MicroTome. The first means of verification is mathematical. The deconvolution algorithms have been published, reviewed and accepted. The reader is invited to read the articles listed in the Bibliography.

The second means of verification is empirical. The algorithms work properly if 1) they image known structures correctly and 2) they produce images similar to those produced by laser scanning confocal microscopes. The reader is directed to the accompanying material illustrating images produced by MicroTome. In addition, readers may send blurred images to VayTek. We will deconvolve them and return the results.

15. How fast can MicroTome deconvolve an image?

Please refer to technical specifications for the latest deconvolution speeds. Times will vary from a few seconds to several minutes depending on the computer platform.

16. Is MicroTome easy to use?

Yes. MicroTome has a friendly, point-and-click interface. MicroTome was designed to make the deconvolution algorithms easy to use and give you feedback of the results as quickly as possible.

17. What are the data acquisition issues?

It is very important to use high quality raw images for deconvolution; otherwise garbage-in, garbage-out. Good raw images mean using a good microscope, an appropriate camera, a good framegrabber, and acquisition software that lets you average and integrate during image capture. VayTek can provide the necessary components for data acquisition. Please consult a VayTek salesperson, the MicroTome manual, or the MicroTome demo program, for a more detailed discussion of data acquisition issues. See our technical papers on data acquisition.

18. What is resolution and what does it have to do with the numerical aperture number of an objective lens...?

This information is a summary of an article on Numerical Aperture and Resolution that appears on the UCLA Brain Research Institute Microscopy Core Facilities website. Resolution can be defined as the ability of a microscope to allow one to distinguish between small objects. In other words, how crisp and sharp is an image at any given magnification? The numerical aperture number is directly related to the cone of light from the specimen at its vertex which is brought into the lens. When light hits an object, it diffracts. A single beam of light will be split into several different diffraction orders bent at increasing angles from the original impinging beam. The higher the numerical aperture of a lens, the better the resolution of a specimen will be which can be obtained with that lens. Using a higher numerical aperture results in more orders of diffraction from the object being brought into the lens. More light is brought into a higher numerical aperture, producing brighter images.

19. What image formats are read by HB/MT?

MicroTome will support most file formats. You specify the header length, height and width and file type. The image data must be 8, 12, 16 or 24 bit integer, binary, raster scan format.

20. Is there technical support for this product?

Yes. Technical support is available at no extra charge for the first year after purchase. After the first year, additional support and new releases are available for a maintenance fee.

21. How can I visualize my data?

MicroTome lets you view the 2D slices as you deconvolve them. VayTek also sells a 3D reconstruction program for the Windows, Macintosh and UNIX based workstations called VoxBlast.

22. Can I get a hard copy print out of my images?

Yes. There are a number of options for printing images. For more information on printers, please consult a VayTek sales representative. It is now possible to also print a 3D Lenticular Panel of an image processed with VayTek Software.

23. How do I obtain 3 color, 3D from Leica stacks?

We have a Leica confocal, TCS-NT. It will make stacks of my images, in three color (FITC, Rhodamine, CY5). I have tried to use NIH-Image. It will call up (import) the stacks, but only the first color appears for all images. So the first question is how to I make NIH-Image recognize the individual colors for each slice? I do not know how to then "overlay" the slices for a 3 color 3D image. Is there any other reasonably priced software out there that anyone can recommend that will do 3 color 3D from Leica stacks?

Answer: Most software will not understand the Leica stack format. Although they are TIFF's, as you know in a 2/3 channel 3D image, the image data is stored as: sections of channel 1, then all sections of channel 2 etc. If you want to see all the images with their particular LUT's, you can open the stack in NT using Imaging
(WangImage) which is located under Accessories.

What you need to do is resave (using the save select command) the optical sections from each channel individually (in TCS). Then in your 3D software import each channel individually.

You don't mention what sort of 3D processing and rendering you intend doing, so it's hard to advise further. NIH Image may well suffice, and if you're working with PC's then you should take a look at ScionImage [http://www.scioncorp.com] which will also work with stacks. Don't forget to load the stack macro before trying stack manipulations (though I can't remember if it does any overlaying/rendering or just projections).

Since we need to make 3D volume/surface area measurements from the rendered images, I routinely use VoxBlast from VayTek, Inc. It is reasonably priced, fast and very powerful.

-- Dr Ian S. Harper, Confocal Microscopy Facility, Department of Biological Sciences, Monash University, Clayton, Vic 3168, Australia, Email: Ian.Harper@sci.monash.edu.au

24. What is the best camera for deconvolution?

What is the best camera for deconvolution? The two most important issues you must consider to answer this question are:

Analog cameras
If you are working with specimens in brightfield or transmission mode, then you can use most any camera. The less expensive, analog cameras will have more noise, however, and create images that will not deconvolve quite as well as images captured with digital cameras. The instances in which VayTek recommends an analog camera are limited to users who have small budgets, who need the analog camera for live recording, and who will be doing only occasional deconvolutions, usually for improving images for publication. Images captured with analog cameras are confined to the nearest neighbor algorithm.

Digital cameras
Deconvolution is really designed for fluorescent specimens. This is because the mathematical formulae for the deconvolution algorithms assume the light in the specimens radiates equally in all directions. Consequently, the implications of specimen characteristics for choosing a camera become more obvious.

If you are working with very bright fluorescent specimens, then you may still be able to work with an analog camera, although the analog camera will not give the best results for deconvolution. A digital camera will be a much better choice, even for bright specimens. There are several reasons for this.

Digital cameras have better dynamic ranges than analog cameras - more gray levels mean more accurate deconvolution. Digital cameras, which are usually cooled, have lower noise and are better for deconvolution - the mathematics of deconvolution will amplify noise in an image so it is important to minimize it during acquisition. Digital cameras integrate over time, thereby letting you work with a wide variety of fluorescent dyes and intensities of dyes. And finally, digital cameras usually have more pixels on the chip and yield images with higher resolution, often at the resolution limits of the microscope.

If you are planning to conduct experiments in which you will be measuring the intensity values of the pixels in the image, you will need to use the constrained iterative algorithm for deconvolution, probably with a measured point spread function. This means you will most certainly need a cooled, digital camera for data acquisition.

Spectral response and sensitivity
Another important specimen-related issue to consider when selecting a camera for digital deconvolution is the spectral response of the camera. CCD chips have a characteristic response curve depending on the color of the light. For example, the Sony interline chips are more sensitive in the blue-green region of the spectrum, and less sensitive in the red spectrum. If you are doing calcium imaging, then you will probably favor a camera that uses the Sony chip. The Kodak chip, however, is more sensitive the red region. This camera is especially good for dyes like Cy3 and Cy5. You can check the spectral response curve of several cameras by consulting the spread sheet on our web site.

You should consider this issue carefully. If you have to integrate longer, then the specimen may bleach over time. This is especially true if you are attempting to acquire a stack of images for 3D reconstruction.

Speed
Camera acquisition and readout speed are other important issues. Different cameras have different readout speeds. If you are working with fixed specimens and dyes that do not photobleach easily, then you can use a camera with slower readout times. The analog cameras work at 30 frames/second, but have noisy images. The cooled digital cameras' readout speeds vary from 1 Mhz (considered slow) to 20 Mhz, or higher (for IEEE 1394 Firewire). A 1 Mhz camera will usually produce a full frame image (about 1 million pixels) in about 1.5 seconds. On the other end of the spectrum, the 20 Mhz, or faster, cameras can operate at 10 frames/second or better. Note that a faster readout increases the noise in the image. If you bin an image, or take a subregion, then the frame rates can approach 100 frames/second, or better.

If you are working with live specimens, doing calcium ratio imaging, or working with specimens that photobleach easily, you will want the greatest sensitivity and the fastest readout times.

Noise and bit depth
Your specimens are related to the noise and bit depth of the camera you choose. If your specimens are dim and you have to integrate a long time, or you have to capture images quickly, then the noise will be greater in the images. The noisier the image, the less accurate the deconvolution. To minimize camera noise under these circumstances, select a camera with the lowest cooling temperature, the largest well capacity, the lowest dark current and the lowest read noise.

It is sometimes difficult to compare cameras on this issue. There is considerable confusion as to the meaning of bit depth and noise. All chips have a bit depth rating from the manufacturer. For example the Sony interline and the Kodak KAF chips are both 12 bit chips. However, the effective bit depth of an image from these two chips is quite different. he Sony chip has a well depth of about 18,000 electrons. Given that it has a readout noise of about 9 electrons per well, the effective signal-to-noise ratio is about 18,000/9 or 2,000. This means the camera can produce an image with about 2000 shades of gray - or an image with a true bit depth of about 11 bits - not the rated 12 bits.

On the other hand, the Kodak chip has a well depth of 40,000 electrons and a read noise of about 10 electrons. This gives it a true bit depth of 40,000/10 = 4000 or 12 bits.

Visually, the images from these two chips are very similar. However, if you perform an exacting deconvolution process on images from these two cameras, you will see slight differences in the results. This is even more obvious if you are working with dim specimens and the dark current noise builds up over time and adds to the readout noise.

Thus, if you are working with very dim specimens, it is better to select a camera with a higher well depth, even if it means sacrificing readout speed.

Again, if you are working in brightfield, you may be able to use an analog camera, but with the caveats mentioned above.

Many customers are now attempting to use color cameras for multiple fluorescent dyes. For general imaging, color cameras are acceptable. However, VayTek is reluctant to recommend a color camera for most deconvolution applications. For color fluorescent imaging, and color deconvolution, it is VayTek's opinion that the best approach is to use a high-quality 12 bit, cooled digital camera and an automated filter wheel. There are several reasons for this.

All color cameras with filters use broadband filters for red, green and blue. On the other hand, a microscope with a monomchrome camera and a filter wheel captures color images that are more accurate because they use a filter wheel with the glass filters designed specifically for the wavelengths of your specimen dyes.

If you use a color camera to collect color fluorescent images you are adding additional filters in the optical path, which causes some additional loss of light and increases integration time. The exact amount of the loss depends on which color camera is used.

The color cameras generally take longer to capture three separate channels and merge them into a single color image than does an excitation filter wheel and 12 bit monochrome camera.

And finally, the color cameras are not as well adapted for precise fluorescent imaging and some of the demanding results required from color images. For example, the color cameras with mechanical filter wheels can introduce vibration into the camera and cause slight misalignment in the color channels. The Bayer filter approach results in a loss of resolution. The acousto-optic filter reduces light transmission by about 30%.

In short, for the purpose of deconvolution, you will have more control of the acquisition process, better control and more accurate images with a good monochrome camera and an excitation filter wheel.

The One-Camera-for-Everything Syndrome
Many customers try to buy a single camera that will serve all purposes. This is often done for budgetary considerations. They want a cooled camera that produces 12 bit monochrome images, or better, for accurate deconvolution. In addition, they want the camera to be fast enough for calcium or live imaging, and they want to do multichannel or color imaging. They often want the camera to double for brightfield applications. And most of all, they want to pay less than $5,000 for it.

Needless to say, such a camera does not yet exist. In fact, the biggest problem VayTek has had to contend with in integrating full systems is this one-camera-for-everything syndrome. The problem has become even more complex with the introduction of several less expensive color cameras that use the Sony interline chips. Some of these cameras are very good and are much more versatile than previous color cameras. However, each of these cameras still has several drawbacks that make them less than desirable for serious deconvolution applications.

Software
The second biggest issue affecting the choice of a camera, and the one that is most often ignored, or at least put off until the last, is the choice of software to be used with the camera.

When considering software, you need to determine if the program will control the camera properly and give you all the functionality you are looking for. For example, the camera may be rated at 10 frames per second, but if the software is not written to operate at that speed, then you will not be able to run the camera at 10 frames per second.

The next issue to consider when evaluating software is how comprehensive it is. Many camera companies offer software packages with their cameras. However, this software usually has minimal functionality. Most of these programs, for example, do not integrate z stage motors, shutters or filter wheels. More complex experiments will also need software control of triggers and annunciators.

The Buy-My-Own-Parts Syndrome
Many customers overlook these points. This leads to the buy-my-own-parts syndrome. When customers begin to shop for a system, they often concentrate on the hardware first. They believe they can save money by getting the best deal on a microscope, then the best deal on a camera, then the best deal on a computer, etc. After they have saved a few thousand dollars by getting the best deals, they then expect that all they need is a piece of software to capture some images and process them.

Unfortunately, this approach often ends up costing the customer more in the long run. The software is key to pulling all the parts of the system together. There are many details, problems and exceptions to getting complex hardware to function together as a system. The best approach is to determine the uses for the system, then talk to an expert about integrating a system around those uses. The most typical process is to determine the research issues and specimens. Then, settle on the microscope and then the camera. This will determine the type of computer, framegrabber board and operating system. Next, a complete understanding of the acquisition paradigms and the analysis software is needed. Finally, if any peripherals, such as filter wheels and stage motors, are needed they can be added to the mix.

It is important to work with someone who has a detailed understanding of microscopes, optics, cameras, microscope peripherals, computers, software, biomedical research, statistics and research design. Such a person will understand which camera is best for your research needs. They will also understand that picking a camera should come early in the process and will determine many of the other components.

25. I'm trying to calibrate the deconvolution software and I get error messages when using the measured PSF in the Volume Constrained Iterative to remove haze in the bead data set using the same bead data set.
Answer: When you deconvolve a bead scan with a bead scan, the theoretical answer is that there will be one pixel with all the energy and the rest will be perfectly black. This is called a delta function in math and like unicorns and fairies they exist in the mind, not in our world. The Constrained Iterative deconvolution may balk at trying to converge into a delta function. Deconvolution will also not work on an object that is infinitely thin (a few pixels wide). Using the Constrained Iterative algorithm for deconvolution will only work when the object to be deconvolved is much larger than the Point Spread Function (PSF). And the PSF must be finite in extent.

One question that is often asked by those who want to do deconvolution is "What resolution should I have for my camera?". This depends on your lens magnification, na, the wavelength of light, the camera chip dimensions, and the camera coupler.

where
d = the smallest resolvable distance between two points under the lens
Lambda = the wavelength of light
NA = the numerical aperture of the lens

The resolvable distance at a conjugate image plane where the camera chip is placed can be calculated with

where
D = is the resolution at the image plane
d = resolving power of the lens
M = the magnification of the lens

In order to get the maximum resolution in the image on the camera chip it is necessary to capture two pixels worth of data. The number of pixels per mm can be calculated as:

Where
R = the number of pixels required on the camera in the x or y direction
N = 4000/Lambda (in microns)
NA = the numerical aperture of the lens
M = the magnification of the lens
X = the chip size in the x direction in mm
Y = the chip size in the y direction in mm
T is the reduction or magnification factor of the camera coupler

For example, a 100x, 1.4 na lens projection onto a 2/3" (8.5 x 6.4 mm) camera chip, using a .63 camera coupler and red light (Lambda = .6) would require a camera with a resolution of 760 x 570 pixels.

CONVERTING SIGNAL TO NOISE FROM DB (DECIBLES)

Here's how you can convert Signal to Noise from dB (decibels) into ratios and back again.
Convert dB to ratio
Divide the dB number by 20. Use this number as an exponent of 10 to find the value of the ratio to 1.
Example: 60 dB / 20 = 3. 10^3 = 1000. Ratio is 1000:1

Convert ratio to dB
Find the log of the ratio and multiply by 20
Example: 2000:1 2000 = 10^3.301. 3.301 x 20 = 66 dB

See our definitions of camera specifications for more information relating to signal-to-noise, chip size, etc.

FAQ: Frequently Asked Questions

1. What is decovolution software?