WO2006125188A1 - Methods and systems for the analysis of 3d microscopic neuron images - Google Patents

Abstract

Methods and apparatus are provided for the automated analysis of 3D neuron images acquired by fluorescence microscopy techniques such as confocal laser scan microscopy and multi-photon laser scan microscopy. The new methods and apparatus can be used for the segmentation and detection of dendrites and dendritic spines, and for the extraction of biologically significant features, including dendrite number, length and volume, spine number and density, spine length, volume, shape, type, and the like.

Description

Methods and Systems for the Analysis of 3D Microscopic Neuron Images

Related Applications

[1] The present application claims priority to Provisional Application No. 60/682,510 filed on May 19, 2005 and entitled "Methods and Systems for the Analysis of 3D Microscopic Neuron Images". The Provisional Application is incorporated herein by reference in its entirety.

Background of the Invention

[2] Neurons are the basic information processing structures in the central nervous system. Each neuron forms thousands of connections with other neurons via axonal and dendritic arborizations. Such connections, known as synapses, are crucial to the biological computations that underlie perception and thought, and provide the means through which the nervous system connects to and controls the other systems of the body (e.g., muscles, glands, etc .). The dendritic tree represents the most substantial receptive area of the nerve cells. Dendritic arborizations receive and integrate incoming synaptic potentials with complex spatio-temporal patterning, and trigger the neuron to generate and transmit time-structured potentials via their axonal arborizations to local and remote target neurons.

[3] Recent research has revealed that morphological characteristics of neuronal structure are closely related to neural functions. Several investigations have described morphological changes in neurons that accompany neuronal activity, and identified a causal relationship for the observed correlations between dendritic geometry and neural properties (B.W. Connors and MJ. Gutnick, Trends Neurosci., 1990, 13: 99- 104; S. Fransceschetti et al, Brain Res., 1995, 696: 127-139; A. Mason et al, J. Neurosci., 1996, 16: 1904-1921). Both macroscopic branching topology (Z.F. Mainen and TJ. Sejnowski, Nature, 1996, 382: 363-366; P.C. Bressloff and B. De Souza, Physica D, 1998, 115: 124-144; P. Vetter et al, J. Neurophysiol., 2001, 85: 926-937; J.L. Krichmar et al, Brain Res., 2002, 941 : 11-28) and microscopic surface irregularities including densities, shapes and distributions of dendritic spines (E. Fifkova, Cell MoI. Neurobiol., 1985, 5: 47-63; CJ. Wilson, J. Electron. Microsc. Tech., 1988, 10: 293-313; W.R. Holmes, Brain Res., 1989, 478: 127-137; S.M. Baer and J. Rinzel, J. NeurophysioL, 1991, 65: 874-890; M.B. Moser et al, Proc. Natl. Acad. Sci. USA₃ 1994, 91: 12673-12675; N. Toni et al, Nature, 1999, 402: 421-425) are thought to contribute to the excitable properties of neurons.

[4] Dendritic spines are morphological specializations that protrude from the main shaft of neuron dendrites. They play an important role in synaptic transmission by creating a local synapse-specific compartment (G.M. Shepherd, J. NeurophysioL, 1996, 75: 2197-2210). Typically 0.5 to 2 microns in length, dendritic spines come in a wide variety of shapes, and are arbitrarily classified on the basis of their structure as filopodium, thin, stubby, mushroom and cup shaped as shown on Figure 1 (H. Hering and M. Sheng, Nature, 2001, 2: 880-888). Since modern biophysical techniques have begun to reveal the structure-function relationship of dendritic spines (H. Hering and M. Sheng, Nature, 2001, 2: 880-888; K. Kasail et al, Trends in Neurosc, 2003, 26: 360-368), research efforts have focused on obtaining detailed information regarding their morphology (G.N. Elston and M.G. Rosa, Cereb. Cortex, 1997, 7: 432-452; B. Jacobs et al, J. Comp. Neurol, 1997, 386: 661-680; G.N. Elston and M.G. Rosa, Cereb. Cortex, 1998, 8: 278-294; G.N. Elston et al, J. Comp. Neurol., 1999, 415: 33- 51; G.N. Elston et al, Proc. R. Soc. London, B, 1999, 266: 1367-1374; G.N. Elston et al, NeuroReport, 1999, 10: 1925-1929; G.N. Elston, J. Neurosc, 2000, 20: 1-4; B. Jacobs et al, Cereb. Cortex, 2001, 11: 558-571; G.N. Elston and K.S. Rockland, Cereb. Cortex, 2002, 12: 1071-1078; E.A. Nimchinsky et al, Annu. Rev. Physiol., 2002, 64: 313-352; M. Matsuzakil et al, Nature, 2004, 429: 761-766; K. Zito et al, Neuron, 2004, 44: 321-334).

[5] Such studies have revealed that many pathologies of cognitive function are associated with subtle changes of dendritic arborization and altered spine densities, shapes, and distributions. For example, dendritic spines are absent or their structures are grossly distorted in the brains of individuals suffering from a variety of neurological diseases, including developmental disorders that lead to mental retardation such as Down's syndrome, inherited metabolic diseases, fetal alcohol syndrome, and fragile X syndrome (M. Marin-Padilla, Brain Res., 1972, 44: 625-629; D.P. Purpura, Science, 1974, 186: 1126-1128; D.P. Purpura, Adv. Neurol, 1975, 12: 91-134M. Marin-Padilla, J. Comp. Neurol, 1976, 167: 63-81; M. Suetsugu and P. Mehraein, Acta Neuropathol, 1980, 50: 207-210; S. Takashima et al, Brain Res., 1981, 225: 1-21; L.E. Becker et al, Ann. Neurol., 1986, 20: 520-526; E. Galofre et al, J. Neurol. ScL, 1987, 81 : 185-195; S. Takashima et α/., Brain Dev., 1989, 11: 131- 133, I. Ferrer and F. Gullotta, Acta Neuropathol., 1990, 79: 680-685; VJ. Hinton et al, Am. J. Med. Genet., 1991, 41: 289-294; P.R. Huttenlocher, Pediatr. Neurol., 1991, 7: 79-85; A. Kamei et al, Pediatr. Neurol, 1992, 18: 145-147; T.A. Comery et al, Proc. Natl. Acad. Sci. USA, 1997, 94: 5401-5404; J. W. S warm et al, Hippocampus, 2000, 10: 617-625; S.A. Irwin et al, Am. J. Med. Genet., 2001, 98: 161-167; E.A. Nimchinsky et al, J. Neurosci., 2001, 21: 5139-5246; J.C. Fiala et al, Brain Res. Rev., 2002, 39: 29-54; E.A. Nimchinsky et al, Ann. Rev. Physiol, 2002, 64: 313- 353). Dendritic spines are also lost or distorted in the brains of patients after seizures and strokes and those suffering from dementia, brain tumors, depression, schizophrenia, and chronic alcoholism (M.E. Scheibel et al, Epilepsia, 1974, 15: 55- 80; J. Spacek, Acta Neuropathol., 1987, 73: 77-85; I. Catala et al, Hum. Neurobiol, 1988, 6: 255-259; I. Ferrer et al, J. Neurol. Neurosurg. Psychiatry, 1991, 54: 932- 934; P. Multani et al, Epilepsia, 1994, 35: 728-736; P. V. Belichenko and A. Dalstrom, J. Neurosci., 1995, 57: 55-61; M. Jiang et al, J. Neurosci., 1998, 18: 8356- 8368; R. Yuste and T. Bonhoeffer, Ann. Rev. Neurosci., 2001, 24: 1071-1089; R. Yuste and W. Denk, 1995, 375: 682-684).

[6] Although the morphological complexity of dendritic and axonal arborizations is known to play an important role in the signal transformation in the neurons and neural networks, and thus in information processing in the central nervous system, it is still far from being understood. A better understanding of the relationship between neuronal structure and neurological changes is not only of great academic interest, it is also crucial to the development of therapies and drugs for the prevention and/or treatment of neural disorders (B.T. Hyman et al, Ann. N.Y. Acad. Sci., 1991, 640: 14-19). Determining exactly how neuron morphology relates to neuronal function requires techniques that can image both global dendritic structure and detailed spine geometry.

[7] Modern fluorescence microscopy methods, such as confocal laser scan microscopy and two-photon laser scan microscopy, provide powerful tools to study dendrites and dendritic spines (B. Lendvai et al, Nature, 2000, 404: 876-881). However, analysis of neuron images generated by these techniques has remained largely manual, extremely time-consuming, and subject to human bias (B. Jacobs et al, J. Comp. Neurol., 1997, 386: 661-680; B. Jacobs et al, Cereb. Cortex, 2001, 11 : 558-571; G.N. Elston and K.S. Rockland, Cereb. Cortex, 2002, 12: 1071-1078; E.A. Nimchinsky et al, Annu. Rev. Physiol., 2002, 64: 313-352; M. Matsuzakil et al, Nature, 2004, 429: 761-766; K. Zito et al, Neuron, 2004, 44: 321-334). Although manual analysis can yield certain features, such as number of dendrites and spines, it is difficult to manually extract other features such as spine volume and surface size. Another disadvantage of manual analysis is that it is not applicable to the study of the large amount of data generated by digital microscopy.

[8] There clearly remains a need for automated systems for the analysis of neuron microscopic images. In particular, tools for accurate and efficient description of neuronal morphological complexities are highly desirable.

Summary of the Invention

[9] The present invention relates to a new, powerful class of informatics tools for efficient neuron images studies. More specifically, improved systems and strategies are described herein that can be used to automatically or semi-automatically process and analyze neuronal image stacks (i.e., 3D images) obtained by optical microscopy. In particular, the present invention provides processes and apparatus with increased capacity to detect and extract neuron components, including axonal arborizations, dendrites, and dendritic spines, and to calculate biologically significant features of dendrites and spines, such as number, density, length, volume, cross-section, surface area, shape, and the like. In addition to allowing large quantities of data to be processed in a short time, thus saving users tremendous time in interpreting microscopic neuron images, the inventive systems reduce human bias to the minimum.

[10] More specifically, the present invention provides a method for characterizing one or more neurons, wherein each neuron comprises one or more dendrites and each dendrite comprises one or more spines. The inventive method comprises steps of: receiving a neuron image showing at least one neuron or part of one neuron; performing a segmentation analysis of the neuron image to obtain a segmented digital image; and extracting one or more parameters from the segmented digital image to characterize the at least one neuron. In certain preferred embodiments, the neuron image is a 3D neuron image, for example, a 3D neuron image obtained using confocal laser scan microscopy or multi-photon laser scan microscopy.

[11] In some embodiments, the inventive method further comprises steps of: processing the neuron image using a median filter to remove noises and obtain a noise-free neuron image; and deconvoluting the noise-free neuron image to restore image distortion to obtain a processed neuron image prior to performing the segmentation analysis. For example, the median filter may be a center-weighted median filter.

[12] The step of performing a segmentation analysis may comprises: applying an unsharp masking filter to the neuron image to obtain a local contrast image; and applying a global threshold process using a preset brightness level to the local contrast image to separate dendrite from background. The segmented digital image obtained comprises a representation of the at least one neuron, each representation comprising a collection of signal intensity values at positions in the image where the neuron is present.

[13] In some embodiments, the inventive method further comprises a step of applying a high level threshold process to the local contrast image to extract dendrite backbone. For example, applying a high threshold process may be performed by using a medial-axis transformation.

[14] In certain embodiments, the inventive method further comprise removing false positive signals from the local contrast image. This can be achieved by: applying a low level threshold process to the local contrast image to obtain a low level processed image showing dendrites, detached objects, attached objects and smears; measuring, for each detached object, the distance between the detached object and the nearest dendrite backbone; and identifying the detached object as a false positive signal if the distance measured is higher than a present distance value D; and measuring, for each attached object of intensity below the preset brightness level used in the global threshold process, the distance between the attached object and the nearest dendrite backbone in the z-direction; and identifying the attached object as a false positive signal if the distance measured is larger than 2 layers; thereby obtaining a processed image showing dendrites, detached objects and attached objects but no false positive signals except for smears. The preset distance value D may be chosen to be higher than the length of the longest spine.

[15] In some embodiments, the inventive method further comprises a step of detecting spines in the processed image with no false positive signals except for smears. Detecting spines may comprise steps of: for each detaches object, measuring the length Li of the detached object in the direction parallel to the nearest dendrite backbone; measuring the length L₂ of the detached object in the direction perpendicular to the nearest dendrite backbone; and identifying the detached object as a smear if Li>∑2 and as a spine head if Li<∑2; and for each object attached to a dendrite, applying a grassfire process from dendrite backbone to dendrite surface to determine voxels of the attached object assigned the locally largest distances by the grassfire process and identifying these voxels as spine base tip; applying a reverse grassfire process from spine base tip to dendrite backbone to obtain sets of voxels of the attached object that are assigned an identical integer by the reverse grassfire process; determining, among the sets of voxels, the set of voxels that comprises the largest number of voxels; determining, within said set of voxels, the voxel whose distance to the dendrite backbone is the largest; andidentifying the segmentation plane between the spine and dendrite as the plane passing through said voxel and parallel to the dendrite backbone.

[16] In certain embodiments, the inventive method further comprises a step of merging spine head and spine base that belong to the same spine. The step of merging spine head and spine base that belong to the same spine comprises steps of: calculating the center of mass, Ri, for each spine head; calculating the center of mass, R₂, for each spine base comprising a spine base tip and a ring of spine-surface boundaries points, measuring the distance between the center of mass Ri and a spine head and the center of mass R₂ of a spine base; and merging the spine head and spine base if the distance measured is lower than a present distance value D ' and the tip of the spine base lies within a cone determined by the center of mass of the spine head and the ring of spine-surface boundaries points. The preset distance value D ' is chosen to be higher than the length of the longest spine. [17] In some embodiments, the step of extracting one or more parameters from the segmented digital image to characterize at least one neuron comprises determining one or more features selected from the group consisting of dendrite number, dendrite density, dendrite length, dendrite volume, dendrite shape, dendrite location, spine number, spine density, spine location, spine length (l_s), spine volume, spine surface area, spine head diameter (d_/i), spine head width, spine neck diameter (d_n), and spine neck width.

[18] In certain embodiments, the inventive method further comprises classifying at least one spine of a dendrite as fϊlopodium, thin, stubby, mushroom-shaped or cup- shaped. Classifying may comprise calculating ratios of spine length (I₁), spine head diameter (d_h), and spine neck diameter (Rvalues for said spine.

[19] The one or more neurons in the neuron images may be primary neurons, secondary neurons or immortalized neurons. In certain embodiments, the neurons are human neurons. The neurons imaged may have been treated under controlled conditions, for example, they may have been treated with a test agent.

[20] In another aspect, the present invention also provides machine-readable media on which are provided program instructions for performing one or more of the inventive processes of neuron image analysis. In still another aspect, the present invention provides computer products comprising a machine-readable medium on which are provided program instructions for performing one or more of the inventive processes. In yet another aspect, the present invention provides a neuron image analysis apparatus comprising a memory or buffer adapted to store, at least temporarily, one or more images acquired by optical microscopy, and a processor configured or designed to perform one or more of the inventive processes.

[21] These and other objects, advantages and features of the present invention will become apparent to those of ordinary skill in the art having read the following detailed description of the preferred embodiments.

Brief Description of the Drawing

[22] Figure 1 shows a scheme illustrating the morphological classification of dendritic spines. [23] Figure 2 is a process flow diagram depicting, at a high level, the system architecture of one embodiment of the inventive method of neuron image analysis.

[24] Figure 3 shows two sets of neuron images submitted to image processing according to the present invention. (A) Raw images; (B) Images obtained after noise removal; and (C) images obtained after deconvolution (5 iterations).

[25] Figure 4 is a process flow diagram depicting, at a high level, the system architecture of one embodiment of the analysis step of the inventive neuron image analysis involving (A) dendrite segmentation and dendrite backbone extraction; (B) false positive signals removal; and (C) spine detection.

[26] Figure 5 shows two sets of images illustrating the problems arising when applying global thresholding on neuron images. (A) Grey level images; and images obtained after (B) segmentation using threshold = 30; (C) segmentation using threshold = 60; (D) segmentation using threshold = 90; and (E) segmentation using threshold = 120.

[27] Figure 6 shows two sets of images illustrating dendrite segmentation based on local contrast map using different threshold values according to the inventive method. (A) Grey level images; (B) corresponding local contrast maps; and images obtained using (C) threshold = 2; (D) threshold = 4; (E) threshold = 6; (F) threshold = 8; and (G) threshold = 10.

[28] Figure 7 shows two sets of images illustrating image segmentation results before and after false positive signals removal. (A) Images superimposed with dendrite backbones; (B) binarized image obtained using with threshold = 4; and (C) binarized image from (B) after false positive signals removal.

[29] Figure 8 is a picture showing several examples of detached spine heads and smears.

[30] Figure 9 is a scheme showing the segmentation of a theoretical attached spine according to the present invention.

[31] Figure 10 presents 4 graphs showing the intersection plane area changes along the center line of the following types of spines: (A) filopodium spine; (B) thin spine; (C) Mushroom spine; and (D) Stubby spine. [32] Figure 11 is a scheme presenting the pipeline of NeuronlQ, one embodiment of the present invention. The pipeline consists of data acquisition from both optical microscopy experiments of neuronal morphology and associated meta-data of genetic information, biological background, and experimental protocols about each image; image pre-processing; image segmentation; extraction of dendrite backbone; spine detection; feature extraction; and statistical analysis. The database includes a relational database and an image repository.

[33] Figure 12 shows two images comparing the spine detection results obtained (A) by manual detection and (B) using NeuronlQ detection.

[34] Figure 13 is a set of two graphs showing spine length measurements comparison (i.e., as obtained by NeuronlQ and by manual analysis) for (A) apical dendrites and (B) basal dendrites.

[35] Figure 14 is an image showing the results of recursive thinning performed using NeuronlQ to extract the backbone of dendrites.

[36] Figure 15 is a set of images. (A) shows a grayscale neuronal image after MIP, (B) shows the segmented image, (C) shows the image obtained using NeuronlQ, wherein spines are marked, and (D) shows the same image where spines are detected manually. All the processing is performed in 3D images; it is only for the purpose of illustration that the results are shown in 2D.

[37] Figure 16 shows a schematic illustration of a spine length, diameter of the spine head and spine neck.

[38] Figure 17 is a set of 4 graphs showing the results of a Kolmogorov-Smirnov test of spine lengths obtained by manual analysis and NeuronlQ from four images stacks (A-D). Blue line, results of NeuronlQ, red line, results of manual analysis. (A) Manual analysis and NeuronlQ detect 83 and 85 spines respectively, (B) manual analysis and NeuronlQ detect 106 and 104 spines respectively, (C) manual analysis and NeuronlQ detect 59 and 58 spines respectively, (D) manual analysis and NeuronlQ detect 78 and 76 spines respectively.

[39] Figure 18 is a graph showing the results of manual analysis and NeuronlQ used to study the effects of TSC2 loss on spine density. [40] Figure 19 presents the tree structure of organizing the neuron image database ofNeuronlQ.

Detailed Description of Certain Preferred Embodiments

[41] Improved systems and strategies for neuron images analysis are described herein. As mentioned above, the present invention more specifically relates to processes (methods) and apparatus with increased capacity to detect and extract neuron components (including dendrites and dendritic spines) and the ability to identify, analyze and quantitate morphological features of these neuron components from large amounts of images acquired using optical microscopy. Furthermore, based on certain extracted features, the inventive processes and apparatus can classify dendritic spines based on their structure. The present invention also relates to machine-readable media on which are provided program instructions, data structures, etc, for performing one or more of the inventive processes.

[42] A high level process flow diagram in accordance with one embodiment of the present invention is depicted in Figure 2. Each step or module of the inventive process is described in detail below.

I - Neuron Images

[43] As shown on Figure 2, the inventive image analysis process starts when one or more image analysis tools (typically logic implemented in hardware and/or software) obtain one or more neuron images showing dendrites and dendritic spines of one or more nerve cells. In certain embodiments, a neuronal image stack (i.e., a 3D image) is obtained at the beginning of the image analysis process.

[44] The image(s) provided at the start of the invention process is/are recorded by an image acquisition system (e.g., a confocal laser scan microscope or multi-photon laser scan microscope). The image acquisition system may be directly coupled with the image analysis tool of the present invention. Alternatively, the image(s) to be processed and analyzed are provided by a remote system unaffiliated with the image acquisition system. For example, the images may be acquired by a remote image analysis tool and stored in a database or other repository until they are ready to be analyzed by the image analysis processes/apparatus of the present invention.

[45] Imaged neurons to be analyzed may be of animal (e.g., primate, dog, cat, goat, horse, pig, mouse, rat, rabbit, and the like) or human origin. Preferably, the neuron images used in the present invention are images of living cells. The terms "live cell" and "living cell" are used herein interchangeably. They refer to a cell which is considered living according to standard criteria for that particular type of cell, such as maintenance of normal membrane potential, energy metabolism, or proliferative capability.

[46] In certain embodiments, neuron images are taken from brain slices (i.e., thin sections of living brain tissue that can be maintained in vitro). Methods for the preparation of brain slices are well known in the art and have been used and reported for all kinds of mammal species, including human (see, for example, D. M. Kacobowitz et al, Brain Res. Bull, 1994, 33: 461-463; M.E. Rice, Methods, 1999, 18: 144-149; T. Wang and LS. Kass, Methods MoI. Biol, 1997, 72: 1-14; R.W. Verwer et al., J. Cell MoI. Med., 2002, 6: 429-432). After removal of the brain from the body, a vibratome or a motorized tissue chopper is used to cut brain slices. Slices can be cut from an isolated portion of the brain (e.g., the hippocampus), or slices can be made of the entire hemibrain. Glutamate antagonists or low-calcium buffer solutions are sometimes employed during slice preparation in order to reduce glutamate-mediated toxicity. After preparation, brain slices are generally placed in an artificial cerebrospinal fluid in a warm oxygenated chamber until use.

[47] Alternatively, images are taken from an assay plate or other cell support mechanism in which neurons are growing or stored. In this case, cells may be primary cells, secondary cells or immortalized cells (i.e., established cell lines). They may have been prepared by techniques well known in the art (for example, isolation from brain tissue) or purchased from immunological and microbiological commercial resources (for example, from the American Type Culture Collection, Manassas, VA).

[48] In certain embodiments, the images used as the starting point of the inventive methods of analysis are obtained from neurons that have been specifically treated and/or imaged under conditions that contrast markers of cellular components of interest from other cellular components and from the background of the image. Preferably, the neurons are specifically treated and/or imaged under conditions that contrast dendrites and dendritic spines from other cellular components and/or the background of the image. For example, images may be obtained of neurons that have been treated with a chemical agent that specifically renders visible (or otherwise detectable in a region of the electromagnetic spectrum) the neurons' dendrites and dendritic spines. Common examples of such agents are colored dyes or fluorescent compounds that bind directly or indirectly (e.g., via antibodies or other intermediate binding agents) to the dendrites.

[49] Alternatively, the nerve cells may have been genetically engineered to express a gene encoding a fluorescent marker, such as the green fluorescent protein, GFP (or any of its derivatives). As a general rule, transgenic expression of GFP within any given cell requires simply placing the GFP coding sequence (or slightly modified versions of the sequence) under the transcriptional control of appropriate regulatory sequences. GFP and its derivatives as well as methods for genetically engineering cells to express these biomolecules are well known in the art.

[50] Preferably, the agent or marker is selected such that it generates a detectable signal whose intensity is related (e.g., proportional) to the amount of cell component to which it is bound. Since the absolute magnitude of signal intensity can vary from image to image due to changes in the cell staining/engineering and/or acquisition procedure and/or apparatus, a correction algorithm may be applied to correct the measured intensities, if desired. Such algorithms can easily be developed based on the known response of the optical system used under a given set of acquisition parameters.

[51] In certain embodiments, the images to be used in the methods and apparatus of the present invention are acquired by fluorescence microscopy techniques such as confocal laser scan microscopy and multi-photon laser scan microscopy. These techniques are particularly useful as they enable visualization deep within both living and fixed cells and tissues and afford the ability to collect sharply defined optical sections from which three-dimensional renderings can be created.

[52] Confocal laser scan microscopy and multi-photon laser scan microscopy are similar in many respects. However, there also exist differences between these techniques. In both methods, a beam of laser light is focused into a small point at the focal plane of the specimen to be imaged, for example inside a cell loaded with a probe. A computer-controlled scanning mirror can move or scan this beam in the xy direction at the focal plane. The fluorescence emission created by the point as it is scanned in the focal plane is detected by a photomultiplier tube (PMT), and this detection input is then reconstructed by computer hardware into an image. Among the most important differences between multi-photon and confocal laser scan microscopy is the type of laser utilized in these techniques. Lasers commonly employed in confocal microscopy are high-intensity monochromatic light sources such as argon-ion lasers, krypton-ion lasers or air-cooled lasers using argon-krypton mixtures. The basic key to the confocal approach is the use of spatial filtering techniques {e.g., involving a pin hole placed between the detector and the specimen being imaged) to eliminate out-of-focus light and glare in specimens whose thickness exceeds the immediate plane of focus. This combination leads to the creation of a sharp image or optic section of the situation inside the specimen. In multi-photon scanning microscopy, solid state pulsed lasers are used (e.g., titanium sapphire lasers). Such lasers emit light off at a longer wavelength than the gas lasers used in confocal microscopy. Thus, the energy of the light is lower and absorption of more than one photon is required for the probe to be promoted to an excited state (e.g., in two-photon laser scan microscopy, excitation of the probe to an excited state requires absorption of two photons). Since the probability of a multi-photon event is extremely low and occurs only where the laser light is the most intense (i.e., at the focal point), essentially all the fluorescent light comes from the focal point of interest and no pin hole is required.

[53] Confocal laser scan microscopy has the advantage that the lasers used in this technique are considerably less expensive and much easier to operate than the light sources employed in multi-photon laser scan microscopy. On the other hand, the light of longer wavelength (i.e., of lower energy) used in multi-photon microscopy is inherently less damaging to biological material and penetrates deeper into the sample than light of shorter wavelength used in confocal microscopy (S. L. Murov et ah, "Handbook of Photochemistry" , 2^nd Ed., 1993, Marcel Dekker; K.C. Smith (Ed.), "The Science of Photobiology" , 2^nd Ed., 1989, Plenum). Furthermore, multi-photon microscopy is more sensitive than confocal microscopy because all the fluorescence light generated to make an image is sent directly to the detector. [54] Selecting an appropriate imaging technique to acquire neuron images to be analyzed by the processes and apparatus of the present invention can easily be performed by one skilled in the art and will depend on several factors including, but not limited to, the nature of the specimen to be imaged (e.g., brain slice vs. cells in culture dish).

II - Image Processing - Noise Removal and Deconvolution

[55] After one or more neuron images (e.g., a 3D image) has/have been obtained, the image analysis tool of the present invention performs an image processing step. Image processing aims at producing clean and clear output images that are ideally noise-free and have little or no artifacts. Any method that leads to clean and clear output images may be used in the practice of the present invention to process neuron images. In certain preferred embodiments, the neuron images are processed by using first a median filter for removing noises, and then a deconvolution processing to restore image distortion.

[56] The raw image (i.e., as obtained by the image acquisition system) can be considered as a blurred version of the real image plus noise. More specifically, considering the physics of imaging, the observed microscopy image can be modeled as follows:

X(x,y,z) = UJ Kx - ξ, y - η, z - 0 S(ξ,η,Q dξdηdζ + N(x,y,z) (1)

[57] where X is the observed 3D image, h is the 3D point spread function, S is the true image, and N is the noise, and where it is assumed that N is statistically independent of A and S. In discrete form, Equation (1) can be written as:

X= h ® S + N (2)

[58] where "®"stands for convolution.

[59] In confocal and multi-photon laser scan microscopy techniques, noise generally results from the imaging mechanism of the photomultiplier tube (PMT) used as detector. A PMT is a high gain, low sensitivity device which, combined with the low number of photons emitted by small structures such as dendritic spines, produces "photon shot noise". To remove this type of noise, a non-linear filter such as a median filter is generally applied (J.W. Tukey, "Exploratory Data Analysis^'", 1971, Addison- Wesley: Reading, MA). Standard median filter works by scanning an image using a filter window (typically 3 x 3, 5 x 5 or 7 x 7 pixels in size). In median filtering, each pixel is determined by the median value of pixels in the filter window.

[60] For a 3D image, center-weighted median filter (CWMF) is preferably used to remove photon shot noise. The filter window W used to scan the 3D image is defined in terms of the image coordinates in the 3D neighborhood centered at the current voxel. For example, a (2L+1) x (2Z+1) x (2Z+1) cubic window is given by:

W= {(r\s\f )|- L < f < L, - L ≤ s' ≤ L, - L ≤ f ≤ L} (3)

[61] If the input and output are denoted as {A(.,.,.)} and (F(.,.,.)}, respectively, then:

F{r,s,t) = median {A(r-r\s-s\t-f )\(r\s\f )e W]. (4)

[62] Though effective at removing shot noise, a median filter tends to destroy fine image details such as thin lines and edges. Loss of image details creates difficulties for image analysis involving classification of fine structures such as spine shapes. To overcome this problem, center-weighted median filter (CWMF) assigns a non- negative weight to the center pixel, and then finds the median value of this expanded input. For a CWMF with a filter window as defined in Equation (3) and a central weight of 2X+1, the operation may be described as follows (T. Chen and H.R. Wu, Signal Proc. Letters., 2001, 8: 1-3):

F(r,s, ή = median {A{r-r\s-s\t-f ), 2K copies of A(r,s,f)} . (5)

[63] When 0 < K≤ L, CWMF can remove shot noise as well as preserve image details (S.-J. Ko and Y.H. Kee, IEEE Trans. Circuits and Systems, 1991, 38: 984- 993).

[64] Figure 3(A) shows the maximum intensity projections of two neuron images, and Figure 3(B) shows the corresponding images obtained after median filtering process. The median filter effectively removes the noise while keeping the dendrites and spines. A deconvolution process may then be applied to the resulting image to restore image distortion caused by the optical point spread function. Any suitable deconvolution system may be used in the practice of the present invention. For example, the present Applicants used a deconvolution package, AutoDeblur (AutoQuant Image, Inc., Troy, NY). Figure 3(C) shows two images obtained after 5 deconvolution iterations performed on the images presented in Figure 3(B).

Ill - Image Analysis

[65] The objective of image analysis is to obtain quantitative information from an image (or image stack) such as number, sizes and shapes of objects. Following image processing as described above, image analysis classifies and extracts biologically significant features about neurons and neuron components from the studied images. These features include, but are not limited to, dendrites and/or spines length, volume, surface area, density, and the like. Dendrite segmentation and spine detection are critical parts of an automated neuron images analysis system as the quality of dendrite segmentation and spine detection affects the accuracy of the subsequent measurements of dendrites and spines features.

[66] A high level process flow diagram in accordance with one embodiment of the inventive image analysis method is depicted in Figure 4. Each step or module of the inventive image analysis process is described in detail below.

Dendrite Segmentation

[67] Segmentation, which transfers a grey-scale image to a black-and-white image, is generally the first step performed by the inventive image analysis process.

[68] For images of homogeneous brightness or close to homogeneous brightness, global thresholding can effectively separate objects from background. However, for images of inhomogeneous brightness, such as neuron images that contain both large contrast variations and low contrast details, application of global thresholding leads to sub-optimal results (J.C. Russ, J. Comp. Assis. Micro., 1995, 7: 141-161). Figure 5, which shows examples of dendrite segmentation obtained using different threshold levels, illustrates the problems arising when applying global thresholding on neuron images. At low threshold levels, dendrite and bright spines become very thick and short spines are inseparable from dendrites, while at high threshold levels, most small spines are lost. [69] Several groups have developed alternative methods for the segmentation of images of inhomogeneous brightness. For example, Weaver and coworkers (C. Weaver et al, Neural Comput, 2004, 16: 1353-1583) employed an indicator kriging technique (W. Oh and W. Lindquist, IEEE Trans. Pattern Anal. Mach. Intell., 1999, 21: 590-602) to find local threshold for neuron image segmentation. However, in order to obtain good segmentation results, this method requires manual selection of the image to be krigged. Several local threshold techniques have been developed for medical image segmentation, in particular for images obtained by digital mammograms. For example, Nishikawa and coworkers have used a global grey-level threshold in a first step, and then a local adaptive thresholding technique in a second step (R.M. Nishikawa et al, Med. Phys., 1993, 20: 1661-1666). Chan and coworkers have applied a local grey level thresholding technique, where the local threshold is varied using the standard deviation of the surrounding pixel values (H.P. Chan et al, Invest. Radiol., 1990, 25: 1102-1110). Kegelmeyer and coworkers have analyzed six different algorithms, including three algorithms based on grey level thresholding and three algorithms that use local contrast estimation (W.P. Kegelmeyer et al, in "Digital Mammography", 1994, Elsevier: Amsterdam, pp. 3-12).

[70] The present invention provides an improved segmentation method particularly suited to the segmentation of neuron images. As shown on Figure 4(A), the inventive segmentation method starts by applying an unsharp masking technique to obtain the local contrast images and then a global thresholding technique is performed to separate neurons from their background.

[71] Unsharp masking techniques are known in the art and have originally been used in the photographic darkroom. Digital unsharp masking filters stimulate the effect of the true photographic technique, which creates a blurred version of the image and then subtracts it from the original to create a sharper, more detailed image. In the field of clinical imaging, unsharp masking technique (A.R. Cowen et al, SPIE Image Processing, 1993, 1898: 833-843) has been used, for example to reduce the large contrast variations in 2D mammograms (WJ. Vedkamp and N. Karssemeijer, IEEE Trans. Med. Imaging, 200, 19: 731-738; S. Dippel et al, IEEE Trans. Med. Imaging, 2002, 21: 343-353). The present invention extends the use of this method to process and analyze neuron 3D images. In the inventive process, the following equation is employed to calculate local contrast C₁ at site i:

[72] where y, is the grey level at site i and di represents a neighbor at site i of size

N.

[73] In confocal laser scan microscopy and multi-photon laser scan microscopy, the xy and z directions are not equally sampled. The sampling space in the z direction is usually 10 times larger than in the xy direction. Selection of d( is based on consideration of the voxels in the layers located nearby site /. For example, when the window size in the xy plane is experimentally determined to be 9 x 9, then 9 x 9 x 3 is used for di (N = 243). As shown in Equation (1), each voxel is normalized by subtracting the result of moving-average filter fromjμ,-.

[74] A global threshold process is then applied to the local contrast image resulting from unsharp masking, in order to separate dendrites from their background. Thresholding is based on simple, well-known concepts. A parameter, called the brightness threshold is chosen and applied to each voxel of the image under consideration as follows: (1) if the intensity of the voxel is higher than the brightness threshold, the voxel is considered as belonging to an object (e.g., a dendrite); (2) if the intensity of the voxel is lower than the brightness threshold, the voxel is considered as belonging to the background.

[75] Voxels within big dendrites that have low local contrast may be removed by application of a global threshold, as high intensity voxels are considered as dendrite even if their contrast is very small, while low local contrast voxels are usually considered as background. To prevent this, the segmentation results are compared to the deconvoluted grey level image and voxels whose values are above a preset dendrite threshold are included in the resulting image.

[76] Figure 6 shows examples of dendrite segmentation obtained at different threshold levels. The segmentation at threshold 2 contains many false positive signals. At threshold 10, most of these false positive signals are removed. The shape of dendrite does not change too much at all as the threshold level is varied. Although spines lose some voxels at a higher threshold level, they still contain enough voxels to be identified as spines. To achieve an accurate segmentation, the segmentation process of the present invention uses a low threshold level to keep most of the spine voxels in the image, and a separate routine is provided herein to remove false positive signals.

[77] False positive signals come from different sources such as disconnected dendrites, axons, and smears caused by the optical point spread function. Disconnected dendrites and axons have the same intensity and contrast as the dendrite being analyzed. Fortunately, they are generally separated in space from the dendrite of interest. Thus, disconnected dendrites and axons can be removed by comparing their distance to the dendrite of interest. Most of the smears appear along the optical axis at different layers from the dendrite. They can be removed by comparing their distance to the dendrite of interest in the z direction. Since the position of the dendrite is critical in removing false positive signals, the dendrite backbone is first extracted.

Dendrite Backbone Extraction

[78] In the inventive process, dendrite backbones are extracted by applying a high threshold to the local contrast image (see Figure 7(A)). Preferably, a medial axis algorithm (T.C. Lee et al, CVGIP: Graph Models Image Proc, 1994, 56: 462-478) is used to obtain the underlying line-branch structure of dendrite. A medial-axis transformation is generally useful in thinning a polygon, or, in other words, finding its skeleton. The medial axis transform of a neuron image contains the skeleton of all dendrites and big spines, including the skeleton of disconnected dendrites and spurs. Preferably, short medial axes are removed and spurs are trimmed. After completion of this process, only the major medial axes, which correspond to dendrite backbones, are left in the image. Figure 7(A) shows examples of dendrite backbones superimposed with the corresponding binary dendrite images.

False Positive Signals Removal

[79] To keep all spine voxels, a low threshold is applied to the local contrast image

(see Figure 7(B)). All the objects present in the image obtained after this low level thresholding, are then analyzed with regards to their distance to dendrite backbones and then separated into different groups (Figure 4(B)).

[80] In the case of detached objects, their distance to the nearest dendrite backbone is measured and the object is considered as a false positive signal if the distance is higher than a preset value. This preset value may be selected such that it corresponds to a distance at which no dendritic spine can be found protruding out of a dendrite shaft. For example, for 0.5 to 2 micron-long spines, such a preset value is preferably chosen to be > 2 microns. In the examples presented on Figure 7, a preset value of 5 microns was used as no spines can protrude out of the dendritic surface more than 5 microns.

[81] Attached objects (or dendrite voxels) are separated according to their intensity level. If the intensity level of a dendrite voxel is below the dendrite intensity threshold (i.e., the brightness level used in the global thresholding technique described above), its distance to the dendrite backbone is measured in the z-direction, and it is considered as a false positive signal if this distance is larger than 2 layers.

[82] Figure 7(C) shows examples of neuron images after removal of false positive signals. After these correction processes have been performed, the only false-positive signals remaining in the image under consideration are smears located within the layer nearby the dendrite backbone. As detailed below, in the analysis process of the present invention these smears are removed during spine detection.

Spine Detection

[83] After dendrites have been segmented, spine detection can be performed.

[84] As reported in the literature, different methods have been applied to perform spine detection. For example, medial axis construction has been used for spine detection (D. A. Rusakov and M.G. Stewart, J. Neurosci. Methods, 1995, 60: 11-21; R. Watzel et al, DAGM-Symposium Bielefeld, 1995, 160-167; A. Herzog et al, Proc. SPIE, 3D Microscopy: Image Acquisition and Processing IV, 1997, 146-157) where spines are identified as the spurs in medial axes. The main drawback of this method is that spine voxels are not separated from dendrite axes. Thus, no further spine parameters except for length can be measured. Koh and coworkers (I. Koh et al, Neural Comput, 2002, 14: 1283-1310) have used protrusion criteria to separate spine voxels from dendrite voxels. However, their method cannot tell the exact segmentation plane between spines and dendrites.

[85] The main difficulty of spine segmentation comes from the shape variation of different spine types as well as dendrite and spine surface irregularities. Mushroom spines are characterized by big heads and thin necks. Their necks are generally too thin to be detected by light microscopy. In the binary dendrite images, they appear as detached head-base pairs. Stubby spines, on the other hand, have very thick necks. Small stubby spines may only protrude out of the dendrite surface by 0.2 microns, which makes them look very similar to dendrite surface irregularities. Quantification of spine parameters such as length, volume, head width, and neck width requires an accurate segmentation between spine and dendrites. The ideal segmentation plane is the dendrite surface. However, it is generally very difficult to estimate the dendrite width locally due to dendrite and spine surface irregularities.

[86] Several multi-level threshold techniques have been developed to detect the transitions from within an object into the background in 2D medical image segmentation. For example, Zheng and coworkers (B. Zheng et ah, Acad. Radiol., 1995, 2: 959-966) used three coarsely spaced thresholds and analyzed features such as size growth and central position shift to determine the regions that best represent the target object. Shiftman and coworkers (S. Shiftman et al, IEEE Trans. Med. Imaging, 2000, 19: 1064-1074), and Amit (Y. Amit, IEEE Trans. Med. Imaging., 1997, 16: 28-40) have applied multi-level threshold and then analyzed the contours pattern to determine the best segmentation point between objects and their background in medical images. The present invention provides a similar method to detect the transition point from spine to dendrite in order to obtain an accurate segmentation between them. As demonstrated herein, spine detection according to the inventive process allows several spine features to be measured.

[87] With respect to their connection to the dendrite under consideration, spine voxels belong to two types of spine components: detached spine components and attached spine components. The present invention provides different methods to detect and identify each type of spine component (see Fig. 4(C)). After detection and identification, a post-processing step is used to merge those spine components that belong to the same spine.

[88] Detection of Detached Spine Component. Dendrite voxels disconnected from the dendrite under consideration are tentatively identified as detached spine heads. As already mentioned, after the image processing step described above has been performed, the remaining false positive signals are smears located in the layers nearby the dendrite backbone. Figure 8 shows several examples of detached spine heads and smears. Smears mostly appear to be parallel to the dendrite backbone and they are longer in the direction of the dendrite backbone and shorter in the direction perpendicular to the dendrite backbone. On the other hand, spines are longer in the direction perpendicular to the dendrite backbone and shorter in the direction parallel to that dendrite backbone.

[89] In the method provided herein, the shape of a detached spine candidate is analyzed and classified based on this observation. A detached spine candidate is identified as a smear if its length in the direction parallel to the dendrite backbone is longer {e.g., 1.2 times longer) than its length in the direction perpendicular to the dendrite backbone. Alternatively, the detached spine candidate is identified as a detached spine component if its length in the direction perpendicular to the dendrite backbone is longer than its length in the direction parallel to the dendrite backbone.

[90] Attached Spine Component Detection. The most difficult part of spine detection is to distinguish attached spine components from dendrite components. Ideally, the segmentation plane between spines and dendrites is the dendrite surface. However, in practice, dendrite surfaces are quite irregular and thus it is generally very difficult to give an accurate estimate of the local dendrite thickness.

[91] A new method is provided herein to detect the transition from spine to dendrite. Starting from dendrite backbone to dendrite surface, each voxel in the dendrite is first labeled with an integer time step using a grassfire technique (R. Leymarie and M.D. Kevine, IEEE Trans. Pattern Anal. Mach. Intell, 1992, 14: 56- 75). In a grassfire technique, the object is imaged to be filled with dry grass and an initial firefront is ignited at the object boundary. It then propagates along the inward normal at constant speed. The time of arrival of the grassfire front at a given point equals the distance of that point from the shape boundary. The shape skeleton is the locus of points where fronts from two or more directions meet.

[92] Using this method, the tips of dendrite surface protrusion are found as those voxels assigned the locally largest distances, and tentatively identified as spine tips. From these spine tips to dendrite backbone a reverse grassfire process is used to label the voxels with another integer time step. Figure 9 shows an example of 2D projection of a piece of hypothetical spine. Each intersection plane contains all the voxels with the same integer time step assigned by the reverse grassfire process.

[93] Figure 10 shows the variation of the number of voxels in the intersection planes as a function of time integer step for 4 different morphological types of spines, i.e., filopodium spine, thin spine, mushroom spine and stubby spine. This figure clearly demonstrates that, for each type of spine, the number of voxels in the intersection plane (or intersection area) is the largest when the plane reaches the dendrite surface. Thus, in the inventive process, the maximum voxel number criterion is used to localize the segmentation plane. However, using this criterion may result in some dendrite voxels being counted as spine voxels. For example, in the case of short protrusions, the segmentation plane localized by this method may go deep into the dendrite. To correct this, the process of the present invention finds the voxel within this plane whose distance to the dendrite backbone is the largest (e.g., distance d shown on Figure 9), and then identifies the segmentation plane between the spine and dendrite as the plane passing through this voxel, and parallel to the dendrite backbone (see Figure 9). The number of voxels within the spine component is then determined and the distance (L) from the spine tip to the segmentation plane is calculated. For example, in the Examples reported herein, spine components with less than 10 voxels or whose length was less than 3 microns were considered as false protrusions and were rejected.

[94] Spine Component Merging. Some spine components may have been identified as detached head and attached base by the previous analysis steps and need to be merged before the final spine counting can be performed. The inventive process checks detached spine components and attached spine components to determine whether they belong to the same spine and should be merged. However, the inventive process does not attempt to merge a detached spine component with another detached spine component since it is usually difficult even for an expert neuroscientist to tell whether they belong to the same spine, due to the resolution limitation of the optical microscopic technique used to acquire the neuron images.

[95] To be merged by the inventive process a detached spine head and an attached spine base must be such that the distance between their centers-of-mass is less than a preset value and the tips of spine base must lie within the cone determined by the center of the spine head and the ring of spine-surface boundaries points. The preset value is preferably selected to be higher than the length of the longest dendritic spines. For example, for 0.5 to 2 micron-long dendritic spines, the preset value may be selected to be 3 microns.

[96] Dendrite and Spine Features Measurements. After dendritic segmentation and spine detection, several dendrite and spine parameters, including, but not limited to, spine density, dendrite length and volume, spine length, volume and surface area, spine head width and neck width, can be determined. Any suitable method can be used to quantify these features.

[97] For example, a spine length may be determined as follows. For a fully attached spine, the spine length may be determined by subtracted the integer time step of the spine tip from the voxel with the smallest integer time step within the spine. For partially attached spines (consisting of a base and a detached head), the spine length is the sum of base component length and detached component length. The base component length may be measured the same way as fully attached spines. The detached component length is determined as the distance from the base component tip to the farthest spine voxel within the detached component. For a detached spine, a line may be drawn between the center of mass of spine to its closest dendrite backbone. The detached spine length is determined as the distance from the dendrite surface voxel on this line to the farthest spine voxel within the spine.

[98] Using the inventive analysis method, spine lengths can be measured in both 2D and 3D. Manual analysis can only measure spine lengths in 2D by projecting the 3D stack of image slices along the optical direction. So in order to be able to compare the manual and automatic methods, spine lengths in 2D should be calculated (see Examples). [99] Volumes of spines may be calculated by multiplying the number of voxels in the spine with the unit volume of a voxel. The unit volume of a voxel is computed as the product of resolution in the x, y and z directions, taking into account that usually resolution in the z-direction is different from that in the x- and ^-directions.

[100] The surface area of a spine may be calculated in a similar manner, by adding the unit area of outward facing side(s) of boundary voxels.

[101] Spines shapes may be classified by calculating the ratios between the spine length I _s, head diameter dj,, and neck diameter d_n.

IV - Software / Hardware

[102] In general, the image analysis methods of the present invention employ various processes involving data stored in or transferred through one or more computer systems. Accordingly, embodiments of the present invention also relate to an apparatus for performing these operations. This apparatus may be specifically constructed for the required purposes, or it may be a general-purpose computer selectively activated or reconfigured by a computer program and/or data structure stored in the computer.

[103] The image analysis processes disclosed herein are not inherently related to any particular computer or other apparatus. Actually, the methods of the present invention may be implemented on various general or specific purpose computing systems. In certain embodiments, the image analysis methods of the present invention may be implemented on a specifically configured personal computer or workstation. In other embodiments, the image analysis methods of the present invention may be implemented on a general-purpose network host machine such as a personal computer or workstation. Alternatively or additionally, the methods of the invention may, at least partially, be implemented on a card for a network device or a general-purpose computing device.

[104] Accordingly, certain embodiments of the present invention relate to computer readable media or computer program products that include program instructions and/or data (including data structures) for performing various computer implemented operations. Examples of computer readable media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROM disks; magneto-optical media; semiconductor memory devices, and hardware devices that are specifically configured to store and perform program instructions, such as read-only memory devices (ROM) and random access memory (RAM). The data and program instructions of the present invention may also be embodied on a carrier wave or other transport medium. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.

V - Neuroinformatics System

[105] In another aspect, the present invention provides a neuroinformatics system, featuring software tools with persistent database in an integrated data processing pipeline, for segmentation, quantitation, correlation, and analysis of microscopic neuronal images. One embodiment of the inventive pipeline (called NeuronlQ, i.e., Neuron Image Quantitator) is comprised of five subsystems, namely data acquisition, image processing, image analysis, data analysis, and database, as shown on Figure 11.

[106] NeuronlQ processes and analyzes high-resolution neuronal and dendritic images acquired by optical microscopy, extracts features from images, and deposits extracted features and other information into a relational database system for subsequent browsing, exploration, retrieval, and statistical analysis (for example, on the Internet). Image features of interest include soma cross-section area, number of primary dendrites, branch number and length of dendrites, dendritic spine density, length, types, shapes, and their changes over time. Based on the results of image analysis, statistical analysis can be performed to investigate relationship between dendrite and spine morphology and experimental conditions (i.e., a given neural disease or condition of the subject from which neurons have been imaged).

Pipeline Structure

[107] NeuronlQ is built on modules for easy maintenance and upgrade. The data acquisition, image processing and image analysis subsystems are implemented in C/C++. The data acquisition relies in identification of regions of interest by the user. The automation effort of Neuron IQ is on the post-processing and management of acquired images (i.e., image processing, analysis, and archival). Data analysis comprises the use of independent statistical packages such as SAS (Statistical Analysis System, http://www.sas.com) and SPSS (http://www.spss.com). The application and web servers of NeuronlQ are based on Appache (http://www.apache.org). The database system is implemented using a structured query language (SQL) relational database. All subsystems work under Linux or Microsoft Windows.

Image Processing and Analysis

[108] The image processing and image analysis subsystems used in NeuronlQ have been described in detail above.

Data Archiving

[109] Measured features from NeuronlQ are organized stored in a neuronal database for further analysis. The neuronal morphometric features that can be extracted by NeuronlQ are as follows: for the primary dendrites: number and cross-section volume of soma; for the dendrites: number of branches and length of branches; for the dendritic spines: density, length, volume, shape and location. The NeuronlQ data model has three parts, i.e., binary image data, meta-data definitions from image analysis, and associated meta-data definitions of the images, e.g., relevant genetics, biological, and experimental information. The data model is instantiated via a relational database in which meta-data are stored in tables as specified by the schema, and both raw and pre-processed binary image data are stored in an image repository. The image repository is indexed by the pointers stored in the database tables.

[110] In addition to extracted image features, the NeuronlQ database contains other biological and experimental parameters of the specimens. These descriptive parameters include: type of species; genotype information; sex; age of subject at the time of samples (e.g., brain slices) preparation; brain region sliced; culture media used; days in vitro (DIV) when transfected; plasmids transfected; DIV when imaged; other drugs and times of application; type of cell imaged, type of cell region imaged; type and model of optical microscopy scanners; data from image acquisition protocols, data and time of acquisition, information about the experimenter; and other administrative information.

Data Mining

[111] After extracting features from image data and archiving them in the database, various statistical techniques may be applied to analyze and mine the features. Currently, statistical analysis of dendrite and spine features is not widely used because quantitating image data is difficult. NeuronlQ provides a tool to quantitate image data, thus facilitating statistical analysis.

[112] Some biological questions can be answered using data mining techniques such as statistical analysis, pattern recognition and data modeling (X. Zhou et ah, J. Franklin Institute-Engineering and Applied Mathematics, 2004, 341: 137-156; X. Zhou et al, IEEE/ ACM Trans, on Computational Biology and Bioinformatics, in press). An example is given below that illustrates how data can be analyzed by the inventive system.

[113] NeuronlQ allows the following features to be extracted: number of spines on each dendrite, length of each dendrite, spine density on each dendrite, and other features for each spine such as layer, volume, head volume, focal volume, length, height, head width, neck width, surface and compactness. To get information regarding the statistical characteristic of all spines on each dendrite, the parameters first need to be extracted based on the features of spines on each dendrite. Since the features are in different levels, the inventive system performs a z-score based normalization on each feature, using the following equation:

X- — X x,. = zscorβ{x_n) = -^{J ' J} std(x. _j)

O)

[114] where Xy is the/'¹ feature of the i^th spine, where I < i ≤ m, l ≤j < n, and m is the total number of spines on one dendrite and n is the total number of features of each spine, and where X_j and std(x._y) are the mean and standard deviation of the j^th feature. In order to estimate the distribution of each feature effectively, the distribution is considered as a mixture model. However, in the example described herein, a simple case is considered, i.e., the distribution of features is modeled by a Gaussian probability density function. The mean and variance of each feature is then used to represent the feature. This process leads to the following features for each dendrite: the number of spines on each dendrite, the length of each dendrite, the spine dendrite on each dendrite, the mean and variance of layer, and compactness. The mean values of these features for all dendrites in each neuron are considered as sufficient statistical features for that neuron. In order to detennine whether neurons are similar or different, Pearson linear correlation coefficients (S. Siegel and NJ. Castellan, "Nonparametric Statistics for the Behavioral Sciences", 1988, 2^nd Ed., McGraw-Hill: New York, xxiii, 399) are then calculated for each pair of neuron cells.

[115] Three neurons imaged from brain slices prepared from a healthy mouse were analyzed using NeuronlQ. The first neuron in apical region was found to have three dendrites, whose lengths were 496, 293, and 155 microns, and which carried 43, 68, and 12 spines, respectively. The second neuron in basal region had six dendrites whose lengths were 477, 315, 301, 98, 44, and 31 microns, carrying 56, 25, 25, 5, 5, and 2 spines, respectively. The third neuron in basal region had one dendrite whose length was 528, and carrying 1 spine. The Pearson linear correlation coefficients were calculated to be 0.9999 between the first and second neurons, 0.9992 between the first and third neurons, and 0.9997 between the second and third neurons, which led to the conclusion that these three studied neurons are similar to each other.

[116] NeuronlQ can be extended to include advanced information processing techniques such as Kolmogorov-Smirnov test, T-test, ANOVA, and other information processing techniques such as feature selection and pattern recognition (X. Zhou et ah, J. Biol. Systems, 2004, 12: 371-386)

IV - Applications of the Neuron Image Analysis Methods

[117] As already mentioned above, modern fluorescence microscopy methods, such as confocal laser scan microscopy and multi-photon laser scan microscopy, provide powerful tools to study dendrites and dendritic spines (B. Lendvai et al, Nature, 2000, 404: 876-881). Since these techniques can image both global dendritic structure and detailed spine geometry, they are particularly suited for the investigation of the subtle changes in dendritic arborization and altered spine densities, shapes, and distributions associated with many pathologies of cognitive function.

[118] By allowing for the automated, improved quantitative analysis of 3D neuron images, the processes and apparatus of the present invention will find numerous applications as powerful informatics tools which will help better understand how neuron morphology relates to neuronal function, which is crucial to the development of therapies and drugs for the prevention and/or treatment of neural disorders. In particular, the inventive processes and apparatus will provide a reliable, non-bias, automated solution to process and analyze large volumes of microcopy neuron image datasets and to investigate dynamic spine shape changes....

[119] Alternatively or additionally, the inventive processes and apparatus of neuron image analysis can be used to investigate the dynamic plasticity of dendritic spines. It has been found that over a time course of seconds to minutes, the majority of spines change their shape, and over a matter of hours, a substantial fraction of spines appear or disappear. The rapid morphological changes of spines has raised the possibility that those categories, rather than being intrinsically different populations of spines, represent instead temporal snapshot of a single dynamic phenomenon. The dynamic behavior of spines has attracted particular attention because they are the only neuronal structures that convincingly show experience-dependent morphological changes in the mammalian brain. The methods provided by the present invention may be used to understand these phenomena.

Examples

[120] The following examples describe some of the preferred modes of making and practicing the present invention. However, it should be understood that these examples are for illustrative purposes only and are not meant to limit the scope of the invention. Furthermore, unless the description in an Example is presented in the past tense, the text, like the rest of the specification, is not intended to suggest that experiments were actually performed or data were actually obtained.

[121] Most of the results presented below have been reported by the present Applicants in scientific abstracts and publications (X Chen et al, "An Automated Dendritic Spine Detection Method", to be presented at the IEEE, 2005; X. Xu et al, "A Computer-base System to Analyze Neuron Images^'", IEEE International Conference on Circuits and Systems, ISCAS, Kobe Japan, 2005; X. Xu and S.T.C. Wong, "Optical Microscopic Image Processing of Dendritic Spines Morphology" , submitted to IEEE Signal Processing Magazine; and X. Xu et al, "NeuronlQ: A Bioinformatics Platform to Analyze High Content Neuronal Image Data", submitted to The Journal of Bioinformatics). Each of these scientific abstracts and publications is incorporated herein by reference in its entirety.

Example 1 : EVALUATION RESULTS

[122] Confocal laser scan microscopy (CLSM) and two-photon laser scan microscopy (2PLSM) provide equivalent challenges to image analysis. However, 2PLSM generally performs better when working with living tissue, which allows high resolution fluorescence imaging of brain slices up to several hundred microns deep with minimal photodamage (B. Lendvai et al, Nature, 2000, 404: 876-881). In the experiments presented herein 2PLSM was used to acquire data.

[123] A set of 20 neuron images from cultured rat brain slices, which were provided by the Department of Neurology, Harvard Medical School, were used to test the inventive automated image analysis method. The xy size of the images was 512 x 512 with various z sizes depending on the neuron being studied. The step size of the microscope was 0.07 microns in both x and y direction and 1 micron in the z direction. Ten of the 20 neuron images were from basal area of pyramidal neurons and the other ten were from apical area.

[124] Using NeuronlQ, the average processing time for a 20 layer-neuron image was 1 minute on a P4 2.4 GHz computer. All the neuron images were processed using NeuronlQ and the results were compared to those obtained by manual analysis. Table 1 lists the experimental results obtained by the two methods. Of the 1134 spines identified by NeuronlQ, 1007 were identified by manual analysis, which matched 93.5% of the 1077 spines identified by the human eye. Table 1. Spine Detection Results

[125] Figure 12 gives an example of spine detection result comparison between a human expert and NeuronlQ. The human expert detected a total of 18 spines whereas NeuronlQ detected all those detected by the expert as well as an additional small spine that the human eye has missed.

[126] A comparison between NeuronlQ and manual analysis on length measurements was made on spines contained in 4 neuron images, which contained 45 apical spines and 51 basal spines, as identified by both methods. Figure 13 shows a comparison of the results obtained for these 95 spines by manual analysis and using NeuronlQ. Due to the small number of spines in each set (45 « 3,000; 51 « 3,000), Shapiro-Wilk tests (S-W-tests) instead of Kolmogorov-Smirnov tests (K-S tests) were performed to test their normality. S-W tests showed that both manual measurements and automated measurements of the samples of apical spines and basal spines were different from normal distributions. As shown in Table 2, spines from apical regions were far from normal distributions, while the ones from basal regions were relatively closer.

Table 2. Shapiro-Wilk Normality Test Results of Spine Length.

[127] As a result, Wilcoxon paired rank sum tests (i.e., Wilcoxon signed rank tests) were applied instead of paired t-test to determine how significant the differences were between the manual analysis and NeuronlQ. The observed v-statistic values were 340.5 and 241 with two-sided p-values equal to 0.6644 and 0.8564, for apical spines and basal spines, respectively. These values suggested no significant differences between results from the two methods. Kendall's rank correlations were calculated instead of Pearson's product moment correlations to test whether results from the two methods were correlated. The correlation values obtained for the apical spines and basal spines were τ = 0.7595633 and τ = 0.9129058, with two-sided p- values equal to 1.896^χlO^"13 and 2.2x10^"16. For these apical spines and basal spines, significant correlations between the automatic measurements and the manual measurements were observed. Therefore, no systematic bias exists between the manual and automatic measurements for these two sets of data. Furthermore, the significant Kendall's rank correlations indicated that the automatic measurement were duplicating the trends found by the manual measurements.

CONCLUSION

[128] Dendrite segmentation and spine detection are the most important and difficult parts for dendritic spine analysis. The accuracy of the parameters measured depends on the quality of dendrite segmentation and spine detection results. In the inventive method, an unsharp mask technique (B. Lendvai et ah, Nature, 2000, 404: 876-881) is used to reduce the high dynamic range of neuron image. A low threshold is then applied to the local contrast map to keep all low contrast dendrite components. A separate procedure is provided to remove the false positive signals caused by disconnected dendrites, axons, and smears.

[129] Spine detection in the inventive method is performed after dendrite segmentation. Detached spine components are detected by checking their relative location to dendrite backbone. Attached spine segmentation which separates the spine component from the dendrite component is the most difficult part of spine detection. A new method for spine segmentation is provided herein that comprises employing a grassfire technique to label a possible spine protrusion from its tip to the dendrite. Voxels with the same label in a spine form one intersection plane. In this method, the transition point from spine to dendrite is determined as the place were the number of voxels in the intersection plane increases the most. The voxel, within this intersection plane, which has the largest distance to the dendrite back is identified, and the segmentation plane between the spine and dendrite is determined to be the plane that is parallel to the dendrite backbone and that passes through this voxel. A post-process is then used to merge detached spine heads with attached spines based on their relative location.

[130] By using the grassfire technique, only voxels within the spine are labeled, no nearby spine voxels contribute to the intersection plane. The inventive method also does not rely on an a priory shape model to find the segmentation boundary, which is an important property considering the shape variation of different spine types.

[131] The results of the experimental evaluation presented above show the high accuracy on spine detection and segmentation obtained by the inventive processes. Many spine parameters such as length, intensity, volume, neck width, head width and surface area can be calculated directly by using the detection results.

Example 2: NEURONIQ

[132] As already mentioned above, it is important to develop a computer-based system to process large amount of neuronal images. By integrating image processing with a database, the system can facilitate statistical analysis to test biological hypothesis. Commercial software like Neurolucida by MicroBrightField (http://www.microbrightfield.com), Imaris by Bitplane AG (http://www.bitplane.ch) and NeuroZoom by Neurome (http://www.neurome.com) have different degrees of sophistication in neuron image analysis. Most of these products focus on tracing and extracting features from dendrites and axons, but lack the capability to analyze and extract features about spines. For example, Neurolucida requires user to manually mark spines on the dendrite. Imaris and NeuroZoom do not have spine analysis functionality. 3DMA, short for 3D Medial Axis, was created by Lindquist et al at SUNY-Stony Brook (CM. Weaver et al, J. Neurosci. Methods, 2003, 124: 197-205; CM. Weaver et al, Neural Comput, 2001, 16: 1353-1383; LY. Y. Koh et al, Neural Comput, 2002, 14: 1283-1310) to analyze neuron images and detect spines. However, as a stand-alone image processing method, 3DMA is not connected to an associated database and modeling. A method that process grayscale neuronal images was developed by Wearne et al (Neuroscience, 2005, 136: 661-680) using Rayburst sampling algorithm. The Rayburst technique was applied to 3D neuronal shape analysis at different scales to identify spines.

[133] NeuronlQ, the neuroinformatics system provided herein, features software tools with persistent database in an integrated data processing pipeline for segmentation, quantitation, correlation, and analysis of optical microscopy neuronal images. The pipeline of NeuronlQ consists of five subsystems, data acquisition, image processing, image analysis, data analysis, and database (as shown in Figure 11). Data were populated from neuroscience experiments and obtained from optical microscopy. The automation effort of NeuronlQ currently is in the post-processing and management of the acquired images, i.e., processing, analysis, and archival. It extracts features from images, and deposits the results and other information into a relational database system. The database can be accessed via Internet. Image features of interest include soma cross-section area, number of primary dendrites, branch number and length of dendrites, dendritic spine density, length, types, shapes, and their changes over time.

Materials and Image Acquisition

[134] Fluorescence label such as green fluorescence (GFP) was used to mark neurons in vitro. GFP absorbs blue light and converts this light to green light, which is of lower energy. The emitted green light can then be captured by optical microscope like confocal laser scanning microscopy (CLSM) and two-photon laser scanning microscope (2PLSM) and reveals details of the specimen.

[135] Dissociated hippocampal neurons were obtained from mice carrying a conditional Tscl allele (S.F. Tavazoie et al, Nature Neurosci., 2005, 8: 1727-1734). Neurons were transfected by GFP in organotypic hippocampla slices. Three dimensional images were acquired using 2PLSM with an excitation wavelength of 910 nm. The detailed acquisition steps have been described in S.F. Tavazoie et al, Nature Neurosci., 2005, 8: 1727-1734. Images were acquired at 5x magnification at spiny regions of basal and apical dendrites and optical sections were taken at 1.0 μm spacing. Image Processing and Analysis

[136] Digital microscopy, coupled with a variety of fluorescence and other labeling techniques, can obtain large amount of image data to localize, identify and characterize neurons. Noise removal is often needed in image processing. For example, fluorescence detection in image acquisition is generally performed by photomultiplier tubes (PMT). PMT is a high gain, low sensitivity device which, combined with the low number of photons emitted by small structures, such as dendritic spines, produces "photon shot noise". To remove this type of noise, a nonlinear filter, such as median filter, is usually applied. Standard median filter scans an image using a filter window. To reduce the blurriness caused by the point spread function of the microscopy, deconvolution is usually applied to restore the image. Commercial deconvolution package, such as AutoDeblurr (AutoQuant, Troy, New York), can be used to reduce the blurriness in the image.

[137] Following image pre-processing, image analysis classifies and extracts important features about dendrites and spines. Segmentation is the first step in image analysis to transfer a grayscale image to black-and-white image. For images of homogeneous brightness or close to homogeneous brightness, global thresholding can effectively separate neurons from background. For image consisting of inhomogeneous brightness, result of global thresholding is sub-optimal. To remove the uneven background, morphological operations like top-hat transformation and bottom-hat transformation can be used before thresholding. Top-hat transformation returns the image minus the binary opening of the original image where binary opening refers to the procedure of processing a grayscale image first by erosion and then by dilation. On the other hand, bottom-hat transformation returns the image minus the binary closing of the original image where the image is first processed by dilation and then by erosion. Given a grayscale image X, the contrast enhanced image Y is given by:

[138] where // and /2 stand for top-hat and bottom-hat transformation, respectively. Methods such as Otsu's criterion and two-means algorithm are used to segment pre-processed images. [139] After segmentation, the next step is to analyze and quantitate the spines which are connected with a neuronal dendrite, the backbone of the dendrite is extracted first. The backbone is approximated by the medial axis of the dendrite, which is found by recursively thinning layers of the dendrite until the thinning process stops (K. Palagyi and Q. Kuba, Pattern Recognition Letters, 1998, 19: 613-627). The recursive thinning process employs a number of templates of size 3 x 3 x 3 centered at each voxel. Each template is rotated and flipped around the central voxel to detect surface voxel and eliminate it from the object. In the NeuronlQ system a 6-direction thinning processing is used with 21 templates (K. Palagyi and Q. Kuba, Pattern Recognition Letters, 1998, 19: 613-627). After computing the medial axis, smoothing and trimming is then performed to remove short spurs to obtain the backbone of dendrites (see, for example, Figure 14 obtained using NeuronlQ).

[140] Once the backbone is found, spines on the dendrite can be detected and classified. To detect spines, a "region growing twice" process is applied. For the first region growing, (1) the process starts from the backbone and region growing is applied to uniformly propagate outward, and (2) the process finds the voxels that are reached in largest number of propagation. The voxel is labeled as tip of protrusion. For the second region growing, the process starts from the tip of protrusion and uses region growing to propagate toward the backbone until the areas of cross-section between two consecutive propagations increase suddenly, i.e., the propagation reaches the surface of the dendrite. The 3D volume covered in the second region growing is determine as the spine.

[141] One problem in spine detection is to discern attached spines from detached heads. In NeuronlQ, spines are detected in the following manner. First, for each detached component, the distance between this component to the nearest spine surface is measured and is used to determine whether this component is an artifact or candidate of spine. Second, for each attached component, two distances are measured. The first distance is measured between the spine candidate protruding out from the dendrite and the dendrite surface. The second distance is measured between the spine candidate protruding into the dendrite and the dendrite surface. The attached component is labeled a spine candidate if it protrudes out farther than it protrudes into the dendrite surface. Third, a merging algorithm checks all the spatial relationships between attached and detached candidates to determine whether some of them are to be merged.

[142] While many salient features of a spine, such as its volume, are difficult to determine through manual analysis, they can be easily obtained using NeuronlQ. NeuronlQ measures volume of spine by multiplying the number of voxels in the spine with the unit volume of a voxel. The unit volume of a voxel is computed as the product of resolution x-, y-, and z-direction (where the resolution in the z-direction is usually different from that of the x-direction and y-direction). NeuronlQ measures the surface area of detected spine in a similar manner, by adding the unit area of outward-facing side(s) of boundary voxels.

[143] Depending on its shape, a spine can be categorized into one of the three types: stubby, thin and mushroom (K. Zito and V.N. Murphy, Current Biology, 2002, 12: R5; LY. Koh et al, Neural Comput, 2002, 14: 1283-1310). Figure 15 shows an example of processed and detected spines. Spine shapes are classified according to the definition given by Harris et al. (J. Neurosci., 1992, 12: 2685-2705). In this method, the spine shape is decided by the length of spine l_s, head diameter d_h, and neck diameter d_n (see Figure 16).

[144] Features measured by NeuronlQ are organized and stored in a neuronal image database for further analysis. Table 3 lists neuronal morphometric features extracted by NeuronlQ. For the purpose of further statistical analysis, the results of NeuronlQ are stored in an associated database as described below.

Table 3. Neuronal morphologic features extracted from dendritic structure.

[145] To test the performance of Neuron IQ, the results obtained the inventive system were compared with those obtained through manual analysis. The manual analysis, which is usually considered as the gold standard, was performed by an expect in neuroscience who did not know the results obtained by NeuronlQ. In both manual analysis and automatic analysis, the lengths of spines in each image stack were measured and the cross-correlation between lengths given by the two methods was computed. Table 4 compares results from manual analysis and NeuronlQ on processing spines on an apical dendrite.

Table 4. Comparison of features extracted by manual analysis and automated NeuronlQ of an apical dendrite.

[146] The results obtained by the two methods are very close, indicating that NeuronlQ can approximate the accuracy of manual analysis by the expert. In addition, as a computerized too, NeuronlQ is able to measure the volume and surface area of spine, which is difficult to achieve in manual analysis.

[147] Another example is shown in Table 5 for basal dendrite. The result show that NeuronlQ can obtain approximately the same results as manual analysis does.

Table 5. Comparison of features extracted by manual analysis and NeuronlQ of a basal dendrite.

[148] Figure 17 presents the results of Kolmogorov-Smirnov (K-S) test of spine lengths of four different images, which provides another way to compare the cumulative distribution of spin length given by manual analysis and NeuronlQ. There were a total of 367 spines in the four images. From the cumulative distribution plots of Figure 17 (A-D), it can be noted that the two methods generated very similar results in spine length. At a level of 0.05, the K-S tests found no evidence of rejecting the hypotheses that the spine lengths given by manual analysis and NeuronlQ are almost the same. [149] Abnormal dendritic spines observed in tuberous sclerosis complex (TSC) were studied using the data archived in the NeuronlQ database. TSC is a hamartomatous disorder in which benign tumors proliferate in many organ systems including the brain, heart, kidney, and skin. TSC is caused by lack of the protein product of either TSCl or TSC2 alleles, which form a heterodimer and act as a negative regulator of mammalian target of rapamycin (mTOR), a kinase implicated as a master regulator of cell growth. It is important to investigate how the lack of either TSCl or TS C2 affects dendritic spine morphology in TSC. NeuronlQ was used to quantitate the decrease of spines due to the loss of TSCl or TSC2 and the results obtained were compared to results obtained through manual analysis. As seen on Figure 18, manual analysis and NeuronlQ achieved the same performance in analyzing the number of spines in experiments where RNA interference (RNAi) was used to express cells.

NeuronlQ Database

[150] NeuronlQ was implemented to comprise a neuronal image database to archive neuron images, extracted image features and associated meta-data of the experiments. The image database runs on an Oracle 1Og enterprise server and a 52- CPU Linux cluster, with a link to a digital image repository that has eight terabytes of network attached storage.

[151] As developed, the NeuronlQ data model has three parts: image data, metadata definitions from image analysis, and associated meta-data definitions of the images, e.g., relevant genetic information, biological background, and experimental protocols. The data model is instantiated via a relational database in which meta-data are stored in tables as specified by the schema, and both raw and pre-processed binary image data are stored in an image repository. Post-processed images are also archived in the database. The image repository is indexed by the pointers stored in the database tables.

[152] In the NeuronlQ system provided herein, each microscopy experiments is defined as an imaging session, to which microscopy configurations, extracted image features, and biological and experimental parameters of the specimens are related. The tables of microscopy configurations and equipment are directly associated to imaging session table. Each imaging session consists of several fields of image taken from the same experiment. A field of images represents an image stack of a particular area of tissue with the same configuration of field of view, resolution in three directions, and zoom factor. A field of view may focus on an apical or basal dendrite and this information is stored in the NeuronlQ database.

[153] Extracted image features are stored in entities of dendrites and spines. The tree structure of the NeuronlQ database for processed datasets is shown in Figure 19. Features of dendrite include branches, tree number, length, surface area, volume, base diameter, average diameter, base coordinate, density of spines, and tortuosity. Dendrite tortuosity is defined as a = d/ a, where d is the length of the straight line between two points in 3D and a is the length along the backbone between two points in 3D. Features of spines include length, volume, position, surface area, head width, and neck width (see Table 6). Information about the experimenter and other administration information are stored in the tables of project, organization and staff.

Table 6. Associated genetics, biological and experimental parameters modeled by the database of NeuronlQ.

[154] Biology and experiment information, such as animal and tissue, are stored in animal or tissue tables. These descriptive parameters include: type of species; genotype information; sex; age of subject at preparation of slices; brain region sliced; culture media used; days in vitro (DIV) where transfected; plasmid(s) transfected; DIV when imaged; other drugs and times of application; types of cells imaged; type of cell regions imaged; type and model of optical microscopy scanners; data from image acquisition protocols; data and time of acquisition.

[155] The first version of a web application was developed by the Applicants on JBoss, an open source J2EE application server. "Apache Torque", a database object persistence code generator, was used to generate Java database access code from an XML database schema definition file. This code generator was chosen because it significantly shortened the development cycle and gave great flexibility in schema change. Apache Torque also can generate database access code for a variety of databases. Porting the web application to support a database other then the current choice of Oracle requires a simple recompilation. Other core technologies used for the web application include Java Servlet, JSP (Java Server Pages), JSTL (Java Standard Tag Library) and JDBC (Java Database Connection). The web application currently supports a simple search on project data by a researcher's name, project keyword and subject species. It also supports experimental data download in CSV (comma-separated values) format.

Discussion

[156] One of the unresolved problems in computational neuroscience is to automatically process, analyze, and archive large amounts of image data generated by confocal or multi-photon laser scanning microscopy. NeuronlQ, which is provided herein, is, to our knowledge, the first integrated neuroinformatics systems that allows for the study of dendritic spine morphology. NeuronlQ can be used to analyze perturbations of neuronal morphology caused by genetic mutations related to neurological diseases. For example, NeuronlQ has been successfully used herein to investigate abnormal dendritic spines observed in TSC using a cell-autonomous model. NeuronlQ is able to provide rich collection of quantitative features about neurons, such as length of dendrites, number and density of spines on a dendrite, and shape of a spine, under various experimental conditions. Such features are then incorporated in a persistent database for subsequent data analysis. Data analysis allows comparative studies to be performed not only within an experiment, but among experiments under different settings.

[157] The main advantages of NeuronlQ include: (1) it extracts reproducible and objective feature measurements from neuronal images, and (2) it provides a mechanism for managing and analyzing large amounts of image datasets generated in high resolution optical microscopy by integrating its image processing capability with data analysis and persistent database management. NeuronlQ and associated databases of various neuroscience applications are still evolving. NeuronlQ is one of the first informatics systems dedicated to solve neuronal image management and analysis problems. The long-term objective is to develop NeuronlQ as an open- source, modular system of multi-functionalities while keeping its interface simple and easy to use for neuroscience researchers.

Other Embodiments

[158] Other embodiments of the invention will be apparent to those skilled in the art from a consideration of the specification or practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope of the invention being indicated by the following claims.

Claims

ClaimsWhat is claimed is:

1. A method for characterizing one or more neurons, wherein each neuron comprises one or more dendrites and each dendrite comprises one or more spines, the method comprising steps of: receiving a neuron image showing at least one neuron or part of one neuron; performing a segmentation analysis of the neuron image to obtain a segmented digital image; and extracting one or more parameters from the segmented digital image to characterize the at least one neuron.

2. The method of claim 1, wherein the neuron image is a 3D neuron image.

3. The method of claim 2, wherein the neuron image is obtained using confocal laser scan microscopy or multi-photon laser scan microscopy.

4. The method of claim 2 further comprising steps of: processing the neuron image using a median filter to remove noises and obtain a noise-free neuron image; and deconvoluting the noise-free neuron image to restore image distortion to obtain a processed neuron image prior to performing the segmentation analysis.

5. The method of claim 4, wherein the median filter is a center- weighted median filter.

6. The method of claim 2, wherein the step of performing a segmentation analysis comprises: applying an unsharp masking filter to the neuron image to obtain a local contrast image; and applying a global threshold process using a preset brightness level to the local contrast image to separate dendrite from background.

7. The method of claim 6, wherein the segmented digital image comprises a representation of the at least one neuron, each representation comprising a collection of signal intensity values at positions in the image where the neuron is present.

8. The method of claim 6 further comprising a step of: applying a high level threshold process to the local contrast image to extract dendrite backbones.

9. The method of claim 8, wherein applying a high threshold process comprises performing a medial-axis transformation.

10. The method of claim 8 further comprising removing false positive signals from the local contrast image.

11. The method of claim 10, wherein removing false positive signals comprises steps of: applying a low level threshold process to the local contrast image to obtain a low level processed image showing dendrites, detached objects, attached objects and smears; for each detached object, measuring the distance between the detached object and the nearest dendrite backbone; and identifying the detached object as a false positive signal if the distance measured is higher than a present distance value D; and for each attached object of intensity below the preset brightness level used in the global threshold process, measuring the distance between the attached object and the nearest dendrite backbone in the z-direction; and identifying the attached object as a false positive signal if the distance measured is larger than 2 layers; thereby obtaining a processed image showing dendrites, detached objects and attached objects but no false positive signals except for smears.

12. The method of claim 11, wherein the preset distance value D is higher than the length of the longest spine.

13. The method of claim 11 further comprising a step of detecting spines in the processed image with no false positive signals except for smears.

14. The method of claim 13, wherein detecting spines comprises steps of: for each detached object, measuring the length Li of the detached object in the direction parallel to the nearest dendrite backbone; measuring the length L₂ of the detached object in the direction perpendicular to the nearest dendrite backbone; and identifying the detached object as a smear if Li>L₂ and as a spine head if Li<∑₂,' and for each object attached to a dendrite, applying a grassfire process from dendrite backbone to dendrite surface to determine voxels of the attached object assigned the locally largest distances by the grassfire process and identifying these voxels as spine base tip; applying a reverse grassfire process from spine base tip to dendrite backbone to obtain sets of voxels of the attached object that are assigned an identical integer by the reverse grassfire process; determining, among the sets of voxels, the set of voxels that comprises the largest number of voxels; determining, within said set of voxels, the voxel whose distance to the dendrite backbone is the largest; and identifying the segmentation plane between the spine and dendrite as the plane passing through said voxel and parallel to the dendrite backbone.

15. The method of claim 14 further comprising merging spine head and spine base that belong the same spine.

16. The method of claim 15, wherein merging spine head and spine base that belong to the same spine comprises steps of: for each spine head, calculating the center of mass, Ri; for each spine base comprising a spine base tip and a ring of spine-surface boundaries points, calculating the center of mass, R₂; measuring the distance between the center of mass Ri and a spine head and the center of mass R₂ of a spine base; and merging the spine head and spine base if the distance measured is lower than a present distance value D ' and the tip of the spine base lies within a cone determined by the center of mass of the spine head and the ring of spine-surface boundaries points.

17. The method of claim 16, wherein the preset distance value D' is higher than the length of longest spine.

18. The method of claim 17, wherein extracting one or more parameters from the segmented digital image to characterize the at least one neuron comprises determining one or more features selected from the group consisting of dendrite number, dendrite density, dendrite length, dendrite volume, dendrite shape, dendrite location, spine number, spine density, spine location, spine length (l_s), spine volume, spine surface area, spine head diameter (d_h), spine head width, spine neck diameter (d_n), and spine neck width.

19. The method of claim 18 further comprising classifying at least one spine of a dendrite as filopodium, thin, stubby, mushroom-shaped or cup-shaped.

20. The method of claim 19, wherein classifying comprises calculating ratios of spine length (l_s), spine head diameter (<#,), and spine neck diameter (devalues for said spine.

21. The method of claim 2, wherein the one or more neurons are primary neurons, secondary neurons or immortalized neurons.

22. The method of claim 21, wherein the one or more neurons are human neurons.

23. The method of claim 21, wherein the one or more neurons comprises neurons treated under controlled conditions.

24. The method of claim 23, wherein the one or more neurons comprise neurons treated with a test agent.

25. A machine readable medium on which are provided program instructions for characterizing at least one neuron, the program instructions comprising: program code for receiving a neuron image showing at lest one neuron or part of one neuron; program code for performing a segmentation analysis of the neuron image to obtain a segmented digital image; and program code for extracting one or more parameters from the segmented digital image to characterize the at least one neuron.

26. The machine readable medium of claim 25, wherein the neuron image is a 3D neuron image.

27. The machine readable medium of claim 26, wherein the neuron image is obtained using confocal laser scan microscopy or multi-photon laser scan microscopy.

28. The machine readable medium of claim 26 further comprising program code for processing the neuron image using a median filter to remove noises and obtain a noise-free neuron image; and program code for deconvoluting the noise-free neuron image to restore image distortion to obtain a processed neuron image.

29 The machine readable medium of claim 28, wherein the median filter is a center-weighted median filter.

30. The machine readable medium of claim 26, wherein program code for performing a segmentation analysis comprises: program code for applying an unsharp masking filter to the neuron image to obtain a local contrast image; and program code for applying a global threshold process using a preset brightness level to the local contrast image to separate dendrite from background.

31 The machine readable medium of claim 30, wherein the segmented digital image comprises a representation of the at least one neuron, each representation comprising a collection of signal intensity values at positions in the image where the neuron is present.

32. The machine readable medium of claim 30 further comprising a step of: program code for applying a high level threshold process to the local contrast image to extract dendrite backbones.

33. The machine readable medium of claim 32, wherein program code for applying a high threshold process comprises program code for performing a medial-axis transformation.

34. The machine readable medium of claim 32 further comprising program code for removing false positive signals from the local contrast image.

35. The machine readable medium of claim 34, wherein program coded for removing false positive signals comprises steps of: program code for applying a low level threshold process to the local contrast image to obtain a low level processed image showing dendrites, detached objects, attached objects and smears; program code for measuring, for each detached object, the distance between the detached object and the nearest dendrite backbone; and identifying the detached object as a false positive signal if the distance measured is higher than a present distance value D; and program code for measuring, for each attached object of intensity below the preset brightness level used in the global threshold process, the distance between the attached object and the nearest dendrite backbone in the z-direction; and identifying the attached object as a false positive signal if the distance measured is larger than 2 layers; thereby obtaining a processed image showing dendrites, detached objects and attached objects but no false positive signals except for smears.

36. The machine readable medium of claim 35, wherein the preset distance value D is higher than the length of the longest spine.

37. The machine readable medium of claim 35 further comprising program code for detecting spines in the processed image with no false positive signals except for smears.

38. The machine readable medium of claim 37, wherein program code for detecting spines comprises steps of: for each detached object, program code for measuring the length L_\ of the detached object in the direction parallel to the nearest dendrite backbone; program code for measuring the length L ₂ of the detached object in the direction perpendicular to the nearest dendrite backbone; and program code for identifying the detached object as a smear if Lι>∑2 and as a spine head if Li<L,2\ and for each object attached to a dendrite, program code for applying a grassfire process from dendrite backbone to dendrite surface to determine voxels of the attached object assigned the locally largest distances by the grassfire process and identifying these voxels as spine base tip; program code for applying a reverse grassfire process from spine base tip to dendrite backbone to obtain sets of voxels of the attached object that are assigned an identical integer by the reverse grassfire process; program code for determining, among the sets of voxels, the set of voxels that comprises the largest number of voxels; program code for determining, within said set of voxels, the voxel whose distance to the dendrite backbone is the largest; and program code for identifying the segmentation plane between the spine and dendrite as the plane passing through said voxel and parallel to the dendrite backbone.

39. The machine readable medium of claim 38 further comprising program code for merging spine head and spine base that belong the same spine.

40. The machine readable medium of claim 39, wherein program coded for merging spine head and spine base that belong to the same spine comprises: program code for calculating the center of mass, Ri, for each spine head; program code for calculating the center of mass, R₂, for each spine base comprising a spine base tip and a ring of spine-surface boundaries points; measuring the distance between the center of mass Rj and a spine head and the center of mass R₂ of a spine base; and merging the spine head and spine base if the distance measured is lower than a present distance value D ' and the tip of the spine base lies within a cone determined by the center of mass of the spine head and the ring of spine-surface boundaries points.

41. The machine readable medium of claim 40, wherein the preset distance value D ' is higher than the length of longest spine.

42. The machine readable medium of claim 41, wherein extracting one or more parameters from the segmented- digital image to characterize the at least one neuron comprises determining one or more features selected from the group consisting of dendrite number, dendrite density, dendrite length, dendrite volume, dendrite shape, dendrite location, spine number, spine density, spine location, spine length (l_s), spine volume, spine surface area, spine head diameter (df,), spine head width, spine neck diameter (d_n), and spine neck width.

43. The machine readable medium of claim 42 further comprising program code for classifying at least one spine of a dendrite as filopodium, thin, stubby, mushroom-shaped or cup-shaped.

44. The machine readable medium of claim 43, wherein program code for classifying comprises program code for calculating ratios of spine length (l_s), spine head diameter (d_h), and spine neck diameter (devalues for said spine.

45. A computer program product comprising a machine readable medium of claim 44.

46. The computer program product of claim 45 further comprising a database system for storing data including binary image data, meta-data definitions from image analysis, and associated meta-data definitions of the images.

46. An image analysis apparatus for characterizing at least one neuron, the apparatus comprising: a memory adapted to store, at least temporarily, a neuron image showing at least one neuron; and a processor configured or designed to characterize said at least one neuron shown on the image.

47. The image analysis apparatus of claim 46, wherein the neuron image is a 3D neuron image.

48. The image analysis apparatus of claim 47 further comprising an interface adapted to receive the neuron image.

49. The image analysis apparatus of claim 48 further comprising an image acquisition system that produces the image.

50. The image analysis apparatus of claim 49, wherein the image acquisition system is a confocal laser scan microscope or a multi-photon laser scan microscope.

51. The image analysis apparatus of claim 48, wherein the processor characterizes the at least one neuron by performing the method of claim 19.