|Collaborators & Students:
and Sensory Information Processing Systems
This program deals with the study and development of systems that are
able to deal with complex sensory processing tasks, such as vision and speech. The
approach followed is to a large extent inspired by the biological system and makes use of
a hierarchy of processing stages. The overall system consists of dedicated sensors
embedded with local processing elements, followed by a neural network that performs higher
levels of processing.
(Viktor Gruev, Zheng Yang, Nader Engheta)
CMOS Camera for Polarimetric Information Extraction and Imaging (in collaboration with Prof. Nader Engheta, Ed Pugh and Viktor Gruev)
We are working on a imaging system for the extraction of polarization information. This work consists of two part. The first aspect focuses on a CMOS based imager that is capable of real-time extraction of polarization information. The imaging system consists of a photo array of active pixels, an analog processing unit at the focal plane for noise suppression and computation of the Stokes parameters. The second aspect deals with the development of a micropolarizer thin film.
- "Image Sensor With Focal Plane Extraction of Polarimetric Information," Proc. of the Proc. IEEE ISCAS, May 2006; V. Gruev, J. Van der Spiegel and N. Engheta.
CMOS Imagers (with Zheng Yang, Viktor Gruev)
We are also studying new approaches for CMOS imagers. We are developing a CMOS image sensor that is capable of both voltage- and current-mode operations. We have also proposed a novel current-mode readout technique using a velocity saturated short-channel transistor, which achieves high linearity. We are also developing new pixels structures and read-out schemes that allow us to reduce the number of transistor per pixels.
- "A CMOS Linear Voltage/Current Dual-Mode Imager," Proc. of the IEEE ISCAS, May 2006; Zheng Yang, Viktor Gruev and Jan Van der Spiegel.
A CMOS image processing sensor for the detection of image features (with Masatoshi Nishimura)
We have been working on a vision sensor that serves as an intelligent front end of a pattern recognition system. The sensor detects higher level image features, such as corners, junctions (T-, X-, Y-type) and linestops. The on-chip detection of these features significantly reduces the data amount and hence facilitates the subsequent processing of pattern recognition. The sensor performs a series of template matching operations in an analog/digital mixed mode for various kinds of image filtering operations including thinning, orientation decomposition, error correction, set operations, and others. The analog operations are done in the current domain. These features serve as input to a system that recognizes characters or categorizes objects in broad categories. The pixels are programmable and consist of photodetectors and circuitry that can be configured to perform a variety of spatial filtering operations.
- "Biologically Inspired Vision Sensor for the Detection of Higher-Level Image Features," M. Nishimura and J. Van der Spiegel, Proceedings of the 2003 IEEE Conference on Electron Devices and Solid-State Circuits, pages 11-16, 2003
- “A CMOS Image Processing Sensor for the Detection of Image Features,” Journal Analog Integrated Circuits and Signal Processing, Vol. 43, pp. 1-17, 2005, M. Nishimura and J. Van der Spiegel.
- "An algorithm for the detection of image features on a smart sensor," Technical Report, M. Nishimura and J. Van der Spiegel, 2003.
- "A VLSI Computational Sensor for the Detection of Image Features,' Masatoshi Nishimura, Ph.D. Thesis, Dept. Electrical Engineering, Univ. of Pennsylvania, Philadelphia, PA
Previous work in Imagers:
Tracking Sensor (in collaboration with Prof. Ralph Etienne-Cummings and Paul Mueller, Corticon Inc.)
We developed a motion perception chip which acquires a two-dimensional images and determines the direction of motion. The chip's architecture and processing elements are biologically motivated. The chip makes use of a spatially variant sampling structure and of an arbitration scheme between smooth pursuit and acquisition to improve the tracking process. A foveal region contains edge detection, contrast normalization circuits and computes divergent 2D velocity. The peripheral region is used to re-capture targets which escape the fovea. The system is used for object tracking.
Photo of the Tracking Chip .More Information:
- "A Foveated Visual Tracking Chip", Technical Digest, Intern. Solid-State Circuits Conference (ISSCC97), pp. 38-39, 1997 (pdf file - 240 kB)"
- A Foveated Silicon Retina for Two-Dimensional Tracking", IEEE Trans. Circuits and Systems II, Vol. 47, pp. 504-517, June 2000; R. Etienne-Cummings, J. Van der Spiegel, P. Mueller and M.Z. Zhang.
- "A Motion Perception and Visual Tracking Chip," Penn Technology News, Vol 5, Fall 1998.
Retina-like CCD Sensor (with Dr. Greg Kreider)
Van der Spiegel and his group developed a foveated retina-like CCD sensor in collaboration with Prof. C. Claeys at IMEC, Prof. G. Sandini at the University of Genova, and Prof. P. Dario at the Scuola Superiore S. Anna, Pisa. This device is an example of a sensor whose computational properties are embedded in the geometry of the structure. This sensor explores the log-polar sampling space and provides significant functional and computational advantages over conventional imagers for real-world vision tasks. The sensor incorporates several unique structures and makes use of the capabilities of CCD technology to implement both optical and electronic functions.
Related work to Space Variant Sensors: Univ. of Genoa - Lira Lab
- "A Retinal CCD Sensor for Fast 2D Shape, Recognition and Tracking," Sensors and Actuators, Vol. A21, pp. 456-460, 1990. I. Debusschere, E. Bronckaers, C. Claeys, G. Kreider, J. Van der Spiegel, G. Sandini, P. Dario, F. Fantini, P. Bellutti, G. Soncini
- "A Foveated Retina-Like Sensor Based on CCD Technology," in "Analog VLSI Implementation of Neural Systems", eds. C. Mead and M. Ismail, Kluwer Academic Publ., Boston, MA, 1989, Chapter 8, pp. 189-210; also in Vision Chips: Implementing Vision Algorithms with Analog VLSI Circuits, pp. 442-465, IEEE Press (Selected Reprint Volume), by C. Koch and H. Li, 1995.J. Van der Spiegel, G. Kreider, C. Claeys, I. Debusschere, G. Sandini, P. Dario, F. Fantini, P. Bellutti and G. Soncini.
Analog to Digital Converters
(Sameer Sonkusale, Jie (George) Yuan, Alper Meric)
Background Calibration of A/D Converters (with Sameer Sonkusale, Alper Meric)
A/D converters suffer from several non-idealities like INL, DNL, offset,harmonic distortion due to capacitor mismatches, finite gain of the opamp, offsets in the comparator, parasitics and several other phenomenon related to a particular architecture. Techniques to increase the resolution of an A/D converter have so far been in terms of self-calibration of the A/D converter, which requires the A/D converter to stop conversion every few cycles to perform calibration. Techniques have to be designed such that calibration can be performed in background without stopping conversion.The scope of this research involves identifying and modeling the non-idealities in an A/D converter efficiently and to develop a speed-power efficient architecture and algorithm to perform calibration in the background.
- "Background Digital Error Correction Technique for Pipelined Analog-Digiital Converters", IEEE International Symposium on Circuits and Systems 2001 (ISCAS 2001) Volume 1, pages 408-411, S. Sonkusale, J. Van der Spiegel, and K. Nagaraj.
- "True Background Calibration Technique for Pipelined ACD", Electronics Letters, Vol. 36, No. 9, pp786-788, 2000, S. Sonkusale, J. Van der Spiegel, and K. Nagaraj."
- 'Mixed-Signal Calibration of Pipelined Analog-Digital Converters," Proceedings of the 2003 IEEE International SOC [Systems-on-Chip] Conference, pages 327-330, S. Sonkusale and J. Van der Spiegel.
- "A Low Distortion MOS Sampling Circuit," Proceedings of the IEEE International Symposium on Circuits and Systems 2002 (ISCAS 2002), Volume 5, pages V-585 - V-588; S. Sonkusale and J. Van der Spiegel
We are investigating several calibration methods. One such background calibration method is based on stage error pattern estimation The method corrects both linear and non-linear errors. The procedure converges in a few ms and requires minimal hardware without the need of highcapacity ROM or RAM.
- "A 50MS/s 12-bit CMOS Pipeline A/D Converter with Nonlinear Background Calibration,” IEEE Custom Integrated Circuit Conference (CICC 2005), Sept. 2005, Jie Yuan, N. Farhat and J. Van der Spiegel.
- "GBOPCAD: A Synthesis Tool for High-Performance Gain-Boosted Opamp Design," IEEE Transactions on Circuits and Systems--I: Regular Papers, Volume 52, Issue 8, August 2005, pages 1535-1544, Jie Yuan, N. Farhat and J. Van der Spiegel.
Modeling of Power Dissipation in delta-sigma converters (with Q. Li and Dr. K. Laker)
This project involves the development of architectures that enable a delta-sigma ADC to realize high dynamic range requirements when implemented in low-voltage technologies and to minimize overall power dissipation. In order to be able to compare different schemes a "Energy-per-Conversion" (EPC) model was developed. The purpose of the EPC model is to provide a figure of merit that enables one to understand the trade-offs that exist in low-voltage, low-power delta-sigma converter design. The model starts with the fundamental lower bound based on thermal noise considerations of the input stage and gives a relationship between the energy-per-conversion and signal-to-noise ratio (SNR).
Signal Adaptive Control Architecture for a Delta-Sigma converter (with Qunying Li, Dr. K. Laker))
We have been working on a Signal Adaptive Control (SAC) architecture for a 2nd order delta-sigma modulator. This scheme reduces the power dissipation and the harmonic distortion in the first stage integrator. The key to the operation is the switching off of the DAC feedback signal to the 1st stage during certain iterations and to compensate the signal at the input of the 2nd stage in an adaptive manner.
- A Low-Voltage/Low-Power Second-Order Delta-Sigma Modulator with Signal Adaptive Control Architecture," IEEE ISCAS, May, 1999, Proc. Vol. II, pp. 41-44 (pdf file - 190kB)"Signal Adaptive Control Architecture of Delta-Sigma Modulator Design", Electronics Letters, Vol. 35, No. 8, p. 610, 1999"
- A 1.2 V, 38 microW Second-Order DeltaSigma Modulator with Signal Adaptive Control Architecture ," IEEE 2nd Dallas CAS Workshop on Low Power/Low Voltage Mixed-Signal Circuits and Systems 2001 (DCAS 2001), pages P23-P26, Qunying Li, K. Laker and J. Van der Spiegel
- "An Overview of Sigma-Delta Converters: How a 1-bit ADC achieves more than 16-bit resolution," Signal Processing Magazine, Volume 13, Issue 1, September 1996, pages 61-84, A. M. Pervez, H. V. Sorensen, J. Van der Spiegel.
Phase-Locked Loop (with Chao Xu, Ken Laker and Winslow Sargeant)
A fully integrated phase-locked loop (PLL) fabricated in a 0.24μm, 2.5v digital CMOS technology has been studied and developed.. The PLL is intended for use in multi-gigabit-per-second clock recovery circuits in fiber-optic communication chips. This PLL first time achieved a very large locking range measured to be from 30MHz up to 2GHz in 0.24μm CMOS technologies. Also it has very low peak-to-peak jitter less than ±35ps at 1.25GHz output frequency.
- "A Fully Integrated CMOS Phase-Locked Loop With 30MHz to 2GHz Locking Range and ±35 ps Jitter," Analog Integrated Circuits and Signal Processing, Volume 36, Issue 1-2, July 2003, pages 91-97, Chao Xu, W. Sargeant, K. Laker and J. Van der Spiegel.
- " An Extended Frequency Range CMOS Voltage-Controlled Oscillator," Proceedings of the 9th IEEE International Conference on Electronics, Circuits and Systems (ICECS 2002), Volume 2, pages 425-428, Chao Xu, W. Sargeant, K. Laker and J. Van der Spiegel.
Speech Recognition and Speech Conversion
(with Dr. A. Mouchtaris, P. Mueller and A. Ali)
Spectral Conversion for Speech Enhancement
Current voice conversion algorithms require a parallel speech corpus that contains the same utterances from the source and target speakers for deriving a conversion function. We have worked on a voice conversion method that does not require a parallel corpus for training, i.e. the spoken utterances by the two speakers need not be the same, by employing speaker adaptation techniques to adapt to a particular pair of source and target speakers, the derived conversion parameters from a different pair of speakers. We have shown that adaptation reduces the error obtained when simply applying the conversion parameters of one pair of speakers to another by a factor that can reach 30% in many cases, and with performance comparable with the ideal case when a parallel corpus is available.
We have also demonstrated that spectral conversion can be successfully applied to the speech enhancement problem as a feature denoising method. The enhanced spectral features can be used in the context of the Kalman filter for estimating the clean speech signal. In essence, instead of estimating the clean speech features and the clean speech signal using the iterative Kalman filter, we show that is more efficient to initially estimate the clean speech features from the noisy speech features using spectral conversion (using a training speech corpus) and then apply the standard Kalman filter. Our results show an average improvement compared to the iterative Kalman filter that can reach 6 dB in the average segmental output Signal-to-Noise Ratio (SNR), in low input SNR’s.
Acoustic-Phonetic Feature Extraction (with Prof.
P. Mueller and Ahmed A. Ali ))
Over the last several years we have also
investigated methods to improve speech recognition. Our goal is to integrate
acoustic-phonetic knowledge into speech recognition systems. A novel strategy is used for
this purpose. We use auditory-based front-end preprocessing that incorporates many of the
human-like effects such as critical band filtering, short-term adaptation forward masking,
non-linearities, etc. The output of such a preprocessing block feeds into a system that
extracts acoustic-phonetic features. The extracted features are chosen based on their
information content and discrimination ability in various tasks. Algorithms are designed
and implemented for phoneme recognition. We have applied this method successfully for
speech segmentation, fricative and stop recognition.
- "Robust Auditory-Based Speech Processing Using the Average Localized Synchrony Detection," IEEE Transactions on Speech and Audio Processing, Volume 10, Issue 5, July 2002, pages 279-292, A. M. Ali, Jan Van der Spiegel and P. Mueller.
- "Acoustic-Phonetic Features for the Automatic
Classification of Fricatives", J. Acoustial Soc of America (JASA), Vol. 109 (5), pp.
2217-2235, May 2001, A. M. Ali, Jan Van der Spiegel and P. Mueller.
- "Robust Classification of Stop Consonants using
Auditory-based Speech Processing", IEEE International Conference on Acoustics, Speech
and Signal Processing (ICASSP-2001), Salt Lake City, May, 2001. A.M. Abdelatty Ali,
Jan Van der Spiegel and Paul Mueller.(pdf file -
- "An Acoustic-Phonetic Feature-based System for Automatic Phoneme
Recognition in Continuous Speech," IEEE ISCAS, May, 1999, Proc. Vol. III, pp.
118-121.(pdf file - 70 kB}
- "Acoustic-Phonetic Features for Automatic Recognition of Stop
Consonants" IEEE Transactions on Speech and Audio Processing, Volume 9, Issue 8, November 2001, pages 833-841; A. M. Ali, Jan Van der Spiegel and P. Mueller.
- "Acoustic-phonetic features for the automatic recognition of stop
consonants" in The Journal of the Acoustical society of America, vol. 103 No.5
May 1998; A. M. Ali, J. Van der Spiegel and Paul Mueller.
General Purpose Analog Neural Computer
(in collaboration with Dr. Paul Mueller, Corticon Inc., D. Blackman, C. Donham, R. Etienne-Cummings)
Our group developed in collaboration with Corticon Inc. a large scale programmable analog neural computer. The architecture is loosely modeled after the pathways in the human brain.. The network consists of over 800 custom VLSI modules and contains programmable neurons, synapses, synaptic time constants and programmable interconnects. The computer runs in analog mode which enables a truly simultaneous summation of many inputs at a single neuron while the programmable time constants permit dynamic computation of temporal patterns as they occur in motion and speech.
The machine is intended for real-time, real world computations such as speed recognition, sonar, ultrasonic, vision, robotics, control automation and other applications requiring computational power and speed exceeding the performance limit of current digital machines. One of the most promising applications is speaker independent speech recognition. The neural network decomposes speech in a sparse set a pattern primitives that will be used to recognize phonemes. Phoneme-based recognition systems are simpler than word-based systems and have the potential to be speaker independent and requires less hardware to be implemented.
- "An Analog Neural Network with Modular Architecture for Real-Time Dynamic Computations," in IEEE J. Solid-State Circuits, Vol. 27, pp.82-92, 1992. J. Van der Spiegel, D. Blackman, P. Chance, C. Donham, R. Etienne-Cummings and P. Kinget.
- "A Programmable Analog Neural Computer and Simulator," in Artificial Neural Networks, E. Sanchez-Sinencio and C. Lau, Eds., IEEE Press (Selected Reprint Volume), pp. 218-224, 1992, P. Mueller, J. Van der Spiegel, D. Blackman, T. Chiu, T. Clare, J. Dao, C. Donham, T-P. Hsieh and M. Loinaz.
- "Real Time Decomposition of Acoustical Patterns with an Analog Neural Computer," SPIE Conf. on Applications of Artificial Neural Networks III, Vol. 1709, pp. 758-769, 1992. P. Mueller, J. Van der Spiegel, D. Blackman, C. Donham and R. Etienne-Cummings.
Summer Undergraduate Research for
I am coordinating the REU SUNFEST program which provides summer research position
for motivated undergraduate students. The area of research is in Sensor Technologies and
includes physical sensors, chemical sensors, neural networks, electronic materials,
nanotechnology and robotics. Typically 10 students participate per summer. For more
information visit the SUNFEST homepage.