Data processing: generic control systems or specific application – Specific application – apparatus or process – Digital audio data processing system
Reexamination Certificate
1999-01-28
2004-04-20
Harvey, Minsun Oh (Department: 2644)
Data processing: generic control systems or specific application
Specific application, apparatus or process
Digital audio data processing system
C381S056000, C084S47000P, C084S47700R
Reexamination Certificate
active
06725108
ABSTRACT:
FIELD OF THE INVENTION
The invention relates to the analysis and understanding of acoustic spectra, particularly the spectra of musical sounds. More specifically, the invention relates to the extraction of pitch and timbre from the spectra of musical sounds.
BACKGROUND OF THE INVENTION
Musicians and others (piano tuners, acousticians) are often concerned with evaluating musical sounds in real time, particularly with regard to pitch and tone quality (timbre). During training, rehearsal, and performance, both singers and instrumentalists make these evaluations continuously, and adjust their technique accordingly to improve the sound. Music teachers, orchestral conductors and choral directors make similar evaluations, and by gesture or verbal instruction indicate how performance should be improved.
In all of these endeavors, the human ear and brain are used to evaluate the sound. Although this mechanism is necessary during performance, and marvelous for judging the “higher” qualities of music such as “expressiveness”, it is hardly ideal for evaluating purely mechanical aspects of sound such as pitch and timbre, because human judgment is subjective. This problem is particularly acute for performing musicians, because the person evaluating the sound is busy producing it. Thus singers and instrumentalists often sing and play off-key while swearing they are in tune, or produce a poor tone quality (timbre) while imagining they are producing a good one.
The tendency to misjudge can be remedied by training, but in the absence of a teacher, in the hours practicing alone, there is typically no objective measure of pitch and timbre. Several techniques may be used but have limitations and drawbacks. For example, a keyboard instrument may be used intermittently to check pitch, but it dos not give continuous pitch feedback to the musician, and says nothing about timbre. Alternatively, recording and playing back may be used to separate the process of sound production from that of sound evaluation, but this is tedious because it is not real time.
To solve these problems, a mechanism is needed to provide real-time visual feedback of pitch and timbre to the musician, based on objective and consistent measurements. Visual feedback is ideal because does not interfere with the auditory feedback that the musician must ultimately use in performance. Rather, the visual feedback should help train the auditory system by showing the musician when pitch and tone quality are good. A personal-computer-based software tool would be ideal, since it is flexible, improves automatically as computer technology progresses, and avoids the cost of dedicated instrumentation.
PROBLEMS WITH THE PRIOR ART
To analyze sound, particularly musical sound, it is essential to begin, as the ear does, with a spectral analysis. All subsequent analysis, such as the extraction of pitch and timbre, depends on the spectral analysis. Yet, as shown below, it is at this fundamental level of spectral analysis that the prior art is deficient. The prior art's technique for doing spectral analysis is the Discrete Fourier Transform (DFT), and its efficient implementation known as the Fast Fourier Transform (FFT). See
Numerical Recipes in C; The Art of Scientific Computing
, William H. Press, Brian P. Flannery, et. al., Cambridge University Press, 1988, ISBN 0-521-35465-X, pp. 403-418, which is herein incorporated by reference in its entirety.
To demonstrate the deficiency of the DFT, it is helpful to summarize some of the mathematics involved. Using the DFT, a signal g(t) (e.g. sound pressure as a function of time) is windowed by a windowing function W(t), such as the Welch Window, which is defined to be non-zero only over the time interval [0, &Dgr;t].
The windowed function
ĝ
(
t
)≡
g
(
t
)
W
(
t
) (1)
is sampled at N discrete times in the interval [0, &Dgr;t], namely
t
n
≡
n
S
=
n
N
Δ
t
,
n
=
0
,
…
,
N
-
1
,
(
2
)
where S is the sampling rate in Hz. Therefore the total time to measure the N-fold ensemble of samples is
Δ
t
meas
=
N
S
.
(
Sound
-
measurement
time
)
(
3
)
Furthermore, using the DFT, the frequency content of ĝ(t) at frequency f, given by
G
^
(
f
)
=
1
S
∑
n
=
0
N
-
1
g
^
(
t
n
)
ⅇ
2
π
ⅈ
f
t
n
,
(
4
)
is evaluated only at certain discrete values of the frequency f namely at the values
f
k
≡
S
N
k
,
k
=
-
N
2
,
…
,
N
2
.
(
5
)
(5)
Therefore the frequency granularity of the DFT (the difference between two adjacent frequencies, f
k+1
−f
k
), is
Δ
f
=
S
N
.
(
Frequency
granularity
for
DFT
)
(
6
)
The deficiency of the DFT is summarized by equations (3) and (6), which together imply that
&Dgr;
f &Dgr;t
meas
=1. (Property of Discrete Fourier Transform) (7)
That is, with the DFT, it is impossible to achieve both a short sound-measurement time &Dgr;t
meas
and a fine frequency granularity &Dgr;f . For example, if it is desired to have a short sound-measurement time of 0.1 seconds, then (7) implies that &Dgr;f must assume the rather coarse value of 10 Hz. Conversely, if a fine frequency granularity of 1 Hz is desired, &Dgr;t
meas
must assume the large value of 1 second. In light of equation (7), it may be concluded that the DFT is inadequate for applications requiring both real-time data acquisition and precise spectral analysis in real time, because such applications require both small &Dgr;t
meas
and small &Dgr;f.
For example, in applications where musical sound need to be measured and analyzed in real time, small &Dgr;t
meas
is necessary to achieve the “real-time” objective. In particular, since fast musical notes are on the order of 80 to 100 milliseconds in length, the application demands
&Dgr;t
meas
≦0.1 seconds. (8)
For the same type of application, small &Dgr;f (frequency granularity) is necessary to achieve accurate results in the computation of pitch. The frequency ratio between two musical notes a half-step apart on the equally tempered scale is
f
+
f
=
2
12
,
(
9
)
where f
+
is the upper of the two notes and f is the lower of the two notes. Thus the frequency difference between two notes a half-step apart is
Δ
f
halfstep
=
f
(
2
12
-
1
)
.
(
10
)
For example, at C131 (i.e. 131 Hz, a note in the middle of the range of a human baritone voice), &Dgr;f
halfstep
is 7.8 Hz. Thus to achieve good pitch resolution of, say, an eighth of a half step, the application demands roughly
&Dgr;
f
≦1 Hz. (11)
Thus, the requirements of such an application with regard to data-acquisition time and frequency granularity, typified by equations (8) and (11), are an order of magnitude more demanding than the capability (7) offered by the DFT. Therefore the DFT is inadequate for such applications, and any prior art that uses it is likewise inadequate. This inadequacy is not dependent on the speed of the computer used to implement the DFT; even if the computer were infinitely fast, the inadequacy would remain the same, because it is inherent in the DFT algorithm itself.
Because the prior art is thus deficient in its ability to perform real-time data acquisition and finely-resolved spectral analysis simultaneously, it is therefore also deficient in its ability to perform accurate, real-time “note analysis”, wherein the pitch and timbre of the sound are extracted, since note analysis uses the output of spectral analysis as its starting point.
PC Programs to acquire and analyze sounds using the DFT certainly exist, such as
CoolEdit
by Syntrillium Software and
Spectrum Analysis
by Sonic Foundary. However, these programs are not typically aimed at real-time applications, and make no attempt to extract pitch and tim
Beck Thomas A.
Harvey Minsun Oh
Herzberg Louis P.
Jacobson Tony M.
LandOfFree
System and method for interpretation and visualization of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for interpretation and visualization of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for interpretation and visualization of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3224409